1 2015-10-29 01:42:01 <paulo_> i would just like to praise the code, it's very readable
  2 2015-10-29 05:12:11 <rusty> CodeShark: Hi!  I'm working on a simpler implementation of versionbits, BTW.
  3 2015-10-29 05:26:48 <CodeShark> rusty: oh?
  4 2015-10-29 05:27:08 <CodeShark> simpler how?
  5 2015-10-29 05:29:11 <rusty> CodeShark: yes, I think it can be significantly simplified.  See https://github.com/rustyrussell/bitcoin/tree/bip-9-versionbits
  6 2015-10-29 05:29:32 <rusty> Just got the unit tests to compile now: almost certainly terminally broken.
  7 2015-10-29 05:32:03 <rusty> It also doesn't do any caching, that's a separate patch on top/
  8 2015-10-29 05:33:14 <CodeShark> hmm
  9 2015-10-29 05:35:07 <CodeShark> you mean using a table?
 10 2015-10-29 05:35:17 <CodeShark> is that the crux of the idea?
 11 2015-10-29 05:39:52 <rusty> CodeShark: that's a big part, yes.  Also having BIP be an object, so you can ask if it's active fairly trivially.
 12 2015-10-29 05:40:08 <rusty> CodeShark: the table for versionbits makes sense, so you can add them trivially.
 13 2015-10-29 05:40:44 <CodeShark> I decided against treating the BIPs as objects because the state of the BIP depends on the blockchain branch
 14 2015-10-29 05:41:09 <sipa> agree - separate code and data
 15 2015-10-29 05:42:19 <sipa> (this is a very uninformed comment, and may or may not be relevant for the code being discussed)
 16 2015-10-29 05:43:05 <rusty> sipa: That's OK, you're allowed to be wrong :)   virtual bool Active(const CBlockIndex* pblockIndex) const is the only BIP method.
 17 2015-10-29 05:43:38 <CodeShark> the state of the BIP is a property of the block - I instead was thinking of the deployments themselves as describing the transition rules
 18 2015-10-29 05:43:46 <CodeShark> not the state
 19 2015-10-29 05:44:28 <sipa> rusty: you expect these BIP objects being instantiated in main/consensus logic then?
 20 2015-10-29 05:44:31 <sipa> as globals?
 21 2015-10-29 05:44:44 <rusty> sipa: well, probably softforks.cpp (the new file), but yeah.
 22 2015-10-29 05:45:32 <rusty> sipa: for versionbits, (aka struct VersionBitsBIP: public BIP) this is most logical.
 23 2015-10-29 05:46:00 <CodeShark> you can describe the transition rules using tables, like you've done - and perhaps that can simplify the description in some ways for the specific cases we're interested in...but we lose polymorphic flexibility
 24 2015-10-29 05:46:06 <CodeShark> perhaps that's not so important, though
 25 2015-10-29 05:47:35 <CodeShark> we can still wrap this mechanism in an abstract interface, I suppose
 26 2015-10-29 05:48:03 <sipa> i'll let you guys come up with something, and review when you agree :p
 27 2015-10-29 05:48:06 <rusty> CodeShark: yeah, we're going to want a C++ wrapper when this logic moves into libconsensus anyway.
 28 2015-10-29 05:48:37 <CodeShark> I'd love to hear your opinions in any case, sipa :)
 29 2015-10-29 05:50:00 <CodeShark> one of the main goals I had was hiding all the specifics of the deployment mechanism from the likes of main.cpp
 30 2015-10-29 05:50:24 <CodeShark> i.e. looking at main.cpp there's no trace whether a particular soft fork was deployed using ISM, versionbits, or something else
 31 2015-10-29 05:50:36 <sipa> rusty: i really don't care about the actual libconsensus API at this point; but good design will keep chain-specific data separate in chainstate, fork-enabled logic within consensus code, ...
 32 2015-10-29 05:51:21 <sipa> CodeShark: that's usually neat software engineering, but i'm not convinced it's something we should thrive for in consensus code... abstractions often hide what is actually going on
 33 2015-10-29 05:51:31 <sipa> (not convinced about the opposite either, to be clear)
 34 2015-10-29 05:52:02 <CodeShark> sipa: the main reasoning here was that we would be able to support further extensions or variations on the scheme or new schemes without having to touch main.cpp at all
 35 2015-10-29 05:52:26 <CodeShark> the logic would be in separate units
 36 2015-10-29 05:52:46 <sipa> having the decision logic for softforks itself separated is useful
 37 2015-10-29 05:53:03 <sipa> but the code enabled by specific forks... that is just consensus code; hiding it away is a bad idea imho
 38 2015-10-29 05:53:13 <CodeShark> oh, yeah - I wasn't suggesting that at all
 39 2015-10-29 05:53:26 <CodeShark> for now I've kept the actual consensus code itself in main.cpp
 40 2015-10-29 05:53:43 <CodeShark> it's only the decision of whether or not to apply a particular rule that has its logic in a separate unit
 41 2015-10-29 05:54:29 <CodeShark> by consensus code, meaning the actual enforcement of the rule itself
 42 2015-10-29 05:55:07 <CodeShark> or not just main.cpp - but whatever other units take part in checking the rules
 43 2015-10-29 05:55:17 <CodeShark> i.e. script stuff
 44 2015-10-29 06:00:43 <CodeShark> the decision logic should probably actually be part of the block tree index logic - each block header should have associated with it the rules that should be enforced - and the actual checking of blocks done once the block header has been added to the tree
 45 2015-10-29 06:02:35 <CodeShark> having a dual block index is sort of ugly...but I wanted to be as noninvasive as possible to existing structures
 46 2015-10-29 06:02:59 <sipa> i think the current CBlockIndex is very ugly because it does so many different things
 47 2015-10-29 06:03:30 <sipa> it has validation information, disk position information, block headers themselves, and proof-of-work related metadata (chainwork)
 48 2015-10-29 06:03:36 <CodeShark> right
 49 2015-10-29 06:04:42 <CodeShark> I'd like to redo much of that - but it would require substantial changes to consensus code
 50 2015-10-29 06:04:59 <sipa> first things first
 51 2015-10-29 06:08:28 <CodeShark> I'd go as far as not even having a CBlock that inherits from CBlockHeader
 52 2015-10-29 06:09:02 <CodeShark> and have a CBlockIndex that points to CBlockHeader instances
 53 2015-10-29 06:10:56 <CodeShark> with caching capabilities
 54 2015-10-29 06:12:12 <CodeShark> CBlock should actually consist of a pointer to a CBlockHeader and a transaction vector
 55 2015-10-29 06:15:55 <CodeShark> but in the interest of being as noninvasive as possible, I'm using a separate map from CBlockHeader* to rule states (which is not very pretty but requires only a handful of changes to consensus code)
 56 2015-10-29 06:16:27 <sipa> that sounds perfectly reasonable to me
 57 2015-10-29 06:33:52 <rusty> Dumb codingstle question: the p and n prefixes.. is there a list of semantics somwhere?
 58 2015-10-29 06:34:22 <rusty> And capitalization rules.
 59 2015-10-29 06:34:24 <Luke-Jr> rusty: one of the coding style docs, before it was removed
 60 2015-10-29 06:34:37 <Luke-Jr> p = pointer; n = number
 61 2015-10-29 06:34:42 <rusty> Luke-Jr: those damned moderators!  Oh, wait...
 62 2015-10-29 06:36:49 <Luke-Jr> rusty: it's called hungarian notation, but google didn't find the exactly flavour we had quickly
 63 2015-10-29 06:37:11 <rusty> Luke-Jr: yeah, I know it, but wondered what variation.
 64 2015-10-29 06:37:27 <Luke-Jr> it's not required anymore, in any case
 65 2015-10-29 06:38:19 <CodeShark> there's something to be said for prefixing pointers with p, but I always found the n to be a little superfluous :)
 66 2015-10-29 06:39:16 <CodeShark> I don't think it's necessary for the variable name to contain the type - pointers do make some sense because it helps with dereferencing
 67 2015-10-29 06:39:51 <CodeShark> and to remind you to delete and null
 68 2015-10-29 06:40:38 <Luke-Jr> pointers already have * and -> for that ;)
 69 2015-10-29 06:41:12 <CodeShark> yeah, point is that it tells you whether to use -> or .
 70 2015-10-29 06:41:27 <CodeShark> pobject->method(); vs. object.method();
 71 2015-10-29 06:51:56 <rusty> OK, tests pass.  Need to test alerts, add caching and clean up commits.  But I've pushed to branch.
 72 2015-10-29 06:52:39 <Luke-Jr> Core with Lightning builtin?
 73 2015-10-29 06:53:25 <aj> Luke-Jr: that would come under "add caching" i think...
 74 2015-10-29 06:53:45 <Luke-Jr> aj: at least it sounds close by!
 75 2015-10-29 09:10:35 <btcdrak> rusty: https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md
 76 2015-10-29 09:12:28 <rusty> btcdrak: thanks!
 77 2015-10-29 09:13:03 <btcdrak> rusty: Note your patch has tabs instead of 4xspaces indentation too.
 78 2015-10-29 09:13:16 <rusty> btcdrak: yeah, thanks.  I'll roll in a fix.
 79 2015-10-29 09:14:54 <rusty> sipa: Hmm, I want to make a minor mod to BIP 009.  It refers to GetMedianTimePast() of the block in the new period, it should be GetMedianTimePast() on the last block in the period.
 80 2015-10-29 09:15:14 <rusty> sipa: I thought GetMedianTimePast(x) was the median for the 11 blocks *before* x.  My bad...
 81 2015-10-29 11:10:14 <jtimon> rusty: re notation also f is for booleans (flag), as has been said it is no longer necessary, but I like that one
 82 2015-10-29 11:16:13 <jtimon> CodeShark: rusty sipa does https://github.com/bitcoin/bitcoin/compare/bitcoin:master...jtimon:softforks make sense to you as an skeleton for a bip9 implementation? what's new in chain.o (VersionBitsIndex) is unimplemented, as well as AreVersionBitsRecognized() and CalculateState() in softforks. Many things taken from CodeShark's branch but making it libconsensus-freindly (tm). Note that VersionBitsIndex is not used in softforks and
 83 2015-10-29 11:16:13 <jtimon> it's out of the consensus folder and namespace (I don't care about having it in chain.o or a new file, I was just being lazzy and didn't want to create a new file).
 84 2015-10-29 11:17:52 <jtimon> the most important parts are the Consensus::VersionBits::State struct and the modifications to Consensus::Params, I think that should be enough to implement all the functionality but feel free to propose more things to put there (no pointers though, please)
 85 2015-10-29 11:22:24 <jtimon> I created the consensus dir and Consensus namespace to help with libconsensus work, please don't mess with that: all files in the consensus dir should be almost-consensus-ready (tm), which means they only have unwanted dependencies that are trivial to solve like util.o, or they depend on CBlockIndex or CCoinsViewCache
 86 2015-10-29 11:24:20 <jtimon> please don't put anything that is not "almost-consensus-ready (tm)" in the consensus dir, if there's no hurry to complete libconsensus, then don't hurry in putting things in the consensus dir (although everything on my version of consensus/softfork.o is almost-libconsensus-ready)
 87 2015-10-29 11:29:14 <jtimon> I'm sure I can refactor CodeShark's branch to fit my skeleton without altering functionality (that's what I've been doing in small increments), but if we can agree on a skeleton as a common base, I think that would be very helpful for effective collaboration (and if it's that skeleton I would be already satisfied no matter how much C++ stuff you use in the implementation of AreVersionBitsRecognized(), CalculateState() and specially
 88 2015-10-29 11:29:14 <jtimon> VersionBitsIndex)
 89 2015-10-29 11:29:32 <jtimon> does that make sense to you guys?
 90 2015-10-29 11:31:51 <jtimon> in my libconsensus document with pictures I want to be able to say things like "everything in the consensus dir..." so please don't put VersionBitsIndex (or another statefull class) in the consensus dir
 91 2015-10-29 11:33:04 <jtimon> cfields: I'm interested in your opinion on this as well (since you've removed a lot of "statefull" dependencies from consensus code already)
 92 2015-10-29 13:32:58 <jbisch> wumpus: Regarding #6900. Don't know if you saw the initial Debian guest support in gitian-builder. Might be good to look at switching to Debian guests long term. Jessie has newer binutils and is supported until 2020 through Debian LTS initiative.
 93 2015-10-29 13:35:56 <wumpus> jbisch: haven't seen that - if gitian supports debian that might be something to consider
 94 2015-10-29 13:36:05 <treehug88> is there a document that explains the architecture of bitcoin core?
 95 2015-10-29 13:38:09 <jbisch> wumpus: It only is KVM atm. And the changes to vmbuilder that allow it to work haven't hit the official Ubuntu package yet. Will be in 16.04 soon. So people would need to install vmbuilder from source to build.
 96 2015-10-29 13:39:02 <wumpus> well once 16.04 is out, we can just as well use that
 97 2015-10-29 13:39:26 <wumpus> treehug88: not that I know
 98 2015-10-29 13:39:52 <treehug88> that's a shame. Thanks wumpus
 99 2015-10-29 13:40:47 <jbisch> wumpus: Yeah, sure. I was also thinking about Debian's reproducible build initiative. Might be good to look at switching anyway at some point, because of that.
100 2015-10-29 13:42:41 <mcelrath> treehug88: Just start with the wikis: e.g. https://en.bitcoin.it/wiki/Protocol_documentation and then https://en.bitcoin.it/wiki/Category:Technical
101 2015-10-29 13:45:50 <skyzer> I got bitcoin core strange behaviour again, can't figure out what to do
102 2015-10-29 13:46:33 <skyzer> https://live.blockcypher.com/btc/tx/649de2497dedcd5e655c30c9729b39310f3ac407e1cd156e590b2e94968b4a7b/ that transaction is double-spend, which was seen by my node. But the malleated transaction which got eventually confirmed c811ec was never seen by my node so I never received a walletnotify for that.
103 2015-10-29 13:47:02 <wumpus> you should have received it when it ended up in a block
104 2015-10-29 13:47:11 <skyzer> nope, did not receive
105 2015-10-29 13:47:50 <skyzer> I have discussed similar situation, gmaxwell or luke-jr told that the confirmed malleated transaction was seen first time in block with already 2 confirms, therefore I never received walletnotify on 0 or 1st confirm.
106 2015-10-29 13:48:05 <skyzer> I wonder how this can be sovled, that is popping up every week
107 2015-10-29 13:49:02 <wumpus> that sounds like a reorg - so you don't get notification for transactions that pop up in a reorg? it's possible...
108 2015-10-29 13:49:32 <skyzer> yeah, and people depositing to my web wallet are never being credited because of that
109 2015-10-29 13:49:43 <skyzer> hurts bitcoin, my website and so on
110 2015-10-29 13:49:45 <dansmith_btc> Running a node with -prune=550 i do with bitcoin-cli getblock 380968 (i checked with getinfo that this is the last downloaded block), the error i get is error: {"code":-5,"message":"Block not found"}. Weren't getblock was supposed to be removed for pruned mode?
111 2015-10-29 13:51:53 <wumpus> dansmith_btc: getblock works fine with pruning, just not with blocks that are pruned
112 2015-10-29 13:52:27 <wumpus> also getblock takes a hash, not a number, do 'getblockhash' first
113 2015-10-29 14:19:53 <danel> a
114 2015-10-29 18:25:34 <complexring> i'm trying to build bitcoin-qt from source.  but am getting a the following (similar) type errors:     "clang: error: unknown argument: '-framework QtNetwork' "
115 2015-10-29 18:26:01 <complexring> using a mac osx, el capitan.  installed Qt-5.6.0alpha from source
116 2015-10-29 18:26:57 <complexring> Qt frameworks are installed in /usr/local/Qt-5.6.0/lib  (default for Qt installed from source if taken from the git repo)
117 2015-10-29 18:27:14 <complexring> surely i am just missing some compiler flag to look for the frameworks in this directory ?
118 2015-10-29 18:28:37 <complexring> should be around for awhile.  let me know if anyone has suggestions.
119 2015-10-29 18:38:49 <complexring> also, appears to be an issue when calling `OBJCXX qt/bitcoin-qt' (which is sitting inside src).
120 2015-10-29 18:39:09 <complexring> anyway, conference talk time.  i'll leave this up if anyone can help
121 2015-10-29 18:58:55 <dstadulis> meeting happening soon, yes?
122 2015-10-29 18:59:11 <jgarzik> da
123 2015-10-29 18:59:41 <dstadulis> here's the google doc if there isn't anything else being used (meeting bot etc) https://docs.google.com/document/d/1t3kGkAUQ-Yui57P29YhDll5WyJuTiGrUhCW8so-E-iQ/edit?usp=sharing
124 2015-10-29 19:00:59 <sipa> present
125 2015-10-29 19:01:41 <petertodd> I managed to schedule a call... so I'm here, yet not here :)
126 2015-10-29 19:01:52 <jeremyrubin> I'm here for once, probably won't say anything tho
127 2015-10-29 19:02:13 <warren> here
128 2015-10-29 19:02:33 <petertodd> if anyone brings up my RBF patches, I'll look at the issues raised soon as I can, and mostly agree with them (though I want to keep the patch really simple - can be improved later)
129 2015-10-29 19:02:43 <wumpus> #meetingstart
130 2015-10-29 19:02:49 <evoskuil> here
131 2015-10-29 19:02:58 <wumpus> #startmeeting
132 2015-10-29 19:02:59 <lightningbot> Meeting started Thu Oct 29 19:02:58 2015 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
133 2015-10-29 19:02:59 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
134 2015-10-29 19:03:20 <Luke-Jr> .
135 2015-10-29 19:03:29 <wumpus> anything to discuss?
136 2015-10-29 19:03:37 <gmaxwell> Hm.
137 2015-10-29 19:03:58 <sipa> suggested topic: upcoming softfork?
138 2015-10-29 19:04:10 <wumpus> #topic upcoming softfork
139 2015-10-29 19:04:14 <morcos> suggeted topic: do we have sufficient agreement on new chain limts
140 2015-10-29 19:04:19 <gmaxwell> There were a couple things I wanted to discuss but my mind is blank at the moment.
141 2015-10-29 19:05:06 <gmaxwell> Are there other things not yet in the old release branches that we think need to be in the point releases for the softfork?
142 2015-10-29 19:05:27 <dstadulis> here's the google doc if desired https://docs.google.com/document/d/1t3kGkAUQ-Yui57P29YhDll5WyJuTiGrUhCW8so-E-iQ/edit?usp=sharing
143 2015-10-29 19:05:30 <Luke-Jr> gmaxwell: I have a PR open with 0.11 backports
144 2015-10-29 19:05:49 <Luke-Jr> but I plan to add more to it until the freeze
145 2015-10-29 19:06:05 <Luke-Jr> (at which point, I'll backport them to 0.10 too as applicable)
146 2015-10-29 19:07:27 <dcousens> is there any coordination in the softfork in regards to other clients? (btcd, bitcoinj, etc)
147 2015-10-29 19:08:23 <jtimon> here
148 2015-10-29 19:08:25 <gmaxwell> davec (btcd) had previously acked the schedule we'd proposed and IIRC it's ready for it.
149 2015-10-29 19:08:31 <warren> I could be wrong, but bitcoinj didn't implement all the previous validation rules.
150 2015-10-29 19:08:39 <sipa> bitcoinj doesn
151 2015-10-29 19:08:47 <gmaxwell> dcousens: historically bitcoinj has not implemented any softforks.
152 2015-10-29 19:08:47 <sipa> bitcoinj doesn't implement any validation rules, afaik
153 2015-10-29 19:08:55 <Luke-Jr> I saw a PR for bitcoinj to pay attention to the block versions
154 2015-10-29 19:08:58 <gmaxwell> sipa: well it has the not really maintained validation support.
155 2015-10-29 19:09:20 <sipa> so what exactly is under consideration: just CLTV?
156 2015-10-29 19:09:59 <morcos> +1 to CLTV alone
157 2015-10-29 19:10:08 <petertodd> +1
158 2015-10-29 19:10:08 <warren> Also for context, what is on the soft fork wish list for the medium term?
159 2015-10-29 19:10:22 <gmaxwell> sipa: I'm not following your question; as you should be aware we'd planned on doing releases with the CLTV soft fork for the 'end of the month'.  No other soft-forks are remotely ready right now.
160 2015-10-29 19:10:30 <gmaxwell> (we also discussed this last week)
161 2015-10-29 19:10:31 <petertodd> i mean, +1 to CLTV alone
162 2015-10-29 19:10:45 <sipa> gmaxwell: that's indeed my assumption, but i want to make sure that's what we're talking about :)
163 2015-10-29 19:10:57 <gmaxwell> I think the mediantime backports haven't been merged in the backport branches yet.
164 2015-10-29 19:11:12 <dcousens> +1 to CLTV alone, CSV is still just complicated enough it may warrant a more in-depth testing first
165 2015-10-29 19:11:14 <sipa> mediantime needs standardness first, right?
166 2015-10-29 19:11:17 <gmaxwell> And thats something I wanted to see in (as a standardness only rule of course)
167 2015-10-29 19:12:01 <gmaxwell> sipa: all thats implemented (and merged in master) is the standardness.
168 2015-10-29 19:12:19 <sipa> ok, so that needs backports too
169 2015-10-29 19:12:24 <Luke-Jr> are we backporting policy in general, or only future-softfork policies?
170 2015-10-29 19:12:29 <gmaxwell> dcousens: That was already discussed last week.
171 2015-10-29 19:12:44 <dcousens> gmaxwell: ok
172 2015-10-29 19:12:48 <gmaxwell> sipa: it's backported at least to 0.11.1 but not merged.
173 2015-10-29 19:12:56 <sipa> ok
174 2015-10-29 19:13:09 <dcousens> sipa: https://github.com/bitcoin/bitcoin/pull/6884 (for 0.11)
175 2015-10-29 19:14:17 <Luke-Jr> (for reference, I'm collecting bugfixes in https://github.com/bitcoin/bitcoin/pull/6825 )
176 2015-10-29 19:14:35 <Luke-Jr> does anyone know the answer to https://github.com/bitcoin/bitcoin/pull/6825#issuecomment-147972253 ?
177 2015-10-29 19:16:39 <jeremyrubin> that sure looks like a bugfix...
178 2015-10-29 19:16:50 <gmaxwell> Luke-Jr: ask petertodd
179 2015-10-29 19:16:56 <gmaxwell> We're straying offtopic severely now.
180 2015-10-29 19:17:36 <Luke-Jr> sorry, thought we were falling short on having topics to discuss
181 2015-10-29 19:17:54 <gmaxwell> Okay so beyond medianpast some effort needs to go in and pull in any straggling bugfixes, then a RC for a 0.11.2 / 0.10.x update?
182 2015-10-29 19:18:23 <sipa> ok
183 2015-10-29 19:18:28 <mcelrath> I'd like to discuss leveldb replacement briefly.
184 2015-10-29 19:19:08 <Luke-Jr> imo better to keep that topic until there's a viable replacement.
185 2015-10-29 19:19:15 <gmaxwell> We can talk about that subject; but I want to be clear. At this time we are not considering replacing leveldb in Bitcoin core.
186 2015-10-29 19:19:33 <mcelrath> I want to ask if anyone else is making a branch for testing with an alternative.
187 2015-10-29 19:19:45 <gmaxwell> (People trying out some other things is great and should always continue.)
188 2015-10-29 19:19:53 <mcelrath> I volunteer to make a LMDB branch.  jgarzik already has a sqlite branch.  We need to make tests.
189 2015-10-29 19:20:04 <jgarzik> 32-bit is a big issue
190 2015-10-29 19:20:10 <sipa> #topic leveldb replacement
191 2015-10-29 19:20:26 <gmaxwell> Holy crap.
192 2015-10-29 19:20:35 <jgarzik> LMDB simply doesn't work on 32-bit
193 2015-10-29 19:20:40 <gmaxwell> stop.
194 2015-10-29 19:20:49 <mcelrath> Why is 32-bit such a big deal?
195 2015-10-29 19:21:07 <mcelrath> Want to keep running on raspbery pis?
196 2015-10-29 19:21:30 <gmaxwell> I don't think that I was clear enough. We are _NOT_ replacing leveldb at this time. People should happily test other things, the code is abstracted in a way that specifically facilitates that. Have fun, report results.
197 2015-10-29 19:21:48 <jgarzik> One of my side projects is a from-scratch COW ordered map dbm https://github.com/jgarzik/pgdb2
198 2015-10-29 19:22:07 <sipa> ok, discussion around exploring that seems to be happening now
199 2015-10-29 19:22:09 <mcelrath> gmaxwell: that is clear, just want to make sure I'm not duplicating someone's effort.
200 2015-10-29 19:22:34 <gmaxwell> The thing that spurred this discussion is a specific bug on windows, which people should stop being shitty developers and fix if they care about and have access to windows. Not run off seeking magic beanstalks.
201 2015-10-29 19:23:00 <mcelrath> Also having db code in the core and having to maintain it is not really a good idea.
202 2015-10-29 19:23:03 <sipa> seems magic beanstalks don't exist at this point, but people are welcome to look for them :)
203 2015-10-29 19:23:06 <Luke-Jr> I suggest we end the meeting early and discuss any alternative db stuff off-meeting. :P
204 2015-10-29 19:23:12 <morcos> chain limits?
205 2015-10-29 19:23:18 <Luke-Jr> mcelrath: 32-bit is the future~ (maybe)
206 2015-10-29 19:23:22 <sipa> #topic chain limits
207 2015-10-29 19:23:26 <Luke-Jr> mcelrath: no matter what we use, we will need to maintain it
208 2015-10-29 19:23:42 <morcos> mostly i want to know what we're looking for to have sufficient consensus to merge them, and maybe i can help try to make that happen, or panic if its not going to
209 2015-10-29 19:23:51 <Luke-Jr> (deferring further discussion on db stuff until post-meeting)
210 2015-10-29 19:23:55 <dcousens> sipa: is there any references to data on average chain length in blocks thus far?
211 2015-10-29 19:24:02 <mcelrath> Luke-Jr: would be nice if it had an upstream maintainer and we weren't the only user. (defferring now to post-meeting)
212 2015-10-29 19:24:03 <gmaxwell> mcelrath: You misunderstand the purpose of the database in the system. This is not a generic database, but a integral part of the implementation of the consensus algorithim. Some challenge cannot be escaped there.
213 2015-10-29 19:24:03 <morcos> i think some people will be angry, but i dont' see an alternative
214 2015-10-29 19:24:05 <sipa> dcousens: good question!
215 2015-10-29 19:24:10 <jgarzik> https://en.wikipedia.org/wiki/LevelDB#Bugs_and_Reliability "LevelDB is widely noted for being unreliable and databases it manages are prone to corruption."
216 2015-10-29 19:24:12 <morcos> dcousens: i posted stats in the pull
217 2015-10-29 19:24:25 <dcousens> morcos: link? (haven't got it open)
218 2015-10-29 19:24:51 <gmaxwell> And if ever something like commited txouts is implemented no off the shelf tool is going to work in any case. So thats just something to get used to there.
219 2015-10-29 19:25:24 <morcos> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-October/011401.html
220 2015-10-29 19:25:36 <morcos> https://github.com/bitcoin/bitcoin/pull/6771
221 2015-10-29 19:26:06 <morcos> i guess i didn't post all my data, but i have %s of txs that were > than chains of certain lengths for various numbers
222 2015-10-29 19:26:30 <gmaxwell> jgarzik: Not consistent with our expirence and extensive testing, outside of windows. (I'm familar with the usenix paper, and brought it to our attention when it came out); but among other things our durability requirement is somewhat unusual.
223 2015-10-29 19:27:06 <gmaxwell> oh I missed the chain limits switch.
224 2015-10-29 19:27:08 <dcousens> morcos: it might be interesting to see the average chain length per block as a graph over time
225 2015-10-29 19:27:33 <dcousens> that will only give us an idea of what was accepted by miners though, and not reflective of the mempool, but, still a good insight?
226 2015-10-29 19:27:37 <sipa> morcos: are these numbers on mempool acceptance, or blocks?
227 2015-10-29 19:27:41 <morcos> dcousens: i'm not sure average is a good measure?  there were some very long chains during some of the spam attacks
228 2015-10-29 19:27:54 <sipa> morcos: i would expect that actual blocks don't contain many long chains, even if the mempool usually does
229 2015-10-29 19:28:03 <dcousens> morcos: overlayed with peak maybe?
230 2015-10-29 19:28:10 <morcos> sipa: mempool acceptance (however historically i don't know that anythign got into mempools that didn't end up in blocks, but yes split over multiple blocks maybe)
231 2015-10-29 19:28:40 <morcos> dcousens: oh is that what you were asking about, longest chain in a block, i didn't measure that
232 2015-10-29 19:29:21 <sipa> morcos: assuming retransmits, all that matters is mempool chains that actually get into a single block
233 2015-10-29 19:29:25 <dcousens> morcos: you could also roll the chain window to evaluate over say 3 blocks, assuming the mempool might have been split over that
234 2015-10-29 19:29:36 <dcousens> sipa: true
235 2015-10-29 19:29:53 <gmaxwell> dcousens: that roll check is not very useful, because wallet behavior changes radically at 1 confirm.
236 2015-10-29 19:30:27 <Luke-Jr> (suggested next topic: block size replacement with cost limits)
237 2015-10-29 19:30:35 <jeremyrubin> btw I'm working on some tools for bitcoin anlaysis on top of spark, I imagine they will help at some point with above analysis. Not cam ready near term, but happy to share what I have if anyone needs that kind of thing sooner.
238 2015-10-29 19:30:42 <gmaxwell> E.g. bitcoin core tries _very_ hard to avoid ever creating an unconfirmed chain, but does less avoidance of spends at 1 confirm.
239 2015-10-29 19:30:43 <dcousens> gmaxwell: indeed, so maybe just average/peak over blockheights, might be worth seeing for insight on limits?
240 2015-10-29 19:30:52 <dstadulis> dcousens why not a histogram?
241 2015-10-29 19:31:28 <dcousens> dstadulis: I suppose once we have the data, there are various ways we could visualize it :)
242 2015-10-29 19:31:50 <morcos> ok i can do that i guess.  do you think that puts this to bed
243 2015-10-29 19:31:57 <gmaxwell> In any case, 25 is very high.  And the recent malleability attacks pretty much would guarentee someone going more than a few deep would be haing a very bad day.
244 2015-10-29 19:32:33 <sipa> my assumption too is that 25 is very high
245 2015-10-29 19:32:45 <jgarzik> +1
246 2015-10-29 19:32:51 <dgenr8> +1 to the more restrictive (but still not very restrictive at all) chain limits
247 2015-10-29 19:32:52 <sipa> but if possible at all, seeing data of how many such chains exist _within a single block_ would be useful
248 2015-10-29 19:33:12 <gmaxwell> From an algorithimic perspective we cannot reasonably support unbounded depth (at least not right now), and depth to the point where it can't even fit in a block undermines our attack prevention assumptions.
249 2015-10-29 19:33:14 <morcos> sipa: ok, will report back on pull, might be a few days though
250 2015-10-29 19:33:31 <dcousens> +1, 25 is too high
251 2015-10-29 19:33:59 <morcos> dcousens: i'm happy with lower
252 2015-10-29 19:34:04 <dcousens> morcos: pm if you need any help
253 2015-10-29 19:34:41 <gmaxwell> there were, unfortunately people complaining even at this patch. In any case, we could always go with a more conservative number now and lower in the future.
254 2015-10-29 19:34:56 <jtimon> do the descendant limits need to be as low as the ascendant limits?
255 2015-10-29 19:35:28 <morcos> jtimon: yeah i wanted 10 for ancestor and 25 for descendant, but seems people are most concerned about the straight line chians
256 2015-10-29 19:35:36 <morcos> and felt 10 was too small
257 2015-10-29 19:35:41 <gmaxwell> I think that if we could make measurements now (e.g. not for the flooding attacks) we'd find lower numbers on account of the malleability attacks.
258 2015-10-29 19:36:16 <morcos> gmaxwell: i can measure over a long time period, but i'll just do the max chain in a block, b/c otherwise it becomes confusing how you count the 10 txs that are all part of a chian of 10
259 2015-10-29 19:36:39 <sipa> morcos: sounds good
260 2015-10-29 19:36:45 <gmaxwell> morcos: okay, though max is a crummy statistic.
261 2015-10-29 19:37:14 <morcos> but if its over a lot of blocks, i think it'll be useful, i seriously doubt most blocks have signficant chians at all
262 2015-10-29 19:37:35 <gmaxwell> e.g. a single 'blockchaing gambling' thing can keep that perpetually inflated, but its probably worth looking at.
263 2015-10-29 19:37:44 <dcousens> gmaxwell: what else can we use? in the end it is max we are limiting?
264 2015-10-29 19:37:55 <gmaxwell> dumping a txid for the longest chain would be nice, so we could go see what it appears to be.
265 2015-10-29 19:38:08 <gmaxwell> dcousens: we are not limiting the max of a _block_
266 2015-10-29 19:38:27 <dcousens> true, but, its likely it will have that effect
267 2015-10-29 19:38:36 <asyn> hi everyone. I was thinking about OP_CHECKLOCKTIMEVERIFY. Will it be possible to send a transaction that freezes funds until a certain block height to any address? Wouldn't that be a problem for exchanges, when they receive funds, credit them but won't be able to spend the funds until a certain block height is reached?
268 2015-10-29 19:38:36 <asyn> Or would using CLTV change the resulting output in a way that it won't result in the deposit address the exchange presented?
269 2015-10-29 19:38:53 <jgarzik> asyn, off topic for meeting
270 2015-10-29 19:38:55 <gmaxwell> asyn: #bitcoin I'll answer you there.
271 2015-10-29 19:39:06 <asyn> sorry
272 2015-10-29 19:39:17 <gmaxwell> dcousens: what I think would be more interesting is how many 'chains' in a block fail the test as a function of the limit. But I don't want to waste morcos' time with a bunch of measurement.
273 2015-10-29 19:39:49 <morcos> gmaxwell: i'll be wasting my time figuring out how to even calculate any of this. :)  i'll report all the stats i can reasonably gather
274 2015-10-29 19:41:54 <dcousens> suggested topic, clang format testing, [strictly] enforce style unit testing or not AND jgarzik's suggestion RE a window for those changes to occur initially
275 2015-10-29 19:41:55 <gmaxwell> morcos: sounds good. Sorry, for my care I'm fine with what you have so far, but process wise, we can avoid some controversy by doing a little more homework I think.
276 2015-10-29 19:42:29 <morcos> gmaxwell: yep, i asked for it. :)
277 2015-10-29 19:43:18 <gmaxwell> #topic clang format
278 2015-10-29 19:43:29 <gmaxwell> dcousens: what is the proposal here?
279 2015-10-29 19:43:51 <dcousens> gmaxwell: https://github.com/bitcoin/bitcoin/pull/6839#issuecomment-151663851
280 2015-10-29 19:44:10 <gmaxwell> Clang format behavior changes "randomly" from version to version.
281 2015-10-29 19:44:15 <jgarzik> History review:  Proposal a while ago was to clang-format file set <a b c ...>   Once done, maintain those files' formatting with automation (git hook checks or whatnot)
282 2015-10-29 19:44:37 <dcousens> gmaxwell: worth locking down a version then?
283 2015-10-29 19:44:40 <jgarzik> +ends coding style complaints     -causes increase in cosmetic traffic
284 2015-10-29 19:44:52 <gmaxwell> This would mean that every contributor ould need the same copy of some selected version of clang format, which seems pretty burdensome to me.
285 2015-10-29 19:44:55 <jgarzik> ends lots of bike shedding and Diapolo (I love you) patches
286 2015-10-29 19:45:06 <jtimon> maybe we need to lock in a version for consistent automation, yes
287 2015-10-29 19:45:09 <gmaxwell> jgarzik: will not end coding style complaints, but might help some of the most uninteresting ones.
288 2015-10-29 19:45:10 <btcdrak> omg I am late
289 2015-10-29 19:45:26 <btcdrak> the clocks went back.
290 2015-10-29 19:45:33 <jgarzik> gmaxwell, it won't end them but we can ignore with greater alacrity ;p
291 2015-10-29 19:45:37 <gmaxwell> it's USes turn nxt week.
292 2015-10-29 19:45:59 <sipa> i really like the idea of a consistent style
293 2015-10-29 19:46:19 <gmaxwell> So-- before we do that we need to have some discussion about style which shouldn't be in this meeting. We use several coding styles which are knon to increase defect rate (e.g. unbraced statements).
294 2015-10-29 19:46:35 <mcelrath> One should then start by clang-formatting the entire repo.
295 2015-10-29 19:46:39 <dcousens> gmaxwell: indeed, this was merely a suggestion to see if its worth getting the ball rolling on that
296 2015-10-29 19:46:45 <wumpus> I really don't like clang-formatting the entire repo
297 2015-10-29 19:46:48 <Luke-Jr> gmaxwell: contributors only need clang-format if we don't write the code in the proper format initially
298 2015-10-29 19:46:49 <jtimon> but it would be nice if some automation was shared in the repo (with or without fixed version)
299 2015-10-29 19:46:58 <jgarzik> A related and relevant piece of this discussion (as noted in PR):  Picking a time window for coding style changes
300 2015-10-29 19:46:58 <jgarzik> wumpus, not at once, certainly
301 2015-10-29 19:47:18 <wumpus> I like code changes to, make actual changes to code, diff noise makes maintenance really annoying
302 2015-10-29 19:47:19 <jgarzik> We can do libconsensus, reformat, other changes inside this time window as discussed previously.
303 2015-10-29 19:47:27 <gmaxwell> jgarzik: time window suggests at once!
304 2015-10-29 19:47:35 <jgarzik> wumpus, they are separate pull obviously
305 2015-10-29 19:47:37 <wumpus> so I'd prefer to not do clang formatting to existing files
306 2015-10-29 19:47:51 <jgarzik> -1   that doesn't solve problems
307 2015-10-29 19:47:51 <Luke-Jr> is there *anything* that supports only looking at changes for style?
308 2015-10-29 19:48:01 <wumpus> or at all really
309 2015-10-29 19:48:05 <jtimon> ideally coding style would be respected with each PR and "code style fixes" would never be necessary (or at least would stop being necessary at some point)
310 2015-10-29 19:48:07 <mcelrath> -1 bikeshedding
311 2015-10-29 19:48:16 <jgarzik> gmaxwell, ?
312 2015-10-29 19:48:21 <gmaxwell> wumpus: a prudent path might be to spec out something for new files, which would also be acceptable on all. And consider a mass reformat for later if we're happy with the result.
313 2015-10-29 19:48:28 <dcousens> mcelrath: the idea is that will end the constant bikeshedding/nits on PRs
314 2015-10-29 19:48:36 <wumpus> I'd really like to not have a mass reformat
315 2015-10-29 19:48:38 <jgarzik> gmaxwell, if you mean all cosmetics are submitted within same time window - yes that is the point
316 2015-10-29 19:48:41 <rusty> As a newcomer, an existing well-defined style would be good.
317 2015-10-29 19:48:42 <wumpus> but supposedly I'm alone in that
318 2015-10-29 19:48:48 <jonasschnelli> wumpus : +1
319 2015-10-29 19:48:55 <jgarzik> gmaxwell, avoiding current situation of constant-rebase-hell was already discussed
320 2015-10-29 19:48:59 <jonasschnelli> what if we enforce clang format code style for new code?
321 2015-10-29 19:49:04 <gmaxwell> wumpus: I certantly don't want to multiple times.
322 2015-10-29 19:49:09 <jtimon> yep, just respect the style on the lines you are touching anyway and little by little the style will be more respected overall
323 2015-10-29 19:49:13 <Luke-Jr> jonasschnelli: does it support that?
324 2015-10-29 19:49:14 <wumpus> I think it's overkill for what amounts to just cosmetic details
325 2015-10-29 19:49:25 <dgenr8> wumpus: +1.  but if you want to make it harder to cherrypick stuff to XT, keep it up with that refactors
326 2015-10-29 19:49:30 <jonasschnelli> let's travis do a diff and check the clang format style of the diff?
327 2015-10-29 19:49:31 <jgarzik> gmaxwell, by extension cosmetics and code movement are rejected outside time window, leaving time for real development with less pain over time
328 2015-10-29 19:49:40 <Luke-Jr> dgenr8: please.
329 2015-10-29 19:49:43 <jgarzik> (general rule - there are always exceptions)
330 2015-10-29 19:49:48 <gmaxwell> Has anyone done the work to see if we can get identical objectfile verification after a clang format?
331 2015-10-29 19:49:48 <wumpus> dgenr8: I'm ok with *sensible* refactors that move around code to make it easier to maintain, but this code style stuff annoys me
332 2015-10-29 19:49:59 <jtimon> someone automating that (apply style only on the lines I've touched for this commit) could share the script
333 2015-10-29 19:50:07 <dcousens> gmaxwell: good question
334 2015-10-29 19:50:08 <jonasschnelli> gmaxwell: i think its possible to get the same output.
335 2015-10-29 19:50:25 <mcelrath> jtimon: that's probably impossible since context from lines outside the lines you touch decide the amount of whitespace.
336 2015-10-29 19:50:29 <jgarzik> IMO objectfile differences indicate clang-format bug...
337 2015-10-29 19:50:45 <Luke-Jr> dgenr8: if anything, we should be making forks *easier*, not harder. just because one fork is misbehaving is not a reason to discourage forks.
338 2015-10-29 19:50:57 <gmaxwell> jgarzik: not so, assertion macros pepper the code with line numbers. special measures must be taken to avoid the object file changing.
339 2015-10-29 19:51:08 <dcousens> wumpus: if your main concern is git blame history being lost, I'm sure we could squash all code changes such that it is only 1 commit deep
340 2015-10-29 19:51:21 <gmaxwell> git blame history is already mostly useless in bitcoin core.
341 2015-10-29 19:51:38 <wumpus> dcousens: that's not my only concern - I just hate all-over-the-place changes, and the more if they don't really achieve anything
342 2015-10-29 19:51:47 <jtimon> mcelrath you can 1) write down the lines you are changing 2) apply clang on the whole file 3) discard any change in the lines you weren't previously changing
343 2015-10-29 19:51:57 <wumpus> gmaxwell: so let's not make it any more useless.
344 2015-10-29 19:51:59 <jgarzik> gmaxwell, good point
345 2015-10-29 19:52:09 <wumpus> 'some windows are broken, let's break them all...'
346 2015-10-29 19:52:13 <jgarzik> 8 minutes left
347 2015-10-29 19:52:13 <jonasschnelli> format everything could ruing the history and make branches/PRs harder to diff. What are the benefits of a clang-format style of every piece of code we have?
348 2015-10-29 19:52:15 <jgarzik> other topics?
349 2015-10-29 19:52:24 <wumpus> agree jonasschnelli
350 2015-10-29 19:52:37 <btcdrak> jgarzik: I need to talk about #6312 and #6564 again
351 2015-10-29 19:52:40 <jtimon> my preference would be to apply purely cosmetic changes ONLY in that way (no time window required)
352 2015-10-29 19:53:12 <wumpus> people will still find purely cosmetic changes to do after clang-format, there's always something it doesn't do, or not yet
353 2015-10-29 19:53:15 <jonasschnelli> cosmetic changes might be applied when touching the code because of a refactoring / change.
354 2015-10-29 19:53:29 <mcelrath> jtimon: your algorithm discounts the possibility that the line numbers change.  e.g. if(foo) { bar; } becoming multiple lines.  it's not so obvious.
355 2015-10-29 19:53:49 <jonasschnelli> and we have already a clang-format checker: Diapolo. :-)
356 2015-10-29 19:54:08 <Luke-Jr> jonasschnelli: lol
357 2015-10-29 19:54:13 <dcousens> jonasschnelli: I think if this is to go forward, careful attention will need to be made as to the timing.  This wasn't an easy question for sure
358 2015-10-29 19:54:48 <jonasschnelli> agree
359 2015-10-29 19:54:57 <wumpus> if the goal is to stop nitpicking about those, then discourage that, not mangle the entire source code to give a few people too franctic about spaces their way
360 2015-10-29 19:55:02 <sipa> my observation: too little consensus about enforcing a consistent style
361 2015-10-29 19:55:13 <jtimon> mcelrath: true, not that simple, but it shouldn't be impossible with some weird git magic
362 2015-10-29 19:55:35 <jgarzik> let's move on to next topic
363 2015-10-29 19:55:39 <dcousens> sipa: the consensus is lacking over the how to initially enforce it
364 2015-10-29 19:55:45 <wumpus> any other topics?
365 2015-10-29 19:55:46 <dcousens> not over enforcement in general, AFAIK
366 2015-10-29 19:55:46 <jgarzik> ### 5 min
367 2015-10-29 19:55:53 <gmaxwell> Yes, I'd be fine with doing something in new files, though I don't know what style we'd use. Going further requires research that no one has done, like getting a determinstic object (which I know for a fact will not just work).
368 2015-10-29 19:55:58 <btcdrak> jgarzik: yes BIP68 and BIP112 implementations
369 2015-10-29 19:56:07 <sipa> dcousens: seems that gmaxwell for example is opposed to requiring a specific clang-format version; without that, there is no way to do it at all
370 2015-10-29 19:56:09 <btcdrak> wumpus:
371 2015-10-29 19:56:10 <dcousens> btcdrak: whats the topic specifically?
372 2015-10-29 19:56:19 <gmaxwell> dcousens: no, there is a lot more under determined.
373 2015-10-29 19:56:50 <gmaxwell> my concern about specific versions would be reduced if someone cooked up a way to make setting up an alien clang format (e.g. cfields dep builder work).
374 2015-10-29 19:57:09 <sipa> BIP68: i still dislike the fact that the checklocktime function has a clause for skipping missing inputs, under the assumption that those only happen in the wallet
375 2015-10-29 19:57:12 <dcousens> gmaxwell: ok, well, maybe something for another time then,  I'd like to grab a hard-copy of your concerns if possible though
376 2015-10-29 19:57:41 <btcdrak> #topic: BIP68 and BIP112 implementations
377 2015-10-29 19:57:49 <sipa> those behaviours should be separated; i think the function should be pure logic (which takes a list of heights/times, and the different call sites can provide that based on wallet, mempool or chainstate)
378 2015-10-29 19:58:21 <btcdrak> sipa: have you reviewed the latest PRs for #6312?
379 2015-10-29 19:58:35 <morcos> sipa: i haven't dived into it yet, but i agree
380 2015-10-29 19:58:47 <petertodd> sipa: ACK, the skipping missing inputs worries me a lot
381 2015-10-29 19:58:48 <sdaftuar> i dived into before and i also agree with sipa
382 2015-10-29 19:59:16 <sipa> btcdrak: it still does that
383 2015-10-29 19:59:37 <btcdrak> It would be helpful to comment on the PR
384 2015-10-29 19:59:44 <sipa> btcdrak: i've commented multiple times about it
385 2015-10-29 20:00:22 <sipa> mark changed the behaviour to making missing inputs cause failure; i commented that that broke the wallet, and then it was reverted to the preexisting skip behaviour
386 2015-10-29 20:00:56 <morcos> i'd love to get BIP68 and BIP112 merged, but without maaku championing it and being responsible for the code, i think we need to be even more cautious
387 2015-10-29 20:01:15 <morcos> btcdrak: i'm not sure how much responsbility you are taking for it?
388 2015-10-29 20:01:17 <btcdrak> well it's important to establish what is really needed to get these merged
389 2015-10-29 20:01:40 <morcos> but i think its important that someone takes up the mantle
390 2015-10-29 20:01:43 <dstadulis> FYI we're at the one hour mark (keep it going for those who can stick around. if people could take a quick read through of the draft of the meeting notes and tweak and add content where they see fit that would help improve coverage and productivity of the meeting: https://docs.google.com/document/d/1t3kGkAUQ-Yui57P29YhDll5WyJuTiGrUhCW8so-E-iQ/edit
391 2015-10-29 20:01:46 <jgarzik> morcos, I think btcdrak is the sole maintainer/mantle now
392 2015-10-29 20:01:46 <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-dev/2015/bitcoin-dev.2015-10-29-19.02.log.html
393 2015-10-29 20:01:46 <lightningbot> Meeting ended Thu Oct 29 20:01:46 2015 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
394 2015-10-29 20:01:46 <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-dev/2015/bitcoin-dev.2015-10-29-19.02.html
395 2015-10-29 20:01:46 <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-dev/2015/bitcoin-dev.2015-10-29-19.02.txt
396 2015-10-29 20:01:46 <wumpus> #endmeeting
397 2015-10-29 20:01:50 <jgarzik> since maaku is out
398 2015-10-29 20:01:51 <btcdrak> I also saw some complaints that it's hard to review both #6312 and  #6564 separately, so I created a combined branch
399 2015-10-29 20:02:14 <btcdrak> https://github.com/bitcoin/bitcoin/compare/master...btcdrak:sequenceandcsv
400 2015-10-29 20:02:56 <btcdrak> morcos: I'm doing my best to get it polished. I was under the impression that the nits for 6312 were addressed
401 2015-10-29 20:03:23 <rusty> sipa: I like the concept of the patch, but I'll have to review and see what a separation looks like.
402 2015-10-29 20:03:48 <btcdrak> so the question is if there should be one combined PR with both 6312 and 6564 as one, or to keep the PRs separated
403 2015-10-29 20:05:24 <jtimon> I think bip68 goes first and then bip112 or both together, but I don't think bip112 makes much sense without bip68
404 2015-10-29 20:05:44 <sipa> bip112 is certainly useless without 68
405 2015-10-29 20:05:49 <rusty> btcdrak: separate is good.  They're conceptually related, but it's nice that they split into two parts so cleanly.  Certainly reduces review burden.  Agreed with jtimon.
406 2015-10-29 20:05:52 <btcdrak> sipa: how is 6564 in your eyes? It certain seems more straightforward, but it also doesnt make sense without 6312
407 2015-10-29 20:05:58 <jgarzik> separate-but-dependent
408 2015-10-29 20:06:09 <sipa> btcdrak: i haven't looked at it; we need 6312 first
409 2015-10-29 20:06:15 <jtimon> btcdrak: anyway, that's a question of review and you already created the branch with both to help review, right?
410 2015-10-29 20:06:48 <btcdrak> I created a branch linked above yes, so people who want to review it as one can do so easily
411 2015-10-29 20:07:01 <btcdrak> I'll keep that rebased if we make any changes to 6312
412 2015-10-29 20:08:11 <btcdrak> My hope is that once we get those nits fixed, we can get them merged and it can sit in master like CLTV did (renamed to OP_HODL by the way), pending a later softfork.
413 2015-10-29 20:08:56 <sipa> why HODL?
414 2015-10-29 20:09:20 <jgarzik> because it's internet-cute like kittens
415 2015-10-29 20:09:31 <jgarzik> (seriously: it's a dumb rename)
416 2015-10-29 20:09:32 <jtimon> btcdrak: thank you, to me it's helpful
417 2015-10-29 20:09:38 <btcdrak> sipa: it's a reddit joke reference. Someone asked how to lock their coins
418 2015-10-29 20:10:03 <btcdrak> sipa: then someone replied saying there is a new feature, but he forgot it's real name, then referred to a joke I'd made calling it OP_HODL.
419 2015-10-29 20:10:11 <sipa> eh ok
420 2015-10-29 20:10:25 <btcdrak> sipa: HODL reference: https://bitcointalk.org/index.php?topic=375643.0
421 2015-10-29 20:10:35 <btcdrak> but anyway :)
422 2015-10-29 20:12:43 <davec> dcousens: I wasn't around, but gmaxwell is correct.  In regards to the CLTV softfork, btcd already has the code in master for it
423 2015-10-29 20:14:43 <jtimon> versionbits?
424 2015-10-29 20:16:06 <jtimon> CodeShark and rusty are probably not around, but if anyone wants to nit the skeleton in https://github.com/bitcoin/bitcoin/compare/bitcoin:master...jtimon:softforks please do so
425 2015-10-29 20:16:23 <rusty> jtimon: am here...
426 2015-10-29 20:16:35 <jtimon> oh, great
427 2015-10-29 20:17:30 <rusty> jtimon: I found CodeShark's approach difficult to follow.  I have an alternate approach, and I'd like him to consider it.  I'm cleaning the commits now for pushing.
428 2015-10-29 20:18:54 <jtimon> ok, I want to review that too, but please have a look at that branch and tell me if it makes sense to you
429 2015-10-29 20:19:14 <jtimon> (when you can)
430 2015-10-29 20:21:28 <rusty> jtimon: my approach is completely different.  You get a global 'struct BIP', eg. BIP3000.  You ask BIP3000.IsActive(Chain *).
431 2015-10-29 20:22:17 <rusty> jtimon:  struct VersionBitsBIP inherits from struct BIP (we can do similar in C)
432 2015-10-29 20:22:34 <rusty> jtimon: I think that makes much, much more sense.
433 2015-10-29 20:23:33 <jtimon> why? you need an array with all the per-chain configurable stuff in Consensus::Params, which should be always available as a const ref in the relevant code
434 2015-10-29 20:23:46 <jtimon> and then another array for the states, done
435 2015-10-29 20:24:37 <jtimon> then you may want an index of those state arrays for given blockindexes for reorgs and stuff
436 2015-10-29 20:25:13 <jtimon> std::map<const CBlockIndex*, Consensus::VersionBits::State> ruleStateMap;
437 2015-10-29 20:25:36 <rusty> jtimon: a BIP can tell internally if it never applies to a chain.
438 2015-10-29 20:26:11 <sipa> BIPs are text, I think the structures should be called rules or forks :)
439 2015-10-29 20:26:14 <jtimon> "internally" it's just semantics, there's no need to encapsulate the attributes here
440 2015-10-29 20:26:21 <jtimon> a function can do just the same
441 2015-10-29 20:26:21 <rusty> sipa: yeah, but BIP is *short*
442 2015-10-29 20:26:31 <sipa> rusty: agree there :)
443 2015-10-29 20:26:45 <jtimon> CodeShark was using Softfork, which I maintain
444 2015-10-29 20:26:59 <rusty> jtimon: asking a bip if it is active at some point is the obvious and clear way to do it.
445 2015-10-29 20:27:20 <jtimon> but that doesn't contain the state (since Consensus::Params will get to you as const)
446 2015-10-29 20:27:20 <rusty> jtimon: exposing tables is a terrible ABI.
447 2015-10-29 20:27:21 <sipa> rusty: i'm not sure i like that overloading in consensus logic
448 2015-10-29 20:27:26 <Luke-Jr> rusty: but BIPs are not necessarily softforks.
449 2015-10-29 20:27:44 <sipa> rusty: though your approach seems pretty clean in general (and short!)
450 2015-10-29 20:28:08 <rusty> sipa: well, in C code it becomes a function pointer rather than a method.  But still trivial.
451 2015-10-29 20:28:24 <sipa> rusty: it would need to be BIP.IsActive(Chainparams&, CBlocKindex*), right?
452 2015-10-29 20:28:27 <sipa> rusty: same thing
453 2015-10-29 20:28:34 <sipa> rusty: it hides what is actually being done
454 2015-10-29 20:28:49 <jtimon> rusty you mean myClass.UseRule() is more obvious and clear than UseRule(myStruct) ?
455 2015-10-29 20:28:53 <rusty> sipa: well, we could make chainparams explicit; I'm using the Params() at the moment.
456 2015-10-29 20:29:04 <sipa> rusty: yeah you need to make chainparams explicit
457 2015-10-29 20:29:11 <rusty> sipa: why?
458 2015-10-29 20:29:15 <sipa> rusty: a library form won't have a global params you can ask
459 2015-10-29 20:29:27 <rusty> sipa: why not?
460 2015-10-29 20:29:35 <rusty> sipa: do you hate your users?
461 2015-10-29 20:29:36 <jtimon> rusty: because we're getting rid of the use of Params() (which is just hiding a global)
462 2015-10-29 20:29:40 <sipa> rusty: please
463 2015-10-29 20:29:49 <jtimon> see #5970
464 2015-10-29 20:29:50 <rusty> sipa: seriously, that's a horrible, horrible API.
465 2015-10-29 20:31:12 <rusty> sipa: avoiding a global does allow you to use the library simulatenously on multple different chains in a threaded program without a lock.
466 2015-10-29 20:31:42 <sipa> rusty: allow you to use multiple different chains simulataneously at all
467 2015-10-29 20:31:56 <jtimon> that's why we want to avoid Params() which is the global, const CChainParams& chainparams as a parameter on the other hand is not
468 2015-10-29 20:31:57 <rusty> sipa: no, you can trivially write a wrapper to override.
469 2015-10-29 20:32:08 <sipa> rusty: not in a threaded program
470 2015-10-29 20:32:16 <rusty> sipa: hence my comment above, yes.
471 2015-10-29 20:32:34 <rusty> sipa: pain for the tiny percentage of users who want to do that, but simplicity for everyone els.e
472 2015-10-29 20:32:57 <sipa> rusty: there is a bigger problem
473 2015-10-29 20:33:04 <rusty> sips/jtimon: is the consensus library going to need other state?
474 2015-10-29 20:33:17 <jtimon> apart from what?
475 2015-10-29 20:33:34 <rusty> jtimon: chainparams?  If so, you probably want a context arg instead.
476 2015-10-29 20:33:55 <sipa> rusty: that is that if the versionbits logic needs access to a Params() (regardless of how it is implemented), it depends on all of the chainparam's dependencies
477 2015-10-29 20:34:00 <jtimon> libconsensus only needs Consensus::Params, not CChainParams
478 2015-10-29 20:34:21 <sipa> rusty: if you want clean modularization, you can create a VersionBitsParams which is passed to the versiobits module
479 2015-10-29 20:34:47 <sipa> and the caller can extract that from the full structure chainparams, which contains versionbitsparams
480 2015-10-29 20:34:56 <jtimon> when you call libconsensus you provide a Consensus::Params
481 2015-10-29 20:35:49 <jtimon> yes, that would work as well, you can just put a VersionBitsParams in Consensus::Params as well
482 2015-10-29 20:35:49 <rusty> sipa: there's one value, so that seems overkill?
483 2015-10-29 20:36:18 <sipa> rusty: ok, so it can take that value as a parameter instead
484 2015-10-29 20:36:27 <sipa> rusty: without depending on all of chainparams
485 2015-10-29 20:36:28 <jtimon> rusty please read https://github.com/bitcoin/bitcoin/compare/bitcoin:master...jtimon:softforks#diff-bc9415451d733499187a3fe59e0fd7e6R77 it's not one value
486 2015-10-29 20:37:21 <jtimon> and if by "one value" you mean the current state for a given blockindex, the state shouldn't be in consensus/params
487 2015-10-29 20:37:40 <rusty> jtimon: https://github.com/rustyrussell/bitcoin/commit/393c87b58838be62000c34dd1bb6a392cecde828
488 2015-10-29 20:37:42 <jtimon> you need your own struct (array) for the current states for each bip
489 2015-10-29 20:38:30 <jtimon> that's one, what about SoftFork vDeployments[MAX_VERSION_BITS_DEPLOYMENTS]; ?
490 2015-10-29 20:39:14 <jtimon> and struct SoftFork {int32_t nBit;uint32_t nDeployTime;uint32_t nExpireTime; };
491 2015-10-29 20:39:29 <rusty> jtimon: you're looking at CodeShark's code.  Don't do that...
492 2015-10-29 20:39:55 <rusty> Child waking, got to go.  Code is done and tested, will push for review once I've squished commits.
493 2015-10-29 20:39:57 <jtimon> which is basically your BIP struct without the current state (which you don't need to duplicate)
494 2015-10-29 20:39:58 <jtimon> ???
495 2015-10-29 20:40:20 <jtimon> please look at my code
496 2015-10-29 21:09:27 <paulo_> hodl or sell?
497 2015-10-29 21:10:26 <btcdrak> paulo_: wrong channel #bitcoin-pricetalk
498 2015-10-29 21:10:49 <paulo_> oops thought this was price talk
499 2015-10-29 21:29:12 <complexring> hello all.  trying to compile bitcoin-qt on el capitan.  have a compiled qt-5.6.0 from source a la git (standard default install location), but am receiving the following error:  "fatal error: 'QLabel' file not found" -- i have passed the QTDIR environment variable and some includes to configure.  seems like i am missing something simple.  using clang.
500 2015-10-29 21:32:06 <jonasschnelli> complexring: did you try install Qt5.5 over homebrew?
501 2015-10-29 21:38:40 <complexring> error: unknown argument: '-framework QtNetwork'  " -- ditto for -framework QtNetwork, QtWidgets, QtGui, QtCore, QtDBus -- and i do pass the -F$QTLOCFRAMEWORKLOCATION flag as well.  this part fails on :  "  OBJCXXLD qt/bitcoin-qt  " , so clang doesn't like -framework flag being passed for objective c++ ?
502 2015-10-29 21:38:40 <complexring> jonasschnelli: i did use brew at some point, and it worked.  but then some other projects failed with the homebrew install, so i went with the option of getting the latest qt from their git repo to make qt work with them.  also, installing from the downloaded .dmg from qt's site worked as well.  in the git install, i can in fact pass the include directory that does contain QLabel, but then run into some clang errors saying :  "  clang:
503 2015-10-29 21:39:41 <jonasschnelli> complexring : ah right.. i also saw this problem on my system... had to go back to qt5.4
504 2015-10-29 21:39:42 <complexring> so, seems like might be an issue with installing qt from source on mac.  but all the files, etc. are installed somewhere (default location), and i would think i just am missing some flag to pass to the compiler.
505 2015-10-29 21:39:47 <jonasschnelli> cfields pointed me to https://github.com/Homebrew/homebrew/commit/fdbb338f2274e093b96209e7ca3ff9bf5460714e
506 2015-10-29 21:39:53 <complexring> o0
507 2015-10-29 21:40:03 <jonasschnelli> but didn't went down that path..
508 2015-10-29 21:40:20 <complexring> let me check that out.
509 2015-10-29 21:40:27 <cfields> yes, it's just a quoting problem
510 2015-10-29 21:40:42 <jonasschnelli> complexring: check this: https://gist.githubusercontent.com/UniqMartin/a54542d666be1983dc83/raw/f235dfb418c3d0d086c3baae520d538bae0b1c70/qtbug-47162.patch
511 2015-10-29 21:41:41 <complexring> so that patch should handle the broken pkg-config install?
512 2015-10-29 21:43:41 <complexring> this doesn't seem to fix the issue if i have qt-5.6.0alpha or does it?
513 2015-10-29 22:35:23 <warren> Other than #5610 and #5865, are people aware of prominent issues, forum or reddit posts where any substantive information about possible causes of the reported Windows db corruption is written?
514 2015-10-29 22:37:20 <warren> https://bitcointalk.org/index.php?topic=337294.0;all  I'm considering repeating something similar to this November 2013 bug bounty for the MacOS X leveldb corruption.
515 2015-10-29 22:37:53 <warren> I need to know if 1) the issue is real (it seems so) and 2) what information already exists about it before deciding if this is a good idea.
516 2015-10-29 22:46:20 <PRab> warren: I'm not sure how useful this is, but I have seen #5610 happen on 2 of 2 Windows machines that I have access to. I posted my log in that issue once, but have seen it occur more times.
517 2015-10-29 22:47:40 <warren> PRab: can you document how to reproduce it consistently?
518 2015-10-29 22:47:46 <warren> PRab: what version of windows?
519 2015-10-29 22:48:04 <PRab> Windows 7 on one machine and Windows 8 on the other.
520 2015-10-29 22:48:26 <PRab> Not 100% consistently, but most of the time a hard power down does it.
521 2015-10-29 22:48:48 <PRab> Usually I'm gone for a weekend, notice that the power went out, and then it needs to do a rebuild.
522 2015-10-29 22:49:30 <warren> PRab: do you have write caching enabled on the drive?
523 2015-10-29 22:50:01 <PRab> Whatever windows defaults to. Checking now.
524 2015-10-29 22:50:31 <warren> a Windows 8.1 machine here defaulted to write caching enabled, which surprised me
525 2015-10-29 22:51:42 <PRab> Write caching is enabled on my machine (Windows 7)
526 2015-10-29 22:52:40 <sipa> why would you not enable write caching on any OS?
527 2015-10-29 22:53:02 <sipa> even DOS 6 did...
528 2015-10-29 22:53:55 <warren> I guess it depends on the type of write caching if it's safe to do so or not.
529 2015-10-29 22:58:57 <btcdrak> is there an historical data of bitcoin full node counts?
530 2015-10-29 23:09:31 <dstadulis> btcdrak define historical data
531 2015-10-29 23:09:59 <dstadulis> btcdrak just a count plot over time?
532 2015-10-29 23:10:20 <btcdrak> plot count over time
533 2015-10-29 23:10:28 <btcdrak> ping CodeShark
534 2015-10-29 23:11:21 <CodeShark> hello :)
535 2015-10-29 23:11:38 <CodeShark> yes, what btcdrak said :)
536 2015-10-29 23:47:36 <warren> Has it been proposed to have a bitcoin.conf option that causes bitciond to refuse to run until the user edits the config and removes it?  I'm thinking about safe packaging standards for Fedora's bitcoin package and I am wondering if that would be a way to force people to configure the system service and not just blindly use it.
537 2015-10-29 23:48:14 <sipa> warren: you can run without bitcoin.conf now, safely
538 2015-10-29 23:48:51 <phantomcircuit> warren, why not just an annoying set of defaults? (server=0 for example)
539 2015-10-29 23:49:14 <sipa> bitcoind ignores server
540 2015-10-29 23:49:26 <warren> sipa: as a system service shipped in a distro, people want us to ship a systemd .service file. I'm personally against doing so but I may be forced into it.
541 2015-10-29 23:49:43 <warren> Thus I may want to force them to look at and configure things.
542 2015-10-29 23:49:57 <phantomcircuit> sipa, server=0 doesn't disable rpc?
543 2015-10-29 23:50:44 <sipa> phantomcircuit: you can't disabled rpc in bitcoind; -server is a bitcoin-qt option
544 2015-10-29 23:51:04 <phantomcircuit> ah
545 2015-10-29 23:51:05 <CodeShark> just automatically create a very long random password for the RPC
546 2015-10-29 23:51:10 <CodeShark> and stick it in the conf file
547 2015-10-29 23:51:11 <Belxjander> warren: assign it a non-created user so it always fails at boot until the user is created and the configuration is then to be generated
548 2015-10-29 23:51:27 <warren> crap, with sys V initscripts it was simple to auto-generate rpcuser and rpcpassword, I'm not sure how to do it with systemd
549 2015-10-29 23:51:36 <Belxjander> warren: systemd won't launch services without a "home" directory to run them from and fails the launch
550 2015-10-29 23:51:38 <sipa> warren: you don't need rpcuser and rpcpassword anymore
551 2015-10-29 23:51:53 <sipa> warren: bitcoind will create a cookie with a token that bitcoin-cli can read
552 2015-10-29 23:52:07 <warren> oh in datadir?
553 2015-10-29 23:52:10 <Belxjander> warren: I suggest looking at the znc.service file as one example of running a service within a specific user
554 2015-10-29 23:52:11 <sipa> yes
555 2015-10-29 23:52:29 <Belxjander> warren: that also fails until configured
556 2015-10-29 23:52:49 <gmaxwell> warren: we have autotokens now. actually, I wonder if the presence of autotokens shouldn't cause us to rename the rpcuser/rpcpassword configuration in 0.12.
557 2015-10-29 23:53:17 <warren> ok this is good
558 2015-10-29 23:53:29 <Belxjander> gmaxwell: separating to a new option name and marking those deprecated for later removal ?
559 2015-10-29 23:54:06 <gmaxwell> Belxjander: yes, since now most users should no longer be setting them.
560 2015-10-29 23:54:18 <Luke-Jr> gmaxwell: why shouldn't they?
561 2015-10-29 23:54:38 <gmaxwell> Luke-Jr: you should only be setting them if you want remote access.
562 2015-10-29 23:54:39 <Luke-Jr> actually, is it even possible for BFGMiner to easily use the autotokens? :x
563 2015-10-29 23:54:46 <gmaxwell> Luke-Jr: absolutely.
564 2015-10-29 23:55:34 <warren> Hmm, the auto-token might not work so well if they intend to query things from this bitcoind from things that aren't bitcoin-cli.
565 2015-10-29 23:56:07 <gmaxwell> warren: works fine for ntp and other services that do roughly the same thing-- including X11 (xauthority),
566 2015-10-29 23:56:19 <warren> I guess I add a README to configure RPC manually if they want that, to query from <random other app>
567 2015-10-29 23:56:26 <gmaxwell> warren: no.
568 2015-10-29 23:56:26 <Luke-Jr> oh, it's just user:pass in a file
569 2015-10-29 23:56:35 <gmaxwell> warren: they should go read the token.
570 2015-10-29 23:56:49 <gmaxwell> warren: and access is controlled by virtue of filesystem permissions.
571 2015-10-29 23:56:51 <Luke-Jr> hm, not so simple actually
572 2015-10-29 23:57:11 <warren> gmaxwell: filesystem permission makes it not simple, the random other app shouldn't be able to read anything in the datadir
573 2015-10-29 23:57:16 <Luke-Jr> right now I just read bitcoin.conf at startup, but that won't work for a token that changes every start
574 2015-10-29 23:57:26 <gmaxwell> Luke-Jr: you should do it at connect time.
575 2015-10-29 23:57:32 <Luke-Jr> gmaxwell: yes, that's why it's complex
576 2015-10-29 23:57:38 <gmaxwell> warren: this is a normal construction used by many daemons.
577 2015-10-29 23:57:45 <gmaxwell> Luke-Jr: move the functional call from site A to B.
578 2015-10-29 23:57:46 <Luke-Jr> gmaxwell: connect-time is for every share/gbt call :/
579 2015-10-29 23:57:57 <gmaxwell> Luke-Jr: so?
580 2015-10-29 23:58:15 <Luke-Jr> just ugly is all
581 2015-10-29 23:58:16 <gmaxwell> Luke-Jr: it won't be measurably different than opening the connection in the first place.
582 2015-10-29 23:58:42 <warren> which PR did autotokens go in?  I'd like to read the change
583 2015-10-29 23:58:48 <Luke-Jr> gmaxwell: right now, connection is only opened once; and that's inside libcurl
584 2015-10-29 23:59:06 <gmaxwell> Luke-Jr: great, you only need to read it when the connection is opened.
585 2015-10-29 23:59:13 <Luke-Jr> yes, but I don't know when that is :P
586 2015-10-29 23:59:35 <Luke-Jr> I suppose I track dead vs alive.. maybe I can mix it in there
587 2015-10-29 23:59:46 <sipa> Luke-Jr: then require a password based connection :)