1 2015-03-10 01:08:02 <kanzure> summed up yesterday's conversation about rpc management of notification hooks and webhooks, https://github.com/bitcoin/bitcoin/issues/5873
  2 2015-03-10 01:47:13 <leakypat> harding: any joy with volunteers for wallet reviewers?
  3 2015-03-10 02:49:03 <Luke-Jr> wumpus: IMO 5861 can be closed, up to you
  4 2015-03-10 04:30:34 <kanzure> vmatekole: fix your client :/
  5 2015-03-10 08:12:05 <b-itcoinssg> regarding the usage of OP_RETURN with a multi-signatures, do you see any problems in terms of pruning or implementation?
  6 2015-03-10 10:29:36 <jtimon> cfields I don't see how you can gt rid of the chainparam globals and still support multiple chains with different values in those constants...I mean apart from using the Params() singleton factory from less places...
  7 2015-03-10 10:42:37 <jtimon> anyway, utACKED #5871, merging it will make #5812 require rebase but that's fine (and expected)
  8 2015-03-10 10:45:45 <jtimon> probably I should rebase that one right away
  9 2015-03-10 12:37:46 <allejuppa> Was doing a "make deploy" on osx 10.10 with boost libs installed with homebrew
 10 2015-03-10 12:37:56 <allejuppa> and it turns out the boost dependencies are not correct in the appbundle
 11 2015-03-10 12:38:15 <allejuppa> boost_thread-mt.dylib is included, but that is dependent on boost_system-mt.dylib, not boost_system.dylib, which is included
 12 2015-03-10 12:38:30 <allejuppa> if I configure it to use boost_system-mt and boost_filesystem-mt, it all works but the defaults do not
 13 2015-03-10 12:43:55 <jonasschnelli> allejuppa, better try gitian build for release builds?
 14 2015-03-10 12:45:19 <allejuppa> indeed, but it does seem funky that an osx build on a normal osx does not produce a functioning appbundle
 15 2015-03-10 12:46:35 <jonasschnelli> allejuppa, maybe dive into gitian-osx.yml and check why the build there includes the boost dep. correctly
 16 2015-03-10 12:47:30 <jonasschnelli> i assume a standalone "make deploy" won't create a proper app bundle
 17 2015-03-10 12:47:59 <jonasschnelli> allejuppa, may i ask why your are trying to build a app bundle?
 18 2015-03-10 12:49:18 <allejuppa> just for the fun of it and better understanding of the buildsystem
 19 2015-03-10 12:50:05 <sipa> the actual release builds are all done on linux systems (including the builds for windows and osx)
 20 2015-03-10 12:50:34 <allejuppa> I know, have a gitian set up here as well
 21 2015-03-10 12:50:46 <sipa> ok
 22 2015-03-10 12:51:53 <allejuppa> did have to tweak it a bit to work with kvm, will post my modification back to the gitian repo
 23 2015-03-10 12:52:02 <allejuppa> not kvm, lxc I mean
 24 2015-03-10 12:52:32 <jonasschnelli> yes. would be nice. Never got lxc running over VMWare or VBox
 25 2015-03-10 12:52:51 <allejuppa> have it running with lxc on VBox
 26 2015-03-10 12:53:36 <sipa> i use lxc inside virtualbox
 27 2015-03-10 12:54:42 <jonasschnelli> hmm... always ended up by almost throwing out my macbook out of the window...
 28 2015-03-10 12:54:58 <jonasschnelli> maybe lxc and osx/mac won't harmonize that good.
 29 2015-03-10 12:55:22 <sipa> jonasschnelli: ubuntu on my hardware, virtual with debian inside, gitian lxc inside that
 30 2015-03-10 12:55:28 <sipa> i doubt the outer host matters
 31 2015-03-10 12:55:37 <sipa> *virtualbox
 32 2015-03-10 12:56:44 <jonasschnelli> sipa, i also thought the main host cannot affect lxc. But I followed multiple time wumpuses manual and also got help from michagogo, etc. Didn't works for me. Neither in VBox then over VMWare.
 33 2015-03-10 12:57:08 <jonasschnelli> now running gitian on debian wheezy. All fine.
 34 2015-03-10 12:57:51 <sipa> weird
 35 2015-03-10 12:58:09 <allejuppa> had to adapt this line, for the rest it worked perfectly:
 36 2015-03-10 12:58:10 <allejuppa> https://github.com/devrandom/gitian-builder/blob/master/libexec/copy-from-target#L53
 37 2015-03-10 12:58:52 <allejuppa> must say that it's been two months ago that I set it up
 38 2015-03-10 12:59:13 <sipa> change to what?
 39 2015-03-10 12:59:18 <sipa> that line changed recently
 40 2015-03-10 12:59:37 <sipa> https://github.com/devrandom/gitian-builder/commit/037d17aff733f1a156ff833cc659ca2c0d2024ac
 41 2015-03-10 13:01:32 <allejuppa> -    sudo $LXC_EXECUTE -n gitian -f var/lxc.config -- sudo -i -u $TUSER tar -C `dirname "$1"` -cf - `basename "$1"` | tar -C "$2" -xkf -
 42 2015-03-10 13:01:33 <allejuppa> +    sudo $LXC_EXECUTE -n gitian -f var/lxc.config -- sudo -i -u $TUSER tar -C `dirname "$1"` -cf - `basename "$1"` | tar -C "$2" -xf -
 43 2015-03-10 13:02:09 <allejuppa> so, added the -k option
 44 2015-03-10 13:02:45 <allejuppa> removed, sorry
 45 2015-03-10 13:02:56 <allejuppa> man, is been a while
 46 2015-03-10 13:18:54 <jonasschnelli> what is the sort order of mempool.mapTx or lets say of RPC "getrawmempool"?
 47 2015-03-10 13:21:13 <sipa> an std::map is a tree map
 48 2015-03-10 13:21:28 <sipa> so it orders based on the keys' ordering
 49 2015-03-10 13:21:31 <sipa> in this case, txid
 50 2015-03-10 13:21:49 <sipa> though txids are swapped before printing, i believe
 51 2015-03-10 13:23:26 <jonasschnelli> hmm... searching for a way for getting latests txs out of the mempool... so i assume i have to loop over and compare CTxMemPoolEntry.nTime?
 52 2015-03-10 13:48:41 <allejuppa> sipa: I removed the -k option since it failed on getting the (already) cached dependencies from the target container
 53 2015-03-10 13:50:19 <allejuppa> it now overwrites them again which is suboptimal, I know, but it does not fail on it
 54 2015-03-10 13:50:48 <sipa> cfields: ^
 55 2015-03-10 13:56:38 <jgarzik> wumpus, I'd like to put libevent http server on the 0.11 list
 56 2015-03-10 13:56:52 <sipa> ack
 57 2015-03-10 13:57:00 <wumpus> jgarzik: I'm not entirely convinced of it yet, but ok
 58 2015-03-10 13:57:29 <sipa> well, putting it on the list only indicates that we currently believe it should go in
 59 2015-03-10 13:57:34 <jgarzik> nod
 60 2015-03-10 13:57:35 <sipa> doesn't mean we can't change our mind
 61 2015-03-10 13:57:37 <wumpus> that's fine with me
 62 2015-03-10 13:58:14 <jgarzik> tested non-rebase version last night, was happy with the basics
 63 2015-03-10 13:58:23 <jgarzik> does it satisfy the miner objections RE keep-alive etc.?
 64 2015-03-10 13:58:33 <jgarzik> (along with recently added option)
 65 2015-03-10 13:58:38 <sipa> i also want univalue
 66 2015-03-10 13:59:14 <wumpus> my main gripe with libevent's http server is its lack of support for multiple threads, which I had to work around in my code with a few hacks to support the worker threads (this is not a limitation of libevent, just of their included http server)
 67 2015-03-10 13:59:33 <wumpus> yes, it supports everything
 68 2015-03-10 13:59:41 <wumpus> I just don't like the code very much :)
 69 2015-03-10 14:00:17 <jtimon> where is the 0.11 list ?
 70 2015-03-10 14:00:19 <jgarzik> ack.  Tree already converted to UniValue: https://github.com/jgarzik/bitcoin/tree/2014_univ_conv    Need to fix UniValue test breakage related to amount-related conversions.
 71 2015-03-10 14:00:32 <wumpus> maybe it's better than what there is, though
 72 2015-03-10 14:00:38 <wumpus> jtimon: milestones -> 0.11.0
 73 2015-03-10 14:00:40 <sipa> jtimon: 0.11 tag on github
 74 2015-03-10 14:01:09 <jgarzik> wumpus, indeed -- the libevent http server is useful, but has limits and is not nginx by any stretch
 75 2015-03-10 14:01:35 <sipa> we can hardly deal with multithreaded processing anyway :)
 76 2015-03-10 14:01:41 <sipa> (which is something to be solved independently)
 77 2015-03-10 14:01:44 <wumpus> sipa: true
 78 2015-03-10 14:02:00 <sipa> i absolutely agree we want multithreaded rpc/http
 79 2015-03-10 14:02:08 <jgarzik> Remember the RPC HTTP server is and has always been horribly bad :)   Not even async
 80 2015-03-10 14:02:09 <sipa> but at this point it's not all that useful
 81 2015-03-10 14:02:20 <jgarzik> it is a step forward
 82 2015-03-10 14:03:12 <wumpus> yes
 83 2015-03-10 14:05:05 <jtimon> I see, I wonder how much of the consensus/policy separation can get into 0.11...
 84 2015-03-10 14:06:55 <jonasschnelli> wumpus, #5677 could use a rebase. I could also test and report.... maybe the rebase is trivial. I didn't check.
 85 2015-03-10 14:07:17 <jonasschnelli> #5677 = new RPC/HTTP
 86 2015-03-10 14:07:32 <wumpus> yes, I'll look at rebasing it soon
 87 2015-03-10 14:09:26 <jonasschnelli> would #5677 also handles http timeout?
 88 2015-03-10 14:14:23 <jonasschnelli> Nevermind. It handles http timeout.
 89 2015-03-10 14:33:24 <wumpus> that was my initial motivation, even :)
 90 2015-03-10 14:35:58 <jonasschnelli> wumpus, okay. :)
 91 2015-03-10 14:37:44 <allejuppa> sipa: sorry, the same fix as I did was committed on the 15th of janurary on the gitian repo apparently
 92 2015-03-10 14:57:24 <jgarzik> jonasschnelli, it handles http timeout and many other abnormal cases much more gracefully than current hand-rolled bitcoind http server
 93 2015-03-10 14:59:34 <jonasschnelli> jgarzik, Should def. go into 0.11. I will do some tests soon.
 94 2015-03-10 15:00:44 <jonasschnelli> Maybe more basic http tests should be added to ensure RPC's underlaying implementation.
 95 2015-03-10 15:01:18 <jonasschnelli> I had some http-timeout and multithread tests in mind.
 96 2015-03-10 15:02:17 <Galaxor> Hi.  I am just curious about the broadcast network for bitcoin.  Documentation I read typically says a lot about the blockchain, but on the topic of broadcasting transactions or new blocks, it usually just says "it is broadcast to the network".  How does that work?
 97 2015-03-10 15:02:57 <Luke-Jr> Galaxor: basically just every peer tells every other peer what they have, and when a peer sees something new it requests it
 98 2015-03-10 15:03:19 <Galaxor> Nobody retransmits a broadcast?  Everybody has a direct connection to everyone else?
 99 2015-03-10 15:03:52 <Luke-Jr> Galaxor: no, usually peers just have 8-100 connections
100 2015-03-10 15:03:58 <sipa> Galaxor: every node relays to their peers after validation
101 2015-03-10 15:04:15 <Luke-Jr> every other peer they're connected to*
102 2015-03-10 15:04:24 <sipa> using a "gossip" mechanism: they tell their peers the hash of the new data they received, and the peer will ask for the actual data only once
103 2015-03-10 15:04:54 <Galaxor> Ohh, okay.  But the new hash could bounce around the network a bunch of times?
104 2015-03-10 15:05:25 <sipa> every peer only relays it only once
105 2015-03-10 15:05:28 <Galaxor> Like, you could theoretically get the same broadcast from each of the peers you're connected to?
106 2015-03-10 15:05:29 <sipa> so it just propagates
107 2015-03-10 15:05:37 <sipa> not could; will
108 2015-03-10 15:06:31 <Luke-Jr> peers don't keep track of what you know, so they just tell you when they get something new regardless
109 2015-03-10 15:06:38 <Galaxor> Like, if A is connected to (B C D), and E is connected to (B C D), but A is not connected to E, then if A makes a broadcast, E will get 3 copies of it:  One from each of (B C D)?
110 2015-03-10 15:07:49 <Galaxor> And if A, B, C, and D each have a connection to each other, then if A makes a broadcast, E will hear it from A, B, C, and D?
111 2015-03-10 15:08:48 <justanotheruser> Galaxor: theres no reason to send data redundantly
112 2015-03-10 15:10:40 <Luke-Jr> Galaxor: E will get 3 announcements that his peers have the data
113 2015-03-10 15:10:57 <sipa> and ask for the actual transaction from the first one he hears from
114 2015-03-10 15:11:02 <Luke-Jr> Galaxor: and E will only request the data from the first one he parses the announcement from, if he's well-behaving
115 2015-03-10 15:11:52 <Galaxor> Okay.  So the announcemnet will get sent redundantly but the data will not?
116 2015-03-10 15:11:58 <Luke-Jr> correct
117 2015-03-10 15:12:11 <Galaxor> Okay, I think I got it.  Thanks.
118 2015-03-10 15:12:33 <Luke-Jr> FYI, even if you don't like trying to read the C++ code, Wireshark has a dissector you can also use to study the protocol
119 2015-03-10 15:12:49 <Galaxor> Luke-Jr: Ooh, that sounds like fun.
120 2015-03-10 15:13:14 <Galaxor> Oh, also, I notice that you request the data from the peer you heard about it from, not the peer that initiated it?
121 2015-03-10 15:14:02 <Galaxor> So that would imply that you don't retransmit the announcement until you've gotten the actual data, right?
122 2015-03-10 15:14:29 <Luke-Jr> Galaxor: right
123 2015-03-10 15:14:36 <Luke-Jr> Galaxor: until you validate the data, even
124 2015-03-10 15:14:44 <Galaxor> Ahh, okay.
125 2015-03-10 15:14:48 <Luke-Jr> if you send peers data that is invalid, they will often ban you
126 2015-03-10 15:15:00 <Galaxor> Makes sense.
127 2015-03-10 15:15:32 <Galaxor> And of course, if you heard an announcement from peer A, you would not send the announcement back to peer A, right?
128 2015-03-10 15:16:11 <sipa> correct
129 2015-03-10 15:16:18 <Galaxor> Cool, thanks.
130 2015-03-10 15:16:18 <Luke-Jr> no reason you couldn't though
131 2015-03-10 15:16:47 <sipa> yeah, they'd just ignore it
132 2015-03-10 15:16:52 <Galaxor> Hm, okay.
133 2015-03-10 16:55:03 <hearn> "The modified version of Xcode, the researchers claimed, could enable spies to steal passwords and grab messages on infected devices. Researchers also claimed the modified Xcode could “force all iOS applications to send embedded data to a listening post.” It remains unclear how intelligence agencies would get developers to use the poisoned version of Xcode"
134 2015-03-10 16:55:07 <hearn> outstanding
135 2015-03-10 16:55:19 <hearn> reproducible builds need more love
136 2015-03-10 17:13:43 <cfields> wumpus: what filesystem/storage medium were you using when you got the arm block corruption?
137 2015-03-10 17:28:58 <wumpus> cfields: ext3, on a USB stick
138 2015-03-10 17:29:24 <wumpus> hearn: yes, reproducible builds + no more closed-source compilers
139 2015-03-10 17:29:28 <cfields> wumpus: thanks. how many cores?
140 2015-03-10 17:29:31 <wumpus> cfields: four
141 2015-03-10 17:29:48 <hearn> Xcode uses gcc/clang though. It's only the IDE that's closed source.
142 2015-03-10 17:29:53 <cfields> i'm trying to link the aws conditions and yours somehow, since it seems to be the exact same problem
143 2015-03-10 17:30:03 <timothy> just use hand-compiled clang or gcc :P
144 2015-03-10 17:30:26 <wumpus> cfields: I've tried killing my node and making it crash in various places, that could not reproduce it
145 2015-03-10 17:32:01 <cfields> wumpus: my inititial thought was that it may have something to do with amazon's de-dup, but that's out the window with your crash
146 2015-03-10 17:34:18 <cfields> wumpus: did you keep that profile around for testing? it'd be a good real-world case for #5864 to see if a re-index gets through it
147 2015-03-10 17:35:19 <wumpus> cfields: ugh, sorry, I only have the corrupted block file not the others
148 2015-03-10 17:35:36 <wumpus> huh, #5846 is failing travis
149 2015-03-10 17:35:39 <cfields> ok
150 2015-03-10 17:36:41 <cfields> wumpus: i thought that issue was fixed yesterday (test order)?
151 2015-03-10 17:37:28 <sipa> i've been a bit out of the loop of the corruption problem
152 2015-03-10 17:37:36 <sipa> is there anything i can do to help?
153 2015-03-10 17:38:36 <wumpus> cfields: well in 8cbe1f4 I fixed a few cases; the randomized order can bring other cases to light where the chainparams are not initialized
154 2015-03-10 17:39:15 <cfields> wumpus: ah sorry, i thought it was fixed up so i didn't look into it.
155 2015-03-10 17:39:39 <wumpus> cfields: I changed the most obvious ones; some other tests may need SelectParams(CBaseChainParams::MAIN); at the top or the full initialization fixture
156 2015-03-10 17:40:12 <cfields> sipa: the one thing i've managed to nail down is that before each case, blocks (or portions, i haven't actually determined that they're correct whole blocks) are written more than once to the same file
157 2015-03-10 17:40:33 <cfields> sipa: i can't come up with any explanation for that, but there are enough cases of the same thing happening to rule it out as a fluke
158 2015-03-10 17:41:05 <wumpus> sipa: I haven't been able to reproduce it yet, we have no idea what causes it
159 2015-03-10 17:41:08 <cfields> i could explain it if they had been previously re-indexed, but it seems that's not the case
160 2015-03-10 17:41:21 <cfields> wumpus: ok, thanks, will have a look
161 2015-03-10 17:42:31 <wumpus> cfields: indeed, blocks were written multiple times and seemingly overlapping
162 2015-03-10 17:43:40 <cfields> sipa: it looks as though it's writing ~8 blocks to the file, bumping up the offset to ~+7.5, not recording the blocks written, then continuing as usual as though it didn't happen, re-writing the "missing" blocks
163 2015-03-10 17:44:22 <cfields> but with some well-formed+recorded blocks in the mix too, just for fun
164 2015-03-10 17:45:12 <sipa> is there any restart involved?
165 2015-03-10 17:45:25 <wumpus> cfields: tried running the tests with export BOOST_TEST_RANDOM=13332 but cannot reproduce the travis failure of https://travis-ci.org/bitcoin/bitcoin/jobs/53692445
166 2015-03-10 17:45:48 <wumpus> cfields: is it possible that it tested without 8cbe1f4?
167 2015-03-10 17:46:02 <cfields> wumpus: should be BOOST_TEST_RANDOM=16, i think
168 2015-03-10 17:46:57 <wumpus> cfields: succeeds, too
169 2015-03-10 17:47:02 <cfields> sipa: unsure. wumpus: you happen to know if you restarted during ibd?
170 2015-03-10 17:47:22 <cfields> wumpus: i'll force a rebuild
171 2015-03-10 17:47:56 <wumpus> cfields: yes, it is a node that is stopped and started fairly often
172 2015-03-10 17:48:29 <sipa> does it have txindex?
173 2015-03-10 17:48:41 <wumpus> nope
174 2015-03-10 17:49:33 <wumpus> eh wait: yes, it has txindex=1
175 2015-03-10 17:49:52 <cfields> sipa: not all do, though
176 2015-03-10 17:49:57 <wumpus> forgot about that
177 2015-03-10 17:50:14 <sipa> because txindex bypasses the normal well-synchronized ordering of data writes
178 2015-03-10 17:50:28 <sipa> but it shouldn't affect anything related to blocks
179 2015-03-10 17:51:18 <cfields> sipa: txindex is disabled in eN0Rm's logs, assuming he's seeing the exact same issue
180 2015-03-10 17:51:42 <cfields> i believe the others said it was enabled though, so it's possible he's seeing something slightly different
181 2015-03-10 17:54:02 <john-dvorak> Any potential downsides to having a "gap limit" of 1000 unused addresses when using an HD wallet? (Assuming performance during a 'recovery' isn't an issue)
182 2015-03-10 17:57:31 <hearn> sipa: i thought OP_RETURN was bumped to 80 bytes in 0.10 .... but now it seems that didn't happen. was i imagining this?
183 2015-03-10 18:00:37 <sipa> hearn: it's bumped in master
184 2015-03-10 18:00:48 <hearn> ah, right, that's why i'm confused
185 2015-03-10 18:00:52 <hearn> thanks
186 2015-03-10 18:53:32 <jtimon> trivia question: are there any transactions before block 1333238400 (BIP16) with unspent outputs whose scripts would return true on CScript::IsPayToScriptHash ??
187 2015-03-10 20:27:09 <jtimon> sorry, 1333238400 wasn't a block number but a timestamp...the question remains similar, are there any unspent outputs that are older than the bip16 softfork and return true on CScript::IsPayToScriptHash() ?
188 2015-03-10 21:16:31 <jcorgan> so, getting some really curious behavior from a 0.10 instance.  it is stuck at a particular block height as a best chain tip, but it looks like it has all the rest of the blocks in the most recent block.dat (maybe).
189 2015-03-10 21:17:06 <jcorgan> the log shows it requesting the next block after the stuck block a couple hundred times, each to a new peer, and it looks like the peer never sends a response to the getdata
190 2015-03-10 21:17:22 <jcorgan> (or it never shows up in the log)
191 2015-03-10 21:18:01 <jcorgan> and the peers that aren't responding are working ok for tx inv and data
192 2015-03-10 21:18:23 <hearn> what are your peers?
193 2015-03-10 21:18:32 <hearn> are they mostly 41.* ips by any chance?
194 2015-03-10 21:18:45 <jcorgan> they're all .onion :)
195 2015-03-10 21:26:02 <jcorgan> it got stuck on 346969, and the best i can tell, it is trying to get 346970 from every new peer it gets, but noboy has it.  meanwhile, it has collected most or all of the blocks that have arrived after that.
196 2015-03-10 21:26:28 <jcorgan> i'll bounce it and see what it does
197 2015-03-10 22:04:56 <jcorgan> hmm. well, bouncing it unstuck it, operating normally now.  i've save the debug.log for future reference if ever anyone want to reverse engineer what broke.