1 2015-01-03 00:42:46 <melik> hello can anyone send me some BTC on the testnet?
2 2015-01-03 00:42:56 <melik> i would like to test/play with things
3 2015-01-03 00:43:26 <kgk> here is a faucet you can use: http://tpfaucet.appspot.com/
4 2015-01-03 00:43:35 <melik> ah perfect merci
5 2015-01-03 00:49:51 <melik> sigh 40 weeks left to sync :)
6 2015-01-03 00:56:21 <moa> melik: using v0.10rc1?
7 2015-01-03 00:56:47 <melik> github*
8 2015-01-03 00:56:47 <melik> moa, most likely i pulled the latest from gi
9 2015-01-03 01:25:58 <swap01> 01swap.com - Make money by selling your files for Bitcoins!
10 2015-01-03 02:20:57 <midnightmagic> ah there go: time ./bench_recover -> min 52431.512us / avg 52433.876us / max 52452.493us : real 2h 54m 51s
11 2015-01-03 02:23:00 <sipa> midnightmagic: ouch
12 2015-01-03 02:23:21 <phantomcircuit> midnightmagic, what is that for?
13 2015-01-03 02:23:25 <sipa> ;;calc 1000000/52433.876
14 2015-01-03 02:23:26 <gribble> 19.0716398689
15 2015-01-03 02:23:44 <sipa> phantomcircuit: secp256k1 verification benchmark
16 2015-01-03 02:24:00 <gmaxwell> sipa: you missed the bench_Verify he gave before.
17 2015-01-03 02:24:03 <sipa> ah, pubkey recovery, not verify
18 2015-01-03 02:24:10 <gmaxwell> verify was similar.
19 2015-01-03 02:25:47 <sipa> what cpu is that?
20 2015-01-03 02:25:57 <gmaxwell> I'm not sure why it's quite so slow, it's something like 210 dmips according to the data sheet, though the 16kb data cache may play a big role and I'm guessing the dram attached is slow.
21 2015-01-03 02:26:06 <gmaxwell> sipa: it's an AVR32.
22 2015-01-03 02:26:14 <gmaxwell> http://www.atmel.com/devices/AT32AP7000.aspx
23 2015-01-03 02:26:21 <sipa> maybe it needs a much smaller g table?
24 2015-01-03 02:26:27 <gmaxwell> probably.
25 2015-01-03 02:27:02 <midnightmagic> FYI, that CPU is EOL'd according to someone who self-identified as a former Atmel employee. Atmel seems to have gone the ARM route for future products.
26 2015-01-03 02:27:10 <gmaxwell> we could put a detting of arch sniffing window sizes, and have a configure override.
27 2015-01-03 02:27:55 <gmaxwell> hm I can't decode my last sentence.
28 2015-01-03 02:28:11 <midnightmagic> that's partly why none of their patches ever survived upstream and avr32 projects in general are virtually dead
29 2015-01-03 02:28:13 <gmaxwell> We could have ifdefs per arch of default window sizes and a configure option to override.
30 2015-01-03 02:28:48 <gmaxwell> midnightmagic: in any case, I expect libsecp256k1's performance story is similar on small cortex m3 and alike.
31 2015-01-03 02:29:45 <midnightmagic> now that I've rediscovered which of my toolchain crosscompile trees does in fact work, I can do a much quicker turnaround if anyone wants me to test something on it.
32 2015-01-03 02:30:43 <sipa> midnightmagic: src/ecmult_impl.h has a define at the top for the table sizes for a and g points in the multiplication
33 2015-01-03 02:30:49 <gmaxwell> midnightmagic: welp you can try reducing the window sizes, thats just a quick tweak.
34 2015-01-03 02:31:02 <midnightmagic> okay
35 2015-01-03 02:31:04 <sipa> decreasing the number for g may help if memory access is slow compared to processing
36 2015-01-03 02:35:41 <midnightmagic> <3 zmodem hah
37 2015-01-03 02:36:57 <sipa> ?
38 2015-01-03 02:37:13 <midnightmagic> it's how I currently have to get stuff onto the board
39 2015-01-03 02:37:27 <midnightmagic> rs-232-only right now
40 2015-01-03 03:42:51 <Genitrust> what does it mean to "augment" a transaction?
41 2015-01-03 05:53:10 <HaltingState> sipa, gmaxwell are curves with small discriminants insecure?
42 2015-01-03 05:57:15 <andytoshi> HaltingState: have you asked in ##crypto?
43 2015-01-03 06:05:07 <gmaxwell> HaltingState: no.
44 2015-01-03 06:05:23 <gmaxwell> (I wish all questions were this easy.)
45 2015-01-03 06:19:40 <firelegend> birthday today?
46 2015-01-03 06:19:45 <firelegend> 6 years and counting :D
47 2015-01-03 06:23:34 <PatrikR> hm, what's wrong with bitcoin.org? I'm trying to download RC1, and I'm getting modem speeds
48 2015-01-03 06:23:57 <PatrikR> "4.8 KB/s - 4.0 MB of 29.2 MB, 1 hour left"
49 2015-01-03 06:28:09 <PatrikR> is there a secondary download site?
50 2015-01-03 06:30:09 <phantomcircuit> PatrikR, iirc rc1 isn't even available from bitcoin.org
51 2015-01-03 06:30:45 <PatrikR> uhhh... https://bitcoin.org/bin/0.10.0/test/
52 2015-01-03 06:33:04 <phantomcircuit> neat
53 2015-01-03 06:33:13 <phantomcircuit> which are you downloading?
54 2015-01-03 06:34:37 <phantomcircuit> PatrikR, the server for that is in la
55 2015-01-03 06:35:04 <phantomcircuit> i can upload somewhere else for you but only if you promise to check the hash
56 2015-01-03 06:42:27 <Luke-Jr> he should be checking the hash from any source :P
57 2015-01-03 06:43:00 <Luke-Jr> I already have a copy on http://luke.dashjr.org/programs/bitcoin/files/bitcoind/0.10.0/test/rc1/ fwiw
58 2015-01-03 06:43:07 <PatrikR> i have the file on my laptop already, i could get it from there if i weren't too lazy to go get it. :) anyway, there's nothing wrong with my network connection, just did a speedtest.net. did someone forget to pay the bandwidth bill for the server?
59 2015-01-03 06:43:56 <phantomcircuit> PatrikR, your problem is the server is in california and you're in finland
60 2015-01-03 06:44:25 <PatrikR> Luke-Jr: thanks, that was much faster
61 2015-01-03 06:44:37 <PatrikR> phantomcircuit: you have no idea what you're talking about
62 2015-01-03 06:44:38 <Luke-Jr> PatrikR: check the hash
63 2015-01-03 06:44:45 <PatrikR> Luke-Jr: of course
64 2015-01-03 06:45:21 <Luke-Jr> PatrikR: I get a good 400+kB/s from bitcoin.org FYI
65 2015-01-03 06:46:08 <phantomcircuit> PatrikR, amazingly i actually do
66 2015-01-03 06:46:14 <phantomcircuit> you're crossing a transatlantic link
67 2015-01-03 06:46:16 <PatrikR> Luke-Jr: huh, interesting.
68 2015-01-03 06:46:21 <phantomcircuit> shit is being traffic shaped
69 2015-01-03 06:46:52 <Luke-Jr> phantomcircuit: my server would cross that too
70 2015-01-03 06:46:59 <Luke-Jr> I wonder if it's the SSL
71 2015-01-03 06:47:08 <phantomcircuit> Luke-Jr, your server where?
72 2015-01-03 06:47:11 <Luke-Jr> PatrikR: downloading system have plenty of CPU time?
73 2015-01-03 06:47:13 <Luke-Jr> phantomcircuit: NY
74 2015-01-03 06:47:23 <phantomcircuit> Luke-Jr, the bitcoin.org server is in california
75 2015-01-03 06:47:30 <phantomcircuit> i mean the submarine cables
76 2015-01-03 06:47:51 <Luke-Jr> phantomcircuit: heh, too bad it isn't smart enough to route across the US ;P
77 2015-01-03 06:47:52 <PatrikR> Luke-Jr: i just did a speedtest.net and got roughly 65 Mbps up and down...
78 2015-01-03 06:47:55 <phantomcircuit> from a server in dallas 52.3M/s
79 2015-01-03 06:48:17 <phantomcircuit> PatrikR, that's not how the internet works...
80 2015-01-03 06:48:26 <gmaxwell> PatrikR: the internet is sometimes a funny place, it's not a uniform network, sometimes there are bottlenecks which are neither you nor the far end...
81 2015-01-03 06:49:45 <PatrikR> gmaxwell: the speed was weirdly consistent at 9.6 kB/s or 14.4 kB/s. looks to me like something was restricting the download to modem speeds on purpose
82 2015-01-03 06:49:58 <phantomcircuit> PatrikR, i just told you
83 2015-01-03 06:50:09 <phantomcircuit> transatlantic submarine cables are *ALL* traffic shaped
84 2015-01-03 06:50:25 <phantomcircuit> your ISP is probably in the slow slow slow lane
85 2015-01-03 06:50:28 <PatrikR> phantomcircuit: Luke-Jr's server is in the USA too
86 2015-01-03 06:50:44 <phantomcircuit> <Luke-Jr> phantomcircuit: heh, too bad it isn't smart enough to route across the US ;P
87 2015-01-03 06:51:43 <Luke-Jr> sometimes i wish nodes chose their own routes
88 2015-01-03 06:51:51 <Luke-Jr> but that would no doubt screw up latency bad for initial stuff
89 2015-01-03 07:00:42 <phantomcircuit> PatrikR, https://mega.co.nz/#!y09i1DKa!Et8lfqNyC37vT7I2Ud_NB6aZxL1O3pe9L8g4DUfMCT8
90 2015-01-03 07:00:43 <phantomcircuit> there
91 2015-01-03 07:00:50 <phantomcircuit> should be much faster
92 2015-01-03 07:01:08 <PatrikR> phantomcircuit: i already got the file from Luke-Jr...
93 2015-01-03 07:01:43 <phantomcircuit> shrug
94 2015-01-03 07:11:13 <midnightmagic> provided USE_ENDOMORPHISM is evaluating to false in my configure and my ability to read *-config.h to verify it is false, is true, changing the window size from 16->10 brings from:
95 2015-01-03 07:11:28 <midnightmagic> time ./bench_verify -> min 52673.949us / avg 52677.212us / max 52705.408us : real 2h 55m 40s (old, default)
96 2015-01-03 07:11:49 <midnightmagic> time ./bench_verify -> min 53630.967us / avg 53634.264us / max 53660.638us : real 2h 58m 47s
97 2015-01-03 07:11:53 <midnightmagic> ^^ new
98 2015-01-03 07:13:43 <gmaxwell> interesting. slower! mind trying a very small value? e.g. 2
99 2015-01-03 07:13:59 <phantomcircuit> i still dont get what exactly is going on here
100 2015-01-03 07:15:27 <midnightmagic> yep, already there, going to 2. 1 results in a compile-time error, some kind of left-shift being negative.
101 2015-01-03 07:23:56 <midnightmagic> phantomcircuit: you mean the benchmarking? a couple people expressed an interest in seeing the new secp library put through its paces on an avr32 platform, since I spent a few weeks a couple months ago carefully rebuilding it.
102 2015-01-03 07:24:43 <gmaxwell> midnightmagic: whats the clock rate of yours?
103 2015-01-03 07:25:31 <midnightmagic> have to wait for the bench to finish, it's a non-muxed serial console
104 2015-01-03 07:25:43 <gmaxwell> sure. no rush.
105 2015-01-03 07:26:21 <phantomcircuit> ah
106 2015-01-03 07:26:36 <phantomcircuit> midnightmagic, neat
107 2015-01-03 07:26:43 <phantomcircuit> also that is lol forever
108 2015-01-03 07:26:47 <midnightmagic> it's supposed to be 210MHz
109 2015-01-03 07:27:36 <midnightmagic> phantomcircuit: i'm using minicom to communicate with it, and zmodem to transfer all files back and forth, it's a lonely-tear-down-the-face throwback into 80s-land lol
110 2015-01-03 07:32:39 <phantomcircuit> midnightmagic, ha
111 2015-01-03 09:28:46 <melvster> happy new year all ... http://i.imgur.com/RwCxZPl.png
112 2015-01-03 09:30:23 <melvster> saw that on reddit, quite nice :)
113 2015-01-03 09:31:35 <sipa> 'technical stuff' ha
114 2015-01-03 09:31:38 <melvster> lol
115 2015-01-03 09:31:59 <melvster> is that the diff / target?
116 2015-01-03 09:33:12 <sipa> coinbase input, which at the time indeed contained a copy of the target bits
117 2015-01-03 09:33:28 <melvster> ah ha
118 2015-01-03 11:00:21 <midnightmagic> cool
119 2015-01-03 11:00:48 <midnightmagic> time ./bench_verify -> min 53630.967us / avg 53634.264us / max 53660.638us : real 2h 58m 47s (last run, g table = 10)
120 2015-01-03 11:01:12 <midnightmagic> time ./bench_verify -> min 63037.704us / avg 63042.201us / max 63080.767us : real 3h 30m 08s (g table = 2)
121 2015-01-03 11:02:33 <midnightmagic> that's interesting. /proc/cpuinfo says the CPU is running at 70MHz.
122 2015-01-03 11:44:02 <hegemoOn> midnightmagic: what are those for ?
123 2015-01-03 11:44:58 <midnightmagic> avr32-based sbc from a few years ago when atmel was still making avr32 devices
124 2015-01-03 12:18:54 <hegemoOn> and what is the use of bench_verify ?
125 2015-01-03 12:42:30 <fanquake> hegemoOn Itâs one of the benchmarking tools in https://github.com/bitcoin/secp256k1/
126 2015-01-03 12:46:35 <Luke-Jr> would it hurt anyone if we redefined the sequence to be a timestamp? :P
127 2015-01-03 14:22:56 <op_mul> Luke-Jr: it would take < 600 seconds before somebody decided to use that to prevent double spending. just take the one that was signed first!
128 2015-01-03 14:35:56 <midnightmagic> time ./bench_verify -> min 53630.967us / avg 53634.264us / max 53660.638us : real 2h 58m 47s (last run, g table = 10)
129 2015-01-03 14:36:15 <midnightmagic> time ./bench_verify -> min 52553.154us / avg 52553.332us / max 52553.522us : real 2h 55m 21s
130 2015-01-03 14:36:23 <midnightmagic> ^^ (g table = 17)
131 2015-01-03 14:36:29 <midnightmagic> default is 16 I think
132 2015-01-03 15:17:17 <firelegend> anything to worry about? https://www.mail-archive.com/openssl-dev@openssl.org/msg37790.html
133 2015-01-03 15:17:47 <Luke-Jr> op_mul: is that a serious concern? <.<
134 2015-01-03 15:18:34 <op_mul> Luke-Jr: oh yeah. you know somebody will do it.
135 2015-01-03 15:32:00 <Luke-Jr> op_mul: I mean: do we care?
136 2015-01-03 15:32:27 <jtimon> Luke-Jr I updated #5180 (and the modified version of #5071) to not include #5114 at all (making it more readable)
137 2015-01-03 15:33:40 <op_mul> Luke-Jr: why do you want a time stamp anyway?
138 2015-01-03 15:33:52 <jtimon> petertodd you may be interested as well (it's basically the Params().DefaultPolicy() that we talked about)
139 2015-01-03 15:33:58 <Luke-Jr> op_mul: to help wallets have a well-defined transaction time
140 2015-01-03 15:34:47 <aschildbach> Saw this flying by HN: https://www.mail-archive.com/openssl-dev@openssl.org/msg37790.html ("EC key generation in broken in all versions")
141 2015-01-03 15:35:16 <op_mul> Luke-Jr: means you need to have two columns though. claimed send time, actual time you saw it.
142 2015-01-03 15:35:43 <Luke-Jr> op_mul: we currently have 3 "columns"
143 2015-01-03 15:35:57 <op_mul> Luke-Jr: columns++ then.
144 2015-01-03 15:37:20 <Luke-Jr> aschildbach: doesn't look like it affects anything we use?
145 2015-01-03 15:37:38 <aschildbach> yeah probably
146 2015-01-03 15:38:07 <aschildbach> I'm not confident with bitcoin core code so I thought I'd mention it just in case.
147 2015-01-03 15:42:28 <firelegend> "I'm not confident with bitcoin core code" which part?
148 2015-01-03 15:44:06 <aschildbach> all except the list of DNS seeds (-:
149 2015-01-03 15:44:16 <aschildbach> I'm a Java coder
150 2015-01-03 15:46:48 <firelegend> your statement made it seem as if you are not confident *in* the bitcoin core code
151 2015-01-03 15:46:54 <Luke-Jr> I think firelegend may have misread aschildbach's statement there as a criticism of Bitcoin Core's code, rather than expressing his own ignorance of it
152 2015-01-03 15:47:12 <Luke-Jr> firelegend: pretty sure the latter ^ is what he meant
153 2015-01-03 15:47:24 <aschildbach> Ah yes, indeed
154 2015-01-03 15:53:45 <jtimon> and Luke-Jr, I also created #5595 which is the simples way to create policy.o that makes sense to me independently
155 2015-01-03 15:53:52 <jtimon> simplest
156 2015-01-03 15:55:03 <jtimon> I'm not sure that policy.cpp should ever include main.h like you do anymore
157 2015-01-03 15:57:31 <jtimon> 5180 is not enough for policies like replace-by-fee like #5071 but as said I don't like policy.o depending on main.o and its locks
158 2015-01-03 16:05:13 <Luke-Jr> jtimon: the cnodepolicy PR I have open was only a small part of what I have locally
159 2015-01-03 16:05:54 <Luke-Jr> mostly because I didn't want to push new code to an existing PR; kinda messes with reviewing
160 2015-01-03 16:08:07 <jtimon> I see, can I see some of that code so that I can help you with rebasing and stuff? I mean, I rebased only what I was seeing on top of my things
161 2015-01-03 16:08:59 <Luke-Jr> yeah, sorry, sec
162 2015-01-03 16:09:16 <jtimon> maybe renaming CNodePolicyBase to CPolicy and CNodePolicy to CStandardPolicy wasn't such a good idea
163 2015-01-03 16:09:36 <Luke-Jr> jtimon: nodepolicy2
164 2015-01-03 16:09:59 <jtimon> ok, thanks
165 2015-01-03 16:14:29 <Luke-Jr> it may need rebasing
166 2015-01-03 16:20:08 <jtimon> yeah, even #5071 needs rebase
167 2015-01-03 16:21:26 <jtimon> I posted a rebased version I think
168 2015-01-03 16:21:54 <jtimon> anyway, I will study this and rebase on top of several stages of what I have
169 2015-01-03 16:30:11 <gmaxwell> aschildbach: firelegend: not related to us. we don't use openssl's encoding functions for that.
170 2015-01-03 16:30:56 <gmaxwell> aschildbach: it's a poorly named message. it's not complaining about key generation, it's complaining that openssl's command line tools don't print leading zeros.
171 2015-01-03 16:46:02 <firelegend> So there seems to be a node that sends me an addr message with a length of the payload that is much higher than 1000 bytes
172 2015-01-03 16:47:18 <gmaxwell> addr message payload is not limited to 1000 bytes; addr messages are ignored when they have a payload with more than 1000 entries.
173 2015-01-03 18:04:15 <BlueMatt> wumpus: strange.....
174 2015-01-03 18:04:23 <BlueMatt> the memory stuff I was dealing with was re: https://github.com/bitcoin/bitcoin/pull/5422
175 2015-01-03 18:04:51 <BlueMatt> after which we're close to running out of travis memory, but still have enough headroom so that it should only happen if a pull increases memory usage significantly
176 2015-01-03 20:16:23 <jtimon> Luke-Jr see luke_nodepolicy2_rebased in my gihub, the last commit breaks the tests, but I'm not sure that's my fault
177 2015-01-03 20:40:41 <jtimon> also luke_nopolicy2_5595
178 2015-01-03 21:34:02 <michagogo> Hmm
179 2015-01-03 21:34:16 <michagogo> My syncing seems to be stalled
180 2015-01-03 21:34:21 <sipa> :(
181 2015-01-03 21:34:38 <michagogo> It's been at 337077 for a very long time now
182 2015-01-03 21:34:44 <michagogo> (this is on v0.10.0rc1)
183 2015-01-03 21:34:52 <sipa> ;;blocks
184 2015-01-03 21:34:53 <gribble> 337339
185 2015-01-03 21:35:20 <michagogo> getblockchaininfo says I have all the headers
186 2015-01-03 21:35:38 <sipa> what does getchaintips say?
187 2015-01-03 21:35:58 <michagogo> https://www.irccloud.com/pastebin/MjcFrbB7
188 2015-01-03 21:36:58 <sipa> and getpeerinfo?
189 2015-01-03 21:37:06 <sipa> any blocks in flight?
190 2015-01-03 21:37:10 <michagogo> https://www.irccloud.com/pastebin/CzcL4eHu
191 2015-01-03 21:37:11 <michagogo> yes
192 2015-01-03 21:37:24 <michagogo> 337078, from the first peer listed
193 2015-01-03 21:40:11 <michagogo> Any ideas?
194 2015-01-03 21:42:56 <michagogo> Ah
195 2015-01-03 21:43:11 <michagogo> Okay, I closed the connection to that peer and now I'm synced up
196 2015-01-03 21:43:20 <michagogo> ;;blocks
197 2015-01-03 21:43:21 <gribble> 337340
198 2015-01-03 21:43:35 <michagogo> Question is, though, why did this happen?
199 2015-01-03 21:43:53 <michagogo> I thought the whole point of headers-first was to not let this kind of thing happen
200 2015-01-03 21:46:05 <sipa> indeed, i don't understand it
201 2015-01-03 21:48:27 <moa> michagogo: i saw it happen once also, back in Oct(?) using git head, got stuck waiting for one block in flight from one peer ... think I restarted and never saw it happen again
202 2015-01-03 21:48:56 <michagogo> sipa: does anything get logged that could help explain this?
203 2015-01-03 21:49:11 <moa> and was on testnet btw
204 2015-01-03 21:50:28 <michagogo> BTW, it took exactly 80 seconds to get up to date after I closed that socket
205 2015-01-03 21:51:39 <moa> how do you disconnect a single peer?
206 2015-01-03 21:51:51 <michagogo> moa: http://www.nirsoft.net/utils/cports.html
207 2015-01-03 21:52:17 <moa> ah, thought you have been referring to a bitcoin rpc command
208 2015-01-03 21:52:21 <michagogo> Ah, nope
209 2015-01-03 21:52:34 <moa> which i believe there is an issue out there somewhere for ..
210 2015-01-03 21:56:25 <morcos> you gon't think this and #5588 could be related to https://github.com/bitcoin/bitcoin/pull/5463 ?
211 2015-01-03 21:56:39 <dfletcher> is walletnotify fired if a fork affects local wallet transactions? and what would the hint that this happened in the json? confirmations would go back to zero or go negative or something? or the tx just dissapears?
212 2015-01-03 22:01:08 <morcos> sipa: not sure if you talked more with ajweiss, but we think #5463 or something that addresses that problem would be good to get in for 0.10
213 2015-01-03 22:14:54 <phantomcircuit> sipa, i would guess that there's some peers which stall sending the block, is there a timeout in asking a different peer for that block?
214 2015-01-03 22:21:05 <Jouke> Hmm, with 10rc1 I was connected to a very slow node and it slowed the syncing a lot (starting from block 0), but an hour later I lost connection to that vps, so couldn't investigate further.
215 2015-01-03 22:21:38 <gmaxwell> Jouke: How do you know that it "it slowed the syncing a lot"
216 2015-01-03 22:22:16 <Jouke> looking at peerinfo, I saw there were blocks inflight from that node
217 2015-01-03 22:22:19 <gmaxwell> By design it cannot slow the syncing more than not having a connection in that slot. (not that there might not be a bug, but thats why I'm skeptical of the claim without some explination as to why you believe that)
218 2015-01-03 22:22:42 <gmaxwell> Jouke: That doesn't mean that it was making it slower, it adapatively requests fewer blocks from slower peers.
219 2015-01-03 22:23:17 <gmaxwell> Jouke: there is a rolling window of 1000 blocks in flight, and if it finds itself with the window full and waiting on a peer, it will disconnect the peer that has stalled the window.
220 2015-01-03 22:23:21 <Jouke> But it was waiting for that peer to finish seeding those blocks that were in flight
221 2015-01-03 22:23:41 <gmaxwell> Jouke: why do you believe it was waiting?
222 2015-01-03 22:24:03 <Jouke> because for minutes it was the only peer with those blocks in flight
223 2015-01-03 22:24:30 <gmaxwell> Jouke: that doesn't mean it was waiting. It downloads blocks out of order.
224 2015-01-03 22:25:24 <Jouke> there were no other blocks in flight at other peers and getinfo was "stuck" while those blocks were in flight
225 2015-01-03 22:26:39 <gmaxwell> Jouke: sounds like a bug then, since by design it shouldn't be able to get into that state. If you observe something like that again, please capture the getpeerinfo output and the debug.log.
226 2015-01-03 22:27:27 <Jouke> yeah, wanted to, but then isp just blocked all trafic on that vps
227 2015-01-03 22:27:52 <Jouke> ($89 dollar per three year vps)
228 2015-01-03 22:28:44 <gmaxwell> huh. did they refund your money?
229 2015-01-03 22:28:49 <phantomcircuit> gmaxwell, is there a timeout for pulling a block from a specific peer?
230 2015-01-03 22:29:30 <gmaxwell> phantomcircuit: at runtime? yes. IIRC 10 minutes. During initial sync it's primarily controlled by your other peers and the download window.
231 2015-01-03 22:29:57 <gmaxwell> phantomcircuit: it can't be too short or you'll make it easy to partition the network with DDOS attacks, and make it impossible to run a node on a slow link.
232 2015-01-03 22:30:55 <gmaxwell> (presumably we could make the timeout configurable; for those who know they are on a fast link... though I think in general the main p2p protocol should be optimizing for anti-partitioning and robustness, not latency. Use the relay network protocol to minimize latency.)
233 2015-01-03 22:31:45 <sdaftuar> i think pr 5463 tries to introduce a timeout, but currently stall detection is just based on a window of certain number of blocks...?
234 2015-01-03 22:32:29 <Jouke> Yeah, it was not a huge problem, on an other new vps it worked just fine.
235 2015-01-03 22:33:44 <phantomcircuit> gmaxwell, if you already have all the headers is it really an issue?
236 2015-01-03 22:35:20 <phantomcircuit> bbl
237 2015-01-03 22:35:49 <gmaxwell> sdaftuar: oh... hm. I thought that made it in.
238 2015-01-03 22:36:53 <sdaftuar> not yet, i have a guess that this issue is what causes 5588 (just posted there). not totally sure without seeing more logs, but i think it fits the evidence
239 2015-01-03 22:37:11 <gmaxwell> sdaftuar: we should still continue when the next block on the network shows up, however.
240 2015-01-03 22:38:18 <sdaftuar> not sure i follow -- i think the headers continue to build up, but we'll never request the block from anyone else, so the tip won't update right?
241 2015-01-03 22:39:13 <sdaftuar> (never request the stuck block, i agree the code would continue to download new blocks on the chain)
242 2015-01-03 22:41:49 <gmaxwell> hm. May be so. Headers first may have regressed our prior protection against getting stuck. (we'd eventually request the block from another peer who told us about a successor block.)
243 2015-01-03 22:46:25 <michagogo> 00:23:01 <gmaxwell> By design it cannot slow the syncing more than not having a connection in that slot. (not that there might not be a bug, but thats why I'm skeptical of the claim without some explination as to why you believe that)
244 2015-01-03 22:46:38 <michagogo> gmaxwell: I was held up on one block inflight from one peer for 2 hours
245 2015-01-03 22:46:50 <michagogo> block 337077
246 2015-01-03 22:47:09 <michagogo> (see 21:30 UTC in here)
247 2015-01-03 22:47:31 <gmaxwell> michagogo: I was referring to during initial download sync, not time at the tip.
248 2015-01-03 22:47:43 <michagogo> gmaxwell: hmm?
249 2015-01-03 22:47:48 <michagogo> This was initial sync
250 2015-01-03 22:47:54 <michagogo> Well, catch-up on starting the node
251 2015-01-03 22:47:55 <gmaxwell> The initial download "can't be worse than just not having the peer" doesn't apply unless there are at least 1000 more headers.
252 2015-01-03 22:48:02 <michagogo> ah
253 2015-01-03 22:49:08 <gmaxwell> Basically it uses all your other sync peers as a dynamic measurement of if a particular peer sucks, instead of having a hardcoded definition of suckyness.
254 2015-01-03 22:49:27 <gmaxwell> But when you're near the tip and there aren't multiple blocks in flight it's not possible to do that.
255 2015-01-03 22:49:42 <michagogo> gmaxwell: there was just one in flight, actually, as far as I could tell
256 2015-01-03 22:49:58 <michagogo> seems a bit broken to me
257 2015-01-03 22:50:22 <michagogo> As soon as I killed the socket to that one peer, I heard my computer fan get louder
258 2015-01-03 22:50:27 <michagogo> 80 seconds later I was synced up
259 2015-01-03 22:50:42 <gmaxwell> michagogo: I just said, that hurestic cannot work when there aren't at least 1000 blocks ahead of you.
260 2015-01-03 22:50:56 <michagogo> gmaxwell: yeah
261 2015-01-03 22:51:01 <gmaxwell> It's not broken, it just doesn't apply there.
262 2015-01-03 22:51:14 <michagogo> gmaxwell: well, something's broken about the mechanism as a whole
263 2015-01-03 22:51:18 <michagogo> (the syncing)
264 2015-01-03 22:51:40 <michagogo> One peer delaying a block for some reason shouldn't freeze sync for 2+ hours :-/
265 2015-01-03 22:52:21 <gmaxwell> michagogo: well what do you want it to do? Lets imagine tha tour internet connection is very slow such that it takes you two hours to download the block. If you terminate fetching you'll never make progress once you reach that point.
266 2015-01-03 22:52:36 <michagogo> gmaxwell: uh
267 2015-01-03 22:52:45 <michagogo> if it takes 2 hours to download a block you have bigger problems
268 2015-01-03 22:53:14 <gmaxwell> (the old behavior is that we'de use announcement of the next block on the network as a beacon to make progress, but it sounds like we lost that behavior)
269 2015-01-03 22:53:36 <michagogo> Even if it didn't terminate, I would expect it to simultaneously try for that block from another peer or something after a couple minutes
270 2015-01-03 22:53:41 <michagogo> or... something
271 2015-01-03 22:53:58 <gmaxwell> michagogo: today it does because there is a maximum block size, so you can say e.g. if you can't download a block in 20 minutes you're screwed anyways. But that doesn't apply where people want to do things like remove or greatly increase the maximum.
272 2015-01-03 22:54:18 <gmaxwell> michagogo: simultaneously means that it now takes 4 hours to fetch that block. :P
273 2015-01-03 22:54:34 <michagogo> gmaxwell: um, in any case you're screwed if it takes 2 hours to fetch a block
274 2015-01-03 22:54:46 <michagogo> Because if you assume that they're coming in every 10 minutes you'll never catch up
275 2015-01-03 22:54:52 <gmaxwell> michagogo: :( did you read what I wrote where it starts with "today"
276 2015-01-03 22:54:59 <michagogo> I did
277 2015-01-03 22:55:17 <michagogo> But if block sizes are increased to the point where it takes more than 10 minutes on average to download a block...
278 2015-01-03 22:55:23 <michagogo> you're still screwed
279 2015-01-03 22:55:32 <michagogo> Unless you mean one huge megablock among many small ones
280 2015-01-03 22:55:35 <Jouke> 10 minutes seems like a goot time to switch a peer
281 2015-01-03 22:55:39 <gmaxwell> michagogo: not all blocks are the average size.
282 2015-01-03 22:55:56 <gmaxwell> (even now there is a 2:1 difference between th average and the maximum)
283 2015-01-03 22:56:15 <michagogo> Hm, do we have any mechanism to learn about the size of a block?
284 2015-01-03 22:56:21 <michagogo> (before fetching it)
285 2015-01-03 22:56:33 <gmaxwell> michagogo: not in advance, which is annoying. If we replace getheaders with something else, that should be fixed)
286 2015-01-03 22:57:06 <gwillen> if you know a rough estimate of the current average transaction rate and size, you can make estimates like "I am currently downloading the bytes of this block more slowly than new bytes of transactions are being created"
287 2015-01-03 22:57:18 <gwillen> "therefore I better give up and try something else"
288 2015-01-03 22:57:39 <michagogo> Somehow, I don't think it's a good idea to release an 0.10.0 that will stall for 2+ hours waiting on a peer to send a block when catching up
289 2015-01-03 22:57:48 <michagogo> ACTION goes to file an issue
290 2015-01-03 22:58:01 <michagogo> What information should I be providing to maximize helpfulness?
291 2015-01-03 22:58:11 <gmaxwell> Getheaders sends a tx_count with the headers but it's always zero.
292 2015-01-03 22:58:21 <michagogo> gmaxwell: m(
293 2015-01-03 22:58:22 <gmaxwell> michagogo: yea sure, I consider this blocking.
294 2015-01-03 22:58:37 <gwillen> another thing you could try would be estimating the user's connection speed
295 2015-01-03 22:58:38 <michagogo> What is that tx_count supposed to mean?
296 2015-01-03 22:58:39 <gmaxwell> My argument above was only trying to make the point that it's not just a trivial thing to handle.
297 2015-01-03 22:58:48 <michagogo> gmaxwell: Okay, fair enough
298 2015-01-03 22:58:51 <gwillen> and get mad if you are currently downloading at less than, say, half that
299 2015-01-03 22:59:00 <michagogo> But I feel like even a 10-15 minute timeout, right now, would be fine
300 2015-01-03 22:59:11 <michagogo> If we introduced larger blocks, we could... change that.
301 2015-01-03 22:59:12 <Jouke> gwillen: but it could be your own connection
302 2015-01-03 22:59:54 <gwillen> well, I handwaved away the process of figuring out your own connection speed
303 2015-01-03 22:59:55 <gmaxwell> (rebroad previously, back in 0.9.x -- where we wouldn't get stuck -- was trying in his usual fashion to get us take a patch that kick (ban?) peers after it took a minute or two to fetch a block.. so I'm a bit defensive about over simplifications of this. :) )
304 2015-01-03 23:00:13 <Jouke> If we introduce lager blocks, you should still have less then 10 minutes time to get blocks
305 2015-01-03 23:00:30 <michagogo> Jouke: right, that's something that would need to be figured out
306 2015-01-03 23:00:48 <gmaxwell> Jouke: on average, but that doesn't speak to any particular block.
307 2015-01-03 23:00:55 <firelegend> why do nodes with newer protocol send an addr message without the timestamp?
308 2015-01-03 23:01:03 <michagogo> gmaxwell: so what information, if any, should I include in the issue?
309 2015-01-03 23:01:06 <gmaxwell> Also, the failure mode if the network is too fast for you should be clean, not be one that lets you get partitioned.
310 2015-01-03 23:01:15 <gmaxwell> michagogo: I think we already have an issue open.
311 2015-01-03 23:01:20 <michagogo> Ah, okay
312 2015-01-03 23:01:21 <michagogo> ACTION looks
313 2015-01-03 23:01:26 <Jouke> gmaxwell: right :)
314 2015-01-03 23:01:31 <gmaxwell> michagogo: https://github.com/bitcoin/bitcoin/issues/5588
315 2015-01-03 23:02:04 <gwillen> gmaxwell: it seems like the failure mode where the network is inherently too fast for you is going to be extremely rare
316 2015-01-03 23:02:17 <gwillen> at lest in the near term
317 2015-01-03 23:02:49 <michagogo> gmaxwell: I'm not sure this is the same issue
318 2015-01-03 23:03:04 <Jouke> maybe if you are only connected through tor? (don't have any experience with that use case)
319 2015-01-03 23:03:10 <gmaxwell> michagogo: pretty sure its exactly the same issue.
320 2015-01-03 23:03:14 <gmaxwell> Jouke: nah, tor is pretty fast.
321 2015-01-03 23:03:28 <gmaxwell> Jouke: keep in mind that the maximum rate for the network is 14kbit/sec.
322 2015-01-03 23:03:59 <michagogo> gmaxwell: hm? I don't see anything there about a wrong or stale block
323 2015-01-03 23:04:17 <michagogo> (no failures logged, etc, the UpdateTips just stop)
324 2015-01-03 23:05:14 <michagogo> What getchaintips showed as active was in the main chain, and it did have the chain up to date headers-only
325 2015-01-03 23:05:55 <gmaxwell> gwillen is indeed correct. But I think that its likely that people who are not thoughtful or are not long term focused will force onto the network a hard fork that makes the maximum size effectively unlimited. It would be disingenuous for me to sit quietly when people propose protocol behavior that will result in network failures with much larger blocks and then later turn around and use those short
326 2015-01-03 23:06:01 <gmaxwell> comings to argue against allowing larger blocks...
327 2015-01-03 23:06:24 <gwillen> heh, *nods*
328 2015-01-03 23:06:41 <michagogo> And looking at the log, I do see block files being switched
329 2015-01-03 23:07:21 <michagogo> Maybe the underlying issue is the same, but our two cases *seem* to be different as far as I can tell
330 2015-01-03 23:07:35 <gmaxwell> michagogo: you see block files being switched because it downloaded all th pending headers beyond that one. There just weren't enough of them to trigger disconnection of the stalled peer. Ultimately I think the behavior is the same: you're waiting on a block to show up, and you'll wait forever.
331 2015-01-03 23:09:57 <Jouke> gmaxwell: in my debug log of that latest node it says that there were peers stalling and they were disconnected. Are those peers really disconnected from the node, or only for the download stage?
332 2015-01-03 23:10:43 <Jouke> Can one get info on old peers?
333 2015-01-03 23:10:49 <moa> so block size is a potentially network splitting problem as well as de-decentralisation?
334 2015-01-03 23:14:39 <Jouke> I think all my tor peers were disconnected because they were too slow.
335 2015-01-03 23:23:39 <Jouke> Ok, found in the logs as well. But indeed, all the tor nodes I was connected to during initial sync were disconnected.
336 2015-01-03 23:31:01 <gmaxwell> Jouke: actually disconnected.
337 2015-01-03 23:32:41 <gmaxwell> moa: yes, but there isn't a clear bright line. Like, pretty obviously almost everyone can keep up with 1MB blocks... and pretty obviously only a large corporations could keep up with 10 gigabyte blocks. Exactly what the decenteralization vs operating cost curve is... is unknown. And the costs change over time as bandwidth/storage/cpu change in price.
338 2015-01-03 23:33:42 <moa> sounds like an optimisation problem
339 2015-01-03 23:35:16 <Jouke> I am nou hosting a full node at a cost of 2 euro's per month. (still have to see how reliable it actually is)
340 2015-01-03 23:36:06 <Jouke> That is the cheapest node I have atm.
341 2015-01-03 23:36:48 <justanotheruser> maybe we can extrapolate on the number of full nodes vs block size and extrapolate to estimate the number of full nodes at any block size
342 2015-01-03 23:37:12 <justanotheruser> s/can extrapolate on/can take/
343 2015-01-03 23:57:42 <gmaxwell> justanotheruser: Doubt it works like that. :)