1 2013-07-21 03:42:34 <swulf--> LevelDB read failure: Corruption: block checksum mismatch / *** System error: Database corrupted / Error: System error: Database corrupted / Error: System error: Database corrupted .. is there a plasant way of handling this?
2 2013-07-21 04:19:42 <Luke-Jr> swulf--: -reindex
3 2013-07-21 04:20:00 <Luke-Jr> although that's still downtime (and misreporting until it's done!)
4 2013-07-21 04:20:34 <swulf--> yeah, running reindex. the wallet.dat is soooooooooo slow...
5 2013-07-21 04:20:42 <swulf--> but it doesn't help that i have 600k+ keys in the wallet
6 2013-07-21 04:21:03 <swulf--> *the wallet.dat load
7 2013-07-21 05:31:58 <CodeShark> Pull Tester: unknown location(0): fatal error in "CreateNewBlock_validity": std::runtime_error: CDB() : can't open database file , error 2
8 2013-07-21 05:34:40 <CodeShark> BlueMatt, you around?
9 2013-07-21 05:34:40 <sipa> heh
10 2013-07-21 05:35:02 <sipa> it should use wallets in a gtem
11 2013-07-21 05:35:07 <sipa> in a temp dir
12 2013-07-21 05:36:49 <sipa> or in memory even
13 2013-07-21 06:24:22 <swulf--> ERROR: AcceptBlock() : AddToBlockIndex failed / ERROR: ProcessBlock() : AcceptBlock FAILED / Loaded 6069 blocks from external file in 187537ms / Reindexing block file blk00008.dat... / Flush(false)
14 2013-07-21 06:24:22 <swulf--> SetBestChain: new best=0000000000000afb91703d071ae630c4d15102026c7a008c704f16e7f42ec39c height=169793 log2_work=67.787176 tx=2560234 date=2012-03-05 20:52:32 progress=0.047699 / ProcessBlock: ACCEPTED / LevelDB read failure: Corruption: block checksum mismatch / *** System error: Database corrupted / Error: System error: Database corrupted / Error: System error: Database corrupted /
15 2013-07-21 06:24:41 <swulf--> How do I resolve this?
16 2013-07-21 06:34:34 <sipa> have you run with -reindex ?
17 2013-07-21 06:34:47 <sipa> apparently yes
18 2013-07-21 06:35:17 <sipa> was this during a reindex?
19 2013-07-21 06:35:41 <swulf--> well
20 2013-07-21 06:35:49 <swulf--> i ran a reindex because of a corruption message
21 2013-07-21 06:35:58 <swulf--> and this message came out and stopped the reindex
22 2013-07-21 06:36:12 <sipa> what os/hardware?
23 2013-07-21 06:36:14 <swulf--> so reindex didn't fix the corruption?
24 2013-07-21 06:36:30 <sipa> it does, it wipes the database and rebuilds it from scratch
25 2013-07-21 06:36:33 <swulf--> Linux bvc 2.6.32-5-686-bigmem #1 SMP Mon Feb 25 01:53:47 UTC 2013 i686 GNU/Linux
26 2013-07-21 06:36:51 <sipa> but there must be something on your system causing it to get corrupted again
27 2013-07-21 06:37:02 <swulf--> interesting?
28 2013-07-21 06:37:04 <swulf--> how
29 2013-07-21 06:37:13 <swulf--> bitcoind is the only thing running
30 2013-07-21 06:37:21 <sipa> what hardware?
31 2013-07-21 06:37:47 <swulf--> specifically?
32 2013-07-21 06:37:58 <swulf--> intel atom cpu
33 2013-07-21 06:38:02 <sipa> cpu, disk, memory
34 2013-07-21 06:38:07 <swulf--> 4gb ram
35 2013-07-21 06:38:39 <sipa> disk?
36 2013-07-21 06:38:50 <swulf--> forgot how to check the disk identifier in linux
37 2013-07-21 06:39:02 <swulf--> 2TB disk
38 2013-07-21 06:39:10 <sipa> encrypted?
39 2013-07-21 06:39:14 <swulf--> yes
40 2013-07-21 06:39:21 <sipa> how?
41 2013-07-21 06:39:27 <swulf--> loop-AES
42 2013-07-21 06:40:34 <sipa> could you try bitcoind outside of the encrypted fs?
43 2013-07-21 06:40:46 <swulf--> hmm
44 2013-07-21 06:40:49 <swulf--> not trivially
45 2013-07-21 06:41:05 <swulf--> but i suppose i could
46 2013-07-21 06:41:18 <sipa> i don't know whether it is related, but i would really like to know what is causimg this
47 2013-07-21 06:41:32 <swulf--> you've seen it before?
48 2013-07-21 06:41:39 <sipa> we frequently get corruption reports, but rarely from people running linux
49 2013-07-21 06:42:17 <swulf--> I would be dumbfounded if the encryption was at fault - it's supposed to be transparent... and the majority of the disk is using loop-aes, even our mysql db, which doesn't have any corruption
50 2013-07-21 06:42:46 <sipa> well the way that leveldb accesses the fs may be different
51 2013-07-21 06:43:09 <sipa> another thing that's perhaps worth trying is runnijng memtest86 for a while
52 2013-07-21 06:43:16 <swulf--> a userspace app would have no access to the disk outside of the mountpoint, though
53 2013-07-21 06:43:19 <sipa> to see if you have memory problems
54 2013-07-21 06:43:34 <sipa> yeah, i know
55 2013-07-21 06:44:03 <sipa> it should be transparent
56 2013-07-21 06:44:05 <swulf--> I'm running reindex a second time to see if the corruption occurs in the same place
57 2013-07-21 06:44:18 <sipa> it won't
58 2013-07-21 06:44:24 <swulf--> if it does...?
59 2013-07-21 06:44:37 <sipa> then something very weird is going on
60 2013-07-21 06:44:48 <sipa> it's likely to fail at a random place
61 2013-07-21 06:44:54 <sipa> if it faiols frewquently
62 2013-07-21 06:44:54 <swulf--> last time it died at 169793... current reindex is up to 150000
63 2013-07-21 06:44:56 <swulf--> so soon
64 2013-07-21 06:45:03 <sipa> oww
65 2013-07-21 06:45:08 <sipa> fails frequently
66 2013-07-21 06:46:37 <sipa> you can increase the db cache size to speed up reindexing
67 2013-07-21 06:46:48 <swulf--> command line or config file?
68 2013-07-21 06:46:53 <sipa> -dbcache=1000 to give it 1 GB
69 2013-07-21 06:46:56 <sipa> either
70 2013-07-21 06:47:05 <swulf--> happen to know the current default?
71 2013-07-21 06:47:09 <sipa> 25
72 2013-07-21 06:47:15 <swulf--> ok
73 2013-07-21 06:47:35 <sipa> oh, 32-bit or 64-bit build?
74 2013-07-21 06:48:20 <swulf--> 32-bit
75 2013-07-21 06:48:48 <sipa> ok, don't go over dbcache 1000 or 1500 or so
76 2013-07-21 06:48:59 <swulf--> right
77 2013-07-21 06:52:26 <swulf--> just surpassed height 170k
78 2013-07-21 06:52:48 <sipa> i have to go, please let me know what happened
79 2013-07-21 06:52:53 <swulf--> ok, will do
80 2013-07-21 06:52:55 <swulf--> thanks
81 2013-07-21 08:21:03 <BlueMatt> CodeShark: pong
82 2013-07-21 09:04:12 <CodeShark> Hi, BlueMatt
83 2013-07-21 09:05:07 <CodeShark> so PullTester is apparently struggling to open the database
84 2013-07-21 09:05:44 <BlueMatt> what database?
85 2013-07-21 09:08:04 <CodeShark> Pull Tester: unknown location(0): fatal error in "CreateNewBlock_validity": std::runtime_error: CDB() : can't open database file , error 2
86 2013-07-21 09:08:12 <BlueMatt> ohh
87 2013-07-21 09:09:42 <BlueMatt> CodeShark: link?
88 2013-07-21 09:10:08 <CodeShark> http://jenkins.bluematt.me/pull-tester/a995ad65d6a584f2f6a7523734b510ebbd7fac68/
89 2013-07-21 09:11:02 <BlueMatt> hmm...I wonder if gavin broke it
90 2013-07-21 09:11:22 <BlueMatt> CodeShark: I reset that commit, but it didnt reset (is it a pull's head?)
91 2013-07-21 09:11:40 <CodeShark> yes
92 2013-07-21 09:12:17 <CodeShark> pull 2841
93 2013-07-21 09:12:46 <CodeShark> or hmm
94 2013-07-21 09:12:56 <CodeShark> sorry, this commit: http://jenkins.bluematt.me/pull-tester/ce2c87ffe48502656eaa4e9cc657afc501aec91d/
95 2013-07-21 09:12:58 <CodeShark> same error
96 2013-07-21 09:20:33 <CodeShark> and now the head of that pull is 476f73e5ac0cc344eeaaa6e6d13e11f992282375
97 2013-07-21 09:43:12 <BlueMatt> CodeShark: no idea, and I cant fix it now....
98 2013-07-21 09:43:55 <CodeShark> oh well - could it at least be turned off as to not clutter up the pull request discussion?
99 2013-07-21 09:44:09 <BlueMatt> yea, pull-tester is off
100 2013-07-21 09:44:33 <CodeShark> excellent, thanks
101 2013-07-21 09:44:50 <CodeShark> could the comments be removed as well?
102 2013-07-21 10:05:07 <CodeShark> why doesn't getbalance "*" 0 return the total balance (including unconfirmed) anymore?
103 2013-07-21 10:08:43 <CodeShark> if (!wtx.IsConfirmed()) continue;
104 2013-07-21 10:08:48 <CodeShark> that doesn't seem like correct behavior
105 2013-07-21 10:09:09 <CodeShark> I used to use getbalance "*" 0 a lot in the past to check balance (including unconfirmed)
106 2013-07-21 10:16:12 <CodeShark> ARGH! https://github.com/bitcoin/bitcoin/issues/172
107 2013-07-21 10:17:12 <CodeShark> so what's the new way to get full balance (including unconfirmed) for all accounts?
108 2013-07-21 11:51:34 <bmcgee> hey with approx. 1300 khash/sec what's a ballpark time for mining a difficulty 1 block?
109 2013-07-21 11:52:30 <gribble> 3303.82099692
110 2013-07-21 11:52:30 <sipa> ;;calc 2**32 / 1300000
111 2013-07-21 11:52:34 <sipa> an hour
112 2013-07-21 11:52:53 <swulf--> sipa: reindex still going this time around...
113 2013-07-21 11:53:01 <swulf--> many, many hours later..
114 2013-07-21 11:53:30 <sipa> ok
115 2013-07-21 11:53:40 <sipa> i don't expect that to do fast on an atom cpu
116 2013-07-21 11:54:05 <swulf--> i wasn't expecting blazing fast, but not this long either :)
117 2013-07-21 11:54:24 <bmcgee> so the general calc is 2^32 / hashes per sec
118 2013-07-21 11:54:39 <sipa> the correct one is actually
119 2013-07-21 11:54:42 <swulf--> it'd be really awesome if reindex could be start/stopped at certain a block or tx index so that it could be interrupted
120 2013-07-21 11:54:57 <sipa> swulf--: you can arbitrarily stop and continue
121 2013-07-21 11:55:04 <sipa> just exit the client, and it will continue next time
122 2013-07-21 11:55:18 <gribble> 3303.87141
123 2013-07-21 11:55:18 <sipa> ;;calc 2**48 / 65535 / 1300000
124 2013-07-21 11:55:30 <sipa> but as you can see, 2**32 is a pretty good approximation :)
125 2013-07-21 11:55:41 <bmcgee> yeah lol cool good to know
126 2013-07-21 11:55:49 <swulf--> sipa: ctrl-z? :P
127 2013-07-21 11:56:04 <sipa> swulf--: KILL
128 2013-07-21 11:56:05 <swulf--> oh, really?
129 2013-07-21 11:56:10 <bmcgee> sipa: seems integration testing with my little mac mini???. not a viable option. If only I had myself an ASIC...
130 2013-07-21 11:56:11 <swulf--> but /without/ -reindex, right?
131 2013-07-21 11:56:26 <sipa> swulf--: indeed, -reindex will start over
132 2013-07-21 11:56:39 <swulf--> if a corruption occurs during the reindex, then i have no choice but to start over?
133 2013-07-21 11:56:45 <sipa> indeed
134 2013-07-21 11:57:01 <sipa> you can create a backup of chainstate
135 2013-07-21 11:57:08 <sipa> (while the client is not running)
136 2013-07-21 11:57:08 <swulf--> is it a single file?
137 2013-07-21 11:57:10 <sipa> no
138 2013-07-21 11:57:36 <swulf--> what should be backed up?
139 2013-07-21 11:57:55 <sipa> chainstate
140 2013-07-21 11:58:04 <sipa> $DATADIR/chainstate
141 2013-07-21 11:58:32 <swulf--> awesome
142 2013-07-21 11:58:52 <swulf--> ~250mb
143 2013-07-21 12:00:43 <swulf--> sipa: what kind of data is stored in the chainstate db?
144 2013-07-21 12:00:51 <sipa> the UTXO set
145 2013-07-21 12:01:25 <swulf--> are utxo's with amount=0 stored?
146 2013-07-21 12:01:31 <sipa> yes, they have to
147 2013-07-21 12:01:38 <sipa> as they can legally be spent
148 2013-07-21 12:01:38 <swulf--> because they can technically be unspent ?
149 2013-07-21 12:01:42 <swulf--> s/un//
150 2013-07-21 12:01:46 <sipa> yes
151 2013-07-21 12:01:51 <swulf--> makes sense
152 2013-07-21 12:10:48 <gmaxwell> I think its surprising that its taking many hours though.
153 2013-07-21 12:11:06 <swulf--> quite literally, about 4 hours now?
154 2013-07-21 12:11:19 <swulf--> with -dbcache=1000
155 2013-07-21 12:11:37 <sipa> gmaxwell: on an atom cpu?
156 2013-07-21 12:11:41 <sipa> in 32-bit mode
157 2013-07-21 12:11:52 <sipa> probably just 1 or 2 cores for sig checking
158 2013-07-21 12:15:22 <gmaxwell> Fair enough. To some extent "many, many hours later" was getting translated to 18 in my mind, not 4.
159 2013-07-21 12:15:32 <swulf--> Sorry :)
160 2013-07-21 12:16:24 <sipa> swulf--: what block are you at?
161 2013-07-21 12:17:26 <swulf--> 230011
162 2013-07-21 12:17:42 <swulf--> since about 225k, it takes about 10 seconds to process 3-4 blocks
163 2013-07-21 12:18:02 <sipa> yeah, signature checking
164 2013-07-21 12:18:10 <swulf--> ooo just squeezed in 5 :P
165 2013-07-21 12:18:17 <swulf--> probably a block with low tx count
166 2013-07-21 12:20:38 <gmaxwell> Atom also has some odd performance properties. E.g. bitscanreverse is like 1 cycle on i7 and like 20 cycles on atom.
167 2013-07-21 12:21:42 <sipa> yes, but who needs _that_ ?
168 2013-07-21 12:23:10 <gmaxwell> sipa: you use it in entropy coders. :) it's not the only thing that is odd on atom though its one of the worst differences.
169 2013-07-21 12:23:32 <gmaxwell> The multipliers on atom have much lower throughput too, though not 20x slower.
170 2013-07-21 12:25:35 <sipa> (it was a probably way too obscure reference to xkcd 619)
171 2013-07-21 12:25:57 <gmaxwell> haha!
172 2013-07-21 12:26:50 <sipa> ACTION tests headers-first sync
173 2013-07-21 12:26:56 <sipa> well, headers-first-and-then-nothing
174 2013-07-21 12:27:03 <sipa> ok, segfault
175 2013-07-21 12:27:05 <gmaxwell> hahah.
176 2013-07-21 12:27:09 <swulf--> sipa: progress!
177 2013-07-21 12:27:32 <gmaxwell> "I have completed the invention! ??? well actually it's not the invention, but the outline of the invention??? Oh, it crashed"
178 2013-07-21 12:27:43 <sipa> gmaxwell: remarkably, i've now removed more code than i added
179 2013-07-21 12:27:52 <sipa> (all stuff for orphan blocks is a lot)
180 2013-07-21 12:27:54 <gmaxwell> sipa: this is usually a sign that you're doing the right thing.
181 2013-07-21 12:29:18 <sipa> ACTION goes on a rampage and removes the GUI
182 2013-07-21 12:29:31 <gmaxwell> but wait! we need that!
183 2013-07-21 12:29:36 <sipa> do you?
184 2013-07-21 12:30:10 <gmaxwell> Me personally? no. I never use it. But!
185 2013-07-21 12:30:21 <sipa> ok, gone
186 2013-07-21 12:30:22 <sipa> sorry
187 2013-07-21 12:30:35 <sipa> removed from git history too
188 2013-07-21 12:30:41 <swulf--> sipa: GUI for what?
189 2013-07-21 12:31:18 <sipa> this -Qt thing
190 2013-07-21 12:31:21 <sipa> nobody needs it
191 2013-07-21 12:31:25 <CodeShark> lol
192 2013-07-21 12:31:35 <swulf--> ACTION shrugs.
193 2013-07-21 12:31:39 <swulf--> I certainly don't
194 2013-07-21 12:32:10 <Luke-Jr> I do!
195 2013-07-21 12:32:24 <Luke-Jr> if you remove it, I'll have to revive Spesmilo
196 2013-07-21 12:32:52 <gmaxwell> wait. so if we remove the GUI you'll make something better?
197 2013-07-21 12:33:04 <gmaxwell> I'm failing to see the problem anymore!
198 2013-07-21 12:33:55 <Luke-Jr> I didn't say better <.<
199 2013-07-21 12:34:24 <CodeShark> what about moving the GUI over to a separate process? :)
200 2013-07-21 12:36:11 <gmaxwell> CodeShark: I think the general consensus has been that it should be seperate for a long time.. and it's been grrradddullllyyy moving in that direction.
201 2013-07-21 12:36:36 <sipa> with headers-first sync, SPV mode comes a lot closer
202 2013-07-21 12:36:40 <Luke-Jr> gmaxwell: you missed an 'a'
203 2013-07-21 12:36:42 <sipa> once SPV is there
204 2013-07-21 12:36:52 <sipa> you can split off full validation to another process easily
205 2013-07-21 12:37:16 <gmaxwell> Really the wallet should become a SPV client, with some special stuff for a richer connection to a trusted backing node when it has one.
206 2013-07-21 12:37:19 <CodeShark> the only question is whether it makes more sense to have the GUI do SPV or to define a trusted protocol that wallet clients can use to connect to the validation engine
207 2013-07-21 12:37:39 <Luke-Jr> ACTION doesn't think the wallet should be in the GUI either <.<
208 2013-07-21 12:37:57 <sipa> validation node <-> wallet <-> GUI/RPC
209 2013-07-21 12:38:00 <Luke-Jr> ^
210 2013-07-21 12:38:05 <sipa> the wallet can be a separate server or not
211 2013-07-21 12:38:26 <gmaxwell> CodeShark: It should probably be SPV, but that doesn't preclude it also speaking the RPC when running against a trusted node.
212 2013-07-21 12:38:34 <Luke-Jr> as long as the wallet<->UI protocol is a single stream, it can easily be the same over stdio or socket
213 2013-07-21 12:39:34 <CodeShark> there are at least two completely different types of wallet use models: single enduser and enterprise
214 2013-07-21 12:39:55 <CodeShark> enterprise wallets need to support high concurrency and good access controls
215 2013-07-21 12:39:55 <gmaxwell> CodeShark: In particular, it's kinda lame to ever have your wallet process exposed directly to potential hostile nodes.
216 2013-07-21 12:40:52 <fanquake> Autonomous agents?
217 2013-07-21 12:40:57 <gmaxwell> So a SPV wallet connecting back only a trusted bitcoind is still a good model to have. .. but if you're doing that you'll want to be able to get things like notification that the bitcoind has no connections.
218 2013-07-21 12:41:32 <Luke-Jr> no OTHER connections <.<
219 2013-07-21 12:41:36 <CodeShark> sure, the protocol would have to be some sort of subscription model
220 2013-07-21 12:43:01 <CodeShark> the client should be able to subscribe to different classes of messages, including messages indicating a change of server state, messages indicating new bitcoin events, etc...
221 2013-07-21 12:43:03 <gmaxwell> CodeShark: I think that polling is adequate for that kind of thing. Blocks and txn get pushed to spv nodes in any case.
222 2013-07-21 12:43:16 <CodeShark> why poll when you can have full duplex communication?
223 2013-07-21 12:43:20 <gmaxwell> (not saying that polling is ideal or anything, but its probably adequate)
224 2013-07-21 12:44:01 <CodeShark> in fact, perhaps it even makes sense to stick some message broker in between
225 2013-07-21 12:44:17 <petertodd> sipa: headers-first-sync, talk to me about my scheme for having nodes do succinct non-interactive proofs of the total work of the block headers they posess
226 2013-07-21 12:44:37 <sipa> petertodd: elaborate
227 2013-07-21 12:44:44 <sipa> \\o/
228 2013-07-21 12:44:49 <sipa> ACTION haz a headers-synced node
229 2013-07-21 12:45:01 <sipa> it took a minute or so
230 2013-07-21 12:45:38 <petertodd> sipa: Build a merkle sum tree (or better yet, merkle sum mountain range) of every block header you have, then use the resulting hash at the end to pick n random block headers, and provide merkle paths to your tip.
231 2013-07-21 12:46:12 <petertodd> sipa: n can be set such that it'd take 2^64 work to falsify p% of the headers with 50% probability
232 2013-07-21 12:46:31 <petertodd> sipa: with an interactive proof, you don't even need a big n
233 2013-07-21 12:46:33 <gmaxwell> CodeShark: mostly a instinctive response on my part.. because complicated protocols are complicated. And subscription messagebusses have a long history of very low third party adoption.
234 2013-07-21 12:47:06 <gmaxwell> CodeShark: certantly don't let my cynical reservations get in the way of actually doing something. :)
235 2013-07-21 12:47:15 <sipa> petertodd: basically tree-ifying the otherwise linear block chain?
236 2013-07-21 12:47:31 <gmaxwell> petertodd: whats N need to be for that?
237 2013-07-21 12:47:43 <sipa> anyway, the easy part is done
238 2013-07-21 12:47:49 <sipa> now getting the actual blocks :D
239 2013-07-21 12:47:52 <CodeShark> message brokers might be overkill for single enduser apps - but might be great in an enterprise setting where you want high redundancy and logical isolation
240 2013-07-21 12:47:56 <gmaxwell> sipa: the treeifying there just make the proofs that the headers are part of the committed tree compact.
241 2013-07-21 12:48:05 <sipa> yeah
242 2013-07-21 12:48:37 <gmaxwell> petertodd: how many do you need in the non-interactive case?
243 2013-07-21 12:48:45 <gmaxwell> ;;bc,blocks
244 2013-07-21 12:48:46 <gribble> 247753
245 2013-07-21 12:49:25 <petertodd> sipa: for instance if p=0.5, then (1-p)^n=2^(-64) if p=0.5 then you need n=64
246 2013-07-21 12:49:43 <sipa> 49 seconds to sync headers
247 2013-07-21 12:49:50 <sipa> and the largest factor is definitely latency
248 2013-07-21 12:50:13 <sipa> (as it can't be pipelined; you need the last headers before you can send a new getheaders)
249 2013-07-21 12:50:25 <sipa> a protocol extension could improve that, but meh
250 2013-07-21 12:50:54 <petertodd> sipa: and with n=64 you can get away with 1% fraud half the time (though remember that's only the interactive case given calculating it is cheap)
251 2013-07-21 12:51:38 <gmaxwell> petertodd: I would think that you'd want to constrain the selection, but not the comitted tree to multiples of 2016.
252 2013-07-21 12:52:13 <petertodd> gmaxwell: Why? To prove diff changes?
253 2013-07-21 12:52:45 <petertodd> gmaxwell: Thing is total work is what you want to prove, constrain it and the attacker is proving less work.
254 2013-07-21 12:52:53 <sipa> my blocks index is now 33 MB
255 2013-07-21 12:54:25 <petertodd> gmaxwell: Note that block selection has to be biased, you don't just pick a random n, rather you pick a set of left-right directions, and then the probability at each step is based on the sum work claimed.
256 2013-07-21 12:54:58 <petertodd> gmaxwell: Essentially doing that evens out the blocks as though they were all the same work.
257 2013-07-21 12:56:08 <petertodd> gmaxwell: Incidentally, afaik this is a advance over what adam back said was the state of the art for providing succinct proofs that work was done with multiple PoW attempts, so I may be missing something...
258 2013-07-21 12:56:18 <gmaxwell> petertodd: sure, you have to do the same thing for value proofs. You have the sum diff left and sum diff right, and you pick a random number % total diff,
259 2013-07-21 12:56:46 <r0sc0e> No suitable long-poll found <- what does that mean? try to solomine with bfgminer
260 2013-07-21 12:56:57 <r0sc0e> is there a misconfiguration in my bitcoin.conf file?
261 2013-07-21 12:57:12 <petertodd> gmaxwell: Yup, and what's really crazy, is you can apply this stuff to a crypto-coin itself, thus making a system where mining is done by the miner providing proof that some subset of transactions were honest. This also institutionalizes fraud, so you can use it as the coin creation mechanism...
262 2013-07-21 12:58:05 <petertodd> gmaxwell: Unfortunately the numbers for a resonable block rate are ugly, like 500,000 proofs needed for a 5% inflation rate if you want 10min block times, and that 5% rate is based on 5% of the total txout value created, not the total amount of coins out there.
263 2013-07-21 12:58:29 <petertodd> gmaxwell: I'm sure if I read more SCIP papers I can improve on that though - I mean, it's what SCIP is all about...
264 2013-07-21 12:59:07 <petertodd> gmaxwell: Solve it though, and you've got a coin that can be mined in a very decentralized manner, and the shared consensus state doesn't scale with transaction volume, only inflation!
265 2013-07-21 13:00:46 <petertodd> gmaxwell: It's also a lot more private... You do need a spent coins list still, but spent coins lists can be sharded by H(txout:n) so no one person needs the full list. (optionally expire old txouts so the list can be eventually pruned)
266 2013-07-21 13:02:06 <petertodd> sipa: Incidentally, credit goes to gmaxwell for suggesting the idea of doing a (n)i proof for block headers, only the implementation is my idea and that's pretty obvious.
267 2013-07-21 13:06:24 <r0sc0e> is there a way to add some line in bitcoin.conf to get long-pool workin?
268 2013-07-21 13:07:23 <petertodd> gmaxwell: Conceptually understanding the proofs is actually really easy: I select 1/n txout spends, and include them in the block I distribute to everyone. Thus if I want to have a 50:50 chance of getting away with making a fraudulent block, I need to make no more than about 1/n transactions fraudulent. Now if creating a block takes 10 minutes of work, well, there's a balance between how many fake tx's I create that give coins to me, how likely my minin
269 2013-07-21 13:07:42 <petertodd> s/takes average 10 minutes of work/
270 2013-07-21 13:08:11 <petertodd> (remember these are coins I'm creating out of thin air, I spend them only by profing that there is a merkle path from the txout to a previous block header)
271 2013-07-21 13:08:16 <petertodd> *proving
272 2013-07-21 13:10:56 <petertodd> One approach could be to fix the total amount of fees you can collect in a block, and then provide a way to punish anyone you tries to put an invalid transaction in a block, and then make wallets get some long history of every txout they get to ensure that someone else isn't going to screw them over by showing *they* committed fraud... but that seems a bit weak.
273 2013-07-21 13:16:34 <r0sc0e> any ideas about my problem?
274 2013-07-21 13:17:39 <sipa> r0sc0e: bitcoind doesn't support long polling
275 2013-07-21 13:18:11 <r0sc0e> so what can i do to do long polling?
276 2013-07-21 13:18:21 <sipa> use a pool server
277 2013-07-21 13:18:28 <petertodd> r0sc0e: p2pool does long-polling
278 2013-07-21 13:18:32 <sipa> but perhaps better ask in #bitcoin-mining
279 2013-07-21 13:18:35 <r0sc0e> i try to solomine with bitcoin-qt and bfgminer
280 2013-07-21 13:18:42 <Luke-Jr> sipa: until someone merges my LP patch ;)
281 2013-07-21 13:19:05 <Luke-Jr> r0sc0e: be sure you look at solo instructions in README btw
282 2013-07-21 13:19:08 <sipa> r0sc0e: what hashrate do you have?
283 2013-07-21 13:19:10 <Luke-Jr> bfg's README I mean
284 2013-07-21 13:19:17 <r0sc0e> i did Luke-Jr
285 2013-07-21 13:19:19 <Luke-Jr> k
286 2013-07-21 13:20:26 <r0sc0e> sipa: about 100 ghash
287 2013-07-21 13:20:38 <sipa> decent :)
288 2013-07-21 13:23:18 <r0sc0e> sipa: yap thats "okay"
289 2013-07-21 13:24:34 <petertodd> gmaxwell: Oh, here this will work: with a fixed inflation rate coin you can have every txout in a block come with it's own proof-of-work, and sum up total work. The ni proof them picks random examples to show how sum work for the whole block. Now wallet software only has to go back far enough in history to show that the sum work done for their txout was > the txout's value to know if the txout is probably valid. What's nice is it ensures that making a t
290 2013-07-21 13:25:05 <gmaxwell> it ensures that making a t
291 2013-07-21 13:25:20 <gmaxwell> petertodd: I note: high words to code ratio today. :P
292 2013-07-21 13:25:41 <petertodd> > transaction is paid for each and every time by the inflation it causes, which in turn forces mining activity. Probably needs demurrange though so that 64-bit ints work... can simply shift coin values to the right periodically though, which then prunes the spent coin list.
293 2013-07-21 13:26:09 <petertodd> gmaxwell: The only difference from a normal day for me is the ratio has to be expressed in a double rather than a single-width float.
294 2013-07-21 13:33:09 <petertodd> gmaxwell: The best part, is all of the above can be shoved into Bitcoin with a soft-fork. Later on we can do a hard-fork to remove the requirement to have a linear block at all, and the hard-fork would then be where we change the economics... unlikely to happen, but it's interesting how it can be done.
295 2013-07-21 13:34:44 <petertodd> gmaxwell: Would get us a properly decentralized crypto-coin because absolutely nothing I've said can't be done by decentralized and distributed low-bandwidth nodes co-operating in pools. (obvs fidelity bonds to ensure nodes don't hold up ni proofs will be a must)
296 2013-07-21 13:36:40 <petertodd> (FWIW a nice trick here is to have your wallet only produce txouts for a particular part of the spent list, which means you can still maintain that part, and provide those you give coins to the proof that the txout being spent wasn't already spent!)
297 2013-07-21 13:37:45 <petertodd> (spent list would be arranged as a patricia tree of course, and it too needs ni proof sampling)
298 2013-07-21 13:40:09 <petertodd> Oh, and that means that as you go up the tree, you can figure out what proofs got sampled at every step... so actually the "miner at the pool goes away" problem isn't a problem - just give the next level up the proof if the ni said you needed one. You do need to make blocks random still, so a second PoW over the tip is probably right, or just constrain H(tip) < target2
299 2013-07-21 13:46:52 <gmaxwell> petertodd: so, if you go and do the math on faking an NI proof that way, it's not so super wonderful for making the proofs compact for a given security bound.
300 2013-07-21 13:48:12 <petertodd> gmaxwell: I know, that's why I'm saying you have to sidestep the problem by either institutionalizing fraud as coin creation, or creating a system where you are expected to have the history of any given txout going far enough back that work > txout value.
301 2013-07-21 13:49:12 <petertodd> gmaxwell: Note how the second essentially means that every time you spend a txout, a certain % of it's value vanishes unless you were the one to do the work to mine it.
302 2013-07-21 13:49:25 <gmaxwell> petertodd: okay thats actually pretty slick??? the institutionalizing fraud as coin creation??? but one problem with it is simultaniously keeping the fraud rate controllable while also not making large spends very expensive.
303 2013-07-21 13:49:39 <gmaxwell> ^ see the latter part
304 2013-07-21 13:50:10 <petertodd> gmaxwell: Indeed, but look at it this way: large spends *should* be very expensive, because they require a lot of security. It actually fixes the bad economics of the Bitcoin system re: the sustainability of mining.
305 2013-07-21 13:50:49 <petertodd> gmaxwell: Yeah, it's back to a % fee on txout value, but I don't think that's all that bad.
306 2013-07-21 13:51:52 <gmaxwell> I guess that gives an ultimate answer to my answer to the suggestion that people with high value txn would pay "performance bonds"... my answer to that is, derp why wouldn't they just mine themselves at least then they'd get to pick a chain that wasn't against their interest.
307 2013-07-21 13:53:02 <petertodd> Indeed, there may also be a way to sanely burn coins and turn that into pseudo-mining-effort too - like what I was saying about consensus level replace-by-fee scorched earth in my Zerocoin proposal.
308 2013-07-21 13:56:10 <petertodd> Of course, often people would just broadcast a tx lacking the PoW, and then miners would have to do the work themselves, with the rules determining the target for the work, and how they can collect that work into a txout.
309 2013-07-21 14:02:54 <petertodd> Oh, and this is cool: see how to do fraud properly we need to set the inflation rate in terms of % per txout created? See at the top level we can't prove anything about the sum of all txouts created with acceptably low levels of fraud, only locally can we do that.
310 2013-07-21 14:03:45 <petertodd> But this is actually good: we want inflation to have something to do with how much money is moving, which is more closely the money supply, (and security needs still) than we want it to be an absolute thing.
311 2013-07-21 14:03:46 <CheckDavid> Anyone wants to earn free Bitcoins just for chatting with me? Send me a PM =)
312 2013-07-21 14:04:06 <Luke-Jr> CheckDavid: spam is off-topic here
313 2013-07-21 14:04:11 <petertodd> If no-one spends money, coin creation actually stops... (aside from a bootstrapping process or somesuch)
314 2013-07-21 14:04:22 <CheckDavid> sorry
315 2013-07-21 14:06:35 <petertodd> Hmm... I guess we'll probably have coin-creation done in as a special transaction, so we can get shared consensus and keep that type of inflation low.
316 2013-07-21 14:07:46 <petertodd> (I think you need some way to create coins out of thin air to satisfy miners and get interest in the 'coin bootstrapped)
317 2013-07-21 14:08:20 <petertodd> (a coin/block forever should be fine)
318 2013-07-21 14:11:31 <Luke-Jr> strUsage += "\\n"; _("SSL options: (see the Bitcoin Wiki for SSL setup instructions)") + "\\n";
319 2013-07-21 14:11:40 <Luke-Jr> am I missing something? how does the string get used? :/
320 2013-07-21 14:11:59 <petertodd> lol, that's gotta be a bug
321 2013-07-21 14:19:51 <sipa> ;;blocks
322 2013-07-21 14:19:52 <gribble> 247767
323 2013-07-21 14:25:59 <gmaxwell> Luke-Jr: wtf why doesn't the compiler warn about a statement with no effect? I guess it thinks the translation macro has side effects? :P
324 2013-07-21 14:26:09 <Luke-Jr> gmaxwell: I guess :x
325 2013-07-21 14:26:10 <swulf--> some compilers will
326 2013-07-21 14:26:36 <gmaxwell> someone should go annotate that transaction function with the pure annotation. :P
327 2013-07-21 14:26:56 <swulf--> "(void)var;" is generally an acceptible method of silencing the "unused parameter" warning, though
328 2013-07-21 14:27:14 <gmaxwell> IIRC GCC will warn about ;pure_function();.
329 2013-07-21 14:27:26 <swulf--> -Wall maybe?
330 2013-07-21 14:28:24 <gmaxwell> swulf--: the translation macro likely calls a function which is probably not annotated as pure. (and perhaps it can't be, e.g. if it has to allocate a string...)
331 2013-07-21 14:28:29 <sipa> it needs some extension the function is pure
332 2013-07-21 14:28:37 <sipa> __attribute__((pure)) or so
333 2013-07-21 14:29:03 <swulf--> is _() being used in a pure context a serious problem?
334 2013-07-21 14:29:17 <sipa> no
335 2013-07-21 14:29:19 <gmaxwell> swulf--: apparently we had the mistake twice! :P
336 2013-07-21 14:29:19 <sipa> it's just pointless
337 2013-07-21 14:29:45 <gmaxwell> swulf--: lots of useful warnings happen for things which themselves are harmless but indicate that you very likely made a mistake.
338 2013-07-21 14:29:45 <swulf--> __attribute__((pure)) could add to difficulties porting?
339 2013-07-21 14:29:51 <gmaxwell> swulf--: nah.
340 2013-07-21 14:30:08 <swulf--> gmaxwell: sure, absolutely agree
341 2013-07-21 14:30:10 <gmaxwell> swulf--: you make it a FOO_PURE macro, and use it only when available, e.g. wrapped in a GCC test.
342 2013-07-21 14:30:18 <swulf--> ACTION nods
343 2013-07-21 14:39:41 <gfawkes> *yawn* internet trollz are so predictable and only mildly amusing in their antics on a user submitted content form
344 2013-07-21 14:39:57 <sipa> "In other news, ..."
345 2013-07-21 14:42:07 <gfawkes> unfortunately as a dev, it means having to waste my time adding stupid content checks to deter the morons
346 2013-07-21 15:06:20 <bmcgee> Hey I'm running bitcoin-qt on a mac. Is it possible to start it with --blocknotify just like with bitcoind?
347 2013-07-21 15:08:39 <Luke-Jr> pretty sure yes
348 2013-07-21 15:08:43 <Luke-Jr> but it's just one dash
349 2013-07-21 15:08:45 <Luke-Jr> -blocknotify='command'
350 2013-07-21 15:09:07 <Luke-Jr> (no, I don't know why we break standard for commandline parsing)
351 2013-07-21 15:11:57 <bmcgee> i don't see how i can pass it a command line param, can't be invoked from the command line. I don't a config option for specifying it either in bitcoin.conf
352 2013-07-21 15:12:05 <bmcgee> *don't see a config
353 2013-07-21 15:12:22 <swulf--> Luke-Jr: because Satoshi?
354 2013-07-21 15:12:43 <Luke-Jr> bmcgee: bitcoin.conf is the command line.
355 2013-07-21 15:13:23 <bmcgee> Luke-Jr: so stick it in bitcoin.conf then?
356 2013-07-21 15:13:41 <bmcgee> it just concatenates the conf file into a set of command line params?
357 2013-07-21 15:14:38 <Luke-Jr> more or less
358 2013-07-21 15:15:05 <Luke-Jr> obviously no - in the config
359 2013-07-21 15:16:10 <bmcgee> yup
360 2013-07-21 15:16:30 <Luke-Jr> I think you don't need quotation either
361 2013-07-21 15:17:06 <gribble> 2.61628756825699E7
362 2013-07-21 15:17:06 <sipa> ;;diff
363 2013-07-21 15:21:31 <fanquake> bmcgee you need to pass --args first then -blocknotify
364 2013-07-21 15:21:43 <sipa> ?
365 2013-07-21 15:22:24 <bmcgee> I second that ?
366 2013-07-21 15:22:52 <fanquake> bmcgee I assume your using
367 2013-07-21 15:23:00 <fanquake> open Bitcoin-Qt.app
368 2013-07-21 15:23:13 <fanquake> if you want to pass command line arguments you need todo
369 2013-07-21 15:23:18 <sipa> oh, osx stuffz
370 2013-07-21 15:23:22 <fanquake> open Bitcoin-Qt.app --args -command
371 2013-07-21 15:23:25 <bmcgee> i was told i could add it bitcoin.conf instead
372 2013-07-21 15:23:34 <bmcgee> sans the -
373 2013-07-21 15:23:37 <sipa> yes, that's easier
374 2013-07-21 15:23:46 <fanquake> yes bitcoin.conf is fine
375 2013-07-21 15:23:55 <bmcgee> cool
376 2013-07-21 15:23:58 <bmcgee> just trying that now
377 2013-07-21 15:58:33 <bmcgee> after faffing around a bit and realising I had the wrong ip address I managed to get the block notify working. Thx Luke-Jr, sipa and fanquake
378 2013-07-21 17:53:32 <bmcgee> Got some basic mining questions, so brace yourselves for some 'stupid'
379 2013-07-21 17:54:31 <bmcgee> Miner hits up pool server for a block template. It can specify a nonces field if it wants specifying the nonce range it needs. Similarly in the response a noncerange can be specified to tell a miner the range of nonces it can scan
380 2013-07-21 17:54:58 <bmcgee> so a miner gets a response, creates a block and starts trying to find a hash which meets the difficulty requirements
381 2013-07-21 17:55:10 <bmcgee> it creates more possibilities by incrementing the nonce
382 2013-07-21 17:55:17 <bmcgee> all correct so far?
383 2013-07-21 17:55:19 <sipa> yes
384 2013-07-21 17:55:28 <bmcgee> cool
385 2013-07-21 17:56:05 <bmcgee> so if you have n miners it's best to ensure they're working with different nonce ranges to maximise the work done. Correct?
386 2013-07-21 17:56:21 <sipa> you just give them different work
387 2013-07-21 17:56:37 <sipa> i think few implementations actually bother limiting nonce ranges
388 2013-07-21 17:56:53 <bmcgee> can you clarify the work in that statement
389 2013-07-21 17:57:03 <sipa> the block being mined
390 2013-07-21 17:57:13 <sipa> in particular, its coinbase transaction
391 2013-07-21 17:57:41 <bmcgee> ah so there is a mutable section in the coinbase that can be tweaked to allow each miner to have effectively a different block?
392 2013-07-21 18:13:33 <davec> bmcgee: yeah the coinbase script is serialized block height (required as of block version 2) then some extra stuff like an extra nonce and some miners also put flags in there for supported features
393 2013-07-21 18:14:58 <bmcgee> davec: the scriptSig is a max of 100 bytes right? i thought version 2 is not widely supported at the moment?
394 2013-07-21 18:15:34 <davec> everything is version 2 for a while (network rule in place where once 95% of the network had upgraded, version 1 are rejected)
395 2013-07-21 18:15:43 <davec> yes, coinbase script between 2 and 100
396 2013-07-21 18:16:40 <bmcgee> BIP 0034 details the contents of scriptSig for version 2 compliance correct?
397 2013-07-21 18:17:15 <davec> https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L2314-L2322 is the relevant code
398 2013-07-21 18:17:48 <bmcgee> thx
399 2013-07-21 18:17:54 <davec> correct on BIP0034
400 2013-07-21 18:20:18 <bmcgee> ha ha if i ONLY want my software to work for the next 300 years I can hardcode the first byte of the serialised height to 0x03??????
401 2013-07-21 18:20:54 <davec> well there are going to be issues before that without code changes anyways
402 2013-07-21 18:21:10 <davec> timestamps in several places use uint32 -- i.e 2106 max
403 2013-07-21 18:21:22 <bmcgee> if my software is running 5 years from now i'll be impressed lol
404 2013-07-21 18:22:07 <bmcgee> usually before then some other developer comes along, says it's all shit, let's rewrite and the circle of life continues...
405 2013-07-21 18:22:54 <sipa> davec: not a problem
406 2013-07-21 18:23:07 <sipa> even with 16 bits timestamps it would be doable
407 2013-07-21 18:23:51 <sipa> just assume the higher bits increase when the lower ones decrease more than a few hours
408 2013-07-21 18:24:32 <sipa> bmcgee: it's not about the software but the protocol
409 2013-07-21 18:24:41 <davec> agreed, it's completely solvable, I was just saying code changes will be needed far before the 300 years when the number of bytes needed for the varint representing the serialized height in the coinbase script happens
410 2013-07-21 18:24:57 <sipa> and everything that is part of bitcoin's consensus rules are extremely hard to change
411 2013-07-21 18:25:04 <sipa> ha, true
412 2013-07-21 18:25:11 <sipa> unlikely to be a problem :)
413 2013-07-21 18:25:12 <bmcgee> sipa: very true. it's impressive the timeframe taking into account when designing bitcoin
414 2013-07-21 18:25:19 <bmcgee> *taken into account
415 2013-07-21 18:26:39 <bmcgee> in the scriptSig is the height encoded as a VarInt then? I see the spec in the BIP but if it's the same as the general var int used throughout bitcoin i already have an implementation for that
416 2013-07-21 18:26:54 <davec> correct
417 2013-07-21 18:26:58 <bmcgee> sweet
418 2013-07-21 18:27:07 <sipa> afaik it'$
419 2013-07-21 18:27:19 <sipa> afaik it is encoded as byte-series push
420 2013-07-21 18:27:23 <sipa> not as a varint
421 2013-07-21 18:27:38 <sipa> a varint would be 5 bytes
422 2013-07-21 18:29:02 <bmcgee> ah
423 2013-07-21 18:29:14 <davec> ah right, number of bytes, little endian height
424 2013-07-21 18:29:37 <sipa> ;;calc 2**24 / 210000 * 4
425 2013-07-21 18:29:38 <gribble> 319.566019048
426 2013-07-21 18:29:41 <bmcgee> yeah, that would have been too easy :). So hardcode number of bytes (0x03), little endian encoding
427 2013-07-21 18:30:40 <davec> would start with a 0xfe if it were a varint iirc
428 2013-07-21 18:30:47 <sipa> correct
429 2013-07-21 18:31:11 <TheLordOfTime> don't... know... why... i was opped...
430 2013-07-21 18:31:12 <TheLordOfTime> o.O
431 2013-07-21 18:33:13 <gmaxwell> wasn't me.
432 2013-07-21 18:34:32 <gmaxwell> bmcgee: there are quite a few 2038 problems in the reference software as it... for example, boost's timers return instantly for times past 2038... which will actually cause issues long before 2038.
433 2013-07-21 18:34:45 <gmaxwell> (because some of the timers are long ones)
434 2013-07-21 18:35:40 <bmcgee> yeah its like sipa said, the protocol has a helluva lot of longevity, implementation has a ways to go and will always need maintenance
435 2013-07-21 19:12:54 <TheLordOfTime> gmaxwell: honestly, I'm baffled as to why i was opped, i mean... unless i missed something while freenode was splitting...
436 2013-07-21 19:14:40 <gmaxwell> TheLordOfTime: it may have been some split weirdness from when you were short term opped previously.
437 2013-07-21 19:22:58 <TheLordOfTime> ACTION shrugs
438 2013-07-21 19:23:03 <TheLordOfTime> gmaxwell: possibly, but i dunno
439 2013-07-21 20:25:32 <serp> hey fellers.. how it goes today?
440 2013-07-21 20:25:51 <serp> well wrong channel but hi anyways
441 2013-07-21 20:27:45 <bitanarchy> wy does bitcoin not require privoxy to run over tor?
442 2013-07-21 20:29:29 <sipa> why would it?
443 2013-07-21 20:30:31 <bitanarchy> sipa: well you need a proxy to run tor right...
444 2013-07-21 20:31:09 <sipa> tor is a SOCKS5 proxy
445 2013-07-21 20:31:12 <mhanne> bitanarchy: tor has direct socks support. privoxy is just needed to clean private info from your http requests
446 2013-07-21 20:31:14 <sipa> that's all we need
447 2013-07-21 20:32:37 <bitanarchy> socks5 proxy is not a web proxy... so socks5 is on 9050 and privoxy is a web proxy which should be on on 9150/9151
448 2013-07-21 20:37:03 <Luke-Jr> bitanarchy: Bitcoin is not web.
449 2013-07-21 20:37:32 <Luke-Jr> SOCKS5 proxy is an internet proxy. Which is plenty fine for Bitcoin.
450 2013-07-21 21:48:34 <Tabaza> i want to build online trading for bitcoin
451 2013-07-21 21:48:46 <Tabaza> and to have many possible options
452 2013-07-21 21:48:53 <Tabaza> iamopen for investments.
453 2013-07-21 21:49:04 <Tabaza> looking for experts in bitcoins to guide me