1 2012-10-22 00:00:01 <Luke-Jr> theymos: what's the excuse for trolls like Diablo-D3 and kano?
  2 2012-10-22 00:00:19 <gmaxwell> theymos: This is not a novel idea; his fearmongering has not made a contribution. And the histeria makes me want to reconsider my involvement with bitcoin.
  3 2012-10-22 00:00:45 <gmaxwell> theymos: or at least if it expanded to more than him and his socks it would.
  4 2012-10-22 00:00:55 <Hasimir> it reminds me of the panic in the Linux world when the NSA announced SELinux
  5 2012-10-22 00:01:06 <gmaxwell> theymos: although the ironic thing is that he's gone so overboard he's actually making prudent people look stupid.
  6 2012-10-22 00:01:16 <Hasimir> panic from ppl who don't uderstand the diff between comsec and comint
  7 2012-10-22 00:01:36 <Hasimir> this is the same, but on a smaller scale
  8 2012-10-22 00:02:17 <theymos> gmaxwell: You wanna moderate Dev&Tech so you can remove those posts of his which are not constructive at all?
  9 2012-10-22 00:02:36 <Luke-Jr> theymos: I vote gmaxwell for global mod
 10 2012-10-22 00:02:38 <gmaxwell> I don't know what I want. :(
 11 2012-10-22 00:02:54 <gmaxwell> meh. my disposition lately is not sutable for moderation.
 12 2012-10-22 00:03:06 <Luke-Jr> gmaxwell: sure, but you don't abuse it even then
 13 2012-10-22 00:03:22 <Luke-Jr> worst case, IMO, you seem to just not moderate
 14 2012-10-22 00:03:24 <BlueMatt> 
ACTION votes for "the forum got ddos'd and we cant afford to keep paying the bills, sorry"

 15 2012-10-22 00:03:27 <Luke-Jr> which is what happens when you can't anyway
 16 2012-10-22 00:03:36 <Luke-Jr> BlueMatt: hah, +1
 17 2012-10-22 00:03:40 <gmaxwell> Luke-Jr: well I do, I got tired of kanos fear mongering about me editing posts, so I started adding silly messages to the end of his messages every time he accused me of it.
 18 2012-10-22 00:03:59 <Luke-Jr> gmaxwell: eh, has anyone complained about that? I thought everyone took it in good humour
 19 2012-10-22 00:03:59 <theymos> Regarding safety, I've always wanted a separate stable release that is 6-12 months behind the main/"unstable" release in terms of features.
 20 2012-10-22 00:04:08 <Luke-Jr> theymos: we have those now???
 21 2012-10-22 00:04:28 <gmaxwell> theymos: stable / unstable is not the right metric for a potentially forking change.
 22 2012-10-22 00:04:31 <Luke-Jr> pretty sure 0.5.x is 6 months old
 23 2012-10-22 00:04:39 <Luke-Jr> 0.4.x certainly is
 24 2012-10-22 00:04:56 <Hasimir> gmaxwell, gotta love the one who can't do anything contructive, so they waste everyone's time attacking others
 25 2012-10-22 00:04:59 <gmaxwell> theymos: worse a lot of our mandatory must deploy now fixes have been of a magnitude that I would not trust backports.
 26 2012-10-22 00:05:38 <theymos> Luke-Jr: Ah, right. There was some reason why I stopped using your stable releases, but I forgot what it was. It should be linked from bitcoin.org somewhere.
 27 2012-10-22 00:05:53 <Luke-Jr> theymos: testnet3 probably
 28 2012-10-22 00:06:23 <gmaxwell> For a potential fork generating change we really want all the mining to be on it or none on it.  Even if its wrong, it's more important to be consistent in bitcoin than correct. :(
 29 2012-10-22 00:06:25 <Luke-Jr> before linking from bitcoin.org, it'd be nice to get more people who can reach the 3 signers policy
 30 2012-10-22 00:06:54 <gmaxwell> And more releases will only further dilute our profoundly inadequate testing concentration.
 31 2012-10-22 00:06:57 <gmaxwell> :(
 32 2012-10-22 00:07:05 <Diablo-D3> theymos: why havent you made me a global admin by now?
 33 2012-10-22 00:07:33 <Hasimir> gmaxwell, what's needed for testing?
 34 2012-10-22 00:07:52 <Luke-Jr> gmaxwell: I've been delaying stable releases to avoid taking testing away from master ones
 35 2012-10-22 00:08:08 <Luke-Jr> Hasimir: more testing for RCs is always helpful
 36 2012-10-22 00:08:31 <gmaxwell> Hasimir: People to actually run pre-release / rc versions and report issues, and maybe beat on them a little.  Testing with testnet (so you can move lots of coin without risking real funds) would also be very good.
 37 2012-10-22 00:08:40 <Hasimir> well if it helps find the solution to the qt client crashing on my mac I'm in
 38 2012-10-22 00:09:19 <gmaxwell> Hasimir: Were you on OSX 10.5 ?
 39 2012-10-22 00:09:25 <Luke-Jr> the situation for Macs is bad all around :P
 40 2012-10-22 00:09:43 <Hasimir> Luke-Jr, yes, we've had that chat  ;)
 41 2012-10-22 00:10:07 <Hasimir> I'd rather not test on the server either
 42 2012-10-22 00:10:24 <Hasimir> but I may be able to cobble some little linux box together
 43 2012-10-22 00:10:47 <Luke-Jr> Hasimir: if you don't want to test bleeding edge master releases on your server, perhaps consider testing the backport for whatever version it's running?
 44 2012-10-22 00:11:14 <Hasimir> yeah, might do
 45 2012-10-22 00:11:39 <Hasimir> where are the rc versions of the code?
 46 2012-10-22 00:11:45 <Luke-Jr> git
 47 2012-10-22 00:11:50 <Hasimir> of course
 48 2012-10-22 00:12:05 <Luke-Jr> http://gitorious.org/bitcoin/bitcoind-stable for the backports
 49 2012-10-22 00:12:14 <Hasimir> cool
 50 2012-10-22 00:12:40 <Hasimir> the server is running CuntOS 5, so it needs an overhaul too
 51 2012-10-22 00:12:57 <Hasimir> definitely going back to slackware after that experiment
 52 2012-10-22 00:13:09 <Luke-Jr> lol
 53 2012-10-22 00:14:18 <Hasimir> Luke-Jr, yeah, that typo is intentional
 54 2012-10-22 00:15:43 <Hasimir> the other little btc project I've taken on is an attempt to port gribble to the v1 API and get real support for multiple currencies
 55 2012-10-22 00:16:28 <Hasimir> that should make a lot of ppl in -otc happy
 56 2012-10-22 00:16:45 <Hasimir> including me, of course
 57 2012-10-22 00:21:11 <gmaxwell> ::sigh:: listunspent  doesn't show coinbase outputs. How the hell did I miss this before.
 58 2012-10-22 00:24:27 <gmaxwell> testnet3 should have a spend at 100 at the block after next.
 59 2012-10-22 00:25:27 <D34TH> atomic batteries to power
 60 2012-10-22 00:25:30 <D34TH> miners to speed
 61 2012-10-22 00:26:37 <gmaxwell> well, it'll probably be an hour, since I'm not going to bother throwing real hashpower at it when I can just wait. :P
 62 2012-10-22 00:32:04 <BlueMatt> yay! working reverse headers sync at the application layer in bitcoinj...lots of orphans, but its all bounded so..wooo
 63 2012-10-22 00:32:17 <BlueMatt> (downloads blocks from all peers +/- slow ones)
 64 2012-10-22 00:32:48 <gmaxwell> BlueMatt: I assume you'll need a new protocol message in order to pipeline the header pulls?
 65 2012-10-22 00:33:15 <BlueMatt> well, the initial sync is headers-only and just uses existing block download code
 66 2012-10-22 00:34:47 <BlueMatt> its more of a two-step download than reverse headers
 67 2012-10-22 00:35:01 <BlueMatt> same general effect though
 68 2012-10-22 00:37:32 <gmaxwell> anyone have a testnet ultraprune running right now?
 69 2012-10-22 00:40:16 <BlueMatt> gmaxwell: I never really understood the reasoning of why headers need to be downloaded in reverse order?
 70 2012-10-22 00:40:35 <BlueMatt> (blocks, sure, populate wallet with txes earlier, etc) but headers too?
 71 2012-10-22 00:41:49 <gmaxwell> BlueMatt: Because it prevents a dos attack where someone gives you an infinite stream of difficulty 1 headers that can't possibly be the best set.
 72 2012-10-22 00:42:06 <BlueMatt> ok, thats what I was thinking...
 73 2012-10-22 00:42:18 <BlueMatt> still, could be avoided with more sanity first...
 74 2012-10-22 00:42:48 <BlueMatt> (ask >1 peer for initial block list and find common ones...)
 75 2012-10-22 00:43:13 <gmaxwell> I'm not sure about that. I mean, there is probably some way of load balancing across multiple header sources... I'd have to think about that. Reverse made it more obviously correct in my mind.
 76 2012-10-22 00:43:14 <BlueMatt> anywhoo...that part is easy, its the multi-peer download part thats fun :)
 77 2012-10-22 00:43:27 <gmaxwell> But yea, that part isn't a super big deal.
 78 2012-10-22 00:47:46 <gmaxwell> freeking sendraw transaction won't take a transaction at 100. this is probably why the network doesn't have one??? the test for the block and mempool are not the same.
 79 2012-10-22 00:48:08 <BlueMatt> ahh, yea, the 120 rule
 80 2012-10-22 00:48:16 <BlueMatt> (it is 120 like wallet, right?)
 81 2012-10-22 00:49:48 <gmaxwell> nah, 120 is the wallet. The mempool is using the current best.
 82 2012-10-22 00:50:01 <gmaxwell> It should be using +1 since the mempool is testing for the next block.
 83 2012-10-22 00:50:19 <gmaxwell> I'm pretty sure I hit this on namecoin and never understood why. :(
 84 2012-10-22 00:50:45 <BlueMatt> ahh, ok
 85 2012-10-22 00:54:27 <gmaxwell> Okay 86eefc0c47a0402d353faf82f5b3dac3995319403aba699ae58e9ed29e412ef2 in
 86 2012-10-22 00:54:30 <gmaxwell> from block 33496, so ultraprune should be dead on testnet now.
 87 2012-10-22 00:54:41 <BlueMatt> nice
 88 2012-10-22 00:55:40 <gmaxwell> sipa: when you fix the height 100 bug, you should also make it allow 99 for the mempool, I think. (I just stuck a +!fBlock into the test but I'm tired so there might be more to worry about)
 89 2012-10-22 00:59:36 <gmaxwell> (this feeds nicely into my belief that if you don't find bugs in the reference implementation you're not testing your new implementation. :P )
 90 2012-10-22 01:01:22 <BlueMatt> gmaxwell: I still disagree with that - finding bugs in the reference implementation (in highly critical code) is still incredibly rare, despite all the smart people writing their own implementations
 91 2012-10-22 01:05:38 <gmaxwell> well, not just highly critical code. Working with this ultraprune bug made me find a non-critical mempool bug due to reusing the same code for block and mempool validation. The locktime checks are probably wrong for the mempool too, though I haven't looked yet.
 92 2012-10-22 01:08:11 <BlueMatt> gmaxwell: I dont see how this situation applies. you found a bug in a fork which showed you more bugs in the original, not an alt client...
 93 2012-10-22 01:09:36 <gmaxwell> Ultraprune is partially an altclient from the exposure perspective... and had a bug like an altclient, which exposed the same bug in the reference (but only for mempool). ::shrugs:: Well, I'm not trying to convince you. :P
 94 2012-10-22 01:11:35 <BlueMatt> fair enough
 95 2012-10-22 01:11:36 <gmaxwell> (in particular, I found the bug by reading the code after my first attempt ended up at offset 101 instead of 100)
 96 2012-10-22 01:12:06 <gmaxwell> but I could have found it just the same by working on ultraprune and thinking about how blocks and mempool differ, though I didn't.
 97 2012-10-22 01:12:29 <BlueMatt> I find this to be a different case because the bug was found because code was copied from one place to another, which I would think would not effect an alt implementation
 98 2012-10-22 01:12:41 <BlueMatt> (or reused)
 99 2012-10-22 01:50:53 <gmaxwell> BlueMatt: you don't use the same code for transaction validity for mempool and blocks?
100 2012-10-22 01:51:29 <BlueMatt> bitcoinj doesnt mine, so there is no (traditional) mempool
101 2012-10-22 01:55:12 <gmaxwell> BlueMatt: but you do accept unconfirmed transactions right?
102 2012-10-22 01:59:27 <Trudel> Hello, I am writing a handout to give out to help promote bitcoins in my local area.  I think we should have such things posted somewhere ready for bitcoin community members to take and use easily and I might as well start here.  Although fixing the wikipedia article should also be a high priority of course.  the handout is here and I am looking for some people to help make sure it is factually error free, that  imo is the most
103 2012-10-22 02:00:03 <Trudel> It is directed at both retailers and users as of course both need to be worked on.
104 2012-10-22 02:00:34 <Trudel> http://piratepad.net/x1h9j4w2Ed my email  for this project is r5t6yuiovbnmj@hmamail.com
105 2012-10-22 02:04:49 <BlueMatt> gmaxwell: yes, but only to wallets and they arent verified (much)
106 2012-10-22 02:05:18 <gmaxwell> Hm. Interesing.
107 2012-10-22 02:05:35 <gmaxwell> It's bad if that could result in transactions being displayed that can't ever be confirmed.
108 2012-10-22 02:05:44 <BlueMatt> (In other words, I havent touched that code)
109 2012-10-22 02:05:54 <gmaxwell> Yea. :P
110 2012-10-22 02:06:14 <BlueMatt> but, yea, thats another thing that has to happen eventually
111 2012-10-22 02:07:22 <BlueMatt> ugg, it took waay less time to write a multi-node block syncer than to write a (still not fully working) dnsseed... :(
112 2012-10-22 02:07:58 <BlueMatt> ^why I hate java
113 2012-10-22 02:27:06 <gmaxwell> ::sigh:: "I'm vehemently opposed to this, as it is economically inefficient and opaque."  Meni on payment protocols where the reciever specifies the transaction fees and includes them in the advertised prices.
114 2012-10-22 02:41:50 <jgarzik> gmaxwell: ignoring the real world, it sounds like
115 2012-10-22 02:47:46 <gmaxwell> he goes on, but I was concerned about forehead bleeding and stopped reading the thread.
116 2012-10-22 03:07:52 <jgarzik> 
ACTION wonders if ultraprune has a higher orphan rate, for some odd reason

117 2012-10-22 03:09:27 <gmaxwell> jgarzik: er. are you mining ultraprune on testnet?
118 2012-10-22 03:09:32 <gmaxwell> lol lol
119 2012-10-22 03:10:31 <gmaxwell> If so I have forked you with with an earlier discovered ultraprune flaw??? I stuck the trigger case in testnet.
120 2012-10-22 03:11:19 <gmaxwell> (also found a minor bug in the preultraprune code along the way)
121 2012-10-22 03:12:23 <gmaxwell> "blocks" : 33600,
122 2012-10-22 03:38:35 <NaruFGT> ;;log
123 2012-10-22 03:38:36 <gribble> Error: "log" is not a valid command.
124 2012-10-22 03:38:39 <NaruFGT> oops
125 2012-10-22 03:38:49 <NaruFGT> wrong irc channel, sorry guys
126 2012-10-22 03:56:28 <echelon> bitcoind is using 50-75% cpu
127 2012-10-22 03:56:30 <echelon> :/
128 2012-10-22 04:37:37 <orion> I'm running v0.7.1-64-g2ef1569-dirty-beta. I would like to reduce the size of my ~/.bitcoin directory now that I am using leveldb. What is the recommended way of doing this?
129 2012-10-22 04:38:07 <gmaxwell> orion: What is the current usage?
130 2012-10-22 04:38:44 <orion> gmaxwell: 7.6G
131 2012-10-22 04:39:31 <gmaxwell> Delete the old blk0001.dat blk0002.dat and blkindex.dat. They aren't used anymore.
132 2012-10-22 04:39:52 <Diablo-D3> gmaxwell: when will 0.7.0 be shipped?
133 2012-10-22 04:40:14 <gmaxwell> Diablo-D3: um. in -1.5 months or so?
134 2012-10-22 04:40:19 <Diablo-D3> k
135 2012-10-22 04:40:30 <gmaxwell> (you note the negative, right?)
136 2012-10-22 04:40:35 <Diablo-D3> erp?
137 2012-10-22 04:40:42 <gmaxwell> 0.7 has been out for a long time.
138 2012-10-22 04:40:47 <Diablo-D3> wtf
139 2012-10-22 04:40:47 <sipa> 0.7.1 is already out
140 2012-10-22 04:40:52 <Diablo-D3> no one ever tells me this shit
141 2012-10-22 04:40:56 <gmaxwell> sipa: good morning.
142 2012-10-22 04:41:06 <sipa> we're now pulling for 0.8.0 :)
143 2012-10-22 04:41:25 <sipa> Diablo-D3: you may want to read the forums :p
144 2012-10-22 04:41:39 <sipa> gmaxwell: thanks *yaaaaawn*
145 2012-10-22 04:41:54 <orion> gmaxwell: Thank you. I am now down to 3.2G.
146 2012-10-22 04:42:02 <jgarzik> gmaxwell: mining on testnet yes, mining with ultraprune, no
147 2012-10-22 04:42:05 <gmaxwell> sipa: dunno if you saw in the backscroll but you should be able to test ultraprune against testnet now.
148 2012-10-22 04:42:41 <orion> Also, I opened a pull request, 1945. :)
149 2012-10-22 04:43:53 <gmaxwell> orion: hm. There is BE freebsd, e.g. ppc.
150 2012-10-22 04:43:57 <gmaxwell> (not that we run on it)
151 2012-10-22 04:44:45 <Diablo-D3> gmaxwell: so what were the major changes in 0.7.0?
152 2012-10-22 04:45:05 <jgarzik> Diablo-D3: switched to tonal
153 2012-10-22 04:45:52 <gmaxwell> Plus the CIA backdoor key.
154 2012-10-22 04:45:55 <gmaxwell> Diablo-D3: https://bitcointalk.org/index.php?topic=110243.0
155 2012-10-22 04:46:07 <Diablo-D3> heee those jokes never get old
156 2012-10-22 04:47:07 <sipa> 0.8.0 will be even more fun, we'll switch to scrypt based hashes with 4 KiB of memory
157 2012-10-22 04:47:16 <Diablo-D3> 
ACTION snorts

158 2012-10-22 04:48:00 <sipa> gmaxwell: hmm, why should we allow 99 conf mempool transactions?
159 2012-10-22 04:48:33 <gmaxwell> sipa: no way to ever mine at 100 if we don't accept at 99.
160 2012-10-22 04:49:02 <sipa> imho, we should only accept coinbase spends with 120 conf...
161 2012-10-22 04:49:07 <sipa> hmm
162 2012-10-22 04:49:14 <gmaxwell> Makes the 100 limit effectively 101, which is almost certantly why we have none.
163 2012-10-22 04:49:32 <Diablo-D3> whats the actual problem?
164 2012-10-22 04:50:56 <gmaxwell> sipa: the same behavior with locktime (I didn't actually check) would also create a weird situation where you could outrun the earliest spend by having a deal with a single modified miner. For the coinbase spends it's not really a big deal... but I don't see any reason to constain spending beyond the protocol rule.
165 2012-10-22 04:51:18 <gmaxwell> The wallet use of 120 is helpful because it makes sure things will propagate even if there is some chain fraying.
166 2012-10-22 04:53:00 <gmaxwell> Diablo-D3: no real problem, while creating a testcase for an ultraprune bug I discovered that stock bitcoin won't mine a minimum age coinbase spend, because to mine it at the minimum point you have to accept it while its still immature.
167 2012-10-22 04:54:01 <sipa> gmaxwell: ACK
168 2012-10-22 04:59:46 <Luke-Jr> gmaxwell: it won't mine a lot of things ;)
169 2012-10-22 05:00:19 <Diablo-D3> gmaxwell: huh.
170 2012-10-22 05:01:54 <gmaxwell> Luke-Jr: its true, but that one seems less intentional. :P
171 2012-10-22 05:02:09 <orion> Luke-Jr: How old are you?
172 2012-10-22 05:02:26 <Luke-Jr> orion: 27, why? O.o
173 2012-10-22 05:02:35 <orion> You look young in your github photo.
174 2012-10-22 05:02:43 <Luke-Jr> it's an old photo
175 2012-10-22 05:02:47 <orion> ahh ok
176 2012-10-22 05:03:08 <Luke-Jr> though I think only 3 or 4 years old
177 2012-10-22 05:03:26 <orion> I was about to say... I am 21 and I thought I was young.
178 2012-10-22 05:07:15 <gmaxwell> Luke-Jr: whaha?!?! "Also note that despite many headers/software making the assumption that endian is fixed at build time, that assumption is apparently not necessarily true in theory"
179 2012-10-22 05:07:33 <gmaxwell> Luke-Jr: do you mean that the _build host_ and the _target_ may not have the same bytesex?
180 2012-10-22 05:07:53 <gmaxwell> Surely you're not saying that it may be runtime different. 0_o
181 2012-10-22 05:07:55 <Luke-Jr> gmaxwell: no, apparently it's possible for the target to have an unknown endianness
182 2012-10-22 05:08:04 <Luke-Jr> at build time
183 2012-10-22 05:08:18 <Luke-Jr> trivia, probably not worth supporting, tho
184 2012-10-22 05:08:51 <gmaxwell> okay, well thats not as bad as runtime endianness.. It's always possible to write neutral code, but it's often not as fast.
185 2012-10-22 05:08:58 <Luke-Jr> right
186 2012-10-22 05:09:57 <orion> sipa: port_posix.h, not env_posix.h
187 2012-10-22 05:10:33 <sipa> orion: thx, fixed
188 2012-10-22 05:10:43 <sipa> (sorry, just woke up)
189 2012-10-22 05:10:51 <orion> NP
190 2012-10-22 05:11:54 <sipa> yeah, i feel particularly non-deterministically polynomial today
191 2012-10-22 05:13:31 <gmaxwell> https://bitcointalk.org/index.php?topic=114585.msg1289314#msg1289314 < nice mining as a lottery success story
192 2012-10-22 05:14:35 <gmaxwell> solomined for 10 months with 400MH/s and found two blocks, used them to preorder a 54GH/s mining device.
193 2012-10-22 05:16:01 <sipa> gmaxwell: mempool accepts non-final transactions anyway, and createnewblock correctly uses the new block's height to determine finalness
194 2012-10-22 05:16:07 <sipa> gmaxwell: so no issue for nLockTime
195 2012-10-22 05:16:22 <gmaxwell> sipa: thank you for being less lazy than me.
196 2012-10-22 05:17:05 <gmaxwell> oh yea, I'd forgotten that we accept non-final transactions, thats one strike aganst using nlocktime when replacement doesn't exist. :(
197 2012-10-22 05:23:09 <sipa> gmaxwell: bugfix_maturity branch pushed
198 2012-10-22 05:25:36 <sipa> well, what's the problem with enabling replacement now?
199 2012-10-22 05:26:05 <sipa> (not a decision we should make lightly, but we should at least have an idea why not to do that)
200 2012-10-22 05:35:06 <_dr> will i find all relevant leveldb changes by looking for 'leveldb' in the github commits?
201 2012-10-22 05:36:15 <sipa> _dr: i think so yes
202 2012-10-22 05:36:35 <_dr> okay, thanks
203 2012-10-22 05:58:02 <sipa> TD: yeah, i should have seen how much more of your leveldb branch was reusable
204 2012-10-22 05:58:26 <TD> the import was a nasty hack. there's a pullreq somewhere from someone that does the rescanning nicer
205 2012-10-22 05:58:46 <TD> i didn't want to change all the code that keeps track of current file position in the leveldb pull to keep it as small as possible
206 2012-10-22 06:00:22 <sipa> well i adapted jgarzik's -reindex for ultraprune now, which does an in-place upgrade
207 2012-10-22 06:01:30 <sipa> in parallel with a running node
208 2012-10-22 06:01:38 <abrkn> i'm working on a card game and i'd like to send the user proof that i'm not cheating him by hashing the result with my pk and sending it upfront. how would i go about this? (node.js and i know nothing of crypto)
209 2012-10-22 06:02:22 <sipa> abrkn: you don't hash with a key - if you do, it is a MAC
210 2012-10-22 06:02:53 <abrkn> ok, my lack of language for this accurately describes how little i know of crypto
211 2012-10-22 08:51:56 <Tykling> do any other clients implement the ability to sign messages ?
212 2012-10-22 08:53:38 <zveda> armoury does I think
213 2012-10-22 08:53:50 <zveda> *armory
214 2012-10-22 08:54:25 <zveda> yeh it does: http://bitcoinarmory.com/index.php/start-page/what-is-armory/features
215 2012-10-22 09:38:53 <orion> I am getting 100% CPU usage. :(
216 2012-10-22 09:39:02 <orion> No blocks are being downloaded and verified.
217 2012-10-22 09:39:21 <orion> The best I can do to track it down is this:
218 2012-10-22 09:39:22 <orion> #2  0x00000000005ce57b in ThreadSocketHandler2 (parg=0x0) at src/net.cpp:779
219 2012-10-22 09:39:30 <orion> Any ideas?
220 2012-10-22 09:42:29 <sipa> git head?
221 2012-10-22 09:55:34 <upb> lol @ Handler2
222 2012-10-22 09:55:45 <upb> is that because of the homegrown stack-aslr ?:P
223 2012-10-22 09:57:39 <sipa> orion: which platform and version, and did have earlier version have the same problem?
224 2012-10-22 10:05:17 <orion> sipa: 2ef15697f84208e05764046dd86bcf067029a9b8
225 2012-10-22 10:05:41 <orion> And yes, I believe 0.7.0 had the same problem.
226 2012-10-22 10:05:52 <sipa> BSD?
227 2012-10-22 10:05:54 <orion> FreeBSD 9-STABLE
228 2012-10-22 10:06:19 <sipa> i suppose it's a BSD-specific issue then
229 2012-10-22 10:06:44 <_dr> you could always run a linux binary
230 2012-10-22 10:06:48 <sipa> we're not actively supporting BSD, but patches are welcome
231 2012-10-22 10:07:55 <orion> I'll open another pull request. I am not familiar with the codebase or anything, so I don't even know where to look.
232 2012-10-22 10:08:23 <orion> What I've done so far is to identify the thread causing the issue and switch to it in gdb.
233 2012-10-22 10:08:31 <orion> From there I get unhelpful backtraces.
234 2012-10-22 10:08:50 <orion> Or at least, it appears unhelpful to me.
235 2012-10-22 10:08:56 <sipa> i assume it's related to the select() call
236 2012-10-22 10:09:20 <orion> I assume that too, since that's the thread using all the CPU.
237 2012-10-22 10:09:20 <sipa> which is what that thread is built around
238 2012-10-22 10:16:56 <orion> hmm
239 2012-10-22 10:21:58 <orion> sipa: I fixed it. A PR is forthcoming.
240 2012-10-22 10:41:44 <orion> sipa: https://github.com/bitcoin/bitcoin/pull/1947
241 2012-10-22 10:55:56 <sipa> orion: oh, again the boost ipc failure?
242 2012-10-22 10:56:07 <sipa> so not the socket handler?
243 2012-10-22 10:56:33 <orion> sipa: Correct. I identified the wrong thread when I was talking to you earlier.
244 2012-10-22 11:00:18 <sipa> orion: strange that no progress was made, in that case
245 2012-10-22 11:32:47 <gmaxwell> sipa: your branch fixes testnet.
246 2012-10-22 11:33:29 <gmaxwell> Also, reindex was useful??? I first reproduced the failure, then restarted with the maturity fix??? node was stuck because got inventory: block 00000000dc3a9cee3f1b  have and didn't refetch it.
247 2012-10-22 11:33:57 <gmaxwell> Reindex made it accept it and it continued on.
248 2012-10-22 11:36:12 <sipa> maybe there can be we -softreindex, which only resets the block database and rescans that, but doesn't wipe the coins db
249 2012-10-22 11:36:32 <sipa> that should allow you to recover from such things much more quickly
250 2012-10-22 11:40:41 <sipa> then again, such things shouldn't happen at all
251 2012-10-22 11:41:22 <Diablo-D3> what exactly is happening?
252 2012-10-22 11:41:36 <gmaxwell> Right, I mean we also could attempt to reattach a detached block when we hear about it from a peer after restart or something.. then it would recover automatically.. but that code would be risker than the problem it solves.
253 2012-10-22 11:42:13 <Diablo-D3> btw, that ultraprune shit? that can be turned off, right?
254 2012-10-22 11:43:12 <sipa> no
255 2012-10-22 11:43:33 <sipa> but it doesn't mean your block chain gets pruned
256 2012-10-22 11:44:01 <sipa> the name refers to the fact that it uses an ultra-pruned copy of the chain for validation, instead of an index into the full one
257 2012-10-22 11:44:52 <Diablo-D3> ahh
258 2012-10-22 11:44:54 <gmaxwell> I propose we rename ultraprune  'weeblix' ... because no one will make assumptions about what weeblix does. :P
259 2012-10-22 11:45:07 <Diablo-D3> because I want to keep the chain history
260 2012-10-22 11:45:26 <gmaxwell> Diablo-D3: good, we want you to do so too.
261 2012-10-22 11:45:40 <sipa> how about "turbo engine" ? :)
262 2012-10-22 11:45:44 <Diablo-D3> well, everyones like they want new clients to start instantly
263 2012-10-22 11:45:50 <Diablo-D3> and not need to store 2gb of data
264 2012-10-22 11:45:59 <Diablo-D3> I dont see how that can be secure =/
265 2012-10-22 11:46:19 <sipa> those are two separate issues
266 2012-10-22 11:46:35 <sipa> a zero-trust node will always need to see the entire history to synchronize
267 2012-10-22 11:46:42 <sipa> but that doesn't mean it needs to keep it
268 2012-10-22 11:47:22 <Diablo-D3> yeah, but if no one has the history anymore, how can you add new nodes?
269 2012-10-22 11:47:54 <sipa> that's another question
270 2012-10-22 11:48:27 <sipa> but surely people on a network connection that only supports very slow uploading, better don't serve the entire history (better for them, and better for the rest as well, as it slows them down)
271 2012-10-22 11:48:48 <Diablo-D3> so we end up doing the whole supernode shit
272 2012-10-22 11:48:51 <Diablo-D3> and that shit always goes wrong
273 2012-10-22 11:49:37 <sipa> i suppsoe i'll start referring to ultraprune as "the 0.8 engine" or something
274 2012-10-22 11:50:47 <gmaxwell> v8 engine, obviously, if that hasn't been taken
275 2012-10-22 11:51:20 <sipa> gmaxwell: bummer!
276 2012-10-22 11:51:29 <gmaxwell> Diablo-D3: in any case thats irrelevant now; we're not doing any of that, we're a long way from being able to do that.
277 2012-10-22 11:52:10 <helo> so the goal is to make it comfortable for everyone to run a bitcoin node to the extent that their resources allow it
278 2012-10-22 11:52:27 <sipa> helo: that would be ideal, i guess
279 2012-10-22 11:52:38 <helo> if nobody ever noticed any negative performance impact from running a node, presumably everyone would
280 2012-10-22 11:52:50 <gmaxwell> helo: thats _my_ vision at least.
281 2012-10-22 11:52:51 <helo> everyone being 'all bitcoin users'
282 2012-10-22 11:53:12 <sipa> well, bandwidth and storage aren't free
283 2012-10-22 11:53:28 <sipa> as long as you're relaying on consumer hardware (and for now, we can), that is typically not a problem
284 2012-10-22 11:54:02 <sipa> but as soon as running a (full) node becomes something that requires investment, i'm afraid things can change
285 2012-10-22 11:54:24 <t7> is the difference between signed and unsigned add just the condition for the carry add being set?
286 2012-10-22 11:54:33 <t7> oh wrong channel, excuse me
287 2012-10-22 11:55:37 <Diablo-D3> t7: there is so much wrong with your statement
288 2012-10-22 11:55:56 <t7> add = flag
289 2012-10-22 11:56:14 <t7> oh and minus :)
290 2012-10-22 11:56:28 <t7> 
ACTION shoots self in head :)

291 2012-10-22 11:56:31 <Diablo-D3> wikipedia twos complement
292 2012-10-22 11:56:40 <sipa> is there an implementation difference between unsigned and signed add?
293 2012-10-22 11:56:51 <sipa> i don't think so
294 2012-10-22 11:56:52 <Diablo-D3> sipa: I dont think so.
295 2012-10-22 11:57:11 <Diablo-D3> not with twos complement math and the way processors implement math
296 2012-10-22 11:58:10 <helo> right now say 90% of nodes are full nodes... so the chance that none of eight connections will be full nodes is 1 in 100,000,000. if things are scaled down so that a lower percentage of the network are full nodes, at which point do we need raise the number of default connections?
297 2012-10-22 11:58:41 <sipa> helo: nodes only connect to full nodes
298 2012-10-22 11:58:57 <sipa> as SPV nodes don't relay, they are not interesting to connect to
299 2012-10-22 11:59:27 <helo> oh... only two tiers?
300 2012-10-22 11:59:31 <sipa> for now, yes
301 2012-10-22 12:00:07 <helo> i was thinking some people will have resources to maybe not serve the full chain, but still be useful for something
302 2012-10-22 12:00:57 <sipa> well, i think a first step is separating full nodes into archive nodes and validation nodes (i assume every archive node will also validate, but that's no techncial requirement)
303 2012-10-22 12:01:09 <sipa> you'd only need archive nodes for IBD
304 2012-10-22 12:43:14 <gmaxwell> helo: right now I expect that something like <20% of users are using full nodes.
305 2012-10-22 12:43:29 <gmaxwell> helo: worse, they've mostly moved to insecure webwallets.
306 2012-10-22 12:44:31 <Diablo-D3> why did my brain try to parse that as wallabes
307 2012-10-22 12:44:42 <helo> because they are delicious?
308 2012-10-22 12:44:54 <Diablo-D3> nothx.
309 2012-10-22 12:45:15 <sipa> well, i don't count those as nodes whatsoever
310 2012-10-22 12:45:32 <gmaxwell> BlueMatt: Some food for thought while you're working on header fetching.  While talking to ThomasV about how electrum is becoming more SPVish and how he'll handler historical headers I thought of a new kind of checkpoint that nodes could use to improve security against isolation.
311 2012-10-22 12:45:47 <Diablo-D3> so wait
312 2012-10-22 12:45:51 <Diablo-D3> people actually use electrum?
313 2012-10-22 12:46:41 <gmaxwell> BlueMatt: the thought would be that you'd checkpoint only the total work and not accept as 'real' any chain which is under that threshold. Because it only sets a minimum it doesn't goof with the consensus algorithm (except letting someone who sets it block the user from working at all)
314 2012-10-22 12:47:13 <gmaxwell> BlueMatt: so someone could be _very_ agressive about updating it. E.g. updating it at the moment a client is downloaded and setting it to their own network tip.
315 2012-10-22 12:48:07 <gmaxwell> (or accepting signed copies of it from the developers over a network connection??? would only give the developers a DOS attack)
316 2012-10-22 12:48:45 <gmaxwell> (but in exchange would give immunity to isolation)
317 2012-10-22 12:51:29 <UukGoblin> gmaxwell, ohai, did you get my messages about the timestamp?
318 2012-10-22 12:51:43 <gmaxwell> UukGoblin: hm. No.
319 2012-10-22 12:51:45 <gmaxwell> 
ACTION looks

320 2012-10-22 13:04:48 <molecular> testing ultraprune (git from 20 hours ago): ./bitcoin-qt -loadblock=/home/nick/.bitcoin/blk0001.dat -loadblock=/home/nick/.bitcoin/blk0002.dat
321 2012-10-22 13:05:25 <molecular> then when I try to start bitcoin-qt again, a popup pops up: "Warning: error reading wallet.dat! Alle keys read correctly, but transaction data or address book entries mieght be missing or incorrect"
322 2012-10-22 13:05:45 <molecular> and ten another one: "failed to connect best block"
323 2012-10-22 13:05:57 <Turingi> so computational proof of work is the only way to build a distributed crypto money system?
324 2012-10-22 13:06:42 <Turingi> or can we call it, protocol-based crypto money system
325 2012-10-22 13:06:54 <UukGoblin> Turingi, I guess not, e.g. a proof-of-stake is another option
326 2012-10-22 13:06:58 <Joric> lol http://www.reddit.com/r/Bitcoin/comments/11vtfk/mining_addresses_is_currently_more_profitable/
327 2012-10-22 13:07:19 <UukGoblin> there's probably lots of other ways
328 2012-10-22 13:09:18 <Joric> looks like mining vanity addresses 10 times more profitable
329 2012-10-22 13:10:38 <Turingi> UukGoblin: thanks, looking at that now https://en.bitcoin.it/wiki/Proof_of_Stake
330 2012-10-22 13:10:42 <sipa> Joric: that is sad...
331 2012-10-22 13:11:12 <Turingi> what's wrong with vanity addresses?
332 2012-10-22 13:11:17 <Joric> can't be done with asics though
333 2012-10-22 13:12:00 <UukGoblin> people are paying THAT MUCH for them? :-O
334 2012-10-22 13:13:10 <gmaxwell> Turingi: They weakly violate the bitcoin design??? bitcoin is highly public, the only source of privacy people have comes from minimizing address reuse.
335 2012-10-22 13:13:28 <Joric> might be the next big thing after block mining pools )
336 2012-10-22 13:13:42 <gmaxwell> Turingi: a cute address here or there is harmless but if you're willing to pay a lot for a vanity address you're going to be sure to use it a lot.
337 2012-10-22 13:13:46 <Turingi> there may be a need for a truly anonymous money transfer system, one that doesn't keep transactions visible to everyone
338 2012-10-22 13:14:42 <Joric> eg 1stJune* costs 0.64 BTC
339 2012-10-22 13:14:46 <gmaxwell> Turingi: if it were as simple as saying there was a need we'd have it. Bitcoin is not too far if used correctly (_every_ transaction to a new address).
340 2012-10-22 13:14:51 <UukGoblin> Turingi, to quote Dan Boneh: "Everything that can be done with a central server, can also be done without it"
341 2012-10-22 13:15:20 <Turingi> well, when you try to spend BTC, all those unique addresses point back to you
342 2012-10-22 13:15:36 <Turingi> so does it matter that you create a new one for each transaction?
343 2012-10-22 13:16:10 <Turingi> what about stuff like mixnets?
344 2012-10-22 13:16:15 <gmaxwell> Turingi: yes, because it minimizes the amount of connected informations, and minimizing crosslink. _only_ the person who paid that address can observe the connection.
345 2012-10-22 13:16:50 <orion> I think people who don't understand bitcoin use it horribly. For example, why on Earth would you EVER get a wallet from some random website instead of running the client on your own machine?
346 2012-10-22 13:17:30 <Turingi> some people need to move lots of BTC, you have to use some trading platform for that (like mtgox)
347 2012-10-22 13:17:32 <UukGoblin> orion, a LOT of people these days choose to use random websites rather than install free clients on their own machines
348 2012-10-22 13:17:40 <gmaxwell> orion: because people reason poorly about risks. We use 'social' proxies to determine safty.  FooWebWallet is "intutively" safe because the website is very polished.
349 2012-10-22 13:17:55 <gmaxwell> orion: and they haven't heard of anyone getting ripped off that way!
350 2012-10-22 13:17:55 <UukGoblin> orion, facebook is a great example
351 2012-10-22 13:18:15 <gmaxwell> Turingi: for the bitcoin inflation-free promise to be upheld the motion of coins must be disclosed so you can tell that the amount isn't being inflated.
352 2012-10-22 13:18:42 <Turingi> gmaxwell: I thought inflation is a tunable parameter
353 2012-10-22 13:18:50 <Joric> gmaxwell, true, coinbase looks so 'reliable'
354 2012-10-22 13:19:26 <Turingi> gmaxwell: you can tune the reward to be slightly inreasing over time with a constant factor (a Milton Friedman-style central bank)
355 2012-10-22 13:19:35 <Joric> not to mention instawallet it was rated 3rd in that bitcoin analysis pdf
356 2012-10-22 13:19:45 <gmaxwell> Turingi: in _bitcoin_, no. In the methods we use, sure??? but if you can't see the transactions, you can't tell what the inflation level is, at least in our kind of distributed system.
357 2012-10-22 13:20:37 <gmaxwell> Turingi: what you're describing is not bitcoin it's something else. Every bitcoin node enforces the scheduled introduction of codes. It's fixed and can't be changed without replacing everyone's software with something else.
358 2012-10-22 13:20:42 <helo> isn't a deflationary bitcoin more valuable than an inflationary bitcoin? why would we want to make bitcoin worth less?
359 2012-10-22 13:21:02 <Turingi> actually, the weak pseudonimity can be exploited by tax authorities, it could make btc legal transactions
360 2012-10-22 13:21:12 <Turingi> since auditing is a matter of data mining
361 2012-10-22 13:21:21 <gmaxwell> But thats an aside and not my point, my point was is that showing the behavior is correct requires the data be available. :(
362 2012-10-22 13:22:26 <echelon> bitcoind cpu usage: 57% :/
363 2012-10-22 13:22:26 <gmaxwell> Turingi: um. Weird idea what all transactions are not 'legal transactions' it's not like coal is illegal because you can trade it without contacting a centeral registration.
364 2012-10-22 13:22:26 <helo> bitcoin used on large scales is taxable just as cash on large scales is
365 2012-10-22 13:22:50 <gmaxwell> echelon: is the node otherwise working correctly? are you current with the chain? What verison? What OS?
366 2012-10-22 13:23:05 <Turingi> just monitor outflows from known addresses associated with people
367 2012-10-22 13:23:23 <Luke-Jr> Turingi: ok, good. tax evasion is bad, don't do it
368 2012-10-22 13:23:44 <gmaxwell> Turingi: that doesn't work so well with current bitcoin usage, since almost every transaction moves funds to new addresses.
369 2012-10-22 13:23:48 <Turingi> hey, everyone is doing it at some level Luke-Jr :)
370 2012-10-22 13:24:06 <Turingi> gmaxwell: the IRS just needs to work backwards from new addresses
371 2012-10-22 13:24:12 <Turingi> from known addresses, that is
372 2012-10-22 13:24:13 <gmaxwell> Turingi: And "making" everyone use constant addresses makes bitcoin very unattractive because it would be highly non-private.
373 2012-10-22 13:24:30 <edcba> it doesn't make it unattractive
374 2012-10-22 13:24:36 <gmaxwell> Turingi: this isn't how tax collection works even on systems where it wouldn't totally ruin privac.
375 2012-10-22 13:24:37 <edcba> it makes it less attractive
376 2012-10-22 13:24:44 <Turingi> when you post a donation addresses associated with your name, any transactions from that address can be monitored
377 2012-10-22 13:24:50 <Turingi> as an example
378 2012-10-22 13:25:00 <gmaxwell> edcba: the first time your mother in law calls and asks why you're buying contraceptives when she wants grandkids you will stop using bitcoin.
379 2012-10-22 13:25:09 <edcba> haha
380 2012-10-22 13:25:23 <helo> afaict because every transaction is public, "laundering" is essentially built into the system
381 2012-10-22 13:25:37 <edcba> hopefully i rarely buy condoms by internet :)
382 2012-10-22 13:25:47 <gmaxwell> Turingi: it's not like every transaction you make at a bank is being constantly streamed to the IRS??? even though it would be completely feasable to do this and would hardly harm banking privacy.
383 2012-10-22 13:25:47 <Luke-Jr> helo: the inverse!
384 2012-10-22 13:25:53 <edcba> i wonder how IRS do with cash...
385 2012-10-22 13:25:55 <Turingi> it is at least one aspect that can make it pallatable potentially to payment processors
386 2012-10-22 13:26:03 <helo> Luke-Jr: because every transaction is private?
387 2012-10-22 13:26:11 <Joric> would it be more appropriate to provide random addresses for donation?
388 2012-10-22 13:26:26 <Turingi> yeah, but eventually you will want to spend the BTC there
389 2012-10-22 13:26:28 <gmaxwell> Turingi: er, generally payment processing _must_ use unique addresses, otherwise you don't know who paid you.
390 2012-10-22 13:26:36 <Turingi> and when you spend it, it's all traced
391 2012-10-22 13:26:39 <Luke-Jr> helo: because every transaction is public
392 2012-10-22 13:27:04 <edcba> i don't see what your problem is
393 2012-10-22 13:27:13 <gmaxwell> Joric: it's a preferred thing to do, but few do it. We haven't created good tools for that yet.
394 2012-10-22 13:27:16 <edcba> either you can justify a transaction either you can't
395 2012-10-22 13:27:28 <sipa> Turingi: but if every address is unique, "tracing" isn't more than linking addresses together
396 2012-10-22 13:27:30 <helo> Luke-Jr: ahh, right :)
397 2012-10-22 13:29:21 <echelon> gmaxwell: yeah, peers are connected to me just fine.. bitcoin ver 0.7.1, linux 2.6.37.6
398 2012-10-22 13:29:22 <helo> Turingi: also, recommended practice is that every donor will receive their own donation address, so there will not be a common address to monitor
399 2012-10-22 13:29:42 <helo> Turingi: it is trivial to never show the same address twice
400 2012-10-22 13:29:49 <echelon> i think the chain is still downloading, and the blk index is being processed
401 2012-10-22 13:30:20 <echelon> 188707523 Oct 22 11:29 blk0004.dat
402 2012-10-22 13:30:24 <echelon> 978403328 Oct 22 11:29 blkindex.dat
403 2012-10-22 13:30:27 <Turingi> helo: ok, in that case you will need to set up your donation page through some obfuscation service, i.e. never post a single address, just a web service of sorts
404 2012-10-22 13:30:28 <gmaxwell> echelon: How many blocks do you have? Hovering over the status will tell you the block count.
405 2012-10-22 13:30:37 <gmaxwell> blk0004.dat ?! you've done something .. odd.
406 2012-10-22 13:30:47 <sipa> gmaxwell: he -loadblock='ed himself, i guess
407 2012-10-22 13:30:48 <echelon> what
408 2012-10-22 13:30:57 <echelon> nope
409 2012-10-22 13:30:59 <Turingi> helo: I see now :) pseudonimity reinforced
410 2012-10-22 13:31:03 <echelon> i aborted loadblock
411 2012-10-22 13:31:34 <gmaxwell> Turingi: thats the normal behavior for payment processing??? if you don't give out new addresses you can't tell which person paid you. For donations it's only important for privacy purposes.
412 2012-10-22 13:31:37 <echelon> gmaxwell: i downloaded the archived blockchain from bitcoincharts :/
413 2012-10-22 13:31:41 <Joric> gmaxwell, it's rather trivial, if you delete blkindex.dat it concatenates a new blockchain to the old one =)
414 2012-10-22 13:32:07 <gmaxwell> Joric: or drop in a chain without an index.
415 2012-10-22 13:32:10 <Joric> ^ this
416 2012-10-22 13:33:00 <Joric> bitcoincharts should add blkindex.dat to the archive
417 2012-10-22 13:33:37 <Luke-Jr> Joric: it was just removed
418 2012-10-22 13:33:40 <Luke-Jr> for good reason
419 2012-10-22 13:34:00 <sipa> they should call their downloaded file bootstrap.dat instead of blk000*.dat
420 2012-10-22 13:34:07 <Luke-Jr> 
ACTION wonders if bitcoincharts redid their thing to be built upon the minimal blk0001.dat

421 2012-10-22 13:35:26 <gmaxwell> sipa: well, I understand that it will be going away.
422 2012-10-22 13:36:30 <echelon> gmaxwell: 197287 blocks
423 2012-10-22 13:37:58 <gmaxwell> echelon: how long has it been syncing since the last time you restarted it?
424 2012-10-22 13:38:12 <echelon> for 2 days now
425 2012-10-22 13:38:34 <echelon> was i supposed to use loadblock for bitcoinchart's tarball'd archive?
426 2012-10-22 13:38:41 <Luke-Jr> of course
427 2012-10-22 13:38:46 <sipa> molecular: is that at startup?
428 2012-10-22 13:40:22 <echelon> so is it just redownloading the entire blockchain from the beginning?
429 2012-10-22 13:40:27 <echelon> i never needed to do that before
430 2012-10-22 13:41:12 <sipa> echelon: if the archive doesn't contain an index, it will assume nothing is there, and just download stuff and append it to whatever file is available
431 2012-10-22 13:41:33 <echelon> bah
432 2012-10-22 13:41:45 <sipa> however, supplying the index is a bad idea, since it means you trusted them to do the indexing for you
433 2012-10-22 13:41:54 <sipa> if you do that, you're better off running a lightweight node
434 2012-10-22 13:42:21 <echelon> so i should just abort then?
435 2012-10-22 13:42:24 <sipa> yes
436 2012-10-22 13:43:29 <sipa> abort, put the downloaded files NOT in the datadir, and -loadblock them
437 2012-10-22 13:43:46 <echelon> gotcha
438 2012-10-22 13:43:55 <echelon> and where can i get the version that has your changes?
439 2012-10-22 13:44:03 <echelon> for faster block indexing thingy ^_^
440 2012-10-22 13:44:28 <sipa> i think there are too many problems with it still for a general preview build
441 2012-10-22 13:45:14 <echelon> oh
442 2012-10-22 13:45:39 <gmaxwell> ask again in a week
443 2012-10-22 13:45:41 <sipa> maybe in a few days i'll put a build online with big warnings on it
444 2012-10-22 13:45:44 <gmaxwell> yea...
445 2012-10-22 13:47:24 <gribble> New news from bitcoinrss: TheBlueMatt opened issue bitcoin/bitcoin#1657 <https://github.com/bitcoin/bitcoin/issues/1657> || apetersson opened issue bitcoin/bitcoin#1656 <https://github.com/bitcoin/bitcoin/issues/1656> || gmaxwell opened pull request bitcoin/bitcoin#1655 <https://github.com/bitcoin/bitcoin/pull/1655> || gmaxwell opened issue bitcoin/bitcoin#1654 <https://github.com/bitcoin/bitcoin/issues/1654> || gmaxwell opened issu
446 2012-10-22 13:52:12 <sipa> gmaxwell: i wonder if there could be logic to automatically -reindex, for example when blocks/blk* is available, but blktree/ isn' t
447 2012-10-22 14:04:06 <molecular> sipa: yes
448 2012-10-22 14:04:41 <molecular> sipa: I posted report in thread: https://bitcointalk.org/index.php?topic=119525.msg1290009#msg1290009
449 2012-10-22 14:05:13 <sipa> molecular: can you try with a clean datadir, for now?
450 2012-10-22 14:05:42 <molecular> sipa: redownload or loadblocks= ?
451 2012-10-22 14:06:30 <sipa> molecular: either is fine
452 2012-10-22 14:07:04 <molecular> it is possible by blk000x.dat are screwed. I wasn't able to start the old version due to "out of memory" error
453 2012-10-22 14:07:22 <molecular> s/by/my
454 2012-10-22 14:07:45 <sipa> you should be able to feed /dev/urandom into -loadblocks without problem
455 2012-10-22 14:07:47 <molecular> I think I will try both, loadblock= first, if that doesn't work, readownload chain
456 2012-10-22 14:08:02 <molecular> is it "loadblock" or "loadblocks"?
457 2012-10-22 14:08:07 <sipa> eh
458 2012-10-22 14:08:13 <sipa> whatever --help says
459 2012-10-22 14:08:15 <Luke-Jr> sipa: in theory, or practice?
460 2012-10-22 14:08:26 <Luke-Jr> I wonder what happens if you get the magic bytes followed by ffffffff
461 2012-10-22 14:08:31 <sipa> Luke-Jr: in theory, but i've seen no counterevidence in practice
462 2012-10-22 14:08:37 <Luke-Jr> or really <any 3>ff
463 2012-10-22 14:08:53 <Luke-Jr> sipa: you run 64-bit I bet ;)
464 2012-10-22 14:09:12 <sipa> Luke-Jr: nothing will happen in that case, as it checks for MAX_BLOCK_SIZE before reading
465 2012-10-22 14:09:21 <gmaxwell> "<user> okay I -loadblock=/dev/urandom and I'm not getting any blocks! it's using LOTS of CPU!"
466 2012-10-22 14:09:42 <sipa> "this sucks, i'm going to use electrum"
467 2012-10-22 14:09:48 <Luke-Jr> sipa: i c
468 2012-10-22 14:10:17 <sipa> Luke-Jr: the difference between theory and practice, however, is that in theory there is none, but in practice there is
469 2012-10-22 14:10:58 <gmaxwell> heh. yea, it just pegs the cpu. :P
470 2012-10-22 14:11:12 <sipa> well, -loadblock does rely on fseek
471 2012-10-22 14:11:24 <gmaxwell> sipa: well, must not catch the failure then.
472 2012-10-22 14:11:45 <Luke-Jr> does fseek fail if you're only moving forward?
473 2012-10-22 14:12:07 <sipa> i don't think you can fseek on a fifo at all, even only forward
474 2012-10-22 14:28:29 <BlueMatt> gmaxwell: re: checkpoint total chain work: sounds like a cool idea...at least downloading a signed copy of the min total chain work on a regular basis...though Im not sure which potential attack scenarios it really prevents...I mean the only real reason for checkpoints is to prevent the "feed you 1 billion diff-1 blocks" dos (and to sanity-test your acceptance), beyond that you should be able to identify sane chains on your own...
475 2012-10-22 14:28:55 <BlueMatt> its nice to be able to update the checkpoints more often, but...meh
476 2012-10-22 14:29:57 <gmaxwell> BlueMatt: nah, the other case checkpoints prevent is "You get your bitcoin from a signed software repo / ssl website/ etc;  then you start it, but the attacker is your ISP/router/etc and intercepts your network connection and doesn't let you see the real chain"
477 2012-10-22 14:31:02 <gmaxwell> One of the security assumptions in bitcoin is that the attacker cant totally deny you access to honest nodes.  In practice attackers really can do this, though it's usually harder than a random sybil attack (also protected against by checkpoints).
478 2012-10-22 14:31:34 <BlueMatt> well, yea, ok...still that falls pretty easily under "put an arbitrary checkpoint back three months"
479 2012-10-22 14:31:46 <BlueMatt> doesnt seem like doing more often updates solves much there
480 2012-10-22 14:32:35 <BlueMatt> (plus performing any actual attacks by rejecting your access to the real chain is hard to pull off, assuming the client is sane and alerts the user that its not getting new blocks)
481 2012-10-22 14:33:16 <gmaxwell> Do any do that yet? It's somewhat hard to do, because long gaps happen so you need to be careful with the warnings.
482 2012-10-22 14:33:47 <gmaxwell> BlueMatt: right, I was pointing out that regular checkpoints defend against this too??? respending to your 'the only real reason for checkpoints' comment.
483 2012-10-22 14:33:51 <BlueMatt> I thought bitcoin-qt notified you if it was running behind...
484 2012-10-22 14:33:56 <BlueMatt> maybe not then...
485 2012-10-22 14:34:28 <gmaxwell> BlueMatt: not really. it'll show that it's syncing onces its a couple hours back, but it's not something cautioning you at all.
486 2012-10-22 14:34:36 <BlueMatt> well, ok, checkpoints really all fall under "sanity test" case, but sure they do prevent a few "attacks"
487 2012-10-22 14:34:46 <BlueMatt> gmaxwell: ahh, ok well that needs fixed then ;)
488 2012-10-22 14:35:23 <BlueMatt> anyway, I suppose a "total height" (and maybe a block height -> total work) checkpoint is probably more useful than static checkpoints if you do reverse header sync
489 2012-10-22 14:35:46 <BlueMatt> (though block height -> total min work is probably no more useful than block height -> hash because it needs to be sufficiently far back anyway)
490 2012-10-22 14:36:32 <sipa> the problem with reverse-header sync is that you can't do much with the data (except verify that it is potentially correct, and was as hard to create as they claim) before it's connected to the genesis
491 2012-10-22 14:36:38 <gmaxwell> yea, It was just a thought. I don't think it's a really important idea. It's just one way to have more frequent checkpoints without worrying about disrupting the consensus algorithim. Dunno that anyone needs more frequent ones. :P
492 2012-10-22 14:36:58 <BlueMatt> gmaxwell: ok, fair enough
493 2012-10-22 14:37:55 <sipa> you could think of an algorithm where you first just ask for the timestamps of the multiple-of-2016 blocks... that's a tiny amount of data, and you'd immediately know the resulting total amount of work
494 2012-10-22 14:38:44 <BlueMatt> sipa: downloading the full set of 200000-some headers is really cheap (you could even throw away everything but the hashes themselves for most blocks)
495 2012-10-22 14:39:01 <sipa> yeah
496 2012-10-22 14:39:09 <sipa> it's probably not worth optimizing this further
497 2012-10-22 14:39:19 <sipa> at least for now
498 2012-10-22 14:40:25 <sipa> anyway - what could be done, is then iteratively requesting block headers in between 2016-apart timestamps, starting with the ranges where difficulty is the highest
499 2012-10-22 14:41:07 <BlueMatt> yea, that would be faster if you were tight on bw
500 2012-10-22 14:41:14 <BlueMatt> (or there were millions of blocks)
501 2012-10-22 14:41:57 <sipa> even with a million blocks, which is at a point in time where moore's law will have pushed technology a lot further than we are today, it's only 84 MB to download all headers
502 2012-10-22 14:42:18 <BlueMatt> yea...
503 2012-10-22 14:42:35 <gmaxwell> sipa: amiller has an algorithim idea that would require some additional commitments but would allow log time chain selection instead of linear. But I think it's mostly a pointless optimization (though academically interesting)
504 2012-10-22 14:42:37 <sipa> faking a longer header chain, requires one that has shorter intervals, thus higher difficulty
505 2012-10-22 14:43:11 <sipa> faking a shorter header chain... well, at most 84 MB of RAM temporarily wasted at the receiver side
506 2012-10-22 14:44:18 <gmaxwell> yea, mostly the concern is that it's a lot of data.. it's that one doofus with an ASIC could potentially make you waste 84mb*n_hosts pulling a bunch of hopeless chains... since you have to pull ~all offered you to sum them up.
507 2012-10-22 14:44:38 <gmaxwell> The stuff about fetching backwards is just a hurestic to make early termination maximally effective.
508 2012-10-22 14:45:49 <sipa> but forward header fetching means you can start downloading actual blocks and validating those, while headers are still being downloaded
509 2012-10-22 14:46:16 <sipa> (and if the validation fails, you also cut of the header download)
510 2012-10-22 14:46:48 <sipa> then again, producing valid low-difficulty blocks is about as hard as creating low-difficulty block headers
511 2012-10-22 14:48:43 <gmaxwell> Yea, I don't think that helps you with header flooding.  Alternatively, just shipping the last 32 bits of all block IDs up to the checkpoint. Would help.
512 2012-10-22 14:49:37 <gmaxwell> I think after things stablize with the reward halving and asic shipping I'm going to float a BIP to increase the minimum difficulty above some height, to mostly kill these stupid flooding attacks.
513 2012-10-22 14:50:14 <sipa> how about just shipping the 2016-cycle timestamp intervals?
514 2012-10-22 14:50:28 <sipa> that means you know what difficulty every block needs to have in advance
515 2012-10-22 14:50:34 <gmaxwell> sipa: this is in fact what thomasv is going to do.
516 2012-10-22 14:50:53 <gmaxwell> Because he plans on only pulling historical headers if the user has older keys.
517 2012-10-22 14:51:41 <gmaxwell> But for the anti flooding I don't know that it helps...
518 2012-10-22 14:52:02 <gmaxwell> Because for the first years worth of blocks you can compute an enormous number of alternative chains which comply with that rule completely.
519 2012-10-22 14:52:02 <sipa> it means that an attacker has to redo as much work as the real chain at every point in time
520 2012-10-22 14:52:07 <sipa> agree
521 2012-10-22 14:52:36 <sipa> well, a combination is also possible
522 2012-10-22 14:52:39 <gmaxwell> Requring a minimum difficulty of 100k after the first point that we hit 100k+, would seriously complicate attacking and bitcoin would be dead if difficulty ever fell below that. Clients could then ship minicheckpoints (32 bit values) to that height.
523 2012-10-22 14:53:19 <gmaxwell> (I'd wait until after the halving and introduction of asics before talking about it just because there is hashrate uncertanty from both of these changes)
524 2012-10-22 14:58:45 <BlueMatt> yea, Im still a fan of shipping the full list of last 32-bit of block hashes up to block N where N has a high diff...
525 2012-10-22 14:59:22 <BlueMatt> means you can do forward headers sync and not get screwed by 1-diff chains
526 2012-10-22 15:03:42 <orion> Flooding attacks?
527 2012-10-22 15:03:50 <orion> Bitcoin is under attack?
528 2012-10-22 15:04:18 <gmaxwell> orion: ... No.
529 2012-10-22 15:04:57 <gmaxwell> orion: bitcoin does not get attacked because we think up the attacks first and make it strong against them.
530 2012-10-22 15:06:59 <sipa> ... hopefully
531 2012-10-22 15:11:49 <Luke-Jr> well, there's the ongoing SDice attack???
532 2012-10-22 15:12:10 <sipa> molecular: also, using ~ after -loadblock= won't work i think; the shell only expands that at the beginning of an argument
533 2012-10-22 15:26:16 <slush> sipa: Can you please point me to place in bitcoind sources, where are different validation rules for testnet blocks?
534 2012-10-22 15:26:51 <slush> I'm still breaking my head on this; code which works on live network don't work on testnet. Even when I'm submitting every block which has at least difficulty 1
535 2012-10-22 15:27:49 <sipa> search for GetNextWorkRequired
536 2012-10-22 15:28:07 <Luke-Jr> slush: are you encoding the minimal-length block height?
537 2012-10-22 15:29:05 <slush> Luke-Jr: I don't know about that, so probably - no
538 2012-10-22 15:29:14 <slush> you mean block height in coinbase?
539 2012-10-22 15:29:16 <Luke-Jr> slush: yes
540 2012-10-22 15:29:21 <slush> yes, I'm doing this
541 2012-10-22 15:29:30 <slush> im producing v2 blocks
542 2012-10-22 15:29:38 <Luke-Jr> with Bitcoin it's always 3 bytes for the near future, but on testnet it's probably 2 bytes
543 2012-10-22 15:29:40 <slush> sipa: thanks, I'll take a look
544 2012-10-22 15:30:04 <slush> Luke-Jr: I know, I should implement that correctly
545 2012-10-22 15:30:19 <slush> btw my code worked on testnet without any issue few weeks ago
546 2012-10-22 15:30:35 <slush> I don't know what happen; I developed stratum pool on testnet3 :-/
547 2012-10-22 15:31:12 <Luke-Jr> slush: does the older version still work?
548 2012-10-22 15:32:00 <slush> Luke-Jr: older version of what?
549 2012-10-22 15:32:05 <gmaxwell> slush: testnet3 may have cross over the header enforcement threshold in that time.
550 2012-10-22 15:32:22 <jgarzik> testnet differences I can see:  (1) difficulty calculation, (2) block nVersion=2 supermajority checking (3) pchMessageStart, (4) bitcoin address id, (5) alert key, (6) checkpoints, (7) IRC usage, (8) testnet mempool accepts non-standard transactions [it does??? sigh], (9) Changing pblock->nTime can change work required on testnet, (10) DNS and hardcoded seeds avoided
551 2012-10-22 15:32:29 <gmaxwell> slush: what error are you getting when your blocks are rejected.
552 2012-10-22 15:32:29 <jgarzik> slush: ^^
553 2012-10-22 15:32:44 <jgarzik> has nVersion==2 requirement kicked in, on testnet3?
554 2012-10-22 15:33:11 <gmaxwell> jgarzik: of course testnet mempool accepts non-standard txns. Thats how you can actually expirement with them there without having to mine all your own blocks.
555 2012-10-22 15:33:38 <gmaxwell> jgarzik: I would be shocked if it hadn't.
556 2012-10-22 15:33:43 <slush> gmaxwell: Luke-Jr: http://blockexplorer.com/testnet/block/000000002076870fe65a2b6eeed84fa892c0db924f1482243a6247d931dcab32 This is block produced by my pool. Can you check if height in coinbase is OK?
557 2012-10-22 15:34:18 <gmaxwell> coinbase there is 020862062f503253482f04b8864e50080800000200000001072f736c7573682f
558 2012-10-22 15:34:20 <Luke-Jr> slush: looks good to me
559 2012-10-22 15:34:40 <slush> ad 1) difficulty calculation: I just re-use nbits what I get from GBT
560 2012-10-22 15:34:50 <slush> ad 2) I use version 2
561 2012-10-22 15:34:58 <slush> ad 3) Have no idea what it is :)
562 2012-10-22 15:35:06 <Luke-Jr> slush: also, might note there is an odd corner-case encoding blocks like 0x8000-0xffff etc
563 2012-10-22 15:35:11 <gmaxwell> slush: okay, so whenever bitcoin rejects a block it logs _why_ it was rejected.
564 2012-10-22 15:35:26 <Luke-Jr> (that isn't the case with the block you linked tho)
565 2012-10-22 15:35:38 <slush> gmaxwell: oh, I completely forgot that there may be anything interesting in debug.log ;)
566 2012-10-22 15:37:01 <slush> hm,  hash doesn't match nBits
567 2012-10-22 15:37:31 <slush> it means "low difficulty", right?
568 2012-10-22 15:38:28 <gmaxwell> Right it means the share is too low difficulty even for the _claimed_ nbits.
569 2012-10-22 15:38:35 <Luke-Jr> "high-hash"
570 2012-10-22 15:43:29 <slush> ok, miner returned hash "1063466275312296339748953154347848099967999507717274656009989480033" as diff1 solution
571 2012-10-22 15:43:43 <Luke-Jr> ???
572 2012-10-22 15:43:45 <slush> and my pool sees current testnet nbits as "133697245282347990925091246082784728529847289082728363897074483200"
573 2012-10-22 15:45:14 <slush> Luke-Jr: ok, target in hexa is 0000000001450000000000000000000000000000000000000000000000000000
574 2012-10-22 15:45:33 <slush> Luke-Jr: it looks much higher than diff1, claimed by bitcoind getinfo, right?
575 2012-10-22 15:46:06 <gmaxwell> getinfo shows the difficulty of whatever the last block was.
576 2012-10-22 15:46:18 <Luke-Jr> 000000000a1924b018a1b8d8aa0471ce043fa2da55a520a0f5fd3601afcd5e61 > 0000000001450000000000000000000000000000000000000000000000000000
577 2012-10-22 15:46:38 <gmaxwell> Testnet is currently at difficulty ~200 except for blocks under the 20 minute exception.
578 2012-10-22 15:46:41 <slush> Luke-Jr: ok, so bitcoind getinfo is here just for fun?
579 2012-10-22 15:46:54 <Luke-Jr> slush: it's not for miners
580 2012-10-22 15:47:06 <Luke-Jr> it's for users to see the network's last difficulty
581 2012-10-22 15:47:15 <gmaxwell> slush: it's always a block behind.
582 2012-10-22 15:47:29 <gmaxwell> E.g. last difficulty, not next difficulty.
583 2012-10-22 15:47:30 <Luke-Jr> slush: testnet3 blocks can change difficutly pretty often tho
584 2012-10-22 15:47:32 <slush> I feel like idiot, really
585 2012-10-22 15:47:57 <Luke-Jr> slush: always use the 'bits' from GBT ;)
586 2012-10-22 15:48:03 <slush> Luke-Jr: yes, I'm using it
587 2012-10-22 15:48:08 <gmaxwell> slush: were you using getinfo to get the target? How did that ever work? :P
588 2012-10-22 15:48:32 <slush> gmaxwell: I expected that nbits and difficulty should be linked together, so I'm using it for human cross-check
589 2012-10-22 15:48:47 <gmaxwell> ah! yea, it's confusing, and more obviously so on testnet.
590 2012-10-22 15:48:51 <slush> of course im using nbits
591 2012-10-22 15:48:56 <slush> yes, totally confusing
592 2012-10-22 15:49:18 <slush> so what? testnet diff is just higher? I expected there are some shortcuts to maintain it lower
593 2012-10-22 15:49:30 <slush> so I wasn't surprised that I see difficulty in getinfo at 1
594 2012-10-22 15:49:35 <Luke-Jr> slush: if 10 minutes pass since the last block, you can mine at diff 1
595 2012-10-22 15:49:40 <slush> but now Im confused, because you're talking that it is above 200
596 2012-10-22 15:49:56 <Luke-Jr> for 1 block
597 2012-10-22 15:50:05 <Luke-Jr> after that block, it goes back to what it "should" be
598 2012-10-22 15:50:08 <Luke-Jr> until 10 mins pass again
599 2012-10-22 15:50:22 <slush> who created these rules?
600 2012-10-22 15:50:29 <Luke-Jr> Pizza Hut
601 2012-10-22 15:50:36 <gmaxwell> Luke-Jr: 20 minutes, not 10. :P
602 2012-10-22 15:50:41 <Luke-Jr> oh
603 2012-10-22 15:51:26 <gmaxwell> slush: unfortunately people keept mining testnet up to several hundred difficulty then walking away and we'd go many days without a block and people couldn't use it for testing.
604 2012-10-22 15:51:42 <slush> yes, testnet is currently unusable for testing
605 2012-10-22 15:52:09 <slush> now I see that my problem was just that I wasnt able to produce diff 200+ block with my testing equipment
606 2012-10-22 15:52:10 <gmaxwell> slush: You can easily make that disabled for yourself with a small additional change.
607 2012-10-22 15:52:21 <gmaxwell> oh you mean the diff is too high
608 2012-10-22 15:52:28 <Luke-Jr> slush: just give it 20 minutes between blocks
609 2012-10-22 15:52:42 <gmaxwell> well if set your time to +20 minutes since the last block you can mine at diff 1 until you hit two hours into the future. :)
610 2012-10-22 15:52:57 <slush> Luke-Jr: I'm mining for 40 minutes, but somebody "withdrawn" that "lucky" 20-minute block for me :-P
611 2012-10-22 15:53:09 <Luke-Jr> slush: TNIAB
612 2012-10-22 15:53:49 <slush> better to setup own testnet :/
613 2012-10-22 15:53:50 <gmaxwell> slush: you can get the 20 minute blocks early by twiddling your time.
614 2012-10-22 15:54:30 <gmaxwell> change to GetAdjustedTime()+1201  ... and you can mine a couple blocks right away.. until you run up against the future test.
615 2012-10-22 15:54:39 <gmaxwell> (main.cpp)
616 2012-10-22 15:54:46 <gmaxwell> (or isolate yourself)
617 2012-10-22 15:54:53 <slush> gmaxwell: Setting up my own rules to testing production software is the way to the hell.
618 2012-10-22 15:55:02 <slush> That resetting rule on testnet sucks, really
619 2012-10-22 15:55:07 <slush> I'm going to setup my own testnet :/
620 2012-10-22 15:55:18 <Luke-Jr> ???
621 2012-10-22 15:55:20 <gmaxwell> slush: you're confusing me.
622 2012-10-22 15:55:36 <gmaxwell> You're complaining about the high difficulty and then also complaining about the only thing that lets you mine a block on it at all?