1 2012-12-22 00:08:23 <da2ce7> ok qmake sort-of works, it generates a broken vs file... lets see if it works if I hack it up.
2 2012-12-22 00:18:28 <da2ce7> O.o a complete mess.
3 2012-12-22 00:20:39 <jrmithdobbs> the bitcoin build process in two words
4 2012-12-22 00:20:42 <jrmithdobbs> ;p
5 2012-12-22 00:21:06 <jrmithdobbs> should have used it before qmake
6 2012-12-22 01:21:06 <wizkid057> ACTION is wasting loads of time in #yandere
7 2012-12-22 03:24:07 <andytoshi> hey all, i just compiled bitcoind with llvm-clang 3.2
8 2012-12-22 03:24:18 <andytoshi> which was released yesterday
9 2012-12-22 03:25:34 <Joric> guys how do you rebase pull requests? it becomes ugly after first few hundred of commits
10 2012-12-22 03:25:58 <Joric> i tried rebasing it locally it just appends the rebase commit on top of others
11 2012-12-22 03:31:32 <gmaxwell> if you rebase after pulling onto an already dirty branch thats been merged that won't get you what you want.
12 2012-12-22 03:32:05 <gmaxwell> It's perhaps not the most proper, but I usually reset the branch pull the clean tree onto it, then cherry pick my changes back into it.
13 2012-12-22 03:32:41 <gmaxwell> though if you have a lot of commits, rebasing _on the branch_ with regular rebase when you haven't pulled the new commits into it with a merge should work.
14 2012-12-22 03:42:22 <Joric> i know the only way of deleting commits from the remote repository it's git push -f origin HEAD^^
15 2012-12-22 03:43:05 <Joric> when i rebase it just puts all rebased shit in the last commit i want to delete previous edits from the pull request
16 2012-12-22 03:44:33 <gmaxwell> then squash them while rebasing.
17 2012-12-22 03:44:52 <gmaxwell> if you do the interactive rebase it lets you specify what action should be taken for every rebased commit.
18 2012-12-22 03:45:05 <gmaxwell> You'd squash all but the ultimate one, and on it you'd edit the commit message.
19 2012-12-22 03:45:11 <gmaxwell> then push -f the whole thing up.
20 2012-12-22 03:47:26 <Joric> yeah i do squash but i just get one more commit after that )
21 2012-12-22 03:47:49 <Joric> git rebase -i HEAD~2 -> push -> one more commit in the history )
22 2012-12-22 03:59:17 <Joric> well i may go back one step git push -f origin HEAD^^ then push but i thought rebase does that itselft
23 2012-12-22 04:39:59 <andytoshi> clang is warning,
24 2012-12-22 04:40:01 <andytoshi> /home/username/dload/code/bitcoin/src/netbase.h:34:7: warning: Null pointer argument in call to memory copy function
25 2012-12-22 04:40:10 <andytoshi> line 34 is "class CNetAddr"
26 2012-12-22 04:40:15 <andytoshi> what in the world could that mean?
27 2012-12-22 04:48:17 <jgarzik> andytoshi: maybe it's referring to IMPLEMENT_SERIALIZE() ?
28 2012-12-22 04:48:22 <jgarzik> ACTION randomly guesses
29 2012-12-22 04:51:45 <andytoshi> i guess so...that's the only place in the file anything might be called
30 2012-12-22 04:51:48 <andytoshi> everything else is definitions
31 2012-12-22 04:55:11 <gmaxwell> andytoshi: run the file through the preprocessor, then compile that and see where the warning is?
32 2012-12-22 04:55:35 <andytoshi> oh, good call
33 2012-12-22 04:55:37 <phantomcircuit> i totally forgot i was still runnign that peer walker
34 2012-12-22 04:55:44 <BTCOxygen> Will a RAM disk speed up the blockchain download?
35 2012-12-22 04:55:50 <gmaxwell> BTCOxygen: yes.
36 2012-12-22 04:56:00 <BTCOxygen> gmaxwell: Thanks
37 2012-12-22 04:56:09 <gmaxwell> well for 0.8 it doesn't make much difference, but it's a very big speedup for older code.
38 2012-12-22 04:56:32 <BTCOxygen> Did 0.8 release?
39 2012-12-22 04:57:00 <BTCOxygen> 7.2 is the latest i see
40 2012-12-22 04:59:28 <gmaxwell> 0.8 meaning the in-development work (you are in #bitcoin-dev, after all)
41 2012-12-22 05:01:31 <BTCOxygen> gmaxwell: Thanks, I will be trying out 0.8 sometime today
42 2012-12-22 05:01:58 <BTCOxygen> Plus i will re-download the blockchain to see the speed improvement you mention
43 2012-12-22 05:05:13 <theymos> I think it'd be pretty easy to replace Bitcoin's use of macros in serialization with templates. That might make errors like andytoshi's more clear.
44 2012-12-22 05:11:33 <Luke-Jr> BTCOxygen: it is not safe to mine with 0.8 yet
45 2012-12-22 05:11:59 <Luke-Jr> theymos: templates tend to make errors less clear :P
46 2012-12-22 05:20:50 <andytoshi> hmm..clang won't run on the preprocessor output for some reason (starts finding errors in system headers all of a sudden)
47 2012-12-22 05:21:06 <andytoshi> if i selectively preprocess the IMPLEMENT_SERIALIZE block, that doesn't change the error
48 2012-12-22 05:23:16 <theymos> Luke-Jr: If you replaced 'READWRITE(FLATDATA(ip));' with something like 'serialize<unsigned char[16]>(out, ip);' (with the template parameter actually being implicit usually), I'd think that any errors would be more clear. If there's an error in 'serialize' you'd (I think) get a line number within the version of the function that is actually called.
49 2012-12-22 05:23:30 <andytoshi> never mind, i was preprocesing the .h file, not the .cpp
50 2012-12-22 05:25:23 <andytoshi> ...i get the exact same error whether or not i preprocess netbase.cpp
51 2012-12-22 05:59:35 <MC1984> is there any work on the P2P stuff going on
52 2012-12-22 05:59:49 <MC1984> peer efficiency, topology etc
53 2012-12-22 06:00:36 <MC1984> some nat traversal would be nice :(
54 2012-12-22 06:01:17 <MC1984> some sort of grouping behaviour for high bandwidth peers maybe
55 2012-12-22 06:02:12 <MC1984> or is it On The List
56 2012-12-22 06:08:03 <gmaxwell> MC1984: uh. topology influence has major security risks.
57 2012-12-22 06:08:27 <gmaxwell> we have upnp ... no recent evidence that we need anything more right now wrt nat.
58 2012-12-22 06:09:07 <gmaxwell> the interesting areas is in just doing the initial block download better, making the networking more concurrent, and picking faster peers to pull blocks from.
59 2012-12-22 06:09:27 <MC1984> even some sort of auto-swarming behavior of good nodes? To gradually form a sort of core of the bitcoin network
60 2012-12-22 06:09:44 <MC1984> like peple are probably doing already with -addnode, but more efficient
61 2012-12-22 06:09:47 <gmaxwell> no. thats bad.
62 2012-12-22 06:10:17 <MC1984> why
63 2012-12-22 06:10:18 <gmaxwell> 'efficient' topologies tend to be vulernable ones.
64 2012-12-22 06:10:39 <MC1984> if the "core" got completely compromised somehow, the netowrk would just form another one with whatever is left
65 2012-12-22 06:10:55 <gmaxwell> ...
66 2012-12-22 06:10:59 <MC1984> but in the meantime it should reduce propagation delays?
67 2012-12-22 06:11:20 <gmaxwell> MC1984: what transactions taking a second to appear in your client is problematic for you?
68 2012-12-22 06:11:52 <MC1984> i heard there was a kerfuffle about propagation delay for ome people
69 2012-12-22 06:11:54 <MC1984> miners?
70 2012-12-22 06:12:07 <gmaxwell> Sounds like personal problems.
71 2012-12-22 06:12:39 <MC1984> well
72 2012-12-22 06:12:51 <gmaxwell> Lets not go about compromising the security of bitcoin because some niche audience is woried about hypermiling their mining returns. They can freely setup fast hubs to peer with if they really care, and that doesn't reduce security.
73 2012-12-22 06:12:51 <MC1984> i suppose this core would probably end up in america.......
74 2012-12-22 06:13:40 <MC1984> in what specific way does that idea reduce security
75 2012-12-22 06:13:52 <MC1984> im sure it does but im too stupid to see it
76 2012-12-22 06:14:54 <gmaxwell> You need to give a specific idea instead of a cloud of general comments if you want a specific analysis.
77 2012-12-22 06:16:25 <gmaxwell> But in _general_ adding structure to the network weakens it in two ways: it decreases the min-cut of the graph over most random wirings, so there are fewer links that need to go down before you get a partition, and it tends to allow nodes to self select important locations where they are in a better position to isolate peers or create partitions.
78 2012-12-22 06:17:41 <MC1984> even when that structure is desined by the network itself according to the capabilities of individual nodes?
79 2012-12-22 06:18:24 <MC1984> i accept your rebuttal, but look at it another way
80 2012-12-22 06:18:54 <gmaxwell> some structures are safer than others but its quite difficult to reason about them, and because of the high fanout it shouldn't be~ any faster in any case.
81 2012-12-22 06:19:18 <gmaxwell> MC1984: because nodes can choose to have 'capabilities' (or lie) to position themselves in preferred places.
82 2012-12-22 06:19:42 <MC1984> people can and are manually doing connect and addnode anyway for whatever reason they want to, forming a sort of higher teir of the network manually
83 2012-12-22 06:19:54 <gmaxwell> So that insted of picking a peer out of a bigger set you pick out of a smaller set, so an attacker has to run fewer clones to get all your connections.
84 2012-12-22 06:20:09 <gmaxwell> MC1984: yes, and addnodes are in addition to the regular topology so they can't hurt it.
85 2012-12-22 06:20:33 <gmaxwell> and??? go back, first you there ought to be a problem worth fixing.
86 2012-12-22 06:20:40 <gmaxwell> s/you//
87 2012-12-22 06:21:29 <gmaxwell> MC1984: manually added connections are interesting precisely because they're manual??? you can be pretty confident that they're not sybil attack clones, vs the automatic ones.
88 2012-12-22 06:22:50 <MC1984> what if some actor threw up lots of high BW nodes in strategic places and watched it become the bitcoin core
89 2012-12-22 06:23:00 <MC1984> i just killed bitcoin :(
90 2012-12-22 06:23:53 <gmaxwell> what are you talking about? You drank that methanol laden booze didn't you? :P
91 2012-12-22 06:24:18 <MC1984> fuck its hard and a bit depressing when you want to create something complicated and beautiful and the foremost thought in you mind must at all times be "How will people try to destroy this?"
92 2012-12-22 06:25:07 <gmaxwell> pfft. The beauty of it is building things that aren't fragile. If postulating malice makes you sad, postulate murphey instead??? anything evil can happen through bad luck or incompetence instead.
93 2012-12-22 06:26:03 <gmaxwell> For example, if nodes preferentially peered with the lowest latency peers you'd have very few peerings crossing the atlantic??? and ISP goes out and perhaps europe and the US partition and start building seperate forks.
94 2012-12-22 06:26:53 <MC1984> yes i just realised the latency stuff is a bad bad idea
95 2012-12-22 06:27:12 <MC1984> also i didnt clarify that nodes would verify each others high capabilities
96 2012-12-22 06:27:17 <MC1984> somehow......
97 2012-12-22 06:27:45 <gmaxwell> Even if they did, the attacker just runs good nodes. Easy to do when you're motivated and expect rewards.
98 2012-12-22 06:28:24 <MC1984> how could a malicious "core" do anything bad though
99 2012-12-22 06:28:59 <gmaxwell> uh, someone that can create partitions can do nasty things.
100 2012-12-22 06:28:59 <MC1984> i also didnt clarify that i wouldnt think all connections would have to be to "known good" nodes
101 2012-12-22 06:29:25 <MC1984> perhaps only 2 of the standard 8 would be selected, rest random?
102 2012-12-22 06:30:02 <gmaxwell> maybe??? if you work through it enough you can get something that isn't obviously dangerous. We hope. But then what good has it done.
103 2012-12-22 06:30:05 <MC1984> im probably implying mroe of a centralisation effect than i really mean by calling it a core
104 2012-12-22 06:30:22 <gmaxwell> Joe random node has no particular need to recieve a transaction in 100ms instead of 300ms.
105 2012-12-22 06:30:39 <MC1984> yeah
106 2012-12-22 06:30:43 <gmaxwell> If anything making it faster decreases privacy by making graph timing analysis more potent.
107 2012-12-22 06:31:02 <gmaxwell> So perhaps we should instead talk about adding more random forwarding delays.
108 2012-12-22 06:31:11 <MC1984> but the miners/exchangers/big merchant processors might want to group up for common benefit
109 2012-12-22 06:31:22 <gmaxwell> then let them.
110 2012-12-22 06:31:36 <gmaxwell> and by doing that they can add vetted links which add something that nothing automatic can add.
111 2012-12-22 06:32:00 <gmaxwell> ??? high confidence of not being partionable by a sybil no matter how many nodes they run.
112 2012-12-22 06:32:27 <MC1984> i see
113 2012-12-22 06:32:44 <MC1984> also forgot about that timing attack shit
114 2012-12-22 06:32:45 <gmaxwell> sorry for being ranty, :P I'm tired.
115 2012-12-22 06:33:08 <MC1984> youre no more ranty than normal :P
116 2012-12-22 06:33:17 <gmaxwell> well "use tor". It's only an argument here because I don't think there is a need to improve network latency generally.
117 2012-12-22 06:34:19 <MC1984> maybe not for tx
118 2012-12-22 06:34:31 <MC1984> but i heard blocks were hitting a good half MB regularly now
119 2012-12-22 06:35:00 <gmaxwell> block latency is only important to miners and they can take care of themselves.
120 2012-12-22 06:35:12 <gmaxwell> Most of them are big pools who earn an income on this stuff.
121 2012-12-22 06:35:46 <gmaxwell> p2pool even took care of itself relative to this.
122 2012-12-22 06:36:07 <MC1984> well
123 2012-12-22 06:36:16 <MC1984> that random forwarding delay sounds like a good idea
124 2012-12-22 06:37:31 <gmaxwell> mean size of the last 100 blocks is 173126 bytes. FWIW
125 2012-12-22 06:41:31 <gmaxwell> (harmonic mean is only 6153.82 .. we're clearly a long way from maximum txn rate. :P)
126 2012-12-22 10:55:52 <sipa> 234 non-standard transactions in the last 68000
127 2012-12-22 11:01:09 <sipa> eh, non-canonical
128 2012-12-22 15:49:20 <etotheipi_> sipa: I odn't know if you saw the message, but I matched all your test vectors
129 2012-12-22 15:49:25 <etotheipi_> sipa: did you hear back from hal?
130 2012-12-22 15:50:59 <sipa> etotheipi_: not yet
131 2012-12-22 15:51:02 <sipa> yes, i saw it
132 2012-12-22 15:51:08 <andytoshi> i've got a dumb github question..
133 2012-12-22 15:51:13 <sipa> nice
134 2012-12-22 15:51:22 <andytoshi> how do i modify a pull request? can i do a push -f to my branch on github
135 2012-12-22 15:51:27 <andytoshi> or should i do a new request
136 2012-12-22 15:52:10 <sipa> andytoshi: push -f
137 2012-12-22 15:52:18 <andytoshi> cool, thanks
138 2012-12-22 15:52:48 <etotheipi_> sipa: the one thing I didn't do in the test was convert the original seed... how were you doing that?
139 2012-12-22 15:53:01 <etotheipi_> sipa: I simply copied in the first priv key and ran the tests from that
140 2012-12-22 15:53:02 <sipa> etotheipi_: see BIP32 :)
141 2012-12-22 15:53:18 <etotheipi_> has it always been specified there?
142 2012-12-22 15:53:21 <sipa> yes
143 2012-12-22 15:53:29 <etotheipi_> I thought we/you were leaving that out
144 2012-12-22 15:54:04 <sipa> no, third time i repeat it: conversion seed->masterkey is in bip32; how you generate the seed isn't (bip32 just says "use a PRNG")
145 2012-12-22 15:54:45 <etotheipi_> okay, sorry about that... I thought both those steps were actually one, undecided-so-far step
146 2012-12-22 15:55:10 <sipa> (the reason being that seeds are arbitrary byte strings, while the master key has some structure)
147 2012-12-22 15:55:25 <sipa> so you just need something that can come up with a bytestring
148 2012-12-22 16:08:27 <sipa> etotheipi_: anyway, nice that the vectors match
149 2012-12-22 16:10:00 <etotheipi_> sipa: so barring any red flags by Hal, are we "done" with this?
150 2012-12-22 16:10:38 <etotheipi_> I probably won't have my wallet format done for a while, but I would like to make sure that nothing major is going to change under me
151 2012-12-22 16:11:24 <sipa> etotheipi_: i suppose i should still post to the dev mailinglist before finalizing, but i'll wait until Hal reacts
152 2012-12-22 16:11:33 <etotheipi_> okay
153 2012-12-22 16:12:40 <sipa> remarkable, a large percentage of non-canonical sigs still come from blockchain.info
154 2012-12-22 16:36:50 <vazakl> http://arxiv.org/pdf/1212.2142v1.pdf
155 2012-12-22 16:55:46 <sipa> vazakl: relevance?
156 2012-12-22 18:36:45 <etotheipi_> sipa: is it okay to share those test vectors? the HW wallet thread from slush seemed interested in seeing an example crypto implementation of BIP 32
157 2012-12-22 19:06:38 <sipa> etotheipi_: with a disclaimer it's not final yet
158 2012-12-22 19:07:51 <etotheipi_> sipa: that's fine... I'm actually just going to point them to the part of my unit-testing code that has the data hardcoded
159 2012-12-22 19:08:04 <etotheipi_> so it won't be all that official-looking
160 2012-12-22 19:46:04 <sipa> da2ce7_d: serms microsoft more or less intentionally made it hard to port applications to winrt
161 2012-12-22 19:46:19 <sipa> *seems
162 2012-12-22 19:48:08 <denisx> sipa: Never attribute to malice that which is adequately explained by stupidity
163 2012-12-22 19:49:33 <sipa> denisx: at least the wikipedia page makes it sound like that
164 2012-12-22 19:49:48 <sipa> which may or may not be a neutral point of view :p
165 2012-12-22 19:57:17 <gmaxwell> sipa: windows has been almost 20 years of unending backwards compatiblity. Without knowing the specifics I expect breaking some stuff could only be good for it!
166 2012-12-22 20:32:42 <midnightmagic> NetBSD has also been backwards-compatible for quite some time.
167 2012-12-22 21:22:06 <bitlane> anybody heard about satan miner
168 2012-12-22 21:22:10 <bitlane> still looking for more info
169 2012-12-22 21:35:18 <bitlane> okay we or a member of my team will post a link once its discovered
170 2012-12-22 21:35:28 <bitlane> should be of great value
171 2012-12-22 21:35:32 <bitlane> possibly millions!
172 2012-12-22 21:35:35 <bitlane> peace
173 2012-12-22 22:16:31 <MC1984> satan miner?
174 2012-12-22 22:18:33 <gmaxwell> hm.
175 2012-12-22 22:18:42 <gmaxwell> 20MB free and this node didn't autoshutdown.
176 2012-12-22 22:19:32 <gmaxwell> I wonder if thats how its getting stuck when it runs out of space??? it's failing to autoshutdown and corrupting the db.
177 2012-12-22 22:23:33 <sipa> which code?
178 2012-12-22 22:23:59 <sipa> i believe checkdiskspace was broken until recenly
179 2012-12-22 22:25:16 <gmaxwell> looks like it's master on 12/13 plus some more recent pulls.
180 2012-12-22 22:25:22 <gmaxwell> any idea what commit fixed it?
181 2012-12-22 22:25:35 <gmaxwell> well ??? I think it's unambigious, thats the issue indeed.
182 2012-12-22 22:25:49 <gmaxwell> heh and yet again I find a real bug thats already fixed. :P
183 2012-12-22 22:26:20 <sipa> 622da5df6ec0efc
184 2012-12-22 22:26:36 <sipa> merged on december 7
185 2012-12-22 22:26:40 <sipa> so you should have it
186 2012-12-22 22:28:48 <gmaxwell> hm. I should have. Will test.
187 2012-12-22 22:29:46 <jgarzik> getheaders 160310 to 0000000000000000000000000000000000000000000000000000000000000000
188 2012-12-22 22:29:52 <jgarzik> does bitcoinj use getheaders?
189 2012-12-22 22:30:05 <sipa> yes
190 2012-12-22 22:31:17 <gmaxwell> sipa: seems to not be working on a current build. I'll fix.
191 2012-12-22 23:52:19 <gmaxwell> sipa: diapolo's patch removed the CheckDiskSpace from accept block. It _only_ checks at start and when it grows preallocations now.
192 2012-12-22 23:53:25 <gmaxwell> and so it can happily completely run out of space and apparently leveldb can become unrecoverably corrupted in that case (which is somewhat problematic regardless of when we CheckDiskSpace since there is still a race there)
193 2012-12-22 23:54:28 <rdymac> the blockchain file size is already at 5.6 GB?
194 2012-12-22 23:55:25 <gmaxwell> rdymac: no, it's about 4.2 GiB.
195 2012-12-22 23:56:59 <rdymac> pruned should be less?
196 2012-12-22 23:57:04 <rdymac> or it doesn't affect the size?
197 2012-12-22 23:57:28 <gmaxwell> rdymac: What are you talking about with respect to pruned?
198 2012-12-22 23:58:47 <gmaxwell> (I'm trying to determine if you're confused and believe that we have implemented blockchain pruning)
199 2012-12-22 23:58:49 <rdymac> in the 0.8 coming version the blockchain file size will be smaller? Or it just will be faster
200 2012-12-22 23:58:56 <gmaxwell> No. It will just be faster.
201 2012-12-22 23:59:44 <gmaxwell> much much faster??? as it doesn't use the blockchain for validating new blocks??? only for serving to other peers and reorganizing the chain.
202 2012-12-22 23:59:46 <rdymac> wouldn't the size be a problem soon?