1 2012-02-02 01:00:45 <gribble> New news from bitcoinrss: mcandre opened issue 793 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/issues/793>
  2 2012-02-02 01:42:10 <luke-jr> I'd like to finish off https://en.bitcoin.it/wiki/P2SH_Votes -- who's still missing on it?
  3 2012-02-02 01:43:00 <midnightmagic> By the way, someone asked what a good well-run open source project looks like: good example is Tahoe LAFS.
  4 2012-02-02 01:43:13 <midnightmagic> That is one of the more highly-process-driven projects I personally have ever observed.
  5 2012-02-02 01:43:37 <midnightmagic> (Successfully process-driven, I might add. Not everyone is capable of pulling it off.)
  6 2012-02-02 01:43:46 <gmaxwell> midnightmagic: It also has a very narrow focus, so I expect it's difficult to have conflicting goals.
  7 2012-02-02 01:45:47 <midnightmagic> gmaxwell: I'm not sure what you mean by that.
  8 2012-02-02 01:48:39 <gmaxwell> midnightmagic: No one is trying to convert Tahoe-LAFS into a distributed currency. It's easier to use a rigorous process when you're actually all trying to accomplish mostly the same things.
  9 2012-02-02 01:52:13 <midnightmagic> gmaxwell: By that definition, all other open source projects don't apply; however, the process itself applies. Strongly test-driven process is applied in all the more successful projects I can personally observe.
 10 2012-02-02 01:55:09 <gmaxwell> midnightmagic: I suspect you misunderstood me.
 11 2012-02-02 01:55:57 <gmaxwell> midnightmagic: I wasn't special casing distributed currency. There are people trying to turn bitcoin into distributed file storage. Or trying to use it to promote alternative number systems, for example.
 12 2012-02-02 01:56:32 <NxTitle> well namecoin can already be considered distributed data storage
 13 2012-02-02 01:56:35 <NxTitle> though not much data
 14 2012-02-02 01:56:41 <gmaxwell> Sure sure.
 15 2012-02-02 01:57:17 <gmaxwell> I don't know that much about tahoe other than it generates an awful lot of noise compared to the amount of actual usage it gets.
 16 2012-02-02 01:57:41 <NxTitle> tahoe seems interesting, however is it one single network?
 17 2012-02-02 01:57:49 <NxTitle> or do you have to privately run your own network?
 18 2012-02-02 01:58:02 <gmaxwell> (I don't mean that in a negative way the communication is a good thing, but we simply don't have anyone doing that kind of communication for us)
 19 2012-02-02 01:58:06 <NxTitle> it looks very similar to freenet
 20 2012-02-02 01:59:15 <gmaxwell> NxTitle: it's more like NFS (or AFS or CODA) but with better security and robustness properties.
 21 2012-02-02 01:59:22 <luke-jr> gmaxwell: I think you have my position slightly incorrect ;)
 22 2012-02-02 02:00:15 <NxTitle> would people be pissed if I just put "yes" for every box in the table for everyone? :P
 23 2012-02-02 02:00:18 <gmaxwell> luke-jr: hah you've said before that the _only_ reason you used bitcoin was to promote Tonal. I suspect thats an exaggeration, but not a complete one. :)
 24 2012-02-02 02:00:19 <luke-jr> gmaxwell: I don't see Bitcoin as a vehicle to promote Tonal, but as a method of providing Tonal with a currency.
 25 2012-02-02 02:00:25 <gmaxwell> oh!
 26 2012-02-02 02:00:32 <NxTitle> what's Tonal?
 27 2012-02-02 02:00:35 <gmaxwell> I'd missed that distinction.
 28 2012-02-02 02:01:14 <gmaxwell> NxTitle: Tonal is a complete alternative to decimal.
 29 2012-02-02 02:01:23 <NxTitle> ohh right
 30 2012-02-02 02:01:26 <NxTitle> that thing
 31 2012-02-02 02:05:22 <splatster> etotheipi_: I am going to try using Joric's build procedure and see if it will work on os x 10.7.2
 32 2012-02-02 02:09:54 <etotheipi_> splatster, great, please do
 33 2012-02-02 02:10:49 <splatster> 10.7 has a different UI than 10.6 so who know what will happen
 34 2012-02-02 02:11:05 <etotheipi_> splatster, also send me an address... toss you a BTC for your efforts ;)
 35 2012-02-02 02:11:35 <splatster> Send it when I launch Armory :)
 36 2012-02-02 02:11:59 <etotheipi_> sipa, you sent me a message about the deterministic wallets, but I didn't totally understand the issue
 37 2012-02-02 02:12:08 <splatster> I'll also try and bundle it up so it has the icon and it can be easily distributed.
 38 2012-02-02 02:15:12 <etotheipi_> splatster, cool
 39 2012-02-02 02:15:53 <splatster> I hope you donated to Joric because I never thought I would see those screenshots he posted
 40 2012-02-02 02:18:28 <sipa> etotheipi_: it's very minor
 41 2012-02-02 02:18:48 <etotheipi_> haha, yes I did
 42 2012-02-02 02:18:59 <etotheipi_> I was very impressed to see those screenshots
 43 2012-02-02 02:19:05 <sipa> etotheipi_: but privkey[n+1] = (H(pubkey[n]) ^ C) * privkey[n], right?
 44 2012-02-02 02:19:08 <etotheipi_> I thought it was never going to happen
 45 2012-02-02 02:19:30 <sipa> etotheipi_: someone who gets access to the wallet can determine C from two subsequent privkeys
 46 2012-02-02 02:20:31 <etotheipi_> sipa, if someone has access to two private keys has the wallet
 47 2012-02-02 02:20:40 <etotheipi_> or one of them...
 48 2012-02-02 02:20:59 <sipa> in your application, it's not a problem
 49 2012-02-02 02:21:04 <etotheipi_> I mean, I think the scenario you are talking about is already game over
 50 2012-02-02 02:21:09 <sipa> because that always means they can just C itself already
 51 2012-02-02 02:21:44 <splatster> etotheipi_: I ran into some errors.  Have those patches been pushed to master yet?
 52 2012-02-02 02:21:56 <sipa> but if you want to make your wallet indet again, that is probably because you want it to diverge in the future, so that it cannot be stolen forever without you knowing, right?
 53 2012-02-02 02:22:17 <etotheipi_> sipa, what does indet mean?
 54 2012-02-02 02:22:21 <sipa> indeterminstic
 55 2012-02-02 02:22:41 <etotheipi_> I don't have non-deterministic wallets, so I never thoguht about that
 56 2012-02-02 02:23:05 <sipa> basically, if the scheme allows to find C from two privkeys, you are not able to completely remove the determinstic sequence from it
 57 2012-02-02 02:23:20 <gmaxwell> sipa: also surprising consequences from dumpprivkey
 58 2012-02-02 02:24:00 <sipa> it's a very minor issue indeed, but i think i'm going to switch to a scheme where C is inside the hash function
 59 2012-02-02 02:24:15 <roconnor> gmaxwell: hey is there any sane way of deploying an improvement in the OP_CHECKMULTISIG count from 20 to something resonable in case it is statically determinable what the number of operations will be?
 60 2012-02-02 02:24:31 <etotheipi_> sipa, next time I upgrade my wallet version, I will consider your recommendation
 61 2012-02-02 02:24:32 <sipa> roconnor: BIP16 does?
 62 2012-02-02 02:24:48 <roconnor> sipa: only inside the internal script.
 63 2012-02-02 02:25:03 <sipa> right, of course
 64 2012-02-02 02:25:14 <sipa> as long as the opcode is OP_CHECKMULTISIG, there is no way around that
 65 2012-02-02 02:25:17 <luke-jr> roconnor: yes and no
 66 2012-02-02 02:25:19 <splatster> etotheipi_: "libtool: for architecture: x86_64 file: algebra.o has no symbols" and many other messages like that
 67 2012-02-02 02:25:36 <roconnor> sipa: huh?
 68 2012-02-02 02:25:39 <luke-jr> roconnor: BIP 17's solution is to avoid that op, and use your script :p
 69 2012-02-02 02:25:51 <etotheipi_> sipa, or rather, let me know what you do to modify it and I'll do it too... there's no reason not to change it, besides the transient writing a version-conversion script
 70 2012-02-02 02:25:59 <roconnor> luke-jr: ya, but it would be good if bitcoin didn't suck so much.
 71 2012-02-02 02:26:02 <etotheipi_> it's a bit easier for you
 72 2012-02-02 02:26:37 <sipa> etotheipi_: in particular, i'm thinking about privkey[n] = SHA256(C | SHA512(C | type | n)) * rootkey
 73 2012-02-02 02:26:51 <luke-jr> roconnor: there's a long list of things to change when we fork the blockchain ;)
 74 2012-02-02 02:27:34 <sipa> gmaxwell: hmm?
 75 2012-02-02 02:27:39 <etotheipi_> sipa, seems like overkill.... (what is type, and n?)
 76 2012-02-02 02:27:49 <sipa> n is from privkey[n]
 77 2012-02-02 02:28:12 <etotheipi_> splatster, I don't even know what libtool is
 78 2012-02-02 02:28:12 <sipa> and if you're going to do two hashes anyway, better do it HMAC-style and prevent whatever the designers of HMAC tried to prevent
 79 2012-02-02 02:28:34 <etotheipi_> sipa, fair enough
 80 2012-02-02 02:28:42 <splatster> I think I need to wipe my system
 81 2012-02-02 02:29:08 <etotheipi_> sipa, is the public key part of that equation?
 82 2012-02-02 02:29:12 <sipa> no
 83 2012-02-02 02:29:43 <sipa> that's a possible alternative of course, use pubkey[n] instead of n inside the hash
 84 2012-02-02 02:31:16 <etotheipi_> ahh... I actually like that better (just n) because it means you can calculate the private keys really fast
 85 2012-02-02 02:31:32 <sipa> indeed, random-access and parallellizable
 86 2012-02-02 02:32:36 <sipa> it's just gmaxwell's original scheme really
 87 2012-02-02 02:32:38 <sipa> but with a double hash
 88 2012-02-02 02:32:49 <etotheipi_> ahh... I never actually read gmaxwell's scheme
 89 2012-02-02 02:33:38 <etotheipi_> wait, what is "type" again?
 90 2012-02-02 02:33:44 <sipa> he also pointed me to why a type is useful: to have several chains inside the same wallet
 91 2012-02-02 02:33:57 <sipa> e.g. all publically usable addresses, and all change adresses
 92 2012-02-02 02:34:15 <etotheipi_> why do you need to separate change addresses?
 93 2012-02-02 02:34:23 <sipa> so you can create a watch-only wallet for the public address, but not let someone see what you do after receiving coins
 94 2012-02-02 02:34:30 <sipa> you know what a change address is?
 95 2012-02-02 02:35:11 <etotheipi_> I don't understand (and I assume you're just talking about a new address for receiving change outputs...?)
 96 2012-02-02 02:35:18 <sipa> yes
 97 2012-02-02 02:35:32 <sipa> see, i create a wallet for my shop
 98 2012-02-02 02:35:47 <sipa> it has two chains in it; C1 and C2
 99 2012-02-02 02:35:59 <sipa> (with type=1 and type=2)
100 2012-02-02 02:36:24 <sipa> wait...
101 2012-02-02 02:36:50 <sipa> gmaxwell: they need a separate rootkey and/or C as well, no?
102 2012-02-02 02:37:56 <sipa> right, of course
103 2012-02-02 02:37:58 <gmaxwell> Indeed, could be accomplished with seperate C just as well as the type.
104 2012-02-02 02:38:12 <gmaxwell> (er, not just as well, but wrt the watch case, thats required)
105 2012-02-02 02:38:58 <gmaxwell> I think when I wrote up the original scheme I hadn't yet thought of hiding the change from the website... only allowing it to avoid using those addresses.
106 2012-02-02 02:39:38 <etotheipi_> I still have no idea what's going on
107 2012-02-02 02:39:46 <etotheipi_> why do you need a separate chain for addresses?
108 2012-02-02 02:39:57 <etotheipi_> *for change addresses?
109 2012-02-02 02:39:57 <sipa> you have a webshop
110 2012-02-02 02:40:09 <sipa> you want to be able to show a partner your income
111 2012-02-02 02:40:41 <sipa> so you give them a watch-only wallet with the chain you use to generate the payment addresses your shop gives to clients
112 2012-02-02 02:40:56 <sipa> all other addresses come from another chain
113 2012-02-02 02:41:26 <sipa> if someone has access to that other chain (even just the pubkeys) they now what you're doing with the money
114 2012-02-02 02:41:32 <sipa> now
115 2012-02-02 02:42:06 <sipa> know
116 2012-02-02 02:42:17 <gmaxwell> Or even, forget the partner: You put the 'watch only' wallet on the webshop itself so it can generate new addresses for customers safely. But you don't want someone who has compromised the shop site to learn more than the absolute minimum about where those funds went after you got them.
117 2012-02-02 02:42:52 <sipa> oh nice one; that was the use case you had in mind when writing the proposal, i assume?
118 2012-02-02 02:43:17 <gmaxwell> sipa: Yes.
119 2012-02-02 02:43:59 <gmaxwell> That was my fundimental motivation behind type-2. Because the FSF asked me how they could give private donation addresses to people without putting their wallet online.
120 2012-02-02 02:44:46 <roconnor> luke-jr: I guess if the miners maintain the old MUTLISIG check until the fixed clients are well deployed, it could be deployed that way perhaps.
121 2012-02-02 02:45:51 <gmaxwell> The other reason, which may not apply to armory.. is that if the wallet is destroyed and recovered we very much want to know which transactions were automatic change and which were direct payments, even if you happen to sometimes be paying funds to your own addresses.
122 2012-02-02 02:46:10 <etotheipi_> gmaxwell, on that last point, I already have that covered
123 2012-02-02 02:46:22 <gmaxwell> etotheipi_: How so?
124 2012-02-02 02:46:30 <etotheipi_> if you use armory, you'll notice that I identify change addresses reliably, even on sent-to-self
125 2012-02-02 02:46:41 <sipa> how so?
126 2012-02-02 02:46:46 <etotheipi_> because the change address is always the one with the higher index
127 2012-02-02 02:46:56 <gmaxwell> erp.
128 2012-02-02 02:47:13 <sipa> that makes it visible to someone with a watch-only wallet
129 2012-02-02 02:47:16 <etotheipi_> arguably, you have to have the transaction history to determine that... but you should have that anyway
130 2012-02-02 02:47:22 <gmaxwell> oh you mean in the send to self case.. ok gotcha.
131 2012-02-02 02:47:26 <sipa> (which is something that you sometimes want)
132 2012-02-02 02:47:46 <etotheipi_> I forget how, but I was able to always determine the change address on every tx in Armory
133 2012-02-02 02:47:50 <etotheipi_> as long as it involved my wallet
134 2012-02-02 02:48:29 <etotheipi_> so here's my thought... everything you just said can be achieved with one upgrade to what I do in Armory:  a setting on the wallet that allows me to specify a different wallet for change outputs
135 2012-02-02 02:48:48 <etotheipi_> when you talk about different chains, you might as well just be talking about different wallets
136 2012-02-02 02:48:54 <gmaxwell> etotheipi_: you may not have the transaction history e.g. if you've had multiple wallets/chains and removed some though. (e.g. deleteprivatekey). But it's true, you can figure it out from the history if you have it and smart index handling in the send to self case.
137 2012-02-02 02:49:35 <etotheipi_> gmaxwell, everytime an address is requested, it gets the next unused index.... the same for th change addresses
138 2012-02-02 02:50:12 <etotheipi_> did I miss something?  I feel like it's not hard to meet that criteria
139 2012-02-02 02:50:53 <gmaxwell> I'm not sure what you're asking.
140 2012-02-02 02:50:55 <sipa> etotheipi_: do you understand the use case of hiding change addresses from a watch-only wallet?
141 2012-02-02 02:51:02 <etotheipi_> (sorry, I didn't mean that in an arrogant way... I"m just trying to figure out if my naive implementation fails to solve this problem in some cases?)
142 2012-02-02 02:51:07 <etotheipi_> sipa, yes
143 2012-02-02 02:51:23 <gmaxwell> Your naive (two wallet) thing sounds like solves it, yes.
144 2012-02-02 02:51:30 <etotheipi_> sipa, I see exactly what you guys are saying, and I recognize the issue
145 2012-02-02 02:51:34 <sipa> if the change address is always the one with the higher index in the chain, it is visible to everyone with a watch-only wallet
146 2012-02-02 02:51:51 <etotheipi_> sipa, and that's why I was saying, I could add a setting to send all change outputs to a different wallet
147 2012-02-02 02:52:00 <gmaxwell> The other case where change seperation matters wrt the webshop case is you very much don't want to have the webshop and the a send use the same address due to unlucky timing.
148 2012-02-02 02:52:13 <gmaxwell> But, likewise, thats solve with 'two wallets' case.
149 2012-02-02 02:52:24 <sipa> right, two wallets would do it
150 2012-02-02 02:52:39 <sipa> but the logic for presenting it in a ledger-style becomes somewhat more complicated then
151 2012-02-02 02:52:40 <etotheipi_> I was envisioning that in a boss-employee system, the boss would generate one wallet for each register, and distribute the watching-only copies
152 2012-02-02 02:53:04 <etotheipi_> sipa, agreed
153 2012-02-02 02:53:16 <gmaxwell> etotheipi_: yea, but then boss sends funds from a register and uses the address for the next customer in line as the change.. confusion happens.
154 2012-02-02 02:53:38 <sipa> so i believe just having two chain codes, one for "public" addresses and one for "internal" addresses per wallet is exactly the solution
155 2012-02-02 02:53:49 <etotheipi_> interesting discussion
156 2012-02-02 02:54:03 <etotheipi_> I'll think about how I would implement this
157 2012-02-02 02:54:46 <etotheipi_> I don't fully appreciate (yet) the complications of moving from my current system to this...
158 2012-02-02 02:54:49 <sipa> you don't lose anything by splitting it up, except 36 bytes (extra chain code, and extra counter)
159 2012-02-02 02:55:00 <sipa> even if not used
160 2012-02-02 02:55:14 <gmaxwell> little extra computation to precalculate for the watching lookahead.
161 2012-02-02 02:55:54 <etotheipi_> doesn't this cause issues for balance/auditing by someone with a watching-only wallet?
162 2012-02-02 02:56:06 <etotheipi_> it seems that every outgoing transaction will look bigger than it actually is
163 2012-02-02 02:56:08 <gmaxwell> etotheipi_: give them a deluxe watching-only wallet.
164 2012-02-02 02:56:20 <gmaxwell> (one that has both chain codes)
165 2012-02-02 02:56:46 <gmaxwell> Both lets you see the real ledger. One lets you generate future addresses but only see incoming funds.
166 2012-02-02 02:57:00 <sipa> a watching-only wallet with only the public chain, and the same ledger-view logic would just see every outgoing transaction disappear fully as a send-to-somewhere
167 2012-02-02 02:57:27 <etotheipi_> right
168 2012-02-02 02:57:48 <etotheipi_> if I'm giving that wallet to them for the purpose of not letting them see what I do with the change, they also can't see how much change was made
169 2012-02-02 02:58:00 <gmaxwell> They can see the income.
170 2012-02-02 02:58:01 <sipa> exactly
171 2012-02-02 02:58:09 <etotheipi_> so the actual value of each transaction is obfuscated (only outgoing transactions)
172 2012-02-02 02:58:12 <sipa> as they can't distinguish real outputs and change outputs
173 2012-02-02 02:58:26 <etotheipi_> I'm seeing that as a problem
174 2012-02-02 02:58:32 <gmaxwell> Then don't do that.
175 2012-02-02 02:58:39 <gmaxwell> Let them see it all, if it's a problem.
176 2012-02-02 02:58:40 <etotheipi_> :)
177 2012-02-02 02:58:50 <gmaxwell> There are usecases where it is and usecases where it isn't.
178 2012-02-02 02:58:56 <sipa> you can have them see it all, if you like - just give both chains
179 2012-02-02 02:58:58 <etotheipi_> fair enough... I just wanted to make sure I understand
180 2012-02-02 02:59:04 <sipa> i like this
181 2012-02-02 03:02:46 <etotheipi_> gmaxwell, how does that happen?
182 2012-02-02 03:02:46 <gmaxwell> sipa: we should also have the ability to flag a wallet so that it only ever pulls addresses from the internal side.  So you don't accidentaly hit getnewaddress and get some address a customer is about to pay to.
183 2012-02-02 03:02:46 <sipa> what about an optional arg to getnewaddress?
184 2012-02-02 03:02:51 <gmaxwell> webshop use case.
185 2012-02-02 03:03:27 <gmaxwell> The webshop has the minimal watching wallet to generate payment addresses for customers. You have a full client with the full wallet.
186 2012-02-02 03:04:16 <gmaxwell> If your full wallet software is unaware that there is a remote address generator, it might.
187 2012-02-02 03:04:50 <etotheipi_> gmaxwell, I don't see why two people would ever use the same wallet at the same time... if I give one wallet to one employee to use, I'll generate a different wallet for myself to use
188 2012-02-02 03:04:53 <sipa> sounds like you want the ability to have N chains in a wallet
189 2012-02-02 03:05:14 <sipa> and we're back to maybe rather wanting separate wallets
190 2012-02-02 03:05:29 <gmaxwell> sipa: maybe.. or as etotheipi_ says.. more wallets. But I want to be able to spend from a collection at once.. not keep them seperate.
191 2012-02-02 03:05:37 <gmaxwell> I think seperate wallets should never comingle inputs.
192 2012-02-02 03:05:47 <gmaxwell> and I think this case it's okay to comingle inputs.
193 2012-02-02 03:05:55 <etotheipi_> well whether it's separate chains or separate wallets is immaterial... one can be made to look like the other, depending on how the interface is designed
194 2012-02-02 03:06:22 <sipa> gmaxwell: so, one inner chain, one public chain, and N alternate chains
195 2012-02-02 03:06:36 <sipa> from the alt chains, no address is ever automatically pulled
196 2012-02-02 03:06:37 <etotheipi_> I just imagine that as the business owner, you would be very interested to deconflict the financial activity of each person... no two people should ever generate from the same wallet
197 2012-02-02 03:06:43 <sipa> but you can give it to webshops
198 2012-02-02 03:07:26 <gmaxwell> sipa: you could also give alternate chains to business partners who make regular payments to you.
199 2012-02-02 03:07:59 <etotheipi_> I've had in mind, all along... that you could use watching-only wallets to simply maintain channels between people who exchange money frequently
200 2012-02-02 03:08:21 <etotheipi_> that I would have one watching-only wallet exchange with my mother, another one with my girlfriend, etc
201 2012-02-02 03:08:37 <poiuh> and your wife!
202 2012-02-02 03:08:38 <etotheipi_> and the balance behind those wallets would not be important, only seeing what transactions had been made
203 2012-02-02 03:08:40 <poiuh> baddum pum
204 2012-02-02 03:09:07 <etotheipi_> (errr... the balance would really not be important, only the ledger)
205 2012-02-02 03:09:22 <poiuh> cool
206 2012-02-02 03:09:32 <gmaxwell> "generate mailslot"
207 2012-02-02 03:09:53 <etotheipi_> I was actually ready to promote that idea with Armory, but it was kind of weird to have such a wallet on the main display with some arbitrary balance... I think I want a separate interface (or info stream) for that
208 2012-02-02 03:11:38 <gmaxwell> well.. I think you don't want it as a seperate wallet. .. it's just part of one wallet .. and you can assign a default label to all those addresses.. on the remote side it's an address book entry, which autoincrements.
209 2012-02-02 03:12:36 <etotheipi_> as far as I'm concerned, we're talking about the same thing... you use a one root key but a different chaincode for each "channel", I use different both for each channel
210 2012-02-02 03:13:13 <gmaxwell> I think what sipa is thinking of is using a metachain to come up with both for all.
211 2012-02-02 03:13:17 <etotheipi_> an interface could be designed that makes both schemes look identical, except for having to backup a 32 more bytes for each "channel"
212 2012-02-02 03:13:24 <gmaxwell> This way adding channels doesn't bugger your full backups.
213 2012-02-02 03:13:57 <poiuh> nice
214 2012-02-02 03:14:01 <etotheipi_> I don't understand... whne you create a new chain, you're going to have to backup that chaincode
215 2012-02-02 03:14:08 <gmaxwell> Nope.
216 2012-02-02 03:14:18 <gmaxwell> You come up with the chaincodes/root keys determinstically.
217 2012-02-02 03:14:27 <sipa> haha!
218 2012-02-02 03:14:46 <gmaxwell> H(big secret+chain_id) = {root,chaincode}
219 2012-02-02 03:14:57 <sipa> if you want your alt chains to automatically see incoming payments, you probably want something pretty much like the current keypool
220 2012-02-02 03:15:09 <sipa> but a keypool per alt chain, but possibly much smaller
221 2012-02-02 03:15:16 <gmaxwell> Yes.
222 2012-02-02 03:15:18 <etotheipi_> so multidimensional determinism
223 2012-02-02 03:15:22 <etotheipi_> determinism^2
224 2012-02-02 03:16:08 <gmaxwell> sipa: The reason you want seperate root&chaincodes for each chain is so that if you start giving away chanins they aren't obviously linked on their face. It's not like its computationally costly or anything.
225 2012-02-02 03:16:32 <etotheipi_> I guess the only concern is that now *all* your wallets are compromised at once
226 2012-02-02 03:16:54 <etotheipi_> but again, same debate as random-vs-deterministic addresses
227 2012-02-02 03:17:05 <sipa> if not, you'll have several wallets on the master computer
228 2012-02-02 03:17:14 <gmaxwell> etotheipi_: this doesn't replace multiple wallets but for true multiple wallets, I think we'd want to provide complete isolation.. e.g. no ability to use inputs in common though the normal UI.
229 2012-02-02 03:17:37 <sipa> this simplifies something else
230 2012-02-02 03:17:48 <sipa> the entire select-coins-for-more-anonimity issue
231 2012-02-02 03:18:04 <sipa> an advanced client interface could let you select which chains to use inputs from
232 2012-02-02 03:18:23 <sipa> far more manageable than showing the user the individual addresses
233 2012-02-02 03:19:49 <gmaxwell> I'm not opposed to that kind of feature but I think for that purpose hard seperated multiple wallets are better.  Though it should result in nice behavior for auto-selection.
234 2012-02-02 03:20:05 <gmaxwell> e.g. all things equal auto selection should prefer to pull from one chain.
235 2012-02-02 03:20:16 <etotheipi_> it sounds like there's applications for both
236 2012-02-02 03:20:20 <gmaxwell> There are.
237 2012-02-02 03:20:35 <etotheipi_> the wallets should have the ability to create multiple chains, but there's still situations you'll be using different wallets
238 2012-02-02 03:20:57 <sipa> of course
239 2012-02-02 03:21:07 <sipa> your personal wallet vs your business wallet, i can imagine
240 2012-02-02 03:21:19 <gmaxwell> I would think of chains more like bitcoinds accounts.. while seperate wallets are really seperate.
241 2012-02-02 03:21:32 <sipa> bingo
242 2012-02-02 03:22:05 <sipa> what about, with the root root key, you can generate new chains
243 2012-02-02 03:22:11 <sipa> but they are really just accounts
244 2012-02-02 03:22:27 <sipa> and their root key / chain code, is derived from the root root key + name of the account
245 2012-02-02 03:22:37 <gmaxwell> No, sadly.. :(
246 2012-02-02 03:22:50 <gmaxwell> Because if you delete your wallet .. "oh shit, what was that account called..."
247 2012-02-02 03:23:08 <sipa> if you delete your wallet, you're screwed...
248 2012-02-02 03:23:23 <gmaxwell> you delete it, recover from a backup before you created the account
249 2012-02-02 03:23:39 <gmaxwell> what should happen then is you should see account "8" appear... with txn in it...
250 2012-02-02 03:24:15 <gmaxwell> and then you can rename it to whatever the right thing is.. though I guess it breaks down a bit if you did inter account transfers.. alas.. those are lost.
251 2012-02-02 03:24:25 <gmaxwell> (backup backup backup)
252 2012-02-02 03:24:53 <gmaxwell> but still .. much better to have the ledgers screwed up than to have actually lost the money in those cases.
253 2012-02-02 03:25:13 <gmaxwell> And there is no way to solve the interior transfer dataloss except via frequent backups.
254 2012-02-02 03:25:24 <gmaxwell> (AND THIS IS ANOTHER REASON WE NEED THE OOPSWALLET! :) )
255 2012-02-02 03:26:44 <luke-jr> roconnor: that's a good idea
256 2012-02-02 03:27:55 <sipa> ok, so new accounts draw from the pool of chains
257 2012-02-02 03:28:31 <gmaxwell> That also suggests that each chain should have public/internal sides.
258 2012-02-02 03:30:06 <sipa> i was more thinking about one special chain for internal addresses
259 2012-02-02 03:30:23 <sipa> but i guess for anonimity purposes, you better keep them entirely separate
260 2012-02-02 03:30:29 <gmaxwell> easier on the keypool. Downside is that there will be more account balance damage if you lose data.
261 2012-02-02 03:30:56 <gmaxwell> also if I want to give a complete-watcher access to some account (public+internal), they need to be seperate.
262 2012-02-02 03:32:38 <gmaxwell> I dub this system, CoinCube(*): nature's harmonic simultaneous multidimensional wallet    (* http://www.timecube.com/)
263 2012-02-02 03:32:50 <luke-jr> roconnor: in fact, since 1000 multisig per block isn't even a problem yet, it's possible that preemptively fixing the client rules now could result in a very graceful transition without resorting to manual multisig scripts
264 2012-02-02 03:32:56 <splatster> etotheipi_: I don't know what I should do.  I'm going to wait until the weekend so I can wipe my drive and start fresh and then hopefully I can get it to run.
265 2012-02-02 03:33:31 <etotheipi_> splatster, maybe you can beg Joric to give you a compiled binary :)
266 2012-02-02 03:33:39 <gmaxwell> luke-jr: we could also just put a fix that takes effect at height $BIG. ... and tell people to upgrade before that height.
267 2012-02-02 03:34:16 <splatster> Joric-o, Joric-o, where art thou?
268 2012-02-02 03:34:21 <gmaxwell> luke-jr: e.g. put it a year out, and use an alert at 11 months and the 12 months minus 1 day targeting clients before the new rule was added.
269 2012-02-02 03:34:26 <luke-jr> gmaxwell: sure
270 2012-02-02 03:34:37 <gmaxwell> we could also fix timewarp this way.
271 2012-02-02 03:34:42 <splatster> Anyone like my reference?
272 2012-02-02 03:35:06 <gmaxwell> I'd like to say we could reenable opcodes that way.. but no one asking for them is going to do the work to make them trustworthy.
273 2012-02-02 03:35:11 <luke-jr> gmaxwell: we could even do hardfork this way potentially
274 2012-02-02 03:36:01 <gmaxwell> luke-jr: yes, but it gets exponentiall harder the more you do  who would oppose fixing sigops count? no one. Who would oppose something more complicated? well look at bip12/16/17
275 2012-02-02 03:36:11 <gmaxwell> er exponentially
276 2012-02-02 03:37:12 <gmaxwell> we also can't put in the rule change until the software is done and proven since those clients will need to survive the change too.. e.g. we can't put in a rule to enable OP_CAT in a year now because OP_CAT safty needs to be validated first. :(
277 2012-02-02 03:37:26 <gmaxwell> at least things like sigops and timewarp are easy to get confidence about.
278 2012-02-02 03:38:45 <splatster> etotheipi_: Here's a pic if you're wondering what a bundled Armory would look like in the dock: http://dl.dropbox.com/u/3533940/Screen%20Shot%202012-02-01%20at%209.35.45%20PM.png
279 2012-02-02 03:39:28 <etotheipi_> splatster, perfect... that was exactly what I wanted when I got the logo designed
280 2012-02-02 03:40:00 <etotheipi_> it's not just another circular or square icon
281 2012-02-02 03:40:02 <splatster> Looks great with the shadow and reflection effects
282 2012-02-02 03:40:30 <luke-jr> gmaxwell: fair enough
283 2012-02-02 03:41:09 <luke-jr> phantomcircuit: O.o
284 2012-02-02 03:41:41 <luke-jr> phantomcircuit: how can you be against all P2SH and also against no-P2SH? -.-
285 2012-02-02 03:41:52 <gmaxwell> That node I started earlier today on luke's vps? "connections" : 67,
286 2012-02-02 03:41:58 <luke-jr> gmaxwell: lol
287 2012-02-02 03:42:15 <luke-jr> isn't that the exact same number you mentioned your IRC node was at?
288 2012-02-02 03:42:20 <gmaxwell> Why is there a BIP22 column there?
289 2012-02-02 03:42:24 <phantomcircuit> luke-jr, both implementations could potentially split the blockchain between new clients and old clients
290 2012-02-02 03:42:27 <phantomcircuit> i dont like that
291 2012-02-02 03:42:39 <luke-jr> phantomcircuit: no they can't& O.o
292 2012-02-02 03:42:43 <gmaxwell> phantomcircuit: no they can't.
293 2012-02-02 03:42:45 <phantomcircuit> i like the basic p2sh idea, but not the current implementations (im not sure it's even possible)
294 2012-02-02 03:42:52 <gmaxwell> Okay, phantomcircuit just disqualified himself.
295 2012-02-02 03:42:56 <luke-jr> phantomcircuit: you're not allowed to vote until reading them :P
296 2012-02-02 03:43:04 <phantomcircuit> i did read them
297 2012-02-02 03:43:07 <luke-jr> gmaxwell: casa added it; I'm ignoring it :P
298 2012-02-02 03:43:25 <phantomcircuit> you could easily create a script which is valid against old clients and invalid against new clients
299 2012-02-02 03:43:39 <luke-jr> phantomcircuit: you missed the majority of miners bit
300 2012-02-02 03:43:42 <sipa> phantomcircuit: that is exactly the intention
301 2012-02-02 03:43:51 <luke-jr> phantomcircuit: those majority of miners will orphan any old miners
302 2012-02-02 03:43:58 <phantomcircuit> luke-jr, the majority of miners makes no difference
303 2012-02-02 03:43:59 <gmaxwell> (if it didn't do that it wouldn't be a change at all :) )
304 2012-02-02 03:44:03 <sipa> every p2sh script will be valid to old clients
305 2012-02-02 03:44:08 <luke-jr> phantomcircuit: the old clients will honour the orphaning
306 2012-02-02 03:44:13 <phantomcircuit> all it takes is enough miners that the old clients dont notice
307 2012-02-02 03:44:23 <gmaxwell> phantomcircuit: no, they'll honor the longer chain, doofus.
308 2012-02-02 03:44:43 <luke-jr> phantomcircuit: if the majority of miners implement the BIP (required for activation), then the BIP will always orphan the pre-BIP
309 2012-02-02 03:44:53 <gmaxwell> phantomcircuit: what you're thinking of wet "enough miners that the old clients dont notice" is what happens if the txn in the new client chain is invalid to old clients.
310 2012-02-02 03:45:05 <gmaxwell> s/wet/wrt/
311 2012-02-02 03:45:05 <sipa> now re-read what i last said
312 2012-02-02 03:45:16 <luke-jr> phantomcircuit: since miners tend to dislike having blocks orphaned, they'll all upgrade fairly fast
313 2012-02-02 03:45:20 <luke-jr> once activation
314 2012-02-02 03:45:39 <phantomcircuit> fair enough
315 2012-02-02 03:45:51 <sipa> phantomcircuit: in short: all the BIP does is make certain scripts invalid that were previously valid, and never the other way around
316 2012-02-02 03:45:58 <gmaxwell> even if they don't _all_ upgrade, it's a non-issue.
317 2012-02-02 03:45:59 <luke-jr> even if they don't, worse case scenario a bad guy gets 2 confirmations on a fake
318 2012-02-02 03:46:03 <phantomcircuit> i've been awake for like 36 hours so i probably am thinking in circles
319 2012-02-02 03:46:10 <luke-jr> >_<
320 2012-02-02 03:46:33 <luke-jr> phantomcircuit: how about sleep and then revise your votes tomorrow? :P
321 2012-02-02 03:46:44 <phantomcircuit> probably
322 2012-02-02 03:46:56 <gmaxwell> phantomcircuit: You know that some studies have shown that doing that has a severly bad impact on life expectancy? :)
323 2012-02-02 03:47:21 <phantomcircuit> gmaxwell, i literally cant help it i wont fall asleep anyways
324 2012-02-02 03:47:53 <BTC_Bear> drink 2 beers and lay down
325 2012-02-02 03:48:05 <phantomcircuit> alcohol wakes me up
326 2012-02-02 03:48:13 <phantomcircuit> right upto the point that it doesn't
327 2012-02-02 03:54:49 <gmaxwell> phantomcircuit: can you go pop off your vote until you've had a nap, lest you inadvertently contribute to confusion? (if after clear minded analysis you reach the same conclusion then great&)
328 2012-02-02 03:55:10 <phantomcircuit> sure
329 2012-02-02 03:56:06 <phantomcircuit> gmaxwell, how about that
330 2012-02-02 03:56:36 <gmaxwell> phantomcircuit: k
331 2012-02-02 03:57:04 <gmaxwell> phantomcircuit: if it helps you BIP22 _does_ have the behavior you were worried about, but I don't think it matters. It's not getting done.
332 2012-02-02 03:57:20 <gmaxwell> or rather, if its getting done, it not getting done by any of the current development team.
333 2012-02-02 03:57:49 <luke-jr> BIP 22 could become a nice solution if we didn't want static analysis
334 2012-02-02 03:57:58 <luke-jr> but if that wasn't the case, we'd use BIP 12
335 2012-02-02 03:58:13 <gmaxwell> Because we're not taking a @#$#@ hard fork just to add a redundant boolean circuit scripting language inside public keys.
336 2012-02-02 03:58:23 <luke-jr> gmaxwell: BIP 22 could be tweaked to avoid a hardfork
337 2012-02-02 03:58:30 <splatster> How is bitcoin-qt bundled for os x
338 2012-02-02 03:58:59 <splatster> I'm going to take a stab at making a blank template for Joric to pop his compiled source into
339 2012-02-02 03:59:01 <gmaxwell> luke-jr: I'm less than convinced. But regardless. come on.. a completely seperate scripting language inside our scripting language? Yo Dawg. I heard you liked scripts.
340 2012-02-02 03:59:21 <etotheipi_> gmaxwell, lol
341 2012-02-02 03:59:40 <luke-jr> gmaxwell: I just mean the concepts are interesting ;)
342 2012-02-02 03:59:45 <luke-jr> not that I want to do anything with ti
343 2012-02-02 04:00:19 <etotheipi_> http://pixelatedgeek.com/2009/02/yo-dawg-i-heard-you-like-macs/
344 2012-02-02 04:00:59 <splatster> etotheipi_: HAHAH
345 2012-02-02 04:01:04 <sipa> gmaxwell: it's not such a bad idea i think
346 2012-02-02 04:01:23 <sipa> but not right now
347 2012-02-02 04:01:26 <sipa> not in a hurry
348 2012-02-02 04:01:33 <gmaxwell> I admit I'm mostly turned off both by the fact that its a hardfork maker, and because it's proposer doesn't think this is a problem.
349 2012-02-02 04:01:43 <luke-jr> if we really needed to replace the language, OP_EVAL or BIP 22 would be a handy way to do it :p
350 2012-02-02 04:01:57 <luke-jr> but that's not an issue afaik
351 2012-02-02 04:01:58 <sipa> but a well-redesigned scripting language that works as an expression evaluator instead of a stack language - i'd very much like that
352 2012-02-02 04:02:26 <gmaxwell> Even with luke suggesting that it can be fixed wrt the forking, I'm not inclined to waste my time thinking about proposals from anyone who would take a hardfork lightly.
353 2012-02-02 04:02:37 <sipa> haha
354 2012-02-02 04:02:45 <luke-jr> gmaxwell: he did say he wasn't seriously proposing it
355 2012-02-02 04:02:59 <lianj> with isstandard templates, i have hard times to call the current scripts a scripting language anymore
356 2012-02-02 04:03:29 <gmaxwell> lianj: er. you know that isstandard is soft security right? You can still have non-isstandard txn in the blockchain.
357 2012-02-02 04:03:32 <gmaxwell> (and we do)
358 2012-02-02 04:03:46 <lianj> but only miners right?
359 2012-02-02 04:04:03 <sipa> you need a miner to accept it
360 2012-02-02 04:04:03 <splatster> etotheipi_: Look what I found: http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html
361 2012-02-02 04:04:42 <gmaxwell> lianj: Only with the cooperation of miners, which inhibits nonstandard scripts from being used for DOS attacks without completely closing expirementation, even in the mainnet, and without creating a big compatbility issue when more is enabled.
362 2012-02-02 04:05:19 <luke-jr> lianj: Eligius tolerates anything
363 2012-02-02 04:05:28 <lianj> luke-jr: so would your nodes accept any valid script i sent them?
364 2012-02-02 04:05:36 <lianj> ah, nice. thanks!
365 2012-02-02 04:05:39 <luke-jr> lianj: so long as it doesn't have a ton of sigops
366 2012-02-02 04:05:53 <gmaxwell> sipa: just need a simple language that lets you describe a directed graphs of CCNOT gates of course.
367 2012-02-02 04:05:54 <lianj> hehe ok, but good to know
368 2012-02-02 04:06:02 <etotheipi_> splatster, I knew you'd like that :)
369 2012-02-02 04:06:22 <gmaxwell> luke-jr: you do have some anti-dos checks too.
370 2012-02-02 04:06:36 <luke-jr> gmaxwell: I do?
371 2012-02-02 04:06:49 <luke-jr> lianj: note I require fees
372 2012-02-02 04:07:00 <lianj> aw :(
373 2012-02-02 04:07:06 <luke-jr> lianj: they're not too expensive
374 2012-02-02 04:07:15 <luke-jr> 0.00004096 BTC per 512 bytes
375 2012-02-02 04:07:25 <gmaxwell> lianj: if you're not willing to spend a fraction of a cent on your transaction 0_o
376 2012-02-02 04:07:37 <gmaxwell> lianj: then I say go away, spammer.
377 2012-02-02 04:07:42 <lianj> gmaxwell: hehe
378 2012-02-02 04:08:13 <lianj> (i'm not, just for the record)
379 2012-02-02 04:08:19 <luke-jr> &
380 2012-02-02 04:09:11 <cjd> what exactly is the benefit of isStandard() check when not all miners run it?
381 2012-02-02 04:09:11 <gmaxwell> lianj: but, really, if you're interested in doing something interesting.. talk in here.. not only will eligius probably mine the transaction, but I'll also mine non-spammy interesting stuff, if you don't mind waiting .. a week or so.. :)
382 2012-02-02 04:09:28 <splatster> etotheipi_: http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html this will make creating an app bundle simple, just did it actually! (though my Armory isn't properly compiled)
383 2012-02-02 04:09:47 <gmaxwell> cjd: because it still is quite effective at ratelimiting dosattacky things. And luke has also replaced it with mandatory fees.
384 2012-02-02 04:09:59 <splatster> Maybe you can pass it off to Joric if I'm not here
385 2012-02-02 04:10:15 <gmaxwell> cjd: and luke is the only non-trivial miner we're aware of that doesn't apply non-standard (based on looking at the blockchain)
386 2012-02-02 04:10:18 <cjd> mm I agree with (isStandard() || higherFee)
387 2012-02-02 04:10:31 <luke-jr> well, if it's really interesting, I'll accept it free too :p
388 2012-02-02 04:11:00 <luke-jr> but then again& cheap is cheap :P
389 2012-02-02 04:11:04 <gmaxwell> cjd: it's hard to reason about that though say someone finds out some script really slows down nodes.. they could easly pay the higher fee to make trouble. At least if it's only luke mining them there won't be many such blocks produced.
390 2012-02-02 04:11:48 <cjd> indeed, and a good argument for miners being awake and alert since these situations require human intervention
391 2012-02-02 04:12:02 <lianj> gmaxwell: thanks, not atm though. but i like that such transactions are not frozen or only available to tinker with by big miners
392 2012-02-02 04:12:15 <gmaxwell> cjd: luke is awake and alert .. less evidence of that with other miners.
393 2012-02-02 04:12:24 <cjd> ofc the hippy in me says "let the scripting do as it wants"
394 2012-02-02 04:12:38 <gmaxwell> lianj: they're available to everyone liberlly on testnet... and available to everyone with a little coperating like coming and asking here.
395 2012-02-02 04:13:39 <gmaxwell> cjd: we'll get back ot that long term, of course. If you want to see that day come faster... help contribute to software QA, build neat things that use the scripts, etc.
396 2012-02-02 04:14:26 <luke-jr> if BIP 17 never amounts to anything more, at least I learned that
397 2012-02-02 04:14:40 <cjd> are we satisfied that there won't be any big surprises w/ the scripting turning into a weird machine?
398 2012-02-02 04:14:51 <luke-jr> cjd: BIP 12 is Skynet
399 2012-02-02 04:15:42 <cjd> heh
400 2012-02-02 04:16:22 <cjd> I wish it was easier for me to understand
401 2012-02-02 04:16:23 <luke-jr> hey, found the bug lurking in Eloipool's bitcoin node implementation
402 2012-02-02 04:16:43 <luke-jr> I was appedning the checksum to the payload, instead of prepending
403 2012-02-02 04:16:45 <luke-jr> <.<
404 2012-02-02 04:16:52 <cjd> C++ abstracts away what's really going on such that one simple line is very complex and nuanced (in my experience)
405 2012-02-02 04:17:21 <luke-jr> cjd: so does C ;)
406 2012-02-02 04:17:34 <sipa> far less so
407 2012-02-02 04:17:43 <sipa> imho
408 2012-02-02 04:17:48 <cjd> +1
409 2012-02-02 04:17:56 <luke-jr> only because C++ has more in its stdlib
410 2012-02-02 04:18:08 <luke-jr> IMO
411 2012-02-02 04:18:11 <sipa> no, because you can overload the assignment operator :P
412 2012-02-02 04:18:13 <gmaxwell> luke-jr: very far less so in C.
413 2012-02-02 04:18:18 <luke-jr> sipa: true
414 2012-02-02 04:18:48 <gmaxwell> You can overload the == operator with something that has side effects. And people do.
415 2012-02-02 04:18:53 <luke-jr> gmaxwell: well, I guess x86 has multiple types, but from my MIPS background, it's funny how everything is really just a 32-bit int
416 2012-02-02 04:18:54 <cjd> :(
417 2012-02-02 04:21:31 <etotheipi_> splatster, py2app sounds good.  much like py2exe
418 2012-02-02 04:21:46 <cjd> I guess my real question is "is there a real benefit to sandboxing the script engine?"
419 2012-02-02 04:22:27 <luke-jr> &
420 2012-02-02 04:22:34 <luke-jr> you want to UNsandbox it?
421 2012-02-02 04:22:39 <gmaxwell> more like seperate out the wallet and sandbox the whole non-wallet part of the node.
422 2012-02-02 04:22:41 <luke-jr> OP_FOPEN
423 2012-02-02 04:23:01 <lianj> luke-jr: that would be great
424 2012-02-02 04:23:04 <luke-jr> OP_FLASHBIOS
425 2012-02-02 04:23:06 <cjd> lol
426 2012-02-02 04:23:33 <gmaxwell> OP_ENDFALSEVACUUM
427 2012-02-02 04:23:53 <cjd> by sandbox I mean put it into a seperate address space and do fun stuff to make sure nomatter how bad it gets in there, it can't own your wallet
428 2012-02-02 04:24:20 <lianj> cjd: it cant
429 2012-02-02 04:24:46 <gmaxwell> cjd: the scripting part is only a fairly modest part of the software and our scripts are more like fancy config files than program code. All that other stuff that talks to the network is about as risky.
430 2012-02-02 04:24:57 <cjd> mm
431 2012-02-02 04:25:10 <cjd> serialize.h is over my head
432 2012-02-02 04:25:42 <cjd> I read it and I gather it means that he has proven it can't have a buffer overflow as long as the templating engine is sound
433 2012-02-02 04:25:55 <cjd> but I can't really say much else about it
434 2012-02-02 04:26:54 <pingdrive> test
435 2012-02-02 04:26:55 <gmaxwell> cjd: yea, but lurking in some corner (irc code perhaps) there could be something that doesn't use the safe types.
436 2012-02-02 04:26:55 <lianj> sure badly implemented in the wrong language, it could own your wallet :P
437 2012-02-02 04:27:23 <cjd> those would be the places to look
438 2012-02-02 04:27:27 <gmaxwell> cjd: obviously we don't know of anything like that... but thats the kind of paranoia that would justify sandboxing things.
439 2012-02-02 04:27:44 <gmaxwell> cjd: people have looked. But as you say C++ is so abstract it can be easy to miss subtle things.
440 2012-02-02 04:28:17 <cjd> ^^^+100
441 2012-02-02 04:28:23 <gmaxwell> "oh. baz() returns an unsafe type and we append to it inside bar()... even though everything defined in bar() is safe"
442 2012-02-02 04:28:35 <cjd> cjdns has an option to set rlimit of file descriptors to 0 on startup, which is a hack but it works great on linux.
443 2012-02-02 04:29:01 <gmaxwell> (and I've personally audited the code for this kind of thing but fat lot thats worth. I've missed every bug we've committed since I've been paying attention :( )
444 2012-02-02 04:29:09 <cjd> doesn't make much sense where the money is in the program, but those are the kind of approaches I like.
445 2012-02-02 04:29:26 <gmaxwell> cjd: have to be careful, sometimes removing privs creates vulnerabilties too.
446 2012-02-02 04:29:47 <gmaxwell> E.g. code that could _never_ overflow .. overflows because an error that could never happen, happens due to dropping privs.
447 2012-02-02 04:31:02 <gmaxwell> if the wallet were process seperable it would be totally great to totally sandbox off the network/blockchain code.
448 2012-02-02 04:31:32 <cjd> hmm
449 2012-02-02 04:31:43 <gmaxwell> "nothign that talks to the network shares VM with your private keys" would be a warm and fuzzy thing indeed.
450 2012-02-02 04:32:24 <cjd> I did that in cjdns too because it needs to have a tcp socket for admin
451 2012-02-02 04:32:51 <gmaxwell> my own personal online wallet is -connect= to a node which runs inside valgrind which valgrind will kill it hit hits anything suspect. (except the things I've excluded because bdb sucks and is not valgrind clean)
452 2012-02-02 04:33:43 <gmaxwell> Though I probably ought to go through bitcoin and make the memset code also valgrind taint, I bet it loses a lot of sensitivity to use after free because of the custom allocator stuff.
453 2012-02-02 04:34:10 <cjd> hmm
454 2012-02-02 04:34:11 <cjd> bdb
455 2012-02-02 04:35:16 <cjd> ofc /me has done a fair amount of java programming and the people who write apache commons stuff are ...
456 2012-02-02 04:39:09 <gmaxwell> luke-jr: you can kill that VPS account I was using ???I've learned what I needed to know from it.
457 2012-02-02 04:42:33 <midnightmagic> lol, as Bjarne said, "C makes it easy to shoot yourself in the foot. C++ makes it harder, but when you do, it blows away your whole leg."
458 2012-02-02 04:43:28 <luke-jr> LOL
459 2012-02-02 04:45:09 <luke-jr> gmaxwell: blown away
460 2012-02-02 04:45:30 <gmaxwell> luke-jr: thanks for your help.
461 2012-02-02 04:45:41 <luke-jr> np
462 2012-02-02 04:51:09 <splatster> http://2.bp.blogspot.com/_SgCbuAVsmS8/TPWU5u1GfiI/AAAAAAAACbg/-QVcxLNXYIA/s1600/Yo+Dawg+-+I+Heard+you+like+dreams+%2528Inception%252C+recursive+dreams%2529.jpg
463 2012-02-02 04:51:27 <splatster> oops maybe not -dev related
464 2012-02-02 05:03:39 <midnightmagic> hey gmaxwell, what's the pps source for your future net4501? did you actually buy an HP Z3801A?
465 2012-02-02 05:06:17 <gmaxwell> midnightmagic: I have a couple trimble thunderbolt's. There is an seamingly unbounded supply of new-old-stock of them due to stock piling for cell site deployments.
466 2012-02-02 05:18:58 <midnightmagic> gmaxwell: I was thinking of getting a couple copernicus modules, as here: http://www.sparkfun.com/products/10923 , PPS length range is 100ns to 500ms.. variable polarity, the works. Looks like the instruction manual is good and detailed too: http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Sensors/GPS/63530-10_Rev-B_Manual_Copernicus-II.pdf
467 2012-02-02 05:21:08 <gmaxwell> ::nods:: you should look one bay for the vendors with the thunderbolts you can talk them into a complete kit with powered antenna and psu for around $100.. and they contain a very stable oxco, they're much better than justa PPS source.
468 2012-02-02 05:26:47 <c_k> ;win 15
469 2012-02-02 05:27:29 <midnightmagic> is the timing noise from rs232 sources partly because it's going through a serial driver or something? that is, if I plug my copernicus (since I already have the devboard for it) into it i'll still get relative superior performance (presuming the copernicus can deliver it)?
470 2012-02-02 05:28:27 <midnightmagic> it would be a fun project.. i have a very unreasonable urge to have accurate time available, even if I don't use it for something that needs it.. :)
471 2012-02-02 05:31:11 <midnightmagic> gmaxwell: do you run a stratum 2 right now? who are you using for your s1 source?
472 2012-02-02 05:32:36 <gmaxwell> I have a S1 now, but with boring unfancy hardware.. just a pps input.
473 2012-02-02 05:33:56 <gmaxwell> midnightmagic: yes. You should get superior performance from 4501 plus any pps input.. even without the 10mhz clock hack. but it'll be limited somewhat by instability of the local onboard osc (because uncertanty in its timing creates uncertanty in the timing between the pps and the packets)
474 2012-02-02 05:36:03 <midnightmagic> gmaxwell: Ah, I understand. You're replacing the crystal entirely..  what happens when GPS is blocked? Does the machine no longer tick?
475 2012-02-02 05:36:58 <midnightmagic> also, can you give me the link to that sun-based timekeeper idea you had one more time?
476 2012-02-02 05:37:18 <gmaxwell> midnightmagic: it free runs on the very stable OXCO, the specsheet has details on the holdover time.
477 2012-02-02 05:37:36 <midnightmagic> huh, cool.
478 2012-02-02 05:38:33 <nathan7> hi midnightmagic
479 2012-02-02 05:38:53 <gmaxwell> you can also by rubidium atomic clocks for about $35 (0_o) but they're power hungry, and less stable than a GPSDO. (but autonomous, at least until the lamp burns out!)
480 2012-02-02 05:39:09 <midnightmagic> ha ha!
481 2012-02-02 05:39:24 <nathan7> I'm being ignored =[
482 2012-02-02 05:39:43 <midnightmagic> !  no, not at all!  hi nathan7!
483 2012-02-02 05:39:44 <gribble> Error: "no," is not a valid command.
484 2012-02-02 05:40:08 <gmaxwell> midnightmagic http://people.xiph.org/~greg/decentralized-time.txt
485 2012-02-02 05:40:09 <nathan7> How's life, midnightmagic?
486 2012-02-02 05:40:19 <midnightmagic> I'm switching back and forth between a web browser, and for some reason your note didn't highlight in my irc client.
487 2012-02-02 05:40:38 <midnightmagic> nathan7: Good, thank you! How are you?
488 2012-02-02 05:40:56 <midnightmagic> that's the one, thanks gm
489 2012-02-02 05:41:03 <nathan7> I'm surviving and printing things
490 2012-02-02 05:41:11 <midnightmagic> printing?
491 2012-02-02 05:41:15 <nathan7> 3D printing!
492 2012-02-02 05:41:22 <midnightmagic> awesome! do you have a mendel?
493 2012-02-02 05:42:01 <midnightmagic> a coworker is getting a mendel with the longer-lasting bearings, but i'm not sure if he's willing to print me off the components
494 2012-02-02 05:42:05 <nathan7> a Prusa Mendel
495 2012-02-02 05:42:12 <nathan7> it's a bit of a mix of a v1 and a v2
496 2012-02-02 05:42:20 <nathan7> pretty much the entire machine was donated to me
497 2012-02-02 05:42:50 <midnightmagic> that's really awesome. ever since my *other* coworker started printing stuff off, all I can see everywhere is formed solutions to problems.
498 2012-02-02 05:42:50 <nathan7> I hang out a *lot* in #reprap, for more than a year now
499 2012-02-02 05:43:12 <nathan7> then I ended up helping out at a build party
500 2012-02-02 05:43:21 <midnightmagic> build parties!!!
501 2012-02-02 05:43:41 <nathan7> and then the guy who organised that and I met up in Amsterdam because I needed some stuff from him
502 2012-02-02 05:43:57 <nathan7> and he's all like "also, here's a bag of printer parts. want them?"
503 2012-02-02 05:44:16 <midnightmagic> LOL
504 2012-02-02 05:44:28 <nathan7> and then I helped out at the next build party
505 2012-02-02 05:44:36 <nathan7> and then at the next, in K??ln
506 2012-02-02 05:44:46 <nathan7> and it was all fun and things [=
507 2012-02-02 05:45:01 <midnightmagic> we are all just the vehicles for the propagation of the organism that is the reprap
508 2012-02-02 05:45:16 <midnightmagic> symbiosis
509 2012-02-02 05:46:48 <nathan7> and prusajr fixes every problem he notices during the build party
510 2012-02-02 05:46:56 <midnightmagic> wow
511 2012-02-02 05:46:58 <nathan7> so they're an opportunity to improve the machine
512 2012-02-02 05:47:17 <nathan7> (the build party crew is ruben-ikmaak, prusajr, Kliment and me)
513 2012-02-02 05:48:20 <nathan7> I'm just in for the fun =p
514 2012-02-02 05:49:09 <midnightmagic> i have some neighbours i think who would love one or two of them. it would be nice to build a community here.
515 2012-02-02 05:49:18 <nathan7> you should ask your coworker to print you a set of parts
516 2012-02-02 05:49:22 <nathan7> or I can print you a set
517 2012-02-02 05:49:55 <midnightmagic> he said he will do a build party once he gets his set up. apparently he sent off to get them professionally formed: he wants one that doesn't need a lot of valibration
518 2012-02-02 05:50:09 <nathan7> mhm
519 2012-02-02 05:50:10 <midnightmagic> $500 CAD
520 2012-02-02 05:50:19 <midnightmagic> so he's getting the parts super-cheap.
521 2012-02-02 05:50:33 <nathan7> that's cheap, yes
522 2012-02-02 05:53:23 <midnightmagic> i'm not sure how he's doing that, i guess the pros have realised that people helping people means everyone wins.
523 2012-02-02 05:53:45 <nathan7> but that's parts
524 2012-02-02 05:53:56 <nathan7> we charge ???850 to build party participants
525 2012-02-02 05:54:10 <nathan7> ???400 in parts, the rest is to pay three people and drinks and food
526 2012-02-02 05:54:11 <midnightmagic> everything all together for this fellow was apparently close to $500 CAD.
527 2012-02-02 05:54:16 <nathan7> (I'm not really paid)
528 2012-02-02 05:54:50 <nathan7> (but i really enjoy the social interaction)
529 2012-02-02 05:55:16 <nathan7> *I 
530 2012-02-02 05:55:43 <midnightmagic> cool beans man, you'll have to post some pics of some forms
531 2012-02-02 05:55:47 <midnightmagic> ttyl
532 2012-02-02 05:55:49 <nathan7> midnightmagic: see ikmaak.nl
533 2012-02-02 05:55:55 <nathan7> for build party pics
534 2012-02-02 05:56:04 <nathan7> it's in Dutch I'm afraid
535 2012-02-02 07:25:00 <luke-jr> doh, 20 Feb hasn't passed yet? :P
536 2012-02-02 07:25:27 <Diablo-D3> whats then?
537 2012-02-02 07:48:05 <luke-jr> Diablo-D3: bitcoin node protocol change
538 2012-02-02 07:49:00 <Diablo-D3> heh
539 2012-02-02 08:05:26 <Prattler> hey, what's the proper way to support p2sh via solo mining? What do I need to do?
540 2012-02-02 08:06:16 <Eliel> apply a patch to your bitcoind or get the latest git version and run that
541 2012-02-02 08:07:09 <luke-jr> latest git is BIP 16 tho
542 2012-02-02 08:07:21 <luke-jr> Prattler: http://luke.dashjr.org/programs/bitcoin/files/bip17/
543 2012-02-02 08:07:25 <Prattler> thanks
544 2012-02-02 08:07:39 <luke-jr> this is BIP 17 week
545 2012-02-02 08:10:53 <Joric> i got a feeling it's a bip 17 month or even an year
546 2012-02-02 08:12:26 <luke-jr> Joric: well, if we achieve majority quickly, it can even be permanent :P
547 2012-02-02 09:09:55 <epscy> what is BIP 17?
548 2012-02-02 09:10:30 <cjd> luke's counterproposal to bip16
549 2012-02-02 12:26:42 <luke-jr> epscy: P2SH done right; I wrote the BIP, but it came out of community discussion
550 2012-02-02 12:43:31 <ThomasV> naive question: how do I dump the string in a CDataStream ?
551 2012-02-02 12:48:26 <smart19885> moin moin
552 2012-02-02 14:34:05 <Joric> did anyone benchmark win32 version of 0.5.2 against 0.5.1?
553 2012-02-02 14:34:20 <Joric> doesn't seem really faster
554 2012-02-02 14:38:12 <pentarh> please comment on this namecoin design improvement proposal https://bitcointalk.org/index.php?topic=62017.msg727656#msg727656 - it possibly could make namecoin better than bitcoin
555 2012-02-02 14:38:25 <luke-jr> Joric: if your bottleneck is network, it won't be
556 2012-02-02 14:40:20 <Joric> yeah sure it's network
557 2012-02-02 14:40:33 <Joric> it could be... 10 years ago
558 2012-02-02 14:42:22 <Joric> why the client is so slow in general? because of recalculating all tx hashes?
559 2012-02-02 14:45:27 <gmaxwell> Joric: haseh are stupidly fast.
560 2012-02-02 14:45:30 <gmaxwell> er hashes.
561 2012-02-02 14:45:33 <gmaxwell> Joric: slow _how_ ?
562 2012-02-02 14:46:01 <gmaxwell> Different kinds of slowness have different causes.
563 2012-02-02 14:46:04 <Joric> it takes two days to download the blockchain
564 2012-02-02 14:46:28 <TD> so use multibit
565 2012-02-02 14:46:37 <TD> or some other lightweight client like electrum
566 2012-02-02 14:46:38 <gmaxwell> TD: pft.
567 2012-02-02 14:47:44 <gmaxwell> Joric: I can sync in ~30 minutes to a ramdisk/ non-fsyncing medium. We're not sure of the exact causes of the slowness, though a major part is just due to the use of synchronous random writes.
568 2012-02-02 14:49:38 <gmaxwell> IO traces of the client show it doing about 23GiB of writes during a full sync from 0, most of them random and most synchronous.  This can be fixed, but the exact details aren't clear yet.
569 2012-02-02 14:50:40 <Eliel> Joric: I hear the libbitcoin version of bitcoind is quite a bit faster with the blockchain download.
570 2012-02-02 14:50:46 <Joric> did you try libcoin? that phd guy forgot his name did he rewrite something or just copied the code
571 2012-02-02 14:51:11 <Eliel> Joric: he refactored the code.
572 2012-02-02 14:51:32 <Eliel> not quite a rewrite but not far :)
573 2012-02-02 14:51:34 <Joric> so no real job was done whatsoever
574 2012-02-02 14:51:42 <gmaxwell> Eliel: have you actually compared it?
575 2012-02-02 14:52:03 <Eliel> gmaxwell: I haven't had time to look at it yet.
576 2012-02-02 14:52:21 <gmaxwell> I spent about an hour going through it and I can't see why it would be (much) faster.
577 2012-02-02 14:52:28 <gmaxwell> I haven't actually tested it, I guess I should.
578 2012-02-02 14:53:06 <Eliel> in the email list he said it's due to removing the inefficiencies associated with threads and locking.
579 2012-02-02 14:53:14 <gmaxwell> He was speculating.
580 2012-02-02 14:53:43 <gmaxwell> My instrumentation indicates that we're not spending basically any time waiting on locks during sync.. could be wrong, but I'm doubtful.
581 2012-02-02 14:54:07 <gmaxwell> Good news there is that if it is as much of an improvement as he said then I suspect a lot is probably just some stupid bug he fixed accidentally.
582 2012-02-02 14:54:21 <Eliel> quite likely :)
583 2012-02-02 15:09:31 <gmaxwell> okay.. I _would_ test it but the cmake fails in a completely opaque way.
584 2012-02-02 15:11:28 <makomk> libcoin?
585 2012-02-02 15:11:30 <gmaxwell> yea, this is likely to be effective:
586 2012-02-02 15:11:31 <gmaxwell> stat("C:/boost/lib64", 0x7fff0db79fd0)  = -1 ENOENT (No such file or directory)
587 2012-02-02 15:12:10 <gmaxwell> makomk: yes.
588 2012-02-02 15:12:27 <makomk> Got any errors like "add_subdirectory given source "coinQt" which is not an existing directory"?
589 2012-02-02 15:13:30 <gmaxwell> I was chasing "Could NOT find Boost" first. actually I don't see what one however. It's whining about qt4/qt3/wxWidgets too though.
590 2012-02-02 15:15:05 <luke-jr> gmaxwell: same problem as me
591 2012-02-02 15:15:11 <makomk> The latest git fixed "Could NOT find Boost" for me, though I think - at least for me - that message was bogus.
592 2012-02-02 15:15:28 <luke-jr> it's not looking in /usr/lib
593 2012-02-02 15:15:32 <luke-jr> everywhere BUT there
594 2012-02-02 15:15:44 <gmaxwell> makomk: I'm on a pull from a few minutes ago.
595 2012-02-02 15:16:20 <gmaxwell> luke-jr: I see it stat /usr/lib64 and even :
596 2012-02-02 15:16:21 <gmaxwell> stat("/usr/lib64/libboost_date_time-mt.a", {st_mode=S_IFREG|0644, st_size=137278, ...}) = 0
597 2012-02-02 15:16:50 <makomk> Interesting. Anyway, you might want to try http://pastebin.ca/2108766 and see if the message about not finding Boost actually matters with that fixed.
598 2012-02-02 15:20:32 <luke-jr> it didn't for me last time
599 2012-02-02 15:20:57 <gmaxwell> makomk: nope.. but I removed the boost checks and it appears to still be getting the includes right perhaps it'll blow up when it links though.
600 2012-02-02 15:21:14 <luke-jr> gmaxwell: yeah, it does seem to find boost includes during cmake&
601 2012-02-02 15:23:26 <gmaxwell> I can't understand how he possibly could have made bitcoin _slower_ to build
602 2012-02-02 15:23:52 <gmaxwell> anyways, I have a binary now.
603 2012-02-02 15:23:55 <luke-jr> :o
604 2012-02-02 15:24:15 <onelineproof> you should make bitcoin-qt build on top of bitcoind
605 2012-02-02 15:26:17 <luke-jr> onelineproof: see also: Spesmilo
606 2012-02-02 15:28:17 <gmaxwell> welp. ... it only seems to want to run as an rpc client for me.
607 2012-02-02 15:29:06 <gmaxwell> ah, apparently it only starts as a daemon if you give it no options.
608 2012-02-02 15:29:06 <luke-jr> aha, it's demanding static libs
609 2012-02-02 15:32:08 <gmaxwell> oohhh kay.. you can't give it a port number with connect.
610 2012-02-02 15:32:33 <makomk> I can't figure out how to pass -datadir to it either.
611 2012-02-02 15:32:39 <gmaxwell> 08:29 < gmaxwell> ah, apparently it only starts as a daemon if you give it no options.
612 2012-02-02 15:33:01 <gmaxwell> makomk: I symlinked .bitcoin from the account I was running it in.
613 2012-02-02 15:35:57 <makomk> Hmmm, makes sense I guess.
614 2012-02-02 15:37:50 <gmaxwell> oh I bet he was benchmarking against the code he forked from.
615 2012-02-02 15:38:16 <gmaxwell> He removed all the secureallocator stuff, so he fixed the mlock behavior as a side effect.
616 2012-02-02 15:39:29 <TD> right
617 2012-02-02 15:39:34 <TD> bip 14 implemented in bitcoinj
618 2012-02-02 15:40:36 <gmaxwell> looks like this is going to be slower than reference. :(
619 2012-02-02 15:44:26 <makomk> I don't think it has the code to skip ECDSA checking for older blocks.
620 2012-02-02 15:44:53 <luke-jr> what's the purpose of multiple checkpoints? doesn't the last one make the earlier ones useless?
621 2012-02-02 15:45:18 <gmaxwell> luke-jr: no, because they stop people from feeding bogus forks that will never win.
622 2012-02-02 15:45:28 <luke-jr> ah
623 2012-02-02 15:45:59 <Diablo-D3> but I like bogus forks! I call them spoons!
624 2012-02-02 15:46:03 <gmaxwell> This could be fixed via a better fetching algo (fetch headers backwards along the most plausable chain, then only fetch headers for the winner)
625 2012-02-02 15:46:28 <gmaxwell> makomk: yea, it's looking like it's going to take about 2x what the refernce client currently takes.
626 2012-02-02 15:46:34 <Diablo-D3> k9quaint: kinky.
627 2012-02-02 16:01:23 <gmaxwell> :(  it's now at only 137695 and has taken as long as the reference software does for me.. maybe 3-4x slower.
628 2012-02-02 16:34:30 <gmaxwell> Okay, almost exactly 2x long as the reference client.
629 2012-02-02 16:37:48 <BlueMatt> wait, libcoin is slower than reference?
630 2012-02-02 16:37:49 <BlueMatt> :*
631 2012-02-02 16:37:56 <BlueMatt> s/:*/:(/
632 2012-02-02 16:38:14 <BlueMatt> ;;seen genjix
633 2012-02-02 16:38:14 <gribble> genjix was last seen in #bitcoin-dev 1 day, 19 hours, 48 minutes, and 6 seconds ago: <genjix> ok off. cya all.
634 2012-02-02 16:38:47 <gmaxwell> BlueMatt: I don't doubt it's faster than the code he forked from but at least for me its slower.
635 2012-02-02 16:39:14 <gmaxwell> BlueMatt: he removed all the secure allocator stuff, so I'm guessing the improvement he saw came entirely from the mlock behavior.
636 2012-02-02 16:39:26 <gmaxwell> well, almost entirely.
637 2012-02-02 16:40:34 <BlueMatt> mmm, shame
638 2012-02-02 16:40:46 <BlueMatt> while you are benchmarking, feel like looking at CBlockStore?
639 2012-02-02 16:40:55 <BlueMatt> ;)
640 2012-02-02 16:41:50 <gmaxwell> Hey, I looked at it before. But yes, I guess I should again do you still have the mystery performance loss?
641 2012-02-02 16:42:07 <BlueMatt> yea, thats pretty much all Im waiting for before I mark the pull ready for merge
642 2012-02-02 16:42:24 <BlueMatt> or at least ready for full pre-merge analysis
643 2012-02-02 16:43:11 <gmaxwell> BlueMatt: okay, I'll pull it again now and go over it and see if I can find the cause.
644 2012-02-02 16:43:20 <BlueMatt> thanks a ton
645 2012-02-02 16:43:25 <gmaxwell> BlueMatt: in the meantime go give my utterly trivial listening patch a sanity look.
646 2012-02-02 16:43:28 <gmaxwell> ;)
647 2012-02-02 16:43:57 <BlueMatt> yea, I was just about to go do that ;)
648 2012-02-02 16:44:39 <gmaxwell> I'm really bummed that this refactor thing didn't magically fix things. :(
649 2012-02-02 16:46:05 <BlueMatt> maybe it'l have more speed hiding under a few minor bugs (like the ecdsa checking thing)
650 2012-02-02 16:47:35 <gmaxwell> I'd guess a bit, but not the kind of large improvements that I want and think we can get through saner IO behavior. (AFAICT his IO behavior is the same as the reference)
651 2012-02-02 16:47:39 <Joric> is it possible to wrap disk operations to write from ram using 10 mb blocks or something
652 2012-02-02 16:47:52 <gmaxwell> Joric: Not trivally.
653 2012-02-02 16:48:27 <BlueMatt> gmaxwell: its much easier when you can split telling other code about new blocks and writing them which you can easily do in eg cblockstore ;)
654 2012-02-02 16:48:46 <gmaxwell> Joric: the software is updating a database on disk while using that database for lookups.
655 2012-02-02 16:49:09 <BlueMatt> well you also have to have a memory list of blocks you can do lookups in first
656 2012-02-02 16:49:45 <gmaxwell> BlueMatt: yea, and also handle reverting changes to your in memory database for reorgs.
657 2012-02-02 16:50:15 <BlueMatt> yea...
658 2012-02-02 16:51:33 <Joric> i just tried ramdrive it speeded up 10-20x
659 2012-02-02 16:51:51 <gmaxwell> Joric: I think I told you this earlier?
660 2012-02-02 16:52:09 <BlueMatt> (has several .bitcoin datadirs always mounted on tmpfs ready to go at any time :) )
661 2012-02-02 16:52:32 <Joric> i only have 2 gb and db weights 1.5 gb so everything else become slow as hell had to cancel it )
662 2012-02-02 16:52:41 <BlueMatt> oh...
663 2012-02-02 16:52:50 <gmaxwell> My gut feel is that it may be easier to solve this by just supporting a 'summary file', then have an external tool that can independantly build the summary in a determinstic and trustworthy way from a compressed blockchain. .. as this will also solve runtime storage as well as syncup speed.
664 2012-02-02 16:52:59 <Joric> it was downloading blocks at 300-400 kb/s
665 2012-02-02 16:53:13 <gmaxwell> Just make the connecting input stuff check the current database, then if there is no hit it checks the summary file.
666 2012-02-02 16:53:28 <gmaxwell> (and lookups in the summary file can be basically O(1)ish)
667 2012-02-02 16:54:27 <BlueMatt> gmaxwell: seems like that would take as much work as doing it right in the first place.  re: reorgs: the buffer idea could be used only during initial syncup.  During that time bitcoin should instead of just downloading blocks, download headers first to get the chain, then dl blocks.  This means you dont have to deal with reorgs until you are on disk, and also prevents the fill disk with a ton of orphan chains that start at 0 attacks
668 2012-02-02 16:55:21 <gmaxwell> BlueMatt: hm. point about only using it initially... the buffering really isn't needed at runtime.. but I'm not sure if the bulk updates really solve the problem.
669 2012-02-02 16:55:32 <gmaxwell> E.g. I don't know if libdb will really batch inserts for index updates.
670 2012-02-02 16:55:49 <gmaxwell> or if it will just do the same zillion writes per record.
671 2012-02-02 16:56:10 <BlueMatt> doesnt matter, you can put the updates in a separate thread so the actual block dls dont take as long
672 2012-02-02 16:56:17 <BlueMatt> s/take as long/block/
673 2012-02-02 16:56:33 <BlueMatt> (well, ok you would still have to block if you start eating 10gb memory, but...)
674 2012-02-02 16:56:39 <gmaxwell> Yes but if you can only write out N updates per second you'll eventually get ahead of it and need to block since taking a gig of ram sucks.
675 2012-02-02 16:56:42 <gmaxwell> right.
676 2012-02-02 16:57:04 <BlueMatt> still, it would help significantly
677 2012-02-02 16:57:23 <gmaxwell> It will, just getting the concurrency will help a ton.
678 2012-02-02 16:57:28 <BlueMatt> yep
679 2012-02-02 16:58:05 <Joric> multibit downloads headers in first 30 seconds or something it's only 17 mb of them
680 2012-02-02 16:58:12 <BlueMatt> anyway, thats my next big cblockstore project
681 2012-02-02 16:58:21 <gmaxwell> Joric: sure, headers are no big deal.
682 2012-02-02 16:58:22 <BlueMatt> (after 0.7 gets cblockstore to begin with)
683 2012-02-02 16:58:36 <gmaxwell> though we need to add a protocol feature to help fetch headers backwards.
684 2012-02-02 16:58:41 <BlueMatt> well, maybe Ill do windows autoupdate first...
685 2012-02-02 16:58:42 <gmaxwell> (with piplining)
686 2012-02-02 17:06:34 <BlueMatt> gmaxwell: the addition of the if(nLastRebroadcast) is there a point to that other than a minuscule optimization?
687 2012-02-02 17:09:49 <gmaxwell> BlueMatt: No, well Kinda. It prevents it from dropping AddrKnown the first time IsInitialBlockDownload passes. The prior behavior ran it once excessively but it was so soon after startup it had little effect. Running it later caused a whole bunch of redundant addr flooding.
688 2012-02-02 17:10:08 <gmaxwell> E.g. my change otherwise made a minorly stupid behavior worse, and that prevents this from happening.
689 2012-02-02 17:10:58 <BlueMatt> mmm, ok
690 2012-02-02 17:11:03 <BlueMatt> that should have been obvious
691 2012-02-02 17:14:33 <BlueMatt> gmaxwell: I dont like adding fNoListen to GetMyExternalIP.  it breaks a ton of if (connecting to myself) dont try; logic.  Better would be to add fNoListen to the two places in irc.cpp where addrLocalHost is used and Im pretty sure that covers everything
692 2012-02-02 17:14:53 <BlueMatt> no, need it in like 3 places in net.cpp too
693 2012-02-02 17:14:59 <luke-jr> DEBUG:BitcoinLink:Wrong checksum on `verack' message (b'f9beb4d9' vs actual:b'5df6e0e2'); ignoring
694 2012-02-02 17:15:08 <luke-jr> is `verack' in some cases checksumless too?
695 2012-02-02 17:15:24 <luke-jr> I suppose if I ignore it until 20 Feb it'll go away& >.>
696 2012-02-02 17:15:40 <gmaxwell> BlueMatt: er.. IRC won't even _run_ if fNoListen is true.
697 2012-02-02 17:15:47 <BlueMatt> actually one place (maybe), just PushVersion
698 2012-02-02 17:15:57 <BlueMatt> oh, ok nvm so Id say remove that dif
699 2012-02-02 17:15:58 <BlueMatt> f
700 2012-02-02 17:16:21 <BlueMatt> luke-jr: whats on 20 feb wrt checksums?
701 2012-02-02 17:16:36 <luke-jr> BlueMatt: 20 Feb, the bitcoin p2p protocol handshake changes
702 2012-02-02 17:16:37 <gmaxwell> BlueMatt: so, with that GetMyExternalIP nolisten makes a bitcoin node completely invisible to everyone but its peers. Which I think is the expected behavior.
703 2012-02-02 17:16:50 <BlueMatt> luke-jr: mmm, how did I miss this?
704 2012-02-02 17:16:58 <luke-jr> BlueMatt: all clients since like 0.3.0 (not sure exactly) automatically change by time
705 2012-02-02 17:17:04 <luke-jr> it's hidden in a .h
706 2012-02-02 17:17:09 <BlueMatt> luke-jr: oh, thats old...
707 2012-02-02 17:17:17 <luke-jr> net.h:        // Version 0.2 obsoletes 20 Feb 2012
708 2012-02-02 17:17:42 <BlueMatt> gmaxwell: it already is
709 2012-02-02 17:17:45 <BlueMatt> afaict
710 2012-02-02 17:18:28 <BlueMatt> gmaxwell: the only place its passed which isnt already covered is the SendVersion
711 2012-02-02 17:18:35 <BlueMatt> s/Send/Push/
712 2012-02-02 17:18:46 <gmaxwell> BlueMatt: did we just cros conversations? :)
713 2012-02-02 17:18:51 <BlueMatt> addrMe, which I dont think is added, but if it is, just send 0.0.0.0
714 2012-02-02 17:18:59 <BlueMatt> gmaxwell: I dont think so
715 2012-02-02 17:19:19 <gmaxwell> I have _no_ clue what you're talking about now.
716 2012-02-02 17:19:53 <BlueMatt> gmaxwell: you say with GetExternalIP returning false if(fNoListen) it becomes invisible, my point is, you dont need that to become invisible
717 2012-02-02 17:19:57 <gmaxwell> BlueMatt: What is passed?
718 2012-02-02 17:20:03 <BlueMatt> your ip
719 2012-02-02 17:20:07 <BlueMatt> (to your peers)
720 2012-02-02 17:20:32 <gmaxwell> BlueMatt: you're visible to the external ip services, and for no good reason.
721 2012-02-02 17:21:17 <BlueMatt> if (addrImAboutToConnectTo == addrMe) dont bother;
722 2012-02-02 17:21:20 <BlueMatt> (is the reason)
723 2012-02-02 17:21:40 <gmaxwell> Yes, but that's only one single connection which will instantly fail (no timeout) since you're not listening.
724 2012-02-02 17:23:08 <BlueMatt> then you are sending addrMe == 127.0.0.1:0 (Im pretty sure) as well
725 2012-02-02 17:23:24 <BlueMatt> maybe set the ip to 0.0.0.0 and then return false?
726 2012-02-02 17:23:40 <BlueMatt> though thats bad design...
727 2012-02-02 17:23:44 <gmaxwell> I think it's a worthwhile tradeoff (even if it takes the normal failure time) for virtue of not having every node announce itself to a couple of centralized services, no?
728 2012-02-02 17:24:51 <gmaxwell> but yea... 0.0.0.0:0 would be better, because we do that for fUseProxy.
729 2012-02-02 17:24:59 <gmaxwell> (in the pushversion at least)
730 2012-02-02 17:25:21 <BlueMatt> gmaxwell: yea probably, but pushing 0.0.0.0 would be nicer imo
731 2012-02-02 17:25:37 <BlueMatt> luke-jr: am I reading this right that all bitcoin message checksums will disappear on that date?
732 2012-02-02 17:25:37 <luke-jr> pushing :: would be best probably?
733 2012-02-02 17:25:47 <luke-jr> BlueMatt: no, all will have checksums
734 2012-02-02 17:25:55 <luke-jr> BlueMatt: right now, at least `version' has no checksum
735 2012-02-02 17:26:16 <BlueMatt> oh, sorry >=, thought it was <
736 2012-02-02 17:26:28 <BlueMatt> shame, we should drop the checksum alltogether
737 2012-02-02 17:26:34 <gmaxwell> 0_o
738 2012-02-02 17:26:35 <luke-jr> TCP checksums don't work
739 2012-02-02 17:26:43 <gmaxwell> TCP checksum is inadequate.
740 2012-02-02 17:27:08 <[eval]> because messages can be split over multiple segments?
741 2012-02-02 17:27:11 <gmaxwell> (go try rsyncing a few TB over the internet.. your ssh connection will eventually drop)
742 2012-02-02 17:27:17 <luke-jr> I've never seen TCP correct corruption
743 2012-02-02 17:27:26 <BlueMatt> well we should drop a lot of checksums, ie block message checksums
744 2012-02-02 17:27:26 <gmaxwell> [eval]: no, because it's just ones compliment addition mode 65536.
745 2012-02-02 17:27:26 <luke-jr> or even detect
746 2012-02-02 17:27:33 <[eval]> ah ok
747 2012-02-02 17:27:37 <gmaxwell> luke-jr: I have, there is a nice paper on it too.
748 2012-02-02 17:27:42 <luke-jr> BlueMatt: no, because we DoS people who send bad blocks
749 2012-02-02 17:27:55 <luke-jr> err
750 2012-02-02 17:27:59 <luke-jr> not DoS, anti-DoS/ban
751 2012-02-02 17:28:19 <BlueMatt> luke-jr: meh, as long as its only DoS on like 2 bad blocks its fine
752 2012-02-02 17:28:19 <gmaxwell> http://www.ir.bbn.com/documents/articles/crc-sigcomm00.ps
753 2012-02-02 17:28:20 <luke-jr> we'd need to consider the possibility of corruption in CheckBlock if we removed the message checksum
754 2012-02-02 17:28:27 <BlueMatt> (if your connection sucks tat bad...)
755 2012-02-02 17:28:44 <luke-jr> BlueMatt: I could see it happening during initial sync
756 2012-02-02 17:28:46 <BlueMatt> gmaxwell: who posts postscript docs?
757 2012-02-02 17:28:56 <gmaxwell> BlueMatt: Real Men"
758 2012-02-02 17:29:00 <luke-jr> lol
759 2012-02-02 17:29:19 <BlueMatt> luke-jr: its rare enough that you might drop one peer every once in a while, which is fine for how nice it would be to not do double sha on every message
760 2012-02-02 17:29:39 <luke-jr> let's put up some BMPs too
761 2012-02-02 17:29:46 <gmaxwell> BlueMatt: the checksum stuff creates a nice way to add authenticated peers too. Provide a secret on each side, and swap the hash for hmac.
762 2012-02-02 17:30:04 <BlueMatt> well thats true
763 2012-02-02 17:30:20 <gmaxwell> probably 20 LOC to add that.
764 2012-02-02 17:30:48 <BlueMatt> probably one LOC to remove the checksums and leave the space there for them so that others can use HMAC
765 2012-02-02 17:30:58 <gmaxwell> true.
766 2012-02-02 17:31:21 <gmaxwell> okay. back to my patch.. is PushVersion the only place you think that should be handled?
767 2012-02-02 17:31:53 <gmaxwell> or should getexternalip return 0:0?  (and then the fUseproxy checks can be removed in pushversion)
768 2012-02-02 17:32:11 <gmaxwell> (well, one of them)
769 2012-02-02 17:32:15 <BlueMatt> actually, nvm GetMyExternalIP is never called if fUseProxy||fNoListen
770 2012-02-02 17:32:36 <gmaxwell> Hm. I thought I checked that
771 2012-02-02 17:32:43 <BlueMatt> if (fUseProxy || mapArgs.count("-connect") || fNoListen)
772 2012-02-02 17:32:46 <BlueMatt> net.cpp:1694
773 2012-02-02 17:32:59 <BlueMatt> so, it doesnt matter
774 2012-02-02 17:33:12 <BlueMatt> gmaxwell: https://github.com/bitcoin/bitcoin/pull/792#issuecomment-3783510
775 2012-02-02 17:33:30 <gmaxwell> Sure enough.
776 2012-02-02 17:34:00 <Joric> how do you determine external ip? from other peers?
777 2012-02-02 17:34:03 <gmaxwell> Though mapArgs.count("-connect") is bogus there.
778 2012-02-02 17:34:13 <gmaxwell> Joric: you shouldn't ask, you won't like the answer.
779 2012-02-02 17:34:16 <BlueMatt> gmaxwell: not -connect implies fNoListen
780 2012-02-02 17:34:21 <BlueMatt> s/not/no/
781 2012-02-02 17:34:27 <BlueMatt> though I suppose its redundant
782 2012-02-02 17:34:30 <gmaxwell> BlueMatt: No it doesn't.
783 2012-02-02 17:34:33 <BlueMatt> hmm?
784 2012-02-02 17:34:37 <BlueMatt> I could have sworn it did
785 2012-02-02 17:34:40 <Joric> i used stun for that
786 2012-02-02 17:35:04 <BlueMatt> I guess I really know nothing about the net code...
787 2012-02-02 17:35:28 <gmaxwell> I don't think anyone editing it does. The behavior is randomly inconsistent in a bunch of places.
788 2012-02-02 17:36:03 <BlueMatt> then someone should rip it out and rewrite it :)
789 2012-02-02 17:36:28 <gmaxwell> Well what behavior do we want?
790 2012-02-02 17:36:47 <BlueMatt> for what?
791 2012-02-02 17:37:37 <gmaxwell> In any case... I don't think we should replumb all that functionality for the next release, but I do think the listen thing should go in.
792 2012-02-02 17:37:50 <gmaxwell> BlueMatt: What connectivity options should we be offering.
793 2012-02-02 17:38:04 <gmaxwell> Should we have "connect" that implies nolisten, etc.
794 2012-02-02 17:38:17 <BlueMatt> you mean if someone were to rewrite it all?
795 2012-02-02 17:38:45 <luke-jr> sipa just rewrote the net code&
796 2012-02-02 17:38:46 <gmaxwell> Right (well, I don't think it needs a complete rewrite), but if we were to change the behavior what should the behavior actually be.
797 2012-02-02 17:39:22 <BlueMatt> I dunno, up to the implementor, there I have little preference
798 2012-02-02 17:40:01 <BlueMatt> I think it would be nice to rewrite the net code to use boost::asio or smth
799 2012-02-02 17:40:14 <BlueMatt> hence why cblockstore moves all the net stuff out of main.cpp
800 2012-02-02 17:41:12 <BlueMatt> but that was a really minor afterthought
801 2012-02-02 17:41:15 <gmaxwell> I actually don't give a shit about api minutia, thats up to whoever sits down and does it.  It's important to know what it should be doing. right now, -connect controls where you connect to, but doesn't change listening except if you have a private IP it will prevent announcements becuase it also doesn't try to get your external IP.
802 2012-02-02 17:41:16 <BlueMatt> anyway, Im off see yall
803 2012-02-02 17:41:20 <gmaxwell> K.
804 2012-02-02 19:44:05 <osmosis> http://seclists.org/fulldisclosure/2012/Feb/0
805 2012-02-02 19:44:48 <Diablo-D3> oh goddamnit
806 2012-02-02 19:45:48 <gmaxwell> osmosis: yes, we know about that post and as I advised on the other channels you're spamming it in, read the reply.