1 2012-10-04 00:00:25 <gmaxwell> This is the common misconception tht bitcoin is a balance tracking system.
  2 2012-10-04 00:01:01 <bcb> gmaxell: so forgive my ignorance can you explain more
  3 2012-10-04 00:01:22 <jose__> Thank gmaxwell again. I still have the feeling of an assimetric api. But is ok. btc is the best there is we just have to make it better.
  4 2012-10-04 00:02:12 <gmaxwell> jose__: I guess you can say the system itself is asymmetric, and intentionally so??? the system needs to know about where funds go for security, but not where they come from.
  5 2012-10-04 00:02:54 <gmaxwell> and bitcoin tries very hard to keep information which isn't essential for security and inflation prevention out of the chain, because that information is bad for privacy/security/scalablity.
  6 2012-10-04 00:04:12 <gmaxwell> bcb: What would you like me to explain?  The basic point there is that bitcoin tracks 'transactions'??? it's helpful to visualize them as coins. When you spend your coins are re-forged by the network??? in the process it can split and merge them, and
  7 2012-10-04 00:04:33 <gmaxwell> and when they're reforced they are minted with the new rules for redeeming them inscribed on them.
  8 2012-10-04 00:05:01 <gmaxwell> Your transaction which directs this process includes the information required to satisify those rules on the original coins you are spending.
  9 2012-10-04 00:05:01 <jose__> There is no fking 'from' in a tx but the money was signed by some one. That what I mean by 'too deep in the design'.
 10 2012-10-04 00:05:13 <gmaxwell> jose__: it can be signed by many people!
 11 2012-10-04 00:05:29 <gmaxwell> and the ownership can and does change invisible to the user
 12 2012-10-04 00:05:43 <jose__> That is part of 'too deep in the desing'.
 13 2012-10-04 00:05:46 <gmaxwell> Thats not a from it's a prior to. It's sometimes but not always relted.
 14 2012-10-04 00:05:58 <gmaxwell> related*
 15 2012-10-04 00:06:02 <gmaxwell> e.g. common web wallets...
 16 2012-10-04 00:07:20 <gmaxwell> I pay Alice, a webwallet user, 5 BTC...  Then later Bob, another user, of that service spends 5 BTC exactly to Tom. The service uses the payment I made to Alice to pay that; because its an exact match that minimizes transaction size and network load.
 17 2012-10-04 00:07:43 <gmaxwell> Now, if Tom is under the misaprehension that the prior-to is a from??? he'll be quite confused.
 18 2012-10-04 00:08:12 <jose__> Meaning signing is the hole point of ownership but the way transactions are constructed makes the reflection of ownership hard to track.
 19 2012-10-04 00:08:17 <gmaxwell> Alternatively, what happens when the prior-to is a script that could be redeemed by multiple people?  ??? any of six keys can spend it?  whats the source then?
 20 2012-10-04 00:08:50 <gmaxwell> or when it can be redeemed by anyone who can provide the preimage under SHA256 of some 256 bit value?
 21 2012-10-04 00:09:10 <bcb> "preimage" ?
 22 2012-10-04 00:09:36 <gmaxwell> H('secret') = 0x12345 ...  I can write a transaction that requires the redeemer to provide 'secret'.
 23 2012-10-04 00:09:44 <gmaxwell> (H == a hash function)
 24 2012-10-04 00:09:47 <bcb> ahh
 25 2012-10-04 00:10:03 <gmaxwell> You can use this to make cheating proof cross-cryptocurrency trades, for example.
 26 2012-10-04 00:10:49 <gmaxwell> You write txns that pay to someone who provides H('yoursecret')=12345 AND H('mysecret')=23456.  Once matching transactions are put in on both chains then the users exchange secrets.
 27 2012-10-04 00:11:10 <gmaxwell> If the guy your trading with doesn't give you his secret you can still get it out of the transaction on the other chain when he redeems it.
 28 2012-10-04 00:13:56 <bcb> gmaxwell:  helpful.  I think the assumption that bitcoind is tracking debits and credits when it is essentially tracking and confirming the transactions?
 29 2012-10-04 00:14:24 <gmaxwell> bcb: correct. Bitcoin tracks only transactions.  But thats not very useful so _all_ bitcoin software and websites show debits and credits.
 30 2012-10-04 00:14:29 <jose__> The only 'secret' should be your private keys. That is the point of cryptography.
 31 2012-10-04 00:14:46 <gmaxwell> jose__: do you hear that wooshing sound?
 32 2012-10-04 00:14:58 <jose__> sorry
 33 2012-10-04 00:15:00 <jose__> i am out
 34 2012-10-04 00:15:02 <bcb> so there is not necessarily a corresponding debit for every credit
 35 2012-10-04 00:15:14 <gmaxwell> Thats you missing the point of that example. Doing a cross currency trade securely and cheat proof is not possible with just private keys.
 36 2012-10-04 00:15:53 <gmaxwell> I am sad that I never got around to saying "No way, Jose"
 37 2012-10-04 00:16:08 <bcb> too late!
 38 2012-10-04 00:16:23 <gmaxwell> bcb: There is certantly coins that go in for any that go out.  But they may not neatly fit into clean debits and credits.
 39 2012-10-04 00:16:50 <gmaxwell> bcb: for example, it's possible for you and me to jointly create a transaction where you and I provide part of the total payment.
 40 2012-10-04 00:17:47 <gmaxwell> So you have two out??? from different people??? and one in.  And 'ownership' isn't clear??? they could be one owner of a txn, or many  .. or an output  could be 'owned' by a collection of people.
 41 2012-10-04 00:18:56 <bcb> this is multisiq and only avail in 0.7 yes
 42 2012-10-04 00:20:04 <bcb> gmaxwell: according to the command you have to know the number of signers before you create the transaction
 43 2012-10-04 00:20:16 <bcb> multisigtransaction
 44 2012-10-04 00:21:30 <gmaxwell> bcb: 'this' meaning the "be 'owned' by a collection of people"  ? Thats multisig... you have to know the candidate signers and how many will be required to create a multisig address.
 45 2012-10-04 00:21:45 <gmaxwell> the system allows a lot more than the rpc allows though
 46 2012-10-04 00:22:22 <gmaxwell> e.g. you can have a txn that can be redeemed by A || (B && Two-Of[C,D,E])
 47 2012-10-04 00:23:04 <bcb> what does that command look like
 48 2012-10-04 00:24:03 <gmaxwell> There isn't a command for it??? you'd have to use external software to write that transaction. The network supports it though.
 49 2012-10-04 00:24:42 <B0g4r7> Is there a way I can cause bitcoind to always "trust" a specific peer (IP address on my LAN), and always exchange new blocks with it as soon as a new one is received by either?
 50 2012-10-04 00:24:57 <B0g4r7> Maybe trust wasn't the right term really.
 51 2012-10-04 00:25:08 <gmaxwell> B0g4r7: you already give blocks to your peers as soon as you get one. (well, send invs)
 52 2012-10-04 00:25:27 <gmaxwell> (you send an INV and if the peer doesn't have the block it will send a get)
 53 2012-10-04 00:25:51 <B0g4r7> One won't snub the other for being too chatty relative to the other peers or something?
 54 2012-10-04 00:26:22 <gmaxwell> No. INV messages are super small so we're maximally chatty about them
 55 2012-10-04 00:26:56 <B0g4r7> Hm, so it should "just work" then huh...
 56 2012-10-04 00:27:11 <B0g4r7> I already have addnode= lines for them in each others' conf files.
 57 2012-10-04 00:28:16 <B0g4r7> I have 2 internet connections, and I plan to have the two nodes each using one, so that if either goes down, they both still remain synchronized through the remaining connection.
 58 2012-10-04 00:29:52 <B0g4r7> Can I expect the peers to remain connected to each other long after startup if I have the addnode= lines present?
 59 2012-10-04 00:30:19 <gmaxwell> yep.
 60 2012-10-04 00:46:14 <B0g4r7> excellent
 61 2012-10-04 01:08:14 <bcb> does bitcoind save your commands like .bash_history
 62 2012-10-04 01:10:16 <gmaxwell> No. But bash history does. :P
 63 2012-10-04 01:10:57 <bcb> just looked they are not in there??
 64 2012-10-04 01:12:19 <bcb> found it
 65 2012-10-04 01:12:28 <gmaxwell> bcb: the .bash_history file isn't written by your shell unless you log out.
 66 2012-10-04 01:12:34 <gmaxwell> bcb: concerned about your passphrase?
 67 2012-10-04 01:13:42 <bcb> yes
 68 2012-10-04 01:15:23 <gmaxwell> https://bitcointalk.org/index.php?topic=54671.0
 69 2012-10-04 01:16:41 <gmaxwell> "Use the CLI w/ encryption? Protect your passphrase with bash's HISTIGNORE"
 70 2012-10-04 01:17:28 <bcb> I'm also trying to find a raw transaction I created to send to the network
 71 2012-10-04 01:18:09 <gmaxwell> ah, the _output_ won't be in the history... so the final signed transaction won't be there unless you already sent it once
 72 2012-10-04 01:28:28 <bcb> gmaxwell:  I didn't send it and I exited the shell
 73 2012-10-04 01:28:54 <bcb> how would I now transmit that to the network
 74 2012-10-04 01:29:22 <bcb> I guess i could sign it again
 75 2012-10-04 01:29:29 <gmaxwell> bcb: yep
 76 2012-10-04 01:31:39 <bcb> code":-22,"message":"TX rejected"
 77 2012-10-04 01:32:03 <bcb> I created both sigs from the same bitoind
 78 2012-10-04 02:09:21 <bcb> gmaxwell: nice: HISTIGNORE
 79 2012-10-04 02:10:26 <bcb> gmaxwell: if multisig escrow does not complete how do you retrieve coins sent to escrow address
 80 2012-10-04 02:11:15 <gmaxwell> precompute a locked refund before announcing the escrow; or have a mediator so you can always complete it.
 81 2012-10-04 02:13:51 <bcb> how would you precompute a locked refund
 82 2012-10-04 02:16:12 <maaku> non-final sequence number and lock time in the future
 83 2012-10-04 02:19:14 <gmaxwell> bcb: you write the escrow transaction but don't announce it, now having its txout you write the refund... a txn both of you sign that returns the funds but it has a non-final sequence number and lock time in the future.
 84 2012-10-04 02:24:08 <stamit> thank you for thanking me for joining the channel...
 85 2012-10-04 04:09:45 <robbak> I've been handed the job of maintainer for the FreeBSD port for bitcoin. The port is currently locked to BDB version 4.7, which I think is wrong.
 86 2012-10-04 04:10:37 <robbak> The system for selecting BDB versions is flexable, defaulting to what is available. What are the reccomended BDB versions?
 87 2012-10-04 04:21:57 <wumpus> 4.8
 88 2012-10-04 04:22:35 <wumpus> using 5.1 is not recommended because the format is incompatible with 4.8, and the binary distributions (on linux and windows) are linked against 4.8
 89 2012-10-04 04:24:37 <robbak> OK. Multiple BDB versions coexist happily on BSD. Is there any reasons not to allow it to build on 4.7 or even earlier?
 90 2012-10-04 04:35:41 <wumpus> it works with at least 4.7 and 4.8, earlier I don't know, that's untested and thus not recommended
 91 2012-10-04 04:38:29 <robbak> OK. I'll set it to 4.7+ in the next update.
 92 2012-10-04 04:39:35 <wumpus> 4.7..4.8 I suppose?
 93 2012-10-04 04:39:54 <maaku> can a 4.8 database run on 4.7?
 94 2012-10-04 04:41:04 <wumpus> don't know
 95 2012-10-04 04:41:19 <wumpus> as I remember, the problems began with 5.x, but I'm by no means a bdb expert
 96 2012-10-04 04:42:14 <robbak> Yes, that's right. The BDB system seems to only deal with 4.x versions, so 4.7+ will build with 4.7 or 4.8, which ever the system has installed.
 97 2012-10-04 04:42:52 <robbak> Don't know about 4.7-4.8 compatability, though. I can find out easily - hold on
 98 2012-10-04 04:43:03 <maaku> yes, but the question is: if I have a wallet backed up from a windows machine (where bitcoind is hard-linked against 4.8), will it work on your 4.7 FreeBSD port?
 99 2012-10-04 04:44:33 <robbak> Answer is no. I just loaded my 4.8 version testnet dir with the 4.7 client and it bombed with a Runaway exception 22DBRunRecoveryException
100 2012-10-04 04:45:21 <wumpus> ok... so better to fix it to 4.8
101 2012-10-04 04:45:28 <robbak> OK, will do.
102 2012-10-04 12:20:25 <MC1984> heres something useful the bitcoin foundation can do
103 2012-10-04 12:20:35 <MC1984> standardise a new non-shitty logo
104 2012-10-04 12:21:02 <MC1984> and an ASCII symbol that isnt already in use by a nation-state on the markets
105 2012-10-04 12:21:05 <kjj_> and by logo, you mean a text glyph, right?
106 2012-10-04 12:21:24 <MC1984> both
107 2012-10-04 12:21:52 <gavinandresen> MC1984: do you like the Foundation's logo?  (the linked square-circles thing)
108 2012-10-04 12:21:52 <MC1984> use the B with the strikethrough, and ditch that shitty old circular logo from 2009
109 2012-10-04 12:22:31 <MC1984> gavinandresen i have no idea what that logo is supposed to mean
110 2012-10-04 12:23:02 <MC1984> does it represent the chain?
111 2012-10-04 12:23:05 <gavinandresen> it's a very abstract reference to the block chain
112 2012-10-04 12:23:24 <MC1984> two mutually recursive basic geometries linked at the corners
113 2012-10-04 12:23:26 <gavinandresen> I don't know nuthin about designing logos.....
114 2012-10-04 12:23:28 <MC1984> oh im right lol
115 2012-10-04 12:23:48 <gavinandresen> ... but that one was designed by professionals (Hansen Belyea)
116 2012-10-04 12:24:27 <MC1984> ive seen on of the forums
117 2012-10-04 12:24:30 <MC1984> i iked
118 2012-10-04 12:24:33 <epscy> did they offer a money back guarantee?
119 2012-10-04 12:24:36 <epscy> i kid
120 2012-10-04 12:24:49 <gavinandresen> epscy: bitcoin-back guarantee... (I'm pretty sure we paid them in bitcoin!)
121 2012-10-04 12:26:05 <thepok> the logo is boring
122 2012-10-04 12:26:23 <thepok> and no connection to bitcoin
123 2012-10-04 12:26:56 <MC1984> gavinandresen ill be honest first time i went to bitcoinfoundation.org, i thought it was a generic parked domain page
124 2012-10-04 12:27:02 <MC1984> purely visually, without reading
125 2012-10-04 12:27:09 <epscy> burn
126 2012-10-04 12:27:12 <Eliel> :D
127 2012-10-04 12:27:16 <thepok> no recall value
128 2012-10-04 12:27:23 <Eliel> I can see how it would look like that.
129 2012-10-04 12:27:24 <epscy> i like the design of the site
130 2012-10-04 12:27:36 <thepok> its about the logo
131 2012-10-04 12:27:41 <epscy> most bootstrap sites have that look about though unfortunately
132 2012-10-04 12:28:17 <MC1984> just my opinion man
133 2012-10-04 12:28:54 <Eliel> there's so little information on the front page that it could easily seem like just a placeholder.
134 2012-10-04 12:29:00 <gavinandresen> okey doke.  I don't get involved in website aesthetic discussions, I have the artistic ability of a cockroach.
135 2012-10-04 12:29:28 <MC1984> its not amazingly important right now
136 2012-10-04 12:29:49 <epscy> to be fair, it's nicer than the linux foundation site
137 2012-10-04 12:29:53 <gavinandresen> It's all my parents fault, I only had the 8-color crayon box growing up.  If they'd given me the 64-color box with the sharpener I'd be a graphic design whiz....
138 2012-10-04 12:30:14 <Eliel> does someone keep a public record of the historical market depth at mtgox by the way?
139 2012-10-04 12:30:16 <epscy> "your parents, they fuck you up"
140 2012-10-04 12:30:27 <epscy> Eliel: bitcoincharts
141 2012-10-04 12:30:37 <epscy> though i am not sure how far back they go now
142 2012-10-04 12:30:44 <Eliel> epscy: market depth too? not just the exchange rate?
143 2012-10-04 12:30:52 <epscy> oh
144 2012-10-04 12:30:54 <epscy> not sure
145 2012-10-04 12:31:15 <epscy> Eliel: to be honest i am not sure of the value of that data
146 2012-10-04 12:31:32 <epscy> a lot of orders are fake
147 2012-10-04 12:31:40 <epscy> and get pulled quickly
148 2012-10-04 12:31:41 <jarpiain> Eliel: bitcoinx.com/charts
149 2012-10-04 12:33:15 <Eliel> jarpiain: yes, I'm aware of that graph. I'd however like to get a) more history and b) the data as numbers.
150 2012-10-04 12:34:23 <epscy> Eliel: that's probably a very large amount of data
151 2012-10-04 12:34:39 <Eliel> epscy: yes, I expect it is.
152 2012-10-04 12:34:51 <epscy> I doubt there is a copy of that outside of mtgoxs database and backups
153 2012-10-04 12:35:07 <epscy> perhaps if you ask MagicalTux nicely
154 2012-10-04 12:44:06 <MC1984> idea: when syncing the chain, what if bitcoin downloaded a chunk of blocks as fast as possible directly to RAM and then started verifying them and building the local database then flush to disk, from there which is shit tons faster than doing it all from disk
155 2012-10-04 12:44:14 <MC1984> say 500 block chunks
156 2012-10-04 12:44:23 <MC1984> repeat until finished
157 2012-10-04 12:44:26 <MC1984> is that stupid
158 2012-10-04 12:44:52 <epscy> depends on how much ram the user has
159 2012-10-04 12:45:18 <epscy> generally the OS tries to use RAM as efficiently as possible
160 2012-10-04 12:45:37 <MC1984> well say 40$ of whatever the machine has
161 2012-10-04 12:45:39 <kjj_> If you've got 4 gigs free for a ramdisk, I'm pretty sure it is quick
162 2012-10-04 12:45:41 <MC1984> 40%
163 2012-10-04 12:45:47 <epscy> and sometimes the bottleneck is the verifying which is cpu bound
164 2012-10-04 12:46:02 <MC1984> currently it is often not
165 2012-10-04 12:46:17 <MC1984> many machines have multi gigabytes of memory now
166 2012-10-04 12:46:42 <MC1984> why not process raw blocks into the local database totally in ram, and then flush out to disk and do it again
167 2012-10-04 12:47:01 <sipa> Eliel: i have a lot of historic mtgox data
168 2012-10-04 12:47:05 <epscy> could end up being worse in terms of IO
169 2012-10-04 12:47:48 <MC1984> the disk flush would be a contiguous write of a chunk of the DB, instead of actually building the DB on the disk
170 2012-10-04 12:48:22 <MC1984> so its like farming off the random access bit of DB building to the RAM which seems like a good idea
171 2012-10-04 12:48:45 <Eliel> sipa: could you send it to me?
172 2012-10-04 12:49:59 <Eliel> MC1984: ultraprune database in memory is probably the most efficient way to do that, assuming you have enough RAM.
173 2012-10-04 12:50:07 <kjj_> MC1984: that isn't a general solution, but for people that know what they are doing and have enough RAM, they can cut hours off their IBD
174 2012-10-04 12:50:35 <MC1984> why cant bitcoin just take care of it automatically
175 2012-10-04 12:50:57 <gmaxwell> MC1984: the operating system already does what you're thinking about there.
176 2012-10-04 12:51:22 <MC1984> does it
177 2012-10-04 12:51:27 <gmaxwell> Yes.
178 2012-10-04 12:51:43 <kjj_> gmaxwell: no, it doesn't.  when you sync, the OS waits for the disk layer to accept the write.
179 2012-10-04 12:51:52 <gmaxwell> The OS uses all the unallocated memory for caching and then reads are satisfied out of it.
180 2012-10-04 12:52:03 <MC1984> why is random write performance still a factor then
181 2012-10-04 12:52:21 <gmaxwell> MC1984: because it's slow.
182 2012-10-04 12:52:26 <gmaxwell> And it still has to get done.
183 2012-10-04 12:52:48 <gmaxwell> The data on disk needs to be consistent at all points because you could get shut down at any point.
184 2012-10-04 12:53:45 <MC1984> am i being stupid
185 2012-10-04 12:53:52 <kjj_> gmaxwell: that's the killer point, really.  when my RAM-only nodes download, they don't care about having consistent state at every point.  that's why I say it isn't a general solution
186 2012-10-04 12:55:55 <kjj_> I did an IBD on a spare windows box with 8 GB RAM, nothing else in particular running.  the disk churn was amazing
187 2012-10-04 12:56:06 <sipa> kjj_: with ultraprune?
188 2012-10-04 12:56:10 <sipa> and/or leveldb?
189 2012-10-04 12:56:17 <kjj_> no, standard 0.7.0
190 2012-10-04 12:56:28 <gmaxwell> kjj_: sure, it's doing lots of random sync writes.
191 2012-10-04 12:56:40 <gmaxwell> But no amount of caching can fix that.
192 2012-10-04 12:56:58 <kjj_> doing it totally in RAM, tmpfs on a linux box, and then copying it to disk at the end is many hours faster, even with a shitty CPU
193 2012-10-04 12:57:34 <kjj_> but that isn't a general solution.  we can't package up a RAMdisk driver for Windows and tell people to have 8+ GB in their machines
194 2012-10-04 12:57:41 <MC1984> 02?3
195 2012-10-04 12:57:58 <MC1984> fuck how do i input ascii codes
196 2012-10-04 12:58:02 <gmaxwell> kjj_: In any case, reports on 0.7 are not longer interesting; we know the bdb backend is slow.
197 2012-10-04 12:58:04 <sipa> kjj_: afaik, with ultraprune+leveldb there is no significant difference (minutes at most) between doing it in tmpfs or on disk
198 2012-10-04 12:58:32 <sipa> given enough RAM to cache everything, of course
199 2012-10-04 12:59:07 <sipa> though it's been a while since i did tmpfs benchmarks
200 2012-10-04 12:59:13 <MC1984> ??
201 2012-10-04 12:59:21 <MC1984> this one
202 2012-10-04 12:59:22 <kjj_> heh, I wasn't reporting it to you, I was telling MC1984 why we can't make the sync faster using the current system
203 2012-10-04 12:59:47 <MC1984> ?? we cant just steal thailands symbol
204 2012-10-04 12:59:48 <kjj_> I look forward to the new speed improvements
205 2012-10-04 13:02:50 <epscy> will ultraprune reduce the size of the blockchain?
206 2012-10-04 13:03:04 <epscy> and if so do we know by how much?
207 2012-10-04 13:03:13 <helo> the blockchain is still the same size
208 2012-10-04 13:03:31 <helo> it is just a way of not including parts of the blockchain that aren't necessary
209 2012-10-04 13:03:39 <epscy> i thought the point of pruning was to make it smaller
210 2012-10-04 13:03:50 <epscy> helo: oh i see, you are being pedantic
211 2012-10-04 13:04:22 <helo> it allows clients to deal with less data, but the blockchain itself is the same... i.e. the block hashes will not be computed based on the ultrapruned data, but the full chain
212 2012-10-04 13:04:34 <sipa> epscy: ultraprune does not do pruning, but it supports it
213 2012-10-04 13:05:12 <epscy> sipa: oh ok, so when UP is released clients won't immediately see a reduction in the size of their blockchain?
214 2012-10-04 13:05:12 <sipa> epscy: that is, implementing it would be trivial on top of it, but requires a few architectural things first, like new service bits on the network to make sure nodes don't try to bootstrap from pruned nodes
215 2012-10-04 13:05:23 <sipa> the default will certainly also be non-pruning
216 2012-10-04 13:05:31 <epscy> i see
217 2012-10-04 13:05:50 <epscy> so we don't really have a solution to the large blockchain problem yet
218 2012-10-04 13:05:56 <sipa> anyway, to give you a number: with pruning, the size requirement is something like 150 MB + 1.1*size_of_retained_blocks
219 2012-10-04 13:06:10 <sipa> where you choose the number or size of the retained blocks yourself
220 2012-10-04 13:06:21 <sipa> however, you cannot rescan or serve pruned blocks
221 2012-10-04 13:06:33 <helo> epscy: the solution is for clients to only keep track of the list of unspent outputs, instead of all transaction history
222 2012-10-04 13:06:40 <kjj_> any chance of the ultraprune system allowing concurrent access to the database?  I would love for my server to only need one copy of the chain
223 2012-10-04 13:06:45 <sipa> and if a reorg larger than your window of non-pruned blocks happens, you node basically dies and needs to restart from scratch
224 2012-10-04 13:06:54 <helo> epscy: the trick is ensuring this list corresponds to the actual blockchain
225 2012-10-04 13:06:56 <epscy> helo: does that have security implications?
226 2012-10-04 13:07:04 <sipa> kjj_: i hope so, but not immediatelty
227 2012-10-04 13:07:46 <kjj_> sipa: cool.  in the mean time, I'm looking forward to running one full chain and one as heavily pruned as possible
228 2012-10-04 13:07:57 <helo> epscy: if done properly (hash tree in coinbase), the security is still very solid
229 2012-10-04 13:08:28 <sipa> helo: with a hash tree in the coinbase, you can do a SPV-level trust instant restart (without processing or downloading the history)
230 2012-10-04 13:08:37 <sipa> this is NOT the same as the current zero-trust level we have
231 2012-10-04 13:09:09 <helo> zero level trust?
232 2012-10-04 13:09:24 <helo> oh, right...
233 2012-10-04 13:09:28 <gmaxwell> helo: validating everything for yourself.
234 2012-10-04 13:09:50 <gmaxwell> So it's not a replacement. But a middle step we don't have.
235 2012-10-04 13:10:30 <phantomcircuit> lol
236 2012-10-04 13:10:37 <helo> the plan is still to use txout-tree-in-coinbase at first, and then slowjam the entire blockchain verification?
237 2012-10-04 13:10:37 <phantomcircuit> pycryptopp has an ecdsa class
238 2012-10-04 13:10:40 <sipa> anyway, having a pruned node which you *know* validated the entire history has exactly the same security guarantee as what we have now
239 2012-10-04 13:10:41 <phantomcircuit> no mention of key length
240 2012-10-04 13:10:46 <phantomcircuit> no mention of which curve it is
241 2012-10-04 13:11:38 <gmaxwell> helo: well, I can't say thats a concrete plan yet, because we don't yet have viable code for determinstic txout trees.
242 2012-10-04 13:12:07 <sipa> helo: in my opinion, we'll first implement headers-sync-first and then full block downloading, which means you start with an SPV-like node, and end up with a potentially-pruned full node
243 2012-10-04 13:12:08 <gmaxwell> Ultraprune structures the node around a txout tree, but uses a conventional database to store them; not a cryptographically certifiable database.
244 2012-10-04 13:12:53 <sipa> once we have utxo-in-coinbase, you can do a shortcut to get a pruned full node, but only at SPV level trust for its history
245 2012-10-04 13:13:07 <epscy> sipa: so when to go back to my original question, how much smaller would a full pruned nodes blockchain be compared to the current blockchain?
246 2012-10-04 13:13:19 <sipa> 17:05:56 < sipa> anyway, to give you a number: with pruning, the size requirement is something like 150 MB + 1.1*size_of_retained_blocks
247 2012-10-04 13:13:34 <epscy> sipa: heh, ok
248 2012-10-04 13:13:47 <sipa> that 150 MB is obviously bound to grow as well
249 2012-10-04 13:14:00 <epscy> so we are still talking about > 2G
250 2012-10-04 13:14:04 <epscy> GB
251 2012-10-04 13:14:09 <gmaxwell> Thats more important futher in the future where catching up could take weeks or months of work even for an efficient implementation. Taking weeks to get to a new full node would kinda suck, utxo tree would allow a node to become 'externally full' very fast.
252 2012-10-04 13:14:10 <sipa> depends how many blocks you want to retain
253 2012-10-04 13:14:29 <sipa> if 10 blocks is enough for you, 200 MB will suffice
254 2012-10-04 13:14:39 <sipa> though i wouldn't recommend that
255 2012-10-04 13:14:39 <stamit> are you enjoying this, gmaxwell ?
256 2012-10-04 13:14:55 <stamit> you seem to be on a roll these days
257 2012-10-04 13:15:17 <MC1984> sipa thats a good idea
258 2012-10-04 13:15:17 <sipa> what's the problem, stamit?
259 2012-10-04 13:15:26 <helo> gmaxwell always explains like a boss
260 2012-10-04 13:15:39 <MC1984> the painful bit of syncing a new node is that its useless until its done
261 2012-10-04 13:15:53 <gmaxwell> MC1984: right, that can be fixed by starting as spv.
262 2012-10-04 13:16:13 <sipa> gtg
263 2012-10-04 13:16:17 <MC1984> if it was useful from minute 1 albeit at a reduced trust level, it could take weeks to sync in the backbround for all i care
264 2012-10-04 13:16:19 <MC1984> its a good plan
265 2012-10-04 13:17:29 <helo> how will the "slow sync" speed be determined?
266 2012-10-04 13:18:00 <gmaxwell> helo: I'd assume it would run as fast as possible in the background unless the users requested otherwise.
267 2012-10-04 13:18:32 <helo> it currently runs as fast as possible, and a lot of users complain about the impact on system responsiveness
268 2012-10-04 13:18:47 <helo> so maybe just a little less quickly :)
269 2012-10-04 13:19:04 <gmaxwell> helo: I believe that is mostly due to the OS having poor handling of thrashy IO load.
270 2012-10-04 13:19:13 <gmaxwell> And it shouldn't be an issue with ultraprune.
271 2012-10-04 13:19:32 <helo> ahh good
272 2012-10-04 13:21:16 <helo> so ultraprune will allow a node to know it has fully verified everything, but what amount of pruning is healthy for the network?
273 2012-10-04 13:21:30 <helo> ideally nobody would prune anything, so everybody would be able to bootstrap a new node
274 2012-10-04 13:21:56 <gmaxwell> I dunno. We actually can't handle pruned nodes at all yet.
275 2012-10-04 13:22:13 <gmaxwell> Because we have no network mechenism to use to tell which nodes are full but can't feed you the whole historic chain.
276 2012-10-04 13:22:25 <gmaxwell> The answer to that question will depend on how we handle that discovery
277 2012-10-04 13:23:00 <helo> so it would be kind of a bittorrent style "partial blocks from many partial copies" approach?
278 2012-10-04 13:23:39 <helo> i guess difference is that bittorrent validates each block separately, but bitcoin has to validate blocks sequentially
279 2012-10-04 13:24:36 <gmaxwell> Yes, that's something that has to be considerd (full validation and downloading would need to be decoupled)... and it can be difficult to create DOS resistance in that kind of system.
280 2012-10-04 13:24:49 <gmaxwell> Bittorrent is basically only survivable because people don't DOS it.
281 2012-10-04 13:25:02 <MC1984> gmaxwell people try
282 2012-10-04 13:25:25 <MC1984> companies exists specifically to fuck with bittorrent
283 2012-10-04 13:25:27 <helo> so pruned nodes in aggregate won't have an even distribution of blocks as with bittorrent... so latter blocks are more at risk for not being found
284 2012-10-04 13:25:58 <MC1984> i do hope pruing remains off by default
285 2012-10-04 13:26:02 <gmaxwell> MC1984: they don't actually screw with the base dht network; doing so would be unlawful, and there are quite a few people who would eagerly begin litigation against those companies.
286 2012-10-04 13:26:16 <helo> sounds like it needs to, but most users would be happy to reduce resource consumption
287 2012-10-04 13:26:22 <helo> (needs to be off by default)
288 2012-10-04 13:26:35 <stamit> gmaxwell, you have a spare $126?
289 2012-10-04 13:26:37 <Luke-Jr> IMO to scale, *eventually* most nodes should (only) randomly prune and verify
290 2012-10-04 13:26:44 <helo> health of network vs happiness of users
291 2012-10-04 13:27:01 <gmaxwell> helo: if you're unhappy an option to turn it off can make you happy
292 2012-10-04 13:27:05 <MC1984> if pruning was default, the actual chain would become endangered maybe
293 2012-10-04 13:27:07 <stamit> my corpse is litigating, gmaxwell
294 2012-10-04 13:27:12 <gmaxwell> helo: the software can also check for available resources.
295 2012-10-04 13:27:50 <helo> thats not a bad idea... "We see you have only 500MB remaining. As much as we'd like you to be a full particpant in the network..."
296 2012-10-04 13:28:01 <gmaxwell> helo: right.
297 2012-10-04 13:28:37 <gmaxwell> helo: and if you still want to crank down your participation even though you have resources left??? you start up with -im-selfish  or so.
298 2012-10-04 13:29:30 <stamit> either $126 or 24 BTC will do: 1B5fLitBtpYrMoyWi2n6wokUT6BBB5SUyq
299 2012-10-04 13:29:48 <helo> it will be nice when the typical pc fully eclipses bitcoin's requirements
300 2012-10-04 13:30:12 <Luke-Jr> ok, I've been ignoring the troll but I really wonder
301 2012-10-04 13:30:20 <Luke-Jr> stamit: why $126 or 24 BTC, when $126 is about 10 BTC?
302 2012-10-04 13:30:40 <stamit> trade took place some time ago
303 2012-10-04 13:30:46 <stamit> price has changed, since that time
304 2012-10-04 13:30:52 <helo> in 10 years we will hopefully be laughing that we spent so much time trying to reduce bitcoin's requirements
305 2012-10-04 13:30:54 <stamit> the right thing is $126
306 2012-10-04 13:31:07 <Luke-Jr> ok, back to ignoring
307 2012-10-04 13:31:08 <stamit> but we can just consider the trade "cancelled" and in that case, i get 24 BTC
308 2012-10-04 13:31:37 <kjj_> stamit: can you please take that elsewhere?  I'm curious about what you are talking about, but whatever it is, itis WAY off topic for this channel
309 2012-10-04 13:31:44 <stamit> this is silly
310 2012-10-04 13:31:52 <helo> ^
311 2012-10-04 13:31:52 <Luke-Jr> kjj_: he's a troll, not sure who unbanned him
312 2012-10-04 13:32:26 <stamit> i ceased being human long ago, i guess
313 2012-10-04 13:32:44 <MC1984> u wot m8
314 2012-10-04 13:32:45 <stamit> didn't pick that
315 2012-10-04 13:35:27 <stamit> i think gmaxwell may be doing me a favor, since, this way i get to chat with people i wouldn't have chatted with otherwise
316 2012-10-04 13:35:47 <stamit> totally misunderstood him
317 2012-10-04 13:35:52 <kjj_> post on the forums or something, but please don't crap up this channel
318 2012-10-04 13:36:26 <gmaxwell> stamit: I'm not doing anything with you except ignoring you.
319 2012-10-04 14:14:55 <epscy> helo: i think there is a good chance that if hardware continues to improve, either bitcoin will grow to use more resources or a better altcoin will crop up that makes use of them
320 2012-10-04 14:15:28 <gmaxwell> epscy: for bitcoin's purposes its important that it be cheap.
321 2012-10-04 14:16:08 <gmaxwell> Otherwise you can't get massive decenteralization of checking??? too much incentive to let 'someone else' handle it??? and the purpose is lost.
322 2012-10-04 14:16:20 <gmaxwell> How cheap it needs to be??? I dunno.
323 2012-10-04 14:23:44 <MC1984> the bitcoins forums have gone full retard
324 2012-10-04 14:23:55 <kjj_> yeah, welcome to 2011
325 2012-10-04 14:23:55 <MC1984> at some point
326 2012-10-04 14:23:57 <gmaxwell> hm?
327 2012-10-04 14:24:02 <MC1984> i dont think theyll ever come back
328 2012-10-04 14:24:35 <Eliel> MC1984: there's still people writing there who're not retards but it's getting more difficult to pick them out from the crowd.
329 2012-10-04 14:27:10 <epscy> thye just have a lot of strong opinions
330 2012-10-04 14:28:00 <phantomcircuit> Eliel, s/more difficult/impossible/
331 2012-10-04 15:42:23 <helo> man, i'm so far behind on the ml
332 2012-10-04 16:53:05 <MaxSan> anyone had any issues getting bitcoinjs-server installed?
333 2012-10-04 17:41:31 <Joric> sipa, how many addresses with a nonzero balance did you get after pruning?
334 2012-10-04 17:41:52 <sipa> Joric: i never calculated that value
335 2012-10-04 17:42:09 <phantomcircuit> so im looking at pynode
336 2012-10-04 17:42:15 <gmaxwell> Joric: ultraprune doesn't calculate balances. Did you mean to ask how many zero value txouts there are?
337 2012-10-04 17:42:30 <phantomcircuit> the ecdsa implementation it's using is about 100x slower than equivalent c implementations
338 2012-10-04 17:43:01 <Joric> a number of txouts can be much bigger than that
339 2012-10-04 17:43:18 <sipa> Joric: yes, so?
340 2012-10-04 17:43:29 <sipa> there are 2414144 unspent transaction outputs, in 1200786 not-fully spent transactions
341 2012-10-04 17:44:08 <sipa> sec, let me calculate the different addresses
342 2012-10-04 17:44:12 <gmaxwell> phantomcircuit: search the logs here for me cursing python for breaking amiller's reasoning about software complexity. :P
343 2012-10-04 17:44:29 <phantomcircuit> lol
344 2012-10-04 17:44:59 <phantomcircuit> sipa, what % of the tx outs can be pruned?
345 2012-10-04 17:45:00 <helo> what reasoning was it?
346 2012-10-04 17:45:42 <sipa> phantomcircuit: 80% or more, afaik
347 2012-10-04 17:45:59 <gmaxwell> helo: I thought he was spending an unhealthy amount of time worrying about how to avoid script validation (which includes the signature checks, I assume) in his efforts to make pynode faster.
348 2012-10-04 17:47:29 <gmaxwell> helo: e.g. he wanted to do stuff like having users manally provide checkpoints so they could skip more validation.  This is not generally an idea I like because it creates stupid behavior opportunity.
349 2012-10-04 17:47:34 <phantomcircuit> gmaxwell, too be fair script validiation is the bulk of the work
350 2012-10-04 17:47:35 <phantomcircuit> :)
351 2012-10-04 17:47:53 <sipa> after recent optimizations it certainly is
352 2012-10-04 17:48:10 <phantomcircuit> i meant in terms of coding effort
353 2012-10-04 17:48:16 <sipa> oh, right
354 2012-10-04 17:48:26 <sipa> not sure; block connection logic can be tricky
355 2012-10-04 17:48:27 <gmaxwell> For an ultraprune-ish system it is??? sure.
356 2012-10-04 17:48:44 <sipa> but script validation requires an unholy amount of testing
357 2012-10-04 17:49:02 <gmaxwell> phantomcircuit: what he wanted to do was 'solve' the performance by having users provide checkpoint values to just skip more of it. Which doesn't address any of the testing concerns.
358 2012-10-04 17:49:23 <gmaxwell> So there, you probably just gave him a 50x speedup in script validation. :P
359 2012-10-04 17:49:32 <stamit> you still talking here?
360 2012-10-04 17:49:40 <stamit> why aren't you banned yet?
361 2012-10-04 17:49:59 <stamit> someone didn't find me on -otc and he thought i wasn't connected
362 2012-10-04 17:50:07 <stamit> you caused me trouble today, gmaxwell
363 2012-10-04 17:51:50 <sipa> Joric: 909461 different txout scripts after pruning
364 2012-10-04 17:52:03 <Joric> cool less than a million
365 2012-10-04 17:52:06 <gmaxwell> Sorry about him. Unfortunately I didn't do anything to deserve this except ban him from -OTC weeks ago because he was earnestly offering people bounties for attack paypal employees. :( Now I'm apparently his preferred target for everything that goes wrong in his life.
366 2012-10-04 17:52:24 <phantomcircuit> gmaxwell, heh
367 2012-10-04 17:52:51 <sipa> Joric: that does include txouts with amount=0, so the number of addresses with nonzero balance may be slightly less
368 2012-10-04 17:52:55 <gmaxwell> sipa: higher reuse factor than I expected.
369 2012-10-04 17:52:58 <sipa> indeed
370 2012-10-04 17:53:04 <sipa> same here
371 2012-10-04 17:53:26 <phantomcircuit> people are lazy
372 2012-10-04 17:53:29 <phantomcircuit> also
373 2012-10-04 17:53:39 <phantomcircuit> satoshidice uses the same addresses over and over again
374 2012-10-04 17:53:54 <sipa> so do almost all personal donations (my own included)
375 2012-10-04 17:54:16 <Joric> ~20 megs considering 20 bytes hash + 4 bytes value
376 2012-10-04 17:54:25 <sipa> what 4 bytes value?
377 2012-10-04 17:55:55 <Joric> oh good thinking it's not needed really
378 2012-10-04 17:56:40 <sipa> not sure what use is a list of all used addresses?
379 2012-10-04 17:57:05 <Joric> i just stumbled upon https://sites.google.com/site/buriedkeys/home was wondering how much memory i'll need for bruteforcing
380 2012-10-04 17:57:50 <gmaxwell> you probably need far less memory.
381 2012-10-04 17:58:50 <gmaxwell> for a million entries and a error rate of only 1e-5 you only need a bloom filter of about 3MB.
382 2012-10-04 17:59:20 <stamit> hey, stupid freak, gmaxwell, how are you today?
383 2012-10-04 17:59:34 <stamit> how are you in your cave?
384 2012-10-04 17:59:53 <stamit> that's how you called me, gmaxwell
385 2012-10-04 17:59:56 <stamit> "stupid" and "freak"
386 2012-10-04 17:59:59 <JFK911> im sure gmaxwell's cia cave is quite comfy
387 2012-10-04 18:00:11 <arij> gmaxwell is CIA?
388 2012-10-04 18:00:15 <arij> 
ACTION posts on the forums

389 2012-10-04 18:00:15 <JFK911> no doubt
390 2012-10-04 18:00:15 <stamit> and i got the cave talk recently too
391 2012-10-04 18:00:31 <Joric> i'm the FBI then
392 2012-10-04 18:00:37 <arij> no one cares
393 2012-10-04 18:00:37 <D34TH> im CFIP
394 2012-10-04 18:00:44 <JFK911> really joric what fbi base do you live on
395 2012-10-04 18:00:49 <stamit> i wonder how much is a CIA salary
396 2012-10-04 18:00:50 <gmaxwell> JFK911: technically it's not a "cave", it's a bunker.
397 2012-10-04 18:01:06 <wizkid057> ...
398 2012-10-04 18:02:22 <phantomcircuit> this will be entertaining
399 2012-10-04 18:02:27 <gmaxwell> :(
400 2012-10-04 18:02:48 <gmaxwell> ("not my kind of entertainment")
401 2012-10-04 18:03:00 <phantomcircuit> i have myself a nice cup of tea
402 2012-10-04 18:03:03 <phantomcircuit> and a giant ban hammer
403 2012-10-04 18:03:16 <phantomcircuit> but instead im just going to kick him around with fun messages
404 2012-10-04 18:03:29 <Joric> oh, addresses probably can be filtered by the date of appearance that will reduce the amount of data
405 2012-10-04 18:04:08 <gmaxwell> Joric: so what I'd do is just take windows of the hash value and use those to set bits in a bloom filter. A 3MB bloom filter will give you a very low hitrate, and it will require only one memory look up for most exclusions.
406 2012-10-04 18:41:27 <helo> does -loadblock empty the previous blk*.dat?
407 2012-10-04 18:41:49 <sipa> no
408 2012-10-04 18:42:28 <sipa> it really just imports blocks from a file
409 2012-10-04 18:42:38 <sipa> as if they were received by network
410 2012-10-04 18:43:17 <helo> ty
411 2012-10-04 18:50:25 <Joric> what's in blkindex.dat? tx_hash -> tx_offset? maybe it's possible to avoid redownloading blockchain without using -loadblock?
412 2012-10-04 18:51:47 <sipa> Joric: blkindex.dat is the thing that is hard to compute; it contains basically everything
413 2012-10-04 18:51:53 <sipa> tx_hash -> tx_offset
414 2012-10-04 18:51:59 <sipa> block_hash -> block_offset
415 2012-10-04 18:52:22 <sipa> and most of all, tx_hash -> [tx_offsets_of_transactions_that_spent_its_outputs]
416 2012-10-04 18:59:12 <Joric> miners must pay 50% of block reward to sipa for improving all that satoshi shit
417 2012-10-04 19:00:03 <CrazyMF> say what?
418 2012-10-04 19:00:22 <CrazyMF> what's sipa?
419 2012-10-04 19:00:51 <edcba> a banking agreement in europe iirc :)
420 2012-10-04 19:01:06 <edcba> also sipa is sipa on this channel...
421 2012-10-04 19:01:09 <eb3kk> wow
422 2012-10-04 19:01:36 <Eliel> CrazyMF: no, the banking agreement is SEPA.
423 2012-10-04 19:01:43 <CrazyMF> so what exactly 50% block reward has to do with SEPA ?
424 2012-10-04 19:01:50 <edcba> lol
425 2012-10-04 19:02:10 <CrazyMF> I'm ref'ing to what Joric said
426 2012-10-04 19:03:09 <sipa> 
ACTION is sipa

427 2012-10-04 19:03:30 <CrazyMF> and keep it that way
428 2012-10-04 19:03:35 <CrazyMF> no way i'm giving you 50%
429 2012-10-04 19:04:01 <sipa> and you shouldn't
430 2012-10-04 19:04:05 <BlueMatt> yes he should
431 2012-10-04 19:04:21 <edcba> sipa will just forget to give you bitcoins :)
432 2012-10-04 19:06:10 <gmaxwell> 14:04 < sipa> and you shouldn't
433 2012-10-04 19:06:24 <gmaxwell> Right, you should give him 75%
434 2012-10-04 19:08:19 <Joric> the speed of sync is ludicrous atm and it's only 2012
435 2012-10-04 19:08:50 <BlueMatt> use a threaded sig checking branch
436 2012-10-04 19:09:02 <BlueMatt> bitcoinj/ultraprune/etc
437 2012-10-04 19:10:37 <Joric> does sig checking really affects speed so much? is there a key to skip it? i wonder how faster it will be
438 2012-10-04 19:11:05 <edcba> i guess the problem is how it retrieve data and check sigs
439 2012-10-04 19:11:14 <edcba> ie alternating retrieve data then check
440 2012-10-04 19:11:16 <BlueMatt> sig checking (after the latest checkpoint) is the speed
441 2012-10-04 19:11:28 <sipa> Joric: in ultraprune on a not ridiculously slow disk, it's like 90%
442 2012-10-04 19:11:32 <BlueMatt> (unless you are doing something stupid like putting .bitcoin on a flash drive or something)
443 2012-10-04 19:11:58 <sipa> BlueMatt: parallel signature checking is for now not in ultraprune, by the way
444 2012-10-04 19:12:09 <BlueMatt> oh, thought it was in that pull...
445 2012-10-04 19:12:11 <MC1984> itsaconspiracy.jpg
446 2012-10-04 19:12:24 <sipa> there were some problems, and i want ultraprune sooner than i may have time to fix parallel sig checking
447 2012-10-04 19:12:35 <sipa> basically, i first want to refactor some things
448 2012-10-04 19:12:52 <jgarzik> gavinandresen sipa: I thought there was a better way to recover thoroughly corrupt BDB databases than dump + restore?
449 2012-10-04 19:12:52 <MC1984> bitcoinforums
450 2012-10-04 19:12:59 <Joric> i wonder does bitcoinfoundation pay the salary now
451 2012-10-04 19:13:05 <BlueMatt> heh...refactor for parallel sig checking
452 2012-10-04 19:13:42 <sipa> BlueMatt: no, i want a global block index cache, which keeps dirty blocks and indexes in memory, and flushes them periodically ad synchronously
453 2012-10-04 19:13:44 <jgarzik> see DB_RECOVER_FATAL
454 2012-10-04 19:13:46 <jgarzik> etc.
455 2012-10-04 19:14:13 <jgarzik> http://docs.oracle.com/cd/E17076_02/html/programmer_reference/transapp_recovery.html
456 2012-10-04 19:14:14 <sipa> BlueMatt: that's the only way i know to make bitcoin fully consistent on disk during IBD
457 2012-10-04 19:14:31 <jgarzik> Joric: no
458 2012-10-04 19:14:49 <sipa> maybe gavin's, but not yet, afaik
459 2012-10-04 19:15:31 <BlueMatt> sipa: is that a leveldb problem or...?
460 2012-10-04 19:16:39 <sipa> with?
461 2012-10-04 19:17:00 <sipa> no bitcoin is currently not disk-consistent at every point in time during IBD
462 2012-10-04 19:17:01 <BlueMatt> that's the only way i know to make bitcoin fully consistent on disk during IBD
463 2012-10-04 19:17:08 <BlueMatt> really? where do we break that?
464 2012-10-04 19:17:29 <sipa> we don't sync after every block write during IBD (it'd kill performance completely)
465 2012-10-04 19:17:43 <sipa> so there is a chance blkindex.dat is updated but the block is never written to disk
466 2012-10-04 19:17:58 <BlueMatt> oh, yea, dur...
467 2012-10-04 19:18:22 <sipa> ultraprune+leveldb improves that (it syncs a bit more), but it also makes the problem harder to solve as there are 2 leveldb database that can get out of sync with eachother
468 2012-10-04 19:18:50 <sipa> such a block cache would both improve performance (block validation doesn't need to wait for it to be written to disk), and consistency
469 2012-10-04 19:19:00 <sipa> and it would make parallel sig checking a lot easier to
470 2012-10-04 19:19:05 <sipa> so i want that first
471 2012-10-04 19:19:09 <BlueMatt> fair enough
472 2012-10-04 19:22:10 <Eliel> was it Slush who built the first mining pool?
473 2012-10-04 19:22:20 <jgarzik> Eliel: first public one, I think so
474 2012-10-04 19:22:50 <jgarzik> sipa: I forget... did you turn on leveldb sync'ing?  Did that kill performance?
475 2012-10-04 19:22:59 <jgarzik> default for leveldb sync is off, iirc
476 2012-10-04 19:23:16 <sipa> jgarzik: i sync leveldb right after syncing the block file
477 2012-10-04 19:23:40 <jgarzik> sipa: fdatasync under the hood, for both, I presume?
478 2012-10-04 19:23:42 <jgarzik> (on Linux)
479 2012-10-04 19:23:55 <phantomcircuit> jgwhat % of the time would you say pynode is spending in ecdsa ops?
480 2012-10-04 19:24:17 <sipa> jgarzik: yes, but that's not under our control for leveldb (it always does fdatasync when available)
481 2012-10-04 19:24:40 <gavinandresen> jgarzik: DB_RECOVER_FATAL doesn't work for the cases I care about (copied .dat files without -detachdb or running incompatible version of BDB)
482 2012-10-04 19:25:20 <sipa> Eliel: puddinpop had an earlier pool, but it used a very different mechanism
483 2012-10-04 19:25:34 <sipa> Eliel: slush's pool was definitely the first to use the concept of shares
484 2012-10-04 19:25:42 <Eliel> how did puddinpop's pool work?
485 2012-10-04 19:26:10 <Eliel> do you have a link to bitcointalk.org thread from the time?
486 2012-10-04 19:26:42 <jgarzik> phantomcircuit: 110%
487 2012-10-04 19:26:48 <jgarzik> phantomcircuit: just kidding.
488 2012-10-04 19:27:01 <jgarzik> phantomcircuit: python sig checking cost is 90% data structure copying, not ECDSA
489 2012-10-04 19:27:17 <jgarzik> gavinandresen: ok
490 2012-10-04 19:28:00 <jgarzik> phantomcircuit: SignatureHash requires copying then modifying CTransaction.  python is achingly slow at this.
491 2012-10-04 19:28:32 <phantomcircuit> using copy.copy?
492 2012-10-04 19:28:42 <jgarzik> phantomcircuit: I've tried all sorts of hacks...  cPickle the CTransaction, then un-pickle during SignatureHash.  copy, deepcopy, ...
493 2012-10-04 19:28:51 <jgarzik> phantomcircuit: the fastest so far is manually coded copy
494 2012-10-04 19:40:45 <Eliel> It's interesting reading the conversation between tcatm and puddinpop at the beginning of this thread :) https://bitcointalk.org/index.php?topic=1458.0
495 2012-10-04 19:41:48 <jgarzik> Eliel: Historical trivia:  I gave puddinpop 10,000 BTC to open source his code
496 2012-10-04 19:42:08 <jgarzik> not worth it, in hindsight, as he largely disappeared rather than continuing to contribute to the community
497 2012-10-04 19:43:26 <Eliel> this metahash stuff he's using will probably need some pondering to understand. Probably not worth the time :)
498 2012-10-04 19:45:50 <Ferroh> bitcoind is switching from BerkelyDB to something else, right?
499 2012-10-04 19:46:45 <Eliel> Ferroh: correct
500 2012-10-04 19:46:56 <Ferroh> is this primarily due to performance concerns?
501 2012-10-04 19:47:12 <MC1984> jgarzik dude thats 130k dollars now
502 2012-10-04 19:48:29 <jgarzik> Ferroh: leveldb, for reasons of performance, stability across versions, ...
503 2012-10-04 19:48:47 <sipa> and general 21st-century-ness
504 2012-10-04 19:48:48 <jgarzik> though if we had imported bdb into the source tree, we would get much of the time benefit ;p
505 2012-10-04 19:49:00 <jgarzik> s/time/same/
506 2012-10-04 19:49:43 <Eliel> yes, but a lot more code to import.
507 2012-10-04 19:49:52 <sipa> and not the performance improvements
508 2012-10-04 19:51:21 <Ferroh> are you saying that leveldb will be imported into the source tree?
509 2012-10-04 19:51:53 <gmaxwell> Yes.
510 2012-10-04 19:53:19 <sipa> it's around 18k lines of code
511 2012-10-04 19:53:36 <sipa> bitcoin itself is twice that
512 2012-10-04 19:53:46 <Ferroh> yeah that's very lightweight
513 2012-10-04 19:54:48 <Ferroh> oh, this isn't an SQL database
514 2012-10-04 19:55:00 <Ferroh> sorry, I'm just reading about leveldb for the first time now
515 2012-10-04 19:56:08 <Ferroh> are you sure this is even 18k lines?
516 2012-10-04 19:56:13 <sipa> it's a key-value store
517 2012-10-04 19:56:14 <Ferroh> are you including the test code in that 18k?
518 2012-10-04 19:56:17 <sipa> yes
519 2012-10-04 19:56:23 <Joric> will it replace berkeley db?
520 2012-10-04 19:56:29 <Ferroh> the unit tests are a massive chunk of the code,
521 2012-10-04 19:56:35 <Ferroh> so you probably dont need to include that
522 2012-10-04 19:56:49 <sipa> Joric: at least for blkindex.dat
523 2012-10-04 19:56:59 <sipa> wallet.dat is a different beast
524 2012-10-04 19:57:13 <Joric> oh right sorry i forgot about wallet
525 2012-10-04 19:57:13 <maaku> wait, why import leveldb into bitcoin source tree?
526 2012-10-04 19:57:23 <maaku> why not static link? am i missing something?
527 2012-10-04 19:57:35 <Joric> good point
528 2012-10-04 19:58:20 <Ferroh> maaku, presumably so that users are guaranteed to have the exact version of leveldb that the bitcoindevs are using?
529 2012-10-04 19:58:44 <maaku> that's why you static link against a specific version
530 2012-10-04 19:58:53 <maaku> there's no need to clutter up the bitcoin source tree
531 2012-10-04 20:01:18 <sipa> maaku: there are voices for both cases; i don't care
532 2012-10-04 20:02:04 <sipa> i mean, we are statically linking against it, after building it from a subdir
533 2012-10-04 20:02:14 <sipa> (in the current pullreq)
534 2012-10-04 20:02:59 <stamit> anyone wants to give /me/ any bitcoins, to open /my/ source code?
535 2012-10-04 20:08:17 <maaku> MC1984: it wasn't back then
536 2012-10-04 20:08:28 <maaku> whoops n/m
537 2012-10-04 20:08:47 <gmaxwell> maaku: static linking does nothing for people who build it themselves; and it's not a normal package that anyone distributes. For better or worse the behavior of the database is part of the distributed algorithim. It's a decision that could be changed later if it made sense though..
538 2012-10-04 20:09:07 <gmaxwell> e.g. if leveldb went back into active development; got shipped by OSes with promises of compatiblity... etc.
539 2012-10-04 20:16:11 <xblitz> I've posted this on bitcointalk dev forum.. but i figured it would be more interactive if i ask here (will post answers on the forum afterwards) ... https://bitcointalk.org/index.php?topic=115488.0    so how can I resolve the address from different inputs of a transaction from a CWalletTx or CMerkleTx ... ?
540 2012-10-04 20:17:34 <sipa> xblitz: look at its vin's, those contain a prevout; look up its hash, fetch the transaction from disk, look at its prevout.n's output's scriptPubKey
541 2012-10-04 20:17:48 <sipa> xblitz: and it's a really bad idea to rely on that information
542 2012-10-04 20:18:10 <xblitz> ohhhh makes sense...
543 2012-10-04 20:18:23 <xblitz> bad idea?  why?
544 2012-10-04 20:18:29 <sipa> what do you need it for?
545 2012-10-04 20:19:03 <xblitz> just to display more information on a transaction.. a bit like what blockchain.info does
546 2012-10-04 20:19:19 <sipa> for debugging purposes it's fine of course
547 2012-10-04 20:19:29 <sipa> but it's not a "from"
548 2012-10-04 20:20:00 <sipa> in case of a web wallet, the from(s) are not addresses that are necessarily under control of the sender, for example
549 2012-10-04 20:20:06 <xblitz> i've seen this before...  but i cant seems to grasp why it wouldnt be a from..
550 2012-10-04 20:20:16 <xblitz> yeah that i know
551 2012-10-04 20:20:36 <xblitz> i dont want to know from "who"  but from "what adress"
552 2012-10-04 20:20:57 <sipa> in that case, there's no problem
553 2012-10-04 20:21:06 <sipa> it's just that many people want to use to send coins back to, for example
554 2012-10-04 20:21:49 <xblitz> ahhh okay okay..  that i understand quite well :) thanks
555 2012-10-04 20:22:04 <jose__> sipa they could if sendfrom only used the private keys corresponding to the tagged addresses in that account. That would be account heaven for me.
556 2012-10-04 20:22:07 <gmaxwell> xblitz: take care that you don't assume you can turn the scriptpubkey into an address.
557 2012-10-04 20:22:29 <sipa> jose__: even then, that would be a bad idea
558 2012-10-04 20:22:38 <sipa> jose__: you're not supposed to reuse addresses anyway
559 2012-10-04 20:23:15 <sipa> jose__: what you're describing is separate wallets; a feature planned for a long time, but just not a priority now
560 2012-10-04 20:23:16 <gmaxwell> xblitz: e.g. make sure you handle the conversion of scriptPubKey -> address failing.
561 2012-10-04 20:23:35 <jose__> kind of mini wallets == accounts
562 2012-10-04 20:23:53 <jose__> optional
563 2012-10-04 20:23:54 <AJNoz_> hello?
564 2012-10-04 20:24:13 <sipa> jose__: no, just separate wallets would be exactly what you want; funds that are entirely separate
565 2012-10-04 20:24:21 <sipa> but that is not what accounts are
566 2012-10-04 20:24:25 <sipa> nor what they are designed for
567 2012-10-04 20:24:34 <AJNoz_> wow, aplacewhere people actually talk andhelp
568 2012-10-04 20:25:40 <AJNoz_> can anyone possiibly help me with pushpool being dumb?
569 2012-10-04 20:26:04 <jose__> how hard is to adapt senfrom behaviour ?
570 2012-10-04 20:27:02 <sipa> jose__: you can do it using the raw transaction interface yourself
571 2012-10-04 20:27:18 <xblitz> gmaxwell: what are the non resolvable scriptPubKey examples?  mining blocks ?
572 2012-10-04 20:28:01 <sipa> xblitz: multisig (non-P2SH), weird transactions, ...
573 2012-10-04 20:28:13 <sipa> P2SH transaction do have an address but not a traditional one
574 2012-10-04 20:29:10 <xblitz> yeah i read about that.. okay seems good.. I'll go try it out now...  I had seen the thing about prevout but i think i forgot to lookin to it
575 2012-10-04 20:29:45 <gmaxwell> xblitz: if you didn't author the transaction you can make no assumptions about the structure of the inputs scripts.. they could be crazy-whatever.
576 2012-10-04 20:30:03 <gmaxwell> 'cause the person sending them to you knew how to satisify them but you might have no clue.
577 2012-10-04 20:30:38 <xblitz> hummm yeah i think i understand....
578 2012-10-04 20:31:30 <xblitz> so in all.. I will handle non resolvable scripts so it doesnt crash..
579 2012-10-04 20:34:32 <kjj_> Did Sergio run his inverse signature attack past anyone?  Is he worried about an extension attack on SHA256?
580 2012-10-04 20:35:51 <jose__> thanks sipa. seems easier to reuse the c++ code for sendfrom and then make the rpc interface. I might do that. my bet is account confusion would be over.
581 2012-10-04 20:36:36 <gmaxwell> kjj_: yes, and your response to him is correct.
582 2012-10-04 20:37:31 <kjj_> gmaxwell: That sounds like the sort of thing that Kamensky or whoever might have found, thinking he was onto something, only to realize that something else made it safe
583 2012-10-04 20:37:41 <gmaxwell> kjj_: and there are many forms of malleability.
584 2012-10-04 20:37:58 <gmaxwell> I'm a little confused by your response to him; did he edit his message?
585 2012-10-04 20:38:25 <kjj_> I don't think so
586 2012-10-04 20:38:39 <kjj_> he mentions the transaction hash specifically
587 2012-10-04 20:39:13 <gmaxwell> Right but why'd you mention "inverted s signature" ?
588 2012-10-04 20:39:24 <kjj_> oh, that
589 2012-10-04 20:39:30 <kjj_> because he mentioned it in a different thread
590 2012-10-04 20:39:35 <gmaxwell> ah. okay.
591 2012-10-04 20:39:41 <kjj_> (r,s) is valid, but so is (r, -s mod K)
592 2012-10-04 20:39:46 <kjj_> or whatever
593 2012-10-04 20:40:31 <sipa> but so is (0x00 + r, s)
594 2012-10-04 20:40:47 <xblitz> oh any good IDE to use to load the whole bitcoind project and have auto complete  for the custom classes/methods?
595 2012-10-04 20:40:50 <gmaxwell> Sure, there are a whole bunch of ways signatures are currently malleiable; as far as any of us all know they're all equivalent and could be used to annoy people but not to usefully attack except as a kind of limited DOS.
596 2012-10-04 20:40:50 <kjj_> but if the txid is the double hash of the FULL transaction, they'll be different transactions, and obvious doubles
597 2012-10-04 20:41:06 <gmaxwell> kjj_: yep.
598 2012-10-04 20:41:32 <kjj_> since he mentions the hash, I'm wondering if he (thinks) there is an extension attack on the hash that can end up with the same signature
599 2012-10-04 20:41:49 <gmaxwell> No, he previously believed that the signature wasn't included in the hash.
600 2012-10-04 20:41:52 <kjj_> er, with the same hash
601 2012-10-04 20:42:26 <kjj_> oh, thank god.  I much prefer when someone else is confused about something instead of me
602 2012-10-04 20:43:05 <kjj_> and I started digging through the source to find out exactly what gets hashed, but I have to leave in a few minutes.  hate to leave a mystery for that long
603 2012-10-04 20:43:26 <jose__> I guess thats like what gmaxwell wanted to say: 'No way, Jose'.
604 2012-10-04 20:46:53 <xblitz> any good IDE to load the whole bitcoind project and have auto complete  for the custom classes/methods?
605 2012-10-04 20:48:01 <sipa> xblitz: never used an IDE for bitcoind, sorry
606 2012-10-04 20:48:06 <galambo> eclispe?
607 2012-10-04 20:48:09 <sipa> i believe wumpus uses Qt creator
608 2012-10-04 20:48:38 <galambo> you have to get it to build in eclipse to get autocomplete
609 2012-10-04 20:48:58 <xblitz> ok.. maybe ill try that...
610 2012-10-04 20:49:08 <galambo> but that doesn't prevent you from switching to QT creator to do the gui build
611 2012-10-04 20:49:50 <xblitz> its just a pain to search through all the source to see what a classes accepts for init and all the inheritance...
612 2012-10-04 20:50:11 <sipa> galambo: why would eclipse be the only IDE that has autocomplete?
613 2012-10-04 20:50:40 <xblitz> i dont think he said it was the only one
614 2012-10-04 20:50:59 <sipa> saying that you have to get it to build in eclipse does kinda imply that, no?
615 2012-10-04 20:51:07 <galambo> its not i just like the autocomplete and
616 2012-10-04 20:51:09 <galambo> search for definiton and references especially
617 2012-10-04 20:51:21 <xblitz> ah no i think he mean that to have the autocomplete IN eclpse it has to build in it
618 2012-10-04 20:51:33 <sipa> oh, sure, yes
619 2012-10-04 20:51:38 <xblitz> thats what i understood
620 2012-10-04 20:52:11 <galambo> sipa: you have to have a successful build for it to pick up all the class names
621 2012-10-04 20:53:19 <sipa> galambo: yes yes, absolutely, i fully agree
622 2012-10-04 20:53:25 <sipa> i just misunderstood you
623 2012-10-04 21:54:55 <sipa> xblitz: the code you pasted on the forum only works when the prevout you're looking up is in your own wallet as well (it was a change output, or a send-to-self)
624 2012-10-04 21:55:29 <Eliel> do any other pools besides eligius and p2pool pay through generation?
625 2012-10-04 21:56:17 <xblitz> sipa: are you sure..  it actualy works on my end
626 2012-10-04 21:57:21 <sipa> xblitz: yes, you're requesting the prevout from pwalletMain
627 2012-10-04 21:57:49 <sipa> so it only works if the transaction is in your wallet
628 2012-10-04 21:58:44 <sipa> oh, it will always work for outgoing transactions, as you'll always have the transactions that credited your with their input
629 2012-10-04 21:58:52 <sipa> but not necessarily for inputs
630 2012-10-04 21:59:40 <xblitz> sipa: ah yes okay.. in my case i am looking for transactions in my wallet.. but i guess you can use MerkleTx instead of CWalletTX and then use the GetTransaction from main.h
631 2012-10-04 22:00:02 <sipa> no, you need CTransaction::ReadFromDisk
632 2012-10-04 22:00:09 <xblitz> sipa: i will try to make a general one for the example so it can work for all transaction
633 2012-10-04 22:00:36 <sipa> oh, GetTransaction probably also works
634 2012-10-04 22:02:59 <xblitz> probably gets unconfirmed ones too
635 2012-10-04 22:08:06 <sipa> yes, but you'll never have a transaction in your wallet whose input is in the mempool but not in the wallet itself
636 2012-10-04 22:08:13 <sipa> ... i think
637 2012-10-04 22:08:34 <sipa> never mind; you're right