1 2012-12-30 04:00:55 <etotheipi_> wow, freakin' slow day on the bitcoin-dev channel today...
  2 2012-12-30 04:02:47 <etotheipi_> gmaxwell, sipa:  I'm becoming more and more swayed by thanke's concern about backing out m/i-privKey given m/i-chaincode and m/i/j-privKey
  3 2012-12-30 04:04:30 <etotheipi_> there's a certain inelegance to that "hole" that makes me think we should fix it if possible... but is it possible?
  4 2012-12-30 04:05:30 <etotheipi_> I can't even think of any real use-cases where it would be an issue, but that doesn't mean there won't be
  5 2012-12-30 04:08:12 <etotheipi_> coudn't it be solved by redefining the CKD_ver2() as CKD_ver1(CKD_ver1()) ?   i.e.  in version 2 m/i/j/k would be m/i/i/j/j/k/k as defined in the version 1 (as BIP 32 is written right now)
  6 2012-12-30 04:12:16 <etotheipi_> nevermind, that doesn't work
  7 2012-12-30 04:20:16 <etotheipi_> actually, we only need to modify the CONOPs for the existing BIP 32 with the same idea -- the crypto doesn't change, but we only allow branching at even levels... and never give out observer or private branches at odd levels
  8 2012-12-30 05:25:20 <wizkid057> gmaxwell: not sure if this issue is still in question, but, that one miner who's address you asked about uses the blockchain.info wallet app
  9 2012-12-30 10:44:52 <stealth222> I opensourced my bitcoin protocol implementation: https://github.com/CodeShark/CoinClasses
 10 2012-12-30 12:28:46 <ssm2017> hello
 11 2012-12-30 12:28:55 <stealth222> hi
 12 2012-12-30 12:29:12 <ssm2017> using the json-rpc connection, is there a way to know the total count of "transactions" ?
 13 2012-12-30 12:29:37 <stealth222> listtransactions
 14 2012-12-30 12:29:40 <sipa> total count of transactions... ever? ... in your wallet? ... that aren't fully redeemed?
 15 2012-12-30 12:30:29 <ssm2017> i have made a php script that is listing transactions but listtransactions is returning only the last 10 ones if i dont precise a quantity and a start point
 16 2012-12-30 12:30:59 <ssm2017> i woulod like to be able to make a pager to get transactions one by one but at first i need to know the total amount of transactions
 17 2012-12-30 12:31:45 <ssm2017> like transactions 0->10 / 2000; 11->20 /2000....
 18 2012-12-30 12:32:14 <stealth222> for all accounts in a wallet? or just a single account?
 19 2012-12-30 12:32:33 <ssm2017> both
 20 2012-12-30 12:32:43 <Scrat> is there a hash list for all the downloads on bitcoin.org?
 21 2012-12-30 12:32:51 <ssm2017> or at first everything that is available :)
 22 2012-12-30 12:33:09 <sipa> Scrat: http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.7.2/
 23 2012-12-30 12:33:48 <sipa> SHASUMS.asc is a GPG-signed list of checksums
 24 2012-12-30 12:33:56 <stealth222> good morning, sipa :)
 25 2012-12-30 12:34:00 <Scrat> sipa: ty
 26 2012-12-30 12:34:09 <sipa> stealth222: eh, you too!
 27 2012-12-30 12:34:25 <stealth222> ssm, unfortunately I don't think you can get the total number of transactions in a wallet with a single RPC call
 28 2012-12-30 12:34:31 <stealth222> am I wrong, sipa?
 29 2012-12-30 12:34:43 <stealth222> wouldn't you have to first query the account names, then call listtransactions for each one?
 30 2012-12-30 12:35:27 <ssm2017> this is what im actually doing
 31 2012-12-30 12:35:31 <sipa> yeah, i think there's something fundamentally wrong with where listtransactions starts counting if you specify an offset
 32 2012-12-30 12:36:06 <stealth222> there's also something fundamentally wrong with getbalance taking the account as the first optional parameter while minconf is the second :)
 33 2012-12-30 12:36:21 <ssm2017> to use the offset in listtransactions, i would need to know first the total amount
 34 2012-12-30 12:36:26 <stealth222> there are so many times I've needed to query the total balance in a wallet with 0 confirmations regardless of account
 35 2012-12-30 12:36:54 <stealth222> I can put in a pull request to expose these things in the RPC :)
 36 2012-12-30 12:37:16 <sipa> ssm2017: ?
 37 2012-12-30 12:37:29 <sipa> ssm2017: offset just means drop the first (or last, can't remember) N transactions
 38 2012-12-30 12:39:15 <ssm2017> ah, ok... i thought that i could be able to ask 10 transactions begining at offset 100 for example
 39 2012-12-30 12:39:16 <stealth222> listtransactions doesn't make sense :p
 40 2012-12-30 12:39:48 <sipa> ssm2017: yes, you can
 41 2012-12-30 12:39:56 <stealth222> listtransactions 10 0 gives me the empty array but listtransactions gives me a bunch
 42 2012-12-30 12:39:59 <stealth222> how's that possible>?
 43 2012-12-30 12:40:28 <sipa> stealth222: you need the account name in between
 44 2012-12-30 12:40:36 <stealth222> oh, right - lol
 45 2012-12-30 12:41:11 <ssm2017> i could not find a way to use count and offset for listtransactions without the account name too
 46 2012-12-30 12:41:11 <stealth222> we should allow count=-1 mean all
 47 2012-12-30 12:41:22 <sipa> ssm2017: there isn't
 48 2012-12-30 12:41:29 <sipa> but you can specify '*' to get all accounts
 49 2012-12-30 12:41:34 <stealth222> oh, can you? lol
 50 2012-12-30 12:41:38 <ssm2017> ah? wildcards ? good :)
 51 2012-12-30 12:41:43 <sipa> not wildcards
 52 2012-12-30 12:41:49 <sipa> just '*' means all
 53 2012-12-30 12:41:54 <ssm2017> ok, noted
 54 2012-12-30 12:42:36 <ssm2017> listrasactions * * 0
 55 2012-12-30 12:42:38 <ssm2017> :)
 56 2012-12-30 12:42:44 <ssm2017> to hang the server
 57 2012-12-30 12:42:50 <stealth222> ok, I see the wildcard now
 58 2012-12-30 12:43:09 <lianj> stealth222: paging > showing all
 59 2012-12-30 12:43:39 <stealth222> depends on where you want to buffer it, lianj
 60 2012-12-30 12:43:56 <ssm2017> there is a missing : gettransactionscount [accountname]
 61 2012-12-30 12:44:23 <sipa> that wouldn't be faster than listtransactions
 62 2012-12-30 12:44:34 <stealth222> it would require less I/O
 63 2012-12-30 12:44:35 <sipa> but it would be convenient, i guess
 64 2012-12-30 12:44:40 <sipa> yeah
 65 2012-12-30 12:44:51 <ssm2017> if i have billions of transactions, there can be a problem to use listtransaction *
 66 2012-12-30 12:45:23 <stealth222> if you only want to get the total count over a network, it would be far faster to perform the count serverside
 67 2012-12-30 12:45:25 <stealth222> :)
 68 2012-12-30 12:46:10 <sipa> sure
 69 2012-12-30 12:46:30 <sipa> but the largest problem is that listtransactions needs to iterate the entire wallet every time anyway
 70 2012-12-30 12:47:03 <stealth222> the transaction count could be stored
 71 2012-12-30 12:47:07 <stealth222> so that the query is fast
 72 2012-12-30 12:47:29 <Scrat> so which operations are cached/indexed?
 73 2012-12-30 12:48:10 <sipa> for the wallet? nothing
 74 2012-12-30 12:48:42 <sipa> oh, total balances and balances per transaction are cached
 75 2012-12-30 12:49:00 <sipa> but not per account
 76 2012-12-30 12:49:02 <Scrat> how about balance per address or account
 77 2012-12-30 12:49:03 <Scrat> oh
 78 2012-12-30 12:49:44 <sipa> 'balance per address' is not needed ever for normal operation
 79 2012-12-30 12:50:03 <Scrat> I really ment received per address :p
 80 2012-12-30 12:50:31 <sipa> that too
 81 2012-12-30 12:51:02 <Scrat> so wrapping every address into an account is the preferred usage
 82 2012-12-30 12:51:13 <sipa> how so?
 83 2012-12-30 12:51:42 <Scrat> this is not for personal use
 84 2012-12-30 12:52:03 <sipa> associate addresses with an account if they're associated with an account
 85 2012-12-30 12:52:08 <sipa> don't if they aren't
 86 2012-12-30 12:52:27 <sipa> (and by account i mean "something you need to track the balance of")
 87 2012-12-30 12:52:36 <sipa> but it's not cached either :(
 88 2012-12-30 12:53:27 <Scrat> if it's only 1 disk seek per transaction that shouldn't be a problem unless you have thousands
 89 2012-12-30 12:53:43 <sipa> wallets are always in memory, there are no disk seeks involved
 90 2012-12-30 12:54:18 <Scrat> even transactions going back to very old blocks?
 91 2012-12-30 12:54:42 <sipa> wallet and blockchain systems are almost entirely separate
 92 2012-12-30 12:54:52 <sipa> and the wallet is stored in memory, for now
 93 2012-12-30 12:59:04 <Scrat> and another thing: is the change address guaranteed to be different than the original address?
 94 2012-12-30 12:59:37 <sipa> yes
 95 2012-12-30 12:59:44 <Scrat> or well not original address since it sends from many
 96 2012-12-30 12:59:45 <sipa> it's always a fresh address
 97 2012-12-30 12:59:48 <Scrat> good
 98 2012-12-30 13:00:05 <sipa> except in case where you restore an old backup, perhaps
 99 2012-12-30 13:01:35 <Joric> sipa i managed to get a fulltime job ) opensource cpp and a salary on a monthly basis how cool is that
100 2012-12-30 13:01:52 <Joric> guess bitcoin foundation still doesn't pay shit
101 2012-12-30 13:02:14 <stealth222> congrats, Joric :)
102 2012-12-30 13:03:25 <sipa> Joric: nice!
103 2012-12-30 13:03:39 <sipa> what does bitcoin foundation have to do with that?
104 2012-12-30 13:04:24 <Joric> well shouldn't it pay atleast for the ultraprune implementation )
105 2012-12-30 13:04:45 <sipa> why?
106 2012-12-30 13:05:27 <Joric> this thing was hard to write wasn't it
107 2012-12-30 13:05:35 <sipa> it took a while :)
108 2012-12-30 13:05:44 <stealth222> difficulty isn't what determines how much it pays :)
109 2012-12-30 13:06:04 <stealth222> in fact, in my experience I've often found the two to be inversely correlated
110 2012-12-30 13:07:35 <Joric> funny how jgarzik just skipped
111 2012-12-30 13:07:40 <Scrat> aren't most devs filthy rich just by being early adopters? 50k btc pizzas come to mind ;)
112 2012-12-30 13:08:06 <sipa> Scrat: i have maybe enough BTC to survive 2 months
113 2012-12-30 13:08:17 <Scrat> damn
114 2012-12-30 13:08:18 <Joric> Scrat, nope they arent )
115 2012-12-30 13:08:40 <sipa> and i wasn't really an early adoper (got my first coins in december 2010 iirc)
116 2012-12-30 13:09:15 <stealth222> the average salary for a software engineer is pretty decent - but that's largely because there's a handful of billionaires, a slightly larger number of millionaires, and then the rest most of which don't make that much :p
117 2012-12-30 13:10:22 <Joric> on the other hand i won't get enough time for getting a PhD now thats for sure
118 2012-12-30 13:12:28 <Joric> jgarzik, was bitcoin foundation considering paying a salary to code devs
119 2012-12-30 13:14:07 <stealth222> code devs should probably be getting paid for giving talks, sitting on company boards, etc...and not for writing open source code :)
120 2012-12-30 13:14:39 <sipa> i suppose you mean core devs?
121 2012-12-30 13:14:43 <stealth222> yer
122 2012-12-30 13:15:22 <stealth222> writing open source code gives you the credentials to do the other things
123 2012-12-30 13:15:28 <stealth222> but it doesn't pay :)
124 2012-12-30 13:15:45 <stealth222> I mean, it doesn't pay directly
125 2012-12-30 13:17:17 <stealth222> how does Linus Torvalds make a living?
126 2012-12-30 13:18:16 <sipa> linux foundation pays him
127 2012-12-30 13:18:23 <roconnor_> what is the median software developer salary?
128 2012-12-30 13:18:50 <Joric> zynga developers get $14k a month for php )
129 2012-12-30 13:19:30 <stealth222> that's sick
130 2012-12-30 13:19:46 <stealth222> are you sure about that?
131 2012-12-30 13:19:52 <Scrat> stealth222: he was given tons of Red Hat shares and options
132 2012-12-30 13:19:59 <Joric> pretty much my friend is there
133 2012-12-30 13:20:09 <Joric> i didn't manage to skip to zynga yet
134 2012-12-30 13:20:58 <Joric> its only $150k a year not that much for a grown man
135 2012-12-30 13:20:59 <Scrat> Joric: what is your friend saying about the management?
136 2012-12-30 13:21:13 <Scrat> I heard they're dicks
137 2012-12-30 13:21:15 <stealth222> the median salary for a senior software engineer at the top companies is about $120k
138 2012-12-30 13:21:28 <Scrat> which is fitting since zynga is one of the most hated companies :p
139 2012-12-30 13:21:39 <roconnor_> stealth222: thanks
140 2012-12-30 13:44:56 <zapsoda> Is there a way to make somthing happen (in PHP) when a address gets BTC without having a script constently running? (I asked this is all the channels so im not ignoreing your answear)
141 2012-12-30 13:45:33 <Joric> zapsoda, no theres no callbacks if you don't use 3rd party services
142 2012-12-30 13:45:49 <zapsoda> :/
143 2012-12-30 13:47:10 <Joric> theoretically it's possble to run bitcoind -printtoconsole + listen for events there just a guess
144 2012-12-30 13:48:10 <Joric> amirite? no callbacks either logs or cron
145 2012-12-30 13:48:27 <zapsoda> As far as i can tell you are
146 2012-12-30 13:48:32 <zapsoda> Ill probably go with cron
147 2012-12-30 13:49:06 <Scrat> can only run cron every 1 minute, correct?
148 2012-12-30 13:49:27 <zapsoda> So like how does this work? http://royalbitcoin.com/  cron job?  It detects when you send BTC and sends back
149 2012-12-30 13:50:03 <Scrat> 1) they might be using a 3rd party api like blockchain.info
150 2012-12-30 13:50:16 <Scrat> 2) listsinceblock maybe
151 2012-12-30 13:52:16 <Scrat> oh its a dice website
152 2012-12-30 13:52:35 <Scrat> probably uses bitcoinj (java client) which also allows it to send with 0 confirmations
153 2012-12-30 13:54:12 <Joric> it's not that hard to modify bitcoind matter of minutes
154 2012-12-30 13:54:15 <Scrat> not sure about listsinceblock, what happens after a reorg? devs?
155 2012-12-30 13:56:14 <Scrat> I can think of a rare case where you miss a transaction
156 2012-12-30 14:00:31 <zapsoda> The other option would to be to use this i guess?
157 2012-12-30 14:00:32 <zapsoda> http://code.google.com/p/bitcoinj/wiki/GettingStarted#Receiving_coins_and_handling_units
158 2012-12-30 14:36:35 <stealth222> heh, zapsoda - that's one of my favorite issues
159 2012-12-30 14:36:54 <stealth222> I've solved it in several ways...unfortunately none of them are the official satoshi client yet
160 2012-12-30 14:37:23 <zapsoda> So you use a diffrent client? Like Bitcoinj?
161 2012-12-30 14:37:24 <stealth222> I've got two pull requests in with the original motivation to be able to get these alerts - and I also wrote my own bitcoin implementation
162 2012-12-30 14:37:34 <stealth222> in order to get these alerts
163 2012-12-30 14:38:26 <stealth222> I just opensourced my bitcoin implementation if you're a C++ programmer and want to use it. You can build a custom filter atop the onTx handler and then use curl to post to a PHP page
164 2012-12-30 14:38:42 <stealth222> that's how I've been doing it
165 2012-12-30 14:38:51 <zapsoda> Sounds to hard for me, Ill stick with the https://en.bitcoin.it/wiki/BitcoinNotify
166 2012-12-30 14:38:56 <zapsoda> I think
167 2012-12-30 14:39:02 <zapsoda> I might do somthing diffrent
168 2012-12-30 14:39:03 <Scrat> stealth222: link plx
169 2012-12-30 14:39:18 <sipa> Note that bitcoinnotify is not in business anymore since it has been sold in December 2011. But there are some good alternatives available which offer similar services:
170 2012-12-30 14:39:32 <stealth222> Scrat: https://github.com/CodeShark/CoinClasses/
171 2012-12-30 14:39:35 <zapsoda> Sorry
172 2012-12-30 14:39:37 <zapsoda> I meeant http://www.bitcoinmonitor.net/
173 2012-12-30 14:39:38 <stealth222> look at the listener example
174 2012-12-30 14:40:10 <sipa> being able to get callbacks when a transaction reaches a preset number of confirmations would be nice, i guess
175 2012-12-30 14:40:24 <stealth222> for that I had to build a database, sipa
176 2012-12-30 14:40:27 <stealth222> lol
177 2012-12-30 14:41:02 <stealth222> although now the satoshi client supports querying confirmation count via RPC
178 2012-12-30 14:41:03 <gmaxwell> stealth222: I don't follow how a notification patch saves zapsoda from having something 'constently running'
179 2012-12-30 14:41:19 <stealth222> you have to have something running
180 2012-12-30 14:41:24 <stealth222> this listener runs as a daemon
181 2012-12-30 14:41:33 <stealth222> but it sleeps most of the time :)
182 2012-12-30 14:41:47 <Scrat> because callbacks > polling
183 2012-12-30 14:42:54 <ne0futur> (16:35) <   zapsoda> Is there a way to make somthing happen (in PHP) when a address gets BTC without having a
184 2012-12-30 14:43:24 <ne0futur> I d use bitping
185 2012-12-30 14:43:43 <ne0futur> https://github.com/neofutur/BitPing.Net
186 2012-12-30 14:43:59 <stealth222> when I first implemented the listener the reference client still didn't have getrawtransaction
187 2012-12-30 14:44:17 <stealth222> but now with getrawtransaction it's possible to check confirmation counts each time a block is received
188 2012-12-30 14:44:26 <stealth222> without having to store the block chain separately
189 2012-12-30 14:45:21 <zapsoda> Thanks guys
190 2012-12-30 14:47:01 <stealth222> I've used bitcoin abe before, ne0futur. I ended up creating a similar schema for my db
191 2012-12-30 14:47:18 <stealth222> optimized for quick searches, not for size
192 2012-12-30 14:47:36 <stealth222> nor for insert speed
193 2012-12-30 14:48:10 <stealth222> but now with getrawtransaction, it's unnecessary to keep a separate block chain database to get confirmation alerts
194 2012-12-30 14:48:46 <sipa> except that git head doesn't support getrawtransaction for arbitrary transactions anymore
195 2012-12-30 14:49:01 <stealth222> are you f#@$ing serious?!??!!
196 2012-12-30 14:49:05 <stealth222> no way...
197 2012-12-30 14:49:13 <stealth222> that was one of the coolest features :)
198 2012-12-30 14:49:15 <sipa> (and with an extra database, it can, but watch-only wallets + notifications is a much nicer solution)
199 2012-12-30 14:49:36 <sipa> well it's fundamentally incompatible with pruning
200 2012-12-30 14:49:50 <sipa> and just to maintain a wallet, you shouldn't need to full block chain
201 2012-12-30 14:49:55 <sipa> *the
202 2012-12-30 14:50:24 <stealth222> yes, true. detecting the status of an unconfirmed transaction is still a problem, though
203 2012-12-30 14:50:33 <stealth222> is it orphaned? double-spent? etc...
204 2012-12-30 14:50:46 <sipa> wallets should track that
205 2012-12-30 14:51:07 <sipa> imho, a wallet should run perfectly on an SPV-style chain database
206 2012-12-30 14:51:07 <stealth222> gmaxwell and I had a discussion the other day about this and concluded that the wallet would have to maintain its own mempool
207 2012-12-30 14:51:36 <sipa> hmm, why?
208 2012-12-30 14:52:14 <stealth222> the wallet only receives transactions that belong to it - meaning it participates in either the inputs or outputs. however, it does not track dependencies
209 2012-12-30 14:52:23 <sipa> sure it does
210 2012-12-30 14:52:29 <stealth222> say A -> B -> C, only C belongs to the wallet
211 2012-12-30 14:52:36 <stealth222> B is double-spent
212 2012-12-30 14:52:41 <sipa> the wallet contains unconfirmed dependencies
213 2012-12-30 14:53:11 <stealth222> hmm...I guess it would need them to make the merkle tree
214 2012-12-30 14:53:28 <gmaxwell> stealth222: no, it doesn't.
215 2012-12-30 14:54:07 <gmaxwell> sipa: which doesn't help you when a height 1 confirmed dependency get reorged out and replaced with a conflict.  (certantly you can monitor all transactions, but that makes all the bloom filtering stuff kinda moot except for ibd)
216 2012-12-30 14:54:25 <gmaxwell> (thats what stealth222 meant about maitain its own mempool)
217 2012-12-30 14:55:47 <stealth222> right, for the merkle tree it is sufficient that you know in which blocks your transactions reside
218 2012-12-30 14:56:02 <sipa> gmaxwell: you can still detect that the transaction you thought was confirmed, isn't anymore
219 2012-12-30 14:56:04 <stealth222> and you trust the network to verify that the inputs connect
220 2012-12-30 14:56:05 <sipa> but indeed
221 2012-12-30 14:56:35 <gmaxwell> sipa: right, it's just an issue for unconfirmed txn and knowing that they've changed between varrious unconfirmed states.
222 2012-12-30 14:59:01 <gmaxwell> sipa: the prior discussion stealth222 and I had where I'd made an aside comment 'while you're working on that, why not have the wallet show negative confirmation counts for conflicted transactions (e.g. -distanct to first conflict)' but it turns out that this is not easy to do from a chainless / indexless wallet.
223 2012-12-30 14:59:40 <sipa> right
224 2012-12-30 15:02:41 <stealth222> you can always run your own block chain database server and query by outpoints :)
225 2012-12-30 15:03:05 <stealth222> that, unfortunately, is the only reliable solution I've so far found
226 2012-12-30 15:08:13 <stealth222> is there any way the protocol could maintain double-spent states without opening itself up to DoS?
227 2012-12-30 15:09:16 <stealth222> i.e. also record at least one conflicting transaction when it occurs but marking it as void
228 2012-12-30 15:09:49 <sipa> gmaxwell: i think the block undo data should have checksums
229 2012-12-30 15:10:34 <sipa> i've implemented a "tolerant" DisconnectBlock that supports rolling back transactions on an inconsistent state
230 2012-12-30 15:10:43 <sipa> but it has to assume the undo data is perfect
231 2012-12-30 15:11:20 <gmaxwell> I think I'd commented before that if we had hashes we could save them when pruning undo data... and then if we added a way to fetch undo data then reorgs below a prunepoint could still be handled.
232 2012-12-30 15:11:35 <gmaxwell> So it would be useful for that too.
233 2012-12-30 15:11:56 <stealth222> so the only serious hurdle is reorgs?
234 2012-12-30 15:12:26 <sipa> afk
235 2012-12-30 15:13:23 <gmaxwell> You still have to have the 'recieve C now go back and get B then A' which gets kinda ugly if someone sends you a really long unconfirmed chain.. but thats just ugly not a barrier.
236 2012-12-30 15:16:27 <stealth222> still going through main.cpp to make sure I fully get how all this works
237 2012-12-30 15:17:43 <stealth222> so when a tx message is received, it attempts to connect inputs and if it succeeds (as well as passing all the other tests) it gets accepted in the mempool. Wallets are only alerted of the transaction when it gets accepted into the mempool, correct?
238 2012-12-30 15:19:17 <stealth222> but the wallet isn't given the entire chain of transactions from the mempool - so what you're saying is that the wallet would have to go back out to the mempool and fetch the chain?
239 2012-12-30 15:20:46 <stealth222> or are you saying that the mempool feeds it not only the transaction that belongs to it but all dependencies, too?
240 2012-12-30 15:22:29 <stealth222> AddToWalletIfInvolvingMe would then have to pass an entire graph structure to the wallet rather than just a single transaction
241 2012-12-30 15:23:51 <stealth222> or am I totally missing something?
242 2012-12-30 15:26:36 <stealth222> actually, it would be sufficient that the wallet keep track of all dependent outpoints
243 2012-12-30 15:26:45 <stealth222> and tx hashes of the txs that claim them
244 2012-12-30 15:27:21 <stealth222> doesn't even really care about the order
245 2012-12-30 15:28:46 <stealth222> the mempool class could handle calculating dependent outpoints
246 2012-12-30 15:39:41 <stealth222> sorry, gmaxwell, I know you've probably been through all these issues a million times already - I'm relatively new to the details of main.cpp.
247 2012-12-30 15:39:50 <stealth222> so I'll study it more and then comment :)
248 2012-12-30 15:47:07 <etotheipi_> sipa: did you have any concerns about thanke's one-level-reversal of private keys?
249 2012-12-30 15:47:22 <etotheipi_> is there a way around it?
250 2012-12-30 15:49:02 <stealth222> etotheipi_: where can I read more about that?
251 2012-12-30 15:49:39 <etotheipi_> stealth222: it's not thoroughly described anywhere.. but you can read it on the second-to-last page of the "Deterministic Wallets" thread on the forums
252 2012-12-30 15:50:15 <etotheipi_> thanke points out that while we rely on  c*PublicKey being non-invertible, c*PrivateKey *is* invertible
253 2012-12-30 15:50:58 <etotheipi_> (actually, c*PrivKey mod N)
254 2012-12-30 15:51:59 <etotheipi_> sipa: is there an operation that requires solving the discrete log problem for both the public and private operations?
255 2012-12-30 15:52:23 <stealth222> etotheipi_: ok, thanks
256 2012-12-30 15:55:20 <etotheipi_> gmaxwell: comments?
257 2012-12-30 15:57:23 <gmaxwell> ::meh:: its not harmful for the proposed uses. I agree it's somewhat surprising and would be good to fix if I saw a way to do so.
258 2012-12-30 15:58:26 <etotheipi_> gmaxwell: that's how I feel, too
259 2012-12-30 15:58:50 <etotheipi_> I think it *should* be fixed, but I'm not sure it's really possible
260 2012-12-30 15:59:41 <etotheipi_> (at least not without breaking type2)
261 2012-12-30 16:05:01 <etotheipi_> is there a way to get RSA-style discrete-log protection on the private key operations?
262 2012-12-30 16:05:23 <etotheipi_> I guess it doesn't work if everyone knows the modulus, N
263 2012-12-30 16:18:05 <stealth222> The private key is a number while the public key is a point. so you're just basically saying that while point multiplication on elliptic curves is not invertible, finding multiplicative inverses mod p is polynomial-time? is that all you're saying? or is there something more?
264 2012-12-30 16:18:25 <etotheipi_> stealth222: that's essentially it
265 2012-12-30 16:19:08 <etotheipi_> and it only applies to a case for which we don't really have any use cases, right now
266 2012-12-30 16:19:36 <etotheipi_> which is that the party must have private chain at one level, and the public chain of the parent
267 2012-12-30 16:32:21 <MC1984> wow after hitting up some android related rooms
268 2012-12-30 16:32:35 <MC1984> i have a new appreciation for the people who are into bitcoin on irc
269 2012-12-30 16:32:48 <MC1984> helpfulness i mean
270 2012-12-30 16:56:37 <stealth222> etotheipi_: your diagrams were invaluable to my understanding of OP_CHECKSIG - thanks so much for publishing them :)
271 2012-12-30 16:56:51 <sipa> etotheipi_: i've memtioned it under thesecurity section of bip32, but i see no way to avoid it
272 2012-12-30 16:57:00 <stealth222> without your help, it would have probably taken me a lot longer to implement a transaction signer
273 2012-12-30 16:58:28 <etotheipi_> stealth222: fantastic!  I knew other people would find it useful
274 2012-12-30 16:59:00 <etotheipi_> it was the result of countless hours of bashing my head over it (mostly the endianness, though, but you never know what the problem is until it's right)
275 2012-12-30 16:59:51 <stealth222> yeah, lol
276 2012-12-30 17:00:13 <stealth222> I had to stick in tracers in bitcoind to get detailed feedback as to why my transactions were rejected
277 2012-12-30 17:00:16 <etotheipi_> sipa: okay, I just wanted to catch up any discussions you may have had about it already
278 2012-12-30 17:00:28 <etotheipi_> apparently you have conceded to it, much like I am about to....
279 2012-12-30 17:01:10 <stealth222> as long as the operation for generating the chain is based on scalar multiplication, the issue seems unavoidable
280 2012-12-30 17:01:23 <etotheipi_> stealth222: eww.. .I didn't have to go that far...
281 2012-12-30 17:01:34 <etotheipi_> but it doesn't surprise me
282 2012-12-30 17:05:54 <etotheipi_> re scalar multiplication:  I guess it will just be an eternal caveat of type2 wallets
283 2012-12-30 17:06:31 <stealth222> unless we move to something other than ECDSA
284 2012-12-30 17:07:11 <etotheipi_> actually, I just realized somethign -- if we move to something else (eventually), there's no guarantee it will have the Type2 property
285 2012-12-30 17:10:40 <stealth222> have you read this paper before? http://middleware.internet2.edu/idtrust/2009/papers/07-perlner-quantum.pdf
286 2012-12-30 17:11:02 <etotheipi_> stealth222: I've seen it before, but never read through it
287 2012-12-30 17:12:35 <stealth222> I wonder whether NTRU might be possible to make type2
288 2012-12-30 17:13:39 <etotheipi_> stealth222: I think even deterministic wallets will be more complicated -- I'm sure it can be done but something like RSA doesn't work with just arbitrary strings as your private key
289 2012-12-30 17:14:21 <etotheipi_> I suppose you could use a deterministic PRNG and use the 32-byte string as the seed for it, but it would be slow
290 2012-12-30 17:14:32 <gmaxwell> etotheipi_: thus??? don't use crap mechenisms. :P
291 2012-12-30 17:14:49 <gmaxwell> (and besides, lamport is fine with $whatever as the private key)
292 2012-12-30 17:15:24 <etotheipi_> gmaxwell: is RSA crap?
293 2012-12-30 17:16:12 <stealth222> other than that it requires much larger keys than ECDSA?
294 2012-12-30 17:16:20 <stealth222> or ECC
295 2012-12-30 17:16:21 <gmaxwell> etotheipi_: the only argument I'm aware of for using it over ECC methods is that it has a wide existing deployment.
296 2012-12-30 17:17:21 <stealth222> perhaps one could also argue that factorization has been more studied than the discrete log problems in ECC...but I'm not sure that's really relevant
297 2012-12-30 17:17:28 <Scrat> isnt key generation on rsa really slow?
298 2012-12-30 17:17:39 <stealth222> RSA requires two large primes and a large random number
299 2012-12-30 17:17:56 <stealth222> and not all pairs of primes are as good
300 2012-12-30 17:18:01 <Scrat> plus multiplying these primes
301 2012-12-30 17:18:40 <stealth222> RSA key generation involves randomly picking large numbers until a set is found that passes the test
302 2012-12-30 17:19:43 <Scrat> is the primality test on the big primes probabilistic?
303 2012-12-30 17:19:54 <stealth222> there's a deterministic test but it's not as fast
304 2012-12-30 17:20:12 <Scrat> yeah you have to test factors up to sqr(num)
305 2012-12-30 17:20:27 <stealth222> no, there's a polynomial time deterministic algorithm
306 2012-12-30 17:21:00 <stealth222> http://www.cse.iitk.ac.in/users/manindra/algebra/primality_v6.pdf
307 2012-12-30 17:21:41 <stealth222> Miller-Rabin is only probabilistic - but in practice it gives good results quickly
308 2012-12-30 17:22:40 <stealth222> AKS is O((log n)^7.5) or thereabouts
309 2012-12-30 17:23:29 <stealth222> Miller-Rabin is about O(k (log n)^2)
310 2012-12-30 17:25:29 <Scrat> stealth222: got it
311 2012-12-30 17:27:25 <Scrat> looks like openssl first does a few division with small numbers and if it passes runs a Miller-Rabin
312 2012-12-30 17:37:27 <Joric> did you see this? http://www.reddit.com/r/Bitcoin/comments/15oakd/the_bitcoin_problem/ the hell was that??
313 2012-12-30 17:38:37 <MC1984> the bitcoin wallet app on the anroid store has nothing to do with you guys right?
314 2012-12-30 17:38:40 <stealth222> looks like nonsense to me, Joric
315 2012-12-30 17:38:41 <Scrat> trolls trolling trolls
316 2012-12-30 17:39:16 <Scrat> buildbox  2593 55.4 20.6 269284 176904 ?       SLsl 18:18  77:29 ./bitcoind -daemon
317 2012-12-30 17:39:23 <Scrat> killing my laptop
318 2012-12-30 17:39:30 <Scrat> and I'm running an ssd, lol
319 2012-12-30 17:40:25 <Joric> TD's diffuculties? Mike Hearn? bitcoinj?
320 2012-12-30 17:40:40 <wereHamster> Scrat: yes, it's very demanding on the disk
321 2012-12-30 17:42:16 <Scrat> satoshidice tx's eating my laptop alive
322 2012-12-30 17:47:00 <Skav> Scrat why you running from your laptop ?
323 2012-12-30 17:47:18 <Scrat> I like coding in bed sometimes
324 2012-12-30 17:47:32 <Skav> oh
325 2012-12-30 17:47:43 <Scrat> and I'm too lazy to setup a LAN RPC thingy
326 2012-12-30 17:47:52 <Scrat> what quality is the 0.8.x branch right now?
327 2012-12-30 19:08:52 <etotheipi_> question about Satoshi client behavior:  node is at block X and receives X+1, add it to the pool, then later receive (X+1)' with the same difficulty
328 2012-12-30 19:08:58 <etotheipi_> it doesn't switch to that block
329 2012-12-30 19:09:14 <etotheipi_> but then let's say it receives (X+2)' and switches to it because now that's the longest chian
330 2012-12-30 19:09:36 <etotheipi_> how does X+1 and X+2 propagate through that node?
331 2012-12-30 19:09:42 <etotheipi_> err (X+1)' and (X+2)'
332 2012-12-30 19:10:04 <etotheipi_> does it forward (X+1)' even though it doesn't treat it as preferred chain?
333 2012-12-30 19:10:28 <etotheipi_> after (X+2)' comes in and it switches, does it broadcast inv messages for both (X+1)' and (X+2)'?
334 2012-12-30 19:20:20 <etotheipi_> okay, more specifically, let's say I'm only listening to the broadcasts of a node, and there is a two-block reorg:  X-->Y-->Z now becomes X-->Y'-->Z'-->A'  ... will my listening node ever see Y' or Z'?  or will it only see A' and have no idea what it is?
335 2012-12-30 19:33:02 <anonibit> reddit is sucking today
336 2012-12-30 19:45:02 <anonibit> WTF is this? http://pastebin.com/aDn7FY4Z
337 2012-12-30 19:46:06 <D34TH> your attempt at drama?
338 2012-12-30 19:49:11 <gmaxwell> etotheipi_: it'll request Z' Y' after getting A'
339 2012-12-30 20:12:21 <etotheipi_> gmaxwell: who'll request it?  is the bitcoind instance going to broadcast knowledge of Y' and Z'?  How does my node know that A' is the longest chain if it's only seen Y and Z?
340 2012-12-30 20:13:45 <gmaxwell> etotheipi_: because it will get A' and not be able to connect it, so then it requests its parents (using prev). Consider: the same question comes up when a node has had connections problems and missed some blocks.
341 2012-12-30 20:18:53 <etotheipi_> gmaxwell: how does it know it's not just getting spammed?  couldn't I create a list of 10,000,000 invalid headers, but it wouldn't know they are invalid until it's retrieved all 10mil of them
342 2012-12-30 20:42:47 <stealth222> I wish blocks stored height in the header
343 2012-12-30 20:43:14 <stealth222> minimal expense in extra data - greatly simplifies many operations
344 2012-12-30 20:44:55 <ThomasV_> stealth222: https://en.bitcoin.it/wiki/BIP_0034
345 2012-12-30 20:45:10 <stealth222> yeah, I've seen that before
346 2012-12-30 20:48:11 <stealth222> would have been way cleaner to just store that in the block header itself rather than the first transaction
347 2012-12-30 20:50:51 <ThomasV_> but that's a hard fork
348 2012-12-30 20:51:02 <stealth222> yes, I know
349 2012-12-30 20:51:09 <stealth222> hence the "I wish"
350 2012-12-30 20:52:03 <etotheipi_> if we accept that we'll have to do a hard fork some day, then we should definitely add that to the list
351 2012-12-30 20:52:10 <etotheipi_> (I'm sure it's already on the hard-fork wishlist, though)
352 2012-12-30 20:52:30 <etotheipi_> https://en.bitcoin.it/wiki/Hardfork_Wishlist
353 2012-12-30 20:53:18 <ThomasV_> I didn't know there was such a wishlist
354 2012-12-30 20:53:25 <ThomasV_> I want a pony!
355 2012-12-30 20:56:38 <etotheipi_> it's gotta happen sometime...
356 2012-12-30 20:57:30 <etotheipi_> a lot of these would be great, though, of course, new clients still have to hardcode the old rules up to block X (whenever the switch is made)
357 2012-12-30 21:02:19 <stealth222> would it be possible to write into the protocol a mechanism for changing the message structures?
358 2012-12-30 21:02:41 <etotheipi_> probably not without huge security holes
359 2012-12-30 21:02:43 <etotheipi_> :)
360 2012-12-30 21:05:25 <stealth222> well, for instance, a block could contain a "proposal" field - where it can propose to, say, add another 8 bytes at the end of the block header structure. and if 999 out of the last 1000 blocks all agree to do this, then the 1001st block must have a header 8 bytes longer :)
361 2012-12-30 21:06:17 <stealth222> not quite sure how to negotiate what that data means and how to apply rules to it
362 2012-12-30 21:06:31 <stealth222> I suppose the rule modification could also be encoded
363 2012-12-30 21:07:04 <stealth222> it would have to require a very large consensus
364 2012-12-30 21:12:12 <gmaxwell> stealth222: uh. no thats bad. As it would break all the existing users, and mining is not a gauge of user consensus, it's a gauge of mining consensus. (which is the applicable consensus for soft forks but not hard ones).
365 2012-12-30 21:12:52 <gmaxwell> stealth222: I wouldn't want the height in the header, it would needlessly make the headers larger... and the headers ought to be as small as possible.
366 2012-12-30 21:13:46 <gmaxwell> etotheipi_: your first barrier is that your headers need to meet the required minimum difficulty.
367 2012-12-30 21:14:01 <stealth222> gmaxwell: it would only add a few bytes to the header - practically negligible
368 2012-12-30 21:14:19 <etotheipi_> gmaxwell: which is "diffculty=1", right?
369 2012-12-30 21:14:23 <gmaxwell> Right.
370 2012-12-30 21:14:45 <gmaxwell> etotheipi_: a sufficiently smart client would always fetch headers and only bother pulling content for something that is potentially the longest chain POW wise.
371 2012-12-30 21:15:25 <stealth222> this is only a major issue for orphaned blocks
372 2012-12-30 21:15:26 <Guest12689> is mining consensus the only one that's possible to measure
373 2012-12-30 21:15:35 <gmaxwell> (though the reference isn't that kind of sufficiently smart yet??? instead we have checkpoints and a permitted difficulty change window that inhibits flooding.
374 2012-12-30 21:15:35 <stealth222> side chains tend not to be very long
375 2012-12-30 21:15:48 <etotheipi_> gmaxwell: you've already answered my question ... I'm just trying to make the simplest listening node possible, but want to make sure it doesn't miss things, either (such as, in the case of a reorg)
376 2012-12-30 21:16:00 <gmaxwell> stealth222: ... I like you, you're obviously a nice guy that doesn't think about malicious acts... :P
377 2012-12-30 21:16:26 <gmaxwell> Guest54088: using the blockchain, sure.
378 2012-12-30 21:19:56 <sipa> gmaxwell: for choosing a checksum for undo data, i'd like to add some identifying information about the block it's about, to make sure no moved data can be accepted as valid with valid checksum, but be for another block
379 2012-12-30 21:21:15 <sipa> but if you want to use it at some point in an authenticated way, you may want to store (via a merkle tree?) in the coinbase of the block itself
380 2012-12-30 21:21:43 <sipa> probably nothing to worry about now
381 2012-12-30 21:23:08 <gmaxwell> yea, the right thing to do is connect it via the coinbase, since that can accommodate what you want as well as being able to SPVly obtain undo data from peers.
382 2012-12-30 21:24:02 <sipa> right
383 2012-12-30 21:24:27 <sipa> but that makes adding the block hash to the undo-data-checksum-calculation impossible
384 2012-12-30 21:24:42 <sipa> you could use the prevblock hash
385 2012-12-30 21:24:49 <sipa> but that's not perfect
386 2012-12-30 21:26:32 <etotheipi_> sipa: is this for undoing pruned-tree ops during a reorg?
387 2012-12-30 21:27:00 <sipa> well, right now it's just about adding a checksum to the undo data
388 2012-12-30 21:27:23 <sipa> and i want to add the block hash of the block it's about to the calculation, so there can be no mistake
389 2012-12-30 21:27:59 <sipa> but that is incompatible with the idea of one day adding the checksum of the undo data to the coinbase (even indirectly)
390 2012-12-30 21:32:57 <etotheipi_> what is the undo data?  is this a BST?
391 2012-12-30 21:33:20 <etotheipi_> or is it agnostic?  (just txouts added and removed?)
392 2012-12-30 21:33:35 <sipa> for every txin, it lists the txout being spent
393 2012-12-30 21:33:44 <sipa> concatenated
394 2012-12-30 21:34:19 <sipa> (and if it's the last txout of a tx being spent, also the tx version, height, and whether it was a coinbase)
395 2012-12-30 21:45:38 <stealth222> who is adding undo data? to undo what?
396 2012-12-30 21:47:25 <stealth222> you mean you want the block to also have to store information on how to undo it if it gets disconnected?
397 2012-12-30 21:47:34 <sipa> no
398 2012-12-30 21:47:41 <sipa> but the client does need to store that data
399 2012-12-30 21:48:20 <stealth222> so you're looking for a way of storing that information short of having to store all the transactions?
400 2012-12-30 21:48:36 <sipa> no, that's already been solved
401 2012-12-30 21:48:47 <sipa> git head works that way
402 2012-12-30 21:49:00 <stealth222> so then what's the issue?
403 2012-12-30 21:49:04 <sipa> i want it to have checksums
404 2012-12-30 21:50:35 <stealth222> I'm still not following - what are the checksums for?
405 2012-12-30 21:50:47 <gmaxwell> to be sure your undo data isn't corrupted or wrong.
406 2012-12-30 21:51:10 <stealth222> so you got the undo data from an untrusted source?
407 2012-12-30 21:51:29 <gmaxwell> thats not the primary motivation, though potentially that might happen in the future.
408 2012-12-30 21:51:55 <sipa> no you get it from the most trusted source imaginable, yourself
409 2012-12-30 21:52:04 <sipa> but i'm working on database integrity checking
410 2012-12-30 21:52:23 <stealth222> oh, ok
411 2012-12-30 21:52:31 <stealth222> I thought you were talking about some protocol issue
412 2012-12-30 21:52:41 <sipa> well, it's related
413 2012-12-30 21:52:57 <sipa> at some point we may wish to have blocks that authenticate their own undo data
414 2012-12-30 21:53:11 <gmaxwell> At some point we many want to support fetching undo data from peers. (say you've deleted yours to save space and now your peer is asking you to reorg past that point)
415 2012-12-30 21:53:24 <stealth222> right, that's what I was getting at
416 2012-12-30 21:53:46 <gmaxwell> So it would be handy if (1) you could just remember the hashes, and then delete the undo data trusting you can fetch it from someone else. ... or fetch undo data in a SPVish way from peers.
417 2012-12-30 21:54:06 <gmaxwell> Sipa would like to make his integrity checking not gratitiously incompatible with that.
418 2012-12-30 21:54:43 <gmaxwell> But it's a little lame because ideally the undo data and block would be bound. and if we make the undo data include the block hash then we can't have the block data eventually commit to the undo data.
419 2012-12-30 21:54:48 <gmaxwell> Thats all.
420 2012-12-30 21:55:41 <stealth222> you mean circularity
421 2012-12-30 21:55:51 <sipa> indeed
422 2012-12-30 21:56:09 <sipa> from the current state of things, the best solution seems making the undo data depend on the block data
423 2012-12-30 21:56:25 <sipa> from a future point of view, making the block data depend on the undo data would be even nicer
424 2012-12-30 21:56:28 <sipa> and you can't have both
425 2012-12-30 21:57:26 <stealth222> I see
426 2012-12-30 22:01:57 <sipa> happy new year's eve's morning!
427 2012-12-30 22:02:16 <stealth222> thank you :)
428 2012-12-30 22:03:35 <stealth222> in 9 hours it will be new years in samoa :)
429 2012-12-30 22:09:35 <sipa> hmmm; how to do this
430 2012-12-30 22:09:43 <sipa> there are basically two types of checks possible
431 2012-12-30 22:10:04 <sipa> 1) verify block integrity (of data in blk00*.dat)
432 2012-12-30 22:10:12 <sipa> 2) verify coindb integrity
433 2012-12-30 22:10:43 <sipa> the second part basically consists of trying to roll back the database, so it requires step 1 of the blocks being rolled back already
434 2012-12-30 22:10:54 <sipa> but you can't do this very far, as it's memory only
435 2012-12-30 22:11:34 <sipa> and i suppose you'll want to verify more blocks than you can roll back
436 2012-12-30 22:12:05 <sipa> separate -checkblocks and -checkcoindb ?