1 2015-03-20 04:42:09 <grassass>    j b b.
  2 2015-03-20 05:23:19 <grassass> sorry, 2 year old got to my laptop
  3 2015-03-20 10:13:44 <gdistasi> hi everyone: a question related to sidechains: what if the main chain dies?
  4 2015-03-20 10:14:37 <sipa> first ask yourself how that could happen
  5 2015-03-20 10:15:08 <sipa> if it is because neither the main chain or the side chains related to it are no longer used for anything, there is not really a problem
  6 2015-03-20 10:17:11 <wumpus> dies as in 'all miners left'?
  7 2015-03-20 10:17:55 <sipa> that's what i assumed
  8 2015-03-20 10:19:22 <wumpus> well, then, there wil be no new blocks. On the other hand the difficulty would become thus low that someone with a historical interest could use a spare 286 for mining...
  9 2015-03-20 10:42:16 <gdistasi> what if someone rewrite history by mining a different parent chain?
 10 2015-03-20 10:42:51 <sipa> what if someone rewrites history by mining a different bitcoin chain now?
 11 2015-03-20 10:43:04 <sipa> they can. there are economic incentives for not doing so
 12 2015-03-20 10:43:15 <gdistasi> that's the point, bitcoin mining is still active and that's impossible
 13 2015-03-20 10:43:35 <sipa> it's perfectly possible
 14 2015-03-20 10:43:38 <sipa> it's just costly
 15 2015-03-20 10:43:46 <gdistasi> I read in the paper that sidechains could be use to migrate bitcoins to a different chain, but if the parent chain dies, the child chain become orphan
 16 2015-03-20 10:43:53 <sipa> you get the security you pay for
 17 2015-03-20 10:44:05 <sipa> bitcoin does that by paying subsidy and fees to miners
 18 2015-03-20 10:44:14 <sipa> sidechains can't pay subsidy, but there are other means
 19 2015-03-20 10:44:55 <sipa> but if it's more profitable to mine a competing chain that to collaborate on mining the 'actual' chain, people may
 20 2015-03-20 10:45:04 <sipa> this is true for bitcoin and for sidechains
 21 2015-03-20 10:45:18 <sipa> and it depends whether you're talking about merged-mined chains or not
 22 2015-03-20 10:45:39 <sipa> in a non-merge mined chain, the block chains and proof of work are independent, so the main chain can die just fine
 23 2015-03-20 10:46:02 <sipa> if people lose interest in it
 24 2015-03-20 10:46:44 <sipa> in a merged-mined chain, if the financial interest of the main chain disappears, but the sidechain remains interestng, then the sidechain provides the incentive to keep mining the main chain with empty blocks
 25 2015-03-20 10:48:29 <gdistasi> so in the child chain there is some kind of reference to a hash on the parent chain. If the parent chain gets rewritten, the hash is not there anymore
 26 2015-03-20 10:49:19 <sipa> yes
 27 2015-03-20 10:49:51 <sipa> but if the sidechain is valuable (to users and to miners, ignoring for the moment how), then miners will not let the main chain get rewritten
 28 2015-03-20 10:49:58 <sipa> the main chain will retain its hash power
 29 2015-03-20 10:50:42 <sipa> and if that becomes meaningless, the sidechain can always undergo a softfork to disable the peg
 30 2015-03-20 10:50:46 <sipa> and become independent
 31 2015-03-20 12:50:34 <the_last> http://www.reddit.com/r/MachineLearning/comments/2zovcw/breaking_bitcoin_mining_machine_learning_to/ -- "Surely if this works it's a reliable attack on sha256 and is probably the most important crypto discovery this decade?"
 32 2015-03-20 12:51:23 <the_last> Can any bitcoin devs comment on that?
 33 2015-03-20 12:54:00 <wumpus> machine learning is utterly ineffective in learning random functions like sha256
 34 2015-03-20 12:54:46 <the_last> So his approach wouldn't be any more effective than a bruteforce?
 35 2015-03-20 12:54:48 <wumpus> it looks like uniform noise to your brain, and it does to any kind of neural network too
 36 2015-03-20 12:55:15 <wumpus> less effective than brute force; it just won't learn the function
 37 2015-03-20 12:56:52 <wumpus> an approach that is slower than brute force, but at least works, would be using a SAT solver: https://jheusser.github.io/2013/02/03/satcoin.html
 38 2015-03-20 12:58:22 <the_last> I shall read this now. This guy seems to think he's onto something amazing, so I'm confused haha. I thought he had found some form of weakness, otherwise how is a neural net anything new :-/ "A pot of bitcoins await at the end of the random forest!"
 39 2015-03-20 12:58:41 <the_last> He be trippin'?
 40 2015-03-20 13:00:18 <wumpus> probably.
 41 2015-03-20 13:01:20 <theorbtwo> Read the comments?
 42 2015-03-20 13:02:21 <theorbtwo> It suggests that the label is still on the training data, and the machine learning just learned to find the label.
 43 2015-03-20 13:03:21 <the_last> Ah, I hadn't seen those comments yet
 44 2015-03-20 13:14:38 <Adlai> aha, we have a new term: "Bitcoin is an online paymeny system"
 45 2015-03-20 13:18:23 <the_last> yay for paymeny's
 46 2015-03-20 13:19:50 <Adlai> the_last: you may also be interested in the HN discussion on satcoin https://news.ycombinator.com/item?id=8401258
 47 2015-03-20 13:21:59 <gmaxwell> the_last: it's yet another person who mistakes prior and posterior probablities.
 48 2015-03-20 13:27:33 <gmaxwell> If there are batches of numbered marbles, each batch having 10000 marbles ... and one in 1000 marbles is red instead of blue with random with equally distributed probablity, and alice checks the marbles starting with the lowest number, saving the red ones and moving onto the next batch any time she finds a marble,  what distribution do you expect for the numbers on the marbles she finds?
 49 2015-03-20 13:39:37 <gavinandresen> gmaxwell… any time she finds a marble what? finds a marble with a number lower than the one she just drew?
 50 2015-03-20 13:40:03 <gavinandresen> anytime she finds a red marble?
 51 2015-03-20 13:40:27 <gmaxwell> she keeps it and moves on to the next batch.
 52 2015-03-20 13:40:53 <gavinandresen> she’s just keeping red marbles?
 53 2015-03-20 13:41:20 <sipa> assuke she takes one ball per second
 54 2015-03-20 13:41:30 <gavinandresen> … I’d like the problem better if there were eleven in each batch.
 55 2015-03-20 13:41:36 <sipa> every how many seconds does she find a red ball?
 56 2015-03-20 13:41:37 <gavinandresen> And the red marbles were orange.
 57 2015-03-20 13:41:37 <gmaxwell> Yep. "saving the red ones, and moving on to the next batch of numbered marbles as soon as she finds one"
 58 2015-03-20 13:41:57 <gavinandresen> and the question is what numbers are on the red marbles?
 59 2015-03-20 13:42:07 <sipa> 11
 60 2015-03-20 13:42:08 <sipa> in hex
 61 2015-03-20 13:42:46 <gavinandresen> Anyway, I don’t understand the problem, but my first answer is “random” and my second is “poisson”
 62 2015-03-20 13:43:19 <gavinandresen> I’d like poisson distributions better if they were called “fish distributions”
 63 2015-03-20 13:43:50 <tramptac> poisson with average of 500 ?
 64 2015-03-20 13:44:24 <gmaxwell> You should expect an exponential distribution with lots of small numbers, since sometimes by chance the first red marble will be really low numbered and she stops looking in that batch when she finds one.
 65 2015-03-20 13:45:11 <gavinandresen> oh, I misread the 10,000 and 1,000 as 10,000 and 10,000
 66 2015-03-20 13:45:33 <gavinandresen> … so I thought there was one red marble per batch.
 67 2015-03-20 13:45:34 <Adlai> probability of n-1 blue followed by one red = (99/100)^n ?
 68 2015-03-20 13:46:01 <Adlai> no, that's definitely wrong
 69 2015-03-20 13:46:26 <gavinandresen> Probability problems are off-topic here, though, I should stop trolling :)
 70 2015-03-20 13:46:43 <Adlai> we're just trying to outsmart bruteforce sha256d!
 71 2015-03-20 13:47:20 <gmaxwell> well, it helped me refine my response on reddit, changed to a percent instead of 1 in 1000.
 72 2015-03-20 13:47:24 <gavinandresen> On-topic, sipa may be interested: I’ve got a patch that limits the number of transactions added back into the mempool if there is a long re-org
 73 2015-03-20 13:48:03 <hearn> is this an interview question or something?
 74 2015-03-20 13:48:55 <gavinandresen> Questions for bitcoin-dev are:  how many blocks deep is it reasonable to throw transactions back into the mempool?  If you’re re-orging 10-blocks-deep to a new 11-block-deep chain, most of the transactions SHOULD be in the new chain, so putting them back in the mempool just bloats memory
 75 2015-03-20 13:48:58 <gmaxwell> hearn: no someone posted on reddit some zomg I compromised sha256 because I noticed a pattern in nonces, see the link above from "the_last".
 76 2015-03-20 13:49:07 <gavinandresen> This is prep-work for a memory-limited mempool, mostly
 77 2015-03-20 13:52:10 <gmaxwell> gavinandresen: unfortunately the ones at the new tip when you reorg are the ones that wouldn't be in it. But you can't have them in the mempool without having their parents there.
 78 2015-03-20 13:53:25 <gavinandresen> gmaxwell: I’m not following…
 79 2015-03-20 13:54:29 <Ge0rG> hi. trying to build bitcoind from source on debian stable, getting this error: http://pastebin.com/mdhi9YCJ - already re-ran "make distclean", autogen.sh, configure etc.
 80 2015-03-20 13:55:37 <gavinandresen> gmaxwell: what’s the scenario?  A transaction in the mempool before re-org that relies on a transaction that is N-deep in the re-org’ed chain and also in the new chain?
 81 2015-03-20 13:55:41 <gmaxwell> so you reorg from 111 down 10 blocks and back to a brand new 112.  Its likely that some of the txn you had in your mempool at the start are not in the new 112 you end up on. It's unlikely that any of the transactions you'd put into the mempool along your way as you disconnect are not in the new chain ending in 112. But you cannot fail to put those along-the-way transactions in your mempool as you go,
 82 2015-03-20 13:55:47 <gmaxwell>  if you want to keep the ones you had there at 111 and end up with them temporarily orphaned.
 83 2015-03-20 13:56:27 <dhill> n/win 5
 84 2015-03-20 13:56:40 <gavinandresen> gmaxwell: do you really care?  Long re-orgs never happen in practice— they’re likely an attack.  In which case losing some mempool transactions is ‘meh'
 85 2015-03-20 13:57:22 <gavinandresen> we lose the entire mempool all the time (every time you restart your node)
 86 2015-03-20 13:57:47 <gmaxwell> Well what do you call large? reorgs two deep happen with pretty good regularity.
 87 2015-03-20 13:57:58 <gmaxwell> We don't lose it on all the nodes at once though.
 88 2015-03-20 13:58:04 <gavinandresen> I picked ‘3’, but am thinking ‘6’ might be better.
 89 2015-03-20 13:58:49 <gavinandresen> … eleven has a nice sound, though....
 90 2015-03-20 13:59:31 <gmaxwell> If you really know the size of the reorg in advance and are only trying to optimize that case, then I think it could basically get away with freezing the mempool after the first disconnected block and only checking it again at the end of the reorg.
 91 2015-03-20 13:59:56 <gavinandresen> gmaxwell: I’m not going to write that code….  way more complicated
 92 2015-03-20 14:00:35 <gmaxwell> Well what /are/ you thinking of?  My response to you saying that you were thinking of a particular depth target was "oh well, yea, but thats going to be complex to implement".
 93 2015-03-20 14:02:04 <gavinandresen> https://github.com/gavinandresen/bitcoin-git/commit/da7fc4f3c3227d36004ec08a3abfbdc9cb3d4c97
 94 2015-03-20 14:05:43 <gavinandresen> I’m looking at the wallet re-broadcast code and trying to figure out a way of testing it.
 95 2015-03-20 14:06:57 <gmaxwell> Oh I see, yea, it should probably be greater than 6 ... 11 or something would be fine. Hardly an increase in work at all, but hopefully takes it into the never happens except in testing realm. (there is a nice parity in matching it up to confirm counts people are looking for, in that a reorg with less than that an the network is at least agressively trying to recover your transaction that falls out)
 96 2015-03-20 14:07:21 <gavinandresen> Six it is.
 97 2015-03-20 14:09:29 <gmaxwell> since we don't store parents in the wallet 'anymore', seems likely that parents will just get lost in big reorgs, so potentially rebroadcasting won't work for people.
 98 2015-03-20 14:10:17 <gavinandresen> there are more common cases where a wallet transction gets stuck:  like parent is double-spent or malleated
 99 2015-03-20 14:10:21 <gmaxwell> e.g. you recieve a transaction whos parent was removed earlier in the reorg.  The sender has gone away and you're unable to rebroadcast then.  (I don't consider this a blocker or anything, just a consequence)
100 2015-03-20 14:11:54 <gavinandresen> dealing better with -1 confirmation transactions is definitely a good idea.  I’m cracking open the ResendWalletTransactions code to check to see if wallet transactions drop out of the mempool, and I’ll LogPrint a useful message if they can’t go back in the mempool for some reason (don’t have parent, etc)
101 2015-03-20 14:12:30 <gavinandresen> Dealing better with those edge cases I’m going to leave for some future MegaWalletImprovement project
102 2015-03-20 14:15:21 <gmaxwell> Related; I'd like to see the wallet code not depend on mempooling itself for the -1 check, just the 'ability to mempool' or something; because I'd like to see possible to disable automatic transaction transmission (so to send a transaction you must getrawtransaction -> sendrawtransaction manually or via a more private announcement path). It's trivial to disable transmission, but right now if you do i
103 2015-03-20 14:15:27 <gmaxwell> t everything ends up -1 which is surprising. (Instead if should be 0, unannounced; instead of -1 conflicted)
104 2015-03-20 14:17:29 <gavinandresen> So… a mempool.WouldAccept(CTransaction) that returns validation state?
105 2015-03-20 14:18:40 <gmaxwell> Or even just a IsConflicted that only checks the input ids?
106 2015-03-20 14:19:37 <morcos> yes, i really like the idea of optionally disabling wallet retransmission
107 2015-03-20 14:20:38 <morcos> how important is it to put transactions back in the mempool at all during a reorg?  it seems like disconnecting blocks is one of the most expensive parts of the code
108 2015-03-20 14:21:07 <morcos> and could possibly be something you'd want to live without if you were running on limited hardware.. (not disconnecting, but saving the tx's from the stale blocks)
109 2015-03-20 14:21:30 <morcos> especially if we're talking about scaling up in the future
110 2015-03-20 14:22:13 <gavinandresen> morcos: gmaxwell’s point about everybody forgetting about recent transactions at the same time is a very good one
111 2015-03-20 14:23:07 <gavinandresen> if nodes don’t remember, then wallets would need to be pushier about rebroadcasting after a re-org. And that would be bad for bandwidth (which will be the likely bottleneck)
112 2015-03-20 14:23:49 <morcos> yeah, when i read that my first thought was hmm maybe your wallet should save its own parents.  but ha, i wasn't around when it used to do that, but assume it was removed for a reason
113 2015-03-20 14:23:54 <gmaxwell> I don't think single nodes forgetting it is a big deal, everyone forgetting is .. uh not great. at least at a few deep, and the cost of retaining is never more than the block size you disconnected.
114 2015-03-20 14:24:31 <gmaxwell> morcos: the code that did that was just broken. It would make sense to do so, but OTOH, it can result in shockingly large wallets.
115 2015-03-20 14:24:31 <hearn> gavinandresen: why not store the mempool in a leveldb during reorgs. allows enormous depth.
116 2015-03-20 14:24:59 <gavinandresen> hearn: because huge reorgs never happen except in stress tests….
117 2015-03-20 14:25:01 <gmaxwell> hearn: because this is already motivated by huge reorgs in testing being crazy slow.
118 2015-03-20 14:25:39 <morcos> i've been meaning to explore exactly what it is that makes reorgs so slow... so you don't think reasonable sized reorgs (say < 6) are slow enough to be a bottleneck?
119 2015-03-20 14:25:56 <hearn> gavinandresen: yes but causing things to break if they do happen is not great - we already have some arbitrary numbers where we assume reorgs can never happen that deep, but chain splitting bugs can happen
120 2015-03-20 14:26:40 <gavinandresen> morcos: I bet it is that the UTXO caching code hasn’t been optimized for re-orgs (because reorgs happen so rarely), but I haven’t benchmarked it either.
121 2015-03-20 14:26:44 <wumpus> we don't really 'forget' about the other fork after a reorg, sure the transactions disappear from view, but the blocks are still stored, and could be accessed  to (say) rebroadcast them
122 2015-03-20 14:26:48 <hearn> gavinandresen: bear in mind a leveldb with memcache on an HDD basically just writes things out to a log file. and you can turn off write consistency if you're willing to accept needing to retry the reorg if there's a disk failure half way through ( seems reasonable ) - so the perf hit should be manageable
123 2015-03-20 14:26:54 <gmaxwell> morcos: it spends a lot of time reupdating the mempool redundantly, at least.
124 2015-03-20 14:27:07 <hearn> i mean i'd want to profile it to be sure, but i think it's not totally certain to be a big hit
125 2015-03-20 14:27:17 <wumpus> (rebroadcast the transactions in them, not the blocks)
126 2015-03-20 14:27:35 <gmaxwell> wumpus: indeed it could all be handled differently, e.g. rescanning the disconnected blocks after the reorg, but thats a lot more code, I think.
127 2015-03-20 14:28:21 <wumpus> gmaxwell: sure, but it could even be done manually I mean; the information is not lost
128 2015-03-20 14:28:23 <gavinandresen> hearn: will bitcoinj rebroadcast unconfirmed transactions?
129 2015-03-20 14:28:36 <wumpus> gmaxwell: it's not forgotten, just sort-of inaccessible
130 2015-03-20 14:28:37 <hearn> yes but it doesn't store the parents.
131 2015-03-20 14:28:45 <gavinandresen> … or, used-to-be-confirmed-but-then-got-unconfirmed-by-a-reorg transactions?
132 2015-03-20 14:29:05 <morcos> i like wumpus's idea as well.  a lot of these things will make mempool pruning more robust if we have other mechanisms
133 2015-03-20 14:29:10 <gavinandresen> hearn: The parents should be rebroadcast by whoever created them if they are unconfirmed, yes?
134 2015-03-20 14:29:27 <hearn> yes when a reorg occurs it does a similar operation to Core - it undoes the blocks, marking the txns as unconfirmed, and then replays the new chain taking them out of the pending state again
135 2015-03-20 14:29:49 <gmaxwell> morcos: so what isn't in blocks on disk is the current mempool... violating the invarent that all txn in the mempool are not orphans is not fun.
136 2015-03-20 14:29:53 <hearn> only if the wallets are online
137 2015-03-20 14:30:02 <hearn> otherwise the transactions become orphans and don't relay, and might get dropped
138 2015-03-20 14:30:12 <hearn> *eventually* things should sort themselves out, but it could take a long time
139 2015-03-20 14:30:24 <hearn> and that's assuming all wallets implement reorgs correctly. i strongly suspect many don't.
140 2015-03-20 14:30:36 <morcos> but couldn't you after a reorg, just scan the blocks that got reorged out and add back / rebroadcast any needed qualifying transactions
141 2015-03-20 14:30:48 <wumpus> morcos: yes, you could
142 2015-03-20 14:31:08 <gavinandresen> hearn: there was a good paper at IFCA’15 about broken wallets.....
143 2015-03-20 14:31:15 <morcos> then the default could be not to add tx's back during the reorg, which would allow the network to all get to the same chain quicker
144 2015-03-20 14:31:21 <hearn> ah, is it online anywhere? i know all kinds of ways bitcoinj is broken :)
145 2015-03-20 14:31:25 <wumpus> morcos: though if you send them all at the same time, you'll again overflow the mempool :-)
146 2015-03-20 14:31:34 <wumpus> morcos: but it could be a delayed process
147 2015-03-20 14:31:58 <gavinandresen> hearn: http://fc15.ifca.ai/preproceedings/bitcoin/paper_9.pdf
148 2015-03-20 14:32:07 <hearn> morcos: the problem is knowing which transactions are needed. bear in mind after a reorg all you have is the ledger/UTXO set and a bunch of unconfirmed transactions.
149 2015-03-20 14:32:21 <alfacent> gmaxwell, hi, do you know of any altcoin that is currently implementing your idea of POW which turns the distributed computation into ticking for timelock encryption?
150 2015-03-20 14:32:21 <gmaxwell> hearn: on the things they were comparing only bitcoin core wasn't broken. There was a real facepalm inducing table... okay broken broken broken broken broken ... but they were only looking at something fairly narrow.
151 2015-03-20 14:32:23 <gavinandresen> hearn:  See Table 1 on page 8….
152 2015-03-20 14:32:27 <morcos> wumpus: i like the idea though that after a reorg, when you scan through the stale blocks...   oh.. i see, you won't be able to identify which ones were in the new chain and spent already
153 2015-03-20 14:32:40 <hearn> morcos: now you want to know which mempool txns are dependent on transactions that are only in the old chain, but the ledger doesn't tell you that
154 2015-03-20 14:33:04 <hearn> gavinandresen: ah yes i think i've seen this one. yeah it only looks at malleability rather than more general wallet stuff
155 2015-03-20 14:33:11 <morcos> hearn: well i was assuming you were still cleaning stuff out of the mempool during the reorg, just not adding back in
156 2015-03-20 14:33:12 <gmaxwell> alfacent: no, there are real challenges in doing that, since it needs asymetric encryption where cracking is fully exponential (progress free with no tmto) to really work well.
157 2015-03-20 14:33:59 <hearn> there's a patch about to go into bitcoinj which adds an equivalent of the conflict state for pending transactions. i think after that work is done and wallets catch up they'd stop being broken in the sense that paper is meaning.
158 2015-03-20 14:34:05 <morcos> while i have all your attentions, do we intend for #5347 to make it imposible to sign a double spend using the rpc call?
159 2015-03-20 14:35:07 <morcos> that seems like a potentially big change to me
160 2015-03-20 14:35:25 <alfacent> Do you think 'homomorphic encryption' may reveal very promising in terms of time-lock encryption?
161 2015-03-20 14:35:35 <hearn> also iirc the paper seemed to be using an old version of Bitcoin Wallet. it is supposed to correctly detect the mutated transaction as double spent once it confirms, and we've seen it do that for real, so i can't quite recall why the paper saw something different. anyway .....
162 2015-03-20 14:36:28 <wumpus> morcos: the whole PR is only meant as a cleanup right? yes, I think such a change in behavior in unacceptable
163 2015-03-20 14:37:21 <morcos> wumpus: ok, i hadn't seen any feedback on that comment the first time i posted it, and wasn't sure whether that was an intentional ability or not
164 2015-03-20 14:37:28 <gmaxwell> morcos: hm. Well I suspect it doesn't actually make it impossible, you'd have to provide the inputs like you do for generally unknown inputs; though that would be a surprising change.
165 2015-03-20 14:38:03 <morcos> gmaxwell: ah, good point. i always forget about that
166 2015-03-20 14:51:54 <dgenr8> a discrete exponential is "geometric" ... hashing is the ultimate example ... the exponential approximation works pretty darn well there
167 2015-03-20 14:56:50 <Lucent> hi devs
168 2015-03-20 15:00:43 <dgenr8> i found it's harder to test malleable txes than it used to be ;)
169 2015-03-20 15:01:34 <dgenr8> the cleanstack test has foiled my latest attempt
170 2015-03-20 15:31:28 <gavinandresen> dgenr8: what are you trying to test?  Re-signing a transaction is a good way to create two identical-but-different-txid transactions
171 2015-03-20 15:32:03 <sipa> gavinandresen: not anymore since deterministic signing :)
172 2015-03-20 15:32:16 <zooko> ☺
173 2015-03-20 15:32:51 <dgenr8> all i've got right now is one with, one without ANYONECANPAY
174 2015-03-20 15:33:06 <dgenr8> as per sipa's master designs
175 2015-03-20 15:33:06 <gavinandresen> git HEAD is deterministically signing now? I don’t know how I missed that….  does 0.10.0 deterministically sign?
176 2015-03-20 15:33:31 <sipa> gavinandresen: ... yes
177 2015-03-20 15:33:44 <sipa> did you read the release notes? :p
178 2015-03-20 15:33:48 <Happzz> does bitcoin core cache peers it used to connect to? i mean, when it restarts, does it seek for peers to connect to all over again, or does it have the last peers it contacted already?
179 2015-03-20 15:34:13 <gavinandresen> spiffy.  Ok, then you could fiddle with nSequence or nLockTime in a way that doesn’t actually lock the transaction to create two .....
180 2015-03-20 15:34:27 <dgenr8> those are signed ...
181 2015-03-20 15:34:29 <gavinandresen> sipa: I keep telling you my brain is full......
182 2015-03-20 15:34:38 <sipa> gavinandresen: good problem to have :)
183 2015-03-20 15:34:54 <sipa> Happzz: it keeps a database of known and heard IP addresses, and statistics about them
184 2015-03-20 15:35:12 <Happzz> sipa is that database kept between restarts?
185 2015-03-20 15:35:15 <sipa> Happzz: yes
186 2015-03-20 15:35:22 <sipa> peers.dat file
187 2015-03-20 15:35:29 <Happzz> does it take that database into consideration every time it needs peers?
188 2015-03-20 15:35:33 <sipa> yes
189 2015-03-20 15:35:39 <gavinandresen> dgenr8: right, fiddle with those and then re-sign (so deterministic signing actually creates a different signature)
190 2015-03-20 15:35:39 <Happzz> alright. thanks.
191 2015-03-20 15:36:07 <Happzz> sipa it doesnt seem like a plaintext. what format is peers.dat?
192 2015-03-20 15:36:20 <sipa> Happzz: see addrman.h; it's rather complicated
193 2015-03-20 15:36:28 <Happzz> okay. thanks.
194 2015-03-20 15:36:58 <sipa> we should have a tool to dump/convert it to something readable
195 2015-03-20 15:37:29 <dgenr8> gavinandresen: they're not equivalent clones at that point, since they have different effects.  your code if IsEquivalentTo is so much prettier btw
196 2015-03-20 15:38:11 <gavinandresen> that’s my goal in life, to write pretty code.....
197 2015-03-20 15:39:46 <dgenr8> well better too, since it adapts to future changes more naturally
198 2015-03-20 15:40:20 <gavinandresen> dgenr8: hmm… the malleability fixes will make testing harder.  sipa, is there some way to get non-deterministic signing?
199 2015-03-20 15:41:16 <sipa> CKey.Sign has an optional argument to 'tweak' the signature
200 2015-03-20 15:41:22 <sipa> but it should absolutely only be used by tests
201 2015-03-20 15:41:40 <sipa> and it's ugly, as it's very low level - exposing it through the transaction signing code looks very ugly
202 2015-03-20 15:42:02 <gavinandresen> dgenr8 is working in Python I bet.  I wonder how big transaction signing is in pure python code………
203 2015-03-20 15:42:09 <sipa> with #5208 it would become a lot easier
204 2015-03-20 15:42:13 <sipa> ah
205 2015-03-20 15:42:50 <dgenr8> since RPC supports sighash type it'll be fine
206 2015-03-20 15:43:07 <gavinandresen> dgenr8: spiffy
207 2015-03-20 15:43:49 <gavinandresen> … you can thank hearn, I think he convinced me the RPC should support hash type (I might be misremembering, my brain is full....)
208 2015-03-20 15:44:36 <hearn> i can't remember either but i'll happily take the credit for it anyway ;)
209 2015-03-20 15:44:50 <dgenr8> morcos: thanks for the idea to use -relaypriority=0 in tests where fees are just a nuisance.  that will make things a lot cleaner
210 2015-03-20 15:45:05 <wumpus> sipa: re abstract BaseSignatureCreator/BaseSignatureChecker: what other data types except transactions do we intend to sign?
211 2015-03-20 15:45:26 <sipa> wumpus: alerts would be nice
212 2015-03-20 15:45:42 <sipa> anything to be signed where multisig makes sense
213 2015-03-20 15:46:01 <wumpus> right, just wondering
214 2015-03-20 16:17:45 <monk___> Der Spammer hat deinen Link gerade im IRCnet gepostet ;D
215 2015-03-20 16:17:45 <monk___> http://5.45.103.49
216 2015-03-20 16:17:46 <monk___> rofl :D sehe ich
217 2015-03-20 16:20:50 <monk__> Der Spammer hat deinen Link gerade im IRCnet gepostet ;D
218 2015-03-20 16:20:52 <monk__> http://5.45.103.49
219 2015-03-20 16:20:55 <monk__> rofl :D sehe ich
220 2015-03-20 16:21:14 <denisx> monk__: Du bist der spammer!
221 2015-03-20 19:38:45 <Naked> hopefully this is not a terribly wrong channel for this, but I'd like to briefly talk about return addresses for refunds in bitcoin
222 2015-03-20 19:39:35 <Naked> bitcoin as such has no return addresses, and sending money back to the address it came from is a bad idea in general - this I know
223 2015-03-20 19:39:40 <helo> Naked: like the payment protocol?
224 2015-03-20 19:40:35 <gmaxwell> Naked: payment protocol lets the payer specify a return address.
225 2015-03-20 19:40:40 <Naked> gavin's payment protocol has provisions for return address, but it adds a lot of other things too - basically requiring https, pki, etc. for merchant authentication
226 2015-03-20 19:41:25 <Naked> also, the return address in that protocol is in no way authenticated - anybody seeing the payment url can eavesdrop the network and submit a payment message with his own return addresses if he can just see the transactions that are needed as outputs
227 2015-03-20 19:41:35 <gmaxwell> Payment protocol doesn't require PKI, you can send a payment request as a file. (though you do need to be able to recieve the response)
228 2015-03-20 19:42:06 <gmaxwell> Yea, indeed, the whole handling of the response in it is kind of lame.
229 2015-03-20 19:42:33 <gmaxwell> I'm fond of sipa's alternative, which is one where you send the payee an encrypted copy of the transaction and metadata, so its impossible for them to get the transaction without getting the metadata.
230 2015-03-20 19:42:38 <Naked> so, my first question is simply that is this the state of return addresses in bitcoin as of now, or am I missing something crucial?
231 2015-03-20 19:43:14 <gmaxwell> sipa: ^ another application for being able to have non-broadcast / non-broadcasting transactions in the wallet.
232 2015-03-20 19:43:24 <gmaxwell> Naked: as far as I know.
233 2015-03-20 19:43:47 <Naked> okay, so, my proposal would be:
234 2015-03-20 19:43:49 <gmaxwell> Naked: payment protocol _can_ be used in a way that doens't have the issues you've raised.
235 2015-03-20 19:44:32 <gmaxwell> (e.g. you do not broadcast the txn seperately-- just pass it back in the https payment response, but there is no way in the protocol to _mandate_ that, AFAIR)
236 2015-03-20 19:44:57 <Naked> have the sender address sign a message which has a transaction id, and a return address - which simply means that *if* this transaction ever has anything refundable, send it here
237 2015-03-20 19:45:21 <Naked> this signed message can either be sent in-band here - or there can be a global website which just collects these signed messages for each bitcoin transaction
238 2015-03-20 19:45:50 <Naked> this also works perfectly for shared wallets such as exchanges, as long as they sign these messages and provide a unique "return address" for each user
239 2015-03-20 19:45:59 <gmaxwell> Naked: sign with _what_ ?  Global website has severe confidentiality problems.
240 2015-03-20 19:46:03 <Naked> (since the signed message identifies the specific transaction)
241 2015-03-20 19:46:20 <Naked> sign with the same private key used to send the bitcoin - you know, the bitcoin signed message stuff
242 2015-03-20 19:46:38 <Naked> this stuff: https://bitcointalk.org/index.php?topic=6430.msg6186816#msg6186816
243 2015-03-20 19:47:04 <gmaxwell> Naked: there may not be any private key, or (very commonly) there isn't one. And the party signing the transaction may not be the logical sender, or the private keys may be smartly embedded in a HSM which doesn't permit signmessages.
244 2015-03-20 19:47:16 <gmaxwell> er (very commonly) there isn't only one.
245 2015-03-20 19:47:41 <gmaxwell> The transaction may have signatures on it from three or four keys controlled by three or for different parties...
246 2015-03-20 19:47:59 <gmaxwell> A pretty substantial chunk of coins are now also controlled by multisignature... etc.
247 2015-03-20 19:48:01 <Naked> granted, this requires changes to anything that is signing transactions - to sign a message in addition to the transaction, *if* return addresses are required for that transaction
248 2015-03-20 19:48:34 <Naked> yeah, might be that there is too much inertia for anything like that anymore...
249 2015-03-20 19:49:18 <gmaxwell> nothing I said has anything to do with intertia except perhaps hsm support. The rest is that what you're thinking about is only compatible with the narrowest possible set of use cases.
250 2015-03-20 19:50:52 <Naked> might be - I really don't know what all is done with bitcoin wallets these days
251 2015-03-20 19:52:16 <Naked> okay, thanks for letting me pitch the idea, though
252 2015-03-20 19:52:25 <Naked> so, back to payment protocol
253 2015-03-20 19:52:28 <gmaxwell> no problem! it's a hard subject area.
254 2015-03-20 19:52:59 <Naked> the use case I am thinking is POS transactions (I make payment terminal software)
255 2015-03-20 19:53:26 <Naked> and there, being able to cancel a transaction is crucial
256 2015-03-20 19:54:39 <Naked> what I mean is that if I show a QR code for a transaction - and the customer maybe uses a phone to scan it and tries to make a payment but fails for some reason
257 2015-03-20 19:55:04 <Naked> when the merchant decides to abort the transaction, and asks the customer to pay with cash, there should be no chance that the merchant ends up holding the money via bitcoin as well
258 2015-03-20 19:55:49 <Naked> which means that there either needs to be a way to ensure that the money cannot be transferred to the merchant anymore - or there needs to be a way to return funds if they are
259 2015-03-20 19:56:53 <Naked> actual refunds can require the customer to just give a new bitcoin address - because then the customer always has a receipt and other identifying information on the transaction
260 2015-03-20 19:57:23 <Naked> but aborted transactions often do not generate a receipt or any other information that can identify the customer later on
261 2015-03-20 19:58:45 <Naked> one way to try and tackle this would be to make it mandatory for payment protocol software to *first* send the Payment message with transactions to the merchant, and only if the merchant responds with a success PaymentAck should the client broadcast the transactions on the bitcoin network
262 2015-03-20 19:59:38 <andytoshi> a few days ago phantomcircuit said checkpoints couldn't be removed without fraud proofs or something ... can someone clarify why headers-first + replacing checkpoints with a "minimum total chain difficulty" is insufficient?
263 2015-03-20 20:00:09 <Naked> this way, if the merchant knows it will reject all Payment messages from this point onwards and there haven't been any so far, he can be sure that he won't end up holding the money unless the user actively is trying to give his money away
264 2015-03-20 20:04:04 <hearn> Naked: originally the payment protocol was intended to work that way. there was some disagreement between myself and gavinandresen about whether tx submission should be exclusively via the payment_url or not. unfortunately i don't quite remember why.
265 2015-03-20 20:04:26 <hearn> Naked: some wallets started out by only submitting this way, but bitpay had problems with their bip70 server several times and some wallets decided to broadcast at the same time to work around their flakyness :(
266 2015-03-20 20:04:34 <hearn> Naked: so to get things back on track now would probably require an extension
267 2015-03-20 20:04:56 <hearn> Naked: re: signing. payment_url is meant to be HTTPS in any reasonable implementation. so i do not think that's a big deal. yes signing with one of the private keys is probably workable but why do it?
268 2015-03-20 20:05:04 <Naked> anyway, handling of transaction cancellations is a big blocker for POS integration - we've done integrations to pretty weird payment methods but they all have to handle this somehow
269 2015-03-20 20:05:30 <hearn> Naked: you should be able to receive return addresses via BIP70. if a wallet isn't implementing BIP70 or not providing a refund address, bug them about it .... make it clear to them that it matters
270 2015-03-20 20:07:19 <Naked> I guess the best way to make it work right now is to just give the QR code that has no bitcoin address, only the payment request https address...
271 2015-03-20 20:08:01 <Naked> that way there is atleast BIP70 support - and if the return address is missing then yes, that can be easier to get fixed
272 2015-03-20 20:09:10 <Naked> however, I am worried about reliability - I mean clients will have to be careful that whenever they pay a payment request, they make sure that a Payment message is sent to the payment_url, and sending that message is retried forever with some reasonable backoff
273 2015-03-20 20:09:34 <Naked> otherwise having the backend down for bit might again mean you are unable to refund
274 2015-03-20 20:10:01 <Naked> and I doubt anything but the server driven wallets do anything like that
275 2015-03-20 20:11:28 <hearn> the standard for wallets is to alert the user that the payment didn't work, if the payment_url is unreachable or returns an error
276 2015-03-20 20:12:07 <hearn> I don't think automatic retry is a great idea .... it's sufficient for a wallet to broadcast *after* successful submission to the payment_url (if needed)
277 2015-03-20 20:12:07 <Naked> hearn: yeah, but if they broadcasted the transactions to the bitcoin network already, that means the merchant has to make the refund
278 2015-03-20 20:12:45 <Naked> hearn: and any sort of error message during the payment is likely to cause the merchant and/or customer to use cash instead, meaning the item will be paid for twice
279 2015-03-20 20:12:46 <hearn> perhaps we can update BIP70 and wallets to say: if a bitcoin URI doesn't specify an address at all and is BIP70 only, then it is REQUIRED to use the payment_url and the transactions MUST NOT be broadcast onto the network. however if the address is present, it's acceptable to broadcast
280 2015-03-20 20:12:58 <hearn> that would largely preserve the existing behaviour and wallets could be adjusted to use this new approach quite easily.
281 2015-03-20 20:13:18 <hearn> would that be ok, do you think?
282 2015-03-20 20:13:18 <Naked> hearn: if BIP70 could be updated to handle this properly, I would be thrilled
283 2015-03-20 20:13:44 <hearn> yeah so far merchants tend to see BIP70 as a way to do UX polish and (hopefully soon) improve security, by using the identity with a second factor
284 2015-03-20 20:13:54 <hearn> i think you're the first one who is using a policy of BIP70-or-die
285 2015-03-20 20:14:13 <Naked> that would be okay... but it is somewhat hard for the wallet side though
286 2015-03-20 20:14:28 <hearn> some wallets don't implement BIP70 at all yet, so you'll lose some sales this way. however AFAIK all wallet developers have committed to supporting it at some point, it's not a controversial feature, so perhaps if you get some popular merchants that would poke them into action
287 2015-03-20 20:14:44 <hearn> nah it should be easy for most wallets if they're properly written. what would be hard about it?
288 2015-03-20 20:15:02 <Naked> because, if you send the transactions to the payment uri - even if the merchant responds with an error, there's nothing preventing the merchant from broadcasting the transactions on the bitcoin network later
289 2015-03-20 20:15:28 <Naked> so the wallet has to emphasize the fact that the merchant did not want the transaction, but might take your money later on anyway
290 2015-03-20 20:16:28 <hearn> the thing that "prevents" them is that the users wallet might double spend the coins.
291 2015-03-20 20:16:49 <hearn> of course it also might not, but they can't rely on that. do you think it's really a common issue that the merchant returns an error but then actually processes the payment anyway?
292 2015-03-20 20:16:55 <hearn> seems like the right fix is to just say, "don't do that"
293 2015-03-20 20:17:36 <Naked> yeah, the wallet software could immediately double spend the coins towards some other wallet address when receiving an error from the payment uri, but that seems a bit complex as well
294 2015-03-20 20:18:43 <Naked> no, it's not a common issue - but just as there are fraudulent customers there are fraudulent merchants - and what is important usually is to conclusively determine the outcome while physically present at the shop
295 2015-03-20 20:19:16 <Naked> so both the merchant and the customer should be certain either that the merchant has the money or the customer has the money, but not that the merchant can take the money later, or the customer can take the money back later
296 2015-03-20 20:19:29 <hearn> i think it'd be sufficient to say that if this ever becomes an actual problem for real users, just spend the coins back to yourself when faced with an error. or blacklist that merchant in the wallet software :)
297 2015-03-20 20:19:45 <hearn> (assuming signing)
298 2015-03-20 20:19:50 <hearn> well, that's what broadcasting the tx does
299 2015-03-20 20:20:09 <hearn> effectively it ensures the merchant has the money. period, end of story. if the merchant then whines because their server crapped out and they didn't get a refund address, ok, run a more reliable server.
300 2015-03-20 20:20:20 <hearn> it'll have to be resolved manually in that case
301 2015-03-20 20:22:31 <Naked> hmmh... because of this "merchant takes the money" later aspect, I think it might be safer to just ensure the merchant gets a refund address - and ensure the merchant refunds the coins right away
302 2015-03-20 20:23:13 <Naked> that way the customer can possibly see the refund in the wallet software before leaving
303 2015-03-20 20:24:37 <Naked> I am more than a bit wary of solutions that expect goodwill or nice behavior on some party
304 2015-03-20 20:25:05 <Naked> credit card transactions always have chargebacks, so they don't need to be as strict
305 2015-03-20 20:25:08 <hearn> why would you refund the coins right away? like, if the customer isn't happy with what they got?
306 2015-03-20 20:25:35 <hearn> well yeah the refund address is meant to be a sort of chargebacks-lite, but without the dispute mediation. the idea is you use it later, rather than immediately after a payment.
307 2015-03-20 20:26:03 <Naked> hearn: user wants to pay with bitcoin, user starts payment with bitcoin, merchant doesn't see the coins, user decides to pay with cash instead, merchant accepts cash, user wants to see the sent bitcoins *returned* before leaving shop
308 2015-03-20 20:26:37 <Naked> hearn: yeah, I get it and that's exactly the right method for ecommerce
309 2015-03-20 20:27:04 <hearn> why would the merchant not see the coins?
310 2015-03-20 20:27:19 <Naked> in ecommerce, you usually don't ship right away, you can ship when the payment arrives instead of having to ship within 60 seconds, and you want to refund later on
311 2015-03-20 20:27:53 <Naked> in face-to-face sales you need the money on the spot, and you need to be able to cancel the transaction on the spot and you have no way of reaching the customer after he's left the shop
312 2015-03-20 20:28:20 <Naked> hearn: a hypothetical scenario where there's a network problem somewhere or similar - that sort of thing happens often enough
313 2015-03-20 20:30:51 <Naked> oh well, this needs more thought in general - and more work to be able to make it plausible to use this
314 2015-03-20 20:31:45 <Naked> for example, right now, it seems that bitpay API does not expose the BIP-70 return addresses in anyway in the ledger or the online messages
315 2015-03-20 20:31:58 <hearn> so far we haven't really seen problems where merchants "don't see coins" beyond occasional cases where they've just had buggy software stacks. in which case they fix things and patch it up manually
316 2015-03-20 20:32:25 <hearn> Naked: which wallet are you testing with? Bitcoin Wallet sets return addresses i think. not sure about the others.
317 2015-03-20 20:32:35 <hearn> Naked: but yeah get on to bitpay and rattle their cage a bit :)
318 2015-03-20 20:33:13 <Naked> hearn: yeah, we will
319 2015-03-20 20:34:15 <Naked> okay, one more point
320 2015-03-20 20:35:42 <Naked> for dispute handling, three proofs are required - proof that the merchant is genuine, proof that the customer has paid and proof that the merchant has given the money back (if there is a cancel/refund)
321 2015-03-20 20:36:16 <Naked> the first is handled by PKI stuff in payment request, the second is handled by just showing the payment request and the transactions in the block chain
322 2015-03-20 20:36:49 <Naked> but the third is not handled in any way - the merchant can transfer the same amount of coins somewhere, but there's nothing to confirm the destionation address belongs to the customer
323 2015-03-20 20:37:16 <Naked> the merchant has no way to prove that he has returned the money
324 2015-03-20 20:38:10 <Naked> if the return address was authenticated in BIP 70 (somehow, difficult I know), then the merchant could prove that he sent the same amount to an authenticated return address
325 2015-03-20 20:38:32 <Naked> which is exactly what I was trying to achieve with my earlier proposal about a signed bitcoin message for the return address
326 2015-03-20 20:40:10 <Naked> the benefit of that would be that it can be provided after the fact - if there's a system error, or anything funky with the actual payment - as long as the customer has the private key to sign the "return address" message, he can provide that later on and the merchant can securely return money
327 2015-03-20 20:41:06 <Naked> so, in brief, it can be provided retroactively if needed
328 2015-03-20 20:42:02 <Naked> and - once again about the difference of face-to-face sales - you usually do not require a return address at all after the customer has left the store
329 2015-03-20 20:42:28 <Naked> because the customer has the product, and a receipt - and you can give the money back to anybody who comes in with the product and the receipt
330 2015-03-20 20:43:23 <Naked> so it is much easier to just ask for a new address when the customer returns
331 2015-03-20 20:43:58 <Naked> this is the way it works with credit cards - for refunds you just insert your card (any card) in to the terminal and the payment is refunded there - it is not a refund of the "original transaction" so to speak
332 2015-03-20 20:46:44 <Naked> but, I think I'm done - sorry for the spam - I hope anybody working on payment requests takes the time to read all this though - and I will be back later again, maybe with more suggestions/questions
333 2015-03-20 20:51:05 <hearn> Naked: i work on bip70 (one of the co-authors and general advocate). i plan to do some more work on extensions for it soon.
334 2015-03-20 20:51:09 <hearn> Naked: so the right people are reading your concerns.
335 2015-03-20 20:51:34 <hearn> Naked: adding authentication of the refund address using one of the keys in a multi-sig coin might be good enough. i don't know - right now we barely use dispute mediation.
336 2015-03-20 20:51:54 <hearn> so it's hard to say what the precise requirements are. just getting people using _any_ kind of dispute mediation would be a step up from the current escrow disaster (see: evolution implosion)
337 2015-03-20 21:54:05 <Naked> a few more thoughts after sauna:
338 2015-03-20 21:54:50 <Naked> if the QR code given to customer does not contain the address, but only the payment request url, the merchant may safely cancel the transaction as long as it knows the payment request has never been fetched
339 2015-03-20 21:55:09 <Naked> in this case the customer cannot have the destination address to send bitcoins, so there is nothing to refund ever
340 2015-03-20 21:57:21 <Naked> another weird solution to the refund address problem would be instead require that a refund address be included in a POST, to which the response is a payment request - that way there can never be a missing refund address as the payment address is never given if the refund address has not been received
341 2015-03-20 21:58:59 <Naked> and a third solution would be to include some sort of a "refund ticket" in the payment request - but payment_url can indeed be made such already, provided it is unguessable
342 2015-03-20 22:00:48 <Naked> but, just random thoughts again, I'm out