1 2014-09-06 01:27:44 <unyo> hello
2 2014-09-06 01:30:02 <unyo> running the 0.9.3 branch for the lulz as a listening node, just wanted to know if of CHECKSUM ERRORs are a common occurrence to anyone else
3 2014-09-06 05:33:47 <Luke-Jr> petertodd: steath addresses should have a single OP_RETURN for all stealth recipients, correct?
4 2014-09-06 05:35:07 <gmaxwell> Luke-Jr: see the updated bip petertodd posted.
5 2014-09-06 05:35:59 <Luke-Jr> gmaxwell: where? I get a 404 on the bips repo
6 2014-09-06 05:36:12 <gmaxwell> bitcoin-development list
7 2014-09-06 05:38:45 <Luke-Jr> gmaxwell: seems to still use a single OP_RETURN..?
8 2014-09-06 05:39:03 <Luke-Jr> most recent I saw was 2014-08-06
9 2014-09-06 05:39:14 <phantomcircuit> petertodd, ^ poke
10 2014-09-06 05:39:37 <gmaxwell> the revised text was a single op_return for all, plus additional bloom bait pushes for each inside that op_return, IIRC
11 2014-09-06 05:39:46 <gmaxwell> Sorry, can't get to a browser to look right now.
12 2014-09-06 05:47:12 <petertodd> it wasn't a "revised bip", it was an idea, notably one that to work requires that op_return not have any particular upper limit. anyway, I've got better and higher-paying things to do right now then fight political battles over op_return
13 2014-09-06 05:48:05 <Luke-Jr> ACTION wonders who's fighting a political battle right now
14 2014-09-06 05:48:21 <Luke-Jr> petertodd: I was only asking because Electrum has a pull request that adds a new OP_RETURN for every output
15 2014-09-06 05:48:24 <petertodd> probably makes more sense to have stealth be <stealth stuff> OP_DROP (rest of script) rather than using P2SH/P2PKH anyway
16 2014-09-06 05:48:36 <Luke-Jr> and I wanted to be sure my comment wasn't ignorantly outdated or something
17 2014-09-06 05:48:38 <petertodd> Luke-Jr: ?
18 2014-09-06 05:48:50 <petertodd> Luke-Jr: oh, you mean a stealth implementation - yeah the darkwallet one does that
19 2014-09-06 05:49:02 <Luke-Jr> so it's valid either way?
20 2014-09-06 05:49:19 <gmaxwell> 'valid' implies something is specced.
21 2014-09-06 05:49:32 <Luke-Jr> gmaxwell: that's what BIPs are for ;)
22 2014-09-06 05:49:33 <petertodd> Luke-Jr: they have a full spec on their wiki that is decent
23 2014-09-06 05:49:35 <gmaxwell> having an extra op_return for every output is supremely inefficient.
24 2014-09-06 05:49:57 <petertodd> gmaxwell: meh, doing it otherwise means the limit on op_return needs to be raised
25 2014-09-06 05:50:38 <gmaxwell> petertodd: yes, I don't see a problem of changing it. E.g. allowing extra indicator words per output.
26 2014-09-06 05:50:39 <Luke-Jr> petertodd: multiple OP_RETURN doesn't work any better than long OP_RETURN afaik
27 2014-09-06 05:51:27 <petertodd> gmaxwell: well, like we discussed, submit a pull-req and change it
28 2014-09-06 05:51:35 <Luke-Jr> gmaxwell: so 40 + (8 * outputs)?
29 2014-09-06 05:52:09 <gmaxwell> petertodd: Yep, been kinda busy, no pull req from me recently, but next week I will again have bandwidth to open several.
30 2014-09-06 05:53:09 <gmaxwell> Luke-Jr: yes, unless someone wants to propose something else.
31 2014-09-06 05:53:58 <Luke-Jr> ACTION wonders if putting the identifier in an OP_DROP of the actual output might be better
32 2014-09-06 05:54:00 <petertodd> gmaxwell: in-output OP_DROP has way less edge cases...
33 2014-09-06 05:54:20 <gmaxwell> petertodd: total preclusion of P2SH though.
34 2014-09-06 05:54:33 <Luke-Jr> ACTION stabs BIP 16
35 2014-09-06 05:54:40 <petertodd> gmaxwell: so what? OP_MAST will exist eventually
36 2014-09-06 05:55:20 <petertodd> gmaxwell: honestly, all this discussion about a waste of 8 bytes per op_return output is bizzare to me
37 2014-09-06 05:55:24 <gmaxwell> petertodd: perhaps, spec as bare pay to pubkey. (oh god, does the electrum pull req continue to use uncompressed pubkeys? :( )
38 2014-09-06 05:55:45 <gmaxwell> petertodd: it's a waste of 34 bytes per output.
39 2014-09-06 05:56:09 <gmaxwell> the darkwallet stuff does a seperate ephemeral key for every op_return
40 2014-09-06 05:56:13 <Luke-Jr> hm, maybe there's a way to generate the ephemeral key in such a way that these numbers are hidden in it?
41 2014-09-06 05:56:43 <gmaxwell> Luke-Jr: yes, but that would only work for one, where almost every transaction will have two outputs (the real one and the change)
42 2014-09-06 05:57:21 <petertodd> gmaxwell: there's tradeoffs re: ephemeral keys too, e.g. proving you actually were the one to pay someone w/ coinjoin
43 2014-09-06 05:57:22 <Luke-Jr> gmaxwell: can't put both in there, and let the inevitable message telling them who paid for what include data on how to decipher it?
44 2014-09-06 05:57:24 <gmaxwell> Luke-Jr: and that requires computation exponential with the data length. (it's the same as vanitygen)
45 2014-09-06 05:59:51 <Luke-Jr> actually, if someone can give out a payment id, they could just give out a different address.. :x
46 2014-09-06 06:00:09 <gmaxwell> petertodd: yes, but they're very narrow and not attractive vs more than doubling every output size on the network... and can be achieved in other ways. E.g. commiting to a hashtree of extradata as part of the emphemeral key. E.g. I can show the ephemeral key is H(extradata)*G + Q and then have the payment info in the extradata tree.
47 2014-09-06 06:00:17 <Luke-Jr> so what exactly is the use case for this?
48 2014-09-06 06:01:19 <gmaxwell> Luke-Jr: inevatably people will use this as static addresses, which is actually fine since it results in new addresses on the network constantly... it's just like BIP32 in that respect: one private key, many addresses.
49 2014-09-06 06:02:08 <Luke-Jr> gmaxwell: I get that, but I don't see the benefit if they have to generate a payment id too
50 2014-09-06 06:02:13 <Luke-Jr> why not just generate a new address?
51 2014-09-06 06:02:23 <gmaxwell> petertodd: that lets you add your coinjoin source proof, without adding a single additional byte.
52 2014-09-06 06:03:02 <gmaxwell> Luke-Jr: safer against some attacks, for example. E.g. if a website gets hacked and changes the static address it is more likely to get noticed; and all the attacker can do otherwise is dork with the payment ids.
53 2014-09-06 06:03:16 <wumpus> gmaxwell: meh, doing it otherwise means the limit on op_return needs to be raised <- as I've said before IMO there is no problem in raising the limit on op_return if there is a good reason
54 2014-09-06 06:03:28 <gmaxwell> plus having the same tool for both the broadcast addresses (donations and such)
55 2014-09-06 06:03:56 <wumpus> it's way preferable to having to work around it with a solution that spams the utxo set
56 2014-09-06 06:04:48 <gmaxwell> wumpus: yea, that was petertodd saying that. I told him previously that I thought raising it would be fine. (esp for this, where it would take the form of additional 8 byte pushes, so we don't dillute the force of 'op_return is for binding hashes (and ephemeral keys)'
57 2014-09-06 06:04:49 <Luke-Jr> wumpus: I think everyone is basically in agreement on those points.
58 2014-09-06 06:06:53 <wumpus> gmaxwell: oh I see now, sorry
59 2014-09-06 06:09:51 <wumpus> 40 + (8 * outputs) sounds okay to me, so there is a way to add output-specific data that won't end up in the utxo
60 2014-09-06 06:17:20 <wumpus> OP_DROP in the output itself would need special handling to make sure the OP_DROPS don't end up in the UTXO set
61 2014-09-06 06:32:09 <gmaxwell> wumpus: re: pull 4230 (just catching up with its discussion while starting some testing on it)- it's been my view that trickle is primarily a bandwidth / packet per second optimization. Without something like it you end up doubling the INV traffic by crossing in flight constant.
62 2014-09-06 06:32:15 <gmaxwell> er constantly.
63 2014-09-06 06:33:18 <gmaxwell> so another argument for not fussing with it right now.
64 2014-09-06 06:41:36 <volante> gmaxwell: its also there for privacy (according to the comments in the code)
65 2014-09-06 06:42:30 <gmaxwell> Yes, it does contribute to that... though for privacy you'd really want a different parameterization
66 2014-09-06 06:45:23 <wumpus> makes sense
67 2014-09-06 06:45:36 <volante> theres an interesting comment about it on that thread by ivanpustogarov (one of the guys who wrote the deanonymisation paper)
68 2014-09-06 07:00:29 <midnightmagic> except the last comment about running over Tor from Ivan is almost completely incorrect.
69 2014-09-06 07:03:51 <wumpus> low-latency is good for privacy even when running over Tor
70 2014-09-06 07:03:58 <wumpus> eh- high-latency
71 2014-09-06 07:04:51 <phantomcircuit> wumpus, only if you assume a global adversary
72 2014-09-06 07:04:57 <phantomcircuit> er
73 2014-09-06 07:05:26 <phantomcircuit> an adversary with the ability to observe traffic on all the tor nodes
74 2014-09-06 07:05:30 <phantomcircuit> otherwise it's silly
75 2014-09-06 07:06:48 <wumpus> no, it doesn't necessarily have to watch all the tor nodes, watching just your node and transactions propagated over the bitcoin network may be enough to make inferences
76 2014-09-06 07:08:21 <wumpus> but only if the network is low-latency, if nodes group and delay transactions (trickle-like) that becomes much less reliable
77 2014-09-06 07:09:02 <phantomcircuit> wumpus, they would need to see the connection to the guard node and the exit node to correlate
78 2014-09-06 07:09:15 <phantomcircuit> which requires a good deal of global awareness
79 2014-09-06 07:09:40 <wumpus> a global adversary could look at timings between individual tor nodes and thus try to folllow the data over tor itself (making use of the fact that tor itself is a low-latency network)
80 2014-09-06 07:09:50 <wumpus> but that's not what I'm talking about
81 2014-09-06 07:11:53 <wumpus> anyhow - trickling will stay for now
82 2014-09-06 07:11:59 <phantomcircuit> wumpus, what kind of inferences could you make about a transaction except for it's origin?
83 2014-09-06 09:15:00 <petertodd> wumpus: "make sure the OP_DROPS don't end up in the UTXO set" <- txo commitments, or in minature, store H(scriptPubKey) and ask for scriptPubKey during tx/block relaying
84 2014-09-06 09:15:57 <petertodd> wumpus: all this obsession about the UTXO set is pretty silly given what we know...
85 2014-09-06 09:16:01 <wumpus> petertodd: yes, sound error-prone to me, I'd prefer standardizing on OP_RETURN
86 2014-09-06 09:16:15 <petertodd> wumpus: error-prone? how so?
87 2014-09-06 09:16:44 <wumpus> easy to introduce a bug if you have to store 'filtered' scripts
88 2014-09-06 09:17:00 <sipa> that's not what petertodd is suggesting
89 2014-09-06 09:17:12 <petertodd> wumpus: these aren't filtered scripts, your storing the hash of the script and asking for the script to accompany a tx
90 2014-09-06 09:17:44 <wumpus> petertodd: oh, you mean storing the hash of the script instead of the script itself
91 2014-09-06 09:18:25 <petertodd> wumpus: yes
92 2014-09-06 09:18:39 <wumpus> it wasn't clear to me what kind of transforation H(x) was, but hash makes sense yes
93 2014-09-06 09:18:43 <petertodd> wumpus: note how that's not even a fork
94 2014-09-06 09:18:53 <sipa> or we could just make p2sh mandatory :)
95 2014-09-06 09:19:12 <petertodd> sipa: I'm not wasting my time on that discussion
96 2014-09-06 09:19:23 <sipa> how do you mean?
97 2014-09-06 09:19:37 <wumpus> sipa: well the point above was to make it possible to add data to the output with OP_DROP instead of adding it to OP_RETURN, that would be foiled :)
98 2014-09-06 09:19:42 <sipa> (i'm not seriously suggesting that; it is just remarkably similar to what you're saying...)
99 2014-09-06 09:20:17 <sipa> wumpus: if the data+drop goes inside the p2sh script, it doesn't end up in the utxo set either
100 2014-09-06 09:20:47 <wumpus> petertodd: but indeed, that wouldn't be a fork or protocol change, just a different way of implementing it?
101 2014-09-06 09:20:59 <petertodd> wumpus: exactly
102 2014-09-06 09:21:09 <petertodd> wumpus: notably a way of implementing it that can be deployed gradually as well
103 2014-09-06 09:21:22 <wumpus> but how would that help? you need the full script for verification
104 2014-09-06 09:21:46 <sipa> you make sure you get the full script whenever you need it
105 2014-09-06 09:21:53 <wumpus> but then you still have to store it
106 2014-09-06 09:22:01 <sipa> yes, but not in the utxo set
107 2014-09-06 09:22:13 <wumpus> well that's just a manner of speaking
108 2014-09-06 09:22:17 <sipa> we store input scripts too, in the blockchain, but not in the utxo set
109 2014-09-06 09:22:26 <petertodd> wumpus: as in, if people start getting concerned about UTXO set size or the data in the UTXO set, full nodes can get the actual scripts on demand from their peers when they need them in conjunction with the tx/blocks's they're relaying
110 2014-09-06 09:22:40 <wumpus> you need to keep it around for verification, whether it is 'in the utxo set' is a technicality
111 2014-09-06 09:22:51 <sipa> wumpus: not really
112 2014-09-06 09:23:00 <sipa> wumpus: the same argument applies to the blockchain itself
113 2014-09-06 09:23:15 <wumpus> petertodd: you're proposing that we should ask a peer to be able to verify some transactions?
114 2014-09-06 09:23:21 <petertodd> wumpus: that's the point: you don't need it for verification, you only need to be given it when you verify something, and you only ever verify things because someone created a tx or block
115 2014-09-06 09:23:23 <sipa> yes, we need the full blockchain around for verification, but it doesn't affect database performence on the utxo set
116 2014-09-06 09:23:38 <petertodd> wumpus: no, I'm saying if a peer wants you to accept a tx, it should give you the scriptPubKey's required to verify that tx
117 2014-09-06 09:23:45 <wumpus> you can currently throw away the block chain and still do verification *now*
118 2014-09-06 09:23:48 <petertodd> wumpus: or a block of course
119 2014-09-06 09:23:51 <sipa> wumpus: no...
120 2014-09-06 09:24:00 <petertodd> you can't even do local verification :(
121 2014-09-06 09:24:10 <petertodd> (though that's easy enough to fix)
122 2014-09-06 09:24:13 <sipa> oh, you mean verification of future things; sure
123 2014-09-06 09:24:14 <wumpus> you just need the utxo set for verification right?
124 2014-09-06 09:24:37 <wumpus> the utxo set is self-contained, if it would contain pointers to input scripts somewhere else you need that as well
125 2014-09-06 09:24:43 <sipa> well, yes, it moves the responsibility of storage from the utxo set to transaction creators
126 2014-09-06 09:24:46 <sipa> like p2sh
127 2014-09-06 09:24:47 <petertodd> wumpus: no, you need to be able to verify that some data is in the UTXO set
128 2014-09-06 09:24:48 <wumpus> (or yo need to ask other peers, but that's kind of crazy...)
129 2014-09-06 09:25:41 <wumpus> yes I mean verification of future things, verifying past things isn't really interesting (apart from reorgs)
130 2014-09-06 09:26:13 <sipa> it's just very different from how bitcoin works now
131 2014-09-06 09:26:30 <petertodd> sipa: I'm not even suggesting a fork... it's really not that different
132 2014-09-06 09:26:35 <wumpus> sipa: sure, if you require p2sh that's different, but that would be a change to how things are done, petertodd implied that it is just an implementation change
133 2014-09-06 09:26:49 <sipa> an implementation change for wallets, nodes, ...
134 2014-09-06 09:26:55 <sipa> but not a consensus change
135 2014-09-06 09:26:55 <wumpus> sipa: I fully agree that if you require P2SH you only need to store the hash
136 2014-09-06 09:27:25 <petertodd> sipa: better than that, because wallets that don't have scriptPubKey data can just stick to connecting to nodes that do have it
137 2014-09-06 09:27:47 <petertodd> sipa: same way they have to stick to connecting to nodes that have blockchain data if they're using SPV w/bloom
138 2014-09-06 09:27:58 <wumpus> but if the script can contain something else you need to store it be able to verify transactions
139 2014-09-06 09:28:31 <sipa> wumpus: petertodd is suggesting that whenever you give someone a tx or a block, you're required to also give them the corresponding full output scripts
140 2014-09-06 09:29:11 <wumpus> sipa: ok...
141 2014-09-06 09:30:07 <wumpus> yes, if you force other people to give you the data every time you process it, you don't need to store it
142 2014-09-06 09:30:34 <sipa> and typically when someone sends you a transaction, they know the output script, because they've just created input for it
143 2014-09-06 09:30:48 <sipa> and when someone sends you a block, they also know those spent output scripts, because they just validated them
144 2014-09-06 09:32:02 <petertodd> sipa: note though the ugly part of this in practice you do need a soft-fork to make proving a specific scriptPubKey of a transaction compact - IE a (U)TXO commitment scheme - otherwise you can construct edge cases where the proof of scriptPubKey(s) are huge transactions
145 2014-09-06 09:32:18 <sipa> right
146 2014-09-06 09:32:35 <sipa> to avoid needing to send the full crediting transactions, and just the output + merkle tree
147 2014-09-06 09:33:02 <sipa> how do you do that as a softfork?
148 2014-09-06 09:33:10 <petertodd> sipa: just add another index to blocks of course
149 2014-09-06 09:33:23 <sipa> ah, per block
150 2014-09-06 09:33:30 <sipa> but that doesn't work for individual transactions
151 2014-09-06 09:33:46 <petertodd> sipa: sure it does - the index is of a new merklized transaction format
152 2014-09-06 09:34:27 <sipa> you're sending me a transaction with a particular input from txidA:n
153 2014-09-06 09:35:00 <sipa> sorry, i need some coffee first :)
154 2014-09-06 09:36:08 <wumpus> so yes, requiring P2SH would be realistic
155 2014-09-06 09:36:16 <wumpus> +more
156 2014-09-06 09:36:38 <sipa> but that doesn't solve petertodd's problem
157 2014-09-06 09:36:57 <sipa> as it means you can't inspect the extra data in transactions
158 2014-09-06 09:37:04 <petertodd> sipa: no, it's specifically designed to not solve my problem - that's the whole point of requiring p2sh
159 2014-09-06 09:37:22 <wumpus> it appears that solving petertodds problem creates a lot of new problems
160 2014-09-06 09:38:01 <wumpus> just put the data in OP_RETURN :)
161 2014-09-06 09:38:41 <petertodd> wumpus: anyway, I pointed out H(scriptPubKey) to lead to my next point, which is that having an infinite sized UTXO set is insane
162 2014-09-06 09:39:20 <petertodd> wumpus: and if you go down the path of thinking about H(scriptPubKey), you realise that how many scriptPubKeys you store in your local UTXO set vs. the set of all possible spendable ones is a bandwidth tradeoff
163 2014-09-06 09:40:13 <wumpus> an infinite sized utxo set would be interesting, would any possible utxo be in there?
164 2014-09-06 09:40:19 <wumpus> :P
165 2014-09-06 09:40:27 <petertodd> wumpus: no, I mean there's no limit on UTXO growth
166 2014-09-06 09:40:34 <sipa> you just preallocate 1 satoshi to every possible output script
167 2014-09-06 09:40:45 <sipa> infinite market cap!
168 2014-09-06 09:42:11 <petertodd> sipa: oh, and reminds me: easy to make tx's merklized by adding an op_ret output to them with the vin/vout merkle trees
169 2014-09-06 09:42:47 <petertodd> sipa: pity that's a client-side change :(
170 2014-09-06 09:43:24 <sipa> right, that's what i was getting at
171 2014-09-06 09:43:39 <sipa> you can't do this without changing txids, which is what is being referred to
172 2014-09-06 09:43:52 <sipa> but if you're inserting something inside transactions, right indeed
173 2014-09-06 09:44:05 <petertodd> sipa: no, you can't do it as bandwidth efficiently
174 2014-09-06 09:44:14 <sipa> perhaps #bitcoin-wizards ?
175 2014-09-06 09:44:24 <petertodd> yup
176 2014-09-06 13:59:17 <kdomanski__> 0*`
177 2014-09-06 13:59:17 <kdomanski__> 'xcm
178 2014-09-06 13:59:18 <kdomanski__> ]tgfg
179 2014-09-06 14:02:39 <sipa> kdomanski__'s cat, is that you on the keyboard?
180 2014-09-06 15:41:45 <kdomanski__> huh, must have been the housemaid
181 2014-09-06 15:41:53 <kdomanski__> sipa: ^
182 2014-09-06 17:41:27 <Adlai> "For Hardy, the most beautiful mathematics was that which had no practical applications in the outside world⦠his own special field of number theory."
183 2014-09-06 17:44:53 <Emcy> tonal
184 2014-09-06 18:16:14 <devrandom> is there a page somewhere with a histogram of blockchain txs binned by BTC sent?
185 2014-09-06 18:18:14 <sipa> devrandom: how do you calculate BTC sent?
186 2014-09-06 18:19:14 <devrandom> good point, it's not possible in general
187 2014-09-06 18:19:39 <sipa> you can bin them by BTC *moved* easily, but that's not the same thing
188 2014-09-06 18:19:46 <Eliel> in practise you can get quite good guesses though.
189 2014-09-06 18:19:47 <devrandom> but is there at least a histogram by BTC moved?
190 2014-09-06 18:20:04 <sipa> would a bash oneliner do?
191 2014-09-06 18:20:15 <devrandom> sure :)
192 2014-09-06 18:21:09 <sipa> hmm, that would take ages
193 2014-09-06 18:23:11 <devrandom> I could do a block range... just interested in ratios
194 2014-09-06 19:14:38 <dhill> http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html
195 2014-09-06 19:40:13 <kdomanski> dhill: that article was discussed here a few days ago
196 2014-09-06 20:48:46 <nullbyte> .
197 2014-09-06 20:59:16 <Eliel> When you're restarting bitcoind that's got 500+MB of itself in swap, is there any reason to have it make a controlled shutdown and not just kill -KILL it? It takes ages to shutdown :P
198 2014-09-06 21:04:54 <phantomcircuit> Eliel, you could screw up the leveldb utxo database
199 2014-09-06 21:05:06 <phantomcircuit> it shouldn't... but weill
200 2014-09-06 21:05:07 <phantomcircuit> well
201 2014-09-06 21:07:30 <Eliel> Ah, it finally shut down. Now it's only 30 minutes wait until it's back running and responsive :P
202 2014-09-06 21:08:54 <Eliel> (yes, this virtual server is not quite big enough for bitcoind anymore)