1 2014-12-25 00:07:23 <tsutsu> gmaxwell: if you don’t lose that state in the first place, it’d be fine, though—not likely in C, but I’d imagine you’d be able to pull it off in e.g. an erlang bitcoind
  2 2014-12-25 00:08:04 <gmaxwell> tsutsu: that defeats the purpose of restarting
  3 2014-12-25 00:08:23 <tsutsu> gmaxwell: we’re talking about loading a new bitcoind executable (upgrading), not restarting
  4 2014-12-25 00:08:25 <gmaxwell> we absolutely would not accept changes that created pressure against changing state in the software.
  5 2014-12-25 00:09:31 <gmaxwell> tsutsu: yes, but there will never be an upgrade which doesn't add or change fields; nor would we accept the complexity of trying to migrate them, or the matrix testing burden of all possible paths in all possible states that might leave the daemon in a latent network fork creating corrupted state.
  6 2014-12-25 00:10:18 <gmaxwell> (Having worked before on commercial products that did live updates like that, I'm comfortable in saying that no one who hasn't already done it has any idea how hard it is)
  7 2014-12-25 00:12:10 <gmaxwell> in any case, bitcoin is not a telephone switch or a DFZ internet router; so I was asking for the motivation before you decided to try debating me on the technical minutia.
  8 2014-12-25 00:12:26 <tsutsu> gmaxwell: eh, that’s just due to the way bitcoind is currently implemented, as the implicit instantiation of a rules engine at a point in time—it could (not easily, but could) be refactored into something that keeps multiple rules engines going against an event stream, and then an upgrade just adds a rules engine. it’s how e.g. server-side mediation of starcraft ii matches works
  9 2014-12-25 00:12:50 <sipa> i have no idea how hard it would be, bit it seems near impossible (how do you even map old datastructures to new ones, if they can have changed; at least you need implicit serialization of every piece of data, but even then - just what state is used changes from version to version)
 10 2014-12-25 00:13:51 <tsutsu> in other words, the client keeps talking the old protocol to the old peers, and starts talking the new protocol to the new peers, and offers to protocol upgrade to the old peers if there is such a command in the protocol, which the other peers accept if they themselves have already upgraded and are just waiting on you
 11 2014-12-25 00:14:25 <gmaxwell> sipa: there are a couple different approaches, all are pretty awful. For example you can have evert state object know how to deseralize and seralize itself into a backwards/forward compatible object.
 12 2014-12-25 00:14:46 <sipa> gmaxwell: even then... just which pieces of data are kept changes
 13 2014-12-25 00:14:59 <gmaxwell> sipa: Or you can log all messages that cause state transaitions and replay. them.
 14 2014-12-25 00:15:11 <gmaxwell> sipa: yep. and you need to handle missing data on any newly introduced fields.
 15 2014-12-25 00:15:36 <sipa> tsutsu: are you now talking about software updates, or protocol changes?
 16 2014-12-25 00:15:51 <sipa> but the latter is pretty much how it already works: clients negotiate the latest version they both know
 17 2014-12-25 00:16:55 <gmaxwell> tsutsu: again, none of this makes any sense for bitcoind, you're not stating a motivation. Just because something can (with great effort and overhead) be done, doesn't mean it should be done.
 18 2014-12-25 00:18:23 <tsutsu> gmaxwell: I never said it *should* be done, I said it would solve the specific problem of enabling seamless in-place upgrades without losing long-lived connections—obviously this isn’t a necessary part of the bitcoin protocol as it stands
 19 2014-12-25 00:18:46 <gmaxwell> tsutsu: what _problem_?
 20 2014-12-25 00:19:15 <sipa> the problem seems to be "upgrades reset connections"; it's a very minor inconvenience at most
 21 2014-12-25 00:19:19 <gmaxwell> Thats not a problem. A problem is a stament like  "I'm running this software and it ends up on an incorrect chain, and people use that to rip me off and I lose money"
 22 2014-12-25 00:19:41 <sipa> the only thing for which long-lived connections mattered was synchronization up to 0.9.x, as resets tended to confuse the process
 23 2014-12-25 00:19:43 <gmaxwell> or "Foo happens and then I need to wait 20 minutes before it's safe to swim, and I like swimming."
 24 2014-12-25 00:20:25 <tsutsu> eh, I guess my mind is full of a related-but-not-quite problem to proof-of-work (proof-of-generational-survival-within-consensus-group)
 25 2014-12-25 00:21:03 <tsutsu> for which it is a real problem
 26 2014-12-25 00:32:16 <Luke-Jr> tsutsu: FWIW, preserving state is easier in C than most languages (at least on non-Windows)
 27 2014-12-25 00:33:16 <sipa> irssi does it
 28 2014-12-25 00:33:38 <gmaxwell> Luke-Jr: the reason he mentioned erlang is because there are specific facilities for non-stop system upgrades in the language and runtime.
 29 2014-12-25 00:33:46 <gmaxwell> (doesn't mean it's not a suicide wish...)
 30 2014-12-25 00:33:59 <ers35> My use case is to quickly test networking related changes without having to wait for peers to reconnect.
 31 2014-12-25 00:34:36 <Luke-Jr> ers35: you know, you can use -addnode for outbound connections..
 32 2014-12-25 00:35:02 <gmaxwell> ers35: such as?
 33 2014-12-25 00:35:19 <sipa> there's even an RPC for addnode afaik
 34 2014-12-25 00:35:20 <gmaxwell> (what kinds of changes are you interested in testing where this is a concern?)
 35 2014-12-25 00:39:55 <ers35> If my node has amassed many inbound connections and I am benchmarking or stress testing the networking code, I can swap out different versions and compare them. However, perhaps a dedicated testing environment on testnet with connections I control make more sense.
 36 2014-12-25 00:40:15 <sipa> sounds like it, yes :)
 37 2014-12-25 00:43:24 <gmaxwell> hard to do any kind of benchmarking without a controlled enviroment. :)
 38 2014-12-25 02:15:46 <gmaxwell> uhh
 39 2014-12-25 02:17:28 <gmaxwell> so I have a transaction in my mempool that has hundreds of inputs, each with a signature... getblocktemplate on one of my 0.10rc test nodes is reporting             "sigops" : 1
 40 2014-12-25 02:17:51 <sipa> uh-oh
 41 2014-12-25 02:17:53 <gmaxwell> is there some strange alternative meaning of sigops that I'm not aware of or is something busted?
 42 2014-12-25 02:22:26 <sipa> no, and i don't see how it could fail
 43 2014-12-25 02:22:36 <sipa> it iterates over all inputs
 44 2014-12-25 02:22:52 <gmaxwell> I don't see it either. Want a copy of the transaction? It's too low prior to relay well so you don't likely have it.
 45 2014-12-25 02:28:55 <gmaxwell> seems consistently wrong on this node. I don't see how, but I'm absolutely sure that its wrong.
 46 2014-12-25 02:28:56 <sipa> i have one too
 47 2014-12-25 02:29:13 <sipa> 2d128b14a7d46e670aa115c264c86a12cc5094c0e8e1b62e6048c538ed163cb2
 48 2014-12-25 02:29:32 <sipa> wait
 49 2014-12-25 02:29:45 <sipa> we don't count sigops in inputs, only in outputs and redeemscripts
 50 2014-12-25 02:30:24 <gmaxwell> oh.
 51 2014-12-25 02:31:22 <gmaxwell> well nevermind then.
 52 2014-12-25 02:36:34 <phantomcircuit> sipa, what's the logic behind that?
 53 2014-12-25 02:37:01 <sipa> iirc, it was just easier in pre-0.8 validation logic, because the sigop count was checked before actually fetching inputs
 54 2014-12-25 02:37:35 <sipa> for p2sh, they're tested at spending time, and non-p2sh is pretty limited due to standardness rules
 55 2014-12-25 02:37:45 <sipa> ... but changing this is a softfork
 56 2014-12-25 02:38:58 <gmaxwell> It's just pure braindamage, ignoring p2sh, it checks precisely everything that doesn't matter. I knew it included a bunch of things that didn't matter, I think I either didn't know or had forgotten that it also excluded the one thing that mattered.
 57 2014-12-25 02:39:53 <sipa> specifially, it includes coinbase inputs
 58 2014-12-25 02:40:03 <sipa> well, it counts inputs + outputs, but inputs typically don't contains sigops
 59 2014-12-25 02:40:43 <gmaxwell> in the case of a coinbase the input isn't even an input, and a checksig there would never get executed, which is kinda loopy but at least consistent.
 60 2014-12-25 02:41:30 <gmaxwell> outputs are pointless to count because they aren't run and may not even be runnable ever.   What counts is scriptSigs (which usually don't have any) and scriptPubkeys that are being spent.
 61 2014-12-25 02:43:15 <petertodd> gmaxwell: heh, that's my goto example when people suggest we need more ability to do static analysis on scripts in Bitcoin - tried that and wounded up with something truly braindamaged
 62 2014-12-25 02:43:22 <petertodd> *wound up
 63 2014-12-25 02:44:36 <petertodd> note how my "accept any sequence of pushdata's in op-return outputs" has to be phrased as pushdata's only, rather than opcodes, due to scriptPubKeys having their sigops counted: https://github.com/bitcoin/bitcoin/pull/5079
 64 2014-12-25 02:45:19 <petertodd> right now we allow the bizzare scriptPubKey: OP_RETURN OP_CHECKMULTISIG due to a bug, which is 20 sigops for only 11 bytes marginal cost
 65 2014-12-25 02:46:08 <gmaxwell> petertodd: yea, sure I knew about the stupid txout scriptubkey counting, which is annoying and dumb, and makes for fun examples of weird effects.
 66 2014-12-25 02:47:45 <gmaxwell> searching around I don't think anyone previously knew or expressed the view that the sigops limit was totally pointless.
 67 2014-12-25 02:49:16 <petertodd> personally I wonder if it's really worth it, given that resources needed to do any of the sigop-counted operations aren't that much unlike the resources needed to process that byte when you take into account networking overheads
 68 2014-12-25 02:49:46 <sipa> well we are limited to 20 sigops per...
 69 2014-12-25 02:49:53 <sipa> byte?
 70 2014-12-25 02:49:57 <petertodd> sipa: exactly
 71 2014-12-25 02:50:02 <sipa> that's still painful
 72 2014-12-25 02:50:11 <gmaxwell> petertodd: well it's not worth it at all because its completely ineffectual in any case.
 73 2014-12-25 02:50:37 <petertodd> gmaxwell: I'm assuming a decent sigops design BTW
 74 2014-12-25 02:50:53 <sipa> that means 5 kilobyte per second with libsecp256k1 on a modern CPU...
 75 2014-12-25 02:51:18 <petertodd> sipa: ?
 76 2014-12-25 02:51:32 <sipa> 5kb of checkmultisig = 100k sigops
 77 2014-12-25 02:51:47 <sipa> wait, that's counted sigops, not executed
 78 2014-12-25 02:52:00 <gmaxwell> petertodd: say your signature verification operations take 1ms (true on slow cpus), that means a limitation of 1000 verifies per second.  Now you can 2dup and get close to one true sigup per two bytes.
 79 2014-12-25 02:52:05 <sipa> to execute them you need at least some inputs, and 1-of-20
 80 2014-12-25 02:52:36 <gmaxwell> so that would suggest then a bandwidth equivlence of 2000 bytes per second. I think that most hosts have more than 2kb/s in bandwidth.
 81 2014-12-25 02:53:12 <sipa> it's not hard to add an extra standardness rule that does accurate sigop counting
 82 2014-12-25 02:53:25 <gmaxwell> yea, so I don't agree that users would hit bandwidth limits before cpu ones, alas.
 83 2014-12-25 02:54:39 <gmaxwell> okay, that pattern would need drops, so it would be three bytes actually, still 3KB/s is not much bandwidth.
 84 2014-12-25 02:55:14 <petertodd> gmaxwell: oh, I thought sig verification was more like 0.1ms on a slow cpu... didn't know it was that bad
 85 2014-12-25 02:55:43 <gmaxwell> now on a state of the art cpu with fastest ecdsa code we have, it would still only be on the order of 1.6mbit/sec.
 86 2014-12-25 02:56:02 <petertodd> which on 1MB blocks isn't bad at all
 87 2014-12-25 02:56:03 <sipa> openssl takes 600us on my reference system (2.6 Ghz modern i7)
 88 2014-12-25 02:56:26 <sipa> libsecp256k1 is now close to 70us iirc
 89 2014-12-25 02:56:34 <sipa> (with endomorphism and gmp)
 90 2014-12-25 02:56:42 <petertodd> sipa: yeah, that might be where I was remembering 100us
 91 2014-12-25 02:57:58 <petertodd> sigops might also be better done as a per-script limit, so we don't have the relatively complex optimization problem at the block creation level
 92 2014-12-25 02:57:59 <sipa> but on x86 it's much more
 93 2014-12-25 02:58:28 <gmaxwell> petertodd: it certantly is annoying to have per block limits when you want to do fractional verification.
 94 2014-12-25 02:58:59 <gmaxwell> petertodd: but unless you can get away with making the limit very low (e.g. 20) I'm not sure that it doesn't make sense to pool it.
 95 2014-12-25 02:59:14 <gmaxwell> E.g. if someone were thinking about having other expensive operations that have a cost other than their size.
 96 2014-12-25 02:59:17 <petertodd> note how you could soft-fork a change to make the limit be per-script by just ensuring you were just under the average
 97 2014-12-25 03:00:06 <sipa> at least make it a limit below MAX_SIGOPS_PER_BLOCK / MAX_BLOCK_SIZE * transaction_byte_size
 98 2014-12-25 03:00:13 <petertodd> sipa: exactly
 99 2014-12-25 03:00:19 <gmaxwell> probably in terms of sane network rules the best thing would be to have a single per block limit, that is some metric that includes all 'costs' for a transaction.
100 2014-12-25 03:00:40 <petertodd> IIRC that's basically ethereums idea
101 2014-12-25 03:01:28 <sipa> let's just switch to schnorr sigs, and merge all signatures of a transaction into one
102 2014-12-25 03:01:33 <gmaxwell> since you really really do at least want a size-like limit. and once you're taking one annoying global limit, you might as well get all the costs in it.
103 2014-12-25 03:01:33 <sipa> Fixed that for you.
104 2014-12-25 03:01:56 <sipa> sorry, all signatures of an input; not all signers necessarily trust eachother
105 2014-12-25 03:02:19 <gmaxwell> sipa: you can still coinjoin single witness per transaction.
106 2014-12-25 03:02:27 <gmaxwell> (thats why multisignature works)
107 2014-12-25 03:02:45 <petertodd> sipa: reminds me, can schnorr sigs after the fact prove who signed? (I mean, optionally?)
108 2014-12-25 03:02:49 <gmaxwell> what you can't do is join things with different scripthash flags.
109 2014-12-25 03:03:15 <petertodd> sipa: in talking to businesses about multisig something that keeps coming up is people really want accountability as to *what* keys signed something
110 2014-12-25 03:03:40 <sipa> if everyone reveals their key, i guess
111 2014-12-25 03:03:48 <sipa> petertodd: note that it can only do m-of-m
112 2014-12-25 03:03:53 <petertodd> sipa: right, which isn't good enough
113 2014-12-25 03:04:05 <petertodd> sipa: oh, then m-of-m is fine - everyone must have signed :)
114 2014-12-25 03:04:09 <sipa> yes
115 2014-12-25 03:04:12 <petertodd> sipa: only m-of-n is interesting there
116 2014-12-25 03:04:39 <gmaxwell> petertodd: we came up with a way of making m of n efficiently out of m of m.
117 2014-12-25 03:04:44 <sipa> petertodd: an idea for a script 2.0 is have opcodes that just "push" which pubkeys should have signed, but have the signature itself separately
118 2014-12-25 03:04:49 <gmaxwell> and yes, you'd know who signed.
119 2014-12-25 03:05:08 <petertodd> gmaxwell: oh neat
120 2014-12-25 03:05:10 <gmaxwell> it doesn't save the transaction from revealing the pubkeys that were involved.
121 2014-12-25 03:05:29 <sipa> but it only costs one sig space, and one verification
122 2014-12-25 03:05:45 <petertodd> sipa: right, so it's basically pushing to a global signature register?
123 2014-12-25 03:05:53 <sipa> petertodd: pubkey register
124 2014-12-25 03:05:58 <petertodd> sipa: right
125 2014-12-25 03:06:04 <sipa> the signature is verified afterwards with the resulting pubkey
126 2014-12-25 03:06:15 <sipa> and yes, it needs one register per sighash flag
127 2014-12-25 03:06:28 <sipa> or per message being signed, rather
128 2014-12-25 03:06:29 <gmaxwell> basically the txn computes the effective m of m pubkey for the active signers on the fly (or reveals it in a hash tree),  and then it has only one signature. (or at least only one signature per unique message being signed, flags would cause you to need more)
129 2014-12-25 03:06:33 <petertodd> sipa: you mean, per message
130 2014-12-25 03:06:37 <sipa> yup
131 2014-12-25 03:06:55 <petertodd> sipa: which could be interesting, as being able to sign arbitrary messages keeps coming up over and over again...
132 2014-12-25 03:07:06 <sipa> note that you can still do for example a (100-of-100 OR 1-of-2), and merge the whole 100-of-100 into a 1-of-1
133 2014-12-25 03:07:38 <sipa> so the optimization can be inside the script level or outside of it
134 2014-12-25 03:08:17 <petertodd> gmaxwell: makes sense - the revealing is *not* zero-knowledge from the point of view of what's in blockchain data correct?
135 2014-12-25 03:09:16 <gmaxwell> petertodd: yea, the blockchain learns who the signers are in this case.   (though there is an interactive protocol with interactive to do m of n schnorr which is actually ZK about who signed)
136 2014-12-25 03:09:25 <gmaxwell> er with interactive setup.
137 2014-12-25 03:09:49 <gmaxwell> what pieter and I are talking about has no setup and only one-pass signing instead of a multi-round interaction.
138 2014-12-25 03:10:26 <sipa> but larger data size (though nearly constant verification cost)
139 2014-12-25 03:10:32 <petertodd> gmaxwell: good - again, businesses are worried that in the event of a hack they won't be able to know for sure which keys were compromised - boring anti-privacy auditing requirement :)
140 2014-12-25 03:11:08 <sipa> btw, does ed25519 allow any such constructs?
141 2014-12-25 03:11:14 <gmaxwell> petertodd: sure sure. I can give you non-zk which is log() scaling on pubkeys, and O(1) on the signature itself and you know exactly what set of keys signed.
142 2014-12-25 03:11:28 <sipa> i guess schnorr over the ed25519 curve would work
143 2014-12-25 03:11:34 <gmaxwell> sipa: not unless you strip out the determinstic nonce and privkey high bit set requirement.
144 2014-12-25 03:11:39 <gmaxwell> right, the curve is fine.
145 2014-12-25 03:11:41 <gmaxwell> The rest is not.
146 2014-12-25 03:12:31 <gmaxwell> the curve having a cofactor might compromise the ZKness of the ZK approaches if someone intentionally generated a key in a different subgroup, I'd have to contemplate that.
147 2014-12-25 03:12:52 <sipa> when using 32-bit code, libsecp256k1 here takes 483us per verification at best
148 2014-12-25 03:13:16 <sipa> (still better than ~1200us for OpenSSL)
149 2014-12-25 03:22:43 <phantomcircuit> wat
150 2014-12-25 03:22:48 <phantomcircuit> ghash had an orphan block
151 2014-12-25 03:22:52 <petertodd> ha
152 2014-12-25 03:57:57 <brand0> lelz @ ghash
153 2014-12-25 06:34:28 <Diablo-D3> Merry Christmas
154 2014-12-25 06:34:39 <bedeho> merry xmas
155 2014-12-25 06:58:38 <Luke-Jr> github down?
156 2014-12-25 06:59:03 <phantomcircuit> Luke-Jr, nope
157 2014-12-25 06:59:04 <cbeams> Luke-Jr: not for me.
158 2014-12-25 06:59:09 <Luke-Jr> hmm (Merry Xmas)
159 2014-12-25 06:59:18 <phantomcircuit> im waiting for someone to ddos them and remember to take down status.github.com also
160 2014-12-25 06:59:29 <phantomcircuit> it's strange that nobody ever does
161 2014-12-25 07:00:14 <phantomcircuit> oh i see it's ec2
162 2014-12-25 07:00:17 <phantomcircuit> that's amusing
163 2014-12-25 07:00:55 <Luke-Jr> well, I can't seem to reach github.com :x
164 2014-12-25 07:41:34 <wumpus> works fine here
165 2014-12-25 08:08:37 <Luke-Jr> wtf is with CC-BY-3.0 section 4c?
166 2014-12-25 08:18:52 <gmaxwell> Luke-Jr: what about it? -by-3.0 is crappy in a number of ways but 4c is just a preserve attribution requirement.
167 2014-12-25 08:20:14 <Luke-Jr> gmaxwell: "You must not distort, mutilate, modify or take other derogatory action in relation to the Work which would be prejudicial to the Original Author's honor or reputation."
168 2014-12-25 08:20:34 <gmaxwell> oh they're using unported and not US.
169 2014-12-25 08:20:40 <gmaxwell> yea... it's problematic.
170 2014-12-25 08:20:43 <Luke-Jr> for example, this could be used to prohibit altcoins the author finds offensive
171 2014-12-25 08:23:07 <gmaxwell> yea, I dunno what to say. I argued agressively against it but CC published it like that anyways. Kat fixed it in cc-by-sa-4.0.
172 2014-12-25 08:23:27 <Luke-Jr> I think we need to find another icon. That one doesn't look particularly unique.
173 2014-12-25 08:24:21 <gmaxwell> so go find one.
174 2014-12-25 08:26:31 <gmaxwell> Luke-Jr: we should probably do some gui overhaul and as a part of it pay some graphic artist to do all new icons. ... but not right now.
175 2014-12-25 08:26:49 <Luke-Jr> gmaxwell: actually.. we just did :o
176 2014-12-25 08:26:56 <gmaxwell> oh we did?
177 2014-12-25 08:27:05 <Luke-Jr> yeah, a bunch of monocolour icons got merged in master
178 2014-12-25 08:27:16 <Luke-Jr> so maybe the guy who did those can make one for this too
179 2014-12-25 08:27:43 <gmaxwell> kinda concerning that I missed that, I intentionally don't pay much attention to the gui stuff, but that rings no bells for me at all.
180 2014-12-25 08:28:02 <Luke-Jr> lol, by the same guy with this PR
181 2014-12-25 08:28:09 <Luke-Jr> (jonasschnelli)
182 2014-12-25 08:28:11 <Luke-Jr> gmaxwell: https://github.com/bitcoin/bitcoin/pull/5219
183 2014-12-25 08:37:42 <gmaxwell> cool.
184 2014-12-25 08:46:38 <wumpus> gmaxwell: if you don't follow it it doesn't really make much sense to comment on it...
185 2014-12-25 08:50:16 <gmaxwell> wumpus: How could I know that it wasn't something I cared about unless I saw it?
186 2014-12-25 08:53:28 <wumpus> gmaxwell: well you say you intentionally pay no attention to it. Which is perfectly fine by itself, but then you say what has to happen without following all the work actually happening that looks a bit strange :)
187 2014-12-25 08:53:58 <gmaxwell> wumpus: huh?
188 2014-12-25 08:54:04 <gmaxwell> I did no such thing.
189 2014-12-25 08:54:10 <wumpus> <gmaxwell> Luke-Jr: we should probably do some gui overhaul and as a part of it pay some graphic artist to do all new icons. ... but not right now.
190 2014-12-25 08:54:50 <wumpus> anyhow, just looked a bit weird, you're welcome to pay more attention to the GUI project in any case
191 2014-12-25 08:56:44 <gmaxwell> I don't pay attention to it in part because it's not really usable for me. :( (I don't run bitcoin on the system I use as an xterminal)
192 2014-12-25 08:57:02 <gmaxwell> I do know that we e.g. never got finished patches from tcatm's very nice looking rework work. Happy to hear that someone was working on icons that I missed though!
193 2014-12-25 08:57:07 <Luke-Jr> 
ACTION notes X11 is network-transparent :P

194 2014-12-25 08:57:23 <wumpus> what would make it usable for you then?
195 2014-12-25 08:58:03 <Luke-Jr> 
ACTION guesses being able to attach the GUI to a remote wallet ;)

196 2014-12-25 08:58:09 <wumpus> it's intentionally using a library that should be very portable
197 2014-12-25 08:58:23 <gmaxwell> wumpus: being able to remote the gui from the daemon would do so.  The problem with tunneling it over X is that its very slow and fragile, any bump and it knocks my daemon out and I'm waiting for it to sync when I want to go use it.
198 2014-12-25 08:58:41 <gmaxwell> (and also uses insane bandwidth if I'm not at home and care about the bandwidth)
199 2014-12-25 08:58:57 <wumpus> ok...
200 2014-12-25 08:59:02 <Luke-Jr> wumpus: besides splitting wallet from core, splitting GUI from wallet is good for security (locked-down wallet server)
201 2014-12-25 08:59:29 <wumpus> Luke-Jr: I intended to do that long time ago, but I have no time for such things anymore
202 2014-12-25 08:59:34 <Luke-Jr> :<
203 2014-12-25 08:59:43 <Luke-Jr> (same here, actually)
204 2014-12-25 08:59:52 <gmaxwell> I'll see what I can do. :)
205 2014-12-25 09:00:15 <Luke-Jr> who remembers https://en.bitcoin.it/wiki/Wallet_protocol ? :P
206 2014-12-25 09:00:31 <wumpus> unfortunately it's much harder to do know than back then when I created that
207 2014-12-25 09:00:36 <wumpus> s/know/now
208 2014-12-25 09:01:05 <wumpus> when I write bitcoin-qt it had hardly any core code in it, by design, later on it seeped into everything
209 2014-12-25 09:01:46 <gmaxwell> Can someone give me a mempool transplant from a publically accessible node that is up >24 hours and running a standard fee policy?
210 2014-12-25 09:01:49 <gmaxwell> ./bitcoin-cli getrawmempool | grep '"' | cut -d'"' -f2   | xargs -n1 ./bitcoin-cli getrawtransaction | xz -c > mempool.xz
211 2014-12-25 09:02:06 <Luke-Jr> jonasschnelli: are you here now, or just GitHub?
212 2014-12-25 09:02:19 <wumpus> gmaxwell: sure
213 2014-12-25 09:02:26 <gmaxwell> wumpus: thanks.
214 2014-12-25 09:02:34 <Luke-Jr> gmaxwell: you know there's a mempool p2p command for that… :P
215 2014-12-25 09:02:50 <gmaxwell> I know but I can't tell how long a random p2p node has been up.
216 2014-12-25 09:03:00 <Luke-Jr> I just send it to every peer :P
217 2014-12-25 09:03:01 <gmaxwell> I'd also have to write a fair bit of software instead of a shell one liner.
218 2014-12-25 09:04:41 <wumpus> https://download.visucore.com/bitcoin/mempool.xz
219 2014-12-25 09:09:46 <wumpus> Luke-Jr: the problem has always been that you can't do everything that the UI does through RPC, although no one ever bothered to make a list of what what be needed
220 2014-12-25 09:10:41 <wumpus> Luke-Jr: ie, a more efficient, incremental way of transaction list synchronization would be needed
221 2014-12-25 09:11:13 <wumpus> Luke-Jr: (not just adding some fields, that'd be easy)
222 2014-12-25 09:11:16 <Luke-Jr> wumpus: yes. I had a rather nice hack for Spesmilo, but it was definitely still a hack
223 2014-12-25 09:14:09 <wumpus> Luke-Jr: right, there's no way to ask 'have there been new wallet transactions since my last poll at timestamp XYZ'
224 2014-12-25 09:14:15 <wumpus> Luke-Jr: the new wallet should have a protocol for that
225 2014-12-25 09:15:32 <wumpus> Luke-Jr: yes I vaguely remember that wiki page
226 2014-12-25 13:42:11 <michagogo> Erm
227 2014-12-25 13:42:18 <michagogo> Why is the new debug window icon a speech bubble?
228 2014-12-25 13:46:27 <wumpus> because speech bubbles signify chat boxes, and in the RPC console you can chat with the core
229 2014-12-25 13:46:47 <michagogo> erm
230 2014-12-25 13:46:55 <wumpus> (honestly I have no idea why jonasschnelli chose a speech bubble, I suppose that's open for discussion)
231 2014-12-25 13:46:56 <michagogo> IMHO the terminal makes more sense for that
232 2014-12-25 13:47:07 <michagogo> And the debug window is a lot more than the RPC console, too...
233 2014-12-25 13:47:19 <michagogo> (especially now in 0.10/master)
234 2014-12-25 13:47:38 <michagogo> (and there may even be more enhancements by the time we reach 0.11)
235 2014-12-25 13:48:11 <wumpus> the new icons in master are experimental, you can propose something else, but be sure to use the same style
236 2014-12-25 13:48:38 <michagogo> 
ACTION is even worse at graphic design than he is at good writing

237 2014-12-25 13:48:43 <wumpus> (eh I mean you can propose something else for specific icons)
238 2014-12-25 13:49:13 <wumpus> same here.
239 2014-12-25 13:49:27 <michagogo> Eh? You're not bad at writing
240 2014-12-25 13:49:41 <michagogo> (from what I've seen, anyway)
241 2014-12-25 13:49:53 <wumpus> not *that* bad, but I'm not native english
242 2014-12-25 14:12:22 <firelegend> michagogo:Same, I can't even draw a circle correctly.
243 2014-12-25 14:39:44 <wumpus> gmaxwell: btw, re: reconnectable X programs that work over laggy connections, xpra (xpra.org) is very useful for that
244 2014-12-25 17:02:20 <s0br> i haven't seen any documentation on if we can mine new bitcoins to a multisig address? from my basic understanding it should be fine?
245 2014-12-25 17:02:37 <s0br> i just want to make sure before i go changing configs
246 2014-12-25 17:09:06 <maaku> s0br: #bitcoin, and yes there is nothing in the protocol which limits what script you can use
247 2014-12-25 17:09:13 <maaku> your pool / mining software would have to support it though
248 2014-12-25 17:11:03 <s0br> great
249 2014-12-25 20:59:44 <shesek> is there any advantage to using bip32's chain codes when the extended master public key is meant to be public?
250 2014-12-25 21:00:03 <shesek> (rather than just deriving from a master public key, without using a chain code)
251 2014-12-25 21:01:01 <shesek> it appears to me like the only reason chain codes were added is as an additional layer of security, which is moot if the master public key its meant to be public and allow third parties to derive keys from it
252 2014-12-25 21:04:49 <phantomcircuit> shesek, s/its/is/ ?
253 2014-12-25 21:06:12 <shesek> phantomcircuit, yep, sorry
254 2014-12-25 21:06:52 <phantomcircuit> shesek, something about the math i believe
255 2014-12-25 21:06:59 <phantomcircuit> see hardened keys
256 2014-12-25 21:08:53 <shesek> phantomcircuit, something about the math? :O
257 2014-12-25 21:09:46 <phantomcircuit> shesek, *hand waving*
258 2014-12-25 21:10:20 <phantomcircuit> oh right
259 2014-12-25 21:10:39 <phantomcircuit> shesek, because this way you have a tree of keypairs
260 2014-12-25 21:10:41 <phantomcircuit> not a chain
261 2014-12-25 21:10:46 <phantomcircuit> which is better
262 2014-12-25 21:11:42 <shesek> phantomcircuit, the derivation index makes it so you have a tree
263 2014-12-25 21:12:07 <shesek> as it worked with the original type-2 deterministic wallets
264 2014-12-25 21:13:42 <gmaxwell> shesek: what do you mean "without using chain codes" ?
265 2014-12-25 21:20:38 <sipa> shesek: the chain code is not meant to be public!
266 2014-12-25 21:21:25 <sipa> without a private key, nobody can steal your coins, but you can lose privacy
267 2014-12-25 21:22:04 <sipa> making your public key + chain code public gives you pretty much the same privacy as just reusing the same key...
268 2014-12-25 21:29:03 <phantomcircuit> sipa, i think people were expecting to be able to give away the extended public key to preserve privacy between the two parties
269 2014-12-25 21:29:09 <phantomcircuit> ie not public but public-ish
270 2014-12-25 21:42:19 <jcorgan> if you create an extended public key per 'customer' and only it to them, then only they can link all the created individual keys together
271 2014-12-25 21:42:27 <jcorgan> *only give
272 2014-12-25 21:57:56 <sipa> yes, so you can give it to them if you want that key to be only used by them
273 2014-12-25 22:01:02 <sipa> at which point there is also no privacy problem
274 2014-12-25 22:04:39 <jcorgan> only a coordination issue on which subkey to use.  but that should be handled by a wallet that keeps track of that.
275 2014-12-25 22:14:47 <shesek> gmaxwell, basically, just doing regular derivation from a master public key with some index (pubkey+G*index)
276 2014-12-25 22:15:31 <shesek> sipa, not really - in my specific case, the deterioration index itself is considered a secret (its not incremental, its an hash of some information only known to the involved parties)
277 2014-12-25 22:15:49 <shesek> blah, autofix fail - s/deterioration/derivation
278 2014-12-25 22:16:54 <shesek> what I'm doing is kinda similar to how stealth addresses are meant to work
279 2014-12-25 22:21:57 <gmaxwell> shesek: I'd be somewhat concerned about signing things with a simple linear relationship.
280 2014-12-25 22:23:18 <gmaxwell> (or, in particular a potentially attacker controlled simple linear relationship)
281 2014-12-25 22:24:27 <shesek> gmaxwell, its not really different with BIP32, though. even with the chain code, its just getting hashed along with some other data to arrive at the final derivation index
282 2014-12-25 22:25:29 <shesek> with bip32, its basically pubkey+G*hmac(pubkey||index, chaincode)
283 2014-12-25 22:25:37 <gmaxwell> shesek:  Yes, I know how it works.
284 2014-12-25 22:25:38 <gmaxwell> ...
285 2014-12-25 22:26:15 <shesek> yeah, of course you do, haha :)
286 2014-12-25 22:26:21 <shesek> I'm just not sure what you meant exactly
287 2014-12-25 22:26:31 <shesek> would you be concerned about using bip32 in general?
288 2014-12-25 22:26:54 <gmaxwell> shesek: That is not "not really different", with a free index an attacker could freely query key plus 1,2,4,8,16, etc. to collect signatures directed and adaptive linear relationships.
289 2014-12-25 22:27:19 <gmaxwell> If this were the nonce it would immediately compromise the security of the signatures, though it's not symmetric.
290 2014-12-25 22:29:35 <gmaxwell> shesek: BIP32 hardened derviation, nah. BIP32 public? I'd only use it where the only option were something less secure. (e.g. leave keys on server, don't use multisig).  It is novel cryptography. We have no proof that it doesn't result in a reduction in security. We've studied it moderately well at this point, however. Some of the tor project folks were working on a proof (in the context of schnorr s
291 2014-12-25 22:29:41 <gmaxwell> ignatures, not ecdsa; but even that would be a help)
292 2014-12-25 22:31:28 <gmaxwell> Part of the reason the HMAC is in there is because of an abundance of caution; if there is some weakness that does arise out of attacker controlled linear relationships between keys the hmac likely makes it impratical to exploit.
293 2014-12-25 22:33:04 <shesek> well, not if he knows the chain code. and if he doesn't, then he can't derive keys and control the relationship between them anyway
294 2014-12-25 22:33:59 <gmaxwell> right if someone doesn't know the chain code and the hmac is in place, then the keys are indistinguishable from random up to a hardness assumption about related hmac values.
295 2014-12-25 22:34:53 <gmaxwell> if they do know the chain code then the key relations will at least not follow any tidy linear relationship, and chosing a particular relationship has exponential complexity (or really just isn't possible because of the limited index size)
296 2014-12-25 22:34:53 <shesek> what if the index is a hash of some private information that can't be known to an attacker? wouldn't you get the exact same security properties then?
297 2014-12-25 22:35:22 <gmaxwell> shesek: thats BIP32 then, with an attacker unknown chaincode.
298 2014-12-25 22:36:10 <shesek> and btw, what about stealth addresses? they seem to be extremely vulnerable according to what you say
299 2014-12-25 22:37:05 <gmaxwell> shesek: no the stealth address construction is also hash mediated.
300 2014-12-25 22:37:30 <gmaxwell> It's P1 + G*H(P2*n)
301 2014-12-25 22:38:38 <gmaxwell> though I have pretty big mixed feelings about stealth addresses; though not related to security.
302 2014-12-25 22:40:04 <shesek> basically, I'm currently using non-bip32 key derivation for to figure out the keys for the other parties in a multi-signature, so that it won't have to be interactive (its for Bitrated v2, meant to allow buyers to make payments immediately, without waiting for the other parties to provide a public key and without reusing public keys)
303 2014-12-25 22:41:22 <gmaxwell> okay so you want your 'index' to just be some other parties pubkeys.
304 2014-12-25 22:41:26 <shesek> which is basically pubkey+G*H(trade_data), where trade_data contains some information only known to the involved parties
305 2014-12-25 22:41:43 <shesek> I was thinking that its probably a good idea to switch over to standard bip32 rather than using my own implementation... but it seems like bip32 doesn't really fit my use case
306 2014-12-25 22:42:12 <gmaxwell> You should be using a pay to contract.
307 2014-12-25 22:42:12 <shesek> the index is a unique trade identifier unknown to third parties
308 2014-12-25 22:42:27 <shesek> pay to contract?
309 2014-12-25 22:42:43 <gmaxwell> which is pubkey+G*HMAC(key=pubkey,data=data)
310 2014-12-25 22:43:38 <gmaxwell> Which is provably binding, e.g. the resulting key is a computationally unforgable hash of the trade data.
311 2014-12-25 22:44:14 <gmaxwell> (thats not so for the pubkey+G*H(trade_data) construction.)
312 2014-12-25 22:45:01 <gmaxwell> You'll probably also want to have some constraint on the data e.g. begins with 'Bitrated2' or something to prevent emulation by other things using a similar construction.
313 2014-12-25 22:45:25 <gmaxwell> shesek: see also https://github.com/Blockstream/contracthashtool
314 2014-12-25 22:47:07 <shesek> yeah, that's probably a good idea. I'm putting "bitrated-..." as a salter for my KDF (for key stretching), I'll do it there too
315 2014-12-25 22:48:06 <gmaxwell> hopefully your KDF isn't just a brainwallet?
316 2014-12-25 22:49:46 <shesek> no; its scrypt (N=16, r=8, p=1) over a password I'm generating for the user (154 bits of entropy)
317 2014-12-25 22:50:42 <shesek> (that's to arrive at the main public key for the user, used for authentication and authorizing operations; that key is then scrypted again with a user-provided password to arrive at the master key I'm using for the multisig itself)
318 2014-12-25 22:51:44 <gmaxwell> key point being "over a password I'm generating" good. :)
319 2014-12-25 22:52:17 <shesek> hmm, so, would you use bip32 or just work with plain derivation instead?
320 2014-12-25 22:52:41 <shesek> I could use H(trade_data) as a one time "chain code" for each derivation... isn't exactly the way bip32 is meant to work, but... it could work
321 2014-12-25 22:53:10 <shesek> alternatively, I could use a known public chain code (or just use none at all, which is pretty much the same I believe?) and use H(trade_data) as the index
322 2014-12-25 22:54:05 <shesek> though, then my H(trade_data) is going to get trimmed to 2^31 bits
323 2014-12-25 22:54:36 <gmaxwell> Well, BIP32 used that way is a valid pay to contract construction.
324 2014-12-25 22:54:48 <gmaxwell> "that way" meaning as the chain code.
325 2014-12-25 22:55:27 <gmaxwell> just use index 0. (or the first one where the derrivation doesn't fail)
326 2014-12-25 22:55:43 <gmaxwell> You'd also gain some basic compatiblity with BIP32 software that way.
327 2014-12-25 22:56:07 <gmaxwell> e.g. have a fighting chance of getting some hardware signer like btcchip working with it.
328 2014-12-25 22:56:47 <shesek> well, it would probably require importing multiple different extended master keys (each with a different chain code) for each regular key
329 2014-12-25 22:57:12 <gmaxwell> yes, I think btcchip is fine with this. e.g. when you ask it to sign you give it the chain code and index.
330 2014-12-25 22:57:34 <shesek> ah, really? that's good
331 2014-12-25 22:57:58 <gmaxwell> I haven't spent much time with it, but thats my recollection.
332 2014-12-25 22:57:59 <shesek> well, I'll go with that, then. BIP32, one-time unique chain codes, index 0
333 2014-12-25 22:58:21 <shesek> thanks for the help, greg. much appreciated!
334 2014-12-25 22:58:42 <gmaxwell> that also gives you the good strong binding property. So you can prove to a third party what the contract was later, if needed.
335 2014-12-25 22:58:46 <gmaxwell> No problem.
336 2014-12-25 22:59:07 <shesek> gmaxwell, yep
337 2014-12-25 22:59:53 <shesek> I also have the users sign the contract, but the contract being hashed into the derivation index as a way to prove what was the contract is also something I had in mind
338 2014-12-25 23:00:44 <gmaxwell> yea, it's just important that the pubkey be in the hash if you're going to do that. or otherwise someone can engage in funny business.
339 2014-12-25 23:03:58 <gmaxwell> shesek: so if you might want to reveal contracts in a dispute I have another construction that you might want to consider using, perhaps in a later version.   It lets you hash a document such that you can reveal only some byte ranges and still check the hash.  E.g. I could show someone else the signed (or pay-to-contract) committed contract but blot out the delivery address.
340 2014-12-25 23:06:59 <shesek> gmaxwell, my trade data is a JSON object (lets say, `{product:<string>, price: <int>, address: <string>}`). I was actually thinking of first hashing each field (`{product:H(<string>), price: H(<int>), ...}`), then hash that whole thing, to arrive at something that provides a similar functionality to what you described
341 2014-12-25 23:07:35 <gmaxwell> shesek: well take care, if some of the fields have low entropy you can guess and check them.
342 2014-12-25 23:07:50 <shesek> yep, I thought about that too
343 2014-12-25 23:08:05 <shesek> was thinking of adding some nonce for that
344 2014-12-25 23:08:41 <gmaxwell> my structure takes it to the logical conclusion, and avoids having extreme inefficiency from sending lots of nonces.
345 2014-12-25 23:09:32 <shesek> sounds great. I would love to learn more about that construction. do you have a writeup somewhere?
346 2014-12-25 23:10:30 <gmaxwell> First notion, let me introduce to you the concept of a tree structured CSPRNG. Silly example, SHA512, put a 256 bit secret in, get a 512 bit derrived secret out, which you take as two 256 bit secrets. So it's an expander.
347 2014-12-25 23:10:58 <gmaxwell> Now take your message, hash it linearly like normal, we'll use this as our master nonce secret.
348 2014-12-25 23:11:45 <gmaxwell> using that master nonce secret, recursively apply the tree-CSPRNG to get yourself a 256 bit secret for every byte in the message. Like building a merkel tree, but backwards.
349 2014-12-25 23:12:39 <gmaxwell> Now build up a hash tree over the byte, nonce pair.   Root of the tree is your hash.
350 2014-12-25 23:13:53 <gmaxwell> To reveal a segment of bytes to a party, find the highest set of nodes in the secret key that cover only the data you want to reveal, and give them them along with the revealed data.
351 2014-12-25 23:14:04 <gmaxwell> and the off-path hashes for the non-revealed data.
352 2014-12-25 23:14:33 <shesek> sorry, I have to run off - be back later
353 2014-12-25 23:14:35 <gmaxwell> and now they can verify agreement. Worst case amount of overhead is O(log2(bytes))
354 2014-12-25 23:14:38 <gmaxwell> K ttyl.