1 2015-09-07 08:39:55 <jonasschnelli> gmaxwell: no problem with the pub auth PR. BitAuth upcounting nonce seem to be a replay attack prevention. But I found out that Copay (and its backend) do not use the nonce. :-)
  2 2015-09-07 08:41:36 <jonasschnelli> So they fully depend on SSL.
  3 2015-09-07 08:42:47 <jonasschnelli> gmaxwell: thanks also for the maxuploadtarget review. Good point with the GetBlockTime() issue. Will do a fix soon.
  4 2015-09-07 08:50:07 <gmaxwell> jonasschnelli: none of their software seems to implement that, and it's just as well because it wouldn't really work for most uses.
  5 2015-09-07 08:50:38 <gmaxwell> E.g. if you have two clients with the same key, or a client that forgets its last value..., or a distributed server, or a server that can't save state.  etc.
  6 2015-09-07 08:50:56 <gmaxwell> Correct solution is to server challenge, but that doesn't fit in the one way request jsonrpc model I guess.
  7 2015-09-07 08:55:10 <jtimon> wumpus: can we merge #6068 now that cfields and Luke-Jr seem to be happy with it simultaneously?
  8 2015-09-07 08:59:38 <jonasschnelli> gmaxwell: what about sending a uuid within a x-header and include this UUID in the signature part? Server would require to keep track of used UUIDs? Would this open a mem dos attack vector?
  9 2015-09-07 09:05:19 <gmaxwell> jonasschnelli: the server sending a per connection then the client counting up from it, and the sever not allowing prior ones would work. Though it would make it harder to implement a client for. I think most of the motivation of the bitpay scheme was that it required not http integration, it just wrapped the requests you already made (and in normal client software your whole request may even be enco
 10 2015-09-07 09:05:25 <gmaxwell> ded before you even open the socket)
 11 2015-09-07 09:07:49 <gmaxwell> past some point the compatiblity is so poor, ... it's best off with a protocol that provides good properties, e.g. confidentiality, ability to not have messages selectively dropped, mutual auth (know you're talking to the right server), and performance (signing and verifying is slow, hundreds of times slower than a MAC). :)  But yes, to confirm your understanding, I do think a server challenge under
 12 2015-09-07 09:07:55 <gmaxwell> the signature would be adequate and could be done in a way with no additional memory usage, except a per connection counter.
 13 2015-09-07 09:51:05 <wumpus> I agree with gmaxwell. Let's not invent our own wheel here. I think it is risky and brings all kinds of new security concerns into our code. Using SSL, or another already existing transport-level security seems preferable, and more straightforward too.
 14 2015-09-07 09:51:26 <wumpus> jtimon: will take a look
 15 2015-09-07 09:52:10 <wumpus> I wouldn't have ever proposed dropping SSL if I know it would result in people trying to invent their own RPC authentication proposals
 16 2015-09-07 09:53:46 <wumpus> SSL + client/server certificates solves this use-case
 17 2015-09-07 09:53:57 <wumpus> so does SSH + pubkeys + tunneling
 18 2015-09-07 09:57:54 <wumpus> and if you hate "legacy" cryptosystems, another option would be tunneling through eg CurveCP. But please, let's not reinvent our own wheel, certainly not for RPC.
 19 2015-09-07 10:15:09 <CodeShark> does anyone know off-hand the block height when BIP66 went into effect?
 20 2015-09-07 10:17:04 <jonasschnelli> wumpus: agreed on all points. Ec pub auth was an experiment and fun to implement. But not really useful for production.
 21 2015-09-07 11:28:27 <CodeShark> the start of pulling out soft forks into their own unit: https://github.com/CodeShark/bitcoin/blob/softforks/src/softforks.cpp
 22 2015-09-07 11:30:25 <CodeShark> and example usage: https://github.com/CodeShark/bitcoin/blob/softforks/src/main.cpp#L1705
 23 2015-09-07 11:31:44 <CodeShark> all the IsSuperMajority crap needs to be pulled out as well - and all the checks for block version
 24 2015-09-07 11:40:09 <CodeShark> then all the version bits logic can be placed in the softforks unit
 25 2015-09-07 12:13:14 <mjerr> on https://bitcoin.org/en/developer-guide#non-standard-transactions it says any other tx type than p2pkh/p2sh/multisig/pubkey/op_return is without further checks directly marked off as non-standard - is this correct?
 26 2015-09-07 12:17:05 <mjerr> so even simple scripts that make use of OP_IF are non-standard and the only way to use any other OP-code is to go with P2SH?
 27 2015-09-07 12:19:23 <jonasschnelli> mjerr: Even a P2SH redeem script needs to conform the "standardness". A isStandard check will be required when spending the output.
 28 2015-09-07 12:19:59 <mjerr> jonasschnelli, thats what I thought as well, but than I found this https://gist.github.com/gavinandresen/88be40c141bc67acb247
 29 2015-09-07 12:20:24 <mjerr> where IsStandard got relaxed a lot
 30 2015-09-07 12:20:30 <mjerr> for P2SH
 31 2015-09-07 12:20:50 <jonasschnelli> Sure... but it's a proposal.
 32 2015-09-07 12:21:00 <mjerr> https://github.com/bitcoin/bitcoin/pull/4365
 33 2015-09-07 12:21:00 <mjerr> oh no, it got merged
 34 2015-09-07 12:23:22 <jgarzik> mjerr, jonasschnelli: Yes you can have just about anything in the P2SH redeem script
 35 2015-09-07 12:24:01 <mjerr> jgarzik, but normal scripts are not possible, unless they fall into p2pkh/p2sh/multisig/pubkey/op_return?
 36 2015-09-07 12:24:26 <jgarzik> mjerr, normal scripts are whitelisted to a tiny set of templates, yes
 37 2015-09-07 12:24:44 <jgarzik> mjerr, normal scripts take up room in UTXO; redeem scripts do not
 38 2015-09-07 12:24:48 <jonasschnelli> thanks @jgarzik, wasn't aware of this.
 39 2015-09-07 12:25:32 <mjerr> jgarzik, ah that was the way I thought it to be, so it's not as black/white - what are the properties of the whitelisting?
 40 2015-09-07 12:26:13 <jgarzik> mjerr, pretty much what the word implies.  there is a whitelist of script templates accepted; all others are rejected for relaying over the network.
 41 2015-09-07 12:26:34 <jgarzik> You can of course mine anything you want in a block.
 42 2015-09-07 12:26:41 <mjerr> oh so it is p2pkh/p2sh/multisig/pubkey/op_return ONLY
 43 2015-09-07 12:26:55 <mjerr> so anything with OP_IF for example would already not pass IsStandard
 44 2015-09-07 12:29:02 <jgarzik> mjerr, correct.  OP_IF would need to be in a P2SH redeem script, or mined manually by a miner.  It would not get relayed across the network by Bitcoin Core.
 45 2015-09-07 12:29:59 <mjerr> hm interesting - so LN is only possible using P2SH
 46 2015-09-07 12:34:46 <mjerr> jgarzik, I guess you are right, LN output scripts can grow quite large, not storing them in the UTXO is good practice.. Still wraping my head around whether LN scripts are fully deterministic, such that you don't have to store the script of all transactions ever
 47 2015-09-07 13:01:21 <jgarzik> mjerr, In some scenarios it is possible to recreate the redeem script rather than store it, yes
 48 2015-09-07 13:01:53 <jgarzik> mjerr, simply requires (1) following a protocol with 100% precision, and (2) knowing all the associated hashes, PKH's and other script inputs
 49 2015-09-07 13:06:39 <jgarzik> jonasschnelli, (no offense intended; I like to avoid PMs)
 50 2015-09-07 13:06:56 <jgarzik> jonasschnelli, wrapping namespace is ideal long term solution -- however it probably requires touching a lot of code
 51 2015-09-07 13:07:20 <jgarzik> class might need a rename too, to avoid referring to "UniValue::UniValue::UniValue" everywhere
 52 2015-09-07 13:07:23 <jonasschnelli> jgarzik: Okay. Sure. We can only wrap the Parsing function in a namespace?
 53 2015-09-07 13:07:57 <jonasschnelli> And s/ParseInt32/UniValue::ParseInt32/?
 54 2015-09-07 13:08:22 <jonasschnelli> not sure if we can use a namespace UniValue while a class with same name exists.. probably no.
 55 2015-09-07 13:08:32 <jgarzik> jonasschnelli, The smallest change could move them to univalue.cpp, their only user, and make them local
 56 2015-09-07 13:08:48 <kanzure> mjerr: you don't have to completely store lightning network scripts. you can use a scheme like p3sh that uses merklized abstract syntax trees where most of the script is "hidden". as long as the participants cooperate, you don't have to reveal the longer scripts.
 57 2015-09-07 13:09:35 <jgarzik> jonasschnelli, then stick those functions in an anonymous local namespace...
 58 2015-09-07 13:10:02 <jonasschnelli> jgarzik: okay. Sounds good. Will try
 59 2015-09-07 13:18:06 <jonasschnelli> jgarzik: what do you think: https://github.com/jgarzik/univalue/pull/5/files
 60 2015-09-07 13:18:38 <jgarzik> jonasschnelli, ACK assuming it  builds and tests :)
 61 2015-09-07 13:18:54 <jonasschnelli> jgarzik: testes on OSX & Debian
 62 2015-09-07 13:20:06 <jgarzik> jonasschnelli, merged
 63 2015-09-07 13:20:19 <jonasschnelli> okay,.. will update the subtree patch.
 64 2015-09-07 13:32:44 <mjerr> channel is closed
 65 2015-09-07 13:32:44 <mjerr> kanzure, hm interesting.. kanzure, jgarzik, yes, it is possible to omit a lot of the statics in the scripts, but you need to store the exact dates and hashes of all payments ever made in this payment channel.. unlike the revocation hashes, these will in generally not build hierarchical, so every payment, whether it was successful or refunded, will add ~30 bytes to the database until the
 66 2015-09-07 13:33:25 <mjerr> but as it is non-discussable to open up isStandard, we just have to accept that
 67 2015-09-07 13:33:52 <mjerr> 30 bytes should be managable, even with 100 thousands of payments
 68 2015-09-07 13:34:02 <kanzure> i believe you can reset the channel if the two parties are still cooperative
 69 2015-09-07 13:34:54 <mjerr> kanzure, if you can't reconstruct the exact script of an old channel state though, the other party can broadcast it and wait for all timeouts to pass
 70 2015-09-07 13:35:10 <kanzure> why would you not be able to reconstruct?
 71 2015-09-07 13:35:23 <kanzure> i might be missing context in this conversation :-)
 72 2015-09-07 13:35:24 <mjerr> you are, but only if you keep exact records of all payments ;)
 73 2015-09-07 13:35:39 <mjerr> nah all good
 74 2015-09-07 13:35:45 <mjerr> still wrapping my head around it as well
 75 2015-09-07 13:38:00 <mjerr> there are currently 4 'variables' in a HTLC script - R-Hash, Revoke-Hash, Revoke-Timeout (relative, probably will be rather constant) and Payment-Timeout (variable, as it is absolute)
 76 2015-09-07 13:38:20 <mjerr> you are able to reconstruct it, but you need to store this data for all payments until the end of the channel
 77 2015-09-07 14:29:48 <jonasschnelli> mjerr: maybe review: https://github.com/bitcoin-dot-org/bitcoin.org/pull/1054?
 78 2015-09-07 16:20:22 <erasmospunk> Is there a way to prove the total BTC value of subset of UTXOs on a specific height without leaking private data?
 79 2015-09-07 16:21:42 <erasmospunk> i.e. the total amount is 1btc but cannot cannot know the specific UTXOs
 80 2015-09-07 16:22:45 <buZz> nope
 81 2015-09-07 16:29:26 <nwilcox> erasmospunk: How is the subset selected?
 82 2015-09-07 16:30:03 <erasmospunk> nwilcox: for all we know, randomly
 83 2015-09-07 16:30:49 <erasmospunk> I think that it is not possible but just wanted to check with the other devs
 84 2015-09-07 16:31:58 <nwilcox> erasmospunk: I believe it is possible to construct a zeroknowledge proving circuit for this.
 85 2015-09-07 16:32:35 <nwilcox> -and I wouldn't be surprised if there's a less heavy-handed alternative.
 86 2015-09-07 16:34:29 <nwilcox> Hm...  also, if there were only 1 possible subset that happened to sum to X, then there'd be no privacy possible.
 87 2015-09-07 16:34:47 <nwilcox> The anonymity set size would be the number of possible subsets which sum to X.
 88 2015-09-07 16:34:50 <erasmospunk> I was thinking about a TX chains that lead up to a specific UTXO, along the partial merkle trees and the headers. But in that case you know the UTXO
 89 2015-09-07 16:36:54 <nwilcox> I said there'd be no privacy possible, but that assumes efficiently summing all possible subsets at each blockheight...
 90 2015-09-07 16:38:59 <nwilcox> erasmospunk: What's your imagined use?  If the proof is "I know an arbitrary subset of UTXOs at height H which hold X BTC." that doesn't seem very useful.
 91 2015-09-07 16:39:05 <nwilcox> -by itself.
 92 2015-09-07 16:39:46 <nwilcox> If it's something like that "-and I know the private key controlling the entire subset" then I can imagine some applications.  ;-)
 93 2015-09-07 16:40:05 <erasmospunk> nwilcox: thinking about proving a wallet balance on a specific height
 94 2015-09-07 16:40:45 <nwilcox> Right, so the subset needs to be constrained by also knowing a controlling scriptSig for all UTXOs within.
 95 2015-09-07 16:41:08 <erasmospunk> so I can prove how much I have, without telling you the specific UTXOs
 96 2015-09-07 16:41:16 <nwilcox> Right.
 97 2015-09-07 16:41:29 <erasmospunk> you are right about the unique sum though
 98 2015-09-07 16:41:48 <erasmospunk> it leaks all private data
 99 2015-09-07 16:41:55 <nwilcox> If you had unlinkable transactions, the wallet could combine all UTXOs into a single one, prove control of that, and then disperse that single one back to other private ones.
100 2015-09-07 16:42:07 <nwilcox> -at the cost/complexity of performing all those transactions.
101 2015-09-07 16:42:50 <nwilcox> erasmospunk: So an obvious augmentation is to prove a range...
102 2015-09-07 16:42:57 <erasmospunk> not on bitcoin blockchain though
103 2015-09-07 16:43:20 <erasmospunk> yeah, range would be the best
104 2015-09-07 16:43:22 <nwilcox> But now I'm wondering about the histogram of wallet totals, and if your proof "needs" to fall into a sparse part of that histogram, you're still in a bind.
105 2015-09-07 16:44:34 <nwilcox> (Oh, this convo is probably better for -wizards, I assume.)
106 2015-09-07 16:51:27 <erasmospunk> (wrote also there)
107 2015-09-07 18:01:49 <jamesob> iinaj in #6613, you mention making an "upgrade process" to obfuscate the entire chainstate db in one go... are there any existing examples of "upgrade processes" in the codebase?
108 2015-09-07 18:02:16 <jamesob> (just wondering if there's an example I can reference)
109 2015-09-07 18:03:03 <iinaj> ? I think you got the wrong guy
110 2015-09-07 18:04:44 <jamesob> ah yeah, haha
111 2015-09-07 18:04:46 <jamesob> sorry :)
112 2015-09-07 18:08:45 <nwilcox> jamesob: I'm not familiar with the history, but I do know that the db switched from berkeleydb to leveldb at some point, so perhaps the same process could be used.
113 2015-09-07 18:09:16 <nwilcox> The switch of db backends probably required replaying history.  I don't know how that would work w/ pruning.  Any dev historians awake?
114 2015-09-07 18:54:11 <gmaxwell> jamesob: as far as upgrade, I think the obfscuation should simply store a flag (or key that can be zero). To activate it, you'd simply reindex. So reindex would be the upgrade process.  nwilcox's example isn't bad-- and in that case, reindex was the upgrade process (though switching there was mandatory)
115 2015-09-07 18:54:41 <gmaxwell> for this if the software is working for someone already I don't think there is an urgent need to immediately activate the obfscuation.
116 2015-09-07 19:40:47 <Luke-Jr> gmaxwell: meh, reindex is slow :/
117 2015-09-07 19:41:32 <Luke-Jr> gmaxwell: consider that the current/pre-obfuscated state may be an exploit we should encourage closing (node can be killed by triggering AV)
118 2015-09-07 19:46:52 <gmaxwell> reindex is slow because it disables checkpoints.
119 2015-09-07 19:47:21 <gmaxwell> Luke-Jr: Any AV killable thing is almost certantly already killed.
120 2015-09-07 20:01:59 <Nickyo9> Bitcoin tomorrow 280$-300$ look at this graph: http://www.coindiagram.com , if you want to follow us on coindiagram our channel is: "BTCTradingGroup"
121 2015-09-07 20:02:18 <jamesob> nwilcox gmaxwell, thanks. I'll work on a draft for the obfuscation itself and then we can discuss migration in-PR
122 2015-09-07 21:10:25 <phantomcircuit> gmaxwell, bleh leveldb spikes memusage to crazy levels
123 2015-09-07 21:10:49 <phantomcircuit> seems to be able to use > 512MB in short bursts
124 2015-09-07 21:11:08 <phantomcircuit> http://0bin.net/paste/nIC3Jj3-UQ6d6g6t#Kw5WDBxDLzxyzQh4F434f0YHJyV6xsQzRYPTCPvIO5A
125 2015-09-07 21:11:19 <phantomcircuit> dbcache=512
126 2015-09-07 21:12:10 <gmaxwell> as an aside, we really need to try out lmdb.
127 2015-09-07 21:12:32 <gmaxwell> phantomcircuit: do you have txindex on?
128 2015-09-07 21:13:02 <phantomcircuit> gmaxwell, nope
129 2015-09-07 21:13:21 <phantomcircuit> 2015-09-07 11:35:25 UpdateTip: new best=00000000000000ae72fef23c91bac347b80e0d7aa5b04c94336ec660b5e2d422  height=235590  log2_work=69.998749  tx=17573118  date=2013-05-11 04:08:22 progress=0.091901  cache=253.5MiB(359023tx)
130 2015-09-07 21:13:21 <phantomcircuit> that happened immediately after
131 2015-09-07 21:43:44 <phantomcircuit> gmaxwell, im guessing that happens when leveldb is combining sorted tables while also still accessing all of the other ones
132 2015-09-07 21:43:55 <bedeho> what is the disctinction between the hash type flag, which postfixes a signature, and a hashtypecode?
133 2015-09-07 21:45:12 <phantomcircuit> bedeho, those sound vaguely like the same thing
134 2015-09-07 21:45:35 <bedeho> https://en.bitcoin.it/w/images/en/7/70/Bitcoin_OpCheckSig_InDetail.png
135 2015-09-07 21:45:35 <bedeho> phantomcircuit, apparently not, one is 1 byte, one is 4 bytes
136 2015-09-07 21:47:31 <bedeho> phantomcircuit, is it just a matter of expanding a one byte value to four bytes by adding a bunch of prefix zeros?
137 2015-09-07 21:47:50 <phantomcircuit> bedeho, no it's the same thing
138 2015-09-07 21:48:02 <bedeho> ok
139 2015-09-07 21:48:24 <phantomcircuit> yes the 1 byte is turned into 4 bytes for purposes of the signature check
140 2015-09-07 21:48:37 <phantomcircuit> the fun of serialize/deserialize not being symetric
141 2015-09-07 21:52:19 <bedeho> phantomcircuit, ok, so how does one go from 1 byte hash type value to 4 byte hash code value, and why is there even a difference
142 2015-09-07 21:52:20 <bedeho> ?
143 2015-09-07 21:52:34 <bedeho> couldnt one just have used the 1byte hash type directly
144 2015-09-07 21:59:43 <phantomcircuit> bedeho, it's just s quirk of the serialize/deserialize logic being different
145 2015-09-07 21:59:53 <phantomcircuit> it goes from 1 byte little endian to 4 bytes little endian
146 2015-09-07 22:00:12 <phantomcircuit> iirc it's actually a signed int
147 2015-09-07 22:00:23 <phantomcircuit> but all of the valid values are unsigned
148 2015-09-07 22:00:27 <phantomcircuit> actually
149 2015-09-07 22:01:13 <phantomcircuit> yes it is signed
150 2015-09-07 22:06:36 <bedeho> so hashcode is just a signed 4 byte integer version of the 1 byte unsigned hash type code postfixing the signature?
151 2015-09-07 22:08:26 <phantomcircuit> bedeho, yes
152 2015-09-07 22:08:50 <phantomcircuit> im thinking the signed ness of it is irrelevant as there's no way to get a negative value
153 2015-09-07 22:08:59 <phantomcircuit> but i could be wrong there
154 2015-09-07 22:09:02 <bedeho> ok
155 2015-09-07 22:09:53 <bedeho> so does this also mean that the signture itself is generated by signing sha^2(tx.serialized() || hashCode) rather than sha^2(tx.serialized() || hashType)
156 2015-09-07 22:11:24 <phantomcircuit> bedeho, hashcode and hashtype are the same thing just serialized differently
157 2015-09-07 22:11:53 <phantomcircuit> bedeho, if you're trying to construct a valid script signature, you probably want to go look at the code directly
158 2015-09-07 22:26:29 <bedeho> phantomcircuit, would this be it? https://github.com/bitcoin/bitcoin/blob/ef30389e2a4a83c2845a2cbdf3a7c2f062404077/src/script/interpreter.cpp#L1078
159 2015-09-07 22:28:21 <phantomcircuit> bedeho, yeah the interesting part is in the CTransactionSignatureSerializer class
160 2015-09-07 22:31:13 <bedeho> phantomcircuit, not sure, its seems it just looks at the passed nHashType  and then picks out which parts out the tx to serialize
161 2015-09-07 22:31:31 <phantomcircuit> bedeho, that's exactly what it does
162 2015-09-07 22:31:39 <bedeho> CHashWriter is the one which actually appends nHashType (unsigned int)
163 2015-09-07 22:32:03 <phantomcircuit> bedeho, what are you trying to do? (i might be more useful if i knew)
164 2015-09-07 22:33:05 <bedeho> I have to properly compute the sighash, basically, and there is this question of how the hash type involved
165 2015-09-07 22:33:39 <bedeho> ss << txTmp << nHashType; indicates that a full 4 byte hashcode is appended
166 2015-09-07 22:33:42 <phantomcircuit> it selects which parts of the transaction you're signing basically
167 2015-09-07 22:33:58 <bedeho> I am aware of the purpose, trying to get the encoding right
168 2015-09-07 22:34:02 <phantomcircuit> the weird encoding of it on the end is just noise
169 2015-09-07 22:34:18 <bedeho> have to get the noise right Im afraid... :/
170 2015-09-07 22:34:18 <phantomcircuit> bedeho, it's 32 bit little endian
171 2015-09-07 22:34:27 <phantomcircuit> what's the problem?
172 2015-09-07 22:35:03 <phantomcircuit> you might want to modify a bitcoind to dump out that ss value
173 2015-09-07 22:35:30 <bedeho> my implementation was just adding the one byte straight up, which I now see is incorrect
174 2015-09-07 22:36:05 <bedeho> does this reveal the encoding ? https://github.com/bitcoin/bitcoin/blob/ef30389e2a4a83c2845a2cbdf3a7c2f062404077/src/script/interpreter.cpp#L998
175 2015-09-07 22:36:15 <bedeho> (nHashTypeIn & 0x1f) == SIGHASH_SINGLE
176 2015-09-07 22:36:34 <phantomcircuit> bedeho, uh
177 2015-09-07 22:36:38 <phantomcircuit> <phantomcircuit> bedeho, it's 32 bit little endian
178 2015-09-07 22:37:07 <bedeho> yes,  Im trying to get how each type is encoded in that 32 bits
179 2015-09-07 22:37:33 <phantomcircuit> it's a bit mask field
180 2015-09-07 22:37:37 <bedeho> yes ok
181 2015-09-07 22:38:33 <bedeho> ah ok, 0x1f == 31d, which the wiki mentions
182 2015-09-07 22:38:50 <bedeho> is there any significance to that value?
183 2015-09-07 22:39:49 <bedeho> phantomcircuit, these were the values I was directly adding
184 2015-09-07 22:39:50 <bedeho> https://github.com/bitcoin/bitcoin/blob/41076aad0cbdfa4c4cf376e345114a5c29086f81/src/script/interpreter.h#L22
185 2015-09-07 22:40:25 <phantomcircuit> bedeho, think about them as bits not numbers
186 2015-09-07 22:41:29 <bedeho> ah ok, that makes more sense :)
187 2015-09-07 22:42:54 <bedeho> phantomcircuit,  so sighash types really just tells you the interpretation of various bits in hash code, I think I got it now... ty
188 2015-09-07 23:22:57 <bedeho> phantomcircuit, am I correct to understand that you in fact use both _all and _none flags at the same time, then that will be interpreted the same as _single?
189 2015-09-07 23:24:03 <bedeho> I guess that is a clever way to avoid the contradiction that would otherwise constitute, as they are contradictory
190 2015-09-07 23:25:12 <nwilcox> bedeho: given the values defined for those identifiers here: https://github.com/bitcoin/bitcoin/blob/ef30389e2a4a83c2845a2cbdf3a7c2f062404077/src/script/interpreter.h#L22
191 2015-09-07 23:25:25 <nwilcox> -you are correct because 1 | 2 == 3.
192 2015-09-07 23:26:52 <bedeho> nwilcox, ty
193 2015-09-07 23:27:52 <nwilcox> Are you implementing transaction signing/verification in another system/language?
194 2015-09-07 23:30:27 <bedeho> no, in c++ actually
195 2015-09-07 23:33:59 <bedeho> so, both the 1 byte hash type value following a signature when in scriptSig is the same bit field as the hash code used when generating the sighash?
196 2015-09-07 23:34:47 <bedeho> nwilcox, do you have any clue why this expansion from 1 byte to 4 byte is even done? makes no sense?
197 2015-09-07 23:37:37 <gmaxwell> bedeho: it's an integer in memory, and thats just how it gets seralized for the hasher.
198 2015-09-07 23:40:01 <nwilcox> bedeho: It's common in c/c++ land to represent bit fields or bytes as "int".
199 2015-09-07 23:40:24 <bedeho> nwilcox, but why not do that when postfixing the signature as well? this is my point
200 2015-09-07 23:40:31 <bedeho> why the discrepancy?
201 2015-09-07 23:40:46 <nwilcox> The serialization format matters for bandwidth and blockchain size purposes.
202 2015-09-07 23:40:50 <gmaxwell> Just a different seralizer. There is an efficiency impact for bits on the wire, none or effectively none in the hashed represetntation.
203 2015-09-07 23:40:51 <Luke-Jr> nwilcox: usually they're unsigned types
204 2015-09-07 23:41:10 <nwilcox> Luke-Jr: Ah, good point.
205 2015-09-07 23:41:38 <bedeho> ah ok, so on wire you want it small, so you make it 1 byte, but as hash input, you leave it as 4b because that is the norm. Get it.
206 2015-09-07 23:42:02 <nwilcox> bedeho: That's my guess, but I don't know the code history well.
207 2015-09-07 23:42:29 <gmaxwell> It's impossible to tell if it was intentional or not in this case, but the same kind of thing does happen intentionally elsewhere.
208 2015-09-07 23:42:31 <bedeho> It sort of makes sense, I gust get a feeling Im missing something when I see stuff like that, so had to ask.
209 2015-09-07 23:43:15 <nwilcox> bedeho: The thing is, no matter what rationalization the original author had, now the consensus rules are stuck with all those choices, intentional-or-not.
210 2015-09-07 23:43:23 <bedeho> :D
211 2015-09-07 23:43:26 <nwilcox> -and sometimes the unintentional, implicit behaviors come back to haunt us.
212 2015-09-07 23:43:46 <gmaxwell> That sort of thing happens by accident too... just because the explicit seralization to put it in the transaction happens to be seperate code from the code that builds up the template for the signature. It's something to be careful of because some kinds of accidental discrepency like that can result in severe security issues. (Old PGP fingerprint scheme being a lovely example)
213 2015-09-07 23:44:30 <nwilcox> I'm not familiar with the PGP fingerprint issue, but given that this bitcoin hashing stuff is fresh in my mind, I'd love to read about the PGP case.
214 2015-09-07 23:44:43 <gmaxwell> In this case it's perfectly harmless.. and already what the signature hasher covers has many differences from the transaction on the wire. (I mean, it has most of its data omitted, and pubkeys from the inputs summoned out of nowhere. :) )
215 2015-09-07 23:45:47 <nwilcox> Yeah, the signature hashing is certainly an area that makes me feel uncomfortable.  I'm still wrapping my head around it.
216 2015-09-07 23:46:25 <nwilcox> gmaxwell: Does the "witness separation" element simplify signature verification code?
217 2015-09-07 23:46:28 <gmaxwell> nwilcox: PGP's old fingerprint was basically H(exponent||modulus) of the RSA pubkey where || is concat. In particular, it did not cover the _length_ descriptor for the exponent. So you could search for an alternative pubkey where the exponent was some larger value and the modulus was some smaller one. The result immediately gives you a trivially factorable modulus, and a public key with an identical
218 2015-09-07 23:46:35 <gmaxwell> fingerprint.
219 2015-09-07 23:46:50 <nwilcox> Doh!
220 2015-09-07 23:46:53 <nwilcox> Nice.
221 2015-09-07 23:48:07 <gmaxwell> The design lesson is that you must be very careful if your hashing seralization is not a bijective encoding of the data being authenticated.
222 2015-09-07 23:48:20 <gmaxwell> And that any onto mapping is intentional and carefully considered.
223 2015-09-07 23:49:59 <gmaxwell> nwilcox: I wouldn't say that witness seg makes things simpler, though it does increase uniformity, in that more of the system uses a signatures-masked hash like the signer does. Though they're still seperate. Problem there is that the signature hasher needs more masking flexibility than you want in general.