1 2017-10-01 03:31:13 <wallet42> esotericnonsense: that's what i though it would do but it felt highly inefficient. but i guess that's the whole point why it's deprecated
  2 2017-10-01 03:33:27 <esotericnonsense> wallet42: i'm not fully up to scratch on the chainstate format (past or present); one thing I remember seeing is that it should be possible to prune spent outputs from it while keeping other outputs within the same transaction
  3 2017-10-01 03:33:56 <wallet42> yes the new format is actually quite straigtforward
  4 2017-10-01 03:33:58 <esotericnonsense> not sure if that already happens; if it does then getrawtransaction could only work for transactions that have all outputs unspent
  5 2017-10-01 03:34:18 <wallet42> yes it needs at least 1 utxo
  6 2017-10-01 03:34:36 <wallet42> getting the block height
  7 2017-10-01 03:34:47 <wallet42> then looking up in blockindex
  8 2017-10-01 03:34:50 <esotericnonsense> ah that's it. right. so it requires 1 utxo _and_ unpruned
  9 2017-10-01 03:35:17 <wallet42> yep
 10 2017-10-01 03:35:41 <esotericnonsense> (thinking about it on a theoretical level rather than how it's actually implemented, i don't know if getrawtransaction works at all on a pruned node)
 11 2017-10-01 03:37:00 <wallet42> it does, but only best effort
 12 2017-10-01 03:38:19 <wallet42> pruning is also just deleting whole blk files
 13 2017-10-01 03:38:29 <wallet42> the blockindex has pointers to those files
 14 2017-10-01 03:38:35 <wallet42> the pointers dont get removed
 15 2017-10-01 03:38:53 <wallet42> but if the data at pointer doesnt exist anymore, it fails
 16 2017-10-01 03:44:47 <esotericnonsense> of course :P
 17 2017-10-01 03:44:49 <esotericnonsense> okay
 18 2017-10-01 18:20:37 <hidden> so quiet
 19 2017-10-01 19:03:42 <djdjjj> question re. TXs: What's the reason for the actual redeemer's pubkey being included in the tx, given that it already can be found by the txin/index info? (I'm going through https://en.bitcoin.it/wiki/Transaction)
 20 2017-10-01 19:15:01 <arubi> djdjjj, you mean at signing time?  you're right.  it doesn't have to be there at all.  the prevtx's txid explicitly passes that information to the signature anyway
 21 2017-10-01 19:15:47 <djdjjj> arubi: signing time, yes. So is there any particular reason it's there?
 22 2017-10-01 19:16:25 <arubi> djdjjj, maybe if you could include input scripts into sighash, but other than that it's really not needed.  security stays the same
 23 2017-10-01 19:18:55 <djdjjj> thx arubi. Another one: when nodes verify if a tx is valid by validating the chain of ownership, how far back (in number of txs) do they go?
 24 2017-10-01 19:19:06 <djdjjj> (I'm a newbie to bitcoin btw)
 25 2017-10-01 19:20:01 <arubi> they just need to look at their known database of spendable outputs
 26 2017-10-01 19:20:23 <arubi> if it isn't there, it's invalid.  full nodes have the most updated database at all times if they're connected to the network
 27 2017-10-01 19:22:30 <djdjjj> So if I send 1BTC to Bob, full nodes have to validate that I'm the rightful owner of my referenced inputs, right?
 28 2017-10-01 19:23:01 <arubi> that's done by verifying you signatures against the inputs' scripts, right
 29 2017-10-01 19:23:05 <arubi> (brb)
 30 2017-10-01 19:23:42 <djdjjj> right. But don't they have to also verify the inputs of my referenced inputs and so on?
 31 2017-10-01 19:24:21 <djdjjj> eg. what if the referenced input of one of my inputs is invalid
 32 2017-10-01 19:24:29 <arubi> no, because you only spend the previous tx's outputs, not the whole chain
 33 2017-10-01 19:24:42 <arubi> then the input itself can't exist in the utxo set
 34 2017-10-01 19:24:50 <arubi> which is the database of spendable outputs
 35 2017-10-01 19:24:52 <djdjjj> come to think of it, this is probably not needed since this validation work of those previous inputs is already done since it's already included in a block
 36 2017-10-01 19:24:59 <arubi> :)
 37 2017-10-01 19:25:52 <djdjjj> oh so this utxo set is a global one? Every node knows all the UTXOs on the chain?
 38 2017-10-01 19:26:48 <arubi> yes, it's what all full nodes agree on
 39 2017-10-01 19:27:02 <djdjjj> makes sense now, thanks :)
 40 2017-10-01 19:27:05 <arubi> yw
 41 2017-10-01 19:27:52 <djdjjj> so there's that invariant: a UTXO's txinputs cannot be UTXO themselves
 42 2017-10-01 19:29:10 <arubi> oh sure, because they haven't been created yet when you try to spend them
 43 2017-10-01 19:30:30 <djdjjj> otherwise that would be double-spending, wouldn't it?
 44 2017-10-01 19:31:01 <arubi> not saying it can't happen, but you'd need to break sha256 to set it up
 45 2017-10-01 19:31:38 <arubi> you're trying to set up a tx that when redeeming an input has the same txid as that input.  sounds possible, but not with current tech :)
 46 2017-10-01 19:32:21 <djdjjj> ah right
 47 2017-10-01 19:34:01 <djdjjj> I didn't knew that UTXO database is part of the consensus protocol btw. I thought only the proof-of-work chain was part of it
 48 2017-10-01 19:35:17 <arubi> right the utxo is the grand thing everyone agrees on because everybody follows the same consensus rules
 49 2017-10-01 19:36:02 <djdjjj> I'm confused by these answers however that state that each node computes the UTXO set on its own https://bitcoin.stackexchange.com/questions/49393/is-the-utxo-set-computed-from-scratch-by-new-full-nodes
 50 2017-10-01 19:37:02 <arubi> each node build their own view of what the utxo set is
 51 2017-10-01 19:37:17 <arubi> but since everybody run the same rules, they will eventually agree
 52 2017-10-01 19:37:31 <arubi> they might disagree sometimes, but it converges very quickly
 53 2017-10-01 19:38:01 <djdjjj> ok I get it. So by looking at the UTXO set you can effectively compute how much BTC exists right now?
 54 2017-10-01 19:38:16 <arubi> yep
 55 2017-10-01 19:39:06 <arubi> there's a specific call in bitcoin core, 'gettxoutsetinfo' that does that
 56 2017-10-01 19:47:04 <djdjjj> re. my initial question, someone replied with a slightly different answer than yours I think: https://www.reddit.com/r/BitcoinBeginners/comments/73nmjr/technical_question_why_is_the_redeemers_pubkey_is/
 57 2017-10-01 19:47:47 <arubi> which answer do you mean?
 58 2017-10-01 19:48:30 <arubi> the address isn't playing a part in anything.  addresses are not part of the actual protocol.  the previous txid:index points to a scriptpubkey
 59 2017-10-01 19:48:52 <arubi> in case it's p2pkh (send to address), it's obvious what should be the script spending it
 60 2017-10-01 19:49:30 <arubi> at signing time, that scriptpubkey is signed, but it's redundant.  it was already hashed as part of the input's txid itself
 61 2017-10-01 19:50:04 <arubi> it already exist in sighash, there's no need to explicitly put it in there again, but now I'm remembering op_codeseparator
 62 2017-10-01 19:50:06 <djdjjj> my mistake then, I was talking about validation time, not signing time.
 63 2017-10-01 19:50:45 <arubi> so actually djdjjj if we didn't have op_codeseparator, I'd be right, but we do and for some special cases, you do need the script signed too
 64 2017-10-01 19:51:21 <arubi> validation is the same.  you know the utxo, you know the scriptpubkey required to spend it
 65 2017-10-01 19:52:13 <djdjjj> I guess my confusion comes from the fact that I thought that addresses were just the public keys
 66 2017-10-01 19:53:01 <arubi> addresses are just strings that when you tell a wallet to "send to them", it actually sends to a standard scriptpubkey that has the public key's hash instead of the public key itself
 67 2017-10-01 19:53:56 <arubi> to redeem it, in the spending transaction, you provide the public key itself, and a valid signature.  so the previous tx's scriptpubkey is executed with these two as inputs
 68 2017-10-01 19:54:04 <djdjjj> so address is a string representation of the hashed value of a public key?
 69 2017-10-01 19:54:24 <arubi> right, and some other non interesting things.  mainly the pubkey's hash
 70 2017-10-01 19:55:43 <djdjjj> so only the redeemer knows his/her own public key?
 71 2017-10-01 19:55:51 <arubi> yes
 72 2017-10-01 19:56:00 <djdjjj> the other peers only know his/her hashed keys (ie. addresses)
 73 2017-10-01 19:56:25 <arubi> right, and to prove ownership of that key, the owner has to sign the hash of the key
 74 2017-10-01 19:56:37 <arubi> which is just as hard as signing the key itself
 75 2017-10-01 19:56:44 <arubi> (or easy if you have the private key)
 76 2017-10-01 19:57:40 <arubi> that's why I initially said you could just use the previous txid, because it already contains the hash of the pubkey, but special singing conditions require the actual executed script too
 77 2017-10-01 19:57:46 <djdjjj> so p2pkh is the simpler format AFAICT, and the most common one
 78 2017-10-01 19:58:05 <arubi> it's most common yes, hopefully not for long as p2wpkh replaces it :)
 79 2017-10-01 19:58:47 <arubi> (same idea, just a more efficient script)
 80 2017-10-01 20:02:33 <djdjjj> it's clear a few paragraphs below in the wiki: "A Bitcoin address is only a hash, so the sender can't provide a full public key in scriptPubKey. When redeeming coins that have been sent to a Bitcoin address, the recipient provides both the signature and the public key. The script verifies that the provided public key does hash to the hash in scriptPubKey, and then it also checks the signature against the public key."
 81 2017-10-01 20:02:51 <djdjjj> it all makes sense now :)
 82 2017-10-01 20:02:55 <djdjjj> thanks a lot
 83 2017-10-01 20:03:00 <arubi> the paragraph is lacking really
 84 2017-10-01 20:03:22 <djdjjj> x_x
 85 2017-10-01 20:03:34 <djdjjj> in which way?
 86 2017-10-01 20:03:39 <arubi> oh sorry, it does
 87 2017-10-01 20:03:44 <arubi> "also checks the signature against the public key"
 88 2017-10-01 20:03:49 <arubi> you're welcome :)
 89 2017-10-01 20:04:26 <arubi> I missed the last "and then.." at first
 90 2017-10-01 20:06:05 <djdjjj> so the actual public keys are not visible in wallets users (they only see addresses) but are visible in full nodes
 91 2017-10-01 20:06:45 <arubi> address owners can run 'ismine <address>' and see the pubkey, but usually well done wallets try to avoid letting the user handle any keys at all
 92 2017-10-01 20:07:34 <djdjjj> and the public keys in TXs appear only in the "scriptSig" part (talking about p2pkh addresses)
 93 2017-10-01 20:07:40 <Sentineo> how did a p2pk addresslook like? was it the pubkey in base56? Or just plain?
 94 2017-10-01 20:08:49 <Sentineo> djdjjj: yes, only in scriptSig (or as I like the more descriptive name unlocking script)
 95 2017-10-01 20:08:50 <arubi> djdjjj, right, not a part of the recorded transaction at all
 96 2017-10-01 20:09:07 <arubi> Sentineo, there isn't an address for that.  it's just a pubkey and checksig after
 97 2017-10-01 20:09:37 <arubi> so sending to a pubkey would just be sending to a script "<pubkey> checksig"
 98 2017-10-01 20:09:38 <Sentineo> ok, so historicaly base58 was cr3ated for p2pkh?
 99 2017-10-01 20:09:48 <arubi> not base58 either, just hex pubkeys :)
100 2017-10-01 20:10:22 <Sentineo> are those still valid?
101 2017-10-01 20:10:29 <arubi> sure, and standard
102 2017-10-01 20:10:31 <Sentineo> need to try on regt3st ;)
103 2017-10-01 20:10:34 <arubi> multisigs too
104 2017-10-01 20:10:45 <arubi> as bare scripts that is, not p2sh wrapped
105 2017-10-01 20:11:03 <Sentineo> yeah, remember those
106 2017-10-01 20:11:11 <Sentineo> been playing with them
107 2017-10-01 20:11:34 <arubi> miners used to pay themselves with a bare 1-of-1 multisig, weird, maybe to make the software count -20 sigops and have some safety margin for sigop count
108 2017-10-01 20:11:43 <djdjjj> so bare public keys are only used to verify the signature and to create addresses?
109 2017-10-01 20:11:46 <arubi> I don't know the reason.  not sure if minres still do :)
110 2017-10-01 20:12:00 <arubi> and multisig scripts
111 2017-10-01 20:12:25 <arubi> well these are wrapped in a hash themselves, so they're not bare, but you do interact with keys for that
112 2017-10-01 20:13:20 <djdjjj> I see
113 2017-10-01 20:13:39 <djdjjj> so `scriptPubKey` is a little misleading name..
114 2017-10-01 20:13:49 <arubi> scriptsig too
115 2017-10-01 20:14:02 <djdjjj> well that's kinda vague too yes
116 2017-10-01 20:14:24 <Sentineo> yep
117 2017-10-01 20:14:30 <Sentineo> well because it was mostly cod3
118 2017-10-01 20:14:32 <arubi> it's all because when these were made up, they had a bit different uses
119 2017-10-01 20:14:32 <Sentineo> code
120 2017-10-01 20:14:49 <Sentineo> yeah
121 2017-10-01 20:14:51 <djdjjj> as a sender, `scriptSig` contains my public key and my signature, and `scriptPubKey` contains the recipients hashed public key (ie. address)
122 2017-10-01 20:14:56 <arubi> scriptsig, you used to be able to execute scripts on there.  you still can to some extent, but not very useful
123 2017-10-01 20:15:12 <arubi> scriptsig is the script that runs before the input's scriptpubkey us run
124 2017-10-01 20:15:35 <arubi> right now, scriptsig is only confined to pushes of data, so the script just pushes stuff like pubkeys and sigs to the stack
125 2017-10-01 20:15:35 <Sentineo> djdjjj: scriptSig js the unlocking script and scriptPubkey, the locking
126 2017-10-01 20:15:49 <arubi> and when the scriptpubkey runs, these are its inputs
127 2017-10-01 20:16:10 <Sentineo> djdjjj: so when you send something you lock it (givr the crypto puzzle)
128 2017-10-01 20:16:11 <arubi> so for a p2pkh, the inputs on the stack, put there by scriptsig would be the pubkey and signature
129 2017-10-01 20:17:04 <arubi> for a p2pk, it's only the signature (pubkey not recorded, but in sighash as previous scriptpubkey), for p2pkh it's both (hash not recorded, but in sighash as previous scriptpbukey)
130 2017-10-01 20:18:05 <Sentineo> djdjjj: and to spend it, you have to unlock it by providing the scriptSig.
131 2017-10-01 20:19:17 <Sentineo> djdjjj: read this and it will make more sense ... https://github.com/bitcoinbook/bitcoinbook/blob/second_edition/ch06.asciidoc
132 2017-10-01 20:20:18 <arubi> no!  read http://enetium.com/resources/Bitcoin.pdf !
133 2017-10-01 20:20:57 <djdjjj> question: the `pubKey` in `scriptSig` is *my* public key, right?
134 2017-10-01 20:21:05 <djdjjj> (not the recipients)
135 2017-10-01 20:21:52 <arubi> yes, yours
136 2017-10-01 20:21:55 <arubi> the one redeeming
137 2017-10-01 20:22:09 <arubi> the recipient's is unknown yet :)
138 2017-10-01 20:22:16 <arubi> (sending to address, right)
139 2017-10-01 20:22:39 <djdjjj> I'm kinda confused by this: https://en.bitcoin.it/wiki/Transaction#Pay-to-PubkeyHash
140 2017-10-01 20:23:08 <arubi> specifically?
141 2017-10-01 20:23:10 <djdjjj> when the OP_EQUALVERIFY comes into play, it says that it compares `pubHashA` with `pubKeyHash`
142 2017-10-01 20:23:26 <arubi> yes, and fails the script if it isn't equal
143 2017-10-01 20:23:27 <djdjjj> why would those be equal? Isn't `pubHashA` _my_ hashed public key?
144 2017-10-01 20:23:45 <djdjjj> and `pubKeyHash` the address of the recipient?
145 2017-10-01 20:23:49 <arubi> yes, it's compared to the hash in the input's scriptpubkey txid:index
146 2017-10-01 20:24:00 <arubi> no no, that pubkeyhash is in the outputs
147 2017-10-01 20:24:36 <arubi> things in scriptsig are input stuff, either scripts at redemption or previous scriptpubkey at signing time
148 2017-10-01 20:25:02 <arubi> look at page 29 in the pdf I linked
149 2017-10-01 20:25:23 <djdjjj> ah sorry I got it
150 2017-10-01 20:25:30 <arubi> oh alright
151 2017-10-01 20:25:32 <djdjjj> this is the whole process that checks that I'm the rightful owner
152 2017-10-01 20:25:32 <Sentineo> djdjjj: you have to concatenate scriptSig and scriptPubkey and lookmat it as one script
153 2017-10-01 20:25:40 <djdjjj> if the coins I'm trying to spend
154 2017-10-01 20:25:49 <Sentineo> djdjjj: if it returns true you transer bitckin ownership
155 2017-10-01 20:26:00 <arubi> ownership is checked by making you sign both the previous txid:index and the script being executed
156 2017-10-01 20:26:31 <djdjjj> I mean, that whole "checking process" described in that page refers only to "checking that I'm the owner of the UTXO's I'm trying to spend"
157 2017-10-01 20:26:53 <arubi> yes, that's what validation of transactions does basically
158 2017-10-01 20:26:54 <djdjjj> I thought that it was validating the whole transaction somehow (meaning involving the sender too)..
159 2017-10-01 20:26:59 <djdjjj> yes yes
160 2017-10-01 20:28:09 <djdjjj> so that `pubKeyHash` there actually refers to the `pubKeyHash` of a _UTXO_ that I'm redeeming, not the `pubKeyHash` that I'll send the coins to
161 2017-10-01 20:29:25 <arubi> yes that's right
162 2017-10-01 20:29:53 <djdjjj> that PDF is awesome btw
163 2017-10-01 20:29:55 <arubi> usually signers sign all the outputs in a transaction, but they always only sign their own input
164 2017-10-01 20:30:24 <arubi> they do sign other txid:index inputs though, just not scriptpubkeys
165 2017-10-01 20:30:42 <arubi> or sequences, you'll see :)
166 2017-10-01 20:54:58 <djdjjj> last one: which data are signed in the scriptSig signature part?
167 2017-10-01 20:55:30 <djdjjj> I guess it's a hash of the tx inputs themselves?
168 2017-10-01 20:56:37 <djdjjj> or the tx-to-be-created?
169 2017-10-01 20:58:48 <arubi> it's everything, prevtxs and current outputs
170 2017-10-01 20:59:29 <arubi> anything that isn't signed is free to change in flight by anyone, so it's pretty thorough if you don't use very exotic setups