1 2016-03-15 00:22:41 <belcher> is there an accepted method for SPV/thin-wallets to have access to the estimatefee method from core? im thinking about writing a wallet that sync's with bip37 bloom filters only to your own node running on your own hardware, but im not sure how such a wallet would estimatefee
  2 2016-03-15 00:45:05 <belcher> i wonder if in the long term, a new method can be added to the p2p protocol that exposed estimatefee
  3 2016-03-15 00:50:23 <wallet42> belcher: the problem here really is how to trust other nodes.
  4 2016-03-15 00:50:42 <belcher> of course, miners especially have an incentive to lie
  5 2016-03-15 00:50:54 <belcher> but my specific situation, ill only connect to my own node on my own hardware
  6 2016-03-15 00:51:01 <belcher> for full privacy, security, trustlessness and so on
  7 2016-03-15 02:16:12 <Luke-Jr> belcher: some authenticated protocol would be desirable for that kind of use case. right now, it basically requires Tor :/
  8 2016-03-15 02:16:36 <belcher> Luke-Jr or ssh tunnel or simply routing in your own home network
  9 2016-03-15 02:17:12 <belcher> bitcoin 0.12 automatically creates a .onion so its even more convenient
 10 2016-03-15 02:17:18 <Luke-Jr> I assume my home network is insecure. :p
 11 2016-03-15 02:17:32 <Luke-Jr> only if Tor is running ;)
 12 2016-03-15 02:17:35 <belcher> really? ok i guess so
 13 2016-03-15 02:17:43 <belcher> well even so, theres no estimatefee
 14 2016-03-15 02:17:54 <belcher> you have to query a web api i think
 15 2016-03-15 02:18:30 <Luke-Jr> you could tunnel the REST or JSON-RPC API
 16 2016-03-15 02:18:58 <belcher> yes
 17 2016-03-15 02:19:11 <belcher> someone said so on #bitcoin, i suppose thats what ill do
 18 2016-03-15 02:19:27 <belcher> its slightly less convieniant setting it all up, but not much can be done about that :)
 19 2016-03-15 13:39:47 <dgenr8> tor is anonymous, not authenticated
 20 2016-03-15 13:43:02 <dgenr8> a rough but trustable measure of fees/kb would be (coinbase amount - block reward) / blocksize in KB.  1000 KB is not a bad estimate for blocksize nowadays.
 21 2016-03-15 13:44:48 <instagibbs> miners may stuff the ballot, so to speak, and would be undetectable
 22 2016-03-15 13:44:49 <helo> can't miners artificially boost coinbase amount by creating (but not broadcasting) transactions that pay large fees?
 23 2016-03-15 13:47:20 <dgenr8> that's too risky for a miner unless they have a really big %
 24 2016-03-15 14:07:01 <instagibbs> dgenr8, you mean for reorgs?
 25 2016-03-15 14:07:14 <instagibbs> s/reorgs/stales ?
 26 2016-03-15 14:11:34 <Chris_Stewart_5> What are the r,s values of an empty ECDSA signature? 0,0?
 27 2016-03-15 14:13:14 <dgenr8> instagibbs: i just mean miners can only pay themselves in their own blocks, much more difficult than a node just spitting out false fee estimates.
 28 2016-03-15 14:13:51 <instagibbs> dgenr8, no argument it's a "better" estimate than randomly querying what appear to be nodes
 29 2016-03-15 14:14:38 <dgenr8> it does appear that today, either method seems pretty good
 30 2016-03-15 14:14:51 <instagibbs> might change if we encourage such behavior
 31 2016-03-15 14:17:30 <dgenr8> and there's always the threat of miners moving fees OOB
 32 2016-03-15 19:32:13 <Chris_Stewart_5> This test case includes a empty pubkey checking this very long signature, correct?
 33 2016-03-15 19:32:15 <Chris_Stewart_5> ["0x4a 0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "0 CHECKSIG NOT", "", "Overly long signature is correctly encoded"]
 34 2016-03-15 19:32:23 <Chris_Stewart_5> it's from script_valid.json
 35 2016-03-15 19:57:53 <waxwing> Chris_Stewart_5: 0,0 is not a valid signature fwiw
 36 2016-03-15 20:02:32 <Chris_Stewart_5> waxwing: Yeah I figured that out, with the next problem a Bitcoinj ECKey cannot be created from an empty byte array (from OP_0)
 37 2016-03-15 20:03:49 <Chris_Stewart_5> actually bouncy castle cannot decode the EC Point which makes sense if they are just trying to decode an empty byte array... not sure how Bitcoin core handles elliptic curve keys that are created from empty byte arrays
 38 2016-03-15 20:06:45 <waxwing> i know nothing about Core, but i guess that line in script_valid.json is just testing the first part of the encoding, i.e. the length byte at the start
 39 2016-03-15 20:12:51 <Chris_Stewart_5> Thanks for the help waxwing
 40 2016-03-15 20:21:15 <Chris_Stewart_5> waxwing: That 0x000.....0000 is equivalent to 0x00 right?
 41 2016-03-15 20:21:36 <kadoban> No, it's a stack item with however many zero bytes in it
 42 2016-03-15 20:22:43 <Chris_Stewart_5> kadoban: I'm referring to when it is evaluated from the OP_CHECKSIG
 43 2016-03-15 20:23:31 <Chris_Stewart_5> at that point, it is a digital signature isn't it?
 44 2016-03-15 20:23:34 <waxwing> i guess OP_CHECKSIG reads it expecting a DER encoded signature. which of course that one isn't :)
 45 2016-03-15 20:24:28 <kadoban> Chris_Stewart_5: Yeah I'm not really sure what OP_CHECKSIG is actually expecting or … what it's supposed to do on nonsense.
 46 2016-03-15 20:25:15 <Chris_Stewart_5> waxwing: Ok thats what I thought but i'm not super familiar with DER encoding but from what i read I didn't think that was valid. I'm wondering if Bitcoin Core just automatically marks the tx as invalid as soon as it sees that the signature isn't a DER encoded
 47 2016-03-15 20:25:18 <waxwing> all the others are (30) (signature?) , (length),(02) <- means integer iirc etc.
 48 2016-03-15 20:25:50 <Chris_Stewart_5> instead of failing at runtime, which bitcoinj does
 49 2016-03-15 20:26:43 <waxwing> failing at runtime? oh. sounds exciting :)
 50 2016-03-15 20:27:11 <Chris_Stewart_5> specifially   at org.bitcoinj.core.ECKey$ECDSASignature.decodeFromDER(ECKey.java:593)
 51 2016-03-15 20:27:18 <Chris_Stewart_5> which make sense now i guess
 52 2016-03-15 20:27:47 <kadoban> Chris_Stewart_5: Well you said this was from the valid test-cases right? So this script is supposed to succeed (get marked as valid)
 53 2016-03-15 20:28:01 <Chris_Stewart_5> kadoban: the script has a OP_NOT at the end
 54 2016-03-15 20:28:26 <Chris_Stewart_5> So I think i'm conflating failing at runtime and the script failing (which is expected)
 55 2016-03-15 20:28:44 <kadoban> Chris_Stewart_5: Yes, but that means the script is continuing to that point, which is different from "automatically marks the tx as invalid", which is more like what OP_RETURN does (nothing after that can make it succeed)
 56 2016-03-15 20:30:05 <arubi> I think the purpose of this being in -valid is deprecated?  it's also in "script_invalid.json"
 57 2016-03-15 20:30:18 <arubi> '"DERSIG", "Overly long signature is incorrectly encoded for DERSIG"'
 58 2016-03-15 20:30:32 <arubi> I mean, I guess it was valid until some point..
 59 2016-03-15 20:30:44 <kadoban> Err, it's in both? That doesn't seem to make much sense. There must be context of some kind?
 60 2016-03-15 20:30:57 <arubi> pre strict-der?
 61 2016-03-15 20:30:59 <Chris_Stewart_5> arubi: Can you link the line on github?
 62 2016-03-15 20:31:08 <arubi> I mean, it should verify until some point, right?
 63 2016-03-15 20:31:24 <arubi> Chris_Stewart_5, src/test/data/script_invalid.json:526
 64 2016-03-15 20:31:26 <waxwing> my guess is that Core doesn't want that kind of thing to be valid since it's a vector for stuffing up the blockchain? *warning no idea..." :)
 65 2016-03-15 20:31:50 <arubi> Chris_Stewart_5, well, at least on my pull it is.  it's pretty new, should still be there
 66 2016-03-15 20:33:52 <Chris_Stewart_5> online for anyone that wants to see
 67 2016-03-15 20:33:54 <Chris_Stewart_5> https://github.com/bitcoin/bitcoin/blob/master/src/test/data/script_invalid.json#L550
 68 2016-03-15 20:35:01 <Chris_Stewart_5> arubi: What do you mean by 'being -valid is deprecated'?
 69 2016-03-15 20:35:46 <Chris_Stewart_5> and was there a recent BIP that enforces strict der encoding?
 70 2016-03-15 20:36:04 <waxwing> Chris_Stewart_5: yeah but that was about enforcing low-S values
 71 2016-03-15 20:36:15 <waxwing> because r,-s works as well as r, s
 72 2016-03-15 20:36:23 <arubi> Chris_Stewart_5, I'm mostly guessing here,  I think these type of signatures (garbage) used to be valid until bip 66 was enforced
 73 2016-03-15 20:36:48 <waxwing> hmm maybe yeah .. i think low S was bip62 and 66 covered some more things
 74 2016-03-15 20:36:55 <arubi> right
 75 2016-03-15 20:37:13 <Chris_Stewart_5> i thought BIP 66 was sipa's attempt to remove tx malleability that failed.. am i thinking of another bip?
 76 2016-03-15 20:37:24 <waxwing> Chris_Stewart_5: yes that's right. same issue.
 77 2016-03-15 20:38:13 <arubi> there's only one (replay-able) form to a signature now, that /is/ helping with malleability somewhat
 78 2016-03-15 20:38:14 <waxwing> i think arubi's right in as much as a total length of 73 is enforced. but wasn't bip66 dropped or something?
 79 2016-03-15 20:38:33 <Chris_Stewart_5> i think bip66 was dropped
 80 2016-03-15 20:38:41 <arubi> ..? really?  that's news to me
 81 2016-03-15 20:38:45 <Chris_Stewart_5> I think most of the work was surpassed by segwit
 82 2016-03-15 20:38:53 <Chris_Stewart_5> or fixed by segwit
 83 2016-03-15 20:38:59 <waxwing> basically the encoding is 30 (length of the rest) 02 (length of r) [00] r 02 (length of s) s
 84 2016-03-15 20:39:13 <waxwing> the [00] is if the value is > 80 hex
 85 2016-03-15 20:39:19 <waxwing> oh i forgot the [00] for the s
 86 2016-03-15 20:39:28 <Chris_Stewart_5> basically this right? https://crypto.stackexchange.com/questions/1795/how-can-i-convert-a-der-ecdsa-signature-to-asn-1
 87 2016-03-15 20:39:34 <waxwing> basically the encoding is 30 (length of the rest) 02 (length of r) [00] r 02 (length of s)[00] s
 88 2016-03-15 20:40:13 <waxwing> Chris_Stewart_5: yes except for the extra 00 if the first byte of r/s is > 80 hex
 89 2016-03-15 20:41:32 <waxwing> meh it's here anyway: https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki#der-encoding
 90 2016-03-15 20:41:41 <waxwing> forgot sighash type at end :)
 91 2016-03-15 20:42:02 <arubi> oh, so bip 62 is withdrawn, not 66, right?
 92 2016-03-15 20:42:29 <arubi> yea, pull request is merged and everything
 93 2016-03-15 20:43:22 <Chris_Stewart_5> had them backwards I guess :-)
 94 2016-03-15 20:44:41 <waxwing> oh i see. yeah same rules are in 66.
 95 2016-03-15 20:45:14 <Chris_Stewart_5> so basically the script_valid.json file is out of date? Is that what I can conclude?
 96 2016-03-15 20:46:43 <arubi> Chris_Stewart_5, I'll rephrase.  I think it still should be there, as the chains must have transactions pre bip66 that act like this "checksig not" script
 97 2016-03-15 20:47:30 <arubi> so, it should be there "but warn" pre bip66 block validation, and both be in _invalid.json for post bip66 block validation, but this time error, and not warn
 98 2016-03-15 20:47:52 <arubi> note, I'm still mostly guessing
 99 2016-03-15 20:48:10 <arubi> s/chains/chain
100 2016-03-15 20:50:24 <Chris_Stewart_5> arubi: I think I understand what you are saying, how is this signature encoded then? Is this called like an non-strict DER encoded signature? Or just a generic ECDSA signature?
101 2016-03-15 20:50:43 <Chris_Stewart_5> because what you said about validation pre and post BIP66 makes sense
102 2016-03-15 20:51:20 <arubi> Chris_Stewart_5, I'd try looking at how an older version of openssl would parse it
103 2016-03-15 20:52:07 <Chris_Stewart_5> arubi: Ok, thanks so much for the help! I appreciate it
104 2016-03-15 20:52:42 <arubi> I think that's what was used in core back then?  I'm not too sure..  I don't know how this would be sanely parsed :(   welcome Chris_Stewart_5, maybe someone more knowledgeable will see your question and correct some stuff I said
105 2016-03-15 21:36:24 <cnnx> Luke-Jr: i guess you would be here too then :)