1 2017-08-31 01:55:23 <nreefode> hi all. I am writing an RPC to get up to speed with bitcoin-core. At some point I would like to be generating an empty (but valid) new block. CreateNewBlock pulls things out of the mempool, yes?
  2 2017-08-31 01:57:10 <nreefode> I'm thinking I can add a flag to CreateNewBlock so that it does not call addPackageTxs, which it looks like is what's grabbing txs from the mempool
  3 2017-08-31 02:01:21 <nreefode> oh you know what, the docs for addPackageTxs explicitly say "we don't remove transactions from the mempool as we select them for block inclusion"
  4 2017-08-31 04:17:42 <esotericnonsense> hm.
  5 2017-08-31 04:18:06 <esotericnonsense> my 'random source' has given me a bip39 seed with three subsequent words identical.
  6 2017-08-31 04:18:30 <esotericnonsense> :>
  7 2017-08-31 15:02:30 <esotericnonsense> after multiple hours of trying to figure out why my bip32 derivation code is not working properly against the test vectors i decide to use a different base58 encoder
  8 2017-08-31 15:02:39 <esotericnonsense> it was always fine
  9 2017-08-31 16:24:43 <dviola> hi
 10 2017-08-31 16:25:12 <dviola> I see my initial sync is going a bit slow today, I'm getting connection refused and such
 11 2017-08-31 16:25:32 <dviola> 2017-08-31 16:23:35 Peer=5 is stalling block download, disconnecting
 12 2017-08-31 16:25:34 <dviola> 2017-08-31 16:23:35 connect() to [2001:0:5ef5:79fb:14be:388:93d8:4eaf]:8333 failed: Network is unreachable (101)
 13 2017-08-31 16:25:47 <dviola> in the debug.log
 14 2017-08-31 16:25:58 <dviola> is it possible my ISP is messing with me?
 15 2017-08-31 16:30:16 <dviola> how do I know if my node is able to connect to other nodes?
 16 2017-08-31 16:33:28 <dviola> I wonder if that time out is simply because my node can't be reached from outside
 17 2017-08-31 17:05:52 <dviola> looks like that was a connection to ipv6 and I can't use ipv6
 18 2017-08-31 18:29:40 <asimplecoder> hey
 19 2017-08-31 18:29:44 <asimplecoder> anyone online?
 20 2017-08-31 18:30:46 <asimplecoder> i need some help with bitcoind
 21 2017-08-31 18:32:56 <adiabat> asimplecoder: people are online, just ask, preambles etc are generally ignored here
 22 2017-08-31 18:34:16 <asimplecoder> i cant be able to figure out how to use bitcoind's sendrawtransaction
 23 2017-08-31 18:34:18 <asimplecoder> bitcoin-cli sendrawtransaction mysignedtransactioninhex allowhighfees=false error: Error parsing JSON:allowhighfees=false
 24 2017-08-31 18:35:56 <adiabat> I think if you want allowhighfees, just put a true at the end?
 25 2017-08-31 18:36:26 <esotericnonsense> looking at bip32 - am I correct in understanding that a hardened public child key cannot be obtained from any public parent key or set of
 26 2017-08-31 18:37:08 <esotericnonsense> that is to say that it can only be derived by knowing some private parent key, determining the hardened private child key and then obtaining the point
 27 2017-08-31 18:37:39 <adiabat> asimplecoder: as in, don't say allowhighfees=false, just put false after the hex string
 28 2017-08-31 18:37:48 <asimplecoder> if i put true at the end i get:
 29 2017-08-31 18:37:49 <asimplecoder> error code: -25 error message: Missing inputs
 30 2017-08-31 18:38:05 <asimplecoder> same as with false
 31 2017-08-31 18:38:17 <adiabat> asimplecoder: same? that's a different error
 32 2017-08-31 18:38:32 <adiabat> try decoderawtransaction to see what's wrong with the tx
 33 2017-08-31 18:38:43 <esotericnonsense> e.g. m/0'/0/0' pub _cannot_ be obtained from m/0'/0 pub nor m/0' pub nor m pub
 34 2017-08-31 18:38:52 <esotericnonsense> you need any of those as prv
 35 2017-08-31 18:39:23 <esotericnonsense> (this logically follows from bip32 CKD's, just confirming that I haven't missed something)
 36 2017-08-31 18:40:23 <asimplecoder> its nothing wrong with the transaction
 37 2017-08-31 18:42:35 <asimplecoder> http://dpaste.com/2T4S9N4.txt
 38 2017-08-31 18:42:40 <asimplecoder> check it out^
 39 2017-08-31 18:44:56 <adiabat> the missing inputs error means the inputs your trying to spend aren't found
 40 2017-08-31 18:45:19 <adiabat> so the tx in isolation may be well formed, but the txo it's spending is gone
 41 2017-08-31 18:45:46 <asimplecoder> so it might have been gone throw?
 42 2017-08-31 18:46:30 <adiabat> the vin:txid:0 is probably not a utxo, you can query that with getrawtransaction
 43 2017-08-31 18:46:37 <adiabat> (if you have txindex=1)
 44 2017-08-31 18:46:41 <arubi> esotericnonsense, right, leafs on the same branch aren't related at all in any way
 45 2017-08-31 18:47:12 <asimplecoder> no it got the bitcoin on it
 46 2017-08-31 18:47:44 <arubi> and specifically hardened paths can not be derived from non hardened paths
 47 2017-08-31 18:47:51 <adiabat> asimplecoder: as a guess, bitcoin txids are "backwards", in that if you're writing your own stuff, the bytes have to be reversed
 48 2017-08-31 18:48:26 <asimplecoder> yupp its a custom script
 49 2017-08-31 18:48:45 <arubi> you can only go forward if you know the private key at all, and can only go backwards with a private key and a non hardened xpub
 50 2017-08-31 18:49:07 <adiabat> the txid in the hex string going into decoderawtransaction is backwards from the hex string you'll see from the json output
 51 2017-08-31 18:49:12 <esotericnonsense> arubi: well you can go forward from pub to pub with pub, just not to a hardened child
 52 2017-08-31 18:49:24 <asimplecoder> i tried to push and i got the error "Validation Error: Insufficient fee. Minimum fee is 1 sat/B."
 53 2017-08-31 18:49:30 <arubi> yea, that's right.  on non hardened paths
 54 2017-08-31 18:50:09 <arubi> sorry, can only go forwards to a hardened path*
 55 2017-08-31 18:50:10 <adiabat> asimplecoder: reduce your output value a bit to up the fee
 56 2017-08-31 18:50:12 <esotericnonsense> i'm unsure what you mean by on non hardened paths. it seems that i could go from pub m/0'/0 to pub m/0'/0/0 just fine.
 57 2017-08-31 18:50:15 <esotericnonsense> yes, indeed
 58 2017-08-31 18:50:29 <esotericnonsense> can only go forwards to a _non_hardened path :P
 59 2017-08-31 18:50:42 <arubi> right, or a hardened path with knowing the xpub
 60 2017-08-31 18:50:47 <arubi> errr
 61 2017-08-31 18:50:48 <arubi> expriv
 62 2017-08-31 18:51:08 <asimplecoder> whaaat "Parse: exception decoding Hex string: String index out of range: 381"
 63 2017-08-31 18:51:55 <arubi> esotericnonsense, sorry, I'm only paying 20% attention, I'm confusing more than helping :)  I think you got it
 64 2017-08-31 18:52:07 <esotericnonsense> lol
 65 2017-08-31 18:52:35 <arubi> try to look into the private key + parent xpub exploit, it's pretty cool
 66 2017-08-31 18:53:38 <arubi> as when a private key on a non hardened path is revealed, then you can derive parent xrpiv's until the non hardened path ends.  I think it's a cool way of applying bip32 stuff :)
 67 2017-08-31 18:56:46 <esotericnonsense> yeah, i've looked at that
 68 2017-08-31 18:58:25 <esotericnonsense> it seems interesting in that you can create a wallet that uses only hardened children, generating say the first 100 pub/priv when the parent is available, then removing the parent (or locking it behind a more secure encryption passphrase than the children)
 69 2017-08-31 18:58:48 <esotericnonsense> you would only need to unlock the parent to generate more pub/priv
 70 2017-08-31 19:00:17 <esotericnonsense> i suppose that actually you could just keep the parent encrypted (both pub and prv) and use non-hardened
 71 2017-08-31 19:01:27 <esotericnonsense> basically it's like having a non-HD wallet with a keypool, but the keypool can be refreshed deterministically, then again it would be easier to just use different parent paths for this sort of thing
 72 2017-08-31 19:02:46 <asimplecoder> fucking hell error code: -25 error message: Missing inputs
 73 2017-08-31 19:09:54 <arubi> esotericnonsense, that's how core does it
 74 2017-08-31 19:10:18 <arubi> it's hardened hd only, and you have to unlock the wallet when the keypool is depleted to generate more
 75 2017-08-31 19:11:06 <arubi> really you can just generate a huge amount the first time and keep encrypted forever, but it gets pretty bit quickly with private keys wrapped in secp256k1 asn.1 for each key :P
 76 2017-08-31 19:12:01 <arubi> well maybe not quickly, iirc it's hundreds of megs at a million keys, which is actually 2 million because of the internal change
 77 2017-08-31 19:14:37 <arubi> electrum uses non hardened derivation, different from core.  really most wallets use bip44 or now starting to use bip49 which are both public derivation
 78 2017-08-31 19:17:12 <esotericnonsense> yes, i'm looking in to bip44/49 now. wanted to cover bip32 first to understand how it works. have reimplemented bip32 in python, passing the vectors
 79 2017-08-31 19:18:09 <arubi> awesome.  the newest added vector is due to a really nasty bugs some implementations had with a leading zero byte for private keys in the extended encoded, be sure to pass that ;)
 80 2017-08-31 19:21:03 <esotericnonsense> yep, all checked out first time :)
 81 2017-08-31 19:21:10 <esotericnonsense> (well, once I got any of them working)
 82 2017-08-31 19:26:02 <arubi> what are you using for ecc esotericnonsense ?
 83 2017-08-31 19:26:40 <esotericnonsense> python-ecdsa, which seems quite slow. but this is kind of a toy for now.
 84 2017-08-31 19:27:38 <esotericnonsense> (it is also slow because i don't optimize, I forget all previous results each time).
 85 2017-08-31 19:29:07 <arubi> cool, slow or not, if you're using it for exploring stuff, then the interesting bits are mostly interactive anyway :)
 86 2017-08-31 19:29:42 <esotericnonsense> i did write the bits of ecdsa I needed for doing this years back but i've lost the code and it was horrendous anyway
 87 2017-08-31 19:30:25 <arubi> heh yea, I wrote my own too.  still using it.  it's a lot slower than anything in python ever will be
 88 2017-08-31 19:30:43 <arubi> mine is in gnu bc
 89 2017-08-31 19:30:48 <esotericnonsense> lol
 90 2017-08-31 19:30:50 <esotericnonsense> ???
 91 2017-08-31 19:30:52 <esotericnonsense> :D
 92 2017-08-31 19:31:10 <esotericnonsense> i could never get on with bc. i tried. always just use python interpreter whenever i need to do quick math stuff.
 93 2017-08-31 19:31:28 <arubi> https://github.com/fivepiece/btc-bash-ng  and bash to tie stuff together :)
 94 2017-08-31 19:31:55 <esotericnonsense> awesome
 95 2017-08-31 19:32:31 <arubi> cheers, if you need some exotic math stuff, I might have it.  if I don't, let me know, I love to implement cool stuff
 96 2017-08-31 19:33:05 <arubi> it's not always 100% correct, but I try :P
 97 2017-08-31 19:33:15 <esotericnonsense> i'm mostly doing this to aid my understanding of how wallets work and what a 'good' wallet storage format should look like
 98 2017-08-31 19:33:34 <arubi> I guess I do too
 99 2017-08-31 19:33:35 <esotericnonsense> after having a fun time trying to extract funds from a bitcoinj wallet, compiling tons of java shit just to get the keys out
100 2017-08-31 19:33:43 <arubi> wait I have something you'd like
101 2017-08-31 19:34:15 <arubi> https://gist.github.com/fivepiece/74b6f8bfa8816ee7e283d25a36947629 , core's wallet with keypool=0
102 2017-08-31 19:35:10 <arubi> I kinda want to dissect the wallet further, my real wish is it being able to use a 64 bytes value as the bip32 seed
103 2017-08-31 19:35:24 <arubi> as the input to the bip32 master xpriv rather
104 2017-08-31 19:36:01 <arubi> currently it's using one valid secp256k1 private key as the 32 bytes to the [hmac-sha512 "value" "Bitcoin seed"] thing
105 2017-08-31 19:36:49 <arubi> if it could be a 64 bytes value, then it could also be an output off a bip39 kdf, and I'd be able to get my core's private key on a hardware wallet! :D
106 2017-08-31 19:40:38 <arubi> brb, sorry, I try to drop this info onto anyone who has an interest in improving the wallet scheme.  I think the current "hey just change the constant" method is difficult.  already folks are asking wallet providers to be able to sign with a "segwit address"
107 2017-08-31 19:41:34 <arubi> what I mean is, this needs fixing.  it's getting to the point it's very hard to maintain..
108 2017-08-31 21:25:42 <esotericnonsense> lessons in cryptography being full of footguns #229384583
109 2017-08-31 21:26:01 <esotericnonsense> https://github.com/petertodd/python-bitcoinlib/blob/master/bitcoin/core/key.py WARNING: This module does not mlock() secrets; your private keys may end up on disk in swap! Use with caution!
110 2017-08-31 21:26:16 <esotericnonsense> such an obvious possibility that I'd never even considered before
111 2017-08-31 21:29:21 <esotericnonsense> i think at least core and electrum now offer the ability to encrypt on boot (otherwise you end up with an unencrypted copy hitting the disk on first startup)
112 2017-08-31 21:29:43 <esotericnonsense> but that one seems to make securing electrum impossible unless there some magic c extension stuff