1 2015-03-30 00:38:31 <phantomcircuit> Luke-Jr, it seems like everything in the transaction object outside of the transaction data itself and the required flag can be calculated by the client with little to no effort
  2 2015-03-30 00:38:52 <phantomcircuit> are those fields provided just to make it easier to implement a minimal client?
  3 2015-03-30 00:52:09 <dgenr8> sipa: you need to check the blockchain inclusion status of every coin you receive (and that of its ancestors) by watching confirmations
  4 2015-03-30 01:05:18 <sipa> dgenr8: right, but you only need to look at confirmations at the tip transaction
  5 2015-03-30 01:06:26 <sipa> dgenr8: if you have transactions that can expire, you need much more complex wallet logic to detect any transacrions in the history of the one paying you which were included close to their expiration
  6 2015-03-30 01:06:48 <sipa> as those are effectivepy somewhat more risky to accept
  7 2015-03-30 01:07:52 <sipa> hi d_t
  8 2015-03-30 01:20:55 <dgenr8> sipa: it's not clear to me that I need watch a regular transaction's parents any less.  a parent that was confirmed one block ago could be double-spend in a 2-block reorg, making the one that pays me unconfirmable, even if not directly double-spent
  9 2015-03-30 01:26:12 <dgenr8> sipa: since we already have to worry about an adversarial scenario, the accidental one may not add any more risk
 10 2015-03-30 01:32:23 <sipa> dgenr8: that's a fair comment
 11 2015-03-30 01:33:09 <sipa> though it may better in cases where you already trust the sender of the transaction through external means
 12 2015-03-30 01:34:00 <sipa> also, having transactions that expire does complicate some of the mempool maintenaince and block construction code
 13 2015-03-30 01:34:54 <Luke-Jr> phantomcircuit: it isn't possible to determine sigops at least
 14 2015-03-30 01:35:22 <phantomcircuit> Luke-Jr, ? cant the client just count the number of sig ops in the script?
 15 2015-03-30 01:35:33 <phantomcircuit> or is that non trivial
 16 2015-03-30 01:35:37 <phantomcircuit> i honestly cant remember
 17 2015-03-30 01:35:43 <Luke-Jr> phantomcircuit: p2sh
 18 2015-03-30 01:36:04 <phantomcircuit> dont the p2sh sig ops count against the spending transaction?
 19 2015-03-30 01:36:09 <Luke-Jr> exactly
 20 2015-03-30 01:36:18 <phantomcircuit> oh right
 21 2015-03-30 01:36:22 <phantomcircuit> hmm
 22 2015-03-30 01:36:46 <Luke-Jr> in theory, you have the scriptSig with the p2sh script in the data, but you have no way to know if it's p2sh in the first place
 23 2015-03-30 01:36:58 <phantomcircuit> so the only way for the client to count p2sh sigops is if they have the outpoint also
 24 2015-03-30 01:46:59 <dgenr8> more RBF travesty tales https://gist.github.com/aalness/a78e3e35b90f52140f0d
 25 2015-03-30 02:17:15 <gmaxwell> dgenr8: uh thats not what that is.
 26 2015-03-30 02:19:04 <gmaxwell> (AFAIK there is no actual non-trivial deployment of RBF on the network currently in any case; though you do have no shortage of people with serious misunderstandings of the non-synchronicity of the network confusing earnest races for it...)
 27 2015-03-30 02:21:07 <gmaxwell> And the same sequence described there (which is just a transaction pattern not a specific attack) works just as well with concurrent announcement of t1 and t2 to orthorgonal fractions of the network.
 28 2015-03-30 02:26:57 <petertodd> gmaxwell: there's ~8 RBF nodes running on the network right now, and another ~15 doublespend-relaying nodes - the latter are just as useful to doublespend as the former modulo miner deployment
 29 2015-03-30 02:27:53 <petertodd> dgenr8: that "travesty" happens every single damn time we change the IsStandard() rules you know... every major release, and most minor releases, open up at least one new way of doublespending people
 30 2015-03-30 02:32:36 <gmaxwell> petertodd: or forget changing them, just any time users twiddle the defaults or run software from different vendors that have slightly different behavior.
 31 2015-03-30 02:32:58 <petertodd> Luke-Jr: specifically, why I think SE implemented via SIGHASH_ANYONECANPAY rather than CPFP is the sane way to opt-into it (or something more clever w/ a soft-fork) - normal wallets doublespend pretty frequently by accident anyway
 32 2015-03-30 02:33:52 <petertodd> gmaxwell: yup, Luke-Jr's gentoo thing probably has exponential complexity there, and that's before they even twiddle things like max datacarrier size
 33 2015-03-30 02:36:21 <petertodd> gmaxwell: speaking of, seems that coinbase is the only non-trivial single unconfirmed tx acceptor out there, as they will accept tx's instantly if you have "convert to fiat instantly" checked in your merchant setup. Mycelium Local Trader too, although they said they'd change the tx confidence language to make it clear that just mean the tx will go through if the sender doesn't actively try to doublespend
 34 2015-03-30 04:28:29 <phantomcircuit> petertodd, in theory in coinbases case they could pull the funds back and thus not lose money
 35 2015-03-30 04:28:35 <phantomcircuit> but i guess they're not?
 36 2015-03-30 04:31:29 <dgenr8> i wandered into Nakamoto's (processor is snapcard) and bought a bitgive t-shirt, 0-conf
 37 2015-03-30 04:39:29 <phantomcircuit> dgenr8, that sounds like a calculated risk
 38 2015-03-30 04:41:14 <dgenr8> research i guess. conclusion: getting double-spent is the least of their worries. i think she had the founder of snapcard on SMS giving tech support just to get the tx done
 39 2015-03-30 04:43:13 <dgenr8> RBF does improve the experience for the attacker.  t1 and t2 can be sent before, then he can hang around leisurely after paying -- no software i know of will alert when the conflict is confirmed
 40 2015-03-30 04:47:06 <gmaxwell> dgenr8: thats true without RBF. T1 and T2 are sent before. T3 is at will.
 41 2015-03-30 04:48:03 <dgenr8> but without RBF, t2 doesn't replace t1 ...
 42 2015-03-30 04:49:21 <gmaxwell> it doesn't have to, T1 and T2 are sent concurrently and some fraction (roughly whatever fraction the attacker chooses) of the nodes will have heard T2 first.
 43 2015-03-30 04:56:23 <dgenr8> RBF makes it more predictable though as you say, the fraction of RBF nodes is small.  i imagine aalness might worry about a scenario where lots of miners run RBF and he'll need some kind of custom relaying
 44 2015-03-30 05:03:44 <gmaxwell> dgenr8: it's completely predictable with what I described, your success rate will be very close to 100% so long as the relay delay between nodes is larger than your network jitter (which it is as relay delay is 10s of ms plus latency, which is tens of ms; typical jitters are a few ms).  And you can pick an arbritary set of nodes, rather than being constrained to ones running particular software.
 45 2015-03-30 05:04:22 <gmaxwell> And this already exists today, has existed for all time, and will always exist because it is _fundimental_. There is no universal view of ordering absent the blockchain.
 46 2015-03-30 05:22:41 <dgenr8> bummer. I was just about to suggest that the first tx should always get mined, and what's all the fuss about. do have a good night.
 47 2015-03-30 08:21:51 <jonasschnelli> wumpus, i think https://github.com/bitcoin/bitcoin/pull/5898 is okay. Travis failure is a github DDOS issue.
 48 2015-03-30 08:22:23 <wumpus> jonasschnelli: is the github DDoS still going?
 49 2015-03-30 08:23:28 <wumpus> well, the change there couldn't cause a travis issue even if it wanted, so yes it's ok
 50 2015-03-30 08:23:46 <jonasschnelli> wumpus, as far as i know yes. China tries to muzzle some repos... :)
 51 2015-03-30 08:25:37 <wumpus> we have this decentralized scm, and then we build a centralized 'hub' for it, serves us right :)
 52 2015-03-30 08:26:50 <jonasschnelli> wumpus, Yesterday i had to think about github and centralization. What if github would go down. What about the issue list, PRs, discussions, etc.
 53 2015-03-30 08:27:09 <wumpus> if wonder if the reason for google deprecating google code was something like this
 54 2015-03-30 08:27:40 <jonasschnelli> We should regular dump github data to preserve decentralization in developing bitcoin
 55 2015-03-30 08:27:46 <wumpus> there's a backup
 56 2015-03-30 08:27:57 <jonasschnelli> ah. Where?
 57 2015-03-30 08:28:41 <wumpus> https://github.com/zw/bitcoin-gh-meta   various people are tracking that repository
 58 2015-03-30 08:29:19 <wumpus> you're of course welcome to do so, too :)
 59 2015-03-30 08:30:31 <jonasschnelli> nice! That's perfect.
 60 2015-03-30 08:34:12 <JackH> you guys are paranoid freaks, lol
 61 2015-03-30 08:36:38 <fanquake> wumpus nice. Never new about that repo.
 62 2015-03-30 08:36:39 <wumpus> hehe, we are, there's no other way in this line of work
 63 2015-03-30 08:36:44 <fanquake> *knew
 64 2015-03-30 08:37:21 <Adlai> JackH: bitcoin itself is much more paranoid than that system. now, if they were to make a new commit every ten minutes...
 65 2015-03-30 08:38:02 <wumpus> fanquake: I thought it was more well-known, maybe we should mention it in some doc
 66 2015-03-30 09:35:49 <wumpus> "Countermeasures against eclipse attacks" https://github.com/bitcoin/bitcoin/pull/5941 needs reviewing, this is the last blocker for 0.10.1
 67 2015-03-30 09:38:36 <jonasschnelli> how do i calculate multisig signature lengths? (n*73)+varint(scriptSigSize)? There is no need to add N*pubkeysize (compressed=34bytes) to the calculation?
 68 2015-03-30 09:39:18 <jonasschnelli> or (n*72) to be aware of the low-s change
 69 2015-03-30 09:51:40 <wumpus> also https://github.com/bitcoin/bitcoin/pull/5900
 70 2015-03-30 09:51:47 <wumpus> jonasschnelli: not sure
 71 2015-03-30 09:53:52 <jonasschnelli> wumpus, i try now the debug/step the sign() code to see how many bytes get added here and there... to compare agains. Will figure it out. :)
 72 2015-03-30 09:54:03 <wumpus> preliminary release notes for 0.10.1, let me know if you have anything to add https://github.com/bitcoin/bitcoin/blob/0.10/doc/release-notes.md
 73 2015-03-30 09:54:06 <jonasschnelli> s/again/against
 74 2015-03-30 09:54:54 <wumpus> jonasschnelli: sounds kind of hard to get right
 75 2015-03-30 09:55:32 <wumpus> jonasschnelli: would another option be to do a dummy 'sign'?
 76 2015-03-30 09:56:03 <wumpus> so to not actually call the secp256k1 signing w/ a key, but do everything apart from that
 77 2015-03-30 09:56:17 <jonasschnelli> wumpus, you mean dummy sign with a dummy keystore? Would also be a option, but would need some code changes to always machetes CKeyIDs...
 78 2015-03-30 09:57:27 <jonasschnelli> For "standard" OP_DUP OP_HASH160 scrip tis the calculation now is okay. I just try to test some multisig stuff.
 79 2015-03-30 09:57:29 <jonasschnelli> But...
 80 2015-03-30 09:57:30 <wumpus> that's one option; but I mean it very general; build the signature without doing any actual signing. It would avoid having two code paths that have to stay in sync
 81 2015-03-30 09:57:50 <jonasschnelli> fundrawtransaction might calculate wrong fees if you fund a custom script transaction
 82 2015-03-30 09:58:26 <wumpus> it should probably fail in that case, as we have no idea how to sign those?
 83 2015-03-30 09:58:57 <wumpus> or, I don't know
 84 2015-03-30 09:59:04 <jonasschnelli> wumpus, yes. probably. But i'd like the idea of dummy sign... but would be mean some code changes in sign.cpp
 85 2015-03-30 09:59:26 <jonasschnelli> skip actually CKey retrieving, skip verifying, etc.
 86 2015-03-30 09:59:52 <wumpus> yes
 87 2015-03-30 10:00:34 <jonasschnelli> i think we just need to handle TX_PUBKEY/TX_PUBKEYHASH/TX_SCRIPTHASH/TX_MULTISIG
 88 2015-03-30 10:00:49 <jonasschnelli> because bitcoind is able to sign TX_NONSTANDARD
 89 2015-03-30 10:01:08 <jonasschnelli> so there is no need for length calculation for stuff that can't be signed later
 90 2015-03-30 10:01:25 <jonasschnelli> *is NOT able
 91 2015-03-30 10:01:30 <wumpus> indeed
 92 2015-03-30 10:29:12 <u20024804> CScriptCheck() : 971b4d9d20553ed5183e807a5aebba036599189b20a6b9a831c7f2c402654552 VerifySignature failed
 93 2015-03-30 10:29:29 <u20024804> ERROR: Non-canonical signature: R value negative
 94 2015-03-30 10:30:10 <u20024804> my bitcoin can not synchronize any nore
 95 2015-03-30 10:30:35 <u20024804> because transaction 971b4d9d20553ed5183e807a5aebba036599189b20a6b9a831c7f2c402654552 has a negative R value
 96 2015-03-30 10:30:54 <wumpus> what version?
 97 2015-03-30 10:32:26 <u20024804> 0.9
 98 2015-03-30 10:32:40 <wumpus> 0.9.what?
 99 2015-03-30 10:33:08 <u20024804> 0.9.3 bitcoin core
100 2015-03-30 10:33:39 <u20024804> Satoshi 0.9.3
101 2015-03-30 10:34:42 <u20024804> I am sorry that I have updated to Satoshi 0.9.4
102 2015-03-30 10:35:19 <wumpus> self-compiled, or binaries from bitcoin.org?
103 2015-03-30 10:35:29 <u20024804> self compiled
104 2015-03-30 10:35:50 <wumpus> in that case you need to upgrade to 0.9.4, 0.9.3 won't work with current openssl
105 2015-03-30 10:38:21 <u20024804> But the function 'IsCanonicalSignature' in script.cpp checks r value, and can't accept negative R value, but not the openssl can not accept negative R value
106 2015-03-30 10:38:41 <u20024804>     if (R[0] & 0x80)         return error("Non-canonical signature: R value negative");
107 2015-03-30 10:39:04 <u20024804> You can see these code in the function IsCanonicalSignature in script.cpp
108 2015-03-30 10:39:25 <wumpus> it's one of the blocks that triggers the openssl issue
109 2015-03-30 10:40:53 <u20024804> But I have aleady updated to 0.9.4
110 2015-03-30 10:42:54 <u20024804> Is negative R value valid ?
111 2015-03-30 10:43:25 <wumpus> it's currently valid, just not canonical
112 2015-03-30 10:44:42 <wumpus> BIP66 will make it invalid
113 2015-03-30 10:44:55 <wumpus> but not back in a block from 2014...
114 2015-03-30 10:45:34 <u20024804> Is there any boot arguments to accept non-canonical transactions ?
115 2015-03-30 10:45:52 <wumpus> no
116 2015-03-30 10:47:22 <u20024804> But how can other 0.9.4 nodes keep synchronize ?
117 2015-03-30 10:48:06 <wumpus> you shouldn't have to change that. I haven't heard anyone else report this so most likely would be a local corruption issue
118 2015-03-30 10:49:50 <u20024804> 971b4d9d20553ed5183e807a5aebba036599189b20a6b9a831c7f2c402654552, could you check is this transaction have a negative R value ?
119 2015-03-30 11:09:04 <wumpus> R value for the signature of the single input of that transaction is b1137f5bc9acc5e1400e3749a66d48da90cf1f31d4802ef3fee7fb3e4b5d0d65 . The first byte is &0x80, so yes, it is negative
120 2015-03-30 11:13:13 <wumpus> so the determination is correct, it just should not be fatal
121 2015-03-30 11:25:13 <wumpus> this is really 0.9.4 or latest 0.9 branch? I'll spin up a node to see if I can reproduce
122 2015-03-30 12:03:32 <hearn> wumpus: did you just receive the "truncated download" mail too?
123 2015-03-30 12:03:47 <hearn> wumpus: i noticed the timestamp on it is three days ago. it looks like sourceforge mailing lists might be dying again
124 2015-03-30 12:03:53 <wumpus> yes, also replied to it
125 2015-03-30 12:04:15 <wumpus> it was hanging in the moderator queue for three days, among 12 spam messages
126 2015-03-30 12:04:18 <hearn> ah i see
127 2015-03-30 12:04:29 <hearn> ok, good to know
128 2015-03-30 12:05:08 <wumpus> the address gets loads of spam; something that is really missing from sourceforge's mailinglist functionality is 'confirm this mail', that would avoid most of the shit
129 2015-03-30 12:05:46 <hearn> perhaps we could have a simple web form to send mail to it instead of a public email address
130 2015-03-30 12:06:20 <wumpus> that may be even easier to spam
131 2015-03-30 12:07:36 <jonasschnelli> implement hashcash for the mailing list. :)
132 2015-03-30 12:08:28 <hearn> it's usually pretty easy to make a web form that isn't spammable, if you're willing to write a bit of custom javascript for it
133 2015-03-30 12:08:56 <wumpus> jonasschnelli: well, even having users confirm the mail (by eg having them click a link) would work fine
134 2015-03-30 12:09:00 <hearn> you know, the arbitrary Q+A trick works quite well as long as you invent a truly unique question for it. xrumer and friends have databases of common puzzles these days so you have to be a little creative but the space of possible questions is so large ...
135 2015-03-30 12:09:05 <wumpus> jonasschnelli: but alas, sourceforge doesn't offer that.
136 2015-03-30 12:09:16 <hearn> yeah. google groups does ;)
137 2015-03-30 12:10:20 <hearn> in this case it seems the problem is:
138 2015-03-30 12:10:21 <hearn> 
139 2015-03-30 12:10:30 <hearn> X-Helo-Check: bad, Forged Random Domain (goodcrypto.com)
140 2015-03-30 12:11:04 <jonasschnelli> but 0.5 should made spamassassin mark this as spam.. i assume threshold will be around 2-3?
141 2015-03-30 12:11:05 <wumpus> there's no problem, all mail ends up in the moderation queue
142 2015-03-30 12:11:15 <wumpus> (except from senders that are on the list)
143 2015-03-30 12:12:22 <hearn> my guess is the server is announcing itself as localhost or something like that, which makes spam filters unhappy
144 2015-03-30 12:12:27 <hearn> email is such a PITA to get right
145 2015-03-30 12:12:32 <hearn> it's no wonder so few people self host
146 2015-03-30 12:13:22 <wumpus> indeed, receiving mail is fairly easy, but sending it... (and not having it end up in spam boxes) is almost impossible these days
147 2015-03-30 12:20:08 <hearn> if only i had a way to double the length of each day, playing with new email systems would be a fun project :)
148 2015-03-30 12:22:01 <jonasschnelli> multisig: why i don't see received multisig coins in my bitcoind wallet?
149 2015-03-30 12:22:59 <jonasschnelli> (i did create a 2of2 msig address on node2, sent 1.2BTC from node0 to the new msig addr of node2, mined some blocks, nothing received)
150 2015-03-30 12:23:21 <jonasschnelli> not in listunspent neither in listreceivedbyaddress(0,True)
151 2015-03-30 12:24:01 <jonasschnelli> (i see the multisig address, but it just says "amount:0"
152 2015-03-30 12:24:21 <hearn> presumably because the wallet doesn't know how to spend them
153 2015-03-30 12:24:47 <hearn> we had the same semantic design issue in bitcoinj: is the balance of a wallet the sum of the outputs that it knows how to *match* or the sum of the outputs it knows how to *spend* ?
154 2015-03-30 12:25:31 <hearn> in the next release the balance becomes the first definition, as otherwise watching wallets end up with a balance of zero (or rather, you had to use a special call to get the balance) which is unintuitive to people.
155 2015-03-30 12:25:52 <hearn> jonasschnelli: btw are you working in zurich this week? want to meet up somewhere if so?
156 2015-03-30 12:26:58 <jonasschnelli> hearn, i'm mostly work in basel to avoid zurich stress level. :) But tmr i'm there but are in hurry, will check... but we def. need to meet soon!
157 2015-03-30 12:27:32 <hearn> ok, cool
158 2015-03-30 12:27:40 <hearn> this is the first time i heard of zurich being stressful :)
159 2015-03-30 12:47:05 <wumpus> jonasschnelli: it does have the redeemscript available?
160 2015-03-30 12:54:51 <jonasschnelli> wumpus, hmm.. i assume the redeemscript was stored in my wallet (i used rpc addmultisigaddress) but there is no listing of the balance anywhere?
161 2015-03-30 13:03:09 <wumpus> the balance only shows that which is directly spendable
162 2015-03-30 13:03:31 <wumpus> you can add watchonly to watch the multisig address
163 2015-03-30 13:05:47 <petertodd> phantomcircuit: indeed, and it sounds like in the case where they don't convert to fiat instantly they do pull the funds back
164 2015-03-30 13:06:24 <jonasschnelli> wumpus, so the coinselection code is not ready for adding/using funds from multisig addresses?
165 2015-03-30 13:07:20 <jonasschnelli> in short: you can't spend received funds over multisigaddresses with "sendtoaddress"?
166 2015-03-30 13:07:46 <petertodd> dgenr8: it's worth noting that I've *delibrately* not made my doublespend.py script in replace-by-fee-tools as good as I could because I don't want to see people get ripped off; it could be way more effective than it is; it doesn't even try to submit txs to multiple parts of the network simultaneously
167 2015-03-30 13:08:35 <hearn> jonasschnelli: i thought you could as long as you have all the keys in the wallet
168 2015-03-30 13:09:24 <jonasschnelli> hearn: i thought the same, but it doesn't look like. Will retest and analyze.
169 2015-03-30 13:09:51 <hearn> jonasschnelli: not that such a setup is terribly useful ....
170 2015-03-30 13:10:10 <jonasschnelli> hearn, Agreed.
171 2015-03-30 13:11:05 <hearn> jonasschnelli: bitcoinj has a notion of pluggable signing engines. they are tried in turn and provide back signatures over what they can do. they are also allowed to serialize data into the wallet. the idea is, a service like BitGo or whatever can write a plugin that knows how to countersign, and then account/server info can be stored in the wallet.
172 2015-03-30 13:11:18 <hearn> jonasschnelli: it'd be neat if the Core wallet had something similar.
173 2015-03-30 13:11:39 <hearn> (nb: we have not actually got any real service plugins yet so that line of work hasn't been fully finished)
174 2015-03-30 13:12:23 <jonasschnelli> hearn, at least it would be nice if there would be a way to co-sign with a 2nd device (like a smartphone, Trezor, etc.). But i think this would be a GUI only thing.
175 2015-03-30 13:12:46 <hearn> yes
176 2015-03-30 13:12:53 <hearn> well, maybe
177 2015-03-30 13:13:02 <hearn> the idea behind pluggable cosigners is you can have e.g. a risk analysis engine
178 2015-03-30 13:13:09 <wumpus> we wouldn't really want to plug third-party code into the wallet though; this would have to be an API to an external process
179 2015-03-30 13:17:59 <jonasschnelli> i'm still thinking how a external bitcoin-core wallet design could look like. Over RPC? No. Use bitcoin-core as library in another wallet repository/project? Not ready, deterministic signing question. zmq? Probably
180 2015-03-30 13:18:31 <hearn> SPV via P2P over localhosdt?
181 2015-03-30 13:18:34 <hearn> *localhost
182 2015-03-30 13:19:36 <wumpus> jonasschnelli: basically bitcoin core + SPV - block storage - full node logic
183 2015-03-30 13:20:15 <wumpus> the first step in any case is implementing an SPV mode for bitcoin core
184 2015-03-30 13:20:56 <jonasschnelli> wumpus, yes. But i'd like to keep the possibility to run a wallet on/over a full-node.
185 2015-03-30 13:21:10 <hearn> SPV wallet connected to a trusted local full node == full node equivalent
186 2015-03-30 13:21:20 <hearn> that's why Lighthouse will sniff localhost for a full node and use it if it can
187 2015-03-30 13:21:29 <hearn> instead of using the p2p network, in that case.
188 2015-03-30 13:21:55 <jonasschnelli> but SPV/Node detaching (or attaching over p2p) would mean you loos information. Like calculating fees.
189 2015-03-30 13:22:19 <hearn> you mean fee estimation?
190 2015-03-30 13:22:32 <jonasschnelli> yes.
191 2015-03-30 13:22:33 <wumpus> sure, information computed from mempool would not be available in SPV mode
192 2015-03-30 13:22:40 <hearn> ok, so expose fee estimates via p2p ;)
193 2015-03-30 13:22:59 <wumpus> that's inherent in any SPV client, you need a synced node for that
194 2015-03-30 13:23:02 <jonasschnelli> no comment on this... (another discussion).
195 2015-03-30 13:23:04 <hearn> the doc for the message can have a big health warning saying "make sure you trust the node and trust the link before you use this"
196 2015-03-30 13:23:22 <wumpus> jonasschnelli: it could request that data through RPC to a trusted node, I suppose
197 2015-03-30 13:23:24 <hearn> or it can implement the design i suggested for SPV calculation of fee estimates, last year.
198 2015-03-30 13:23:35 <wumpus> no need to do it over P2P at all
199 2015-03-30 13:23:38 <hearn> but that's complicated and needs the consensus changes for v2 transactions.
200 2015-03-30 13:23:58 <wumpus> could use any protocol, even just something slapped over http
201 2015-03-30 13:25:22 <jonasschnelli> so imaging a bitcoin-core spv mode could mean: running a bitcoin-core full node and also running a bitcoin-core-spv+wallet thread on the same machine... yes..
202 2015-03-30 13:26:39 <hearn> yes
203 2015-03-30 13:26:48 <wumpus> indeed, although as said, the first step would be to implement SPV mode in bitcoin core itself, for now it's not the biggest issue to have them in one process
204 2015-03-30 13:26:55 <hearn> you can always use an in-process message queue or something like that, rather than an actual network socket.
205 2015-03-30 13:27:18 <jonasschnelli> i played around some days with a possible spv mode for bitcoin-core. It doesn't look that hard. The question is more to use main.cpp as core logic or to start a spv.cpp...
206 2015-03-30 13:27:25 <wumpus> right, QLocalPipe or something like that
207 2015-03-30 13:27:37 <jonasschnelli> it would be nice to reuse the whole pow/reorg stuff
208 2015-03-30 13:27:46 <hearn> be careful
209 2015-03-30 13:28:00 <hearn> the bitcoinj wallet internally looks quite different to the Core wallet (bcj is more complicated). partly because you have to handle partial reorgs.
210 2015-03-30 13:28:07 <hearn> that is, you know a re-org occurred but you don't see all transactions
211 2015-03-30 13:28:16 <hearn> you also don't have a full mempool
212 2015-03-30 13:28:30 <hearn> and if you want to be able to properly handle double spends and the like in this environment, well, it's a bit painful.
213 2015-03-30 13:29:28 <hearn> btw i am slowly rewriting the bcj wallet to try and make it work entirely in terms of outputs and tx hashes. no actual transactions stored anywhere (we are not there yet, it's a slow refactoring)
214 2015-03-30 13:29:43 <hearn> the reason is some users receive absurd quantities of giant transactions and then their wallets get big fat and slow. and then they whinge :)
215 2015-03-30 13:31:06 <jcorgan> i had once come to the conclusion that one could implement the full capability of a wallet using the existing RPC interface, with the exception of asynchronous notification of incoming transactions and reorgs, and the latter could be done by a simple ZMQ channel.
216 2015-03-30 13:31:18 <jcorgan> the delta in bitcoind to support that is very small.
217 2015-03-30 13:31:39 <jcorgan> it seemed an unpopular idea, however.
218 2015-03-30 13:31:41 <wumpus> you could do it all over P2P
219 2015-03-30 13:32:21 <jcorgan> you could do it all by hand crafting Ethernet packets; but why require someone to re-implement that low-level of code?
220 2015-03-30 13:32:33 <wumpus> it clear that that would work, but the point is, you don't *need* an extra channel, it would unnecessarily bind it to a full node
221 2015-03-30 13:32:56 <hearn> does anyone have contact details for coinify.com?
222 2015-03-30 13:33:11 <wumpus> you don't need to reimplement it, the code exists already
223 2015-03-30 13:36:27 <petertodd> hearn: is this urgent? I probably have it in my business card pile at home
224 2015-03-30 13:36:31 <hearn> it's not urgent
225 2015-03-30 13:36:36 <hearn> i probably have it in my pile too :) i am just being lazy
226 2015-03-30 13:38:18 <jcorgan> you'd still need to design your application logic as a layer on top of a "P2P-node" library.  i'd expect that to be more work than RPC calls and about five lines to listen on a ZMQ socket.
227 2015-03-30 13:38:22 <wumpus> jcorgan: that idea would be to offer CValidationInterface notifications over a local pipe; sure, then you can move the wallet to a separate process, which is an improvement from a modularity point, but it doesn't help you run bitcoin core as SPV wallet , or start as a SPV wallet and backfill into a full node, as seems to be requested a lot
228 2015-03-30 13:39:23 <jcorgan> my use case would be to leave bitcoind as a full-node "border router", then you could have any number of wallets running behind it talking RPC and ZMQ.
229 2015-03-30 13:40:09 <wumpus> sure sure but the same is possible with a SPV wallet already, without having to use a 'proprietary' interface
230 2015-03-30 13:40:56 <wumpus> (bascially it's just the electrum idea)
231 2015-03-30 13:41:05 <petertodd> jcorgan: amir taaki's libbitcoin uses ZMQ fwiw
232 2015-03-30 13:41:09 <wumpus> many setups use electrum + a local electrum server
233 2015-03-30 13:41:14 <jcorgan> perhaps it is just a matter of perspective.  it seems that requiring a wallet to use SPV to talk to a trusted node is a artificially high barrier to writing a wallet
234 2015-03-30 13:41:26 <jcorgan> petertodd: yes, i've seen it
235 2015-03-30 13:41:32 <wumpus> writing a wallet is easy with bitcoinj
236 2015-03-30 13:41:38 <wumpus> (well, not easy but...)
237 2015-03-30 13:41:55 <hearn> it's pretty easy. there's a class called WalletAppKit. you instantiate that, call start and you have a synced spv wallet.
238 2015-03-30 13:42:06 <hearn> as long as your preferred language runs on the jvm you are good to go
239 2015-03-30 13:42:29 <hearn> if you want to use C, C++, Objective-C, Go or Rust or other languages that don't have a JVM equivalent then you're back to needing manual binding goop
240 2015-03-30 13:42:33 <buZz> your preferred java
241 2015-03-30 13:43:11 <hearn> nah, lots of languages run on the JVM these days. you can use javascript, python, ruby, a dialect of haskell, scala, kotlin, R ....
242 2015-03-30 13:43:24 <hearn> a dynamic dialect of lisp
243 2015-03-30 13:43:46 <hearn> even C# and object pascal, if you are OK with buying a proprietary compiler!
244 2015-03-30 13:43:48 <wumpus> you can also run java outside the JVM by using GNU java compiler
245 2015-03-30 13:43:58 <hearn> http://en.wikipedia.org/wiki/List_of_JVM_languages
246 2015-03-30 13:44:24 <hearn> yes. GCJ is a little bitrotted these days. but it can be done. a more modern version of the same thing is RoboVM but it targets MacOS and iOS only, afaik
247 2015-03-30 13:45:12 <hearn> there's actually a research project that runs C on the JVM too. it's interpreted and then just in time compiled (*boggle*)
248 2015-03-30 13:45:41 <wumpus> turing tarpit
249 2015-03-30 13:46:13 <jcorgan> i could use a similar argument and just say that RPC and ZMQ have bindings in dozens of languages
250 2015-03-30 13:47:11 <hearn> yes, but the bulk of the complicated code in bitcoinj is not networking related. it's to do with actually handling and correctly managing the notifications from the network.
251 2015-03-30 13:47:17 <jcorgan> (perhaps not *dozens*)
252 2015-03-30 13:47:21 <wumpus> jcorgan: so just use electrum? that's basically what you're describing, except less hacky
253 2015-03-30 13:47:25 <hearn> if you bind to a full node via ZMQ you still have to implement all that logic
254 2015-03-30 13:48:42 <jcorgan> wumpus: electrum server is very heavyweight.  bitcoind is not :)
255 2015-03-30 13:48:42 <wumpus> hearn: indeed
256 2015-03-30 13:48:53 <jcorgan> hearn: yeah, that wouldn't change
257 2015-03-30 13:49:37 <wumpus> jcorgan: bitcoind is also not exactly lightweight
258 2015-03-30 13:49:39 <jcorgan> but you're arguing, just use bitcoinj; i'm arguing to allow a small delta to bitcoind (adding the ZMQ notifications) and let people do whatever they want
259 2015-03-30 13:49:59 <wumpus> sure, electrum adds some weight, but you get some convenience back for that, the good thing is that it already exists and you can use it
260 2015-03-30 13:50:44 <wumpus> people can already do what they want, no need for us to argue about that :)
261 2015-03-30 13:50:47 <jcorgan> this is an old argument, i wasn't successful in convincing anyone the first time i tried to make it last year, and don't seem to be doing any better this time around.
262 2015-03-30 13:50:55 <hearn> jcorgan: well, features have a maintenance cost. so to justify it someone would have to implement a wallet that worked that way.
263 2015-03-30 13:50:59 <jcorgan> even through there was code implemented
264 2015-03-30 13:51:02 <jcorgan> though
265 2015-03-30 13:51:24 <wumpus> it's just not a route that I want to persue, but other people can do anything they want
266 2015-03-30 13:51:35 <hearn> huh fascinating. someone actually implemented an LLVM -> JVM converter engine. so you can actually run C programs on the JVM. crazy hacks!
267 2015-03-30 13:51:38 <wumpus> open source and such
268 2015-03-30 13:52:35 <wumpus> hearn: why would someone do that instead of LLVM->native?
269 2015-03-30 13:53:03 <jcorgan> wumpus: of course, but maintaining a local fork of bitcoind is tiring.  i did it for a while with the addrindex patch and with the zmq patch, but without the code being visible to developers, it gets stomped on frequently
270 2015-03-30 13:53:49 <hearn> wumpus: he doesn't seem to provide much justification for it beyond C and Java being able to call each other without binding overhead. it's also possible that you would get faster code because the JITC can e.g. inline java into C and vice-versa when doing optimisations
271 2015-03-30 13:53:52 <hearn> http://da.vidr.cc/projects/lljvm/
272 2015-03-30 13:53:54 <wumpus> jcorgan: there's even now a pull open for zmq notification in bitcoind, but there just isn't much interest in it
273 2015-03-30 13:54:16 <jcorgan> wumpus: i think you're referring to the one i wrote and jgarzik picked up
274 2015-03-30 13:55:30 <hearn> for example the research project that JIT compiles interpreted C, is designed to let you use C Ruby extensions from within their souped-up Ruby-on-the-JVM implementation. and when they run code that does e.g. image processing in Ruby calling out to a C extension, the final code runs faster than the original ruby interpreter+C extension does
275 2015-03-30 13:55:53 <hearn> because the compiler can see both C and Ruby simultaneously and optimise across the transitions to remove all the abstraction and interop overhead
276 2015-03-30 14:00:16 <hearn> jcorgan: what kind of wallet would you implement if that feature was integrated into Core? out of curiousity
277 2015-03-30 14:05:16 <jcorgan> nothing radically different; at one point i was working on a "stealth address" transaction scanner that listened to zmq, also a "cold storage manager" that would sync up Electrum style to for watching wallets, using RPC, a couple of other half-finished ideas.  the idea itself back at the very beginning was inspired by the question, "What would be needed to allow Armory wallet to not keep an entirely separate copy of the blockchain?"
278 2015-03-30 14:06:06 <jcorgan> and the conclusion was, just incoming tx/block notification, which my zmq patch made very simple
279 2015-03-30 14:07:37 <wumpus> I don't see how tx/block notification would change armory having to store a copy of the block chain?
280 2015-03-30 14:08:00 <jcorgan> well, with txindex and addrindex, bitcoind would be doing that
281 2015-03-30 14:08:27 <wumpus> that'd basically change bitcoind into electrum server
282 2015-03-30 14:08:59 <jcorgan> lighterweight than that, and you're already running it.  it was just a couple of indexes added to the db.
283 2015-03-30 14:09:39 <wumpus> bitcoind is meant as infrastructure, it is not meant to accomedate any possible use case that people could have with bitcoin
284 2015-03-30 14:10:31 <jcorgan> right, we're retreading old ground.  i'm arguing for some incremental features that enable others to build on top of bitcoind, but those features, as small as they are in code, seem to cross a line that few people find acceptable.
285 2015-03-30 14:10:39 <wumpus> it's easy to add it but no one is going to maintain all that extra code
286 2015-03-30 14:11:06 <jcorgan> addrindex was pretty small, zmq was even smaller
287 2015-03-30 14:11:08 <wumpus> even now there is hardly interest in say, most wallet changes
288 2015-03-30 14:11:20 <wumpus> yes but it's all incremental
289 2015-03-30 14:11:23 <wumpus> let's add this let's add that
290 2015-03-30 14:12:57 <wumpus> all optional features that possibly interact with each other and have to be tested in all combinations, even more that can break from version to version
291 2015-03-30 14:13:25 <wumpus> even more dependencies
292 2015-03-30 14:13:57 <jcorgan> of course i agree with those as general principles
293 2015-03-30 14:14:23 <jcorgan> which is why i was advocating the inclusion of these as interfaces, to allow actual logic to be implemented elsewhere
294 2015-03-30 14:14:47 <jcorgan> none of it touches consensus or data structures, just hooks to expose bitcoind
295 2015-03-30 14:15:10 <jcorgan> anyway, this is probably not very productive here.
296 2015-03-30 14:15:48 <wumpus> sure - I've always liked the idea of async notifications, this could be done through http as well (say, with longpolling or websocket)... but I'm less excited about dragging in zmq
297 2015-03-30 14:18:17 <jcorgan> "dragging" is a loaded word :)
298 2015-03-30 14:18:25 <wumpus> if I were to redesign bitcoind I'd make it a full node daemon only, with a simple REST interface to query its state and listen for notifications
299 2015-03-30 14:18:49 <jgarzik> +1
300 2015-03-30 14:18:55 <jcorgan> ah, we agree wholeheartedly on that
301 2015-03-30 14:19:13 <jcorgan> can you listen asynchronously though for notifications over a REST API?
302 2015-03-30 14:19:13 <wumpus> but I'm not, we have to do with what we have, and it's already a crazy complicated thing and I'm very conservative about bolting on more...
303 2015-03-30 14:19:54 <wumpus> jcorgan: no, would be nice to have
304 2015-03-30 14:20:25 <jcorgan> i mean in principle.  i didn't think you could.  hence...message passing asynchronous interfaces like zmq.
305 2015-03-30 14:20:36 <wumpus> jcorgan: websocket, longpolling
306 2015-03-30 14:20:58 <wumpus> 'wake me up when there is a new block'.. you really don't need zmq for a simple notification socket
307 2015-03-30 14:21:00 <jcorgan> i admit i'm less familiar with those
308 2015-03-30 14:21:40 <jcorgan> i need to run, off to hack for a week on a completely different code base :)
309 2015-03-30 14:22:52 <wumpus> it's not that I have anything specific against zmq, the concept is nice; but it's another library to learn, potentially opens up attack surface, it doesn't seem like something that belongs in bitcoind directly
310 2015-03-30 14:22:54 <wumpus> okay have fun
311 2015-03-30 14:31:28 <jcorgan> just a final comment==the real desire is for a simple asynchronous notification mechanism.  i happen to think zmq is it, but something else would also work for what i was advocating.  thanks for the discussion.
312 2015-03-30 14:31:59 <wumpus> agree on that - the current invoke-a-command ntoification mechanism has always bothered me :)
313 2015-03-30 14:33:07 <wumpus> (mostly because it's inflexible; you can't plug in anything to be notified after starting bitcoind)
314 2015-03-30 14:42:17 <jonasschnelli> back to bitcoin-core multisig addresses balance: bitcoin-core DOES handle the balance correct (when you have all according keys = tx spenable). Was a mistake on my side! Sorry for the noise.
315 2015-03-30 14:43:26 <wumpus> ok, good to hear
316 2015-03-30 14:46:07 <jonasschnelli> and the coin-selection will also use these funds.
317 2015-03-30 14:47:04 <hearn> jonasschnelli: right, so balance == spendable outputs
318 2015-03-30 14:47:38 <jonasschnelli> hearn, it looks like. Didn't analyze the behavior completely yet.
319 2015-03-30 15:06:20 <wumpus> yes, balance is what is directly spendable
320 2015-03-30 15:18:18 <dgenr8> how difficult is it to construct a useful transaction that is relayed by 99% of the network?
321 2015-03-30 15:19:52 <wumpus> ... in what context? bitcoind seems to have no difficulty doing so, generally
322 2015-03-30 15:21:18 <wumpus> although no hard guarantees are possible on a P2P network
323 2015-03-30 15:23:50 <dgenr8> just roughly knowing the isstandard changes in still-relevant releases of core ... does anything come to mind about today's default created txes that won't relay on those releases?
324 2015-03-30 16:17:04 <dgenr8> looks like quite a bit changed code-wise in .10.  the P2SH relaxations, scriptsig size check, possible bare multisig ban, and a big change where Solver was moved to standard.cpp that github is no help analyzing...
325 2015-03-30 16:18:06 <dgenr8> now to become the nth person to analyze propagation empirically ;)
326 2015-03-30 16:22:09 <gmaxwell> dgenr8: surprises, a year of development actually changed things.
327 2015-03-30 16:25:03 <dgenr8> and in very positive ways.  anything I missed?
328 2015-03-30 19:25:27 <ThomasV> cal
329 2015-03-30 19:33:29 <benjyz1> hello. can someone tell me how can write a python script to decode a transaction?
330 2015-03-30 20:11:21 <Guest97134> benjyz1: use one of the many python bitcoin libraries
331 2015-03-30 20:11:46 <benjyz1> I want to decode transactions to one address
332 2015-03-30 20:12:00 <benjyz1> I'm now using blockchain info's api
333 2015-03-30 20:51:07 <benjyz1> hello bitcoin-devs
334 2015-03-30 20:51:35 <benjyz1> can someone tell me, if a message is attached how I decode that from the raw script?
335 2015-03-30 22:03:35 <phantomcircuit> Luke-Jr, after going through most of the gbt protocol
336 2015-03-30 22:04:10 <phantomcircuit> it strikes me that the minimum protocol would be something in which the client selects virtually everything except for a set of coinbase outputs
337 2015-03-30 22:04:49 <phantomcircuit> and that the rest of the protocol is about communicating the pools policies to miners
338 2015-03-30 22:06:59 <Luke-Jr> phantomcircuit: right. GBT was designed initially with the goal of being a replacement for getwork, so it had a lot of cruft to try to remain otherwise compatible with it