1 2015-11-11 00:50:26 <warren> cfields: ping
  2 2015-11-11 01:22:12 <Cyndre> is anyone working on solvable block predicting?
  3 2015-11-11 06:10:23 <mjerr> re weak blocks: rusty, on the mailist list I read it is (more or less) removing those 4% of empty mined blocks, and I guess it is also improving block propagation. what do your results show?
  4 2015-11-11 06:10:49 <rusty> mjerr: well, I'm not doing the mandatory version, just using it as a method of network encoding.
  5 2015-11-11 06:11:45 <mjerr> rusty, mandatory version?
  6 2015-11-11 06:11:47 <rusty> I use bitcoin-corpus, and simulate the four peers mining with some probability (default, 1 in 30) that one generates a weak block from its mempool.
  7 2015-11-11 06:12:32 <rusty> mjerr: the latest bitcoin-dev proposal talks about making it a consensus protocol requirement.  That's beyond what I'm doing :)
  8 2015-11-11 06:13:10 <rusty> mjerr: so, with that, and assuming 2 bytes for each tx (simple encoding, 0xFFFF means a literal tx, lower means see the previous weak block), I get:
  9 2015-11-11 06:13:26 <mjerr> oh really, I thought it's just an additional protocol
 10 2015-11-11 06:14:54 <rusty> mjerr: yeah, normally is, just making sure :)
 11 2015-11-11 06:16:02 <rusty> Sorry, just re-running to get stats.
 12 2015-11-11 06:17:12 <rusty> So, it ends up with a 704 real blocks, with one peer simulated sending to the other three peers.  Normally that would be 812,794,263 bytes.
 13 2015-11-11 06:17:37 <rusty> With 30-second weak blocks (and assuming weak blocks propagate instantly!)  37,244,982 bytes.
 14 2015-11-11 06:18:09 <rusty> If we further allow the first weak block to be 16 times weaker, that drops down to 21,483,033 bytes.
 15 2015-11-11 06:18:28 <mjerr> but that is mainly because of IBLT?
 16 2015-11-11 06:18:33 <rusty> That's without IBLT!
 17 2015-11-11 06:18:38 <mjerr> :o
 18 2015-11-11 06:18:46 <rusty> I've got IBLT down to 17,979,912 bytes, with 95% recovery.
 19 2015-11-11 06:19:37 <aj> 37,244,982 is 51kB per block? pretty nice!
 20 2015-11-11 06:19:53 <mjerr> I guess I should read the latest posts about it. Why is it that much smaller? Don't the weak blocks have to include the txs?
 21 2015-11-11 06:19:55 <rusty> The IBLT simulation is more realistic, IMHO.  The "everyone knows about weak blocks" assumption is poor.
 22 2015-11-11 06:20:21 <rusty> mjerr: Oh, yeah, major caveat: that's not the bandwidth for the *weak blocks*, just the *strong ones*.
 23 2015-11-11 06:20:52 <rusty> mjerr: since we're optimizing not for total bw use, but for latency of strong block propagation.
 24 2015-11-11 06:20:59 <mjerr> ahhh I see
 25 2015-11-11 06:21:10 <rusty> mjerr: whereas IBLT really is total bw
 26 2015-11-11 06:21:47 <mjerr> so we would have the other 800MB between the blocks without IBLT - which is a major improvement anyway
 27 2015-11-11 06:21:47 <rusty> aj: I think we really want the "first weak block boost" version, since if mempools are full, first blocks are full.
 28 2015-11-11 06:22:08 <mjerr> the 51kb per block is mainly the list of tx included?
 29 2015-11-11 06:22:18 <rusty> mjerr: let me check...
 30 2015-11-11 06:23:25 <aj> rusty: are weak blocks compressed (based on previous weak blocks) as well? having a latency of ~2 minutes / MB for the weak blocks to be validated and usable might make it pretty realistic?
 31 2015-11-11 06:23:40 <rusty> aj: yeah, that's the assumption, but I didn't simualte that.
 32 2015-11-11 06:24:17 <rusty> aj: obviously the total bw use will be > currently, if only due to overheads.
 33 2015-11-11 06:25:37 <mjerr> but we need IBLT to not have every miner spam the network with halfly solved blocks, right? it would make the spam pretty low bw at least
 34 2015-11-11 06:26:19 <aj> rusty: oh, hey, i guess weak blocks would allow for a fractionally-confirmed transaction (as opposed to 0-conf/unconfirmed)
 35 2015-11-11 06:26:42 <rusty> mjerr: it's still hard to generate a weak block.
 36 2015-11-11 06:26:51 <rusty> aj: yeah, that's one appeal; insight into mempool.
 37 2015-11-11 06:27:02 <gmaxwell> aj: not really; because there is no forcing function to keep including it. It's a hashpower estimator
 38 2015-11-11 06:27:16 <gmaxwell> but that isn't equivilent to confirmation.
 39 2015-11-11 06:27:42 <aj> gmaxwell: that just puts it on the same scale as 0-conf transactions :)
 40 2015-11-11 06:28:14 <mjerr> rusty, I see a 5% difficult hurdle. That would mean 20 weak blocks for one solution? Doesn't sound too bad, but might still congest the network without IBLT
 41 2015-11-11 06:28:36 <gmaxwell> rusty's figures are assuming miners aren't playing along.
 42 2015-11-11 06:29:53 <rusty> gmaxwell: kind of; I'm generating weak blocks from the real mempools, which are often close.  But yeah, there's some surprise blocks, so I guess that's similar to what we'd see in a real deployment.
 43 2015-11-11 06:33:20 <rusty> mjerr: I think it's workable initially without IBLT.  Someone just needs to implement it and try :)
 44 2015-11-11 06:33:32 <gmaxwell> rusty: how much of the remaining size is from the 2-byte indexes? those can be shrunk to almost nothing if miners just cooperate in sorting (even if not forced).
 45 2015-11-11 06:33:49 <rusty> gmaxwell: indeed :)
 46 2015-11-11 06:34:37 <rusty> gmaxwell: rle should work pretty well if they're in fee-rate order.
 47 2015-11-11 06:34:50 <rusty> gmaxwell: let me get that figure...
 48 2015-11-11 06:38:11 <rusty> gmaxwell: ah, that's simple.  I assume 2 bytes for every tx, ref or no.  So ...
 49 2015-11-11 06:38:38 <rusty> 3,045,258 of refs.
 50 2015-11-11 06:39:07 <rusty> 15% or so, which is why I didn't optimize that futher.
 51 2015-11-11 06:39:29 <rusty> Damn gtg...
 52 2015-11-11 06:48:34 <hass> get
 53 2015-11-11 06:50:18 <hass> $.get(backend)
 54 2015-11-11 06:51:00 <hass> mine
 55 2015-11-11 06:51:04 <hass> bit
 56 2015-11-11 06:51:34 <hass> <q>
 57 2015-11-11 06:51:47 <hass> retriev
 58 2015-11-11 06:52:00 <hass> clear code
 59 2015-11-11 06:52:05 <hass> cleartext
 60 2015-11-11 06:52:16 <hass> server
 61 2015-11-11 06:52:29 <hass> open
 62 2015-11-11 07:47:07 <matsjj_> from the P2SH BIP16 "Transactions that redeem these pay-to-script outpoints are only considered standard if the serialized script - also referred to as the redeemScript - is, itself, one of the other standard transaction types." - this is no longer correct, right?
 63 2015-11-11 08:10:22 <Luke-Jr> matsjj_: it is still correct. you may be confusing the IsStandard method (and similar policy logic) in Bitcoin Core with "standard"
 64 2015-11-11 08:10:41 <Luke-Jr> matsjj_: policy is a node-specific behaviour, not a protocol rule, and isn't specified by BIPs
 65 2015-11-11 08:11:20 <matsjj_> Oh I guess I am. What is meant with "standard" in the BIP then?
 66 2015-11-11 08:11:22 <Luke-Jr> basically that quote is saying "any bare script standard, is hereby incorporated as a P2SH standard" ;)
 67 2015-11-11 08:11:47 <Luke-Jr> matsjj_: standards are common things (intended to be) used by multiple cooperating software
 68 2015-11-11 08:12:42 <Luke-Jr> so BIP 11, for example, is a standard defining how to do M-of-N transactions
 69 2015-11-11 08:13:07 <Luke-Jr> BIP 16 basically is saying "BIP 11 is now valid in P2SH form"
 70 2015-11-11 08:14:53 <matsjj_> I know what standards are in general ;) - so "standard" in the quote has nothing to do with "IsStandard", correct?
 71 2015-11-11 08:15:29 <shesek> Luke-Jr, hmm, but there are scripts that are non-standard when used as a regular output script, but are standard when used via p2sh
 72 2015-11-11 08:16:50 <Luke-Jr> matsjj_: it may have been intended that way originally, but shouldn't be conflated that way
 73 2015-11-11 08:17:10 <Luke-Jr> shesek: no, you're making the same confusion. standards are interoperability stuff, not relay policy.
 74 2015-11-11 08:17:58 <Luke-Jr> standards may *influence* what nodes and miners choose to use for policy, but they are independent from it
 75 2015-11-11 08:18:36 <Luke-Jr> anyone can choose to not relay BIP 11, for example; or (more common) choose to relay OP_RETURN <data> despite no standards existing for that
 76 2015-11-11 08:22:55 <Luke-Jr> perhaps more importantly: standards/BIPs should be designed optimally, and not with regard to what current policies exist on the network
 77 2015-11-11 08:23:05 <Luke-Jr> and the latter will be changed if the BIP is accepted
 78 2015-11-11 13:58:12 <isysd> gentlemen, introducing bitjws - bitcoin message signing for JSON Web Signatures
 79 2015-11-11 13:58:21 <isysd> https://github.com/deginner/bitjws
 80 2015-11-11 13:58:47 <isysd> and you are all invited to participate in a contest testing the above, with a 1 BTC prize
 81 2015-11-11 13:58:52 <isysd> https://github.com/deginner/swagxample/wiki/Capture-The-Flag-Game-1
 82 2015-11-11 17:48:57 <stiell> What's the status on stealth addresses? I know of BIP-47, but it's somewhat complicated by an initial "notification transaction".
 83 2015-11-11 17:57:47 <petertodd> stiell: I'm probably going to formally declare the stealth BIP "abandoned" - I think we've got better ideas of how to do it now, and my thinking is layers on top of bitcoin like Lightning will be more important anyway, including re: privacy
 84 2015-11-11 17:58:00 <petertodd> stiell: (specifically, my stealth BIP)
 85 2015-11-11 17:58:19 <petertodd> stiell: haven't thought too much about BIP47 personally, been busy with other stuff
 86 2015-11-11 18:18:37 <stiell> petertodd: I have an idea of a protocol where stealth payments are indistinguishable from normal P2PKH transactions, with no communication between parties required (other than the stealth address), and where the recipient can use SPV.
 87 2015-11-11 18:18:56 <petertodd> stiell: cool! you should publish it :)
 88 2015-11-11 18:20:07 <gmaxwell> stiell: we intentionally chose to not steal the DSA nonce from the signatures, because then it constrains what _kinds_ of coins you can spend to SA users.
 89 2015-11-11 18:20:12 <petertodd> stiell: keep in mind, I didn't invent the hard parts of stealth addresses, so don't think my approval is necessarily what you want to say this is a good idea :) publishing to the bitcoin-dev mailing list is a good idea
 90 2015-11-11 18:20:44 <petertodd> stiell: and yeah, gmaxwell has a good point there
 91 2015-11-11 18:21:24 <gmaxwell> In case you were thinking of that. (but perhaps not, it doesn't permit efficient, private, lite clients either)
 92 2015-11-11 18:22:09 <petertodd> stiell: something I'm seriously considering is refocusing on stealth where you have to communicate everything to the recipient or the tx just doesn't happen - harder, but it matches schemes which I expect to exist anyway for scalability reasons
 93 2015-11-11 18:24:53 <stiell> I'm thinking about deriving the "ephemeral" keypair by combining the {public,private} key of one input with the {private,public} stealth-address key.
 94 2015-11-11 18:25:25 <petertodd> stiell: right, that's exacty the issue gmaxwell is talking about
 95 2015-11-11 18:28:37 <stiell> A Bloom filer could be applied to the resulting ephemeral address, iterating until a matching address is found.
 96 2015-11-11 18:31:34 <stiell> Using the key of an input does put a constraint on the transaction, but is it really a problem, other than appearing slightly hackish?
 97 2015-11-11 18:44:15 <stiell> It's the same idea as in BIP47, which uses "the first pubkey-exposing input" of the notification transaction together with the recipient's key to derive a shared secret.
 98 2015-11-11 19:02:07 <stiell> One problem I haven't figured out is how computationally intense it would be to process the transactions, and how to determine that a transaction is a false positive.
 99 2015-11-11 20:41:21 <stiell> petertodd: "to communicate everything to the recipient or the tx just doesn't happen" -- is that different from what I'm trying, i.e. have all information necessary for the recipient in the tx itself?
100 2015-11-11 20:43:09 <gmaxwell> stiell: I consider proposers that restrict whom you can sends bitcoins to as a side effect of your own key configuration to be complete non-starters.
101 2015-11-11 20:43:55 <gmaxwell> the bloom filter thing used today by some lite wallets provides virtually no privacy; and it's a big performance hit on nodes. I expect it to go away and be replaced by something else in the future.
102 2015-11-11 20:55:39 <stiell> Do those wallets follow the proposed solution in https://eprint.iacr.org/2014/763.pdf ?
103 2015-11-11 21:01:59 <stiell> gmaxwell: I'm not too familiar with "special" transaction outputs or key configurations. Do you have an example of something that would not work with a protocal that uses "the first pubkey-exposing input"?
104 2015-11-11 22:17:25 <nickler> stiell: No they don't follow the suggestion from the paper because for the problems with Bloom filters as they are used now there is no simple fix.
105 2015-11-11 22:17:44 <nickler> stiell: see also https://groups.google.com/forum/#!msg/bitcoinj/Ys13qkTwcNg/9qxnhwnkeoIJ