1 2015-06-11 00:07:24 <null_radix> sipa; how extremely unlikely? like hash collision unlikely? or something that might get encoutered by normal use once a while?
  2 2015-06-11 00:08:17 <gmaxwell> null_radix: as the test demonstrates it can be intentionally constructed.
  3 2015-06-11 00:08:36 <gmaxwell> So the liklyhood is proportional to how bad a day its going to cause for you.
  4 2015-06-11 00:08:57 <null_radix> haha
  5 2015-06-11 00:09:13 <gmaxwell> Perhaps reading too much tone into your text----  that if getting the details right is an irritation, maybe you should let someone else write that stuff? :)
  6 2015-06-11 00:09:17 <sipa> that's a very fundamental principle, not a joke
  7 2015-06-11 00:10:55 <null_radix> gmaxwell; maybe, but I don't want to give up on doing things just b/c they irritate me
  8 2015-06-11 00:11:45 <gmaxwell> in an adversarial context its often the case that you can't reason about 'failure probablity' like you can for a normal industrial process.  Like if your webpage has ten billion pixels you can click, and one of them pays 1 BTC to the clicker, modling the clicks as a random walk or uniform or whatever suggests you'll never lose any money. :)  But in reality your funds would be untouched for a while an
  9 2015-06-11 00:11:51 <gmaxwell> d then all be gone in an instant. :)
 10 2015-06-11 00:12:48 <gmaxwell> null_radix: for cryptosystem work virtually all the doing is in getting the fine details _exactly_ right, and then making sure they're right-- it's probably 99% of the work.  I see some people who hop in and just want to get it working and have fun with that, but hate the rest, and the rest is most of it.
 11 2015-06-11 01:34:20 <gruez> for bitcoin core, would there be any point in using CryptProtectMemory in windows to prevent dumping private keys from memory?
 12 2015-06-11 01:34:49 <gruez> CryptProtectMemory reference: https://msdn.microsoft.com/en-us/library/windows/desktop/aa380262%28v=vs.85%29.aspx
 13 2015-06-11 01:39:40 <teward> gruez: how would that translate to other operating systems?  Would there be equivalent functions elsewhere?  (no idea what's implemented, but that's a valid question either way)
 14 2015-06-11 01:42:42 <gruez> would it be a problem if it was only available to windows?
 15 2015-06-11 01:43:06 <gruez> a quick search reveals there's no linux/mac equivalent
 16 2015-06-11 01:45:18 <teward> gruez: immediate end user complaint from a security perspective:  "Wait, so I need to have WINDOWS to prevent a private key dump from memory?"
 17 2015-06-11 01:45:35 <teward> "Why do I need Windows just to get that?"
 18 2015-06-11 01:45:49 <gruez> right, but the current situation is no memory protection for everyone
 19 2015-06-11 01:46:15 <teward> not saying it's a bad idea, saying how do you extend that to the other OSes that Bitcoin can run on
 20 2015-06-11 01:46:55 <teward> adding a feature for just Windows is... in my mind, a 'why?' if the same protection can't be replicated elsewhere, but i'm not a dev, just a techie/enduser who would be asking that question i posed since I"m a *nix guy
 21 2015-06-11 01:48:26 <phantomcircuit> gruez, https://github.com/bitcoin/bitcoin/blob/master/src/support/pagelocker.cpp
 22 2015-06-11 01:54:28 <gruez> phantomcircuit, virtuallock doesn't protect against readprocessmemory
 23 2015-06-11 02:18:53 <phantomcircuit> gruez, not sure that's super useful
 24 2015-06-11 02:19:04 <phantomcircuit> but i guess if the kernel holds the key it's maybe interesting
 25 2015-06-11 02:19:17 <phantomcircuit> except that you can inject code to decrypt the memory if you're running as the same user
 26 2015-06-11 02:19:26 <phantomcircuit> so it really just makes it harder to do
 27 2015-06-11 02:19:34 <phantomcircuit> not actually impossible
 28 2015-06-11 05:54:10 <gribble> hearn was last seen in #bitcoin-dev 1 week, 0 days, 13 hours, 28 minutes, and 23 seconds ago: <hearn> as long as some code has a reference to it, the object will live
 29 2015-06-11 05:54:10 <jonasschnelli> !seen hearn
 30 2015-06-11 06:19:16 <jonasschnelli> How would that be (idea): allowing a SPV mode on top of the current peers managing. 3-4 nodes could be uses exclusive for SPV communication (or maybe a node could be used for both modes simultaneous), the SPV mode would allow the wallet to request blocks and headers to do spv while the verification process is still going on...
 31 2015-06-11 06:20:10 <jonasschnelli> The verification process could be trotted (CPU, net) because for a wallet user, the time required to be back in sync is no longer such important
 32 2015-06-11 06:20:29 <jonasschnelli> The verification process could even be disabled to use spv only
 33 2015-06-11 06:24:59 <gmaxwell> I don't see why you're thinking in terms of node counts?
 34 2015-06-11 06:26:10 <jonasschnelli> gmaxwell: i wasn't sure if a node can be used for SPV (bloom filtering) as well as for the standard verification behavior.
 35 2015-06-11 06:26:15 <jonasschnelli> but i think a node can
 36 2015-06-11 06:27:15 <gmaxwell> You do not want bloom filtering if you're going to be verifying anyways, all bloom filtering would do there is waste bandwidth and completely hose your privacy.
 37 2015-06-11 06:29:28 <jonasschnelli> gmaxwell: agreed. If bandwidth doesnt matter, bloomfiltering makes no sense. But while trying to add a SPV mode for bitcoind i always have think about having a  libbitcoincorespv which woudl allow to create smartphone apps (iOS / Android)
 38 2015-06-11 06:31:11 <gmaxwell> Even still-- I wouldn't recommend a new implementer use the bloom filters today they provide no privacy, they're also quite expensive to serve for, and less bandwidth efficient than e.g. electrum.  (presumably why we se so few clients on the network)
 39 2015-06-11 06:32:21 <gmaxwell> (there are several papers that show even with the false positive rate turned way up, the bloom filter is basically completely identifying)
 40 2015-06-11 06:33:35 <gmaxwell> I believe some academics were talking to the electrum folks about implementing a PIR server for address lookup.
 41 2015-06-11 06:34:09 <jonasschnelli> gmaxwell: Agreed. Recently had a chat with one of cdeckers (ETH Zurich) assistants and he was also telling me that he can easily find all your coins as long as you provide him the bloomfilters
 42 2015-06-11 06:34:20 <gmaxwell> But yea, in any case, I still don't see mixed mode making sense, if you're running a SPV wallet that hopes to someday become a full node, might as well recieve whole blocks and save them for when you need them.
 43 2015-06-11 06:35:44 <midnightmagic> People don't like waiting. Giving them a reason to fiddle with the software and distract them from the fact it's sync'ing would give them another reason t stay with bitcoin-core.
 44 2015-06-11 06:36:47 <gmaxwell> midnightmagic: there is really no reason they should ever have to wait.
 45 2015-06-11 06:37:56 <gmaxwell> A relatively small amount of code would need to be changed changed to make it so you'd accept blocks at the tip, on a new wallet while it syncs in the background.
 46 2015-06-11 06:38:20 <midnightmagic> gmaxwell: sounds like you're implying a modified spv-like client that just reorganizes its block fetch handling
 47 2015-06-11 06:38:51 <midnightmagic> ah, indeed you are.
 48 2015-06-11 06:39:40 <midnightmagic> wow brutal, some of my comments in -wizards don't appear in logs..
 49 2015-06-11 06:39:44 <gmaxwell> midnightmagic: pretty much, just also fetch blocks at the tip, scan them for transactions and throw them out. Make the confirmation counter in the wallet report headers valid ahead of; maybe with some rpc/visual flag that its still in SPV mode (e.g. in RPC, "headers-confirmations". Tada.
 50 2015-06-11 06:40:05 <gmaxwell> midnightmagic: Andytoshi's logbot isn't super reliable.
 51 2015-06-11 06:40:14 <midnightmagic> doh
 52 2015-06-11 06:40:27 <gmaxwell> if you can figure out the gap, whine and someone will fix.
 53 2015-06-11 06:44:11 <midnightmagic> andy's logbot has it. botbot.me is missing most of June 4th. Long as andy's has it I'm fine.
 54 2015-06-11 06:52:55 <BlueMatt> any objections to making the importaddress parser also take pubkeys?
 55 2015-06-11 06:53:06 <BlueMatt> it currently takes scripts, so there is a minute chance some scripts cant be watched
 56 2015-06-11 06:53:15 <BlueMatt> but...really, if your script is a valid pubkey...wow
 57 2015-06-11 07:01:34 <gmaxwell> BlueMatt: remind me again for what cases pubkeys are needed?
 58 2015-06-11 07:02:05 <BlueMatt> gmaxwell: the only time you care is when you're fundrawtransaction'ing a transaction...you need to calculate size of the thing to calculate fee
 59 2015-06-11 07:02:15 <BlueMatt> gmaxwell: so you need to know, essentially, if the pubkey is compressed or not
 60 2015-06-11 07:02:44 <BlueMatt> if the default were the larger one, I would just say use that constant size and move on....except the default is compressed, so.....
 61 2015-06-11 07:03:19 <BlueMatt> also, if you have the pubkey you can execute the script and check that the whole thing works with a sigchecker of { return true; }
 62 2015-06-11 07:03:21 <BlueMatt> which is kinda nice
 63 2015-06-11 07:06:45 <gmaxwell> hm in a worst case, being conservative for that would be an 18% difference, which isn't as trivial as I expected.
 64 2015-06-11 07:07:16 <gmaxwell> deciding between script and pubkey by sniffing the content is really ugly and ... uh seems to me like something that could cause funds loss though.
 65 2015-06-11 07:08:32 <BlueMatt> well its not either-or
 66 2015-06-11 07:08:49 <gmaxwell> Clarify?
 67 2015-06-11 07:09:09 <BlueMatt> you can still keep around the set of known pubkeys via a new importpubkey or by requiring people to import by using the p2-pubkey script (which is already supported)
 68 2015-06-11 07:10:19 <gmaxwell> uh, so if you import using the p2pubkey script its not treated as p2sh?!
 69 2015-06-11 07:10:35 <BlueMatt> huh?
 70 2015-06-11 07:10:56 <BlueMatt> no, there is no way to import a redeemScript via importaddress (though my existing pull included that, and I plan to keep that option)
 71 2015-06-11 07:11:39 <gmaxwell> Isn't clear what you mean by "by requring people to import by using the p2-pubkey script (which is already supported)
 72 2015-06-11 07:11:42 <gmaxwell> "
 73 2015-06-11 07:12:12 <BlueMatt> ie importaddress ToHex(OP_PUSH* pushCount pubkey OP_CHECKSIG)
 74 2015-06-11 07:12:20 <gmaxwell> sipa: vergraph is broken.
 75 2015-06-11 07:12:47 <gmaxwell> BlueMatt: and how does importing a redeemscript work?
 76 2015-06-11 07:13:06 <BlueMatt> gmaxwell: afaik there is no way currently to import an arbitrary redeemscript
 77 2015-06-11 07:13:33 <gmaxwell> How do you propose it works, I'm trying to make sure that you're not conflating importing a redeemscript and a p2pubkey.
 78 2015-06-11 07:13:36 <BlueMatt> you can call addmultisig
 79 2015-06-11 07:15:58 <BlueMatt> currently - you can call addmultisig to add a redeemscript to the wallet (this is the only way for the wallet to be aware of a redeemscript, period). you can call importaddress with either an address (which is immediately mangled into a script) or a hex script, which will be watched...
 80 2015-06-11 07:16:22 <BlueMatt> with fundrawtransaction you go through the motions of creating the script in every way, except your signature creator/checker will return true for all sigs and will populate sigs with a string of appropriate size that passes the der checks
 81 2015-06-11 07:16:36 <BlueMatt> in order for that to work, you need to know redeemscripts, and you need to know pubkeys
 82 2015-06-11 07:16:56 <BlueMatt> you could theoretically skip the pubkeys and just hack around it, but that requires some bigger refactors
 83 2015-06-11 07:17:17 <gmaxwell> BlueMatt: ugh. that isn't how that should work.  The creation of the transaction should not be signing, thats a brain damaged approach we have right now, if it's going to be touched it should be fixed.
 84 2015-06-11 07:17:33 <BlueMatt> in any case, an option I added was to importaddress "hex script" "" true, which will ALSO add the p2sh-version to watch addresses and will store the redeemScript
 85 2015-06-11 07:18:06 <BlueMatt> gmaxwell: lol, good luck fixing that code....it calls through solver stuff, which you absolutely need to call to figure out redeemScripts
 86 2015-06-11 07:19:01 <gmaxwell> you do _not_ need to sign, to draft the transaction, making it not sign is not a big deal. You just have to assume the signature will be 74 bytes. The difference there is negligble, less so for compressed/uncompressed though.
 87 2015-06-11 07:19:29 <BlueMatt> gmaxwell: the code i have now obviously doesnt sign (it doesnt have privkeys)....
 88 2015-06-11 07:19:46 <BlueMatt> gmaxwell: but it calls the signing code because it calls through to the Solver stuff
 89 2015-06-11 07:19:59 <BlueMatt> it just uses a transaction signer that creates dummy sigs of the right length
 90 2015-06-11 07:20:02 <gmaxwell> thats nuts. Take the signing code out of the solver.
 91 2015-06-11 07:20:26 <BlueMatt> gmaxwell: huh? thats the only point of the solver stuff
 92 2015-06-11 07:20:35 <BlueMatt> it calls to produce a valid scriptSig
 93 2015-06-11 07:20:59 <BlueMatt> you totally want to go through the same routines because you want to make sure you have the right scriptSig
 94 2015-06-11 07:21:40 <gmaxwell> What I'm saying is that CreateTransaction should not call SignSignature.
 95 2015-06-11 07:21:45 <BlueMatt> gmaxwell: otherwise we get a second codepath in the transaction creation code to do its own Solver stuff
 96 2015-06-11 07:22:05 <BlueMatt> gmaxwell: it needs to know the final size, so I dont see why it shouldnt use the same codepath
 97 2015-06-11 07:22:12 <gmaxwell> (or if it does for not having to change a billion things reasons it should only call it possibly a maximum of once at the very end controlled by a bool or something.
 98 2015-06-11 07:22:17 <BlueMatt> otherwise its duplicated code
 99 2015-06-11 07:23:00 <gmaxwell> BlueMatt: it does not, it should assume the size is 74 in that inner loop. The current design is throughly brain damaged and has bad performance (because if it has to loop many times to increment the key its stupidly signing in the inner loop)
100 2015-06-11 07:23:07 <BlueMatt> gmaxwell: oh, you mean that it shouldnt sign while calculating fees in the inner loop?
101 2015-06-11 07:23:16 <BlueMatt> gmaxwell: oh, yea, well thats easily fixable
102 2015-06-11 07:23:16 <gmaxwell> Yes.
103 2015-06-11 07:23:25 <BlueMatt> hell, in the code Ive got its like a 5 line diff
104 2015-06-11 07:23:25 <gmaxwell> hurrah. Communication! :)
105 2015-06-11 07:23:33 <BlueMatt> but that doesnt change any of the things I was asking about :p
106 2015-06-11 07:23:45 <BlueMatt> you still go through the Solver stuff just dont actually create ecdsa sigs
107 2015-06-11 07:23:55 <BlueMatt> because otherwise you're duplicating a ton of code for no reason
108 2015-06-11 07:24:19 <gmaxwell> not I know you're asking about taking in the pubkey, sure sure. so in the interface you have "importaddress "hex script" "" true,"  where are you planning on taking a pubkey instead?
109 2015-06-11 07:25:38 <gmaxwell> perhaps it should just be split to importpubkey/importscript?  It shouldn't try to detect pubkey vs script because you can easily make one thats valid as the other.
110 2015-06-11 07:26:04 <BlueMatt> yea, i was just thinking you're doing it on purpose if you make such a script
111 2015-06-11 07:26:10 <BlueMatt> but, yea, ok, I'll just make a second rpc
112 2015-06-11 08:47:51 <jonasschnelli> BlueMatt: so the dummy signing was a bad idea?
113 2015-06-11 08:51:00 <jonasschnelli> The dummy signing goes over sign but not over the solver (it bypasses VerifyScript()). I think it's a flexible that is easy to maintain. Otherwise i assume you can't just use 74 bytes. You need at least check the inputs script type.
114 2015-06-11 08:51:21 <jonasschnelli> s/flexible ans is easy to maintain
115 2015-06-11 08:51:26 <jonasschnelli> s/ans/and
116 2015-06-11 08:51:48 <jonasschnelli> s/is/it's (damit german autocorretion!)
117 2015-06-11 09:07:22 <wumpus> gruez: sure, you could use that to allocate memory for private keys - if that is a call that locks memory, you could use it in MemoryPageLocker::Lock, currently it uses VirtualLock to avoid swapping to disk - otherwise it's a matter of rewiring the secure allocator
118 2015-06-11 09:10:45 <LeMiner> Can someone explain me a easy way to understand the code part of promissory note/simulfund?
119 2015-06-11 09:10:53 <LeMiner> I know it works since I use it from time to time...
120 2015-06-11 10:59:08 <aliakbar> Hi! Short question: an inv message is always sent either as an answer to getblocks or unsolicitedly (Source: https://bitcoin.org/en/developer-reference#inv) - what is the frequency of unsolicited inv messages?like in what time intervals are they sent?
121 2015-06-11 11:06:50 <harding> aliakbar: for block messages on new blocks, Bitcoin Core sends them to all of peers as soon as it finishes validating the block (but only to peers that don't send it an inv for that block first)
122 2015-06-11 11:08:28 <harding> aliakbar: if you want to see it in action, Wireshark has a nice Bitcoin filter.  You can start it up and watch what your full node does.  I used it extensively while writing that section of the docs.
123 2015-06-11 11:12:43 <aliakbar> harding: thanks, but the thing is I came up with the question on looking at this post - says inv messages every 100 sec (http://bitcoin.stackexchange.com/questions/3273/when-does-the-main-client-relay-its-knowledge-of-txs-in-inv-message#answer-3277)
124 2015-06-11 11:13:59 <aliakbar> harding: as i'm working on 0.10.0 i thought maybe there's also a timer thing that triggers inv messages but didn't find any other relevant appearance of "SendMessages" in the source code
125 2015-06-11 11:14:52 <aliakbar> harding: except for some "boost nodesignal" but a timer-like structure is not really visible
126 2015-06-11 11:20:02 <harding> aliakbar: I don't know what the current tx inv interval is, although I seem to recall something about it maybe in the 0.10.0 release notes.  Maybe you care to check: https://bitcoin.org/en/release/v0.10.0
127 2015-06-11 11:21:30 <aliakbar> harding: ok thanks I'll read through this. Could you also send me a link to the Wireshark bitcoin filter?
128 2015-06-11 11:22:29 <fanquake> harding Interesting you should bring that up. I was trying to find the 0.10.0 release notes on bitcoin.org today, they seem to be pretty well hidden.. or am I missing some obvious link?
129 2015-06-11 11:22:36 <harding> aliakbar: https://en.bitcoin.it/wiki/Protocol_specification#See_Also but it was provided by default in the Wireshark on Debian.
130 2015-06-11 11:24:02 <harding> fanquake: sadly, they are pretty well hidden---I'm working on a branch that fixes them.  Currently the click trail is: home page, get started, choose your wallet, Bitcoin Core, show version history --- a ridiculous number of clicks.
131 2015-06-11 11:24:35 <harding> fanquake: that last click (show version history) is from the Download page: https://bitcoin.org/en/download
132 2015-06-11 11:25:31 <fanquake> harding Thanks, suspected it was something silly like that. It really needs to be far easier to find.
133 2015-06-11 11:25:46 <fanquake> Good to know your working on it.
134 2015-06-11 11:26:01 <aliakbar> harding: thanks! will this dissector also work on regtest?!? probably not?
135 2015-06-11 11:26:10 <harding> fanquake: agreed.  On the new layout, it'll be: home page -> bitcoin core -> news
136 2015-06-11 11:27:02 <harding> aliakbar: I think it worked on testnet which I think has the same prefix bytes as regtest.  If so, it should work.
137 2015-06-11 11:28:06 <aliakbar> harding: ok i'll just give it a try!
138 2015-06-11 12:16:04 <Diablo-D3> http://www.telegraph.co.uk/news/celebritynews/11666316/christopher-lee-dies-live.html
139 2015-06-11 13:22:35 <rodmar> when running bitcoind with txindex=1 do I have to reindex again if I restart bitcoind?
140 2015-06-11 13:23:05 <hearn> no, that should not be necessary. the index is written to disk.
141 2015-06-11 13:23:41 <rodmar> ok. but what about blocks that I may have missed while the daemon was down?
142 2015-06-11 13:25:53 <hearn> it will catch up when you restart it, as normal for any bitcoin node
143 2015-06-11 13:26:56 <rodmar> ok so having txindex=1 in bitcoin.conf and reindexing once should be enough, right?
144 2015-06-11 13:27:57 <sipa> yes, reindexing once is enough
145 2015-06-11 13:28:20 <rodmar> ok thanks
146 2015-06-11 13:35:39 <hearn> hey sipa
147 2015-06-11 14:21:18 <jonasschnelli> Hi hearn! I was waiting for your.. :)
148 2015-06-11 14:21:31 <hearn> hello! i wish i had been able to make it to your talk the other week
149 2015-06-11 14:21:39 <hearn> unfortunately i had plans for that evening already
150 2015-06-11 14:21:46 <jonasschnelli> I try to get a better understanding of SPV fork/rorg handling...
151 2015-06-11 14:22:40 <jonasschnelli> If a reorg happens, can i just remove all tx (in and out) to the dept where the reorg happend?
152 2015-06-11 14:22:54 <jonasschnelli> s/dept/depth
153 2015-06-11 14:22:55 <hearn> it's a pain in the ass
154 2015-06-11 14:23:08 <hearn> no, you can't just remove them. you have to consider them as pending/awaiting re-inclusion
155 2015-06-11 14:23:18 <hearn> because when you roll forward onto the new chain, not every transaction may be present in the new chain
156 2015-06-11 14:24:19 <jonasschnelli> hearn: but i don't get the point why i have to mark them as pending. Because they are on the in-valid chain and should be removed therefore? not?
157 2015-06-11 14:24:58 <hearn> look at how Core does it:   when a re-org occurs, all transactions are put back into the memory pool. a re-org does not delete transactions
158 2015-06-11 14:25:02 <hearn> (unless they conflict)
159 2015-06-11 14:25:11 <hearn> consider the following chain
160 2015-06-11 14:25:22 <hearn> b1 -> b2 (with my tx) -> b3
161 2015-06-11 14:25:25 <hearn> -> b4
162 2015-06-11 14:25:41 <hearn> now myTX is still unaffected, right? b4 is just a side chain at this point
163 2015-06-11 14:25:55 <hearn> now we see b5->b6 but none of those blocks include myTX
164 2015-06-11 14:26:09 <hearn> what happens is, myTX goes pending/unconfirmed. other miners will put it back in their mempool and try to reinclude it, assuming it's still valid
165 2015-06-11 14:26:16 <hearn> therefore the user interface of the wallet should indicate that
166 2015-06-11 14:26:46 <hearn> does that make sense?
167 2015-06-11 14:27:33 <jonasschnelli> hearn: okay. I think i got the point now. I didn't thought about the tx are going back to the mempool.
168 2015-06-11 14:27:36 <hearn> this is the easy part, by the way :-) the fun really starts when you want to handle double spends and conflicts in the SPV scenario
169 2015-06-11 14:27:57 <hearn> btw don't pay too much attention to the bitcoinj wallet code. it's overly complex for what it does .... when I wrote it, it was back in 2010/2011 and the only reference was Satoshi's code.
170 2015-06-11 14:28:02 <hearn> hey there aschildbach
171 2015-06-11 14:28:17 <aschildbach> hi mike!
172 2015-06-11 14:28:26 <hearn> jonasschnelli: since then the bcj wallet code got more features and got ever more complex. the current code is half way through a rewrite to only track unspent outputs and never deal with transactions directly at all
173 2015-06-11 14:28:29 <jonasschnelli> hearn: okay. Will do. I wonder if i can reuse the blocks/undos from bitcoin-core
174 2015-06-11 14:28:35 <hearn> jonasschnelli: however that transition is not finished
175 2015-06-11 14:28:46 <hearn> jonasschnelli: well in an SPV wallet you do not have full blocks, right? so you cannot reuse that
176 2015-06-11 14:29:14 <hearn> jonasschnelli: you have to record the position in the block chain of each transaction, so when a re-org occurs, you can do the right changes just using the headers
177 2015-06-11 14:29:22 <jonasschnelli> hearn: what if i would keep the last 288 full blocks (don't think of a smartphone bloomfiltering SPV).
178 2015-06-11 14:30:43 <hearn> how is that different to just running a full node with pruning of older blocks?
179 2015-06-11 14:31:01 <hearn> because you wouldn't calculate the utxo set in that case?
180 2015-06-11 14:31:10 <jonasschnelli> hearn: no validation, no cpu usage, only bandwidth.
181 2015-06-11 14:31:25 <hearn> i dunno, never thought about it. i guess that might simplify some things.
182 2015-06-11 14:31:55 <hearn> though if you still have to download the full block chain when you start the node, some users might be surprised if it was being advertised as "SPV"
183 2015-06-11 14:32:02 <hearn> because all existing SPV wallets are much faster than that would be
184 2015-06-11 14:32:08 <hearn> but it could be a useful half-way goalpost
185 2015-06-11 14:32:31 <jonasschnelli> hearn: i have serval modes in mind: a) pruned full node with validation, b) no validation, no bloom filtering = full blocks c) bloom filtering (bitcoinjs way of doing it)
186 2015-06-11 14:33:22 <jonasschnelli> the ETH academics convinced me that SPV with bloomf really hurts the privacy. I don't create use-cased, i just try to give users the option to choose.
187 2015-06-11 14:33:39 <jonasschnelli> Thats why i think a bloom-filter less SPV mode could be the choice for some of the users.
188 2015-06-11 14:33:46 <hearn> well, it "hurts" privacy only in the sense that any approach that isn't "download the entire block chain" hurts privacy :)
189 2015-06-11 14:33:50 <jonasschnelli> I would chose it for my desktop wallet (bloomless SPV)
190 2015-06-11 14:34:44 <hearn> you can always just set a BF with very high FP rate to get that model too, of course. BF with FP rate of 1 == full download
191 2015-06-11 14:34:48 <jonasschnelli> hearn: but i would say that using bloom-filter exposed (in a high chance with current fp rates) my wallet content
192 2015-06-11 14:35:08 <hearn> AFAIK all filtering-capable SPV wallets use very low FP rates currently because users prefer performance to wire privacy
193 2015-06-11 14:35:09 <jonasschnelli> but bloom with height FP could endup in bloomless SPV?
194 2015-06-11 14:35:38 <hearn> but yes obviously it's tunable. the tricky stuff starts when you want a low and controlled FP rate e.g. because of constrained bandwidth, without giving any information away to a long term observer
195 2015-06-11 14:35:43 <jonasschnelli> i mean i see the reason of BF, especially on smartphones. But here, where i'm sitting behind a 150MB line, i don't need bloomf.
196 2015-06-11 14:35:47 <hearn> sure
197 2015-06-11 14:35:53 <hearn> well, you don't :)
198 2015-06-11 14:35:58 <hearn> users are impatient on all devices, in my experience.
199 2015-06-11 14:36:24 <aschildbach> hearn: Any plans for a Bitcoin XT based on 0.11?
200 2015-06-11 14:36:54 <jonasschnelli> but lets say i catch up the last 20 day. 20*3*24 = 1440 blocks = 1GB of data
201 2015-06-11 14:36:55 <tegila> i'm searching for some kind of cheap hardware wallet and not convinced about using this heavy customised for BTC already done..
202 2015-06-11 14:37:00 <hearn> aschildbach: sure, once it's out
203 2015-06-11 14:37:01 <tegila> why not use the ISO 7816-4 cards?
204 2015-06-11 14:37:08 <hearn> aschildbach: it'll take a while to rebase everything though
205 2015-06-11 14:37:14 <tegila> it can store 4 RSA keys with safety
206 2015-06-11 14:37:43 <hearn> jonasschnelli: yeah exactly. 1GB of data is quite a bit! user will be waiting for quite some time to see their latest balance, if they do not have their wallet open regularly
207 2015-06-11 14:38:07 <hearn> jonasschnelli: in the mobile context, we sync the chain at night in the background to keep things snappy when users open up the app. aschildbach can tell more about these experiments.
208 2015-06-11 14:38:26 <hearn> jonasschnelli: you could perhaps use the same approach
209 2015-06-11 14:39:00 <aschildbach> hearn: It's not at night. It's a degressive schedule.
210 2015-06-11 14:39:06 <jonasschnelli> hearn: Right. 1GB is to much on a smartphone, but no problem on a standard line in our region.
211 2015-06-11 14:39:28 <hearn> aschildbach: right. i thought we used to have logic in there that preferred to sync when charging and on wifi?
212 2015-06-11 14:39:34 <aschildbach> We dropped the "at night", on WLAN, on a charger approach when we introduced bloom filters.
213 2015-06-11 14:39:34 <jonasschnelli> hearn: But thanks. I now got the point of keeping the position of a tx in a block in case of a reorg.
214 2015-06-11 14:39:41 <hearn> ah ok
215 2015-06-11 14:39:45 <hearn> i thought we still used it
216 2015-06-11 14:39:52 <aschildbach> No, not really worth it
217 2015-06-11 14:39:53 <hearn> the android scheduler framework should approximate the same thing, if we used it, i think
218 2015-06-11 14:40:26 <hearn> because we can register with the OS to say "we have low priority intensive work to do, please batch me up with other apps that want to sync things", right?
219 2015-06-11 14:40:51 <aschildbach> We are using the INEXACT constants when it comes to rescheduling.
220 2015-06-11 14:41:04 <aschildbach> The docs says it will queue that with other activity, yes.
221 2015-06-11 14:41:07 <hearn> cool
222 2015-06-11 14:41:13 <hearn> that's good enough then. android takes care of it for us
223 2015-06-11 14:41:45 <aschildbach> However I'm concerned that we will get problems with the new doze stuff (Android M)
224 2015-06-11 14:42:09 <aschildbach> apparently you're now forced to send keep-alive-messages via Google Cloud messaging
225 2015-06-11 14:43:49 <aschildbach> hearn: How did the issue migration come out?
226 2015-06-11 14:44:12 <hearn> even with the --rate_limit flag, it got rate limited and died. i contacted github support to ask them to turn off the rate limits for a bit
227 2015-06-11 14:44:22 <hearn> aschildbach: well we shouldn't sync when dozing anyway
228 2015-06-11 14:44:27 <hearn> aschildbach: that's kind of the point of dozing :)
229 2015-06-11 14:44:45 <hearn> aschildbach: i guess the phone will not doze when charging
230 2015-06-11 14:44:47 <aschildbach> hearn: How are we supposed to receive payments then?
231 2015-06-11 14:45:01 <aschildbach> let's hope at least that's the case
232 2015-06-11 14:45:14 <hearn> open the app? i mean, we already sync only every few hours, right? so it's not like you receive payments in real time with the current design anyway
233 2015-06-11 14:45:32 <hearn> for that we'd need a relay from bitcoin p2p to GCM, or do our own always-on connections that listen for transactions (ugh)
234 2015-06-11 14:45:53 <aschildbach> Of course we also want to receive payments without user intervention!
235 2015-06-11 14:46:11 <hearn> yes but it only happens eventually with the current app, dozing wouldn't change that
236 2015-06-11 14:47:01 <hearn> i think at some point we'll need to run a GCM relay that accepts bloom filters and basically applies the same logic to tell the app when to wake up and do a chain sync, if we want insta-payments. either that OR a Subspace-esque system where the servers that receive BIP70 payments know how to do a GCM submission
237 2015-06-11 14:47:23 <aschildbach> How do you mean eventually? It depends on how many clients the user has.
238 2015-06-11 14:47:48 <aschildbach> For example, I got donations every other hour when I was allowed to get them.
239 2015-06-11 14:47:53 <hearn> i mean if i send you a payment right now to a stored address, your Bitcoin Wallet won't immediately make the jingle sound, will it? it'll make the sound and notify you next time it's scheduled to wake up and sync
240 2015-06-11 14:48:17 <aschildbach> GCM is closed source.
241 2015-06-11 14:48:18 <hearn> yeah exactly - but those tips aren't so urgent you need to know the second they are sent. it's ok for notification to be delayed a bit.
242 2015-06-11 14:48:23 <aschildbach> And centralized.
243 2015-06-11 14:48:24 <hearn> well GCM is a service, not a piece of software.
244 2015-06-11 14:48:40 <hearn> so being open source wouldn't achieve much. you still wouldn't be able to run it yourself.
245 2015-06-11 14:49:54 <aschildbach> Linking closed source code into the app is out of question.
246 2015-06-11 14:50:20 <aschildbach> If there is an alternative OSS implementation, let me know.
247 2015-06-11 14:50:32 <tegila> It's possible to do a Hardware Wallet using a FunCard Smartcard?
248 2015-06-11 14:50:36 <hearn> *shrug* i'd disagree, you already rely on closed source radio firmwares and so on. the GCM client library is just a thin shim that listens for intents anyway, i thought
249 2015-06-11 14:50:47 <hearn> but there won't be an open/decentraised GCM anytime soon
250 2015-06-11 14:50:51 <hearn> that's a challenge for some other time
251 2015-06-11 14:51:06 <aschildbach> That's fine with me, just what I expected.
252 2015-06-11 14:51:17 <hearn> tegila: some people have made smartcard based wallets, but not using that card iirc. however the best hardware wallets have screens.
253 2015-06-11 14:51:33 <hearn> tegila: a hardware wallet with no trusted display is not that useful, as the malware can make it sign whatever it wants and you would never know ...
254 2015-06-11 14:51:35 <tegila> they are expensive
255 2015-06-11 14:51:44 <hearn> yes, they are
256 2015-06-11 14:51:54 <tegila> today we have multi-sign wallets
257 2015-06-11 14:52:06 <tegila> so we can do a smartphone + smart card signing
258 2015-06-11 14:52:10 <aschildbach> The radio firmware is sandboxed afaik.
259 2015-06-11 14:52:17 <tegila> don't need for displays in the wallet
260 2015-06-11 14:53:09 <tegila> or even we can do a PINPAD that print a report of what is the balance of the wallet
261 2015-06-11 14:53:46 <hearn> aschildbach: yes, i think so
262 2015-06-11 14:53:47 <tegila> i'm trying to encourage some shops to use bitcoin
263 2015-06-11 14:53:57 <tegila> but need be fast-forward
264 2015-06-11 14:54:11 <hearn> aschildbach: if the only concern is that google might put wallet-stealing code into the GCM client library, disassembling it should be sufficient to resolve that doubt, i'd guess. it should be reimplementable anyway given that people have to statically link it
265 2015-06-11 14:54:36 <hearn> tegila: you do need a display. otherwise the wallet doesn't really improve security as much as you may expect
266 2015-06-11 14:54:55 <hearn> tegila: nothing stops a virus waiting until you want to send some money, and then rewriting the request sent to the card from "send 1 BTC to the shop" to "send all BTC to virus author"
267 2015-06-11 14:54:59 <aschildbach> hearn: I tried disassembling half a year ago, but it was heavily obfuscated.
268 2015-06-11 14:55:10 <hearn> tegila: with no screen the user cannot tell and will press the button to confirm both
269 2015-06-11 14:55:38 <tegila> hearn, is it can happen with VISA also
270 2015-06-11 14:55:46 <aschildbach> That said, on the Droidcon a week ago I learned of a powerful tool against obfuscation. A friend acquired a license and we'll try again.
271 2015-06-11 14:56:45 <tegila> hearn, if this work that way no credit card PINPAD machine will be working for all this years
272 2015-06-11 14:57:05 <tegila> in poor countries the bitcoin is not developing as it can
273 2015-06-11 14:57:08 <hearn> tegila: credit cards rely on the assumption of trusted merchant hardware that is tamper resistant.
274 2015-06-11 14:57:26 <hearn> tegila: i.e. the *receiver* provides trusted hardware. with bitcoin you are sending over the internet, the receiver may not have any screen to give you
275 2015-06-11 14:57:52 <hearn> tegila: in poor countries the right solution is SMS based 2-factor, i think, or just using smartphones
276 2015-06-11 14:57:52 <tegila> hearn, in hardware wallets you trust the merchant as well
277 2015-06-11 14:57:54 <hearn> tegila: you probably don't need a full blown hardware wallet for individual users if they use a decent smartphone
278 2015-06-11 14:58:04 <tegila> hearn, imagine dummies trying to make his payments sending SMS...
279 2015-06-11 14:58:16 <hearn> tegila: no *receiving* sms from some third party auth service
280 2015-06-11 14:58:19 <hearn> e.g. bitgo
281 2015-06-11 14:59:26 <tegila> its not working that way
282 2015-06-11 14:59:48 <tegila> dummies don't care about technology specification
283 2015-06-11 15:00:25 <tegila> and not all of them have a smartphone
284 2015-06-11 15:00:29 <hearn> aschildbach: https://github.com/ayltai/Android-Lib-GCM/tree/master/GCM
285 2015-06-11 15:00:51 <hearn> in india
286 2015-06-11 15:00:51 <hearn> tegila: yes people with no smartphone are out of luck presently. but that's ok. smartphones are getting so cheap, they can soon afford one. now you can buy one for just $30
287 2015-06-11 15:01:02 <tegila> hearn, a funcard can provide security for private key
288 2015-06-11 15:01:41 <tegila> you can put and algo inside and they will assign the transaction for you
289 2015-06-11 15:01:48 <hearn> if you want to re-create the credit card security model with bitcoin, go right ahead, but you'll be starting from scratch and by the time you get anywhere smartphones will have sold a few more hundred million units
290 2015-06-11 15:01:56 <hearn> so imo it's not worth it
291 2015-06-11 15:02:11 <tegila> you see the transaction in the PINPAD machine and provide your PIN to unlock the sign-alto
292 2015-06-11 15:02:48 <aschildbach> hearn: Thanks, doesn't look like it implements GCM. It just seems to *use* it.
293 2015-06-11 15:02:59 <hearn> yes, it's the client code you link into your app
294 2015-06-11 15:03:03 <tegila> hearn, I don't trust smartphone-online devices running foolish software to put my money in a file there
295 2015-06-11 15:03:11 <hearn> so the bit that shares the address space with your process
296 2015-06-11 15:03:30 <hearn> i thought that was the issue? gcm has a kind of daemon thing that runs in the background all the time too, but that is not open source indeed. however it has no privileged access to your app
297 2015-06-11 15:03:54 <hearn> tegila: ok .... that's fine. but then you will have to trust (in your model) some random box that is given to you by the merchant.
298 2015-06-11 15:04:02 <aschildbach> hearn: It imports com.google.android.gms.common.* and that code is missing.
299 2015-06-11 15:04:10 <aschildbach> So it won't run without linking stuff.
300 2015-06-11 15:04:11 <hearn> aschildbach: just for some constants, it seems?
301 2015-06-11 15:04:18 <hearn> aschildbach: they could just be copied. i think it's just intent names
302 2015-06-11 15:04:26 <hearn> anyway i haven't looked in depth
303 2015-06-11 15:04:35 <tegila> hearn, not a random box, if the merchant start robbing his consumers He will be jail
304 2015-06-11 15:04:37 <hearn> it should not be hard to re-build the client shim anyway.
305 2015-06-11 15:05:02 <tegila> as the way it happens today with credit card machines
306 2015-06-11 15:05:18 <hearn> tegila: yes to do that model you would need to basically become a new VISA/bank and do ID verification of merchants, get them to adopt your magic box, issue people with bitcoin cards, etc...... it's a totally different technical model to how bitcoin works today.
307 2015-06-11 15:05:21 <tegila> even TV-STB that are heavy secure have some ways to get inside it
308 2015-06-11 15:05:40 <aschildbach> GoogleCloudMessaging.getInstance(GcmRegistrar.this.context).register(params[0]);
309 2015-06-11 15:05:50 <hearn> tegila: and BTW the credit card machines don't have the ability to withdraw arbitrary funds. they just relay txn data to the trusted third party.
310 2015-06-11 15:06:04 <hearn> aschildbach: oh, ok
311 2015-06-11 15:06:14 <hearn> aschildbach: darn. it looked like it was replacing most stuff
312 2015-06-11 15:06:16 <aschildbach> And a couple of others
313 2015-06-11 15:06:17 <tegila> hearn, not totally different, it still use the blockchain technology
314 2015-06-11 15:07:05 <tegila> i'm planning use the Atmega inside this cards to program an signing algo who don't give the PINPAD the access to the private-key
315 2015-06-11 15:07:37 <tegila> and searching for help
316 2015-06-11 15:08:48 <tegila> the signing algorithm is very exclusive for bitcoin system
317 2015-06-11 15:09:26 <tegila> my doubt is if i can use the already existed cards that store a RSA key to store a bitcoin-key
318 2015-06-11 15:09:32 <tegila> mostly sure i can't
319 2015-06-11 15:13:47 <hearn> brb
320 2015-06-11 15:33:30 <flound1129> are wallets disabled with prune in rc1?
321 2015-06-11 15:43:31 <sipa> yes
322 2015-06-11 16:03:16 <wumpus> in master it's enabled now (so will be in 0.12) - but be careful, it's experimental
323 2015-06-11 18:28:03 <BlueMatt> jonasschnelli: yea, the dummy signing was broken...wasnt strictly a "bad idea" but missed some stuff, and sipa's suggestion made the issue visible :)
324 2015-06-11 18:28:47 <jonasschnelli> BlueMatt: okay. what was broken?
325 2015-06-11 18:29:30 <BlueMatt> when creating the scriptSig to check the size of the tx, instead of pushing the pubkey and the sig (where the sig was 7X 0-bytes), it pushed the constant 0 and the sig (where the sig was 7X 0-bytes)
326 2015-06-11 18:29:38 <BlueMatt> so it was short $SIZE_OF_PUBKEY in size
327 2015-06-11 18:39:39 <jonasschnelli> BlueMatt: but this should have been got detected over the rpc test at: https://github.com/jonasschnelli/bitcoin/commit/5ef0b3306a4293832a23e2390972aa36d1d3e112#diff-87236049b3bc83377a2cffc06a5f4aa6R291  and i assumed it always took 72 bytes per sig (https://github.com/jonasschnelli/bitcoin/commit/5ef0b3306a4293832a23e2390972aa36d1d3e112#diff-d546634098d8003b6397c8643b74e47dR26)?
328 2015-06-11 18:56:48 <BlueMatt> jonasschnelli: hmm, well all I can say is that test doesnt work and the sig is 72 bytes, but you also have to push the pubkey to pay-to-pubkeyhash, and when GetPubKey fails, it just pushes OP_0
329 2015-06-11 18:58:23 <jonasschnelli> BlueMatt: hmm.. but in this solution (https://github.com/jonasschnelli/bitcoin/commit/5ef0b3306a4293832a23e2390972aa36d1d3e112#diff-d546634098d8003b6397c8643b74e47dR24) GetKey() never gets called because the idea of the dummy sign is that you just use it to calculate the signature size. Or did you use a different dummy singe code then that one?
330 2015-06-11 18:59:30 <BlueMatt> jonasschnelli: no, in SignStep case TX_PUBKEYHASH: first it pushes the pubkey, then it calls Sign1
331 2015-06-11 19:01:21 <jonasschnelli> BlueMatt: ah. Damit. Right (https://github.com/jonasschnelli/bitcoin/blob/5ef0b3306a4293832a23e2390972aa36d1d3e112/src/script/sign.cpp#L96). This also needs a pad/add of 72 bytes.
332 2015-06-11 19:01:48 <jonasschnelli> no. not 72. 32. sry
333 2015-06-11 19:05:02 <BlueMatt> not 32 either, 33 usually, but not always
334 2015-06-11 19:05:42 <sipa> 33 or 65
335 2015-06-11 19:05:52 <sipa> depending on compressed pubkey or not
336 2015-06-11 19:06:13 <jonasschnelli> Right.
337 2015-06-11 19:13:40 <jonasschnelli> BlueMatt: so the missing 33 bytes did influenced the fee calculation so that the transaction got no longer broadcaster/relayed?
338 2015-06-11 19:19:09 <BlueMatt> jonasschnelli: no, i didnt see it effect the fee, nor did I test that, but it certainly could effect the fee
339 2015-06-11 19:19:21 <BlueMatt> and it would reliably result in lower fee than you want
340 2015-06-11 19:20:46 <jonasschnelli> agreed. 33 (or 65) bytes could affect the fee in a unexpected way and might result in not getting accept into the mempool.
341 2015-06-11 19:21:52 <BlueMatt> anyway, Ive got a branch nearly together that should work
342 2015-06-11 19:24:02 <jonasschnelli> BlueMatt: Cool. And sorry for missing that thing during the dummy sign implementation.
343 2015-06-11 19:24:20 <BlueMatt> lol, i missed it too
344 2015-06-11 19:24:35 <BlueMatt> it took me a while of debugging to even find it when i had code that was failing because of it :p
345 2015-06-11 19:25:46 <jonasschnelli> Indeed hard to debug! but how did you detect this? I mean 33/1024 is a very low fee delta. And there is also a tiny delta if you assume every sig is 72 bytes.
346 2015-06-11 21:04:16 <qpayct> hi
347 2015-06-11 21:37:58 <Luke-Jr> I'm pondering if there should be an option to define the policy soft limit based on time. eg, -blockavgmaxsize=400 to use 400k for 10 minutes, 200k for 5 minutes, and 800k for 20 minutes (etc). Thoughts?
348 2015-06-11 21:38:23 <Luke-Jr> IMO this seems like a sensible option for miners to use regardless of the whole hard limit discussion
349 2015-06-11 21:38:59 <gmaxwell> Luke-Jr: you mean in terms of time since last block?
350 2015-06-11 21:39:02 <Luke-Jr> yes
351 2015-06-11 21:39:12 <Luke-Jr> perhaps using the seen-time to avoid DoS risks
352 2015-06-11 21:39:22 <gmaxwell> yea, I assumed seen time.
353 2015-06-11 21:42:54 <Luke-Jr> gmaxwell: seem like a good idea?
354 2015-06-11 21:48:41 <gmaxwell> Luke-Jr: sure, though it should also apply some correction based on the network behavior over the last 24 hours. E.g. if the network is below your target you should be bigger, or above you should be smaller.
355 2015-06-11 21:48:51 <gmaxwell> Luke-Jr: well perhaps what it should be is the lowest level is set so that the average size over 24 hours is measured. And the difference between your designed size and that size is computed.  then your target is linearly adjusted based on the wait time;  e.g. target = wait*target/10. And then the value you use is.. target + within(0.5*target,(that_value/144),2*target).  e.g.
356 2015-06-11 21:50:10 <Luke-Jr> gmaxwell: eh, so if other miners have a higher target you're under-mining? not quite what I was thinking..
357 2015-06-11 21:50:35 <Luke-Jr> eg, if other miners are bloating their blocks with spam, you don't want to start making empty ones
358 2015-06-11 21:53:24 <gmaxwell> Luke-Jr: thats why it should be limited.
359 2015-06-11 21:53:56 <gmaxwell> Luke-Jr: e.g. target + within(0.5*target,(24hour_error_value/144),2*target).
360 2015-06-11 21:54:01 <Luke-Jr> near-empty* :p
361 2015-06-11 21:54:22 <gmaxwell> so at most you'd make a block 50% smaller than your target (at 10 minutes) or 3x larger (at 10 minutes).
362 2015-06-11 21:54:57 <Luke-Jr> right, my disagreement is mainly with other miners influencing your own policy
363 2015-06-11 21:56:18 <gmaxwell> well is your policy "how fast should the chain be growing" vs just "how big this single block I make is", I think the policy encompasses both of these factors.
364 2015-06-11 21:56:37 <gmaxwell> To the extent that its the first one, you want some correction based on existing behavior.
365 2015-06-11 22:00:48 <Luke-Jr> the former might work, if you kept track of what size those blocks would be if you had mined them; but that's even more complex, and the alternative seems to be problematic IMO
366 2015-06-11 22:01:31 <Luke-Jr> evem if you're making 50%-size blocks, the other (possibly hostile) miners have reduced your influence and fees to 50% too
367 2015-06-11 23:42:25 <dgenr8> isn't it a question of building a cost function for including the next-lowest fee paying tx, and cutting off the flow when cost > fee?
368 2015-06-11 23:50:04 <gmaxwell> dgenr8: if you do not assume block relay protocol the answer is that you'd likely include no transactions at all under that model. And if you do assume it, the cost is completely negligible.
369 2015-06-11 23:59:20 <dgenr8> gmaxwell: well, block relay protocol exists, so I guess it's fill-em-up