1 2014-06-23 00:00:44 <dekalo> ok, my fault
2 2014-06-23 00:00:47 <dekalo> :)
3 2014-06-23 00:02:19 <dekalo> just learned that same privk could results in 2 different WIF keys with 2 different final addresses
4 2014-06-23 00:02:50 <jcorgan> yep
5 2014-06-23 00:03:12 <dekalo> the difference reside in the pubkey format so, that is the one hashed and pt in scriptPubKey
6 2014-06-23 00:03:15 <dekalo> right
7 2014-06-23 00:03:17 <dekalo> ?
8 2014-06-23 00:03:59 <jcorgan> didn't quite understand that
9 2014-06-23 00:04:18 <jcorgan> each privkey can generate either a compressed or uncompressed pubkey
10 2014-06-23 00:04:26 <jcorgan> which each hash to a different address
11 2014-06-23 00:04:45 <jcorgan> so you indicate in the WIF which one you want generated when you import it
12 2014-06-23 00:05:11 <dekalo> ok, seems that bitcoinj standard use compressed
13 2014-06-23 00:05:53 <jcorgan> i wish all address generating software did, apparently some did not get the memo two years ago
14 2014-06-23 00:07:44 <dekalo> bitcoinj do it as i see, nice
15 2014-06-23 00:09:54 <sipa> from bitcoin's perspective, a public key is identified by its serialization
16 2014-06-23 00:10:25 <sipa> that means that to retain the 1:1 correspondence between private keys and public keys, private keys must "know" whether their corresponding public key is to be compressed or not
17 2014-06-23 00:31:52 <dekalo> thank you sipa for explaination
18 2014-06-23 00:32:07 <dekalo> i'm still a bit confused
19 2014-06-23 00:34:24 <ahmed_> does anyone here know how i can set the coinbase tags in eloipool?
20 2014-06-23 00:38:32 <sipa> dekalo: try in testnet, and paste your code
21 2014-06-23 00:40:52 <embicoin> are you planning to upgrade bdb version in bitcoin?
22 2014-06-23 00:41:22 <sipa> no
23 2014-06-23 00:41:24 <embicoin> I know the compatibility thing, just asking
24 2014-06-23 00:41:25 <embicoin> ok
25 2014-06-23 00:44:03 <embicoin> I must reprogram my mind to stop thinking newest versions are better :P
26 2014-06-23 00:45:54 <sipa> yes, you should
27 2014-06-23 00:46:02 <sipa> we basically use 0 features bdb offers
28 2014-06-23 00:51:55 <maaku> embicoin: we are planning to get rid of bdb, eventually
29 2014-06-23 00:52:44 <justanotheruser> what db will be used? One from scratch?
30 2014-06-23 00:53:04 <sipa> a simple append-only key-value store
31 2014-06-23 00:53:09 <sipa> no need for a db
32 2014-06-23 00:59:55 <dekalo> sipa: i've done some test in testnet and it works, from privkey generated with ECKey package 87807d633e347fe1b9fce7b25fc79af9a90b93df40d6ad11304bbea05e880756 it generates WIF: 92cbNrtQVeYLwDobhM9eXZNMzynsXgyMYiwuPwJzEnupocabtUb(uncompr), cS86jVBthShLzKWg95wAdssHgufU2TRjh29LGEiPdse9zczgyvbQ(compr). The address obtained from privkey with toAddress() is mk5wtFGNgAdo1NgR5TBNHzsT284Crm1B2a, the same showed from bitcoind after i've used im
33 2014-06-23 00:59:56 <dekalo> portprivkey with compr-key as input
34 2014-06-23 01:00:16 <lechuga_> everything will just use leveldb?
35 2014-06-23 01:00:26 <sipa> lechuga_: no, wallet will not
36 2014-06-23 01:00:38 <sipa> leveldb is overkill for the wallet file
37 2014-06-23 01:02:12 <dekalo> i want to be sure to obtain the same pubkeys when I generate a key with ECKey and i call getPubKey and when i get it from validateaddress in bitcoind. I'm doing some thing with p2sh, so I need to do this :)
38 2014-06-23 03:01:34 <rebroad> Hi.. quick question.. I'm trying to CAddrMan::Add, and I currently have a variable "addr" which is a CService. How do I pass this to Add please?
39 2014-06-23 06:31:45 <Krellan> Nice
40 2014-06-23 06:32:05 <Krellan> 0.9.2 had a bad bug on my Mac with the fonts
41 2014-06-23 06:32:10 <Krellan> 0.9.2.1 has fixed it
42 2014-06-23 06:32:19 <Krellan> Thanks
43 2014-06-23 06:45:27 <wumpus> Krellan: yes, seemingly no one with MacOSX 10.9 tested the RC, so the font issue only came to light in the release
44 2014-06-23 07:01:19 <Krellan> wumpus: Yikes, good to know. Will have to pay more attention to the RC's when they come out.
45 2014-06-23 10:04:26 <jtimon> in rpcmining.cpp in function getblocktemplate pblock->nBits is used for "bits" and "target" returned fields, but it is only set to anything different than 0 in the testnet case (inside UpdateTime)
46 2014-06-23 10:04:51 <jtimon> does it really need to be set in that case or a 0 will do it like in the other cases?
47 2014-06-23 10:05:39 <sipa> yes, as updating the time may change the difficulty for testnet
48 2014-06-23 10:05:48 <sipa> because of the 20-minute rule
49 2014-06-23 10:06:10 <jtimon> pblock->nNonce is also set to 0 but not used (it was 0 from the constructor)
50 2014-06-23 10:07:19 <jtimon> sipa: but unless I'm missing something getblocktemplate will always return nBits = 0 for the mainnet and regtest cases, why would a 0 be wrong in testnet?
51 2014-06-23 10:09:05 <sipa> that would be a serious bug
52 2014-06-23 10:10:03 <sipa> it works fine here on my head-as-of-a-few-weeks-ago instance
53 2014-06-23 10:11:52 <jtimon> I'm circling on "pblock->nBits" and "pblocktemplate" after PR #4100 and nBits doesn't seem to be set to GetNextWorkRequired(pindexPrev, pblock) anywhere (besides, as said, UpdateTime in testnet case)
54 2014-06-23 10:12:39 <jtimon> well, I'm at yesterday's head
55 2014-06-23 10:13:03 <jtimon> origin/master master 6b40eabb Merge pull request #4381
56 2014-06-23 10:14:59 <sipa> maybe a recently introduced bug
57 2014-06-23 10:20:04 <jtimon> I'll review the history of the file
58 2014-06-23 10:21:47 <sipa> 4100 changed quite some things, it may be the offender
59 2014-06-23 10:22:55 <michagogo> Gah, I need to remember to rebase my PR when I'm at my computer
60 2014-06-23 10:23:29 <michagogo> To do that I just use `git rebase upstream/master`, right?
61 2014-06-23 10:23:57 <michagogo> (with the PR branch checked out)
62 2014-06-23 10:26:47 <jtimon> sipa, that's what I thought, but it seems previous to merging #3824
63 2014-06-23 10:31:01 <jtimon> seems quite old, in 450cbb09 Ultraprune rpcmining.cpp didn't contained any set on pblock->nBits/pblocktemplate->nBits
64 2014-06-23 10:31:24 <sipa> are you actually calling the getblocktemplate RPC and looking at the result?
65 2014-06-23 10:31:43 <jtimon> are we 100% sure that the nBits must be set on rpcmining.cpp?
66 2014-06-23 10:31:52 <jtimon> sipa, no, just looking at the code
67 2014-06-23 10:32:35 <sipa> i don't see nBits being set in rpcminining
68 2014-06-23 10:36:24 <sipa> jtimon: i don't really have an idea of what code you're looking at
69 2014-06-23 10:36:30 <sipa> the internal miner, or rpc?
70 2014-06-23 10:36:43 <sipa> are you wondering about the code path that sets nBits in the block being constructed>
71 2014-06-23 10:36:49 <jtimon> so maybe it's not necessary at all to do it there, neither in the testnet case
72 2014-06-23 10:36:59 <sipa> in where?
73 2014-06-23 10:37:09 <jtimon> yep, I'm wondering about htat code path
74 2014-06-23 10:37:13 <michagogo> Hm, I wonder what's identifying itself as /Snoopy:0.1/
75 2014-06-23 10:37:14 <jtimon> in rpcmining.cpp
76 2014-06-23 10:37:14 <sipa> which?
77 2014-06-23 10:37:18 <sipa> ok
78 2014-06-23 10:37:22 <sipa> in getblocktemplate?
79 2014-06-23 10:37:34 <jtimon> yep
80 2014-06-23 10:37:37 <michagogo> Looks like address is 192.33.90.253
81 2014-06-23 10:37:53 <jtimon> I can't find any set to nBits besides UpdateTime
82 2014-06-23 10:38:14 <upb> descr: ETHZ, Swiss Federal Institute of Technology Zurich
83 2014-06-23 10:38:14 <upb> descr: Zurich, Switzerland
84 2014-06-23 10:38:17 <upb> heh
85 2014-06-23 10:38:44 <michagogo> upb: interesting
86 2014-06-23 10:39:03 <michagogo> I have 3 peers with the subver "/ETHZBitshark:0.9.1/",
87 2014-06-23 10:39:13 <GAit> I think I know one of the guys that heads that group
88 2014-06-23 10:40:22 <sipa> jtimon: CreateNewBlock calls UpdateTime
89 2014-06-23 10:40:37 <michagogo> I don't see snoopy in getpeerinfo, just in version messages in the log
90 2014-06-23 10:41:59 <michagogo> Hm, why does getaddr.bitnodes.io claim to have 290000 blocks?
91 2014-06-23 10:42:27 <sipa> jtimon: and calls GetNextWorkRequired directly too
92 2014-06-23 10:42:33 <GAit> actually, i am not sure anymore - maybe some other institute
93 2014-06-23 10:44:38 <michagogo> Whoa
94 2014-06-23 10:44:41 <michagogo> Massive flood of ERROR: AcceptToMemoryPool : inputs already spent
95 2014-06-23 10:48:01 <jtimon> sipa oh, sorry, yeah, getblocktemplate calls CreateNewBlock
96 2014-06-23 10:48:01 <jtimon> Then I don't see the purpose of the additional call to UpdateTime ant to set nNonce to zero...
97 2014-06-23 10:48:25 <jtimon> maybe I can just delete that
98 2014-06-23 10:50:35 <hearn> good morning
99 2014-06-23 10:50:54 <sipa> jtimon: ah, now i see what you mean; yes, that looks safe
100 2014-06-23 10:54:11 <jtimon> cool, thanks
101 2014-06-23 10:58:52 <hearn> sipa: would still be interested in your thoughts on the spv floating fee estimator algorithm i posted about, if you have time
102 2014-06-23 11:04:29 <sipa> hearn: haven't read through the entire idea yet, but an attacker can just give you "return true" utxo's, which will always validate
103 2014-06-23 11:04:44 <sipa> hearn: not a comment on the whole proposal, but the claim "thus authenticating it" isn't valid
104 2014-06-23 11:04:51 <hearn> the script is covered in the signature hash
105 2014-06-23 11:04:57 <hearn> as in, the correct script
106 2014-06-23 11:05:12 <hearn> i thought i mentioned this in the proposal. maybe finish reading it first?
107 2014-06-23 11:06:55 <sipa> if the output being spent has no checksig, there is no signature, though as long as we rely on no such transaction being standard relayable, that is not a problem
108 2014-06-23 11:08:00 <hearn> any such outputs are stealable anyway, unless we assume complicated new additions to script
109 2014-06-23 11:08:07 <hearn> so that is an edge case that doesn't worry me much
110 2014-06-23 11:08:09 <sipa> agree, let's ignore that problem
111 2014-06-23 11:08:34 <Luke-Jr> wow, this describes a common thing http://mywiki.wooledge.org/XyProblem
112 2014-06-23 11:14:39 <hearn> sipa: so do you agree that it actually is retroactively authenticated, for transactions that require a signature?
113 2014-06-23 11:14:51 <hearn> for *outputs* that require a signature
114 2014-06-23 11:15:03 <gdm85> morning everybody
115 2014-06-23 11:15:05 <sipa> hearn: the script, yes
116 2014-06-23 11:15:12 <sipa> hearn: but it's the amount you're interested in
117 2014-06-23 11:15:27 <hearn> indeed, which is why I said it also needs the value to be covered by the signature hash as well.
118 2014-06-23 11:15:27 <sipa> and yes, you know the that the spending transaction is valid, and used the right input amounts
119 2014-06-23 11:15:31 <gdm85> nice to see discussion on mailing list about wallet separation
120 2014-06-23 11:15:32 <sipa> because of that
121 2014-06-23 11:15:50 <hearn> however, once that task is done, getutxo doesn't need any changes as the value is already included.
122 2014-06-23 11:16:01 <gdm85> wumpus: online?
123 2014-06-23 11:16:01 <sipa> right
124 2014-06-23 11:17:34 <wumpus> sipa: https://github.com/bitcoin/bitcoin/pull/4198/files would make nLastBlockProcess update only when receiving a block; in principle it makes sense I think?
125 2014-06-23 11:17:36 <sipa> hearn: so, what you really need is authenticated information about transactions' spent coins values?
126 2014-06-23 11:17:50 <sipa> wumpus: i tried that first, and it disconnects everyone :)
127 2014-06-23 11:18:15 <wumpus> ok...
128 2014-06-23 11:18:26 <sipa> wumpus: though the current solution is certainly suboptimal
129 2014-06-23 11:18:37 <hearn> sipa: no. for my current app what i need is getutxo. but later if/when we implement more decentralised floating fee logic, the same command will be useful then too.
130 2014-06-23 11:18:47 <sipa> hearn: right, i'm talking about this use case
131 2014-06-23 11:18:52 <sipa> not about lighthouse
132 2014-06-23 11:18:54 <hearn> right
133 2014-06-23 11:19:11 <hearn> yes, what we actually need to know is the fee that was paid for arbitrary spent transactions
134 2014-06-23 11:19:38 <sipa> i did hear another idea about fee logic for SPV
135 2014-06-23 11:19:43 <hearn> a simpler/more direct method would be for tx3 transactions to redundantly specify their fee size in the serialized format
136 2014-06-23 11:19:43 <wumpus> I suppose it shouldn't disconnect all nodes it is not receiving blocks from, probably only if it is the node it has requested blocks from
137 2014-06-23 11:19:57 <sipa> wumpus: right, i was exaggerating
138 2014-06-23 11:19:58 <hearn> however that bloats up the block chain and introduces potential for divergence. it may still be a better solution though
139 2014-06-23 11:20:19 <sipa> wumpus: the problem is that block processing can take a really long time if orphans are being reconnected
140 2014-06-23 11:20:24 <sipa> wumpus: which can trigger the timeout
141 2014-06-23 11:20:35 <hearn> but given we want to put value under the sighash anyway, for TREZOR which also wants to know the fee, it seems best to just reuse that
142 2014-06-23 11:20:49 <wumpus> sipa: ah
143 2014-06-23 11:21:07 <sipa> hearn: so, no objection at all to including the value in the sighash; adding it to transactions... less sure
144 2014-06-23 11:21:23 <hearn> right. if we change the sighash rule there is no need to change tx serialization format.
145 2014-06-23 11:21:38 <hearn> which i'd prefer, because all existing software that parses txns but doesn't sign them (i guess this is a lot of small tools and libraries) can work as-is
146 2014-06-23 11:21:47 <sipa> exactly
147 2014-06-23 11:21:53 <sipa> it's just a script update
148 2014-06-23 11:22:01 <sipa> which SPV nodes already don't care about
149 2014-06-23 11:22:25 <hearn> right, mostly. though note that bitcoinj can run scripts and in fact lighthouse does run them
150 2014-06-23 11:22:38 <hearn> (it runs the scripts fetched from getutxo)
151 2014-06-23 11:23:48 <sipa> however, what about this idea: when creating a block, a miner writes its minimum fee-per-byte used in a block (plus maybe some other constants relating to priority) to the coinbase, and blocks are invalid if they include a transaction with lower fee/byte
152 2014-06-23 11:24:02 <petertodd> sipa: why?
153 2014-06-23 11:24:36 <sipa> petertodd: as that can be observed by SPV nodes, which then learn what fee policy is being used
154 2014-06-23 11:25:01 <sipa> (not saying the idea is perfect, btw)
155 2014-06-23 11:25:02 <petertodd> sipa: thing is the invalid part is unenforcable - you have no idea if the tx was actually broadcast
156 2014-06-23 11:25:22 <sipa> true, but how does that change anything?
157 2014-06-23 11:25:27 <hearn> we could do both
158 2014-06-23 11:25:39 <hearn> the idea behind the pending fee estimator pull is to observe actual behaviour of the network rather than stated behaviour
159 2014-06-23 11:25:43 <petertodd> sipa: better to let miners advertise a higher fee/byte to let people know the actual cost, e.g. if they're getting a lot of people paying for tx mining out of bound
160 2014-06-23 11:25:49 <hearn> however, the problem is, you have to wait for a while to learn what the behaviour is
161 2014-06-23 11:25:50 <petertodd> sipa: also, your idea doesn't work with child-pays-for-parent
162 2014-06-23 11:25:52 <sipa> a miner can add high-fee transactions himself to the block, but that doesn't change the fee he can claim
163 2014-06-23 11:26:00 <sipa> petertodd: indeed, it fails child-pays-for-parent
164 2014-06-23 11:26:44 <petertodd> sipa: basically if I'm to accept any out-of-band payments, I'll probably have to set my advertised fee to zero or near zero
165 2014-06-23 11:26:44 <sipa> also, i really dislike encoding fee policies into a consensus rule
166 2014-06-23 11:27:00 <sipa> petertodd: good point
167 2014-06-23 11:27:38 <hearn> well, the policy wouldn't be a consensus rule right, just a "Truth in advertising" rule
168 2014-06-23 11:28:07 <sipa> hearn: what i suggested would be a consensus rule, a block must be invalid if its state minfee is wrong
169 2014-06-23 11:28:20 <petertodd> one way to think about this, is if you take the view that the relay network is meant to be operating on behalf of miners, then the relay network wants to know what fees are being accepted by miners, so if miners are encoding in their blocks that only tx's having a fee > x BTC/KB are accepted, it's not unreasonable to say maybe the relay network shouldn't bother relaying tx's with a fee less than that
170 2014-06-23 11:28:21 <sipa> anyway, just food for thought, not actually proposing this
171 2014-06-23 11:28:23 <hearn> yes, exactly. the policy can be anything. but your summary of your policy must be correct
172 2014-06-23 11:30:07 <sipa> hearn: so i think that once we have transactions committing to input values, i guess it makes sense to make the input values data available through some means
173 2014-06-23 11:30:19 <hearn> you can't define "miners" so easily. if one block in 100 says it accept any transaction without any fee, then the network should arguably try and get transactions to that miner, up to its capacity, which brings us back to capacity-based prioritisation instead of the current anti-DoS system.
174 2014-06-23 11:30:25 <petertodd> sipa: note how per-block txo indexes do that, solving both problems
175 2014-06-23 11:30:43 <sipa> hearn: though i dislike that it basically requires full script validation to authenticate
176 2014-06-23 11:31:22 <hearn> sipa: well, realistically it seems script is people's favourite part of bitcoin to reimplement .... it's also quite libraryfiable.
177 2014-06-23 11:31:38 <hearn> so sure, that requirement is not ideal but it seems the most obvious next move
178 2014-06-23 11:31:39 <sipa> and probably the most tricky part to get right
179 2014-06-23 11:31:59 <petertodd> sipa: it's not tricky to get it
180 2014-06-23 11:32:11 <petertodd> sipa: to get it "right enough" for validating common-usage scriptSigs
181 2014-06-23 11:32:22 <sipa> right, and that may suffice here
182 2014-06-23 11:32:25 <petertodd> sipa: a false failure isn't a big deal in that application
183 2014-06-23 11:33:58 <hearn> the thing i like about this way of estimating fees is that it's of somewhat implementable size. getutxo is a simple message, already written. putting value in the sighash is not a hard piece of code either, the hard part is doing the fork and waiting for people to upgrade and start making the new transactions. then implementing the algorithm i proposed is a volunteer sized piece of work
184 2014-06-23 11:34:13 <hearn> additionally things can be done in paralle
185 2014-06-23 11:34:29 <hearn> *parallel. always a benefit for spread out groups of people.
186 2014-06-23 11:34:35 <sipa> an alternative would be to just relay txin values along with transactions
187 2014-06-23 11:34:47 <sipa> without needing several roundtrips for that information
188 2014-06-23 11:35:13 <sipa> (with the race condition that they may already be confirmed by the time you get to ask for it, pruning the data)
189 2014-06-23 11:35:30 <sipa> sorry, 1 roundtrip
190 2014-06-23 11:35:35 <hearn> you'd need the full CTxOut because you can't run the scripts without the full data. but yes the getutxo answer format can be glued to the end of a mempool-relayed tx given a version bump
191 2014-06-23 11:36:00 <hearn> however not all clients would want it
192 2014-06-23 11:36:06 <hearn> e.g. for full nodes it's just a waste of bandwidth
193 2014-06-23 11:36:16 <sipa> agree
194 2014-06-23 11:37:34 <petertodd> sipa: so that's with a new CHECKSIG mode to hash the CTxOut? sounds useful to me, especially with a flag to let the COutPoint not be hashed if desired
195 2014-06-23 11:37:57 <petertodd> sipa: oh, right, SignatureHash() already does that...
196 2014-06-23 11:37:58 <hearn> petertodd: we are discussing the algorithm i proposed on the getutxo thread for fee estimation
197 2014-06-23 11:38:18 <petertodd> hearn: I know
198 2014-06-23 14:16:23 <gavinandresen> petertodd: I'm still trying to understand how you think somebody can Sybil attack the fee estimation code on the main bitcoin network.
199 2014-06-23 14:17:02 <gavinandresen> petertodd: ⦠So I control somebody's connections when their fee estimates happen to be zero (they've just started up maybe? what's the scenario there?)
200 2014-06-23 14:17:28 <sipa> gavinandresen: and then you only forward very high fee transactions to them, so they get a filtered view
201 2014-06-23 14:17:39 <petertodd> gavinandresen: ^
202 2014-06-23 14:18:18 <gavinandresen> I see, you control what goes into their memory pool.
203 2014-06-23 14:18:24 <petertodd> yup
204 2014-06-23 14:18:44 <gavinandresen> petertodd: you need to get better at communicating, I did not get that at all from your github comment.
205 2014-06-23 14:18:47 <petertodd> and anyway, that I could trick it into sending a 1BTC fee simply has to be fixed - there must be some sane max limit
206 2014-06-23 14:19:23 <gavinandresen> I'll set the flag that does the fee sanity check for the wallet code
207 2014-06-23 14:19:42 <hearn> hello
208 2014-06-23 14:19:47 <petertodd> gavinandresen: sorry, just seemed obvious to me
209 2014-06-23 14:19:48 <gavinandresen> howdy mike
210 2014-06-23 14:20:06 <gavinandresen> petertodd: everything is obvious when it comes out of your own head....
211 2014-06-23 14:20:15 <petertodd> gavinandresen: we probably want a lower sanity check limit - 0.1BTC fees are still a lot of money.
212 2014-06-23 14:20:18 <petertodd> gavinandresen: indeed
213 2014-06-23 14:20:34 <gavinandresen> petertodd: lower sanity limit: fine, separate pull request
214 2014-06-23 14:21:48 <petertodd> gavinandresen: well, I'd call this a merge-blocker myself - you can lose a lot of money very quickly with that bug, even just with a fluke estimate
215 2014-06-23 14:22:13 <petertodd> gavinandresen: but anyway, with sane limits I'm ACK on that patch
216 2014-06-23 14:22:15 <gavinandresen> petertodd: no, you really can't.
217 2014-06-23 14:23:11 <petertodd> gavinandresen: sure you can - get a bad estimate that's a bit under the 0.1BTC sanity limit and you'll be spending about $50 per transaction sent
218 2014-06-23 14:23:47 <gavinandresen> petertodd: okey dokey. Good luck with that Sybil at EXACTLY the time when you delete your fee estimates..........
219 2014-06-23 14:24:34 <sipa> is there some protection to not give any estimations before there's a sizable observed population?
220 2014-06-23 14:24:36 <petertodd> gavinandresen: indeed, good "luck" - lowish probability, but *far* from zero.
221 2014-06-23 14:24:39 <petertodd> sipa: none
222 2014-06-23 14:24:43 <sipa> sorry, i didn't get to lock at tje code yet
223 2014-06-23 14:25:05 <hearn> the estimate is just iirc the default
224 2014-06-23 14:25:15 <gavinandresen> sipa: there used to be, I think that code got lost in some reworking somewhere. I don't think that will make Peter happy, though, because "infinite Sybil!!!!"
225 2014-06-23 14:25:18 <petertodd> sipa: basically I demonstrated a 1BTC fee based on a single transaction confirming into a block, samplesize = 1
226 2014-06-23 14:25:59 <hearn> huh, when i reviewed it there was indeed a codepath to give some hard-coded reasonable default until enough time had passed to observe the network
227 2014-06-23 14:26:21 <gavinandresen> if it makes y'all happy, let me know how many samples would make you happy and I'd be happy to change the == 0 to < N
228 2014-06-23 14:26:42 <gavinandresen> Frankly, I'm tired of the nit-picking "must be perfect" attitude here.
229 2014-06-23 14:27:01 <petertodd> gavinandresen: it's not a matter of samples, just put a sane upper limit on it that's configurable. that's sufficient to give users the tools they need to protect themselves.
230 2014-06-23 14:27:16 <gavinandresen> yippee, another configuration option....
231 2014-06-23 14:27:28 <hearn> ACTION shrugs
232 2014-06-23 14:27:30 <sipa> we already have a "sane max fee" limit somewhere afaik
233 2014-06-23 14:27:33 <hearn> it can be hard-coded as well
234 2014-06-23 14:27:51 <petertodd> sipa: yeah, it's hardcoded to make 0.1BTC fee per tx - it's not even a per-byte limit
235 2014-06-23 14:27:52 <hearn> i see simple safeties like that more as a protection against accidental bugs rather than unlikely attacks
236 2014-06-23 14:28:02 <sipa> hearn: agree
237 2014-06-23 14:28:13 <sipa> though it's nice to have as assurance against attacks as well
238 2014-06-23 14:28:19 <gavinandresen> Ok. I will change the code to require at least 11 samples.
239 2014-06-23 14:29:02 <gavinandresen> I think minimum sane fee is already configurable, I'll leave that alone.
240 2014-06-23 14:29:42 <hearn> cool
241 2014-06-23 14:30:42 <hearn> so who gets to be the lucky, lucky guy who starts the "getfees" SPV message fight? is that me, gavin or wumpus? :)
242 2014-06-23 14:31:13 <sipa> ACTION suggests: asking it from the same source you get exchange rates from, until we have an authenticated mechanism for it
243 2014-06-23 14:32:23 <gavinandresen> rebasing against master this morning, I'm getting:
244 2014-06-23 14:32:24 <gavinandresen> configure.ac:699: warning: 'INCLUDES' is the old name for 'AM_CPPFLAGS' (or '*_CPPFLAGS')
245 2014-06-23 14:32:25 <gavinandresen> configure.ac:114: error: required file 'src/config/bitcoin-config.h.in' not found
246 2014-06-23 14:32:36 <sipa> gavinandresen: rerun autogen
247 2014-06-23 14:32:47 <gavinandresen> sipa: thanks
248 2014-06-23 14:32:59 <petertodd> sipa: +1
249 2014-06-23 14:33:20 <sipa> if you're going to rely on untrusted data, better go with the devil you know
250 2014-06-23 14:33:53 <hearn> bitcoin community in 2012: https://github.com/bitcoin/bitcoin/pull/1641
251 2014-06-23 14:34:07 <hearn> bitcoin community now: find some random guy and trust him!
252 2014-06-23 14:34:11 <gavinandresen> sipa: what about Mike's argument that peers already give you lots of unauthenticated data, and we manage to deal with that OK?
253 2014-06-23 14:34:47 <hearn> i miss the days when we could go from "spv wallets need this" to a merged pull req in the span of about 20 days and 9 comments :)
254 2014-06-23 14:34:47 <sipa> gavinandresen: doesn't mean it's a practice we should encourage or extend
255 2014-06-23 14:34:56 <gavinandresen> "perfect enemy of the better" and all that
256 2014-06-23 14:35:05 <wumpus> hearn: me too, maybe we should fork bitcoin core :P
257 2014-06-23 14:35:14 <hearn> you mean and gavin together, right? :) hehe
258 2014-06-23 14:35:14 <petertodd> sipa: yup. anyway, the important thing is to make sure interactive users a) notice what fees they're sending if they're unexpectedly large, and b) automated users can set limits so they can't find themselves spending significantly more than expected without a human getting involved
259 2014-06-23 14:35:17 <gavinandresen> Why not encourage/extend if it makes the Bitcoin network more valuable?
260 2014-06-23 14:35:47 <hearn> sipa: so realistically, the source we use for exchange rates (by which i mean my clients use - bitcoinj has no centralised server dependencies at all), has in the past returned garbage several times.
261 2014-06-23 14:35:51 <hearn> i also don't trust it to not get hacked
262 2014-06-23 14:35:56 <jgarzik> "don't keep going in the wrong direction"
263 2014-06-23 14:36:05 <jgarzik> one wrong doesn't make another wrong right.
264 2014-06-23 14:36:06 <petertodd> wumpus: I'll call my fork Genuine Bitcoin
265 2014-06-23 14:36:10 <hearn> "trust some guy you never met" is certainly a temptingly simple security model, but it'd require safety and sanity checks no matter what
266 2014-06-23 14:36:30 <hearn> jgarzik: well you added the mempool command! and you know what, i've never had reports of problems with it
267 2014-06-23 14:37:06 <hearn> in contrast to bitcoinaverage.com which actually crashed all android wallets after it started returning exchange rates of zero :)
268 2014-06-23 14:37:20 <hearn> luckily exchange rates tend to get sanity checked by humans when they need them
269 2014-06-23 14:37:53 <sipa> and so should fees
270 2014-06-23 14:37:56 <hearn> we can certainly do averaging and sanity checking of what comes back from the p2p network, until the day we implement my proposed lower trust method
271 2014-06-23 14:38:03 <wumpus> petertodd: hah, BitcoinGenuineAdvantage
272 2014-06-23 14:38:14 <hearn> sipa: yes and i fully expect them to .... all spv wallets do display fees to the user iirc
273 2014-06-23 14:38:26 <petertodd> wumpus: no! It's Genuine Bitcoin that has the Genuine Bitcoin Advantage!
274 2014-06-23 14:38:37 <sipa> the mempool command just gives you data you could have had before by connecting earlier and seeing inv messages; the contents of the mempool has no consensus, and doesn't pretend to be (though it seems mike would like it to be)
275 2014-06-23 14:38:39 <hearn> so in practice i'm not too worried about attacks on this. yes, it can happen, and we ideally will defend against it, and we have a plan.
276 2014-06-23 14:39:13 <sipa> utxo worries me more, as it's about the most critical information in the system
277 2014-06-23 14:39:38 <t7> hearn: haha i didnt hear about that
278 2014-06-23 14:40:02 <sipa> fees... i don't really care, they're also just mempool data, but i still think if it's unverifiable, it's better fetched from a place that has a reputation to suffer rather than random peers
279 2014-06-23 14:40:03 <hearn> sipa: for a full node yes. not for every app.
280 2014-06-23 14:40:15 <hearn> well
281 2014-06-23 14:40:17 <hearn> look at it like this
282 2014-06-23 14:40:17 <petertodd> sipa: +1 - note how getutxo would be more useful if it returned the entire transaction so you could get some proof the TXO actually existed. (e.g. for checking fees paid by a tx)
283 2014-06-23 14:40:41 <hearn> the p2p network itself has a reputation, which indeed if lots of nodes return garbage or users keep reporting sybil attacks, the "reputation" of the p2p network as a useful decentralised place to get unauthenticated data will suffer.
284 2014-06-23 14:40:41 <sipa> i basically worry that someone will implement a wallet based on getutxos, by fetching from random peers
285 2014-06-23 14:40:42 <wumpus> petertodd: I don't think we have the entire transaction
286 2014-06-23 14:40:56 <hearn> then i will have to find another source with a better reputation like bitcoinaverage, my own servers, electrum etc.
287 2014-06-23 14:41:01 <jgarzik> hearn, poor example
288 2014-06-23 14:41:10 <petertodd> wumpus: I thought we do? getrawtransaction returns the tx so long as at least one output is unspent (although I'd hate to make that a requirement)
289 2014-06-23 14:41:12 <hearn> the argument i keep making, which doesn't seem to sink in, is that in fact the p2p network's reputation so far has been OK. good enough, for sure.
290 2014-06-23 14:41:20 <jgarzik> hearn, "mempool" is for diagnostics and network sampling, not finance
291 2014-06-23 14:41:25 <sipa> petertodd: and it's awfully inefficient
292 2014-06-23 14:41:33 <jgarzik> hearn, if you are using that for "trust", you are doing something really wrong
293 2014-06-23 14:41:44 <sipa> petertodd: we just know the height, read the entire block from disk, and scan through its transactions
294 2014-06-23 14:41:46 <hearn> jgarzik: the only actual user of it today, that i know of, is SPV wallets, which use it to find transactions broadcast before they started up
295 2014-06-23 14:42:16 <sipa> hearn: right, and i don't want to encourage relying on something that we don't know how it will evolve in the future
296 2014-06-23 14:42:31 <sipa> bitcoin as an experiment is just so much more interesting the less assumptions it has to make
297 2014-06-23 14:42:41 <petertodd> sipa: yup, though actually, given that getutxo now returns height/blockhash, you can do that with a getblock
298 2014-06-23 14:42:44 <hearn> yes, ideally bitcoin would have no value etc etc.
299 2014-06-23 14:43:38 <hearn> now back in the real world, unfortunate though it may be, people are actually using this experiment even though we keep telling them not to :) so we do have to work with that reality.
300 2014-06-23 14:44:09 <hearn> or at least me and gavinandresen and wumpus do because our incomes rely on bitcoin working. now get back to writing cluster control scripts sipa :)
301 2014-06-23 14:44:22 <hearn> ACTION remembers being an SRE fondly
302 2014-06-23 14:44:31 <sipa> ACTION drains hearn
303 2014-06-23 14:44:56 <hearn> you youngsters. i remember when i had to file a ticket and wait for a grumpy human every time i wanted to drain a cluster ....
304 2014-06-23 14:45:01 <hearn> anyway
305 2014-06-23 14:45:15 <AndersAA> ACTION is getting some good stuff on how decision-making happens in the Bitcoin-project for his masters thesis
306 2014-06-23 14:45:51 <hearn> sipa: there is no way to implement a wallet based on getutxos. for that we'd need an address/key indexed UTXO set
307 2014-06-23 14:45:54 <hearn> sipa: which we don't have.
308 2014-06-23 14:46:09 <jgarzik> AndersAA, In open source projects, all the dirty laundry is out in public for everyone to view. To some that appears chaotic, but it's just a different chaos than other software engineering models. ;p
309 2014-06-23 14:46:11 <hearn> sipa: i mean, there is a way, it's called making a web wallet and lots of people have done that. but there's no way to do it with the p2p network even after my patch
310 2014-06-23 14:46:48 <hearn> jgarzik: well, in the corporate model eventually a manager steps in and bangs heads together :) or linus ...
311 2014-06-23 14:46:48 <sipa> good point
312 2014-06-23 14:47:04 <hearn> jgarzik: or the company goes bankrupt :)
313 2014-06-23 14:47:22 <AndersAA> jgarzik: In my experience people tend to yell louder at each other behind closed doors :)
314 2014-06-23 14:47:56 <jgarzik> hearn, RE linux not really
315 2014-06-23 14:47:59 <hearn> sipa: basically i wrote getutxos for two purposes. one is, lighthouse uses it as a UI hint and a performance optimisation. it helps the UI greatly. it does not, ultimately, impact the security, because if the results of getutxo are bogus then the resulting contract tx won't work: it'll be rejected or treated as an orphan and the issue can be detected at that point. secondly, it's a part of my SPV fee estimation plan.
316 2014-06-23 14:48:24 <sipa> yes, i understand your reasons
317 2014-06-23 14:48:25 <jgarzik> Linux works like this: Throw spaghetti at a wall, and see if it sticks.
318 2014-06-23 14:48:31 <hearn> sipa: so in both cases we can tolerate some noise in the results, because the worst case scenario is the app gets stuck a bit and the user then learns either bitcoin or their internet connection is hosed. and in the latter it's just a waste of disk space.
319 2014-06-23 14:48:53 <jgarzik> getutxos is very easy to use poorly
320 2014-06-23 14:48:59 <hearn> so is "tx" :)
321 2014-06-23 14:49:06 <wumpus> I think it's fine to have P2P messages with different security assumptions, as long as well documented
322 2014-06-23 14:49:35 <jgarzik> And I simply cannot see a responsible funds collector/administrator preferring an untrusted interface over simply running bitcoind+REST
323 2014-06-23 14:49:43 <wumpus> right, as we've seen, bitcoin itself is very easy to use poorly
324 2014-06-23 14:50:23 <hearn> jgarzik: well, that's sort of like saying a responsible funds collector would not accept unconfirmed transactions as payment, yet your employer does! i am certainly more than happy to make lighthouse use a local bitcoind if one is detected
325 2014-06-23 14:50:28 <wumpus> in practice I think getutxos will be ignored by everyone except hearn
326 2014-06-23 14:50:46 <hearn> and in fact i might go implement that this week! but i suspect most people will not want to run Core for the usual reasons. i can, however, advise that it is a useful thing to do, for those who have it
327 2014-06-23 14:50:57 <hearn> wumpus: yes that's more or less what i'm expecting.
328 2014-06-23 14:51:27 <wumpus> I really doubt people will start pounding out extra-insecure wallets as soon as the message appears
329 2014-06-23 14:51:59 <Eliel> is there a reason to use uncofirmed transactions as the base for the fee? On first glance, I'd use the transactions from the last block as the base if no prior info.
330 2014-06-23 14:52:04 <hearn> right, it's not usable for a normal wallet anyway, at least not without an address index. even if an address index was created, if getutxo returned a bogus answer, the resulting tx would merely be invalid and the app would get stuck
331 2014-06-23 14:52:05 <GAit> i am worried we consider todays incentives the same as tomorrow's incentives and i am not sure i feel i identified what incentives there are in providing correct data here, FWIW
332 2014-06-23 14:52:18 <hearn> Eliel: you can't calculate fees paid from a block in isolation, nor how long it took to confirm
333 2014-06-23 14:52:20 <jgarzik> So, new logic: It's ok to add crap, since it will just be used by this one odd duck coder/app.
334 2014-06-23 14:52:21 <sipa> Eliel: miners can cheat that by putting their own high-fee transactions in
335 2014-06-23 14:53:00 <hearn> jgarzik: eh, that's a silly argument too - seen that way bloom filtering is only used by me for nearly 2 years already. it's transitively been used by ~2 million people
336 2014-06-23 14:53:10 <petertodd> sipa: I mentioned elsewhere that taking the min fee measured by both methods probably made a lot of sense
337 2014-06-23 14:53:12 <wumpus> jgarzik: it seems more like 'deny and inhibit all experimentation until a perfect solution (TM) appears'
338 2014-06-23 14:53:20 <jgarzik> hearn, the universe of fund administrators is tiny
339 2014-06-23 14:53:21 <Eliel> sipa: several blocks then? :)
340 2014-06-23 14:53:31 <sipa> Eliel: equally cheatable
341 2014-06-23 14:53:56 <jgarzik> hearn, the universe of fund administrators that prefer untrusted interfaces over trusted ones...
342 2014-06-23 14:54:11 <sipa> Eliel: see it this way: we want to measure the conditions for transactions migrating from the memory pool to the block chain
343 2014-06-23 14:54:27 <hearn> jgarzik: well, fund administrators today accept credit card payments .... and the people funding them accept only promises.
344 2014-06-23 14:54:32 <jgarzik> wumpus, no. Several trusted alternative ways to do fund administration already exist.
345 2014-06-23 14:54:33 <Eliel> sipa: yes, if you look at blocks, you can see a collection where 100% made it in.
346 2014-06-23 14:54:35 <wumpus> hearn: I suppose you could try getting it added in one of the other node implementations
347 2014-06-23 14:55:01 <sipa> Eliel: there are other ways through which transactions can end up in blocks, and that's not what we want to measure (private deals with miners, their own transactions, anything out-of-band)
348 2014-06-23 14:55:02 <Eliel> thus, you can deduce the minimum fee by looking at which transaction paid the least.
349 2014-06-23 14:55:23 <jgarzik> wumpus, This could have been done a month ago with existing bitcoin software, no new APIs, trusted.
350 2014-06-23 14:55:26 <hearn> wumpus: we have to tackle all these arguments sooner or later anyway. otherwise there won't be any decentralised wallets anymore. once your wallet can't function without magicwebsite.com being up and reliable we may as well just give up and all use blockchain.info
351 2014-06-23 14:55:32 <jgarzik> wumpus, That != shutting down experiments
352 2014-06-23 14:55:59 <hearn> jgarzik: ah, but the entire point of my product is that it's decentralised. obviously it *can* be done with a trusted third party ,as can all finance. and it sure does simplify things
353 2014-06-23 14:56:04 <Eliel> sipa: that's true, but I think that'd be good enough to start with. definitely harder to cheat than just using the last n unconfirmed transactions
354 2014-06-23 14:56:27 <sipa> Eliel: it uses the combination: watch unconfirmed transaction come and and measure how long they take to confirm
355 2014-06-23 14:56:30 <jgarzik> hearn, Did I say trusted third party? No. Stop imagining words.
356 2014-06-23 14:56:44 <jgarzik> hearn, google insight and install it. Insight is not a trusted third party, it is a trusted first party.
357 2014-06-23 14:56:46 <hearn> jgarzik: then i'm not sure what you mean. are you still talking about running Core locally?
358 2014-06-23 14:57:32 <hearn> like i said, i'm happy to support that for people who want to run Core. i do not wish to insist upon it, for the same reason that i don't insist people run Core as their wallet for other cases too.
359 2014-06-23 14:58:17 <wumpus> why would you need insight to do a simple UTXO lookup? every node keeps track of that already
360 2014-06-23 14:58:29 <jgarzik> wumpus, you don't with a simple REST patch
361 2014-06-23 14:58:32 <hearn> also Insight is a full blown block explorer. unless you found a way to make them 100x more efficient than the others i know about, setting up Insight for the main network would easily take days or weeks on consumer hardware, assuming it can be set up at all. the biteasy guys told me reindexing their main chain (using some fancy nosql system) takes forever
362 2014-06-23 14:58:40 <hearn> so how about this
363 2014-06-23 14:58:43 <jgarzik> Fine, I will implement the REST patch today.
364 2014-06-23 14:58:43 <wumpus> no need for a big transaction index, or extra software
365 2014-06-23 14:58:52 <jgarzik> Then no need for big transaction index, or extra software.
366 2014-06-23 14:58:52 <wumpus> isn't there a getutxo RPC call?
367 2014-06-23 14:58:54 <hearn> i will use a local bitcoind if found on port 8333
368 2014-06-23 14:59:09 <hearn> if not, it'll use the main network. then users can decide for themselves the overhead/security tradeoffs
369 2014-06-23 14:59:10 <wumpus> no need for even a REST patch
370 2014-06-23 14:59:16 <jgarzik> gettxout
371 2014-06-23 14:59:17 <jgarzik> yes
372 2014-06-23 14:59:39 <jgarzik> and gettxout RPC can be batched
373 2014-06-23 14:59:58 <wumpus> great
374 2014-06-23 15:00:44 <hearn> anything that requires you to run Core yourself is basically DoA for regular users at this point.
375 2014-06-23 15:00:50 <hearn> so that implies anything RPC based also is
376 2014-06-23 15:01:00 <jgarzik> hearn, why?
377 2014-06-23 15:01:12 <gavinandresen> petertodd sipa: https://github.com/gavinandresen/bitcoin-git/commit/853e3fab33b9934c88360cf07e9f67664fc0e025
378 2014-06-23 15:01:41 <jgarzik> A fund administration wants to collect tens of thousands of dollars (in bitcoin), but cannot run a -nolisten node? Come on.
379 2014-06-23 15:02:09 <Apocalyptic> heh
380 2014-06-23 15:02:13 <jgarzik> gettxout RPC works today, is trusted, no need to wait for P2P rollout of an untrusted feature
381 2014-06-23 15:02:15 <Apocalyptic> sounds like reality
382 2014-06-23 15:02:27 <hearn> two points. one, you'd be amazed at how much money some people carry around on android wallets. i've seen hundreds of BTC sized wallets! yes, people don't want to wait even if they're collecting lots of money.
383 2014-06-23 15:02:34 <hearn> two, who says these contracts will all be tens of thousands of dollars?
384 2014-06-23 15:02:46 <hearn> there's no particular minimum size requirement when using bitcoin for this, it's one of the big advantages
385 2014-06-23 15:02:50 <petertodd> gavinandresen: how about we make that part a separate pull-req, along with the maxfee setting?
386 2014-06-23 15:03:04 <hearn> one of the examples i give for how it *could* be used (not saying it would) is friends organising a party or day out
387 2014-06-23 15:03:23 <hearn> "we need at least 10 people to make this fun, tickets cost $20, please attach your pledges to your reply to this email" kind of thing
388 2014-06-23 15:03:42 <hearn> anyway, brb
389 2014-06-23 15:04:00 <petertodd> hearn: anyway, I still see no reason why Lighthouse needs getutxo, but I just ACKed the patch all the same for the other uses of it
390 2014-06-23 15:04:08 <Luke-Jr> [15:02:26] <hearn> two points. one, you'd be amazed at how much money some people carry around on android wallets. i've seen hundreds of BTC sized wallets! yes, people don't want to wait even if they're collecting lots of money. <-- this sounds like an argument to make it harder to do :P
391 2014-06-23 15:07:07 <hearn> hah, well, andreas' wallet already has "don't store lots of money here" in the safety notes that appear when you first run the app
392 2014-06-23 15:07:38 <hearn> if someone sends a lot of money anyway, not much we can do about that. besides who are we to say it's a bad idea? maybe the phones are kept at home as dedicated hardware wallets, with no apps. it's hard to outsmart all our users all the time
393 2014-06-23 15:08:11 <Apocalyptic> "maybe the phones are kept at home as dedicated hardware wallets, with no apps." do you really believe that ?
394 2014-06-23 15:08:29 <hearn> jgarzik: how about this? if the user tries to create an assurance contract of larger than X BTC where X is something i'll think about, i'll display a warning asking them to install a local version of Bitcoin Core and i'll use it when it's available
395 2014-06-23 15:08:33 <hearn> and e.g. alert the user if it's not running
396 2014-06-23 15:08:58 <hearn> jgarzik: this should satisfy the use case of someone who really is running a contract of tens of thousands of dollars, whilst not inconveniencing users who have more casual use cases
397 2014-06-23 15:09:03 <petertodd> hearn: heh, I like that
398 2014-06-23 15:09:17 <jgarzik> hearn, default to using gettxout RPC, make them take extra effort to use the untrusted crapola
399 2014-06-23 15:09:19 <hearn> Apocalyptic: well people have told us they actually do ..... so yes. how many of the big wallets correlate with such use cases, i can't say.
400 2014-06-23 15:09:31 <jgarzik> don't make users work harder, just to get better security
401 2014-06-23 15:09:36 <jgarzik> you want the opposite
402 2014-06-23 15:09:37 <hearn> jgarzik: how about, i'll do that, when bitpay insists on 6 confirmations for all its payments? :)
403 2014-06-23 15:09:44 <petertodd> hearn: kinda reminds me of a big scary warning screen DarkWallet has at startup where it gives you the option of using mainnet or testnet
404 2014-06-23 15:09:56 <hearn> petertodd: i keep meaning to play with that. thought it was testnet only for now
405 2014-06-23 15:10:03 <jgarzik> hearn, Do we really need to review all the ways that 0conf is just fine for certain businesses?
406 2014-06-23 15:10:16 <jgarzik> default secure
407 2014-06-23 15:10:20 <jgarzik> default trusted
408 2014-06-23 15:10:51 <hearn> jgarzik: you don't see any contradiction here? you're telling me to make my users wait a day or two always, regardless of their use case, because "security is good". and now you're saying it's fine that bitpay uses 0conf for everything because it's fine for certain businesses. ok, seems the same to me.
409 2014-06-23 15:11:11 <GAit> isn't the difference in the defaults?
410 2014-06-23 15:11:14 <petertodd> hearn: mainnet too, although even I don't trust it enough to run it in anything but a VM with small amounts of money
411 2014-06-23 15:11:26 <jgarzik> hearn, I think it's a stupid and unrelated comparison. BitPay is not decentralized software rolled out to the entire bitcoin community.
412 2014-06-23 15:11:40 <jgarzik> hearn, Bitcoin Core necessarily has a higher standard
413 2014-06-23 15:12:52 <jgarzik> Default:trusted is obviously superior to default:untrusted
414 2014-06-23 15:13:05 <hearn> i see no distinction. it's all software, designed to be used by ordinary humans who have varying security requirements.
415 2014-06-23 15:14:07 <hearn> the computer industry is littered with software that killed itself by attempting to be too secure, also. it's easy to insist on ever higher security standards, then you get the famous yellow sticky note with passwords on it
416 2014-06-23 15:14:28 <hearn> the security/usability balance is _very_ tricky and subtle. it's not something that can be summed up in a simple one liner.
417 2014-06-23 15:14:42 <hearn> heck o'reilly have an entire book that consists just of essays and research papers on this topic. i read it once, it's very good.
418 2014-06-23 15:15:08 <Apocalyptic> not sure it's very relevant to bitcoin's case hearn
419 2014-06-23 15:15:36 <hearn> how can it not be relevant? we're making software that tries to be both usable and secure. it by necessity means exploring this balance.
420 2014-06-23 15:15:47 <Apocalyptic> common software didn't use to hold billions worth of irreversible money
421 2014-06-23 15:16:14 <petertodd> Apocalyptic: that doesn't preclude a balance you know, it just makes the balance have a different center than you would otherwise expect
422 2014-06-23 15:16:20 <Apocalyptic> indeed
423 2014-06-23 15:16:26 <hearn> online banking and trading systems certainly do, along with people's most private data, military secrets, etc
424 2014-06-23 15:16:53 <hearn> i mean i don't think you can say bitcoin is totally different to everything else and all prior experience and research simply gets thrown out the window because moneyzz
425 2014-06-23 15:16:54 <Apocalyptic> petertodd, I suspect o'reilly center is quite far from what we should consider
426 2014-06-23 15:17:15 <hearn> the research is really about human factors in software design
427 2014-06-23 15:17:19 <hearn> and humans are the same no matter what
428 2014-06-23 15:18:56 <dabura667> hearn: I think most online banks don't come at the drawing board for their online banking sites saying "ok, how much should we compromise security to make an awesome usability for the customer?"... I think it starts with "How to we secure our customers?" then once they get the necessities down, they ask "OK, how usable can we make this site within these security constraints?"
429 2014-06-23 15:20:24 <Luke-Jr> trezor's gitian reinvention isn't even free software? why would anyone trust this? @.@
430 2014-06-23 15:20:53 <netg_> wat, wasnt it supposed to ALL code being free
431 2014-06-23 15:20:55 <hearn> dabura667: it'd be nice if banks really did work like that. look at the USA. an entire country where hardware 2-factor auth is hardly known! even in places where things are further along, banks routinely compromise security for usability, consciously. for instance in allowing signature fallbacks if there's something wrong with the card chip, allowing country-internal payments without a hardware auth, etc.
432 2014-06-23 15:20:55 <petertodd> dabura667: you have to remember that in many countries a *lot* of political effort has been spent making banks not liable for fraud. There was a fascinating talk on that subject as the Financial Crypto conference, which pretty much boiled down to "Banks spend more effort making it impossible to prove ATM fraud then they do preventing it."
433 2014-06-23 15:21:00 <petertodd> Luke-Jr: WTF?
434 2014-06-23 15:21:44 <hearn> dabura667: same thing for web browsers, corporate VPNs, medical systems ... there is always a balance. otherwise nothing stops the designers from requiring ever more strict security until people just give up and go use a competitor
435 2014-06-23 15:22:08 <hearn> dabura667: i went through this same argument when i worked on google account security. some people said, "wtf why bother doing risk analysis. everyone should use 2-factor authentication on their gmail accounts, problem solved"
436 2014-06-23 15:22:30 <hearn> dabura667: that had the very practical problem that lots of users didn't want to use 2-factor auth and either refused to use it entirely, or tried it and then switched it off.
437 2014-06-23 15:22:34 <petertodd> dabura667: for instance, there are examples in the UK where banks used to keep records that duplicated information, or at least made cross-checking possible, and they stopped doing that because it was being used as evidence that their chip-card security wasn't as good as they were claiming
438 2014-06-23 15:22:56 <hearn> dabura667: so in the end a "middle road" was still necessary.
439 2014-06-23 15:22:58 <Luke-Jr> petertodd: dunno, now someone is saying it's Apache licensed. but their website wants $$ for it. I don't get it..
440 2014-06-23 15:23:11 <dabura667> whoa... so much green... o_o
441 2014-06-23 15:23:14 <petertodd> Luke-Jr: how much $? buy a copy and release it
442 2014-06-23 15:23:22 <Luke-Jr> petertodd: it's per-repository O.o
443 2014-06-23 15:23:31 <petertodd> Luke-Jr: link?
444 2014-06-23 15:23:34 <Luke-Jr> http://www.docker.com/resources/howtobuy/
445 2014-06-23 15:23:45 <hearn> dabura667: indeed the latest Big Thing in the traditional payments world is NFC payments, which can be summed up as "lower security for faster performance when the risk seems low"
446 2014-06-23 15:24:03 <hearn> dabura667: so our competitors are very consciously exploring the security/usability spectrum
447 2014-06-23 15:24:40 <petertodd> Luke-Jr: ah, so deterministic builds are a feature of docker. hmm, I'd almost be inclined to sign up for their offer for independent experts to do deterministic builds just to raise that issue...
448 2014-06-23 15:26:34 <netg_> finding out the right level of security in code und proposing general risk analysis standards is a perfect job, for the chief scientist IMHO
449 2014-06-23 15:26:50 <netg_> as its obvious your views differ much
450 2014-06-23 15:27:14 <petertodd> netg_: yes, I think it would be great to add yet another view to that discussion :P
451 2014-06-23 15:27:49 <dabura667> hearn: The point I am trying to get at was what jgarzik was mentioning about defaulting to trusted, at least as far as Core is concerned. If an SPV client or some other piece of software wants to make exchanges of security for usability, then I think that's fine and as you mentioned should be encouraged to compete. But leaving the core protocol open to security weaknesses in the name of usability is dangerous imo.
452 2014-06-23 15:27:51 <hearn> some of this stuff just has to be tried. i don't think it's always possible for someone to just Get It Right.
453 2014-06-23 15:28:35 <sipa> especially with floating fees i agree with that... we'll just have to see how the interaction of software with an economy behaves...
454 2014-06-23 15:28:46 <dabura667> hearn: true. Can't knock it 'til you try it, I guess.
455 2014-06-23 15:28:57 <epscy> is there a rollback procedure? :p
456 2014-06-23 15:29:03 <hearn> dabura667: also i think there's a small misunderstanding. getutxo does not add any security weaknesses to the protocol (ignoring the usual DoS stuff which can be triggered via many other ways). jeff's objection is that people might use it to write insecure apps
457 2014-06-23 15:29:05 <sipa> epscy: sure, just use a time machine
458 2014-06-23 15:29:13 <petertodd> sipa: yeah, floating fees is fine, but as I said on day 1, it must be sanity checked or you'll find out that interaction didn't go as planned...
459 2014-06-23 15:29:17 <hearn> dabura667: but this is the old "SPV vs full" debate again
460 2014-06-23 15:29:33 <dabura667> hearn: ok, I didn't catch on that.
461 2014-06-23 15:29:34 <hearn> SPV is less secure than full and more secure than web wallets. we let the market decide. outcome: mixed.
462 2014-06-23 15:30:16 <hearn> so far it's sort of a race between mostly centralised web wallets, entirely centralised web wallets, and SPV wallets. not sure how many people use Core as a wallet these days, it's a bit hard to measure. probably still a lot. i'm guessing in the hundreds of thousands range.
463 2014-06-23 15:30:30 <hearn> ACTION wonders why anyone would choose "so" as their nick
464 2014-06-23 15:30:45 <Apocalyptic> I hope there is a lot of them
465 2014-06-23 15:31:32 <hearn> Apocalyptic: i think the numbers will fluctuate as Core's resource requirements do. allowing chain pruning and upgrade-from-SPV would help a lot, of course. though that'd muddy the waters of who is using what security model even more :)
466 2014-06-23 15:31:56 <dabura667> I can't wait til BIP32 + BIP39 with Trezor English dict. is just in at least one wallet on every platform... Then I think we'll see SPV wallets soar into first place.
467 2014-06-23 15:32:27 <hearn> dabura667: good news! someone is working on married wallets for bitcoinj. making good progress, writing good code. that's BIP32+multisig wallets that are "married" to a third party risk analysis operator.
468 2014-06-23 15:32:29 <dabura667> And part of the reason I like Electrum. Even though the current Android client is really cruddy, I can have the same wallet on my PC and android
469 2014-06-23 15:32:57 <dabura667> hearn: *squee*
470 2014-06-23 15:33:16 <hearn> ah yes. well i'm working on HD wallets for bitcoinj at the moment so then cloning wallets,at least at the financial level, will become possible. we still need to find a nice way to do electrum style metadata sync
471 2014-06-23 15:33:30 <hearn> as metadata is becoming more and more important over time.
472 2014-06-23 15:33:41 <sipa> use the cloud!
473 2014-06-23 15:33:47 <hearn> heh
474 2014-06-23 15:33:54 <Luke-Jr> +1 (jk)
475 2014-06-23 15:34:00 <dabura667> Electrum doesn't do metadata sync... I think theres a plugin for contacts but thats about it
476 2014-06-23 15:34:05 <hearn> i thought it did label syncing
477 2014-06-23 15:34:10 <Luke-Jr> sipa: the obvious solution is to embed your wallet in the blockchain
478 2014-06-23 15:34:15 <dabura667> yeah that's a plugin not main
479 2014-06-23 15:34:15 <Luke-Jr> we can make OP_RETURN bigger
480 2014-06-23 15:34:20 <hearn> ah ok
481 2014-06-23 15:34:21 <dabura667> and only for PC client
482 2014-06-23 15:34:33 <sipa> Luke-Jr: right;it's the obvious best solution
483 2014-06-23 15:34:50 <Luke-Jr> sipa: it's even log-based, perfect format!
484 2014-06-23 15:34:53 <dabura667> Luke-Jr: OP_RETURN ALL THE DATAS!
485 2014-06-23 15:34:57 <hearn> wallets aren't very big. i think we could hash the wallet into an OP_RETURN output and then use ASICs to reverse the hash into the real wallet via brute force
486 2014-06-23 15:35:01 <hearn> it's decentralised and elegant ;)
487 2014-06-23 15:35:11 <sipa> hearn: lol!
488 2014-06-23 15:35:16 <gdm85> btw, wumpus, I noticed you contributed some other repository on github: etna_viv. mind if I PM you about it? not for a long talk
489 2014-06-23 15:36:04 <dabura667> I'm going to go play Monaco, I just bought on Steam summer sale. Nice talking to you guys, cya.
490 2014-06-23 15:36:22 <wumpus> gdm85: sure
491 2014-06-23 15:36:35 <hearn> later
492 2014-06-23 15:36:37 <petertodd> Luke-Jr, sipa: I actually semi-seriously suggested that as a way to upgrade from non-HD to HD wallets...
493 2014-06-23 15:38:20 <hearn> ACTION -> home
494 2014-06-23 15:38:56 <dsnrk> Luke-Jr: why do you need a wallet file?? all I need is my brainwallet, lets store that in a OP_RETURN output
495 2014-06-23 15:39:18 <Luke-Jr> dsnrk: only if you memorise the blockchain .->
496 2014-06-23 15:39:22 <Luke-Jr> ._. *
497 2014-06-23 15:39:36 <petertodd> Luke-Jr: nah, other people do that for you
498 2014-06-23 15:55:01 <sipa> this has to be the most confusingly wrong notation for the bitcoind version numbers yet:
499 2014-06-23 15:55:03 <sipa> "I downloaded the latest 0.92 BTC source code "
500 2014-06-23 15:56:14 <Luke-Jr> ROFL
501 2014-06-23 15:56:40 <Luke-Jr> sipa: bet it's an altcoin fan
502 2014-06-23 15:57:01 <Luke-Jr> (altcoins often use their base unit as an abbreviation)
503 2014-06-23 15:57:13 <Luke-Jr> (and don't understand the concept of decentralisation)
504 2014-06-23 16:01:13 <petertodd> sipa: sounds better when you say it's version 920,000 bits
505 2014-06-23 16:01:40 <wumpus> i wonder who he paid those 0.92BTC to
506 2014-06-23 16:01:47 <sipa> i'd say he paid way too much for some crappy piece of opensource code...
507 2014-06-23 16:02:08 <petertodd> sipa: but it's enterprise!
508 2014-06-23 16:03:50 <wumpus> petertodd: enterprise licenses are much more expensive!
509 2014-06-23 16:04:55 <petertodd> wumpus: well that's just a single seat obviously, and the price is a few months out of date...
510 2014-06-23 16:06:01 <Luke-Jr> wumpus: will we charge for 0.10?
511 2014-06-23 16:06:09 <sipa> yes, 0.10 BTC
512 2014-06-23 16:06:22 <Luke-Jr> cool
513 2014-06-23 16:06:26 <sipa> significantly cheaper than the previous version, which was 0.92 :)
514 2014-06-23 16:06:27 <Luke-Jr> how much will 0.10 BTC be worth then?
515 2014-06-23 16:06:44 <petertodd> Luke-Jr: 0.10 BTC
516 2014-06-23 16:06:47 <Luke-Jr> ACTION tries to trick sipa into speculating on future prices :P
517 2014-06-23 16:07:16 <dsnrk> sipa: just wait, if "bits" catches on we'll be in a world of hurt. "bitcoin uses 256bit private keys.." "what if I want to store more than 256 bits, I need two!????"
518 2014-06-23 16:07:44 <Luke-Jr> dsnrk: sounds like win for TBC
519 2014-06-23 16:08:45 <dsnrk> Luke-Jr: come on, we have a hard enough time with decimals. I'd love everything to be expressed as power of two, butttt it's not going to happen I don't think.
520 2014-06-23 16:09:15 <dekalo> hello
521 2014-06-23 16:09:22 <dsnrk> Luke-Jr: maybe eligius can only mine outputs that are of the size 2^n? ;)
522 2014-06-23 16:09:48 <Luke-Jr> dsnrk: or at least prioritise them :P
523 2014-06-23 16:10:15 <netg_> sound selfish
524 2014-06-23 16:10:16 <Luke-Jr> but then there's silly people like phantomcircuit who will represent every decimal number in multiple 2^n outputs
525 2014-06-23 16:10:17 <netg_> ;)
526 2014-06-23 16:14:18 <AndersAA> I don't like how Aaron Voisne hardcoded "bits" into Breadwallet for iOS. Only a matter of time before that gets forked. Great app though.
527 2014-06-23 16:15:33 <dsnrk> seeds aren't compatible with anything though. I thought they would be transportable to Electrum, but nope.
528 2014-06-23 16:15:47 <netg_> chief scientist should talk with him and sort this issue out :)
529 2014-06-23 16:16:29 <dsnrk> I misunderstood BIP32 and thought it would allow for people to easily transport between software (something which we don't have now, everybody uses their own lame format), but it looks like Electrum doesn't even make seeds that are the same from version to version, let alone with anybody else.
530 2014-06-23 16:16:41 <dabura667> dsnrk: Electrum uses its own thing... breadwallet is BIP32
531 2014-06-23 16:17:28 <dabura667> Electrum version 2.0 will support BIP32... but its mnemonic phrase will use a different dictionary and will include a version byte at the beginning... so not BIP32 standard per se.
532 2014-06-23 16:17:38 <dsnrk> exactly.
533 2014-06-23 16:18:04 <dsnrk> so then wallets have to do 32 different bip32 versions, and we're back to incompatible wallet files.
534 2014-06-23 16:18:05 <dabura667> However, if you can decode the mnemonic, remove the version byte and reencode it with the Trezor dictionary which seems to be what everyone is using, then you could use it.
535 2014-06-23 16:18:23 <dabura667> It would be trivial to create a Electrum <> BIP32 conversion tool.
536 2014-06-23 16:20:14 <dabura667> unfortunately, the mnemonic BIP (39) needs to set a 1 language 1 dictionary standard... imo.
537 2014-06-23 16:20:29 <dabura667> anywho, venturing into offtopic land.
538 2014-06-23 16:20:34 <dsnrk> I don't want to have to explain that to users. the seeds look and act the same. people should have sat down for more than 5 minutes and worked out a standard everybody agreed on.
539 2014-06-23 16:21:05 <dabura667> > people should have sat down for more than 5 minutes and worked out a standard everybody agreed on.
540 2014-06-23 16:21:06 <sipa> people argued for months...
541 2014-06-23 16:21:13 <dabura667> The story of my professional life...
542 2014-06-23 16:22:00 <AndersAA> http://xkcd.com/927/
543 2014-06-23 16:22:17 <dsnrk> sipa: I realise (I read that), but the pain is not just going to be one parties. it's something that's pretty much destroyed that use of BIP32 in my eyes.
544 2014-06-23 16:22:24 <dabura667> Thomas was saying that he views BIP39 as trivial and that BIP0044 is more important... however he said he didn't like the current BIP0044 draft.
545 2014-06-23 16:23:07 <dabura667> But he's busy with 2.0... and starting his company and 2FA service... etc.etc.
546 2014-06-23 16:24:28 <dsnrk> sorry this is straying into OT for this channel.
547 2014-06-23 16:24:49 <dabura667> I am gonna go on record and +1 a 1-language 1-dictionary rule for BIP39. Though BIPs are OT.
548 2014-06-23 16:26:41 <dabura667> AndersAA: UTF-8 was amazing... then UTF-16 came out, etc... endless loop. Anywho I'm going to bed, I have to code a silly Excel Macro for my boss... (shoot me)
549 2014-06-23 16:29:34 <hearn> UTF-16 predates UTF-8
550 2014-06-23 16:29:42 <hearn> oh, he's gone.
551 2014-06-23 16:29:45 <wumpus> yes, one can never have enough standards, and one can never invent the wheel enough times
552 2014-06-23 16:30:19 <hearn> in fairness, UTF-16 was the "obvious" way, and when UTF-8 was invented everyone agreed it was better and switched to it. the only time i encounter UTF-16 these days is in software that predates the widespread adoption of UTF-8.
553 2014-06-23 16:30:32 <hearn> (the exception being when performance is absolutely critical....)
554 2014-06-23 16:31:34 <wumpus> UTF-16 was the obvious way until it was discovered that 16 bits isn't enough to represent every glyph that humans use either
555 2014-06-23 16:31:44 <hearn> right
556 2014-06-23 16:32:10 <wumpus> UTF-32 never caught on for some reason :-)
557 2014-06-23 16:33:07 <dsnrk> I don't think UTF-32 went far enough. UTF-64 is where the party is at.
558 2014-06-23 16:34:56 <wumpus> dsnrk: that will be introduced along with the switch to 128 bit timestamps
559 2014-06-23 16:38:15 <dsnrk> wumpus: ah yes, the impending year 292,277,026,596 problem.
560 2014-06-23 16:41:36 <phantomcircuit> hearn, that would be...all of windows
561 2014-06-23 16:41:50 <hearn> yes, i know
562 2014-06-23 16:41:55 <hearn> i used to do win32 programming
563 2014-06-23 16:42:07 <hearn> heck a bit of win16 to
564 2014-06-23 16:42:08 <hearn> *too
565 2014-06-23 16:43:01 <hearn> java is the same, annoyingly
566 2014-06-23 16:47:23 <wumpus> well no matter what utf8/utf16/utf32 at least it's a solved issue now, no need for per-locale character sets anymore, yay
567 2014-06-23 16:48:04 <gavinandresen> current master unit tests are failing for me: test/DoS_tests.cpp:293: error: in "DoS_checkSig": Signature cache timing failed
568 2014-06-23 16:48:07 <gavinandresen> is it just me?
569 2014-06-23 16:48:33 <dsnrk> I'll check.
570 2014-06-23 16:49:06 <wumpus> gavinandresen: no errors detected here
571 2014-06-23 16:50:30 <phantomcircuit> wumpus, that's cheating, you gotta checkout upstream/master, autogen.sh, configure, make clean, make, then run the tests :P
572 2014-06-23 16:51:40 <wumpus> phantomcircuit: I do that all day
573 2014-06-23 16:52:03 <wumpus> :p