1 2015-01-21 01:58:43 <Luke-Jr> wumpus: harding: might make sense to upload/bump quietly, without a release notice?
  2 2015-01-21 02:02:47 <gmaxwell> 0.10 is close enough that it doesn't concern me.
  3 2015-01-21 02:03:07 <gmaxwell> I really don't want to send j-random-user through an upgrade cycle twice in a month; as that will undermine 0.10 deployment.
  4 2015-01-21 02:03:51 <cfields> gmaxwell: that's a very good point
  5 2015-01-21 02:05:21 <gmaxwell> I want to share a link to Pieter's strictder bitcoin-development post; but sourceforge's archive is being insanely slow.
  6 2015-01-21 02:05:57 <phantomcircuit> maybe just https://gist.github.com/sipa/5d12c343746dad376c80
  7 2015-01-21 02:05:59 <phantomcircuit> for now
  8 2015-01-21 02:06:13 <gmaxwell> I didn't want to do that; simply because I want to move discussion to the list. :)
  9 2015-01-21 02:06:20 <gmaxwell> So don't comment here on it. Comment on the list. :)
 10 2015-01-21 02:06:45 <gmaxwell> But indeed, thats the link the link I wanted linked to.
 11 2015-01-21 02:06:45 <phantomcircuit> sorry i ruined your plans
 12 2015-01-21 02:06:47 <phantomcircuit> :P
 13 2015-01-21 02:06:59 <Diablo-D3> MR SPEAKER, THE PRESIDENT OF THE... oh wait, its just some black guy, nm
 14 2015-01-21 02:17:30 <cfields> gmaxwell: do you still think it'd be worthwhile to run some of the unit-tests at startup for sanity? I was thinking about moving some of those over today but got distracted by something shinier
 15 2015-01-21 02:20:54 <fanquake> cfields Are happy to accept prs straight to you trivial repo?
 16 2015-01-21 02:21:20 <cfields> fanquake: sorry, i missed a few responses to that
 17 2015-01-21 02:21:32 <cfields> fanquake: yes, but let me shut down my other repos so it's more obvious where they should go
 18 2015-01-21 02:22:25 <cfields> fanquake: you PR'd to an experimental tree i was playing with. I merged it into my current one
 19 2015-01-21 02:22:50 <cfields> if this proves to be worth doing, we should setup a bitcoin/bitcoin-trivial to avoid confusion
 20 2015-01-21 02:23:52 <fanquake> No worries. I think it’d be good to move trivial pulls away from the main repo, reduce the pull request noise a bit.
 21 2015-01-21 02:24:12 <cfields> fanquake: yes, that was exactly the intention
 22 2015-01-21 02:24:53 <cfields> fanquake: https://github.com/theuni/bitcoin/tree/trivial-next is where i'm pulling em in at the moment
 23 2015-01-21 02:26:24 <fanquake> cfields Alright. Can pr against that for now. What’s the plan for merging this back into the main repo?
 24 2015-01-21 02:26:56 <cfields> fanquake: need to talk to wumpus about it. I'm keeping em rebased. I figure maybe PR into master every 2-3 weeks?
 25 2015-01-21 02:27:34 <fanquake> cfields Sounds good.
 26 2015-01-21 02:28:05 <moa> i'd be careful with seemingly 'trivial' changes ...
 27 2015-01-21 02:28:31 <fanquake> cfields Nice working on getting the osx SDK bumped to 10.9 btw. Now that we’ve dropped 10.6 support, can we revisit pulls like #5366
 28 2015-01-21 02:28:51 <cfields> moa: it's only for docs, spelling/grammar of strings, etc
 29 2015-01-21 02:29:00 <moa> ah ok
 30 2015-01-21 02:29:15 <fanquake> Assuming we’re explicitly saying that < 10.6 is now unsupported.
 31 2015-01-21 02:29:16 <moa> no code or build code
 32 2015-01-21 02:29:38 <fanquake> moa correct. Everything that needs ACK/review still goes to the main repo.
 33 2015-01-21 02:30:34 <cfields> fanquake: yes. I'll start by adding a check to configure to verify that the sdk is >=10.7
 34 2015-01-21 02:31:23 <fanquake> There are a few places in the code we’re we’ve got 10.6 specific work arounds, will be good to pull them out.
 35 2015-01-21 02:31:33 <cfields> fanquake: mm, actually, i'm not sure about that...
 36 2015-01-21 02:32:49 <cfields> fanquake: i do think that we should try to build everywhere possible. atm 10.5/10.6 builds work fine afaik, other than the recent font issues. is there a benefit in neutering them?
 37 2015-01-21 02:33:30 <fanquake> cfields I didn’t think 10.5 built at all anymore? I need a VM to be able to comfirm though.
 38 2015-01-21 02:34:52 <fanquake> Although, if it is the case that building still works, I guess we can leave the code as is for now.
 39 2015-01-21 02:35:07 <cfields> fanquake: ok, i'm not sure either. But 10.6 definitely does. Until there's something that absolutely requires >=10.7, i don't see much benefit in nuking it
 40 2015-01-21 02:35:46 <cfields> fanquake: i'm torn on it, depending on which day you ask :)
 41 2015-01-21 02:36:14 <fanquake> cfields heh, I’ll keep it in mind for tomorrow.
 42 2015-01-21 02:36:33 <cfields> heh
 43 2015-01-21 02:37:08 <fanquake> cfields What’s the status on something like #5477 now?
 44 2015-01-21 02:38:26 <cfields> fanquake: i'm not sure it's changed, but i did learn a good bit about the weak syms while fiddling with sdks. let me see if there's a simpler way there
 45 2015-01-21 02:39:50 <fanquake> cfields Is all the code worth it just to supress a deprecation warning?
 46 2015-01-21 02:40:15 <fanquake> OSX seems to be the worst offender for having to maintain lots of OS specific code.
 47 2015-01-21 02:40:35 <cfields> fanquake: well presumably they'll yank the deprecated function in 10.11 or 10.12
 48 2015-01-21 02:41:29 <Diablo-D3> can we just yank osx altogether in 10.11 or 10.12?
 49 2015-01-21 02:42:33 <fanquake> cfields Probably, if not sooner in a point release.
 50 2015-01-21 02:45:20 <gmaxwell> cfields: I think we should run all the tests we can that do not unduely delay startup.
 51 2015-01-21 02:45:35 <gmaxwell> (or do weird things like have /tmp/ races)
 52 2015-01-21 02:46:09 <cfields> gmaxwell: ok. so you'd be in favor of moving many of them into sanity classes, and having the unit tests simply wrap those?
 53 2015-01-21 02:46:18 <cfields> that's what i had in mind, anyway
 54 2015-01-21 02:47:29 <gmaxwell> perhaps, unfortunately the boost instrumentation stuff is really rigidly setup to be running in a seperate test executable.
 55 2015-01-21 02:47:56 <gmaxwell> so I /think/ that means removing boostisms, which is always fine with me.
 56 2015-01-21 02:57:10 <echelon> hey
 57 2015-01-21 02:57:42 <echelon> any trading systems developers here?
 58 2015-01-21 02:58:00 <cfields> fanquake: http://pastebin.com/raw.php?i=s9yWZ0cs
 59 2015-01-21 02:58:19 <cfields> fanquake: i think that should do it. i dunno if the compiler is able to figure out that it should ignore the deprecation there. i assume not.
 60 2015-01-21 02:58:53 <cfields> oops, that's not right. sec
 61 2015-01-21 03:00:17 <cfields> there: http://pastebin.com/raw.php?i=MNA8WBgu
 62 2015-01-21 03:01:39 <fanquake> cfields nice. Much cleaner.
 63 2015-01-21 07:26:14 <unicodesnowman> i like how gavinandresen doesn't consider that that a "medium-range internet connection" in the US is vastly different to a medium range internet connection, in, say Australia
 64 2015-01-21 07:26:36 <unicodesnowman> the average net connection here, to foreign hosts, is around 150 kb/s
 65 2015-01-21 07:27:08 <unicodesnowman> 20000 / 150 / 60 = more than 2 minutes to download a block
 66 2015-01-21 08:47:03 <wumpus> does anyone know of software to benchmark/fuzz/stress test http servers? I'd like to compare #5677 to the old http server, but I feel something like this must already exist
 67 2015-01-21 08:48:41 <poutine> jmeter
 68 2015-01-21 08:51:02 <wumpus> thanks
 69 2015-01-21 09:11:12 <wumpus> httperf and siege look usefu too
 70 2015-01-21 09:13:13 <wumpus> gmaxwell: cfields: probably we want to run the basic crypto and consensus tests, but not the higher level ones for e.g. wallet, rpc, networking and such
 71 2015-01-21 09:21:15 <wumpus> e.g. nothing with side effects or i/o
 72 2015-01-21 09:21:39 <wumpus> those can be converted to sanity test functions and called at every start
 73 2015-01-21 10:03:39 <op_mul> wumpus: I'll work on testing #5677 soonish. good to see SSL going away.
 74 2015-01-21 10:04:58 <op_mul> is that an actual thing, or just a limit of the code today? if it's a lasting change we might want to fail startup of nodes with rpcssl= defined.
 75 2015-01-21 10:07:59 <op_mul> unicodesnowman: if you're running the fast relay node the time is probablt a lot less than that due to pre-fetching of block contents before they arrive.
 76 2015-01-21 10:08:31 <op_mul> ideally nodes would be using this sort of system to deal with larger block sizes.
 77 2015-01-21 10:08:42 <wumpus> op_mul: yes, it should fail with that option given
 78 2015-01-21 10:09:48 <op_mul> wumpus: oh, I didn't see that on the first passthrough, sorry.
 79 2015-01-21 10:10:00 <wumpus> op_mul: no, it *should*, it doesn't at this point
 80 2015-01-21 10:10:20 <op_mul> right
 81 2015-01-21 10:13:01 <wumpus> I haven't heard anyone argue for keeping RPC SSL support yet
 82 2015-01-21 10:14:14 <op_mul> I suspect it's probably one of those things that I don't *want* to know where people are using it.
 83 2015-01-21 10:14:54 <wumpus> everyone is happy to see openssl go as a dependency for bitcoind in the longer run, and it's easy enough to use stunnel, or e.g ssh tunnels
 84 2015-01-21 10:15:02 <wumpus> right
 85 2015-01-21 11:31:15 <wumpus> jonasschnelli: lol, discussion about how to do the wallet transition is as usual going nowhere, just keep on developing we'll make sure it lands in some way
 86 2015-01-21 12:23:29 <jonasschnelli> wumpus, will do. :)
 87 2015-01-21 12:23:38 <jonasschnelli> wumpus, but i think we found a/the way.
 88 2015-01-21 12:27:20 <jonasschnelli> is it true, that encrypting a bitcoind wallet (say over RPC) will leave already generated key unencrypted?
 89 2015-01-21 12:28:59 <op_mul> I don't think so.
 90 2015-01-21 12:29:57 <op_mul> encrypting a wallet flushes the keypool, and naturally the keys which existed unencrypted in other files are still exposed.
 91 2015-01-21 12:32:46 <jonasschnelli> op_mul, hmm... there should be a warning about that (at least).
 92 2015-01-21 12:33:47 <op_mul> encryping a wallet has quite low value unless you plan to store it unused on remote storage.
 93 2015-01-21 12:36:01 <op_mul> most people don't realise that you can see the contents of an encrypted wallet anyway, it's only the private keys which are encrypted and not the public.
 94 2015-01-21 12:37:23 <wumpus> jonasschnelli: no, it will not leave anything unencrypted
 95 2015-01-21 12:37:36 <wumpus> (no keys at least)
 96 2015-01-21 12:38:03 <jonasschnelli> wumpus, yes. just saw it in the code EncryptKeys(vMasterKey);
 97 2015-01-21 12:38:32 <op_mul> I should have worded that better. previously unencrypted keys are encrypted, but that doesn't mean they are safe (ie, that's possibly not the only copy of them)
 98 2015-01-21 12:38:47 <wumpus> of course at a lower level it is still possible for unencrypted files to be on the file system
 99 2015-01-21 12:39:07 <wumpus> for the best security you should encrypt the wallet as the first thing you do
100 2015-01-21 12:39:41 <jonasschnelli> wumpus, for the wallet transition. Would it make sense to allow switching the wallets with a cmd arg? So make init.cpp and rpcwallet.cpp flexible enough to switch wallet classes at runtime?
101 2015-01-21 12:39:53 <wumpus> e.g. it will no longer deal out  keys from before the encryption
102 2015-01-21 12:40:11 <wumpus> jonasschnelli: maybe
103 2015-01-21 12:41:31 <wumpus> jonasschnelli: as said in the pull, make the switching build-time configurable first, so that it's possible to build without bdb. If it's only run time configurable you're still always stuck with the old shit
104 2015-01-21 12:42:02 <wumpus> jonasschnelli: it also makes sure that no code is accidentally shared
105 2015-01-21 12:42:20 <jonasschnelli> wumpus, yes. but we could just build with the new wallet if bdb lib was not found
106 2015-01-21 12:42:59 <jonasschnelli> wumpus, bdb will be required (at least for the official binaries) in near futue anyway because of the migration-tool
107 2015-01-21 12:43:03 <wumpus> jonasschnelli: no, make it explicit
108 2015-01-21 12:43:26 <wumpus> jonasschnelli: no automatic choosing of this
109 2015-01-21 12:43:39 <jonasschnelli> wumpus, okay. Sounds more save.
110 2015-01-21 12:44:34 <wumpus> migration tool would be a separate executable and configuration option, e.g. --enable-wallet-migration-tool
111 2015-01-21 12:44:35 <jonasschnelli> wumpus, i just finish the logdb integration while the legacy handling can be discussed. I think the decisions won't be a big impact on the current implementation.
112 2015-01-21 12:45:13 <jonasschnelli> wumpus, yes. i thought about "db-tool" or "bitcoin-tool" because using the name "wallet" would not allow to do any leveldb stuff in there
113 2015-01-21 12:45:18 <wumpus> jonasschnelli: sure, it's fine to do it on a completely separate branch for now without any switching possibility
114 2015-01-21 12:45:44 <jonasschnelli> wumpus, i also see zapwallettxes and rescue and salvage options within this new tool
115 2015-01-21 12:45:50 <wumpus> jonasschnelli: but it will be more work to keep up with other changes if there's multiple development branches
116 2015-01-21 12:45:55 <jonasschnelli> all this "one shot" startup operations
117 2015-01-21 12:46:03 <jonasschnelli> s/this/these/
118 2015-01-21 12:46:11 <wumpus> jonasschnelli: why would it need to do leveldb stuff in there?
119 2015-01-21 12:47:01 <jonasschnelli> wumpus, just thought there could be other tasks in future which could be done in a such tool. If we name it "wallet-tool", it would be not that flexible... but maybe there will never be a such need
120 2015-01-21 12:47:07 <wumpus> if we ever need to migrate leveldb to something else, that'd require a completely different tool, this tool would just be for wallet migrations
121 2015-01-21 12:47:11 <wumpus> nonono
122 2015-01-21 12:47:15 <wumpus> it's a one-shot tool
123 2015-01-21 12:47:19 <wumpus> just for wallet conversion
124 2015-01-21 12:47:47 <jonasschnelli> wumpus, so you don't think there will be any of these one-shot-tasks outside the wallet?
125 2015-01-21 12:47:51 <wumpus> let's not complicate this unnecesarrily, if some other tool is needed in the future just add another tool
126 2015-01-21 12:48:03 <wumpus> there could be, but let's worry about it then.
127 2015-01-21 12:48:15 <jonasschnelli> wumpus, okay. also fine for me. Then i think "wallet-tool" could be the right name
128 2015-01-21 12:48:28 <wumpus> wallet-migration?
129 2015-01-21 12:48:41 <jonasschnelli> but maybe it should start with "bitcoin" so "bitcoin-wallet-tool"? too long?
130 2015-01-21 12:48:54 <wumpus> fine with me too
131 2015-01-21 12:48:58 <jonasschnelli> wallet-migration would somehow not be in the line with "rescue" and "zapwallettxes"
132 2015-01-21 12:49:24 <wumpus> well the rescue and zap would be in series with the migration in that case
133 2015-01-21 12:49:40 <wumpus> the goal is to migrate, if you need to rescue and zap before that, that's optional
134 2015-01-21 12:49:57 <jonasschnelli> okay. But i'd see the -zapwalltxes gone from bitcoind and added to bitcoin-wallet-tool
135 2015-01-21 12:50:06 <wumpus> so: copy the database to a temporary file, do  whatever you need to make it readable, read it, write out new database in new format
136 2015-01-21 12:50:19 <wumpus> sure
137 2015-01-21 12:51:09 <wumpus> those options would be part of the legacy wallet, but not the new one
138 2015-01-21 12:52:28 <wumpus> (although it may also need rescue options, but there's no need in keeping them option name compatible)
139 2015-01-21 12:52:36 <jonasschnelli> hmm... the new wallet could also make use of 'zapwallettxes' or a 'rescue' in case of a corrupt wallet. So i think the "bitcoin-wallet-tool" should be capable to handle both wallet types and therefor needs to have the legacy stuff compile in with
140 2015-01-21 12:55:36 <jonasschnelli> however, let me continue and see if it really works. :)
141 2015-01-21 12:55:39 <wumpus> hmm I don't know. If you're trying to make some swiss army knife for wallet manipulation that's great, but let's not confuse it with the migration tool.
142 2015-01-21 12:55:57 <jonasschnelli> wumpus, hmm.. yes. Agreed.
143 2015-01-21 12:56:22 <op_mul> is there any reason for not making "new" wallet P2SH by default?
144 2015-01-21 12:56:23 <wumpus> e.g. the migration tool should be careful to not modify the source wallet at all
145 2015-01-21 12:56:40 <wumpus> op_mul: any concrete reason to?
146 2015-01-21 12:57:44 <wumpus> op_mul: not that I'm against it
147 2015-01-21 12:57:58 <op_mul> slightly less UTXO pressure?
148 2015-01-21 13:01:13 <op_mul> I don't think there's any massive reason to.
149 2015-01-21 13:01:42 <wumpus> could be made easier/more transparent to the user to create a P2SH address
150 2015-01-21 13:01:55 <wumpus> but enable it by default, I don't know, it's not terribly urgent imo
151 2015-01-21 14:28:34 <jonasschnelli> concurrency: what happens if a RPC thread does – let's say – getnewaddress, while another thread is doing a wallet encryption? Is it right that there is no locking of critical wallet sections?
152 2015-01-21 14:35:09 <gavinandresen> jonasschnelli: all the wallet RPC should take the cs_wallet lock, either explicitly or implicity via a flag in the RPC function dispatch table
153 2015-01-21 14:36:35 <jonasschnelli> gavinandresen, thanks. I see it now.
154 2015-01-21 14:52:58 <jonasschnelli> just thinking: would it make sense to have the wallet encyption also outside of bitcoind in a new tool "bitcoin-migration-tool"?
155 2015-01-21 14:56:11 <sipa> how would that work?
156 2015-01-21 14:59:42 <jonasschnelli> sipa, calling a cmd tool "./bitcoin-wallet-tool --enc-wallet --file=myWallet.wal", there it ask the user for a passphrase, etc. and will create a MasterKey and call EncryptKeys(MasterKey).
157 2015-01-21 15:03:16 <sipa> jonasschnelli: the wallet itself still needs code to access encrypted wallets
158 2015-01-21 15:03:29 <sipa> so not sure how much you'd win
159 2015-01-21 15:04:39 <jonasschnelli> sipa, Yes. Maybe it's a bad idea. I just stepped over the encryption code and saw, that there is need for a tmp wallet copy and a bitcoind restart.
160 2015-01-21 15:05:00 <jonasschnelli> but maybe bip32 will change that anyway.
161 2015-01-21 15:05:58 <sipa> no
162 2015-01-21 15:06:53 <sipa> the reason for the copy is to prevent the old keys from remaining in some unused sector of the bdb file
163 2015-01-21 15:07:00 <sipa> bip32 changes that
164 2015-01-21 15:07:13 <sipa> an append-only file format does not change that
165 2015-01-21 15:07:31 <sipa> eh, bip32 doesn't fhange that either i mean
166 2015-01-21 15:09:13 <op_mul> for a BIP32 wallet would it make sense to encrypt the MPK and keep a cache of public keys outside?
167 2015-01-21 15:09:57 <jonasschnelli> sipa, after encrypting the wallte i new do a logdb compact rewrite (sipa, okay. After the encryption i'll do a Rewrite/Compact write now: https://github.com/jonasschnelli/bitcoin/commit/6b8f069d4d6d769cdce1ba41079dfef269bc42d0#diff-f217b695d6b8dfcad1b45846b1fb0d32R572)
168 2015-01-21 15:10:38 <gmaxwell> op_mul: we wouldn't be using type-2 normally.
169 2015-01-21 15:10:42 <jonasschnelli> sipa, i assume CKeyingMaterial does somehow delete the key in memory
170 2015-01-21 15:11:11 <op_mul> gmaxwell: oh right of course not. yeah. ignore that totally. why would we be using type 2.
171 2015-01-21 16:17:22 <rfreeman_w> gavinandresen, hi did you yet implemented the voting on blockchain size thing, with 80% from 1000 blocks version as vote? Don't see it in  https://github.com/gavinandresen/bitcoin-git/commits/megablocks
172 2015-01-21 16:19:29 <gavinandresen> rfreeman_w: the megablocks branch is just for testing, there is no code to actually implement the increase for the testnet and main chains.
173 2015-01-21 16:20:17 <gavinandresen> rfreeman_w: … but see the IsSuperMajority function in main.cpp, we used the same technique in the past to upgrade.
174 2015-01-21 16:25:35 <justanotheruser> gavinandresen: but in the past a supermajority of miners was the only requirement.
175 2015-01-21 16:26:25 <justanotheruser> Don't you think it would be better to have a big delay for the hardfork so all the full nodes are onboard if we're going to have it at all?
176 2015-01-21 16:28:08 <gavinandresen> justanotheruser: all full nodes will NEVER get on board. Giving people time to upgrade is a good idea, I’d be fine with either a “bigger blocks OK after THIS date” rule (it is simpler)
177 2015-01-21 16:28:56 <gavinandresen> justanotheruser: I propose the wait for 80% of miners because past experience is it takes many months for that to happen, which should be plenty of time for full nodes to upgrade.
178 2015-01-21 16:29:19 <rfreeman_w> gavinandresen, I would propose to check not last 1000 blocks, but last 1000 blocks starting with blocks at least 6 blocks old, to avoid fluctuation of allowing/disallowing vote if the 999th block gets sidechained
179 2015-01-21 16:30:47 <gavinandresen> rfreeman_w: “meh” : fluctuation is not a problem, since “are big blocks allowed” is a property of whichever fork you’re on.
180 2015-01-21 16:31:06 <rfreeman_w> or in this case, 20 blocks old.  And after finding the vote_block (so that vote_block-1000 .. vote_block has majority),  start applying the new rule after say 3 months, since vote_block + 6*24*30*3
181 2015-01-21 16:31:43 <gavinandresen> rfreeman_w: that’d be fine with me, too.
182 2015-01-21 16:32:24 <gavinandresen> rfreeman_w: … although that starts to get complicated and annoying to code and test....
183 2015-01-21 16:33:04 <gmaxwell> rfreeman_w: I think you're confused; the introduction of the rule is the hardfork: it changes the network to allow currently invalid things; all the rest is just an effort to prevent fraying of the chain in the likely case; it's chain specific and the rules do not cross different tips.
184 2015-01-21 16:34:07 <gavinandresen> Making sure people have time to upgrade can be fixed in simpler ways— like put out a release of Core with all the upgrading logic, but that produces only 1MB old-version blocks.
185 2015-01-21 16:34:25 <gavinandresen> Then N months later ship a release that produces the new-version blocks, to begin the vote.
186 2015-01-21 16:34:39 <gmaxwell> Vote? Thats confused.
187 2015-01-21 16:34:42 <sipa> there should be no need for that
188 2015-01-21 16:35:13 <sipa> if the network has already upgraded to the new rules , miners are free to do what they want
189 2015-01-21 16:35:19 <gavinandresen> How do we tell if a supermajority of hashing has upgraded if there is no on-chain vote?
190 2015-01-21 16:35:22 <sipa> including not creating large block
191 2015-01-21 16:35:31 <sipa> i don't see why a supermajority of hashing matters at all?
192 2015-01-21 16:35:35 <gmaxwell> gavinandresen: who cares if a supermajority of hashing has upgraded?
193 2015-01-21 16:35:46 <gmaxwell> This is not a soft fork.
194 2015-01-21 16:35:47 <sipa> after the fork, only miners who have upgraded are relevant
195 2015-01-21 16:35:59 <sipa> so by definition 100%
196 2015-01-21 16:36:27 <rfreeman_w> gmaxwell, I thoght the idea here is to let hashers decide, instead bitcoin developers saying if/when there will be new limit
197 2015-01-21 16:36:30 <gavinandresen> Lets discuss more in front of a whiteboard…..
198 2015-01-21 16:36:32 <sipa> rfreeman_w: neither
199 2015-01-21 16:36:43 <sipa> rfreeman_w: the network has to decide, by choosing to run a new version
200 2015-01-21 16:37:05 <sipa> developers can suggest, and miners can impose stronger rules than the network enforces (which is a softfork)
201 2015-01-21 16:37:15 <sipa> rfreeman_w: but you're completely confuse dif you think miners set bitcoin's rules
202 2015-01-21 16:37:52 <justanotheruser> gavinandresen: We can't get everyone to upgrade, but I don't think relying on miners numbers is the best way to go since it's the stakeholders that matter. There is probably a correlation between number of miners running the update and number of full nodes running the update, but I don't think 80% miners is good enough. What if that means only 70% of the stakeholders are running a new client? The consequences of that are ...
203 2015-01-21 16:37:53 <gavinandresen> Hmm.  SPV clients care about supermajority of hashpower
204 2015-01-21 16:37:58 <justanotheruser> ... pretty bad.
205 2015-01-21 16:38:23 <gavinandresen> Interesting thing about changing the blocksize is it does NOT require SPV wallets/clients to change anything.
206 2015-01-21 16:38:35 <rfreeman_w> the blog post said the vote is 80% of block.version field, and that field is decided by the pool owners in practice
207 2015-01-21 16:38:52 <sipa> rfreeman_w: which is a fine mechanism for soft forks, because there what matters is a supermajority of hash power
208 2015-01-21 16:38:59 <justanotheruser> rfreeman_w: that's only true if the block size is decreasing
209 2015-01-21 16:39:22 <sipa> rfreeman_w: soft forks are backwards compatible, namely every new block is accepted by old code
210 2015-01-21 16:39:37 <gmaxwell> gavinandresen: not necessarilly true that they don't require changes (quite possible that some have limits on the size of the fragments they'll accept), but in practice it may well be true.
211 2015-01-21 16:39:40 <sipa> hard forks are different, because they're not just making miners enforce additional rules
212 2015-01-21 16:39:43 <op_mul> gavinandresen: I wouldn't make that assumption, I'm aware of at least one piece of "lite" wallet code which enforces limits on the block size it will process.
213 2015-01-21 16:39:56 <sipa> they require everyone to upgrade, because everyone who doesn't excludes themselves from the network
214 2015-01-21 16:40:00 <gavinandresen> op_mul: which one?
215 2015-01-21 16:40:02 <justanotheruser> op_mul: how does it enforce that without being a full node?
216 2015-01-21 16:40:31 <sipa> a non-full node can download blocks and do part of the consensus rules checking still
217 2015-01-21 16:40:53 <gavinandresen> sipa: sure, but in practice the ones I know about don’t.  Because 1MB is too big....
218 2015-01-21 16:41:07 <op_mul> justanotheruser: normally SPV nodes won't know the size, but they can still assume that a block breaking the size rules is invalid.
219 2015-01-21 16:41:10 <gmaxwell> sipa: this is specifically mentioned in the Bitcoin whitepaper, though without a good mechenism to go and do it.
220 2015-01-21 16:41:23 <sipa> gavinandresen: yes, same; i was speaking theoretically
221 2015-01-21 16:41:31 <gavinandresen> Anyway, even if a miner vote is not necessary, it might be a good idea anyway.  Needs more thought.
222 2015-01-21 16:41:46 <sipa> giving miners sufficient time to upgrade is good
223 2015-01-21 16:41:56 <sipa> but it's equally true for every other piece of bitcoin infrastructure
224 2015-01-21 16:42:05 <sipa> miners are not privileged in any way when it comes to a hardfork
225 2015-01-21 16:42:11 <op_mul> justanotheruser: as a node on the network I can happily just ignore the filter a client sends me and just push the entire block to them.
226 2015-01-21 16:42:22 <gmaxwell> "One strategy to protect against this would be to accept alerts from network nodes when they
227 2015-01-21 16:42:25 <gmaxwell> detect an invalid block, prompting the user's software to download the full
228 2015-01-21 16:42:28 <gmaxwell> block and alerted transactions to confirm the inconsistency"
229 2015-01-21 16:42:46 <gmaxwell> (just mentioning it because most people don't recall it)
230 2015-01-21 16:43:04 <sipa> i didn't, indeed
231 2015-01-21 16:43:58 <op_mul> gavinandresen: mine.
232 2015-01-21 16:44:10 <rfreeman_w> btw some users will likelly run old bitcoin client for months, or start it up after a year suddenly, can we protected from fraud here, e.g. if someone would maintain old hardfok with small maxsize, at small cost (tiny hashpower) just to scam people into buying btc on invalid chain?
233 2015-01-21 16:45:06 <op_mul> network clients do make local alerts if they see a long chain that is invalid but has a valid proof of work.
234 2015-01-21 16:45:07 <sipa> if not everyone upgrades, that is a risk
235 2015-01-21 16:45:11 <rfreeman_w> just thinking out loud - I know few friends who use bitcoin sporadically, like yearly "what is the price now - oh ok I will buy some after xmass"
236 2015-01-21 16:45:18 <gmaxwell> rfreeman_w: Well careful with language there; there are some people now loudly saying that a larger blocksize chain is the 'scam invalid chain'; better to avoid value loaded words.  To your question; at least bitcoin core yells at you when there is a long high work chain tip that your node is rejecting.
237 2015-01-21 16:45:35 <gmaxwell> What you do about that is up to you.
238 2015-01-21 16:48:27 <rfreeman_w> gmaxwell, ok;  To clarify I only meant that someone could *lie* on purpose to buyer that btc he sells is not spent, while knowing he transferred it to someone else (on the new chain).  Yeah the node yelling should help
239 2015-01-21 16:49:21 <gmaxwell> rfreeman_w: if the chains are seperate they can just have orthorgonal coins; e.g. from subsidy. So its not limited to actual double spending.
240 2015-01-21 16:49:45 <sipa> rfreeman_w: the fundamental problem is that bitcoin nodes consider "the longest chain that does not violate any rules" as the current state of the world
241 2015-01-21 16:49:55 <sipa> rfreeman_w: a hard fork changes what thuse rules are
242 2015-01-21 16:50:17 <gmaxwell> rfreeman_w: e.g. I can pay you 100 BTC plus 0.01 BTC from a coinbase on a particular chain that isn't on another and it guarentees the transaction will only show up on one side. The coins on the other side could remain unmoved.
243 2015-01-21 16:50:22 <sipa> so 'longest' is not longer relevant to distinguish the two chains - one is just valid and the other isn't
244 2015-01-21 16:51:29 <gmaxwell> rfreeman_w: If you think there may be a substantial long lasting fork, it may be prudent to orthorgonalize your coins that way; so that you'd preserve the abilit to spend the coins on the alternative chain, should one arise. :(
245 2015-01-21 16:55:04 <justanotheruser> Couldn't stakeholders just include an OP_RETURN output with their vote for the hardfork? Coin days destroyed weighed votes is probably a better metric for number of stakeholding full nodes on board.
246 2015-01-21 16:55:16 <op_mul> I thought it might be interesting to look into how altcoins do hard forks.. there's nothing to be gained from that. they generally just say "hard fork tomorrow", push new binaries, and then everybody sits around in a daze for a while wondering why their wallets are behind.
247 2015-01-21 16:56:08 <gmaxwell> op_mul: situation is sadly pretty different on systems without much actual use.
248 2015-01-21 16:56:15 <sipa> justanotheruser: or post signed messages that prove control over coins
249 2015-01-21 16:56:33 <sipa> justanotheruser: but control of coins != ownership of coins
250 2015-01-21 16:57:14 <justanotheruser> sipa: control of coins is more important than ownership during a hardfork
251 2015-01-21 16:57:59 <sipa> how so? even though my coins are stored at MySuperWebWallet doesn't mean i necessarily have the same opinion about a fork as they do
252 2015-01-21 16:58:08 <gmaxwell> justanotheruser: there was some incomplete and somewhat complex proposal about signaling support for larger blocks on an ongoing with a flag in transactions. Coins days destroyed as you said. But beyond the technical complexity you have points like sipas. Though; presumably control of coins is a more interesting metric than hashpower. A problem you have though using anything in transactions is that m
253 2015-01-21 16:58:09 <op_mul> gmaxwell: I thought there might be something to be gained from observing how one of the big altcoins managed it like darkcoin or dogecoin.
254 2015-01-21 16:58:14 <gmaxwell> iners can censor transactions.
255 2015-01-21 16:58:16 <sipa> and it's my money that would be affected by the potential economic consequences, not theirs
256 2015-01-21 16:58:38 <gmaxwell> op_mul: would be good to do some research. I know dogecoin had really severe forking problems, and issues getting miners to upgrade.
257 2015-01-21 16:59:06 <rfreeman_w> people wanting to vote could withdraw, cast vote, and send again.  Btw that could cause a bank run / liquidity prove, for the better or the worse :)
258 2015-01-21 16:59:21 <gmaxwell> Somewhat similar to the issues we've found recently where some miners were staying on 0.8.1 due to having RPC issues with newer versions that they never really bothered looking into. Just "oops, this doesn't work, fallback"
259 2015-01-21 16:59:34 <sipa> rfreeman_w: doesn't prevent the one with control from already voting
260 2015-01-21 16:59:41 <op_mul> gmaxwell: the major dogecoin forking problem was due to the developers pushing a binary with a hard fork in it with 1-2 days notice. they hit just the wrong time when there was 50% hashpower on each side so the network caploded.
261 2015-01-21 17:00:24 <rfreeman_w> sipa, right, unless the voting period (say 1 month) would be announced say 2 months ahead, so everyone could take their coins before they are used to vote for them?
262 2015-01-21 17:00:54 <justanotheruser> sipa: in the case that you disagree and they agree, you are upset, but your coins aren't on a fork
263 2015-01-21 17:01:34 <sipa> justanotheruser: my coins' value is potentially affected by the change
264 2015-01-21 17:01:40 <sipa> (in both ways)
265 2015-01-21 17:02:50 <justanotheruser> sipa: right, your opinion is important, but not as important as the controllers IMO. Either way, it's better than you just being on the wrong fork because the state of your stake wasn't considered by anyone
266 2015-01-21 17:03:06 <gmaxwell> rfreeman_w: you don't know that the delegation won't go the way you want until its too late; also see my comment on miner censorship.  The prior proposal on this front dealt with that by saying the default (absent a transaction) is that everyone votes no increase; the rational is that miners can already produce smaller blocks rules be damned, so the only conflict of interest that you can do anything
267 2015-01-21 17:03:12 <gmaxwell> about is if miners want an increase but other users of the system do not.
268 2015-01-21 17:03:12 <op_mul> gmaxwell: developers comment was "That whole "update your client to v1.3, it's mandatory" thing meant, update your client to v1.3, it's mandatory." somehow I suspect if that happened with Bitcoin the topic would not be titled "much concern".
269 2015-01-21 17:03:42 <op_mul> gmaxwell: and it was 10 days, not 2 like I thought.
270 2015-01-21 17:04:10 <gmaxwell> op_mul: did they suffer serious divergence with the MM deployment?
271 2015-01-21 17:10:12 <op_mul> gmaxwell: as far as I can tell, no. they scheduled the switch in a new major version, specified a block height (371,337) for the fork. there's a small number of people complaining their wallets got stuck, but I can't find anybody mentioning major services getting lost.
272 2015-01-21 17:18:22 <justanotheruser> gmaxwell: I would expect the tx to work their way into a block eventually unless you're talking about a 51% attack
273 2015-01-21 17:20:17 <gmaxwell> justanotheruser: why would you? common interest. depending on a tx inclusion is isomorphic to "just let the majority of miners decide." since if the majority of miners disapprove they can exclude it.
274 2015-01-21 17:21:46 <justanotheruser> gmaxwell: so you're talking about a 51% attack? In that case, at least it collapses to gavins idea where miners vote on the hardfork. In the case it isn't an attack, you have stakeholders voting.
275 2015-01-21 17:22:08 <op_mul> 
ACTION mutters about the term 51% attack

276 2015-01-21 17:23:08 <justanotheruser> op_mul: what, would you prefer I say miners chose collectively to softfork so OP_RETURN <vote for/against hardfork message> was an invalid tx?
277 2015-01-21 17:24:34 <op_mul> yes, just the phrase 51% attack doesn't make a pile of sense, that's all.
278 2015-01-21 17:27:00 <justanotheruser> It seems some softforks are considered by some coredevs to be 51% attacks based on the ethical implications of them. I'm not an expert on Bitcoin ethics, so I can't comment on that.
279 2015-01-21 17:29:59 <Luke-Jr> justanotheruser: even if there was no softfork, enough miners could use their policy to control favour on such a vote
280 2015-01-21 17:31:31 <gmaxwell> justanotheruser: you should probably just never use the words "51% attack" it's meaningless by itself.
281 2015-01-21 17:39:45 <justanotheruser> Luke-Jr: Why couldn't the miners who aren't ignoring votes accept the ignored tx and make a big block?
282 2015-01-21 17:40:06 <Luke-Jr> justanotheruser: they could.
283 2015-01-21 17:40:20 <Luke-Jr> but if 80% of miners want it one way, 20% won't necessarily be able to counter it
284 2015-01-21 17:40:58 <justanotheruser> It's basically proportional to the block size afaics
285 2015-01-21 17:42:02 <justanotheruser> .2 mb blocks on average means .2 of the hashrate needs to be honest for votes to be counted
286 2015-01-21 17:43:20 <justanotheruser> It gets more complex when someone has such an economic interest in the hardfork that they make a ton of spam tx that vote one way and miners get paid more if they ignore some votes
287 2015-01-21 18:04:08 <jps> :)
288 2015-01-21 18:04:16 <jps> lol oop
289 2015-01-21 19:53:57 <wumpus> interesting, libevent has built-in support for rate limiting connections, may come in handy if we ever want to use it for P2P too
290 2015-01-21 20:08:30 <andytoshi> dumb question about the dersig bip: it says R cannot start with null bytes unless the first byte after that is 0x80 or higher (in which case one null is needed). this seems to exclude numbers ≥ 0x8000..000 and also numbers ≤ 0x008000..000, for both r and s
291 2015-01-21 20:08:32 <andytoshi> why?
292 2015-01-21 20:08:56 <andytoshi> i thought, s needed to be restricted to half its range to avoid the (r, s) → (r, -s) malleability, but that's it?
293 2015-01-21 20:09:48 <sipa> andytoshi: the _encoding_ cannot start with null bytes
294 2015-01-21 20:09:52 <sipa> but it's variable length
295 2015-01-21 20:10:15 <sipa> the half-range restriction of s has nothing to do with DER
296 2015-01-21 20:10:35 <sipa> this no-leading-null bytes requirement comes directly from the DER standard
297 2015-01-21 20:10:43 <andytoshi> ok. why is the encoding like this then? to avoid misparses as negative numbers?
298 2015-01-21 20:11:31 <sipa> DER guarantees a unique encoding
299 2015-01-21 20:11:42 <sipa> so it forces you to use the shortest possible one
300 2015-01-21 20:12:02 <andytoshi> naively i would think "shortest possible" means no leading 0's unconditionally
301 2015-01-21 20:12:15 <sipa> unfortunately, BER only has signed integers
302 2015-01-21 20:12:28 <andytoshi> ah ok, i'm happy
303 2015-01-21 20:12:32 <sipa> if the highest bit is set, it's interpreted as two's complement
304 2015-01-21 20:12:36 <andytoshi> gotcha
305 2015-01-21 20:12:40 <sipa> well, it always is two's complement
306 2015-01-21 20:13:16 <andytoshi> thx for the answers, sorry, i should've looked this up..
307 2015-01-21 20:13:30 <andytoshi> i was confused because i got the idea that it had something to do with s vs -s in my head..
308 2015-01-21 20:13:50 <sipa> well if it wasn't clear from the document, perhaps it needs clarification
309 2015-01-21 20:14:50 <andytoshi> it's in the comments in the code, but not in the summary comment up top
310 2015-01-21 20:16:38 <andytoshi> i think after "R: arbitrary-length big-endian ... single null byte is required." you should add "This is the unique shortest encoding of a positive signed integer."
311 2015-01-21 20:18:05 <sipa> good
312 2015-01-21 20:22:16 <sipa> andytoshi: how do you like it now?
313 2015-01-21 20:22:37 <sipa> ah, you want it in the comment; nvm
314 2015-01-21 20:25:47 <sipa> andytoshi: check again
315 2015-01-21 20:25:49 <andytoshi> sipa: aside from that comment i'm happy with the whole thing. at first i thought you said `lenR` in a couple places where you meant `lenS` but it turned out i was wrong on both counts :)
316 2015-01-21 20:26:15 <cfields> andytoshi: heh, i did the same
317 2015-01-21 20:26:17 <andytoshi> sipa: great, thx :)
318 2015-01-21 20:26:21 <andytoshi> i'll ACK on the list
319 2015-01-21 22:39:03 <cfields> jtimon: ping
320 2015-01-21 22:57:07 <jtimon> cfields pong
321 2015-01-21 22:57:59 <cfields> jtimon: i'd like to spend a few days furiously moving files out into a consensus module as we discussed the other day
322 2015-01-21 22:58:11 <cfields> but it's going to conflict with your work a good bit obviously
323 2015-01-21 22:58:21 <cfields> i'd like to find some way to work together a little better
324 2015-01-21 22:58:58 <cfields> thoughts/suggestions?
325 2015-01-21 23:00:45 <jtimon> that sounds great
326 2015-01-21 23:01:42 <cfields> would it help for us to work in the same branch?
327 2015-01-21 23:01:54 <jtimon> I'm sorry that right now is a moving target
328 2015-01-21 23:02:25 <jtimon> yes, a common branch would definitely help I think
329 2015-01-21 23:03:08 <cfields> ok
330 2015-01-21 23:03:24 <jtimon> I'm just cleaning up consensus_policy
331 2015-01-21 23:05:06 <jtimon> I'll push it and we can talk about which parts belong to a common base and talk about some nits or something
332 2015-01-21 23:06:26 <cfields> sounds good, thanks
333 2015-01-21 23:09:10 <jtimon> great, I'll ping you again in a moment
334 2015-01-21 23:14:38 <jtimon> + e03890f...95e2750 consensus_policy -> consensus_policy (forced update)
335 2015-01-21 23:15:57 <jtimon> cfields so at the top (after f07dd126) there's many refactor things
336 2015-01-21 23:16:37 <jtimon> that I'm just considering, probably not worth it to put them in a common branch
337 2015-01-21 23:19:18 <jtimon> 77a2a4d1 probably needs squash, but maybe it was touching too much...
338 2015-01-21 23:20:26 <cfields> looking
339 2015-01-21 23:21:44 <cfields> heh, no idea how we can work together here :)
340 2015-01-21 23:21:57 <jtimon> so I would suggest something like "f07dd126 LOCAL: notes on include main.h" as something reasonable
341 2015-01-21 23:22:12 <jtimon> maybe without that last commits with notes
342 2015-01-21 23:23:23 <cfields> basing my work on that, you mean?
343 2015-01-21 23:24:00 <jtimon> or I can rebase that on top of your work
344 2015-01-21 23:24:35 <cfields> hmm
345 2015-01-21 23:25:08 <cfields> how about i hack for a day or so, and come up with some realistic conflicts. then we can see what makes sense
346 2015-01-21 23:25:12 <jtimon> but if you see something you would nack specially on the first commits, please tell me
347 2015-01-21 23:26:49 <cfields> jtimon: i'd skip stuff like 881260472 until the refactors have been merged
348 2015-01-21 23:27:02 <cfields> that just serves to complicate things
349 2015-01-21 23:27:58 <jtimon> it was a nit, but sure, it's separated so that's easier to move last
350 2015-01-21 23:31:45 <jtimon> so would rebasing your work on top of say 7acf782c 6c2ac8ca or c9bd1dc0 (after whatever nits you have) be hard?
351 2015-01-21 23:32:07 <jtimon> or do you think it's a bad idea?
352 2015-01-21 23:36:29 <cfields> not sure yet. i'd like to somehow order it by likelihood of being merged so we can just pop things off the bottom, but that's really hard to predict
353 2015-01-21 23:38:48 <Dizzle> Hey guys. #bitcoin topic says 0.9.4, website only has 0.9.3 binaries. what's up?
354 2015-01-21 23:39:23 <gmaxwell> Dizzle: website actually does have 0.9.4, though 0.9.4 is really only a release needed for people who were building their own.
355 2015-01-21 23:39:27 <sipa> Dizzle: 0.9.4 only has very minor fixes apart from one change that is only relevant to people who build from source
356 2015-01-21 23:39:47 <sipa> but we should publish 0.9.4 binaries imho
357 2015-01-21 23:41:15 <Dizzle> Ahh, ok. Thanks! Yea, I just noticed that https://bitcoin.org/en/download says "Latest version: 0.9.3" is all.
358 2015-01-21 23:41:36 <jtimon> ok cfields another try without a few potential (and one confirmed) offenders: 94656864 in jtimon/consensus_policy2
359 2015-01-21 23:41:36 <sipa> wumpus, cfields: where are we with publishing 0.9.4 binaries?
360 2015-01-21 23:41:54 <gmaxwell> There will be a 0.10 soon; so we have been avoiding creating a totally pointless release cycle for people who are running bitcoin.org binaries.
361 2015-01-21 23:42:08 <gmaxwell> Since they don't need to upgrade and if they do they may be slower in installing 0.10.
362 2015-01-21 23:42:47 <sipa> yeah, the main reason for publishing  0.9.4 is to avoid confusion afaik
363 2015-01-21 23:42:51 <sipa> s/afaik/imho/
364 2015-01-21 23:43:00 <cfields> sipa: i agree with gmaxwell
365 2015-01-21 23:43:22 <cfields> sipa: if anything, i'd prefer to push 0.9.4 alongside 0.10, for those who want to say on 0.9
366 2015-01-21 23:43:32 <sipa> ok
367 2015-01-21 23:44:03 <cfields> (that's assuming we get 0.10 out reasonably quickly)
368 2015-01-21 23:44:40 <jtimon> what the main 0.10 blocker right now?
369 2015-01-21 23:44:49 <jtimon> what's
370 2015-01-21 23:44:52 <sipa> i'd like to see the DERSIG softfork in
371 2015-01-21 23:45:08 <phantomcircuit> and also just time
372 2015-01-21 23:45:24 <phantomcircuit> would be annoying to do a 0.10.0 release and then immediately do a 0.10.1 release
373 2015-01-21 23:45:32 <jtimon> I see
374 2015-01-21 23:45:37 <moa> field testing
375 2015-01-21 23:46:02 <sipa> ok, reformulate: DERSIG is an rc4 blocker
376 2015-01-21 23:46:31 <moa> anyone know how many rc clients have been seen on the main net?
377 2015-01-21 23:48:38 <Luke-Jr>      72 70002 "/Satoshi:0.10.0/" unknown
378 2015-01-21 23:49:09 <moa> so 72??
379 2015-01-21 23:49:15 <Luke-Jr> yes
380 2015-01-21 23:49:17 <Luke-Jr> listening
381 2015-01-21 23:49:18 <moa> thnx
382 2015-01-21 23:49:28 <Luke-Jr> can't tell non-listening
383 2015-01-21 23:59:12 <benjamindees> so, here's an idea to think about with regards to block size
384 2015-01-21 23:59:27 <benjamindees> (not sure if it's been proposed before because frankly I haven't been paying close attention)
385 2015-01-21 23:59:55 <benjamindees> one main chain limited to something reasonable, 5mb or so