1 2015-04-14 01:04:06 <tdryja> Hi, question about testnet3
  2 2015-04-14 01:04:28 <tdryja> anyone else seeing lots of orphans...?
  3 2015-04-14 01:04:38 <tdryja> I can't even tell what the current best height is
  4 2015-04-14 01:05:09 <tdryja> public block explorer websites:
  5 2015-04-14 01:05:19 <tdryja> test-insight.bitpay.com says 333019
  6 2015-04-14 01:05:22 <j4m13> Hey, i know not to discuss Altcoin-dev here however, does anyone know the any good IRC addresses where i could discuss?
  7 2015-04-14 01:05:34 <tdryja> www.biteasy.com says 332934
  8 2015-04-14 01:05:56 <tdryja> blockexplorer.com/testnet says 333500
  9 2015-04-14 01:06:38 <tdryja> http://test.webbtc.com/ says 330626
 10 2015-04-14 01:07:12 <tdryja> I have a btcd testnet node which reports 339824
 11 2015-04-14 01:13:58 <tdryja> I have another btcd node
 12 2015-04-14 01:14:03 <tdryja> reports height 340946
 13 2015-04-14 01:15:05 <tdryja> If this were mainnet, this would be, like, panic... nobody cares because it's testnet I guess, but... I kind of want to know what's going on
 14 2015-04-14 01:16:06 <tdryja> maybe just some mining power making a mess of things
 15 2015-04-14 01:29:15 <tdryja> I'm getting a block every ~2 secs
 16 2015-04-14 01:30:14 <tdryja> someone's mining quite a bit; the block explorers are all broken.  Maybe it's their fault, but I wonder what people with testnet bitcoind's are reportin
 17 2015-04-14 01:30:15 <tdryja> g
 18 2015-04-14 01:59:47 <davec> tdryja: There is clearly one or more ASICs on testnet having fun with reorgs.  I have two different BCs reporting different values as well, one is a few hundred blocks behind the other.  I suspet it's because that's a slower machine
 19 2015-04-14 02:23:40 <gmaxwell> tdryja: I looked earlier and it appeared to just be an unintentional warp to diff 1 and then high block rate catching back up to a sane difficulty.
 20 2015-04-14 02:24:20 <gmaxwell> If anyone has a bitcoin core node rejecting a chain on testnet now, I'd love to hear about it.
 21 2015-04-14 02:25:21 <davec> same here with a btcd node.  I show BC and btcd on the same machine in sync.
 22 2015-04-14 02:25:24 <davec> $ btcctl -C ~/btcdtestnet.conf  getblockcount; btcctl -C ~/bitcoincoretestnet.conf  getblockcount
 23 2015-04-14 02:25:27 <davec> 352033
 24 2015-04-14 02:25:29 <davec> 352033
 25 2015-04-14 02:28:31 <gmaxwell> there are some explorers that just won't handle moderate length reorgs at all, e.g. blockexplorer IIRC would only handle 6 blocks or something like that.
 26 2015-04-14 02:28:44 <gmaxwell> (or 12? I forget, I've broken it a bunch of times)
 27 2015-04-14 02:28:58 <gmaxwell> (not intentionally of course)
 28 2015-04-14 02:29:59 <davec> oh I didn't put testnet in my conf there. That was mainnet.  Let me do that again
 29 2015-04-14 02:30:24 <davec> $ btcctl -C ~/btcdtestnet.conf getblockcount; btcctl -C ~/bitcoincoretestnet.conf getblockcount
 30 2015-04-14 02:30:27 <davec> 342299
 31 2015-04-14 02:30:29 <davec> 342299
 32 2015-04-14 02:32:18 <davec> does that match what you're seeing as well gmaxwell?  (give or take a few blocks due to the speed at which they're coming in)
 33 2015-04-14 03:35:43 <CodeShark> https://docs.google.com/document/d/1BlRmpeR5iaeqKGjkIht96blpVoI3QPWxX_P6hJxmG0Y
 34 2015-04-14 03:44:18 <moa> CodeShark: crypto-economic
 35 2015-04-14 03:44:57 <CodeShark> I guess it makes it easier to read, so why not? :p
 36 2015-04-14 03:45:01 <moa> dehyphenating back-to-back vowels is bad form ... without a long history of such usage
 37 2015-04-14 03:45:36 <CodeShark> yes, you raise a good point
 38 2015-04-14 03:45:39 <CodeShark> thanks :)
 39 2015-04-14 03:45:57 <moa> np
 40 2015-04-14 03:46:50 <gmaxwell> CodeShark: thats BS.
 41 2015-04-14 03:47:06 <gmaxwell> CodeShark: there have been tremendous improvements to Bitcoin since 2009.
 42 2015-04-14 03:47:07 <CodeShark> what is?
 43 2015-04-14 03:47:13 <CodeShark> ok, name one
 44 2015-04-14 03:47:35 <moa> multi-sig
 45 2015-04-14 03:47:47 <gmaxwell> CodeShark: P2SH, for example. And the fact that the implemention is well over 100 times faster.
 46 2015-04-14 03:47:49 <CodeShark> multisig was considered in the original implementation but was disabled deliberately
 47 2015-04-14 03:48:04 <gmaxwell> CodeShark: wtf. It wasn't disabled.
 48 2015-04-14 03:48:18 <CodeShark> well, it was considered "nonstandard"
 49 2015-04-14 03:48:23 <moa> though of != implemented
 50 2015-04-14 03:48:46 <CodeShark> the use case was already covered, in principle, in the script
 51 2015-04-14 03:49:01 <CodeShark> but only a handful of script types were really allowed
 52 2015-04-14 03:49:22 <gmaxwell> CodeShark: no, base multisig was only non-standard during a narrow window. (just because all scriptpubkeys other than pay to pubkey/pay to pubkey hash were the only things permitted for a few versions)
 53 2015-04-14 03:49:25 <CodeShark> and regarding the sync optimizations, those are not protocol improvements
 54 2015-04-14 03:49:41 <CodeShark> those are sipa rewriting entire chunks of code
 55 2015-04-14 03:49:57 <CodeShark> to basically have the same behavior, just faster
 56 2015-04-14 03:50:05 <moa> this is a meta discussion for #bitcoin-wizards ?
 57 2015-04-14 03:50:06 <gmaxwell> CodeShark: addition of bloom filters; which save enormous bandwidth for spv clients.
 58 2015-04-14 03:50:23 <sipa> payment protocol
 59 2015-04-14 03:50:27 <CodeShark> bloom filters are currently the only way to support SPV...but they are not a long-term solution to scalability
 60 2015-04-14 03:50:34 <gmaxwell> CodeShark: Thats not strictly correct, we use the protocol differently today.
 61 2015-04-14 03:50:39 <CodeShark> the payment protocol is not a core protocol improvement
 62 2015-04-14 03:50:44 <CodeShark> it's a UI thing, mainly
 63 2015-04-14 03:50:57 <sipa> CodeShark: so what core improvements do you think are needed?
 64 2015-04-14 03:51:00 <gmaxwell> (e.g. headers first; the getheaders stuff wasn't even in the protocol orignially)
 65 2015-04-14 03:51:27 <Luke-Jr> CodeShark: why worry about protocol improvements when *the existing protocol* is not being used?
 66 2015-04-14 03:51:53 <sipa> i read your manifesto as "we need a procedure to introduce hard forks"
 67 2015-04-14 03:52:05 <CodeShark> much greater scalability with partitioned storage, greater availability of data requested by thin clients along with a way to construct short proofs of their validity
 68 2015-04-14 03:52:28 <CodeShark> greater extensibility of scripts to be able to make use of stateful data
 69 2015-04-14 03:52:40 <gmaxwell> Such bullshit; CodeShark you can't just define out all progress. I see you've defly ignored P2SH and thigs like the block relay protocol.
 70 2015-04-14 03:52:56 <sipa> partitioned storage will not offer brtter scalability, unless you also trade in security
 71 2015-04-14 03:53:32 <CodeShark> I'm not so sure, sipa - in any case, I think these topics have been all but ignored by core devs because they are at best wishful thinking with our current dev process
 72 2015-04-14 03:53:43 <gmaxwell> All but ignored?
 73 2015-04-14 03:53:54 <CodeShark> well, they have not gone beyond a few online discussions
 74 2015-04-14 03:53:59 <CodeShark> there are no implementations, no tests
 75 2015-04-14 03:54:23 <gmaxwell> You realize that most of the "scaling improvements" out there that have gone in discussions before were initially suggested by us right?
 76 2015-04-14 03:54:35 <CodeShark> if you want, gmaxwell, I can change the language a bit to not sound so dismissive
 77 2015-04-14 03:54:36 <gmaxwell> And yes, going out and building them hasn't been a priority absent needs.
 78 2015-04-14 03:55:02 <CodeShark> but let's face it, satoshi's protocol is an experimental prototype that's become ossified and nearly impossible to evolve
 79 2015-04-14 03:55:11 <gmaxwell> CodeShark: You can first try changing your understanding to not be so completely ignorant; as it stands anyone clueful will just throw your writing in the trash.
 80 2015-04-14 03:55:24 <sipa> CodeShark: have you read the sidechains paper? :)
 81 2015-04-14 03:55:45 <Apocalyptic> this discussion is painfull to read, so much bs...
 82 2015-04-14 03:55:47 <gmaxwell> (and thing coming from someone who also agrees that there is a great need for greater ability to try different things!)
 83 2015-04-14 03:57:01 <Luke-Jr> CodeShark: what sipa said; sidechains are aimed to solve the difficulty of protocol evolution
 84 2015-04-14 03:57:05 <CodeShark> yes, I have, sipa
 85 2015-04-14 03:57:11 <Luke-Jr> (well, at least make it significantly easier)
 86 2015-04-14 03:57:19 <sipa> CodeShark: i agree partially... i think we cannot just make changes to the consensus rules, but because i do not think we have authority over it (if we do, what's the point in decentralization?)
 87 2015-04-14 03:57:20 <gmaxwell> Luke-Jr: our at least one area of improvement;
 88 2015-04-14 03:57:28 <CodeShark> sidechains are a potentially useful tool for this...but it's just one component
 89 2015-04-14 03:57:43 <CodeShark> not to be dismissive - I support what you're doing
 90 2015-04-14 03:58:03 <Luke-Jr> CodeShark: the other component is the BIP process
 91 2015-04-14 03:58:29 <sipa> CodeShark: i think many of the things you are sayig are definitely considered... but just not ready. not technically, and not politicaly
 92 2015-04-14 03:58:52 <sipa> CodeShark: one way to make them ready is being able to experimemt with them
 93 2015-04-14 03:59:05 <CodeShark> sipa: yes, absolutely - we need many testnets
 94 2015-04-14 03:59:07 <CodeShark> not just one
 95 2015-04-14 03:59:08 <sipa> which is currently just not happening
 96 2015-04-14 03:59:52 <sipa> testnets can experiment with some tecnhnical aspects, but without economic value, that's only part of the system
 97 2015-04-14 04:00:07 <CodeShark> right - the incentives need recalibration
 98 2015-04-14 04:00:13 <sipa> (and Bitcoin only works and is secure due to economic arguments)
 99 2015-04-14 04:00:21 <CodeShark> yes, indeed
100 2015-04-14 04:00:44 <sipa> if what you are saying is that we need more people money and time to work on fundamental improvements, i fully agree
101 2015-04-14 04:01:00 <CodeShark> I just want more people to be conscious of these issues
102 2015-04-14 04:01:10 <sipa> i am very conscious
103 2015-04-14 04:01:17 <CodeShark> I know most of the core devs are at least somewhat conscious of them :p
104 2015-04-14 04:01:41 <CodeShark> but we need more support from the community at large for big changes to occur
105 2015-04-14 04:02:02 <gmaxwell> Misleading it about the current level of activity is only going to make people flame out.
106 2015-04-14 04:02:17 <CodeShark> ?
107 2015-04-14 04:03:07 <sipa> you present it as if there is no evolution and no research
108 2015-04-14 04:03:35 <gmaxwell> Reading your screed there about other ideas being proposed elsewhere made it sound like (1) we were doing nothing (and ignoring the actual protocol improvements; both at the consensus level, and the non-consensus ones like the block relay protocol) (2) making it sound like we already weren't the origin of many of the scalability proposals that exist in the space at all-- the actual research the event
109 2015-04-14 04:03:41 <gmaxwell> ually leads to new things in deployment.
110 2015-04-14 04:04:24 <sipa> there is evolution, but indeed - no total reworkings. But that is not because we do not want to, just brcause they are not ready. Some people are researching very significant changes.
111 2015-04-14 04:04:35 <gmaxwell> It's doubly insulting being presented by an emergency from someone who is able to contribute, but doesn't, ... leaving those of us who do busy as heck keeping things going.
112 2015-04-14 04:04:36 <CodeShark> gmaxwell: your reaction somewhat surprizes me considering the fact you've obviously invested a lot of time and effort into thinking of things we want to try but can't do with the current network
113 2015-04-14 04:04:39 <moa> CodeShark: do you have specific improvements or "missed technologies" that bitcoin should already have incorporated ... as the Introduction seems to be suggesting?
114 2015-04-14 04:05:18 <CodeShark> the hardfork wishlist and your alt ideas are some of my favorite links, gmaxwell
115 2015-04-14 04:05:25 <gmaxwell> CodeShark: sorry then, to read your writing it sounded like you were saying no one has put in that effort.  And it's not a question of "can't try" (well some things are)
116 2015-04-14 04:05:35 <CodeShark> no, that's not what I'm saying at all
117 2015-04-14 04:05:54 <CodeShark> there's been tremendous effort...but as sipa said, nobody has the authority nor the ability to make certain changes...and so we're stuck just talking about them
118 2015-04-14 04:06:06 <gmaxwell> (I've mostly stopped updating them, since they were getting copy and pasted into altcoin whitepapers; latest thinking goes a fair bit beyond them.)
119 2015-04-14 04:06:09 <CodeShark> so rather than evading this issue we should codify it
120 2015-04-14 04:06:43 <gmaxwell> I actually don't feel that there is an authority problem at all in fact, somewhat the opposite. There is a challenge in engineering software for such a critical deployment.
121 2015-04-14 04:07:13 <CodeShark> gmaxwell: well, there's both a technical and a political challenge
122 2015-04-14 04:07:13 <gmaxwell> (there should be more of an authority problem as there as, so far it looks like we write a bip and test things out and people generally go along with it)
123 2015-04-14 04:07:28 <moa> CodeShark: codifying may just move the "ossify" to the political process ...
124 2015-04-14 04:07:34 <moa> more likely imho
125 2015-04-14 04:07:45 <CodeShark> moa: indeed! so it's extremely tricky
126 2015-04-14 04:08:32 <moa> just wait until you have committe meetings with govt. departments over every line of code change
127 2015-04-14 04:08:37 <gmaxwell> I've never had a proposal die on 'political' grounds, I can only think of one: the getutxo, which we rejected because it makes the network more dependant on unverifable data; and risks incentivzing storing crap in the utxo set as a random access database.
128 2015-04-14 04:09:01 <moa> the golden days of experimentation are behind us ... it happens to all mission-critical software
129 2015-04-14 04:09:18 <gmaxwell> It's just difficult to deploy things under value, not enough isolation. And I'm disinterested in deploying code which will just end up in a competing system. (the expirence with coiledcoin was very negative)
130 2015-04-14 04:09:28 <CodeShark> moa: then I'm afraid we're going to crash
131 2015-04-14 04:09:28 <sipa> CodeShark: i am not totally happy about the current way thigs are. On one hand it is not just hard (which is fine) but also untransparent how a non-trivial hardfork would happen. On the other gand, i am even more afraid of having a committee that can dictate changes.
132 2015-04-14 04:10:02 <CodeShark> sipa: I was thinking of something more along the lines of a decentralized voting scheme
133 2015-04-14 04:10:15 <sipa> CodeShark: who would get to votem
134 2015-04-14 04:10:16 <sipa> ?
135 2015-04-14 04:10:21 <Luke-Jr> 
ACTION wonders if there is a decentralised votign scheme

136 2015-04-14 04:10:22 <CodeShark> with perhaps some technical tools to prove correctness for certain aspects of the implementation
137 2015-04-14 04:10:45 <gmaxwell> I vote to transfer all codeshark's coins to me. All in favor? :P
138 2015-04-14 04:10:53 <sipa> aye.
139 2015-04-14 04:10:56 <CodeShark> lol
140 2015-04-14 04:11:34 <CodeShark> the SchellingCoin attack?
141 2015-04-14 04:11:58 <CodeShark> anyhow, I don't have answers to all these issues - but they ARE issues and we cannot continue to pretend they don't exist
142 2015-04-14 04:12:35 <gmaxwell> who said there is any pretending anything? It's statements like that that make it sound like you're saying people aren't working to advance the ecosystem.
143 2015-04-14 04:12:48 <CodeShark> but that's not what I'm trying to say - I'm sorry if it sounds that way
144 2015-04-14 04:12:56 <gmaxwell> CodeShark: sorry for barking at you; I now realize you're not trying to be a jerk. Your writeup sounded to me probably 180 degrees off from how you intended it; and I believe it will be read by others that way.
145 2015-04-14 04:13:36 <CodeShark> I am trying to help all you guys
146 2015-04-14 04:13:57 <CodeShark> if you have some suggestions on how to word things, please feel free to suggest and comment
147 2015-04-14 04:14:12 <sipa> you could help by implementing and testing big changes you'd like to see
148 2015-04-14 04:14:22 <sipa> and show people that they work
149 2015-04-14 04:14:36 <sipa> and convince people that they are necessary
150 2015-04-14 04:14:41 <gmaxwell> and perhaps I'm being a bit touchy because I am fed up with people asking me when I'm going to consider $random_idea by $press_fiend which was really something that I described in some detail in 2012 or 2013; which has about 200 layers of deep thought that go into it which make it not so easy to deploy or obviously a win once you work out the details.
151 2015-04-14 04:15:32 <sipa> almost all big changes make fundamental tradeoffs
152 2015-04-14 04:16:05 <sipa> utxo commitments offer pretty nice advantages, allowing things which are not possible now, with way lower costs
153 2015-04-14 04:16:23 <CodeShark> yes, absolutely
154 2015-04-14 04:16:28 <CodeShark> I've loved that one for a long time
155 2015-04-14 04:16:31 <CodeShark> been a huge advocate of it
156 2015-04-14 04:16:43 <gmaxwell> (E.g. an example UTXO commitments, they're neat and cool and enable some helpful things; but when you actually go to implement you find they have something like an >>10 fold IO overhead for full verifying nodes and suddenly now it's not so obvious a huge win, and there are several disjoint designs which are plausable and have different advantages)
157 2015-04-14 04:16:57 <gmaxwell> CodeShark: Were you aware of the huge IO cost?
158 2015-04-14 04:17:04 <CodeShark> can't merkle tree modifications be performed in logarithmic time?
159 2015-04-14 04:17:08 <sipa> on the other hand, they all require much higher cpu and io bandwidth from full nodes, increasing storage, oprrating costs and relay time, all potentially resultig in higher centralization
160 2015-04-14 04:17:25 <gmaxwell> CodeShark: yes, it's logarithmic, which is why it's not a million fold overhead.
161 2015-04-14 04:17:40 <CodeShark> well, right now initial sync time carries linear overhead
162 2015-04-14 04:17:49 <CodeShark> and that's assuming a constant transaction volume
163 2015-04-14 04:17:59 <sipa> CodeShark: and it would remain linear!
164 2015-04-14 04:18:18 <CodeShark> it can never go mainstream
165 2015-04-14 04:18:23 <sipa> only a *lower* security model would be enabled, which is faster
166 2015-04-14 04:18:34 <CodeShark> there's just no fricking way we can process the volume of transactions we would like for a global currency
167 2015-04-14 04:18:36 <sipa> and this may or may not be wort it
168 2015-04-14 04:18:53 <gmaxwell> CodeShark: they don't replace initial sync though: not without a substantially changed security model (one that looks like the sidechains one, where a hashpower majority can steal coins via reorgs).  And if you're really willing to take a lower security model, we have SPV in bitcoin.
169 2015-04-14 04:19:05 <CodeShark> it also makes client implementations much more difficult to pull off without relying on a trusted central blockchain server
170 2015-04-14 04:19:14 <CodeShark> so now we have the incentive for far greater centralization
171 2015-04-14 04:19:32 <sipa> CodeShark: perhaps
172 2015-04-14 04:19:34 <gmaxwell> CodeShark: for 'a global currency' that can never be in a decenteralized consensus network directly;  but thats what micropayment hubs/lightning are for.
173 2015-04-14 04:19:50 <gmaxwell> CodeShark: android wallet works pretty great, so thats not clear to me.
174 2015-04-14 04:20:07 <sipa> CodeShark: nobody is sayig that UTXO commitments are bad; just that they change the tradeoffs significantly
175 2015-04-14 04:20:14 <CodeShark> how many users does android wallet have? how many users do CoinBase and bc.i have?
176 2015-04-14 04:20:33 <CodeShark> if you don't think that's a problem, I don't know what to say :p
177 2015-04-14 04:20:49 <sipa> i think it is an unsolvable problem
178 2015-04-14 04:21:00 <gmaxwell> CodeShark: come on, be reasonable; there isn't anything about android wallet's SPVness that matters for that comparison.
179 2015-04-14 04:21:08 <Luke-Jr> CodeShark: I don't think those numbers are really public
180 2015-04-14 04:21:13 <sipa> centralized services are inherenty more scalable
181 2015-04-14 04:21:17 <CodeShark> point is it's a losing battle with the current approaches
182 2015-04-14 04:21:23 <gmaxwell> CodeShark: yea, I like UTXO commitments; I mean I proposed the idea first.  But I also didn't know the cost then, it's a harder problem that I thought.
183 2015-04-14 04:21:51 <gmaxwell> CodeShark: Whats a losing battle?
184 2015-04-14 04:22:03 <CodeShark> the current trend towards centralization of verification
185 2015-04-14 04:22:22 <CodeShark> and the shrinking numbers of full nodes
186 2015-04-14 04:22:33 <gmaxwell> CodeShark: I doubt that. and if so, oh well, "we tried", I mean if you give up on decenteralization there is no point to bitcoin.
187 2015-04-14 04:22:47 <sipa> so you want more full nodes... by making it harder to run one?
188 2015-04-14 04:23:06 <gmaxwell> (thus all the effort going into performance improvements in Bitcoin Core; and the concern about utxo costs, and the implementation of pruning)
189 2015-04-14 04:23:19 <sipa> utxo commitments may make it easier to start one (b y trusing miners about history)
190 2015-04-14 04:23:30 <sipa> but it makes it a lot harder to run one
191 2015-04-14 04:23:40 <sipa> *by trusting
192 2015-04-14 04:23:49 <CodeShark> gmaxwell: far from that - just saying that the economic incentives are all out of whack. sipa: the issue of providing incentives to validators is another big one that needs a solution
193 2015-04-14 04:23:52 <gmaxwell> sipa: it's also easy to 'start' just by starting as a SPV node and catching up in the background.
194 2015-04-14 04:24:14 <sipa> CodeShark: i do not think so
195 2015-04-14 04:24:30 <sipa> CodeShark: i do not want to pay someone to run a validation node for me
196 2015-04-14 04:24:45 <sipa> *that* is centralization pressure
197 2015-04-14 04:25:00 <sipa> i want to be able to run one myself, if i want to
198 2015-04-14 04:25:02 <CodeShark> in the very, very early history of bitcoin, the economic assumptions and the economic reality *almost* lined up - the incentives lined up well enough to encourage decentralized growth
199 2015-04-14 04:25:12 <CodeShark> but the economic assumptions are now far from the current economic reality
200 2015-04-14 04:25:12 <gmaxwell> The only validation that matters to you is your own, otherwise you're just trusting, and if you want to trust you can trust coinbase or what have you (who's throat you might be able to choke), rather than anonymous 'incentivized' people in network space.
201 2015-04-14 04:25:29 <sipa> do not confuse decentralization with trustlessness
202 2015-04-14 04:26:00 <sipa> you need miners and service providers to be decentralized, as you need to trust them
203 2015-04-14 04:26:47 <sipa> you do not need anyone else to run a full node - the number of nodes does not matter; only how easy it is to run one yourself
204 2015-04-14 04:27:24 <CodeShark> unfortunately, it seems that full validators will more and more be subsidized by private interests and less and less by the network as a whole
205 2015-04-14 04:27:45 <gmaxwell> CodeShark: Cost of running a client (e.g. non-listening) full node is very low in any case: like a dollar of diskspace and cents per month in power and bandwidth.  There are significant tripping points, needless rough edges that discourage it, plus a lot of misunderstanding... but it's not about the cost, I mean you can easily go find bitcoin companies with millions in funding that tell you they use
206 2015-04-14 04:27:51 <gmaxwell> hosted node services.
207 2015-04-14 04:28:22 <CodeShark> but perhaps the greater issue is the substantially greater complexity and significantly different trust model required to operate SPV
208 2015-04-14 04:28:45 <CodeShark> which will continue to encourage app developers to rely on trusted servers
209 2015-04-14 04:28:55 <CodeShark> remember, most programmers aren't as good as you guys :p
210 2015-04-14 04:29:41 <sipa> CodeShark: $BIG_COMPANY has enough money to run a full node (and does), but runs entitely on AWS
211 2015-04-14 04:29:47 <gmaxwell> It's not about being good, its about awareness and care. Programmers are usually in a problem solving mode, and protecting the decenteralization of bitcoin (and your own security) isn't on the path of least resistance to getting $random_service_up.
212 2015-04-14 04:30:23 <CodeShark> sipa: my point exactly - these nodes are more and more subsidized by private interests that don't necessarily have the good of the network as a whole as their incentive
213 2015-04-14 04:30:23 <sipa> CodeShark: if they trust amason, the could easily SPV as well... the cost of running a full node is not the problem
214 2015-04-14 04:30:34 <gmaxwell> but thats not an issue with any software or protocol or anyting; it's a fundimental barrier for the hope of a successful decenteralized system. :(
215 2015-04-14 04:30:50 <sipa> CodeShark: running a full node hardly contributes to the network
216 2015-04-14 04:31:03 <sipa> CodeShark: it does service SPV nodes
217 2015-04-14 04:31:21 <gmaxwell> CodeShark: subsidized is the wrong word, when it isn't a question of costs.
218 2015-04-14 04:32:27 <sipa> the problem imho is that most people do not understand the benefits that trustless systems provide... and do not ask for them
219 2015-04-14 04:32:44 <sipa> and for companies it is just easier to not use them
220 2015-04-14 04:32:57 <gmaxwell> it's not monetary costs that keep companies with millions in funding from running their own node(s); they spend more on free lunches for their staff in a day then the costs of running a dozen nodes for years.
221 2015-04-14 04:33:16 <sipa> easier in terms of develoment costs, perhaps, but the operating costs are not the problem
222 2015-04-14 04:33:35 <CodeShark> yes, that's why I said earlier that this was the bigger issue
223 2015-04-14 04:34:14 <CodeShark> attempting a trustless model that is sufficiently scalable given the current network requires significantly more sophisticated logic than just querying for txouts from a trusted server
224 2015-04-14 04:34:21 <gmaxwell> well not even development so much at least for those using APIs that are 1:1 matches for bitcoind. (plus if there were specific features they really needed, we'll often develop them at no cost)
225 2015-04-14 04:34:47 <sipa> CodeShark: yup, and i have no solution for that
226 2015-04-14 04:35:05 <gmaxwell> CodeShark: but that isn't the issue _today_ even if it will be the issue later, and the problem we have today is real and more fundimental and perhaps moots the rest (and also explains the non-investment in the future)
227 2015-04-14 04:35:11 <sipa> CodeShark: but incentivizing full nodes running does not help
228 2015-04-14 04:35:30 <sipa> CodeShark: you need to incentivize people to _use_ a full node
229 2015-04-14 04:35:55 <CodeShark> sipa: you earlier were saying that adding the utxo root hash could make running a full node more expensive, and thus make their numbers shrink even more
230 2015-04-14 04:36:04 <CodeShark> this is why I suggested that perhaps there needs to be a counterincentive
231 2015-04-14 04:36:20 <sipa> such as?
232 2015-04-14 04:36:25 <gmaxwell> CodeShark: e.g. you can just take an existing full node (sophication of the logic you must write: zero), and run it, and query it.  Thats pretty easy and cheap.
233 2015-04-14 04:37:04 <sipa> CodeShark: if everyone in the world would run a full node, but everyone would still query bc.i, would we have gained anything?
234 2015-04-14 04:37:54 <gmaxwell> (at least from the service operator position; the reason many large commercial players are outsourcing isn't because writing a SPV client takes a bit of work; surely more advanced scaling techniquies will just make that expoentially harder; but because they don't see value in the non-centeralized systems)
235 2015-04-14 04:38:12 <sipa> ^ that
236 2015-04-14 04:38:24 <kanzure> i suspect that the number one reason why companies are not using full nodes is because of marketing and propaganda from api service providers telling them that bitcoind is too complex to operate
237 2015-04-14 04:38:34 <CodeShark> gmaxwell: then we've failed :(
238 2015-04-14 04:38:45 <sipa> kanzure: well they're likely right
239 2015-04-14 04:38:49 <kanzure> they aren't
240 2015-04-14 04:38:51 <CodeShark> unless we can present a good case for decentralized trustless models
241 2015-04-14 04:39:13 <Apocalyptic> CodeShark, why would you need to present a case ?
242 2015-04-14 04:39:25 <CodeShark> Apocalyptic: someone needs to do it
243 2015-04-14 04:39:29 <kanzure> huh?
244 2015-04-14 04:39:48 <CodeShark> what's so obvious to many in here isn't really so obvious to everyone out there
245 2015-04-14 04:39:49 <sipa> kanzure: running bitcoind is not hard. using it is probably harder than necessary
246 2015-04-14 04:40:06 <gmaxwell> It's not like much of anyone is marketing these values; while there is marketing going against them.
247 2015-04-14 04:40:32 <sipa> so we need a trustlessness marketing fund?
248 2015-04-14 04:40:38 <kanzure> sipa: my only recommendation on that front is https://github.com/bitcoin/bitcoin/issues/5873
249 2015-04-14 04:40:54 <Apocalyptic> CodeShark, if there were no case then nobody would bother about bitcoin
250 2015-04-14 04:40:55 <gmaxwell> sipa: I'd believe kanzure at least; ... (though I'm biased, I don't think it's hard particularly because I can't imagine it's harder than coping with the inevitable outages of a third party service)
251 2015-04-14 04:40:56 <kanzure> yeah i agree that proactive marketing isn't strictly necessary
252 2015-04-14 04:41:06 <kanzure> it's the anti-marketing that is the most annoying
253 2015-04-14 04:41:13 <kanzure> e.g., ripple's bullshit
254 2015-04-14 04:41:54 <gmaxwell> It's not just marketing though, we don't make enough of an effort to mitigate stupid stumbling blocks; things that throw people off but really aren't fundimental issues.  Most of the things I push for lately are along those lines.
255 2015-04-14 04:42:10 <sipa> Apocalyptic: people seem more excited about the currency than about the grounds it was created for
256 2015-04-14 04:42:22 <kanzure> i also think that developes learning about bitcoin from journalists is the dumbest idea in the history of mankind
257 2015-04-14 04:42:25 <kanzure> *developers
258 2015-04-14 04:42:28 <gmaxwell> It's easy to believe some FUD that running your own node is 'hard', if also the damn bufferbloat problem hits you right after starting it.
259 2015-04-14 04:43:06 <gmaxwell> or when it takes > a day to sync (e.g. pre-headersfirst), it's easy to believe it'll never work right.
260 2015-04-14 04:43:51 <kanzure> "well these guy raised $50M and they say don't use bitcoind, they must have been vetted right? right guys?"
261 2015-04-14 04:44:12 <gmaxwell> I'd like to make a better push for trustlessness to the bitcoin fans out there; but think we need to remove a few more warts first. (e.g. libsecp256k1 verification, pruning, some solution to the bufferbloat issue)
262 2015-04-14 04:44:45 <kanzure> i have never had a developer complain about openssl (except for stupid dependency/packaging problems)
263 2015-04-14 04:44:57 <kanzure> er, at least in the context of bitcoind
264 2015-04-14 04:45:15 <gmaxwell> kanzure: it's not openssl, the verification is >5x faster. :)
265 2015-04-14 04:45:49 <kanzure> right right, i understnad that
266 2015-04-14 04:45:51 <gmaxwell> which makes a meaningful syncup speed improvement.
267 2015-04-14 04:46:37 <gmaxwell> and people do complain about syncup speed; also when they don't directly complain it gives them a bad impression. E.g. that it won't work well because it's 'slow' (nevermind that it's actually verifying at a couple hundred times the maximum theoretical rate for the network)
268 2015-04-14 04:47:09 <kanzure> maybe show them bitcoin news during sync -_-
269 2015-04-14 04:47:24 <gmaxwell> kanzure: interesting thing we've seen esp with windows users: using 100% cpu for more than a few seconds makes them think the software has hung. ... they're conditioned by interactive software that any actual sustained use of the computer means its broken. :)
270 2015-04-14 04:47:25 <moa> kanzure: this evil but brilliant
271 2015-04-14 04:47:40 <gmaxwell> kanzure: sync should ultimately be more in the background anyways.
272 2015-04-14 04:48:13 <kanzure> if they are using the box as a desktop then maybe running a full node should not be a priority?
273 2015-04-14 04:48:43 <kanzure> moa: perhaps documentation and tutorials, then. like using regtest.
274 2015-04-14 04:49:11 <gmaxwell> kanzure: pfft, it should work fine though. And thats the thing; why should they be trusting some third party site?  They'll just translate that same practice to their work when they go to deploy a bitcoin service commercially.
275 2015-04-14 04:49:13 <moa> yeah ... or a suggestion to go make a cup of coffee, take the dog for a walk and an estimate of how long it might take
276 2015-04-14 04:49:28 <moa> plus a scrolling r/bitcoin feed
277 2015-04-14 04:49:42 <gmaxwell> One of the major success of free software is people took the things they learned casually and translate it into their offices.
278 2015-04-14 04:50:23 <kanzure> actually an interactive regtest tutorial might work
279 2015-04-14 04:50:28 <kanzure> during initial sync
280 2015-04-14 04:50:51 <moa> or how to secure/backup your wallet
281 2015-04-14 04:50:54 <CodeShark> gmaxwell: I reworded the intro a bit - I hope it doesn't sound as dismissive of the core devs
282 2015-04-14 04:50:57 <moa> tutorial
283 2015-04-14 04:51:02 <CodeShark> that was not my intention at all
284 2015-04-14 04:51:38 <moa> voice-over, smooth flash graphics, etc
285 2015-04-14 04:53:17 <gmaxwell> CodeShark: thanks, it's improved.  So it's not clear what you mean by "Seperate the protocol from the currency"... so a read on that sounds like you're advocating "blockchain without the bitcoin" or suggesting that we should be supporting altcoins.
286 2015-04-14 04:53:56 <gmaxwell> If you're just saying that the terminiolgy should be clear, you should make that more explicit.
287 2015-04-14 04:55:21 <CodeShark> I'm suggesting that protocol research should be conducted on many different testnets...and that each network be able to specify what currencies it accepts and which protocol changes it incorporates
288 2015-04-14 04:56:19 <gmaxwell> People copying my work into random altcoins which then went and pumped themselves as "Better than bitcoin" on the basis of it has pretty much left me ready to leave the ecosystem more than once.
289 2015-04-14 04:57:02 <gmaxwell> I believe strongly in free software; which is why I didn't switch to "bitcoin only" licenses for my code, but I'm really not enthused about going out of my way to support competing things that only want to eat the bitcoin network effect for their own gain.
290 2015-04-14 04:57:07 <CodeShark> lol - I sincerely hope that people wise up to all these hacksters
291 2015-04-14 04:57:49 <gmaxwell> yea, most people have, by now I suppose; but it's still an outsized impact becuase of incentives.
292 2015-04-14 04:58:12 <CodeShark> most alts are total bunk...but a few do try out at least interesting, if not elegantly implemented, innovations
293 2015-04-14 04:58:52 <gmaxwell> CodeShark: that list is reaaallly thin, but sure. (you have to read the code too, lots of stuff is claimed that isn't real)
294 2015-04-14 05:00:12 <phantomcircuit> CodeShark, huckersters
295 2015-04-14 05:00:14 <phantomcircuit> not hacksters
296 2015-04-14 05:00:16 <moa> i think sidechains will help a lot with bringing the innovation energy back into alignment
297 2015-04-14 05:00:24 <CodeShark> phantomcircuit: frauds
298 2015-04-14 05:00:26 <CodeShark> is that better? :p
299 2015-04-14 05:00:34 <phantomcircuit> yes!
300 2015-04-14 05:02:02 <moa> i, for one, regret showing that guy how to make a genesis block ... he turned around and made a windows GUI template out of it :(
301 2015-04-14 05:02:29 <gmaxwell> moa: hm. its actually unclear to me, simply because I don't know how much 'energy' there actually is.  It it for the lack of sidechains that something like 95% of lines of code in diffs in bitcoin core come from like 8 people total.
302 2015-04-14 05:03:25 <gmaxwell> it isn't for lack of sidechains that there are major commercial services handling third party funds that use remote bitcoind like APIs instead of even running their own full node(s).
303 2015-04-14 05:03:38 <gmaxwell> It isn't for lack of sidechains that the existing script functionality is mostly unused.
304 2015-04-14 05:05:09 <gmaxwell> Or the situation around hashpower/control consolidation.
305 2015-04-14 05:05:32 <gmaxwell> If we solve whatever causes these other issues we'd run right into the problems sidechains may help with, sure. (In particular, it'll be super easy to do codesharks' 'testnet' that can carry bitcoins; in a reduced security model at least for some kinds of changes). So thats why I think sidechains work is important, too.. but the rest is also a consideration.
306 2015-04-14 05:06:18 <moa> all good points, I was referring more to the lack of test env for new protocols/features ... snap, you just said it
307 2015-04-14 05:09:05 <moa> "there are major commercial services handling third party funds that use remote bitcoind like APIs instead of even running their own full node(s)" - I find this baffling, why is this?
308 2015-04-14 05:11:32 <phantomcircuit> moa, the answer is depressing
309 2015-04-14 05:11:33 <phantomcircuit> :/
310 2015-04-14 05:11:55 <gmaxwell> I doubt is a single reason. There have been some factors I can identify, I don't know their weight.
311 2015-04-14 05:12:29 <moa> mediocity, incompetence, laziness?
312 2015-04-14 05:12:41 <gmaxwell> One is that a service is at least theoretically supported (I can say that at least some of the api services provide usless support), you could purchase bitcoind support too-- but no one advertises it.
313 2015-04-14 05:13:34 <gmaxwell> Another factor is that if you have nothing to start with and are from a background which already is heavy into api service (a current trend in web development), the first thing you think of is to find a RPC to call, ... not look for a daemon to install.
314 2015-04-14 05:13:48 <moa> software-as-a-service groupthink infection
315 2015-04-14 05:13:53 <gmaxwell> right.
316 2015-04-14 05:14:30 <gmaxwell> Another thing is that "but for a <feature>", sometimes the services are 99% like bitcoind but have an extra feature like watching wallets; we don't even get asked for the missing feature in most cases. soooo.
317 2015-04-14 05:14:50 <moa> hmmm, seems like money to made operating on-site bitcoind remotely
318 2015-04-14 05:15:02 <moa> like a legit bot-net
319 2015-04-14 05:15:56 <gmaxwell> kanzure as seen some anti-bitcoind marketing. Telling people that its complex to run and such, and that no one should ever run it.
320 2015-04-14 05:16:12 <moa> i saw that at a conference
321 2015-04-14 05:18:22 <gmaxwell> In some cases I've encountered people who chose hosting enviroments which made running code written in anything but a small set of scripting languages (e.g. python/ruby/php) hard.... (why anyone would use such a host is beyond me, but they exist and are widely used; and thats probably a factor for some people)
322 2015-04-14 05:19:59 <gmaxwell> I'm sure there are other factors.
323 2015-04-14 05:20:06 <moa> yeah that sounds alien to me but maybe it's like that now ...
324 2015-04-14 05:21:36 <gmaxwell> moa: yea, I couldn't believe the hosting thing when I heard it, but nope, it's true. I guess the hosts motivation is by focusing on the most popular sandboxed scripting enviroments it lowers support costs (and potentially improves security; e.g. if they can sandbox the scripting enviroment tighter than arbritary native programs)
325 2015-04-14 05:22:33 <jrick> oops up enter
326 2015-04-14 05:22:38 <jrick> damnit
327 2015-04-14 05:22:40 <moa> but straitjacket to any flexibility needed
328 2015-04-14 05:24:27 <gmaxwell> besides, if you let people run arbritary code, they could perhaps run a debugger without a license from the proper authorities...  :-/
329 2015-04-14 05:24:46 <moa> I seem to recall it was Chain API saying "you don't want to run bitcoind" ... and that's what their business model is
330 2015-04-14 05:25:33 <moa> so not surprising
331 2015-04-14 05:26:19 <moa> their argument might be that they offer some other features as well as usual RPC calls
332 2015-04-14 08:25:30 <wumpus> so, we're at the point of nailing manifestos to the church door now
333 2015-04-14 08:31:11 <priidu> was wondering if there's a way to change account names after they've already been created in the wallet.dat?
334 2015-04-14 08:32:57 <wumpus> setaccount
335 2015-04-14 08:34:17 <priidu> i checked pywallet for that option but didn't notice it at first glance
336 2015-04-14 08:34:37 <wumpus> I'm talking ahout the RPC API
337 2015-04-14 08:35:18 <sipa> that does not change the account for sent payments afaik
338 2015-04-14 08:35:36 <sipa> only the account received ones are credited to
339 2015-04-14 08:35:48 <priidu> ok, thanks to you both
340 2015-04-14 08:35:49 <sipa> but i may misremember
341 2015-04-14 08:35:55 <priidu> completely forgot about setaccount
342 2015-04-14 08:35:56 <sipa> eh, ofcourse
343 2015-04-14 08:36:26 <sipa> because the sent account does not involve an address at all
344 2015-04-14 08:36:27 <wumpus> right, all it does is change the account (label) associated with an address
345 2015-04-14 08:37:17 <wumpus> you cannot change accounting entries in the past
346 2015-04-14 08:39:43 <CodeShark> never use the bitcoind wallet account system - it makes swapping out the wallet extremely painful later on...meaning that key compromise protocols and procedures are very unpleasant
347 2015-04-14 08:40:18 <sipa> yeah, the account system is scheduled for deprecation
348 2015-04-14 08:40:29 <priidu> yep + scaling issues etc
349 2015-04-14 08:40:33 <wumpus> using accounts as labels for addresses is fine though, that will stay
350 2015-04-14 08:40:45 <priidu> using the accounts just for a demo though
351 2015-04-14 08:40:57 <wumpus> but indeed, don't use them for actual accounting
352 2015-04-14 08:41:35 <wumpus> but that should be redundant to everyone who has been in this channel for longer than five minutes
353 2015-04-14 08:41:40 <CodeShark> lol
354 2015-04-14 08:41:53 <Luke-Jr> [08:25:30] <wumpus> so, we're at the point of nailing manifestos to the church door now <-- ? :|
355 2015-04-14 08:42:25 <wumpus> Luke-Jr: read back a bit
356 2015-04-14 08:42:36 <priidu> yeah, was wondering about that too, lol
357 2015-04-14 08:43:01 <Luke-Jr> 
ACTION read the only prior convo earlier

358 2015-04-14 08:46:03 <wumpus> it was just an offhand comment on CodeShark's manifesto
359 2015-04-14 09:15:29 <CodeShark> wumpus: do you have any comments or suggestions or obscenities or insults?
360 2015-04-14 09:17:08 <wumpus> CodeShark: my first question would always be 'who is going to do this'? there are so many ideas already as to what has to be done
361 2015-04-14 09:17:56 <CodeShark> the first point is mostly a matter of terminology coinage (pun intended) and education
362 2015-04-14 09:17:59 <wumpus> the problem is finding, and also incentivizing people that can execute on the plans
363 2015-04-14 09:19:51 <CodeShark> well, I would like to offer these suggestions first to the Bitcoin Foundation...but if they are unwilling or incapable of delivering we'll have to figure out how to organize an effort
364 2015-04-14 09:20:16 <CodeShark> don't worry, wumpus - I'd definitely include you in any effort :)
365 2015-04-14 09:21:12 <CodeShark> but setting aside the means, for a moment, what do you think about the ends?
366 2015-04-14 09:21:47 <wumpus> well the bitcoin foundation, due to monetary issues, is not funding any development or research on bitcoin anymore, so that would mean very little
367 2015-04-14 09:22:08 <CodeShark> ok, so yeah - I suppose such support would be symbolic at best
368 2015-04-14 09:23:02 <CodeShark> but before trying to muster the full effort, I want to know what you think about the aims and whether they make sense to you or whether you would change anything or remove anything or add anything
369 2015-04-14 09:27:15 <wumpus> personally I really like the sidechains proposal, it would address the 'experimentation is difficult' part
370 2015-04-14 09:27:53 <wumpus> "Protocol research should be carried out on test networks using test tokens, freeing up developers to experiment without risk of economic repercussions" I see that the other way around ,for serious experimentation there would have to be economic repercussions
371 2015-04-14 09:28:07 <CodeShark> hmmm...good point
372 2015-04-14 09:28:08 <wumpus> creating testnets with different rules is easy
373 2015-04-14 09:28:18 <wumpus> (eg that's altcoins)
374 2015-04-14 09:28:34 <CodeShark> I suppose we could simulate economic incentives artificially
375 2015-04-14 09:29:21 <CodeShark> we could impose hard expiration times on the network and reward network participants at that point
376 2015-04-14 09:29:34 <CodeShark> so that the tokens do not end up getting traded on markets perpetually
377 2015-04-14 09:29:37 <wumpus> it is clear that the more serious bitcoin is taken, the harder it is to change the rules of the original network, a kind of annealing process
378 2015-04-14 09:30:47 <CodeShark> yes, and it's a vicious cycle...because the less we change it the harder it becomes to change it later
379 2015-04-14 09:30:50 <wumpus> people just won't agree to changes as they are invested into how things are, in the system working as it does - you'd also be against changing universal constants, as it would make life impossible
380 2015-04-14 09:31:10 <wumpus> this was one of bitcoin's selling points: stability
381 2015-04-14 09:31:38 <wumpus> "so that the tokens do not end up getting traded on markets perpetually" a two-way peg would make trading unnecessary
382 2015-04-14 09:32:01 <CodeShark> problem is the potential for pump-and-dump
383 2015-04-14 09:32:15 <CodeShark> if the token has an expiration, it's a limited experiment
384 2015-04-14 09:32:17 <wumpus> no, there would not, the sidechains don't have their own tokens
385 2015-04-14 09:32:32 <wumpus> you'd transfer bitcoins to there, and back when you're done with the experiment
386 2015-04-14 09:33:40 <wumpus> introducing other token types is indeed problematic
387 2015-04-14 09:35:04 <CodeShark> using bitcoins exclusively has several problems: we cannot as easily tweak economic parameters (i.e. generation and rewards in particular), it would be the same exact people that hold most of the bitcoins today being able to fund these experiments
388 2015-04-14 09:35:21 <CodeShark> I'd like to get fresh people in
389 2015-04-14 09:35:59 <wumpus> sidechains would be great to make a kind of darwinian selection possible without risking the integrity of the entire sysetm
390 2015-04-14 09:36:41 <wumpus> ugh, creating a new token type to fund development is an extremely bad idea
391 2015-04-14 09:36:48 <CodeShark> how do we incorporate non-zero-sum rewards in sidechains?
392 2015-04-14 09:37:43 <CodeShark> if we just merge mine them then we haven't really done much to improve the economic security model
393 2015-04-14 09:39:58 <CodeShark> one possible way to subsidize testnets is to reward the winning candidate by allowing it to convert its own currency into bitcoins when it gets merged back into the mainnet
394 2015-04-14 09:40:12 <CodeShark> of course, this is fraught with a bunch of other problems
395 2015-04-14 09:40:42 <wumpus> just that bitcoin community has been incapable of maintaining an organization (or multiple) for development and research into the network doesn't mean encouraging boiler room practices is  a good idea. If so, you should already be happy with the cambrian explosion altcoins (which are, in practice, essentially mostly get quick rich schemes without any real substance).
396 2015-04-14 09:42:16 <CodeShark> I fully agree, wumpus - I just hate to see bitcoin losing to newer networks that incorporate many powerful results from recent theoretical research
397 2015-04-14 09:42:38 <wumpus> ok ok, there's blockstream, which is far from "incapable", so I take that back :)
398 2015-04-14 09:42:50 <CodeShark> as painful as some of these things may seem, if we don't do them I fear the value of bitcoin will ultimately be $0
399 2015-04-14 09:43:07 <CodeShark> and the satoshi protocol relegated to comp sci textbooks
400 2015-04-14 09:43:23 <wumpus> bitcoin is exactly useful because it provides a stable foundation for cryptocurrency
401 2015-04-14 09:43:36 <wumpus> many experiments will live and die, but bitcoin will just be there to fall back on
402 2015-04-14 09:43:57 <CodeShark> but it isn't really stable...it's extremely volatile and rests on an economic security model that deviates more and more from the initial assumptions everyday
403 2015-04-14 09:44:12 <wumpus> it is as stable as realistically possible.
404 2015-04-14 09:45:05 <wumpus> I mean, experimenting with the main network will by definition not make it more stable
405 2015-04-14 09:45:12 <CodeShark> I predict that before the end of 2016 a far superior technology with very tangible benefits for endusers will emerge...and if bitcoin cannot incorporate these innovations it will drop to $0
406 2015-04-14 09:45:43 <CodeShark> I like the sidechain ideas...but I'm not sure they go far enough
407 2015-04-14 09:46:15 <CodeShark> specifically, we need ways of experimenting with far more sophisticated game-theoretic models for economic incentives
408 2015-04-14 09:46:18 <wumpus> That's always possible. Risk of the trade. This is not the channel to get worried about price fluctiatuions
409 2015-04-14 09:46:40 <CodeShark> it's not really the price I'm concerned about - I'm not in this to get rich from currency speculation
410 2015-04-14 09:47:10 <CodeShark> rather, I should say I'm concerned about the price because of its indirect effects on the entire industry
411 2015-04-14 09:47:43 <wumpus> looking historically economic changes have been very, very slow
412 2015-04-14 09:47:58 <CodeShark> not in technology
413 2015-04-14 09:49:34 <CodeShark> I'll just be frank - there are other groups of very smart people working on ideas to replace the current bitcoin network. unless bitcoin core dev can incorporate new insights at a significantly faster rate, it will be caught off guard and will lose its authority in this space
414 2015-04-14 09:49:38 <wumpus> you don't need to convince me of anything, if you think things are not moving fast enough, get organized, get thing done
415 2015-04-14 09:50:29 <CodeShark> you do raise a great point regarding economic incentives for testnets...this is tricky...but I'll think about it some more :)
416 2015-04-14 09:50:53 <wumpus> that's how it works in a decentralized system
417 2015-04-14 09:52:40 <wumpus> personally I have been bogged down in day-to-day source code issues it is a difficult and risky enough project to even maintain, so I've grown very conservative
418 2015-04-14 09:54:33 <CodeShark> yeah, I don't blame you - I understand
419 2015-04-14 09:55:09 <CodeShark> I just feel we're on the cusp of a major pivot in the industry
420 2015-04-14 09:55:17 <jtimon> gmaxwell, if you're talking about services like heroku, they allow you to run compiled languages, they just don't give official support https://devcenter.heroku.com/articles/third-party-buildpacks so theoretically you could run bitcoind there if you wanted to (but it will be probably easier for you to just run it somewhere else)
421 2015-04-14 09:56:14 <CodeShark> whatever this ends up becoming, whether it's bitcoin or not, I'd like to see you play a big role, wumpus
422 2015-04-14 09:56:56 <jtimon> CodeShark what's wrong with running full nodes for private interests? Isn't that precisely the reason to run one?
423 2015-04-14 09:57:33 <jtimon> like my private interest in saying if I've been really paid and the like?
424 2015-04-14 09:58:31 <jtimon> on hardforks, I believe there should be hardforks at some point, but they will probably have to be completely uncontroversial hardforks like say, the timetravel attack fix
425 2015-04-14 09:58:48 <wumpus> CodeShark: the part I agree on is that it 's always good to get more (qualified) people on this, and more energy
426 2015-04-14 09:59:19 <jtimon> about 4 lines of code, tested by many altcoins...and still, it wouldn't be trivial to deploy
427 2015-04-14 10:00:00 <wumpus> jtimon: yes, there are good reasons to run your own node, and software improvements are going to make it easier and less resource intensive
428 2015-04-14 10:00:22 <wumpus> a lot of improvements don't even need forks, just work on the code
429 2015-04-14 10:00:53 <jtimon> wumpus sure, he was complaining about people doing it "only for private interests" and I don't see the problem there
430 2015-04-14 10:01:04 <sipa> tecivally trivial, but the deployment risk for a fork is probablyt worth the benefit of fixing timewarp
431 2015-04-14 10:01:20 <sipa> *not worth
432 2015-04-14 10:01:20 <wumpus> jtimon: oh I agree on that, private interests are great motivators
433 2015-04-14 10:02:45 <jtimon> well, I agree that hardfork deployments are hard, but I think they could be relatively safe, say, schedule thaat 2 years worth of blocks into the future or something
434 2015-04-14 10:03:55 <wumpus> jtimon: but would a 'timetravel attack fix' result in any user visible improvements, or is it just asthetics?  it's not just about risk, it's about risk/gains
435 2015-04-14 10:04:04 <jtimon> in any case my main point is that if it's hard for an implemented, tested and uncontroversial 4-line-change, it's much hardder for anything else
436 2015-04-14 10:04:12 <wumpus> jtimon: sure agreed there
437 2015-04-14 10:04:35 <sipa> well... costs and benefits
438 2015-04-14 10:04:54 <wumpus> yes
439 2015-04-14 10:05:52 <sipa> i think the may 2013 hardfork was relatively risky due to its short timeframe
440 2015-04-14 10:06:24 <sipa> but also very uncontroversial and necessary
441 2015-04-14 10:08:16 <wumpus> that was easy, the risk of not doing it was much higher
442 2015-04-14 10:09:09 <jtimon> well, even if the benefits of a timetravel hardfork would be small, I guess another "benefit" is having succesfuly planned and deployed a hardfork
443 2015-04-14 10:09:47 <jtimon> I mean, the 2013 harfork was also planned but...
444 2015-04-14 10:10:38 <wumpus> but how would you convince the rest of the users and developers to partake in it, if the benefits would be so small, I don't think doing a hardfork for sake of doing a hardfork is that great an argument :)
445 2015-04-14 10:11:23 <sipa> i think the timewarp thing could be fixed in a hardfork together with something more "needed"
446 2015-04-14 10:11:47 <jtimon> well, more than "risks" as such I think it's a huge coordination effort, if anyone says no, it's no, so you would have to ask to many people
447 2015-04-14 10:11:47 <wumpus> then again, maybe if you plan it years ahead, it won't be much of a controversy
448 2015-04-14 10:12:33 <jtimon> sipa I agree, although I think other people wouldn't like a "cherrypicking hardfork"
449 2015-04-14 10:12:41 <wumpus> e.g. we add code now to switch around the rules at block XXX two years in the future, and inform everyone of that, that could be quite painless
450 2015-04-14 10:12:54 <CodeShark> wumpus: exactly
451 2015-04-14 10:13:00 <CodeShark> scheduled updates
452 2015-04-14 10:13:20 <wumpus> (but only if the change itself is noncontroversial, like the time travel fix)
453 2015-04-14 10:13:35 <jtimon> but you need to make sure the changes are uncontroversial first
454 2015-04-14 10:13:41 <CodeShark> uncontroversial fixes would be a great start to test this deployment model
455 2015-04-14 10:13:45 <jtimon> exactly
456 2015-04-14 10:14:03 <sipa> i like the idea of scheduled upgrades
457 2015-04-14 10:14:16 <sipa> but i fear the politics of who gets to decide what goes in
458 2015-04-14 10:14:28 <sipa> i think hard forks should be hard
459 2015-04-14 10:14:46 <sipa> they change the rules everyone implicitly agreed to
460 2015-04-14 10:14:47 <wumpus> right, that's why I think it will only work for completely uncontroversial changes
461 2015-04-14 10:15:02 <wumpus> not anything remotely related to anything economic, and so should it be
462 2015-04-14 10:15:08 <CodeShark> hardforks would be less necessary if the protocol supported certain things like header extensions
463 2015-04-14 10:15:33 <jtimon> yeah, the coordination effort is finding out which changes are comletely uncontroversial
464 2015-04-14 10:16:15 <wumpus> the resulting discussion when you propose the BIP will tell a lot
465 2015-04-14 10:16:33 <jtimon> header extensions? like putting the nHeight in the "extended header" (ie the header + coinbase tx)?
466 2015-04-14 10:16:38 <CodeShark> yes
467 2015-04-14 10:17:05 <jtimon> well, we have the height sofforked in the coinbase now, no?
468 2015-04-14 10:17:13 <CodeShark> that's so fricking ugly :p
469 2015-04-14 10:17:43 <CodeShark> it requires deep knowledge of transaction structure to read that information
470 2015-04-14 10:17:47 <sipa> and expensive to produce spv proofs for
471 2015-04-14 10:17:57 <sipa> if you have many coinbase outputs
472 2015-04-14 10:18:01 <jtimon> well, I agree, but what was the alternative? making the headers only chain bigger?
473 2015-04-14 10:18:18 <CodeShark> we could have a field in the main header for the hash of an extension header
474 2015-04-14 10:18:35 <CodeShark> older versions would simply ignore new fields
475 2015-04-14 10:18:39 <sipa> CodeShark: um... and change every bitcoin software on the planet?
476 2015-04-14 10:18:47 <CodeShark> well, it requires a hard fork now :p
477 2015-04-14 10:18:59 <CodeShark> just saying if we had that we wouldn't need a hard fork to add a new header field
478 2015-04-14 10:19:00 <sipa> that's not just a hard fork
479 2015-04-14 10:19:10 <CodeShark> well, it also causes problems for miners
480 2015-04-14 10:19:12 <sipa> that is also changing non-consensus software
481 2015-04-14 10:19:14 <sipa> miners
482 2015-04-14 10:19:17 <sipa> wallets
483 2015-04-14 10:19:22 <CodeShark> yes, of course
484 2015-04-14 10:19:22 <sipa> indexers
485 2015-04-14 10:19:39 <sipa> most interesting hardforks only affect full nodes
486 2015-04-14 10:20:09 <CodeShark> I wasn't speaking to the deployment strategy - clearly this scheduled update thing should NOT be first tried out with this extension header proposal - lol
487 2015-04-14 10:20:26 <CodeShark> it's just something on my wishlist
488 2015-04-14 10:20:46 <jtimon> CodeShark mainly your proposal is related to this https://github.com/bitcoin/bitcoin/pull/3977 ?
489 2015-04-14 10:20:53 <jtimon> s/mainly/maybe
490 2015-04-14 10:21:04 <wumpus> but it's not really exciting change if it just makes things less 'ugly'
491 2015-04-14 10:21:10 <sipa> the cleanest *viable* extra commitment scheme i know is the low-number-of-bits-in-ntx proposal by iirc tiernolan
492 2015-04-14 10:21:21 <sipa> i consider extension headers just not viablr
493 2015-04-14 10:21:57 <CodeShark> yes, regrettably the original protocol did not provision an extra 256 bits in the block header to use for whatever we might not have known about then
494 2015-04-14 10:22:29 <sipa> well or elsewhere
495 2015-04-14 10:22:41 <sipa> some place in the merkle tree for transactions would suffice
496 2015-04-14 10:22:48 <CodeShark> yes, true
497 2015-04-14 10:22:49 <jtimon> why not on the coinbase? only ugliness?
498 2015-04-14 10:23:58 <CodeShark> well, consider headers-first sync
499 2015-04-14 10:24:16 <jtimon> yes, what's with it?
500 2015-04-14 10:24:25 <CodeShark> a headers-first sync unit shouldn't have to download the coinbase transaction to get this information
501 2015-04-14 10:24:53 <CodeShark> sure, it could derive it ...but what about out-of-order downloads?
502 2015-04-14 10:25:01 <sipa> i do not understand
503 2015-04-14 10:25:10 <jtimon> well say you have headers-first sync and extended-headers-first sync
504 2015-04-14 10:25:27 <sipa> why does a headers-only node need the rxtra commitments at all?
505 2015-04-14 10:25:35 <wumpus> the coinbase is kind of an extended header
506 2015-04-14 10:25:37 <jtimon> I think CodeShark wants to force everyone to use extended-headers-first sync ?
507 2015-04-14 10:25:38 <CodeShark> parallel out-of-order downloads from multiple peers
508 2015-04-14 10:25:58 <CodeShark> at least you know you are not looking at overlapping ranges
509 2015-04-14 10:26:07 <sipa> CodeShark: sure, we do that now...
510 2015-04-14 10:26:58 <jtimon> you could have extended-headers-first sync putting that data in the coinbase, but not everyone will want that extra data when syncing
511 2015-04-14 10:27:27 <CodeShark> yeah - anyhow, this is not the best example...and it's not the #1 priority
512 2015-04-14 10:27:45 <CodeShark> it's on my wishlist but it isn't the top item :)
513 2015-04-14 10:27:47 <sipa> CodeShark: i am completely confused about what you want
514 2015-04-14 10:28:01 <sipa> why do you want extended headers?
515 2015-04-14 10:28:37 <CodeShark> we could use them to store other useful data...like txout indices or utxo merkle roots or who-knows-what-else-we'll-eventually-think-of
516 2015-04-14 10:28:38 <sipa> and why does a coinbase commitment not suffice for it
517 2015-04-14 10:28:54 <sipa> or better, tiernolan's proposal which has smaller proofs
518 2015-04-14 10:29:28 <sipa> extended headers require every piece of bitcoin software to change
519 2015-04-14 10:29:39 <CodeShark> I already agreed it isn't practical :p
520 2015-04-14 10:29:41 <sipa> coinbase commitments... none
521 2015-04-14 10:30:01 <sipa> yes, technical debt and ugliness
522 2015-04-14 10:30:06 <sipa> but also practical
523 2015-04-14 10:30:22 <sipa> changing the header format is completely out of the question iho
524 2015-04-14 10:30:36 <jtimon> unrelated: I'm sorry for being always beaging for review, but luke already acked #5595 can you guys take a look? I would really like to get finally started with the policy code encapsulation
525 2015-04-14 10:30:37 <sipa> it is the single most difficult change to make
526 2015-04-14 10:30:49 <CodeShark> yes, sipa - more the reason to have thought about adding an extra unused field :p
527 2015-04-14 10:30:52 <CodeShark> lol
528 2015-04-14 10:30:59 <CodeShark> this is common practice in systems architecture
529 2015-04-14 10:31:09 <sipa> you need to change every asic every wallet every node
530 2015-04-14 10:31:20 <sipa> well sure, would have been nice
531 2015-04-14 10:31:26 <sipa> can't change it now, sorry
532 2015-04-14 10:31:30 <CodeShark> I know, it's totally impractical to do now - it's just a frustration on my part because it's not a new concept and satoshi missed it completely
533 2015-04-14 10:31:32 <sipa> i wish it were different
534 2015-04-14 10:31:56 <jtimon> I'm not convinced I would add the header extension if we were designing bitcoin from scratch...
535 2015-04-14 10:31:58 <sipa> satoshi did not miss it
536 2015-04-14 10:32:10 <sipa> satoshi added version fields for this reason everywhere
537 2015-04-14 10:32:18 <sipa> so more fields could be added over time
538 2015-04-14 10:32:38 <CodeShark> but you already said it - the header structure cannot be changed without severely disrupting existing infrastructure
539 2015-04-14 10:32:39 <sipa> satoshi just thougt that what we call hardforks now were something easy
540 2015-04-14 10:32:40 <wumpus> if there were an extra field, people would just be fighting what to use it for :-) header space is at an enormous premium
541 2015-04-14 10:33:06 <sipa> yeah, it could just have been an extra entry in the tx merkle tree
542 2015-04-14 10:33:10 <CodeShark> sipa: I'm not so sure - didn't he claim somewhere that the protocol rules would essentially have to remain fixed?
543 2015-04-14 10:33:22 <CodeShark> wasn't his push at extensibility mainly through the script?
544 2015-04-14 10:33:38 <sipa> i think that what he called protocol rules were subjective economic rules
545 2015-04-14 10:33:46 <sipa> not implementation details
546 2015-04-14 10:33:50 <wumpus> yes, the economic rules have to remain fixed
547 2015-04-14 10:34:51 <sipa> CodeShark: every data structure in the protocol has version fields, which used to be integrated with the derialization code
548 2015-04-14 10:34:57 <jtimon> I always thought about "protocol rules" as "consensus rules" but maybe I'm wrong, I prefer the later term now
549 2015-04-14 10:35:07 <sipa> imho that means he wanted to be able to extend fields
550 2015-04-14 10:35:20 <CodeShark> sipa: really? so you're saying that the hardcoding of core structures came later?
551 2015-04-14 10:35:37 <sipa> not realizing that part of them were consensus critical, and could not just be changed without fork risk
552 2015-04-14 10:36:04 <sipa> the wallet and the block headers use the exact same structure!
553 2015-04-14 10:36:20 <CodeShark> huh?!
554 2015-04-14 10:36:40 <sipa> they used to be in the same file even
555 2015-04-14 10:36:42 <CodeShark> lol
556 2015-04-14 10:37:09 <sipa> imho, satoshi thought there would be just one implementation, which he could change at will, and people would update
557 2015-04-14 10:37:28 <sipa> i do not think he understood the distinctoon between wgat we now call softforks and hardforks
558 2015-04-14 10:37:29 <jtimon> well, everything used to be in main.h and maincpp, no ? ;)
559 2015-04-14 10:37:36 <sipa> at least not initially
560 2015-04-14 10:38:19 <CodeShark> it turns out that the development process itself is at least as critical as the protocol
561 2015-04-14 10:38:28 <CodeShark> and needs to be carefully considered as part of the design
562 2015-04-14 10:38:35 <jtimon> well, maybe he didn't knew about softforks but knew about hardforks (and if he was assuming 1 implementation, those are not so painful)
563 2015-04-14 10:38:55 <sipa> CodeShark: well there is one thing i agree with... consensus changes should be understood to be different from regular software improvememts
564 2015-04-14 10:39:16 <CodeShark> sipa: yes, absolutely
565 2015-04-14 10:39:38 <sipa> it's why we (mostly jtimon and cfields) are working on abstracting consensus critical parts out
566 2015-04-14 10:39:58 <gdm85> sipa: does that also correspond in separation of objects/classes (where possible)?
567 2015-04-14 10:40:16 <gdm85> I can imagine it must not be easy on the eyes/understanding when one wants also to be extremely conservative
568 2015-04-14 10:40:16 <sipa> gdm85: of course
569 2015-04-14 10:40:34 <wumpus> yes, everyone agrees there, consensus critical code should be isolated
570 2015-04-14 10:40:49 <sipa> well isolation is just one iart
571 2015-04-14 10:40:57 <wumpus> gdm85: indeed
572 2015-04-14 10:41:13 <sipa> i believe it should follow a very fifferent (and less conventional) development process
573 2015-04-14 10:41:21 <jtimon> CodeShark: ehmm, making rules about the development process or somecing? what about this new rule, "you all go ahead and review my PRs as soon as I ask for it"
574 2015-04-14 10:41:21 <jtimon> now, channel, go review #5595 #6009 #5968 #5975 #5669 #5709
575 2015-04-14 10:41:30 <wumpus> sipa: that would be helped along by isolation
576 2015-04-14 10:41:37 <sipa> wumpus: absolutely
577 2015-04-14 10:41:46 <sipa> but isolation is just one step
578 2015-04-14 10:41:58 <CodeShark> jtimon: I'm talking about architectural features that specifically are there to reduce the need for hardforks and simplify mass scheduled deployments
579 2015-04-14 10:42:02 <gdm85> sipa: I believe it needs a new branch of science for that process :P
580 2015-04-14 10:42:20 <sipa> CodeShark: example?
581 2015-04-14 10:42:21 <wumpus> sipa: agreed
582 2015-04-14 10:42:46 <sipa> gdm85: i believe it is called politics, i fear
583 2015-04-14 10:42:54 <gdm85> :\
584 2015-04-14 10:42:55 <sipa> nobody has solved this problem.so far
585 2015-04-14 10:43:04 <sipa> despite huge public interest
586 2015-04-14 10:43:10 <sipa> and tons of money involved
587 2015-04-14 10:43:46 <moa> money happens at the intersection of politics and economics
588 2015-04-14 10:45:41 <CodeShark> sipa: or at least to better isolate the different consensus changes into different units
589 2015-04-14 10:45:53 <CodeShark> i.e. block header verification doesn't really care about transaction structure
590 2015-04-14 10:46:09 <CodeShark> pow verification, that is
591 2015-04-14 10:46:35 <CodeShark> so any changes to the transaction structures do not affect any of the code that deals with mining or checking pow
592 2015-04-14 10:47:54 <CodeShark> we could completely replace the script format and never have to worry about the code that verifies pow
593 2015-04-14 10:47:59 <jtimon> "block header verification doesn't really care about transaction structure" ehem, #5995
594 2015-04-14 10:48:51 <CodeShark> looking, jtimon
595 2015-04-14 10:49:28 <jtimon> well, the smaller ones this one depends on without the moveonly or the function pointer are easier to review
596 2015-04-14 10:50:03 <jtimon> but I think what you want is more or less trying to be done
597 2015-04-14 10:50:21 <CodeShark> yes, I'm glad to see this
598 2015-04-14 10:51:02 <jtimon> well, the bottleneck is still review, many of the commtis are quite trivial
599 2015-04-14 10:51:53 <CodeShark> yes, good code reviewers are much harder to find than decent coders :)
600 2015-04-14 10:51:59 <jtimon> but trivial changes on consensus code, so...
601 2015-04-14 10:52:22 <wumpus> yes, if you're serious about helping things forward, doing code review would help a lot CodeShark:)
602 2015-04-14 10:52:53 <CodeShark> where shall I start?
603 2015-04-14 10:53:08 <sipa> :)
604 2015-04-14 10:53:19 <wumpus> whatever you think is important
605 2015-04-14 10:53:49 <CodeShark> well, dependency fixes and things that improve modularity are definitely high up on my list :)
606 2015-04-14 10:53:50 <sipa> CodeShark: and thr best way to understand tje codebase enough so you are a good reviewer is by contributing to it yourself :)
607 2015-04-14 10:54:44 <CodeShark> I've done far more of that than the former, sipa...
608 2015-04-14 10:54:51 <sipa> :)
609 2015-04-14 10:54:53 <sipa> i know
610 2015-04-14 10:55:10 <CodeShark> but the former is actually in many ways a more refined art :)
611 2015-04-14 10:55:15 <jtimon> CodeShark there's 3 modularity efforts I'm following: separating the consensus, the policy and the wallet code
612 2015-04-14 10:55:51 <CodeShark> what do you mean by policy?
613 2015-04-14 10:56:38 <jtimon> things that are not protocol rules but are part of the standard policy
614 2015-04-14 10:56:42 <wumpus> decisions that a node makes about e.g. relaying transactions
615 2015-04-14 10:56:54 <CodeShark> so things like minimum fees and DoS scores and such?
616 2015-04-14 10:57:09 <wumpus> minimum fees, yes, DoS scores not so much
617 2015-04-14 10:57:17 <wumpus> those are completely internal
618 2015-04-14 10:57:40 <sipa> CodeShark: policy is what node operators could change without breaking consensus
619 2015-04-14 10:57:48 <CodeShark> ok, got it
620 2015-04-14 10:58:01 <CodeShark> excluding the peer management stuff
621 2015-04-14 10:58:04 <wumpus> isstandard rules is another one
622 2015-04-14 10:58:07 <jtimon> CodeShark this branch is outdated, but I mean this https://github.com/bitcoin/bitcoin/blob/61145b4817963da2f03b623db7991461867f8d70/src/policy/policy.cpp
623 2015-04-14 10:58:07 <sipa> rules for mempool acceptance, relay of transactions, mining decisions
624 2015-04-14 10:58:17 <wumpus> yes, network code is a completely different concern
625 2015-04-14 11:00:00 <CodeShark> from a practical standpoint, the transaction fee issue is something I need a better solution for in applications...but the consensus stuff is far more fundamental to the aims we were discussing earlier
626 2015-04-14 11:00:55 <CodeShark> right now the only way I can tell whether a transaction will get relayed is to try to relay it
627 2015-04-14 11:01:01 <wumpus> (after all, even if you completely swapped out the network code for say, a different transport protocol, policy would still be the same)
628 2015-04-14 11:01:24 <CodeShark> but I have no way of constructing an unsigned transaction and then asking a relay node how much of a fee it will require once it has signatures
629 2015-04-14 11:01:25 <jtimon> I believe #6008 is the longest one on wallet movements
630 2015-04-14 11:01:56 <wumpus> asking a relay node would be a bad idea in any case, they could easily lie.
631 2015-04-14 11:02:12 <CodeShark> well, my current security model in most real deployments use a trusted node I run myself
632 2015-04-14 11:02:14 <wumpus> nodes should not deal out unverifyable information
633 2015-04-14 11:02:17 <CodeShark> but I do not use any RPC
634 2015-04-14 11:02:51 <wumpus> for a trusted node you can ask using a private protocol
635 2015-04-14 11:03:08 <CodeShark> yes, indeed - but I would like general support
636 2015-04-14 11:03:16 <CodeShark> I would prefer not to require a custom build for the node
637 2015-04-14 11:03:22 <CodeShark> anyhow...
638 2015-04-14 11:03:28 <CodeShark> it's not super urgent :p
639 2015-04-14 11:03:40 <wumpus> maybe add a queryfeestatistics in the REST interface?
640 2015-04-14 11:04:12 <CodeShark> yes, but perhaps I'll just end up implementing the same fee calculation logic on my end
641 2015-04-14 11:04:23 <jtimon> maybe we can add some policy RPCs after encapsulating the code
642 2015-04-14 11:05:18 <CodeShark> yeah, I suppose an RPC method for it would be doable...I could just disable the feature when connected to a node that doesn't expose this RPC
643 2015-04-14 11:05:25 <jtimon> in any case, making the policy more configurable mainly helps people that want to run full nodes without all the standard checks and things like that
644 2015-04-14 11:05:52 <jtimon> making it easier to implement replace-by-fee, child-ays-for-parents, etc
645 2015-04-14 11:05:53 <CodeShark> just being able to easily implement the same policy on my end and update parameters as needed when they change would suffice
646 2015-04-14 11:06:10 <CodeShark> but I guess the heuristics aren't that simple
647 2015-04-14 11:07:32 <jtimon> I hate to hear complains about "bitcoin development centralization" and then hear custom policies they want to be implemented as an "example" of something developers are "blocking" or something similar
648 2015-04-14 11:08:52 <jtimon> the policy encapsulation should solve that: "don't like the standard policy? use the testnet policy, tweak the policy parameters, completely disable the policy checks or just implement your own policy from scratch or by extending CStandardPolicy"
649 2015-04-14 11:08:59 <wumpus> jtimon: right, there is a lot of controversy about policies esp. replace-by-fee, making it switchable makes sense
650 2015-04-14 11:09:06 <CodeShark> heh, yes - it's a good idea
651 2015-04-14 11:10:35 <CodeShark> the third thing you mentioned is the wallet code - I think I want to stay away from that part as much as possible :p
652 2015-04-14 11:10:42 <jtimon> I mean, I'm not there yet, luke was closer but I want to avoid the txmempool dependency for policy if possible
653 2015-04-14 11:10:42 <sipa> jtimon: the worst controversy about a policy was however something thatbwas even changeable with a config option (the dust limit)
654 2015-04-14 11:10:54 <wumpus> CodeShark: if you do get involved, please coordinate with jonasschnelli
655 2015-04-14 11:11:00 <wumpus> (with the wallet, that is)
656 2015-04-14 11:11:04 <CodeShark> I already tried to do too many things with the wallet...and ended up just writing my own :p
657 2015-04-14 11:11:26 <sipa> jtimon: making the code easier to change will not help
658 2015-04-14 11:11:37 <CodeShark> if I get too involved with the wallet I'm liable to end up writing another wallet :p
659 2015-04-14 11:11:40 <jonasschnelli> CodeShark: come on. Join the new wallet team! :)
660 2015-04-14 11:12:03 <sipa> (which is by no means saying that the policy should not be abstracted out, but you may be overestimating the result)
661 2015-04-14 11:12:06 <jonasschnelli> (which is not acctually a team right now)
662 2015-04-14 11:12:06 <jtimon> sipa: yes, that's a good point, the dust and the op_return are the most controversial and are already configurable
663 2015-04-14 11:13:34 <wumpus> sipa: well it helps in that you can shift the blame away from the developers :)
664 2015-04-14 11:13:57 <jtimon> hopefully having the policy abstracted out and the policy config options listed under the same "policy options" section will make that clearer, but maybe they just keep complaining
665 2015-04-14 11:14:10 <wumpus> people wil always complain
666 2015-04-14 11:14:35 <wumpus> complaining is free, after all
667 2015-04-14 11:16:05 <jonasschnelli> I still try to understand the use cases of a Bip32 wallet (sorry if i'm having a long line). Would it make sense to allow Bip32 PubKey only wallets? Usecase: Alice has two wallets (multiwallet setup), a) normal Bip32 wallet, b) only pubykey Bip32 wallet (privkey belongs to Bob). Alice can now create multisig addresses (2of2 in this case) but cannot spend them.
668 2015-04-14 11:16:28 <CodeShark> jonasschnelli: I do that kind of thing all the time :)
669 2015-04-14 11:17:04 <jtimon> sounds like it makes sense, yes
670 2015-04-14 11:17:24 <CodeShark> the point of a bip32 pubkey wallet is that you can generate a virtually unlimited number of addresses subject to the exact same security policy but cannot spend anything
671 2015-04-14 11:17:32 <CodeShark> very useful for inbound payment processing
672 2015-04-14 11:17:43 <sipa> jonasschnelli: yes
673 2015-04-14 11:20:17 <jonasschnelli> Okay. And does this also makes sense: when creating a new wallet (RPC: "addwallet") allow user to set a desired chainpath over a RPC arg. Like (m/44/0/0/x) or (m/x) or (m/0/0/0/0/x)?
674 2015-04-14 11:20:58 <jonasschnelli> (and maybe (m/44/0/0/200+) to set the nChild offset)
675 2015-04-14 11:21:18 <CodeShark> if you want full flexibility, sure - I've implemented stuff like that for testing but don't use it very much in production
676 2015-04-14 11:22:48 <jonasschnelli> CodeShark: when creating a new wallet, how do you handle the seed? Show it once to the users (raw hex or mnemonic)?
677 2015-04-14 11:22:58 <jonasschnelli> * a new Bip32 master key
678 2015-04-14 11:23:20 <CodeShark> I can show a mnemonic...I have BIP39 support. I can also show the base58 serialized format in the BIP32 spec
679 2015-04-14 11:24:21 <CodeShark> but usability studies strongly seem to suggest the former is generally better :)
680 2015-04-14 11:24:22 <jonasschnelli> CodeShark: okay. When creating a address, do you keep the priv/pub key pair or do you only keep the parents key with nDepth, nChild and regenerate the privkey when required?
681 2015-04-14 11:24:24 <CodeShark> although less flexible
682 2015-04-14 11:25:04 <CodeShark> jonasschnelli: I only keep the parents and regenerate the child keys...but store the addresses for fast indexing
683 2015-04-14 11:25:18 <CodeShark> actually, I store the full redeemscript
684 2015-04-14 11:25:22 <CodeShark> for p2sh
685 2015-04-14 11:25:55 <jonasschnelli> CodeShark: whats the benefit of not storing the derived pub/privkey?
686 2015-04-14 11:26:28 <CodeShark> mostly space, but with privkeys there could be some attack vectors if you're not careful with how you encrypt them
687 2015-04-14 11:27:17 <jonasschnelli> CodeShark: but isn't it that (parentkey + nDepth + nChild) = privatekey in case of security?
688 2015-04-14 11:27:29 <CodeShark> it's easier to securely store a small amount of entropy and generate child keys as needed with a fair amount of security than it is to store a large number of private keys securely
689 2015-04-14 11:27:57 <CodeShark> at least, the latter requires a lot more care to make sure you don't leak info
690 2015-04-14 11:28:31 <CodeShark> and is also more prone to potential data corruption
691 2015-04-14 11:28:38 <jonasschnelli> CodeShark: okay. I'll see. So even the masterkey you don't keep (only the m/entropy)?
692 2015-04-14 11:29:28 <CodeShark> I do keep the master key right now because in my original schema I didn't keep the entropy, only the key and the chain code
693 2015-04-14 11:30:03 <CodeShark> but in principle I could get rid of it now.
694 2015-04-14 11:31:05 <jonasschnelli> so the m/seed and chainpath in a persistant store has less attack vectors then the raw pub/privkey
695 2015-04-14 11:31:46 <CodeShark> well, I would qualify that statement - recalculating could leak info via either memory buffer issues or timing attacks
696 2015-04-14 11:32:05 <jonasschnelli> CodeShark: hmm.. right.
697 2015-04-14 11:32:37 <CodeShark> but if someone can pull off these kinds of attacks on your computer chances are your system is already compromised - in practice for the kinds of applications I'm doing they seem extremely difficult to pull off
698 2015-04-14 11:33:06 <jonasschnelli> Agreed. But no performance concerns while regenerating keys?
699 2015-04-14 11:33:31 <CodeShark> I guess it could become an issue if there was a very long derivation path - but the only time I really need to do that is when signing transactions
700 2015-04-14 11:34:01 <CodeShark> and I haven't really gone more than a few levels deep in the HD nesting
701 2015-04-14 11:35:23 <jonasschnelli> hmm.. somehome my humbel opinion is more about storing the whole priv/pubkey instead of re-Deriving keys.
702 2015-04-14 11:36:01 <CodeShark> I do keep derived scripts
703 2015-04-14 11:36:03 <jonasschnelli> if the wallet is compromised, parentkey+path+level = privatkey. But regenerating keys might open the door for time-attacks from "outside"
704 2015-04-14 11:37:04 <CodeShark> I guess it depends on your application and execution environment
705 2015-04-14 11:38:05 <CodeShark> if you're continuously signing over a network using the same key it's probably far more dangerous than if you're sporatically signing different child keys locally
706 2015-04-14 11:38:11 <jonasschnelli> corewallet = same process as the fullnode (at the moment). Thats why i'd like to avoid regenerating.
707 2015-04-14 11:39:03 <CodeShark> or wait, I take that back :p
708 2015-04-14 11:39:14 <CodeShark> perhaps signing with different child keys over a network is more dangerous
709 2015-04-14 11:39:29 <CodeShark> since each roundtrip might leak different info
710 2015-04-14 11:39:48 <CodeShark> anyhow, my hunch is that such an attack is extremely difficult to pull off in practice
711 2015-04-14 11:40:18 <jonasschnelli> CodeShark: Agreed. I just try to find out how i should start implementing Bip32 for corewallet
712 2015-04-14 11:40:31 <jonasschnelli> Thanks for the chat!
713 2015-04-14 11:40:35 <sipa> jonasschnelli: have you seen my bip32 branch?
714 2015-04-14 11:40:53 <sipa> i implemented part of a simple bip32 key generation for the wallet
715 2015-04-14 11:41:03 <jonasschnelli> sipa: no! :)
716 2015-04-14 11:41:12 <jonasschnelli> sipa: Let me search
717 2015-04-14 11:41:27 <sipa> not sure what the name of the branch is
718 2015-04-14 11:41:29 <jonasschnelli> https://github.com/sipa/bitcoin/tree/bip32?
719 2015-04-14 11:41:41 <sipa> hdw or hd or bip32 or detwallet or ...
720 2015-04-14 11:42:06 <sipa> no
721 2015-04-14 11:42:13 <sipa> that's just the derivation
722 2015-04-14 11:42:17 <sipa> not the wallet code
723 2015-04-14 11:42:39 <jonasschnelli> "hwd" branch is only HMAC stuff
724 2015-04-14 11:42:45 <CodeShark> do we have a bip32 derivation unit implemented yet for bitcoind?
725 2015-04-14 11:43:07 <CodeShark> I guess sipa's
726 2015-04-14 11:43:09 <sipa> CodeShark: yes
727 2015-04-14 11:43:11 <sipa> for ages
728 2015-04-14 11:43:14 <CodeShark> ok ;)
729 2015-04-14 11:43:18 <sipa> jonasschnelli: detwallet2
730 2015-04-14 11:43:52 <jonasschnelli> sipa: thanks. Nice!
731 2015-04-14 11:45:01 <sipa> i don't remember what the state was
732 2015-04-14 11:45:11 <jonasschnelli> will figure it out. :)
733 2015-04-14 11:46:01 <jonasschnelli> jtimon: regarding https://github.com/bitcoin/bitcoin/pull/5994#issuecomment-92775415, what's the benefit of passing CChainParams to the miners functions?
734 2015-04-14 11:46:28 <wumpus> jonasschnelli: the main idea of passing the params around is less reliance on global state
735 2015-04-14 11:46:42 <jtimon> yes, see https://github.com/bitcoin/bitcoin/pull/5970 for example
736 2015-04-14 11:47:08 <jonasschnelli> wwu
737 2015-04-14 11:47:17 <jonasschnelli> (sry, misstype)
738 2015-04-14 11:47:37 <jonasschnelli> jtimon: wumpus: okay. I see. This makes sense.
739 2015-04-14 11:47:51 <jtimon> well, I'll update that now, I've been replacing params for chainparams and consensusParams as suggested by cfields to avoid confusion
740 2015-04-14 11:48:14 <jonasschnelli> okay...
741 2015-04-14 11:49:36 <jtimon> ideally one day we will be able to get rid of https://github.com/bitcoin/bitcoin/blob/master/src/test/test_bitcoin.cpp#L34
742 2015-04-14 11:51:13 <jtimon> or at least directly use Params(CBaseChainParams::MAIN) to initialize the wallet instead of calling SelectParams() or something
743 2015-04-14 11:51:56 <jonasschnelli> jtimon: but in case of PR5994, miner detaching, what whould i do with the chainparams parameter? Isn't the rest of the code relying on the global state?
744 2015-04-14 11:52:59 <sipa> jonasschnelli: one step at a time :)
745 2015-04-14 11:53:12 <sipa> there are tons of global state remaining
746 2015-04-14 11:53:38 <jtimon> well, ProcessBlockFound for example won't use the new parameter yet, but when main::ProcessNewBlock takes it explicitly, one less change will be required
747 2015-04-14 11:54:19 <jonasschnelli> okay. Agreed. But it would look like that we pass something which is unused in the rest of the function code (which might bring Diapolo to the point of adding another PR to remove it).
748 2015-04-14 11:55:31 <jtimon> maybe diapolo is happy to take over #5970 instead or something, I don't know
749 2015-04-14 11:56:15 <jtimon> some of the other functions could use it already though
750 2015-04-14 11:56:53 <jonasschnelli> jtimon: okay. I see the point. Will try to add a commit.
751 2015-04-14 11:57:20 <jtimon> you can also pass Consensus::Params to UpdateTime if you want to, like in #5998
752 2015-04-14 11:57:45 <jtimon> cool, thanks
753 2015-04-14 12:15:46 <jonasschnelli> jtimon: https://github.com/jonasschnelli/bitcoin/commit/35a97d8692e1ae4694d58307f7b5f6946c0c7104
754 2015-04-14 12:17:11 <jtimon> great
755 2015-04-14 12:18:24 <jtimon> mhmm, at this point, what is CreateNewBlockWithScript exactly doing? isn't it identical to CreateNewBlock ?
756 2015-04-14 12:19:10 <jtimon> I think you can just remove it and call CreateNewBlock directly instead
757 2015-04-14 12:21:30 <jonasschnelli> hmm... ;-)
758 2015-04-14 12:22:24 <jonasschnelli> jtimon: right. Kinda makes no sense. But changing CreateNewBlock to add CChainParams will result in a bigger changeset because it's called from tests / rpcmining etc.
759 2015-04-14 12:24:05 <jonasschnelli> i'll remove CreateNewBlockWithScript and use CreateNewBlock (without CChainParams [can be changed later in a sep. PR|)
760 2015-04-14 12:30:11 <jtimon> yes, can be changed later since you don't need to change the interface then
761 2015-04-14 12:30:32 <jtimon> but you already did in the commit you show me
762 2015-04-14 12:31:19 <jtimon> whatever, my request was to do so when you're already changing the arguments for the function
763 2015-04-14 12:32:00 <jtimon> and pobably just squash it in those same commits since the number of lines changed will be the same
764 2015-04-14 12:32:23 <jonasschnelli> jtimon: force pushed, new commit hash: https://github.com/bitcoin/bitcoin/commit/e898656fa48cf8a1824eeef880654bc1516a68ce
765 2015-04-14 12:39:20 <jtimon> jonasschnelli looks good to me