1 2015-12-02 11:10:13 <jtoomim> anyone online who understands sync.h?
  2 2015-12-02 11:10:23 <sipa> sure
  3 2015-12-02 11:10:46 <sipa> at least i used to :)
  4 2015-12-02 11:10:57 <jtoomim> that might be good enough
  5 2015-12-02 11:11:13 <jtoomim> so we've been running into some issues with cs_main locking in our testnet tests with big blocks
  6 2015-12-02 11:11:37 <jtoomim> prattler uncovered the most recent one
  7 2015-12-02 11:11:58 <sipa> there is a lot of room for improvement there for sure
  8 2015-12-02 11:12:11 <jtoomim> in this instance, LOCK(cs_main) is required in order to respond to a getheaders request
  9 2015-12-02 11:12:20 <sipa> yup
 10 2015-12-02 11:12:31 <sipa> because chainActive
 11 2015-12-02 11:12:37 <jtoomim> and my mining node has GBT latencies of about 12 seconds
 12 2015-12-02 11:12:40 <sipa> i think?
 13 2015-12-02 11:12:56 <jtoomim> and my mining node almost always starts a new GBT immediately after finishing UpdateTip
 14 2015-12-02 11:12:57 <gmaxwell> jtoomim: not a very big mempool I guess?
 15 2015-12-02 11:13:08 <jtoomim> we're trying to keep the mempool below 20 MB
 16 2015-12-02 11:13:13 <sipa> update to bitcoin core master, the GBT speed is dramtically improved
 17 2015-12-02 11:13:21 <jtoomim> but it's hard to go smaller than that with block sizes of 9 MB
 18 2015-12-02 11:13:33 <jtoomim> sipa yes, that's an option, but it's a lot of work
 19 2015-12-02 11:13:43 <jtoomim> since we
 20 2015-12-02 11:13:55 <jtoomim> have a lot of modifications on bitcoinxt that we're using to get the log info we need
 21 2015-12-02 11:14:22 <jtoomim> anyway, i thought it might be a good idea to start working on converting some of the locks to readers-writers locks
 22 2015-12-02 11:14:47 <jtoomim> because both getheaders and GBT only need to read chainactive etc, so they don't need to block each other
 23 2015-12-02 11:15:13 <sipa> that will be a lot of work, as cs_main is often used recursively (which should also get fixed, no doubt)
 24 2015-12-02 11:15:27 <sipa> and i don't think recursive rw locks exist
 25 2015-12-02 11:15:44 <sipa> but absolutely
 26 2015-12-02 11:16:12 <gmaxwell> though with caching impacts fewer things are 'read only' than you might guess.
 27 2015-12-02 11:16:18 <jtoomim> i figured i could leave it as a unique_lock for most stuff, but hack in a quick-and-dirty RW lock mechanism
 28 2015-12-02 11:16:32 <sipa> jtoomim: unique lock is not recursive
 29 2015-12-02 11:17:00 <sipa> and anything that touches the chainstate needs a single lock
 30 2015-12-02 11:17:51 <jtoomim> i don't follow what you mean by "often used recursively", sorry
 31 2015-12-02 11:18:10 <sipa> a recursive lock is one that allows locking from the same thread a second time
 32 2015-12-02 11:18:14 <sipa> without blocking
 33 2015-12-02 11:18:48 <sipa> so you can just write "LOCK(cs_main) all over the place, regardless of whether the caller already had the lock or not
 34 2015-12-02 11:18:59 <jtoomim> ok, like how GBT and CNB both lock cs_main
 35 2015-12-02 11:19:06 <sipa> right
 36 2015-12-02 11:19:31 <sipa> the solution is to split up the functions into ones that only do an assertlockheld
 37 2015-12-02 11:19:38 <sipa> and then wrappers around it that lock
 38 2015-12-02 11:20:26 <sipa> recursive locks are less efficient and lead to badly designed logic (you lose view of where the boundaries of your lock-touching code are)
 39 2015-12-02 11:21:27 <sipa> but if that is done, yes, absolutely, cs_main and menpool.cs etc should become rwlocks
 40 2015-12-02 11:21:57 <sipa> there is also a lot of network/msg processing logic that unnecessarily uses cs_main
 41 2015-12-02 11:22:23 <sipa> CNodeState for example is covered by cs_main but really should have a separate lock
 42 2015-12-02 11:25:44 <jtoomim> gmaxwell can you clarify your comment about caching?
 43 2015-12-02 11:26:35 <sipa> many things that do not seem to touch the chainstate for example do have mutatingf effects
 44 2015-12-02 11:26:43 <sipa> as they modify the caches underneath
 45 2015-12-02 11:26:53 <gmaxwell> ^
 46 2015-12-02 11:26:55 <sipa> and this is true for CNB in 0.11
 47 2015-12-02 11:27:06 <sipa> not anymore for CNB in 0.12 iirc
 48 2015-12-02 11:27:25 <sipa> it does not touch the utxo set, only data cached inside the mempool
 49 2015-12-02 11:27:26 <jtoomim> hmm, okay.
 50 2015-12-02 11:29:00 <gmaxwell> jtoomim: I know you're just trying to get things working; but I hope you keep in mind that a 20MB mempool isn't really realistic for a 9mb block. (because of posson distributed blocks it will need to be many times it; even when the offered and target blocksize are not dissimilar.
 51 2015-12-02 11:30:25 <jtoomim> gmaxwell do you mean for a scenario in which there is a continuous stream of 9 MB blocks?
 52 2015-12-02 11:32:02 <jtoomim> under BIP101 the max blocksize should generally be larger than the mempool size, so i would expect that 20 MB mempool would be greater than we would typically see for a 9 MB block
 53 2015-12-02 11:32:25 <gmaxwell> jtoomim: I mean we had mempools much over that when blocks were 500k max typically, and a lot of variance in sizes in bitcoin. Blocks arrive randomly, and many blocks arrive very quickly, and yet blocks that happen 12x later than usual happens ones to a couple times a week.
 54 2015-12-02 11:33:00 <tulip> jtoomim: do you still hold the 100KB transaction size limit, or was that just discussed?
 55 2015-12-02 11:33:04 <jtoomim> yes, and i expect those really slow blocks are the ones that would end up being 8 or 9 MB
 56 2015-12-02 11:33:25 <jtoomim> i believe BitcoinXT still has the 100 kB IsStandard check
 57 2015-12-02 11:33:38 <tulip> consensus, not IsStandard.
 58 2015-12-02 11:34:00 <gmaxwell> e.g. if offered load is 9 mb, and all blocks are choosing to clear it (no policy that tried to lower orphan rate); the minimum mempool size will be 108mb, and probably actually a fair bit more, since there would have been tx in before then.
 59 2015-12-02 11:34:31 <gmaxwell> jtoomim: okay, so you're looking at offered loads much lower than the size? if so then a small multiple makes more sense.
 60 2015-12-02 11:35:12 <sipa> the basic question is what transaction rate are you supporting; the block size and mempool size follow from that
 61 2015-12-02 11:35:14 <jtoomim> tulip i think a 100 kB max transaction size consensus rule would eliminate some useful edge applications like lighthouse, and that a separate sigops/verification time limit should be used on a per-block level (like what gavin already coded into BIP101)
 62 2015-12-02 11:36:24 <jtoomim> gmaxwell yes, i expect if BIP101 gets adopted that it would grow faster than demand for the near future, and that should remain true for longer than it takes to fix GBT/CNB and the locking mess completely
 63 2015-12-02 11:37:00 <jtoomim> my model of tx demand growth is approximately doubling every year or slower
 64 2015-12-02 11:37:32 <jtoomim> with BIP101's scaling algo, that would give us about 6 years before we hit block congestion again
 65 2015-12-02 11:37:38 <sipa> my model for tx demand growth is "fill the available space, as long as it is nearly free"
 66 2015-12-02 11:38:15 <jtoomim> sipa i don't think that matches with the data that we've collected over the last 5 years
 67 2015-12-02 11:38:31 <jtoomim> people take time to find and implement uses for block space
 68 2015-12-02 11:38:39 <sipa> sure, not instantly
 69 2015-12-02 11:39:12 <sipa> but there are so many people with ill-advised use cases that a sudden extra space would imho very quickly be filled
 70 2015-12-02 11:39:20 <sipa> and we'll have the same question again
 71 2015-12-02 11:39:49 <jtoomim> and they have to pay miners enough for that space to make it worth the miners' while
 72 2015-12-02 11:39:52 <midnightmagic> it's like i'm wandering through the chocolate factory and augustus just got sucked up the tube. why, mr. wonka, why don't you care that kid almost drowned..?
 73 2015-12-02 11:39:54 <jtoomim> and make it worth the orphan rates
 74 2015-12-02 11:40:17 <sipa> jtoomim: in a decentralized setting
 75 2015-12-02 11:40:42 <sipa> with sufficiently centralized mining, block space has no effect on orphan rates
 76 2015-12-02 11:41:06 <jtoomim> in a centralized setting, they have to make it worth the miners' loss of real USD income from the economic/political fallout of centralized mining
 77 2015-12-02 11:41:31 <jtoomim> also, it's possible to decrease the block size without the cooperation of the hashrate majority
 78 2015-12-02 11:41:37 <sipa> that's true
 79 2015-12-02 11:41:43 <jtoomim> but increasing the blocksize max requires their cooperation
 80 2015-12-02 11:41:49 <sipa> but i don't see either of those happening
 81 2015-12-02 11:42:16 <jtoomim> it's not too hard to soft-fork down the block size
 82 2015-12-02 11:42:34 <sipa> a soft fork requires miner consensus
 83 2015-12-02 11:42:38 <jtoomim> you just convince all the exchanges to use a bitcoind with a lower block size limit, and let miners mine on whatever chain they want
 84 2015-12-02 11:42:55 <jtoomim> nah, you just need enough miners to make sure that the short chain continues forward
 85 2015-12-02 11:42:59 <jtoomim> it's not exactly a soft fork
 86 2015-12-02 11:43:07 <jtoomim> more like a reverse-order hard fork
 87 2015-12-02 11:43:25 <sipa> no, you need to convince every full node operator in the world to lower the limit
 88 2015-12-02 11:43:31 <sipa> or they will be forked off
 89 2015-12-02 11:43:53 <jtoomim> that's the point, you get the economic majority to fork off
 90 2015-12-02 11:44:07 <Luke-Jr> miner majority* for softfork, not consensus..
 91 2015-12-02 11:44:35 <gmaxwell> jtoomim: uh at virtually every point in bitcoin's history the used capacity has been very close to what miners were actually allowing via the soft limits.
 92 2015-12-02 11:44:42 <jtoomim> miners can continue to make their large-block chain, but they can't spend their revenue anywhere that matters, so they stop making big blocks
 93 2015-12-02 11:44:55 <gmaxwell> jtoomim: when we incresed the soft limit from 250k to 750k; the average usage increased 3x.
 94 2015-12-02 11:45:12 <jtoomim> gmaxwell i think that confuses cause and effect
 95 2015-12-02 11:45:24 <jtoomim> the soft limits were increased when usage got close to them
 96 2015-12-02 11:45:27 <gmaxwell> it's a time series, so you can actually see the direction.
 97 2015-12-02 11:45:30 <tulip> jtoomim: you might find this graph enlightening. http://webbtc.com/graphs/block_size.png
 98 2015-12-02 11:45:49 <midnightmagic> ... is it raining, is it snowing ...
 99 2015-12-02 11:46:42 <gmaxwell> jtoomim: consider, if someont offered you free (externalized costs) highly replicated perpetual storage... how much would you use? ... I'd have streaming backups of my computer into it.
100 2015-12-02 11:46:59 <jtoomim> depends on how fast it was
101 2015-12-02 11:47:44 <jtoomim> i mean, gmail gives me, what, 5 GB free?
102 2015-12-02 11:47:50 <jtoomim> i think I probably only use about 100 MB of that
103 2015-12-02 11:48:05 <jtoomim> that's free highly replicated perpetual storage
104 2015-12-02 11:48:11 <midnightmagic> gmail != perpetual, or even safe storage.
105 2015-12-02 11:48:13 <gmaxwell> wow. really? only 100mb?
106 2015-12-02 11:48:25 <gmaxwell> (and indeed gmail has lost data)
107 2015-12-02 11:48:34 <gmaxwell> (and bans tools for using it for backup)
108 2015-12-02 11:48:37 <jtoomim> yeah, i keep most of my email on my own servers
109 2015-12-02 11:48:39 <tulip> gmaxwell: I was banned from flickr when they had a free 1TB of storage offering, I encoded my backups as PNGs and uploaded several hundred GB of them.
110 2015-12-02 11:48:49 <gmaxwell> "14.5 GB (96%) of 15 GB used"
111 2015-12-02 11:48:54 <Diablo-D3> http://www.danielsen.com/jokes/objecttoaster.txt
112 2015-12-02 11:49:00 <Diablo-D3> gmaxwell, midnightmagic: ^
113 2015-12-02 11:49:01 <gmaxwell> ^ my gmail account.
114 2015-12-02 11:49:03 <jtoomim> "free" isn't the most important attribute to me
115 2015-12-02 11:49:12 <Diablo-D3> gmaxwell: also, _how the fuck did you do that_
116 2015-12-02 11:49:14 <gmaxwell> jtoomim: who said you had to keep email there.
117 2015-12-02 11:49:28 <gmaxwell> Diablo-D3: mailing lists, really.
118 2015-12-02 11:49:29 <sipa> to many people, convenience is the important attribute
119 2015-12-02 11:49:37 <Diablo-D3> gmaxwell: I use like half a gig at most, and thats entirely due to it backing up original photos
120 2015-12-02 11:49:54 <Diablo-D3> virtually none of it is because of gmail
121 2015-12-02 11:49:56 <sipa> i have no doubt that convenient tools for blockchain backup will appear if enough space is available
122 2015-12-02 11:49:57 <gmaxwell> jtoomim: lots of people, for example, want to use bitcoin transactions for ephemeral messaging; because the p2p network is accessible from everywhere.
123 2015-12-02 11:50:35 <jtoomim> anyway, blockchain space isn't and never will be free
124 2015-12-02 11:50:35 <tulip> sipa: there's some available today, one service floods the UTXO set by encoding images as PKH fragments.
125 2015-12-02 11:50:43 <gmaxwell> There have already been blockchain VFS drivers people have created. :(
126 2015-12-02 11:50:44 <jtoomim> right now, blockchain space costs about $0.02 per kB
127 2015-12-02 11:51:20 <jtoomim> even at 1/1000th of that price, it's a lot more expensive than rolling out your own distributed storage system
128 2015-12-02 11:51:22 <tulip> jtoomim: that cost is paid to miners who don't have to store any blocks at all.
129 2015-12-02 11:51:27 <gmaxwell> And having to tell someone "no, node operators really will begin blocking your pict sharing thing if you run it on the bitcoin network" is something I encounter every couple weeks.
130 2015-12-02 11:51:44 <jtoomim> it's an externalized cost, yes, but miners still have to pay the cost of orphans and processing delays
131 2015-12-02 11:51:53 <gmaxwell> jtoomim: yes, thats what it costs in a competatively loaded system.
132 2015-12-02 11:52:16 <gmaxwell> Thats not what it costs when the offered load is well below capacity.
133 2015-12-02 11:52:24 <jtoomim> citation needed
134 2015-12-02 11:52:50 <gmaxwell> jtoomim: they don't so long as they centeralize; which has been their reaction to orphaning in bitcoin. (e.g. rise of Ghash is a nice example of that)
135 2015-12-02 11:53:02 <jtoomim> the data that i've seen for fees indicates that the real cost has never gone below $0.02 even in times when blocks had a lot of empty space
136 2015-12-02 11:53:09 <Diablo-D3> speaking of which
137 2015-12-02 11:53:13 <tulip> jtoomim: no, not really, with validation-free mining the "orphan" cost of making enormous or hard to validate blocks is zero. no amount of obstruction stops a majority of the hashrate from building on top of the blocks.
138 2015-12-02 11:53:15 <Diablo-D3> wtf is the thash of the netowrk now?
139 2015-12-02 11:54:08 <sipa> and of course validation-free market requires miners to trust each other, turning the market into something that's even harder to emter
140 2015-12-02 11:54:09 <midnightmagic> 579,586 TH?
141 2015-12-02 11:55:35 <jtoomim> from what i've seen, the strongest effect on per-tx fees is what the default setting on wallet software is
142 2015-12-02 11:55:55 <gmaxwell> jtoomim: I've made many thousands of transactions on bitcoin mainnet and almost never paid a fee (beyond throwing away dust.)
143 2015-12-02 11:56:16 <gmaxwell> (I'd actually give a total from a wallet of mine, but coinjoins are making the counting bonkers)
144 2015-12-02 11:56:35 <jtoomim> e.g. the per-tx fees dropped by about 4x when 0.8.2 was released
145 2015-12-02 11:57:12 <gmaxwell> There are a bunch of things paying static fees, which are unnecessarily high; but low enough the users don't bother defeating them.
146 2015-12-02 11:57:52 <Diablo-D3> midnightmagic: wtf? we're up to half a ph?
147 2015-12-02 11:58:01 <gmaxwell> But all this time you can transact much more cheaply, and this is part of why offered load has always met capacity.
148 2015-12-02 11:58:34 <jtoomim> i don't buy the "offered load has always met capacity" argument
149 2015-12-02 11:58:39 <midnightmagic> Diablo-D3: Huh? We're at 579 ph.
150 2015-12-02 11:58:51 <jtoomim> the capacity was changed in response to load getting close to soft limits, not the other way around, as i see it
151 2015-12-02 11:59:09 <sipa> jtoomim: so you suggest we continue that?
152 2015-12-02 11:59:21 <jtoomim> and there was a delay each time in between load reaching that level and capacity increasing
153 2015-12-02 11:59:57 <moa> Diablo-D3: half an exahash
154 2015-12-02 12:00:34 <jtoomim> sipa i don't see why having headroom in the block size limit is a bad thing
155 2015-12-02 12:00:37 <gmaxwell> so for example, in a wallet with 275 transactions here, my current online wallet, recieved and sent. there are 22 that paid a fee at all, and 5 of those are below 1000 base units.
156 2015-12-02 12:00:59 <sipa> jtoomim: if there is headroom, the marginal cost for filling it is zero
157 2015-12-02 12:01:10 <jtoomim> not zero, no
158 2015-12-02 12:01:28 <sipa> jtoomim: in a system that is already designed to handle the maximum capacity anyway
159 2015-12-02 12:02:04 <jtoomim> zero is a way of saying that you're too lazy to calculate the actual cost, or that you think the cost is very small relative to other factors, but it is never zero
160 2015-12-02 12:02:27 <gmaxwell> jtoomim: well it actually can be negative, not zero.
161 2015-12-02 12:02:42 <Diablo-D3> midnightmagic: WTF WE'RE IN PH?!
162 2015-12-02 12:02:42 <jtoomim> for selfish mining, sure
163 2015-12-02 12:02:44 <gmaxwell> For example if iBLT is in use and the transaction is in the mempool it has cost to exclude it.
164 2015-12-02 12:02:53 <gmaxwell> no not for selfish mining.
165 2015-12-02 12:02:59 <Diablo-D3> where the fuck are all you asshats putting the mining hardware!
166 2015-12-02 12:03:40 <gmaxwell> you already pay the verification cost on reciept, and iBLT penalizes you for building a block which is inconsistent with nodes mempools; you have to send as much data as you add or remove.
167 2015-12-02 12:03:57 <jtoomim> interesting
168 2015-12-02 12:04:28 <jtoomim> so then other miners can vote on whether that transaction should be in the next block by including it or not in their mempools
169 2015-12-02 12:04:40 <gmaxwell> (technically rusty came up with an optimization to make removal somewhat cheaper; but it still has a cost.. so it can still be negative to add it)
170 2015-12-02 12:04:52 <jtoomim> maybe having a multi-tiered mempool, the top tier of which goes into the IBLT and the lower tiers are just a cache
171 2015-12-02 12:05:00 <gmaxwell> Not humans, just miners... since you only really care that your block reaches 30% of the hashrate quickly.
172 2015-12-02 12:05:09 <gmaxwell> Everyone else can catch up in their own sweet time.
173 2015-12-02 12:06:30 <gmaxwell> With orphaning fixed, your own income is maximized by including all you can right now. So a single feerate sorted mempool is a pretty good predictor.
174 2015-12-02 12:06:51 <gmaxwell> though you can augment it with extra signaling to make it cheaper to remove things.
175 2015-12-02 12:07:18 <sipa> iirc rusty's optimization means removal costs 1/4 of adding
176 2015-12-02 12:07:35 <gmaxwell> And maybe get the cost to exclude to effectively zero-ish again.
177 2015-12-02 12:08:12 <gmaxwell> sipa: I only mentioned it so if rusty mentioned it later jtoomim wouldn't feel I mislead him; I don't think it was important for the point being made. :)
178 2015-12-02 12:08:36 <gmaxwell> any any case, food for thought.
179 2015-12-02 12:11:26 <jtoomim> i wonder if that might provide a mechanism for collective bargaining on fee floors
180 2015-12-02 12:12:01 <jtoomim> miners might set a different minrelaytxfee or equivalent
181 2015-12-02 12:12:09 <jtoomim> or might limit their mempools to a different size
182 2015-12-02 12:12:27 <jtoomim> this would slow down IBLTs for other miners who used a different setting
183 2015-12-02 12:12:39 <jtoomim> and the amount of slowdown would be proportional to the magnitude of the difference
184 2015-12-02 12:13:02 <jtoomim> this would let miners choose a point on the supply and demand curves that maximizes revenues
185 2015-12-02 12:13:27 <tulip> you can't rationalise like that due to the existence of validation free mining.
186 2015-12-02 12:13:53 <jtoomim> by punishing miners who deviate from the mean
187 2015-12-02 12:14:18 <jtoomim> tulip only if all miners are using validation-free mining, and only if the risk of using validation-free mining is zero
188 2015-12-02 12:14:51 <jtoomim> if a uses validation free and b does not, a is still subject to propagation costs for blocks that a creates and which b may be slow to mine off of
189 2015-12-02 12:15:17 <sipa> only if the group doing vfm does not have a near majority of the hashrate
190 2015-12-02 12:15:28 <sipa> once that is the case, it's the only way around
191 2015-12-02 12:16:11 <jtoomim> if vfm miners have a supermajority, then it can become advantageous for non-vfm miners to inject invalid blocks with valid PoW to try to make a long invalid chain... which is not at all nice.
192 2015-12-02 12:16:32 <sipa> vfm miners only trust each other
193 2015-12-02 12:16:42 <tulip> VFM works by trusting the headers of other trusted miners, not anybodies headers.
194 2015-12-02 12:17:00 <jtoomim> i think that f2pool is taking the headers from almost all of the major pools
195 2015-12-02 12:17:23 <jtoomim> i've seen them mine off of slush ridiculously quickly before
196 2015-12-02 12:17:32 <jtoomim> trust is easily broken
197 2015-12-02 12:17:35 <sipa> they'll learn their lesson after a few incidents
198 2015-12-02 12:17:45 <sipa> they don't need trust if they can sue
199 2015-12-02 12:19:39 <sipa> and apparently, even after the bip66 incident it is still more profitable to do vfm
200 2015-12-02 12:19:40 <midnightmagic> f2pool isn't (or wasn't) mining on anyone's headers, just specific miner
201 2015-12-02 12:19:52 <sipa> though better relay protocols can change that
202 2015-12-02 12:20:57 <jtoomim> can you think of any other ways in which collective bargaining can be achieved?
203 2015-12-02 12:22:02 <gmaxwell> jtoomim: careful with the collective bargaining; since encouraging collusion out side of the rules and physics of the system can also enable a lot of much more obviously bad things. E.g. effective demurrage by forcing a percentage of tx value to fees, or things like the wikileaks payment processing block.
204 2015-12-02 12:22:05 <jtoomim> it seems to me that having miners work together to maximize total fee revenue would probably result in a transaction cost and volume that was acceptable to both miners and transaction generators
205 2015-12-02 12:22:45 <gmaxwell> Even if we can't completely prevent collusion, I think it's perferable to not instutionalize it such that it will be warm and ready for scope creep.
206 2015-12-02 12:23:41 <jtoomim> sure, it's something that we need to be careful about
207 2015-12-02 12:23:51 <gmaxwell> In effect a blocksize limit is a shelling point to drive fees and also keep resource usage at level that nodes (which verify but don't mine and so can only express their will be rejecting blocks) can support.  It's not the only one though, and things like flexcap proposals try to seek out other ones.
208 2015-12-02 12:24:33 <jtoomim> but if there were a system in the code or the protocol that defined the rules by which a price floor was determined, then there would not need to be an out-of-band negotiation system
209 2015-12-02 12:24:43 <gmaxwell> (and maybe if one of those proposals work then the static block size limits are only needed as software engineering targets, hardware resource planning, or emergency backstops)
210 2015-12-02 12:25:45 <gmaxwell> yes. hm. well so usually any rules that do anything with fees can be defeated by out of band fees... but this is in the other direction where you're setting a floor. I guess they can be bypassed with refunds but thats a bit ugly.
211 2015-12-02 12:26:32 <gmaxwell> e.g. a miner could break rank and charge lower fees by coinjoining with anyone who wants to be in his block, and pay the mandatory fees himself; or more trusty just refund after the fact either out of band or with seperate bitcoin payments.
212 2015-12-02 12:26:42 <tulip> just to be clear, I don't blame mining pools for VFM. it's symptomatic of other problems.
213 2015-12-02 12:26:59 <jtoomim> but the miner would have to race with other miners for that transaction
214 2015-12-02 12:27:28 <midnightmagic> i do!
215 2015-12-02 12:29:10 <jtoomim> gmaxwell so the tradeoff for users would be trust vs cost for colluding with these scabs
216 2015-12-02 12:29:31 <gmaxwell> well the trust could be elimianted by coinjoin but that has its own overhead.
217 2015-12-02 12:31:04 <jtoomim> the coinjoin would probably be detectable
218 2015-12-02 12:31:18 <jtoomim> it would be hard for a miner to do this without other miners knowing
219 2015-12-02 12:31:43 <gmaxwell> I think I've probably not given fee floors an adequate amount of thought because they don't address the issue that it's easy for miners to use huge pools then support mega super blocks (just buy a 24 core 256 GB ram host for the uberpool); so they'd potentially support a pretty low floor to max out that income; and the verifiers that don't see that income (and are decenteralized rather thana few mega
220 2015-12-02 12:31:49 <gmaxwell> servers) get left in the dust.
221 2015-12-02 12:32:18 <gmaxwell> jtoomim: yea though miners blackballing other miners blocks... also pretty bad news, (and hurts convergence.. you were confirmed, now you're not :))
222 2015-12-02 12:32:42 <jtoomim> the miners/pools want to maximize the fee * tx_count product
223 2015-12-02 12:32:52 <gmaxwell> but assuming the leaving verifiers in the dust and thus leaving the miners without a check were solved some other ways perhaps protocol agreed floors might be useful.
224 2015-12-02 12:32:54 <jtoomim> lowering the fee is not a straightforward way to maximize that product
225 2015-12-02 12:33:22 <gmaxwell> jtoomim: I know. But it's not not a straightforward way, depends on the demand curve.
226 2015-12-02 12:33:44 <jtoomim> i think maybe the threat of blackballing might be enough to prevent people from trying this coinjoining/scab scheme
227 2015-12-02 12:33:57 <jtoomim> also, the effect that it would have on the bitcoin exchange rate
228 2015-12-02 12:34:24 <jtoomim> and the demand curve might be exactly the right thing to determine that fee floor
229 2015-12-02 12:34:53 <jtoomim> goodnight gmaxwell, thanks for engaging
230 2015-12-02 12:35:28 <gmaxwell> jtoomim: I've got (before I go) pretty powerful counter evidence in the exchange rate thing; when ghash.io had 30% hashrate they were used to perform a 1000 BTC (lots at the time!) finney attack against a bitcoin user; for months after both their hashrate and the exchange rate grew.
231 2015-12-02 12:35:53 <jtoomim> that was the gambling website, right?
232 2015-12-02 12:36:06 <gmaxwell> https://bitcointalk.org/index.php?topic=321630.0  I think this changed my thinking about that element of ecosystem being strongly self correcting.
233 2015-12-02 12:36:15 <jtoomim> supposedly by a rogue employee?
234 2015-12-02 12:36:38 <gmaxwell> Yes, that was the explination that came a long time after
235 2015-12-02 12:37:23 <gmaxwell> So I don't think that its at all clear that playing chicken with undermining the value of bitcoin isn't a perfectly good strategy.
236 2015-12-02 12:37:51 <gmaxwell> You can also look at the significant levels of centeralization in mining as an example of taking that risk too...
237 2015-12-02 12:38:19 <jtoomim> miners are risk-averse, but pools are not
238 2015-12-02 12:38:22 <gmaxwell> If you want your transactions to be secure against someone who might have compromised one of the larger miners; you really need to wait a LOT of confirmations.
239 2015-12-02 12:38:35 <jtoomim> pools have thin margins, and would be willing to take these kinds of risks to make a quick buck
240 2015-12-02 12:39:16 <gmaxwell> right, and lots of hashrate goes through them (also... non-trivial amounts are on service that you can buy and point at anything.. which also suggests that hardware owners are not too worried about buring things down)
241 2015-12-02 12:39:35 <jtoomim> yeah, i hate the hashrate rentals too
242 2015-12-02 12:39:41 <jtoomim> a lot of my customers use them
243 2015-12-02 12:39:50 <gmaxwell> 0.1% hashrate is perfectly fine for a finney attack, and in your one block you hit lots of spends you can rob many zero conf things at once.
244 2015-12-02 12:40:24 <gmaxwell> (then you have that virutally all mining is unauthenticated; and in altcoin land address space hijacking has happened several times.. though so far only to divert hashpower)
245 2015-12-02 12:40:40 <jtoomim> if they become common, i wonder if there might be a service offered by pools for anti-finney protection
246 2015-12-02 13:32:20 <instagibbs> The formation of Finney Protection Rackets. Progress!
247 2015-12-02 14:08:20 <jtoomim> sipa i ended up switching over to my own hacked-together fast CNB version. that's getting me 700 ms CNB latency on a 7 MB mempool, which is maybe good enough for now. locks another day...
248 2015-12-02 14:31:22 <what_if_tho> Is there some agreed upon method of watching an address for incoming payments? Running your own softwarr that is
249 2015-12-02 14:32:21 <sipa> what_if_tho: bitcoin core has a importaddress function to make its wallet watch for payments
250 2015-12-02 20:56:41 <jcorgan> kanzure: ping
251 2015-12-02 20:58:23 <kanzure> jcorgan: pong
252 2015-12-02 20:58:24 <kanzure> sup
253 2015-12-02 20:58:52 <jcorgan> i sent a bitcoin-dev moderated message to the wrong address
254 2015-12-02 20:59:22 <jcorgan> can you PM me the proper one so they should up on the bitcoin-dev-moderated site?
255 2015-12-02 20:59:35 <jcorgan> *show up
256 2015-12-02 20:59:39 <kanzure> you mean the proper address?
257 2015-12-02 20:59:45 <jcorgan> yes
258 2015-12-02 21:00:12 <kanzure> done. imho this address should be the default value in the interface, nobody wants to memorize yet another email address.
259 2015-12-02 21:00:40 <jcorgan> right.  need rusty to change.
260 2015-12-02 21:00:41 <kanzure> in fact, this could maybe be fixed by making it a rule that all messages- whether moderated or not- get sent to the other mailing list.
261 2015-12-02 21:00:55 <kanzure> but this makes it slightly harder to see what is going on
262 2015-12-02 21:00:56 <jcorgan> nah, would lose its purpose
263 2015-12-02 21:00:59 <kanzure> yea ok
264 2015-12-02 21:01:11 <kanzure> all in favor of deleting all email clients and implementations?
265 2015-12-02 21:01:26 <jcorgan> you mean, like, *talk* to each other?
266 2015-12-02 21:01:52 <kanzure> no way, i just mean let's get some ACKs for collectively deciding to never use smtp again
267 2015-12-02 21:07:58 <aj> kanzure: MAIL TO: kanzure\r\nRCPT FROM: aj\r\nDATA\r\nSubject: vote\r\n\r\nNACK\r\n.\r\n