1 2015-11-17 01:21:19 <kenrestivo> is this a valid solution for doing a merchant store, or has it got issues? https://github.com/MyceliumGear/straight-server/blob/master/README.md
  2 2015-11-17 01:50:15 <Squidicc> holy crap. Core 11.2 fixes the windows derp on shutdown thing?! <3 :)
  3 2015-11-17 01:57:25 <phantomcircuit> Squidicc, yes wumpus got a windows machine from somewhere
  4 2015-11-17 01:57:38 <Squidicc> hahaha!
  5 2015-11-17 01:57:47 <Squidicc> awesome.
  6 2015-11-17 01:58:18 <sipa> Squidicc: there is still a known source of corruption (overly active anti virus software)
  7 2015-11-17 01:58:27 <sipa> which will be fixed in 0.12, but isn't in 0.11.2
  8 2015-11-17 01:58:37 <Squidicc> right. have exceptions for that already.
  9 2015-11-17 01:58:39 <Squidicc> ah cool.
 10 2015-11-17 01:58:48 <Squidicc> now, I just need to get rid of windows .__.
 11 2015-11-17 02:07:49 <kenrestivo> someone would run a security-related project on windows?
 12 2015-11-17 02:08:19 <kenrestivo> anyway, that straight-server seems like it's even less secure than running an SPV node. it uses blockchain.info.
 13 2015-11-17 02:08:22 <gmaxwell> kenrestivo: what usage of your computer isn't security related?
 14 2015-11-17 02:08:38 <kenrestivo> gmaxwell: cat pictures?
 15 2015-11-17 02:08:43 <kenrestivo> facebook?
 16 2015-11-17 02:08:45 <gmaxwell> you can get 0wned via them. :P
 17 2015-11-17 02:09:22 <gmaxwell> kenrestivo: I don't know anything about that "straight-server" thing but I normally direct people to baron: https://github.com/slickage/baron
 18 2015-11-17 02:11:22 <kenrestivo> thanks!
 19 2015-11-17 02:24:54 <kenrestivo> alas though, baron seems to want a full node. i have nether bandwidth nor hardware available to dedicate to that, so, oh well, mycelium gear
 20 2015-11-17 02:25:36 <kenrestivo> gmaxwell: freedom's just another word for, nothing left to 0wn
 21 2015-11-17 02:26:21 <sipa> kenrestivo: you don't have 50 kilobit/s or a CPU produced the past 5 years, or a 1 GB hard disk?
 22 2015-11-17 02:27:05 <sipa> and 1 GB RAM might be needed too
 23 2015-11-17 02:32:51 <midnightmagic> sipa: How is the AV corruption problem being fixed?
 24 2015-11-17 02:33:34 <sipa> midnightmagic: chainstate obfuscation
 25 2015-11-17 02:34:28 <midnightmagic> sipa: Ahh..  so the realtime write hooks that AV interrupts is not the source of corruption of which you speak when you say it'll be fixed.
 26 2015-11-17 02:38:01 <gmaxwell> actually damn, midnightmagic is right, there is one more source of corruption we haven't addressed.
 27 2015-11-17 02:38:25 <gmaxwell> there is AV software that when you close a file, it begins scanning it, and the idiot windows file locking behavior means that if we try to open it again we get an error.
 28 2015-11-17 02:38:47 <gmaxwell> And at least last time I looked we do close and reopen a lot.  But-- I think this was only an issue for a single AV package.
 29 2015-11-17 02:50:00 <midnightmagic> they're all over the place. some of them scan non-deterministically. sometimes open. sometimes hooked. will it block access? who knows?
 30 2015-11-17 02:50:55 <midnightmagic> "But I disabled realtime AV detection" Psyche! Its hooks are still in, it still runs, and the time in which it decides to let you do your thing is unpredictable. Maybe seconds. Maybe milliseconds. Who knows?
 31 2015-11-17 02:52:06 <midnightmagic> file-based locking mechanisms are therefore impossible to rely on on Windows machines that run AV. As far as I know, there is no solution except to switch to inter-process self-mitigated locking and hope nobody else (read: no other process) ever touches your files ever.
 32 2015-11-17 02:52:24 <sipa> actually
 33 2015-11-17 02:52:41 <sipa> we could switch the file env code to open files with sharing flags enabled
 34 2015-11-17 02:52:57 <sipa> i think the leveldb code opens for exclusive access
 35 2015-11-17 03:00:39 <gmaxwell> it was block files I've noticed this for before.
 36 2015-11-17 03:07:26 <midnightmagic> midnightmagic: MISMATCH
 37 2015-11-17 03:07:32 <midnightmagic> so many mismatch.
 38 2015-11-17 03:08:50 <midnightmagic> 68283ce35cc16c53a28c9211c09784cba4e445f3c9de73636bed7838dc21c5e6  bitcoin-0.10.4-linux32.tar.gz
 39 2015-11-17 03:13:18 <kenrestivo> sipa: wait what? the bitcoind docs say 100GB disk, and 500kbps upload bandwidth
 40 2015-11-17 03:13:45 <kenrestivo> 100gb disk is not so hard to find cheap, but the bandwidth isn't going to work in the situation i'm in.
 41 2015-11-17 03:35:27 <sipa> kenrestivo: you don't listen for incoming connections, you won't serve historical blocks to other nodes, and have less connections, but for your use case that doesn't matter
 42 2015-11-17 03:35:48 <sipa> kenrestivo: and since 0.11 you can run in pruned mode where you don't serve historicap blocks at all
 43 2015-11-17 03:36:16 <sipa> kenrestivo: don't *store* historical blocks at all
 44 2015-11-17 03:36:54 <sipa> kenrestivo: though it's only in 0.12 that you can run the wallet in combination with pruning, so for baron i guess it won't work yet
 45 2015-11-17 03:38:07 <sipa> kenrestivo: and in practice there will certainly be some overhead, but 50 kbit/s should ve enough in theory to stay in sync with the network
 46 2015-11-17 07:04:02 <kenrestivo> that's fantastic, thanks
 47 2015-11-17 10:26:01 <CodeShark> Comments appreciated: https://github.com/bitcoin/bips/pull/242
 48 2015-11-17 11:37:06 <btcdrak> CodeShark: you should mail the devlist also please
 49 2015-11-17 11:37:30 <btcdrak> gmaxwell: ping, just in case you miss it.
 50 2015-11-17 11:41:09 <CodeShark> oops, I emailed from the wrong account :(
 51 2015-11-17 11:41:33 <CodeShark> I emailed again - moderators, please throw away the one that is not from my subscribed account
 52 2015-11-17 12:57:25 <btcdrak> gmaxwell: this can be merged now I believe https://github.com/bitcoin/bips/pull/219
 53 2015-11-17 15:39:00 <jtimon> ping https://github.com/bitcoin/bitcoin/pull/6625#issuecomment-157403741
 54 2015-11-17 15:39:47 <jtimon> also, should I open anything else from #5970 or can I rebase one last time and close until 0.12 is forked?
 55 2015-11-17 15:43:29 <jtimon> well, not necessarily close since it's labeled as "DEPENDENT: ", but stop rebasing
 56 2015-11-17 15:49:15 <morcos> gmaxwell: phantomcircuit: I determined that the hashMerkleRoot mismatch happens with master too.  So I'm going to take a pause on continuing to debug it for now.  It looks like what happens is cgminer tries to submit a block that is missing a couple of transactions from the block template it received.
 57 2015-11-17 15:49:42 <sipa> morcos: context?
 58 2015-11-17 15:49:51 <morcos> It populates the # txs field with the number of txs it actually put in the block, but this is a few less than what gbt returned
 59 2015-11-17 15:50:44 <morcos> sipa: i was trying to mine with my new mining code and i got an invalid block.  but i was using the cgminer on an Antminer S7 to connect directly to bitcoind's getblocktemplate, which phantomcircuit said wasn't suprising it didn't work right
 60 2015-11-17 15:50:58 <morcos> But I just wanted to be 100% sure it wasn't something very strange from my new code
 61 2015-11-17 15:51:12 <sipa> morcos: well does it work when you do the same on old code? :)
 62 2015-11-17 15:51:16 <morcos> it appears to happen on about 1 out of every 20 blocks or so
 63 2015-11-17 15:51:37 <morcos> that was my first sentence...  happens with master too
 64 2015-11-17 15:51:39 <morcos> :)
 65 2015-11-17 15:52:12 <morcos> it didn't really seem possible that it was related, but i was just trying to be sure....
 66 2015-11-17 15:52:43 <sipa> morcos: ok, sorry, backwards parsing is hard :)
 67 2015-11-17 15:55:28 <phantomcircuit> morcos, i'll debug it if you can get me a pcap of the rpc calls
 68 2015-11-17 15:57:47 <morcos> phantomcircuit: ok thanks, but at this point i might not get around to that until december..  now i know its not related to my code!
 69 2015-11-17 16:53:33 <instagibbs> is there reasoning behind using nop2 for cltv instead of say nop1?
 70 2015-11-17 17:01:03 <nwilcox> instagibbs: I don't know the answer, but I'm also interested in this question.
 71 2015-11-17 17:01:12 <nwilcox> Basically I'm interested in every rationale question.
 72 2015-11-17 17:01:27 <sipa> I believe it was decided by a fair dice roll.
 73 2015-11-17 17:01:33 <nwilcox> It'd be nice if the CLTV bip had a rationale for the op choice.
 74 2015-11-17 17:02:12 <nwilcox> hehe... ok.
 75 2015-11-17 17:02:13 <nwilcox> That's also useful information.
 76 2015-11-17 17:03:13 <wumpus> bip17 says: "OP_NOP2 is used, so existing OP_EVAL (BIP 12) transactions in the block chain can still be redeemed."
 77 2015-11-17 17:03:37 <wumpus> eh wait, wrong bip
 78 2015-11-17 17:03:58 <wumpus> was already wondering :)
 79 2015-11-17 17:04:10 <sipa> well, that rationale applies to CLTV as well... apparently there are OP_NOP1 scripts in the chain
 80 2015-11-17 17:04:11 <instagibbs> lol
 81 2015-11-17 17:04:23 <sipa> (according to BIP17)
 82 2015-11-17 17:04:49 <nwilcox> Actually... having explicit rationales may also be a kind of red-herring, since consensus is whatever the code does, not what we humans intend for it to do. ;-)
 83 2015-11-17 17:05:08 <instagibbs> yeah I was more interested in the decision-making and planning side of things
 84 2015-11-17 17:06:06 <sipa> I think rationales are very useful, even if they're not part of the decision itself (as in: if the rationale was found to be incorrect afterwards, it won't change the accepted behaviour, but it does remain useful information)
 85 2015-11-17 17:07:55 <nwilcox> sipa: Yeah. They're also helpful for anticipating future decisions.
 86 2015-11-17 17:09:21 <nwilcox> eg, if that rationale wumpus quoted will affect every future reallocation of a NOP...
 87 2015-11-17 17:09:29 <nwilcox> -then bip12.
 88 2015-11-17 17:10:10 <instagibbs> These bips are before my time, so this is helpful
 89 2015-11-17 17:10:46 <nwilcox> Hm. bip12 is "withdrawn". Is this the same as saying: "We propose to never implement BIP12" or does it mean "At this time, we don't propose to implement BIP12" ?
 90 2015-11-17 17:11:48 <kanzure> rationales are also useful for when you go back and refactor and want to check whether you are solving the same thing that the previous person was trying to solve
 91 2015-11-17 17:11:52 <nwilcox> So if bip12 will never be re-considered, then it's claim on NOP1 no longer holds, and the rationale in bip17 is no longer relevant for new bips?
 92 2015-11-17 17:12:01 <nwilcox> kanzure: +1
 93 2015-11-17 17:12:02 <instagibbs> I think it means Luke-Jr abandons efforts to get it deployed?
 94 2015-11-17 17:13:20 <nwilcox> kanzure: Because I'm working on an alt, I'm wary of changes to the protocol violating explicit rationales, because even if it works in the short term, it may lead to future Bitcoin designs colliding with the alt protocol.
 95 2015-11-17 17:13:58 <nwilcox> This is kind of similar to refactoring, or also introducing protocol changes.
 96 2015-11-17 17:14:06 <Luke-Jr> instagibbs: you mean Gavin. I never proposed BIP 12 :P
 97 2015-11-17 17:14:17 <instagibbs> oh im looking at 17
 98 2015-11-17 17:14:33 <instagibbs> had my wires crossed
 99 2015-11-17 17:14:44 <sipa> nwilcox: well, if somebody would have created a BIP12 output at some point, and got it into the blockchain, reusing OP_NOP1 for something else would make these outputs very likely unspendable
100 2015-11-17 17:14:46 <Luke-Jr> I didn't catch that CLTV used the same NOP as BIP17, but oh well
101 2015-11-17 17:15:03 <Luke-Jr> perhaps someone should determine if any BIP17 UTXOs exist?
102 2015-11-17 17:15:22 <Luke-Jr> is there an easy way to scan the UTXO set for those?
103 2015-11-17 17:15:23 <sipa> on the other hand... somebody would have been pretty stupid to do so, as it would be an anyone-can-spend
104 2015-11-17 17:15:45 <instagibbs> so chances of unspent seems low...
105 2015-11-17 17:15:58 <sipa> Luke-Jr: add a few lines of code in the txdb.cpp code that computes the gettxoutsetinfo data
106 2015-11-17 17:47:43 <nwilcox> sipa: But since BIP12 was withdrawn, I assume long before it came close to a protocol change, does this mean current and future BIPs are free to reclaim NOP1?
107 2015-11-17 17:48:56 <sipa> nwilcox: under the assumption that someone created a BIP12 output (a bad idea, as it would be a NOP at the time), hoping it would be redeemable by a BIP12 script, no
108 2015-11-17 17:49:09 <sipa> but maybe that's an unreasonable assumption
109 2015-11-17 17:49:19 <sipa> s/unreasonable/incorrect/
110 2015-11-17 17:50:44 <nwilcox> sipa: I see.
111 2015-11-17 17:51:19 <nwilcox> I guess it's quite reasonable not to assume anything about the users while also striving not to surprise them with unexpected semantic changes to their property.
112 2015-11-17 17:52:20 <sipa> all unassigned NOP opcodes are currently treated as non-standard by bitcoin core (even when the isstandard check is disabled, as it's done on a different level in the code)
113 2015-11-17 18:41:22 <aschildbach> Is there anything going on on mainnet that could blocks show up as version 536870919 ? (2^29+7)
114 2015-11-17 18:45:51 <aj> aschildbach: that's bip101/bitcoin xt
115 2015-11-17 18:47:11 <aschildbach> aj: So that's a block which includes a vote for bip101?
116 2015-11-17 18:48:48 <aj> aschildbach: yeah
117 2015-11-17 18:49:08 <aschildbach> Ok thanks
118 2015-11-17 18:50:27 <aschildbach> Is this usage of high bits somehow standardized? I mean is there a bitmask that should be used to determine the actual block version?
119 2015-11-17 18:51:35 <sipa> aschildbach: it uses something inspired by a pre-final version of BIP9 (which imho is misguided, it's a hardfork, miners don't get to vote for what code the rest of the network should run)
120 2015-11-17 19:18:55 <Luke-Jr> sipa: BIP12 OP_EVAL is not a NOP
121 2015-11-17 19:19:41 <sipa> oh
122 2015-11-17 19:19:55 <Luke-Jr> well, the opcode itself is
123 2015-11-17 19:20:00 <Luke-Jr> but not the expected usage
124 2015-11-17 19:20:31 <sipa> right, that's all that matters
125 2015-11-17 19:20:36 <Luke-Jr> not quite
126 2015-11-17 19:20:45 <Luke-Jr> with BIP17, you could literally just spend any such outputs
127 2015-11-17 19:20:52 <Luke-Jr> BIP12, you'd need to know the redeemScript
128 2015-11-17 19:21:11 <Luke-Jr> ie, we can just cleanup any BIP17 UTXO entries, but not BIP12
129 2015-11-17 19:22:40 <sipa> my expectation is that there are none of either :)
130 2015-11-17 19:24:30 <Luke-Jr> hopefully. I'll try to check that in a bit
131 2015-11-17 19:25:22 <gmaxwell> All this speculation, from what I recall for CLTV we checked the blockchain and found a bunch of OP_NOP1 in ramdom places (I don't think BIP12) and thought it better to use the next one.
132 2015-11-17 19:28:07 <Luke-Jr> gmaxwell: well, OP_NOP2 (BIP17) is definitely in the blockchain, just (hopefully) not the UTXO set
133 2015-11-17 19:29:36 <nwilcox> Does it make sense to have a "no, really, this is really a NOP forever" opcode?
134 2015-11-17 19:30:16 <nwilcox> -also, doesn't the soft-fork workflow alter a nop at some blockheight, so older txns aren't affected anyway?
135 2015-11-17 19:30:44 <nwilcox> Or is this a concern about existing non-upgraded infrastructure that expects to continue emitting txns with those opcodes?
136 2015-11-17 19:31:04 <Luke-Jr> nwilcox: UTXOs aren't necessarily aged right now
137 2015-11-17 19:31:20 <nwilcox> Oh, right!  Silly me.
138 2015-11-17 19:31:59 <nwilcox> So a conditional on block height in the Script evaluator would be ...  unpalatable, I suppose.
139 2015-11-17 19:35:56 <gmaxwell> nwilcox: not there is no reason to have a NOP forever; but there is no particular need to use them in order.
140 2015-11-17 19:38:50 <rusty> Confused about opt-in RBF.  Seems like bitcoin-core v11.0 is already "opting in" which doesn't seem like opt-in in practice.  But I'm still pre-coffee, so maybe I misunderstand.
141 2015-11-17 19:41:34 <nwilcox> gmaxwell: I'm not convinced there's no reason...
142 2015-11-17 19:42:01 <nwilcox> Suppose there's some need for NOPs out there, and there's an agreement that NOP_FOREVER will not be reallocated.
143 2015-11-17 19:42:09 <nwilcox> Doesn't this make the decision about which NOPs to reallocate for soft-forks simpler?
144 2015-11-17 19:43:06 <gmaxwell> nwilcox: I'm pretty convinced there is no reason.
145 2015-11-17 19:43:28 <gmaxwell> I think it's likely that we'd leave NOP_1 alone until we run out, however.
146 2015-11-17 20:06:13 <instagibbs> rusty, not totally sure what you're referring to but the "opt-in" means you mark a transaction using a field. So if you see the field marked, beware 0-conf
147 2015-11-17 20:08:34 <gmaxwell> rusty: bitcoin core uses max-int -1 which is opt-out.
148 2015-11-17 20:09:02 <gmaxwell> rusty: also bitcoin core is, I believe, the only implementation that produces locked transactions by default.
149 2015-11-17 20:25:36 <lclc> petertodd: Starting with 0.12 wallets will have to check for nSequence < 0xFFFFFFFF-1 on any inputs to and display to the user that this transaction might get replaced?
150 2015-11-17 20:25:53 <lclc> If it's not confirmed
151 2015-11-17 20:26:22 <sipa> lclc: transactions can always be replaced...
152 2015-11-17 20:27:17 <sipa> though if opt-in RBF gets widely adopted then that test will indeed mean higher chance for replacement
153 2015-11-17 20:27:38 <lclc> sipa: Yes but so far nodes didn't accept any new transactions using the same inputs, so not that big of a risk to accept 0 conf
154 2015-11-17 20:28:04 <sipa> lclc: they have in the past, in cases where it was profitable enough...
155 2015-11-17 20:28:12 <sipa> s/nodes/miners/
156 2015-11-17 20:31:28 <rusty> gmaxwell: hmm, "Bitcoin Core has been producing transactions with non-maxint nSequence since v0.11.0 to discourage fee sniping" + "A transaction is considered to have opted into full-RBF semantics if nSequence < 0xFFFFFFFF-1" => 0.11.0 is opting into RBF?
157 2015-11-17 20:33:00 <sipa> rusty: non-maxint nLockTime for fee sniping
158 2015-11-17 20:33:14 <sipa> and nSequence=0xFFFFFFFE
159 2015-11-17 20:33:16 <Luke-Jr> hmm, do we actually need OpenSSL in master anymore?
160 2015-11-17 20:33:20 <sipa> Luke-Jr: PRNG
161 2015-11-17 20:34:05 <rusty> sipa: I was quoting from petertodd's post... hence my confusion.  You're right, of course.
162 2015-11-17 20:34:33 <sipa> so yes, non-maxint nSequence... but 0xFFFFFFFE
163 2015-11-17 20:34:52 <sipa> rusty: however, BIP68 transactions will opt-in
164 2015-11-17 20:34:56 <rusty> sipa: Hence the < 0xFFFFFFFF *-1*, makes sense.  Would be nice to be spelled out for people like me.
165 2015-11-17 20:39:11 <sturles> Starting with 0.12 wallets need to warn users if the fee on an incoming transaction is too low as well, due to setting priorityspace to 0.  In the past transactions which weren't double spent would eventually confirm as the priority increased.  This won't happen in 0.12.  As of the current version, Bitcoin Core will even reuse the input if the transaction is removed from it's mempool.
166 2015-11-17 20:39:51 <sturles> So you can't accept unconfirmed transactions even from users you trust, or for small amounts like payment for a beer.
167 2015-11-17 20:40:01 <sipa> sturles: estimatepriority will detect that
168 2015-11-17 20:40:04 <sturles> Unless the fee is high enoug.
169 2015-11-17 20:40:22 <sturles> sipa: No, only the fee matters in current git.
170 2015-11-17 20:40:33 <sturles> There are several open issues about it.
171 2015-11-17 20:40:37 <gmaxwell> lclc: You're expressing an incorrect understanding of the network; both pratically and theoretically. It is _trivial_ to replace transactions now, and you can easily get 50% success rate with no great work.
172 2015-11-17 20:40:54 <sturles> Mempool trimming will remove local wallet transactions where the fee is deemed too low, and reuse the inputs.
173 2015-11-17 20:41:03 <sipa> sturles: sure, there are bugs in the current bitcoin core wallet handling, which needs to be fixed
174 2015-11-17 20:41:21 <sturles> And in mempool handling.
175 2015-11-17 20:41:22 <gmaxwell> lclc: All you do is concurrently announce your double spends send one version to half the nodes another version to the other half (though this won't get you 50%, you have to identify a particular mining pool to get 50%)
176 2015-11-17 20:41:23 <sipa> sturles: but estimatefee will detect the fact that miner's priority space is shrinking (or not)
177 2015-11-17 20:41:32 <sipa> sturles: eh, estimatepriority
178 2015-11-17 20:41:48 <gmaxwell> lclc: the statement of "nodes didn't accept any new" implies synchronicity, as if nodes could see things in the same order, they don't.
179 2015-11-17 20:41:57 <sturles> sipa: Not reliably is the memorypool is trimmed.
180 2015-11-17 20:42:21 <sipa> sturles: i don't understand how that matters
181 2015-11-17 20:42:43 <sturles> Then why did you mention estimatepriority at all?
182 2015-11-17 20:42:49 <sipa> sturles: if we get to a situation where no transactions are accepted based on priority anymore, estimatepriority will fail to find a good number
183 2015-11-17 20:42:58 <sturles> I know estimatepriority don't matter.
184 2015-11-17 20:43:30 <sturles> Yes, and if you sent the transaction two seconds before your mempool was trimmed?
185 2015-11-17 20:43:50 <sipa> then you'll rebroadcast?
186 2015-11-17 20:44:03 <Luke-Jr> sturles: 0.12 does not remove priority
187 2015-11-17 20:44:03 <sturles> Bitcoin Core won't.
188 2015-11-17 20:44:19 <Luke-Jr> unless something I NACK'd got merged..
189 2015-11-17 20:44:25 <sipa> sturles: it will try to add it back to the mempool, no?
190 2015-11-17 20:44:25 <sturles> Luke-Jr: No, but it will introduce bugs related to priority.
191 2015-11-17 20:44:30 <Luke-Jr> sturles: ?
192 2015-11-17 20:44:41 <sturles> See my comments in https://github.com/bitcoin/bitcoin/pull/6992
193 2015-11-17 20:44:56 <Luke-Jr> sturles: pull != in 0.12
194 2015-11-17 20:45:05 <sturles> ?
195 2015-11-17 20:45:06 <Luke-Jr> oh
196 2015-11-17 20:45:17 <sturles> It was closed.  Not pulled.  It is needed.
197 2015-11-17 20:45:55 <sturles> Unless the current mempool trimming code has been redone elsewhere?
198 2015-11-17 20:46:34 <sipa> sturles: ah, you're talking about the fact that it won't get into your own mempool
199 2015-11-17 20:46:46 <sturles> Right now it doesn't care about priority, and when your own transactions are gone they won't be rebroadcasted until you restart Bitcoin Core.  And then only if the inputs haven't been spent in new transactions.
200 2015-11-17 20:47:10 <sturles> sipa: No, the fact that they may be removed from my own mempool.
201 2015-11-17 20:47:39 <sturles> Making them not get in there is trivial.
202 2015-11-17 20:48:00 <Luke-Jr> sturles: as long as they get to miners, they should eventually confirm. your wallet should continue rebroadcasting until that time, as well. that it doesn't right now is a clear bug that needs fixing.
203 2015-11-17 20:48:03 <gmaxwell> sturles: that they get removed isn't important.
204 2015-11-17 20:48:09 <sturles> Or preventing Bitcoin Core from making transactions that won't make the memory pool.
205 2015-11-17 20:48:22 <Luke-Jr> gmaxwell: well, right now, removing your own transactions from the mempool breaks things badly
206 2015-11-17 20:48:27 <sipa> sturles: that last thing should definitely happen
207 2015-11-17 20:48:49 <gmaxwell> sturles: there is a pull req from morcos that fixes the fee logic to not do an idiotic thing with respec to that.
208 2015-11-17 20:48:59 <sturles> No, it doens't fix it.
209 2015-11-17 20:49:48 <Luke-Jr> gmaxwell: the bug isn't in fee logic
210 2015-11-17 20:50:18 <gmaxwell> sturles: It will prevent the wallet from creating a transaction that cannot make it into the mempool.
211 2015-11-17 20:50:26 <sturles> E.g. if I have a very old 10 BTC UTXO which I spend to sent 9.99999 BTC.  The entire input is spent, and the rest is fee.  Since the transaction has high priority, it will be sent with no extra inputs to make the fee requirement.  This transaction may easily get evicted on the nest mempool trimming event.  Even seconds after I made it.'
212 2015-11-17 20:50:39 <Luke-Jr> gmaxwell: the mempool trimming logic can change its rules post-hoc
213 2015-11-17 20:50:56 <gmaxwell> sturles: yes, so? you'll have already broadcast it at that point.
214 2015-11-17 20:50:56 <sturles> Exactly.
215 2015-11-17 20:51:06 <morcos> To be clear what #6134 does is prevent you from accidentally placing a tx that won't be accepted into your mempool in the first place (if you use the smart fee or priority estimation)
216 2015-11-17 20:51:07 <Luke-Jr> …
217 2015-11-17 20:51:08 <gmaxwell> it doesn't matter if its in your mempool at that point.
218 2015-11-17 20:51:20 <Luke-Jr> gmaxwell: right now, your wallet will DOUBLE SPEND transactions not in the mempool
219 2015-11-17 20:51:27 <sipa> Luke-Jr: that's a separate bug
220 2015-11-17 20:51:36 <morcos> It does not address the problem of what happens if a prviously accepted tx is later evicted or you use a fixed fee to create your tx
221 2015-11-17 20:51:37 <Luke-Jr> and won't rebroadcast them
222 2015-11-17 20:51:37 <sturles> gmaxwell: Why does Bitcoin Core normally rebroadcast transactions every half hour?
223 2015-11-17 20:51:49 <gmaxwell> Luke-Jr: thats just busted in any case.
224 2015-11-17 20:51:52 <Luke-Jr> yes
225 2015-11-17 20:52:22 <sipa> that absolutely needs to get fixed; the wallet shouldn't consider a transaction output spendable again because a spending wallet tx was evicted from the mempool
226 2015-11-17 20:52:32 <sturles> What if the transaction only hits other nodes with heavily trimmed mempools at the first attempt?
227 2015-11-17 20:52:44 <sipa> and it also absolutely needs to be fixed that the wallet doesn't create transactions it won't accept itself
228 2015-11-17 20:52:45 <gmaxwell> sturles: it's an effort to get the transaction to go if the initial broadcast failed but it's kind of broken and should probably go away; but replacing it is somewhat complex so it hasn't been done yet.
229 2015-11-17 20:52:50 <phantomcircuit> sipa, it's a failure of the "conflicted" logic
230 2015-11-17 20:52:54 <morcos> sipa: i agree it needs to be fixed, but i don't think its true that it shouldn't consider it spendable again
231 2015-11-17 20:52:55 <sipa> phantomcircuit: i know
232 2015-11-17 20:53:05 <morcos> it probably should.. you probably need to RBF it
233 2015-11-17 20:53:36 <Luke-Jr> gmaxwell: are you saying we shouldn't rebroadcast wtx?
234 2015-11-17 20:53:56 <sturles> Without the memory trimming, or with pull request #6992, this won't happen.
235 2015-11-17 20:54:00 <gmaxwell> Luke-Jr: we shouldn't-- in the longrun we should mempool sync new peers.
236 2015-11-17 20:54:00 <Luke-Jr> morcos: RBF logic is independent from making it spendable IMO
237 2015-11-17 20:54:23 <gmaxwell> sturles: there has _never_ been a guarentee that your peers would take your transactions.
238 2015-11-17 20:54:25 <sturles> And IMHO all priority bugs should be fixed as long as Bitcoin Core has the concept of priority.
239 2015-11-17 20:54:31 <Luke-Jr> gmaxwell: if wallets don't rebroadcast, then transactions really will never confirm sometimes..
240 2015-11-17 20:54:32 <morcos> i think before we can properly fix the wallet to deal with evicted txs or otherwise missing txs we should add RBF, and then make the UI consistent with all of these things.  of course if you didn't opt-in to RBF, then that is a different problem...
241 2015-11-17 20:54:34 <gmaxwell> And none could ever be provided.
242 2015-11-17 20:54:52 <sipa> Luke-Jr: mempool syncing fixes it by having others "relay" for you
243 2015-11-17 20:54:58 <sturles> gmaxwell: Of course not, but at least Bitcoin Core used to make some effort in that regard.
244 2015-11-17 20:55:15 <gmaxwell> Luke-Jr: oh gosh, please. Stop.  Let me make this clear:  The world does not end if everything doesn't work exactly as it did yesturday.
245 2015-11-17 20:55:21 <Luke-Jr> sturles: I'm still planning to rework the mempool that will hopefully fix the priority ignorance in it, but it won't make 0.12 likely. In the meantime, this shouldn't be a huge issue.
246 2015-11-17 20:55:34 <Luke-Jr> sipa: not if everyone trims it
247 2015-11-17 20:55:41 <sturles> The bug has hit me many times with the current git.
248 2015-11-17 20:55:44 <gmaxwell> Luke-Jr: I was trying to continue and say that what it should do is mempool sync on connect with new peers, not rebroadcasts, but the channel is a flood of text.
249 2015-11-17 20:55:46 <sipa> Luke-Jr: if everyone trims it, it won't get into a block
250 2015-11-17 20:56:42 <phantomcircuit> gmaxwell, if we're going to do that we need to significantly improve the mechanism
251 2015-11-17 20:56:50 <gmaxwell> phantomcircuit: yes yes, not for today.
252 2015-11-17 20:56:53 <phantomcircuit> simply issuing a mempool command is not sufficient
253 2015-11-17 20:57:00 <gmaxwell> lol, I know.
254 2015-11-17 20:57:04 <sturles> sipa: Correct.  And this is new behaviour.
255 2015-11-17 20:57:22 <phantomcircuit> gmaxwell, yes... that ends comically
256 2015-11-17 20:57:33 <gmaxwell> the issue sturles is pointing out that priority is unusable with the trimming immediately kills priority behavior.
257 2015-11-17 20:57:39 <sipa> ^
258 2015-11-17 20:57:47 <sturles> Correct.
259 2015-11-17 20:58:00 <sturles> So either fix mempool trimming or remove priority entirely.
260 2015-11-17 20:58:10 <sipa> not only does it prevent you from using it, it also prevents the estimatepriority from computing things correctly
261 2015-11-17 20:58:11 <sturles> The current mix is broken.
262 2015-11-17 20:58:21 <gmaxwell> sturles: okay, agreed.
263 2015-11-17 20:59:58 <sturles> Thanks!  This has been a very frustrating testing period with 0.11.99.  Pull #6992 has fixed it for me.  And as a bonus it is much better at keeping transactions which will get into the next block.
264 2015-11-17 21:01:08 <sipa> I think #6992 may temporarily be necessary.
265 2015-11-17 21:02:06 <sipa> the alternative is saying that everyone running 0.12 will not be able to do priority transactions at all anymore
266 2015-11-17 21:03:47 <sturles> Unfortunately it will mess upp broadcasting of those transactions from older nodes when they are sent to 0.12 nodes with trimmed mempools.
267 2015-11-17 21:05:07 <morcos> sipa: thats not exactly the alternative.  with a large enough mempool or a newly started mempool, you'll still be able to place priority txs
268 2015-11-17 21:05:27 <sipa> morcos: that's not very reliable...
269 2015-11-17 21:05:31 <morcos> ok, maybe not with newly started b/c of bad estimates
270 2015-11-17 21:06:05 <morcos> the problem of your txs getting evicted is irrelevant to this priority issue
271 2015-11-17 21:06:13 <morcos> not irrelevant, wrong word, independent
272 2015-11-17 21:06:28 <morcos> it'll happen with fee txs, old txs, etc... it has to be fixed period.
273 2015-11-17 21:06:39 <sipa> i think that we need to deal with evicted wallet transactions differently (just not considering conflicted until an actual conflict is in the chain would be one way...)
274 2015-11-17 21:06:59 <morcos> so i thought we decided in the IRC meeting that sending priority txs was no longer supported
275 2015-11-17 21:07:12 <morcos> you are right that it will not be reliable or barely work at all with 0.12 code
276 2015-11-17 21:07:19 <morcos> but i thought that was the whole point
277 2015-11-17 21:07:22 <sipa> that's not what i understood, but if that's the case, we need to rip out the estimatepriority code now
278 2015-11-17 21:07:43 <morcos> why, it'll essentially do the right thing?
279 2015-11-17 21:07:48 <gmaxwell> if it's not reliable we need to turn if off in the wallet.   But that also wasn't my take from the meeting; but my perspective may be biased because I don't agree with removing it.
280 2015-11-17 21:07:54 <morcos> its defaulted to off
281 2015-11-17 21:08:19 <sipa> i thought the idea was to deprecate it _for mining_, but leave it supported otherwise
282 2015-11-17 21:08:28 <sipa> and then see in 0.13 about completely removing it
283 2015-11-17 21:08:35 <Luke-Jr> morcos: mining priority txs is still supported, and for that to work it needs to at least relay correctly in normal conditions
284 2015-11-17 21:08:52 <Luke-Jr> definitely do not want to remove it for mining.
285 2015-11-17 21:09:02 <morcos> sipa: the first part of the IRC meeting was about #6992 and we decided to close it
286 2015-11-17 21:09:14 <morcos> without #6992 or similar, we can not support priority
287 2015-11-17 21:09:16 <sipa> morcos: i was surprised to see it closed
288 2015-11-17 21:09:21 <sturles> morcos: If it is only defaulted to off, it should still work for those who choose to turn it on.  Either that or remove it entirely.
289 2015-11-17 21:09:36 <morcos> sturles: release notes were going to say priority was deprecated
290 2015-11-17 21:10:11 <sipa> morcos: i thought the plan was to set the default priority area to 0, and make approximations to simplify priority calculations (like #7008), but otherwise keep it work
291 2015-11-17 21:10:14 <morcos> sipa: sdaftuar and i both interpreted the meeting (and combined wiht the comments on the pull) as deciding that this was introducing too much complexity to continue to support priority
292 2015-11-17 21:10:55 <Luke-Jr> morcos: remember that nothing is ever to be decided at meetings; in any case, mining will not deprecate/remove priority.
293 2015-11-17 21:11:22 <morcos> sigh...
294 2015-11-17 21:11:50 <morcos> i think we should all go reread what was said in the meeting rather than hash this out over and over again
295 2015-11-17 21:11:55 <gmaxwell> sipa's view reflects my understanding.
296 2015-11-17 21:12:08 <sipa> morcos: if we deprecate it both for mining and wallets, it should just be ripped out
297 2015-11-17 21:12:25 <Lightsword_> is priority going to be useful in 0.12 without the same type of massive GBT latency issues in 0.11.2?
298 2015-11-17 21:12:26 <morcos> sipa: the reason not to rip it out entirely is to not make major code changes
299 2015-11-17 21:12:30 <morcos> this was also discussed at the meeting
300 2015-11-17 21:12:31 <sturles> I read through it, and as far as I can see only one person supported removing priority support entirely in 0.12.
301 2015-11-17 21:12:41 <morcos> Lightsword_: remains to be seen
302 2015-11-17 21:12:55 <sturles> The same number of people were completely against removing priority.
303 2015-11-17 21:13:07 <Luke-Jr> I will strongly discourage miners to upgrade to 0.12 if priority is removed from mining.
304 2015-11-17 21:13:11 <sipa> morcos: i thought it was to give time for people to transition to not having priority
305 2015-11-17 21:13:15 <sturles> The rest had various opinions about replacing priority with something else.
306 2015-11-17 21:13:23 <gmaxwell> Lightsword_: priority without the constant updating everytime a block shows up is not fundimentally incompatible with performance.
307 2015-11-17 21:13:28 <morcos> sipa: yes thats why i thought it would still be supported in mining
308 2015-11-17 21:13:38 <Lightsword_> my thoughts are it may not be worth maintaining since pools have already been phasing it out
309 2015-11-17 21:13:46 <morcos> so that the usefulness of priority just didn't disappear entirely from the network instantly
310 2015-11-17 21:13:55 <Luke-Jr> Lightsword_: pools should not be phrasing it out.
311 2015-11-17 21:14:04 <sturles> Lightsword_: Far from all pools.
312 2015-11-17 21:14:05 <sipa> morcos: but the current plan is something where it is unusuable in the wallet (and nearly useless for mining)... if that's the case, it should go
313 2015-11-17 21:14:13 <morcos> but it seemed to me, that if we made a decision to  deprecate it, why add a lot of complication to continue to support sending priority txs for a few 0.12 users
314 2015-11-17 21:14:36 <sipa> morcos: the problem is that the mempool is used both for the wallet logic (currently) and for the mining logic
315 2015-11-17 21:14:45 <morcos> sipa: the current plan is essentially that it is unusable for 0.12 users, but still perfectly usable for mining even if mining on 0.12
316 2015-11-17 21:15:10 <sipa> morcos: that's fine by me... but then estimatefee should go, as it can't return any usable result anymore
317 2015-11-17 21:15:11 <Luke-Jr> morcos: when the wallet supports RBF, priority should be reintroduced there
318 2015-11-17 21:15:11 <morcos> sipa: ah, well i guess in my mind no miner is going to have a limited mempool
319 2015-11-17 21:15:14 <Lightsword_> Luke-Jr, currently it doesn’t seem possible to run it with unpatched 0.11.2 without massive GBT latency
320 2015-11-17 21:15:56 <Lightsword_> which seems to be because those transactions aren’t getting mined fast enough
321 2015-11-17 21:15:57 <morcos> well i don't know why i'm arguing about this. i was in favor of 6992.
322 2015-11-17 21:16:00 <Luke-Jr> Lightsword_: but GBT latency is not a problem, so that's okay, even if not ideal.
323 2015-11-17 21:16:06 <morcos> but i would like to have a clear plan
324 2015-11-17 21:16:28 <Lightsword_> Luke-Jr, it is for probably half of all mining pools AFAIK
325 2015-11-17 21:16:42 <sipa> morcos: i thought the meeting was about "deprecate it for mining, but keep it working to the extent possible otherwise", which is what I thought I agreed to :)
326 2015-11-17 21:17:08 <Luke-Jr> Lightsword_: well, they should fix that. GBT will never be zero-latency, because that's impossible.
327 2015-11-17 21:17:14 <morcos> sipa: i should go reread myself, but i think we explicitly decided not to support maintaining a priority space in the mempool.
328 2015-11-17 21:17:21 <Lyth0s_> x-post from #bitcoin
329 2015-11-17 21:17:23 <Lyth0s_> I think most dev's would agree that the block size limit needs to be increased by some amount (there are a few exceptions). Let's just make it simple and increase the block size limit to 2MB now and let it double every 2 years for a total of 5 cycles up to a total of 32MB per block down the road. This will buy us some time to figure out what the perfect algorithm will be for block size limits and in the mean time we can get to
330 2015-11-17 21:17:23 <Lyth0s_> the more important issue that will affect us all very soon and should be a core principle of Bitcoin.....fungibility. We need new and integrated ways to keep/make Bitcoin fungible otherwise somewhere down the road, and very soon, there will be companies and governments blacklisting coins and users and suddenly Bitcoin falls to pieces much faster than waiting a few hours for a transaction to go through (which is still faster
331 2015-11-17 21:17:24 <Lightsword_> Luke-Jr, if filtered it can be pretty darn close
332 2015-11-17 21:17:24 <Lyth0s_> than ACH transfers anyways).
333 2015-11-17 21:18:27 <Luke-Jr> Lightsword_: tired of discussing lazy pools' negligence. is there anything new to add to that discussion?
334 2015-11-17 21:18:51 <gmaxwell> Luke-Jr: not productive.
335 2015-11-17 21:18:59 <morcos> sipa: i think we should just be consistent.   1) we need to deal with evicted txs.  2) we need to clearly communicate what level of support priority has
336 2015-11-17 21:19:16 <sipa> Luke-Jr: i believe that CNB's speed is contributing to mining centralization... whether or not that's miner's fault, it's everyone's problem
337 2015-11-17 21:19:21 <Luke-Jr> gmaxwell: having the same discussion over and over with nothing new added, is also not productive.
338 2015-11-17 21:19:29 <Lightsword_> Luke-Jr, from my testing there is little difference between pools skipping GBT and those that build directly off of it
339 2015-11-17 21:19:34 <gmaxwell> The fact of the matter is that ckpool is dependant on gbt latency; it won't change, instead its users will either refuse upgrades or continue to apply poorly constructed and risky patches that may cause harm to the network.
340 2015-11-17 21:19:35 <morcos> sipa: with 6992, i think we could have mostly full support (its only starting priority, no aging) .   Mining could have full support (or only originally in chain aging)
341 2015-11-17 21:19:48 <gmaxwell> Luke-Jr: minimizing gbt latency is an important goal which we should satisify.
342 2015-11-17 21:19:54 <Lightsword_> Luke-Jr, the only thing that I see as making a difference is fully skipping validation
343 2015-11-17 21:20:00 <morcos> sipa: without 6992, we just need to note it is not supported.  We can remove estimatepriority, but I think it'll basically do the right thing.
344 2015-11-17 21:20:03 <Luke-Jr> gmaxwell: yes, that's been worked on and is being worked on
345 2015-11-17 21:20:16 <gmaxwell> Luke-Jr: then don't assume its okay for it to be high.
346 2015-11-17 21:20:53 <gmaxwell> As far as what sturles is bringing up, a week ago I did not realize that the current mempool behavior broke priority as bad as it did even ignoring the wallet not checking the current rule (which is a simple fix)
347 2015-11-17 21:21:03 <gmaxwell> So some discussion on this is needed.
348 2015-11-17 21:21:47 <Lightsword_> Luke-Jr, you have to look at this from a practical point of view, a lot mining pool operators are unlikely to implement a complex p2p component into their stratum servers for a very minor latency improvement
349 2015-11-17 21:21:48 <sipa> morcos: (no #6992, only mining priority if you have a huge mempool, and no wallet support for priority) is ne consistent choice
350 2015-11-17 21:21:59 <Luke-Jr> gmaxwell: high GBT latency is Lightsword_'s rationale for disabling priority in his pool; I'm just saying that is entirely irrelevant to the discussion of priority itself, since the motivation in disabling it in 0.11 is both gone and irrelevant for 0.12
351 2015-11-17 21:22:23 <gmaxwell> Luke-Jr: okay. true, there isn't a fundimental incompatiblity.
352 2015-11-17 21:22:23 <Luke-Jr> Lightsword_: blocknotify is sufficient I think?
353 2015-11-17 21:23:28 <Lightsword_> Luke-Jr, I’m not sure, I thought there wasn’t enough info there, but I haven’t looked into it in detail
354 2015-11-17 21:23:40 <morcos> sipa: sure i think the reason to not remove support from the wallet entirely was it was a less invasive change this late in the process.   but maybe if its only the wallet we remove it from thats reasonable.
355 2015-11-17 21:24:18 <Luke-Jr> Lightsword_: also, neither implementing p2p nor deploying Eloipool are impractical. in any case, tangent to the topic
356 2015-11-17 21:25:21 <Lightsword_> Luke-Jr, most pools run custom stratum server software, just saying deploy a different software stack isn’t really practical
357 2015-11-17 21:26:55 <Lightsword_> anyways I still don’t see any evidence of it making any significant difference, eloipool based pools seem to be fairly average in my benchmarking
358 2015-11-17 21:27:31 <Luke-Jr> except with higher GBT latency, I presume
359 2015-11-17 21:30:08 <Luke-Jr> I wonder if #6978 (qt -fPIC stuff) should be backported?
360 2015-11-17 21:30:30 <Lightsword_> depends I guess on how they are filtering their mempools if it has much of an effect
361 2015-11-17 21:32:44 <wumpus> Luke-Jr: the -fPIC stuff should be backported to versions that support qt5
362 2015-11-17 21:33:04 <Luke-Jr> wumpus: isn't that all versions currently maintained?
363 2015-11-17 21:33:20 <wumpus> Luke-Jr: I think so, yes, 0.10 and 0.11
364 2015-11-17 21:33:28 <Luke-Jr> k
365 2015-11-17 21:33:49 <wumpus> Luke-Jr: it doesn't matter for the binary releases but for people building from source it's useful
366 2015-11-17 21:34:09 <Luke-Jr> just checking before I do it, since it looks like it needs special care
367 2015-11-17 21:34:42 <wumpus> yes it may be non-trivial
368 2015-11-17 21:39:12 <Luke-Jr> ok, doesn't look that bad after all
369 2015-11-17 21:56:01 <kenrestivo> what's the option to bitcoind to make it just listen, not serve blocks?
370 2015-11-17 22:01:04 <wumpus> -nolisten will usually do the job (most clients won't request blocks from incoming connections), to 100% disable serving of blocks currently the options are to enable pruning, or to use a low upload limit (only master)
371 2015-11-17 22:14:21 <kenrestivo> pruning is on, thanks, sounds like that should do it.
372 2015-11-17 22:14:53 <kenrestivo> running 0.11 right now, will probably build and run git HEAD later, and see if i can get that working with baron.
373 2015-11-17 22:36:23 <petertodd> lclc: the success rate of standard wallets detecting conditions where transactions could be doublespent with high probability is abysmally low
374 2015-11-17 23:49:55 <jtoomim> can anyone help me understand the first line of DisconnectBlock?
375 2015-11-17 23:49:57 <jtoomim> assert(pindex->GetBlockHash() == view.GetBestBlock());
376 2015-11-17 23:50:15 <jtoomim> sipa, git blame marks you as the last one to modify that line
377 2015-11-17 23:50:58 <sipa> jtoomim: you can only disconnect the block that is the last one connected
378 2015-11-17 23:51:25 <sipa> so the tip known by the utxo set view must be exactly the block whose effects you're trying to undo
379 2015-11-17 23:52:00 <jtoomim> this is failing for us on a massive reorg on testnet right now
380 2015-11-17 23:52:52 <sipa> that means something is wrong :)
381 2015-11-17 23:53:26 <jtoomim> yes, i figured as much
382 2015-11-17 23:53:39 <jtoomim> i'm curious what
383 2015-11-17 23:53:49 <sipa> hard to say without more information
384 2015-11-17 23:54:03 <jtoomim> any hypotheses?
385 2015-11-17 23:54:41 <jtoomim> this is not happening on all of my nodes, but it's happening on a few of them
386 2015-11-17 23:55:21 <gmaxwell> no problem on my testnet nodes.
387 2015-11-17 23:57:02 <jtoomim> gmaxwell i expect yours didn't just go through a 400 block reorg
388 2015-11-17 23:57:44 <gmaxwell> well we certantly test reorgs bigger than that normally.
389 2015-11-17 23:58:33 <gmaxwell> but sounds interesting, let us know if you find out why its doing that.
390 2015-11-17 23:58:54 <jtoomim> oh, i have a hypothesis
391 2015-11-17 23:59:02 <jtoomim> i added cltv support recently
392 2015-11-17 23:59:11 <jtoomim> (it's not a great hypothesis)
393 2015-11-17 23:59:27 <jtoomim> i added cltv support and low-S to this branch after most of the chain had been built
394 2015-11-17 23:59:45 <jtoomim> perhaps there's a difference in how the chain was built and how it's being broken down that's causing trouble