1 2015-09-28 00:32:57 <ploopkazoo> anyone have any idea what could possibly be going wrong here? https://bpaste.net/raw/075e67a26b73
  2 2015-09-28 00:41:43 <ploopkazoo> oh goddammit
  3 2015-09-28 00:42:00 <ploopkazoo> I was connected to the wrong node and it doesn't have wallet support compiled in
  4 2015-09-28 00:42:02 <ploopkazoo> never mind
  5 2015-09-28 00:42:04 <ploopkazoo> :|
  6 2015-09-28 01:11:11 <gribble> Admin, Alias, Anonymous, AutoMode, BadWords, BitcoinData, Channel, ChannelLogger, ChannelStats, Conditional, Config, Debug, Dict, Dunno, Factoids, Filter, Format, GPG, Games, Gatekeeper, Google, Herald, Internet, Later, Market, Math, MessageParser, Misc, Network, OTCOrderBook, Owner, Plugin, RSS, RatingSystem, Reply, Scheduler, Seen, Services, Status, String, Time, Topic, URL, Unix, User, (1 more message)
  7 2015-09-28 01:11:11 <hex__> ;;list
  8 2015-09-28 01:11:27 <hex__> ;;hextarget
  9 2015-09-28 01:11:31 <gribble> 00000000000000001287BA000000000000000000000000000000000000000000
 10 2015-09-28 01:12:15 <hex__> ;;bcstats
 11 2015-09-28 01:12:18 <gribble> Current Blocks: 376448 | Current Difficulty: 5.933535123386657E10 | Next Difficulty At Block: 376991 | Next Difficulty In: 543 blocks | Next Difficulty In About: 3 days, 16 hours, 39 minutes, and 11 seconds | Next Difficulty Estimate: None | Estimated Percent Change: None
 12 2015-09-28 01:13:04 <gribble> (bounty takes no arguments) -- Get current block bounty.
 13 2015-09-28 01:13:04 <hex__> ;;bounty help
 14 2015-09-28 01:13:11 <hex__> ;;bounty
 15 2015-09-28 01:13:12 <gribble> 25
 16 2015-09-28 01:13:39 <gribble> (genrate <hashrate> [<difficulty>]) -- Calculate expected bitcoin generation rate using <hashrate> Mhps, at current difficulty. If optional <difficulty> argument is provided, expected generation time is for supplied difficulty.
 17 2015-09-28 01:13:39 <hex__> ;;genrate
 18 2015-09-28 01:14:04 <hex__> ;;genrate 3000
 19 2015-09-28 01:14:06 <gribble> The expected generation output, at 3000.0 Mhps, given difficulty of 59335351233.9, is 2.54269928115e-05 BTC per day and 1.05945803381e-06 BTC per hour.
 20 2015-09-28 01:14:44 <ploopkazoo> why is there no "haswalletsupport" rpc call?
 21 2015-09-28 01:14:46 <ploopkazoo> that would be nice
 22 2015-09-28 01:15:22 <hex__> ;;nethash
 23 2015-09-28 01:15:23 <gribble> 448115619.594
 24 2015-09-28 01:15:58 <gribble> 14661250.0
 25 2015-09-28 01:15:58 <hex__> ;;totalbc
 26 2015-09-28 06:34:26 <jonasschnelli> ploopkazoo: call "getinfo" and if you'll find a "walletversion" K/V, you know that a wallet is available.
 27 2015-09-28 06:34:51 <jonasschnelli> although "getinfo" is a bit expensive to call... so better don't call it to often.
 28 2015-09-28 08:37:25 <gribble> The operation succeeded.
 29 2015-09-28 08:37:25 <mircea_popescu> ;;later tell gavinandresen https://www.youtube.com/watch?v=L8d9pBvtr8s fucktard.
 30 2015-09-28 13:18:07 <gavinandresen> coryfields: I want to add a 'make bench' target that is like the existing 'make check'; what is the magic autotools sauce?  https://github.com/gavinandresen/bitcoinxt/commit/9d167b1a754fdd6448bc77d7250e6277e4dcdaa6
 31 2015-09-28 13:19:52 <jgarzik> gavinandresen, That seems like it would do it
 32 2015-09-28 13:19:56 <jgarzik> gavinandresen, what is missing?
 33 2015-09-28 13:20:59 <gavinandresen> jgarzik: I want 'make bench' at top-level of source tree to work...
 34 2015-09-28 13:22:08 <jgarzik> gavinandresen, Your commit adds a top level makefile target 'bench'.  Does that not work?
 35 2015-09-28 13:22:35 <gavinandresen> It'll work if I'm in the src/ directory, not in the top-level directory
 36 2015-09-28 13:22:55 <jgarzik> gavinandresen, There is a lot of custom code built into automake to turn 'TESTS' into the 'make check' stuff.  That is too much to hope for.  :)
 37 2015-09-28 13:23:54 <gavinandresen> jgarzik: mmm... I got lost trying to figure out how 'make check' at the very top-level works, which is why I'm asking
 38 2015-09-28 13:24:02 <jgarzik> gavinandresen, "cd src" seems like a reasonable bar for developers to attain to execute benchmarks?
 39 2015-09-28 13:24:16 <jgarzik> gavinandresen, "make check" hunts through TESTS in */Makefile.am
 40 2015-09-28 13:24:28 <jgarzik> gavinandresen, "make check" isn't coded by Cory but by automake
 41 2015-09-28 13:24:54 <gavinandresen> jgarzik: I suppose my question then becomes: is there magic benchmarking stuff in automake I should know about?
 42 2015-09-28 13:24:55 <jgarzik> (I know a lot about autoconf/automake/libtool)
 43 2015-09-28 13:25:06 <jgarzik> gavinandresen, easy answer - no
 44 2015-09-28 13:25:20 <gavinandresen> I'm happy to leave it at "cd src; make bench" to run benchmarks
 45 2015-09-28 13:25:32 <jgarzik> gavinandresen, You are basically Doing It The Right Way already...
 46 2015-09-28 13:25:38 <gavinandresen> spiffy, thanks!
 47 2015-09-28 13:25:41 <jgarzik> ;p
 48 2015-09-28 13:27:20 <wumpus> a lot of the top-level commands just redirect to src, eg  "bench:" "$(MAKE) -C src bench"
 49 2015-09-28 13:27:41 <wumpus> but yes, having to cd to src/ isn't that much of a problem
 50 2015-09-28 13:27:42 <jgarzik> yep
 51 2015-09-28 14:04:00 <hearn> jgarzik: ping
 52 2015-09-28 14:12:10 <jgarzik> hearn, I'm disappearing-in-five-minutes pong :)
 53 2015-09-28 16:21:31 <ProfMac> write a better make.
 54 2015-09-28 16:45:55 <jgarzik> ProfMac, working on it
 55 2015-09-28 17:37:27 <SpeakFreely> Could someone tell me how I can sign an input that has already been spent? I'm trying to double spend a transaction from days ago because I accidentaly put the fee too low
 56 2015-09-28 17:37:47 <SpeakFreely> Running Bitcoin core, but it won't let me sign it since it says "Input not found or already spent"
 57 2015-09-28 17:38:16 <sipa> with signrawtransaction?
 58 2015-09-28 17:38:20 <SpeakFreely> Yes
 59 2015-09-28 17:38:35 <sipa> you can pass in the scriptpubkey and amount explicitly to the RPC
 60 2015-09-28 17:38:42 <sipa> rather than letting it fetch it from the UTXO set
 61 2015-09-28 17:39:04 <sipa> alternatively, you can use bitcoin-tx tool, which does the same thing as signrawtransaction but doesn't even have access to the UTXO set
 62 2015-09-28 17:39:12 <SpeakFreely> I did createrawtransaction with the txid of the input and the address I want to pay to
 63 2015-09-28 17:39:31 <sipa> that's not enough
 64 2015-09-28 17:39:39 <sipa> the signing code needs access to the output being spent
 65 2015-09-28 17:39:49 <sipa> either it needs to fetch it from the UTXO set, or you pass it in
 66 2015-09-28 17:40:15 <SpeakFreely> How do I pass it? Sorry, I just recently learned how to create a raw txn
 67 2015-09-28 17:41:14 <sipa> do help signrawtransaction
 68 2015-09-28 17:41:21 <sipa> argument 2, prevtxs
 69 2015-09-28 17:43:16 <SpeakFreely> Ah I see, and each argument of that type requires the 4 fields txid,vout,scriptpubkey,redeemscript?
 70 2015-09-28 17:44:03 <arubi> redeemscript is only needed if you're dealing with p2sh
 71 2015-09-28 17:44:04 <sipa> the redeemscript is only necessary if it's a P2SH output
 72 2015-09-28 17:44:33 <SpeakFreely> I see...thanks. How can I get the scriptPubKey?
 73 2015-09-28 17:45:01 <sipa> gettransaction
 74 2015-09-28 17:45:09 <sipa> though it may give the parsed version, not the hex one
 75 2015-09-28 17:45:19 <sipa> typically you use listunspent, but that will not work if it's spent
 76 2015-09-28 17:45:47 <dgenr8> gettransaction returns both versions
 77 2015-09-28 17:46:20 <SpeakFreely> so its the hex that it returns right?
 78 2015-09-28 17:53:30 <SpeakFreely> crap I'm not doing this right lol...are there any good tools to double spend?
 79 2015-09-28 17:54:11 <SpeakFreely> I got an "opcode missing" error
 80 2015-09-28 17:58:22 <SpeakFreely> Okay think I signed it correctly...however where can I push this? Blockr gets an error pushing it b/c its a double spend
 81 2015-09-28 17:59:11 <dgenr8> SpeakFreely: question, if the answer were "just wait two hours" would that have been ok in your use case?
 82 2015-09-28 17:59:36 <SpeakFreely> dgenr8, The answer to what?
 83 2015-09-28 17:59:42 <dgenr8> how to double spend
 84 2015-09-28 18:00:29 <SpeakFreely> I'm not sure what waiting would do...as the txn I'm trying to double spend against was created a few days ago
 85 2015-09-28 18:00:55 <dgenr8> ok, thanks.  that's a yes.  are you just trying to bump fees?
 86 2015-09-28 18:01:07 <SpeakFreely> Yup
 87 2015-09-28 18:04:18 <SpeakFreely> so i have everything signed it just seems I'm having trouble broadcasting it to the network..
 88 2015-09-28 18:05:45 <dgenr8> you can try 1) back up wallet 2) launch with -salvagewallet.  you'll lose ephemeral tx data and unconfirmed txes (which is what you want)
 89 2015-09-28 18:07:17 <sipa> SpeakFreely: how old is the transaction you're double spending?
 90 2015-09-28 18:07:35 <SpeakFreely> So I'm not sure what happened but now that I pushed that tx my old 1 with the low fee just got put into a block lol
 91 2015-09-28 18:07:52 <SpeakFreely> Not sure if that was coincidence or what
 92 2015-09-28 18:07:58 <sipa> it was coincidence
 93 2015-09-28 18:08:01 <sipa> how old was it?
 94 2015-09-28 18:08:14 <SpeakFreely> 3 days I believe
 95 2015-09-28 18:09:50 <SpeakFreely> sipa, how does that amount of time affect the double spend?
 96 2015-09-28 18:15:13 <morcos> sipa: wumpus: BlueMatt: gmaxwell: and anyone else interested in the nitty gritty of mempool limiting.  I was hoping that by this weeks IRC meeting we could "decide" on a way forward.  However after spneding some time reviewing BlueMatt's 6722 I feel unsure of what direction we should go.
 97 2015-09-28 18:15:49 <morcos> There are some things I like better about his strategy, although I'm a bit concerned it hasn't had as much scrutiny as the 6557 approach.
 98 2015-09-28 18:16:16 <morcos> I'm hoping you guys can look at the strategy behind 6722 and give us your thoughts?
 99 2015-09-28 18:17:10 <morcos> Regardless of the approach we take, I think we need to add more pieces.  I can write up arguments for them later, but we need tighter limits on package sizes, and we need to consider ancestor package fee rate (not just current tx) when looking at a new tx
100 2015-09-28 18:17:40 <morcos> There are just too many ways to game these things, we need as many tools at our disposal to make that difficult.
101 2015-09-28 18:20:03 <morcos> BlueMatt: I know you're a bit under the weather, there are several tweaks needed to your code aside from that.  Do you prefer that I leave the feedback on the pull for you to make the changes?  Or should sdaftuar and I grab it and make changes ourselves?  Nothing that changes how to think about the strategy, just obscure failure modes that need to be covered.
102 2015-09-28 19:30:12 <dgenr8> morcos: agreed wrt ancestor package, and it's a short hop to extend that to txes already in the pool.  if feerate(tx) is generally the feerate of the ancestor package, the whole process can proceed at the tx level.
103 2015-09-28 19:34:23 <morcos> its not so simple.  there is a tradeoff between optimizing what's in your mempool from the standpoint of what miners would want to mine, and protecting against attacks
104 2015-09-28 19:34:50 <morcos> for 6557, i believe you need to consider ancestor fee rate to protect against attacks.
105 2015-09-28 19:35:56 <morcos> for 6722, its not clear, requiring ancestor fee rate to get in can sometimes lead to more optimal mempool and sometimes less, so i don't think you should consider it if its not needed for attacks
106 2015-09-28 19:41:03 <dgenr8> 6722 appears not to have the requirement to pay for what was removed in addition to itself. +1
107 2015-09-28 19:47:56 <morcos> yes it accomplishes that in effect by making future transactions pay for it.  it is nice in that its much simpler to reason about what fee rate will be needed to get into the mempool.   the key is balancing the decay rate so that future txs pay for the evicted txs relay enough to prevent bandwidth attacks vs attackers being able to pay a one time cost to boost relay fees high for an extended period
108 2015-09-28 20:15:51 <maaku> sipa: so aparantly calling (CBlockIndex* NULL)->GetMedianTimePast() does not segfault, but returns uninitialized memory
109 2015-09-28 20:19:09 <sipa> maaku: that surorises me, but... same thing :)
110 2015-09-28 20:20:28 <maaku> oh yeah needs to be fixed, but it would have saved me tons of time trying to figure out why it was *not* segfaulting
111 2015-09-28 20:20:43 <maaku> TIL: you can call methods on a NULL pointer
112 2015-09-28 20:23:15 <sipa> non-virtual methods, sure
113 2015-09-28 20:23:27 <sipa> there is no lookup table involved
114 2015-09-28 20:23:42 <sipa> it's really just a function that gets an implicit this argument
115 2015-09-28 20:24:27 <jrick> derefencing null is always undefined behavior, there's never a guarantee of a segfault
116 2015-09-28 20:24:40 <sipa> jrick: it's not even dereferenced
117 2015-09-28 20:24:49 <maaku> jrick: this is true, but it was a surprise to me that it wasn't dereferenced at all
118 2015-09-28 20:25:12 <jrick> i am pretty sure that calling the static method on a null instance is UB too
119 2015-09-28 20:25:24 <maaku> probably
120 2015-09-28 20:25:35 <sipa> jrick: probably yes, but i'm not sure
121 2015-09-28 20:25:54 <maaku> returning uninitialized stack space (which happens here) most definately is undefined as well
122 2015-09-28 20:25:57 <sipa> it mau just literally be defined as a function that gets a this argument
123 2015-09-28 20:26:11 <sipa> yup, that certainly is undefined
124 2015-09-28 20:26:25 <jrick> obj->static_method(); // 'obj->' implies a dereference, and then a pointer of that dereference is passed as the this pointer
125 2015-09-28 20:26:52 <sipa> i think you are right
126 2015-09-28 20:27:24 <sipa> it's just convertee into a null reference before the call happens, which is undefined, but works in oractice
127 2015-09-28 20:27:42 <jrick> until it doesn't
128 2015-09-28 20:28:08 <sipa> and even if it is defined, it would be pretty bad practice
129 2015-09-28 20:31:00 <sipa> jrick: did some googling; pretty sure you're right
130 2015-09-28 20:38:23 <jrick> I believe that c++ added classes before references existed, which is why this is a pointer and not a reference, but the language guarantees that it is always non-null (or UB has already been hit)
131 2015-09-28 20:59:31 <maaku> sipa: so there does not appear to be a clean, easy way to get the height of the best block from a view
132 2015-09-28 21:04:24 <maaku> other than calling GetStats which serializes the entire utxo database
133 2015-09-28 21:07:45 <sipa> maaku: add a GetHeight function to CCoinsView :)
134 2015-09-28 21:08:23 <maaku> sipa: yeah, for bitcoin/master for sure. should I keep this PR ugly with a comment, for the purpose of back-porting?
135 2015-09-28 21:09:26 <sipa> i think that's fine
136 2015-09-28 22:39:35 <BlueMatt> morcos: I can make changes
137 2015-09-28 22:39:41 <BlueMatt> feel free to leave it on that pull or email me
138 2015-09-28 22:39:43 <BlueMatt> either way is fine
139 2015-09-28 22:40:01 <morcos> Ok, I was about to start leaving on the pull and I realized the one I was most concerned about wasn't actually a problem
140 2015-09-28 22:40:37 <morcos> Have you thought whether there is any case where the code could fail to evict anything for a particularly high fee rate transaction for some edge reason?
141 2015-09-28 22:40:40 <BlueMatt> there are lots of those there :)
142 2015-09-28 22:41:00 <BlueMatt> I have not spent significant time thinking about that, no?
143 2015-09-28 22:41:01 <morcos> I was worried about dirty descendant package statistics, but i guess if the set fails you just keep on going
144 2015-09-28 22:41:07 <BlueMatt> why would that code fail to evict anything?
145 2015-09-28 22:41:14 <BlueMatt> oh?
146 2015-09-28 22:41:43 <morcos> I'm worried about the cases where you stop looking for new things, but pretty much that can't happen unless you've evicted enough or you will find nothing to evict
147 2015-09-28 22:41:47 <BlueMatt> why would the stats be particularly dirty at that run?
148 2015-09-28 22:41:55 <BlueMatt> yea
149 2015-09-28 22:42:18 <morcos> I was thinking if you hit a dirty one and it failed then you would stop and the fact that it was sorted at the wrong place would hurt you
150 2015-09-28 22:42:35 <morcos> If you have a moment I'll describe some other concerns here, so you could ask questions?
151 2015-09-28 22:42:55 <BlueMatt> sure
152 2015-09-28 22:43:08 <BlueMatt> I cant guarantee I'll be useful, but I can try to think :)
153 2015-09-28 22:44:02 <morcos> Some are maybe just comments
154 2015-09-28 22:44:15 <morcos> I notice you moved your check to the end of the ATMP call
155 2015-09-28 22:45:08 <morcos> That's important, because if you hadn't already called CalculateMempoolAncestors, then it would be possible to have a transaction which joined an unfair number of big packages together which faield to evict anything (b/c it was protected by everything) and bumped the relay fee very high
156 2015-09-28 22:45:40 <morcos> Similarly its important that since you go ahead and evict in your code, that there are no other checks in ATMP that could fail after your code
157 2015-09-28 22:45:57 <morcos> maybe you already knew all this, but I'd just want to be sure anyone editing the code later caught it
158 2015-09-28 22:47:09 <morcos> Perhaps the biggest attack thats unique to your strategy that worries me is a miner pumping the mempool full of high fee txs
159 2015-09-28 22:47:48 <morcos> Let's say he's a 25% miner, then he'll recover 25% of the fees he paid to do the attack right off the bat, and he'll make 25% of all excess fees that are paid by other participants until the decay goes back down
160 2015-09-28 22:48:01 <morcos> depending on the elasticity of transaction supply, this could be a lot of fees
161 2015-09-28 22:48:31 <morcos> The network right now depends on tragedy of the commons preventing miners from just submitting lots of high fee txs to drive the prevailing fee rate up
162 2015-09-28 22:48:49 <morcos> So we need to make sure that it remains hard for any 1 miner to do it at break even or close to it
163 2015-09-28 22:49:37 <morcos> I think your code should bypass evictions during a reorg, and then just evict the lowest stuff from the mempool to bring it back to size afterwards
164 2015-09-28 22:50:00 <BlueMatt> ATMP?
165 2015-09-28 22:50:00 <morcos> I think time based expirations is good to add on top of anything just to capture any misalignment of miner policy with mempool sorting
166 2015-09-28 22:50:06 <morcos> AcceptToMemoryPool
167 2015-09-28 22:50:08 <BlueMatt> ahh, yes
168 2015-09-28 22:50:55 <morcos> miner nit that trackRemovedOrAddFailed gets called for all txs even if the mempool isn't full
169 2015-09-28 22:51:52 <BlueMatt> oops, yea, thats not supposed to happen
170 2015-09-28 22:52:04 <BlueMatt> that actually does break things
171 2015-09-28 22:52:17 <morcos> i may be leaning towards your approach of ours though...  the benefit of simplicity is huge
172 2015-09-28 22:52:21 <morcos> why does it break things?
173 2015-09-28 22:52:41 <BlueMatt> it breaks the "this patch disables free relay of high prio txn unless mempool is full"
174 2015-09-28 22:52:57 <BlueMatt> specifically the if (mempoolRejectFee > 0...) check
175 2015-09-28 22:53:07 <morcos> the mempoolRejectFee stays 0
176 2015-09-28 22:53:26 <BlueMatt> wait, what?
177 2015-09-28 22:53:32 <BlueMatt> oh, because bestFeeRateRemoved is 0?
178 2015-09-28 22:53:34 <BlueMatt> yea, ok, whatever
179 2015-09-28 22:53:37 <BlueMatt> should be fixed, but ok
180 2015-09-28 22:53:52 <morcos> anyway i think the key is going to be figuring out the right way to decay that feerate
181 2015-09-28 22:53:57 <BlueMatt> yea, so I'd air on the side of the feerate decay going down fast
182 2015-09-28 22:54:06 <morcos> perhaps exponential decay is not the right answer, maybe it should be linear
183 2015-09-28 22:54:28 <BlueMatt> the idea that patrick (I think) had to decrease the decay only after a block has been found since last bump was really good
184 2015-09-28 22:54:35 <BlueMatt> and, yea, because of that probably exponential isnt right anymore
185 2015-09-28 22:54:44 <morcos> yeah i like that part
186 2015-09-28 22:55:39 <morcos> anyway, i have to run, dinner is served, but thanks for getting this out there...  i'll keep hacking away on it
187 2015-09-28 22:55:49 <BlueMatt> might even add a failsafe that says eg "if the mempool size limit is set to at least ~100 blocks, and it is now <= 3 blocks, drop feerate significantly"
188 2015-09-28 22:56:00 <BlueMatt> yea, i should probably go find breakfast
189 2015-09-28 22:56:05 <morcos> ha ha
190 2015-09-28 23:02:47 <maaku> BlueMatt: isn't it 4pm where you are?
191 2015-09-28 23:03:00 <BlueMatt> maaku: yes, but flu makes me sleep late :/
192 2015-09-28 23:03:25 <maaku> aw :( I hope you feel better soon
193 2015-09-28 23:03:50 <BlueMatt> yea, the 13 hours of sleep should help...