1 2015-10-20 01:09:25 <morcos> gmaxwell: i'm not so sure there isn't some issue here related to the memory leaks
  2 2015-10-20 01:09:47 <morcos> i'm just now really looking at getblocktemplate for the first time, so i have a limited understanding
  3 2015-10-20 01:10:28 <morcos> but i just set up a little script on a freshly started node which calls getblocktemplate every 30 secs and watches the resident memory usage
  4 2015-10-20 01:10:35 <morcos> its climbing pretty quickly
  5 2015-10-20 01:12:02 <morcos> much faster than the usage of the mempool
  6 2015-10-20 01:12:24 <morcos> anyway i'm going to look into it some more and i'll report back
  7 2015-10-20 01:15:25 <jgarzik> morcos, there's a cache related to getblocktemplate
  8 2015-10-20 01:16:22 <morcos> jgarzik: there is?  in what part of the code?
  9 2015-10-20 01:16:32 <Luke-Jr> just the most recent result, IIRC
 10 2015-10-20 01:17:30 <morcos> yes, the most recent blocktemplate isn't deleted until you're ready to create a new one
 11 2015-10-20 01:17:31 <Luke-Jr> morcos: running in Valgrind?
 12 2015-10-20 01:18:03 <morcos> Luke-Jr: no, maybe that'll be next
 13 2015-10-20 01:18:42 <Luke-Jr> target remote | vgdb
 14 2015-10-20 01:18:44 <Luke-Jr> monitor leak_check full any unlimited
 15 2015-10-20 01:18:45 <Luke-Jr> ^ useful trick
 16 2015-10-20 01:18:56 <morcos> Luke-Jr: one thing that surprised me is I took a long running bitcoind with a low min relay fee, that had a 2GB+ mempool already...  It's resident size was already 5GB
 17 2015-10-20 01:19:08 <morcos> I ran getblocktemplate once, and the resident size jumped to 2GB
 18 2015-10-20 01:19:13 <Luke-Jr> (dumps leaks of a running Valgrind process, without exiting it; which means you don't get stuff "leaked" at shutdown)
 19 2015-10-20 01:19:20 <morcos> jumped by 2, to 7
 20 2015-10-20 01:19:35 <Luke-Jr> O.o
 21 2015-10-20 01:20:08 <Luke-Jr> I'd expect it to grow, but only 10 MB or so, not GBs
 22 2015-10-20 01:20:29 <morcos> why would that be the case, i guess the heap in CreateNewBlock maybe isn't unallocated right away or something
 23 2015-10-20 01:21:11 <Luke-Jr> not sure, I've never had a need to look at it
 24 2015-10-20 01:21:34 <Luke-Jr> (memory usage of GBT, that is)
 25 2015-10-20 01:21:38 <morcos> ok, but you confirmed how i thought it should work, the block template should be a small factor the size of a block
 26 2015-10-20 01:22:08 <morcos> i wasn't sure if i was missing something where it kept track of all the txs somewhere, and not just the ones CreateNewBlock put into the block for it
 27 2015-10-20 01:22:12 <Luke-Jr> yeah, off-hand there'd be the entire block, possibly in hex (although I doubt it), plus dependency info
 28 2015-10-20 01:22:14 <gmaxwell> morcos: oh. hm. I was about to say "I don't see that"
 29 2015-10-20 01:22:21 <gmaxwell> morcos: but I do. interesting.
 30 2015-10-20 01:22:47 <gmaxwell> My normal mining node is up at 3.416 resident, it has a quite high relay fee.. and it certantly wasn't like that a few weeks ago.
 31 2015-10-20 01:22:52 <gmaxwell> (3.415 GB)
 32 2015-10-20 01:23:26 <gmaxwell> cpus is pegged on it too.
 33 2015-10-20 01:24:01 <gmaxwell> (cpu pegged seems to be significantly due to mempool checking)
 34 2015-10-20 01:24:05 <gmaxwell> uh wtf.
 35 2015-10-20 01:24:07 <gmaxwell> 2015-10-20 01:23:49 Checking mempool with 1623 transactions and 248745 inputs
 36 2015-10-20 01:24:49 <jgarzik> $certainBigMiner emailed me privately two days ago, noting they cannot mine (GBT) on current master
 37 2015-10-20 01:25:10 <morcos> hmm, i wonder if the recent spammer took to creating some higher fee txs
 38 2015-10-20 01:25:15 <gmaxwell> So that makes me doubt a memory leak and just suspect a new transaction pattern that uses craptons of ram.
 39 2015-10-20 01:25:22 <morcos> but many of his were 100txins -> 1 out
 40 2015-10-20 01:25:46 <Luke-Jr> src/txmempool.h:    bool fSanityCheck; //! Normally false, true if -checkmempool or -regtest
 41 2015-10-20 01:25:58 <jgarzik> so
 42 2015-10-20 01:26:02 <gmaxwell> Luke-Jr: yes, I have checkmpool running on this node.
 43 2015-10-20 01:26:06 <jgarzik> What are known #6722 merge blockers?
 44 2015-10-20 01:26:10 <jgarzik> Upon re-review, I don't see any
 45 2015-10-20 01:26:11 <gmaxwell> That wasn't my wtf, my wtf was the 248745
 46 2015-10-20 01:26:11 <morcos> gmaxwell: yeah i've made that mistake before
 47 2015-10-20 01:26:15 <Luke-Jr> i c
 48 2015-10-20 01:26:15 <morcos> peers kept dropping me
 49 2015-10-20 01:26:25 <Luke-Jr> gmaxwell: it's not the reason for CPU being pegged though?
 50 2015-10-20 01:26:31 <gmaxwell> morcos: this host is fast enough that it won't get disconnected.
 51 2015-10-20 01:26:31 <jgarzik> #6722 has more than enough ACKs.
 52 2015-10-20 01:26:38 <gmaxwell> Luke-Jr: 18:24 < gmaxwell> (cpu pegged seems to be significantly due to mempool checking)
 53 2015-10-20 01:26:54 <Luke-Jr> oh, missed that
 54 2015-10-20 01:27:22 <jgarzik> My test node has been running it for ~6 days without CPU peg
 55 2015-10-20 01:27:26 <gmaxwell> morcos: this hos is fast enough it won't get dropped with mempool checking (at least not with high limits)
 56 2015-10-20 01:27:52 <morcos> s/is and won't/was and wouldn't/
 57 2015-10-20 01:28:39 <gmaxwell> hah. Well it's not getting disconnected right now. :P
 58 2015-10-20 01:28:52 <gmaxwell> in any case with that many txins loaded, that along might explain the memory usage; there is quite a bit of overhead
 59 2015-10-20 01:29:40 <morcos> gmaxwell: but what does getmempoolinfo say?
 60 2015-10-20 01:30:08 <gmaxwell> morcos: 78MB usage, but it's counting the mempool transactions themselves, not the transactions pulled into the view cache to verify them-- I think
 61 2015-10-20 01:30:43 <morcos> gmaxwell: hmm, but that can't be a signficant multiple can it
 62 2015-10-20 01:31:35 <gmaxwell> hm. no even that is big, if I assume 2.5GB is used by this and divide by that txin count, it's 10kb per.
 63 2015-10-20 01:34:09 <gmaxwell> jgarzik: wrt that miner, kind of them to test-- though I hope they're not regularly mining on master. :)
 64 2015-10-20 01:34:39 <gmaxwell> reports on the mailing list are on 0.11 I thought? (in particular, they're users without mempool 'usage' vs size info)
 65 2015-10-20 01:35:15 <gmaxwell> my mining node is on master, but some number of weeks back and carrying some other patches at the moment.
 66 2015-10-20 01:35:33 <gmaxwell> it's 536207f3167daad1fa3d60a1de7d9cb55db28ac9 plus -maxuploadtarget
 67 2015-10-20 01:35:54 <gmaxwell> and running with a very high relay fee. (0.0001 or something)
 68 2015-10-20 01:36:12 <morcos> i have to run for now, will check back later
 69 2015-10-20 02:23:12 <warren> jcorgan: sent you the bitcoin-discuss admin password, please coordinate with the others on how it is to be managed
 70 2015-10-20 02:26:59 <jcorgan> got it, but looks like you encrypted to wrong key id
 71 2015-10-20 02:28:15 <jcorgan> nm, i derped
 72 2015-10-20 04:52:24 <btcdrak> warren: great!
 73 2015-10-20 06:58:26 <jonasschnelli> ping cfields_
 74 2015-10-20 06:59:42 <jonasschnelli> Is this assumption correct: I can pay to a non-standard P2SH script, but spending the non-standard P2SH script could be impossible (or at least hard to find a miner)?
 75 2015-10-20 07:02:26 <Luke-Jr> jonasschnelli: no, default policy accepts even non-standard P2SH
 76 2015-10-20 07:02:50 <Luke-Jr> unless it's crazy sigops
 77 2015-10-20 07:08:37 <jonasschnelli> Luke-Jr: ah, okay. So expect for the sigops limit, i can create crazy non-standard script over P2SH outputs and be able to spend them afterwards?
 78 2015-10-20 07:08:47 <Luke-Jr> yes, you should be able to
 79 2015-10-20 07:11:06 <jonasschnelli> Thanks Luke-Jr.
 80 2015-10-20 07:24:14 <jonasschnelli> I'd like to link against libbtc (https://github.com/jonasschnelli/libbtc) dynamic. Anyone knows the reason why my symbols table of libbtc.dynlib (or .so on linux, checked with nm -gU ) is empty while "nm -gU libbtc.a" prints the correct sym-table?
 81 2015-10-20 07:38:42 <jonasschnelli> nm: found the problem: -fvisibility was = hidden.
 82 2015-10-20 09:33:55 <rodmar> hi
 83 2015-10-20 09:34:04 <rodmar> I am using bitcoind in regtest mode for testing my code. It was working fine but now I get a bunch of error messages: "{u'message': u'64: non-mandatory-script-verify-flag (No error)', u'code': -26}"
 84 2015-10-20 09:35:34 <rodmar> also in my log files I get a lot of "ERROR: CheckProofOfWork(): hash doesn't match nBits"
 85 2015-10-20 10:00:06 <rodmar> anyone?
 86 2015-10-20 10:03:34 <wumpus> rodmar: what is the block number?
 87 2015-10-20 10:04:29 <wumpus> rodmar: if the failure is due to retargeting going wrong on regtest, try https://github.com/bitcoin/bitcoin/pull/6853
 88 2015-10-20 10:08:09 <rodmar> I am not sure. what I am doing in my tests is creating a new block with generate 1 every time I create a transaction
 89 2015-10-20 10:08:26 <wumpus> then it is most likely that issue.
 90 2015-10-20 10:09:36 <rodmar> the fact that I generate a new block after I push a transaction?
 91 2015-10-20 10:09:48 <wumpus> see the link I pasted
 92 2015-10-20 10:10:14 <wumpus> regtest retargets were broken, I've just merged code into master to fix that
 93 2015-10-20 10:10:27 <wumpus> (well, to not do retargets at all)
 94 2015-10-20 10:11:03 <rodmar> I can try the master branch but I don't really understand the problem. What is a retarget?
 95 2015-10-20 11:18:16 <jtimon> wumpus: ping #6235
 96 2015-10-20 12:02:38 <jtimon> wumpus: should I translate initialization errors? https://github.com/bitcoin/bitcoin/pull/6235/files#r42484244
 97 2015-10-20 12:03:03 <wumpus> jtimon:  no, don't translate errors
 98 2015-10-20 12:03:10 <wumpus> (they must be google-able)
 99 2015-10-20 12:04:09 <jtimon> it seems we're currently translating many errors in init.cpp, but thanks, I will change that too
100 2015-10-20 12:04:15 <wumpus> the only exception would be if  it's something that directly affects the user of the interface and fits in the application area eg 'insufficient funds'
101 2015-10-20 12:12:29 <wumpus> if it looks too technical for translators and users and is more troubleshooting, leave it untranslated
102 2015-10-20 12:21:16 <rodmar> wumpus, I am using the latest master and I am still getting {u'message': u'64: non-mandatory-script-verify-flag (No error)', u'code': -26}
103 2015-10-20 12:21:36 <jtimon> wumpus: well "Unkown chain" doesn't sound too technical for me (specially since now it should be impossible to get that error and in the future it could be "unkown chain AAA" when the user introduces -chain=AAA)
104 2015-10-20 12:22:03 <jtimon> but I don't have a strong opinion here
105 2015-10-20 12:22:07 <wumpus> jtimon: it is technical - normal users will never change the chain
106 2015-10-20 12:22:16 <wumpus> they use mainnet, and have no clue it's even called that way :)
107 2015-10-20 12:22:25 <jtimon> fair enough, removing translation then