1 2015-07-15 01:34:59 <phantomcircuit> sipa, do you happen to know offhand why there is a CBlockHeader::GetBlockTime method which converts the uint32_t to int64_t ?
  2 2015-07-15 01:36:49 <sipa> phantomcircuit: i guess because someone once wanted to convert the codebase from 32-bit timestamps to 64-bit once, except within protocol data structures
  3 2015-07-15 01:37:05 <phantomcircuit> hmm
  4 2015-07-15 01:37:08 <phantomcircuit> sipa, thanks
  5 2015-07-15 01:37:34 <sipa> the question should perhaps be "why the hell are we accessing private data members almost everywhere without encapsulated getters?"
  6 2015-07-15 01:37:41 <sipa> to which GetBlockTime is an exception :p
  7 2015-07-15 01:39:40 <phantomcircuit> heh
  8 2015-07-15 01:55:31 <phantomcircuit> sipa, hmm there's no easy way to use the primitive data structures is there
  9 2015-07-15 01:56:45 <sipa> ?
 10 2015-07-15 01:57:03 <phantomcircuit> sipa, CBlockHeader isn't in a library or anything that gets installed
 11 2015-07-15 01:57:28 <sipa> you mean as in libbitcoinconsensus? no
 12 2015-07-15 02:08:00 <warren> jgarzik: FYI, mailman at linuxfoundation.org now should be munging DMARC so posting from your bitpay.com domain should theoretically work now without ending up in people's spam bins.  Could you please test it?
 13 2015-07-15 02:08:20 <morcos> sipa: i'd really like it if we got a couple of chances at least to find a transactin to kick out.
 14 2015-07-15 02:08:41 <sipa> morcos: ok?
 15 2015-07-15 02:08:46 <sipa> don't we?
 16 2015-07-15 02:08:52 <morcos> it seems like you shouldn't be able to use all of your fails on your first try
 17 2015-07-15 02:09:59 <morcos> i like the 10% chance of checking, but if you get unlucky and check one with > 10 children and you end up failing, you won't make it in the mempool.
 18 2015-07-15 02:10:15 <sipa> hmm, yes
 19 2015-07-15 02:11:14 <sipa> i consider this very-large-package to be something exceptional
 20 2015-07-15 02:12:20 <morcos> what if the fails was only incremented by 1 (not iternow) and you capped it at like 3 or 4... and then you left the iternow > 20 check, so worst case you check (3 or 4) x 20txs, if you happen to check all big chains that fail
 21 2015-07-15 02:15:40 <sipa> sgtm
 22 2015-07-15 02:15:51 <morcos> the other thing sdaftuar and i were contemplating is you can put a fast check at the beginning which will help.  you check the fee rate of the bottom tx in the mempool = LowRate.  you know that the total fee you need on your transaction must be > LowRate * (your txsize + size you must expel to be under the limit).
 23 2015-07-15 02:16:36 <sipa> i think that's what i was talking about yesterday
 24 2015-07-15 02:16:46 <sipa> except that you know how much usage to expel, not how much txsize
 25 2015-07-15 02:18:10 <morcos> yes, but you can use that as a proportion right...  if you have to expel A bytes of usage, and you're guessing dyanmic usage of B for your own transaciton, then assume you're expelling A/B * (your tx size) of txsize?
 26 2015-07-15 02:18:42 <morcos> maybe you were talking about that yesterday, it went over my head at the time then..
 27 2015-07-15 02:21:11 <morcos> so then you have to decide that if your nFees-nDeleted is not > minRelayTxFee, should you take a chance at free relay, or try to use up a few more fails first to find a better package to expel
 28 2015-07-15 02:21:52 <sipa> i think the free relay is generally broken already...
 29 2015-07-15 02:22:02 <sipa> (not that i think it should be)
 30 2015-07-15 02:22:16 <morcos> why broken?
 31 2015-07-15 02:23:28 <sipa> well the current code will very likely kick out free-relay transactions first from the mempool
 32 2015-07-15 02:23:35 <sipa> in my pr i mean
 33 2015-07-15 02:23:45 <morcos> oh yeah good point
 34 2015-07-15 02:23:56 <sipa> and in general, the fact that "free transactions" is separate from the normal mechanism is not very meaningful
 35 2015-07-15 02:24:16 <sipa> why wouldn't i be able to get some better processing by paying both some fee AND have some old coins to spend
 36 2015-07-15 02:25:32 <morcos> so break it to potentially add it back later in better form?
 37 2015-07-15 02:26:04 <sipa> i'm not intentionally breaking it further, but i don't think it's worth much effort to keep working either
 38 2015-07-15 02:26:11 <morcos> if we're lucky we won't hit the mempool limits, ha. if there is one thing that will make sure we hit the limit, its having it
 39 2015-07-15 02:26:38 <sipa> a correct solution is to account for priority by making it adjust the fee
 40 2015-07-15 02:27:02 <morcos> yeah thats what i meant by better form
 41 2015-07-15 02:27:51 <morcos> so here's another consideration.  if we reject a tx from peer 1 because we randomly don't find packages to kick out that pass the test
 42 2015-07-15 02:28:11 <morcos> aren't we just going to have to try again if peer 2 did accept it and then relays it to us
 43 2015-07-15 02:28:18 <morcos> seems pretty inefficient
 44 2015-07-15 02:29:01 <morcos> i think that argues for trying harder to include it the first time round
 45 2015-07-15 02:29:12 <morcos> (tolerating more fails)
 46 2015-07-15 02:30:56 <morcos> maybe we need some heuristic on whether it seems like it should be accepted (such as the fast test mentioned before, but maybe not with the rock bottom feerate) and then we tolerate more fails than if it fails that heuristic.
 47 2015-07-15 02:32:27 <sdaftuar> for that to have a chance we still would need to move the fee tests that currently happen after the call to StageTrimToSize into that function, so that we can use fail attempts to find better packages to remove
 48 2015-07-15 02:32:31 <sdaftuar> (have we discussed that yet?)
 49 2015-07-15 03:38:12 <hey_joe> anyone have experience with the OS X platform
 50 2015-07-15 03:38:35 <hey_joe> running it own system launchdaemon instead of user etc?
 51 2015-07-15 03:38:56 <hey_joe> or even... where the log file is perhaps
 52 2015-07-15 03:38:59 <hey_joe> would be a start
 53 2015-07-15 11:33:17 <Lightsword> how long should a GBT rpc call normally take after a block notify?
 54 2015-07-15 12:17:56 <jonasschnelli> Lightsword: GBT?
 55 2015-07-15 12:18:03 <Lightsword> getblocktemplate
 56 2015-07-15 12:18:39 <jonasschnelli> you mean the response-time after you requested over rpc?
 57 2015-07-15 12:19:29 <Lightsword> yeah
 58 2015-07-15 12:46:18 <jonasschnelli> is it possible to re-encode a mainnet CExtKey (xpub) with a testnet Base58check (tpub)? I assume skipping the 4 check bytes, de-base58 and re-base58check with the testnet magic 0x043587CF should work?
 59 2015-07-15 12:47:41 <sipa> why would youbwant to?
 60 2015-07-15 12:53:10 <jonasschnelli> sipa: hardware wallet can only export mainnet ext keys, need to test on testnet.
 61 2015-07-15 12:56:05 <gavinandresen> git HEAD build is failing for me on OSX, pristine build environment. Compiling primitives/transaction.cpp, <boost/foreach.hpp> is not found because for some reason -I/usr/local/include isn't being passed
 62 2015-07-15 12:57:45 <gavinandresen> Has there been a change to libbitcoinconsensus build stuff in the last week or so?
 63 2015-07-15 13:04:48 <jonasschnelli> gavinandresen: Build is also failing here. But because of yesterdays/todays Q_EMIT/emit change.
 64 2015-07-15 13:05:35 <gavinandresen> I'll git bisect to see where it started failing for me....
 65 2015-07-15 13:05:47 <wumpus> no problems building here, travis passes too
 66 2015-07-15 13:07:21 <wumpus> jonasschnelli: are you sure that's the cause? any errors regarding macros?
 67 2015-07-15 13:07:55 <jonasschnelli> wumpus: yes. One emit left over is in macdockiconhandler.mm.
 68 2015-07-15 13:08:00 <jonasschnelli> I can form a PR if you like
 69 2015-07-15 13:08:13 <jonasschnelli> travis is not compiling the mac part...
 70 2015-07-15 13:09:00 <jonasschnelli> L133
 71 2015-07-15 13:09:03 <wumpus> ok
 72 2015-07-15 13:09:22 <jonasschnelli> but it looks like that gavin has a different issue.
 73 2015-07-15 13:10:14 <wumpus> jonasschnelli: fixed
 74 2015-07-15 13:10:42 <jonasschnelli> wumpus: wow. This was quick. Thanks
 75 2015-07-15 13:11:50 <jonasschnelli> Yes. Compiling works now.
 76 2015-07-15 13:11:51 <wumpus> but I'm not sure how anything with regard to boost/foreach.hpp can have changed, that indeed sounds like a different issue
 77 2015-07-15 13:13:43 <gavinandresen> mmm... I found a commit that works, will run bisect and see what pops up. That'll take a while, though...
 78 2015-07-15 13:20:22 <wumpus> <jonasschnelli> travis is not compiling the mac part... <- it should, right? ("#.7 Cross-Mac")
 79 2015-07-15 13:45:43 <gavinandresen> 5098c47b2430ded299c21620527ebd0544fe51e2 is the first bad commit  (sipa's accurate memory accounting for mempool)  -->  because primitives/transaction.cpp added #include "memusage.h" and memusage.h includes boost stuff.
 80 2015-07-15 13:46:51 <gavinandresen> Is libbitcoinconsensus supposed to be boost-free ?
 81 2015-07-15 13:48:52 <sipa> oh, yes indeed
 82 2015-07-15 13:49:00 <sipa> i think that can be avoided
 83 2015-07-15 13:49:08 <wumpus> yes, I think so
 84 2015-07-15 13:55:06 <gavinandresen> jonasschnelli: are you using homebrew for your mac builds? I'm wondering why it worked for you...
 85 2015-07-15 13:56:45 <sipa> easy solution: only have memusage.h define boost stuff when a MEMUSAGE_BOOST_DEPENDENCY is defined
 86 2015-07-15 13:56:53 <sipa> and not define that inside primitives
 87 2015-07-15 13:57:21 <wumpus> maybe easier: memusage_boost.h ?
 88 2015-07-15 13:57:26 <sipa> won't work
 89 2015-07-15 13:57:40 <sipa> at least if you want the recursive definitions
 90 2015-07-15 13:57:49 <wumpus> hm okay
 91 2015-07-15 13:57:58 <sipa> you can't add a new template instantiation in the middle of already defined ones
 92 2015-07-15 13:58:15 <sipa> we can avoid the recursion, though
 93 2015-07-15 13:58:29 <sipa> but i figure it's something that will become more useful over time
 94 2015-07-15 13:58:34 <sipa> if we want to add more accounting
 95 2015-07-15 13:58:53 <wumpus> right, you'd need some way to be able to 'plug in' data types later, and that's not possible
 96 2015-07-15 13:58:54 <sipa> however, it seems near impossible to do for multiindex anyway, so perhaps we need a more manual approach
 97 2015-07-15 13:59:43 <sipa> so, perhaps there can be a memusage_boost, but it won't take part in the recursion
 98 2015-07-15 14:00:36 <wumpus> but yeah, MEMUSAGE_BOOST_DEPENDENCY for all compilation units that use boost would work too
 99 2015-07-15 14:00:54 <wumpus> s/use boost/are allowed to use boost/
100 2015-07-15 14:01:20 <sipa> the alternative is to make primitives entirely memusage free
101 2015-07-15 14:01:24 <wumpus> but it may be overkill if a memusage_boost solution works
102 2015-07-15 14:02:13 <wumpus> oh, that'd be best
103 2015-07-15 14:02:25 <sipa> and have some core_memusage module that implements memory usage computation
104 2015-07-15 14:02:32 <wumpus> moves memusage out of the consensus code itself
105 2015-07-15 14:02:56 <sipa> but for CBlock, i think we want to have cached memory usage, rather than recomputing it by iterating over transactions
106 2015-07-15 14:03:01 <wumpus> I suppose that does need friend declarations to make sure the memory usage computation code can access all fields
107 2015-07-15 14:03:26 <sipa> i'll fix that in a few hours
108 2015-07-15 14:03:55 <jonasschnelli> gavinandresen: sorry. I'm more or less afk. Yes. I'm using homebrew.
109 2015-07-15 14:04:40 <jonasschnelli> wumpus: i think the specific part where it needs the MacOSX10.9 framework is not covered over travis IIRC?
110 2015-07-15 14:05:15 <wumpus> jonasschnelli: that would explain it
111 2015-07-15 14:28:38 <morcos> sipa: sdaftuar and i have been discussing 6421 and mempool limitation in general.  it seems like it would be nice to build in some way to amortize the cost of kicking large transaction packages from the bottom of the mempool.
112 2015-07-15 14:28:53 <morcos> we have an idea on how to use the dynamic mempool growth to do this
113 2015-07-15 14:30:08 <morcos> how set are you on pushing forward with 6421 as is, or should we take a crack at modifying it to include both the soft cap (could also include a hard cap) and a cleanup which uses the tx's beyond the soft cap as a reservoir of excess fees to boot things at the bottom.
114 2015-07-15 14:32:21 <gavinandresen> sipa: I'll file an issue and assign it to you, just so it doesn't get forgotten and in case anybody else runs into the same problem....
115 2015-07-15 14:41:24 <Wegglesworth> anyone know why maxconnections flag isnt working in 0.11 ?
116 2015-07-15 14:42:05 <Wegglesworth> my client is using a ton of bandwidth, i used to use  -maxconnections=16 to manage data usage
117 2015-07-15 14:42:33 <Wegglesworth> i launched with that flag but core is showing 43 connections
118 2015-07-15 14:49:17 <wumpus> Wegglesworth: any "Using at most %i connections " message in debug.log? what does it show?
119 2015-07-15 14:52:56 <Wegglesworth> wumpus, http://puu.sh/j09bE/c2fe89c9fa.png
120 2015-07-15 14:53:59 <Apocalyptic> Wegglesworth, the "%i" was meant to indicate an integer
121 2015-07-15 14:54:10 <Apocalyptic> of course the log won't print %i, but a number instead
122 2015-07-15 14:55:13 <Luke-Jr> dgenr8: not sure we should be encouraging people to confess to crimes on the dev ML :/
123 2015-07-15 14:55:39 <Wegglesworth> Apocalyptic, what search string should i use then? i'm not a programmer sry
124 2015-07-15 14:55:47 <Wegglesworth> network admin
125 2015-07-15 14:56:17 <Apocalyptic> "Using at most " seems fine
126 2015-07-15 14:57:24 <Wegglesworth> ok, that did find something: http://puu.sh/j09pA/510153bd94.png
127 2015-07-15 14:58:26 <Apocalyptic> so there you go, it seems your maxconnections is set to 125
128 2015-07-15 15:00:06 <Wegglesworth> weird, i launched it using a shortcut that always worked in the past: http://puu.sh/j09yv/d8e6aca44b.png
129 2015-07-15 15:05:27 <sipa> Wegglesworth: strange, that should work
130 2015-07-15 15:05:33 <hdcn1> can anyone explain to me why mike hearn is still involved in the bitcoin core project?
131 2015-07-15 15:05:49 <Wegglesworth> ill try and relaunch the software
132 2015-07-15 15:06:33 <hdcn1> he is directly responsible for two soft forks and pushing features which create vulnerabilities in bitcoin
133 2015-07-15 15:07:34 <hdcn1> is this some kind of group solidarity where the project is protecting an incompetent developer?
134 2015-07-15 15:07:39 <cysm> off topic
135 2015-07-15 15:08:45 <hdcn1> https://mail-archive.com/bitcoin-dev@lists.linuxfoundation.org/msg00500.html
136 2015-07-15 15:09:30 <Wegglesworth> looks like that may have worked http://puu.sh/j0a3n/47a9dc9d3e.png
137 2015-07-15 15:11:17 <sipa> Wegglesworth: looks good
138 2015-07-15 15:11:26 <jcorgan> hdcn1: please take this elsewhere, like #bitcoin, and don't be so trollish
139 2015-07-15 15:12:06 <hdcn1> the Bitcoin Core project endangers its credibility and political influence every time these issues occur and summarily dismissed as a mistake; no onlookers can reasonably see this and determine it occurred through sheer incompetence rather than malice
140 2015-07-15 15:21:21 <morcos> Does the call to CheckInputs in CreateNewBlock (miner.cpp:282) ever fail or is that just belt-and-suspenders in case we allowed something in our mempool we shouldn't have?
141 2015-07-15 15:26:00 <sipa> morcos: iirc it should never fail
142 2015-07-15 15:50:42 <Luke-Jr> morcos: or if any other part of block creation went wrong
143 2015-07-15 15:51:12 <Luke-Jr> err, I may be thinking of a different part
144 2015-07-15 18:02:08 <StephenM347> Peter todd's python-bitcoinlib seems to keep an RPC connection, or some kind of connection that prevents my daemon from shutting down. Does anyone know anything about this?
145 2015-07-15 18:02:52 <StephenM347> AFAIK, the bitcoin server RPC doesn't support keep-alive connections, but maybe I'm wrong
146 2015-07-15 18:06:43 <StephenM347> "Connection": "keep-alive" in the header doesn't seem to be having the desired effect...
147 2015-07-15 18:10:08 <jonasschnelli> StephenM347: it does support http keep alive. Check qa/rpc-tests/httpbasics.py
148 2015-07-15 18:11:57 <StephenM347> jonasschnelli: Do I have to start with -rpckeepalive?
149 2015-07-15 18:13:01 <petertodd> Steph~.~.
150 2015-07-15 18:13:12 <StephenM347> I guess I just assumed it was enabled by default, because why wouldn't it be?
151 2015-07-15 18:14:02 <jonasschnelli> StephenM347: it on by default
152 2015-07-15 18:15:11 <petertodd> StephenM347: sorry, shit wifi connection here...
153 2015-07-15 18:15:48 <petertodd> StephenM347: anyway, I think http-keepalive is the answer; if that works for you would you please do a pull-req mentioning that in the python-bitcoinlib docs or something?
154 2015-07-15 18:18:04 <StephenM347> petertodd: python-bitcoinlib is working correctly, keeping the connection alive, for some reason adding the keep-alive header in a Node.js project isn't keeping the connection alive
155 2015-07-15 18:24:31 <petertodd> StephenM347: ah! misread you
156 2015-07-15 18:42:51 <wumpus> rpckeepalive is enabled by default, it can be disabled because some http libraries don't handle it properly and keep connections open beyond their usefulness, leading to clogged servers
157 2015-07-15 19:01:44 <StephenM347> jonasschnelli: my telnet connection is automatically closed http://pastebin.com/g2HVRk0K
158 2015-07-15 19:02:31 <jonasschnelli> StephenM347: what version of core are you using?
159 2015-07-15 19:02:54 <StephenM347> jonasschnelli: v0.10.2
160 2015-07-15 19:04:57 <jonasschnelli> StephenM347: hmm... yous response looks good. You got a Connection: keep-alive...
161 2015-07-15 19:05:00 <jonasschnelli> let me try
162 2015-07-15 19:06:12 <StephenM347> jonasschnelli: haha, thanks for looking into i.
163 2015-07-15 19:06:14 <StephenM347> it
164 2015-07-15 19:09:16 <jonasschnelli> works fine on my end... but was to tired to use telnet... with curl it works
165 2015-07-15 19:09:20 <jonasschnelli> i post a gist, wait
166 2015-07-15 19:10:23 <jonasschnelli> https://gist.github.com/jonasschnelli/fcac061a44e8d35bea4b
167 2015-07-15 19:10:25 <jonasschnelli> mind line 27
168 2015-07-15 19:11:06 <jonasschnelli> So you probably did wrong implement HTTP 1.1 persistent connections on your end?
169 2015-07-15 19:12:11 <StephenM347> jonasschnelli: Strange, I'll try the same.
170 2015-07-15 19:14:29 <jonasschnelli> Maybe its hard to simulate over telnet because a whitespace or a newline could lead to a close connection from the server
171 2015-07-15 19:15:42 <StephenM347> I ran the same with curl and it seemed to mostly work, but gave this: Warning: Keep-alive functionality somewhat crippled due to missing support in your operating system!
172 2015-07-15 19:16:03 <StephenM347> Strange... (OS X, btw)
173 2015-07-15 19:23:31 <jonasschnelli> StephenM347: huh... yeah. This looks strange. Did you google after missing keep alive support?
174 2015-07-15 19:23:51 <jonasschnelli> I have tested on debian 7... but will do now on osx
175 2015-07-15 19:24:13 <StephenM347> I'm looking around now. I mean, it did reuse the connection, just gave that warning
176 2015-07-15 19:25:40 <jonasschnelli> Warning: Keep-alive functionality somewhat crippled due to missing support in
177 2015-07-15 19:25:44 <jonasschnelli> ^^ ha..
178 2015-07-15 19:25:56 <jonasschnelli> but it works...
179 2015-07-15 19:26:00 <StephenM347> jonasschnelli: the strange thing is that --keepalive-time 60 seemed to work, but curling with -H 'connection: keep-alive;' doesn't
180 2015-07-15 19:26:11 <StephenM347> jonasschnelli: glad to see it's a reproducable issue
181 2015-07-15 19:26:15 <wumpus> why is the ';' there?
182 2015-07-15 19:26:37 <jonasschnelli> indeed
183 2015-07-15 19:26:51 <wumpus> the connection header should exactly match 'keep-alive', trailing junk will make it mismatch
184 2015-07-15 19:26:53 <jonasschnelli> its terminated by \r\n
185 2015-07-15 19:27:19 <StephenM347> Maybe that's my issue..
186 2015-07-15 19:27:21 <jonasschnelli> -H "connection: keep-alive" works on my end
187 2015-07-15 19:28:24 <jonasschnelli> maybe --keepalive-time 60 does keep the connection persistent over multiple curl interactions... not sure though.
188 2015-07-15 19:28:40 <jonasschnelli> But curl's last verbose output is "* Connection #0 to host 127.0.0.1 left intact"
189 2015-07-15 19:30:21 <StephenM347> Yeah, working now. Anyway, thanks for the help
190 2015-07-15 19:39:43 <jonasschnelli> Somehow the clients needs a way to tell the server to finish the connection-resuing.
191 2015-07-15 19:40:36 <wumpus> connection: close on the last request
192 2015-07-15 20:09:54 <SeXploit> @jgarzik you need to twitter pm me your mailing address or here so i can ship you out these chips for the sats
193 2015-07-15 21:03:34 <phantomcircuit> wumpus, there's a guy who keeps spamming pull requests and issues with nonsense
194 2015-07-15 21:03:44 <phantomcircuit> is there any existing protocol for banning people from github?
195 2015-07-15 21:05:02 <wumpus> if it's really spam the best way is to report them to github admin
196 2015-07-15 21:05:23 <wumpus> (or clearly abusive)
197 2015-07-15 21:05:50 <wumpus> also I think it's possible to ban people from just one repository, but I've never done that yet
198 2015-07-15 21:06:16 <phantomcircuit> wumpus, it's this guy https://github.com/bitcoin/bitcoin/pull/3753#issuecomment-121559904
199 2015-07-15 21:08:04 <sipa> he's harmless :)
200 2015-07-15 21:08:28 <phantomcircuit> sipa, harmless yes, annoying, also yes
201 2015-07-15 21:09:01 <wumpus> yeah, he sprouts a lot of nonsense, but he's never abusive
202 2015-07-15 23:32:48 <jtimon> some optimizations around AcceptToMemoryPool can be found in https://github.com/bitcoin/bitcoin/pull/6445