1 2015-04-15 00:04:33 <BlueMatt> Lightsword: those nodes are likely to be better connected, on average, yes....however,
  2 2015-04-15 00:04:34 <BlueMatt> <BlueMatt> also, a miner who connects to *all* the nodes will find that their overly-bursty bandwidth usage will cause you to lose packets when you're sending blocks, resulting in seconds more to get the block out to the first few peers
  3 2015-04-15 00:05:34 <BlueMatt> I dont think your conclusion is completely invalid, I'm just afraid that any non-giant increase in outbound connections will result in very negligible decrease in orphan rates, whereas finding good peers will result in a huge one
  4 2015-04-15 00:05:59 <Lightsword> BlueMatt how many connections do you think a pool should be making?
  5 2015-04-15 00:06:03 <BlueMatt> 8
  6 2015-04-15 00:06:15 <Lightsword> and no inbound connections?
  7 2015-04-15 00:06:26 <BlueMatt> sure, accept inbound
  8 2015-04-15 00:06:58 <BlueMatt> frankly, no matter what you do, on the p2p network, youre not gonna do *great* hop-to-other-miners-wise
  9 2015-04-15 00:06:59 <Lightsword> Why would a pool want to accept inbound connections instead of making more outbound connections(which should statistically be better connected nodes)?
 10 2015-04-15 00:07:03 <BlueMatt> but you'll do plenty well
 11 2015-04-15 00:07:41 <BlueMatt> because you're accepting inbound connections as a service to the network, and only barely as a way to make more connections :)
 12 2015-04-15 00:09:10 <BlueMatt> 
ACTION -> out

 13 2015-04-15 00:09:25 <Lightsword> BlueMatt, I think thats almost a separate issue, a pools job is not to provide connections to users on NAT restricted networks, its to confirm transactions on the network which works better if one uses well connected nodes
 14 2015-04-15 00:09:53 <BlueMatt> sure, but if you have the node already, might as well do both :)
 15 2015-04-15 00:11:55 <Lightsword> BlueMatt, I have a node with tons of excess bandwidth and I’m only at 30 connections(I have the limit set to 125), I’m sure there are many other nodes in the same situation, if it would speed up transactions and block propogation why aren’t we using those connections?
 16 2015-04-15 00:13:29 <BlueMatt> <BlueMatt> its harder to attack a network with headroom than one running at capacity (for people actively trying to take the whole thing down)
 17 2015-04-15 00:13:35 <BlueMatt> 
ACTION -> actually out now

 18 2015-04-15 00:14:20 <Lightsword> BlueMatt, so one solution may be to have 2 types of connections, reserved and unreseved or something like that
 19 2015-04-15 00:15:09 <Lightsword> outbound connections over 8 are *droppable* connections if a user that needs needs a *non-droppable* connection wants to connect
 20 2015-04-15 00:28:35 <lechuga_> does 0.10 no longer track orphan blocks
 21 2015-04-15 00:38:07 <lechuga_> to answer my own first question this used to be the solution:
 22 2015-04-15 00:38:08 <lechuga_> https://github.com/bitcoin/bitcoin/blob/0.8/src/main.cpp#L3423-L3427
 23 2015-04-15 00:39:21 <Luke-Jr> lechuga_: there are no orphan blocks in 0.10
 24 2015-04-15 00:39:31 <lechuga_> thx
 25 2015-04-15 00:39:45 <lechuga_> seemed 2 be case from reading but good to confirm
 26 2015-04-15 00:41:16 <lechuga_> trying to figure out how the above captioned code is handled in 0.10
 27 2015-04-15 00:41:24 <lechuga_> or rather the situation the captioned code is meant to handle
 28 2015-04-15 00:42:37 <lechuga_> still a mystery in my mind over why not all old nodes didnt get sycned on testnet after yesterday
 29 2015-04-15 00:50:44 <lechuga_> hmm dont see it
 30 2015-04-15 01:13:18 <lechuga_> hmm ok this commit changed the behavior
 31 2015-04-15 01:13:18 <lechuga_> https://github.com/bitcoin/bitcoin/commit/f59d8f0b644d49324cabd19c58cf2262d49e1392
 32 2015-04-15 01:17:09 <lechuga_> sipa: whenever you're up and if u happen to recall, do you know why you deleted this line/branch: https://github.com/bitcoin/bitcoin/commit/f59d8f0b644d49324cabd19c58cf2262d49e1392#diff-7ec3c68a81efff79b6ca22ac1f1eabbaL3459
 33 2015-04-15 01:17:21 <lechuga_> cant figure out what handles that situation now
 34 2015-04-15 01:29:53 <sipa> lechuga_: it's headers first; we already know the entire chain before start fetching blocks
 35 2015-04-15 01:30:13 <sipa> no need for hacks to discover the chain anymore
 36 2015-04-15 01:30:19 <lechuga_> ic
 37 2015-04-15 01:30:36 <lechuga_> well i didnt impl headers first so i need ot add this hack :)
 38 2015-04-15 01:30:37 <lechuga_> thx for info
 39 2015-04-15 01:40:38 <gmaxwell> Lightsword: due to recent abusive behavior you can expect that opening many outbound connections will just get you banned by nodes; already some do this and I've been working on tools to make it easier for nodes to ban people that open excessive outbound connections.
 40 2015-04-15 01:41:54 <Lightsword> gmaxwell, how nodes know another node is opening too many outbound connections?
 41 2015-04-15 01:55:59 <gmaxwell> Lightsword: by observing it on the network (people run multiple nodes or cooperate with other nodes and compare notes)
 42 2015-04-15 01:57:05 <Lightsword> gmaxwell, would that be an option only enabled on nodes with very limited incoming connections?
 43 2015-04-15 01:58:43 <gmaxwell> No.
 44 2015-04-15 01:59:22 <gmaxwell> There is no reason for any host to open more than a fairly small number of outbound connections; doing so is abusive on the network and harmful to other users.
 45 2015-04-15 02:01:28 <Lightsword> gmaxwell, faster block propagation would seem to be a reason, limiting connections too much can increase the chance of orphans it would seem
 46 2015-04-15 02:15:39 <gmaxwell> Lightsword: having too many connections increases orphans. A single slow peer will block and delay your transmission.
 47 2015-04-15 02:16:15 <Lightsword> gmaxwell, so bitcoind can’t transmit simultaniously?
 48 2015-04-15 02:18:00 <gmaxwell> Lightsword: a new block is transmitted sequentially to each peer.
 49 2015-04-15 02:19:02 <Lightsword> gmaxwell, couldn’t you DOS by locking up a nodes connection then?
 50 2015-04-15 02:21:23 <gmaxwell> Lightsword: perhaps you should have listened a day ago when I told you that it was insane to have a pool node exposed to untrusted nodes ... because people could and would DOS it?
 51 2015-04-15 02:21:27 <gmaxwell> :)
 52 2015-04-15 02:23:09 <Lightsword> gmaxwell, well I don’t advertize the node is a pool or anything like that but it sounds insane for bitcoind to wait on a dialup user if there is plenty of bandwidth available
 53 2015-04-15 02:23:28 <Lightsword> why can’t it just transfer concurrently across all connections?
 54 2015-04-15 02:26:54 <gmaxwell> Because it's not simple and not a priority, your optimizing the linear part of an expoential formula. The important thing for relay behavior is to get it to a couple nodes very fast, the propation cone expands expoentially. having more places on the first step is only a linear improvement.
 55 2015-04-15 02:32:12 <Lightsword> gmaxwell, sounds like that dealing with that should be a priority since it is a DOS, how long can one user lock up a node for?
 56 2015-04-15 02:36:12 <gmaxwell> Not a long time compared to the normal interblock variance in the network, it's only a big concern for miners, which can address it (and a thousand other issues) by following the standard best practice advice of NOT connecting directly to untrusted nodes.
 57 2015-04-15 02:38:10 <Lightsword> gmaxwell, how are miners supposed to find trusted nodes?
 58 2015-04-15 02:40:23 <gmaxwell> Run your own, in part, use the relay network.
 59 2015-04-15 02:42:35 <Lightsword> gmaxwell, so bitcoind can do parallel block downloads but not uploads?
 60 2015-04-15 02:44:53 <rusty> Dumb q: What bounds are there on the txmempool in the core code?  Does it just accumulate cruft over time?
 61 2015-04-15 02:47:20 <gmaxwell> rusty: gets forgotten on restart, the priority/fee stuff forms both a pratical limit and a pedantic technical limit too. (meaning: in practice the usage doesn't become very large, though it could in theory, but even in theory there is a 'limit')
 62 2015-04-15 02:47:46 <gmaxwell> rusty: better handling of that is a todo. It's a little tricky, since too eager reeping encourages doublespending races at the expiration point.
 63 2015-04-15 02:50:21 <gmaxwell> rusty: e.g. if you have a expiration there is some incentive to try to race the replacement, if the replacement is unpredictable, you can use it to get a conflicting transaction to end up only in part of the mempools. Probably if the limit is 'big' then these issues are moot. But it may be better to support storing the mempool on disk first, so there isn't artifical pressure to define 'big' as not ve
 64 2015-04-15 02:50:27 <gmaxwell> ry big at all. :)
 65 2015-04-15 02:51:05 <gmaxwell> rusty: right now we actually don't have a good handle on where all the memory is going in core; basic heap instrumentation (e.g. tcmalloc stuff) seems to only explain about half the usage.
 66 2015-04-15 02:52:03 <gmaxwell> We know we lose a fair bit to eager allocation in vectors, and other assorted heap overheaders (e.g. from fragmentation and padding and pointers from tiny objects).  sipa did some stl magic a while back to dump the true size of many objects (and we went around optimizing some), but I suspect that work has since bitrotted.
 67 2015-04-15 02:53:24 <gmaxwell> We overuse heap allocation of small short lived objects, though this is a side effect of a programming style that is also fairly safe... it has a large cost both in memory overhead, and-- seemingly, speed, if profiles are to be believed malloc is a major consideration in initial block download, at least with the signatures disabled.
 68 2015-04-15 02:54:29 <rusty> gmaxwell: ah, I found the penny flooding stuff in main.cpp.  So, we limit what goes in, but never expire.  Does that mean there are stuck txs in long-running bitcoinds in practice?
 69 2015-04-15 02:56:37 <gmaxwell> rusty: the rate limiter isn't the primary mechenism; to get it a transaction must have sufficient fees, or must have sufficient priority. (priority comes from spending coins that have say around without moving); priority is backstopped by a rate limiter.
 70 2015-04-15 02:57:16 <gmaxwell> And sure there are txn that sit around for a while, though the network policy is fairly consistent so not usually. If you make your own relay policy more relaxed than the miners you'll end up with tx in your mempool for a long time.
 71 2015-04-15 02:59:40 <rusty> gmaxwell: this plays with IBLT.  It's easy to just throw everything in the mempool into the IBLT, but when that doesn't work, you have potential to use heuristics to try to make more progress (ie. "let's assume this TX isn't in the block...")
 72 2015-04-15 03:02:49 <rusty> gmaxwell: I'm recording behaviour on 4 nodes in different locations, to get real data.  If anyone has a longrunning node, could they tell me the getmempoolinfo?  I'd like to figure out how long I should run before considering the mempool to have stabilized.
 73 2015-04-15 03:03:19 <gmaxwell> all mine are recently restarted due to 0.10rc testing.
 74 2015-04-15 03:03:23 <gmaxwell> BlueMatt: ^ ?
 75 2015-04-15 03:34:56 <BlueMatt> gmaxwell/rusty: oops, thats also true for me :(
 76 2015-04-15 03:35:38 <rusty> BlueMatt: NP
 77 2015-04-15 05:35:38 <nicelys> test
 78 2015-04-15 05:55:05 <wumpus> rusty: getmempoolinfo? {  "size" : 1208,    "bytes" : 734708 }, node has been running since 2015-04-01
 79 2015-04-15 05:58:28 <rusty> wumpus: thanks...
 80 2015-04-15 11:30:21 <wallet42> is there a livestream from devcore?
 81 2015-04-15 12:53:08 <aliakbar> Hi guys - I got a questio on [../qa/rpc-tests/util.py]: how exactly does "os.getpid()" work in the way of port assigning?
 82 2015-04-15 13:17:14 <wumpus> it makes sure that multiple tests can be run in parallel
 83 2015-04-15 13:17:22 <wumpus> by giving them a port based on the process id
 84 2015-04-15 14:41:31 <aliakbar> wumpus: ok thats clear to me but...does this function os.getpid() also manage free ports? does it instantly look for free ports?
 85 2015-04-15 14:42:14 <wumpus> no, it does nothing with ports, I suggest looking into the python library documentation if you need a detailed description beyond what I said
 86 2015-04-15 14:44:17 <aliakbar> ok i hope i can find better results by googling this time
 87 2015-04-15 14:44:19 <aliakbar> thanks
 88 2015-04-15 14:56:21 <priidu> wonder what the "correct" MIME type for bitcoind request is
 89 2015-04-15 14:56:42 <priidu> I think Bitcoin Core responds in application/json
 90 2015-04-15 14:57:16 <priidu> but the help command recommends using text/plain
 91 2015-04-15 14:57:25 <priidu> in the end it doesn't really matter, I suppose
 92 2015-04-15 15:06:07 <priidu> forgot to mention that I was talking about the JSON-RPC api :p
 93 2015-04-15 15:10:30 <wumpus> application/json
 94 2015-04-15 15:11:19 <wumpus> where does it recommend using text/plain?
 95 2015-04-15 15:16:29 <priidu> the "As a json rpc call" section for 'help sendfrom' for example
 96 2015-04-15 15:16:47 <priidu> although it's passed through curl, shouldn't it still be "application/json"?
 97 2015-04-15 15:25:20 <j4m13> Hi, I am looking for a good altcoin developer to fix a broken POS coin. I am willing to pay for their help. Anyone interested PM Me.
 98 2015-04-15 15:27:27 <adlai> j4m13: you may have better luck asking for a mathematician to fix a broken consensus algorithm
 99 2015-04-15 15:27:58 <j4m13> adlai it certainly is beginning to feel that way :-)
100 2015-04-15 15:28:15 <adlai> (although the jury's out (some say they've already decided!) on whether it's fixable)
101 2015-04-15 15:28:59 <j4m13> lol
102 2015-04-15 15:30:05 <adlai> the problem correctly identified is half solved, and the other 90% of success is showing up
103 2015-04-15 15:42:05 <berndj> priidu, i made a demo a while ago and it serves the request as Content-Type: application/bitcoin-paymentrequest
104 2015-04-15 15:45:14 <phantomcircuit> can anybody point me in the direction of share validation in p2pool? specifically that the coinbase pays to the previous shares
105 2015-04-15 15:46:18 <gmaxwell> phantomcircuit: email forrestv. :) (he responds)
106 2015-04-15 16:12:53 <phantomcircuit> gmaxwell, hmm so for p2pool every node needs to see all of the transaction data for all of the current blocks shares
107 2015-04-15 16:30:40 <adlai> phantomcircuit: yes, because each p2pool node independently verifies the block content
108 2015-04-15 16:35:02 <gmaxwell> adlai: it's not not about the blocks that he's asking, it's about verifying the shares.
109 2015-04-15 16:35:21 <adlai> aren't p2pool shares each a candidate block?
110 2015-04-15 16:36:11 <adlai> to ensure that p2pool miners don't cheat eachother, each one has to verify that each share could have been a valid block, if not for the lower difficulty
111 2015-04-15 16:36:25 <gmaxwell> thats not strictly true.
112 2015-04-15 16:36:32 <adlai> how not so?
113 2015-04-15 16:37:09 <gmaxwell> adlai: someone can always withhold attack and its impossible to prevent it in that model (or the current bitcoin mining in general)
114 2015-04-15 16:37:54 <adlai> ok, true... so replace "cheat eachother" with "get credit for more valid shares than they produced"
115 2015-04-15 16:38:11 <adlai> is that correct?
116 2015-04-15 16:39:46 <gmaxwell> adlai: so doing something to prevent attacks which are isomorphic to withholding is pretty pointless.
117 2015-04-15 16:41:18 <gmaxwell> adlai: there is no need to check validity in ways which are just equal to a withholding attack. Say you include a double spend in your block; that candidate block would be invalid if it were a block sure.  Thats sad. But it's exactly the same as if the user, having a valid block, just discarded it.
118 2015-04-15 16:41:39 <gmaxwell> P2Pool discourages it by having a half percent of your income be solomining.
119 2015-04-15 16:42:30 <adlai> 
ACTION doesn't understand why they're equivalent; isn't it much easier to generate garbage blocks, than to withhold valid ones?

120 2015-04-15 16:43:26 <sipa> define easy?
121 2015-04-15 16:43:40 <sipa> both require a simple mining software patch
122 2015-04-15 16:44:03 <adlai> easier = requires less computation
123 2015-04-15 16:44:23 <sipa> how does withholding require any computation?
124 2015-04-15 16:44:32 <sipa> you just don't submit winning shares...
125 2015-04-15 16:44:54 <adlai> withholding a valid block/share only happens if you have enough hashpower to compute a valid one
126 2015-04-15 16:45:10 <sipa> so does generating a garbage block
127 2015-04-15 16:45:46 <adlai> 
ACTION seems to be confusing validating block contents, and validating PoW difficulty

128 2015-04-15 16:45:55 <gmaxwell> it requires zero computation to not transmit a block.
129 2015-04-15 16:47:05 <sipa> if your share doesn't even meet the PoW requirement, it will just be discarded and you won't get paid
130 2015-04-15 16:47:34 <sipa> if it has valid PoW, but corresponds to an invalid block contents, it's equivalent to a withholding attack
131 2015-04-15 16:48:19 <adlai> so: nodes do validate the PoW requirement, but don't validate full block contents, because it's cheaper to disincentivize withholding and garbage attacks by incentivizing publishing a valid block?
132 2015-04-15 16:49:00 <sipa> bitcoin full nodes obviously validate everything
133 2015-04-15 16:49:25 <sipa> gmaxwell is arguing that mining pool software shouldn't bother validating block contents however before deciding whether to accept a share
134 2015-04-15 16:49:29 <adlai> right, iiuc phantomcircuit was asking about p2pool itself
135 2015-04-15 16:49:57 <sipa> because even if they did that, a vandalizing miner could equally easily do a withholding attack, with the same costs
136 2015-04-15 16:50:15 <sipa> the only reason to do so is perhaps to detect broken software
137 2015-04-15 16:51:42 <priidu> btw, noticed that "bitcoin-cli help walletpassphrase" and "bitcoin-cli help walletpassphrasechange" return "help: unknown command: ..." the other day
138 2015-04-15 16:51:55 <priidu> not sure if it's deliberate or not, just caught my eye
139 2015-04-15 16:52:19 <gmaxwell> adlai: you have to ask what risk you're protecting against;  "The miner isn't really working for the pool, he's just mining junk."  this is the same ham as "The miner isn't really working for the pool, he's withholding blocks when he finds them"
140 2015-04-15 16:52:38 <adlai> priidu: on which version? they return documentation for me
141 2015-04-15 16:52:53 <gmaxwell> (other than perhaps an unintentional error)
142 2015-04-15 16:53:01 <priidu> hmm, i'm using 0.10.0
143 2015-04-15 16:53:07 <sipa> adlai, priidu: they will only return documentation in case of an encrypted wallet
144 2015-04-15 16:53:11 <sipa> i think this is bad
145 2015-04-15 16:53:14 <priidu> oh, right :D
146 2015-04-15 16:53:34 <priidu> yes, i think i was on an unencrypted testnet wallet
147 2015-04-15 16:53:37 <priidu> when i noticed it
148 2015-04-15 16:54:31 <phantomcircuit> so does p2pool check transactions with bitcoind?
149 2015-04-15 16:54:49 <gmaxwell> phantomcircuit: no.
150 2015-04-15 16:59:20 <phantomcircuit> gmaxwell, so p2pool wont count claimed transaction fees unless they're in the local bitcoin's getblock result
151 2015-04-15 16:59:43 <phantomcircuit> which seems like it probably breaks the payout policy if fees become large
152 2015-04-15 17:20:16 <adlai> phantomcircuit: iiuc, the consensus policy is only an upper bound. if p2pool nodes don't collect the maximum possible block fee, they've "burned" the difference
153 2015-04-15 17:43:41 <paveljanik> jonasschnelli, mempool tests added.
154 2015-04-15 18:03:56 <jonasschnelli> paveljanik: nice. Will give it a proper test soon. Nice work!
155 2015-04-15 18:05:01 <paveljanik> jonasschnelli, thanks. It is easy the the infrastructure is already there and you fill in the empty gaps ;-)
156 2015-04-15 18:55:19 <jtimon> wumpus cfields_ can you take another look at #5968 ? I have a couple of commits there pending squashing or removal
157 2015-04-15 20:22:56 <mrkent_> Is there an operational website for visualizing connections between bitcoin addresses (similar to https://coinkite.com/faq/btclook)