1 2015-04-06 00:54:44 <phantomcircuit> gmaxwell, so i noticed that bandwidth usage is a lot more spikey with the latency_fix patch
  2 2015-04-06 00:55:05 <phantomcircuit> this is on a 1gbps lan so it's no where near saturation
  3 2015-04-06 02:49:01 <phantomcircuit> gmaxwell, benchmark posted
  4 2015-04-06 02:49:20 <phantomcircuit> ~30% performance improvement
  5 2015-04-06 02:51:18 <gmaxwell> \0/
  6 2015-04-06 05:28:28 <fanquake> ;;blocks
  7 2015-04-06 05:28:28 <gribble> 350939
  8 2015-04-06 05:29:54 <phantomcircuit> gmaxwell, i dont understand the "addr" trickle logic
  9 2015-04-06 05:29:59 <phantomcircuit> what's the goal there?
 10 2015-04-06 05:33:30 <gmaxwell> batch addr flooding. I guess.
 11 2015-04-06 05:39:41 <phantomcircuit> gmaxwell, i'd ask the original author but well it's satoshi...
 12 2015-04-06 05:39:56 <phantomcircuit> possible there isn't really a good reason for it
 13 2015-04-06 05:45:09 <gmaxwell> phantomcircuit: we don't want new addr message broadcasts to facilitate analysis of the network topology.
 14 2015-04-06 05:45:34 <gmaxwell> and we also don't want to waste bandwidth with them crossing in flight both directions on the same link.
 15 2015-04-06 05:51:06 <phantomcircuit> <gmaxwell> phantomcircuit: we don't want new addr message broadcasts to facilitate analysis of the network topology.
 16 2015-04-06 05:51:24 <phantomcircuit> you mean we dont want to broadcast address of newly connected peers?
 17 2015-04-06 05:54:42 <gmaxwell> when a peer announces itself if you're connected to the whole network you could get a pretty good idea of who is connected to who by how the new announcement propagated.
 18 2015-04-06 06:12:30 <phantomcircuit> gmaxwell, it seems like the trickle logic is a fairly weak protection there
 19 2015-04-06 06:18:45 <gmaxwell> Not sure what else could be done; other than for addr it could have much greater delays. it should be pretty good a breaking up the cross in trafer.
 20 2015-04-06 06:45:06 <phantomcircuit> gmaxwell, i guess part of the question is why does the node advertise itself?
 21 2015-04-06 06:46:12 <gmaxwell> phantomcircuit: so other people know it exists and can connect to it.
 22 2015-04-06 07:08:28 <phantomcircuit> gmaxwell, it's connected to other nodes, they know it exists
 23 2015-04-06 07:15:09 <phantomcircuit> gmaxwell, it seems to be like if everybody just responded to getaddr messages through addrman
 24 2015-04-06 07:15:19 <phantomcircuit> that would be better than peers explicitly pushing their own addr
 25 2015-04-06 07:59:55 <wumpus> "<phantomcircuit> gmaxwell, it's connected to other nodes, they know it exists" -> but initially only with outgoing connections, so it has to advertise its own listening address
 26 2015-04-06 08:01:19 <wumpus> the TCP source ip does not tell you everything about how to connect to it; at the least not the listening port, and on privacy network such as Tor there is no source IP at all
 27 2015-04-06 08:01:46 <wumpus> so yes, nodes need to advertize themselves to get incoming connections
 28 2015-04-06 08:39:39 <phantomcircuit> wumpus, ok
 29 2015-04-06 08:40:24 <phantomcircuit> and i can see why the broadcast of the peers addr would be included in the normal queue to disguise it a bit
 30 2015-04-06 08:40:39 <wumpus> exactly
 31 2015-04-06 08:40:43 <phantomcircuit> but i dont see why the normal queue would be rate trickled
 32 2015-04-06 08:40:52 <phantomcircuit> instead the address being put into the queue should be trickled
 33 2015-04-06 08:40:57 <wumpus> why not?
 34 2015-04-06 08:41:26 <wumpus> gmaxwell gave some good reasons to trickle. What would, according to you, be the advantage of not trickling?
 35 2015-04-06 08:41:50 <wumpus> there is no necessity to make address propagation faster
 36 2015-04-06 08:42:41 <phantomcircuit> uh well for one as it's currently running all responses to getaddr are trickled
 37 2015-04-06 08:42:51 <phantomcircuit> the only reason it works is that the trickling logic is broken
 38 2015-04-06 08:45:23 <wumpus> trickly just means that the answers will come in slower
 39 2015-04-06 08:45:36 <wumpus> I don't see how it is broken
 40 2015-04-06 08:46:38 <phantomcircuit> wumpus, answers dont come in slower...
 41 2015-04-06 08:47:01 <phantomcircuit> at least not enough for it to make a difference
 42 2015-04-06 08:47:11 <wumpus> but that is the point isn't it?
 43 2015-04-06 08:47:28 <phantomcircuit> wumpus, that's why i said the trickle logic is broken
 44 2015-04-06 08:47:58 <phantomcircuit> i actually dont think it ever worked very well since at the sleep was 100ms
 45 2015-04-06 08:48:03 <wumpus> it's very possible for there to be bugs in the logic, but you were arguing against the concept
 46 2015-04-06 08:48:34 <wumpus> if there are bugs they obviously need to be solved
 47 2015-04-06 08:49:10 <phantomcircuit> no im saying the trickling should be of the thing that is sensitive, our address, not of all the addr responses
 48 2015-04-06 08:49:26 <phantomcircuit> yeah im fixing it that's why all the questions
 49 2015-04-06 08:50:28 <wumpus> but if you'd only trickle our own address it'd stand out like a sore thumb
 50 2015-04-06 08:50:43 <phantomcircuit> wumpus, ah i see the confusion
 51 2015-04-06 08:50:51 <wumpus> the current logic stashes it somewhere between the other addresses to avoid that
 52 2015-04-06 08:51:19 <phantomcircuit> i meant that there would just be a flag or something indicating our address should be included in whatever the next addr message is
 53 2015-04-06 08:51:35 <phantomcircuit> instead of putting it in the queue and then applying the trickle logic to the entire queue
 54 2015-04-06 08:51:47 <phantomcircuit> (it's not a queue it's a list)
 55 2015-04-06 08:51:49 <wumpus> right
 56 2015-04-06 08:51:53 <phantomcircuit> set?
 57 2015-04-06 08:51:56 <phantomcircuit> whatever
 58 2015-04-06 08:52:04 <phantomcircuit> wumpus, does that make sense now?
 59 2015-04-06 08:52:55 <wumpus> but what would be the essential difference? in both cases it's added to one of the next addr messages
 60 2015-04-06 08:53:09 <wumpus> a flag implies extra state to take care of
 61 2015-04-06 08:53:34 <phantomcircuit> if i just fix the trickle logic then responses to getaddr will be at random
 62 2015-04-06 08:53:56 <wumpus> while if if you stash your address with the other addresses, you're virtually guaranteed it's not treated any differently
 63 2015-04-06 08:54:24 <wumpus> the gossip of addresses over the network should be random-ish
 64 2015-04-06 08:54:33 <phantomcircuit> except that it is being treated differently
 65 2015-04-06 08:55:37 <wumpus> okay
 66 2015-04-06 08:56:46 <phantomcircuit> wumpus, if responses to getaddr are delayed and trickled then probably they should just be ignored entirely and the sharing of addresses done entirely without prompting
 67 2015-04-06 08:57:29 <phantomcircuit> which isn't something im necessarily opposed to
 68 2015-04-06 09:19:19 <wumpus> well, getaddr does provide one bit of information: the other node is interested in hearing about addresses
 69 2015-04-06 09:19:35 <phantomcircuit> wumpus, it could just set a CNode flag
 70 2015-04-06 09:19:39 <phantomcircuit> easy enough
 71 2015-04-06 09:19:47 <wumpus> apart from that it may indeed make sense to delay responses, this avoids some kinds of harvesting
 72 2015-04-06 09:21:01 <wumpus> e.g. a drive-by getaddr wouldn't get you any addresses anymore. On the other hand, I think we use this to query the seed nodes ourselves when there are no DNS seeds.
 73 2015-04-06 09:21:30 <phantomcircuit> wumpus, seed nodes are all custom logic anyways
 74 2015-04-06 09:21:50 <wumpus> no, not really. The DNS seeds are custom logic, the normal seed nodes are simply stable nodes.
 75 2015-04-06 09:22:30 <wumpus> (those in chainparamsseeds.h)
 76 2015-04-06 09:22:47 <phantomcircuit> oh
 77 2015-04-06 09:23:35 <wumpus> in the case the DNS seeds are unreachable we rely on connecting to those, and them giving out further addresses to connect to
 78 2015-04-06 09:25:32 <wumpus> in case of a behavior change to getaddr, one of the things to verify is that that mechanism stil works
 79 2015-04-06 09:26:16 <phantomcircuit> wumpus, seems like actually trickling would cause any node that hit this logic to more than likely connect to only the fixed seed nodes
 80 2015-04-06 09:27:04 <wumpus> possibly; although it depends on how long the response is delayed
 81 2015-04-06 09:27:06 <phantomcircuit> under each rock is a new rock
 82 2015-04-06 09:27:28 <wumpus> you could e.g. reply with some addresses to getaddr directly then trickle the rest
 83 2015-04-06 09:27:56 <wumpus> (that's effectively what happens right now?)
 84 2015-04-06 09:28:37 <phantomcircuit> oh and i think the timeout logic for using the fixed seed nodes more than likely never gets triggered
 85 2015-04-06 09:28:44 <phantomcircuit> except for the first time it runs
 86 2015-04-06 09:28:50 <wumpus> it actually does
 87 2015-04-06 09:29:14 <phantomcircuit> wumpus, 100 tries in the loop below is is almost certainly going to take > 60 seconds
 88 2015-04-06 09:29:30 <phantomcircuit> unless you're not connected to the network at all in which case seed nodes do you no good
 89 2015-04-06 09:29:47 <wumpus> so what if it takes more than 60 seconds?
 90 2015-04-06 09:30:04 <phantomcircuit> then... nvm it doesn't matter
 91 2015-04-06 09:30:06 <wumpus> if it has found no connections after a while it will use those seed nodes, I'm sure that works
 92 2015-04-06 09:30:28 <wumpus> I've reached that code path many times
 93 2015-04-06 09:30:56 <phantomcircuit> yeah im not saying it doesn't im saying the 60 second limit is probably superceded by the loop below it
 94 2015-04-06 09:31:03 <phantomcircuit> but it doesn't matter
 95 2015-04-06 14:28:58 <sdaftuar> wumpus: around?
 96 2015-04-06 14:29:05 <wumpus> sort of
 97 2015-04-06 14:29:45 <sdaftuar> i have a question about software licenses -- i'm working on a pull that will build on our existing python testing framework to build a framework for p2p tests
 98 2015-04-06 14:30:05 <sdaftuar> i found it useful to grab and modify the script processing tools from python-bitcoinlib
 99 2015-04-06 14:30:09 <sdaftuar> but that is under LGPL
100 2015-04-06 14:30:23 <wumpus> LGPL is not compatible with MIT, sorry
101 2015-04-06 14:30:27 <sdaftuar> is it a non-starter to try to include that in the bitcoin project repo?
102 2015-04-06 14:30:28 <sdaftuar> ah ok
103 2015-04-06 14:31:03 <wumpus> eg: reasoning would be that it's a stricter license, so it's no longer possible to license the whole project under MIT
104 2015-04-06 14:31:26 <sdaftuar> ok, i wasn't sure if it was limited to the qa/rpc-tests directory, if that might change the answer at all
105 2015-04-06 14:32:32 <wumpus> well being part of the tests makes it something more of a grey area, but I think it's confusion best avoided
106 2015-04-06 14:33:01 <sdaftuar> got it.  i will drop it for the time being...
107 2015-04-06 14:33:08 <wumpus> it's kind of unfortunate that python-bitcoinliib is LGPL
108 2015-04-06 14:33:09 <harding> sdaftuar: IIRC, python-bitcoinlib has an older version that's MIT licensed.
109 2015-04-06 14:33:20 <wumpus> harding: hah, great
110 2015-04-06 14:33:40 <sdaftuar> interesting -- i didn't see that
111 2015-04-06 14:33:41 <wumpus> if that's true you can simply use the older version, if it has the functionality you need
112 2015-04-06 14:33:47 <sdaftuar> i will look, thanks!
113 2015-04-06 14:34:26 <harding> sdaftuar: https://github.com/jgarzik/python-bitcoinlib/blob/master/COPYING
114 2015-04-06 14:35:00 <sdaftuar> i think i need petertodd's work -- script.py
115 2015-04-06 14:35:08 <sdaftuar> oh, maybe not
116 2015-04-06 14:35:14 <sdaftuar> never mind, thank you!  i think this might be just what i need
117 2015-04-06 15:59:59 <mkarrer_> Is a locktime > 0 standard (relayed)?
118 2015-04-06 16:00:17 <belcher> only if the lock time is in the past
119 2015-04-06 16:00:33 <belcher> i.e. the miners wont store it in their mempool unless they can mine it right now
120 2015-04-06 16:01:05 <belcher> so you need to save the signed tx locally, then broadcast when the lock time passes
121 2015-04-06 16:03:41 <StephenM347> mkarrer_: locktime is only in effect if at least one of the inputs has non-maximum value, as well.
122 2015-04-06 16:03:46 <aakselrod> although, if all inputs' sequences are UINT_MAX, the transaction is considered final even if its locktime is in the future
123 2015-04-06 16:04:23 <aakselrod> er. :>
124 2015-04-06 16:09:02 <jtimon> cfields does #5975 conflict with your stuff on checkpoints? maybe makes it easier?
125 2015-04-06 16:09:38 <jtimon> coryfields_
126 2015-04-06 16:12:34 <mkarrer_> belcher, aakselrod: thanks, thats what I assumed...
127 2015-04-06 16:31:20 <maaku_> is it true that if tx.nLockTime=N, tx isn't valid until block N+1 ?
128 2015-04-06 16:36:28 <maaku> all documentation and my own understanding is that the first moment a timelocked tx can be valid is the block of height nTimeLock (ignoring timstamps)
129 2015-04-06 16:36:52 <maaku> but that doesn't seem to be what IsFinalTx is doing (< not <=)
130 2015-04-06 16:54:06 <mkarrer_> for me in regtest it does not work. I set the nTimeLock to 2 blocks in the future (getbestblockhash + 2) but I get the tx included int he block in the next block doing setgenerate true in the regtest client. are there any differences for regtest? Is isStandard enforced automatically or do I need to set it in regtest?
131 2015-04-06 16:54:40 <hearn> I think IsStandard is not enforced in regtest mode, iirc
132 2015-04-06 16:54:58 <maaku> mkarrer_: you set the sequence numbers appropriately?
133 2015-04-06 16:55:11 <maaku> hearn: this is consensus rules, not IsStandard
134 2015-04-06 16:55:33 <maaku> mkarrer_: you need to set a sequence number to be non-final (e.g. 0)
135 2015-04-06 16:55:43 <mkarrer_> no, i use BitcoinJ and dont change the sequence number from the default. but will check that, maybe thats the reason
136 2015-04-06 16:57:49 <mkarrer_> ah yeah the sequence nr. is set to max, so thats the reason, thanks for the hint!
137 2015-04-06 16:59:16 <hearn> perhaps we should add an API check for that
138 2015-04-06 17:00:03 <hearn> heh. in fact it's a TODO in the source code.
139 2015-04-06 17:03:37 <mkarrer_> yes, true
140 2015-04-06 17:54:11 <familiar> ThomasV: are you available for a quick question about electrum?
141 2015-04-06 17:54:20 <ThomasV> sure
142 2015-04-06 17:55:15 <familiar> I placed a plugin in C:\Program Files\Electrum\plugins but it doesn't show up in Tools->Plugins. It's with version 2.0.4 and windows 8. Same plugin worked with mac and linux
143 2015-04-06 17:56:07 <ThomasV> I don't use windows, sorry
144 2015-04-06 17:57:15 <familiar> Yeah me neither :/ But do you happen to know if the windows installer just runs the python files that are in the directory? Or does it prepackage everything into the .exe
145 2015-04-06 17:57:56 <ThomasV> you'd need to ask Animazing, in #electrum
146 2015-04-06 17:58:03 <familiar> Thanks!
147 2015-04-06 18:11:16 <maaku> sipa: what is the least ugly way to get the block header containing the original transaction of a CCoin?
148 2015-04-06 18:13:36 <sipa> maaku: chainActive.GetAncestor(coin.nHeight) ?
149 2015-04-06 18:16:08 <maaku> sipa: this is for a relative-locktime check in IsFinal() -- is there ever a case (e.g. reorg, mempool, ?)  where you might be able to pull a CCoins from a CCoinsViewCache but the block containing the tx is not in chainActive
150 2015-04-06 18:16:27 <sipa> no
151 2015-04-06 18:16:47 <sipa> pcoinsTip and chainActive are synchronzed
152 2015-04-06 18:17:17 <sipa> well CCoindViewCache can be anything; there are a dozen caches used
153 2015-04-06 18:17:38 <sipa> but pcoinsTip is a cache representing the current state
154 2015-04-06 18:27:19 <lewellyn> heh. that jajaj spammed a youtube link in privmsg and it is already removed due to ToS violation. :D
155 2015-04-06 18:33:53 <jtimon> is there any reason to ignore nlocktime when the seq numbers are not max? always looking at nlocktime (forget about the seq numbers) would be a softfork, right?
156 2015-04-06 18:34:28 <maaku> jtimon: no, and correct
157 2015-04-06 18:46:09 <TonzZzy> hey
158 2015-04-06 21:56:10 <phantomcircuit> maaku, sure if there's a bug
159 2015-04-06 23:12:23 <phantomcircuit> wumpus, so the current behavior of CAddrMan::GetAddr() is to return a random list of addresses which pass the !IsTerrible test
160 2015-04-06 23:12:55 <phantomcircuit> so currently an inbound peer can get us to dump more or less the entire contents of CAddrMan
161 2015-04-06 23:55:14 <phantomcircuit> it seems like the optimal solution here is for response to getaddr to be fixed for each peer
162 2015-04-06 23:55:31 <phantomcircuit> and then to randomly include our own address for a random subset of those peers
163 2015-04-06 23:56:11 <phantomcircuit> and to spontaneously push this list of addrs to all the peers on a random period such as once every hour
164 2015-04-06 23:56:36 <phantomcircuit> with the list of peers changing only when the peers addrman has changes also
165 2015-04-06 23:57:36 <phantomcircuit> and maybe do something smarter like weight the random function to preferentially select tor nodes/whitelisted nodes