1 2014-11-16 04:33:59 <truky> Hi
  2 2014-11-16 04:35:29 <truky> i need help for understand howto i can limit tranfer count of block in my nex altcoin
  3 2014-11-16 04:36:56 <justanotheruser> truky: ##altcoin-dev
  4 2014-11-16 10:07:04 <elichai2> hey
  5 2014-11-16 10:07:15 <elichai2> in what tech does the nodes talk to each other?
  6 2014-11-16 10:07:40 <elichai2> (i mean in RPC and JSON? or in something else?)
  7 2014-11-16 10:08:54 <Krellan> The nodes talk to each other in their own cute little peer-to-peer protocol
  8 2014-11-16 10:09:23 <Krellan> I finally got around to adding an option -whiteconnections=<n>
  9 2014-11-16 10:09:42 <Krellan> This gives whitelisted peers the ability to exceed the -maxconnections limit
 10 2014-11-16 10:09:53 <elichai2> Krellan, can you send me a link to how it's operating? i want to pull myself the mempool from a specific node
 11 2014-11-16 10:10:02 <elichai2> (without a client of my own)
 12 2014-11-16 10:10:25 <Krellan> Hmm, I'm not sure where the best writeup of the protocol is
 13 2014-11-16 10:10:30 <Krellan> Best bet is to check the source yourself
 14 2014-11-16 10:11:15 <elichai2> i'm not very good at C++ :\
 15 2014-11-16 10:11:21 <elichai2> i'm writing my code in python
 16 2014-11-16 10:14:03 <Krellan> Nice, I'm not good at Python
 17 2014-11-16 10:15:39 <Krellan> I don't know if there's any clear documentation of how the protocol works, other than the code itself
 18 2014-11-16 10:15:46 <elichai2> here they write a lot of theoretical things about how it's works but it's no use for me :\ https://en.bitcoin.it/wiki/Network
 19 2014-11-16 10:16:50 <elichai2> and here is the specific thing about the mempool https://en.bitcoin.it/wiki/Protocol_specification#mempool
 20 2014-11-16 10:16:57 <elichai2> but still nothing concrete
 21 2014-11-16 10:17:15 <elichai2> maybe i'll wait until the US and Canada will wake up and then there will be here some devs :)
 22 2014-11-16 10:29:06 <Krellan> Aye, me too
 23 2014-11-16 11:07:33 <truky> hi, i can help for modify code for force free to all transaction
 24 2014-11-16 11:07:56 <truky> force fee
 25 2014-11-16 11:47:26 <Krellan> Yay, it works!
 26 2014-11-16 11:50:12 <Krellan> https://github.com/bitcoin/bitcoin/pull/5288
 27 2014-11-16 15:00:18 <bitbit> Hi. I'm trying to figure out what is unique about the Blockchain. The concept of decentralized networks is not a new one, what has changed with Bitcoin and the Blockchain that makes it unique? every place I try to research, like documentries about Bitcoin, and videos about Ethereum, they always stop short of explaining the technical details. I'm a wevdev programmer and can grasp these concepts, but don't know where to start
 28 2014-11-16 15:01:52 <sipa_> bitbit: the problem that the blockchain essentially solves is decentralized consensus
 29 2014-11-16 15:02:24 <sipa_> the ability to agree on a single state with random participants in a network without trusted party
 30 2014-11-16 15:02:45 <sipa_> which was previously thought to be impossible
 31 2014-11-16 15:03:01 <bitbit> how is it achived techically
 32 2014-11-16 15:03:04 <bitbit> ?*
 33 2014-11-16 15:03:15 <sipa_> and technically, it is impossible: bitcoin does not actually solve it; it only solves it after some time, only probabilistically, and only under certain assumptions
 34 2014-11-16 15:03:29 <sipa_> the blockchain is a version of history that people may or may not agree on
 35 2014-11-16 15:03:55 <sipa_> nodes add episodes of history to it, and by making this hard, they essentially vote on accepting that version by extending it
 36 2014-11-16 15:04:14 <sipa_> sometimes two nodes disagree about what the next episode (=block) is, but this is temporary
 37 2014-11-16 15:04:30 <sipa_> the one that gets extended first later on is retroactively accepted as history
 38 2014-11-16 15:11:20 <dgenr8> elichai2: the p2p protocol has a "mempool" command to do that.  you can send it from any local language of course.
 39 2014-11-16 15:15:46 <nickler> What would you estimate is the ratio of successful malleability attacks on OP_CHECKMULTISIG (OP_0 -> OP_1, currently standard) given that the attacker has a polite number of connections (125 to 250) to well connected peers?
 40 2014-11-16 15:18:53 <nickler> In my experiment not a single one of around 100 malleated transactions successfully landed in a block. I am just wondering if this is expected.
 41 2014-11-16 15:30:09 <dgenr8> nickler: not multisig, but there were quite a lot of these confirmed recently http://respends.thinlink.com/tx/b15f3f23be26349e4a4030f3fdc9c4547d1f728440a0f5a5fc271c25cfa55afb
 42 2014-11-16 15:31:33 <bitbit> sipa_: thanks!
 43 2014-11-16 15:39:04 <nickler> dgenr8: thanks, interesting. Is this "malleability by new signature"?
 44 2014-11-16 15:44:06 <chmod755> gavinandresen, http://www.reddit.com/r/Bitcoin/comments/2mgi7w/well_known_ponzi_scheme_become_silver_member_on/cm42sxx?context=10000
 45 2014-11-16 15:53:33 <jhns> this may be a stupid question, but shouldn't the comparision in https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L4354 use ">", not "<"?
 46 2014-11-16 15:56:16 <jhns> no, sorry
 47 2014-11-16 16:05:03 <jhns> I'm not sure if I'm totally stupid right now, but "nPingUsecStart + PING_INTERVAL * 1000000" is less than GetTimeMicros() directly after the last pong has been received?
 48 2014-11-16 16:07:31 <jhns> yeah, stupid thinking error of mine. sorry
 49 2014-11-16 16:58:15 <elichai2> my question again :)
 50 2014-11-16 16:58:25 <elichai2> in what protocol does the bitcoin nodes talking?
 51 2014-11-16 16:58:44 <elichai2> (RPC with JSON? or HTTP? or something else? :) )
 52 2014-11-16 16:59:11 <dhill> elichai2: https://en.bitcoin.it/wiki/Protocol_specification
 53 2014-11-16 17:00:08 <elichai2> dhill, read that already
 54 2014-11-16 17:00:20 <elichai2> it's only theoretically
 55 2014-11-16 17:01:03 <Apocalyptic> no, it's very explicit and practical
 56 2014-11-16 17:02:39 <elichai2> Apocalyptic, I'm trying to talk with a node via python. it's just tells me how the protocol is working. not how to use it.(it's even tcp or udp? the answer is json?)
 57 2014-11-16 17:03:57 <Apocalyptic> you wouldn't ask about json if you have read that page
 58 2014-11-16 17:04:34 <elichai2> i haven't read all of it, i have a look on it
 59 2014-11-16 17:04:43 <elichai2> maybe i'll spare some time and read all of it
 60 2014-11-16 17:05:16 <dhill> elichai2: tcp
 61 2014-11-16 17:05:43 <Apocalyptic> as for TCP/UDP it's also stated
 62 2014-11-16 17:05:50 <Apocalyptic> "The ping message is sent primarily to confirm that the TCP/IP connection is still valid. An error in transmission is presumed to be a closed connection and the address is removed as a current peer."
 63 2014-11-16 17:06:15 <Apocalyptic> so before throwing up claims like " it's just tells me how the protocol is working. not how to use it", read the damn thing
 64 2014-11-16 17:06:40 <elichai2> LOL, ok sorry
 65 2014-11-16 17:07:59 <sipa_> there are plenty of python implementations of the p2p protocol
 66 2014-11-16 17:08:21 <sipa_> sounds silly to reimplememt the whole thing
 67 2014-11-16 17:08:48 <Apocalyptic> unless it's for educational purposes
 68 2014-11-16 17:11:44 <elichai2> sipa_, nothing that can communicate directly with other nodes
 69 2014-11-16 17:12:19 <elichai2> not Peter Todd's implementation and not icook's
 70 2014-11-16 18:28:57 <Luke-Jr> 
ACTION kicks Travis.. github.com: Temporary failure in name resolution

 71 2014-11-16 19:18:32 <Krellan> jhns: Bitcoin ping is different than TCP/IP ping, they're at totally different layers of the IP stack
 72 2014-11-16 19:18:45 <Krellan> Bitcoin ping is also regularly sent as a keepalive
 73 2014-11-16 19:19:08 <Krellan> I looked at that code and see nothing wrong with it, it's seeing if it has been long enough duration since last, to send again
 74 2014-11-16 19:19:23 <Krellan> Only thing I'd do differently is not call GetTimeMicros() twice in a row, call it once then save and reuse result.
 75 2014-11-16 20:01:03 <jtimon_> can we merge #4793 already, it has been there for long after replacing #4423, is there anything I can do to make it move ahead? should I squash the last commit already? should I squash them all? I know most people don't care about this but I do
 76 2014-11-16 20:02:09 <jtimon_> also, if we merge it before #5171 (also pretty old code) I can extend that one instead of having to create another PR later...
 77 2014-11-16 20:09:42 <jtimon_> jaromil I know you care about encapsulating the proof of work, can you review #4793 and #5171, please?
 78 2014-11-16 20:46:22 <BlueMatt> sipa_: re: (whatever bug it was that was complaining about 0-bytes in blk****.dat): we should actually just put in 0-bytes to make blocks start on sector boundaries
 79 2014-11-16 20:46:55 <sipa_> why?
 80 2014-11-16 20:47:00 <BlueMatt> not that it matters, but instead of fixing it to have no 0-bytes, might as well
 81 2014-11-16 20:47:12 <BlueMatt> save 1 sector-read half the time...
 82 2014-11-16 20:47:17 <BlueMatt> again, barely matters, but...why not
 83 2014-11-16 20:48:18 <sipa_> heh, you could even do it adaptively... if the size of the block on disk is $SECTORSIZE*Y+X, allow inserting up to $SECTORSIZE-X 0 bytes
 84 2014-11-16 20:49:04 <BlueMatt> heh, one could
 85 2014-11-16 21:25:00 <jhns> Krellan: yes, I know. I was just confused at the calculation
 86 2014-11-16 21:29:38 <jhns> Krellan: I just thought wrong ;)
 87 2014-11-16 21:35:22 <Krellan> jhns: no problem
 88 2014-11-16 21:35:58 <jhns> thanks for looking at it though :)
 89 2014-11-16 21:51:47 <Krellan> OK, I tested it and it's running well, added warning/debugging messages also.
 90 2014-11-16 21:51:48 <Krellan> https://github.com/bitcoin/bitcoin/pull/5288
 91 2014-11-16 21:52:08 <Krellan> This adds -whiteconnections=<n> which lets you exceed -maxconnections but only for whitelisted peers.
 92 2014-11-16 21:52:38 <sipa_> Krellan: will review
 93 2014-11-16 21:55:55 <Krellan> sipa_: Nice, thanks!
 94 2014-11-16 21:55:58 <Krellan> Appreciate that.
 95 2014-11-16 21:56:47 <Krellan> It's useful for my p2pool node, then you don't get the failure to connect if your bitcoind has been up for a long time and all of its slots are already taken.
 96 2014-11-16 21:57:20 <gmaxwell> Krellan: I'd rather just reserve some of the max connections slots for whitelisted peers. Freely exceeding maxconnections will cause memory corruption.
 97 2014-11-16 21:57:30 <gmaxwell> (e.g. if it makes you use more than the FD limit)
 98 2014-11-16 21:57:34 <sipa_> gmaxwell: read the patch
 99 2014-11-16 21:58:10 <gmaxwell> ah okay
100 2014-11-16 21:58:53 <gmaxwell> sipa_: you're wearing a cape.
101 2014-11-16 21:59:04 <sipa_> ...?
102 2014-11-16 21:59:10 <gmaxwell> sipa_
103 2014-11-16 21:59:13 <Krellan> gmaxwell: Thanks, I thought of that as well from our earlier talk about it
104 2014-11-16 21:59:40 <Krellan> I silently boost maxconnections to the greater of (maxconnections, whiteconnections)
105 2014-11-16 21:59:55 <gmaxwell> perhaps this should default to having a few whiteconnections over the max?
106 2014-11-16 21:59:57 <Krellan> and it's still subject to all the same system FD counts
107 2014-11-16 22:00:24 <Krellan> The test for whitelist is done at connection time. Essentially there's a smaller maxconnections that non-whitelisted peers are subjected to, it's smaller than the "real" maxconnections.
108 2014-11-16 22:01:36 <gmaxwell> hm. So this becomes ineffectie when you run into the limit, which might be unwelcome.
109 2014-11-16 22:02:06 <gmaxwell> E.g. if you set maxconnections to 2048 you'll end up with no whitelist reserved connections... at least you warn though.
110 2014-11-16 22:02:16 <Krellan> And if you request something goofy like 100000 connections, you're still clamped to the system max. On my linux it's 873.
111 2014-11-16 22:03:19 <Krellan> True. Part of it is that I didn't want to change any existing semantics of -maxconnections because there's a lot of users using it already.
112 2014-11-16 22:05:25 <Krellan> Maybe -whiteconnections parameter should work subtractive from -maxconnections, instead of establishing a new total limit?
113 2014-11-16 22:05:41 <gmaxwell> Krellan: Perhaps it would be less confusing though if setting your white too high made you lose all your inbound sockets.
114 2014-11-16 22:05:49 <Krellan> so -maxconnections=100 -whiteconnections=20 --> outside world gets 80 connection slots, but overall there's a limit of 100 connection slots
115 2014-11-16 22:06:03 <gmaxwell> Yea, I think the fd limit should be reserved in this order:  outbound (limited to 8 in any case), white, then non-white inbound.
116 2014-11-16 22:06:46 <gmaxwell> hm. yea, and -whiteconnections is "reserve this many inbound connections for whitelisted peers".
117 2014-11-16 22:06:55 <Krellan> OK, good point!
118 2014-11-16 22:07:01 <Krellan> That would improve semantics somewhat.
119 2014-11-16 22:07:24 <Krellan> Will do that.
120 2014-11-16 22:07:33 <gmaxwell> We could also make that default to, say, 8 and increase the default maxconnections by 8... so people shouldn't need to fuss with that knob to benefit from it.
121 2014-11-16 22:08:08 <Krellan> Also, I've been curious, what is the magic of the number 8 for outbound connections? It's hardcoded everywhere, seemingly.
122 2014-11-16 22:08:45 <gmaxwell> or maybe 4.  I guess you can't avoid slightly breaking some people who've set maxconnections very low.
123 2014-11-16 22:08:56 <Krellan> If we make default to reserve 8 connections for whitelisted, should maxconnections default be also raised to 133, or should it stay 125? Anything significant about the number 125?
124 2014-11-16 22:09:36 <Krellan> I set my maxconnections pretty tight, my DSL was running with -maxconnections=12 for a while (only 4 allowed incoming)
125 2014-11-16 22:09:54 <gmaxwell> Krellan: it's not really magic, it has reasonable good properties without being an enormous number.   We shouldn't increase the connection count for the default configuration for most users. (e.g. ones without whitelisted peers)... simply to avoid using more resources.
126 2014-11-16 22:11:11 <Krellan> Aye. that's why I default to not reserving any extra at all.
127 2014-11-16 22:11:42 <gmaxwell> I don't mind increasing it some for those which are actually using whitelisting, since they're in a non-standard configuration... and we've reduced resource usage per connection a lot in the last couple years.
128 2014-11-16 22:14:29 <Krellan> Makes sense.  So if whitelisted connections are allowed 8 extra, for example, then maxconnections would silently default to 133 and not 125 (but still be capped at 125 for non-whitelisted).
129 2014-11-16 22:16:26 <gmaxwell> yea. e.g. so that would be -whiteconnections=8  ("8 connections reserved for whitelisted peers") and -maxconnections=133 (which is effectively 125 because of the reserve).  And whitelisted is first capped at fd_limit-outbound.. so if you set whitelisted to 99999 you just get 8 out, limit-8 white, and no non-white inbound.
130 2014-11-16 22:17:34 <Krellan> Makes sense to me. Will go and do that.
131 2014-11-16 22:17:39 <Krellan> Thanks for review!
132 2014-11-16 22:19:18 <BlueMatt> sipa: also, aligning blocks to sector starts instead of randomly makes block-level deduplication work
133 2014-11-16 22:19:34 <BlueMatt> 
ACTION knew there was a better reason for doing that that he was thinking of last night

134 2014-11-16 22:24:07 <gmaxwell> BlueMatt: what actual level of alignment is needed for that? Wasting 4k per block adds 1.5GB to the block files in the worst case, which seems pretty bad.
135 2014-11-16 22:24:26 <BlueMatt> nfc, but i figure alignment is needed
136 2014-11-16 22:24:43 <gmaxwell> we do get alignment from the new block files.
137 2014-11-16 22:25:06 <BlueMatt> true, but that is pretty rare (how many blocks do we shove in each file?)
138 2014-11-16 22:26:04 <gmaxwell> yea, it's rare though it would be completely effective if we had some process to repack old block files at startup, since they'd all end up file wise identical.
139 2014-11-16 22:26:19 <BlueMatt> yup
140 2014-11-16 22:26:54 <BlueMatt> problem is, headers-first is gonna make such dedup much more difficult because, previously, where blocks were almost always packed (ie during ibd), now they never are
141 2014-11-16 22:26:59 <gmaxwell> e.g. if at startup we look for the oldest block file with orpahns which is at least two blockfiles back and repack it and the block before it...
142 2014-11-16 22:27:12 <gmaxwell> ah right, reordering too.
143 2014-11-16 22:27:18 <BlueMatt> we should do that, if only it werent hard with -txindex, no?
144 2014-11-16 22:27:59 <BlueMatt> I mean, you can update the -txindex too, but if you only have to touch the mapBlockIndex entries, its not too bad
145 2014-11-16 22:30:33 <gmaxwell> Irritating. yea, txindex should use mapBlockIndex to get the positions.
146 2014-11-16 22:31:08 <BlueMatt> yea, block_start+block_offset is easy enough...
147 2014-11-16 22:31:23 <gmaxwell> e.g. it should be a blockindex and a offset in the block. Then look up the block position using mapBlockIndex. I don't think this would have a measurable performance impact since the blockmap is already in memory.
148 2014-11-16 22:31:37 <BlueMatt> jinx
149 2014-11-16 22:32:12 <gmaxwell> then we could repack the block files so that they were eventually the same on all hosts and free of orphans.
150 2014-11-16 22:32:23 <BlueMatt> yupyup
151 2014-11-16 22:32:58 <BlueMatt> we could force txindex recalculate on 0.10 load....
152 2014-11-16 22:33:24 <gmaxwell> well there are perhaps other reasons to do that. (leveldb bloom filter format changing)
153 2014-11-16 22:33:47 <BlueMatt> then offline/init-option block compress could come later, or even in python script
154 2014-11-16 22:33:53 <BlueMatt> hmm, indeed
155 2014-11-16 22:34:06 <gmaxwell> right now there is a presumably-big performance hit from upgrading to 0.10 which probably takes fairly long time to self correct.
156 2014-11-16 22:34:30 <gmaxwell> on the chainstate we'd reasoned that it would correct pretty fast, but that reasoning probably is less true for txindex.
157 2014-11-16 22:34:58 <gmaxwell> we have some other needs to reindex things though. E.g. corrupted version information in utxos.
158 2014-11-16 22:35:01 <BlueMatt> well, good opportunity to slightly tweak the on-disk format then
159 2014-11-16 22:36:00 <BlueMatt> huh? we have corrupted version information in utxos?
160 2014-11-16 22:36:23 <gmaxwell> yea, the seralizer for versions doesn't correctly handle negative version numbers.
161 2014-11-16 22:36:38 <BlueMatt> ahh, I do recall seing code working around that
162 2014-11-16 22:36:50 <gmaxwell> there is explicit code in the start up sanity check to work around that (basically be letting the utxo database be wrong)
163 2014-11-16 22:36:55 <gmaxwell> s/be/by/
164 2014-11-16 22:49:16 <maraoz> I read Mike Hearn wrote the original wiki article on micropayment channels. Did he also create the concept?
165 2014-11-16 22:54:18 <livegnik> Hi everyone. Would it be a proper question in this channel for me to ask some advice on where to find senior devs, familiar with bitcoind / C++ (& SQL, javascript, node.js, DHT, P2P) who'd like to work on an infrastructure-project?
166 2014-11-16 22:55:31 <yolandiVisser> livegnik: #bitcoin-otc would probably be better
167 2014-11-16 22:55:52 <livegnik> yolandiVisser: Much appreciated! Thanks :)