1 2013-06-12 00:33:11 <undecim> I'm interested in creating a synchronized decentralized database (like bitcoin), and think that an alternative block chain (like NameCoin) is the way to go, but I'm having a little trouble understanding the technical aspects of it...
2 2013-06-12 00:35:04 <undecim> So using namecoin as an example... There is a deterministic set of rules dictating what is a valid database transaction. And for a transaction to be valid, its hash to be included in a bitcoin transaction?
3 2013-06-12 00:35:32 <undecim> Doesn't that require a transaction fee?
4 2013-06-12 00:40:40 <Luke-Jr> undecim: Namecoin was implemented independent of Bitcoin entirely, though you could potentially do that
5 2013-06-12 00:40:51 <Luke-Jr> undecim: and yes, any system will need some kind of anti-spam measure
6 2013-06-12 00:46:27 <undecim> I'm just trying to get my head around how you can do POW for two different sets of data without doing double the work
7 2013-06-12 00:46:59 <Luke-Jr> merkle trees
8 2013-06-12 00:47:29 <gmaxwell> undecim: I offer you a question as an answer: how does the POW cover second byte of a transaction as well as the first without double the effort?
9 2013-06-12 00:47:48 <Diablo-D3> gmaxwell: instruction parallelism?
10 2013-06-12 00:48:27 <undecim> gmaxwell: Well each byte is part of the same hash operation. You can't get that hash unless you include EVERY bit used in the hash
11 2013-06-12 00:48:53 <gmaxwell> undecim: right.
12 2013-06-12 02:38:50 <zw> the kitco gold live mobile app has bitcoin prices from mtgox exchange listed under oil & indices as GOX, yeah it's promoting the exchange but seems catchy
13 2013-06-12 09:41:14 <warren> TheUni: btw, make still doesn't take advantage of multiple cores
14 2013-06-12 09:46:11 <tumak_> warren: -j works for me
15 2013-06-12 09:46:29 <warren> tumak_: talking about his repo for gitian
16 2013-06-12 09:46:40 <tumak_> ah
17 2013-06-12 09:47:11 <warren> heh. irc.lfnet.net: #bitcoin-dev 1 We've been slashdotted! Yay! | http://bitcoin.org/ - Bitcoin Development - We're here to help support the Bitcoin system and its use.
18 2013-06-12 09:47:38 <tumak_> recorded history :)
19 2013-06-12 09:47:47 <warren> that's still the topic there
20 2013-06-12 09:47:52 <tumak_> yup
21 2013-06-12 09:47:57 <warren> there's one person in the channel
22 2013-06-12 09:48:36 <epscy> you?
23 2013-06-12 09:48:51 <coingenuity> so, who's familiar with bitcoin's leveldb impl
24 2013-06-12 09:48:55 <warren> "solar"
25 2013-06-12 09:49:28 <tumak_> warren: 13:47 -!- Channel #bitcoin-dev created Fri Jun 18 16:41:08 2010
26 2013-06-12 09:49:47 <tumak_> recorded history :)
27 2013-06-12 09:50:27 <sipa> coingenuity: i guess i qualify
28 2013-06-12 09:51:30 <tumak_> grau: just a bit taunting (repeating stuff somebody already repeated on forums is actually quite common). follow the context wrt 'first rule of bitcoin-dev'
29 2013-06-12 09:52:03 <epscy> sipa: how feasible would it be to write a SQL query interface to the leveldb blockchain?, is it possible to do with reasonable performance?
30 2013-06-12 09:52:06 <coingenuity> sipa: you qualify a lot, actually :D
31 2013-06-12 09:52:20 <coingenuity> sipa: so i'm looking at the /chainstate dir's leveldb
32 2013-06-12 09:52:27 <coingenuity> can't figure out what data is encoded in
33 2013-06-12 09:52:36 <coingenuity> any idear?
34 2013-06-12 09:52:36 <sipa> coingenuity: the unspent transaction output set
35 2013-06-12 09:52:48 <sipa> epscy: leveldb is a key-value store, not an sql database
36 2013-06-12 09:53:00 <coingenuity> i mean, i dont know what character encoding it's using
37 2013-06-12 09:53:04 <epscy> sipa: yes i know
38 2013-06-12 09:53:20 <sipa> coingenuity: there is no text in it, there is no character encoding
39 2013-06-12 09:54:14 <epscy> sipa: i'm wondering if it is possible to do that kind of querying in place (without modifying or extracting the data), with reasonable performance
40 2013-06-12 09:54:38 <sipa> epscy: what kind of query?
41 2013-06-12 09:54:51 <coingenuity> when you say output set, what kind of data is that?
42 2013-06-12 09:54:51 <sipa> epscy: there is the gettxout RPC that allows you to query the chainstate
43 2013-06-12 09:54:56 <epscy> sipa: the kinds blockchain.info does
44 2013-06-12 09:55:15 <sipa> epscy: it's not indexed, only direct lookups of a given txid are possible
45 2013-06-12 09:55:17 <epscy> sipa: i have an address, find me all transactions that the address was involved in
46 2013-06-12 09:55:19 <sipa> coingenuity: transaction outputs
47 2013-06-12 09:55:22 <sipa> epscy: NAK
48 2013-06-12 09:55:30 <epscy> sipa: ok, as i thought
49 2013-06-12 09:55:40 <epscy> thanks
50 2013-06-12 09:56:40 <epscy> sipa: so theoretically, could you build a separate index and use that?, or at this point are you better off just extracting the data?
51 2013-06-12 09:56:56 <sipa> epscy: first of all, are you talking about the blockchain or the utxo set?
52 2013-06-12 09:57:02 <epscy> blockchain
53 2013-06-12 09:57:37 <sipa> well we have no blockchain index whatsoever, except for the block position on disk for each block
54 2013-06-12 09:58:08 <sipa> unless you enable -txindex=1, in which case also the disk position for each individual tx is present
55 2013-06-12 09:58:40 <sipa> epscy: imho, any application that relies on having a full address-to-txid index, is broken
56 2013-06-12 09:58:47 <sipa> epscy: though it's very useful for debugging :)
57 2013-06-12 09:58:53 <epscy> right so basically to do what i want i would need to add another index
58 2013-06-12 10:00:08 <epscy> i realise the data is all in a format that is most efficient for the bitcoin client, I am just interested in leveraging it for blockchain analysis
59 2013-06-12 10:00:25 <epscy> and trying to work out how possible that is
60 2013-06-12 10:00:30 <coingenuity> epscy: same
61 2013-06-12 10:00:33 <sipa> i once made a patch that added an address-to-txid index, but i didn't like the implementation
62 2013-06-12 10:00:49 <sipa> and i'm not convinced it belongs in bitcoind anyway
63 2013-06-12 10:00:55 <epscy> sipa: oh i agree
64 2013-06-12 10:01:17 <epscy> i'm trying to work out if i could write something external, that operates on the data
65 2013-06-12 10:01:28 <sipa> using RPC it would be extremely slow
66 2013-06-12 10:01:33 <sipa> ask phantomcircuit :p
67 2013-06-12 10:01:49 <epscy> well i was thinking it would read the leveldb files
68 2013-06-12 10:02:02 <epscy> would read only access interfere with the client?
69 2013-06-12 10:02:05 <sipa> the leveldb files don't contain any information you want
70 2013-06-12 10:02:14 <epscy> oh
71 2013-06-12 10:02:26 <epscy> where are the blocks stored?
72 2013-06-12 10:02:33 <epscy> blocks and transactions
73 2013-06-12 10:02:34 <sipa> in the blocks/blk*.dat files
74 2013-06-12 10:02:39 <sipa> unindexed
75 2013-06-12 10:02:43 <epscy> i see
76 2013-06-12 10:02:53 <epscy> again thanks for your help
77 2013-06-12 10:02:57 <sipa> you can parse those just fine; armory does that, for example
78 2013-06-12 10:03:00 <sipa> they're append-only
79 2013-06-12 10:03:42 <sipa> coingenuity: i still don't know what you want to know
80 2013-06-12 10:04:13 <sipa> coingenuity: the chainstate database contains for each txid which is not yet completely spent, all unspent output and a tiny bit of transaction metadata
81 2013-06-12 10:04:34 <sipa> coingenuity: in a custom designed efficient format
82 2013-06-12 10:04:47 <coingenuity> well, i understand what that format is, basically :X
83 2013-06-12 10:04:53 <coingenuity> er, am trying to*
84 2013-06-12 10:05:08 <sipa> you mean the format of the values?
85 2013-06-12 10:05:17 <coingenuity> it just looks like junk data when i iterate through the db itself
86 2013-06-12 10:05:29 <sipa> coingenuity: sec
87 2013-06-12 10:05:30 <coingenuity> i could paste some here but i dont want to make everyone vomit :D
88 2013-06-12 10:05:42 <sipa> https://github.com/bitcoin/bitcoin/blob/master/src/core.h#L425
89 2013-06-12 10:06:12 <tumak_> epscy: why not use -txindex?
90 2013-06-12 10:06:41 <sipa> tumak_: that still only given you a txid-based index, not an address-based one
91 2013-06-12 10:07:09 <coingenuity> sipa: yeah, it doesn't look like ASCII as it does in the examples
92 2013-06-12 10:07:15 <grau> epscy: I have API to query leveldb by address
93 2013-06-12 10:07:16 <sipa> coingenuity: ascii???
94 2013-06-12 10:07:27 <sipa> coingenuity: the examples show the data in hex
95 2013-06-12 10:07:34 <sipa> coingenuity: on disk it's binary of course
96 2013-06-12 10:07:41 <tumak_> sipa: oic
97 2013-06-12 10:08:23 <tumak_> was curious how electrum-server does it, external index?
98 2013-06-12 10:08:39 <coingenuity> ah, that helps clear things up a bit, ty
99 2013-06-12 10:08:41 <sipa> i think so, yes
100 2013-06-12 10:10:34 <epscy> grau: i'm interested in the whole blockchain
101 2013-06-12 10:10:39 <jgarzik> word
102 2013-06-12 10:10:50 <grau> epscy: thats what i have
103 2013-06-12 10:11:06 <coingenuity> ah, beautiful
104 2013-06-12 10:11:09 <epscy> grau: is it on github?
105 2013-06-12 10:11:09 <grau> the full blockchain in leveldb queriable by andy data element in scripts
106 2013-06-12 10:11:16 <coingenuity> thats great sipa
107 2013-06-12 10:11:19 <sipa> grau: how large is that?
108 2013-06-12 10:11:22 <coingenuity> i seem to have some output from leveldb now
109 2013-06-12 10:11:23 <coingenuity> :)
110 2013-06-12 10:11:27 <grau> 11.8GB
111 2013-06-12 10:11:34 <sipa> that's quite good
112 2013-06-12 10:11:55 <epscy> grau: interesting
113 2013-06-12 10:12:09 <grau> https://github.com/bitsofproof/supernode
114 2013-06-12 10:13:26 <epscy> grau: thanks, i will take a look later
115 2013-06-12 10:14:10 <grau> You can query my servers without the hassle to set up yours if thats your preference
116 2013-06-12 10:20:58 <tumak_> grau: do you actually evaluate prevtx stack result or just look for op_dup op_hash 160 pubkeyhash sequence?
117 2013-06-12 10:21:31 <tumak_> (proper adress-tx mapping implies full script chain verify)
118 2013-06-12 10:22:18 <grau> the query is more generic. Not by address but by any data element in script. That might be key or script hash or address or....
119 2013-06-12 10:23:04 <tumak_> ah
120 2013-06-12 10:23:12 <tumak_> so you index script contents
121 2013-06-12 10:23:23 <grau> it is also more of a scan than a query since it optionally picks up not only tx matching the criteria but tx spending a tx that matches the criteria
122 2013-06-12 10:24:15 <grau> it is an optimized query to get series of transactions in single pass that have some property and build on each other.
123 2013-06-12 10:25:12 <grau> since the most relevant use case for such a query is to figure a balance for an address that requires not only spends to the address but tx spending those.
124 2013-06-12 10:26:33 <grau> feel free to repeat whatever I said, I wont call it plagiarism
125 2013-06-12 10:30:27 <grau> tha scan is however supported by probabilistic indices (bloom filter)
126 2013-06-12 10:33:48 <sipa> grau: then what is stored in leveldb?
127 2013-06-12 10:34:00 <grau> everything
128 2013-06-12 10:34:19 <grau> key is tx or block hash (with an added byte)
129 2013-06-12 10:34:43 <grau> value is serialized in protobuf format
130 2013-06-12 10:34:58 <grau> tx a tx serialization
131 2013-06-12 10:35:01 <sipa> and for blocks, you list the txids?
132 2013-06-12 10:35:25 <grau> block is block header + tx hash list + bloom filter computed out of script data
133 2013-06-12 10:35:32 <sipa> ic!
134 2013-06-12 10:36:51 <grau> I keep the bloom filter in mem for queries and parse positive blocks to evaluate match
135 2013-06-12 10:41:43 <tumak_> grau: so if i need to query from set of 100M addresses, i need to keep ~5MB bloom filter for each block (just for pubkeyhash)?
136 2013-06-12 10:42:05 <tumak_> (not very space efficient, but the generalism of this solution is neat)
137 2013-06-12 10:42:27 <grau> no, there is a bloom filter per block that helps you to determine if the block contains a transaction you care
138 2013-06-12 10:42:56 <grau> a single filter is <32kb usually much less
139 2013-06-12 10:43:56 <tumak_> ah, i see
140 2013-06-12 10:44:06 <tumak_> so when i have hash160 and i want to get all txes in a block matching it
141 2013-06-12 10:44:18 <tumak_> i have to query all the blocks, because the hash function cannot be stable, hmm
142 2013-06-12 10:44:51 <grau> you have to query the filter of all blocks to figure which blocks contain a tx you care
143 2013-06-12 10:45:18 <grau> those queries hoewver are all in memory
144 2013-06-12 10:45:26 <tumak_> yup
145 2013-06-12 10:45:42 <grau> and can be furthe roprimized since you ask the same hash from all of them
146 2013-06-12 10:45:56 <tumak_> still not convinced this is faster than old fashioned hash index (which can be in memory as well)
147 2013-06-12 10:46:26 <grau> A hash index on what?
148 2013-06-12 10:47:40 <tumak_> take last 20 bits and use it as offset to skiplist
149 2013-06-12 10:47:48 <tumak_> (once again, highly specialized db)
150 2013-06-12 10:48:05 <tumak_> but bit-exact, i imagine bloom filter could be quite efficient assuming you mantain reasonable fanout
151 2013-06-12 10:50:14 <grau> Since there are tens of millions of data elements, you can not build an exact index, but some sort of hash map
152 2013-06-12 10:51:04 <grau> the block itself is already a skip list of transactions and my filter just identify where to look into
153 2013-06-12 10:52:06 <tumak_> grau: you just add a bit when collision happens :)
154 2013-06-12 10:52:12 <tumak_> thats how you do append-only indexes
155 2013-06-12 10:52:30 <grau> the other reason is that you do not really need random access but a supported scan
156 2013-06-12 10:52:33 <tumak_> abusing blockchain itself is neat, though i'm worried about trashing
157 2013-06-12 10:53:08 <tumak_> uh, supported scan?
158 2013-06-12 10:53:16 <tumak_> the bloom filter just points at random places
159 2013-06-12 10:53:19 <grau> imagine you locate a transaction that sends to an address, now you would want to find the next tx spending it
160 2013-06-12 10:53:28 <tumak_> and you have to check half of them on average
161 2013-06-12 10:53:30 <grau> it is inefficient to look for that behind the first
162 2013-06-12 10:53:32 <tumak_> :)
163 2013-06-12 10:53:32 <tumak_> i guess thats not linear at all
164 2013-06-12 10:53:43 <tumak_> (as far as disk is concerned)
165 2013-06-12 10:55:08 <tumak_> grau: you can query only single branch of btree as well :)
166 2013-06-12 10:55:21 <grau> the filter identifies about 10 out of 240,000 blocks for an address.
167 2013-06-12 10:55:24 <tumak_> needless to say i'll give your code a try, i'm fairly impressed
168 2013-06-12 10:55:38 <grau> I think you do not get what the use case is
169 2013-06-12 10:55:59 <tumak_> grau: when i put 1dice address there
170 2013-06-12 10:56:02 <grau> Usually you look for a transaction sequence
171 2013-06-12 10:56:04 <tumak_> i must get all blocks
172 2013-06-12 10:56:19 <tumak_> :)
173 2013-06-12 10:56:19 <tumak_> since early 2012 or so
174 2013-06-12 10:56:24 <tubby2> Hello, can anyone help me with a BitcoinX question http://pastebin.com/7ZDKsZQF ?
175 2013-06-12 10:56:42 <grau> yes you get all blocks for 1dice since it started
176 2013-06-12 10:57:01 <grau> that would be similarly edge case for any other type of index
177 2013-06-12 10:57:39 <grau> the point is that you usually look for the first tx matching a criteria then txs that use that output and so on.
178 2013-06-12 10:57:55 <grau> so queries are rather scans then random access
179 2013-06-12 10:58:14 <grau> filter supported scans
180 2013-06-12 11:04:23 <tubby2>
181 2013-06-12 11:04:23 <tubby2> I am currently testing BitoinX (Armory with color definitions). If I didn't get it wrong, nothing prevents me from creating a color definition with a genesis transaction not related at all to me. Is this a desired feature?
182 2013-06-12 11:04:23 <tubby2> Maybe the colors' id should be signed by the account that collected the inputs?
183 2013-06-12 11:05:28 <tubby2> account(s)*
184 2013-06-12 11:06:54 <nsh> grau, what is the effect of false positives from the bloom filter on your queries? are they just discarded?
185 2013-06-12 11:07:21 <nsh> (so, efficiency loss, but no accuracy loss)
186 2013-06-12 11:07:34 <grau> nsh: yes, its efficiency loss
187 2013-06-12 11:07:39 <nsh> ok
188 2013-06-12 11:08:08 <nsh> ty
189 2013-06-12 11:08:25 <tumak_> tubby2: the trick is, nobody should install rogue colordefs :)
190 2013-06-12 11:09:54 <tumak_> because you could also create one with same name, the hash of issue is identity of issuer, not genesis
191 2013-06-12 11:11:29 <tubby2> Can't a malicious user create a definition to tamper with another one? Doesn't the color id involve a hashing of all its values (including genesis transaction)?
192 2013-06-12 11:11:39 <tumak_> yup
193 2013-06-12 11:12:13 <tumak_> simply use colordefs you trust and only those
194 2013-06-12 11:12:31 <tumak_> they can be alternatively signed by key
195 2013-06-12 11:12:45 <tumak_> and then you just trust the key, simplyfying the trust management
196 2013-06-12 12:07:40 <jgarzik> CheckForkWarningConditionsOnNewFork()
197 2013-06-12 12:07:44 <jgarzik> ACTION looks at BlueMatt
198 2013-06-12 12:07:54 <BlueMatt> wat?
199 2013-06-12 12:08:12 <BlueMatt> ACTION has been writing too much java...it shows
200 2013-06-12 12:08:24 <jgarzik> ThatSureIsALongFunctionNameItsAlmostASentence
201 2013-06-12 12:08:26 <jgarzik> :)
202 2013-06-12 12:08:40 <sipa> just add an AbstractFactorySingletonVisitorAdapter
203 2013-06-12 12:08:45 <tumak_> i think it's time bitcoin switches to check_for_warning_conditions_on_new_fork()
204 2013-06-12 12:08:57 <BlueMatt> youre missing a k
205 2013-06-12 12:09:35 <tumak_> CamelCaseIncreasedIdentifierLengthRelation
206 2013-06-12 12:09:53 <tumak_> BlueMatt: btw, should it be checkForWarning?
207 2013-06-12 12:10:03 <BlueMatt> not in bitcoin
208 2013-06-12 12:10:12 <tumak_> ah
209 2013-06-12 12:10:20 <BlueMatt> our coding standards are straight out of the 90s
210 2013-06-12 12:10:24 <BlueMatt> well...maybe 80s
211 2013-06-12 12:10:24 <sipa> MethodsAreLikeThis, CClassesAreLikeThink strVariablesAreLikeThis
212 2013-06-12 12:10:25 <TD> sipa: if you want a laugh do an internal code search for .*FactoryFactory.*
213 2013-06-12 12:11:03 <sipa> TD: i expected worse :D
214 2013-06-12 12:11:06 <tumak_> sipa: oh i see, there is that lovely C in front of case, instead of capital
215 2013-06-12 12:11:13 <tumak_> s/case/class/
216 2013-06-12 12:15:39 <jgarzik> it's not CamelCase
217 2013-06-12 12:15:42 <jgarzik> it's StudlyCaps
218 2013-06-12 12:16:01 <sipa> TD: WrapperWrapper is also fun
219 2013-06-12 12:16:21 <TD> lol
220 2013-06-12 12:16:56 <TD> also works for AdapterAdapter
221 2013-06-12 12:16:58 <BlueMatt> RapperInfoStorageFactoryFactoryFactoryWrapper
222 2013-06-12 12:17:16 <TD> although in fairness there is only one adapteradapter
223 2013-06-12 12:17:32 <tumak_> i thought CamelCase was invented by case insensitive languages, like pascal
224 2013-06-12 12:17:34 <tumak_> it was great there
225 2013-06-12 12:17:52 <tumak_> everything went avry when people got it wrong in c++ and java :(
226 2013-06-12 12:18:11 <tumak_> s/av/aw/
227 2013-06-12 12:19:21 <sipa> (Wrapper|Adapter|Factory|Singleton|Visitor){3}
228 2013-06-12 12:20:28 <TD> hahaha
229 2013-06-12 12:20:29 <TD> good one
230 2013-06-12 12:20:52 <TD> i like the TreeTableWrapperFactorySingleton
231 2013-06-12 12:21:03 <jgarzik> heh
232 2013-06-12 12:21:42 <jgarzik> ACTION regenerates the blockchain torrent @ height 238000
233 2013-06-12 12:21:59 <jgarzik> beyond the last checkpoint, but that's OK I think
234 2013-06-12 12:22:21 <BlueMatt> can you give me like an rss so I dont have to reset my seedbox when a new one comes out?
235 2013-06-12 12:23:01 <jgarzik> BlueMatt, how would that work, ideally?
236 2013-06-12 12:23:15 <jgarzik> BlueMatt, should be able to swap out old .torrent for new, and the existing file is just appended
237 2013-06-12 12:23:16 <kjj> jgarzik: I was thinking about automating the torrents
238 2013-06-12 12:23:39 <BlueMatt> jgarzik: for that to work, someone would have to write code to handle it
239 2013-06-12 12:23:45 <BlueMatt> ACTION nomiates jgarzik to be that someone
240 2013-06-12 12:23:49 <jgarzik> heh
241 2013-06-12 12:23:56 <jgarzik> kjj, cool
242 2013-06-12 12:24:00 <jgarzik> automation good
243 2013-06-12 12:24:24 <nsh> why isn't -logtimestamps default?
244 2013-06-12 12:24:32 <kjj> jgarzik: if we pad the block files to the torrent piece size, we can replace the torrents without invalidating the old ones
245 2013-06-12 12:24:53 <tumak_> wont report as "finished" though
246 2013-06-12 12:24:54 <sipa> nsh: satoshi was concerned about privacy issues with that, if log files aren't well protected
247 2013-06-12 12:25:00 <nsh> hmm
248 2013-06-12 12:25:09 <nsh> i suppose
249 2013-06-12 12:25:10 <tumak_> kjj: how much to pad? 1gb 10gb?
250 2013-06-12 12:25:27 <kjj> tumak_: to the piece size
251 2013-06-12 12:25:35 <tumak_> huh?
252 2013-06-12 12:25:37 <tumak_> i dont follow
253 2013-06-12 12:25:48 <jgarzik> kjj, not sure I follow. what is invalidated? the current torrent setup works fine, with replacement.
254 2013-06-12 12:25:54 <jgarzik> kjj, with no padding
255 2013-06-12 12:26:01 <tumak_> yup, just redownload torrent
256 2013-06-12 12:26:08 <tumak_> it catches up the previous file
257 2013-06-12 12:26:13 <kjj> the hash of the final piece changes unless you pad it out to zero
258 2013-06-12 12:26:37 <tumak_> not all clients support padding btw
259 2013-06-12 12:26:55 <jgarzik> The announcement comes after a while of testing and PGP signing and whatnot, but,
260 2013-06-12 12:27:02 <jgarzik> this is the updated torrent: http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent
261 2013-06-12 12:27:04 <kjj> you pad the block file. if there are block parsers that can't handle padding, they fail and need to be fixed
262 2013-06-12 12:27:20 <tumak_> i dont know what block file is
263 2013-06-12 12:27:25 <BlueMatt> sipa: no, it was after satoshi
264 2013-06-12 12:27:32 <tumak_> kjj: do you mean bootstrap.dat?
265 2013-06-12 12:27:36 <jgarzik> kjj, why?
266 2013-06-12 12:27:40 <kjj> tumak_: yes, the bootstrap.dat file
267 2013-06-12 12:27:45 <tumak_> why to pad it?
268 2013-06-12 12:27:53 <jgarzik> Block files can of course be zero padded; bitcoind does this.
269 2013-06-12 12:27:55 <tumak_> you need to republish the torrent anyway, since infohash changes :(
270 2013-06-12 12:28:11 <kjj> so that each replacement torrent is a strict extension of the last
271 2013-06-12 12:28:25 <kjj> with the current replacement, the hash of the last piece changes
272 2013-06-12 12:28:29 <sipa> BlueMatt: ?
273 2013-06-12 12:28:31 <tumak_> jgarzik: frankly, using something like bootstrap1.dat, bootstrap2.dat etc would be probably for the best
274 2013-06-12 12:28:40 <tumak_> jgarzik: saves the need for rehash
275 2013-06-12 12:28:47 <BlueMatt> sipa: logtimestamps discussion happened A.S.
276 2013-06-12 12:28:48 <sipa> how about blk00000.dat, blk00001.dat, ...?
277 2013-06-12 12:28:54 <sipa> BlueMatt: sure
278 2013-06-12 12:29:05 <tumak_> sipa: those are in wire format? :)
279 2013-06-12 12:29:11 <sipa> tumak_: yes
280 2013-06-12 12:29:20 <BlueMatt> <sipa> nsh: satoshi was concerned about privacy issues with that, if log files aren't well protected
281 2013-06-12 12:29:28 <BlueMatt> no, satoshi wasnt, he was gone
282 2013-06-12 12:29:30 <sipa> BlueMatt: but i thought the reason it wasn't there before was because satoshi was concerned about that
283 2013-06-12 12:29:42 <BlueMatt> well, I know gmaxwell and jgarzik were concerned about it
284 2013-06-12 12:29:47 <BlueMatt> or at least echoed satoshi's concern
285 2013-06-12 12:29:47 <jgarzik> kjj, Does not seem like a problem
286 2013-06-12 12:29:53 <jgarzik> kjj, The info hash changes anyway
287 2013-06-12 12:29:57 <sipa> BlueMatt: ah, ok
288 2013-06-12 12:31:09 <kjj> jgarzik: with padding, we can run every torrent off of the same single file on the hard drive
289 2013-06-12 12:31:34 <jgarzik> RE torrent and multiple files: doesn't work automatically. Requires manual specification on command line, or in config file. Becomes tedious: loadblock=blk0001.dat, loadblock=blk0002.dat, ...
290 2013-06-12 12:32:04 <jgarzik> kjj, IMO it's ok that incentives exist to -not- run the older torrents
291 2013-06-12 12:32:21 <jgarzik> it's an append-only dataset after all, so the current is always a superset
292 2013-06-12 12:33:14 <kjj> people have different paranoia levels. letting them pick where they want to start is probably a good thing
293 2013-06-12 12:34:27 <SomeoneWeird> anyone got a link handy (or a couple minutes) that explains how the leveldb metadata works?
294 2013-06-12 12:34:28 <BlueMatt> the point of the torrent is you dont have to trust the provider of the torrent
295 2013-06-12 12:34:34 <kjj> if I write scripts for this, I could easily have torrents available up to midnight last night. people might not want that one, they may want the one from two weeks ago
296 2013-06-12 12:34:46 <BlueMatt> why?
297 2013-06-12 12:35:03 <kjj> BlueMatt: because some people are paranoid. :)
298 2013-06-12 12:35:17 <BlueMatt> my point is: there is no measurable difference in security
299 2013-06-12 12:35:44 <BlueMatt> in fact, its probably more secure to "trust" the torrent because your trust is in something more trustworthy than $RANDOM_P2P_NODE
300 2013-06-12 12:35:48 <kjj> but huge differences in unmeasurable differences in perceived security. :(
301 2013-06-12 12:36:07 <kjj> wow. I really butchered that sentence. let me try again
302 2013-06-12 12:36:19 <kjj> but huge differences in unmeasurable perceived security. :(
303 2013-06-12 12:37:56 <jgarzik> SomeoneWeird, looking for bitcoin-specific metadata stored in leveldb, or the leveldb metadata itself, used for internal db workings?
304 2013-06-12 12:38:18 <jgarzik> SomeoneWeird, for the former, it's mainly when we need to lookup a transaction id
305 2013-06-12 12:38:22 <SomeoneWeird> jgarzik, oh right, sorry, the data stored in leveldb for bitcoin
306 2013-06-12 12:38:45 <SomeoneWeird> so it's just a reverse lookup?
307 2013-06-12 12:39:09 <jgarzik> SomeoneWeird, it stores [roughly] transaction id -> unspent transaction outputs, and some transaction metadata such as block height
308 2013-06-12 12:40:00 <jgarzik> SomeoneWeird, transaction-id is the key, transaction information is the value, in the key-value pair stored
309 2013-06-12 12:40:07 <jgarzik> so not a reverse lookup
310 2013-06-12 12:40:15 <SomeoneWeird> right
311 2013-06-12 12:40:38 <SomeoneWeird> any docs about reading blk00000.dat? :P
312 2013-06-12 12:41:03 <sipa> it's [network magic] [4-byte LE block length] [block]
313 2013-06-12 12:41:07 <jgarzik> kjj, not worth the torrent constantly changing, IMO. It is definitely a balance -- the more torrents, the fewer seeders on any one torrent
314 2013-06-12 12:41:12 <sipa> lather, rinse, repeat
315 2013-06-12 12:41:30 <jgarzik> kjj, right now the torrent should probably be updated more frequently, but certainly not every night. Perhaps every 30 days.
316 2013-06-12 12:41:52 <nsh> *28
317 2013-06-12 12:42:05 <jgarzik> kjj, seeders ultimately key off the info hash, so special logic for sharing doesn't really help
318 2013-06-12 12:42:37 <jgarzik> want to concentrate the power on a few torrents, rather than spread widely across many torrents. Users will see greater download speeds, with fewer torrents.
319 2013-06-12 12:43:09 <warren> TheUni: around?
320 2013-06-12 12:43:20 <sipa> ACTION votes: 24 days, 6 hours, 32 minutes, 32 seconds
321 2013-06-12 12:43:51 <jgarzik> what is that? 3000 blocks?
322 2013-06-12 12:44:30 <sipa> 2^21 seconds
323 2013-06-12 12:44:43 <nsh> <yoleaux> (24 days 6 hours 32 minutes 32 seconds)/(10 minutes): 3495
324 2013-06-12 12:44:49 <kjj> jgarzik: we'll see how it works out.
325 2013-06-12 12:44:50 <michagogo> Are the logs linked in the topic updated live?
326 2013-06-12 12:45:09 <nsh> :)
327 2013-06-12 12:45:09 <nsh> michagogo, there's probably a way to check that
328 2013-06-12 12:45:26 <sipa> jgarzik: an integral multiple of (intended) blocks would be a multiple of 10 minutes :)
329 2013-06-12 12:45:34 <michagogo> Yeah, but it involves waiting for my browser to load :-D
330 2013-06-12 12:46:00 <warren> TheUni: unping
331 2013-06-12 12:46:16 <nsh> 14:45\tmichagogo\tYeah, but it involves waiting for my browser to load :-D - http://bitcoinstats.com/irc/bitcoin-dev/logs/2013/06/12
332 2013-06-12 12:46:37 <nsh> sipa, in theory.
333 2013-06-12 12:47:14 <nsh> could you not have rolling torrents with split blockchains?
334 2013-06-12 12:48:00 <nsh> nm, not sure that makes sense
335 2013-06-12 12:48:53 <nsh> (you can selectively download certain files within a torrent, but i'm not sure you can make parts of two different torrents employ the same hive)
336 2013-06-12 12:49:38 <vrs> hive? you mean swarm?
337 2013-06-12 12:49:54 <helo> colony.
338 2013-06-12 12:50:38 <nsh> hoard
339 2013-06-12 12:51:09 <helo> might as well go with murder i guess
340 2013-06-12 12:51:46 <nsh> all solution converge asymptotically on murder
341 2013-06-12 12:51:49 <nsh> *solutions
342 2013-06-12 12:52:42 <tumak_> Messages may be hidden in the capital and lower-case letters such as "ShoEboX" which spells "SEX" in capitals and "hobo" in lower-case.
343 2013-06-12 12:52:45 <tumak_> aha!
344 2013-06-12 12:52:47 <jgarzik> Torrent post updated with new info: https://bitcointalk.org/index.php?topic=145386.0 Please seed!
345 2013-06-12 12:52:56 <tumak_> ACTION looks for hidden messages from satoshi in bitcoin
346 2013-06-12 12:53:09 <jgarzik> Direct torrent link http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent (but magnet / DHT preferred)
347 2013-06-12 12:53:13 <nsh> tumak_, context?
348 2013-06-12 12:53:23 <nsh> ACTION seeds
349 2013-06-12 12:53:26 <tumak_> nsh: StudlyCaps
350 2013-06-12 12:53:37 <vrs> nsh: with a compliant bittorrent client, you can't do really fancy stuff like announcing the seeder of one file to another swarm, peer handshake includes info_hash and that is torrent specific
351 2013-06-12 12:53:54 <nsh> right
352 2013-06-12 12:54:19 <michagogo> jgarzik: How come you didn't mention the magnet link in here?
353 2013-06-12 12:54:28 <michagogo> I just grabbed the torrent file
354 2013-06-12 12:54:41 <michagogo> Anyway, it says it's "Checked 37%"
355 2013-06-12 12:54:45 <michagogo> with the number rising
356 2013-06-12 12:55:22 <jgarzik> michagogo, laziness? The forum link includes the magnet link
357 2013-06-12 12:55:24 <jgarzik> :)
358 2013-06-12 12:55:57 <michagogo> jgarzik: Sure, but you also posted the torrent link in here
359 2013-06-12 12:56:06 <michagogo> With a request to use the magnet link, but not the link itself
360 2013-06-12 12:56:21 <michagogo> So that same laziness leads to noncompliance with your request
361 2013-06-12 12:56:23 <nsh> jgarzik, can bootsrap.dat be used with pynode?
362 2013-06-12 12:56:30 <nsh> *bootstrap.dat
363 2013-06-12 12:56:31 <jgarzik> michagogo, hehe
364 2013-06-12 12:56:32 <nsh> (directly)
365 2013-06-12 12:56:36 <michagogo> Hmm, no trackers?
366 2013-06-12 12:56:51 <jgarzik> nsh, yes. loadblock=bootstrap.dat in your configuration file
367 2013-06-12 12:56:57 <nsh> ok, ty
368 2013-06-12 12:57:15 <jgarzik> michagogo, DHT preferred. magnet link adds some trackers though
369 2013-06-12 12:59:42 <vrs> I think tracker.ccc.de hasn't been up for a year or two
370 2013-06-12 12:59:47 <vrs> I could be wrong though
371 2013-06-12 13:00:35 <jgarzik> vrs, dunno. Input welcome? I don't use the trackers usually.
372 2013-06-12 13:00:36 <michagogo> Well, DHT hasn't found me any peers yet
373 2013-06-12 13:00:53 <nsh> .head http://tracker.ccc.de ---> 404, text/html, 345 bytes
374 2013-06-12 13:01:09 <jgarzik> nsh, is UDP down?
375 2013-06-12 13:01:30 <nsh> for?
376 2013-06-12 13:01:49 <jgarzik> nsh, that tracker you just posted. the magnet link uses udp://
377 2013-06-12 13:01:49 <nsh> i had one peer for a while there but gone now
378 2013-06-12 13:01:56 <nsh> oh, right, unsure
379 2013-06-12 13:02:01 <jgarzik> michagogo, if it matters, my peer is us2.exmulti.net
380 2013-06-12 13:02:22 <michagogo> jgarzik: Can you add a peer to a torrent?
381 2013-06-12 13:02:31 <vrs> michagogo: depends on your client
382 2013-06-12 13:02:42 <jgarzik> michagogo, depends on the software. Not sure about transmission.
383 2013-06-12 13:02:51 <jgarzik> lemme try something locally, adding trackers
384 2013-06-12 13:02:54 <michagogo> ??Torrent here
385 2013-06-12 13:03:37 <vrs> jgarzik: which port?
386 2013-06-12 13:04:03 <nsh> jgarzik, the A record for tracker.ccc.de is set to 127.0.0.1
387 2013-06-12 13:04:12 <jgarzik> vrs, 51413
388 2013-06-12 13:04:22 <jgarzik> default transmission port, I think
389 2013-06-12 13:04:28 <jgarzik> us2.exmulti.net:51413
390 2013-06-12 13:06:00 <vrs> hm rtorrent still doesn't seem to have that
391 2013-06-12 13:06:01 <nsh> ACTION is downloading from 1 peer (us2.exmulti.net)
392 2013-06-12 13:06:16 <vrs> but as soon as one bootstrapped peer is on the dht, things should start to work
393 2013-06-12 13:06:57 <coingenuity> im afraid that i'm utterly retarded jgarzik, trying to figure out what part of this is the txid: 630003daa50026213d668b9338edd40cedf93513cdd323bf006626678fec624a97
394 2013-06-12 13:07:15 <sipa> coingenuity: all of it
395 2013-06-12 13:07:36 <sipa> hmm, it's one character too much
396 2013-06-12 13:07:45 <coingenuity> sipa: hm, blockchain doesn't like that for some reason :/
397 2013-06-12 13:07:52 <vrs> nsh: do you have working dht and are you announcing on it?
398 2013-06-12 13:08:02 <sipa> coingenuity: is that a key in chainstate?
399 2013-06-12 13:08:05 <SomeoneWeird> {"key":"c?????????\\u0001??????\\\\Bv??????]\\u0007??????{???\\u0018???y<???\\u0018??????\\u0010??????\\u000e\\u0000???V","value":"\\u0001\\u00032\\u0003n+p?????????\\b??V??? \\u001a???4v5???????????O??????T\\u001bA\\u0019???5\\n?????? "}
400 2013-06-12 13:08:07 <coingenuity> sipa: yep
401 2013-06-12 13:08:12 <nsh> vrs, unsure how to check in transmission
402 2013-06-12 13:08:12 <SomeoneWeird> raw
403 2013-06-12 13:08:12 <SomeoneWeird> that's key + value from chainstate
404 2013-06-12 13:08:28 <nsh> i don't have any trackers, so i assume it got the peer from DHT
405 2013-06-12 13:08:44 <sipa> coingenuity: the '63' in front is the key type ('c' in ascii, for coin)
406 2013-06-12 13:08:47 <vrs> hm ok then it isn't in my dht yet
407 2013-06-12 13:08:51 <sipa> coingenuity: so take all but the first character
408 2013-06-12 13:08:55 <vrs> aha
409 2013-06-12 13:08:57 <michagogo> Is DHT something that should be propagating globally?
410 2013-06-12 13:09:11 <vrs> I got an answer from the dht
411 2013-06-12 13:09:31 <coingenuity> sipa: i've tried a bunch of things like that, just tried your suggestion - still "invalid transaction hash"
412 2013-06-12 13:09:44 <sipa> there may be a byte order issue
413 2013-06-12 13:10:01 <jgarzik> michagogo, in theory yes
414 2013-06-12 13:10:11 <michagogo> jgarzik: Looks like several of those trackers aren't working: http://i.imgur.com/yczZCfT.png
415 2013-06-12 13:10:56 <coingenuity> sipa: hrm, strange...not sure how i can figure this out now xD
416 2013-06-12 13:11:17 <coingenuity> the value of the keys seems to conform to data standards in that file you linked me to before
417 2013-06-12 13:11:19 <jgarzik> michagogo, noted thanks :/
418 2013-06-12 13:11:28 <sipa> coingenuity: swap the bytes
419 2013-06-12 13:11:36 <vrs> ok downloading from an israeli peer
420 2013-06-12 13:11:38 <sipa> so 974a62ec8f ...
421 2013-06-12 13:11:49 <nsh> http://www.trackon.org/ Trackon is a service to monitor the status and health of existing open and public trackers that anyone can use. A meta-tracker if you will. You can add any of the tracker announce URLs listed here to any of your torrents, or submit any other open/public trackers you might know of.
422 2013-06-12 13:11:55 <michagogo> vrs: Hmm, maybe that's me?
423 2013-06-12 13:12:06 <michagogo> vrs: are you amnesia.lambda.name?
424 2013-06-12 13:12:09 <vrs> aye
425 2013-06-12 13:12:14 <coingenuity> ah, thats a good idea sipa
426 2013-06-12 13:12:17 <michagogo> Okay, so we found each other
427 2013-06-12 13:12:26 <jgarzik> got eu3.exmulti.net going on torrent too
428 2013-06-12 13:12:26 <nsh> (either they don't update often or bittorrent is not so healthy these days)
429 2013-06-12 13:12:32 <jgarzik> DHT works there
430 2013-06-12 13:13:38 <nsh> ACTION has us2.exmulti.net and alster013.startdedicated.com now
431 2013-06-12 13:13:40 <michagogo> Ah, found Add Peer
432 2013-06-12 13:13:41 <jgarzik> two more peers here
433 2013-06-12 13:13:51 <michagogo> I'll connect in a sec
434 2013-06-12 13:14:06 <michagogo> Oh, it already found us2
435 2013-06-12 13:14:22 <nsh> also get a semidecent speed which means british telecom are feeling generous or something
436 2013-06-12 13:14:31 <michagogo> jgarzik: What port?
437 2013-06-12 13:14:41 <jgarzik> michagogo, 51413
438 2013-06-12 13:14:55 <jgarzik> anybody running rtorrent?
439 2013-06-12 13:15:04 <jgarzik> got a forum report that it doesn't like the .torrent
440 2013-06-12 13:15:08 <nsh> hmm
441 2013-06-12 13:15:47 <nsh> ACTION checks, one min
442 2013-06-12 13:16:58 <michagogo> Oh, that's why I couldn't seem to add eu3 to the torrent
443 2013-06-12 13:17:00 <nsh> rtorrent complains: (15:14:41) Could not create download: Could not find any trackers
444 2013-06-12 13:17:10 <michagogo> Because alster013.startdedicated.com is eu3 and it already found that for me
445 2013-06-12 13:17:34 <ll> yeah not loading with rtorrent, same msg as nsh
446 2013-06-12 13:17:56 <michagogo> rtorrent requires trackers? o_O
447 2013-06-12 13:18:00 <vrs> downloading at 8MB/s :)
448 2013-06-12 13:18:01 <jgarzik> nsh, ll: turn on DHT?
449 2013-06-12 13:18:03 <michagogo> Does the magnet link work?
450 2013-06-12 13:18:07 <nsh> ACTION figures out how to do so
451 2013-06-12 13:18:09 <jgarzik> michagogo, yes
452 2013-06-12 13:18:17 <michagogo> To what?
453 2013-06-12 13:18:19 <jgarzik> rtorrent disables DHT by default or somesuch
454 2013-06-12 13:18:20 <vrs> turn on the dht and turn off the trackers
455 2013-06-12 13:18:22 <michagogo> Ah
456 2013-06-12 13:18:34 <michagogo> I look to have 5 peers
457 2013-06-12 13:18:52 <michagogo> jgarzik, jgarzik, vrs, and 2 more
458 2013-06-12 13:19:43 <michagogo> Ah, I see eu3 doesn't have the whole thing yet
459 2013-06-12 13:19:57 <michagogo> I appear to be sending it ~250 kB/s
460 2013-06-12 13:20:02 <jgarzik> indeed
461 2013-06-12 13:20:08 <jgarzik> us2 has the only full copy
462 2013-06-12 13:20:27 <jgarzik> eu3 has the old bootstrap.dat, so > 50%
463 2013-06-12 13:21:25 <warren> how big is it these days? (doesn't sync often)
464 2013-06-12 13:21:50 <jgarzik> warren, 7.83 GiB
465 2013-06-12 13:21:55 <nsh> jgarzik, even with dht = on \\n dht_port = 6881 in .rtorrent.rc it still complains about not finding trackers. may need to configure a session directory to bootstrap the DHT or something
466 2013-06-12 13:22:18 <sipa> didn't rtorrent use a differnt dht mechanism or so
467 2013-06-12 13:22:32 <nsh> or wait until it's bootstrapped then add the torrent
468 2013-06-12 13:22:47 <jgarzik> sipa, no, it's just stupid ;p I tested rtorrent back when I originally did this. It worked -- once you manually configured a setting or two
469 2013-06-12 13:23:02 <jgarzik> the fix is to regen the .torrent with trackers, it sounds like
470 2013-06-12 13:23:13 <michagogo> Oh, I didn't even notice that ??Torrent tells you how much of the file your peers have
471 2013-06-12 13:23:21 <jgarzik> sipa, rtorrent's DHT does work fine, once setup
472 2013-06-12 13:23:24 <sipa> ok
473 2013-06-12 13:23:37 <michagogo> I seem to have the second most after us2, tied with eu3
474 2013-06-12 13:23:38 <vrs> sipa: rtorrent doesn't use bootstrap nodes afaik
475 2013-06-12 13:24:06 <tumak_> rtorrent's dht is super sloppy
476 2013-06-12 13:24:11 <tumak_> but eventually it catches up
477 2013-06-12 13:24:22 <vrs> or, used to but it worked fine this time
478 2013-06-12 13:24:23 <tumak_> it indees needs to discover dht nodes from torrents with tracker first :/
479 2013-06-12 13:24:48 <vrs> tumak_: yeah, this
480 2013-06-12 13:24:54 <nsh> you can embed DHT nodes in the torrent file itself
481 2013-06-12 13:24:57 <nsh> i believe
482 2013-06-12 13:25:01 <vrs> yes
483 2013-06-12 13:25:03 <nsh> also you can add them manually in rtorrent
484 2013-06-12 13:25:07 <jgarzik> If done smartly, that wouldn't be half bad. Pick a few open trackers, pick some info hashes at random, bootstrap the DHT that way.
485 2013-06-12 13:25:11 <vrs> dht nodes? how?
486 2013-06-12 13:25:17 <nsh> http://libtorrent.rakshasa.no/wiki/RTorrentUsingDHT
487 2013-06-12 13:25:18 <michagogo> Anyone know what the Flags column is?
488 2013-06-12 13:25:25 <nsh> If you know host name/IP and port of a DHT node, you can enter it manually with the command "dht_add_node=host:port". This command only adds the node to the routing table if there is space for it.
489 2013-06-12 13:26:03 <nsh> (command switch)
490 2013-06-12 13:26:32 <vrs> nice, I just used to copy dht caches between rtorrent instances when I needed a quick bootstrap
491 2013-06-12 13:26:44 <nsh> ACTION nods
492 2013-06-12 13:26:54 <nsh> rarely use rtorrent, only when i need to get around ISP restrictions
493 2013-06-12 13:28:15 <jgarzik> OK, heading to lunch. When I return (~ 1 hour), will regenerate the .torrent. infohash should not change [unless I'm an idiot, which is always possible]
494 2013-06-12 13:28:30 <jgarzik> and in an hour, hopefully there will be more seeds, both data and DHT-wise
495 2013-06-12 13:29:06 <nsh> :)
496 2013-06-12 13:29:26 <michagogo> jgarzik: Will we need to swap out the torrent?
497 2013-06-12 13:29:41 <nsh> probably not
498 2013-06-12 13:30:00 <jgarzik> michagogo, only if you want to advertise to the trackers that are added to the new .torrent
499 2013-06-12 13:30:01 <michagogo> (??????????????, btw)
500 2013-06-12 13:30:11 <jgarzik> michagogo, so, not REQUIRED to swap out .torrent, no
501 2013-06-12 13:30:19 <michagogo> But better to do it?
502 2013-06-12 13:30:22 <jgarzik> yes
503 2013-06-12 13:30:26 <michagogo> k
504 2013-06-12 13:30:43 <jgarzik> advertises to more trackers, making the torrent easier to find by weirdo rtorrent users
505 2013-06-12 13:30:44 <jgarzik> :)
506 2013-06-12 13:30:44 <jgarzik> ACTION runs
507 2013-06-12 13:31:34 <SomeoneWeird> lul
508 2013-06-12 13:31:40 <michagogo> I wonder if ??Torrent can seed the old torrent off of the new .dat
509 2013-06-12 13:32:14 <michagogo> It seems it can't while it's being brought up to date, since it's a .!ut file while that happens
510 2013-06-12 13:32:22 <michagogo> But afterwards... I wonder...
511 2013-06-12 13:34:14 <tumak_> meh, dht still deaf
512 2013-06-12 13:34:26 <tumak_> Id: Counters: 0f / 1s (0) on S/L/D: 0/0/0 (2/2)
513 2013-06-12 13:35:04 <tumak_> ACTION holds t
514 2013-06-12 13:35:56 <michagogo> Hmm, told it to update DHT
515 2013-06-12 13:36:00 <michagogo> Picked up another peer
516 2013-06-12 13:38:07 <TD> jgarzik: once you regenerate the torrent, maybe you could submit a pull for the website to inform people that it exists and how to use it
517 2013-06-12 13:40:38 <tumak_> huh
518 2013-06-12 13:40:44 <tumak_> are all bitcoiners mac users? :)
519 2013-06-12 13:40:56 <tumak_> transmission 4x, utorrent 1x :)
520 2013-06-12 13:41:37 <sipa> tumak_: eh, no?
521 2013-06-12 13:41:48 <tumak_> just prejudice
522 2013-06-12 13:41:57 <tumak_> transmission dont always imply mac user :)
523 2013-06-12 13:43:09 <sipa> i also use transmission, but on ubuntu
524 2013-06-12 13:44:00 <tumak_> ACTION is faithful to rtorrent
525 2013-06-12 13:44:15 <tumak_> transmission refused to to run 2k+ torrents for me :(
526 2013-06-12 13:44:51 <michagogo> tumak_: If the ??Torrent is Israeli, that's me
527 2013-06-12 13:44:56 <michagogo> Windows 7 here
528 2013-06-12 13:45:51 <michagogo> Though I don't see anything I recognize as you
529 2013-06-12 13:46:27 <tumak_> :)
530 2013-06-12 13:46:27 <tumak_> libtorrent cz
531 2013-06-12 13:46:51 <tumak_> and i see you :)
532 2013-06-12 13:46:54 <michagogo> tumak_: I see 2 libtorrents
533 2013-06-12 13:47:08 <michagogo> One is vrs, and the other is router-sun-nat-f.pilsfree.net internet address = 212.79.110.21
534 2013-06-12 13:47:22 <michagogo> And the second one doesn't match your host
535 2013-06-12 13:47:33 <michagogo> (or is that you somewhere else?)
536 2013-06-12 13:47:34 <tumak_> yeah, i irc from dev box
537 2013-06-12 13:47:47 <tumak_> router-sun is torrent box
538 2013-06-12 13:48:08 <michagogo> Ah, so you're on a server in Germany
539 2013-06-12 13:48:24 <tumak_> dunno, hetzner is all over the place
540 2013-06-12 13:48:31 <michagogo> Odd, the torrent box is showing as being in the US according to ??Torrent
541 2013-06-12 13:48:45 <michagogo> Oh, you know what?
542 2013-06-12 13:48:50 <michagogo> I think it's just using the tld
543 2013-06-12 13:49:07 <tumak_> yeah, it never bothered with geoip :/
544 2013-06-12 13:49:23 <michagogo> Ah
545 2013-06-12 13:49:28 <michagogo> Okay, I turned on an advanced option
546 2013-06-12 13:49:36 <nsh> ACTION used to have an ip address located in antarctica
547 2013-06-12 13:49:40 <michagogo> peer.resolve_country
548 2013-06-12 13:49:53 <nsh> but then the australian antarctic survey wanted their server back :/
549 2013-06-12 13:49:54 <tumak_> * 5216.0 / 8013.0 MB Rate: 4898.3 / 392.2 KB Uploaded: 1963.7 MB [65%] 0d 2:01 [ R: 0.38]
550 2013-06-12 13:49:57 <tumak_> ugh
551 2013-06-12 13:50:00 <michagogo> Now it seems to show the flags properly
552 2013-06-12 13:50:02 <tumak_> 2 hours? really? :(
553 2013-06-12 13:50:14 <tumak_> ACTION wants teh blockchains nao
554 2013-06-12 13:50:24 <michagogo> Is the Czech flag a blue triangle from the left, with a white stripe above a red one?
555 2013-06-12 13:50:42 <vrs> nsh: was the physical server there too or was it just the geolocation db?
556 2013-06-12 13:50:56 <tumak_> michagogo: like philippino, except upside down
557 2013-06-12 13:51:01 <tumak_> michagogo: we have phillipinos
558 2013-06-12 13:51:03 <nsh> i think the server was there. it served a lot of photos of ice sheets :)
559 2013-06-12 13:51:04 <tumak_> *hate
560 2013-06-12 13:51:19 <michagogo> tumak_: I'm not familiar with most world flags
561 2013-06-12 13:51:20 <michagogo> :-/
562 2013-06-12 13:52:08 <nsh> (i didn't legitimately own it. i fell out in the course of some security auditing)
563 2013-06-12 13:52:11 <nsh> *it
564 2013-06-12 13:52:47 <tumak_> you stole it!
565 2013-06-12 13:52:48 <tumak_> damn
566 2013-06-12 13:53:06 <tumak_> do you have any idea how expensive australian servers are?!
567 2013-06-12 13:53:18 <tumak_> nsh: i hope you gave it back to rightful owner
568 2013-06-12 13:53:29 <nsh> of course
569 2013-06-12 14:27:32 <michagogo> ACTION pokes jgarzik
570 2013-06-12 14:28:22 <SomeoneWeird> ACTION pokes michagogo
571 2013-06-12 14:28:24 <michagogo> ACTION slaps SomeoneWeird around a bit with a large trout
572 2013-06-12 14:28:39 <SomeoneWeird> ACTION eats trout
573 2013-06-12 14:30:44 <dugo> someone not done dumping 500 at a time
574 2013-06-12 14:32:46 <dugo> ;; market buy 700
575 2013-06-12 14:32:49 <dugo> ;; market sell 700
576 2013-06-12 14:32:52 <gribble> A market order to buy 700 bitcoins right now would take 77881.0105 USD and would take the last price up to 111.5000 USD, resulting in an average price of 111.2586 USD/BTC. | Data vintage: 0.0008 seconds
577 2013-06-12 14:32:52 <gribble> A market order to sell 700 bitcoins right now would net 75884.0478 USD and would take the last price down to 108.1600 USD, resulting in an average price of 108.4058 USD/BTC. | Data vintage: 0.0040 seconds
578 2013-06-12 14:33:05 <dugo> wc
579 2013-06-12 14:42:11 <nsh> jgarzik, do you have any thoughts yet on how wallet functionality with work in pynode? will it be roughly analogous to the reference client implementation?
580 2013-06-12 14:45:01 <michagogo> Still only got those 6 peers
581 2013-06-12 14:45:53 <nsh> ACTION has 5 now
582 2013-06-12 14:46:10 <nsh> getting 700kB/s though (which is not bad for my connection)
583 2013-06-12 14:46:30 <nsh> 3.20/8.40GB complete
584 2013-06-12 14:47:04 <michagogo> nsh: are you 91.202.254.170?
585 2013-06-12 14:47:29 <michagogo> or 86.142.61.176?
586 2013-06-12 14:47:57 <SomeoneWeird> >_>
587 2013-06-12 14:47:57 <SomeoneWeird> maybe he's the one who wants to keep his IP private
588 2013-06-12 14:48:00 <nsh> ACTION smiles
589 2013-06-12 14:48:23 <michagogo> I'm your Israeli peer
590 2013-06-12 14:48:30 <nsh> yay :)
591 2013-06-12 14:48:44 <michagogo> Don't seem to be uploading to you, though
592 2013-06-12 14:49:33 <michagogo> ...and now you've dropped off the peer list
593 2013-06-12 14:50:20 <michagogo> Okay, updating dht got you back
594 2013-06-12 14:56:31 <petertodd> 'away .
595 2013-06-12 14:57:18 <jgarzik> sigh. OSX demands a restart.
596 2013-06-12 14:59:55 <SomeoneWeird> OSX
597 2013-06-12 14:59:59 <SomeoneWeird> lol @ quit message
598 2013-06-12 15:05:43 <jgarzik> ok, back. Will look at regenerating the torrent.
599 2013-06-12 15:05:57 <SomeoneWeird> APPLE APPLE APPLE
600 2013-06-12 15:06:07 <nsh> BANANA HAMMOCK
601 2013-06-12 15:06:20 <SomeoneWeird> OI OI OI
602 2013-06-12 15:06:33 <jgarzik> TD, meh, I just cannot summon the effort to update the html and send in a pull req. will happily email text to saivann or someone else though
603 2013-06-12 15:07:00 <jgarzik> TD, advertising the torrent in that way is OK
604 2013-06-12 15:08:11 <michagogo> jgarzik: If it's not too much HTML stuff (I'm not great at that beyond the basics) I can do it
605 2013-06-12 15:09:27 <jgarzik> michagogo, it's basically: clone bitcoin.org repo, update html, submit pull req. I think basic html works. open question about where to submit the text, WRT site navigation.
606 2013-06-12 15:09:42 <jgarzik> i.e. new page, update existing page, I don't know.
607 2013-06-12 15:09:44 <michagogo> With respect to?
608 2013-06-12 15:10:02 <jgarzik> With regards to. Same basic idea. :)
609 2013-06-12 15:10:06 <michagogo> k
610 2013-06-12 15:10:13 <michagogo> Forking right now
611 2013-06-12 15:10:43 <jgarzik> main question is trackers
612 2013-06-12 15:11:07 <jgarzik> from the imgur: http/h33t.com udp/openbittorrent are OK
613 2013-06-12 15:11:19 <jgarzik> ccc.de, publicbt.com, istole.it NOT OK
614 2013-06-12 15:12:16 <jgarzik> unfortunately I am not an expert on "what is a good public tracker for bit coin block chain torrent?"
615 2013-06-12 15:12:45 <michagogo> istole.it seems to be working atm
616 2013-06-12 15:13:07 <michagogo> and publicbt.com
617 2013-06-12 15:13:08 <jgarzik> ok
618 2013-06-12 15:13:16 <michagogo> Idk
619 2013-06-12 15:13:41 <gmaxwell> "But DHT's fix everything!" :P
620 2013-06-12 15:14:00 <kauzu> hey
621 2013-06-12 15:14:26 <michagogo> gmaxwell: Except rtorrent apparently requires trackers
622 2013-06-12 15:14:40 <nsh> (or witchery)
623 2013-06-12 15:14:42 <kauzu> https://blockchain.info/de/tx/79c6730ac0698886e2021a13f0099f183ce266a5f139922f0c52627477b569ac?show_adv=true why is this tx not accepted? i know there is a double spend but one of them both should be accepted by time or am i wrong?
624 2013-06-12 15:14:56 <gmaxwell> michagogo: it doesn't??? but you have to bootstrap it somehow... if only the problems were limited to rtorrent. :(
625 2013-06-12 15:15:20 <jgarzik> gmaxwell, hehe
626 2013-06-12 15:15:46 <nsh> kauzu, transactions are not partially valid
627 2013-06-12 15:16:08 <jgarzik> I need to get on Cafe Press or somewhere that has an online t-shirt shop. Need shirts and coffee mugs with taglines like "DHT will fix it" or "have DHT, will travel"
628 2013-06-12 15:16:19 <jgarzik> bonus points if t-shirt shop takes bitcoins
629 2013-06-12 15:16:33 <michagogo> Looks like my node has seen 79c6730ac0698886e2021a13f0099f183ce266a5f139922f0c52627477b569ac but not 4ed48eea852c943d8ae13a09a18fb209af36350f43884854a481132abc81d172
630 2013-06-12 15:17:09 <kauzu> nsh: what does "not partially valid" mean?
631 2013-06-12 15:17:24 <nsh> they are either valid or not
632 2013-06-12 15:17:53 <michagogo> nsh: I think he's asking about confirmation
633 2013-06-12 15:17:53 <nsh> (perhaps i misunderstood your query)
634 2013-06-12 15:18:02 <michagogo> As in, why hasn't one of them confirmed
635 2013-06-12 15:18:08 <nsh> right, my bad
636 2013-06-12 15:19:04 <kauzu> is there a way to know if one of them will be accepted? it does not matter for me which one
637 2013-06-12 15:22:58 <kauzu> :/
638 2013-06-12 15:25:49 <jgarzik> OK
639 2013-06-12 15:26:00 <jgarzik> Updated torrent file w/ trackers: http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent
640 2013-06-12 15:26:05 <jgarzik> Please test, especially with rtorrent
641 2013-06-12 15:26:26 <jgarzik> info hash is same as earlier release today
642 2013-06-12 15:26:40 <jgarzik> so same DHT collection, same peers, same bytes
643 2013-06-12 15:27:35 <michagogo> jgarzik: Would the same magnetlink work?
644 2013-06-12 15:27:58 <michagogo> Also, nothing appears to have changed as far as I can tell
645 2013-06-12 15:28:18 <jgarzik> michagogo, magnet link obtains the .torrent from another peer, so? probably not
646 2013-06-12 15:28:22 <jgarzik> ACTION is restarting his two peers
647 2013-06-12 15:28:56 <michagogo> It said something about it already existing, and do I want to add trackers from it
648 2013-06-12 15:29:32 <michagogo> Erm
649 2013-06-12 15:29:33 <michagogo> Oops...
650 2013-06-12 15:29:39 <michagogo> I think I just accidentally the torrent
651 2013-06-12 15:30:16 <SomeoneWeird> :O
652 2013-06-12 15:30:47 <kauzu> nsh: is there a way to know if one of them will be accepted? it does not matter for me which one
653 2013-06-12 15:31:08 <michagogo> It's okay, just needs to rescan the file, it looks like
654 2013-06-12 15:40:06 <jgarzik> I wonder if rsync is smart enough to sync old->new bootstrap.dat, without completely transferring all ~7GB
655 2013-06-12 15:40:18 <jgarzik> ACTION seems to recall "yes"
656 2013-06-12 15:40:30 <Diablo-D3> define "all"
657 2013-06-12 15:40:36 <sipa> jgarzik: sure
658 2013-06-12 15:40:49 <sipa> but rsync is not super efficient about that, as it searched for moved data
659 2013-06-12 15:41:12 <jgarzik> seems like noticing an append situation would be straightforward
660 2013-06-12 15:41:35 <Diablo-D3> sipa: yeah, but rsync can execute rsync on the other side
661 2013-06-12 15:41:39 <Diablo-D3> which is where the magic is
662 2013-06-12 15:41:52 <Diablo-D3> so it can check blocks without transferring data
663 2013-06-12 15:43:41 <sipa> jgarzik: it is, but it will first compute checksums of the entire receiver-side file, send those to the sender, let the sender search for diffs, and then send those
664 2013-06-12 15:43:57 <sipa> so on the sender side you may see minutes of nothing before anything is really transmitted
665 2013-06-12 15:44:17 <jgarzik> as long as the total time is decreased...
666 2013-06-12 15:45:04 <sipa> surely it will
667 2013-06-12 15:45:28 <nsh> rsync is also somehow faster at deleting large numbers of files than rm
668 2013-06-12 15:45:52 <nsh> (probably because of the NSA backdoor in GNU rm that does a backup first)
669 2013-06-12 15:48:29 <sturles> rsync doesn't delete the files at all. Just makes them invisible to everyone but NSA.
670 2013-06-12 15:48:38 <sturles> Obviously faster.
671 2013-06-12 15:48:44 <jgarzik> Two peers have 92%
672 2013-06-12 15:49:11 <michagogo> I have 87.6
673 2013-06-12 15:49:16 <nsh> ACTION at 69%
674 2013-06-12 15:49:24 <michagogo> Doesn't seem to be moving much, though
675 2013-06-12 15:49:40 <michagogo> jgarzik: Looks like 3 peers have that
676 2013-06-12 15:49:50 <michagogo> 4*
677 2013-06-12 15:50:29 <nsh> btw: <nsh> jgarzik, do you have any thoughts yet on how wallet functionality with work in pynode? will it be roughly analogous to the reference client implementation?
678 2013-06-12 15:51:26 <jgarzik> nsh, completely open question. pynode has zero wallet code right now, so it could be SPV or full
679 2013-06-12 15:51:37 <nsh> right
680 2013-06-12 15:51:58 <nsh> ideally a choice of both :)
681 2013-06-12 15:52:13 <michagogo> Erm, both of jgarzik's peers and vrs's peer have an S in the flags section
682 2013-06-12 15:52:24 <michagogo> According to the help thing, that means they're snubbed
683 2013-06-12 15:52:27 <michagogo> Why is that?
684 2013-06-12 15:53:13 <jgarzik> ACTION tries to recall. I think snubbed is just a temporary, flow control condition
685 2013-06-12 15:53:49 <michagogo> ;;google define snub bittorrent
686 2013-06-12 15:53:50 <gribble> Glossary of BitTorrent terms - Wikipedia, the free encyclopedia: <http://en.wikipedia.org/wiki/Glossary_of_BitTorrent_terms>; Bittorrent - Bit Torrent terms and definitions - Vladd44.com: <http://www.vladd44.com/torrent/terms.php>; how to avoid being snubbed using bt? - BitTorrent - Peer to peer ...: <http://forums.whirlpool.net.au/archive/229945>
687 2013-06-12 15:53:53 <nsh> "An uploading client is flagged as snubbed if the downloading client has not received any data from it in over 60 seconds."
688 2013-06-12 15:53:56 <nsh> -http://en.wikipedia.org/wiki/Glossary_of_BitTorrent_terms#Snubbed
689 2013-06-12 15:54:04 <nsh> probably link quality problem
690 2013-06-12 15:54:14 <michagogo> How do I fix it?
691 2013-06-12 15:54:21 <michagogo> Or have it retry or something?
692 2013-06-12 15:54:45 <nsh> not sure :/
693 2013-06-12 15:54:53 <jgarzik> not sure. easy guess: restart client.
694 2013-06-12 15:55:05 <jgarzik> but probably just wait
695 2013-06-12 15:55:12 <jgarzik> The Swarm Will Fix It
696 2013-06-12 15:55:22 <michagogo> I'll try that
697 2013-06-12 15:55:40 <nsh> might be modifiable in preferences somewhere