1 2012-10-20 00:03:28 <jgarzik> now this is _really_ strange
2 2012-10-20 00:03:48 <jgarzik> getpeerinfo reports all satoshi clients, all with "startingheight" >= 204066
3 2012-10-20 00:04:00 <jgarzik> however...
4 2012-10-20 00:04:01 <jgarzik> 10/20/12 02:03:51 getblocks 198388 to 00000000000000000000 limit 500
5 2012-10-20 00:04:01 <jgarzik> 10/20/12 02:03:51 getblocks stopping at limit 198887 00000000000002772ce1
6 2012-10-20 00:04:03 <jgarzik> 10/20/12 02:03:52 getblocks stopping at limit 199887 000000000000005e7e63
7 2012-10-20 00:04:15 <jgarzik> a bunch of getblocks
8 2012-10-20 00:05:05 <jgarzik> client(s) appear to be in a loop, repeatedly requesting 192388 through 203888
9 2012-10-20 00:05:16 <jgarzik> they are served those blocks, then the getblocks restarts
10 2012-10-20 02:02:30 <Joric> http://eprint.iacr.org/2012/584.pdf 'We acquired the complete state of the Bitcoin transaction system (...) This required downloading 180,001
11 2012-10-20 02:02:31 <Joric> '
12 2012-10-20 02:02:47 <Joric> can't... find... words...
13 2012-10-20 02:03:21 <Joric> did they scrape blockexplorer to death
14 2012-10-20 02:25:19 <weex> yes
15 2012-10-20 02:31:11 <jgarzik> Joric: sad, eh?
16 2012-10-20 02:58:50 <Joric> they are analysts, not technichians
17 2012-10-20 03:01:05 <Joric> how do they call it... IR, information retrieval specialists :D
18 2012-10-20 04:38:05 <wumpus> so that's why block explorer was down all the time :')
19 2012-10-20 04:42:28 <wumpus> well, "not being technicians" is not an excuse, they could have just asked here, or on the forums...
20 2012-10-20 04:45:12 <MagicalTux> lol
21 2012-10-20 04:45:23 <MagicalTux> they actually parsed the blockexplorer pages ?
22 2012-10-20 04:49:01 <wumpus> yes :D
23 2012-10-20 12:38:19 <Luke-Jr> gavinandresen: found a way to (tediously) get transifex contributors; what date range do you want?
24 2012-10-20 12:39:05 <gavinandresen> Luke-Jr: meh. if it is tedious, not worth it to do every release....
25 2012-10-20 12:39:37 <Luke-Jr> gavinandresen: basically the equivalent of "git log" is available via HTML with like 20 entries per page
26 2012-10-20 12:40:51 <sipa> you have a script to scrape that?
27 2012-10-20 12:40:54 <gavinandresen> I wonder if any other projects thank their translators. I really DO appreciate their effort, it'd be nice to thank them.
28 2012-10-20 12:41:07 <Luke-Jr> sipa: I don't, but I doubt it'd be hard to make
29 2012-10-20 12:42:09 <Luke-Jr> especially if we just want usernames
30 2012-10-20 12:43:30 <sipa> how large is blk0001.dat, blk0002.dat, blkindex.dat together these days?
31 2012-10-20 12:44:24 <Luke-Jr> 4-5 GB I think
32 2012-10-20 12:44:59 <sipa> du -h --total on ultraprune's equivalent files: 4071804 KiB
33 2012-10-20 12:45:14 <Luke-Jr> :/
34 2012-10-20 12:45:35 <abrkn> heya, i was thinking of making a card game played with a "real" (pseudo random) deck of cards. the player receives a hash of the deck before the game starts to prove that the game is not "rigged" for him to lose. has this been done? are there any holes in my logic?
35 2012-10-20 12:45:42 <sipa> without any pruning, obviously
36 2012-10-20 12:51:51 <sipa> anyone know the numbers for the current databases?
37 2012-10-20 12:52:06 <Luke-Jr> ?
38 2012-10-20 12:52:25 <sipa> du -h --total ~/.bitcoin/{blk*.dat}
39 2012-10-20 12:53:45 <Luke-Jr> [14:44:21] <Luke-Jr> 4-5 GB I think
40 2012-10-20 12:54:12 <sipa> i'm asking for somewhat higher precision :)
41 2012-10-20 12:54:29 <Luke-Jr> it's not that precise :p
42 2012-10-20 12:54:50 <Luke-Jr> depends on how many orphans etc someone's seen
43 2012-10-20 12:54:56 <Luke-Jr> how recently they did IBD
44 2012-10-20 12:55:07 <sipa> sure, that may make a few % difference
45 2012-10-20 12:55:22 <Luke-Jr> mine is 4.0 GB
46 2012-10-20 12:55:30 <Luke-Jr> with pruned blk0001 index
47 2012-10-20 12:56:16 <sipa> by pruned you mean removed spent entries from blkindex.dat, or no stale blocks in blk000*.dat ?
48 2012-10-20 12:56:27 <Luke-Jr> both
49 2012-10-20 12:56:41 <Luke-Jr> but only up to the end of blk0001
50 2012-10-20 12:56:55 <sipa> ok
51 2012-10-20 12:57:04 <Luke-Jr> also, note that du uses sane 1024-units, not SI
52 2012-10-20 12:57:20 <sipa> that's why I said KiB :)
53 2012-10-20 12:58:20 <Luke-Jr> just clarifying my GB so it doesn't confuse you ;)
54 2012-10-20 13:01:08 <sipa> ok
55 2012-10-20 13:02:54 <Luke-Jr> gavinandresen: do you want a date range or "all time"?
56 2012-10-20 13:03:15 <gavinandresen> date range, last release to current release
57 2012-10-20 13:04:54 <Luke-Jr> so from 0.7.0 to 0.7.1, or 0.6.3 to 0.7.1 ? I don't think much changed between the 0.7.x
58 2012-10-20 13:07:23 <gavinandresen> I dunno, what do you think? Maybe somebody should post a "Thank you Translators" that does all-time, and we'll do a thank-you incremental for future releases.
59 2012-10-20 13:07:58 <Luke-Jr> shrug
60 2012-10-20 13:08:27 <gavinandresen> exactly, if it is a pain in the ass it's not worth spending time on it.
61 2012-10-20 13:09:33 <Luke-Jr> { for i in {1..21}; do curl -s -i -b 'sessionid=bd4010e6a781c81af98283c49f32a165' 'https://www.transifex.com/projects/p/bitcoin/timeline/?page=$i'; done; } | perl -nle 'm[/accounts/profile/([^/]+)] && print $1'|sort|uniq -c|sort -n
62 2012-10-20 13:09:34 <Luke-Jr> :p
63 2012-10-20 13:10:29 <Luke-Jr> hmm, odd
64 2012-10-20 13:10:40 <Luke-Jr> I probably should void that sessionid :x
65 2012-10-20 13:11:36 <Luke-Jr> seems to be missing some users somehow; I gotta run tho, will look at it a bit more when I get back
66 2012-10-20 13:13:44 <Luke-Jr> actually, it was $i being inside single quotes
67 2012-10-20 13:14:46 <abrkn> how can i get a receiving address that starts with "abrkn"? i noticed "1ninja" has an address starting with "1ninja"
68 2012-10-20 13:15:02 <sipa> abrkn: search for vanity addresses
69 2012-10-20 13:16:00 <abrkn> that's cool as shit.
70 2012-10-20 13:16:38 <sipa> yes and no; i admin i have some of those myself, but as addresses aren't really intended to be reused multiple times, they should be generated on the spot
71 2012-10-20 13:16:59 <sipa> but for donation addresses and such, there's no nicer solution currently, imho
72 2012-10-20 13:17:11 <sipa> s/admin/admit/
73 2012-10-20 13:18:20 <abrkn> is "brain wallet" on bitadress.org any good? like, if that site goes down can i still use my money?
74 2012-10-20 13:18:23 <Luke-Jr> 85 mauron, 57 Diapolo, 50 lukedashjr, 49 birgerhedman, 38 penknife, 34 Dr_Nix, 32 tcatm, 30 iongchun, 27 Janjko, 16 eurekafag, 15 Amink, 10 yomismo, 10 xHire, 10 transifexuser, 10 finway_china, 9 DrHaribo, 7 pishtov, 7 kevinxw, 6 menirosenfeld, 6 jondoh, 6 bitgarden, 5 doobrik, 5 Zyzzy, 5 HostFat, 4 stikonas, 4 kr105, 4 jui, 4 ariesgo, 4 Legogris, 4 Iltane, 4 Eisenaxt, 3 slobodanmiskovic, 3 jaau, 3 Eliel, 3 Deafboy, 2 sulaymanf, 2
75 2012-10-20 13:18:25 <Luke-Jr> runeks, 2 palsecam, 2 opresco, 2 olea, 2 mrmx, 2 mila, 2 matshenricson, 2 hendi, 2 david113, 2 bitcoinhu, 2 akira0, 2 Jrnr601, 2 Fernando, 2 Alissa, 1 vitor.de.mello.freitas, 1 tester, 1 tengo, 1 stergium, 1 reezer, 1 ownermadepa, 1 mysteq, 1 mikolaj, 1 linjaaho, 1 jeremiaskangas, 1 iavael, 1 h2010n, 1 eugemjj, 1 emuLOAD, 1 cj, 1 cesarfdez123, 1 cande, 1 bttfmcf, 1 bottleb, 1 beoswind, 1 Thiriel, 1 Stemby, 1 Serenata, 1
76 2012-10-20 13:18:26 <Luke-Jr> NedjeljniKomentar, 1 Funkin, 1 DH_, 1 BAWLAW
77 2012-10-20 13:18:34 <Luke-Jr> sorry for spam, ttyl
78 2012-10-20 13:18:44 <sipa> abrkn: yes, but you shouldn't ever use a website for generating addresses
79 2012-10-20 13:18:55 <abrkn> it claims to be client sife
80 2012-10-20 13:18:56 <abrkn> side*
81 2012-10-20 13:19:05 <sipa> doesn't matter - they can change it on the fly
82 2012-10-20 13:19:07 <Luke-Jr> for 0.7.0-0.7.1: 7 birgerhedman, 5 mauron, 5 Diapolo, 3 penknife, 2 xHire, 2 iongchun, 2 Dr_Nix, 1 pishtov, 1 menirosenfeld, 1 eurekafag, 1 emuLOAD
83 2012-10-20 13:19:15 <sipa> you can save the site locally though, and use that
84 2012-10-20 13:19:19 <sipa> if you trust the code
85 2012-10-20 13:19:52 <Luke-Jr> for 0.6.3-0.7.1: 50 lukedashjr, 32 mauron, 22 birgerhedman, 20 Diapolo, 11 iongchun, 9 Dr_Nix, 9 Amink, 7 penknife, 6 finway_china, 5 xHire, 5 eurekafag, 4 stikonas, 4 jui, 4 ariesgo, 3 menirosenfeld, 3 Eliel, 2 pishtov, 2 palsecam, 2 DrHaribo, 1 tengo, 1 tcatm, 1 olea, 1 mysteq, 1 mikolaj, 1 linjaaho, 1 h2010n, 1 emuLOAD, 1 beoswind, 1 Serenata
86 2012-10-20 13:20:03 <sipa> Luke-Jr: put that in a gist or a pastebin or so
87 2012-10-20 13:37:07 <abrkn> heya, i was thinking of making a card game played with a "real" (pseudo random) deck of cards. the player receives a hash of the deck before the game starts to prove that the game is not "rigged" for him to lose. has this been done? are there any holes in my logic?
88 2012-10-20 13:40:36 <vazakl> your logic is correct
89 2012-10-20 13:41:34 <abrkn> so for roulette, i could make up a game id before every spin, generate result on server, send client crypted gameid+result before he bets?
90 2012-10-20 13:41:48 <vazakl> you could
91 2012-10-20 13:41:59 <abrkn> interesting. i assume this method is in use already?
92 2012-10-20 13:42:27 <fiesh> it is
93 2012-10-20 13:42:33 <abrkn> any examples?
94 2012-10-20 13:42:39 <vazakl> im not up to date on all the bitcoin sites. but what youre saying makes perfect sense
95 2012-10-20 13:42:42 <fiesh> check the bitcoin wiki, section gambling
96 2012-10-20 13:42:57 <abrkn> ok. i really want to make a card game you can beat if you play it perfectly
97 2012-10-20 13:43:30 <abrkn> just need to find a way not to get overrun by bots that isnt as boring as text captcha
98 2012-10-20 13:46:16 <abrkn> what's the most interesting way to prove that someone is a human?
99 2012-10-20 13:54:18 <SomeoneWeird> abrkn, get them to take a picture of themselves with a shoe on their head with a game id written on their face while holding a dog
100 2012-10-20 13:55:09 <abrkn> SomeoneWeird: genius, is that technology patented?
101 2012-10-20 13:55:37 <SomeoneWeird> abrkn, get them to take a picture of themselves with a shoe on their head with a game id written on their face while holding a dog?
102 2012-10-20 13:55:40 <SomeoneWeird> it is now
103 2012-10-20 13:55:49 <abrkn> touche
104 2012-10-20 14:38:13 <gmaxwell> sipa: just didn't want us to forget it.
105 2012-10-20 14:43:29 <sipa> sure
106 2012-10-20 15:11:37 <jgarzik> <sipa> By the way: this pull request is rebased on top of 'threadimport' and 'canonical'. <<-- would be nice to reference the pull req #'s
107 2012-10-20 15:11:44 <jgarzik> in github
108 2012-10-20 15:15:09 <sipa> jgarzik: done
109 2012-10-20 15:16:11 <jgarzik> Meni's letter to Ron/Shamir got a response: https://bitcointalk.org/index.php?topic=118797.msg1286015#msg1286015
110 2012-10-20 15:37:10 <Titanium2> abrkn have them do somethign only a humnan can do, like somethign in the real world that generates a news article
111 2012-10-20 15:37:23 <Titanium2> obits are hard to fake
112 2012-10-20 15:45:55 <_dr> wow, these guys sure are annoying
113 2012-10-20 15:51:14 <_dr> first they claim that if someone has money in their wallet they are hoarding it; and when someone comes along and tells them that's BS they accuse the community of wanting to manipulate the methodology to fit their own 'political' (whatever that means) agenda
114 2012-10-20 15:51:58 <_dr> yeah right, and after they insult the community they ask for help! science, bitches!
115 2012-10-20 16:07:48 <senseless> is there anyway to rebroadcast a transaction?
116 2012-10-20 16:08:26 <sipa> it happens automatically from time to time
117 2012-10-20 16:08:37 <sipa> at least once every half an hour
118 2012-10-20 16:11:19 <senseless> alright, thanks
119 2012-10-20 16:24:52 <sipa> hmm, 33 incoming onion connections to bitcoin.sipa.be, and 4 IPv6 connections ...
120 2012-10-20 16:25:25 <sipa> while my crawler only knows of 14 onion nodes that accept connections
121 2012-10-20 16:25:32 <sipa> ACTION thinks more onion nodes are needed
122 2012-10-20 16:27:03 <gmaxwell> sipa: It takes work to make them go. We can ask people to run more... but unless tor adds the ability to ask for an onion inbound over the proxy port, it'll remain hard to get people to do.
123 2012-10-20 16:27:12 <sipa> true
124 2012-10-20 16:32:04 <jgarzik> gavinandresen: ACK troll for https://github.com/bitcoin/bitcoin/pull/1880 (Move external block import to separate thread)
125 2012-10-20 16:43:45 <Luke-Jr> jgarzik: your last post almost implies p2pool is the only decentralized pool -.-
126 2012-10-20 16:46:45 <sipa> it's the only pool i know that decentralizes payouts
127 2012-10-20 16:50:06 <gmaxwell> decentralizes payouts, for better or worse!
128 2012-10-20 16:50:27 <Luke-Jr> not really; payouts are decided by forrestv's rules, for better or worse
129 2012-10-20 16:50:41 <Luke-Jr> decentralizes the checking of every share, sure, but that's also irrelevant to Bitcoin
130 2012-10-20 16:51:22 <sipa> well if you consider the author of software a point of centralization, surely Bitcoin is completely centralized as well
131 2012-10-20 16:51:53 <Luke-Jr> sipa: for Bitcoin, it's an acknowledged problem the community is working to overcome :p
132 2012-10-20 16:52:19 <sipa> of course, but the same is true for any other software
133 2012-10-20 16:52:51 <gmaxwell> Luke-Jr: you can't change the rules of bitcoin by making more versions.
134 2012-10-20 16:53:20 <gmaxwell> also, unlike bitcoin it's perfectly reasonable to just start your own p2pool.. and there are people using it 'privately' on forks.
135 2012-10-20 16:53:28 <Luke-Jr> gmaxwell: that's not p2pool then
136 2012-10-20 16:53:43 <gmaxwell> It's the p2pool software.
137 2012-10-20 16:53:52 <sipa> i'm not saying that centralizing payouts cannot have advantages, but using software that does payout calculation according to verifiable and verified-by-every-node software is certainly more decentralized than what any central server can achieve
138 2012-10-20 16:53:57 <Luke-Jr> anyhow, there is no reason to prefer p2pool over other decentralized pools
139 2012-10-20 16:54:37 <gmaxwell> Luke-Jr: Some people don't trust pool operators not to rob them.
140 2012-10-20 16:54:58 <gmaxwell> And there are perfectly reasonable reasons to do that.
141 2012-10-20 16:55:02 <sipa> i really don't consider a GBT pool to be decentralized (even though it's certainly superior to a pool that does not allow block inspection or modification)
142 2012-10-20 16:55:05 <Luke-Jr> gmaxwell: Eligius has required zero trust in that regard since it launched
143 2012-10-20 16:55:44 <Luke-Jr> sipa: as far as Bitcoin is concerned, it is identical to p2pool
144 2012-10-20 16:55:45 <gmaxwell> Luke-Jr: er. Thats not true. You were originally proportional and could have been stuffing shares.
145 2012-10-20 16:56:03 <Luke-Jr> gmaxwell: no, because the share database including hashes of every share has been public
146 2012-10-20 16:56:04 <sipa> Luke-Jr: i'm not Bitcoin
147 2012-10-20 16:56:13 <gmaxwell> Luke-Jr: And later with SMPPS you're still a kind of proportional payout when you go negative.
148 2012-10-20 16:56:54 <sipa> Luke-Jr: i fully support the idea of GBT pools, but claiming it is as decentralized as P2Pool really makes no sense to me - one connects to a central server, no?
149 2012-10-20 16:57:24 <Eliel> Luke-Jr: your pool is pretty well auditable but it does require trust unless you're willing to spend hours auditing it.
150 2012-10-20 16:57:32 <Eliel> p2pool automatically audits itself.
151 2012-10-20 16:57:35 <sipa> i know the advantages, and I like them, but i think you'll get little respect by claiming it is not centralized
152 2012-10-20 16:57:47 <Luke-Jr> Eliel: p2pool doesn't audit itself, you have to spend even more hours reading its unreadable code :p
153 2012-10-20 16:58:08 <Luke-Jr> sipa: miners control and make their own blocks; as far as Bitcoin goes, that's as decentralized as it gets
154 2012-10-20 16:58:08 <sipa> Luke-Jr: compared to looking at every GBT response your server gives?
155 2012-10-20 16:58:30 <sipa> Luke-Jr: and as I said, I am not Bitcoin - I'm a human that considers a central server to be something centralized
156 2012-10-20 16:58:38 <Eliel> Luke-Jr: that is much more doable than writing your own software to audit eligius data.
157 2012-10-20 16:59:04 <sipa> and I think very few people will disagree with me there
158 2012-10-20 16:59:37 <Luke-Jr> *shrug* the only practical difference is that forrestv saves on server costs
159 2012-10-20 16:59:51 <Luke-Jr> at the expense of many other more important factors
160 2012-10-20 17:01:08 <sipa> I think a GBT pool should be called "solo mining with coordinated payout" or something
161 2012-10-20 17:03:45 <Luke-Jr> or just "decentralized pool" for short
162 2012-10-20 17:04:14 <Luke-Jr> p2pool can continue to use "p2p" for its tradeoffs as it has been
163 2012-10-20 17:04:25 <sipa> Do you really think you' re going to convince *anyone* that a service with a central server is decentralized?
164 2012-10-20 17:04:35 <sipa> even though I know what you mean, it is confusing
165 2012-10-20 17:05:39 <Eliel> the problem is trying to assign a binary statement about decentralizedness when one bit is not enough.
166 2012-10-20 17:05:47 <sipa> exactly
167 2012-10-20 17:05:52 <Luke-Jr> sipa: it's not exactly been disputed in general
168 2012-10-20 17:07:13 <Eliel> and you two are arguing about whether the generalized one bit statement should be formed by anding the bits or orring them :)
169 2012-10-20 17:14:34 <gmaxwell> There are multiple reasons to want decenteralized mining.
170 2012-10-20 17:15:26 <gmaxwell> Protection of bitcoin, attack resistance (downtime), invulnerablity to theft are some of the most obvious ones. Different solutions have different degrees of these things.
171 2012-10-20 17:16:07 <Luke-Jr> gmaxwell: p2p is *less* resistant to attacks
172 2012-10-20 17:16:51 <sipa> One reason to prefer a centralized (but inspectable) payout system, is performance - I'm not sure to what extent p2pool can reasonably scale
173 2012-10-20 17:16:52 <gmaxwell> Luke-Jr: p2p has less *motivation* to attack.
174 2012-10-20 17:17:23 <Luke-Jr> sipa: p2pool should be fine for constant performance I expect, but variance will skyrocket
175 2012-10-20 17:17:32 <gmaxwell> sipa: it just runs out of varience reduction at some point. The PPLNS window is such that small miners will just not get shares in every block.
176 2012-10-20 17:17:38 <Luke-Jr> gmaxwell: only because p2pool has good PR ;)
177 2012-10-20 17:18:08 <gmaxwell> Luke-Jr: not so. You got people trying to attack you because they thought they could extort you.
178 2012-10-20 17:18:28 <Luke-Jr> gmaxwell: they could extort p2p miners easier IMO
179 2012-10-20 17:18:41 <gmaxwell> Luke-Jr: And p2p can have very strong attack resistance. A good chunk of p2pool's hash rate is connected via non-public IPs on a darknet I run.
180 2012-10-20 17:19:03 <Luke-Jr> gmaxwell: p2pool wouldn't work if everyone did that
181 2012-10-20 17:19:10 <gmaxwell> Sure it would.
182 2012-10-20 17:19:33 <gmaxwell> there can be any number of darknets. We still mine the same pool, but you can't sever our connectivity.
183 2012-10-20 17:20:10 <gmaxwell> you'd have to saturate the connection of every node in a darknet to partition it from p2pool.. And we'd still be mining our own darkpool that you couldn't attack.
184 2012-10-20 17:21:37 <gmaxwell> and I'm sure the software itself has all kinds of dos weaknesses that your pool lacks, since it simply hasn't been attacked.
185 2012-10-20 17:22:08 <gmaxwell> There is just less motivation. All the big centeralized pools get dos attacked, and p2pool doesn't. The payoff isn't there, and the effort required for a minimum attack is somewhat greater.
186 2012-10-20 17:22:32 <Luke-Jr> I doubt that.
187 2012-10-20 17:22:54 <Luke-Jr> all the p2pool nodes combined can be overwhelmed easier than a single dedicated server with DDoS protection
188 2012-10-20 17:23:10 <gmaxwell> You can't even _find_ all the p2pool nodes.
189 2012-10-20 17:37:39 <bcb> how do you remove the bitcoin wallet passphrase?
190 2012-10-20 17:37:51 <sipa> that wasn't ever implemented
191 2012-10-20 17:38:32 <sipa> it wouldn't be hard, but nobody found it worthwhile to do i suppose
192 2012-10-20 17:39:34 <gmaxwell> It's just another risky edge case that would be undertested.
193 2012-10-20 17:40:32 <sipa> I think in general it's better to have encrypted wallets anyway - the only reason not to require that initially is that people are likely to forget the password when they're experimenting and don't have many coins yet
194 2012-10-20 17:41:16 <gmaxwell> You can also always make the password something trivial.
195 2012-10-20 17:41:28 <gmaxwell> Like the empty string.
196 2012-10-20 17:42:03 <sipa> I'm not sure the GUI accepts an empty string.
197 2012-10-20 17:42:29 <gmaxwell> hm. I thought I'd seen someone say thats what they'd done.
198 2012-10-20 17:42:43 <gmaxwell> might not have been the gui.
199 2012-10-20 17:46:34 <MC-Eeepc> p2p isnt really all that, claims operator of a competing centralised pool
200 2012-10-20 17:47:31 <gmaxwell> MC-Eeepc: I'm sure luke's views are earnest. It's not like his pool is a big money maker in any case.
201 2012-10-20 17:48:01 <gmaxwell> E.g. rather than his pool justifying his views, his views justify the pool.
202 2012-10-20 17:49:02 <MC-Eeepc> he seems to trot out the same distortions every time someone mentions p2pool
203 2012-10-20 17:49:16 <Luke-Jr> MC-Eeepc: Eligius is a decentralized pool.
204 2012-10-20 17:49:30 <MC-Eeepc> strange definitions no one here seems to agree with
205 2012-10-20 17:49:58 <MC-Eeepc> its partially decentralised at best from what i understand
206 2012-10-20 17:50:16 <MC-Eeepc> and you could say it is not autonomously decentralised, which p2pool is
207 2012-10-20 17:50:31 <MC-Eeepc> big distinction i think
208 2012-10-20 17:51:04 <Luke-Jr> MC-Eeepc: p2pool gives up a lot of major benefits of pools in exchange for what are mostly vulnerabilities
209 2012-10-20 17:51:41 <gmaxwell> Luke-Jr: what percentage of your hashrate is on getwork? What percentage of !getwork miners have _ever_ selected their own transactions?
210 2012-10-20 17:51:52 <gmaxwell> For p2pool these figures are 100% and 100%.
211 2012-10-20 17:52:02 <Luke-Jr> gmaxwell: I don't even have a way to measure that.
212 2012-10-20 17:52:31 <Luke-Jr> gmaxwell: if Stratum hadn't come around, it could have been 100% when ASICs went live :/
213 2012-10-20 17:52:35 <gmaxwell> Even if you ignore the centeralization of the payout, in practice you're not very decenteralized??? though I know you're working on it with things like GBT support in bfgminer.
214 2012-10-20 17:52:47 <gmaxwell> Yea. ::sigh::
215 2012-10-20 17:52:50 <Luke-Jr> (100% of all mining, I mean)
216 2012-10-20 17:53:03 <gmaxwell> Well, it still wouldn't result in people actually doing anything with the decenteralization.
217 2012-10-20 17:53:11 <Luke-Jr> gmaxwell: neither does p2pool
218 2012-10-20 17:53:15 <gmaxwell> Vs p2pool which won't mine if bitcoin disagrees.
219 2012-10-20 17:53:17 <Luke-Jr> but it does enable automated checks
220 2012-10-20 17:53:36 <Luke-Jr> gmaxwell: well, bfgminer could easily be made to solo mine in such a scenario
221 2012-10-20 17:53:41 <rdponticelli> They are just different things and both aproach are needed to build a richer ecosystem
222 2012-10-20 17:53:46 <gmaxwell> And where all the nodes are picking their own transactions, though only a few miners diverge from the bitcoind defaults.
223 2012-10-20 17:54:06 <Luke-Jr> gmaxwell: I intentionally made libblkmaker's API such that it can combine GBT templates from multiple sources
224 2012-10-20 17:54:20 <sipa> Luke-Jr: every time someone challenges that eligius is decentralizes, your answer seems to be "it's not centralized because it has all advantages decentralization has" - that's like saying that airplanes are actually boats, as they have all advantages of boats compared to cars
225 2012-10-20 17:54:27 <gmaxwell> Yea, I agree with rdponticelli. I'm happy both exist. Keep in mind your work for GBT probably never would have happened without p2pool prodding you along.
226 2012-10-20 17:55:01 <Luke-Jr> gmaxwell: I don't believe "what if" is possible to talk about :p
227 2012-10-20 17:55:16 <darkip> Does anyone know what size window blockchain.info uses for their hash rate calculation?
228 2012-10-20 17:56:29 <gmaxwell> Luke-Jr: and regardless of what you're doing, none of the other formerally fully centeralized pools seem to have much interest in going your route. Since you've never had more than 1/6th the hashrate or so, clearly "eligius is (somewhat) decenteralized" doesn't solve the problem of pool centeralization.
229 2012-10-20 17:56:48 <Luke-Jr> gmaxwell: EclipseMC is GBT-enabled
230 2012-10-20 17:56:59 <gmaxwell> Oh, I wasn't aware of that. Cool.
231 2012-10-20 17:57:12 <gmaxwell> P2pool has done quite well, especially considering its vastly increased startup costs.
232 2012-10-20 17:59:02 <MC-Eeepc> gmaxwell startup costs?
233 2012-10-20 17:59:14 <sipa> I think there are two types of miners (yeah, generalizations, I know): those who care about Bitcoin itself, and those who mainly care about their income; the former (given that they know about it, and don't mind the effort) are likely to choose a solution that decentralizes everything (even making them blind for the disadvantages, perhaps), the latter will just pick a centralized pool and some others as backups
234 2012-10-20 18:00:42 <MC-Eeepc> the latter put a couple of radeons and bitcoin 0.3 in a cupboard 18 months ago and forgot all about it while it prints money for them
235 2012-10-20 18:00:55 <MC-Eeepc> far too many of those it seems
236 2012-10-20 18:03:14 <Luke-Jr> sipa: I think there are some miners who would care about Bitcoin if it was convenient and didn't disadvantage them
237 2012-10-20 18:04:06 <sipa> sure, i've overgeneralized
238 2012-10-20 18:10:49 <Luke-Jr> gmaxwell: fwiw, looks like at least 6 blocks (globally) found with BFGMiner 2.8.1+ over GBT :p
239 2012-10-20 18:11:11 <gmaxwell> Pretty snazzy!
240 2012-10-20 18:11:36 <MC-Eeepc> thats why p2pool should be made brain dead easy to get up and running
241 2012-10-20 18:11:53 <gmaxwell> MC-Eeepc: you need a bitcoin full node.
242 2012-10-20 18:12:03 <MC-Eeepc> at least as easy and typing in a central pool address
243 2012-10-20 18:12:31 <MC-Eeepc> gmaxwell yeah that is a pain
244 2012-10-20 18:12:55 <darkip> What is the recommended number of previous blocks to consider when calculating network hash rate?
245 2012-10-20 18:13:12 <MC-Eeepc> perhaps as part of the SPV > full process, it could mine to a p2pool node and switch to normal p2pool if/when it becomes a full node
246 2012-10-20 18:14:22 <Luke-Jr> lol
247 2012-10-20 18:14:35 <Luke-Jr> using a "p2pool node" is LESS decentralized than GBT pools
248 2012-10-20 18:15:06 <sipa> Luke-Jr: that is the one claim you keep making that I do not understand at all
249 2012-10-20 18:15:34 <Luke-Jr> sipa: what's not to understand there?
250 2012-10-20 18:16:00 <Luke-Jr> sipa: all those miners are doing is solving getworks for some 3rd party, like any other centralized pool
251 2012-10-20 18:16:47 <Eliel> Luke-Jr: who's this 3rd party?
252 2012-10-20 18:16:57 <Luke-Jr> Eliel: the person running the p2pool node
253 2012-10-20 18:18:29 <gmaxwell> sipa: by p2pool node he means getwork mining against someone elses.
254 2012-10-20 18:19:14 <gmaxwell> Luke-Jr: no reason p2pool nodes couldn't offer GBT to connected hosts, same as you do... There has just been no demand.
255 2012-10-20 18:19:28 <gmaxwell> but thats like??? worst of all worlds.
256 2012-10-20 18:19:57 <sipa> Luke-Jr: oh - i thought you were talking about people running their own p2pool client
257 2012-10-20 18:20:07 <Luke-Jr> sipa: I was responding to MC-Eeepc's comment
258 2012-10-20 18:20:16 <Luke-Jr> where he is contrasting the two
259 2012-10-20 18:20:18 <sipa> ok - nevermind in that case
260 2012-10-20 18:21:47 <MC-Eeepc> and SPV sucks compared to a real client, but tis just a temporary measure
261 2012-10-20 18:23:08 <Luke-Jr> MC-Eeepc: but insofar as Bitcoin being secured goes, GBT is equal to p2pool, plus uses a standardized protocol that isn't pool-specific
262 2012-10-20 18:24:53 <gmaxwell> Luke-Jr: it's not equal in practice, it's only equal if people start doing things with GBT that they currently don't.
263 2012-10-20 18:25:05 <MC-Eeepc> "GBT is equal to p2pool" - luke jr, 2012
264 2012-10-20 18:25:39 <Luke-Jr> gmaxwell: they don't do those things with p2pool either
265 2012-10-20 18:26:09 <gmaxwell> Luke-Jr: e.g. lets have a bet, move all your GBT users onto a fork 10 blocks back. I'll bet you they all happily mine it.
266 2012-10-20 18:26:24 <gmaxwell> p2pool can't have that done to them.
267 2012-10-20 18:26:27 <Luke-Jr> gmaxwell: I'll bet they won't.
268 2012-10-20 18:26:33 <gmaxwell> Why won't they?
269 2012-10-20 18:26:45 <gmaxwell> I expect they don't even have a bitcoin to compare with.
270 2012-10-20 18:27:06 <Luke-Jr> BFGMiner will consider the pool broken and disable it
271 2012-10-20 18:27:14 <sipa> how will it now?
272 2012-10-20 18:27:16 <sipa> *know
273 2012-10-20 18:27:19 <Luke-Jr> it keeps a log of past blocks
274 2012-10-20 18:27:30 <Luke-Jr> and ideally can get info from other pools
275 2012-10-20 18:27:36 <gmaxwell> Well you could do that for getwork too (which was the context under which I'd suggested that behavior!) :P
276 2012-10-20 18:27:44 <gmaxwell> Hm. I didn't realize you were actually enforcing it.
277 2012-10-20 18:27:53 <gmaxwell> I though it was still just a warning.
278 2012-10-20 18:28:17 <gmaxwell> How are you dealing with psycho stuff from weird load balancing?
279 2012-10-20 18:28:24 <Luke-Jr> gmaxwell: not well :<
280 2012-10-20 18:29:01 <Luke-Jr> I need to rewrite the work fetching someday to handle these corner cases better
281 2012-10-20 18:29:15 <Luke-Jr> I think it ends up flooding the pool while it does that right now
282 2012-10-20 18:30:52 <gmaxwell> Okay, so fine, move them on to a fork that started ten blocks back, but skipping the first block of it. (which could have just been mined with private hash power, or random non-bfgminers)
283 2012-10-20 18:31:15 <gmaxwell> again, p2pool nodes have no compariable vulnerability. You'd have to isolate each miner's bitcoin node.
284 2012-10-20 18:47:23 <sipa> Luke-Jr: how well are ztex fpga's supported by bfgminer now?
285 2012-10-20 18:48:37 <Luke-Jr> sipa: works for me, if you overlook the false hw errors
286 2012-10-20 18:48:59 <sipa> i'll give it a try then
287 2012-10-20 18:50:26 <jgarzik> sipa: all ultraprune reqs are merged now?
288 2012-10-20 18:50:38 <Diablo-D3> fuck.
289 2012-10-20 18:50:41 <Diablo-D3> okay so
290 2012-10-20 18:50:44 <Diablo-D3> Im going to make my own language
291 2012-10-20 18:50:51 <Diablo-D3> and the only feature I can think of that would be nifty is
292 2012-10-20 18:51:04 <Diablo-D3> @json " ";
293 2012-10-20 18:53:34 <Diablo-D3> actually, that probably should be @json { }
294 2012-10-20 18:54:13 <Diablo-D3> and it gets compiled to a JSONTree, built inside the compiler adn static
295 2012-10-20 19:07:00 <sipa> jgarzik: yes
296 2012-10-20 19:13:24 <sipa> gmaxwell: ACK on ultraprune?
297 2012-10-20 19:18:20 <sipa> Diablo-D3: i think you want metaprogramming
298 2012-10-20 19:19:31 <Diablo-D3> sipa: probably
299 2012-10-20 19:19:38 <Diablo-D3> I just want the fucking compiler to do my bidding
300 2012-10-20 19:20:01 <Diablo-D3> seriously, my compiler (if you can call it that, it'll probably just end up being a perl script with a ton of regex that turn it into C)...
301 2012-10-20 19:20:05 <Diablo-D3> will have plugins.
302 2012-10-20 19:24:44 <Diablo-D3> dont like how the language works?
303 2012-10-20 19:24:47 <Diablo-D3> redfine it, asshole
304 2012-10-20 19:26:13 <midnightmagic> I've always wanted a compile to swear at me and call me stupid.
305 2012-10-20 19:33:56 <Diablo-D3> midnightmagic: hah
306 2012-10-20 19:34:07 <Diablo-D3> I should call the language Seaking
307 2012-10-20 19:35:38 <midnightmagic> or you could call it.. like.. PoppaVic or greycat or something..
308 2012-10-20 19:36:31 <Diablo-D3> tf?
309 2012-10-20 19:42:14 <gmaxwell> sipa: acked
310 2012-10-20 20:12:27 <gmaxwell> sipa: Can you send out a quick note to bitcoin-dev and say that ultraprune has been merged and many pull requests need to be rebased?
311 2012-10-20 20:12:38 <sipa> yes, was about to
312 2012-10-20 20:12:44 <gmaxwell> sipa: and congrats, good work. Now the testing fun begins. :P
313 2012-10-20 20:12:56 <Luke-Jr> XD
314 2012-10-20 20:19:38 <gmaxwell> ACTION starts updating public nodes
315 2012-10-20 20:25:07 <gmaxwell> Hm. I bet that bitcoin's search for sha2 partial preimages has now recieved more computational effort than any other single 'problem'. We're almost to 69 bits of work now.
316 2012-10-20 20:26:25 <rdponticelli> I've been trying ultraprune as another username. Can I copy blocks directory to my main user and it will take it out of the box?
317 2012-10-20 20:27:07 <gmaxwell> No. it's need to resync either from the network, a bootstrap, or via loadblock. Fortunately it resyncs fast.
318 2012-10-20 20:27:20 <rdponticelli> Ok
319 2012-10-20 20:27:29 <sipa> rdponticelli: no, but you can use -loadblock=<path>, where path is the old blk0001.dat file
320 2012-10-20 20:27:41 <sipa> and specify it twice, if you also want to import blk0002.dat
321 2012-10-20 20:28:02 <rdponticelli> Thx, will do it
322 2012-10-20 20:40:31 <sipa> gmaxwell: given that each bitcoin hash attempt is around 122 SHA256 rounds, I'd say we're at 69.6 bits
323 2012-10-20 20:40:41 <Luke-Jr> sipa: I suggest a followup email (or maybe adding a doc/* file) detailing how upgrade/downgrade is handled?
324 2012-10-20 20:41:01 <sipa> Luke-Jr: not yet implemented :)
325 2012-10-20 20:41:15 <Luke-Jr> :<
326 2012-10-20 20:41:41 <sipa> (there is no overlap between the data files, so at worst your node ends up in what seems to be an empty datadir, except for peers.dat and wallet.dat)
327 2012-10-20 20:43:00 <Luke-Jr> I presume wallet.dat works as-is both directions for now?
328 2012-10-20 20:43:13 <sipa> yes
329 2012-10-20 20:43:15 <sipa> no change
330 2012-10-20 20:43:22 <sipa> same for peers.dat
331 2012-10-20 20:43:46 <sipa> Luke-Jr: but my intention is to have some mechanism move (and maybe split) the block files, delete blkindex.dat, and then run an automated -reindex
332 2012-10-20 20:46:28 <Luke-Jr> sipa: would be nice if it didn't need to break older versions; there's almost certainly going to be a reason some people want to go back
333 2012-10-20 20:50:27 <sipa> the choice is either removing the blocks from the pre-ultraprune view, or keeping 3+ GB of data duplicated
334 2012-10-20 20:50:57 <sipa> the GUI could ask whether you want to move or copy, i suppose
335 2012-10-20 20:52:43 <Luke-Jr> Maybe a menu item for "Remove old data" that is hidden if it's missing already? (and a firstrun popup to let users know it exists)
336 2012-10-20 20:53:03 <sipa> perhaps - I'll let the GUI people worry about that
337 2012-10-20 21:12:26 <echelon> hi, how do i prevent the client from trying to get the external ip?
338 2012-10-20 21:12:36 <sipa> by specifying it explicitly
339 2012-10-20 21:12:45 <echelon> ah
340 2012-10-20 21:13:17 <sipa> or -nodiscover
341 2012-10-20 21:13:33 <echelon> cool thanks :)
342 2012-10-20 21:14:03 <echelon> i'm in the middle of repairing my index, how long should it take? :/
343 2012-10-20 21:14:24 <echelon> it's making everything sluggish
344 2012-10-20 21:17:38 <jgarzik> sipa: yay :) congrats, it's finally merged
345 2012-10-20 21:17:47 <jgarzik> sipa: well done
346 2012-10-20 21:21:28 <jgarzik> sipa: oh yeah
347 2012-10-20 21:21:35 <jgarzik> sipa: if not already done... bump the client version
348 2012-10-20 21:21:47 <jgarzik> sipa: that will help determine behavior differences
349 2012-10-20 21:21:55 <sipa> good idea
350 2012-10-20 21:23:01 <gmaxwell> hey, now that the wallet is the only thing in the bdb enviroment.. we could realistically have a -walletfile that even let you use another path.
351 2012-10-20 21:23:36 <jgarzik> yep
352 2012-10-20 21:24:06 <jgarzik> ACTION ponders blockindex as a flat file
353 2012-10-20 21:24:35 <sipa> jgarzik: i considered that too before changing the block index entries
354 2012-10-20 21:24:50 <jgarzik> sipa: how often are blockindex entries updated?
355 2012-10-20 21:25:11 <jgarzik> looking at CBlockIndex right now... not seeing much that is highly variable
356 2012-10-20 21:25:13 <sipa> right now: when adding them, and when connecting them
357 2012-10-20 21:25:27 <sipa> nStatus is variable, and nFile/nBlockPos/nUndoPos
358 2012-10-20 21:25:56 <sipa> but there's also CBlockFileInfo in the block index now
359 2012-10-20 21:26:32 <jgarzik> sipa: I don't see it in CBlockIndex?
360 2012-10-20 21:26:37 <sipa> it's not
361 2012-10-20 21:27:11 <sipa> it's a per-blockfile entry
362 2012-10-20 21:28:04 <sipa> when doing headers-first mode, with blocks signatures checked in the background, the number of times data is written will increase
363 2012-10-20 21:28:21 <jgarzik> ah, I see. multiple, distinct datasets merged into a single leveldb database
364 2012-10-20 21:28:26 <jgarzik> similar to blkindex.dat
365 2012-10-20 21:28:36 <sipa> all metadata associated with the block tree
366 2012-10-20 21:29:38 <sipa> this means for example that space occupied by a partially-written block will be reused afterwards, as the fact that that byte range was used will not have been written to the index
367 2012-10-20 21:39:20 <jgarzik> c_k: As just noted in https://bitcointalk.org/index.php?topic=108854.msg1286724#msg1286724 I am open to GBT improvements
368 2012-10-20 21:40:30 <sipa> gmaxwell: there's still a (less serious) issue if someone has a wallet file on a USB stick, has an unclean shutdown, moves the stick elsewhere, and tries to open it
369 2012-10-20 21:40:31 <jgarzik> sipa: docs on upgrade
370 2012-10-20 21:40:48 <gmaxwell> sipa: it should move the whole database enviroment.
371 2012-10-20 21:41:06 <sipa> gmaxwell: oh, duh!
372 2012-10-20 21:41:07 <jgarzik> sipa: Your post does not appear to include any instructions or advice or description of what happens during upgrade
373 2012-10-20 21:41:28 <gmaxwell> sipa: thats what I meant by 'now that the wallet is the only thing in the bdb enviroment'
374 2012-10-20 21:41:31 <jgarzik> sipa: like "if you run it on an existing install, it will re-download the blockchain"
375 2012-10-20 21:41:45 <sipa> right; i'll mention that
376 2012-10-20 21:42:18 <jgarzik> Building LevelDB ...
377 2012-10-20 21:42:20 <jgarzik> lame!
378 2012-10-20 21:42:21 <jgarzik> ;p
379 2012-10-20 21:42:50 <sipa> true, but compatibility >> complexity (and it's already a huge hack)
380 2012-10-20 21:42:58 <sipa> and you only need to do it once anyway
381 2012-10-20 21:43:34 <gmaxwell> sipa: did you ever sort out the memory corruption with parallel signature validation?
382 2012-10-20 21:43:39 <sipa> no
383 2012-10-20 21:44:13 <jgarzik> hum
384 2012-10-20 21:44:15 <gmaxwell> okay, when the dust settles point me at the patches and I'll see if I can find it. I think thats now the limiting factor on blocksync from local peers.
385 2012-10-20 21:44:26 <jgarzik> I wonder if I hit a bad peer... things are going more slowly than with BDB
386 2012-10-20 21:44:28 <jgarzik> 31870 jgarzik 39 19 857m 43m 4752 S 7.6 1.1 0:08.57 bitcoind
387 2012-10-20 21:44:32 <jgarzik> not doing much there
388 2012-10-20 21:44:45 <gmaxwell> jgarzik: yea, I -connect mine when syncing otherwise it's slow. :(
389 2012-10-20 21:44:56 <jgarzik> ACTION -addnode's
390 2012-10-20 21:45:04 <BlueMatt> sipa: for comparison's sake, Im assuming your dnsseed never throws out ips so X/N available means its heard of a total of N ips? Also, how much checking does it do per peer and what kind of timeout does it have?
391 2012-10-20 21:45:17 <gmaxwell> jgarzik: addnode isn't enough, you'll switch to some moron peer at a random point.
392 2012-10-20 21:45:50 <sipa> BlueMatt: it does throw out IPs, but only very rarely (like no response seen in a month, i should check the actual criteria)
393 2012-10-20 21:46:08 <sipa> BlueMatt: X/N indeed means a total of N IPs seen
394 2012-10-20 21:46:27 <BlueMatt> fair enough; what about block download checking/etc?
395 2012-10-20 21:46:28 <sipa> how much checking: correct version/verack cycle, and answer to addr
396 2012-10-20 21:46:35 <BlueMatt> ok
397 2012-10-20 21:46:38 <BlueMatt> timeout?
398 2012-10-20 21:46:42 <jgarzik> OK
399 2012-10-20 21:46:42 <sipa> 10s or so
400 2012-10-20 21:46:47 <jgarzik> -loadblock gets me full speed
401 2012-10-20 21:46:50 <sipa> i planned to add some checking that asked for a random recent block, or something
402 2012-10-20 21:46:51 <BlueMatt> sipa: thanks
403 2012-10-20 21:46:56 <sipa> but not yet done
404 2012-10-20 21:47:03 <echelon> jgarzik: full speed?
405 2012-10-20 21:47:12 <jgarzik> echelon: yes
406 2012-10-20 21:47:41 <echelon> how do you mean
407 2012-10-20 21:47:44 <sipa> jgarzik: -loadblock's (wall clock) speed has gone down significantly since it happens in parallel with normal operation
408 2012-10-20 21:47:58 <jgarzik> echelon: we are all playing with the just-merged database backend rewrite
409 2012-10-20 21:47:58 <sipa> solution: reducing cs_main
410 2012-10-20 21:48:15 <jgarzik> sipa: agreed... long been on the todo list
411 2012-10-20 21:48:21 <echelon> oh, me too i guess
412 2012-10-20 21:48:35 <echelon> but i left this running since early this morning
413 2012-10-20 21:48:58 <jgarzik> sipa: what is tx= in SetBestChain log line? Total number of unspent transactions?
414 2012-10-20 21:49:07 <sipa> jgarzik: total number of transactions ever
415 2012-10-20 21:49:12 <jgarzik> echelon: probably don't have it, then
416 2012-10-20 21:49:21 <echelon> don't have what
417 2012-10-20 21:49:30 <sipa> jgarzik: for use in a future post-headers-first progress bar, mainly
418 2012-10-20 21:49:41 <sipa> as transactions are much better measure than blocks
419 2012-10-20 21:49:49 <jgarzik> echelon: the just-merged database backend rewrite
420 2012-10-20 21:50:25 <echelon> and that makes the loadblock feature work faster?
421 2012-10-20 21:50:35 <sipa> it makes everything faster :)
422 2012-10-20 21:50:46 <echelon> oh :/
423 2012-10-20 21:51:00 <sipa> but mostly block validation, and -loadblock is purely that
424 2012-10-20 21:51:08 <sipa> so there it should be very noticable
425 2012-10-20 21:51:58 <echelon> so should i abort this and try your thingy
426 2012-10-20 21:52:05 <jgarzik> he said thingy
427 2012-10-20 21:52:14 <sipa> haha :)
428 2012-10-20 21:52:26 <sipa> i don't dare counting how many man-hours it took :)
429 2012-10-20 22:18:12 <sipa> gmaxwell: better suggest that on pullreq page #1889
430 2012-10-20 22:19:52 <c_k> jgarzik: oops, I think you meant ckolivas
431 2012-10-20 22:20:01 <c_k> jgarzik: I am not him :)
432 2012-10-20 22:22:32 <Arnavion> Building src/leveldb fails if I pass CXXFLAGS to make
433 2012-10-20 22:22:41 <Arnavion> I suppose the workaround used in makefile.unix needs to be used there too
434 2012-10-20 22:23:05 <sipa> Arnavion: building leveldb automatically is somewhat of a hack
435 2012-10-20 22:23:24 <sipa> alternatively, build it yourself by first going into leveldb/ and running make there
436 2012-10-20 22:23:47 <Arnavion> Well it was just one line of change I needed to do
437 2012-10-20 22:23:49 <sipa> make libleveldb.a libmemenv.a
438 2012-10-20 22:23:53 <Arnavion> Not very hacky
439 2012-10-20 22:24:05 <sipa> ah
440 2012-10-20 22:24:11 <sipa> feel free to pullreq it then
441 2012-10-20 22:24:17 <Arnavion> Like I said, makefule.unix already has a workaround for the same reason
442 2012-10-20 22:24:23 <Arnavion> It uses xCXXFLAGS instead of CXXFLAGS
443 2012-10-20 22:24:25 <jgarzik> c_k: oh, sorry
444 2012-10-20 22:24:31 <jgarzik> thought he changed nicks
445 2012-10-20 22:24:43 <c_k> all good :)
446 2012-10-20 22:24:51 <echelon> i thought there was going to be a way to trim old transactions to keep the size of the db down?
447 2012-10-20 22:25:31 <echelon> i think i remember reading it in the white paper
448 2012-10-20 22:25:38 <jgarzik> ACTION returns to check on IBD... wow, a ton of orphans and already-have-block errors
449 2012-10-20 22:25:56 <sipa> echelon: the type of pruning supported by ultraprune is different from what the paper describes
450 2012-10-20 22:26:10 <sipa> (though it's not implemented)
451 2012-10-20 22:26:13 <echelon> oh
452 2012-10-20 22:26:17 <gmaxwell> jgarzik: you didn't -connect. :P
453 2012-10-20 22:26:29 <sipa> echelon: it will remove full blocks
454 2012-10-20 22:26:39 <sipa> echelon: but always keep unspent transaction outputs
455 2012-10-20 22:27:41 <jgarzik> gmaxwell: <shrug> this is a test of what others will see. my local node is the one that initiated the requests for these dups and orphans
456 2012-10-20 22:27:52 <jgarzik> it's not like it's the remote node's fault for that
457 2012-10-20 22:28:17 <jgarzik> [jgarzik@bd ~]$ grep -c 'ERROR: ProcessBlock() : already have block ' /spare/bitcoin/data/debug.log
458 2012-10-20 22:28:20 <jgarzik> that's just one run
459 2012-10-20 22:28:22 <gmaxwell> jgarzik: what happens is the node you're pull from gets a new block from the network, and then your node switches to pulling from it, trying to resolve from the new head.
460 2012-10-20 22:28:40 <gmaxwell> well at least thats what makes the orphans. Not sure about the already haves.
461 2012-10-20 22:35:36 <Arnavion> sipa: Making a pullreq is effort though, so have a gist of the patch. https://gist.github.com/3925338
462 2012-10-20 22:37:54 <gmaxwell> 10/20/12 23:21:57 SetBestChain: new best=00000000839a8e6886ab height=1 work=8590065666 tx=2 date=01/09/09 02:54:25
463 2012-10-20 22:37:57 <gmaxwell> 10/21/12 00:31:02 SetBestChain: new best=00000000000002f68b1e height=204208 work=547427517987481385917 tx=8170493 date=10/20/12 23:08:46
464 2012-10-20 22:39:24 <sipa> gmaxwell: synced how?
465 2012-10-20 22:40:17 <gmaxwell> p2p acrocess gig-e, w/ connect.
466 2012-10-20 22:41:07 <sipa> shouldn't be slower than loadblock
467 2012-10-20 22:41:47 <sipa> (i know it is - just saying that it can be improved)
468 2012-10-20 22:45:49 <jgarzik> gmaxwell: remote node have blocks in pagecache?
469 2012-10-20 22:46:16 <jgarzik> well I guess gige is probably sufficient for the disk at any rate?
470 2012-10-20 22:49:30 <gmaxwell> I think peopple will be pretty happy about the startup time improvement.
471 2012-10-20 22:55:06 <MC1984> does the ibd improvements bring the curve of the chain growth below the curve of the march of technology
472 2012-10-20 22:55:15 <MC1984> has anyone ever quantified that
473 2012-10-20 22:55:49 <jgarzik> ultraprune upgrades the database backend, not IBD
474 2012-10-20 22:56:29 <jgarzik> indirectly speaking, IBD will go faster iff you are disk-bound with BDB (which many were, including on of my main boxes)
475 2012-10-20 22:56:53 <jgarzik> but work remains on peer selection; you can still get stuck
476 2012-10-20 22:56:58 <gmaxwell> MC1984: since no one was ever seeing it take >10 minutes on average to validate any block there wasn't any risk of it getting away.
477 2012-10-20 22:56:58 <MC1984> no the leveldb
478 2012-10-20 22:57:24 <sipa> MC1984: there are two changes now; ultraprune and leveldb
479 2012-10-20 22:57:33 <sipa> neither have any influence on the size of the block chain
480 2012-10-20 22:57:36 <gmaxwell> jgarzik: I still protest the use of 'stuck' there. "paused". :P
481 2012-10-20 22:57:41 <MC1984> I KNOW
482 2012-10-20 22:58:01 <sipa> well i don't understand your question in that case
483 2012-10-20 22:58:28 <gmaxwell> sipa: he was under the mistaken impression that the growth of the blockchain was faster than people could keep up with and was asking if we were back under it now.
484 2012-10-20 22:58:28 <MC1984> gmaxwell is right
485 2012-10-20 22:59:20 <sipa> MC1984: oh you mean people where the IBD system failed to do 1 block per 10 minutes?
486 2012-10-20 23:00:04 <MC1984> no i know its not getting away, but is IBD getting shorter or longer, given chain growth vs some sort of metric for average level of technology out there
487 2012-10-20 23:00:16 <MC1984> say just in the west for now, to make it simpler
488 2012-10-20 23:01:36 <MC1984> it strikes me the long term viability of bitcoin is tied to the r&d depts of the likes of intel, amd, seagate etc
489 2012-10-20 23:02:00 <MC1984> oh shit its them, they are satoshi
490 2012-10-20 23:02:02 <MC1984> !
491 2012-10-20 23:03:01 <jgarzik> sipa: thinking about headers-first, fill-in-blocks-later syncing... what, if anything, blows up if we only have headers for range X-infinity (where <X blocks are full, >=X blocks are headers only)?
492 2012-10-20 23:03:04 <jgarzik> what explodes
493 2012-10-20 23:03:19 <jgarzik> triggering an iterative header download seems straightforward
494 2012-10-20 23:03:29 <sipa> jgarzik: ?
495 2012-10-20 23:03:47 <jgarzik> sipa: does ultraprune support that configuration now?
496 2012-10-20 23:04:15 <jgarzik> would a header download need to write to block data (an empty block), or just to block index?
497 2012-10-20 23:04:39 <sipa> the blktree database format supports anything you like (see the enum BlockStatus in main.h)
498 2012-10-20 23:05:02 <sipa> no block data needs to be written if there is none
499 2012-10-20 23:05:16 <sipa> the blocks/ directory is there for blocks; block headers are in blktree/
500 2012-10-20 23:05:44 <jgarzik> ACTION was thinking more about impact to P2P and RPC commands
501 2012-10-20 23:05:59 <jgarzik> there is no longer a global fClient state, but a per-block one
502 2012-10-20 23:06:18 <sipa> well the "current" block, as exposes by P2P and RPC, is still the tip of the currently-connected best chain
503 2012-10-20 23:06:27 <sipa> so you may have headers after that point
504 2012-10-20 23:07:18 <jgarzik> sipa: does the definition of "currently connected best chain" require full block data?
505 2012-10-20 23:07:22 <sipa> yes
506 2012-10-20 23:07:26 <jgarzik> ok
507 2012-10-20 23:07:38 <jgarzik> that's doable
508 2012-10-20 23:07:57 <sipa> it means the block whose UTXO state is currently reflected by pcoinsTip (aka coins/)
509 2012-10-20 23:08:50 <sipa> there are other modes of operation; if you allow the "current block" to be something for which you don't have a UTXO state, you're basically building an SPV client
510 2012-10-20 23:08:59 <sipa> i suppose that's also possible, but further ahead
511 2012-10-20 23:09:22 <jgarzik> sipa: what happens when importing a full block... if the block index is already present? block index is updated? not touched? overwritten?
512 2012-10-20 23:09:27 <jgarzik> s/if/is/
513 2012-10-20 23:09:47 <sipa> well, if it is marked with HAVE_BLOCK, it won't be written again
514 2012-10-20 23:10:28 <sipa> if BLOCK_VALID_CHAIN is already present, no ConnectBlock would be necessary anymore
515 2012-10-20 23:10:46 <jgarzik> sipa: presumably the headers-only download would add a bunch of VALID_HEADER
516 2012-10-20 23:11:10 <jgarzik> sipa: then pass #2 downloads, so as to transform VALID_HEADER -> HAVE_DATA
517 2012-10-20 23:12:06 <sipa> not really; i added it because enum states are cheap, but I don't think you want to store a header that can't be connected to the tree (what we currently call a n orphan block)
518 2012-10-20 23:12:35 <jgarzik> sipa: presumably it can be connected to the tree
519 2012-10-20 23:13:25 <sipa> how i see it: first, we switch the current getblocks logic to a getheaaders logic, and add processing for an incoming header to do the same as an incoming block does now, but only up to BLOCK_VALIDT_TREE
520 2012-10-20 23:13:41 <sipa> so that will build you a block tree in memory/blktree
521 2012-10-20 23:14:07 <jgarzik> consider a full HAVE_DATA dataset to height H=120000. the node would start up, download headers from H+1 to infinity, then come back, and iterate through each header, pulling block data.
522 2012-10-20 23:14:30 <sipa> then a process continuously searches for the best tree tip, and fetches blocks (from any peers known to have that many blocks)
523 2012-10-20 23:14:31 <jgarzik> those headers would not be accepted, if not connectable
524 2012-10-20 23:14:50 <sipa> along the best known path
525 2012-10-20 23:14:58 <sipa> this happens in parallel with the normal block-header sync
526 2012-10-20 23:15:52 <sipa> then a third process performs what is currently done in ConnectBlock, to take them from state VALID_TRANSACTIONS (reached when the block was downloaded, and before it got stored) to BLOCK_VALID_CHAIN
527 2012-10-20 23:16:12 <sipa> this moves the "current block" pointer
528 2012-10-20 23:16:44 <sipa> finally, a fourth process performs signature checking along the path, for blocks for which this hasn't been done yet
529 2012-10-20 23:16:54 <sipa> well, fourth, fifth, sixth, ...
530 2012-10-20 23:22:40 <sipa> you'd basically get several block pointers: best-known-header, best-known-header-with-known-data, currently-connected-block and currently-sigchecked-block
531 2012-10-20 23:22:58 <sipa> each being a predecessor of the previous
532 2012-10-20 23:23:18 <sipa> and each with one or more associated processes to catch up with it