1 2012-09-04 00:06:38 <arij> iw ill pay for a satoshi dice clone
2 2012-09-04 00:06:45 <arij> pm me
3 2012-09-04 00:07:03 <arij> it just needs two addresses
4 2012-09-04 00:08:55 <sipa> btctrader22: yes, but not starting over is often not an option
5 2012-09-04 00:10:46 <Luke-Jr> ACTION glares at arij
6 2012-09-04 00:11:54 <Luke-Jr> woohoo, I get about 8 kH/s using libgcrypt for CPU mining
7 2012-09-04 00:12:36 <arij> its just for me
8 2012-09-04 00:13:03 <arij> :/
9 2012-09-04 00:16:07 <sipa> arij: i don't think many developers here are fan of satoshiDICE; if you want to pay someone to create a clone for you, you can ask of course, but try #bitcoin-otc or the forums, not here
10 2012-09-04 00:16:53 <Luke-Jr> (of course, you might get better reception anywhere if it doesn't need to be a SD clone, but can be an equivalent gambling site without the DDoS)
11 2012-09-04 00:25:36 <sipa> Luke-Jr: also libgcrypt CPU mining... what CPU?
12 2012-09-04 00:26:37 <Luke-Jr> sipa: i5-2400
13 2012-09-04 00:27:12 <sipa> sounds mildly unimpressive
14 2012-09-04 00:27:18 <Luke-Jr> :p
15 2012-09-04 00:27:27 <Luke-Jr> sipa: trying to build a *simple* test case for libblkmaker
16 2012-09-04 00:28:08 <sipa> there we go, 9m53s for importing 193k blocks, with database checksum checking enabled
17 2012-09-04 00:39:53 <btctrader22> sounds mildly unimpressive :p
18 2012-09-04 00:40:13 <sipa> haha
19 2012-09-04 00:40:49 <arij> sipa: what do you mean an eqivalent without the ddos
20 2012-09-04 00:41:41 <sipa> arij: Luke-Jr said that; and he is referring to the fact that there are ways to have a gambling service like SD which doesn't put as much load on the blockchain as SD does
21 2012-09-04 00:41:57 <arij> oh
22 2012-09-04 00:42:31 <Luke-Jr> arij: Bitcoin doesn't work well with unbalanced load right now; better to have deposits and withdrawls
23 2012-09-04 00:42:42 <arij> i see
24 2012-09-04 00:43:15 <Luke-Jr> unbalanced with reference to transaction volume vs real human users
25 2012-09-04 00:43:43 <Luke-Jr> (iow, while it's much less than VISA in raw transaction counts, it's HUGE in percentages)
26 2012-09-04 04:51:43 <Diablo-D3> http://www.isidewith.com/presidential-election-quiz
27 2012-09-04 04:52:01 <Diablo-D3> this is how I did: http://imgs.isidewith.com/results-image/85031537.jpg
28 2012-09-04 05:55:22 <sturles> Jill Stein for president!
29 2012-09-04 05:55:29 <sturles> How come I never heard of her?
30 2012-09-04 05:56:35 <Diablo-D3> shes the green candidate this year
31 2012-09-04 05:58:06 <freewil> http://therealnews.com/t2/index.php?option=com_content&task=view&id=31&Itemid=74&jumival=8102
32 2012-09-04 05:59:36 <ThomasV> Dr. Jill Stein
33 2012-09-04 06:22:56 <edcba> so why does she deserve that ?
34 2012-09-04 06:24:15 <Diablo-D3> deserve what?
35 2012-09-04 06:24:24 <edcba> to be president
36 2012-09-04 06:29:42 <sturles> edcba: Try the quiz Diablo-D3 linked to.
37 2012-09-04 06:31:28 <edcba> i don't have the right to vote in usa...
38 2012-09-04 06:35:15 <Diablo-D3> edcba: okay lemme screw your head on straight
39 2012-09-04 06:35:20 <Diablo-D3> a) the quiz is anonymous
40 2012-09-04 06:35:32 <Diablo-D3> b) no one deserves to be president, but we deserve a better one
41 2012-09-04 06:36:18 <_dr> do you really think a new president can still save your country?
42 2012-09-04 06:36:40 <_dr> without meaining to be offensive, i think it's just better to let it fail
43 2012-09-04 06:37:00 <Diablo-D3> _dr: if its a president who will charge congress for gross violation of every crime we have laws for? yes
44 2012-09-04 06:38:19 <_dr> i really hope such a thing would be an option, but honestly i think the structures in place will prevent that from ever happening
45 2012-09-04 06:41:02 <_dr> but then again, i'm probably biased by the extrinsic media view of your country. in the last X years there hasn't been any good news in the press about america in europe (certainly not 100% true, but the trend)
46 2012-09-04 06:41:16 <_dr> of your -> on your
47 2012-09-04 06:42:04 <edcba> i don't see how america affected positively europe recently
48 2012-09-04 06:42:30 <edcba> i mean more positively than before
49 2012-09-04 06:43:25 <_dr> not about affecting something, the news in general. any news about the usa we get around here is: mass shooting, corporate crimes, guy shot because we was trying to video-tape a cop, etc.
50 2012-09-04 06:44:02 <edcba> obama taking down reddit
51 2012-09-04 06:44:08 <_dr> :-)
52 2012-09-04 06:48:25 <Luke-Jr> [08:40:12] <CIA-45> bitcoin: Michael Gronager * ra1a0fe779654 libcoin/ (19 files in 10 dirs): Added SQLite dependency - libcoin is moving towards exclusive use of SQLite instead of various key value pair databases like bdb and leveldb http://tinyurl.com/c2p7s96 <-- fail
53 2012-09-04 06:48:41 <sturles> How many got shot in New York because the Police couldn't aim at that guy?
54 2012-09-04 06:49:02 <sturles> In Norway the police has fired one shot in 2012.
55 2012-09-04 06:49:03 <sturles> Total.
56 2012-09-04 06:49:18 <sturles> (Training excluded.)
57 2012-09-04 06:49:51 <sturles> They don't need to fire sixteen times to shoot someone.
58 2012-09-04 06:52:51 <_dr> hardly a fair comparison
59 2012-09-04 06:53:44 <_dr> if citizens in the us were not allowed to carry a firearm, police wouldn't have to be so trigger-happy i guess
60 2012-09-04 06:54:10 <edcba> that's not the problem i guess
61 2012-09-04 06:58:40 <Luke-Jr> _dr: how silly, crime is far lower when citizens are armed
62 2012-09-04 06:59:53 <Luke-Jr> _dr: thankfully, the US (in theory) doesn't have the same kind of "super-class" where police are considered above citizens (in fact, any citizen has the authority to make arrests)
63 2012-09-04 07:00:39 <_dr> i believe you, it's just what the media reports over here in the last years has become very biased against the us
64 2012-09-04 07:01:06 <Luke-Jr> _dr: even the US media nonsense is biased against the US
65 2012-09-04 07:01:17 <Luke-Jr> but media is worthless, so..
66 2012-09-04 09:17:41 <doublec> getmemorypool is gone in v0.7?
67 2012-09-04 09:22:45 <sturles> Luke-Jr: By what statistics? The USA comes out pretty bad e.g. on this: http://en.wikipedia.org/wiki/List_of_countries_by_intentional_homicide_rate#By_country
68 2012-09-04 09:25:57 <sturles> And here: http://www.nationmaster.com/graph/cri_rob-crime-robberies
69 2012-09-04 09:27:25 <Luke-Jr> doublec: https://en.bitcoin.it/wiki/BIP_0022
70 2012-09-04 09:27:58 <doublec> Luke-Jr: thanks. has the major pool software update to that?
71 2012-09-04 09:28:24 <doublec> I'm just testing 0.7 on my pool and noticed it was gone
72 2012-09-04 09:28:52 <Luke-Jr> doublec: Eloipool has, IIRC p2k said he was going to update ecoinpool
73 2012-09-04 09:31:08 <sturles> Perhaps the ultimate crime statistics: http://www.nationmaster.com/graph/cri_pri_per_cap-crime-prisoners-per-capita
74 2012-09-04 09:31:12 <Luke-Jr> doublec: I think it should mostly be backward compatible if you just rename the call, maybe need to add an empty params Object
75 2012-09-04 09:34:48 <doublec> I think getmemorypool had a field with the value of the number of generated coins but I didn't see that in the new call from a quick scan
76 2012-09-04 09:35:18 <sturles> Of course a problem with U.S. police is their inability to aim. Recently they had to fire 16(!) times to kill someone pulling up a gun in New York. Close up. And by their inability to aim properly and finish him in one or two shots, they hurt a lot of innocent bystanders.
77 2012-09-04 09:35:30 <_dr> sturles: table of freedom by country, put together by the department of we know best
78 2012-09-04 09:35:33 <_dr> usa 100%
79 2012-09-04 09:35:34 <_dr> rest 0%
80 2012-09-04 09:35:53 <sturles> Table of freedom per country?
81 2012-09-04 09:35:55 <sturles> Where?
82 2012-09-04 09:37:36 <sipa> doublec: coinbasevalue
83 2012-09-04 09:38:56 <doublec> sipa: ah, I must have missed it, thanks
84 2012-09-04 09:52:01 <wao> oh hai.
85 2012-09-04 09:52:27 <wao> is there list of some irc bots supporting bitcoin functions?
86 2012-09-04 09:52:30 <wao> like watch on some wallet
87 2012-09-04 09:59:29 <wao> bbe
88 2012-09-04 09:59:36 <gribble> http://blockexplorer.com
89 2012-09-04 09:59:36 <wao> !bbe
90 2012-09-04 10:04:52 <Luke-Jr> _dr: US 100%? must be relative - I don't think anything is 100%
91 2012-09-04 10:10:29 <_dr> i better use a sarcasm xml-tag next time :)
92 2012-09-04 10:35:21 <sipa> TD: benchmarks yesterday: git head: 41m, ultraprune: 19m, leveldb+ultraprune: 10m; batch block connection makes almost no difference
93 2012-09-04 10:35:30 <sipa> (for 193k blocks)
94 2012-09-04 13:04:50 <TD> sipa: a 5x speed increase is definitely worth celebrating!
95 2012-09-04 13:05:05 <edcba> depends
96 2012-09-04 13:05:19 <TD> although this is artificial of course. 41m is still way faster than most people see. the bottleneck is now surely the network and the randomness of whether you pick a good peer to sync from
97 2012-09-04 13:05:22 <edcba> if 5x speed on a 1ms operation happening every century...
98 2012-09-04 13:05:29 <TD> 5x faster block chain sync vs git head
99 2012-09-04 13:05:49 <sipa> TD: on my VPS it's much more (even just ultraprune it took close to a day on BDB), now it's finished in less than an hour...
100 2012-09-04 13:06:07 <TD> hmm
101 2012-09-04 13:06:17 <sipa> it horribly slow I/
102 2012-09-04 13:06:18 <TD> yeah when i benchmarked leveldb originally i was seeing more like 90mins for git head+bdb
103 2012-09-04 13:06:19 <sipa> I/O
104 2012-09-04 13:06:29 <jgarzik> FWIW, I noted in IRC the other day that I prefer to merge leveldb first, then ultraprune second
105 2012-09-04 13:06:37 <TD> bdb was 45 mins with no sig checking
106 2012-09-04 13:06:45 <jgarzik> makes for better comparisons and sanity checks, in that order, IMO
107 2012-09-04 13:07:33 <sipa> well, the leveldb code import is identical, but apart from that and a few implementations, the integration in current head and ultraprune are quite independent
108 2012-09-04 13:07:45 <BlueMatt> TD: its only somewhat artificial, keep in mind the block download is entirely sequential, so it would still help the speed people see anyway
109 2012-09-04 13:08:06 <sipa> so if leveldb gets merged first, i'd add a patch to ultraprune that first reverted all changes to core anyway
110 2012-09-04 13:09:36 <TD> presumably at least some of the code is the same (to load the block headers, initialize leveldb, etc)
111 2012-09-04 13:10:29 <sipa> i wrote most of it from scratch, but used your code as a guideline (you're mentioned in the commit)
112 2012-09-04 13:12:05 <sipa> jgarzik: so i don't mind doing leveldb first (in fact, if ultraprune would be deemed too experimental for the next release, i still hope leveldb gets in), but for sanity checking... the database changes anyway
113 2012-09-04 13:13:01 <gmaxwell> at least at the moment I would prefer to not release-stage leveldb and ulraprune; I think the have a lot of shared risk and I'd rather take it once rather than twice. They also both have migration cost I'd rather take once rather than twice.
114 2012-09-04 13:13:22 <jgarzik> agreed; I simply feel it is one fewer variables, to compare behavior between vanilla and +leveldb, as the structure of the indexes themselves does not matter.
115 2012-09-04 13:13:48 <jgarzik> gmaxwell: sounds like a strawman; no one AFAIK has proposed separating leveldb and ultraprune by a full release
116 2012-09-04 13:14:46 <gmaxwell> jgarzik: er. Sure; I only commented because sipa just mentioned doing that.
117 2012-09-04 13:16:23 <TD> it's really up to whoever reviews the code
118 2012-09-04 13:16:53 <BlueMatt> ACTION jumps up and down screaming unit tests repeatedly for 10 minutes
119 2012-09-04 13:17:10 <BlueMatt> :)
120 2012-09-04 13:17:17 <sipa> BlueMatt: every commit in my ultraprune branch has succeeding unit tests here
121 2012-09-04 13:17:30 <BlueMatt> I still want more unit tests...
122 2012-09-04 13:17:51 <sipa> (which is far from enough; whole-program tests like your bitcoinj-based tests does are obviously very welcome)
123 2012-09-04 13:19:09 <BlueMatt> ACTION sees no/few leveldb-specific unit tests anywhere...
124 2012-09-04 13:19:32 <TD> do there need to be any?
125 2012-09-04 13:19:40 <TD> leveldb is basically a big hashmap
126 2012-09-04 13:19:47 <gmaxwell> So, whats the plan if we find that leveldb gets corrupted by crashes?
127 2012-09-04 13:19:49 <BlueMatt> not for leveldb itself, but around bitcoin's leveldb api
128 2012-09-04 13:20:16 <TD> yeah i don't think there's any internal leveldb wrapping api that isn't tested after sipas work. there was indeed that problem in my branch
129 2012-09-04 13:20:49 <TD> gmaxwell: good question. there is a function you can call that tries to repair the db but i don't know what it does - with the migration code i wrote (guess sipa rewrote it), blowing away the txdb directory would cause it to be recreated
130 2012-09-04 13:21:28 <BlueMatt> is the ultraprune+leveldb stuff in the ultraprune pull? I see no test-cases which are really testing the db
131 2012-09-04 13:21:32 <jgarzik> that's indeed the standard question for switching "preferred filesystems" in the kernel as well. the long term plan is to switch away from ext* (currently ext4) to btrfs in distros... but recoverability after crash must be rock solid, including disaster recovery tools
132 2012-09-04 13:21:35 <jgarzik> i.e. fsck
133 2012-09-04 13:21:50 <jgarzik> BDB analogue 'db_recover'
134 2012-09-04 13:22:11 <TD> there's no explicit tool to recover a leveldb. it simply isn't supposed to get corrupted by crashes
135 2012-09-04 13:22:26 <TD> given that it's basically the same design bigtable uses and bigtable servers are forcibly terminated in flight all the time, i guess whatever it does works
136 2012-09-04 13:22:42 <jgarzik> <TD> leveldb is basically a big hashmap <<-- are you sure? I thought it was a big sorted, chunked data structure
137 2012-09-04 13:22:46 <TD> i recall no cases of db corruption in bigtable that could be traced to crashes
138 2012-09-04 13:22:50 <TD> jgarzik: well yes. i meant in terms of API
139 2012-09-04 13:22:54 <jgarzik> where sorted keys are adjacent
140 2012-09-04 13:23:44 <jgarzik> TD: does leveldb verify checksums on data read from disk?
141 2012-09-04 13:23:44 <TD> right
142 2012-09-04 13:24:07 <jgarzik> TD: i.e. does it do metadata checksumming, data checksumming, or none of the above?
143 2012-09-04 13:24:34 <TD> there is a unit test in the leveldb tree called corruption_test.cc
144 2012-09-04 13:25:14 <TD> it does all of the above, as far as i know
145 2012-09-04 13:25:22 <TD> the corruption test randomly scrambles some bytes
146 2012-09-04 13:25:26 <gavinandresen> RE: crash recovery: I think I agree with the paper somebody linked to here a few months ago, arguing that applications should always do a quick&dirty shutdown and always perform recovery at startup.
147 2012-09-04 13:25:35 <TD> and then it checks that the affected records are skipped
148 2012-09-04 13:26:03 <devrandom> hi TD
149 2012-09-04 13:26:05 <jgarzik> gavinandresen: I've seen some early Google papers saying they often do just that
150 2012-09-04 13:26:12 <TD> ok
151 2012-09-04 13:26:17 <TD> so there's a db option called paranoid_checks
152 2012-09-04 13:26:20 <TD> hey devrandom
153 2012-09-04 13:26:22 <gavinandresen> It's a good way to make sure your crash recovery code actually works.
154 2012-09-04 13:26:27 <TD> // If true, the implementation will do aggressive checking of the
155 2012-09-04 13:26:28 <TD> // errors. This may have unforeseen ramifications: for example, a
156 2012-09-04 13:26:51 <TD> // If true, all data read from underlying storage will be
157 2012-09-04 13:26:51 <TD> struct ReadOptions {
158 2012-09-04 13:26:54 <TD> so there is the answer
159 2012-09-04 13:27:01 <jgarzik> gavinandresen: that's what pagedb does, in fact: https://github.com/jgarzik/pagedb (I mention this only for academic interest... not suggesting pagedb for bitcoin)
160 2012-09-04 13:27:03 <TD> by default, no. i don't know what kind of performance impact it'd have to enable it
161 2012-09-04 13:27:20 <jgarzik> TD: mainly CPU impact, I would presume?
162 2012-09-04 13:27:33 <TD> yes
163 2012-09-04 13:28:59 <TD> i think "random bytes get scrambled" is not a particularly common failure mode for hard disks unless they're about to die totally
164 2012-09-04 13:29:13 <TD> the process being zapped in the middle of writing seems more common
165 2012-09-04 13:29:35 <TD> and ~all google software is built on the assumption that it can be killed at any time, because that's exactly what our datacenter control system does
166 2012-09-04 13:29:38 <TD> (very regularly)
167 2012-09-04 13:30:43 <jgarzik> yeah, I think I read that in the Chubby paper
168 2012-09-04 13:30:56 <jgarzik> TD: does Google still use Chubby, or something quite like it?
169 2012-09-04 13:30:59 <TD> yes
170 2012-09-04 13:31:05 <jgarzik> cool
171 2012-09-04 13:31:10 <TD> we still use chubby. it has some scalability changes these days
172 2012-09-04 13:31:15 <jgarzik> TD: I have a shitty Chubby clone ;-)
173 2012-09-04 13:31:17 <TD> but it's still the same idea
174 2012-09-04 13:31:19 <TD> heh
175 2012-09-04 13:31:35 <jgarzik> TD: core is still multi-paxos?
176 2012-09-04 13:32:31 <TD> yes. these days there are a lot of proxies in front of it
177 2012-09-04 13:32:36 <sipa> BlueMatt: it's in a separate branch, leveldb+ultraprune, for now
178 2012-09-04 13:32:39 <TD> otherwise the single master would get overloaded
179 2012-09-04 13:32:46 <jgarzik> right
180 2012-09-04 13:33:49 <sipa> TD: i've enabled verify_checksums; no significant impact on IBD speed
181 2012-09-04 13:33:59 <jgarzik> the cache coherence was a surprising property, when I first read the paper. many of us kernel vfs hackers wished that there existed a POSIX-compliant, cache coherence remote network filesystem.
182 2012-09-04 13:34:19 <sipa> TD: it was 2 seconds slower, iirc, which is probably less than the random noise on the measurement anyway
183 2012-09-04 13:34:25 <jgarzik> (obviously chubby's mission is quite different than such a filesystem, just noting...)
184 2012-09-04 13:34:42 <jgarzik> *coherent
185 2012-09-04 13:35:03 <jgarzik> anyway... has anyone done fuzz testing with bitcoin+leveldb?
186 2012-09-04 13:35:16 <jgarzik> or even simple truncate-the-file testing?
187 2012-09-04 13:35:45 <sipa> BlueMatt: suggestions for a leveldb-specific unit test?
188 2012-09-04 13:35:53 <sipa> BlueMatt: apart from the tests leveldb itself already has
189 2012-09-04 13:36:00 <BlueMatt> jgarzik: feel free: https://github.com/TheBlueMatt/test-scripts
190 2012-09-04 13:36:41 <BlueMatt> sipa: I was thinking largely simple tests just to put things like the commit/abort stuff through its paces
191 2012-09-04 13:36:52 <sipa> BlueMatt: leveldb has no commit or abort
192 2012-09-04 13:36:57 <sipa> just batch writes
193 2012-09-04 13:37:21 <sipa> TD's code put a layer for transactions on top of it, but ultraprune doesn't need that
194 2012-09-04 13:37:22 <BlueMatt> sipa: yes, but I was under the impression CDB for leveldb wrapped batch writes into such, so it could use testing ;)
195 2012-09-04 13:37:27 <BlueMatt> ah
196 2012-09-04 13:38:20 <TD> i think just killing the process at random times and restarting it might be useful
197 2012-09-04 13:38:40 <sipa> let's see!
198 2012-09-04 13:38:52 <TD> that said, i expect it to reveal no problems unless there is code that does non-batched writes that expects them to be batched (application level issue)
199 2012-09-04 13:40:18 <TD> jgarzik: yeah chubby isn't really a filesystem despite the API it exposes
200 2012-09-04 13:40:26 <jgarzik> indeed
201 2012-09-04 13:40:27 <TD> jgarzik: when I was an SRE this fact was reliably the cause of a few outages per year
202 2012-09-04 13:40:30 <TD> :)
203 2012-09-04 13:40:37 <TD> (cluster-level outages only, fortunately)
204 2012-09-04 13:41:43 <TD> i managed to nuke the entire Google Earth service once by doing a "fileutil touch" on a file in chubby. i just wanted to update the modification time. the command worked ... but it turned out to be implemented by simply erasing the contents of the file.
205 2012-09-04 13:42:13 <sipa> outch :)
206 2012-09-04 13:42:15 <jgarzik> rofl
207 2012-09-04 13:42:58 <jgarzik> TD: I've been meaning to experiment with a scalable design, where bits of the namespace are leased to downstream masters, to distribute the load
208 2012-09-04 13:43:26 <TD> yeah. i think that's a pretty sensible design. it often wished chubby actually worked like that.
209 2012-09-04 13:43:27 <jgarzik> the 'master master' cluster doesnt do much but refer, unless '/' changes
210 2012-09-04 13:43:56 <jgarzik> use some metrics / hueristics to decide when a new downstream master is needed, based on load
211 2012-09-04 13:44:42 <jgarzik> over time, 99% of clients simply 'know' what master/cluster is needed for any given portion of the namespace + their app-specific usage pattern
212 2012-09-04 13:44:52 <TD> yes. that is much like bigtable.
213 2012-09-04 13:45:52 <BlueMatt> god...seems like the chrome guys have never tested chrome's battery performance with rc6...it nearly doubles my battery rate...
214 2012-09-04 13:46:01 <TD> did anyone investigate why there are so many duplicate blocks flying around? i see "ERROR: ProcessBlock() : already have block" pretty often in my logs
215 2012-09-04 13:46:22 <BlueMatt> TD: its probably two nodes announcing at nearly the same time
216 2012-09-04 13:46:32 <BlueMatt> (we dont bother to deduplicate requests for blocks when that happens)
217 2012-09-04 13:46:42 <TD> yeah maybe
218 2012-09-04 13:46:51 <sipa> i kinda decided not to touch the IBD sync mechanism again until we implemente reverse-header sync
219 2012-09-04 13:46:55 <BlueMatt> s/nearly the same time/in the same period during which we were processing some message from another node"
220 2012-09-04 13:47:15 <BlueMatt> sipa: heh, oops...
221 2012-09-04 13:47:44 <sipa> BlueMatt: i don't mind others touching it, if there are improvements possible
222 2012-09-04 13:48:29 <sipa> but the only real solution is downloading headers first, and then syncing a known long chain through the block tree
223 2012-09-04 13:48:51 <BlueMatt> that is true
224 2012-09-04 13:49:43 <sipa> ultraprune's database design already allows that, by the way, but header sync is not implemented yet obviously
225 2012-09-04 13:52:07 <MC-Eeepc> oh wow this ultraprune is now synched
226 2012-09-04 13:52:16 <MC-Eeepc> id say that was about 3 days total
227 2012-09-04 13:53:24 <TD> it took you 3 days to sync a block chain with sipas branch?
228 2012-09-04 13:53:43 <sipa> TD: on an EEE pc with datadir on a usb stick, iirc :)
229 2012-09-04 13:53:44 <gmaxwell> WRT: crash recover, the paper gavinandresen mentioned is linked from here: https://lwn.net/Articles/191059/
230 2012-09-04 13:55:09 <MC-Eeepc> still a lot faster than normal
231 2012-09-04 13:55:24 <TD> oh
232 2012-09-04 13:57:01 <sipa> MC-Eeepc: i'll soon have a branch that should be even faster (a lot faster in your setting, probably)
233 2012-09-04 13:57:15 <TD> what branch is that?
234 2012-09-04 13:57:32 <BlueMatt> ultraprune+leveldb, Id guess
235 2012-09-04 13:57:37 <sipa> TD: he's currently using ultraprune without leveldb
236 2012-09-04 13:58:15 <TD> oh
237 2012-09-04 13:59:11 <MC-Eeepc> sipa hit me up when its ready
238 2012-09-04 13:59:33 <MC-Eeepc> is there any way to get a total download time out of logfiles on this or what
239 2012-09-04 13:59:59 <BlueMatt> use -logtimestamps
240 2012-09-04 14:01:37 <MC-Eeepc> this is stuck on 1 block remaining now hmmmm
241 2012-09-04 14:02:22 <TD> gavinandresen: the bitcoin price has clearly gone over some kind of threshold. maybe payouts should be lower? or a function of exchange rate?
242 2012-09-04 14:02:46 <copumpkin> TD: the network doesn't know about price
243 2012-09-04 14:03:17 <sipa> edcba: regarding 5x speed increase; we're talking 41min vs 10 min for syncing the blockchain up to 193k
244 2012-09-04 14:05:13 <MC-Eeepc> yep pretty well stuck on 1 block remaining
245 2012-09-04 14:05:16 <MC-Eeepc> strange
246 2012-09-04 14:05:19 <MC-Eeepc> it ws synced
247 2012-09-04 14:05:46 <MC-Eeepc> choked on the next produced block?
248 2012-09-04 14:05:50 <sipa> doubtful
249 2012-09-04 14:06:33 <denisx> sipa: can I test it on freebsd with zfs?
250 2012-09-04 14:08:03 <MC-Eeepc> why has it stopped then
251 2012-09-04 14:08:11 <MC-Eeepc> also i keep losing my peer connections
252 2012-09-04 14:08:35 <sipa> i suspect timeouts?
253 2012-09-04 14:09:01 <sipa> because your node is so slow for processing things, the network threads blocks so long that connections timeout?
254 2012-09-04 14:09:50 <MC-Eeepc> maybe
255 2012-09-04 14:10:04 <MC-Eeepc> getting a lot of misbehaving IPs in the log
256 2012-09-04 14:10:25 <gmaxwell> 0_o
257 2012-09-04 14:10:35 <gmaxwell> Uh oh.
258 2012-09-04 14:10:43 <gmaxwell> MC-Eeepc: pastebin log please.
259 2012-09-04 14:10:46 <sipa> oooh i get it
260 2012-09-04 14:10:51 <MC-Eeepc> Misbehaving: 176.212.16.27:8333 (0 -> 100) DISCONNECTING
261 2012-09-04 14:11:01 <TD> that's not very helpful ...
262 2012-09-04 14:11:03 <gmaxwell> MC-Eeepc: I need to know what happens before that.
263 2012-09-04 14:11:09 <sipa> you're using an older ultraprune build that had canonical sig checking enabled
264 2012-09-04 14:11:13 <gmaxwell> oh.
265 2012-09-04 14:11:14 <gmaxwell> hah
266 2012-09-04 14:11:23 <MC-Eeepc> how much do you want this is a 7mb logfile
267 2012-09-04 14:11:34 <sipa> the last 50 lines
268 2012-09-04 14:13:20 <MC-Eeepc> http://pastebin.com/f8YptJNw
269 2012-09-04 14:13:23 <jgarzik> dammit
270 2012-09-04 14:13:31 <jgarzik> ACTION broke the build... fixing
271 2012-09-04 14:13:42 <MC-Eeepc> well there are 2 misbehaving things in there but ive seen more
272 2012-09-04 14:14:22 <sipa> MC-Eeepc: yes, what i expected; my fault and already fixed in later code
273 2012-09-04 14:14:43 <MC-Eeepc> alrighty
274 2012-09-04 14:16:50 <jgarzik> uh oh
275 2012-09-04 14:16:58 <jgarzik> us4's public node crashed: http://pastebin.com/PZXWfZ3A
276 2012-09-04 14:16:58 <sipa> jgarzik: what did you do?
277 2012-09-04 14:17:23 <sipa> jgarzik: i see that message often, but it doesn't crash
278 2012-09-04 14:18:53 <jgarzik> sipa: it crashed here. I worry about remote peers triggering uncaught exceptions :/
279 2012-09-04 14:18:58 <jgarzik> that's ThreadMessageHandler
280 2012-09-04 14:19:43 <jgarzik> ProcessMessages(ping, 0 bytes) : Exception 'CDataStream::read() : end of data' caught, normally caused by a message being shorter than its stated length
281 2012-09-04 14:19:51 <jgarzik> ^^ obviously _some_ problems are caught
282 2012-09-04 14:20:04 <sipa> strange
283 2012-09-04 14:20:10 <TD> i've seen that. it must be due to the ping nonce change
284 2012-09-04 14:20:19 <jgarzik> TD: yep, and a bug in the reference impl
285 2012-09-04 14:20:20 <TD> somehow
286 2012-09-04 14:20:25 <jgarzik> known bug
287 2012-09-04 14:20:39 <TD> is that the one that got fixed to do with 0 being uint32 instead of uint64 or something?
288 2012-09-04 14:20:40 <BlueMatt> jgarzik: sorry...jenkins should be back up in a few hours...
289 2012-09-04 14:21:42 <Diapolo> BlueMatt: Good news, indeed!
290 2012-09-04 14:22:08 <sipa> jgarzik: can you let me know when the build is fixed? seems i rebased against master at exactly the wrong time
291 2012-09-04 14:22:24 <jgarzik> sipa: RSN
292 2012-09-04 14:22:39 <jgarzik> building test_bitcoin takes forever for some reason
293 2012-09-04 14:22:55 <BlueMatt> its bitcoind+test stuff, so its just longer bitcoind...
294 2012-09-04 14:23:28 <BlueMatt> someone wanna finally take a hatchet to bitcoinrpc.cpp?
295 2012-09-04 14:23:44 <Diapolo> sipa: Could you take a look at my network-init pulls at least the fixing-stuff one(s) ;)?
296 2012-09-04 14:23:56 <sipa> Diapolo: soon
297 2012-09-04 14:24:28 <Diapolo> Thanks, getting zero feedback is a tad ... counter motivating in the end ^^. Which is no personal critic.
298 2012-09-04 14:24:58 <jgarzik> sipa: ok
299 2012-09-04 14:25:19 <jgarzik> BlueMatt: bitcoinrpc.cpp already hatcheted quite a bit...
300 2012-09-04 14:25:28 <jgarzik> BlueMatt: did you have something specific in mind?
301 2012-09-04 14:25:59 <BlueMatt> oh, hey, it is now...how did I miss that?
302 2012-09-04 14:26:16 <BlueMatt> ACTION needs to pay more attention
303 2012-09-04 14:26:34 <gmaxwell> BlueMatt: jeff slipped in that breaks-all-pull-request change when no one was looking. :P
304 2012-09-04 14:26:49 <Diapolo> Which one was that btw.?
305 2012-09-04 14:27:14 <BlueMatt> gmaxwell: heh
306 2012-09-04 14:27:31 <BlueMatt> well done jgarzik
307 2012-09-04 14:28:55 <jgarzik> HTTP server and client still need separating ;p
308 2012-09-04 14:36:56 <jgarzik> CTxMemPool::accept() : accepted 0b80e93e7d (poolsz 3689)
309 2012-09-04 14:37:01 <jgarzik> tx pool is quite large
310 2012-09-04 14:38:05 <sipa> denisx: please do! (branch leveldb+ultraprune on sipa/bitcoin.git)
311 2012-09-04 14:38:14 <BlueMatt> wasnt it >9k at one point?
312 2012-09-04 14:39:20 <sipa> MC-Eeepc: ok, building
313 2012-09-04 14:46:17 <sipa> hmm, leveldb needs snappy
314 2012-09-04 14:46:42 <sipa> any gitian-builder expert that knows how to get that available in gitian?
315 2012-09-04 14:47:14 <sipa> especially for windows i suppose that's an extra dependency
316 2012-09-04 14:47:41 <gavinandresen> what is snappy?
317 2012-09-04 14:48:07 <sipa> google's very fast compression library
318 2012-09-04 14:48:18 <BlueMatt> usually: just build with faketime, sometimes also need -j1 forced, sometimes need to do more debugging
319 2012-09-04 14:48:35 <sipa> here i just did apt-get install libsnappy-dev :)
320 2012-09-04 14:48:44 <BlueMatt> well that works for linux...
321 2012-09-04 14:48:47 <gmaxwell> sipa: I assume snappy does ~nothing useful for us? Can we just disable it?
322 2012-09-04 14:49:31 <sipa> gmaxwell: maybe, but that will make our leveldb instance incompatible with maybe other programs that exist for interacting with leveldb, and do use compression
323 2012-09-04 14:51:00 <BlueMatt> we are already static linking...
324 2012-09-04 14:51:13 <sipa> exactly because of that
325 2012-09-04 14:51:26 <BlueMatt> oh, you mean the db itself
326 2012-09-04 14:51:29 <BlueMatt> file
327 2012-09-04 14:51:31 <sipa> yes
328 2012-09-04 14:51:34 <TD> i don't think there are any programs that access leveldb directly
329 2012-09-04 14:51:37 <BlueMatt> meh
330 2012-09-04 14:51:41 <sipa> hmm, ok
331 2012-09-04 14:51:47 <TD> given that the db consists almost entirely of random numbers .... i'd simplify by removing it
332 2012-09-04 14:51:49 <TD> though i don't remember this issue
333 2012-09-04 14:52:01 <TD> why do you say it needs snappy?
334 2012-09-04 14:52:35 <sipa> because i get link errors against symbols from libsnappy otherwise
335 2012-09-04 14:54:41 <sipa> hmm, that build script autodetects whether snappy is available
336 2012-09-04 14:59:18 <Diapolo> jgarzik: Can you have a look at your last merge in the Github comments.
337 2012-09-04 15:00:40 <BlueMatt> heh, was going to kernel.org for other reasons and saw jgarzik's branch titled "FUSE fs w/ Berkeley DB backend."
338 2012-09-04 15:01:45 <sipa> oh dear
339 2012-09-04 15:08:19 <Diapolo> reminds me of Half-life 1 sipa ^^
340 2012-09-04 15:11:49 <BlueMattBot> Project Bitcoin build #55: ABORTED in 1 hr 21 min: http://jenkins.bluematt.me/job/Bitcoin/55/
341 2012-09-04 16:48:52 <BlueMatt> TD: adding a few test-cases to fullverif stuff now: looks like mostly all thats missing is Transaction.verify (aka CheckTransaction) tx.IsFinal and all the MoneyRange stuff
342 2012-09-04 16:48:55 <BlueMatt> ?
343 2012-09-04 16:49:15 <TD> ok
344 2012-09-04 16:49:55 <BlueMatt> do you have any objections to adding those as block test-cases for bitcoindcomparisontool instead of standard test-cases?
345 2012-09-04 16:50:08 <BlueMatt> (they are still run as a part of junit)
346 2012-09-04 16:53:08 <sipa> TD: mind if i'd just comment the code for detecting snappy out?
347 2012-09-04 16:53:19 <sipa> in build_detect_platform
348 2012-09-04 16:53:23 <TD> why doesn't it work for you ?
349 2012-09-04 16:53:48 <sipa> it works; i have snappy installed, so it builds and links against it
350 2012-09-04 16:53:58 <sipa> but building bitcoin then requires a -lsnappy too
351 2012-09-04 16:54:09 <TD> oh, i see
352 2012-09-04 16:54:23 <TD> i think there is a linker flag that lets you do optional linking
353 2012-09-04 16:54:31 <TD> it's been years since i did this kind of stuff
354 2012-09-04 16:54:34 <TD> sure comment it out for now
355 2012-09-04 16:54:37 <sipa> there is --as-needed
356 2012-09-04 16:54:43 <sipa> or something like that
357 2012-09-04 16:54:56 <sipa> but i'm sure it'll be some work before it works on all platforms
358 2012-09-04 16:55:20 <sipa> so if we decide not to use snappy anyway (it doesn't help), we may as well disable it entirely (for now)
359 2012-09-04 16:55:46 <sipa> though i don't like messing with the leveldb code
360 2012-09-04 16:55:56 <sipa> s/code/source tree/
361 2012-09-04 16:58:55 <TD> ok
362 2012-09-04 16:59:01 <TD> it's already patched for windows anyway
363 2012-09-04 17:46:32 <Diapolo> Do you guys also gett a message of frigg when login in? That thing spams me everytime I login... with a version message or sth. like that.
364 2012-09-04 17:52:41 <shhh> does the rpc interface allow multiple clients, and, is that safe ?
365 2012-09-04 17:53:41 <shhh> i mean as long as one would read only the other one doing commands which modify state ?
366 2012-09-04 17:54:06 <sipa> commands are still only processed one at a time
367 2012-09-04 17:54:33 <shhh> ok, no matter how much clients they get queued up and executed in the order they came in ?
368 2012-09-04 17:54:47 <shhh> and if one takes time, the other client hangs there waiting i guess
369 2012-09-04 17:55:25 <sipa> yes
370 2012-09-04 17:55:35 <sipa> there are efforts to gradually reduce that constraint
371 2012-09-04 17:55:55 <sipa> at least the RPC processing happens multithreaded now
372 2012-09-04 17:56:08 <sipa> but the actual execution still needs a global lock for almost all commands
373 2012-09-04 17:56:18 <jgarzik> each RPC connection runs in a separate thread. however, the threads all take a global lock, as sipa notes.
374 2012-09-04 17:56:40 <jgarzik> the lock is only held during RPC execution, and not during network I/O
375 2012-09-04 17:59:29 <shhh> sounds good for me
376 2012-09-04 18:00:09 <shhh> i am using the rawtransaction stuff permanently with a lot of statechanges and need to update a website too, which will ocassionally read balance
377 2012-09-04 18:06:04 <shhh> i would like to export selectcoins over rpc, maybe it could be useful, shoud i clone it in git so you could maybe merge it back if you want ?
378 2012-09-04 18:06:34 <shhh> i do not want to do the coinselection most of the time ;) and the fee calculation ...
379 2012-09-04 18:13:08 <gmaxwell> shhh: then why are you using the raw transaction API at all?
380 2012-09-04 18:14:03 <shhh> because the higherlevel api does not let me select the exact coins
381 2012-09-04 18:14:38 <shhh> i would patch selectcoins to be able to specify "a coin from this transaction must be included"
382 2012-09-04 18:14:52 <kjj_> I think he is asking why you need to select them in the first place
383 2012-09-04 18:14:55 <shhh> so it will be safe to work 0-confirms
384 2012-09-04 18:15:13 <gmaxwell> shhh: That doesn't make it 'safe' to work with 0-confirms.
385 2012-09-04 18:15:36 <kjj_> he wasnt to make an outgoing green address
386 2012-09-04 18:15:44 <kjj_> er, wants
387 2012-09-04 18:15:47 <sipa> he wants to do what SD does
388 2012-09-04 18:15:52 <gmaxwell> shhh: it just shifts the risk of a dead transaction onto the recipents of your funds.
389 2012-09-04 18:15:55 <shhh> right sipa ;)
390 2012-09-04 18:15:56 <sipa> payouts taken from the inputs sent
391 2012-09-04 18:16:26 <gmaxwell> shhh: you shouldn't do that, it's bad for bitcoin because it creates a bunch of unnecessary traffic, and also leave your customers exposed to attack.
392 2012-09-04 18:16:48 <shhh> if i include the bet in the output it will be safe as if it was fake/doublespend it would not be included in the blockchain, and therefore my answer would not be included too
393 2012-09-04 18:17:09 <sipa> why oh who does suddenly everyone want to create a betting site?
394 2012-09-04 18:17:32 <gmaxwell> shhh: yea, fantastic, but then you use a child output from that transaction to supply coins for another user's output and they get screwed if some other user produces a conflicing transaction.
395 2012-09-04 18:17:34 <Eliel> it's the new fad :P
396 2012-09-04 18:17:35 <jgarzik> <Diapolo> jgarzik: Can you have a look at your last merge in the Github comments. <<-- can you be more specific?
397 2012-09-04 18:18:05 <gmaxwell> sipa: because existing betting sites displaing (read: potentially fabricating) evidence of enormous income.
398 2012-09-04 18:18:32 <shhh> gmaxwell, no i wont, i put those childoutputs in my "bank", and wait for them to be confirmed
399 2012-09-04 18:18:40 <btctrader22> devs against market forces
400 2012-09-04 18:18:54 <gmaxwell> btctrader22: don't be an idiot.
401 2012-09-04 18:19:07 <gmaxwell> btctrader22: if there wasn't technology there would be no market at all.
402 2012-09-04 18:19:26 <sipa> still, he has a point
403 2012-09-04 18:19:42 <gmaxwell> No, I don't agree that he does.
404 2012-09-04 18:19:44 <Diapolo> jgarzik: Yes, we now have 2x StartupTime printfs
405 2012-09-04 18:19:59 <gmaxwell> Go ahead, happily remove the anti-dos code; and bitcoin can just be inoperable for everyone.
406 2012-09-04 18:20:07 <Diapolo> jgarzik: https://github.com/bitcoin/bitcoin/commit/dcb14198bb2b55a673aa27bc5fa036d809556dbe#L0L472
407 2012-09-04 18:20:29 <gmaxwell> The exact boundary at which the anti-dos behavior activates isn't some golden rule handed down by the flawless hand of the free market; it's a complicated compromise.
408 2012-09-04 18:21:40 <gmaxwell> If someone advocated free market then they should also be demanding that users pick which transactions their nodes process... and a bunch of inconsistent processing rules would make the network much less secure and reliable.
409 2012-09-04 18:22:47 <shhh> gmaxwell, any point agains putting childoutput in a bank and waiting for it to be confirmed before being put in another output for a differnt user ?
410 2012-09-04 18:22:56 <shhh> should be safe i guess...
411 2012-09-04 18:23:00 <btctrader22> you're missing my point. satoshi dice success is a clear indicator of what bitcoin users want to do with the system.
412 2012-09-04 18:23:16 <btctrader22> devs not being happy about it seems kinda counterproductive to me
413 2012-09-04 18:23:19 <has_many> "get rich quick"
414 2012-09-04 18:23:40 <shhh> wont get rich, its an experiment here and will be a low user site
415 2012-09-04 18:23:44 <shhh> but
416 2012-09-04 18:23:50 <btctrader22> and devs beinggrumpy about satoshi dice gets in the way of bitcoin getting bigger.
417 2012-09-04 18:23:56 <gmaxwell> btctrader22: As far as I know the 'success' is just BS traffic created to pump the IPO. And regardless, it's activity that consumes enormous transaction load for a tiny number of users; it's not viable.
418 2012-09-04 18:23:58 <shhh> 0 confirm is just more usable
419 2012-09-04 18:24:12 <shhh> i hate to wait if i want to play and so do a lot of people
420 2012-09-04 18:24:13 <btctrader22> we should work out a solution to allow dice-like solution to be feasible without hurting usage.
421 2012-09-04 18:24:16 <gmaxwell> shhh: well it means your bank can be robbed.
422 2012-09-04 18:24:25 <amiller> satoshi dice is eating up some of our "startup fund"
423 2012-09-04 18:24:37 <gmaxwell> ^ that.
424 2012-09-04 18:24:49 <sipa> btctrader22: that's the point; without intervention, what SD does will cause bitcoin to become less usable for everyone
425 2012-09-04 18:24:51 <shhh> gmaxwell, my bank could be robbed ? as in if my server was broken into ?
426 2012-09-04 18:25:05 <gmaxwell> It's basically increasing the cost of running a bitcoin node _enormously_; while the actual usage of bitcoin is not growing enough to support that cost.
427 2012-09-04 18:25:19 <sipa> btctrader22: of course we'll do what we can, but being grumpy is hopefully because we have a bit better view of the consequences
428 2012-09-04 18:26:04 <btctrader22> maybe, but I'm not convinced that throttling something like that is necessarily the optimal response.
429 2012-09-04 18:26:17 <gmaxwell> shhh: no, it means that people can take back funds they'd lose to you??? selectively only when they lose.
430 2012-09-04 18:26:58 <gmaxwell> shhh: the risk can't eliminated without waiting for confirmations before taking _any_ action; all you could hope to do is externalize it by hotpotatoing those inputs off to someone else.
431 2012-09-04 18:26:58 <sipa> btctrader22: and throttling is also not what is being done
432 2012-09-04 18:27:56 <Luke-Jr> btctrader22: educating people to not fund DDoS attacks like SD would help
433 2012-09-04 18:28:19 <shhh> gmaxwell, but if they send an invalid transaction at first it will never be confirmed, how would they stop a valid transaction from being confirmed ?
434 2012-09-04 18:28:20 <Luke-Jr> btctrader22: as would more developers or paid bounties to get the problems ironed out
435 2012-09-04 18:28:35 <gmaxwell> And, of course, encouraging people to operate normal sites that don't create tremendous transaction bloat is helpful.
436 2012-09-04 18:29:02 <gmaxwell> shhh: via a finney attack, or simply broadcasting conflicting transactions directly to many nodes.
437 2012-09-04 18:29:37 <Vinnie_win> Can someone point me in the direction of understanding what a "share" is in the mining context?
438 2012-09-04 18:29:56 <sipa> ;;google share site:bitcoin.it
439 2012-09-04 18:29:57 <gribble> Pooled mining - Bitcoin: <https://en.bitcoin.it/wiki/Pooled_mining>; Comparison of mining pools - Bitcoin: <https://en.bitcoin.it/wiki/Comparison_of_mining_pools>; Mining pool reward FAQ - Bitcoin: <https://en.bitcoin.it/wiki/Mining_pool_reward_FAQ>
440 2012-09-04 18:30:14 <Vinnie_win> thanks
441 2012-09-04 18:30:40 <shhh> gmaxwell, as my bank will have funds closely adjusted i could live with a finney attack
442 2012-09-04 18:30:52 <Luke-Jr> Vinnie_win: a "block" mined at a higher target
443 2012-09-04 18:31:05 <gmaxwell> shhh: I don't thikn I understood what you were saying there.
444 2012-09-04 18:31:06 <shhh> if they put that effort into it, and get a miner involved, congrats, take the coins ;)
445 2012-09-04 18:31:27 <gmaxwell> shhh: people can just purchase mining via gpumax.
446 2012-09-04 18:31:30 <Vinnie_win> Luke-Jr: I dont really understand. You mean a found hash collision at a higher difficulty?
447 2012-09-04 18:31:42 <gmaxwell> shhh: and they don't need a miner involved to just broadcast multiple conflicting versions.
448 2012-09-04 18:31:44 <sipa> Vinnie_win: we don't look for collisions; we look for a preimage
449 2012-09-04 18:31:52 <shhh> gmaxwell, would not make much sense for 0.1BTC max bets ... :)
450 2012-09-04 18:31:56 <sipa> Vinnie_win: and higher target means lower difficulty
451 2012-09-04 18:32:21 <Vinnie_win> ACTION googles
452 2012-09-04 18:32:42 <gmaxwell> shhh: It's O(1) effort to reverse any number of unconfirmed transactions (well, up to the size of the block you generate)
453 2012-09-04 18:32:45 <sipa> Vinnie_win: a collision is looking for two different inputs with the same hash
454 2012-09-04 18:32:55 <sipa> Vinnie_win: a preimage is looing for one input that has a given hash
455 2012-09-04 18:33:09 <Vinnie_win> sipa: So a share is a piece of data which, when hashed, has a certain number of zeroes in the result (defined by the difficulty)
456 2012-09-04 18:33:18 <sipa> Vinnie_win: yes
457 2012-09-04 18:33:20 <gmaxwell> shhh: Why not a regular deposit/withdraw site? If you want instant payments you could still do that??? and you could also accept mtgox codes.
458 2012-09-04 18:33:29 <sipa> Vinnie_win: it's an almost-valid block
459 2012-09-04 18:33:39 <Vinnie_win> sipa: How much lower than the difficulty does it go? Is it always 32 bits of zeroes?
460 2012-09-04 18:33:53 <sipa> Vinnie_win: shares are typically +- difficulty 1
461 2012-09-04 18:34:03 <sipa> which means 32 bits of zeroes
462 2012-09-04 18:34:10 <shhh> gmaxwell
463 2012-09-04 18:34:16 <gmaxwell> Vinnie_win: some miners can't do less than difficulty 1.
464 2012-09-04 18:34:21 <sipa> (not exactly though, as difficulty 1 is 0x00000000FFFF0000....)
465 2012-09-04 18:34:28 <shhh> so one should rob satoshi dice ? ;)
466 2012-09-04 18:34:52 <Vinnie_win> sipa: I see. So along the way to trying to solve a block, shares are produced according to the distribution of the hash function and they serve as proof that you did work
467 2012-09-04 18:35:00 <sipa> Vinnie_win: exactly
468 2012-09-04 18:35:23 <Vinnie_win> sipa: I thought that pool servers kept the block contents secret, and only transmitted a partial hash to miners
469 2012-09-04 18:35:29 <sipa> Vinnie_win: and pools use different formulas to use those shares to determine the distribution of income from shares that match the actual target, and thus result in real income
470 2012-09-04 18:35:43 <Vinnie_win> sipa: i.e. there's no way for a miner in a pool to produce the solved block when they find a preimage
471 2012-09-04 18:35:51 <Vinnie_win> sipa: Yes, I"m reading the paper from Meni
472 2012-09-04 18:36:03 <sipa> Vinnie_win: indeed, most pools only send the header info, and not the transactions used to create it
473 2012-09-04 18:36:07 <gmaxwell> Vinnie_win: No. thats not how pool security works. (it might be true in some cases that the miner can't; but thats not what makes it secure)
474 2012-09-04 18:36:18 <sipa> what gmaxwell says
475 2012-09-04 18:36:20 <Vinnie_win> What stops a miner from submitting the block themselves, if they solve it
476 2012-09-04 18:36:36 <sipa> they can
477 2012-09-04 18:36:39 <kjj_> uh, the coinbase isn't for their wallet, so there is no point
478 2012-09-04 18:36:40 <sipa> if they had the data
479 2012-09-04 18:36:40 <Vinnie_win> the address in the header pays the pool operator?
480 2012-09-04 18:36:50 <sipa> Vinnie_win: not the header, but the coinbase
481 2012-09-04 18:37:07 <sipa> but indeed, you need to decide the destination of the subsidy before starting to mine
482 2012-09-04 18:37:19 <sipa> as that is encoded in the block itself, and influences the hash
483 2012-09-04 18:37:21 <Vinnie_win> I see
484 2012-09-04 18:37:47 <Vinnie_win> how does p2pool work? Doesn't there need to be a central authority to handle the payouts?
485 2012-09-04 18:38:04 <sipa> no, it uses a coinbase that pays out everyone
486 2012-09-04 18:38:11 <sipa> and everyone agrees how that payout should happen
487 2012-09-04 18:38:25 <sipa> so they can see shares by others that have the correct payouts
488 2012-09-04 18:38:26 <Luke-Jr> Vinnie_win: it uses PPLNS while adjusting share target to make an effective minimum payout
489 2012-09-04 18:38:54 <Vinnie_win> How does everyone agree on the same set of transactions to include
490 2012-09-04 18:38:59 <kjj_> they don't
491 2012-09-04 18:39:02 <Luke-Jr> they don't need to
492 2012-09-04 18:39:09 <kjj_> each node has full autonomy except on the coinbase
493 2012-09-04 18:39:13 <gmaxwell> p2pool uses basically the same consensus algorithim to agree on what the payouts are, and miners then build blocks according to that. Attempted blocks that paid out correctly are accepted by peers as evidence of work, which gets them included in the consensus payout.
494 2012-09-04 18:39:28 <gmaxwell> er. same consensus algorithim as bitcoin.
495 2012-09-04 18:39:42 <sipa> meh. gitian build of leveldb failed
496 2012-09-04 18:39:49 <Vinnie_win> Thanks for enlightening me
497 2012-09-04 18:39:55 <gmaxwell> The shares form a chain, each extending the last, and the longest valid sharechain is the agreed one.
498 2012-09-04 18:41:16 <Vinnie_win> is there a doc that explains how p2pool works
499 2012-09-04 18:42:00 <gmaxwell> https://en.bitcoin.it/wiki/P2Pool
500 2012-09-04 18:42:14 <Vinnie_win> oh...p2pool has its own blockchain ??
501 2012-09-04 18:42:25 <sipa> yup
502 2012-09-04 18:42:39 <gmaxwell> 'sharechain' really, but whatever you want to call it. :P
503 2012-09-04 18:43:03 <Vinnie_win> And all participants see the chain so they can validate the contents of the coinbase of a solved block
504 2012-09-04 18:45:21 <MC-Eeepc> https://en.bitcoin.it/wiki/Scalability is this page been updated anytime recently
505 2012-09-04 18:46:09 <gmaxwell> There were a bunch of things added to it that TD fobbed off onto the talk page.
506 2012-09-04 18:46:24 <gmaxwell> (perhaps I shouldn't complain, I'm certantly guilty of that myself :P )
507 2012-09-04 18:46:47 <sipa> last change seems to be at july 28
508 2012-09-04 18:49:46 <MC-Eeepc> fairly recent
509 2012-09-04 18:50:06 <sipa> TD wanted to update the page, afaik
510 2012-09-04 18:50:24 <TD> yes
511 2012-09-04 18:50:39 <sipa> leveldb's makefile insists on using -std=c++0x for windows
512 2012-09-04 18:50:50 <sipa> i wonder why
513 2012-09-04 18:51:06 <sipa> (gitian's gcc doesn't support that yet, ancient beast)
514 2012-09-04 18:52:09 <Luke-Jr> (future gccs probably won't support that either, since it's c++11 :p)
515 2012-09-04 18:52:57 <shhh> why does bitcoins makefile.unix have -ldl ?
516 2012-09-04 18:53:26 <shhh> libdl == linux only
517 2012-09-04 18:54:10 <sipa> patches welcome, but linux is currently the only maintained platform for which makefile.unix is used
518 2012-09-04 18:55:12 <shhh> oh i see, i can do solaris and free/openbsd have it running there
519 2012-09-04 18:55:19 <MC-Eeepc> Vector76 attack
520 2012-09-04 18:55:24 <MC-Eeepc> FUCK
521 2012-09-04 18:56:14 <sipa> shhh: afaik 0.7.0rc1 compiles without problems on some bsd at least
522 2012-09-04 18:56:41 <gmaxwell> shhh: libdl is not 'linux only', e.g. it's how you get dlopen in solaris oo.
523 2012-09-04 18:56:42 <shhh> sipa, does not on freebsd and openbsd, had to remove -ldl :)
524 2012-09-04 19:02:50 <Luke-Jr> shhh: you can be the new BSD maintainer! :P
525 2012-09-04 19:02:59 <Luke-Jr> shhh: hint: there's some code commented because it needs testing on BSD first
526 2012-09-04 19:04:35 <shhh> Luke-Jr, there is a macosx maintainer already ? i could do that too ;)
527 2012-09-04 19:05:13 <Luke-Jr> shhh: Mac != BSD
528 2012-09-04 19:05:30 <shhh> luke, got osx here, and earn my money on it
529 2012-09-04 19:05:39 <shhh> osx is closer to freebsd as you might think ;)
530 2012-09-04 19:05:48 <sipa> gavin is OSX maintainer currently
531 2012-09-04 19:06:14 <shhh> i see, maybe i can look into the osx stuff which was done, some things should be recycleable for freebsd
532 2012-09-04 19:07:03 <Luke-Jr> shhh: I'm aware of the myth that OSX and FreeBSD are closely related.
533 2012-09-04 19:07:49 <sipa> well OSX and BSD are definitely more closely related than any other combination of two platforms from the list (Windows,Linux,OSX,BSD)
534 2012-09-04 19:08:27 <sipa> let's keep it at that
535 2012-09-04 19:09:26 <Luke-Jr> ACTION thinks Linux and BSD are closer, but meh
536 2012-09-04 19:25:12 <shhh> actually OSX is more closely related to older versions of freebsd
537 2012-09-04 19:25:47 <shhh> atleast from the unixish side of things , the mach side is a different story
538 2012-09-04 19:26:58 <Luke-Jr> shhh: older versions, as in before it was FreeBSD at all ;)
539 2012-09-04 19:27:17 <shhh> no, as in FreeBSD 4
540 2012-09-04 19:27:18 <Luke-Jr> you could just as well say FreeBSD is based on OSX
541 2012-09-04 19:27:26 <shhh> but they retrofitted a lot of things
542 2012-09-04 19:27:53 <shhh> OSX is not based on FreeBSD , its mach and FreeBSD used for its unix compatibility layer
543 2012-09-04 19:28:24 <shhh> the BSD part runs as task inside the mach kernel
544 2012-09-04 19:31:18 <shhh> one can even patch the bsd part runtime by using mach_write on the bsd task to modify its memory ;)
545 2012-09-04 19:48:49 <gmaxwell> BlueMatt: https://github.com/blog/1227-commit-status-api
546 2012-09-04 19:49:53 <BlueMatt> gmaxwell: nice, does gavinandresen wanna hook BitcoinPullTester up with whatever permissions are required to set that flag?
547 2012-09-04 19:50:49 <BlueMatt> or can you do that?
548 2012-09-04 19:55:35 <gmaxwell> BlueMatt: I _think_ anyone who can commit can create an auth.
549 2012-09-04 19:55:58 <BlueMatt> well, BitcoinPullTester cant ;)
550 2012-09-04 19:56:50 <gmaxwell> Right, I'm looking.
551 2012-09-04 19:58:07 <sipa> \\o/ leveldb windows gitian build
552 2012-09-04 19:58:40 <sipa> BlueMatt: how hard would it be to make the gitian windows build start using that more recent g++?
553 2012-09-04 19:58:57 <sipa> wumpus said something about a more recent one being available in another packages
554 2012-09-04 19:59:22 <BlueMatt> not very, would have to s/i586-mingw32msvc-/w64-something.../ and upgrade to 12.04, but that shouldnt be bad
555 2012-09-04 19:59:52 <BlueMatt> i686-w64-mingw32- is what it is on my system
556 2012-09-04 20:00:00 <sipa> right, moving to 12.04 is not a problem for the windows builds
557 2012-09-04 20:00:05 <BlueMatt> yep
558 2012-09-04 20:00:31 <sipa> right now i had to manually hack some leveldb-win code, as it relied on -std=c++0x
559 2012-09-04 20:02:11 <BlueMatt> gmaxwell: hmm...it may only be possible to limit it further with oath...which Im much too lazy to set up for pull tester...you are welcome to do that part if you want (Ill send you the script)
560 2012-09-04 20:02:49 <gmaxwell> it looks like the oauth stuff is required to use the api at all.
561 2012-09-04 20:02:59 <BlueMatt> its not (but they want you to think it is)
562 2012-09-04 20:03:07 <BlueMatt> you can use regular http auth if you are lazy
563 2012-09-04 20:04:13 <gmaxwell> BlueMatt: hm. bleh. I'd prefer to not leave my own authentication keys online for it to run either. Kinda annoying.
564 2012-09-04 20:05:40 <BlueMatt> yea...
565 2012-09-04 20:09:03 <gmaxwell> maybe it's possible to authorize an account with only that? I don't have permissions to see those setting on the bitcoin repo, I think.
566 2012-09-04 20:09:23 <BlueMatt> yea, that would be best, I think...have to ask gavinandresen
567 2012-09-04 20:31:14 <sipa> MC-Eeepc: http://bitcoin.sipa.be/builds/ultraprune/0.7.0rc1-81-g33a92bf/
568 2012-09-04 20:31:25 <sipa> (no guarantees)
569 2012-09-04 20:31:50 <MC-Eeepc> whats different in this one
570 2012-09-04 20:32:07 <sipa> leveldb instead of bdb, mostly
571 2012-09-04 20:32:11 <sipa> and some bugfixes
572 2012-09-04 20:32:30 <MC-Eeepc> so this is ldb and ultraprune?
573 2012-09-04 20:32:35 <sipa> yes
574 2012-09-04 20:36:32 <MC-Eeepc> welp its downloading
575 2012-09-04 20:40:51 <MC-Eeepc> seems like this database work is going faster than expected
576 2012-09-04 20:43:33 <shhh> how is the txid generated ?
577 2012-09-04 20:43:55 <sipa> it's the double-SHA256 of the tx data
578 2012-09-04 20:46:08 <shhh> so if one would want his txid to have a specific criteria he could just do slight variations of txdata
579 2012-09-04 20:46:25 <sipa> what criteria?
580 2012-09-04 20:46:36 <gmaxwell> sipa: stop helping the spammer.
581 2012-09-04 20:46:38 <gmaxwell> :P
582 2012-09-04 20:46:58 <gmaxwell> https://github.com/bitcoin/bitcoin/issues/1783 < bleh
583 2012-09-04 20:47:04 <shhh> i just thought that dice site might cheat
584 2012-09-04 20:47:21 <sipa> shhh: they use a secret that's revealed afterwards
585 2012-09-04 20:47:41 <shhh> yeah they have it
586 2012-09-04 20:47:48 <shhh> so the know if a txid would win or loose
587 2012-09-04 20:48:01 <sipa> yes
588 2012-09-04 20:48:05 <sipa> of course
589 2012-09-04 20:48:21 <sipa> gmaxwell: yes, i'm afraid of that when people go mass-upgrading to 0.7.0
590 2012-09-04 20:48:22 <shhh> therefore they could have a box with a client sitting somewere generating tx hashes until one "wins", and only if it wins send it off
591 2012-09-04 20:48:24 <shhh> ;)
592 2012-09-04 20:48:38 <sipa> shhh: i don't see how that helps
593 2012-09-04 20:48:40 <gmaxwell> shhh: sure, or loses.
594 2012-09-04 20:49:02 <sipa> shhh: it's those that do not have the secret that need to create a transaction that wins
595 2012-09-04 20:49:07 <sipa> and they don't know the secret
596 2012-09-04 20:49:25 <sipa> so they can't try multiple tx's
597 2012-09-04 20:49:47 <shhh> sipa, the satoshi guy have the secret, so they could cheat their own site ? ;)
598 2012-09-04 20:49:53 <shhh> thus maximize their winnings
599 2012-09-04 20:49:56 <sipa> what would be the point?
600 2012-09-04 20:50:05 <sipa> they'd have to pay to gains or losses to themself
601 2012-09-04 20:50:26 <shhh> they wont loose if they have the secret ?!
602 2012-09-04 20:50:43 <sipa> i have no idea what you're getting at
603 2012-09-04 20:50:57 <sipa> you're saying that the quizmaster can always win his own game
604 2012-09-04 20:50:59 <sipa> Duhh.
605 2012-09-04 20:51:20 <gmaxwell> ACTION imagines shhh sitting alone in a padded room, with a penny.. muttering HEADS I WIN, TAILS I WIN. and flipping the coin while laughing manically.
606 2012-09-04 20:51:20 <shhh> they claim its proveable that they do not cheat their users
607 2012-09-04 20:51:33 <amiller> they also try to steer people to a particular item
608 2012-09-04 20:51:37 <sipa> shhh: afterwards, yes
609 2012-09-04 20:51:40 <amiller> the biggest payoff one
610 2012-09-04 20:51:46 <sipa> shhh: they could cheat for one day
611 2012-09-04 20:51:48 <amiller> that would be the only one to try to win at
612 2012-09-04 20:51:56 <sipa> and i guess that afterwards they'd be out of business
613 2012-09-04 20:52:00 <shhh> sipa, they could cheat forever ...
614 2012-09-04 20:52:08 <sipa> how so?
615 2012-09-04 20:52:29 <sipa> yes they can create as many winning bets as they want against their own system
616 2012-09-04 20:52:32 <sipa> who cares?
617 2012-09-04 20:52:51 <sipa> that is in no way cheating its users
618 2012-09-04 20:53:01 <gmaxwell> Or losing, for that matter.
619 2012-09-04 20:53:10 <gmaxwell> Which might be cheating their investors but not the users.
620 2012-09-04 20:53:17 <sipa> (except nicer statistics to show, but they might already be doing that now)
621 2012-09-04 20:53:43 <shhh> Guest73506:50 versus 51:49 can be a lot of money
622 2012-09-04 20:53:51 <shhh> argh
623 2012-09-04 20:53:52 <gmaxwell> There are some fun things; e.g. they could use the malleability of transactions to convert winners to losers.
624 2012-09-04 20:54:02 <gmaxwell> but that would leave evidence.
625 2012-09-04 20:54:20 <gmaxwell> they could also simply guess which users won't notice and then simply not pay some of those.
626 2012-09-04 20:57:35 <shhh> looks like someone is already trying to trick them , they just had a lot of 0.001 bets on "less than 1" , still unconfirmed ...
627 2012-09-04 20:57:41 <shhh> ;)
628 2012-09-04 20:59:20 <sipa> why is that an attack?
629 2012-09-04 20:59:26 <sipa> sounds more like a fool...
630 2012-09-04 21:00:07 <shhh> because its an hour ago and all those 50 something transactions are not included in the blockchain
631 2012-09-04 21:00:11 <shhh> 4 of them are
632 2012-09-04 21:00:13 <shhh> hehe
633 2012-09-04 21:00:16 <sipa> so?
634 2012-09-04 21:01:03 <gmaxwell> shhh: a number (who knows how many?) miners and nodes won't relay or mine those transactions.
635 2012-09-04 21:01:28 <gmaxwell> There are patches floating around to block them, and some people run them. Some who aren't blocking them entirely deprioritize them.
636 2012-09-04 21:02:01 <shhh> i see
637 2012-09-04 21:02:44 <shhh> is that really a good solution ? filtering or depriorzing selected payments ?
638 2012-09-04 21:02:53 <gmaxwell> No, its not.
639 2012-09-04 21:03:05 <gmaxwell> But, the world is full of adhoc solutions.
640 2012-09-04 21:03:34 <gmaxwell> It doesn't have to be completely effective to help improve transaction processing time for the bulk of regular transactions that actually _care_ about their processing time.
641 2012-09-04 21:04:27 <gmaxwell> My personal preference is to depriortize based on repeated address use, so that implicitly standing relationships that repeat addresses are treated with the lower priority they probably need.
642 2012-09-04 21:04:35 <gmaxwell> But I think I'm roughly alone on that one.
643 2012-09-04 21:04:42 <doublec> why is repeated address use bad?
644 2012-09-04 21:04:59 <doublec> I'm surprised they don't do a deal with a friendly miner and just relay their transactions through them
645 2012-09-04 21:05:21 <gmaxwell> doublec: hm, it's not _bad_
646 2012-09-04 21:05:34 <gmaxwell> You don't understand me at all if you're thinking in terms of good and bad.
647 2012-09-04 21:05:40 <gmaxwell> :(
648 2012-09-04 21:06:03 <doublec> well by "depriortize" I'm assuming those are "less good"
649 2012-09-04 21:06:13 <doublec> or less important
650 2012-09-04 21:06:20 <sipa> no, rather "less interested in priority"
651 2012-09-04 21:06:21 <doublec> I'm wondering why repeated address use is an indicator of that
652 2012-09-04 21:06:46 <sipa> also, i do think repeated address use should be discouraged, but that's quite independent from this issue
653 2012-09-04 21:06:57 <doublec> or is it just that the services your interested in de-prioritzing do that
654 2012-09-04 21:06:57 <gmaxwell> Repeated address reuse generally indicates that either there is a standing relationship (fast confirmations aren't needed), OR that the communications between parties are one-way (again, fast confirmations are not needed)
655 2012-09-04 21:07:21 <doublec> ok, thanks
656 2012-09-04 21:08:56 <shhh> gmaxwell, will still be just a workaround, they could just generate a new address for every pageview of the website ?
657 2012-09-04 21:09:07 <fiesh> isnt's satoshi dice susceptible to a Finney attack? It immedaitely tells if an, even unconfirmed, transaction has won or lost, no?
658 2012-09-04 21:09:16 <gmaxwell> Another way of looking at it is that, absent meaningful fees??? we'd like to give all the currently active users equal access to the available capacity. You normally can't tell if two txn are the same person or not, but when there is address reuse people are voluntarily telling you that they're the same partys.
659 2012-09-04 21:09:42 <sipa> makes sense
660 2012-09-04 21:09:44 <gmaxwell> shhh: sure but I don't think they would? why would they? They'd still get confirmed, it would just tend to take a bit longer, filling in slack space in the capacity.
661 2012-09-04 21:10:46 <sipa> MC-Eeepc: by the way, what was your hardware again, and how fast is it going?
662 2012-09-04 21:10:52 <shhh> gmaxwell, if they still get confirmed it makes sense, if they wont get confirmed at all ...
663 2012-09-04 21:11:05 <doublec> gmaxwell: that's a good explanation, thanks
664 2012-09-04 21:11:30 <fiesh> hmm no that doesn't help actually, don't think it is
665 2012-09-04 21:11:34 <gmaxwell> I don't propose it as a way to "fight dice"??? I don't care to fight it, I don't dislike it fundimentally or anything like that??? I just want to mitigate the harm on all the other users. And I expect that the operator would support that too.
666 2012-09-04 21:11:54 <sipa> fiesh: since they use the coins sent to them in payouts, a finney attack would also invalidate the payouts you got
667 2012-09-04 21:12:07 <fiesh> sipa: yes, just realized that myself, thanks
668 2012-09-04 21:12:07 <sipa> (and probably payouts of others too...)
669 2012-09-04 21:12:09 <shhh> gmaxwell, actually dice might improve things midterm ...
670 2012-09-04 21:12:32 <gmaxwell> shhh: it does not.
671 2012-09-04 21:12:52 <MC-Eeepc> atom 1.6 2gb rams
672 2012-09-04 21:13:04 <shhh> then something is fundamently flawed if it cant cope with it
673 2012-09-04 21:13:14 <MC-Eeepc> its going about 25k blocks since you posted the link
674 2012-09-04 21:13:16 <gmaxwell> shhh: nonsense.
675 2012-09-04 21:13:36 <sipa> MC-Eeepc: and storage is on a USB stick?
676 2012-09-04 21:13:48 <MC-Eeepc> yes, slow as shit usb stick
677 2012-09-04 21:14:32 <sipa> MC-Eeepc: to minimize impact of network delays, you can try to -addnode or -connect to a known fast node
678 2012-09-04 21:14:36 <gmaxwell> shhh: is democracy fundimentally flawed if it can't surivive every voter being assassinated by ninjas? Why not? No amount of smart ballot design can stop ninja stars. A system is more than its most simple technical rules.
679 2012-09-04 21:15:12 <amiller> what's the largest size payout that satoshi dice has ever made?
680 2012-09-04 21:15:32 <gmaxwell> amiller: who knows?
681 2012-09-04 21:15:33 <sipa> shhh: i assume you're referring to "the load SD generates forces the software to be improved" ?
682 2012-09-04 21:16:17 <MC-Eeepc> i dont think network delays are slowing me down
683 2012-09-04 21:16:37 <sipa> MC-Eeepc: maybe not
684 2012-09-04 21:16:42 <gmaxwell> MC-Eeepc: hows it going now? Faster than the prior attempt?
685 2012-09-04 21:17:00 <midnightmagic> +1 ninjas and pirates
686 2012-09-04 21:17:05 <midnightmagic> ACTION is sorry for that one
687 2012-09-04 21:17:10 <MC-Eeepc> seems to be i think
688 2012-09-04 21:17:19 <MC-Eeepc> cant really remember what it was doing last time
689 2012-09-04 21:17:26 <gmaxwell> amiller: I should clarify my who-knows. That stats thread probably has the largest transaction... but it's not clear what it means: as shhh pointed out before, they could just have been playing themselves.
690 2012-09-04 21:17:28 <MC-Eeepc> seems to be about 20 blox second atm
691 2012-09-04 21:18:01 <sipa> I just synced 193k blocks from the internet (from a fixed fast node, though) in 23 minutes
692 2012-09-04 21:18:16 <amiller> they're probably putting most of their effort into playing against themselves at the highest value
693 2012-09-04 21:18:23 <sipa> so even in that setting, the network delay *outweighs* importing time
694 2012-09-04 21:18:36 <sipa> (as doing the same from a local disk took 10 minutes)
695 2012-09-04 21:19:02 <gmaxwell> amiller: there was a very odd statistical difference in the payoff rates of large vs small bets previously.. that appeared to go away after people pointed it out and said wtf.
696 2012-09-04 21:19:47 <gmaxwell> amiller: dooglus said he was wondering at one point if they weren't intentionally losing against themselves to make the service look more attractive.
697 2012-09-04 21:20:06 <amiller> then why don't they lose to themselves at the highest level
698 2012-09-04 21:21:09 <MC-Eeepc> you shouldnt beckmark it to 193k
699 2012-09-04 21:21:24 <MC-Eeepc> that relies on the checkpoint doesnt it
700 2012-09-04 21:21:35 <MC-Eeepc> checkpoints are merely a necessary evil
701 2012-09-04 21:21:50 <sipa> yes, but afterwards the speed is dominated by sig checking
702 2012-09-04 21:22:03 <gmaxwell> which is interesting and all, but not the stuff being changed here.
703 2012-09-04 21:22:04 <sipa> so there isn't much the benchmark anymore
704 2012-09-04 21:22:26 <MC-Eeepc> so cpu bound instead of i/o bound?
705 2012-09-04 21:22:38 <gmaxwell> (and plus sigchecking is artifically slow at the moment because it's single threaded and synchronous with other activity)
706 2012-09-04 21:23:01 <MC-Eeepc> my cpu usage was about zero for the last 50k blox last time
707 2012-09-04 21:23:02 <sipa> indeed, parallel sigchecking may make things a lot more interesting
708 2012-09-04 21:23:22 <sipa> MC-Eeepc: in your case it's most certainly IO bound
709 2012-09-04 21:23:39 <MC-Eeepc> for sure
710 2012-09-04 21:24:02 <sipa> gmaxwell: i wonder whether we'll even need the no-sig-check-before-checkpoint rule when all CPUs are fully quadcore and we add Hal's improved ECDSA code
711 2012-09-04 21:24:23 <sipa> the sig check time would be very close to raw import time without sigchecking
712 2012-09-04 21:24:49 <MC-Eeepc> theres no way to use the crypto logic on newer chips for ecdsa right
713 2012-09-04 21:24:53 <gmaxwell> I don't think we will.. dunno. My impression was that we mostly got it because we were sort of grasping at why it was slow and it was all gavin found at first that helped at all.
714 2012-09-04 21:25:03 <gavinandresen> RE: sigchecking: I bet it'd be easy to write a -spotchecksigs=n argument, to only check (probabilistically) 1-out-of-n accepted-in-the-blockchain signatures. I might even use that myself....
715 2012-09-04 21:26:06 <gmaxwell> gavinandresen: yea, thats really what I think we should do. And really we could default it to something like 1:1000 and not make any visible impact but have the certificational strength that all particiants are really checking with high probablity.
716 2012-09-04 21:26:08 <gavinandresen> actually, -spotchecktransactions is what we want, just assume the whole transaction is valid
717 2012-09-04 21:26:58 <gavinandresen> 1:10 would give 10x speedup, that would be a good place to start. Might want some special cases, like always check the coinbase
718 2012-09-04 21:26:59 <sipa> well, if you accept transactions, you probably want sigchecks a few levels deep in the ancestry of incoming transactions
719 2012-09-04 21:27:27 <sipa> and if you mine, you probably want to check everything
720 2012-09-04 21:27:34 <gmaxwell> ah, you mean for recent blocks too. meh. I'd rather go in the direction of provisional acceptance in that case.
721 2012-09-04 21:27:51 <gmaxwell> E.g. you accept right away, but check in the background, then retrospectively reject the block if it fails.
722 2012-09-04 21:28:04 <gavinandresen> I meant in the case of catching up from the last checkpoint. Transactions on the wire aught to be checked before relaying
723 2012-09-04 21:28:04 <gmaxwell> So long as you can catch up eventually then they all get checked.
724 2012-09-04 21:28:24 <gavinandresen> Smarter relaying code aught to be written, though...
725 2012-09-04 21:28:50 <sipa> ACTION has long since wondered whether 'aught' is english, and whether it means the same thing as 'ought'
726 2012-09-04 21:28:52 <gavinandresen> e.g. use transaction priority and fees to throttle transaction relaying
727 2012-09-04 21:29:35 <gavinandresen> hey, wow, it is ought....
728 2012-09-04 21:29:43 <gavinandresen> aught is something completely different!
729 2012-09-04 21:30:03 <gavinandresen> I ought to have known that
730 2012-09-04 21:30:05 <ErnestoJuarell> How would I got about getting this kind of data? http://blockchain.info/address/1BD1BFcK3tuA48u7Bu2CAdCm8acfAubbQY?format=json
731 2012-09-04 21:30:07 <MC-Eeepc> aught means nothing
732 2012-09-04 21:30:14 <MC-Eeepc> zero
733 2012-09-04 21:30:17 <ErnestoJuarell> directly from the chain
734 2012-09-04 21:30:31 <midnightmagic> MC-Eeepc: Or "anything".
735 2012-09-04 21:30:47 <MC-Eeepc> sipa whats your expected benefit from adding leveldb
736 2012-09-04 21:30:51 <gmaxwell> ErnestoJuarell: getrawtransaction <txid> 1
737 2012-09-04 21:30:57 <sipa> MC-Eeepc: in your case: a lot
738 2012-09-04 21:31:15 <gavinandresen> ErnestoJuarell: ... but you'll need an index of <address> to <txid>
739 2012-09-04 21:31:30 <gavinandresen> ErnestoJuarell: or you'll need to iterate over every transaction in every block.
740 2012-09-04 21:31:41 <ErnestoJuarell> so those sites analyzed the blockchain and built a db?
741 2012-09-04 21:31:45 <gavinandresen> yup
742 2012-09-04 21:32:03 <sipa> gavinandresen: after ultraprune/leveldb are out of the way, i want to split off a coins.* and blktree.* (and potentially) mempool.* from main, and then do reverse-headers sync (sync blocktree instead of blocks, and then have a process fetch blocks along the best known chain through the tree)
743 2012-09-04 21:32:14 <gavinandresen> sipa: excellent!
744 2012-09-04 21:32:55 <midnightmagic> ErnestoJuarell: The "relayed by" is something which requires universal connectivity, or a nearly-perfect map of the network.
745 2012-09-04 21:33:20 <sipa> gavinandresen: i think that will simplify a lot (like preview relaying, and parallel sigchecking)
746 2012-09-04 21:33:24 <midnightmagic> .. which of course is going to piss some people off who will then work hard to frustrate your efforts. :)
747 2012-09-04 21:33:32 <sipa> as processing of blocks can happen in stages in separate threads
748 2012-09-04 21:33:35 <gavinandresen> I have been eavesdropping on gmaxwell's concerns about blockchain stability, though. I think ideally we'd have a "for miners" fork and a "for users" fork. And a library in between.