1 2011-03-31 00:00:48 <Kiba`> what is this multiplier?
  2 2011-03-31 00:00:55 <devrandom> "democratic" == "hard to get a large multiplier from capital expenditure"
  3 2011-03-31 00:01:13 <Kiba`> don't forget Peter Thiel!
  4 2011-03-31 00:01:23 <devrandom> Kiba` - how much more efficient is your hardware if you spend money on full-custom silicon
  5 2011-03-31 00:01:37 <lfm> I forget, who is Peter Thiel?
  6 2011-03-31 00:01:48 <devrandom> (Mh/s per $)
  7 2011-03-31 00:02:04 <[Tycho]> gasteve, what problem ? I didn't said anything about problems.
  8 2011-03-31 00:02:18 <Kiba`> the epitome of a libertarian billionaire techno-geek
  9 2011-03-31 00:02:45 <gasteve> by that I mean that I doubt there is a way to implement an algo that will make it impossible to subvert bitcoin when there is a relatively small economy...so the solution is to grow the economy (and the clients and mining power) and make it such that it's virtually impossible to subvert even if you had say the annual GDP of all nations available to you
 10 2011-03-31 00:02:45 <lfm> there is no problem, bitcoin is working fine
 11 2011-03-31 00:02:48 <Kiba`> if he isn't rich, he would be the kind of guy who would fit in our community
 12 2011-03-31 00:03:10 <lfm> Kiba`: maybe he is here already then
 13 2011-03-31 00:03:23 <Kiba`> that mean we must grow an economy bigger than several nations' GDP?
 14 2011-03-31 00:04:03 <Kiba`> kinda hard
 15 2011-03-31 00:04:16 <kiba> the US is one of the biggest economy in the world next to EU
 16 2011-03-31 00:04:37 <lfm> kiba oh he was behind paypal. If he still has a lot of stock he might not like bitcoin
 17 2011-03-31 00:04:41 <gasteve> well, the bar would actually be far less ...but you get the idea
 18 2011-03-31 00:05:21 <gasteve> make it so you would require a compute power that is impractical for any single entity to control
 19 2011-03-31 00:05:24 <kiba> why would he hate bitcoin?
 20 2011-03-31 00:05:33 <kiba> paypal failed to acheive his libertarian dream
 21 2011-03-31 00:05:46 <devrandom> lfm - he left before Paypal became evil, AFAIK.  Paypal was supposed to be something alot more similar to bitcoin/digicash.  I'm sure he'll be into this.
 22 2011-03-31 00:05:49 <lfm> kiba if he has a lot of stock in paypal, bitcoin is kinda a competitor
 23 2011-03-31 00:06:19 <devrandom> gasteve - I agree that growing the bitcoin economy is critical
 24 2011-03-31 00:06:28 <kiba> paypal is owned by ebay
 25 2011-03-31 00:07:18 <kiba> the great advantage of being programmers is that we're force multiplers
 26 2011-03-31 00:07:48 <devrandom> BTW, Thiel is funding this: http://seasteading.org/
 27 2011-03-31 00:08:25 <kiba> I am sure we can defeat a bunch of breaucractic slow acting 3 letter government agencies
 28 2011-03-31 00:08:57 <kiba> our OODA loop for growing the economy just have to be faster than their
 29 2011-03-31 00:10:53 <gasteve> I guess the middle of an ocean is about the only place left where you can try to bootstrap a new socio-economic system...until we colonize mars I guess
 30 2011-03-31 00:12:37 <[Tycho]> Rignt above the R'lyeh, sure
 31 2011-03-31 00:22:10 <forrestv> <lfm> forrestv: nonsense.  Unless you mean the transaction pruning mentioned in Satoshi's white paper, then no - I dont think anyone is doing it yet
 32 2011-03-31 00:22:12 <forrestv> in fact, entire series of blocks could be eliminated
 33 2011-03-31 00:22:40 <jgarzik> you cannot eliminate block headers
 34 2011-03-31 00:22:46 <jgarzik> thus entire blocks
 35 2011-03-31 00:22:57 <forrestv> why not?
 36 2011-03-31 00:23:34 <forrestv> a message like 'block with hash X is redundant, continue at block with hash Y' could be added to the chain
 37 2011-03-31 00:24:23 <lfm> well you dont need old block headers till someone trys to give you a txn based on a very old input txn, to verify it you could be downloading a lot
 38 2011-03-31 00:25:44 <ArtForz> depends
 39 2011-03-31 00:26:26 <lfm> depends how much you trust your block server(s)
 40 2011-03-31 00:27:04 <forrestv> if the skip instructions were confirmed by miners and embedded into the chain, there wouldn't need to be any more trust than usual ..
 41 2011-03-31 00:27:18 <ArtForz> you dont have to trust them
 42 2011-03-31 00:27:34 <ArtForz> all really old tx are covered by the hardcoded checkpoint
 43 2011-03-31 00:27:43 <bd_> forrestv: Knowing the blocks up to point X are valid isn't enough. You also need to know what transactions were used in that period
 44 2011-03-31 00:27:55 <bd_> Analyzing the blocks to get this list of transactions takes time.
 45 2011-03-31 00:28:25 <bd_> I suppose you could speed it up somewhat if you assumed that every signature before point X was valid, though.
 46 2011-03-31 00:28:37 <lfm> if the block servers let you ask "which block has this txn hash in it" then you just get the merkle tree for that block and the txn you want
 47 2011-03-31 00:28:51 <jgarzik> still need to know unspent tx, even < hardcoded checkpoint
 48 2011-03-31 00:28:56 <ArtForz> why?
 49 2011-03-31 00:28:57 <bd_> lfm: What happens if the block server lies to you, then? :)
 50 2011-03-31 00:29:06 <ArtForz> how could it?
 51 2011-03-31 00:29:34 <lfm> dbitcoin: then it wouldnt verify (assuming you had the block header chain already
 52 2011-03-31 00:29:36 <ArtForz> you have the whole chain of headers from genesis, so you know the blocks are ok
 53 2011-03-31 00:29:45 <bd_> lfm: Your client: "I have a txn trying to spend CTxOut 4 of transaction 0xDEADBEEF. What do you know about it?" Server: "0xDEADBEEF, valid txn in block 4883. Never used."
 54 2011-03-31 00:29:46 <ArtForz> otherwise no miners would have built on them
 55 2011-03-31 00:29:52 <bd_> In reality, it was used in block 4884. Oops!
 56 2011-03-31 00:30:07 <ArtForz> which gains you nothing
 57 2011-03-31 00:30:22 <bd_> ArtForz: It does if you have a POS terminal or something that wants to verify a transaction quickly.
 58 2011-03-31 00:30:32 <lfm> bd_: the merkle tree connects the block to the txns
 59 2011-03-31 00:30:52 <bd_> lfm: Yes, the server can't lie about the transaction being part of block 4883.
 60 2011-03-31 00:31:02 <bd_> My point is it can lie and say "This coin has never been spent"
 61 2011-03-31 00:31:21 <ArtForz> hrrrmmm... actually thats a neat idea
 62 2011-03-31 00:31:35 <lfm> bd if the txn hash is not in the merkle tree you write off that sever as untrustworthy and go to another
 63 2011-03-31 00:31:47 <bd_> lfm: You're not listening to me. the txn _is_ in the merkle tree.
 64 2011-03-31 00:32:03 <bd_> The server's just lying and saying it was never spent, so the server operator can get away with double spending.
 65 2011-03-31 00:32:06 <ArtForz> have each block contain a chain hash of all unspent tx before that block
 66 2011-03-31 00:32:19 <bd_> ArtForz: That can get really big really fast.
 67 2011-03-31 00:32:30 <lfm> bd_: if it was never spent it should be able to show you the txn with the unspent output
 68 2011-03-31 00:32:37 <ArtForz> how?
 69 2011-03-31 00:32:50 <bd_> lfm: Of course. I can _always_ show you the txn with the unspent output, even if it's spent.
 70 2011-03-31 00:32:55 <bd_> The original txn never changes.
 71 2011-03-31 00:33:11 <bd_> The point is, unless you have personally examined EVERY block AFTER that point, you cannot know that the txn has not been spent
 72 2011-03-31 00:33:19 <lfm> bd_: oh i see
 73 2011-03-31 00:33:57 <bd_> Now, if you assume that all block generators will do the full verify, then you'll figure it out eventually when everyone refuses to include the txn in their block chain
 74 2011-03-31 00:34:04 <ArtForz> I dont think theres any way to verify if a tx is unspent without a) keeping track of em or b) trusting someone else to do that
 75 2011-03-31 00:34:30 <bd_> But if you want that snap decision then you really need to keep a database of unspent txns.
 76 2011-03-31 00:34:37 <ArtForz> yep
 77 2011-03-31 00:34:55 <lfm> and you have to at least look at all the spent ones
 78 2011-03-31 00:35:30 <bd_> Now here's the thing: If you include this data in the block chain, while you could increase confidence in the data without spending too much computation time on verification, it makes the job of the generators MUCH more difficult
 79 2011-03-31 00:35:45 <bd_> They have to dump out and hash their entire unspent txn database every block, after all.
 80 2011-03-31 00:36:11 <ArtForz> not quite
 81 2011-03-31 00:36:59 <lfm> if you start with a "trusted" block chain that has already been pruned you should be ok
 82 2011-03-31 00:37:46 <bd_> ArtForz: If you're going to say, "But we can make a merkle tree out of it!", remember that you still need to keep around all those merkle blocks in case a client asks you for them later
 83 2011-03-31 00:38:27 <lfm> bd_: you can just pass on the request to someone else till it finds someone with everything
 84 2011-03-31 00:38:29 <bd_> You also have to hand them to other generators (or have those generators compute the prior state of the unspent-txn-pool) so they can verify the chain
 85 2011-03-31 00:38:31 <ArtForz> which ones?
 86 2011-03-31 00:38:43 <bd_> And you need to keep this data around for a while because the most recent data is the least trustworthy
 87 2011-03-31 00:39:11 <lfm> bd_: yes the pruning algo can never prune "newish" stuff
 88 2011-03-31 00:39:14 <bd_> lfm: also, block chains are never pruned. The system relies on the entire block chain remaining verifiable forever.
 89 2011-03-31 00:39:50 <lfm> bd_: thats just 80 bytes per block tho so it is managable
 90 2011-03-31 00:39:53 <bd_> Not every client needs to do the entire verification, but they need to trust that _someone_ has run that verification
 91 2011-03-31 00:40:03 <phantomcircuit> ;;bc,stats
 92 2011-03-31 00:40:05 <gribble> Current Blocks: 115862 | Current Difficulty: 68978.89245792 | Next Difficulty At Block: 116927 | Next Difficulty In: 1065 blocks | Next Difficulty In About: 6 days, 17 hours, 13 minutes, and 45 seconds | Next Difficulty Estimate: 75956.80814940
 93 2011-03-31 00:40:13 <dirtyfilthy> not every client needs to store the whole blockchain either
 94 2011-03-31 00:40:17 <bd_> lfm: No, it's 80 bytes, PLUS the merkle tree's contents which need to be available on request
 95 2011-03-31 00:40:19 <jgarzik> well its 80 bytes + merkle, if we are pruning TX's, right?
 96 2011-03-31 00:40:21 <jgarzik> yeah
 97 2011-03-31 00:40:54 <phantomcircuit> -rw-r--r-- 1 phantomcircuit phantomcircuit 269M Mar 30 19:39 bitcoin.sqlite3
 98 2011-03-31 00:40:56 <bd_> so you need to either store that merkle or compute it
 99 2011-03-31 00:40:57 <phantomcircuit> holy crap
100 2011-03-31 00:41:05 <bd_> Other generators should be able to generate them easily enough, of course
101 2011-03-31 00:41:11 <bd_> So you only need to actually send it to clients
102 2011-03-31 00:41:29 <lfm> bd_: requests for pruned merkle tree contents can be passed on too, you usually will not have requests for spent txns even of the merkle trees
103 2011-03-31 00:41:49 <bd_> what do you mean by passed on?
104 2011-03-31 00:41:59 <jgarzik> phantomcircuit: why not do like bitcoin, and store blocks in their own file?
105 2011-03-31 00:42:00 <bd_> Just say, "Oh, go ask this other generator, they'll probably know."
106 2011-03-31 00:42:01 <bd_> ?
107 2011-03-31 00:42:14 <jgarzik> phantomcircuit: just index w/ db software
108 2011-03-31 00:42:15 <lfm> bd forward the request to a node you know which kept historical data
109 2011-03-31 00:42:46 <bd_> I suppose you can limit the long-term storage to archival nodes
110 2011-03-31 00:42:50 <phantomcircuit> jgarzik, i doubt that would do much good
111 2011-03-31 00:42:56 <bd_> Or, well, "verification servers"
112 2011-03-31 00:43:09 <jgarzik> phantomcircuit: are you storing the entire block in a single SQL column?
113 2011-03-31 00:43:13 <bd_> Since the generators _can_ keep around the most-recent data without too much difficulty... hm
114 2011-03-31 00:43:31 <phantomcircuit> also running bitcoin with -daemon crashes on my system with a master/HEAD build
115 2011-03-31 00:43:39 <lfm> if the new nodes start with a chain pre-pruned to a checkpoint, then requests for spent txn befor thos should be minimal
116 2011-03-31 00:44:04 <bd_> lfm: That said, there is a bootstrapping problem
117 2011-03-31 00:44:21 <bd_> If you see two _partial_ block chains floating around, how do you know which to trust?
118 2011-03-31 00:44:35 <lfm> ya if you want a dead start from the geneis block, make sure you connect to an archival node
119 2011-03-31 00:44:48 <bd_> lfm: And the dead start is exactly what we're trying to avoid :)
120 2011-03-31 00:45:03 <lfm> it should be rare after a while
121 2011-03-31 00:45:09 <bd_> I suppose you could work out the total difficulty for the block chain that's visible
122 2011-03-31 00:45:54 <lfm> ya its still easy to calulate the chain with the most POW
123 2011-03-31 00:46:28 <lfm> same as any other fork
124 2011-03-31 00:46:41 <Diablo-D3> http://fukung.net/v/32943/db5e3f15884878d2c385faeca3e8b534.jpg
125 2011-03-31 00:47:06 <Validus> lol
126 2011-03-31 00:47:47 <bd_> lfm: that said, it does add O(lg n) to the cost of processing a transaction for generators
127 2011-03-31 00:47:48 <grbgout> hah
128 2011-03-31 00:47:56 <grbgout> who is that woman?
129 2011-03-31 00:48:04 <bd_> where n = number of unspent txns
130 2011-03-31 00:48:59 <lfm> bd_: if the txn are filed by hash you can just find the inputs directly
131 2011-03-31 00:50:17 <bd_> Imean, in maintaining the merkle tree
132 2011-03-31 00:51:59 <lfm> oh i see ya, simple enuf to limit the "reblocking" to only txn which have been verified and verify at lower priority. only do re-blocking periodiclly (a few seconds apart or whatever)
133 2011-03-31 00:53:50 <phantomcircuit> rofl
134 2011-03-31 00:53:55 <phantomcircuit> db contention is killing me
135 2011-03-31 00:54:00 <grbgout> bd_, lfm: how far back does your conversation go?  I'd like to catch up, but I'm feeling lazy at the moment >_>
136 2011-03-31 00:54:00 <phantomcircuit> 20 seconds to insert a single block
137 2011-03-31 00:54:24 <jgarzik> phantomcircuit: contention with what?  are you doing a parallel add?
138 2011-03-31 00:55:09 <phantomcircuit> jgarzik, the logic for connecting/requesting blocks is all parallel
139 2011-03-31 00:55:27 <phantomcircuit> so it requests the same thing from everybody
140 2011-03-31 00:55:28 <phantomcircuit> lol
141 2011-03-31 00:56:13 <phantomcircuit> i just need a simple synchronization between all the connected peers
142 2011-03-31 00:56:19 <phantomcircuit> or possibly to move the logic out of there
143 2011-03-31 00:56:23 <phantomcircuit> which ever
144 2011-03-31 00:58:48 <lfm> grbgout: the conversations all run into one another so go back as far as you want
145 2011-03-31 01:03:37 <grbgout> lfm: well yeah, I was wondering more about how long ago your and bd_'s conversation began.
146 2011-03-31 01:03:49 <grbgout> lfm: ultimately it doesn't matter, I'll page up eventually
147 2011-03-31 01:04:27 <bd_> about a half hour or so ago maybe?
148 2011-03-31 01:05:09 <lfm> grbgout: just about as hard for me to figure out how far it goes as you and you have more motivation
149 2011-03-31 01:05:12 <grbgout> ah, thanks --- yeah, to lazy to go that far back atm ^_^
150 2011-03-31 01:05:21 <grbgout> lfm: indeed
151 2011-03-31 01:05:29 <grbgout> *too
152 2011-03-31 01:11:01 <manveru> hm
153 2011-03-31 01:11:23 <manveru> i wonder how hard it would be to implement a simple client for bitcoin... only to see my transactions
154 2011-03-31 01:11:55 <manveru> do i really need to implement the whole protocol for that?
155 2011-03-31 01:12:09 <grbgout> manveru: dunno, have you looked at google's java client?
156 2011-03-31 01:12:42 <manveru> no... it's java
157 2011-03-31 01:12:50 <grbgout> hehe
158 2011-03-31 01:13:06 <manveru> sorry :)
159 2011-03-31 01:13:17 <phantomcircuit> manveru, depends on what you cant to know about your transactions, you like python?
160 2011-03-31 01:13:23 <grbgout> I think that's a good thing: should make an android app simple, and just on the horizon.
161 2011-03-31 01:13:32 <lfm> manveru: depends, do you want to trust a server?
162 2011-03-31 01:13:56 <manveru> heh
163 2011-03-31 01:14:02 <manveru> phantomcircuit: i'm ok with python
164 2011-03-31 01:14:20 <manveru> lfm: and yeah, something easily curl-able would be cool
165 2011-03-31 01:14:32 <grbgout> lfm: he could set it up to query his local bitcoind, that is: setup the 'server' to be his home, and establish decent security measures to prevent the odd-unauthorized connection.
166 2011-03-31 01:14:56 <manveru> oh, right
167 2011-03-31 01:15:07 <manveru> there's a REST api for the bitcoind, right?
168 2011-03-31 01:15:12 <lfm> manveru: no, a bitcoin server, for bitcoin queries. If you can trust a bitcoin server you can make a very minimal client that just requests certain data
169 2011-03-31 01:15:31 <grbgout> manveru: probably, what do you mean by REST?
170 2011-03-31 01:15:44 <manveru> grbgout: well, HTTP
171 2011-03-31 01:16:02 <grbgout> manveru: ah, yeah.  The RCP JSON API.
172 2011-03-31 01:16:20 <manveru> gotta read that again
173 2011-03-31 01:16:37 <manveru> i'm mostly worried about having to implement all those encryption methods
174 2011-03-31 01:17:01 <manveru> since i'm more likely to get it wrong than right, i'll try to stand on the shoulders of giants for that
175 2011-03-31 01:17:09 <lfm> manveru: if you dont have a server you can trust then you need to implement pretty much the whole protocol
176 2011-03-31 01:17:25 <manveru> lfm: well, i have my local machine
177 2011-03-31 01:17:34 <manveru> i also have a couple of servers i do trust
178 2011-03-31 01:17:35 <grbgout> manveru: you can probably just implement the JSON API, and setup secure RPC on your local machine's bitcoind.
179 2011-03-31 01:17:35 <phantomcircuit> manveru, https://github.com/phantomcircuit/bitcoin-alt/tree/sqlite3
180 2011-03-31 01:17:57 <grbgout> i thought I had the API bookmarked, but it seems not
181 2011-03-31 01:18:04 <phantomcircuit> manveru, downloads blocks/transactions, doesn't parse scripts
182 2011-03-31 01:18:32 <lfm> manveru: there is also the option of something like mybitcoin.com. If you can trust it (or something like it) then you can use your browser to access your account
183 2011-03-31 01:18:36 <grbgout> phantomcircuit: well, why waste the bandwidth/effort: the local bitcoind should be doing that anyway, right?
184 2011-03-31 01:18:49 <phantomcircuit> grbgout, facepalm
185 2011-03-31 01:18:56 <phantomcircuit> because im implementing my own bitcoind?
186 2011-03-31 01:18:57 <manveru> lfm: i'd rather not do that :)
187 2011-03-31 01:19:05 <grbgout> phantomcircuit: I mean just for his purposes, not as a reflection on your project.
188 2011-03-31 01:19:26 <phantomcircuit> grbgout, i guess
189 2011-03-31 01:20:12 <grbgout> manveru: it seems to me that ... $ bitcoind help is probably more than what you'd need to have access to.
190 2011-03-31 01:20:41 <manveru> grbgout: yes
191 2011-03-31 01:20:42 <grbgout> hmm, alright, now I'm annoyed.  I know I bookmarked the old wiki page on the API, which discussed doing SSL stuff, and now I can't find it in my lists. >_<
192 2011-03-31 01:20:54 <manveru> https://en.bitcoin.it/wiki/API_reference_(JSON-RPC)
193 2011-03-31 01:21:04 <manveru> and i'm in a VPN already
194 2011-03-31 01:21:10 <grbgout> manveru: so then it's just a matter of access a bitcoind server you like, and implementing the least amount of the JSON API you'll need.
195 2011-03-31 01:21:21 <manveru> exactly
196 2011-03-31 01:21:23 <lfm> or just clone mybitcoin.com and run your own server, no need to develop yet another client since your browser is it
197 2011-03-31 01:21:28 <manveru> and replacing the ruby code on the wiki...
198 2011-03-31 01:21:31 <phantomcircuit> ;;bc,stats
199 2011-03-31 01:21:33 <gribble> Current Blocks: 115869 | Current Difficulty: 68978.89245792 | Next Difficulty At Block: 116927 | Next Difficulty In: 1058 blocks | Next Difficulty In About: 6 days, 15 hours, 52 minutes, and 32 seconds | Next Difficulty Estimate: 76118.34963845
200 2011-03-31 01:21:49 <grbgout> there's already a JavaScript thing just for what you're after, too, manveru (I think: didn't look at the project too closley).
201 2011-03-31 01:21:54 <phantomcircuit> neat
202 2011-03-31 01:22:00 <grbgout> https://github.com/tcatm/bitcoin-js-remote
203 2011-03-31 01:22:06 <phantomcircuit> my client has one more block than gribble
204 2011-03-31 01:22:31 <gribble> 115869
205 2011-03-31 01:22:31 <lfm> ;;bc,blocks
206 2011-03-31 01:22:32 <grbgout> manveru: how did your tests with the testnet work out?  Anything exciting?
207 2011-03-31 01:22:56 <lfm> phantomcircuit: are you running a very old version?
208 2011-03-31 01:23:09 <manveru> grbgout: i didn't get that far... we started another project
209 2011-03-31 01:23:09 <phantomcircuit> lfm, hmm
210 2011-03-31 01:23:19 <manveru> grbgout: http://ebeats.org/
211 2011-03-31 01:23:21 <phantomcircuit> lfm, oh i hard coded a version number
212 2011-03-31 01:23:22 <grbgout> manveru: ah.  Who are "we", if you don't mind my asking?
213 2011-03-31 01:23:24 <lfm> phantomcircuit: the block numbering changed a while back
214 2011-03-31 01:23:35 <phantomcircuit> lfm, no i meant *my* python client
215 2011-03-31 01:23:47 <phantomcircuit> oh it's reporting the 0 based top block?
216 2011-03-31 01:23:50 <phantomcircuit> that's dumb
217 2011-03-31 01:24:16 <lfm> phantomcircuit: ya some clients count the genisis as a regular block and some dont
218 2011-03-31 01:24:34 <phantomcircuit> weird
219 2011-03-31 01:24:43 <phantomcircuit> well either way
220 2011-03-31 01:24:51 <lfm> and bitcoin(d) used to but quit a while back
221 2011-03-31 01:24:51 <phantomcircuit> i have all the blocks/transactions in my loverly sqlite db
222 2011-03-31 01:25:05 <lfm> phantomcircuit: wtg
223 2011-03-31 01:25:37 <phantomcircuit> now i have to figure out why adding blocks/transactions randomly take longer
224 2011-03-31 01:26:15 <lfm> ya, typical database, once you got it loaded, its time to redisign it
225 2011-03-31 01:26:45 <grbgout> hehe
226 2011-03-31 01:27:58 <phantomcircuit> lfm, im guessing i just need an insert/commit lock
227 2011-03-31 01:28:33 <grbgout> manveru: The "solar day" that ebeats measure is taken at UTC/GMT/Zulu, correct?
228 2011-03-31 01:28:53 <bd_> sqlite is bad if you have multiple threads operating on the database - it's a single monolithic lock with starvation issues
229 2011-03-31 01:29:10 <phantomcircuit> bd_, yeah im aware
230 2011-03-31 01:29:15 <phantomcircuit> acutely >.>
231 2011-03-31 01:29:23 <bd_> You can mitigate it a bit with batching and your own starvation-avoiding locking schemes
232 2011-03-31 01:29:31 <manveru> grbgout: yes
233 2011-03-31 01:29:46 <bd_> eg, when you have incoming txns, don't process them immediately. Every 30 seconds or so, take all the new txns and blocks and process them all at once, in a single sqlite-level transaction
234 2011-03-31 01:29:49 <phantomcircuit> batching has gotten me most of the way
235 2011-03-31 01:29:54 <manveru> it's a simple algorithm from UTC to ebeats
236 2011-03-31 01:30:01 <lfm> grbgout: normally a solar day is a local time from noon to noon or sinrise to sunrise
237 2011-03-31 01:30:15 <bd_> phantomcircuit: Are you having contention between readers and writers or something?
238 2011-03-31 01:30:18 <nanotube> manveru: have you seen js-remote?
239 2011-03-31 01:30:26 <manveru> nanotube: no
240 2011-03-31 01:30:37 <grbgout> manveru: I just provided the link to it!
241 2011-03-31 01:30:42 <manveru> sorry, gotta hack some smalltalk... brb
242 2011-03-31 01:30:44 <nanotube> ;;sl github tcatm js remote
243 2011-03-31 01:30:45 <bd_> phantomcircuit: because, you can batch reads as well :)
244 2011-03-31 01:30:45 <gribble> http://tcatm.github.com/bitcoin-js-remote/ | bitcoin-js-remote is a user interface for Bitcoin written in JavaScript and released under the ... git clone git://github.com/tcatm/bitcoin-js-remote ...
245 2011-03-31 01:30:49 <nanotube> manveru: ^^
246 2011-03-31 01:30:54 <grbgout> 23:19:33 < grbgout> https://github.com/tcatm/bitcoin-js-remote
247 2011-03-31 01:30:57 <grbgout> >_<
248 2011-03-31 01:31:00 <nanotube> hehe
249 2011-03-31 01:31:02 <nanotube> grbgout: :)
250 2011-03-31 01:31:21 <nanotube> manveru: basically... a fully-functional client, based on accessing a remote bitcoind node via jsonrpc
251 2011-03-31 01:31:22 <phantomcircuit> bd_, yeah but the only reasonable way to do it is with joins
252 2011-03-31 01:31:26 <phantomcircuit> bd_, and well
253 2011-03-31 01:31:27 <phantomcircuit> meh
254 2011-03-31 01:31:34 <bd_> phantomcircuit: how are joins involved?
255 2011-03-31 01:32:15 <phantomcircuit> bd_, selects are almost all blocks/transactions/transaction input/outputs
256 2011-03-31 01:32:26 <phantomcircuit> bd_, but afaict the selects are trivial
257 2011-03-31 01:32:55 <bd_> Yes. bitcoin doesn't really need much in the way of complex join
258 2011-03-31 01:32:56 <bd_> s
259 2011-03-31 01:33:12 <lfm> lfm: but ya it looks like ebeats means UTC date/time
260 2011-03-31 01:33:40 <phantomcircuit> http://pastebin.com/PTVfrkQp
261 2011-03-31 01:33:41 <phantomcircuit> bah
262 2011-03-31 01:33:47 <bd_> phantomcircuit: Anyway, joins aren't magical. They're just index lookups, possibly several in a row.
263 2011-03-31 01:33:48 <phantomcircuit> 2 seconds
264 2011-03-31 01:33:54 <phantomcircuit> 40 seconds
265 2011-03-31 01:34:01 <phantomcircuit> 60 seconds
266 2011-03-31 01:34:18 <bd_> phantomcircuit: Have you profiled to see where your time is being spent?
267 2011-03-31 01:34:26 <bd_> actually
268 2011-03-31 01:34:30 <phantomcircuit> that is my profiling actually
269 2011-03-31 01:34:32 <bd_> it might be syncs on commit
270 2011-03-31 01:34:45 <bd_> phantomcircuit: profiling tells you -where- the time is being spent, not just how much :)
271 2011-03-31 01:35:12 <phantomcircuit> bd_, that is telling me
272 2011-03-31 01:35:23 <phantomcircuit> the time is being spent in handling "block" commands
273 2011-03-31 01:36:49 <bd_> phantomcircuit: Or it's being spent holding a lock that the block commands need.
274 2011-03-31 01:37:02 <bd_> So - what lock is it, and who is holding the lock?
275 2011-03-31 01:37:09 <bd_> -that- is profiling :)
276 2011-03-31 01:37:40 <phantomcircuit> bd_, this is python
277 2011-03-31 01:37:44 <phantomcircuit> good luck with that
278 2011-03-31 01:38:13 <bd_> *shrug* you can always start instrumenting other paths
279 2011-03-31 01:38:26 <bd_> but at a guess - maybe syncs on database commits are taking a lot of time?
280 2011-03-31 01:38:40 <bd_> Or you might be missing some important indices
281 2011-03-31 01:40:11 <phantomcircuit> bd_, yes and no
282 2011-03-31 01:40:12 <phantomcircuit> :P
283 2011-03-31 01:40:48 <xenon481> Dear God, I wish that FairUser and Geebus would get a clue.......
284 2011-03-31 01:41:02 <grbgout> what have they done now?
285 2011-03-31 01:41:22 <xenon481> They are saying that Raulo's attack isn't real.
286 2011-03-31 01:41:24 <LobsterMan> ;;bc,stats
287 2011-03-31 01:41:25 <phantomcircuit> bd_, turns out the problem is i made a bad assumption
288 2011-03-31 01:41:26 <gribble> Current Blocks: 115871 | Current Difficulty: 68978.89245792 | Next Difficulty At Block: 116927 | Next Difficulty In: 1056 blocks | Next Difficulty In About: 6 days, 15 hours, 34 minutes, and 24 seconds | Next Difficulty Estimate: 76183.15879097
289 2011-03-31 01:41:26 <phantomcircuit> hmm
290 2011-03-31 01:41:34 <LobsterMan> just generated a block XD
291 2011-03-31 01:41:52 <bd_> phantomcircuit: yes, most programming bugs originate from bad assumptions :)
292 2011-03-31 01:42:02 <lfm> LobsterMan: wtg
293 2011-03-31 01:42:06 <LobsterMan> hehehe ^______^
294 2011-03-31 01:42:08 <grbgout> Haven't heard of Raulo's attack --- well, maybe vaguely; the name does seem familiar.  Care to elaborate, or point to a relavent page (not about them denying it, rather about it)?
295 2011-03-31 01:42:12 <phantomcircuit> bd_, yeah
296 2011-03-31 01:42:16 <LobsterMan> ty ty
297 2011-03-31 01:42:33 <phantomcircuit> bd_, to be fair i had predicted it was a bad assumption
298 2011-03-31 01:42:33 <xenon481> It's the pool-hopping attack that Share based pools are vulnerable to.
299 2011-03-31 01:42:34 <phantomcircuit> lol
300 2011-03-31 01:42:40 <LobsterMan> is it possible that stopping and starting my miners frequently can actually increase the chances of generating a block?
301 2011-03-31 01:42:47 <LobsterMan> due to changes in entropy/whatnot?
302 2011-03-31 01:42:54 <LobsterMan> like when i play a game i usually close my miners
303 2011-03-31 01:43:01 <phantomcircuit> LobsterMan, no
304 2011-03-31 01:43:46 <lfm> LobsterMan: nope, the more you run the more your chances
305 2011-03-31 01:43:49 <grbgout> xenon481: interesting.
306 2011-03-31 01:43:59 <xenon481> LobsterMan: Your miners aren't doing anything random, so there is no possibility that they could be exhibiting semi-random behavior as opposed to random behavior.
307 2011-03-31 01:44:43 <lfm> xenon481: that doesnt quite sound like an explanation! :-)
308 2011-03-31 01:45:29 <xenon481> LobsterMan: Restarting your miners would only be useful if you felt that they were somehow being caught in some sort of trap of your computer's random number table (which is only semi random). Restarting them would seed them to a different part of the table.
309 2011-03-31 01:45:55 <LobsterMan> so i suppose then it does change it up, so to speak
310 2011-03-31 01:46:07 <LobsterMan> but whether or not it helps me in the long run is indeterminante?
311 2011-03-31 01:46:21 <xenon481> LobsterMan: No, due to the fact that the miners don't use the random number table.
312 2011-03-31 01:46:23 <grbgout> or, more to the point, is completely random :)
313 2011-03-31 01:46:25 <phantomcircuit> https://github.com/phantomcircuit/bitcoin-alt/blob/sqlite3/bitcoin/peer.py#L137
314 2011-03-31 01:46:49 <xenon481> Lobsterman: It would only be useful IF they were using the random number table.
315 2011-03-31 01:46:54 <LobsterMan> oh...
316 2011-03-31 01:47:13 <lfm> LobsterMan: nope, if you stop it for one sec you have lost one sec worth of chances and wont get em back
317 2011-03-31 01:47:55 <LobsterMan> so those calculations that would be performed in that one second of miner stoppage, is that same calculation deferred and just happens when the miner starts up again?
318 2011-03-31 01:48:01 <xenon481> and only if you were certain that the part of the random number table you would have been using was somehow semi-random in a way that specifically causes you non-optimal processing.
319 2011-03-31 01:48:28 <lfm> LobsterMan: no, nothing is defered. you just start new
320 2011-03-31 01:48:42 <xenon481> LobsterMan: No, the GetWork server just assigns you another "random" piece of work.
321 2011-03-31 01:49:06 <LobsterMan> see that is what i mean by "random"
322 2011-03-31 01:49:20 <LobsterMan> due to it starting at a different time, something different is being calculated upon
323 2011-03-31 01:49:28 <xenon481> None of the calculations you are doing are in any way dependent upon the previous calculations that you have done, so you don't need to work in order, but there is also no reason not to work in order.
324 2011-03-31 01:49:36 <lfm> LobsterMan: note the khash/s number? each hash is separate and a separate chance to find a good hash. that whole thing is done thousands of times per sec
325 2011-03-31 01:49:39 <phantomcircuit> bd_, https://github.com/phantomcircuit/bitcoin-alt/blob/sqlite3/bitcoin/peer.py#L137
326 2011-03-31 01:49:57 <LobsterMan> yes but the hashes it's trying, how is that determined?
327 2011-03-31 01:50:06 <LobsterMan> is it just trying "random" solutins to see if it solves a block?
328 2011-03-31 01:50:13 <LobsterMan> solutions*
329 2011-03-31 01:50:28 <lfm> LobsterMan: every try is different. when you start new it starts at a new place
330 2011-03-31 01:50:39 <bd_> phantomcircuit: hah, that would do it :)
331 2011-03-31 01:51:09 <LobsterMan> i think i still don't totally understand exactly what my miner is doing at each hash attempt
332 2011-03-31 01:51:21 <xenon481> The miner doesn't decide where, though. The GetWork server does.
333 2011-03-31 01:51:36 <LobsterMan> and the getwork server is managed by bitcoin.exe?
334 2011-03-31 01:51:38 <bd_> phantomcircuit: Next time actually use the python profiler (http://docs.python.org/library/profile.html) and you'd see where the time was going from the start ;)
335 2011-03-31 01:51:48 <lfm> LobsterMan: one part of the block header is a timestamp, changes every second
336 2011-03-31 01:51:55 <jgarzik> LobsterMan: the getwork server == the bitcoin.exe JSON-RPC server
337 2011-03-31 01:52:26 <phantomcircuit> bd_, doesn't particularly help actually
338 2011-03-31 01:52:35 <LobsterMan> so is the sequence of attempts it makes totally deterministic?
339 2011-03-31 01:52:48 <phantomcircuit> (i tried it)
340 2011-03-31 01:53:35 <lfm> LobsterMan: kinda ya. the inputs are sequential deterministic but the hash algorith produces a cryptogrphicly secure random output
341 2011-03-31 01:53:45 <bd_> phantomcircuit: oh?
342 2011-03-31 01:54:04 <xenon481> There are a non-infinite number of possible solutions, but the number is so large that it is essentially infinite for all calculation purposes.
343 2011-03-31 01:54:18 <phantomcircuit> bd_, not sure why but cProfile says that almost no time is spent in put_blocks
344 2011-03-31 01:54:21 <phantomcircuit> bd_, so
345 2011-03-31 01:54:23 <phantomcircuit> yeah
346 2011-03-31 01:55:05 <lfm> phantomcircuit: is your time cpu time or elapsed time?
347 2011-03-31 01:56:30 <phantomcircuit> hmm?
348 2011-03-31 01:57:50 <phantomcircuit> lfm, very little cpu time is used, so long as there is little contention
349 2011-03-31 02:00:06 <LobsterMan> let me try to phrase this differently:
350 2011-03-31 02:00:07 <LobsterMan> what i mean by "random", as in...if i start the miner (say at 1pm), stop it for 10 mins, and then start it again (at 1:10pm), and then it has been running for 10 mins (now 1:20pm); as opposed if I just let it run from from 1pm to 1:10pm
351 2011-03-31 02:00:09 <LobsterMan> in the first case where i stop it, after the 10 minutes is up and then start mining again, is it going to try the same hashes (during computation from 1:10 to 1:20) that would have been tried if i just did not close it in the first place? (same hashes as if I just ran it from 1:00 to 1:10?)
352 2011-03-31 02:01:06 <LobsterMan> i hope that makes sense...lol
353 2011-03-31 02:01:13 <xenon481> LobsterMan: It will never work the same hashes twice.
354 2011-03-31 02:01:15 <xenon481> ever.
355 2011-03-31 02:01:16 <lfm> LobsterMan: it doesnt really matter WHICHJ hashes it does since the chance of finding a good one is the same for every hash
356 2011-03-31 02:01:20 <xenon481> in all likelyhood.
357 2011-03-31 02:01:37 <LobsterMan> yes but that is what i mean by "random"
358 2011-03-31 02:01:41 <LobsterMan> i use random loosely
359 2011-03-31 02:01:44 <phantomcircuit> LobsterMan, in those 10 minutes the timestamp and merkle root will have changed
360 2011-03-31 02:01:59 <LobsterMan> it's like the monty hall problem kind of
361 2011-03-31 02:02:08 <LobsterMan> http://en.wikipedia.org/wiki/Monty_hall_problem
362 2011-03-31 02:02:12 <grbgout> >_<
363 2011-03-31 02:02:21 <grbgout> I was keeping the monty hall paradox to myself, damn it, LobsterMan!
364 2011-03-31 02:02:29 <LobsterMan> haha
365 2011-03-31 02:02:32 <lfm> LobsterMan: you will never go back and fill in missed ones and you shouldnt
366 2011-03-31 02:02:42 <xenon481> also what phantomcircuit said. Even if you did process the same hashes twice, what you are hashing completely changed, so it doesn't matter.
367 2011-03-31 02:05:35 <LobsterMan> so if i start the miner at 1:00, then stop at 1:05, then start again at 1:10, it's going to be trying the same sequence of hashes at 1:11 as if I would have just left it go from 1:00 to 1:11?
368 2011-03-31 02:05:47 <lfm> ok total number of hashes tried by all the miners since bitcoin started more than 2 years ago is about 2.93566e+18 and the total number of different hash results possible is 1.16E+77 so the chances of two hashes being the same is rather small
369 2011-03-31 02:07:33 <LobsterMan> i guess what i'm trying to get at is how does the miner choose which numbers to try in its calculations?
370 2011-03-31 02:07:47 <LobsterMan> and does stopping/starting it at different times alter this sequence?
371 2011-03-31 02:07:58 <lfm> LobsterMan: have you heard of "the nonce"?
372 2011-03-31 02:08:11 <LobsterMan> yes, but i don't know exactly what it means
373 2011-03-31 02:08:35 <LobsterMan> http://en.wikipedia.org/wiki/Cryptographic_nonce this?
374 2011-03-31 02:08:50 <phantomcircuit> LobsterMan, so you're asking technical questions without having actually learned how it works?
375 2011-03-31 02:08:52 <phantomcircuit> facepalm
376 2011-03-31 02:08:59 <grbgout> xenon481: are you aware of any pages that explain how the Raulo attack works (preferably in detail)?
377 2011-03-31 02:08:59 <LobsterMan> lol
378 2011-03-31 02:09:03 <LobsterMan> i'm trying to understand how it works :P
379 2011-03-31 02:09:16 <grbgout> LobsterMan: you should read the white paper.
380 2011-03-31 02:09:17 <LobsterMan> learn/understand*
381 2011-03-31 02:09:27 <LobsterMan> hmm probably
382 2011-03-31 02:09:30 <phantomcircuit> https://en.bitcoin.it/wiki/Block_hashing_algorithm
383 2011-03-31 02:09:33 <grbgout> LobsterMan: http://www.bitcoin.org/faq#How_does_Bitcoin_work
384 2011-03-31 02:09:33 <lfm> ok each time you start you have a different time stamp of course and ever miner has a different address for their input txn so no one starts with the same block. then you just set the nonce to zero and increase it by one for each hash try
385 2011-03-31 02:09:35 <phantomcircuit> LobsterMan, https://en.bitcoin.it/wiki/Block_hashing_algorithm
386 2011-03-31 02:09:42 <grbgout> I recommend the white paper!
387 2011-03-31 02:09:45 <xenon481> grbgout: there is an entire white paper on it that explains all of the math.
388 2011-03-31 02:09:55 <grbgout> xenon481: awesome! Do you recall the title?
389 2011-03-31 02:10:10 <grbgout> maybe googling Raulo Attack white paper would work?
390 2011-03-31 02:10:16 <grbgout> I tried raulo bitcoin attack thus far
391 2011-03-31 02:10:28 <xenon481> http://bitcoin.atspace.com/poolcheating.pdf
392 2011-03-31 02:10:35 <grbgout> xenon481: you're the best, thank you.
393 2011-03-31 02:10:49 <xenon481> as mentioned in this and many other threads: http://www.bitcoin.org/smf/index.php?topic=4787.0
394 2011-03-31 02:11:41 <grbgout> ah, I skimmed that thread when I first looked into BitCoin, but it seems I simply determined "okay, pooling is safe: depending on the pool."
395 2011-03-31 02:11:43 <xenon481> Here is the original thread: http://www.bitcoin.org/smf/index.php?topic=3165.0
396 2011-03-31 02:11:58 <grbgout> awesome, thanks again
397 2011-03-31 02:12:06 <phantomcircuit> bd_, yeah it's that
398 2011-03-31 02:12:07 <phantomcircuit> darn
399 2011-03-31 02:19:44 <LobsterMan> but yes....the monty hall problem comparison is really what i was getting at
400 2011-03-31 02:19:45 <LobsterMan> i may not actually know what i'm talking about, but i think a read through the bitcoin spec and a bit about merkle trees is in order... :)
401 2011-03-31 02:20:27 <xenon481> LobsterMan: Monty Hall doesn't fit in this scenario.
402 2011-03-31 02:20:35 <lfm> LobsterMan: you get different hashses if you dont stop too. there is no advantage to stopping
403 2011-03-31 02:21:00 <LobsterMan> i don't totally understand how the monty hall problem works either :D
404 2011-03-31 02:21:13 <xenon481> Monty Hall only fits with a known and limited set and only if you somebody is purposefully showing you something that you already know is bad.
405 2011-03-31 02:21:21 <LobsterMan> hehe
406 2011-03-31 02:22:18 <LobsterMan> but the set of all possible hashes is known and limited as (i think) lfm said
407 2011-03-31 02:22:20 <xenon481> This is more like Deal or No Deal instead of Monty Hall
408 2011-03-31 02:23:08 <grbgout> LobsterMan: dude! Just learn about it, and test your hypothesis on the testnet, stop givin' away ur secretz
409 2011-03-31 02:23:16 <lfm> LobsterMan: huh? the set of all possible hashes is 2^256 so it isnt really known and it isnt in any order
410 2011-03-31 02:23:17 <xenon481> LobsterMan: Yes, the hashes are known and limited, but only for one particular thing you are working on.
411 2011-03-31 02:23:27 <xenon481> the thing you are working on is essentially constantly changing.
412 2011-03-31 02:23:57 <xenon481> and yes, 2^256 is essentially infinite for any calculation purposes.
413 2011-03-31 02:24:05 <grbgout> As I understand it, the timestamps, nonces, etc., is what cripples the monty hall paradox in this case.
414 2011-03-31 02:24:17 <xenon481> Not just that.
415 2011-03-31 02:24:20 <lfm> only limited in a theoretical sense. in practise there are so many we will never see them all
416 2011-03-31 02:24:27 <grbgout> xenon481: "etc." means et cetera
417 2011-03-31 02:24:45 <xenon481> right, but the problem is more fundamental than that.
418 2011-03-31 02:25:05 <xenon481> This is the difference between Deal or No Deal and Monty Hall
419 2011-03-31 02:25:09 <grbgout> elaborate, if you would be so kind.  I realize that's /exactly/ what you've been doing for about an hour now ^_^
420 2011-03-31 02:25:22 <grbgout> one sec
421 2011-03-31 02:25:26 <grbgout> let me look up DoND
422 2011-03-31 02:25:46 <lfm> I dont think game shows are any help
423 2011-03-31 02:25:55 <xenon481> Monty Hall, no matter what door you pick, you know that Monty is going to reveal a door that was not the winner.
424 2011-03-31 02:26:11 <grbgout> I'm familiar.
425 2011-03-31 02:26:38 <xenon481> In DoND, there is no guarantee that the briefcase they open to show you is or isn't going to be the winner.
426 2011-03-31 02:26:44 <Netsniper> montyhallproblem
427 2011-03-31 02:27:07 <xenon481> hashing is DoND.
428 2011-03-31 02:29:27 <grbgout> I really want to jump in with my hypotheses on this matter, but I'd rather keep them to myself, investigate with testnet, and save us all the trouble. Ultimately publishing to the forum to put it to rest.
429 2011-03-31 02:30:24 <LobsterMan> i'm going to have to do some testnet testing too lol
430 2011-03-31 02:30:25 <grbgout> LobsterMan: perhaps we can work together.
431 2011-03-31 02:30:31 <grbgout> Anyway, I haven't asked about the paradox yet because I'm not familiar enough with the technical details of btc.  I've been assuming that once I become so, then the paradox will quite obviously not apply.
432 2011-03-31 02:30:39 <LobsterMan> yeah same here
433 2011-03-31 02:30:46 <LobsterMan> i don't really understand all the technical details of bitcoin as well
434 2011-03-31 02:30:54 <grbgout> LobsterMan: no, not same there, you totally mentioned it! Yur given away our sekretz ;)
435 2011-03-31 02:30:56 <nanotube> grbgout: monty hall: requires that there be a party that knows ahead of time what's behind all doors.
436 2011-03-31 02:31:44 <nanotube> grbgout: block hashing: nobody knows ahead of time
437 2011-03-31 02:31:48 <grbgout> nanotube: I'm aware of monty hall.
438 2011-03-31 02:31:51 <grbgout> nanotube: I'm aware.
439 2011-03-31 02:31:53 <nanotube> hehe ok
440 2011-03-31 02:31:54 <grbgout> I'll address that in time :)
441 2011-03-31 02:31:54 <nanotube> but you said 'i'm not familiar with technical details, ... paradox, etc.
442 2011-03-31 02:31:55 <grbgout> by which I mean the nonces, merkle trees, block headers, protocols, etc.  In other words: I have yet to finish reading the white paper ('cause I'm a jerk), and I havne't read the source yet.
443 2011-03-31 02:31:55 <nanotube> heh k