1 2012-04-23 02:17:26 <thunderchao> good evening, all
  2 2012-04-23 02:18:29 <thunderchao> i had a question about upgrading, since i've never done that before with bitcoin. do i just reinstall over the current version?
  3 2012-04-23 02:18:46 <setkeh> hey guys im trying to get mtgox data using hook.io using the video as an example but i keen running in to this issue http://screenshots.setkeh.com/screenshot-04.23.12-14.17.47.png any idea's ??
  4 2012-04-23 02:20:59 <BlueMatt> #mtgox ?
  5 2012-04-23 02:21:46 <setkeh> ok
  6 2012-04-23 04:06:46 <Cory> EXCEPTION: 11DbException
  7 2012-04-23 04:06:50 <Cory> Is that significant?
  8 2012-04-23 04:07:04 <Cory> I think Windows deleted some stuff.
  9 2012-04-23 04:07:37 <wumpus> yes, they could
 10 2012-04-23 06:58:07 <setkeh> Hey guys im trying to setup hook.io and hook.io-mtgox to work like the in the video but i keep running into these issues http://screenshots.setkeh.com/screenshot-04.23.12-18.48.27.png any ideas ?? Video: http://www.youtube.com/watch?v=KD5ljtNK72U and the mtgox hookio http://github.com/cronopio/hook.io-mtgox
 11 2012-04-23 07:47:51 <DiabloD3> ;;bc,gen 85000000
 12 2012-04-23 07:47:52 <gribble> The expected generation output, at 85000000 Khps, given current difficulty of 1577913.4856767 , is 54.1825728253 BTC per day and 2.25760720105 BTC per hour.
 13 2012-04-23 07:48:33 <DiabloD3> !ticker
 14 2012-04-23 07:48:34 <gribble> Best bid: 5.14996, Best ask: 5.16889, Bid-ask spread: 0.01893, Last trade: 5.15, 24 hour volume: 35375, 24 hour low: 5.101, 24 hour high: 5.25
 15 2012-04-23 07:48:42 <DiabloD3> ;;bc,calc 5.14 * 54.18
 16 2012-04-23 07:48:43 <gribble> The average time to generate a block at 5.14 * 54.18 Khps, given current difficulty of 1577913.4856767 , is 771 years, 35 weeks, 1 day, 7 hours, 19 minutes, and 2 seconds
 17 2012-04-23 07:48:45 <DiabloD3> err
 18 2012-04-23 07:48:48 <DiabloD3> ;;calc 5.14 * 54.18
 19 2012-04-23 07:48:48 <gribble> 278.4852
 20 2012-04-23 11:08:27 <ovidiusoft> anyone else experiencing random crashes with 0.6.0? (linux, daemon mode)
 21 2012-04-23 11:08:55 <ovidiusoft> once in a while such a crash will corrupt the data files and then bitcoind will not start anymore
 22 2012-04-23 11:25:14 <luke-jr> ovidiusoft: addr.dat?
 23 2012-04-23 11:25:45 <abortretry> hello.... i am starting to understand and reading through the bitcoin client source. i have an experimental project of creating a new currency. and i am puzzled how to speed up creating the genesis block. can someone give me a helping hand
 24 2012-04-23 11:26:01 <ovidiusoft> luke-jr: i think so, yes
 25 2012-04-23 11:26:12 <ovidiusoft> i deletec the corefile by mistake
 26 2012-04-23 11:26:39 <ovidiusoft> i'll post a trace if it crashes again
 27 2012-04-23 11:27:30 <ovidiusoft> the only thing that seems extraordinary in my setup is the large number of concurrent connections, limit is 500 and i usually have over 200
 28 2012-04-23 11:59:48 <hazek> can someone help me please
 29 2012-04-23 11:59:55 <hazek> is this statement correct:
 30 2012-04-23 12:00:12 <hazek> valid bitcoin txs must be logged into the blockchain
 31 2012-04-23 12:00:34 <helo> false
 32 2012-04-23 12:00:44 <helo> there are two principles: validity and confirmatin
 33 2012-04-23 12:01:16 <hazek> ah
 34 2012-04-23 12:01:20 <helo> confirmation is achieved by entry into the blockchain.
 35 2012-04-23 12:01:26 <hazek> so being logged has nothing to do with validity
 36 2012-04-23 12:02:05 <hazek> if I were to list txs rules in Bitcoin, would these be it:
 37 2012-04-23 12:02:09 <hazek> -no double spending
 38 2012-04-23 12:02:14 <hazek> -irreversible
 39 2012-04-23 12:02:18 <helo> right... validity confirms that the creator of the transaction has control of the coin being sent, and that it has not been sent thus far
 40 2012-04-23 12:02:27 <hazek> -no blocking
 41 2012-04-23 12:02:44 <hazek> -must have a valid cryptographic digital signature
 42 2012-04-23 12:02:54 <hazek> -must be addressed to a valid address
 43 2012-04-23 12:03:00 <hazek> am I forgetting soemthing?
 44 2012-04-23 12:03:35 <luke-jr> hazek: txns don't need to be to an address
 45 2012-04-23 12:03:44 <hazek> no?
 46 2012-04-23 12:04:14 <luke-jr> nor are they irrevesible or doublespend-free until they are confirmed at depth 6
 47 2012-04-23 12:04:18 <hazek> I don't mean strictly the code rules
 48 2012-04-23 12:04:21 <hazek> I mean practically
 49 2012-04-23 12:05:23 <hazek> hmmmm
 50 2012-04-23 12:06:05 <hazek> I see I need to do some more thinking about this
 51 2012-04-23 12:06:16 <hazek> (i'm trying to write a guide for laypeople)
 52 2012-04-23 12:07:21 <hazek> maybe listing rules isn't the best approach to explaning how transactions work in bitcoin
 53 2012-04-23 12:11:19 <helo> the rules in the code result in some emergent properties that allow the system to function as a currency
 54 2012-04-23 12:12:24 <hazek> right right
 55 2012-04-23 12:12:27 <hazek> ty guys
 56 2012-04-23 15:28:27 <Diapolo> BlueMatt: a little ping ;)?
 57 2012-04-23 15:28:30 <Diapolo> and hello all
 58 2012-04-23 16:22:54 <jgarzik> sipa, tcatm, gmaxwell: do we really need a 0.6.1 release?  why not go straight to 0.7...
 59 2012-04-23 16:27:49 <paulo_> how does a bitcoin address get correlated with a public key?
 60 2012-04-23 16:28:10 <luke-jr> it doesn't.
 61 2012-04-23 16:28:22 <luke-jr> what exactly do you mean?
 62 2012-04-23 16:28:22 <paulo_> or does a transaction contain both public key and address?
 63 2012-04-23 16:28:30 <sipa> that
 64 2012-04-23 16:28:39 <luke-jr> to send from an address, you need to provide the pubkey
 65 2012-04-23 16:28:52 <[Tycho]> Hello, devs :)
 66 2012-04-23 16:29:14 <paulo_> so, does a transaction contain both public key and address?
 67 2012-04-23 16:29:42 <paulo_> i mean the sending address
 68 2012-04-23 16:29:42 <sipa> paulo_: yes
 69 2012-04-23 16:29:56 <sipa> transaction outputs contain an address
 70 2012-04-23 16:30:04 <sipa> transaction inputs contains a signature + public key
 71 2012-04-23 16:30:19 <sipa> the signature must be valid for the given public key, and the public key must hash to the address
 72 2012-04-23 16:32:54 <sipa> jgarzik: hmm, i believe the current code is better than 0.6.0 and probably doesn't take long to go from rc to final
 73 2012-04-23 16:33:20 <sipa> while 0.7.0 would take longer
 74 2012-04-23 16:34:17 <sipa> now, BlueMatt closed cblockstore (pity...), so unless there's some changes there soon, that makes the changes for 0.7.0 smaller
 75 2012-04-23 16:34:25 <luke-jr> imo, should be merged for 0.6.1: 936, 1032, 1090, 917, 1002, 1119, 1133, 1134, 1135, 1136, 1139; maybe 1128
 76 2012-04-23 16:35:02 <gmaxwell> jgarzik: Right now we're pretty ripe for a bunch of cleanups and fixes.  And we don't have new major features in shipping ready state yet.
 77 2012-04-23 16:35:32 <paulo_> another question
 78 2012-04-23 16:35:41 <gmaxwell> Also, it seems like simply churning out more releases get more people ot upgrade to the improved versions we already have out.
 79 2012-04-23 16:35:44 <paulo_> how do you prevent resending of a transaction?
 80 2012-04-23 16:35:55 <sipa> paulo_: why would you want to prevent that?
 81 2012-04-23 16:36:28 <sipa> paulo_: or do you mean how to prevent double spending (i.e., another transaction but with the same input)
 82 2012-04-23 16:36:54 <gavinandresen> jgarzik: besides the time it takes to spin a release, is there any other reason NOT to do a 0.6.1rc1 in the next day or two?
 83 2012-04-23 16:37:09 <jgarzik> works for me...
 84 2012-04-23 16:37:09 <paulo_> no, the exact same transaction.
 85 2012-04-23 16:37:21 <sipa> paulo_: no need to prevent that
 86 2012-04-23 16:37:45 <paulo_> I mean, I can easily resend a transaction, since it still has the same signatures
 87 2012-04-23 16:37:59 <sipa> paulo_: yes, so?
 88 2012-04-23 16:38:24 <sipa> paulo_: if others already have it in their memory pools or blockchain, they ignore it
 89 2012-04-23 16:38:40 <sipa> if the receiver already has it in his wallet, nothing changes
 90 2012-04-23 16:39:40 <paulo_> sipa: but what if I really do want to make the same transaction?
 91 2012-04-23 16:39:47 <paulo_> same inputs and outputs
 92 2012-04-23 16:40:22 <sipa> paulo_: not sure what you're trying to do or trying to get at
 93 2012-04-23 16:40:34 <sipa> the same transaction is the same transaction
 94 2012-04-23 16:41:50 <paulo_> how will you differentiate two transactions with the exact same amount, inputs, and outputs?
 95 2012-04-23 16:42:19 <sipa> paulo_: if they are exactly the same, it's just the same transaction; if they're not exactly the same, it's a double spend
 96 2012-04-23 16:43:08 <paulo_> what does "not exactly the same" mean?
 97 2012-04-23 16:43:15 <sipa> paulo_: not byte for byte the same
 98 2012-04-23 16:43:33 <paulo_> what would differentiate them?
 99 2012-04-23 16:43:34 <jgarzik> luke-jr: I agree on: #1090, #1002
100 2012-04-23 16:43:40 <jgarzik> gavinandresen: ^^
101 2012-04-23 16:43:45 <paulo_> a timestamp?
102 2012-04-23 16:43:48 <sipa> paulo_: for example ECDSA signatures are not deterministic, so you could have the same in- and outputs, but different signatures
103 2012-04-23 16:43:58 <sipa> paulo_: or the order of inputs could differ
104 2012-04-23 16:44:03 <jgarzik> the other stuff luke-jr mentioned is either (a) already closed or (b) under discussion or (c) typical luke-jr-is-only-audience stuff
105 2012-04-23 16:44:29 <sipa> ACK 936, but for 0.7.0
106 2012-04-23 16:44:38 <paulo_> non-deterministic signatures... interesting
107 2012-04-23 16:44:58 <jgarzik> some other stuff on luke-jr's list will probably be ready by 0.7
108 2012-04-23 16:45:05 <gavinandresen> 1090: ACK
109 2012-04-23 16:45:47 <sipa> i don't comment to much on the GUI things; i assume they will get merged when they are ready
110 2012-04-23 16:46:01 <luke-jr> 1032 needs some time for Diapolo to go through the translations
111 2012-04-23 16:46:40 <jgarzik> sipa: same here
112 2012-04-23 16:47:08 <gavinandresen> 1002: ok. URI handling doesn't work for the vast majority of our users now anyway....
113 2012-04-23 16:47:33 <gavinandresen> I tagges 1101 for 0.6.1
114 2012-04-23 16:47:44 <paulo_> beats RSA for this purpose.
115 2012-04-23 16:48:12 <luke-jr> jgarzik: what's the problem with 917 and 1139?
116 2012-04-23 16:48:28 <gavinandresen> 917 is a new feature
117 2012-04-23 16:49:01 <gavinandresen> 1139 will require new translations, which is more trouble than it is worth IMHO
118 2012-04-23 16:49:15 <jgarzik> 917 needs ACKs
119 2012-04-23 16:49:27 <jgarzik> what gavinandresen said, on 1139
120 2012-04-23 16:49:30 <luke-jr> gavinandresen: well, the tooltip in 1139 is just plain wrong, right now
121 2012-04-23 16:49:56 <sipa> ACK 1139
122 2012-04-23 16:50:00 <luke-jr> if it's not worth translating for 0.6.1, it should probably be removed until 0.7
123 2012-04-23 16:50:58 <sipa> how about #1119?
124 2012-04-23 16:51:39 <gavinandresen> I REALLY want to pul 1119 but it scares me
125 2012-04-23 16:52:21 <sipa> yes, i understand, but i'm not sure what can be done to improve it any further
126 2012-04-23 16:52:35 <gavinandresen> Has anybody looked at subversion's bdb code?  I used subversion for years and never saw a DB_RUNRECOVERY or other error....
127 2012-04-23 16:52:45 <jgarzik> what gavinandresen said, on 1119 ;-)
128 2012-04-23 16:52:48 <jgarzik> makes me nervous
129 2012-04-23 16:53:06 <luke-jr> gavinandresen: IIRC, Subversion stopped using BDB a long time ago?
130 2012-04-23 16:54:38 <gmaxwell> luke-jr: yes, back when it stopped losing data! :(
131 2012-04-23 16:55:09 <luke-jr> LOL
132 2012-04-23 16:55:58 <gmaxwell> I've been quiet w/ acks lately because lots of the recent pulls felt like they needed testing and I juts haven't had time the last couple weeks. :-/ (I'm very glad to have seen luke objdump/diffing the asm on changes that shouldn't have changed the code)
133 2012-04-23 16:55:58 <sipa> there's still a bdb directory in the latest source code
134 2012-04-23 16:59:44 <sipa> gavinandresen: SVN doesn't call lsn_reset anywhere, though
135 2012-04-23 16:59:55 <gavinandresen> sipa: very interesting...
136 2012-04-23 16:59:58 <sipa> but they assume their database directory untouchable by the outside world
137 2012-04-23 17:00:09 <sipa> as we should too, if it weren't for wallet files in it
138 2012-04-23 17:02:00 <gavinandresen> I think you're right.
139 2012-04-23 17:02:23 <jrmithdobbs> gavinandresen: i've seen svn eat it's bdb backend
140 2012-04-23 17:02:24 <jrmithdobbs> fwiw
141 2012-04-23 17:02:49 <jrmithdobbs> all caused by bugs in svn though, as far as I can remember
142 2012-04-23 17:03:00 <gavinandresen> jrmithdobbs: sure, with enough customers you'll see EVERYTHING eventually, including "cosmic ray changed exactly the wrong bit(s)"
143 2012-04-23 17:03:05 <jrmithdobbs> especially <=1.5 had lots of weird binary corruption stuff that could happen in certain instances
144 2012-04-23 17:03:19 <gavinandresen> good thing we're not using <= 1.5
145 2012-04-23 17:03:36 <sipa> i didn't know we were still using SVN at all?
146 2012-04-23 17:03:49 <jrmithdobbs> well, i haven't had to touch an svn repo since we upgraded that one to 1.6 (and I left the company like 3 months later)
147 2012-04-23 17:03:55 <jrmithdobbs> I have no idea the current state/stability of svn ;p
148 2012-04-23 17:04:07 <gavinandresen> (sorry, right, svn 1.5 not bdb 1.5....)
149 2012-04-23 17:04:16 <jrmithdobbs> (other than the obvious: "Why are you using svn in 2012?! lol" comment ...)
150 2012-04-23 17:04:20 <gavinandresen> back in half hour or so....
151 2012-04-23 17:05:23 <sipa> jrmithdobbs: i was surprised to learn that subversion didn't migrate its repository to git yet ;)
152 2012-04-23 17:05:32 <gmaxwell> haha
153 2012-04-23 17:05:48 <jrmithdobbs> sipa: equally surprised the merc guys haven't either ;p
154 2012-04-23 17:05:57 <sipa> Revision: 1329369
155 2012-04-23 17:06:00 <sipa> ... wow
156 2012-04-23 17:06:19 <gmaxwell> In any case, if we split the wallet and rest of the enviroment, might we make this stuff safer? (e.g. other stuff being unclean not breaking the wallet)
157 2012-04-23 17:07:24 <sipa> gmaxwell: regarding #1119?
158 2012-04-23 17:07:44 <jrmithdobbs> 1119 scares me too fwiw
159 2012-04-23 17:08:22 <gmaxwell> sipa: yes, in particular, but also generally. E.g. I worry that people with unclean databases swapping around wallet files will get corrupted.
160 2012-04-23 17:08:31 <sipa> it shouldn't - the only thing that changes is this lsn_reset call; that call is not part of normal operation, it's only necessary for moving files in and out of environments
161 2012-04-23 17:08:36 <gmaxwell> Esp since a lot of users now kill the process due to slow shutdowns.
162 2012-04-23 17:08:50 <jrmithdobbs> eww, why would you do that? it's a bdb app
163 2012-04-23 17:08:54 <jrmithdobbs> that's asking for trouble
164 2012-04-23 17:09:16 <gmaxwell> Because we don't have coincontrol. Also to test backups.
165 2012-04-23 17:09:27 <jrmithdobbs> coincontrol?
166 2012-04-23 17:09:30 <sipa> yes, wallet files are being moved around
167 2012-04-23 17:09:39 <sipa> so there is an lsn_reset on wallet files
168 2012-04-23 17:10:00 <sipa> but there is absolutely no need for such a call on the block database
169 2012-04-23 17:10:19 <sipa> unless you do funky stuff like resetting block databases for particular test cases
170 2012-04-23 17:11:13 <sipa> gmaxwell: have you seen how fast shutdown is without the lsn_reset call?
171 2012-04-23 17:11:31 <sipa> (unless you shutdown right in the middle of an initial block chain sync)
172 2012-04-23 17:11:33 <luke-jr> jrmithdobbs: ironically, svn still has features missing from all DVCS
173 2012-04-23 17:14:20 <gmaxwell> jrmithdobbs: coin control = feature that lets you control which inputs you're using in transactions.
174 2012-04-23 17:15:25 <gmaxwell> sipa: right, but that was my point if we split the enviroment perhaps it will still be fast if we're just lsn resetting on the wallet.
175 2012-04-23 17:16:31 <sipa> gmaxwell: lsn resets are already per file
176 2012-04-23 17:16:49 <sipa> nothing will change by moving the wallet out of it (though we absolutely should do so for other reasons)
177 2012-04-23 17:16:52 <gmaxwell> oh for some reason I thought it was for the whole enviroment. Okay, so why is it slow then?
178 2012-04-23 17:17:10 <sipa> it must traverse the entire database file
179 2012-04-23 17:17:17 <sipa> large database file, slow lsn_reset
180 2012-04-23 17:17:37 <sipa> read the comments on the pull request
181 2012-04-23 17:19:34 <sipa> WOAH https://forums.oracle.com/forums/thread.jspa?threadID=874276&tstart=0#3489748
182 2012-04-23 17:20:01 <sipa> there seems to be a flag DB_AM_NOT_DURABLE with removes LSNs altogether
183 2012-04-23 17:20:15 <sipa> we don't care about durability of the blockchain database
184 2012-04-23 17:36:27 <gavinandresen> sipa:  DB_TXN_NOT_DURABLE  looks like a very good idea for blkindex.dat
185 2012-04-23 17:36:48 <sipa> gavinandresen: i remember we looked at it before
186 2012-04-23 17:37:04 <sipa> This means that updates of this database exhibit the ACI (atomicity, consistency, and isolation) properties, but not D (durability); that is, database integrity will be maintained, but if the application or system fails, integrity will not persist. The database file must be verified and/or restored from backup after a failure.
187 2012-04-23 17:37:38 <sipa> my question is: what do they mean by integrity?
188 2012-04-23 17:38:37 <sipa> if they mean: some committed transactions will be in the database, and some will not; that's perfect
189 2012-04-23 17:38:43 <gavinandresen> yup
190 2012-04-23 17:38:57 <sipa> if they mean: the database may be in an arbitrary state; that's terrible
191 2012-04-23 17:40:21 <gavinandresen> The question for me is:  how terrible?  If it only happens when you're writing to blkindex.dat and the drive loses power halfway through a write, that's not so terrible.
192 2012-04-23 17:41:02 <gavinandresen> ... especially if the result is an error on subsequent startup (a corrupted-but-reads-without-an-error would be worse)
193 2012-04-23 17:41:50 <sipa> interesting: i kill -9'd while downloading the block database and while using DB_TXN_NOT_DURABLE
194 2012-04-23 17:42:01 <sipa> result: not a single block was in the file
195 2012-04-23 17:42:13 <sipa> and no error either
196 2012-04-23 17:42:59 <sipa> but it's shutting down extremely quickly
197 2012-04-23 17:43:17 <sipa> even in the middle of block chain database download
198 2012-04-23 17:43:20 <gavinandresen> clean shutdown with DB_TXN_NOT_DURABLE does the right thing I assume?
199 2012-04-23 17:43:26 <sipa> of course
200 2012-04-23 17:43:58 <gavinandresen> I'd be OK with "dirty shutdown means you lose blockchain download progress"
201 2012-04-23 17:44:07 <sipa> same
202 2012-04-23 17:44:51 <gavinandresen> ... and "power outage at exactly the wrong time means you might have to delete blkindex.dat and database/* and re-download the whole chain"
203 2012-04-23 17:45:46 <sipa> ok, another kill -9, and now i get EXCEPTION: 11DbException
204 2012-04-23 17:45:48 <sipa> bitcoin in AppInit()
205 2012-04-23 17:49:05 <sipa> not sure i like that...
206 2012-04-23 17:49:53 <gavinandresen> yeah, what happened to the C in ACI....
207 2012-04-23 17:51:18 <sipa> my preference is just no lsn_reset for blkindex.dat
208 2012-04-23 17:51:56 <sipa> DN_TXN_NOT_DURABLE is certainly faster and easier, but will likely result in many more people "meh, the program just doesn't start anymore... bleh"
209 2012-04-23 17:53:11 <gavinandresen> Sigh.  We aught to do a much better job of catching errors and helping users recover
210 2012-04-23 17:59:54 <sipa> also, i wonder why we don't use DB_PRIVATE
211 2012-04-23 18:00:21 <sipa> that would prevent some interprocess locking issues like the one we had with rewriting addr.dat, i assume
212 2012-04-23 18:00:52 <gavinandresen> if I recall, we did once DB_PRIVATE once upon a time, I don't remember why that was changed
213 2012-04-23 18:01:37 <sipa> why does github have no "search code history" function? :)
214 2012-04-23 18:02:03 <jrmithdobbs> sipa: doesn't DB_PRIVATE lock it to a thread?
215 2012-04-23 18:02:20 <sipa> yes
216 2012-04-23 18:02:28 <jrmithdobbs> that's probably why
217 2012-04-23 18:02:43 <gavinandresen> commit c8ad9b83
218 2012-04-23 18:02:55 <gavinandresen> "dropped DB_PRIVATE from dbenv.open"  (satoshi)
219 2012-04-23 18:03:31 <sipa> helpful commit messages :)
220 2012-04-23 18:04:32 <jrmithdobbs> bet it was done to allow updating addr.dat from the connection threads
221 2012-04-23 18:04:47 <gavinandresen> I'm about to disappear, I'm meeting with Dries (Mr. Drupal) again this evening
222 2012-04-23 18:05:03 <jrmithdobbs> (as in, eventually causing the problem you just mentioned sipa)
223 2012-04-23 18:28:01 <Diapolo> sipa: I made some progress with my approach and was able to now write to the blkindex.dat DB
224 2012-04-23 19:05:31 <sipa> Diapolo: link?
225 2012-04-23 19:06:58 <Diapolo> sipa: https://github.com/Diapolo/bitcoin/commit/95a4056c8b683698f955f39cacdd11dc7627b12b
226 2012-04-23 19:08:26 <Diapolo> I added 2 new functions in CTxDB that read / write the file-pointer position and removed the code I had in for creating that extra file
227 2012-04-23 19:09:28 <Diapolo> seems like I didn't add db.cpp?
228 2012-04-23 19:09:29 <Diapolo> lol
229 2012-04-23 19:09:57 <sipa> yes
230 2012-04-23 19:10:24 <sipa> Diapolo: you'll want a file pointer per block file
231 2012-04-23 19:10:34 <sipa> not just a global one
232 2012-04-23 19:12:25 <Diapolo> sipa: yes currently it only works for blk0001.dat
233 2012-04-23 19:13:42 <Diapolo> but I'm happy I figured out that db stuff even if it's perhaps unoptimized
234 2012-04-23 19:13:47 <Diapolo> https://github.com/Diapolo/bitcoin/commit/37e85743d705b94932afb716173f9a7c6e22161c
235 2012-04-23 19:16:27 <Diapolo> sipa: fileBlockchain is global but it can be re-used for other files, too ... I
236 2012-04-23 19:16:47 <Diapolo> 'I'm not sure if I understood the "per block file"
237 2012-04-23 19:18:28 <gmaxwell> Diapolo: there will be a blk0002.dat once blk0001.dat becomes to large (to avoid FS maximum file size limitations)
238 2012-04-23 19:18:58 <gmaxwell> (for some reason I was thinking the threshold was at 1g, but I guess not)
239 2012-04-23 19:19:02 <makomk> Well, there should be a blk0002.dat. Has that code been tested?
240 2012-04-23 19:19:10 <sipa> makomk: nope
241 2012-04-23 19:19:18 <Diapolo> and they have then to be open simultaneously?
242 2012-04-23 19:19:24 <sipa> yes
243 2012-04-23 19:19:46 <Diapolo> alright, at least there is a little time left for me to mature my experiment ^^
244 2012-04-23 19:19:54 <makomk> I think some SolidCoin nodes got up to 16 GB or more without creating a blk0002.dat, though obviously that's a modified version of the Bitcoin code.
245 2012-04-23 19:20:08 <sipa> haha
246 2012-04-23 19:20:11 <gmaxwell> what .. were you doing to those poor solidcoin nodes?
247 2012-04-23 19:21:04 <makomk> I'm not sure any of my nodes reached that point due to insufficient Internet connectivity, but remember their block spam issues of a while ago?
248 2012-04-23 19:22:30 <Diapolo> Is there any compression going on in the blk000x.dat files currently?
249 2012-04-23 19:22:40 <sipa> Diapolo: no
250 2012-04-23 19:23:55 <gmaxwell> sipa: erp, I can't find any code to increment nFile.
251 2012-04-23 19:25:18 <gmaxwell> oh there it is.
252 2012-04-23 19:25:26 <Diapolo> gmaxwell: there is an nCurrentBlockFile++ in the main.cpp
253 2012-04-23 19:25:42 <gmaxwell> // FAT32 filesize max 4GB, fseek and ftell max 2GB, so we must stay under 2GB
254 2012-04-23 19:25:45 <gmaxwell> if (ftell(file) < 0x7F000000 - MAX_SIZE)
255 2012-04-23 19:26:28 <Diapolo> is this code tested and working so I could take a look at how you currently handle multiple files?
256 2012-04-23 19:27:12 <gmaxwell> Well, I think that codes wrong.
257 2012-04-23 19:27:31 <gmaxwell> I think it should be - MAX_BLOCK_SIZE  not MAX_SIZE, no?
258 2012-04-23 19:27:49 <gmaxwell> Diapolo: in any case, lower the limit.. it should work. I've never tested it. You tell me!
259 2012-04-23 19:29:36 <sipa> what was MAX_SIZE again?
260 2012-04-23 19:30:09 <makomk> Maximum size of serialized data, I think.
261 2012-04-23 19:30:40 <Diapolo> sipa said the files should be opened simultaneously, but is this true or are the opened / closed serially?
262 2012-04-23 19:30:49 <Diapolo> the == they
263 2012-04-23 19:31:11 <Diapolo> static const unsigned int MAX_SIZE = 0x02000000;
264 2012-04-23 19:32:01 <makomk> I think it closes and reopens the block file every time, actually.
265 2012-04-23 19:32:08 <Diapolo> MAX_SIZE is 32 MB then?
266 2012-04-23 19:32:21 <Diapolo> makom: that's what I think too
267 2012-04-23 19:32:28 <sipa> Diapolo: they're being used simultaneously; the current code always opens and closes files
268 2012-04-23 19:32:43 <sipa> Diapolo: but you changed that, no?
269 2012-04-23 19:33:03 <Diapolo> sipa: but no 2 files are open at the same time right? talking about current code to understand how it works
270 2012-04-23 19:33:11 <sipa> no
271 2012-04-23 19:33:42 <sipa> it's just a CAutoFile being opened when something needs to be read or written
272 2012-04-23 19:34:22 <Diapolo> right and this should work with a global object of type CAutoFile/Stream or a local one, too
273 2012-04-23 19:34:54 <sipa> i meant "simultaneously" in the sense that for the processing of a single transaction multiple block files may be necessary
274 2012-04-23 19:35:19 <Diapolo> sipa: got it and that what was I thought, too
275 2012-04-23 19:35:59 <sipa> but you changed it to a global, so you'd have all files open simultaneously, i guess
276 2012-04-23 19:36:10 <sipa> not sure that's necessary though
277 2012-04-23 19:36:41 <Diapolo> the global object can be fed with the file I need I just need to re-add the code to handle multiple files
278 2012-04-23 19:36:53 <sipa> ?
279 2012-04-23 19:37:13 <Diapolo> I thought it was better performance wise to not open and close it everytime durint the block-chain download.
280 2012-04-23 19:37:28 <sipa> i doubt it matters much
281 2012-04-23 19:39:01 <gmaxwell> oh I fail at reading. yea MAX_SIZE is fine.
282 2012-04-23 19:39:22 <Diapolo> with multiple files you are perhaps right with a single one the difference is perhaps messureable
283 2012-04-23 19:39:27 <gmaxwell> (I thought it was 2000 for some reason)
284 2012-04-23 19:39:57 <Diapolo> fileBlockchain.setFile() can be used to feed the global with a new file, so that should work
285 2012-04-23 19:40:15 <kish> is there a way to run database recovery so i dont have to start from scratch?
286 2012-04-23 19:41:16 <Diapolo> sipa: is that CTxDB code I use okay or can it be made any faster / better?
287 2012-04-23 19:41:44 <Diapolo> I don't want to learn bad usage.
288 2012-04-23 19:44:19 <sipa> Diapolo: if you're only keeping one open block file around in the global var, you need synchronisation code to prevent different threads wanting to access it simultaneously
289 2012-04-23 19:44:42 <sipa> i guess it will always be locked by cs_main in practice
290 2012-04-23 19:45:24 <sipa> but it's bad design to keep local information in a global variable
291 2012-04-23 19:47:29 <Diapolo> it would be easy to use a local one and I have to admit I even dislike globals
292 2012-04-23 19:51:54 <Diapolo> damn it's again to late here ^^ ... I'm off bye
293 2012-04-23 20:46:23 <zooko> I just posted a bounty for some Python hacking on a cool distributed storage project: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/20#comment:13
294 2012-04-23 21:12:00 <xixor> sup
295 2012-04-23 21:14:37 <xixor> sup
296 2012-04-23 21:14:55 <xixor> so, I am trying to understand bitcoin mining... what is the deal?
297 2012-04-23 21:15:09 <copumpkin> what have you read on the subject?
298 2012-04-23 21:16:17 <xixor> this article: http://www.weusecoins.com/mining-guide.php
299 2012-04-23 21:16:26 <xixor> this article: http://en.wikipedia.org/wiki/Bitcoin#Covert_.22mining.22
300 2012-04-23 21:16:42 <xixor> and this one: https://en.bitcoin.it/wiki/How_bitcoin_works
301 2012-04-23 21:16:53 <copumpkin> what specific questions have those left you?
302 2012-04-23 21:17:37 <xixor> I don't understand it at all.  So the bitcoin miners have to solve some crytopgraphic system to generate "new" bitcoins for the market?
303 2012-04-23 21:17:56 <xixor> and they are paid, in bitcoins, by some sort of central bank?
304 2012-04-23 21:18:05 <copumpkin> no central bank
305 2012-04-23 21:18:09 <copumpkin> just an agreed-upon protocol
306 2012-04-23 21:18:34 <copumpkin> everyone agrees that if you can prove you solved a certain kind of cryptographic problem, you can give yourself more coins
307 2012-04-23 21:18:48 <gmaxwell> Gah.  (as usual)
308 2012-04-23 21:18:57 <copumpkin> no?
309 2012-04-23 21:19:03 <xixor> how does this work in an economic sense?
310 2012-04-23 21:19:14 <xixor> doesn't this just cause inflation on the currency?
311 2012-04-23 21:19:23 <copumpkin> these questions are all answered in those articles...
312 2012-04-23 21:19:30 <copumpkin> in many different ways
313 2012-04-23 21:19:36 <gmaxwell> Mining is the process of securing bitcoin transactions against reversal by computing proofs of work needed to extent the bitcoin blockchain, a database of transactions.
314 2012-04-23 21:19:46 <Eliel> xixor: that's the point. That's the only way new bitcoins come into being. It won't be forever though. The reward halves every 4 years.
315 2012-04-23 21:20:05 <luke-jr> xixor: there's a fixed total of 21 million Bitcoins, being generated gradually
316 2012-04-23 21:20:19 <gmaxwell> Because the original bitcoins have to come from somewhere, and we don't want something centeralized like a centeral bank the mining process lets the miners produce a metered amount of new coins 'from thin air' as they add blocks.
317 2012-04-23 21:21:18 <gmaxwell> The software in all bitcoin nodes regulates the creation of new coins to follow a fixed program: 50 bitcoins per block halving every 210000 blocks, 2016 blocks per two weeks resulting in a maximum of 21 million coins ever.
318 2012-04-23 21:21:26 <xixor> so the bitcoins have a planned schedule of inflation then?
319 2012-04-23 21:21:52 <gmaxwell> So at the moment the supply is inflating by about 35%/yr, a rate that will exponentially fall over time.
320 2012-04-23 21:22:11 <Eliel> xixor: the inflation will only be around 12% by next year.
321 2012-04-23 21:22:13 <gmaxwell> The first halving should happen sometime in decemember this year.
322 2012-04-23 21:22:28 <Eliel> and will keep dropping.
323 2012-04-23 21:22:59 <xixor> I see
324 2012-04-23 21:23:12 <Eliel> xixor: although, if you look at bitcoin's value, it looks like it's deflating though.
325 2012-04-23 21:23:42 <sipa> xixor: gmaxwell raises a good point: the purpose of mining is not currency creation, it is much more essential to the system; it's tied to currency introduction though to make it economically interesting for people to help secure the chain
326 2012-04-23 21:23:42 <xixor> Eliel: yes, of course
327 2012-04-23 21:24:35 <xixor> what happens when all 21-million bitcoins are created?
328 2012-04-23 21:24:44 <gmaxwell> Well, and absent tying it to mining it's not clear how you could introduce currency in a decenteralized way otherwise.
329 2012-04-23 21:25:11 <xixor> this is extremely fascinating
330 2012-04-23 21:25:17 <Eliel> xixor: no more new bitcoins will be created.
331 2012-04-23 21:25:23 <DiabloD3> bitcoin is a rather uninteresting proof of work system
332 2012-04-23 21:25:32 <gmaxwell> xixor: well 21 million arises as the limit of that geometric series... technically it doesn't ever have to reach it.   But more pratically nothing special.
333 2012-04-23 21:25:34 <sipa> xixor: everything stays exactly the same; blocks will keep being generated at the same rate as ever, they just won't introduce new currency anymore
334 2012-04-23 21:26:01 <xixor> oh, somehow I thought blocks were bitcoins
335 2012-04-23 21:26:17 <sipa> no, blocks are lists of transactions
336 2012-04-23 21:26:25 <Eliel> xixor: the blocks are more like the entire bookkeeping of bitcoin network.
337 2012-04-23 21:26:28 <sipa> blocks also refer to a previous block, forming a chain
338 2012-04-23 21:26:31 <gmaxwell> That was basically the cause for my 'gah'.
339 2012-04-23 21:26:37 <xixor> ah, I see
340 2012-04-23 21:26:58 <xixor> well, I'd be lying if I said I understand it
341 2012-04-23 21:27:24 <Eliel> and mining exists so the whole network has a way to agree which version of bookkeeping to agree on :)
342 2012-04-23 21:28:00 <xixor> version to agree on?  hasn't the protocol already been decided?  The miners are creating the crytographic machinery to keep it honest?
343 2012-04-23 21:28:48 <sipa> is there some standard STL or Boost type to represent a-or-b? (a tagged union) ?
344 2012-04-23 21:29:35 <sipa> boost::variant, it seems
345 2012-04-23 21:29:39 <Eliel> xixor: the network rules for accepting blocks keep it honest. However, mining is used to make sure everyone has the same honest bookkeeping.
346 2012-04-23 21:30:16 <gmaxwell> xixor: the key challenge in digital currency is the "double spending" problem, without it ecash would be trivial.
347 2012-04-23 21:30:28 <sipa> xixor: you can consider mining to be voting for a particular order of transactions, by your computation power
348 2012-04-23 21:30:43 <sipa> a particular order of otherwise valid transactions, by the way
349 2012-04-23 21:30:52 <gmaxwell> the double spending problem is that if I give you some digital money whats to stop me from giving sipa the _same_ digital money, thus creating inflation.
350 2012-04-23 21:31:36 <xixor> ok
351 2012-04-23 21:31:43 <gmaxwell> The obvious way to solve double spening is to order the transactions and only consider the first one.  E.g. If I pay you digital money via a centeralized bank the bank can timestamp the payments.
352 2012-04-23 21:32:17 <gmaxwell> But thats centralized. Bitcoin solves it by having a distributed network figure out the ordering.
353 2012-04-23 21:32:58 <gmaxwell> But this is tricky to do because physical law means that spatially seperated parties will view events as happening in different orders depending on their locations.
354 2012-04-23 21:33:11 <xixor> do you have to be online to query the distributed network to make a transaction?
355 2012-04-23 21:33:34 <sipa> you want to be online when you receive a transaction, and want to be sure it's not a double spend
356 2012-04-23 21:33:40 <gmaxwell> Bitcoin solves all this by using mining as a kind of election where nodes commit to an ordering of the transactions which takes lots of computation so it can't be faked.
357 2012-04-23 21:34:19 <xixor> very interesting
358 2012-04-23 21:34:21 <gmaxwell> xixor: you can send and recieve them offline, but you can't validate them against double spending that way. If you only trade with people you trust you can defer the announcement of your transactions.
359 2012-04-23 21:34:34 <xixor> ok
360 2012-04-23 21:34:52 <xixor> won't this type of book keeping mean that every transaction is in the open?  everyone know who bought what from whom?
361 2012-04-23 21:35:11 <gmaxwell> ha!
362 2012-04-23 21:35:15 <Eliel> xixor: yes and no :)
363 2012-04-23 21:35:30 <gmaxwell> You're perceptive. Bitcoin uses pseudoanonymity to replace the privacy this system loses.
364 2012-04-23 21:35:57 <Eliel> xixor: bitcoin accounts look like random numbers. Unless you reveal your addresses as yours, no-one will know who owns them.
365 2012-04-23 21:36:04 <gmaxwell> Ideally every transaction is paid to a fresh address... and no one has any idea who owns which addresses.
366 2012-04-23 21:36:10 <luke-jr> Eliel: except blockchain.info
367 2012-04-23 21:36:14 <gmaxwell> And the addresses are random.
368 2012-04-23 21:36:16 <xixor> can you get different random numbers for each transaction then?
369 2012-04-23 21:36:30 <xixor> this is very interesting, did one of you write a phd dissertation on this or something?
370 2012-04-23 21:36:35 <Eliel> luke-jr: I don't buy that.
371 2012-04-23 21:36:40 <gmaxwell> You can. though people do reuse addresses because it's easy to do so.
372 2012-04-23 21:36:54 <gmaxwell> xixor: You may be interested in the bitcoin design paper http://bitcoin.org/bitcoin.pdf
373 2012-04-23 21:36:57 <luke-jr> Eliel: 90% of my addresses are linked by spends
374 2012-04-23 21:37:12 <xixor> gmaxwell: cool, thanks
375 2012-04-23 21:37:18 <gmaxwell> xixor: it covers the core concepts, though not the implementation details.
376 2012-04-23 21:37:21 <luke-jr> Eliel: Coin Control is a nice view of ones own lack of privacy
377 2012-04-23 21:37:29 <gmaxwell> (your exact privacy question is covered in section 10, for example)
378 2012-04-23 21:37:38 <luke-jr> xixor: #bitcoin-watch may be handy :p
379 2012-04-23 21:38:05 <xixor> well, I had heard of bitcoin mining for a while now... but I had zero idea of what it meant
380 2012-04-23 21:38:22 <gmaxwell> luke-jr: we could probably bias the selection against linking with almost no cost... I only haven't bothered looking into it yet because the coincontrol detection of linkage is a prereq. :)
381 2012-04-23 21:38:38 <Eliel> luke-jr: so, isn't anyone working on making the input choosing algorithm more intelligent so it doesn't link the addresses so much?
382 2012-04-23 21:38:50 <luke-jr> Eliel: not afaik
383 2012-04-23 21:39:12 <Eliel> not a priority for anyone I guess.
384 2012-04-23 21:39:24 <gmaxwell> Eliel: see my comment above ^
385 2012-04-23 21:39:58 <Eliel> ah
386 2012-04-23 21:40:39 <Eliel> now, what's this coin control thing?
387 2012-04-23 21:40:52 <luke-jr> Eliel: coderrr's work
388 2012-04-23 21:41:17 <luke-jr> shows you which of your addresses are linked, and lets you pick which ones you send from
389 2012-04-23 21:41:33 <xixor> well thank you for all the explanations, I will take a read of that bitcoin design paper
390 2012-04-23 21:41:34 <luke-jr> http://coderrr.wordpress.com/2011/06/30/patching-the-bitcoin-client-to-make-it-more-anonymous/
391 2012-04-23 21:41:37 <sipa> i wonder how it deals with multisig addresses
392 2012-04-23 21:41:41 <xixor> it seems like an exciting project
393 2012-04-23 21:41:53 <gmaxwell> xixor:  It is!  feel free to ask questions.
394 2012-04-23 21:43:01 <luke-jr> wtf
395 2012-04-23 21:43:03 <luke-jr> "Add a +1 to the pull request if you believe this should be added to the official client: https://github.com/bitcoin/bitcoin/pull/415"
396 2012-04-23 21:43:07 <luke-jr> that's where the spam came from