1 2012-03-29 02:26:03 <amiller> alright here's my first draft at explaining my proposed alternate proof of work scheme https://gist.github.com/2232636
  2 2012-03-29 02:29:14 <gmaxwell> amiller: You sort of undermine my interest in bothering to read the rest by misrepresneting checkpointing as a mechenism "for coping with the increasing storage burden".
  3 2012-03-29 02:29:27 <amiller> hrm
  4 2012-03-29 02:31:20 <gmaxwell> As I misunderstanding that you also require them to retain spent transactions?  If so that would completely inhibit pruning.
  5 2012-03-29 02:32:49 <amiller> i don't think it would be hard to modify this to skip over pruned txoutputs
  6 2012-03-29 02:33:10 <gmaxwell> I didn't suggest it was.
  7 2012-03-29 02:33:43 <amiller> but pruning doesn't reduce any costs if it's part of the proof of work scheme
  8 2012-03-29 02:33:52 <gmaxwell> But it's not much of a proposal if you have to keep changing it in response to weaknesses which take me 10 seconds to spot.
  9 2012-03-29 02:37:44 <gmaxwell> Your POW can't be validated by nodes that lack a complete copy of the blockchain.
 10 2012-03-29 02:39:22 <gmaxwell> This would be horrific for decentralization and independence and would prevent the existance of secure SPV nodes. In Bitcoin, all full nodes can independantly validate the protocol rules with this particular scheme you couldn't even prevent yourself from being dos attacked by someone flooding you with junk blocks.
 11 2012-03-29 02:39:54 <gmaxwell> You could fix this by chaning how the binding works. But,  I pointed this potential weakness out to you already, so I don't understand why you didn't fix it.
 12 2012-03-29 02:54:01 <amiller> ok i think i understand your comment from yesterday better now, that the cost of validation should be low enough that lots of people besides miners can validate it
 13 2012-03-29 02:54:29 <amiller> so far i have that it's exponentially cheaper to validate than it is to mine competitively.. but it still seems to require the whole block chain which is a lot
 14 2012-03-29 02:57:20 <nanotube> the other issue is that bitcoin is never going to do it, it'll have to be an alt-chain. so trying to glom it onto the existing block chain such as gmaxwells proposal is probably more likely to happen. (if it is determined that something like this should happen at all)...
 15 2012-03-29 02:57:50 <nanotube> but it is an interesting thought experiment i guess ;)
 16 2012-03-29 02:58:44 <gmaxwell> amiller: for anti-dos sake it has to be ultra-cheap to check that the party providing the block did a lot of work to produce it, but for general decenteralization it also should not require the chain to validate partially, and not require the unpruned chain to validate fully (so it doesn't increase the full validation requirements)
 17 2012-03-29 03:02:38 <amiller> ok, i'll have answers for those next time i come back
 18 2012-03-29 04:01:23 <gribble> New news from bitcoinrss: Diapolo opened pull request 1006 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1006>
 19 2012-03-29 04:21:44 <hctib> How do I set up a website to check if a payment has been sent?  Can I do this without having a client on the server?
 20 2012-03-29 10:41:15 <sipa> gavinandresen: max 5 files? that's quite impressive (i had max 12 files, when doing it manually, but checking every 2 seconds)
 21 2012-03-29 10:41:44 <gavinandresen> yup.  I think AUTO_REMOVE is the right answer.
 22 2012-03-29 10:45:01 <gavinandresen> So.... pull translations from transifex, AUTO_REMOVE... and what about the addr.dat hang?  We could just never bother erasing the old entries
 23 2012-03-29 10:45:28 <sipa> i'm not convinced that is the actual solution
 24 2012-03-29 10:45:53 <gavinandresen> Do you think it would avoid the problem?  Actual solution could wait for a 0.6.1 or 0.7
 25 2012-03-29 10:46:07 <sipa> probably
 26 2012-03-29 10:46:20 <sipa> but the actual problem is locks surviving process crashes
 27 2012-03-29 10:46:33 <gavinandresen> agreed
 28 2012-03-29 10:46:46 <sipa> and there is no reason to assume this is limited to addr.datr
 29 2012-03-29 10:46:59 <sipa> though addrman's rewrite of it exposes it
 30 2012-03-29 10:47:12 <gavinandresen> weird we've never seen it before
 31 2012-03-29 10:47:49 <gavinandresen> I suppose we're mostly write-only in our use of bdb
 32 2012-03-29 10:48:12 <gavinandresen> ... or, at least, we don't do a lot of erasing
 33 2012-03-29 11:12:06 <gribble> New news from bitcoinrss: gavinandresen opened pull request 1007 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1007>
 34 2012-03-29 11:13:18 <sipa> gavinandresen: i don't get it, i'm unable to reproduce luke's problem now
 35 2012-03-29 11:14:39 <gavinandresen> I'll see if I can...
 36 2012-03-29 11:15:04 <sipa> i wanted to try whether dbenv.set_timeout helped
 37 2012-03-29 11:16:26 <sipa> ok, never mind
 38 2012-03-29 11:19:21 <sipa> Although Berkeley DB allows an application to have multiple outstanding transactions active within a single thread of control, great care must be taken to ensure that the transactions do not block each other (for example, attempt to obtain conflicting locks on the same data). If two concurrently active transactions in the same thread of control do encounter a lock conflict, the thread of control will deadlock so that the deadlock detector cannot...
 39 2012-03-29 11:19:27 <sipa> detect the problem. In this case, there is no true deadlock, but because the transaction on which a transaction is waiting is in the same thread of control, no forward progress can be made.
 40 2012-03-29 11:23:35 <helo> at first glance "in the same thread" and "concurrently active" seem contradictory
 41 2012-03-29 11:24:26 <sipa> if you create two independent DbTxn's in a single application
 42 2012-03-29 11:24:34 <sipa> *thread
 43 2012-03-29 11:27:37 <helo> ahh... "thread of control" != "db thread"
 44 2012-03-29 11:28:24 <helo> maybe one application (the thread of control) could have multiple transactions that are being processed concurrently in separate db threads
 45 2012-03-29 11:28:40 <sipa> sure
 46 2012-03-29 11:32:00 <jeremias> https://en.bitcoin.it/wiki/URI_Scheme#Python
 47 2012-03-29 11:32:22 <jeremias> it seems that the reference implementation for python sucks
 48 2012-03-29 11:32:30 <jeremias> and otherwise, the URI scheme sucks
 49 2012-03-29 11:32:39 <jeremias> who invented this shit
 50 2012-03-29 11:32:59 <jeremias> If exponent is omitted, implementations SHOULD assume X8 for decimal numbers, and X4 for hexadecimal numbers. I.e. amount=50.00 is treated as 50 BTC, and amount=x40 is treated as 40 TBC. When specifying bitcoin base units, "X0" SHOULD be used.
 51 2012-03-29 11:33:24 <gavinandresen> yeah, that's luke-jr.   We nuked all that crap.
 52 2012-03-29 11:33:51 <gavinandresen> jeremias: see https://en.bitcoin.it/wiki/BIP_0021   for mostly up-to-date spec
 53 2012-03-29 11:33:58 <jeremias> so, when you google bitcoin uri, that shit turns out first
 54 2012-03-29 11:34:01 <jeremias> how to change that
 55 2012-03-29 11:34:07 <gavinandresen> it's a wiki....
 56 2012-03-29 11:35:20 <jeremias> well, I try to edit it so others don't waste their precious time like I did... fuck
 57 2012-03-29 11:35:27 <gavinandresen> nice, thanks
 58 2012-03-29 11:46:05 <sipa> gavinandresen: how long did that sync take, by the way?
 59 2012-03-29 11:46:19 <gavinandresen> I don't know, I was asleep when it finished.
 60 2012-03-29 11:46:40 <sipa> ah, i always run benchmarks with -logtimestamps :)
 61 2012-03-29 11:46:48 <gavinandresen> good idea
 62 2012-03-29 11:47:30 <gavinandresen> I haven't been setting up for benchmark conditions, though-- I'm downloading from a random peer, etc
 63 2012-03-29 11:48:19 <sipa> ok, because the slower you download, the more checkpoints will be created, hence more chances for log removal
 64 2012-03-29 11:48:49 <sipa> i can't do it with less than 12 log files, when loading the block chain from a local file
 65 2012-03-29 11:49:07 <gavinandresen> 12 is not bad-- 120MB
 66 2012-03-29 11:49:12 <sipa> now, i doubt anyone cares whether it's 5 or 12 log files
 67 2012-03-29 11:50:04 <gavinandresen> I just pulled translations, the only issue left is the addr.dat hang.
 68 2012-03-29 11:50:20 <gavinandresen> (assuming we're all happy with the AUTO_REMOVE pull)
 69 2012-03-29 11:52:16 <sipa> AUTO_REMOVE seems the way to go; i'll do a benchmark with it still
 70 2012-03-29 11:52:50 <sipa> by the way: subversion with bdb backend sets AUTO_REMOVE by default, unless you specify --keep-db-logs or something like that
 71 2012-03-29 11:53:20 <gavinandresen> interesting...
 72 2012-03-29 11:55:29 <sipa> i wonder how much lsn_reset plays a role
 73 2012-03-29 11:56:02 <sipa> it makes the .dat files clean for transport to a different environnement, but it also seems take some time
 74 2012-03-29 11:56:11 <sipa> so i wonder whether it's also a form of flushing
 75 2012-03-29 12:03:53 <sipa> some sort of counter for indexing into the log files; if you don't reset them, the db files can still reference the log files, and cause failure when those aren't available anymore
 76 2012-03-29 12:03:58 <sipa> even though there was a checkpoint
 77 2012-03-29 12:04:01 <sipa> or something
 78 2012-03-29 12:06:11 <sipa> doesn't sound bad to also do that occassionally
 79 2012-03-29 12:21:07 <gavinandresen> Thinking about it... we should set DB_TXN_NOT_DURABLE for blkindex.dat. And maybe addr.dat.  http://docs.oracle.com/cd/E17276_01/html/api_reference/CXX/dbset_flags.html
 80 2012-03-29 12:21:54 <gavinandresen> That avoids the entire fill-up-disk-with-log-files issue entirely.
 81 2012-03-29 12:22:42 <sipa> no integrity when the system crashes? i don't like that...
 82 2012-03-29 12:23:05 <gavinandresen> meh.  You lose some blocks and have to re-download them....
 83 2012-03-29 12:23:39 <gavinandresen> blkindex.dat will still be consistent.  blk00001.dat might end up containing duplicates
 84 2012-03-29 12:23:39 <sipa> no, that's durability
 85 2012-03-29 12:23:50 <sipa> integrity means that the file may be inconsistent
 86 2012-03-29 12:24:06 <gavinandresen> ACID : TXN_NOT_DURABLE just drops the D
 87 2012-03-29 12:24:24 <sipa> This means that updates of this database exhibit the ACI (atomicity, consistency, and isolation) properties, but not D (durability); that is, database integrity will be maintained, but if the application or system fails, integrity will not persist.
 88 2012-03-29 12:25:42 <gavinandresen> right, isolation not integrity....  (I shoulda taken a database course in college)
 89 2012-03-29 12:27:37 <sipa> gavinandresen: it seems i was running with logsize 1M instead of 10M
 90 2012-03-29 12:27:45 <helo> shouldn't it be: amountparam    = "amount=" +digit [ "." +digit ]
 91 2012-03-29 12:27:55 <sipa> gavinandresen: wait, not sure
 92 2012-03-29 12:28:07 <helo> instead of amountparam    = "amount=" *digit [ "." *digit ]
 93 2012-03-29 12:28:36 <sipa> gavinandresen: oh, no, it's you who changed the bsize to 1M
 94 2012-03-29 12:28:50 <helo> well, i guess it's not really necessary
 95 2012-03-29 12:29:27 <gavinandresen> sipa:  yes, I just dropped a zero from the 10M/100M setting...
 96 2012-03-29 12:29:42 <gavinandresen> sipa:  let me know what they should be
 97 2012-03-29 12:31:14 <helo> although amount=. would be kind of weird
 98 2012-03-29 12:32:30 <gavinandresen> helo:  we should follow JSON or some other well-known standard
 99 2012-03-29 12:33:14 <helo> ye
100 2012-03-29 12:35:33 <gavinandresen> json.org has a nice BNF definition for 'number' that doesn't allow "." as a valid number.  You can pull request BIP changes by forking https://github.com/genjix/bips
101 2012-03-29 12:37:59 <sipa> gavinandresen: benchmarking your patch now
102 2012-03-29 12:39:15 <gavinandresen> sipa: thanks.  I'm out for a bit, back in a hour or two
103 2012-03-29 12:41:54 <gavinandresen> one thought on lsn_reset:  if DB_RECOVER happens at environment-opening time, then would it ever hurt to call lsn_reset before opening a .dat? It'd either be a no-op (if clean shutdown) or might be better than a failure to open if there was a dirty shutdown and user copied the .dat files separate from the logs.....
104 2012-03-29 12:43:42 <sipa> i doubt you can do lsn_reset when the database environment is unclean
105 2012-03-29 12:58:05 <t7> whats being done about the 0 tx issue?
106 2012-03-29 13:03:24 <t7> will the getwork command return the same hash, if the blockchain head hasnt changed?
107 2012-03-29 13:06:50 <sipa> t7: no
108 2012-03-29 13:06:58 <t7> its random?
109 2012-03-29 13:07:07 <sipa> well, getwork doesn't return a hash; it returns work for you to find a hash
110 2012-03-29 13:07:18 <sipa> but the work it gives will be different every time
111 2012-03-29 13:07:25 <t7> cool
112 2012-03-29 13:07:38 <sipa> otherwise, how would you cope with doing more than 2^32 hashes per 10 minutes?
113 2012-03-29 13:07:48 <sipa> you'd be doing the same work over and over
114 2012-03-29 13:08:07 <t7> whats the network rate atm?
115 2012-03-29 13:08:23 <sipa> ;;bc,nethash
116 2012-03-29 13:08:24 <gribble> 11398.633639951546
117 2012-03-29 13:08:28 <sipa> (in Ghash/s)
118 2012-03-29 13:08:32 <t7> wow
119 2012-03-29 13:08:57 <t7> 11,398 trillion a second?
120 2012-03-29 13:09:01 <sipa> yes
121 2012-03-29 13:10:37 <t7> i wonder how many computers at work run opencl
122 2012-03-29 13:11:00 <sipa> i think most is done by dedicated miners
123 2012-03-29 13:28:57 <t7> sipa do i hash the getwork 'data' as ab=n ascii string?
124 2012-03-29 13:29:08 <sipa> no
125 2012-03-29 13:31:20 <sipa> t7: it's a weird hex encoding
126 2012-03-29 13:41:57 <t7> ah theres a python miner in the git repo :)
127 2012-03-29 13:42:05 <t7> i can translate that
128 2012-03-29 13:44:18 <t7> how many calculations are needed to calculate a hash on avg? is that number known?
129 2012-03-29 13:46:37 <sipa> difficulty * 2^48 / 65535
130 2012-03-29 13:46:49 <sipa> so currently:
131 2012-03-29 13:46:57 <sipa> ;;calc [bc,diff] * 2^48 / 65535
132 2012-03-29 13:47:09 <sipa> ;;calc [bc,diff] * 2**48 / 65535
133 2012-03-29 13:47:18 <sipa> hello, gribble?
134 2012-03-29 13:47:22 <gribble> Error: Something in there wasn't a valid number.
135 2012-03-29 13:47:24 <t7> no i mean x86 instructions
136 2012-03-29 13:47:27 <gribble> 6986100606938533
137 2012-03-29 13:47:44 <t7> then i can work out the speed of the bitcoin network :]
138 2012-03-29 13:47:48 <gribble> Math is hard. Lets go shopping!
139 2012-03-29 13:47:54 <sipa> no, you don't know what hardware it consists of
140 2012-03-29 13:48:02 <sipa> and it sure as hell isn't mostly x86
141 2012-03-29 13:48:37 <sipa> it's around 1000 x86 instructions, i assume
142 2012-03-29 13:48:43 <sipa> roughly
143 2012-03-29 13:49:22 <t7> must be more!?
144 2012-03-29 13:50:19 <gmaxwell> sipa: well the simd ones have throughput that high, but it's actually more instructions (it just does four at once)
145 2012-03-29 13:51:14 <helo> t7: it's probably more sensical to look at "work=...." in debug.log, and take that as the number of hashes the bitcoin network has done
146 2012-03-29 13:52:19 <helo> t7: for example, currently it is enough to have brute-forced all 11-character passwords (that use double-sha256)
147 2012-03-29 13:52:48 <t7> wow
148 2012-03-29 13:52:50 <helo> at least that is my favorite bitcoin stat :)
149 2012-03-29 13:53:41 <helo> if each character can be any of 72 different values (lower, upper, numbers, 10 symbols)
150 2012-03-29 13:54:15 <t7> home time
151 2012-03-29 13:54:52 <sipa> and within 4 months it will have done enough work for all 12-character alphabetical passwords
152 2012-03-29 13:55:01 <sipa> at the current difficulty
153 2012-03-29 13:56:07 <gmaxwell> The password bruteforcing comparisons are a bit unfortunate, since they play into the common misconception that bitcoin is actually bruteforcing passwords.
154 2012-03-29 13:57:02 <sipa> let's try to compare with the number of computations seti@home has done in its history
155 2012-03-29 13:57:13 <sipa> though that's floating-point operations
156 2012-03-29 14:06:54 <sipa> gavinandresen: https://github.com/sipa/bitcoin/commit/c8b1bd8657cfc1b04dec2bd0461b3ace0b2a16e2
157 2012-03-29 14:07:06 <sipa> benchmarked at 31 minutes, with max 12 10M logfiles
158 2012-03-29 14:08:42 <sipa> the lower max_lg_bsize doesn't seem to matter
159 2012-03-29 14:09:40 <gmaxwell> was "max 12 10M logfiles" with the lower max_lg_bsize, and you're saying it didn't hurt performance?
160 2012-03-29 14:10:18 <sipa> gmaxwell: yes
161 2012-03-29 14:10:49 <sipa> at least, 1M or 10M max_lg_bsize doesn't matter (i guess OS write cache completely takes over where bdb stops)
162 2012-03-29 14:11:40 <sipa> gmaxwell: that's not the same as lg_max (the size of the log files)
163 2012-03-29 14:12:03 <gmaxwell> ah. indeed, I was confusing the two.
164 2012-03-29 14:13:14 <sipa> gavinandresen: (it depends on your pullreq 1007)
165 2012-03-29 14:17:15 <gmaxwell> It looks like there is going to be pretty awesome chain splitting on the first. I wish we already had 0.5.4/0.6.0 out and widely deployed.
166 2012-03-29 14:22:05 <copumpkin> oh?
167 2012-03-29 14:22:38 <gmaxwell> copumpkin: why are you saying oh?
168 2012-03-29 14:22:45 <copumpkin> oh about the chain splitting
169 2012-03-29 14:22:51 <copumpkin> just wanted more information :)
170 2012-03-29 14:23:14 <gmaxwell> There is still a goodly amount of hash power without p2sh enforcement. They'll be willing to mine invalid transactions and their blocks will get orphaned.
171 2012-03-29 14:23:45 <gmaxwell> (I think it's safe to assume that someone will happily supply them with eligible transactions)
172 2012-03-29 14:24:14 <lh77> whats chainsplitting?
173 2012-03-29 14:25:07 <gmaxwell> when mined blocks don't form a single linear chain due to blocks being orphaned.
174 2012-03-29 14:25:13 <gmaxwell> It happens all the time in small amounts.
175 2012-03-29 14:25:18 <helo> lh77: when one group of bitcoin nodes disagree with another about what is valid, they go off on their own forks of events
176 2012-03-29 14:25:54 <lh77> oh
177 2012-03-29 14:25:57 <lh77> that doesnt sound good
178 2012-03-29 14:26:22 <gmaxwell> helo: these forks aren't permanent. the split nodes will graft back on at the next block.
179 2012-03-29 14:26:50 <gmaxwell> (or, rather, when the majority chain overtakes them)
180 2012-03-29 14:26:52 <helo> as long as the majority of the miners are updated and know which is correct, everything will be ok
181 2012-03-29 14:27:03 <gmaxwell> Right, thats not in question.
182 2012-03-29 14:27:32 <copumpkin> a temporary blocktree
183 2012-03-29 14:27:56 <gmaxwell> But basically a fifth of blocks starting the 1st are going to be dead, which may be confusing to users who aren't running software that knows to ignore them.
184 2012-03-29 15:22:39 <gavinandresen> sipa: I'm not sure counting transactions is worth it... seems to me checkpointing every 5 minutes aught to be fine.  Also: can we remove the special-case code for addr.dat now that CAddrMan isn't constantly writing to addr.dat?
185 2012-03-29 15:43:47 <gribble> New news from bitcoinrss: gavinandresen opened pull request 1008 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1008>
186 2012-03-29 16:04:13 <gribble> New news from bitcoinrss: gavinandresen opened pull request 1009 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1009>
187 2012-03-29 16:04:58 <luke-jr> &
188 2012-03-29 16:05:20 <luke-jr> gavinandresen: I think it's safe to assume you can bump the rc without a pullreq :p
189 2012-03-29 16:05:52 <gavinandresen> but it's not safe to assume I'll actually REMEMBER to do it without  a pullreq
190 2012-03-29 16:06:06 <luke-jr> heh
191 2012-03-29 16:07:31 <gavinandresen> sipa: can I get an ack on https://github.com/bitcoin/bitcoin/pull/1008  ?  And does the blkindex.dat txn_checkpoint code need changing for good performance, or can we get away with it as-is for 0.6 ?
192 2012-03-29 16:09:19 <gavinandresen> lazy sipa out eating dinner or having fun or something.....  we mustn't have any of that!
193 2012-03-29 16:10:25 <kinlo> :p
194 2012-03-29 16:10:58 <t7> which company runs bitcoin? google or microsoft?
195 2012-03-29 16:11:28 <t7> :)
196 2012-03-29 16:12:03 <gavinandresen> didn't you hear? bitcoin is secretly run by the NSA, they're building a big mining operation in Utah
197 2012-03-29 16:12:24 <t7> NSA are the 15% of blocks without tx
198 2012-03-29 16:12:52 <Diablo-D3> gavinandresen: its not in utah though
199 2012-03-29 16:12:54 <Diablo-D3> er
200 2012-03-29 16:12:58 <Diablo-D3> goddamnit.
201 2012-03-29 16:13:27 <kinlo> Stick to the coverstory people :)
202 2012-03-29 16:13:34 <gavinandresen> Hmm?  http://www.foxnews.com/politics/2012/03/28/nsa-dismisses-claims-utah-data-center-watches-average-americans/
203 2012-03-29 16:16:20 <gmaxwell> those hangs on erase are scarry. :(
204 2012-03-29 16:16:31 <gavinandresen> agreed
205 2012-03-29 16:18:26 <gavinandresen> If we had a dedicated QA team I'd ask them to test more failure scenarios; turning off power, corrupting a disk sector or two...
206 2012-03-29 16:18:53 <t7> il do it for x btc
207 2012-03-29 16:19:04 <gavinandresen> But, then again, I already know we do a lousy job of backing up the critical data, and we're probably better off making that better
208 2012-03-29 16:19:04 <t7> where x > 0
209 2012-03-29 16:21:34 <gmaxwell> gavinandresen: the bdb stuff is brittle enough that it's almost not worth testing. E.g. deleting the db log files will (sometimes) make your wallet unreadable, even if you haven't generated a new key in months.
210 2012-03-29 16:23:31 <gavinandresen> That's exactly the type of thing a good QA team would keep track of (keep the datadir so we can reproduce, etc)
211 2012-03-29 16:24:02 <BlueMatt> gavinandresen: werent you the one saying you didnt want to move off bdb for wallets?
212 2012-03-29 16:24:32 <gavinandresen> Moving off too often means "lets swap this known set of problems for another unknown set of problems."
213 2012-03-29 16:24:44 <Diablo-D3> known unknowns
214 2012-03-29 16:24:52 <Diablo-D3> or unknown unknowns?
215 2012-03-29 16:25:32 <zooko> I wrestled bdb into submission one time.
216 2012-03-29 16:25:44 <zooko> The layers and layers of error recovery that I implemented were ridiculous.
217 2012-03-29 16:25:46 <gavinandresen> what was the secret?
218 2012-03-29 16:25:46 <gmaxwell> In that particular example it's not even an unexpected behavior. We have writes to the wallet which are unrelated to keys, and it's a known/documented/expected behavior that after an unclean shutdown deleting the logs will make BDB unable to open the database.
219 2012-03-29 16:26:25 <gmaxwell> It doesn't help that a lot of people (and shutdown scripts) kill -9 bitcoin because it often takes forever to shut down.
220 2012-03-29 16:26:57 <gavinandresen> I actually have a private branch that tries to make shutdown much faster....
221 2012-03-29 16:27:00 <BlueMatt> gavinandresen: fair enough, still moving onto something that we can more easily control with fewer unknown...
222 2012-03-29 16:27:08 <gavinandresen> (switching all our threads to use boost::thread)
223 2012-03-29 16:27:12 <zooko> gavinandresen: the only reason I considered it even vaguely okay
224 2012-03-29 16:27:51 <t7> Diablo-D3: you cant know about an unknown unknown
225 2012-03-29 16:27:58 <zooko> was that the loss of the db was not fatal -- I could, at a cost of grinding the hard drive for a few hours -- reconstruct the db if it died.
226 2012-03-29 16:28:07 <Diablo-D3> t7: you can know they exist, however'
227 2012-03-29 16:28:07 <gavinandresen> zooko: open source code?  Can I look at it?
228 2012-03-29 16:28:09 <zooko> Therefore, I just had all sorts of double-checks to detect corruption of the db and blow it away.
229 2012-03-29 16:28:17 <t7> unless we are talking about known unknown unknowns
230 2012-03-29 16:28:20 <zooko> gavinandresen: ancient and crufty -- I'm embarassed -- but yes.
231 2012-03-29 16:28:21 <zooko> Let's see...
232 2012-03-29 16:28:26 <zooko> Actually that wasn't open sourced damn.
233 2012-03-29 16:28:30 <Diablo-D3> it only makes them known unknowns when you know what it is you dont know
234 2012-03-29 16:28:38 <t7> Diablo-D3: how can you know they exist
235 2012-03-29 16:28:41 <Diablo-D3> but you know there are unknown unknowns out there
236 2012-03-29 16:28:46 <t7> if you did they are known unknowns
237 2012-03-29 16:28:50 <Diablo-D3> you simply dont know everything, that is a known unknown
238 2012-03-29 16:28:58 <t7> you dont know that
239 2012-03-29 16:29:12 <Diablo-D3> well, _I_ dont know that because I really do know everything
240 2012-03-29 16:29:12 <gavinandresen> It IS darn tempting to switch to sqlite, and maybe a flat-file for private keys.....
241 2012-03-29 16:29:21 <Diablo-D3> gavinandresen: DO IT
242 2012-03-29 16:29:22 <Diablo-D3> IT
243 2012-03-29 16:29:23 <Diablo-D3> DO IT
244 2012-03-29 16:29:28 <Diablo-D3> DOOOOOooOooOooOooOoooOO EEEEEEEEEeeeeEeeEEeeeeeEEeeeET
245 2012-03-29 16:29:41 <t7> is sqlite in-memory?
246 2012-03-29 16:29:48 <zooko> gavinandresen: I'm sorry, I don't have it.
247 2012-03-29 16:29:54 <gavinandresen> zooko: no problem
248 2012-03-29 16:29:56 <zooko> Most of it would probably not be relevant to you.
249 2012-03-29 16:30:04 <zooko> A lot of it had to do with the Python wrappers to bsddb.
250 2012-03-29 16:30:08 <zooko> I recommend sqlite.
251 2012-03-29 16:30:21 <zooko> Do you know that they allegedly have branch-level 100% code coverage of their tests?
252 2012-03-29 16:30:32 <kinlo> I think a seperate file with an extreamly simple key/value store mapping pub/privkeys would guarantee that you dont loose coins ever
253 2012-03-29 16:30:51 <kinlo> If you'd make that file append only....
254 2012-03-29 16:30:57 <Diablo-D3> sqlite can be in memory
255 2012-03-29 16:31:17 <t7> and disk backed?
256 2012-03-29 16:31:22 <kinlo> That could be a plain text file too
257 2012-03-29 16:31:23 <gavinandresen> kinlo: then you're all set until your hard disk catches on fire.
258 2012-03-29 16:31:26 <t7> i mean does the whole db have to live in memory?
259 2012-03-29 16:31:34 <zooko> We implemented a simple flat file for something in Tahoe-LAFS, and later I adjusted some of the arithmetic calculating where the boundary of one part and the next part lay, and thus introduced one of the few security bugs that we've shipped in Tahoe-LAFS.
260 2012-03-29 16:31:35 <gavinandresen> or a couple sectors in the middle go bad or or or
261 2012-03-29 16:31:53 <zooko> I later concluded that I should avoid doing arithmetic which is necessary to keep more-sensitive stuff apart from less-sensitive stuff, and should instead consider using sqlite if it fits.
262 2012-03-29 16:31:56 <kinlo> gavinandresen: You need to backup it ofcourse but it wont go corrupt
263 2012-03-29 16:31:57 <Diablo-D3> fuck, tahoe
264 2012-03-29 16:32:22 <gavinandresen> .... and D3 goes back on my ignore list....
265 2012-03-29 16:32:39 <Diablo-D3> gavinandresen: what the fuck dude
266 2012-03-29 16:32:41 <kinlo> If it's one key per line you can always recover it with a text editor
267 2012-03-29 16:33:29 <zooko> So what is the proximate issue which is forcing y'all to think about local storage design?
268 2012-03-29 16:33:42 <Diablo-D3> zooko: it needs to work.
269 2012-03-29 16:33:42 <zooko> Is it the block chain, including log of all transactions, getting too big?
270 2012-03-29 16:34:00 <Diablo-D3> and libbdb needs to stop being badly designed
271 2012-03-29 16:34:21 <Diablo-D3> and my HN karma is now above 1200. Fuck yeah.
272 2012-03-29 16:34:40 <gavinandresen> Unclean shutdowns leading to failures at startup are the big issue
273 2012-03-29 16:34:47 <zooko> Yeah, nowadays I would consider using bsddb only if it were a cache -- not a canonical storage the loss of which would be unrecoverable -- and if it fit the "key - value" paradigm really well.
274 2012-03-29 16:34:48 <gmaxwell> zooko: no, it's libdb being unreliable and scarry, especially related to wallet data it's also the differences between our requirements for wallet data and the chain data.
275 2012-03-29 16:34:53 <zooko> And even then I think I'd try sqlite first. :-)
276 2012-03-29 16:34:54 <gavinandresen> The smaller is annoying bdb version incompatibility
277 2012-03-29 16:35:06 <zooko> gavinandresen: ah, yeah, that's a trouble.
278 2012-03-29 16:35:15 <Diablo-D3> and thus, my comment about bad design
279 2012-03-29 16:35:31 <zooko> I think I had some horrible hacks of doing my own two-phase commit *on top* of bsddb's by cp'ing the file before opening it and later relinking it or something. Ha
280 2012-03-29 16:35:42 <zooko> Fortunately time has relieved my mind of having to remember the gory details.
281 2012-03-29 16:36:07 <Diablo-D3> zooko: except bdb uses transaction integrity files
282 2012-03-29 16:36:11 <Diablo-D3> outside of the db
283 2012-03-29 16:36:18 <zooko> Yes, I remember that now that you mention it.
284 2012-03-29 16:36:48 <zooko> If you change the local storage format in the mainline client, will you have to provide a function to transfer all the data over to the new format?
285 2012-03-29 16:37:03 <zooko> I guess that makes sense. And extract the keys?
286 2012-03-29 16:37:12 <gavinandresen> yup
287 2012-03-29 16:38:16 <gavinandresen> wouldn't (probably) need to decrypt the keys, so that shouldn't be hard to do in an external little utility
288 2012-03-29 16:39:01 <gavinandresen> But it's not clear to me that it is a big enough problem to be first or second or fifth on the priority list.
289 2012-03-29 16:39:16 <zooko> Okay, so what needs to be stored: keys, and the entire history of blockchain including transactions? And, if this is a miner, floating transactions?
290 2012-03-29 16:39:48 <jgarzik> block chain.  wallet.  addresses on P2P network.
291 2012-03-29 16:39:49 <gavinandresen> We already store the blockchain data in a flat-file.  blkindex.dat stores indices into that flat-file.
292 2012-03-29 16:39:51 <jgarzik> and indices
293 2012-03-29 16:40:10 <kinlo> Transactions...
294 2012-03-29 16:40:14 <gavinandresen> (actually, blockchain data is a series of at-most-2GB flat files, we just have one so far)
295 2012-03-29 16:40:41 <jgarzik> memory pool transactions are not stored
296 2012-03-29 16:40:50 <jgarzik> only completed blocks (w/ transactions)
297 2012-03-29 16:41:52 <kinlo> But client-made transactions that are not yet confirmed on the network are stored too right?
298 2012-03-29 16:42:00 <gavinandresen> just in memory
299 2012-03-29 16:42:10 <gavinandresen> unless they're your own
300 2012-03-29 16:42:25 <kinlo> Your own yes
301 2012-03-29 16:42:43 <gavinandresen> Yes, transactions involving you are stored in wallet.dat
302 2012-03-29 16:43:34 <kinlo> I've seen problems with that, transactions with not enough fees never getting confirmed and locking people out not being able to spend those coins
303 2012-03-29 16:44:10 <gavinandresen> The moral to that story is: don't hack the fee policy if you don't know what you're doing...
304 2012-03-29 16:44:22 <kinlo> True for now
305 2012-03-29 16:44:37 <gavinandresen> (now jgarzik is going to say how he proposed fixing that a long time ago.... :)
306 2012-03-29 16:44:59 <kinlo> Perhaps not if the free block part is difficult to get into
307 2012-03-29 16:45:34 <gmaxwell> It'll always be true there will be some kinds of transactions which will always go nowhere, if you modify your client you allow them you're going to have issues end of story. The space of safe changes may grow, but changing things requires some clue.
308 2012-03-29 16:45:55 <kinlo> True
309 2012-03-29 16:46:32 <gavinandresen> But there probably aught to be a failsafe "failed transaction" mode, where if your txn doesn't get into a block in X days the client assumes it will never get in, and un-spends the inputs.
310 2012-03-29 16:47:06 <gmaxwell> Indeed, but bumping into that will still make you have a bad day. :)
311 2012-03-29 16:47:30 <gavinandresen> yup, which is why that isn't first or fifth or eleventh on the priority list either
312 2012-03-29 16:48:38 <zooko> So "wallet" is a set of private keys?
313 2012-03-29 16:49:25 <kinlo> much more, own transactions, accounts...
314 2012-03-29 16:49:29 <gmaxwell> zooko: more than that private keys, some user configured settings from the GUI, and your own sent/recieved transactions, address to account mappings, transactions comments.
315 2012-03-29 16:49:32 <zooko> What are "accounts"?
316 2012-03-29 16:49:40 <zooko> Ah.
317 2012-03-29 16:49:44 <zooko> address to account mappings.
318 2012-03-29 16:49:46 <zooko> Huh.
319 2012-03-29 16:49:49 <gmaxwell> zooko: locally significant bookkeeping.
320 2012-03-29 16:49:53 <kinlo> yes
321 2012-03-29 16:50:14 <zooko> So, I hope I'm not saying a bunch of obvious and/or obviously-wrong things, but if I were starting on this I would first think about putting more-sensitive stuff in a separate file from less-sensitive stuff.
322 2012-03-29 16:50:15 <luke-jr> gmaxwell: GUI settings are outside the wallet now
323 2012-03-29 16:50:17 <gmaxwell> E.g. you can assign addresses to be MoneyFromZooko and then the client will keep track of how much btc is assigned to that account.
324 2012-03-29 16:50:26 <zooko> So I would probably start by putting private keys and only private keys in one sqlite db file.
325 2012-03-29 16:50:49 <luke-jr> zooko: you can encrypt private keys
326 2012-03-29 16:50:50 <zooko> Then, maybe keep using the block chain flat file -- unless there's reason to think it is all by itself a problem -- and replace the bsddb index to it with a sqlite index to it.
327 2012-03-29 16:50:51 <kinlo> i would make that plaintext
328 2012-03-29 16:50:55 <zooko> Then benchmark. :-)
329 2012-03-29 16:51:09 <luke-jr> zooko: sqlite isn't known for performance&
330 2012-03-29 16:51:22 <zooko> luke-jr: I'm willing to bet it is good enough.
331 2012-03-29 16:51:31 <luke-jr> it's also SQL
332 2012-03-29 16:51:35 <luke-jr> there's no reason to use SQL
333 2012-03-29 16:51:41 <kinlo> for wallet data it should be
334 2012-03-29 16:51:47 <gmaxwell> I can't even express how horrible I think that all sounds.
335 2012-03-29 16:51:47 <zooko> In fact, I'll bet 10?? that it is within 2% of the bsddb version. :-)
336 2012-03-29 16:52:02 <zooko> gmaxwell: try! I'd be interested to hear it.
337 2012-03-29 16:52:03 <gmaxwell> Switching one enormous chunk of unauditable code for another is not a win.
338 2012-03-29 16:52:12 <zooko> That's a good argument.
339 2012-03-29 16:52:14 <luke-jr> zooko: um, 2% improvement wouldn't be worth it, and 2% worse is worse
340 2012-03-29 16:52:45 <zooko> gmaxwell: I myself conservatively stuck with bsddb far too long, then went through a phase of "I'll do my own flat files thank you very much", and just now I'm finally coming around to using sqlite for at least some things.
341 2012-03-29 16:53:07 <gmaxwell> Especially for key management which is basically append only, fixed size records, entirely in memory during use.
342 2012-03-29 16:53:10 <zooko> luke-jr: that's also a good argument. If there were something a lot like sqlite but without SQL, I would consider that, too.
343 2012-03-29 16:53:26 <zooko> gmaxwell: yeah, that one could definitely be a flat file.
344 2012-03-29 16:53:35 <gmaxwell> ::nods::
345 2012-03-29 16:53:41 <zooko> On the other hand, it could also work as a sqlite file. The fact that it imposes such light requirements makes it work fine with either.
346 2012-03-29 16:54:16 <kinlo> the keys file is the single most important file, the rest one can just redownload
347 2012-03-29 16:55:02 <kinlo> so i'd go for a simple file format that is less prone to corruption
348 2012-03-29 16:55:07 <gmaxwell> zooko: but then we have issues like the wallet encryption bug bdb makes it basically impossible to reliably delete data in its databases, doesn't disclosed this fact and solving it be deleting the whole database was problematic because closing and reopening was making it crash.
349 2012-03-29 16:55:22 <zooko> gmaxwell: yeah, good point.
350 2012-03-29 16:55:38 <zooko> So it was cleartext keys stored in the bsddb from the previous release that was the source of the leak?
351 2012-03-29 16:55:56 <gmaxwell> Yes.
352 2012-03-29 16:56:04 <zooko> kinlo: what I've recently learned is that my own flat files, while less prone to corruption than bsddb files, are not immune to corruption. :-)
353 2012-03-29 16:56:20 <zooko> I.e., I wrote a bug in the handling of mine.
354 2012-03-29 16:56:39 <gmaxwell> as far as the rest of the usage thats less concerning except bugs/unexpected behavior there leak into the distributed algorithim.
355 2012-03-29 16:56:39 <kinlo> zooko: that's why i'd go for append only
356 2012-03-29 16:56:50 <gmaxwell> kinlo: thats still not a magical cureall.
357 2012-03-29 16:57:00 <kinlo> ofcourse not
358 2012-03-29 16:57:12 <kinlo> but a good start
359 2012-03-29 16:57:43 <zooko> kinlo: yeah, it sounds like the private keys file would be even simpler than the flat files that I managed to screw up.
360 2012-03-29 16:57:58 <zooko> Another option is to rely on the filesystem -- a different file for each private key.
361 2012-03-29 16:58:26 <zooko> Might cause a problem with performance if people start using more than 10,000 private keys in one process.
362 2012-03-29 16:58:50 <gmaxwell> zooko: encourages footgunning. E.g. doing things like removing keys you've spent from creating negative balances and all kinds of other untested and potentially untestable cornercases.
363 2012-03-29 16:58:50 <kinlo> zooko: afaik they are loaded into ram every startup
364 2012-03-29 16:58:52 <gavinandresen> That starts to get to the "bitcoin core is too many things to too many people" problem
365 2012-03-29 16:59:04 <gavinandresen> (merchants and miners and end-users have very different requirements)
366 2012-03-29 16:59:25 <kinlo> sequential reading is fastest anyway if you need all
367 2012-03-29 17:00:09 <zooko> IMHO bsddb is a bad choice for storing priv keys, flat files and sqlite are both good choices, and keeping priv keys separate from everything of lesser sensitivity is wise.
368 2012-03-29 17:01:38 <gavinandresen> zooko:  agreed.  Although I suspect we'll be moving away from lots of private keys and towards "deterministic wallets"
369 2012-03-29 17:02:35 <zooko> gavinandresen: ah, that's interesting.
370 2012-03-29 17:03:00 <gavinandresen> ... mostly because backing up lots of private keys becomes an issue pretty quickly.
371 2012-03-29 17:03:07 <zooko> If you're not going to be handing out individual private keys to someone else, then you might as well just remember a single secret and deterministically generate any private key you need from that.
372 2012-03-29 17:03:32 <gmaxwell> Where is sipa's spec?
373 2012-03-29 17:03:52 <sipa> gmaxwell: of?
374 2012-03-29 17:03:53 <gmaxwell> zooko: we're looking at being a bit smarter than that, in fact.
375 2012-03-29 17:04:10 <gmaxwell> sipa: the lastest determinstic wallet stuff, you should show it to zooko.
376 2012-03-29 17:04:10 <kinlo> deterministic keys
377 2012-03-29 17:04:19 <zooko> Intriguing!
378 2012-03-29 17:04:21 <zooko> I love that stuff.
379 2012-03-29 17:04:41 <sipa> https://gist.github.com/1799467
380 2012-03-29 17:04:43 <sipa> it's WIP
381 2012-03-29 17:04:54 <gmaxwell> zooko: What we'll probably be doing allows the public keys to be generated independantly of the private keys. E.g. so you could give a third party (or an untrusted web server) the ability to generate public keys fr you.
382 2012-03-29 17:05:01 <gmaxwell> s/fr/for/
383 2012-03-29 17:05:15 <gribble> New news from bitcoinrss: sipa opened pull request 1010 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1010>
384 2012-03-29 17:05:47 <zooko> gmaxwell: what? That's crazy talk.
385 2012-03-29 17:05:59 <zooko> Do you know about Identity-Based Encryption? Crazy talk. :-)
386 2012-03-29 17:06:19 <gmaxwell> I know about identity based encryption. But this is simpler. :)
387 2012-03-29 17:07:22 <sipa> note that sections 0, 1 and 2 are already "rewritten", while sections 3, 2, 3 aren't yet :)
388 2012-03-29 17:07:25 <zooko> Well I look forward to learning about your insane impossible perpetual motion machine.
389 2012-03-29 17:07:48 <gavinandresen> We've got a car that runs on nothing but seawater, too!
390 2012-03-29 17:07:59 <sipa> zooko: it's a bit inconsistent, but all is explained there
391 2012-03-29 17:08:16 <zooko> The whole idea of Bitcoin is something that I struggled to invent for about 12 years, and had started to think it was impossible.
392 2012-03-29 17:08:28 <zooko> So, it's a wonderful feeling of possibilities opening up, to see it actually work.
393 2012-03-29 17:08:42 <zooko> I wish I could remember exactly what Mark S. Miller said along those lines.
394 2012-03-29 17:09:41 <gavinandresen> sipa gmaxwell jgarzik : I'm going to pull 1008 1009 and 1010, tag a rc6, and start gitian-building.....
395 2012-03-29 17:09:55 <gmaxwell> gavinandresen: ACK
396 2012-03-29 17:10:52 <sipa> gmaxwell: ACK
397 2012-03-29 17:10:54 <sipa> eh, gavinandresen
398 2012-03-29 17:13:22 <gavinandresen> * [new tag]         v0.6.0rc6 -> v0.6.0rc6
399 2012-03-29 17:14:54 <gavinandresen> zooko: You might also be interested in 2-party escrow design I'm working on:  https://gist.github.com/830ca16758fb9ad496d7
400 2012-03-29 17:16:58 <zooko> gavinandresen: sounds great!
401 2012-03-29 17:17:09 <zooko> gavinandresen: I'm really excited about the "ripped certificates" that you and I talked about.
402 2012-03-29 17:17:23 <zooko> If I understand correctly, Silk Road is offering an escrow service for fee.
403 2012-03-29 17:17:52 <zooko> Not to imply any approval or other moral judgment about their product, but I imagine that ripped certificates could replace *any* escrow service with a no-trusted-third-party, cheap, reliable substitute.
404 2012-03-29 17:18:03 <helo> is lockTime currently implemented?
405 2012-03-29 17:18:15 <gavinandresen> yes, it is
406 2012-03-29 17:20:55 <sipa> helo: transaction replacement isn't, but nLockTime is fully enabled
407 2012-03-29 17:21:00 <sipa> gavinandresen: building
408 2012-03-29 17:21:47 <luke-jr> not sure about fully. isn't there something limiting it for backward compat?
409 2012-03-29 17:22:14 <sipa> luke-jr: below a certain number it's interpreted as a block number, above a certain number it's interpreted as a timestamp
410 2012-03-29 17:22:28 <sipa> that's all i know about it
411 2012-03-29 17:22:58 <helo> two-party escrow is pretty neat stuff... thanks for the explanation writeup gavin
412 2012-03-29 17:23:05 <luke-jr> // To help v0.1.5 clients who would see it as a negative number
413 2012-03-29 17:23:06 <gavinandresen> I'm not sure miners will accept transactions with locktime's < now into their memory pools or not, or if they'll get relayed, but that's not necessary
414 2012-03-29 17:23:06 <luke-jr> if ((int64)nLockTime > std::numeric_limits<int>::max())
415 2012-03-29 17:23:07 <luke-jr> return error("AcceptToMemoryPool() : not accepting nLockTime beyond 2038 yet");
416 2012-03-29 17:23:44 <luke-jr> hmm, I wonder if that even works considering the int64 cast will make it signed
417 2012-03-29 17:24:12 <sipa> hey, didn't we already break compatibility with pre-v0.2.10 clients?
418 2012-03-29 17:24:23 <luke-jr> sipa: :D
419 2012-03-29 17:24:43 <gavinandresen> cool, more code we can remove.
420 2012-03-29 17:28:19 <gavinandresen> hmmm... RCC: Error in 'src/qt/bitcoin.qrc': Cannot find file 'locale/bitcoin_he.qm'
421 2012-03-29 17:28:54 <gavinandresen> ah, right, need to touch bitcoin-qt.pro to get the Makefile rebuilt....
422 2012-03-29 17:36:56 <gavinandresen> mac build uploaded to sourceforge, halfway done with the linux gitian build....
423 2012-03-29 17:39:53 <user> gavinandresen: i was reading your Two-Party Escrow. Is not possible implement some situation where the two parts need to fund bitcoin?
424 2012-03-29 17:42:37 <gavinandresen> user: you mean have both the buyer and seller commit coins, like is proposed here: https://bitcointalk.org/index.php?topic=33615.msg421527#msg421527
425 2012-03-29 17:42:58 <user> yes. will read
426 2012-03-29 17:43:22 <zooko> gavinandresen: do you know about Dominant Assurance Contracts?
427 2012-03-29 17:44:49 <gavinandresen> zooko:  no, but there's a nice short description at http://p2pfoundation.net/Dominant_Assurance_Contract
428 2012-03-29 17:45:32 <zooko> Heh heh. p2pfoundation.net are freaks.
429 2012-03-29 17:46:23 <zooko> gavinandresen: it is an idea that has great potential, has never been tried, and good escrow/ripped-certs in Bitcoin might make it very easy to try it using Bitcoin.
430 2012-03-29 17:46:36 <zooko> Without that sort of tech, it can still be tried using Bitcoin by relying on a trusted escrow agent.
431 2012-03-29 17:47:27 <copumpkin> do any forum admins hang out in here, other than Maged and theymos? I wanted to ask for an account whitelist
432 2012-03-29 17:47:47 <Blitzboom> gavinandresen = admin
433 2012-03-29 17:49:21 <gavinandresen> zooko:  I'm a big fan of crowdfunding, especially now it is going to become more legal.
434 2012-03-29 17:49:58 <gavinandresen> zooko: I wonder if dominance assurance contracts would run into legal trouble if people used it as a form of gambling/investing....
435 2012-03-29 17:50:29 <gavinandresen> (I'm thinking of people gambling that projects will NOT get funded so they'd get more back than they put in)
436 2012-03-29 17:54:51 <jgarzik> gavinandresen: ACK
437 2012-03-29 17:56:55 <gmaxwell> I think gambling usually only runs into legal trouble when it victimizes the poor and unwary ... we seem to be perfectly happy with securities markets, parts of which are pretty much pure gambling though I suppose they are somewhat heavily regulated.
438 2012-03-29 17:58:45 <kinlo> gmaxwell: it's also too difficult to start with to be attractive for the weaker minds
439 2012-03-29 17:58:51 <kinlo> iek
440 2012-03-29 18:00:23 <BTC_Bear> Does it victimize the poor when they WIN $500,000,000 Million ?
441 2012-03-29 18:01:03 <BTC_Bear> btw: >70% of winners are quick picks.
442 2012-03-29 18:10:18 <gavinandresen> sipa: just pushed my rc6 gitian.sigs
443 2012-03-29 18:13:25 <zooko> gavinandresen: I'm not an expert on the relevant laws, but one important difference between dominant assurance contracts and "gambling" is whether there is an explicit element of chance -- i.e. random lotto numbers, die rolls, and whether there is an element of skill.
444 2012-03-29 18:13:40 <zooko> From that perspective, dominant assurance contracts don't look much like gambling.
445 2012-03-29 18:13:54 <zooko> However, I don't have confidence that actual laws, much less regulations, much less police, interpret things the way I do. :-(
446 2012-03-29 18:14:06 <gavinandresen> zooko: agreed.
447 2012-03-29 18:14:16 <zooko> Because the same argument tells you that Idea Futures are also not gambling. :-/
448 2012-03-29 18:14:19 <gavinandresen> zooko:  I think we need more prediction markets, too....
449 2012-03-29 18:14:32 <zooko> Exactly!
450 2012-03-29 18:14:45 <zooko> And those have definitely been chilled out of business inthe USA by the gambling regs. :-(
451 2012-03-29 18:15:05 <gavinandresen> Yup.  Happily the Internet isn't a US-only thing
452 2012-03-29 18:15:20 <gavinandresen> ... and I think things will change here over time.
453 2012-03-29 18:16:08 <gmaxwell> (Idea-futures have basically the same gambling problem that securities markets have they're one part gambling, one part actually economically productive work, and it's impossible to distinguish the part but securities markets have the advantage of being well established and supported by powerful entities, ironically ones that make a lot of money from the gambling component)
454 2012-03-29 18:16:44 <zooko> gmaxwell: good point.
455 2012-03-29 18:17:28 <Diablo-D3> yes, but its still gambling
456 2012-03-29 18:17:31 <Diablo-D3> its a boolean state
457 2012-03-29 18:17:38 <Diablo-D3> if its even 0.1% gambling, its still gambling
458 2012-03-29 18:17:40 <gmaxwell> zooko: I was really hoping to make some coin on the mispricing of skein on bets of bitcoin, but it seems that people with access to bitcoin are less irrationally exuberant for skein than the general public!
459 2012-03-29 18:18:07 <Diablo-D3> gmaxwell: well, its even better
460 2012-03-29 18:18:14 <Diablo-D3> everyone in bitcoin land can use the predictinator
461 2012-03-29 18:18:48 <gmaxwell> Diablo-D3: you ought to expand your snakeoil business to also predict bets of bitcoin outcomes.
462 2012-03-29 18:19:08 <Diablo-D3> I make no money off predictinator.
463 2012-03-29 18:21:13 <gmaxwell> Yet.
464 2012-03-29 18:21:56 <iz> a bitcoin futures market for sports teams :b
465 2012-03-29 18:23:31 <gribble> New news from bitcoinrss: davout opened issue 1011 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/issues/1011>
466 2012-03-29 18:24:33 <Diablo-D3> gmaxwell: well, the problem is
467 2012-03-29 18:24:36 <Diablo-D3> its very accurate.
468 2012-03-29 18:29:39 <Blitzboom> can you also make 60d and 90d predictions?
469 2012-03-29 18:29:59 <Diablo-D3> no.
470 2012-03-29 18:30:04 <Diablo-D3> you're lucky it works at 30.
471 2012-03-29 18:31:13 <Blitzboom> i havent been watching it & does it change its future projection (same 30d)?
472 2012-03-29 18:31:24 <Blitzboom> based on daily changes
473 2012-03-29 18:31:48 <Diablo-D3> yes
474 2012-03-29 18:31:50 <Diablo-D3> once a day
475 2012-03-29 18:54:12 <gribble> New news from bitcoinrss: dooglus opened issue 1012 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/issues/1012>
476 2012-03-29 19:15:36 <helo> http://www.rdmag.com/News/Feeds/2012/03/general-sciences-bitcoin-currency-system-offers-negative-incentive/
477 2012-03-29 19:16:07 <helo> rather, use http://www.news.cornell.edu/stories/March12/Bitcoin.html
478 2012-03-29 19:17:51 <luke-jr> https://www.okpay.com/en/company/news/bitcoin-okpay.html
479 2012-03-29 19:17:52 <andytoshi> helo, that article makes no sense
480 2012-03-29 19:18:01 <andytoshi> as i recall, neither did the original study..
481 2012-03-29 19:18:49 <gmaxwell> helo: thats just repeating the redballoons paper which like a lot of academic work was a beautiful snowflake in its own right, and yet totally inapplicable in reality.
482 2012-03-29 19:18:52 <gavinandresen> the study makes sense-- there IS an incentive problem with relaying transactions.  I don't like their proposed solution, though, and it may never be a problem in practice.
483 2012-03-29 19:19:17 <gmaxwell> (inapplicable in the "the soviets used a pencil" sort of sense)
484 2012-03-29 19:20:03 <andytoshi> in principle, there is an incentive problem, but you'd need a -lot- of people to stop relaying messages before it would be effective
485 2012-03-29 19:20:17 <andytoshi> so an attacker would need to put effort into modifying his node to be an asshole
486 2012-03-29 19:20:25 <andytoshi> but probably wouldn't get any gain from it
487 2012-03-29 19:20:36 <gmaxwell> The miners will greedily not share transactions problem, should it ever be a problem, is trivially solved by "give your transactions to many miners directly, or via a simple clearing house (miners would pay to access)"
488 2012-03-29 19:20:49 <gavinandresen> ... and it wouldn't be hard to detect that they're being an asshole and drop their connection....
489 2012-03-29 19:21:22 <gmaxwell> The red balloons paper proposes a system which would enormously increase the cost or relaying and processing transactions as well the cost of maintaining the blockchain.
490 2012-03-29 19:21:24 <helo> aren't most full nodes not miners? i.e. no incentive to not forward transactions
491 2012-03-29 19:21:38 <andytoshi> right now, yes
492 2012-03-29 19:21:54 <andytoshi> but that's not really sustainable - even now, the blockchain takes hours for many people to download
493 2012-03-29 19:22:05 <helo> there's not really much of an incentive to run a full node, aside from helping the network...
494 2012-03-29 19:22:06 <gmaxwell> andytoshi: I disaggree.
495 2012-03-29 19:22:49 <helo> i bet there are thousands of people who would like to run a full node for that purpose alone though
496 2012-03-29 19:22:52 <gmaxwell> It's perfectly sustainable so long as we don't increase the maximum block size (much).  Perhaps not the almost-every-user-is-a-full-node, but at least full nodes being common and drastically outnumbering miners.
497 2012-03-29 19:23:25 <luke-jr> gmaxwell: but we need to increase the max block size eventually
498 2012-03-29 19:23:47 <gmaxwell> luke-jr: I don't see why you take that as axiomatic.
499 2012-03-29 19:24:13 <andytoshi> having said all that, i actually like their "solution"
500 2012-03-29 19:24:18 <gmaxwell> (there are degrees of increase too e.g. a 10x increase would probably not totally bust decenteralization)
501 2012-03-29 19:24:21 <andytoshi> it would be nice to incentivize all parts of running the network
502 2012-03-29 19:24:26 <andytoshi> not just verification
503 2012-03-29 19:24:29 <gmaxwell> andytoshi: If you like it you haven't thought it through.
504 2012-03-29 19:24:41 <gmaxwell> andytoshi: not your fault, because they don't spell it out clearly.
505 2012-03-29 19:24:57 <andytoshi> true, i actually hadn't heard of it until just now
506 2012-03-29 19:25:41 <gmaxwell> andytoshi: go think about what the details of the implementation need to be in order to prevent the final miner from just unwrapping the transaction and mining all the fees to themselves.
507 2012-03-29 19:25:49 <helo> maybe that summary is overlooking something, but just saying "we put a limit on the chain length to be sure nobody can sybil higher rewards"
508 2012-03-29 19:25:56 <helo> seems insufficient
509 2012-03-29 19:26:52 <gmaxwell> helo: yea, no, it's totally insufficient I talked to them. They know this. What they're describing can be implemented but the cost is horiffic. You basically increase transaction processing and storage costs N fold, where N is the number of hops in the chain.
510 2012-03-29 19:27:31 <gavinandresen> sipa: builds done?
511 2012-03-29 19:28:22 <gmaxwell> (actually worse, it's N*M fold processing where N is the number of hops, and M is how many connections each hop has. :( )
512 2012-03-29 19:28:25 <helo> so the appropriate response is: "Have fun implementing that system on your own chain!"
513 2012-03-29 19:30:30 <gmaxwell> (the implementation needs to be roughly "you have the sender include the key of the party he relays to in his signature, and the relaying parties add the key they relay to and sign (M times). eventually this chain ends up in the blockchain and everyone can validate the right parties were credited")
514 2012-03-29 19:31:44 <helo> yuck
515 2012-03-29 19:33:55 <gmaxwell> and even once you do that, their economic incentives argument only applies to tree structured relaying networks (not at all like bitcoin, in reality you're free to connect to whoever you want). I assume if I thought about it I'd conclude that your best strategy is to connect directly to known miners and maxsibyl-1 every time on your connections to them.
516 2012-03-29 19:34:55 <luke-jr> gavinandresen: is there a reason you're not merging the stupid s/URL/URI/ fix (#987)? it'll be more painful for 0.7 since 0.6 will have used the wrong name
517 2012-03-29 19:35:26 <gavinandresen> luke-jr: just trying to minimize code changes to critical fixes, so we actually get 0.6 out the door.
518 2012-03-29 19:35:52 <gavinandresen> Changing "little things" like URL->URI can break things like...oh, translations... in unexpected ways
519 2012-03-29 19:36:06 <luke-jr> it's not user-visible, but it's used for IPC filename
520 2012-03-29 19:36:47 <gavinandresen> IPC for the URI-opening?  That doesn't really work right now, anyway
521 2012-03-29 19:37:04 <gavinandresen> (I suppose drag and drop does, but does anybody actually know to do that?)
522 2012-03-29 19:37:28 <luke-jr> gavinandresen: oh, true enough
523 2012-03-29 19:37:32 <luke-jr> drag & drop doesn't use IPC anyway
524 2012-03-29 19:37:50 <luke-jr> guess that makes it painless for 0.7 then
525 2012-03-29 19:42:24 <helo> i get the impression that they assume evenly distributed hashing power across all bitcoin nodes
526 2012-03-29 19:47:19 <Cryo> seeing this on osx jhfs+ https://github.com/bitcoin/bitcoin/issues/776
527 2012-03-29 19:51:14 <gavinandresen> Cryo: can you try https://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.6.0/test/   and see if it is any better?
528 2012-03-29 19:52:17 <Cryo> after the defrag finishes, sure.
529 2012-03-29 21:11:10 <sipa> gavinandresen: sorry, was out for a while; linux builds done already, now doing windows builds
530 2012-03-29 21:17:19 <sipa> wow, the entire bitcoin git history of v0.6.0rc5 is only 5.4 MB
531 2012-03-29 21:20:38 <gmaxwell> sipa: you can get it under 5 if you do a git repack -a -d -f -F --window=1024 --depth=256
532 2012-03-29 21:23:33 <luke-jr> lol
533 2012-03-29 21:26:47 <sipa> strange, repacking the repository itself seems to influence the size of exported packs as well
534 2012-03-29 21:31:09 <sipa> gavinandresen: sigs pushed for 0.6.0rc6; they match
535 2012-03-29 21:51:16 <sipa> http://bitcoin.sipa.be/builds/v0.6.0rc6
536 2012-03-29 23:20:26 <gribble> New news from bitcoinrss: nanotube opened pull request 29 on bitcoin/bitcoin.org <https://github.com/bitcoin/bitcoin.org/pull/29> || nanotube opened pull request 28 on bitcoin/bitcoin.org <https://github.com/bitcoin/bitcoin.org/pull/28>
537 2012-03-29 23:29:14 <gavinandresen> sipa: thanks, I put binaries up on sourceforge a while ago anticipating the sigs would match
538 2012-03-29 23:55:26 <sipa> gavinandresen: just realized that txn_checkpoint actually takes as argument the max number of log kbytes written, so the approximation with counting transactions is not ideal (though better than doing it via the number of blocks)
539 2012-03-29 23:55:58 <gavinandresen> sipa: good thing 0.6 won't be our last release ever, then.
540 2012-03-29 23:56:14 <gavinandresen> (i can live with not ideal for now)