1 2013-01-25 00:00:30 <CodeShark> I'd like to have bitcoind run as a service in the background that UI processes can connect to as needed
  2 2013-01-25 00:01:17 <CodeShark> I envision at least three different types of UI apps: wallets, network monitoring tools, and database query tools
  3 2013-01-25 00:03:12 <gavinandresen> man bitcoin transactions are fast, I'm always surprised when I send a test transaction from bitcoind on my machine to my blockchain.info wallet; poke return and get a "beep, new transaction" right away???.
  4 2013-01-25 00:04:06 <CodeShark> what's the average hop number?
  5 2013-01-25 00:04:36 <HM> this reminds me, i need to look in to the networking code of bitcoind
  6 2013-01-25 00:05:00 <HM> how are messages routed? is it just flood with a hop count?
  7 2013-01-25 00:05:21 <gavinandresen> CodeShark: good question.  If I took a networking class in college I'd probably know the approximate answer for a randomly connected mesh network....
  8 2013-01-25 00:05:31 <gavinandresen> HM: it is just flood.  No hop count.
  9 2013-01-25 00:06:10 <HM> So a timeout?
 10 2013-01-25 00:06:15 <HM> :\\
 11 2013-01-25 00:08:00 <gavinandresen> no, you tell all your peers when you have new data.  If they already know about the data, then they ignore the message.  If the don't, then they ask you for the data, then flood the "there is new data" message to their peers
 12 2013-01-25 00:08:13 <gavinandresen> (who either ignore it or request the data??? etc)
 13 2013-01-25 00:09:10 <HM> makes sense
 14 2013-01-25 00:09:23 <gavinandresen> the new data ('inv') messages are tiny, so receiving them repeatedly isn't a big deal.
 15 2013-01-25 00:10:26 <gavinandresen> And I may have the details wrong, the networking code isn't my strong suit
 16 2013-01-25 00:10:40 <HM> right, and the chance of a tree topology to connections where all the leaves come back to you is slim
 17 2013-01-25 00:11:29 <gavinandresen> yes, random-ness for the win
 18 2013-01-25 00:11:44 <HM> not to you rather, but to one poor SOAB
 19 2013-01-25 00:15:18 <HM> the network bootstrapping looks gnarly
 20 2013-01-25 00:16:59 <etotheipi_> CodeShark: leaving it open is easy... I can have it specified in the options
 21 2013-01-25 00:18:20 <etotheipi_> so if the user has never installed Bitcoin-Qt, do I just bundle bitcoind binary, and run it in the default datadir?  also create a random bitcoin.conf username/pass?  rpc port should be standard so that trying to run installed bitcoin-qt fails
 22 2013-01-25 00:19:51 <CodeShark> I usually just create a random password and block external access to RPC
 23 2013-01-25 00:20:50 <CodeShark> and if I'm really paranoid, I create a special user just for the bitcoind instance and only give that user permissions to the datadir
 24 2013-01-25 00:21:08 <etotheipi_> oh, is there a way to specify a specific wallet.dat?  I want to make sure my bitcoind does not use their wallet.day
 25 2013-01-25 00:21:22 <CodeShark> specify a specific :)
 26 2013-01-25 00:21:43 <CodeShark> no, unfortunately the datadir is the datadir...for now
 27 2013-01-25 00:21:57 <CodeShark> but you can change the datadir in your own custom build
 28 2013-01-25 00:22:34 <etotheipi_> CodeShark: yes, but I want to avoid having blk files duplicated
 29 2013-01-25 00:24:17 <CodeShark> yeah, unfortunately the wallet stuff is mixed in with the blk stuff in the data dir right now
 30 2013-01-25 00:24:22 <CodeShark> it would be nice to decouple the two
 31 2013-01-25 00:24:39 <CodeShark> it would be entirely possible to do so
 32 2013-01-25 00:24:44 <CodeShark> they aren't even using the same db env
 33 2013-01-25 00:25:44 <CodeShark> the multiwallet version of bitcoin-qt I made can load arbitrary wallet files anywhere on the file system
 34 2013-01-25 00:50:35 <etotheipi_> CodeShark: does it need the wallet before it loads?  does it need to do a rescan?  how long does the rescan take?
 35 2013-01-25 01:17:19 <CodeShark> etotheipi_: "does it need the wallet before it loads?" not sure what you mean. "does it need to do a rescan?" it does if you want to know about transactions that occured since the last time you loaded that particular wallet. "how long dies the rescan take?" it only needs to rescan since the last block the wallet remembers, so not that long.
 36 2013-01-25 01:19:25 <etotheipi_> ugh, the "rescan from last block" scares me
 37 2013-01-25 01:19:53 <etotheipi_> i.e. wallet is created from offline computer whose clock was never synchrnoized
 38 2013-01-25 01:20:13 <etotheipi_> wallet is recovered after HDD failure and reports zero balance
 39 2013-01-25 01:20:39 <etotheipi_> or only partial balance, and maybe they don't notice...
 40 2013-01-25 01:21:21 <CodeShark> in principle it would be possible to only scan the utxo
 41 2013-01-25 01:21:35 <CodeShark> if all you're interested in is balance
 42 2013-01-25 01:22:23 <CodeShark> well, perhaps not...
 43 2013-01-25 01:22:30 <CodeShark> you need all unspent transactions - not just txouts
 44 2013-01-25 01:22:46 <CodeShark> wait, no...you also need the transactions that you've spent :p
 45 2013-01-25 01:23:19 <etotheipi_> CodeShark: yeah, this is why I really want to see the Ultimate compression/trust-free lite nodes
 46 2013-01-25 01:23:27 <etotheipi_> it totally changes the CONOPs for using Bitcoin, period
 47 2013-01-25 01:23:30 <etotheipi_> never need to rescan
 48 2013-01-25 01:23:38 <etotheipi_> no resource requirements for regular users
 49 2013-01-25 01:24:19 <CodeShark> the rescan isn't too terrible if you're not concerned with doing signature validation, though
 50 2013-01-25 01:24:22 <etotheipi_> but until then... I'm always rescanning because it sounds like a mess to risk losing coins like that
 51 2013-01-25 01:24:48 <etotheipi_> well presumably, even bitcoin-qt doesn't need to do signature validation on a sweep/import rescan
 52 2013-01-25 01:24:51 <CodeShark> you use a sliding checkpoint (say, 10 blocks ago)
 53 2013-01-25 01:24:58 <etotheipi_> since it's already verified the blockchain
 54 2013-01-25 01:25:37 <CodeShark> yeah
 55 2013-01-25 01:30:27 <CodeShark> actually, I take that back about knowing transactions you spent - if all you care about is balance (you don't care about history) then unspent outputs is enough
 56 2013-01-25 01:31:18 <CodeShark> in practice, users won't really find the block chain to give them much useful history
 57 2013-01-25 01:31:41 <CodeShark> they'll need to store account names and transaction history themselves
 58 2013-01-25 01:31:45 <CodeShark> and back it up
 59 2013-01-25 01:34:23 <CodeShark> about the only clues that the block chain would provide are amounts and timestamp
 60 2013-01-25 01:35:06 <CodeShark> in practice, they won't remember to whom they gave the receiving address 1GtrAiHQLoMFV2WTLJkETsvcw4YeUj326B
 61 2013-01-25 01:35:49 <gmaxwell> gah
 62 2013-01-25 01:35:54 <gmaxwell> virgins debating sex
 63 2013-01-25 01:36:12 <gmaxwell> The rescan required to update the walt has ABSOLUTELY NOTHING TO DO WITH SIGNATURE VALIDATION
 64 2013-01-25 01:36:25 <CodeShark> that's what we both agreed upon
 65 2013-01-25 01:36:37 <gmaxwell> In the wallet it's by height??? and scans a little before the recorded height. So there is no risk of clockwhatever.
 66 2013-01-25 01:37:02 <CodeShark> but we're not just talking about bitcoind
 67 2013-01-25 01:37:03 <gmaxwell> A scan of the whole chain (not just the recent blocks) is fairly fast??? it takes a couple minutes.
 68 2013-01-25 01:37:30 <gmaxwell> Once the optional indexes are merged then those could be used if available.
 69 2013-01-25 01:39:41 <CodeShark> the rescan isn't a serious issue for the multiwallet bitcoin-qt stuff as of yet...at most it takes a few seconds to load a wallet
 70 2013-01-25 01:40:30 <CodeShark> even without the extra index stuff
 71 2013-01-25 01:40:52 <etotheipi_> gmaxwell: at least in my case, most of the wallets have never seen the blockchain
 72 2013-01-25 01:41:09 <CodeShark> etotheipi_ was referring to a difference scenario, though
 73 2013-01-25 01:41:20 <etotheipi_> and since rescanning the full chain takes 1-2 min, I just do it anyway... but I know it's not sustainable
 74 2013-01-25 01:42:48 <CodeShark> you only really need to do that the first time - or you could give the user the option to either not rescan or give a height
 75 2013-01-25 01:43:15 <etotheipi_> eventually...
 76 2013-01-25 01:43:25 <etotheipi_> right now it's so fast it's not worth bothering the user about it
 77 2013-01-25 01:43:59 <etotheipi_> but I do know I'll need to address it eventually... I was hoping by just convincing the Bitcoin world to adopt ultimate blockchain compression and then no one deals with it ever again :)
 78 2013-01-25 01:44:11 <gmaxwell> etotheipi_: For your use case I'd expect you to require the bitcoind to have the optional indexes enabled.
 79 2013-01-25 01:44:53 <etotheipi_> but gmaxwell rightfully points out that's a big burden for miners... but I still think we should push for it anyway
 80 2013-01-25 01:45:25 <gmaxwell> You 'ultimate blockchain compression' doesn't solve this??? you wouldn't get the history... and if you don't want the history things could already be obtained fast from the utxo set.
 81 2013-01-25 01:45:27 <etotheipi_> gmaxwell: well for me, I'm only using bitcoind as a regular peer...
 82 2013-01-25 01:45:41 <gmaxwell> etotheipi_: ah, well I mean assuming you used it over rpc.
 83 2013-01-25 01:45:58 <etotheipi_> gmaxwell: yes, it only gets the balance
 84 2013-01-25 01:46:06 <etotheipi_> but the utxo set is indexed by txid, not address
 85 2013-01-25 01:46:13 <etotheipi_> so you still have to scan the whole thing
 86 2013-01-25 01:46:24 <etotheipi_> so lite nodes get no benefit unless peers are willing to scan the whole thing for you
 87 2013-01-25 01:47:07 <gmaxwell> etotheipi_: I'm not referring to lite nodes??? indeed thats different. But nothing prevents a node for keeping whatever indexes it wants for its own usage, without burdening the network.
 88 2013-01-25 01:49:28 <etotheipi_> meh, well then you get centralization of the network around the few super-nodes that are willing to supply address indexes to all the lite nodes
 89 2013-01-25 01:49:41 <gmaxwell> As far as burdening miners go.... well it starts by ~doubling the storage and working set required... and makes a bunch of theoretically O(1)ish operations O(log n)ish.  I think it's hard to even talk about it being worth it without some difficult assumptions about the future uxto size.
 90 2013-01-25 01:50:38 <etotheipi_> gmaxwell: we've been through this before... I was just kind of recapping for CodeShark ... I don't have time to debate this again
 91 2013-01-25 01:50:41 <gmaxwell> etotheipi_: the price of index centeralization is a DOS attack on lite nodes though.. SPV security alone means all they could do is hide parts of the balance... and if the resonses are signed its easy to prove when someone lied.
 92 2013-01-25 01:51:37 <gmaxwell> oohkay.
 93 2013-01-25 01:51:41 <etotheipi_> I just think it's worth an awful lot to have a network that is usable by default in a fully decentralized fashion
 94 2013-01-25 01:51:58 <gmaxwell> I am giving you a dirty look.
 95 2013-01-25 01:52:03 <CodeShark> lol
 96 2013-01-25 01:52:11 <etotheipi_> haha
 97 2013-01-25 01:52:24 <etotheipi_> 
ACTION runs back to his quadruple vim session

 98 2013-01-25 01:52:43 <gmaxwell> I think it's worth an awful lot to have a decenteralized network at all??? rather than having a centeralized one because an additional mandatory index increased update costs by as much as 50x.
 99 2013-01-25 01:53:04 <etotheipi_> gmaxwell: if it was 50x, I would agree with you
100 2013-01-25 01:53:30 <gmaxwell> Thats log2(21e14)/1. It's an insanse example but its on the order of the upper bound possible.
101 2013-01-25 01:53:34 <etotheipi_> but I guess I haven't plotted out worst cases yet
102 2013-01-25 01:53:36 <etotheipi_> I'm too buy
103 2013-01-25 01:53:42 <etotheipi_> *busy
104 2013-01-25 01:54:17 <etotheipi_> gmaxwell: you are not oging to have more addresses than utxos
105 2013-01-25 01:54:26 <gmaxwell> Realistically O(1) isn't quite the right case... making atomic group updates to non-tree datastrutures without writing the whole thing is hard.
106 2013-01-25 01:54:40 <gmaxwell> etotheipi_: yes, thats assuming there is 1 address per 1 utxo per 1 satoshi.
107 2013-01-25 01:55:06 <etotheipi_> gmaxwell: where does the 50x come from, then?
108 2013-01-25 01:55:15 <CodeShark> don't forget it's also possible to send 0 satoshis :p
109 2013-01-25 01:55:21 <gmaxwell> And in that case being forced to use a tree (for a efficient merkelized data structure) over a hash table lookup is a factor of 50.
110 2013-01-25 01:55:30 <jaakkos> I was thinking, transaction scripts could be used to implement a potentially powerful password cracking service. the script implements a hash function that is being attacked, requiring the input to be the plaintext that's hashed into the target hash. to claim the transaction's value, the plain password needs to be recovered and published.
111 2013-01-25 01:55:50 <gmaxwell> jaakkos: https://en.bitcoin.it/wiki/User:Gmaxwell/why_hash_locked
112 2013-01-25 01:56:15 <gmaxwell> CodeShark: yea, my evil roadmap has removing that on it.
113 2013-01-25 01:56:26 <gmaxwell> (or rather making it impossible to create zero value outputs)
114 2013-01-25 01:56:59 <gmaxwell> Dunno if that'll ever happen, but I think it should. It would be premature to do it now.
115 2013-01-25 01:57:18 <CodeShark> it should have been done from the beginning, IMHO
116 2013-01-25 01:57:23 <etotheipi_> gmaxwell: I still don't see it... PATRICIA trees are O(1) for query, insert and delete
117 2013-01-25 01:57:27 <etotheipi_> it's more a question of constant factors
118 2013-01-25 01:57:37 <gmaxwell> CodeShark: it was an oversight. The refernce code will never create one.
119 2013-01-25 01:57:54 <gmaxwell> etotheipi_: they are not O(1) I/O for query. you have to visit the intermediate nodes.
120 2013-01-25 01:57:56 <etotheipi_> I agree there may be a large constant factor on the patricia ops
121 2013-01-25 01:58:14 <gmaxwell> Well okay, you're just hiding it in the constant factors then.
122 2013-01-25 01:58:30 <etotheipi_> gmaxwell: if you use a 256-way trie, you do exactly 20 hops every time no matter how many nodes there are
123 2013-01-25 01:58:45 <etotheipi_> the fact that patricia trees are compressed is a bonus... even though it looks like you're doing more work with more nodes
124 2013-01-25 01:58:55 <etotheipi_> it's bounded by O(1)
125 2013-01-25 01:59:41 <gmaxwell> Everything is bounded by O(1)??? we have one universe. Everything we do can be completely during its life.   Step. {run universe} done. :P
126 2013-01-25 01:59:54 <gmaxwell> etotheipi_: in any case, thats my concern.
127 2013-01-25 02:00:12 <etotheipi_> gmaxwell: my point is you wouldn't be making that statement if I was using a trie
128 2013-01-25 02:00:19 <etotheipi_> because there's no question a trie is O(1)
129 2013-01-25 02:00:25 <etotheipi_> and it would be on average 6 hops
130 2013-01-25 02:00:33 <etotheipi_> to get from root to node
131 2013-01-25 02:00:34 <CodeShark> gmaxwell, does lim{n->oo} exist in that universe?
132 2013-01-25 02:01:09 <etotheipi_> the maximum is probably 10 (if two addresses have the same first 10 bytes)
133 2013-01-25 02:01:33 <etotheipi_> (err 6 avg, probably 10 max... for the compressed trie)
134 2013-01-25 02:01:39 <gmaxwell> etotheipi_: and perhaps you've forgotten??? I proposed a committed utxo query a long time before you??? its not that I don't like the idea. I'm just concerned that it may not quite be viable.
135 2013-01-25 02:01:55 <etotheipi_> gmaxwell: I know you have proposed such ideas
136 2013-01-25 02:02:11 <etotheipi_> I just think you're overstating the resource requirements for it
137 2013-01-25 02:02:16 <etotheipi_> I don't disagree it's a burden
138 2013-01-25 02:02:23 <jaakkos> gmaxwell: indeed that is a major problem with such service ;) thanks for your interesting wiki article!
139 2013-01-25 02:02:45 <etotheipi_> but I think it's worth consideration, given the optimality it provides all other users of the network
140 2013-01-25 02:02:59 <gmaxwell> jaakkos: we don't actually have hash functions that are useful for password cracking, though as I show??? with some deep crypto magic, we don't have to.
141 2013-01-25 02:03:23 <jaakkos> gmaxwell: perhaps you could implement them in the script.
142 2013-01-25 02:03:34 <gmaxwell> jaakkos: Totally undesirable and unneeded.
143 2013-01-25 02:03:34 <jaakkos> gmaxwell: you could unroll loops unless it's something like pbkdf2
144 2013-01-25 02:04:13 <gmaxwell> Bitcoin is a currency not a password cracking service??? many bitcoin users would be _opposed_ to it being used for that, but besides, increasing the cost of script validation would harm the network.
145 2013-01-25 02:04:55 <gmaxwell> But as I pointed out, existing script is at least theoretically sufficient to bind any computer proof to a transaction.
146 2013-01-25 02:06:03 <gmaxwell> etotheipi_: absolutely. It's certantly worth consideration??? I think at this point it needs implementation, not consideration.
147 2013-01-25 02:06:39 <etotheipi_> gmaxwell: fair enough
148 2013-01-25 02:06:48 <etotheipi_> I thought you had pretty much just written it off
149 2013-01-25 02:06:48 <gmaxwell> There are a bunch of questions like "how does lookup not become linear complexity??? and totally unbalance the hash tree??? when many txn are sent to one address" which are probably easily answered with an implementation.
150 2013-01-25 02:07:37 <etotheipi_> gmaxwell: I totally want to make an implementation, I just want to make sure you understand the properties of what I'm proposing... you're welcome to still not like it after that
151 2013-01-25 02:07:38 <etotheipi_> :)
152 2013-01-25 02:07:51 <etotheipi_> how does it get unbalanced?
153 2013-01-25 02:08:02 <etotheipi_> you can't have an unbalanced trie-structure
154 2013-01-25 02:08:11 <etotheipi_> or rather, you can't have anything that would cause greater than 20 hops
155 2013-01-25 02:08:12 <gmaxwell> etotheipi_: what happens when you have 1000 million outputs assigned to the same key?
156 2013-01-25 02:08:16 <jaakkos> gmaxwell: note that i wasn't proposing any modifications to the language, though (except perhaps enabling some of the disabled functions). but of course the validation would be very expensive when it gets complicated.
157 2013-01-25 02:08:46 <etotheipi_> gmaxwell: those outputs are stored in the same O(1)-bounded data structure
158 2013-01-25 02:08:52 <gmaxwell> jaakkos: we don't have looping for a reason, likewise the small maximum size. :P
159 2013-01-25 02:09:38 <etotheipi_> it still feels like you are taking a one-time 2x increase in HDD consumption...
160 2013-01-25 02:09:40 <petertodd> jakkos: that's the problem with your idea: password algorithms are designed to be difficult to evaluate, and it's unacceptable for any tx operation to take any length of time
161 2013-01-25 02:09:55 <asoltys> hi, i'm running a relay-only node on a vps and i just ran out of disk space.  is there anything i can do yet to prune or compress the blockchain? any documentation you can point me to?
162 2013-01-25 02:10:02 <gmaxwell> asoltys: No.
163 2013-01-25 02:10:10 <asoltys> gmaxwell: ok, thanks. coming soon though right?
164 2013-01-25 02:10:14 <gmaxwell> asoltys: No.
165 2013-01-25 02:10:16 <asoltys> doh
166 2013-01-25 02:10:39 <gmaxwell> (0.8.x does use a little less space, but it won't buy you much time on its own)
167 2013-01-25 02:11:57 <gmaxwell> etotheipi_: if I thought it were only the 2x increase... Bt I don't see how you're 20 hop bounded if an address can have a trillion payments assigned to it. Think about updating the hash root when adding one more. Also, of course, when you query that node you get an instant dos attack... but thats an implementation more than a datastructure issue.
168 2013-01-25 02:12:59 <etotheipi_> gmaxwell... if you were using raw tries... it would be 20 hops to get to any address leaf, and then 32 hops to get the whichever utxo you're looking for in the subtree
169 2013-01-25 02:13:17 <etotheipi_> but you're right, it's not 2x, because I didn't realize you were using hashtables for the tx-indexed UTXO set
170 2013-01-25 02:13:34 <gmaxwell> We're not currently. We _could_.
171 2013-01-25 02:13:42 <etotheipi_> err.. yeah
172 2013-01-25 02:13:48 <etotheipi_> I know you're not, but I didn't know that's what you were comparing against
173 2013-01-25 02:14:43 <etotheipi_> err.. 20 + 36 actually
174 2013-01-25 02:15:17 <etotheipi_> even if you used a raw trie... in terms of computation time (not HDD, which would be a disaster for a raw trie), your doing 56 dereferences to get to your UTXO
175 2013-01-25 02:15:17 <gmaxwell> okay, so if you make the txid/vout part of the key then yes, I see that staying balanced at least... but yor constant factor goes up a lot.
176 2013-01-25 02:15:50 <etotheipi_> in reality, most addresses will only have a couple UTXOs and it will be 2 hops...
177 2013-01-25 02:16:07 <gmaxwell> no no, can't think that way. Assume the data is malicious.
178 2013-01-25 02:16:26 <etotheipi_> gmaxwell: not sure about that
179 2013-01-25 02:16:43 <etotheipi_> unless you're assuming that the *majority* of your nodes are malicious
180 2013-01-25 02:16:45 <gmaxwell> etotheipi_: an implementation of just the core??? some dookhicky which could take atomic updates and return a hash root, and queries and return a SPV fragment and result..  expose it to the internet and we get bounties on breaking it. :P
181 2013-01-25 02:17:27 <etotheipi_> my point was that worst case is still nicely bounded (in compute time, not HDD space)... and thus malicious entities creating addresses will 100million UTXOs are not to be feared
182 2013-01-25 02:17:28 <gmaxwell> etotheipi_: I don't see how majority of nodes matters, I mean??? I can buy that attackers can't get a lot of off-by-one txids or addresses... but they can surely send lots of txs to a single address.
183 2013-01-25 02:17:49 <etotheipi_> gmaxwell: my point is that a majority of addresses will not have 100 million addresses
184 2013-01-25 02:18:14 <etotheipi_> I'm only concerned that worst case is nicely bounded so that there's no real threat from some malicious people doing htis
185 2013-01-25 02:18:44 <gmaxwell> etotheipi_: if having one breaks you, I create one that is like that.. and I query it a lot and knock over nodes. ... and sure, so long as you're worrying about the bounds of the worst case thats fine??? your statement that the average won't be like that was what concerned me.
186 2013-01-25 02:18:48 <etotheipi_> but even if 100 people started cramming 100 addresses each with millions of UTXOs, the majority of the network is still operating with non-mailicious nodes
187 2013-01-25 02:19:37 <gmaxwell> I think we're talking past each other. I apparently misunderstood you when you mentioned average.
188 2013-01-25 02:19:38 <etotheipi_> some of your transactions take 20 hops after finding the address... but most only take 2
189 2013-01-25 02:20:22 <etotheipi_> gmaxwell: okay... I'm a little brain-cluttered tonight
190 2013-01-25 02:20:35 <etotheipi_> I keep mistyping things too... but I think this is constructive
191 2013-01-25 02:21:04 <etotheipi_> gmaxwell: my operating assumption was that computation should not be an issue at all... only disk space
192 2013-01-25 02:21:29 <gmaxwell> I think that I/O bandwidth is by far more concerning that space, assuming that space is at least vaguely reasonable.
193 2013-01-25 02:21:30 <etotheipi_> you're challenging it and making me justify it, but I still think it's valid
194 2013-01-25 02:21:59 <etotheipi_> disk I/O?  or network I/O?
195 2013-01-25 02:22:01 <gmaxwell> Overall growths of storage and local computation have been very rapid.. memory and storage bandwidth seem to be growing hardly better than linearly.
196 2013-01-25 02:22:47 <gmaxwell> disk/memory IO.  Less so network (or at leat that matters too, but if you don't have horrible disk IO requirements the network won't be bad)
197 2013-01-25 02:23:16 <gmaxwell> E.g. if your alg has to visit 100 nodes then the SPV proof will be quite big.
198 2013-01-25 02:23:32 <gmaxwell> But before you'd even care about the proof size??? doing 100 seeks would already kill you.
199 2013-01-25 02:23:39 <etotheipi_> well how bad can it be?  what are plans for expanding block sizes
200 2013-01-25 02:24:07 <etotheipi_> I mean, 1 MB every 10 minutes (expanded maybe 10x for space-inefficient algorithms) doesn't seem like a lot
201 2013-01-25 02:24:38 <gmaxwell> There are currently no plans, and I'm of the opinion that it should either never be done, or only be done very conservatively.  But obviously it shouldn't be designed assuming that it won't be increased.
202 2013-01-25 02:24:49 <gmaxwell> 1MB/10minutes does add up!
203 2013-01-25 02:25:13 <etotheipi_> gmaxwell: but that's only 10MB/10min of *updates*
204 2013-01-25 02:25:17 <etotheipi_> not space increase
205 2013-01-25 02:25:50 <etotheipi_> I like visualizing the network as a bubble... the blockchain is its area, and the UTXO set is the outer surface
206 2013-01-25 02:25:50 <gmaxwell> Yes, though right now most txn are increasing the utxo set size. :(
207 2013-01-25 02:26:10 <gmaxwell> etotheipi_: that scaling law doesn't reflect current behavior. :(
208 2013-01-25 02:26:21 <gmaxwell> But maybe thats short term.
209 2013-01-25 02:26:42 <etotheipi_> gmaxwell: yes, there is probably some constant growth term
210 2013-01-25 02:26:46 <etotheipi_> added on
211 2013-01-25 02:27:17 <gmaxwell> well if we don't find a way to discourage blockchain-for-IM (e.g. by providing alternatives) that term will be large and become larger.
212 2013-01-25 02:27:27 <etotheipi_> lol
213 2013-01-25 02:27:38 <etotheipi_> I already had someone pushing me to help implement that in Armory
214 2013-01-25 02:27:54 <gmaxwell> Bitcoin accidentally invented another kind of network service that doesn't otherwise exist: an anonymous flooding messaging network.
215 2013-01-25 02:28:27 <gmaxwell> etotheipi_: you laugh but effectively thats what SD is doing with their 1e-8 payments to signify 'you lost'.. They never get spent. They're just am IM in the blockchain.
216 2013-01-25 02:28:42 <gmaxwell> and of course lots of other people are trying to do it??? and either failing or getting talked out of it.
217 2013-01-25 02:30:16 <etotheipi_> gmaxwell: I bet there's a way to actually pay users to clean up their dust
218 2013-01-25 02:30:37 <etotheipi_> find a way to actually give users money for transactions that have 300 inputs and 1-2 outputs
219 2013-01-25 02:30:39 <gmaxwell> Yep. The problem is how to align the incentives for both users and miners.
220 2013-01-25 02:31:04 <gmaxwell> One thing I want us to do very soon (perhaps for 0.8) is changing the priority for free transactions to be strongly based on the UTXO impact.
221 2013-01-25 02:31:25 <gmaxwell> Once thats done it'll be very justifyable for wallet software to pick inputs heedful of that.
222 2013-01-25 02:31:54 <gmaxwell> etotheipi_: one thing you need to worry about though is whatever you do you don't want people to create utxos just to later spend them for whatever benefit you provide.
223 2013-01-25 02:32:12 <etotheipi_> like maybe miners can have a gentleman's agreement that they will add a 0.05 output from their coinbase to an address that combines hundreds of outputs into exactly 1 output
224 2013-01-25 02:32:57 <etotheipi_> given how crazy phobic users are from paying a $0.001 fee... I bet they would get irrationally excited about making $0.001 for simply executing a tx
225 2013-01-25 02:33:01 <petertodd> It's too bad the core algorithms aren't such that reducing the UTXO out set actually makes a block inherently more likely to be confirmed... possibly something worth thinking about for any UTXO proposals.
226 2013-01-25 02:33:40 <etotheipi_> it would definitely cost you more to create 300 outputs than you would gain for converting 300->1
227 2013-01-25 02:33:50 <petertodd> For instance, deletions of the UTXO set need to be at least as efficient, and preferably more, than adding.
228 2013-01-25 02:34:25 <petertodd> etotheipi: So that's inherently true with these trie and what not data structures?
229 2013-01-25 02:35:02 <etotheipi_> petertodd: yes, they're strictly bounded O(1) for query, insert and deletion
230 2013-01-25 02:35:12 <etotheipi_> the question is whether feasible worst case is too much, evne if it's O(1)
231 2013-01-25 02:35:17 <gmaxwell> etotheipi_: unfortunately most miners are not thinking deeply about the long term future of the network.
232 2013-01-25 02:35:24 <petertodd> etotheipi: good to hear
233 2013-01-25 02:35:29 <gmaxwell> "MY GPU SHITS MONEY! HURRAY!"
234 2013-01-25 02:35:40 <petertodd> Long-term is someone elses problem.
235 2013-01-25 02:36:00 <petertodd> The mining section of the forums is one of the worst cesspits.
236 2013-01-25 02:36:02 <etotheipi_> petertodd: it's true that even if there were 1e30 UTXOs it would be less than 100 disk seeks
237 2013-01-25 02:36:13 <etotheipi_> but 100 disk seeks is probably too much
238 2013-01-25 02:36:23 <etotheipi_> we.. 56 disk seeks would be the max
239 2013-01-25 02:36:28 <petertodd> Yeah, 100*1mS=0.1s...
240 2013-01-25 02:36:45 <gmaxwell> etotheipi_: maybe we can get away with default fee rules that increase the fees required for bloating txn... Maybe. But the first time someone sees a txn with 0.1 BTC fees and 400 outputs not get mined they'll be looking for the switch to disable that feature.
241 2013-01-25 02:37:17 <petertodd> Keep in mind that there are solid physical reasons why io for *any* storage type will increase much slower than storage capacity. Basically, the world is 3d, and heat dissippation is limited by surface area.
242 2013-01-25 02:37:52 <petertodd> Ultimately the problem is there aren't any other ways of doing BTC micropayments, and there won't be for some time.
243 2013-01-25 02:37:55 <etotheipi_> petertodd: but it looks like even for 21e14 UTXOs, there would be a max of 8 seeks, period
244 2013-01-25 02:38:05 <etotheipi_> not max, but like 95th percentile
245 2013-01-25 02:38:17 <etotheipi_> and it might be possible to optimize the disk layout to make it very efficient
246 2013-01-25 02:38:34 <petertodd> So long as you can keep the disk layout efficient as it gets updated...
247 2013-01-25 02:38:40 <etotheipi_> and on the upside, it's very easy to parallelize, but you don't want to rely on that
248 2013-01-25 02:38:53 <etotheipi_> the parallelization should be a bonus, not a requirement
249 2013-01-25 02:39:23 <petertodd> Parallization will come naturally as we move away from harddrives anyway, but the underlying 3d vs 2d surface area will be a constraint.
250 2013-01-25 02:40:22 <petertodd> Personally I think we're getting off focusing on making ways for people to move away from working with Bitcon proper directly, and moving to stuff linked to it. Or at least having that as an option.
251 2013-01-25 02:40:30 <etotheipi_> on the upside.... as I just pointed out... the number of seeks/hops is pretty much constant
252 2013-01-25 02:40:37 <kjj> in Hausdorff terms, most storage is very nearly 2d, and the busses are 1d
253 2013-01-25 02:40:42 <etotheipi_> the question is whether that constant fits into the current systems
254 2013-01-25 02:40:56 <etotheipi_> if it does, it should only get better relative to future growth
255 2013-01-25 02:41:49 <etotheipi_> dammit... I ended up spending half my night debating UTXO trees again!
256 2013-01-25 02:42:29 <petertodd> kjj: yup, we still make IC's by stacking a relatively small number of layers. We may never find ways to make the Z axis density anything like the X and Y.
257 2013-01-25 02:43:20 <etotheipi_> gmaxwell: I don't get it.... " But the first time someone sees a txn with 0.1 BTC fees and 400 outputs not get mined they'll be looking for the switch to disable that feature."
258 2013-01-25 02:43:30 <kjj> still lots of room at the bottom.  the cool part is that 2d interfaces are coming soon
259 2013-01-25 02:43:53 <etotheipi_> who turns off what feature?
260 2013-01-25 02:44:16 <petertodd> kjj: even the limited extent that 2d interfaces already exist is a horrid nightmare: what axis do you use for your test probes?
261 2013-01-25 02:44:17 <kjj> the user.  they want that 0.1 BTC fee.  that's friggin huge
262 2013-01-25 02:44:45 <petertodd> Well, the user will find a miner who have turned the feature off.
263 2013-01-25 02:45:24 <petertodd> The problem is that the only incentive for miners is not to create big blocks, the block's effect ont he utxo set is irrelevant.
264 2013-01-25 02:45:55 <etotheipi_> petertodd: that's not true... in the future miners will be able to operate with only the UTXO set
265 2013-01-25 02:46:00 <petertodd> You could try systems that discourage blocks, but they risk forking the chain.
266 2013-01-25 02:46:46 <petertodd> etotheipi: Right, but the miner operating the the UTXO set only has nothing to do with the content of the block.
267 2013-01-25 02:47:07 <petertodd> etotheipi: Any individual block changes the set so little that it's hard to have a real reason to discourage.
268 2013-01-25 03:00:44 <jgarzik> 
ACTION karma-tips gmaxwell, for having the patience to bother arguing with kano

269 2013-01-25 03:01:36 <SomeoneWeird> lol
270 2013-01-25 03:33:29 <Luke-Jr> whoa, it's *possible* to argue with kano?
271 2013-01-25 03:40:59 <vazakl> sup
272 2013-01-25 04:52:10 <JyZyXEL> i wonder if its normal to have the blockchain download and verify take days
273 2013-01-25 04:53:13 <yareyare> yes
274 2013-01-25 04:55:40 <JyZyXEL> isn't this a problem?
275 2013-01-25 04:56:46 <CodeShark> yes, it is - which is why in the future, most clients probably won't be doing a full verification of the entire block chain
276 2013-01-25 04:58:27 <andytoshi> JyZyXEL: version 0.8 will have some dramatic speedups
277 2013-01-25 04:58:33 <andytoshi> but yes, there will be a transition to lite clients
278 2013-01-25 04:58:39 <JyZyXEL> it would seem it is a storage i/o bandwidth limited operation
279 2013-01-25 05:00:52 <CodeShark> the biggest bottleneck is actually the signature validation
280 2013-01-25 05:01:14 <andytoshi> i don't think it needs to be so I/O limited..the new database should improve that
281 2013-01-25 05:01:25 <andytoshi> it is validation that takes forever, but you can see the HDD light blinking
282 2013-01-25 05:01:30 <andytoshi> so i dunno what it's up to
283 2013-01-25 05:02:05 <CodeShark> it could use a sliding checkpoint - say, only validate signatures for the last n blocks
284 2013-01-25 05:02:06 <JyZyXEL> well at least in 0.6.2 i don't see much CPU usage but the hard disks are making noises that sound like they are under a serious load
285 2013-01-25 05:02:33 <JyZyXEL> and the whole OS is a bit sluggish because of it
286 2013-01-25 05:03:44 <CodeShark> 6.2 is still using berkeley db for blktree, right?
287 2013-01-25 05:03:58 <JyZyXEL> ill try upgrading to 0.7.2
288 2013-01-25 05:04:07 <CodeShark> JyZyXEL: the latest version is using leveldb instead of berkeley db
289 2013-01-25 05:04:34 <CodeShark> it's a lot faster
290 2013-01-25 05:19:40 <SomeoneWeird> oh it's not berkeley db anymore?
291 2013-01-25 05:19:41 <SomeoneWeird> TIL
292 2013-01-25 05:20:18 <andytoshi> SomeoneWeird: it's berkeley up to v0.8
293 2013-01-25 05:20:23 <gmaxwell> CodeShark: no, the not yet released version.  0.7.2 is the same as 0.6.2 except less horrible fatal bugs.
294 2013-01-25 05:20:24 <andytoshi> which i don't think is out yet
295 2013-01-25 05:20:39 <gmaxwell> as andytoshi says
296 2013-01-25 05:20:56 <gmaxwell> 21:57 < CodeShark> yes, it is - which is why in the future, most clients probably won't be doing a full verification of the entire block chain
297 2013-01-25 05:21:19 <gmaxwell> There is absolutely no plan for that.
298 2013-01-25 05:21:30 <gmaxwell> (in the reference software)
299 2013-01-25 05:21:56 <gmaxwell> Things like starting off as SPV and verifying the chain in the background is a fine sort of thing.
300 2013-01-25 05:22:04 <SomeoneWeird> ah, right
301 2013-01-25 05:22:13 <CodeShark> gmaxwell: I wasn't speaking of bitcoind
302 2013-01-25 05:22:30 <gmaxwell> CodeShark: Yes, and there is no plan for that in bitcoind.
303 2013-01-25 05:22:41 <CodeShark> I didn't say there was
304 2013-01-25 05:23:09 <CodeShark> although bitcoind does have checkpoints
305 2013-01-25 05:23:24 <gmaxwell> CodeShark: which do not disable validation.
306 2013-01-25 05:23:37 <gmaxwell> E.g. you can't inflate the currency with fradulent checkpoints.
307 2013-01-25 05:23:40 <CodeShark> so it isn't too big a stretch to think that eventually the checkpoint might just be n blocks ago
308 2013-01-25 05:23:48 <SomeoneWeird> why don't they, gmaxwell ?
309 2013-01-25 05:23:52 <CodeShark> whatever the current block is
310 2013-01-25 05:24:08 <gmaxwell> CodeShark: Yes actually it's an enormous streach... since they don't actually do what you seem to be thinking they do.
311 2013-01-25 05:24:43 <andytoshi> SomeoneWeird: it wouldn't fit with the trust model of bitcoin
312 2013-01-25 05:24:55 <gmaxwell> You can't actually verify _any_ blocks unless you've observed all of them and advanced a coins database up to current.
313 2013-01-25 05:26:04 <gmaxwell> as andytoshi says, philosophically??? bitcoin is based on zero trust. Even the software is totally transparent and auditable, you're not depending on any magic numbers.  But more technically, it doesn't buy you much??? as you can't validate new blocks without doing all the work on old ones in any case.
314 2013-01-25 05:26:32 <SomeoneWeird> right
315 2013-01-25 05:26:40 <SomeoneWeird> except depending on the genesis block :PP
316 2013-01-25 05:27:27 <gmaxwell> CodeShark: And generally??? it's not a question of startup time, it's a question of bandwidth (mostly) and storage (for the coins database).  There is no reason that all the validation can't be backgrounded in any client... but what security model you're able to use depends on the system capacity.
317 2013-01-25 05:27:43 <gmaxwell> SomeoneWeird: the genesis block is actually created by the node at startup. :P
318 2013-01-25 05:27:49 <gmaxwell> the onlything magic in it is the nonce.
319 2013-01-25 05:27:58 <SomeoneWeird> lmao
320 2013-01-25 05:33:39 <CodeShark> gmaxwell: I'm looking right now at the code for CTransaction::CheckInputs() - and I see it skips ECDSA verification for blocks before the last checkpoint
321 2013-01-25 05:33:53 <CodeShark> so how is it different from the way I was understanding it?
322 2013-01-25 05:34:07 <gmaxwell> checkpoints do a two fold thing??? they force a particular chain??? which mostly has the benefit of preventing a bunch of idiotic dos attacks where someone floods you with tens of gigabytes of blocks at difficulty 1 forked off the genesis.  And they disable the scriptsig (ECDSA) checks because they're computationally slow.
323 2013-01-25 05:34:37 <gmaxwell> But all the other checks??? including, importantly, the checks that coins are not double spent, that outputs sum to inputs, that the subsidy is correct are the same.
324 2013-01-25 05:35:42 <CodeShark> I was only speaking of signature verification
325 2013-01-25 05:36:08 <andytoshi> in theory, could you disable those checks and only do the merkle tree? would that give you something to compare against the checkpoint hash?
326 2013-01-25 05:36:25 <gmaxwell> CodeShark: Then you're confused about how slow signature verification is... on a system here is only increases the time to sync by about 40% without checkpoints.
327 2013-01-25 05:36:56 <CodeShark> 40% isn't insignificant and will probably grow as the average block size increases
328 2013-01-25 05:37:28 <gmaxwell> It's cpu bound, it's much smaller with more cores.
329 2013-01-25 05:37:47 <gmaxwell> on the 0.6.2 code you're talking about before it hardly makes a difference, all the time is spent on the DB.
330 2013-01-25 05:38:22 <CodeShark> actually, JyZyXEL was talking about 0.6.2
331 2013-01-25 05:38:37 <CodeShark> I quickly pointed out that 0.7.2's DB code is much faster
332 2013-01-25 05:40:58 <gmaxwell> CodeShark: Well I guess I misunderstood you.  Someone complained it was taking days to validate, you responded "most clients probably won't be doing a full verification of the entire block chain" ... but performing the signature can't result in in that kind of slowness. There is only one the order of one cpu hour of signatures in the whole chain. So I mistakingly assumed you meant it wouldn't validate doublespends.
333 2013-01-25 05:41:56 <CodeShark> SVP clients won't
334 2013-01-25 05:42:05 <gmaxwell> SPV
335 2013-01-25 05:42:14 <CodeShark> that's what I meant :p
336 2013-01-25 05:42:36 <gmaxwell> CodeShark: ah, well if you want a spv client you can have one today. No future tense.
337 2013-01-25 05:42:39 <gmaxwell> :P
338 2013-01-25 05:42:47 <CodeShark> future tense on the "most"
339 2013-01-25 05:43:03 <CodeShark> in the future, most clients will probably be SPV
340 2013-01-25 05:43:08 <gmaxwell> Sorry. Been having a bad day here.
341 2013-01-25 05:43:23 <CodeShark> it's ok, gmaxwell - all's good :)
342 2013-01-25 05:45:21 <CodeShark> I sold bitcoins yesterday at 18.5 and bought back at 16.3 today :)
343 2013-01-25 05:45:57 <gmaxwell> Spiffy.
344 2013-01-25 05:45:59 <gmaxwell> ;;ticker
345 2013-01-25 05:46:00 <gribble> BTCUSD ticker | Best bid: 16.34848, Best ask: 16.46900, Bid-ask spread: 0.12052, Last trade: 16.34848, 24 hour volume: 172154.00718409, 24 hour low: 15.38613, 24 hour high: 19.18999, 24 hour vwap: 17.29215
346 2013-01-25 05:46:31 <gmaxwell> ;;calc [ticker --last]/[ticker --high]
347 2013-01-25 05:46:31 <gribble> 0.85192748928
348 2013-01-25 06:59:08 <CodeShark> Say you have a resource R which is accessed by A, B, and C. Furthermore, say you don't care if B and C access it at the same time but don't want either of these accessing it at the same time as A. What's the best synchronization structure to use?
349 2013-01-25 07:00:15 <CodeShark> you could use two separate mutexes - but it seems there must be a better way
350 2013-01-25 07:00:43 <CodeShark> moreover, using two mutexes won't really handle the general case
351 2013-01-25 07:00:53 <ThomasV> A needs two locks to access it, lockB and lockC
352 2013-01-25 07:01:32 <CodeShark> but that won't really handle the general case
353 2013-01-25 07:01:42 <ThomasV> define the general case
354 2013-01-25 07:03:04 <CodeShark> ok, you have entities E = {A1, A2, A3, ...} which access resource R. define F(subset of E) == true iff all entities in the subset are allowed to access R at the same time
355 2013-01-25 07:03:29 <SomeoneWeird> 0_o
356 2013-01-25 07:03:30 <ThomasV> use an incremented lock
357 2013-01-25 07:04:11 <CodeShark> how would incrementing help? it's not the number of accesses that's restricted - it's the combination of accesses
358 2013-01-25 07:05:22 <CodeShark> I guess perhaps you need a separate lock for each maximal subset of E such that F(subset) == true
359 2013-01-25 07:06:02 <CodeShark> seems like it could be optimized by using some sort of bitmask, though
360 2013-01-25 07:09:12 <CodeShark> the actual use scenario I envision is not quite so general, though. For instance, say you have a function A which inserts an object into a list and function B which removes an object from a list. and functions f1, f2, f3, f4, ... which iterate over the objects in the list but don't alter the list itself
361 2013-01-25 07:09:43 <CodeShark> you don't care if any of the f's run concurrently with other f's
362 2013-01-25 07:10:12 <CodeShark> but you certainly never want an f running concurrently with either A nor B...nor would you want A and B running concurrently
363 2013-01-25 07:10:46 <SomeoneWeird> when a starts increment a variable and when b starts increment it again and decrement when both are done, then when f{1-4} run check that the variable != 2
364 2013-01-25 07:10:47 <SomeoneWeird> or something
365 2013-01-25 07:10:49 <SomeoneWeird> idno
366 2013-01-25 07:13:09 <CodeShark> a wait condition?
367 2013-01-25 07:13:24 <CodeShark> sounds about right
368 2013-01-25 07:14:29 <CodeShark> hmmm - so when A or B start, they own a lock. fns all wait until a condition variable is signaled, which occurs when A and B exit.
369 2013-01-25 07:15:33 <CodeShark> but A and B also need to know when the f has exited. I know this is probably something covered in multithreading101...but I haven't really worked through many of these examples
370 2013-01-25 07:41:42 <sipa> CodeShark: there is something called a shared mutex
371 2013-01-25 07:42:40 <sipa> which supports any number of threads taking a shared lock, or one process taking an exclusive lock
372 2013-01-25 07:43:14 <sipa> but not shared and exclusive locks at the same time
373 2013-01-25 07:43:26 <CodeShark> aha, I knew sipa would have the answer :)
374 2013-01-25 07:46:24 <CodeShark> you're talking about boost::shared_lock?
375 2013-01-25 07:51:31 <sipa> CodeShark: yes, that's one implementation
376 2013-01-25 07:52:00 <CodeShark> just making sure it refers to the same concept
377 2013-01-25 08:05:16 <CodeShark> strange...on a separate note, I've got a function written in two ways: one way takes the output as a reference parameter, the other returns the output. For some reason, returning the output is a lot faster and I'm not sure why: http://pastebin.com/LNjnyvdQ
378 2013-01-25 08:07:00 <CodeShark> I would think that the for loop is where it spends the bulk of its time
379 2013-01-25 08:07:56 <CodeShark> is accessing the referenced variable slower than accessing a local variable?
380 2013-01-25 08:28:27 <CodeShark> hmmm, in ubuntu the difference isn't as stark as in OS X
381 2013-01-25 08:48:59 <sipa> CodeShark: which are 32 and 64 bit?
382 2013-01-25 08:49:06 <CodeShark> 64 bit
383 2013-01-25 08:49:10 <CodeShark> both
384 2013-01-25 08:49:14 <sipa> hmm
385 2013-01-25 08:49:18 <sipa> weird
386 2013-01-25 08:49:31 <sipa> (that there's a difference)
387 2013-01-25 08:49:58 <CodeShark> yeah, and I ran it several times and even swapped the order in which I did the two and consistently passing the reference variable takes longer
388 2013-01-25 08:51:32 <CodeShark> here's another weird one: http://pastebin.com/6x1rZhee
389 2013-01-25 08:52:31 <CodeShark> this one makes absolutely no sense whatsoever
390 2013-01-25 08:52:49 <CodeShark> I even checked the CPU utilization to make sure that two cores are being used
391 2013-01-25 09:06:23 <CodeShark> it takes almost twice as long to do it with two threads than with one? WTF?!?!?!
392 2013-01-25 09:06:34 <SomeoneWeird> lols
393 2013-01-25 09:12:07 <sipa> CodeShark: is that in a VM or not?
394 2013-01-25 09:12:18 <CodeShark> I tried it both in and out of a VM
395 2013-01-25 09:12:24 <sipa> ok
396 2013-01-25 09:12:27 <CodeShark> in both cases, there was nearly a factor of 2
397 2013-01-25 09:12:35 <CodeShark> and I'm sure I'm giving the VM two processors
398 2013-01-25 09:12:53 <sipa> oo, that means the limiting factor is not CPU but memory access?
399 2013-01-25 09:13:15 <sipa> seems reasonable for such a tight loop
400 2013-01-25 09:13:33 <CodeShark> but the number of memory accesses is about the same in both cases, no?
401 2013-01-25 09:13:59 <CodeShark> besides, for such a tight loop, wouldn't CPU registers be used?
402 2013-01-25 09:14:03 <CodeShark> and the CPU cache?
403 2013-01-25 09:14:31 <CodeShark> I mean, unless the compiler absolutely sucks
404 2013-01-25 09:15:05 <sipa> if both memory addresses are in the same cache row, the per-core cache becomes useless
405 2013-01-25 09:15:37 <sipa> and i don't think the compiler can just turn explicit memory accesses into register updates
406 2013-01-25 09:16:06 <CodeShark> so you're saying that it might be faster to use a local variable and then copy the result?
407 2013-01-25 09:16:25 <sipa> yes
408 2013-01-25 09:16:44 <CodeShark> let's test that hypothesis then :)
409 2013-01-25 09:18:17 <CodeShark> oh wow!
410 2013-01-25 09:18:28 <CodeShark> you're right :)
411 2013-01-25 09:18:50 <SomeoneWeird> sipa's always right
412 2013-01-25 09:19:18 <sipa> 0 + 1 = 2
413 2013-01-25 09:19:26 <SomeoneWeird> holy shit you broke maths
414 2013-01-25 09:21:06 <CodeShark> http://pastebin.com/Mi9mnfKz
415 2013-01-25 09:21:25 <CodeShark> I still don't understand why such a difference, though
416 2013-01-25 09:21:32 <CodeShark> more than three times as fast using two threads
417 2013-01-25 09:21:45 <CodeShark> nearly four times faster
418 2013-01-25 09:22:44 <CodeShark> wide variations, though
419 2013-01-25 09:22:53 <CodeShark> I ran it again and it only took 10.5 secs to do one thread
420 2013-01-25 09:23:10 <CodeShark> but that's inside a VM
421 2013-01-25 09:23:14 <CodeShark> so it might be a scheduling issue
422 2013-01-25 09:23:45 <CodeShark> but yeah, in general I'm getting > 2x
423 2013-01-25 09:23:47 <CodeShark> which is strange
424 2013-01-25 09:24:00 <CodeShark> I would think it would be slightly less than 2x
425 2013-01-25 09:28:52 <CodeShark> that might have to do with virtualization and scheduling, though
426 2013-01-25 09:28:53 <CodeShark> dunno
427 2013-01-25 09:29:03 <ThomasV> blockchain.info does display multisig addresses, but it seems to fail with recent (unconfirmed) ones: http://blockchain.info/address/3Lip6sxQymNr9LD2cAVp6wLrw8xdKBdYFG
428 2013-01-25 09:29:31 <CodeShark> when I run it on the host OS, I do get a factor slightly below 2x
429 2013-01-25 09:43:12 <CodeShark> anyhow, you solved the first issue as well, sipa
430 2013-01-25 09:43:41 <CodeShark> moral of the story: use local variables whenever performing CPU-intensive operations on primitive types whenever possible :)
431 2013-01-25 09:44:28 <CodeShark> avoid dereferencing
432 2013-01-25 09:57:33 <gigavps> can anyone help me understand what happened in this debug.log from bitcoind -> http://pastebin.com/zeHUdncX
433 2013-01-25 10:07:40 <gmaxwell> gigavps: that node mined block 217900 successfully (well, a poolsever rpc connected to that node at least)
434 2013-01-25 10:08:50 <gmaxwell> gigavps: other than solving a block (which is not normally considered a bad thing, esp was it wasn't orphaned) nothing looks unusual there.
435 2013-01-25 10:09:01 <gmaxwell> s/was/as/
436 2013-01-25 10:22:34 <gigavps> gmaxwell, thanks
437 2013-01-25 10:22:51 <gigavps> i guess the log shows that i received a block directly before finding one
438 2013-01-25 10:23:07 <gigavps> what is weird is that bitcoind reported back that the block was accepted
439 2013-01-25 12:44:49 <Cylta> What the relation between this two type of how to show transaction number: abe526a03baf86e212b21da8f424cdae5c258b2568723e2bc3b1ca88e9cdaaa2 (from blockchain.info) and a2aacde988cab1c32b3e7268258b255caecd24f4a81db212e286af3ba026e5ab (from other service). Looks like it's the same number but mixed and reversed...
440 2013-01-25 12:59:29 <kjj> just different endianness
441 2013-01-25 13:00:44 <kjj> In both numbers, each pair of digits is a byte.  one number has the biggest byte at the beginning, the other starts with the smallest
442 2013-01-25 13:02:25 <kjj> biggest here means "highest power".  in the number 19, the 1 is bigger (has the higher power) because it means 1*10^1, while the 9 means 9*10^0
443 2013-01-25 13:08:11 <HM> i hate bigints
444 2013-01-25 13:08:28 <HM> 64 bits should be enough for every one
445 2013-01-25 13:18:11 <Cylta> HM: 640Kb of memory will be enough for everyone --- Sounds similar, eh?
446 2013-01-25 13:18:32 <sipa> there's maybe place for 5 computers in this world
447 2013-01-25 13:18:54 <Cylta> HM: just wait untill we will have golographic memory spheres with exzabytes of data, and 64bits will not be enough to address all that memory.
448 2013-01-25 13:18:54 <SomeoneWeird> or 5 billion
449 2013-01-25 13:27:02 <HM> OK, but i'd be grateful if we could keep it to powers of 2
450 2013-01-25 13:27:11 <HM> Base58 :| who's idea was that?
451 2013-01-25 13:28:15 <SomeoneWeird> SATOSHIS
452 2013-01-25 13:29:21 <HM> Base32 is case insensitive and only 17% longer
453 2013-01-25 13:29:37 <SomeoneWeird> >.>
454 2013-01-25 13:29:37 <SomeoneWeird> a whole 17%
455 2013-01-25 13:29:39 <HM> it hardly seems that many people are hand typing Base58 addresses anyway
456 2013-01-25 13:29:50 <SomeoneWeird> qr ftw
457 2013-01-25 13:29:57 <HM> alternatively Base64 only requires 2 characters outside alphanum
458 2013-01-25 13:31:23 <Cylta> do not even think about typing addresses manually, in any encoding :P
459 2013-01-25 13:31:39 <HM> well there's a check digit isn't there? and the Base58 encoded excludes I
460 2013-01-25 13:31:45 <HM> it seems that was the intent, or at least for OCR
461 2013-01-25 13:31:45 <SomeoneWeird> yes
462 2013-01-25 13:31:55 <HM> *encoding
463 2013-01-25 13:32:54 <HM> Crockfords base32 seems to be a better balance between simple encoding and decoding and human typo protection
464 2013-01-25 13:37:24 <sipa> i'd have liked base32
465 2013-01-25 13:37:46 <sipa> that would also mean we could use e.g. a CRC-30 instead of truncated-double-SHA256 for checksums
466 2013-01-25 13:38:12 <sipa> and have strong guarantees on the errors that would be caught
467 2013-01-25 13:45:56 <kuzetsa> really, OCR on base32 / base58 seems silly when you could just use ISO standard #18004 with 65http://goo.gl/7CPKx
468 2013-01-25 13:45:59 <kuzetsa> ack
469 2013-01-25 13:46:17 <kuzetsa> 65% (level H) error correction, THEN the link after a space. --> http://goo.gl/7CPKx
470 2013-01-25 13:46:20 <kuzetsa> stupid mouse
471 2013-01-25 13:47:19 <HM> QR codes and such are great but you always run in to the case where the user doesn't have a cell phone or a printer
472 2013-01-25 13:47:40 <HM> if you have a pen and some skin you can record a string :P
473 2013-01-25 13:47:43 <kuzetsa> right, but QR code is just a way of encoding / representing it.
474 2013-01-25 13:48:33 <kuzetsa> the 160 bits can be encoded in hex or base32 or base58 and written with pen and paper same as it could be converted to qr code (in raw 160 bit format or a qr code representing the hex or base32 or base58. either way is fine)
475 2013-01-25 13:49:31 <kuzetsa> and at that point, handwriting styles and kerning can throw off the OCR anyway
476 2013-01-25 13:51:57 <sipa> for optimal efficiency when encoding as a QR code, you need base36
477 2013-01-25 13:52:59 <HM> Morse code is another option
478 2013-01-25 13:53:06 <kuzetsa> haha
479 2013-01-25 13:53:28 <kuzetsa> they don't force radio operators to know morse code anymore
480 2013-01-25 13:53:36 <kuzetsa> it's a dying practice to use morse code
481 2013-01-25 13:53:46 <HM> Morse code is base36 , i think
482 2013-01-25 13:53:59 <kuzetsa> no, it's really not
483 2013-01-25 13:54:11 <HM> it's only alphanum though isn't it
484 2013-01-25 13:54:27 <HM> 36 symbols
485 2013-01-25 13:54:37 <HM> well 2 symbols
486 2013-01-25 13:54:40 <HM> whatever
487 2013-01-25 13:54:58 <HM> i'm busy thinking about converting QR codes to morse code
488 2013-01-25 13:56:17 <kuzetsa> HM: morse code has extensions for non-roman letters, and punctuatin too :P
489 2013-01-25 13:57:39 <HM> cool
490 2013-01-25 13:58:09 <kuzetsa> I suspect you were being satirical though / dead-pan humor & not serious.
491 2013-01-25 13:59:26 <HM> well, it's interesting
492 2013-01-25 14:00:29 <kuzetsa> fair enough
493 2013-01-25 14:00:36 <gavinandresen> There's another good April 1 BIP :   "Bitcoin Addresses version --.---.-.  : Morse-Encoded"
494 2013-01-25 14:01:03 <HM> . and - aren't optimal for tapping though, too far away from one another on the keyboard
495 2013-01-25 14:01:12 <HM> should use , and .
496 2013-01-25 14:01:17 <HM> :D
497 2013-01-25 14:01:28 <gavinandresen> ??? put that in the Issues section...
498 2013-01-25 14:03:24 <HM> I'll put it after my idea for a wrinkle resistant tatoo encoding for private keys
499 2013-01-25 14:03:26 <HM> brb
500 2013-01-25 14:18:46 <jgarzik> hmmm, I could tattoo the kids' saving accounts on their bum
501 2013-01-25 14:18:58 <jgarzik> with bitcoin, possibilities are endless
502 2013-01-25 14:19:45 <SomeoneWeird> lollllll
503 2013-01-25 14:42:03 <ThomasV> is it posible to redeem multisig and classical op_checksig inputs in the same tx?
504 2013-01-25 14:43:14 <kjj> yes
505 2013-01-25 15:01:05 <SomeoneWeird> https://www.destroyallsoftware.com/talks/wat
506 2013-01-25 15:01:08 <SomeoneWeird> this should be added to the topic
507 2013-01-25 15:01:36 <sipa> what's that?
508 2013-01-25 15:01:48 <SomeoneWeird> lol
509 2013-01-25 15:01:50 <SomeoneWeird> watch it
510 2013-01-25 15:01:52 <SomeoneWeird> best talk ever
511 2013-01-25 15:02:04 <sipa> This webpage has a redirect loop
512 2013-01-25 15:02:05 <sipa> The webpage at https://www.destroyallsoftware.com/talks/wat has resulted in too many redirects.
513 2013-01-25 15:02:11 <SomeoneWeird> 0.o
514 2013-01-25 15:02:14 <SomeoneWeird> works for me
515 2013-01-25 15:02:41 <SomeoneWeird> https://s3.amazonaws.com/destroyallsoftware-talks/wat.mov?AWSAccessKeyId=AKIAIKRVCECXBC4ZGHIQ&Expires=1359131526&Signature=hoa%2Ffi0fPnOh1jViQysFMWzaoeo%3D
516 2013-01-25 15:32:35 <BlueMatt> gmaxwell: did github kill search in response to you?
517 2013-01-25 15:32:41 <BlueMatt> or was it just a coincidence?
518 2013-01-25 15:33:08 <BlueMatt> or did you see a link and post here?
519 2013-01-25 15:40:51 <SomeoneWeird> coincidence
520 2013-01-25 15:45:51 <sipa> coin cidence?
521 2013-01-25 15:54:25 <SomeoneWeird> possibly
522 2013-01-25 16:36:42 <mappum> i'm keeping track of address balances manually via getblock, but i want to make sure i'm doing this correctly
523 2013-01-25 16:37:23 <mappum> for each new block, i'm going through the transactions, and checking each input. if it is from an address i am tracking, i subtract that from the balance
524 2013-01-25 16:37:43 <mappum> then i go through the outputs, and if one is for an address i am tracking, i add it to the balance
525 2013-01-25 16:38:09 <mappum> am i doing any of that incorrectly?
526 2013-01-25 16:39:15 <sipa> that's correct, but be sure to deal with reorganisations
527 2013-01-25 16:39:21 <mappum> yep
528 2013-01-25 16:39:53 <mappum> if i detect one, i go backwards through the blocks, reversing the txs until i get to where it branched
529 2013-01-25 16:40:12 <sipa> yes, that's correct
530 2013-01-25 16:40:47 <mappum> anything else like that i have to think about?
531 2013-01-25 16:43:25 <sipa> no, that's basically what happens internally as well
532 2013-01-25 16:43:33 <sipa> except it's not indexed by address
533 2013-01-25 17:05:32 <kuzetsa> helo / gmaxwell --> the thing I was saying about block limit, I just realized something (I think... but I'm not sure my understanding of the math is right)
534 2013-01-25 17:05:37 <muhoo> what's the status of making the fee behavior in bitcoij match that of the c++ client?
535 2013-01-25 17:05:51 <muhoo> er bitcoinj (slow ssh connection)
536 2013-01-25 17:07:05 <kuzetsa> ...basically, the value of MAX_BLOCK_SIZE ... can it be exceeded?
537 2013-01-25 17:07:33 <helo> kuzetsa: nope. blocks that are larger than that aren't valid
538 2013-01-25 17:07:42 <kuzetsa> oh my
539 2013-01-25 17:07:53 <kuzetsa> ... then ultimately, my argument was valid after all
540 2013-01-25 17:08:11 <kuzetsa> bitcoin could theoretically break if it was suddenly being used for high volume and needed more than would fit in blocks
541 2013-01-25 17:08:49 <kuzetsa> and raising fees and/or using coins that are more aged (higher priority) would only be a stopgap measure to make transactions go in the block first
542 2013-01-25 17:08:58 <muhoo> more miners then?
543 2013-01-25 17:09:25 <Cylta> I sthere any limit about amount of transactions per block?
544 2013-01-25 17:09:26 <kuzetsa> muhoo: not really. the network would adjust difficulty, and blocks would slow back down to aprox 144 blocks per day
545 2013-01-25 17:09:34 <kuzetsa> Cylta: yes.
546 2013-01-25 17:09:38 <helo> kuzetsa: it wouldn't break... i think the devs have addressed the DOS concerns of a bunch of unconfirmed transactions
547 2013-01-25 17:09:49 <Cylta> kuzetsa: how much of that limit are we using right now?
548 2013-01-25 17:09:51 <helo> rather, i should say "how could it break?"
549 2013-01-25 17:09:57 <lianj> Cylta: the block has a max size
550 2013-01-25 17:10:06 <lianj> (in bytes that is)
551 2013-01-25 17:10:15 <kuzetsa> Cylta: we often hit 1/4 full blocks on like a daily basis from what I've seen
552 2013-01-25 17:10:24 <jgarzik> kuzetsa: MAX_BLOCK_SIZE cannot be exceeded.  Clients will reject blocks larger than that as invalid.
553 2013-01-25 17:10:34 <muhoo> i don't remember if it's possible to generate a transaction that spends outputs that haven't been in a block yet
554 2013-01-25 17:10:40 <kuzetsa> jgarzik: this seems like a systemic design flaw to me
555 2013-01-25 17:11:04 <kuzetsa> ... unless the difficulty adjustment algorithm could be adjusted to have blocks come out faster
556 2013-01-25 17:11:04 <sipa> muhoo: it is possible to spend unconfirmed outputs (according to the protocol, the client only allows this in specific circumstances)
557 2013-01-25 17:11:06 <helo> kuzetsa: limiting the data growth of a decentralized system is the only way to keep a system decentralized
558 2013-01-25 17:11:12 <Cylta> So, why not simply increase MAX_BLOCK_SIZE?
559 2013-01-25 17:11:23 <sipa> Cylta: requires a hard fork
560 2013-01-25 17:11:25 <lianj> Cylta: fork
561 2013-01-25 17:11:39 <Cylta> sipa: lianj: just update all clients simultaniously
562 2013-01-25 17:11:52 <sipa> Cylta: that is indeed what would be necessary
563 2013-01-25 17:12:02 <helo> Cylta: presently the current MAX_BLOCK_SIZE is high enough
564 2013-01-25 17:12:08 <lianj> works well with new windows versions too
565 2013-01-25 17:12:15 <kuzetsa> hmm
566 2013-01-25 17:12:32 <Cylta> IF we already hit 25% of MAX_BLOCK_SIZE, it mean that bitcoin will die in several years...
567 2013-01-25 17:12:36 <sipa> jgarzik: has you asic arrived yet?
568 2013-01-25 17:12:43 <Cylta> I mean, smaller transactions will not be accepted
569 2013-01-25 17:12:51 <sipa> Cylta: no, it means that either there will be a hard fork, or more transactions will happen off-chain
570 2013-01-25 17:13:07 <helo> Cylta: and increasing it once it isn't "high enough" would cut into mining profits, reducing the hashrate... and also ultimately reduce decentralization due to increased resource requirements
571 2013-01-25 17:13:10 <muhoo> won't that cause re-orgs though?
572 2013-01-25 17:13:13 <jgarzik> sipa: https://bitcointalk.org/index.php?topic=137534.msg1478626#msg1478626
573 2013-01-25 17:13:15 <Cylta> sipa: I think simply increasing fees for the transaction could help
574 2013-01-25 17:13:16 <jgarzik> short answer: no
575 2013-01-25 17:13:28 <Cylta> (just stop satoshidice)
576 2013-01-25 17:13:37 <helo> Cylta: when the blocks are full, people just have to pay more fees to get a transaction in a block
577 2013-01-25 17:13:38 <sipa> Cylta: fees will automatically increase if block space becomes scarce
578 2013-01-25 17:13:47 <muhoo> a ton of transactions happpening off-chain, unconfirmed outpus being spent, and then someone tries to get those into a chain?
579 2013-01-25 17:14:04 <kuzetsa> Cylta: that sort of policitcally motivated censorship doesn't seem to set a good precedent.
580 2013-01-25 17:14:11 <helo> Cylta: think of it like a market for block space... miners have 1MB of transactions, so they pick the ones with the highest fees
581 2013-01-25 17:14:12 <kuzetsa> "stop satoshidice" ... meh
582 2013-01-25 17:14:37 <jgarzik> muhoo: that's fine
583 2013-01-25 17:14:43 <jgarzik> muhoo: as long as the dependencies are all in the chain
584 2013-01-25 17:14:58 <Cylta> helo: I understand this. 1M is max? So, simply pay more for your transaction and it will be instant...
585 2013-01-25 17:14:59 <jgarzik> muhoo: You are allowed to have a bunch of off-chain transactions
586 2013-01-25 17:15:11 <jgarzik> muhoo: It's only when decentralized consensus is required, that the chain is required
587 2013-01-25 17:15:21 <Luke-Jr> kuzetsa: it's not political to stop a DDoS
588 2013-01-25 17:15:50 <sipa> and decentralized consensus is only required when you do not want to trust anyone
589 2013-01-25 17:15:54 <helo> Cylta: it won't be instant, but it will make it into an upcoming block if the fee is high enough
590 2013-01-25 17:16:00 <jgarzik> tcatm: bitcoinwatch.com block counter is stuck
591 2013-01-25 17:16:37 <helo> Cylta: if blocks are persistently full, and the fee isn't high enough, it may never make it into a block
592 2013-01-25 17:17:06 <Cylta> helo: I get it. So, there will be "free-market" fee
593 2013-01-25 17:17:11 <Cylta> helo: as for me that's okay
594 2013-01-25 17:17:11 <helo> yup
595 2013-01-25 17:17:21 <muhoo> oh, i see. this so reminds me of git. can run it in decentralized mode, or use github, same protocol, can be used either way
596 2013-01-25 17:17:24 <kuzetsa> Luke-Jr: calling it a pejorative term like DoS isn't based on a clear, universally accepted definition. The exchange of bitcoin for satoshi dice is a valid use / business model which is just as valid as people exchanging on mtgox, or renting advertizing impressions for bitcoin, or buying a pizza or anything else
597 2013-01-25 17:17:25 <sipa> yes, fees are intended to evolve to a free market
598 2013-01-25 17:17:27 <Cylta> stop the satochidice! hehe
599 2013-01-25 17:17:50 <Cylta> I think SD are making about half of current transactions
600 2013-01-25 17:18:05 <helo> probably the biggest consequence will be that lower-valued transactions will be less viable
601 2013-01-25 17:18:11 <muhoo> i keep forgetting that the chain is not the only way bitcoin can be used
602 2013-01-25 17:18:14 <Luke-Jr> kuzetsa: SD doesn't use transactions as exchange, they use it as an action signal
603 2013-01-25 17:18:48 <sipa> muhoo: the chain is the only way bitcoin-the-technology can be used
604 2013-01-25 17:18:56 <sipa> muhoo: but bitcoin-the-currency is wider
605 2013-01-25 17:19:35 <kuzetsa> Luke-Jr: no, I'm pretty sure they transactions are actually exchanging bitcoin.. as a side effect, they're using the blockchain as an entropy source for their "provably fair gambling" business model
606 2013-01-25 17:19:42 <Luke-Jr> kuzetsa: and it does fit the clear objective definition of DDoS in fact
607 2013-01-25 17:20:10 <sipa> calling it a DoS imho implies malicious intent, and it is ridiculous to claim this is the case, imho
608 2013-01-25 17:20:33 <sipa> they may be damaging in unintentional way nonetheless, though
609 2013-01-25 17:20:40 <muhoo> in taht case, porn is a DDos agains the interenet
610 2013-01-25 17:21:08 <sipa> or perhaps better, an ununderstood way
611 2013-01-25 17:21:13 <muhoo> gawd, character munching. in that case, porn is a DDoS against the internet
612 2013-01-25 17:21:38 <Luke-Jr> muhoo: how is porn denying service to the internet?
613 2013-01-25 17:21:39 <kuzetsa> "...an attempt to make a machine or network resource unavailable to its intended users." <--- it's only an attack if you don't intend for gambling small amounts to be allowed, therefore applying a definition to satoshi dice business model as "not a valid service or use / unauthorized user / not an intented user"
614 2013-01-25 17:21:48 <kuzetsa> that's a formal definition of DoS
615 2013-01-25 17:22:48 <Luke-Jr> kuzetsa: and SD does in fact make the Bitcoin network degraded for its intended uses.
616 2013-01-25 17:22:50 <helo> miners should probably be happy about SD, given that it will drive up fees
617 2013-01-25 17:23:00 <muhoo> Luke-Jr: i was around in the 90s. ISPs did consider porn, and then napster, and bittorrent, etc, to be a DDoS. it cloged up the pipes so that "legitimate" traffic was slowed or hampered. i remember selling gear to ISPs t try to cope with this. lots of it.
618 2013-01-25 17:23:57 <Luke-Jr> muhoo: I guess you could argue that??? except that ISPs just sell connectivity in theory
619 2013-01-25 17:24:15 <Luke-Jr> muhoo: Bitcoin, on the other hand, was intended specifically for financial transactions
620 2013-01-25 17:24:15 <muhoo> even before that, in the late 80s and early 90s, anything that was not related to government or academic use was considered to be a DDoS. i knew guys who ran NNTP servers in those days.
621 2013-01-25 17:24:33 <muhoo> you would not believe the shit they went through.
622 2013-01-25 17:24:46 <kuzetsa> right
623 2013-01-25 17:24:55 <kuzetsa> progressive, creative, artistic new-use for existing platforms
624 2013-01-25 17:25:16 <kuzetsa> political evolution of culture and practice
625 2013-01-25 17:25:55 <muhoo> my point is, this argument has been had several times before. might be helpful to look at the prior art, what happened, how people coped, and how we got here today. it's not an easy problem, but it's solvable.
626 2013-01-25 17:27:07 <helo> i think, if the blockchain growing at 1MB is sustainable, SD traffic is tolerable
627 2013-01-25 17:27:53 <kuzetsa> muhoo: ultimately, I'm pretty sure that creating disparity / unfair advantage or disadvantage, or even outright setting a censorship policy for certain practices and/or uses for bitcoin is not an actual goal anyone will openly admit to having.
628 2013-01-25 17:28:09 <Luke-Jr> helo: SD would just flood more
629 2013-01-25 17:29:22 <Luke-Jr> muhoo: there really is no preceding case like Bitcoin though
630 2013-01-25 17:29:37 <kuzetsa> oh... now I feel silly. the DVR has been on pause for the past half hour / was watching a show.
631 2013-01-25 17:29:53 <Luke-Jr> probably Bitcoin will survive because Erik gets arrested finally
632 2013-01-25 17:30:38 <muhoo> Luke-Jr: as a technology, it's prety innovative. but the general problems you're coping with are essentially capacity problems, not new to the internet, or to engineering in general.
633 2013-01-25 17:30:42 <kuzetsa> ok, before I go...
634 2013-01-25 17:31:10 <Luke-Jr> muhoo: when has something ever had "data that must be stored by all peers forever and grows with time"?
635 2013-01-25 17:31:26 <muhoo> Luke-Jr: porn? :-)
636 2013-01-25 17:31:48 <kuzetsa> up until the (currently hard-coded) MAX_BLOCK_SIZE = 1000000 is reached, basically, the transactions with higher priority (coins which have been aged longer, and/or higher fees on the transaction) will be included first, right?
637 2013-01-25 17:31:50 <Luke-Jr> porn can easily be deleted
638 2013-01-25 17:31:52 <jgarzik> Luke-Jr: sadly I think Erik will eventually taint Bitcoin Foundation
639 2013-01-25 17:32:12 <Luke-Jr> jgarzik: hopefully the first election round replaces BitInstant with BitPay
640 2013-01-25 17:32:17 <Luke-Jr> my hope, at least
641 2013-01-25 17:32:20 <jgarzik> kuzetsa: there is only a small area (27k?) for free transactions
642 2013-01-25 17:32:31 <jgarzik> kuzetsa: and inside that "free zone", transactions are sorted by priority
643 2013-01-25 17:32:47 <kuzetsa> jgarzik: priority doesn't matter except for "free" transactions?
644 2013-01-25 17:33:22 <helo> Luke-Jr: it won't ever flood more than 1MB, though... if MAX_BLOCK_SIZE is the appropriate value, then the system will be fine with SD or without it
645 2013-01-25 17:33:22 <jgarzik> kuzetsa: outside the free pool, it's largely sorted by fee-per-kb, IIRC
646 2013-01-25 17:33:30 <jgarzik> but each miner chooses, and may not have the latest rules
647 2013-01-25 17:33:55 <kuzetsa> jgarzik: right, and "fee-per-kb" is basically also linked to the age of the source coins in the blockchain I thought.
648 2013-01-25 17:34:06 <helo> if block space starts to become more scarce, SD will probably fix its system to not send the satoshi notification
649 2013-01-25 17:34:14 <helo> just to stay viable
650 2013-01-25 17:34:14 <jgarzik> kuzetsa: not AFAIK
651 2013-01-25 17:34:33 <kuzetsa> jgarzik: oh, then the "age" only affects free transactions? ... huh, weird.
652 2013-01-25 17:34:52 <Luke-Jr> jgarzik: and Matonis is picking a fight with the IRS in Feb Bitcoin Magazine (unless Mihai takes it out)
653 2013-01-25 17:34:54 <jgarzik> kuzetsa: fee == you are paying to avoid all those pesky anti-spam and age rules :)
654 2013-01-25 17:35:09 <jgarzik> Luke-Jr: not new.  Matonis is vocally anti-tax
655 2013-01-25 17:35:12 <jgarzik> sadly
656 2013-01-25 17:35:23 <Luke-Jr> jgarzik: I mean explicitly encouraging tax evasion
657 2013-01-25 17:35:38 <jgarzik> I've yelled about that before
658 2013-01-25 17:35:42 <jgarzik> c'est la vie
659 2013-01-25 17:35:46 <kuzetsa> hmm
660 2013-01-25 17:36:00 <jgarzik> Luke-Jr: He's done that, or close, in at least one Forbes article
661 2013-01-25 17:36:22 <Luke-Jr> sigh
662 2013-01-25 17:36:28 <jgarzik> I think it's bollocks, but he's high profile and this damages bitcoin's image, IMO
663 2013-01-25 17:36:33 <kuzetsa> I coulda sworn the age rules affected priority for being included in a block when the blocksize was getting higher than 250k as it approaches the 1M limit
664 2013-01-25 17:36:48 <Luke-Jr> jgarzik: more worrisome, it gives the government a reason to go after Bitcoin
665 2013-01-25 17:36:49 <jgarzik> kuzetsa: I don't see any code that does that
666 2013-01-25 17:36:57 <kuzetsa> ouch
667 2013-01-25 17:37:23 <kuzetsa> ... this seems even less sustainable than I thought it looked yesterday :(
668 2013-01-25 17:37:58 <jgarzik> kuzetsa: right now SatoshiDICE happily pays a fee and spams the shit out of the blockchain
669 2013-01-25 17:38:02 <helo> kuzetsa: bitcoin was never supposed to be used for everyday transactions by the entire population of the earth
670 2013-01-25 17:38:20 <kuzetsa> helo: who said so?
671 2013-01-25 17:38:32 <kuzetsa> ... also, the internet was originally for academic / government use :P
672 2013-01-25 17:38:46 <jgarzik> it is inherent in the bitcoin design
673 2013-01-25 17:38:48 <helo> kuzetsa: the existential realities of a decentralized system say so
674 2013-01-25 17:39:12 <kuzetsa> helo: tossing the word "existential" in there doesn't help explain anything
675 2013-01-25 17:39:13 <helo> kuzetsa: there's no way to be decentralized with terabytes of data per day being added to the blockchain
676 2013-01-25 17:39:30 <kuzetsa> helo: sure there is
677 2013-01-25 17:40:13 <kuzetsa> ... storage is gonna be cheaper in 100 years, that might be a fraction of a penny worth of storage when you already have a few exabytes to work with
678 2013-01-25 17:40:36 <helo> kuzetsa: it decentralization can't exist if the data grows too quickly. thus, "existential"
679 2013-01-25 17:41:15 <muhoo> 
ACTION suspects he's the only old fart here who remembers when a terabyte drive was science fiction.

680 2013-01-25 17:41:24 <kuzetsa> muhoo: same
681 2013-01-25 17:41:30 <kuzetsa> I remember when a gigabyte was a big deal
682 2013-01-25 17:41:42 <kuzetsa> ... I'll be 32 years young in a couple years *shrug*
683 2013-01-25 17:42:07 <kuzetsa> blah I really need to get back to watching this show. sorry for such a lengthy diversion
684 2013-01-25 17:42:41 <kuzetsa> helo: also, terabytes per day won't happen
685 2013-01-25 17:43:50 <kuzetsa> even if the population of the planet was somehow increased 10 fold, the overall volume of transaction in ANY currency (fiat, commodities, bartering or otherwise) wouldn't likely generate terabytes per day
686 2013-01-25 17:44:24 <helo> three transactions per day per human over the age of 15 will be a lot of data
687 2013-01-25 17:44:42 <muhoo> back to more mundane things: fees. should i implement fees on my own outside of bitcoinj (transliterating the code in the c++ client), submit a patch to bitcoinj to do it, or wait for bitcoinj to implement it?
688 2013-01-25 17:45:12 <muhoo> is it sitting in someone's branch somwhere waiting to be merged?
689 2013-01-25 17:45:45 <helo> kuzetsa: 6 billion * 3 * 0.5kb = 8 terabytes
690 2013-01-25 17:46:00 <kuzetsa> helo yeah, I just fact-checked that math myself... touche
691 2013-01-25 17:47:44 <helo> kuzetsa: there is plenty of a market for higher-valued financial transactions that bitcoin's 1MB blocks could handle
692 2013-01-25 17:47:49 <kuzetsa> now I just find myself wondering why there's still no pruning algorithm or compression or other space-saving algorithm(s) on the horizon for older transactions in the blockchain
693 2013-01-25 17:48:21 <muhoo> wait, i thought pruning was coming soon?
694 2013-01-25 17:48:21 <sturles> From the wiki: If the blocksize is more than 250 kB, transactions get increasingly more expensive as the blocksize approaches the limit of 500 kB. Sending a transaction when the blocksize is 400 kB will cost 5 times the normal amount; sending when it's 499 kB will cost 500x, etc.
695 2013-01-25 17:48:28 <helo> there is pruning
696 2013-01-25 17:48:30 <kuzetsa> it is? oh neat.
697 2013-01-25 17:48:35 <kuzetsa> that'll be nice
698 2013-01-25 17:48:38 <sturles> Perhaps the progressive fees should start earlier?
699 2013-01-25 17:48:46 <sturles> E.g. at 100 kB?
700 2013-01-25 17:49:53 <kuzetsa> iunno... I'm gonna drop this subject until pruning or other space-saving measures have been tested and deployed
701 2013-01-25 17:50:40 <Luke-Jr> sturles: they're not in effect anymore since 0.6 IIRC
702 2013-01-25 17:51:18 <jgarzik> muhoo: I think bitcoinj + better fee handling is in the "todo/anybody" category. TD[gone]`
703 2013-01-25 17:51:49 <Luke-Jr> jgarzik: it's in the "code exists and is tested, but needs formal unit tests" category
704 2013-01-25 17:51:56 <jgarzik> ah, cool
705 2013-01-25 17:52:05 <Luke-Jr> of course, it can always be improved even more I'm sure
706 2013-01-25 17:52:17 <Luke-Jr> (the existing code has been sitting idle for months)
707 2013-01-25 17:52:54 <jgarzik> Luke-Jr: url?
708 2013-01-25 17:53:50 <Luke-Jr> jgarzik: https://github.com/bitcoin/bitcoin/pull/1647
709 2013-01-25 17:55:23 <jgarzik> Luke-Jr: The discussion was about bitcoinj
710 2013-01-25 17:57:44 <Luke-Jr> jgarzik: oh, I get it. nm
711 2013-01-25 18:05:44 <muhoo> right, it looks like bitcoind already supports it. i guess i'll try to confirm later when td or bluematt are around
712 2013-01-25 18:06:13 <muhoo> what the status of bitcoinj implementation is
713 2013-01-25 18:06:42 <gavinandresen> status of bitcoinj implementation is "healthy, good"
714 2013-01-25 18:07:10 <muhoo> gavinandresen: of fee handling?
715 2013-01-25 18:07:24 <gavinandresen> oh, of fee handling: I believe that is near the top of TD's TODO
716 2013-01-25 18:08:18 <muhoo> right, i was trying to find out if i should wait, submit a patch, or hack around it outside the library
717 2013-01-25 18:09:00 <muhoo> i'll ask when he's around
718 2013-01-25 18:10:36 <muhoo> 
ACTION just got confirmation of  his first 2 transactions via bitcoinj (on testnet)

719 2013-01-25 18:12:40 <BlueMatt> muhoo: whats up?
720 2013-01-25 18:13:25 <BlueMatt> oh, bitcoinj fee stuff...I dunno exactly, but its up there (if not in master...)
721 2013-01-25 18:55:45 <grau> Will you be in Bratislava on Sunday ?
722 2013-01-25 18:56:42 <grau> I mean this event: http://www.facebook.com/events/425780117494016/
723 2013-01-25 19:01:37 <gmaxwell> grau: non-facebook users can't even get a subject line from what you're linking to there.
724 2013-01-25 19:02:02 <Luke-Jr> UNsystem Bitcoin Conference Preview 2013
725 2013-01-25 19:02:18 <Luke-Jr> On Sunday 27th January 2013, we will have a great opportunity to see and meet core organizers of the upcoming Bitcoin conference 2013 in our new shiny Progressbar at Michalsk?? 3 in the centre of Bratislava:
726 2013-01-25 19:02:21 <Luke-Jr> Amir Taaki (Bitcoin developer, Bitcoin conference organizer)
727 2013-01-25 19:02:25 <Luke-Jr> Cody R Wilson (The Wiki Weapon Project) - Crypto-banking / mutualist political economy
728 2013-01-25 19:03:29 <Luke-Jr> Juraj Variny, Marek "slush" Palatinus, Mihai Alisie, Mike Gogulski, Nicolas Fischer , Pavol Rusnak, Peter Surda, yossarian, Jamileh Taaki , Mark David Lamb, Stefan Thomas, Andreas Schildbach, Frank Braun , Joerg Platzer
729 2013-01-25 19:03:33 <grau> Juraj Variny - How to invest into the biggest bitcoin stock exchange and the broker service coinbr.com
730 2013-01-25 19:03:33 <grau> Marek "slush" Palatinus (Author of Bitcoin.cz pool) - Trezor hardware bitcoin wallet
731 2013-01-25 19:03:33 <grau> Mihai Alisie (Bitcoin magazine)
732 2013-01-25 19:03:35 <grau> Mike Gogulski (Bitcoin laundry) - panel moderator
733 2013-01-25 19:03:37 <grau> Nicolas Fischer (Bitcoin conference organizer)
734 2013-01-25 19:03:39 <grau> Pavol Rusnak - Trezor hardware bitcoin wallet
735 2013-01-25 19:03:41 <grau> Peter Surda - Economics of Bitcoin
736 2013-01-25 19:03:43 <grau> yossarian - lawyer and Bitcoin conference organiser. He's a big local dealer in Berlin with lots of interesting stories.
737 2013-01-25 19:03:45 <grau> Jamileh Taaki - studying astrophysics at London university
738 2013-01-25 19:03:47 <grau> Mark David Lamb - Bitcoin entrepreneur from London
739 2013-01-25 19:03:49 <grau> Stefan Thomas - author of BitcoinJS and WeUseCoins Bitcoin video / website
740 2013-01-25 19:03:51 <grau> Andreas Schildbach - created Bitcoin for Android and developer for BitcoinJ (he's recently taken over) which is the Bitcoinimplementation used on most phone clients and MultiBit. Bitcoin Wallet for Android
741 2013-01-25 19:03:53 <grau> Frank Braun - the masked privacy extremist who was at the London conference doing a lecture on cryptoanarchism. Also his buddy Smuggler.
742 2013-01-25 19:03:55 <grau> Joerg Platzer - started accepting Bitcoin at his restaurant; Room77. It's famous for its delicious hamburgers in a posh part of Berlin called Kreuzberg. The area was bohemian and retains that feel. Local shops dominate. Joerg was getting popular in the press, and other curious shop-owners became interested. All along the road, shops are now accepting Bitcoin. You can walk the Bitcoin mile in
743 2013-01-25 19:03:57 <grau> Berlin's Kreuzberg.
744 2013-01-25 19:03:57 <Luke-Jr> ???
745 2013-01-25 19:04:03 <Luke-Jr> grau: I intentionally avoided doing that :P
746 2013-01-25 19:04:15 <grau> why?
747 2013-01-25 19:04:24 <Luke-Jr> it's considered bad ettiquite on IRC :P
748 2013-01-25 19:04:32 <BlueMatt> because you just pasted like 30 lines....
749 2013-01-25 19:04:52 <grau> I know, since I was asked to carry over what was not visible.
750 2013-01-25 19:05:16 <grau> will I now be banned :)?
751 2013-01-25 19:06:02 <Luke-Jr> if you weren't grau, you might have been. instead now we're all just glaring at you :P
752 2013-01-25 19:06:29 <grau> ok, shaming...
753 2013-01-25 19:07:25 <grau> I will be there and, so I was curious who else
754 2013-01-25 19:08:03 <Luke-Jr> grau: the Foundation is doing a different Bitcoin Conference I think