1 2012-09-03 06:02:05 <killerstorm> Do I understand it right that creating CTxDB in places which don't get it might cause deadlocks?
2 2012-09-03 06:02:33 <killerstorm> I.e. I created CTxDB txdb("r"); in CWallet method to fetch some information about transaction, and now client hangs.
3 2012-09-03 06:05:19 <killerstorm> Is that expected behaviour?
4 2012-09-03 06:05:41 <killerstorm> It seems to hang somewhere around: REORGANIZE
5 2012-09-03 06:05:43 <killerstorm> REORGANIZE: Disconnect 0 blocks; 000000003d9b4a29995a..000000003d9b4a29995a
6 2012-09-03 06:05:45 <killerstorm> REORGANIZE: Connect 1 blocks; 000000003d9b4a29995a..000000000a05755e41b4
7 2012-09-03 07:11:04 <robbak> I'm currently helping get bitcoin 0.6.3 into FreeBSD. Could someone help me by lobbing some 0.6.3 version testnet coins at mfi2Er7T2MBgAJRJmi8aL4BpPodt98t3fu ?
8 2012-09-03 07:12:29 <robbak> Or myuKAcyHpvWhBVoLzW2fUAWS6NuXk67iSb , or both. This one is running on BSD.
9 2012-09-03 07:36:05 <fluke7350> is there any way to force "sendtoaddress" to include a specific input in the wallet ?
10 2012-09-03 07:37:32 <fluke7350> i want to relay unconfirmed coins and this way i would be able to do it instantly, because it poses no risk to me (if the input does not confirm, the output wont be included in the block too)
11 2012-09-03 07:44:28 <robbak> I'm currently helping get bitcoin 0.6.3 into FreeBSD. Could someone help me by lobbing some 0.6.3 version testnet coins at mfi2Er7T2MBgAJRJmi8aL4BpPodt98t3fu or myuKAcyHpvWhBVoLzW2fUAWS6NuXk67iSb
12 2012-09-03 07:44:38 <robbak> That latter one is running on BSD.
13 2012-09-03 07:47:14 <robbak> It is also using 50% of each cpu constantly, even when the download is finished. Is that normal? We are patching qtipserver.cpp to apply the OSX fix, that takes it down from 100%
14 2012-09-03 07:47:48 <fluke7350> robbak, i am running my bitcoind on freebsd too
15 2012-09-03 07:48:10 <fluke7350> i guess that cpu usage is while its fetching the blockchain
16 2012-09-03 07:48:18 <fluke7350> it has to verify it
17 2012-09-03 07:48:28 <robbak> This is after it finishes getting the blockchain.
18 2012-09-03 07:48:46 <fluke7350> strange
19 2012-09-03 07:50:00 <fluke7350> 95351 root 7 82 2 507M 448M RUN 264:14 27.98% bitcoind
20 2012-09-03 07:50:02 <fluke7350> but still fetching
21 2012-09-03 07:51:35 <_dr> you can always use a profiler to find out where it spends the cpu time
22 2012-09-03 07:52:00 <robbak> thanks. afk,sorry
23 2012-09-03 08:13:26 <robbak> Seems like I've found it. A Patch to bitcoin.cpp disabling qtipcserver fixes it. The patch is :
24 2012-09-03 08:13:31 <robbak> --- src/qt/bitcoin.cpp.orig 2012-09-03 19:41:52.000000000 +1000 +++ src/qt/bitcoin.cpp 2012-09-03 19:42:14.000000000 +1000 @@ -134,7 +134,7 @@ #ifndef BITCOIN_QT_TEST int main(int argc, char *argv[]) { -#if !defined(MAC_OSX) && !defined(WIN32) +#if !defined(MAC_OSX) && !defined(WIN32) && !defined(BSD) // TODO: implement qtipcserver.cpp for Mac and Windows // Do this early as we don't want to bother initializing
25 2012-09-03 08:13:58 <robbak> So, just what does qtipserver do?
26 2012-09-03 08:17:15 <sipa> robbak: it's needed for receiving URLs sent to it
27 2012-09-03 08:27:45 <sipa> fluke7350: see the raw transaction APIs that were added for 0.7.0
28 2012-09-03 08:32:01 <fluke7350> sipa, i have seen them, but i will look into the code and maybe patch the sendtoaddress
29 2012-09-03 08:32:14 <fluke7350> i just dont want to do my own fee-guessing and other stuff ;)
30 2012-09-03 08:58:48 <killerstorm> can somebody send my testnet3 coins to moEaekk95oJrdH6JWb7KYEtJEyksKy7Hs1? thanks. (clinet 0.7)
31 2012-09-03 08:59:23 <killerstorm> I'm trying to implement colored bitcoins :)
32 2012-09-03 08:59:44 <sipa> what are colored bitcoins?
33 2012-09-03 09:00:59 <killerstorm> Basically some bitcoins are declared to represent something else, i.e. asset of some sort.
34 2012-09-03 09:01:23 <sipa> how will you prevent them being mixed with other non-colored ones?
35 2012-09-03 09:02:01 <killerstorm> I.e. I declare that bitcoins spend in tx with a certain hash are "killerstorm's bitcoins" now and bitcoins which can be traced backwards to them have a special meaning.
36 2012-09-03 09:02:06 <killerstorm> i.e. a different exchange rate.
37 2012-09-03 09:02:18 <killerstorm> Separate wallet for each color.
38 2012-09-03 09:03:10 <theorbtwo> Or, possibly, that they are tainted coins, the result of a known illicit transaction, and should be refused / sent back to some configured account.
39 2012-09-03 09:03:11 <killerstorm> Client which is aware of colored bitcoins will send transactions without mixing colors. If person with generic client gets them, he can mix them.
40 2012-09-03 09:03:37 <theorbtwo> (In which case you need to support uncolouring rules as well, so that configured account can then spend them.
41 2012-09-03 09:04:37 <sipa> killerstorm: amiller was thinking about schemes where the script language could be used to enforce such things, afaik
42 2012-09-03 09:05:06 <killerstorm> Tainting has somewhat different semantics, basically taint is contagious, while coloring is anti-contagious.
43 2012-09-03 09:05:39 <sipa> mixing colored coins with others removes the coloring?
44 2012-09-03 09:05:46 <killerstorm> I think so.
45 2012-09-03 09:06:36 <sipa> so it's a bit like bullion coins that have a legal tender face value, but are sold for (a lot) more
46 2012-09-03 09:06:40 <killerstorm> People should be careful not to do that as their rare coins might lose their value.
47 2012-09-03 09:07:27 <killerstorm> Yep.
48 2012-09-03 09:08:21 <killerstorm> Here's a forum thread: https://bitcointalk.org/index.php?topic=101197.0
49 2012-09-03 09:09:20 <killerstorm> Potential use cases are coins with exchange rate pegged to something by some company which issues them, representation of company shares, etc.
50 2012-09-03 09:09:29 <sipa> did you read https://en.bitcoin.it/wiki/Smart_Property ?
51 2012-09-03 09:09:36 <killerstorm> So it might be an alternative to GLBSE/OpenTransactions eventually
52 2012-09-03 09:12:40 <killerstorm> Yep I looked it through at some point.
53 2012-09-03 09:13:30 <killerstorm> If I understand you correctly, there is some overlap, but colored coins is simple, robust, practical solution, while hardcore contract stuff is largely a theoretic one for now.
54 2012-09-03 09:14:57 <amiller> killerstorm, when you say "in tx with a certain hash" do you mean like proof-of-work / vanity address?
55 2012-09-03 09:16:27 <sipa> i think he means just tracking txout
56 2012-09-03 09:17:11 <amiller> okay i see, i misparsed
57 2012-09-03 09:33:04 <MC-Eeepc> quite a lot of spam going on in #bitcoin
58 2012-09-03 09:34:21 <MC-Eeepc> cant freenode do anything
59 2012-09-03 09:35:38 <nathan7> Has anyone contacted an IRCop?
60 2012-09-03 09:36:52 <diki> Few days ago I was looking at the bitcoinj source or perhaps a fork where there were files called HashUtils.java,PrivateKey.java,PublicKey.java. But upon looking at the source today, I cannot find them anymore.
61 2012-09-03 09:37:48 <diki> So I thought I might have looked at some fork. Was wondering if anyone knows such a fork of the project.
62 2012-09-03 09:39:12 <MC-Eeepc> well how do i disable PM for myself
63 2012-09-03 09:39:38 <sipa> http://freenode.net/faq.shtml#spambots
64 2012-09-03 09:42:16 <amiller> killerstorm, i keep looking for a "killer app" that would justify and help focus on making stronger validation scripts
65 2012-09-03 09:42:40 <amiller> so far, for every candidate i've found, there's some way of doing it just using the current rules that at least mostly works
66 2012-09-03 09:44:12 <amiller> the simplest solution for any of the 'colored coins' use cases is to self-issue some currency with a 'conservation' rule encoded in the validation script
67 2012-09-03 09:47:14 <killerstorm> what do you mean by stronger validation scripts?
68 2012-09-03 09:48:34 <sipa> he's talking about a hypothetical successor to bitcoin in which scripts are more powerful
69 2012-09-03 09:49:35 <amiller> it's a lot more complicated to try to add new layers of rules, that are supposed to work even though the miners don't treat the transactions any differently
70 2012-09-03 09:50:29 <amiller> "we take each of these transaction and crawl the block chain to see if this transaction was generated from the genesis transaction." this is the colored coins wiki i think http://yoniassia.com/bitcoin-2-x-or-currency-2-x-initial-specs/
71 2012-09-03 09:50:55 <amiller> as described, that's very inefficient
72 2012-09-03 09:51:42 <sipa> well obviously you implement it by adding a color flag to the utxo set database
73 2012-09-03 09:53:17 <amiller> i think it wouldn't be enough just to add an extra index
74 2012-09-03 09:53:30 <amiller> because you'd also have to make that index exclusive
75 2012-09-03 09:53:46 <sipa> hmm?
76 2012-09-03 09:54:05 <sipa> killerstorm: what if a mix a red coin with a blue coin?
77 2012-09-03 09:54:31 <sipa> do i get a purple coin, or a non-colored coin, or a blue coin, or a red coin?
78 2012-09-03 09:55:10 <amiller> (i misinterpreted you again, disregard those lines containing 'index')
79 2012-09-03 09:55:51 <killerstorm> haha good question. I assume that coins of mixed heritage are same as non-colored, ordinary coins.
80 2012-09-03 09:56:09 <amiller> sipa, only people who are 'full red nodes' and/or 'full blue nodes' will have color flags in their database
81 2012-09-03 09:56:47 <killerstorm> I implemented it as a meta-info in TxDB
82 2012-09-03 09:56:57 <killerstorm> basically an extra index.
83 2012-09-03 09:57:20 <sipa> TxDB will soon be gone, however :)
84 2012-09-03 09:57:57 <killerstorm> wow
85 2012-09-03 09:58:00 <killerstorm> what replaces it?
86 2012-09-03 09:59:40 <sipa> in my current branch, CChainDB and CCoinsDB, but that may still change
87 2012-09-03 10:07:18 <amiller> killerstorm, one sad thing about colored coins is that you can't arrange for an atomic-transaction that exchanges some blue coins for some red coins
88 2012-09-03 10:08:05 <amiller> you could easily do that with actual custom validation rules
89 2012-09-03 10:08:20 <killerstorm> there is a solution through contracts. E.g. https://en.bitcoin.it/wiki/Contracts#Example_5:_Trading_across_chains
90 2012-09-03 10:08:21 <sipa> you could have a rule that a transaction with identical input and output values in txin and txout, retains colors
91 2012-09-03 10:08:28 <killerstorm> Although maybe there is an easier one within one chain.
92 2012-09-03 10:08:46 <sipa> killerstorm: except your color coins do not form a separate chain
93 2012-09-03 10:09:25 <killerstorm> sipa: yes I've thought about it too.
94 2012-09-03 10:09:51 <killerstorm> sipa: it is possible even on separate chain, it's only easier if it is in one chain.
95 2012-09-03 10:10:04 <sipa> ideally, you'd have a marker in each txout that specifies its color
96 2012-09-03 10:10:07 <amiller> sipa, the only problem with that rule is that everyone would need to maintain color flags for all the possible colors
97 2012-09-03 10:10:53 <sipa> and a validation rule that constrains sum(txouts, color_X) <= sum(txins, color_X)
98 2012-09-03 10:11:06 <sipa> amiller: yeah
99 2012-09-03 10:11:18 <sipa> especially when people want colors for their own private purposes
100 2012-09-03 10:11:48 <killerstorm> amiller: why? If I only care about red color I just need to see that 10 red coins go in and 10 red coins go out. I.e. 10 coins which go out are red coins. I don't care about other coins.
101 2012-09-03 10:11:53 <killerstorm> It is messy, but doable.
102 2012-09-03 10:12:46 <killerstorm> It can require a fingerprint in txn amount, i.e. we would transfer 10.0111011 coins to 10.0111011 coins.
103 2012-09-03 10:12:50 <amiller> if 10 red coins go out, but the transaction is invalid, then the issuer would not honor it
104 2012-09-03 10:13:06 <amiller> even if the transaction is invalid only because of the blue coins not being valid
105 2012-09-03 10:13:15 <edcba> tainted bitcoins ?
106 2012-09-03 10:13:23 <amiller> colored is like anti-tainted
107 2012-09-03 10:13:59 <edcba> anti tainted ???
108 2012-09-03 10:14:02 <amiller> "colored": a mark that washes off easily if you mix with something else "marked" something that sticks for a while "tainted" it's contagious
109 2012-09-03 10:14:32 <edcba> ok
110 2012-09-03 10:14:59 <edcba> is it for privacy ?
111 2012-09-03 10:15:45 <edcba> to avoid spending obvious origin coins ?
112 2012-09-03 10:15:49 <killerstorm> No, it's for issuance of notes which have different value from bitcoins.
113 2012-09-03 10:15:50 <amiller> it's for increased liquidity, because you can create IOU/issuer currencies that are useful in trusting networks
114 2012-09-03 10:26:19 <killerstorm> amiller: speaking of advanced scripts, what do you think about opcode which would return time or blockchain height? (use case is time-dependent scripts)
115 2012-09-03 10:27:35 <sipa> killerstorm: satoshi was strongly against that, because it would allow a transaction which changes from valid to invalid
116 2012-09-03 10:27:54 <sipa> which could cancel an entire tree of transactions derived from iut
117 2012-09-03 10:28:04 <amiller> killerstorm, i'll tell you what quantity i like, is "sum difficulty", or "total work" or w/e you call it
118 2012-09-03 10:28:11 <amiller> ultraprune reports it in its debug logs
119 2012-09-03 10:28:18 <sipa> amiller: so does bitcoin
120 2012-09-03 10:28:22 <amiller> oh.
121 2012-09-03 10:28:43 <sipa> it's used to find the best chain
122 2012-09-03 10:29:12 <sipa> but in satoshi's model, transactions could deliberately not observe any blockchain state
123 2012-09-03 10:29:22 <sipa> as they sometimes need to be moved in the chain
124 2012-09-03 10:30:23 <denisx> re
125 2012-09-03 10:30:58 <killerstorm> I've discussed it with Mike Hearn on forum, and I think we came to conclusion that if implemented properly, opcode which returns height is not a problem
126 2012-09-03 10:31:01 <amiller> i don't see the point of that, transaction validity is already dependent on the context of the blockchain because of the utxos
127 2012-09-03 10:31:17 <sipa> it's not
128 2012-09-03 10:31:28 <sipa> transaction validity depends on the previous transactions it refers to
129 2012-09-03 10:31:34 <sipa> in whatever context you're looking at it
130 2012-09-03 10:31:45 <sipa> but that context is not necessarily (just) the blockchain
131 2012-09-03 10:32:12 <sipa> mempool transactions' validity also depends on other mempool transactions in particular
132 2012-09-03 10:32:25 <killerstorm> I really do not care what Satoshi thought about it, I just see no way it 'induces fork' or something.
133 2012-09-03 10:32:50 <sipa> ok, have a script that says BLOCKCHAIN_HEIGHT <= 200000
134 2012-09-03 10:33:00 <sipa> the tx gets propagated
135 2012-09-03 10:33:01 <killerstorm> The only complaint is that nodes cannot identify whether it is a valid txn or not.
136 2012-09-03 10:33:12 <sipa> it is included in mempools, because it looks valid
137 2012-09-03 10:33:18 <sipa> it is mined in block 199999
138 2012-09-03 10:33:29 <sipa> at block 200001 we get a 3-block fork
139 2012-09-03 10:33:53 <sipa> and suddenly the transaction, which was valid and included, and has potentially many dependent transactions already
140 2012-09-03 10:33:56 <sipa> becomes invalid
141 2012-09-03 10:34:04 <amiller> if there's a 3 block fork, then there could be a double-spend that preempts 3 blocks worth of transactions too
142 2012-09-03 10:34:19 <sipa> of course, but a double spend implies foul play
143 2012-09-03 10:34:21 <killerstorm> Double spend will have exactly these properties.
144 2012-09-03 10:34:24 <sipa> yes, and attacker can do that
145 2012-09-03 10:34:28 <killerstorm> It was valid once, but now it is double-spend.
146 2012-09-03 10:34:51 <amiller> sipa, but it doesn't necessarily indicate whose fault it is
147 2012-09-03 10:35:08 <sipa> but this can lead to transactions becoming invalid without anyone behaving less than ideally
148 2012-09-03 10:35:25 <amiller> only if you wait till near the last minute
149 2012-09-03 10:35:42 <sipa> how do you mean?
150 2012-09-03 10:36:47 <killerstorm> If one spends it near expiration date it is essentially a race condition.
151 2012-09-03 10:36:48 <amiller> an example usage would be that i give you a bitcoin that you can spend, or it gets sent back to my address after work=200000megahashes
152 2012-09-03 10:36:56 <amiller> suppose that's in two weeks
153 2012-09-03 10:37:23 <amiller> there's a race condition like killerstorm says if you wait for 2 weeks - a few hours
154 2012-09-03 10:37:37 <sipa> but it's out of your control
155 2012-09-03 10:37:44 <sipa> you don't decide when it gets mined
156 2012-09-03 10:38:03 <sipa> and if there is an expiration date, you inevitably risk it being mined close to it
157 2012-09-03 10:38:27 <amiller> miners refusing or deliberately delaying valid transactions counts as 'foul play'
158 2012-09-03 10:38:35 <killerstorm> Bitcoin security model implies that transactions are not considered a valid payment before they have enough confirmations.
159 2012-09-03 10:38:41 <amiller> also this is a good reason not to prefer receiving bitcoins with expiration dates
160 2012-09-03 10:38:46 <sipa> killerstorm: i disagree completely
161 2012-09-03 10:39:02 <sipa> they are not considered confirmed before enough confirmations
162 2012-09-03 10:39:10 <kjj_> heh. like you have a say in the transactions that come to you
163 2012-09-03 10:39:11 <sipa> which is what you need if you don't trust the sender
164 2012-09-03 10:39:46 <sipa> exactly like doing a deal with someone you don't trust over paypal; you need a few months to be sure he won't try to reverse the transaction
165 2012-09-03 10:39:55 <sipa> but if you trust him, you can spend what you received immediately
166 2012-09-03 10:41:07 <killerstorm> ok, so do we agree that objections to OP_HEIGHT or however it is called are entirely on a meta level?
167 2012-09-03 10:41:10 <amiller> kjj_, you do have a say if you're a merchant, if you make a deal with someone that says "must be paid in non-expiring bitcoin tx" and someone sends you something that expires in an hour, then you just disregard it
168 2012-09-03 10:41:21 <killerstorm> I.e. there is no fundamental reasons to not implement this.
169 2012-09-03 10:41:24 <amiller> OP_WORK :p
170 2012-09-03 10:41:27 <sipa> killerstorm: i have no problem with transactions that have a deadline for remaining in the mempool
171 2012-09-03 10:42:21 <kjj_> amiller: better not to have an option to expire. what's the point?
172 2012-09-03 10:42:45 <killerstorm> The point is that it is possible to implement escrow with timeout.
173 2012-09-03 10:42:50 <kjj_> killerstorm: reorgs happen
174 2012-09-03 10:43:02 <killerstorm> E.g. I send money to escrow, but if escrow dies I get money back in a year, for example.
175 2012-09-03 10:43:05 <sipa> amiller: ok, let's say we have a (stupid, or for whatever reason) merchant who does accept expiring transactions
176 2012-09-03 10:43:20 <sipa> amiller: now that merchant pays me with a non-expiring transaction
177 2012-09-03 10:43:52 <sipa> if i don't want to accept expiring transactions, i also have to check the entire ancestry of what comes to me
178 2012-09-03 10:43:54 <killerstorm> multisig is fun and games until somebody loses his private keys or just dies.
179 2012-09-03 10:44:03 <amiller> sipa, you probably should be checking the ancestry anyway
180 2012-09-03 10:44:14 <amiller> sipa, if someone sends you a coin that they just received, then it's liable to be double-spent out from under them
181 2012-09-03 10:44:45 <sipa> amiller: sure, but that would imply not just tracking a UTXO set, but tracking an UTXO augmented with expirability information
182 2012-09-03 10:44:57 <killerstorm> sipa: can't you just wait like 10 confirmations? if there is 10-deep reorg, well, it's a problem...
183 2012-09-03 10:45:11 <sipa> killerstorm: why would i want to wait 10 confirmations before spending my money?
184 2012-09-03 10:45:33 <amiller> sipa, that's easy :p
185 2012-09-03 10:45:46 <sipa> amiller: but you can't expect anyone to do it for you
186 2012-09-03 10:45:48 <kjj_> ok, how about we back up a bit. what do you hope to gain by having scripts aware of their position in the blockchain?
187 2012-09-03 10:46:36 <killerstorm> see above: person can get money locked in escrow forever.
188 2012-09-03 10:46:45 <amiller> sipa, how it would work is you would have a 'work-timed event' index in the utxo table
189 2012-09-03 10:47:11 <amiller> when miners publish a block, they execute all the transactions scheduled for that range of work
190 2012-09-03 10:47:17 <sipa> killerstorm: you know about nLockTime ?
191 2012-09-03 10:48:03 <killerstorm> Yes. but I guess there is a reason why no contracts are implemented so far.
192 2012-09-03 10:48:20 <killerstorm> Or are they?
193 2012-09-03 10:49:32 <sipa> if you're going to use that as an argument, i'll ask you what you think will be implemented first: contracts or the more complicated infrastructure that is necessary for a bitcoin-like system which provides time-dependent scripts?
194 2012-09-03 10:49:33 <killerstorm> Well I was wondering about theoretic problems with OP_TIME/OP_WORK/whatever.
195 2012-09-03 10:49:53 <killerstorm> I can implement time-dependent scripts in about a hour.
196 2012-09-03 10:50:04 <killerstorm> It's a trivial patch, I think.
197 2012-09-03 10:50:17 <sipa> will you also write the more complicated logic needed to decide whether an incoming payment is trustable?
198 2012-09-03 10:50:36 <sipa> as you need to verify none of its ancestry depends on time-dependant outputs
199 2012-09-03 10:50:38 <killerstorm> No, I just trust payments which have enough confirmations.
200 2012-09-03 10:51:01 <sipa> so you're essentially removing the ability to accept 0-conf payments from trusted sources?
201 2012-09-03 10:51:37 <killerstorm> If source is trusted you can also trust it to not accept a timebomb.
202 2012-09-03 10:52:00 <killerstorm> If reorg comes we just start over.
203 2012-09-03 10:52:12 <sipa> there's a difference between trusting a source not to be malicious, and the source not being stupid
204 2012-09-03 10:52:41 <amiller> sipa, you have the same problem currently
205 2012-09-03 10:52:44 <killerstorm> Ok, I see, these are valid concerns.
206 2012-09-03 10:52:50 <sipa> amiller: how so?
207 2012-09-03 10:53:00 <amiller> if you take a 0-conf payment from someone who you trust not to be malicious, but who might be stupid,
208 2012-09-03 10:53:12 <amiller> then in their stupidity, they may have sent you a coin they just received, and that coin can be double-spent out from under them
209 2012-09-03 10:53:28 <killerstorm> Your trusted source would send you money which later turns out to be a double-spend after reorg.
210 2012-09-03 10:53:32 <amiller> which also makes your received coin invalid, even though the stupid mark didn't do anything deliberately wrong
211 2012-09-03 10:53:57 <sipa> good point
212 2012-09-03 10:55:01 <sipa> of course you can limit yourself to only accept 0-conf transactions from a trusted source, whose txins have enough confirmations already (or are known to come from the same source)
213 2012-09-03 10:55:22 <sipa> but TD knows more about these use cases
214 2012-09-03 10:56:20 <killerstorm> Who is TD?
215 2012-09-03 10:56:24 <sipa> Mike Hearn
216 2012-09-03 10:56:39 <sipa> (who wrote the contracts and smart property pages)
217 2012-09-03 10:56:48 <sipa> and BitcoinJ
218 2012-09-03 10:57:54 <sipa> eh?
219 2012-09-03 10:59:20 <kjj_> meh. transactions not caring which blocks they end up in is a very good thing. shouldn't be thrown out casually
220 2012-09-03 11:00:30 <amiller> the main thing i would want to be able to do is for the validation rule to condition on the other txins and txouts in the transaction
221 2012-09-03 11:00:51 <amiller> actually that's exactly what you would need to implement the 'mark' 'color' and 'taint' kinds of rules
222 2012-09-03 11:01:02 <sipa> well, there's no problem with transactions being able to observe their inputs
223 2012-09-03 11:01:24 <sipa> their outputs/decendents is more tricky
224 2012-09-03 11:02:00 <amiller> you wouldn't want to accept a bitcoin as payment unless it were unencumbered and had no expiry (or at least a very very long expiry)
225 2012-09-03 11:02:07 <kjj_> do mark, color and tain mean what I think they mean?
226 2012-09-03 11:02:21 <kjj_> er, taint
227 2012-09-03 11:02:46 <sipa> amiller: agree
228 2012-09-03 11:03:03 <sipa> if there's an easy way to verify those properties, there is not really a problem
229 2012-09-03 11:04:42 <amiller> it shouldn't be any harder than the ancestry check that we should probably be doing now anyway
230 2012-09-03 11:05:19 <sipa> we are doing that ancestry check
231 2012-09-03 11:05:25 <sipa> as it is binary: valid or not
232 2012-09-03 11:05:28 <kjj_> please don't bloat the system by checking anything more than the math. plenty of us think that "taint" is a really retarded idea
233 2012-09-03 11:05:43 <sipa> kjj_: it's not about taint
234 2012-09-03 11:06:05 <sipa> (taint is similar implementation-wise, but not what they want to achieve)
235 2012-09-03 11:07:09 <amiller> sipa, i mean ancestry as in the number of confirmations 'behind' a 0-conf-but-trusted-source kind of thing
236 2012-09-03 11:08:08 <sipa> right
237 2012-09-03 14:02:50 <arij> hello
238 2012-09-03 14:03:12 <arij> there is a project which i need a developer for
239 2012-09-03 14:03:17 <arij> i am paying in btc
240 2012-09-03 14:03:25 <arij> please message me for more details
241 2012-09-03 14:30:22 <arij> any freelance php coders offering their services?
242 2012-09-03 14:30:24 <arij> please pm me
243 2012-09-03 14:34:39 <kjj_> what's your project?
244 2012-09-03 14:48:23 <epscy> it's like facebook
245 2012-09-03 14:48:28 <epscy> but with bitcoins
246 2012-09-03 14:48:41 <epscy> anyone interested?
247 2012-09-03 15:09:16 <has_many> so bitcoins post there their cat and baby pictures?
248 2012-09-03 15:25:04 <MC1984> intredasting question
249 2012-09-03 15:25:37 <MC1984> what is the longest password that could be cracked using the total hashpower expended in the chain so far
250 2012-09-03 15:26:56 <MC1984> or if that is retarded, hash collisions or something
251 2012-09-03 15:27:09 <MC1984> inb4 protons and the universe and shit
252 2012-09-03 15:27:14 <amiller> MC1984, that follows directly from the 'total work' value of the chain
253 2012-09-03 15:27:36 <gmaxwell> MC1984: "longest password" ??? potentially gigabytes long. The length of a password only sets an upper bound on the entropy.
254 2012-09-03 15:28:20 <MC1984> ok longest realistic pw
255 2012-09-03 15:28:51 <gmaxwell> If you were instead to ask for "lentgh of a uniformly chosen random key" or something you'd have what amiller was answering.
256 2012-09-03 15:28:51 <sipa> the answer is: a bit less than 71 bits of entropy
257 2012-09-03 15:29:06 <gmaxwell> we're up to 71 bits now? impressive!
258 2012-09-03 15:29:12 <sipa> no, close to 70
259 2012-09-03 15:29:18 <sipa> but we're doing double SHA256 :)
260 2012-09-03 15:29:21 <MC1984> gmaxwell i only have the faintest idea what im even asking
261 2012-09-03 15:29:31 <MC1984> sipa is a boss tho, how do you work it out?
262 2012-09-03 15:29:46 <gmaxwell> log2(work number from logs)
263 2012-09-03 15:30:42 <MC1984> it sounds like someone has asked about this before
264 2012-09-03 15:30:56 <sipa> MC1984: assuming we have no better information about how people chose their passwords than just assuming every byte string is equally likely, 71 bits of entropy means that something of 9 bytes is potentially already our of reach
265 2012-09-03 15:31:31 <sipa> MC1984: if we limit ourselves to let's say alphanumeric passwords, case sensitive, 71 bits means up to 12 characters
266 2012-09-03 15:31:45 <sipa> if you use a real model for how humans choose passwords: very long
267 2012-09-03 15:32:27 <MC1984> yeah humans u suck
268 2012-09-03 15:33:21 <MC1984> wow the password paradigm is fucked isnt it
269 2012-09-03 15:33:27 <MC1984> like within 10 years for sure
270 2012-09-03 15:33:54 <MC1984> yeah you can make strong ones, but people dont
271 2012-09-03 15:34:48 <MC1984> whats the next step
272 2012-09-03 15:35:02 <sipa> i was wrong by the way; the total work in the chain is around 2^68.64
273 2012-09-03 15:35:26 <MC1984> yea we wrked this out last night
274 2012-09-03 15:35:40 <sipa> 4 octoquintillion bihashes!
275 2012-09-03 15:35:46 <sipa> eh
276 2012-09-03 15:35:50 <sipa> 4 hectoquintillion bihashes!
277 2012-09-03 15:36:08 <kjj_> exponentials sneak up on you
278 2012-09-03 15:36:54 <MC1984> whats funny is that that will look like piss in the ocean compared to whats coming
279 2012-09-03 15:37:02 <sipa> Maybe.
280 2012-09-03 15:37:58 <MC1984> whats the projection for when bfl get off thier asses
281 2012-09-03 15:38:05 <MC1984> 200thash?
282 2012-09-03 15:39:23 <sipa> at 200 Thash/s, it would take 26 days to do as much work as the entire chain did up to now
283 2012-09-03 15:40:07 <kjj_> the fun part is that it just adds 1 bit. and the next bit will take two more months
284 2012-09-03 15:40:40 <sipa> that's certainly true, but i fail to see the humor :)
285 2012-09-03 15:43:06 <kjj_> the easy bits are done.
286 2012-09-03 15:52:38 <TD> is anyone doing anything about the pirate spam?
287 2012-09-03 15:52:46 <jgarzik> ?
288 2012-09-03 15:53:10 <sipa> people in #bitcoin and #bitcoin-otc are being spammed in PM by some goatser
289 2012-09-03 15:53:42 <Evilmax> i too
290 2012-09-03 15:53:45 <Evilmax> yes
291 2012-09-03 15:53:50 <Evilmax> me too
292 2012-09-03 15:54:14 <sipa> quick solution: /umode +R
293 2012-09-03 15:54:23 <sipa> (block PM from unregistered nicks)
294 2012-09-03 15:54:36 <Evilmax> /mode yournick +R
295 2012-09-03 15:54:43 <Evilmax> i have solved
296 2012-09-03 15:54:44 <TD> ah ha
297 2012-09-03 15:54:52 <TD> or just /part #bitcoin
298 2012-09-03 15:56:20 <jgarzik> yeah, /mode yournick +R is the solution for me... they are appearing on all different IP addresses, so you cannot even ban the IP address :/
299 2012-09-03 15:57:40 <sipa> i've seen people talk about it in #freenode, so i suppose at least some are aware of the problem
300 2012-09-03 16:00:42 <MC1984> problem is he got like 40btc on the first address he put up
301 2012-09-03 16:00:57 <sipa> seriously?
302 2012-09-03 16:01:01 <MC1984> or maybe he was a dumbass and used a previous address and thats why he then changed it
303 2012-09-03 16:01:53 <MC1984> s just some skid
304 2012-09-03 16:07:17 <MC1984> http://blockchain.info/address/12SBihiWqxWRqkdU7zaGSuxhCyvcU6Mg8B this is the original address that was in the spam
305 2012-09-03 16:09:07 <Optimo> is that spammer still on the loose?
306 2012-09-03 16:10:30 <MC1984> http://blockchain.info/address/1GoatSe6pWoeTXugzofi7QUG3SFskFH2yp this is his new vanity one
307 2012-09-03 16:10:46 <Optimo> yeah.. that's why I left a few hours ago lol
308 2012-09-03 16:11:04 <smiddi> there are people really paying? wtf
309 2012-09-03 16:11:25 <MC1984> ha he seems to be manually sending between these two in order to make ascii bytes after the decimal like that other hack
310 2012-09-03 16:11:32 <MC1984> shit is adorable if not original
311 2012-09-03 16:15:45 <jrmithdobbs> gmaxwell: ban that tiggr faggot already
312 2012-09-03 16:16:17 <copumpkin> there are nicer ways to put that
313 2012-09-03 16:16:25 <MC1984> oh its not ascii
314 2012-09-03 16:17:39 <MC1984> 46 aa 86 96 b2 9e 55 this is the hex
315 2012-09-03 16:19:58 <jrmithdobbs> Tiggr: fix your shit moron
316 2012-09-03 16:36:44 <Karmaon> lol
317 2012-09-03 18:08:11 <Diapolo> sipa: May I as another network related question ^^?
318 2012-09-03 18:08:14 <Diapolo> +k
319 2012-09-03 18:09:52 <Diapolo> sipa: SetProxy() contains an address.IsValid() check, but before that happens there is another !addrProxy.IsValid() check for the initError ... why not just use the return value from SetProxy() instead?
320 2012-09-03 18:13:01 <sipa> i suppose because it SetProxy can fail for another reason
321 2012-09-03 18:13:07 <sipa> but i don't really care
322 2012-09-03 18:14:36 <Diapolo> SetProxy only fails when a wrong SOCKS version is supplied, for which an earlier check is in init :D
323 2012-09-03 18:17:13 <btctrader22> Hi guys
324 2012-09-03 18:17:38 <btctrader22> Question: is there a list of current implementations of bitcoin sumwhere ?
325 2012-09-03 18:18:29 <sipa> what are you looking for? a library? a node? an implementation of the validation engine?
326 2012-09-03 18:19:39 <sipa> a wallet? a GUI?
327 2012-09-03 18:19:54 <c_k> btctrader22: http://bitcoin.org/clients.html
328 2012-09-03 18:20:05 <btctrader22> ideally, some sort of wiki page listing projects that implement some sort of bitcoin related fuctionality that I can go plunder.
329 2012-09-03 18:20:14 <btctrader22> Preferably in C, can live with C++
330 2012-09-03 18:20:40 <btctrader22> Something with minimal dependencies would also be nice
331 2012-09-03 18:21:51 <btctrader22> eh, just found http://en.bitcoin.it/wiki/Software. sorry for the noise.
332 2012-09-03 18:27:58 <MC1984> swiftcoinwalletinstaller.exe
333 2012-09-03 18:28:04 <MC1984> ok this is shady as fuck
334 2012-09-03 18:30:51 <btctrader22> oOoo. Just found bitcoinjs-lib. yum.
335 2012-09-03 18:40:37 <MC1984> no really what is this swiftcoin shit
336 2012-09-03 18:41:29 <btctrader22> s/swiftcoinwalletinstaller/mailmywallettoukraine.exe/
337 2012-09-03 18:42:35 <MC1984> website is cagey as hell
338 2012-09-03 18:42:37 <MC1984> fuck this shit
339 2012-09-03 18:52:54 <Eliel> MC1984: If it's a wallet stealer, it's very well camouflaged. The data-file structure it creates is complex enough that you wouldn't think someone would do that just to make a wallet stealer look more credible.
340 2012-09-03 18:53:17 <Eliel> my friend checked it out in a virtual machine.
341 2012-09-03 18:53:52 <MC1984> still shady as shit
342 2012-09-03 18:53:55 <sipa> TD: benchmarking leveldb+ultraprune :)
343 2012-09-03 18:54:01 <TD> sweet
344 2012-09-03 18:54:10 <MC1984> i was on thier site 20 mins and i have no idea what theyre even trying to do
345 2012-09-03 18:56:42 <sipa> TD: cpu usage >100% is a nice sign
346 2012-09-03 18:57:01 <TD> i tended to see the second cpu core spend around 80% of its time in iowait
347 2012-09-03 18:57:13 <sipa> that's expected, i suppose
348 2012-09-03 18:57:15 <TD> yes
349 2012-09-03 18:57:41 <sipa> 4 minutes for 161k blocks
350 2012-09-03 18:58:01 <sipa> but i think the speedup will be mostly at the end
351 2012-09-03 18:58:37 <MC1984> shit doesnt get real until block 180k
352 2012-09-03 18:58:44 <sipa> indeed
353 2012-09-03 18:59:18 <MC1984> im under 1000 blocks now
354 2012-09-03 18:59:39 <MC1984> its basically choking on every block like they were 11 inch dicks
355 2012-09-03 19:02:17 <sipa> 7m32s for 185k blocks... good
356 2012-09-03 19:03:40 <sipa> 9m51s for 193k blocks
357 2012-09-03 19:04:49 <sipa> and now it's just 100% CPU, to do sig checking
358 2012-09-03 19:05:51 <MC1984> gogogogogo
359 2012-09-03 19:16:45 <sipa> TD: hardly any difference between -dbcache=25 and -dbcache=200
360 2012-09-03 19:17:13 <TD> i think the primary thing to tweak is bloom filter size
361 2012-09-03 19:17:27 <TD> more bits == fewer disk seeks to find data but that dataset gets larger, pretty much
362 2012-09-03 19:17:29 <TD> so less fits into ram
363 2012-09-03 19:17:37 <TD> the current setting was chosen by me arbitrarily
364 2012-09-03 19:17:41 <TD> i suspect it can be smaller
365 2012-09-03 19:17:49 <sipa> well, i suppose the database at this point is so small it's just cached by the OS entirely
366 2012-09-03 19:18:10 <sipa> i'd need a low-memory setting to see the effects of a bloom filter vs cache size
367 2012-09-03 19:25:56 <TD> db size i suspect fluctuates a lot
368 2012-09-03 19:26:06 <TD> as whenever something is deleted it gets a "deletion marker" inserted into the db
369 2012-09-03 19:26:14 <TD> the data actually gets removed during compaction
370 2012-09-03 19:26:27 <TD> but it's good to know that it got faster
371 2012-09-03 19:26:32 <TD> what sort of times were you seeing at the end with bdb?
372 2012-09-03 19:26:58 <sipa> i think i remember close to 20 min for what took 7m here
373 2012-09-03 19:27:17 <sipa> but i'll benchmark again, because other things changed in my code too
374 2012-09-03 19:27:19 <TD> good
375 2012-09-03 19:27:23 <TD> ok
376 2012-09-03 19:29:16 <sipa> also: database size seems a lot smaller than with bdb
377 2012-09-03 19:30:35 <sipa> TD: regarding bloom filter size; those bloom filters are also stored on disk, or in memory?
378 2012-09-03 19:31:00 <gmaxwell> sipa: how big?
379 2012-09-03 19:31:23 <gmaxwell> (hopefully we'll still get reduction once a tx index is reintroduced then!)
380 2012-09-03 19:31:38 <TD> in memory
381 2012-09-03 19:31:51 <TD> http://leveldb.googlecode.com/svn/trunk/doc/index.html
382 2012-09-03 19:32:00 <sipa> gmaxwell: fluctuation between 99 and 120 MiB
383 2012-09-03 19:32:04 <TD> it says
384 2012-09-03 19:32:05 <TD> "We recommend that applications whose working set does not fit in memory and that do a lot of random reads set a filter policy."
385 2012-09-03 19:32:22 <TD> you're correct that right now it probably makes no difference, or even makes things worse
386 2012-09-03 19:32:38 <TD> the real question is what should it be after working set size > most nodes available ram
387 2012-09-03 19:32:41 <sipa> that may just mean that the filter is on disk, but is assumed to remain in the OS cache or LevelDB cache
388 2012-09-03 19:33:10 <sipa> gmaxwell: in BDB it was over 160 MiB
389 2012-09-03 19:33:34 <sipa> (i know there's around 85-90 MB of actual data in there)
390 2012-09-03 19:34:42 <sipa> ;;bc,blocks
391 2012-09-03 19:34:43 <gribble> 197082
392 2012-09-03 19:34:46 <TD> yeah i'd have to check the code
393 2012-09-03 19:34:53 <TD> the filters are stored in sstable blocks
394 2012-09-03 19:34:58 <TD> and i think those blocks are typically mmapped
395 2012-09-03 19:35:04 <TD> so yes. it may assume that the OS file cache stores it
396 2012-09-03 19:35:08 <sipa> right
397 2012-09-03 19:36:38 <sipa> well, ultraprune's database entries are around 80 bytes on average (of which 32 are key), 10 bits means a 1.5% increase
398 2012-09-03 19:39:20 <TD> hmm
399 2012-09-03 19:39:31 <TD> for a 100x reduction in disk seeks when the db gets larger, that may be worth it
400 2012-09-03 19:39:32 <sipa> ok, final size: 102 MB coindb, 29 MB blockdb
401 2012-09-03 19:39:57 <sipa> that's MiB, actually
402 2012-09-03 19:40:38 <sipa> and the full sync (including a part from network) took 33 minutes
403 2012-09-03 19:40:52 <Diablo-D3> [05:32:05] <TD> "We recommend that applications whose working set does not fit in memory STOP DOING THAT RIGHT NOW"
404 2012-09-03 19:40:54 <Diablo-D3> ftfy
405 2012-09-03 19:41:19 <TD> sipa: cheers :)
406 2012-09-03 19:41:32 <sipa> now, let's try that again with BDB
407 2012-09-03 19:45:17 <MC1984> so do all these improvements bring the it to the point where future technology outstrips projected blockchain growth
408 2012-09-03 19:45:38 <TD> projected blockchain growth?
409 2012-09-03 19:45:42 <sipa> without a blockchain fork, the blockchain growth is quite limited actually, anyway
410 2012-09-03 19:45:47 <TD> isn't that the million bitcoin question :-)
411 2012-09-03 19:45:57 <sipa> as blocks are max 1 MB, and contain max 20000 sigops
412 2012-09-03 19:46:42 <MC1984> hmm 1mb every 10 mins
413 2012-09-03 19:46:49 <MC1984> forever
414 2012-09-03 19:46:53 <MC1984> seems doable
415 2012-09-03 19:48:34 <MC1984> i wonder if the 1mb limit + minsers subsidy halving willl kick start the fees market
416 2012-09-03 19:48:41 <sipa> TD: by the way, i just have a singleton instance of the database objects left, so no need for all the global pdb object stuff
417 2012-09-03 19:49:37 <MC1984> already 0 fee txs dont seem to be getting into first available block
418 2012-09-03 19:50:09 <sipa> that's mostly because there are enough fee-paying transactions available to mine, i guess
419 2012-09-03 19:50:19 <TD> great
420 2012-09-03 19:50:33 <TD> MC1984: en.bitcoin.it/wiki/Scalability
421 2012-09-03 19:50:43 <TD> also see the thread on the forum about theoretical max ECDSA speed
422 2012-09-03 19:50:59 <TD> ultraprune+leveldb are a good start but there are many more improvements that can be made
423 2012-09-03 19:51:16 <TD> i'm starting to think that a single high-end server could outpace all of VISA and MasterCard combined with sufficient software optimization
424 2012-09-03 19:52:27 <sipa> 7m instead of 4m for the first 161k blocks
425 2012-09-03 19:52:52 <edcba> theoretical max speed of ecdsa ?
426 2012-09-03 19:53:02 <edcba> oh before being broken too easily i guess
427 2012-09-03 19:53:20 <sipa> edcba: no, it's about practical implementations
428 2012-09-03 19:53:35 <sipa> the current OpenSSL-based code is far from the most performant ECDSA-verify code
429 2012-09-03 19:53:36 <edcba> but cpu aren't really limited in speed
430 2012-09-03 19:53:52 <sipa> i'm not sure the word "theoretical" is rightly placed there
431 2012-09-03 20:01:35 <TD> hmm
432 2012-09-03 20:01:38 <TD> cpus ARE limited in speed
433 2012-09-03 20:01:52 <TD> sipa: ok so it's looking like maybe a slightly less than 2x speedup?
434 2012-09-03 20:02:22 <sipa> TD: i think the speedup if higher in the later parts of the chain
435 2012-09-03 20:02:49 <TD> ok
436 2012-09-03 20:03:57 <sipa> looks like it's going to be 10min vs 19min for 193k blocks
437 2012-09-03 20:04:21 <sipa> and the speedup is probably largely dependent on the ratio CPU/IO speed
438 2012-09-03 20:05:24 <TD> not anywhere near as significant as ultraprune, but i can live with a 2x speedup
439 2012-09-03 20:05:25 <TD> :)
440 2012-09-03 20:05:51 <sipa> actually, good idea; i'll do the same benchmark with git head
441 2012-09-03 20:06:08 <sipa> i have no idea how that compares anymore
442 2012-09-03 20:06:54 <gmaxwell> certantly much larger than that.
443 2012-09-03 20:07:17 <sipa> i do feel that batch block connection during IBD is cheating, though :)
444 2012-09-03 20:07:27 <sipa> (though it may not matter too much on leveldb)
445 2012-09-03 20:16:20 <btctrader22> so, after much looking around at various implementations ... looks like linking against satoshi's client bits and pieces of code is still the easiest.
446 2012-09-03 20:16:42 <sipa> btctrader22: what do you want to do?
447 2012-09-03 20:16:58 <btctrader22> question for devs familiar with the satoshi codebase : is it safe to link directly with ,cpp files from the satoshi code base
448 2012-09-03 20:17:10 <btctrader22> or is there some init sequence to ensure the code behaves ?
449 2012-09-03 20:17:44 <sipa> yeah, init.cpp is kinda humongous
450 2012-09-03 20:17:46 <btctrader22> sipa: various things. first thing I coded was something that generates a list of N (pub,priv) keypairs in plain bsae58
451 2012-09-03 20:18:30 <btctrader22> sipa: humongous maybe, but ... doesn't really answer my question. I'm mostly interested in low-level address manipulation stuff.
452 2012-09-03 20:18:39 <btctrader22> Don't particularly care about the block chain data
453 2012-09-03 20:18:52 <sipa> oh, just use base58.h and key.cpp/.h
454 2012-09-03 20:19:04 <btctrader22> But would like to be able to eventually construct arbitrary TXs
455 2012-09-03 20:19:32 <btctrader22> sipa: yeah, just did that. But I have no clue if there isn't some hidden init sequence I missed somwhere
456 2012-09-03 20:20:51 <edcba> if you want to do some stuff easily best option is to run stock client to validate everything and send commands with your own client connected to it
457 2012-09-03 20:21:37 <btctrader22> edcba: nope. standard client is lame in that it works with one wallet and in many places seems to assume the blockchain data is available.
458 2012-09-03 20:22:01 <btctrader22> edcba: also, the RPC interface is horribly limiting.
459 2012-09-03 20:22:17 <edcba> that's why i say connect to it using bitcoin protocol
460 2012-09-03 20:22:36 <sipa> btctrader22: blockchain data is actually only need for serving blocks, reorganising and rescanning
461 2012-09-03 20:22:40 <edcba> you don't have to implement validation since stock client will handle that for you
462 2012-09-03 20:22:52 <edcba> and forward you only valid stuff
463 2012-09-03 20:22:53 <btctrader22> edcba: what I want to build, is a bunch of small binaries, shell-style that perform elementary operations and that can be glued together via shell scripts
464 2012-09-03 20:23:48 <btctrader22> ecdba : ???? Can I generate 100 (pub,priv) keypairs using the native bitcoin protocol ? I don't think so.
465 2012-09-03 20:24:21 <sipa> through RPC? no
466 2012-09-03 20:24:27 <btctrader22> sipa: yeah, but that was the essence of my question. It's rather unclear which part of code actually require what data to be around.
467 2012-09-03 20:24:33 <sipa> but that's a task much more easily done with a library
468 2012-09-03 20:24:42 <btctrader22> sipa:neither RPC nor protocol.
469 2012-09-03 20:25:00 <edcba> of course since it's quite oblivious to bitcoin protocol
470 2012-09-03 20:25:02 <sipa> what protocol apart from RPC are you talking about?
471 2012-09-03 20:25:23 <sipa> keypairs is purely a local wallet-related task
472 2012-09-03 20:25:39 <btctrader22> sipa: the one edcba mentions, which I guess is the client to client p2p stuff
473 2012-09-03 20:25:59 <sipa> oh no, of course
474 2012-09-03 20:26:33 <sipa> i'd say interact with bitcoind for anything that does need broadcasting on the p2p network or validating transactions, or information about blocks
475 2012-09-03 20:27:37 <btctrader22> sipa: mmh. Can I create an arbitrary wallet.dat from a list of addresses using bitcoind ? Don't think so. The code can do that though.
476 2012-09-03 20:28:08 <sipa> using importprivkey you can
477 2012-09-03 20:28:25 <edcba> sipa: while running bitcoind ?
478 2012-09-03 20:28:27 <btctrader22> sipa: nope. I'd have to restart the client each time.
479 2012-09-03 20:28:33 <sipa> btctrader22: no you don't
480 2012-09-03 20:28:52 <btctrader22> sipa: and importprivkey is dogslow because of the dumb decision of forcing a rescan at each import
481 2012-09-03 20:29:08 <sipa> yeah, should make that optional
482 2012-09-03 20:29:33 <sipa> or finish importwallet
483 2012-09-03 20:29:44 <edcba> the problem is bitcoin client hasn't been well architectured
484 2012-09-03 20:29:50 <btctrader22> sipa: how can I possibly create multiple wallets with a running instance of bitcoind ??? (I mean without restart)
485 2012-09-03 20:30:00 <btctrader22> edcba: I'd strongly agree with that statement.
486 2012-09-03 20:30:10 <sipa> btctrader22: oh, multiple wallets? not yet
487 2012-09-03 20:30:22 <sipa> yeah, refactorings get low priority
488 2012-09-03 20:30:31 <sipa> though there's certainly improvement
489 2012-09-03 20:30:40 <edcba> not an easy task anyway
490 2012-09-03 20:30:45 <btctrader22> and so, while waiting for it to become better architected, I have to link directly agianst bits and pieces of the code to get it to do what I want.
491 2012-09-03 20:31:04 <sipa> btctrader22: though it shouldn't be too hard to have multiple wallets;
492 2012-09-03 20:31:14 <sipa> create multiple CWallet objects, backed by different files
493 2012-09-03 20:31:15 <btctrader22> hence my original question about the what needs to get executed before various bits and pieces of the codebase become functional
494 2012-09-03 20:31:43 <sipa> i can certainly help when you have specific problems with running things, but it's kinda hard to say this and this
495 2012-09-03 20:31:52 <btctrader22> sipa: ahah. now we're getting to where I'm alread at: linking against the codebase directly. My init question stands.
496 2012-09-03 20:32:04 <edcba> want multiple wallets ? run multiple vms !
497 2012-09-03 20:32:07 <btctrader22> *initial
498 2012-09-03 20:32:15 <sipa> i and just answered it :)
499 2012-09-03 20:32:16 <btctrader22> edcba: LOL
500 2012-09-03 20:32:20 <TD> sipa: a 10 minute initial sync is definitely something that would be neat to demo at the conf
501 2012-09-03 20:32:32 <btctrader22> edcba: you're clearly the simple and lightweight solution dude.
502 2012-09-03 20:32:36 <TD> i wonder how hard it'd be to whip up a cooler way of presenting the sync
503 2012-09-03 20:32:40 <TD> like a pie chart or something
504 2012-09-03 20:32:42 <edcba> also don't use vmware since bitcoin client share the vmware web administration console...
505 2012-09-03 20:32:46 <edcba> port
506 2012-09-03 20:32:51 <edcba> so handy
507 2012-09-03 20:33:11 <sipa> TD: reverse-header sync would be nice, as it'd allow a progressbar that counts the number of transactions imported instead of the number of blocks :)
508 2012-09-03 20:33:25 <sipa> but i think i want some refactoring first... :S
509 2012-09-03 20:33:27 <btctrader22> sipa: so ... the answer is : no one really knows and I've got to experiment. OK, fair enough.
510 2012-09-03 20:33:28 <TD> one thing at a time
511 2012-09-03 20:33:30 <edcba> i think satoshi is working for oracle
512 2012-09-03 20:33:44 <sipa> yeah
513 2012-09-03 20:33:47 <edcba> using bdb and preventing to use vmware
514 2012-09-03 20:33:52 <TD> right now just getting 0.8 with ultraprune + leveldb + bloom filtering would be a big achievement
515 2012-09-03 20:34:01 <TD> i suspect the bottleneck right now is gavin feeling overwhelmed with review work
516 2012-09-03 20:34:08 <TD> piling even more on top .... :/
517 2012-09-03 20:34:17 <sipa> yes, i think so
518 2012-09-03 20:34:19 <TD> anyway, multi-threading sig checking seems like the next biggest win
519 2012-09-03 20:34:35 <edcba> i hope he mined enough coins to be full time dev on bitcoin
520 2012-09-03 20:35:47 <sipa> gmaxwell once suggested having a "turbo edition" based on recent improvements, being a bit more experimental
521 2012-09-03 20:36:06 <gmaxwell> 15:23 < btctrader22> ecdba : ???? Can I generate 100 (pub,priv) keypairs using the native bitcoin protocol ? I don't think so.
522 2012-09-03 20:36:13 <gmaxwell> Via the rpc? sure you can.
523 2012-09-03 20:36:41 <sipa> not without adding them to a wallet
524 2012-09-03 20:36:57 <gmaxwell> No, indeed, but did I miss that requirement? :)
525 2012-09-03 20:37:05 <btctrader22> gmaxwell: nope. You missed the start of the thread. I wanted the (pub,priv) keypairs to juts be dumped in bse58. not added to wallet.
526 2012-09-03 20:37:13 <gmaxwell> ah, okay.
527 2012-09-03 20:37:14 <btctrader22> Anyway I got that part working.
528 2012-09-03 20:37:27 <btctrader22> Apparently, that prt of the code seems to work standalone.
529 2012-09-03 20:37:35 <btctrader22> We'll see if the rest holds together as well
530 2012-09-03 20:37:43 <btctrader22> for example the CWallet stack
531 2012-09-03 20:38:10 <sipa> mostly the tx validation / block chain management is very intertwined
532 2012-09-03 20:38:35 <TD> ACTION -> bed
533 2012-09-03 20:38:50 <gmaxwell> 15:35 < sipa> gmaxwell once suggested having a "turbo edition" based on recent improvements, being a bit more experimental
534 2012-09-03 20:39:09 <gmaxwell> mostly I'm concerned that leveldb and ultraprune have significant, difficulty to test, network partitioning risks.
535 2012-09-03 20:39:12 <btctrader22> sipa: ok. like I said, going to experiment, we'll see what breaks.
536 2012-09-03 20:39:43 <gmaxwell> but there is a huge swath of nodes (e.g. end user clients) which both need the speedup and which aren't harmed greatly if there is some partitioning issue.
537 2012-09-03 20:40:18 <btctrader22> gmaxwell: just out of curiosity, what will the upgrade path to ultraprune+leveldb look like for joe user ?
538 2012-09-03 20:40:33 <gmaxwell> btctrader22: please report issues we find, we do want to further isolate hunks of the code, eventually making it possible to run the wallet standalone (/remotely) from the blockchain entry.
539 2012-09-03 20:40:55 <sipa> btctrader22: at startup, it'd import your existing blockchain files
540 2012-09-03 20:41:02 <gmaxwell> btctrader22: a long initial startup that rebuilds/reindexes local data.
541 2012-09-03 20:41:04 <sipa> as the database layout is incompatible
542 2012-09-03 20:41:16 <btctrader22> gmaxwell: will do as I find them. Items : The global fTestNet us butt-ugly. So is that macro that replaces printf.
543 2012-09-03 20:41:27 <sipa> agree
544 2012-09-03 20:42:08 <sipa> btctrader22: i dare you to look at the 0.3.19 code, which had wallet handling code mixed inside main :)
545 2012-09-03 20:42:21 <btctrader22> Not to mentui that uiInterface thing that's dragged in when you link in utils.cpp
546 2012-09-03 20:42:31 <btctrader22> *mention
547 2012-09-03 20:42:58 <gmaxwell> btctrader22: there is a ton of hideous global stuff, but that isn't news.
548 2012-09-03 20:43:33 <btctrader22> gmaxwell: I'm sure you guys noticed them, yeah :D
549 2012-09-03 20:44:27 <sipa> damn, around 190k the speed of git head becomes soo slow...
550 2012-09-03 20:45:13 <btctrader22> Next stop : small app to read an ascii list of (base58,Pub, base58Priv, Comment), make a wallet.dat out of it. Then the other way round.
551 2012-09-03 20:45:27 <btctrader22> See if that works.
552 2012-09-03 20:50:37 <sipa> gmaxwell: git head: 41m34s, ultraprune: 19m, ultraprune+leveldb: 10m
553 2012-09-03 20:50:49 <sipa> for 193k blocks
554 2012-09-03 21:00:53 <btctrader22> ugh. linking in wallet.cpp and dependencies drags in a list of globals half a mile long
555 2012-09-03 21:01:21 <sipa> if you want the wallet code, you need just about everything
556 2012-09-03 21:01:28 <btctrader22> yup
557 2012-09-03 21:01:34 <btctrader22> just figuring that out now
558 2012-09-03 21:01:57 <gmaxwell> btctrader22: right; but the question is does it work if you don't call any of that stuff, don't have p2p up, or a blockchain, etc.
559 2012-09-03 21:02:20 <sipa> i think it'll work
560 2012-09-03 21:02:25 <btctrader22> I know. Got a link a binary before I get to answering that question
561 2012-09-03 21:02:56 <btctrader22> sipa: don't know the code base well enough to make a prediction on that. gut feeling given what I've seen is not optimistic
562 2012-09-03 21:03:34 <sipa> the wallet pulls in the code from main, because it can create transactions that need broadcasting, and because it inspects the blockchain for confirmation
563 2012-09-03 21:03:41 <gmaxwell> btctrader22: well, we've certantly been moving in a direction that would allow that, but I expect some issues because I'm not aware of anyone else doing it before.
564 2012-09-03 21:04:06 <sipa> but the keystore management and transaction creation should work fine without any active node or block database
565 2012-09-03 21:04:18 <sipa> actually, the unit tests do that
566 2012-09-03 21:04:35 <btctrader22> exploratory work, to say the least. but goes to show the initial client code was slapped together in a rather ad-hoc fashion.
567 2012-09-03 21:04:56 <sipa> well, i advise against looking at what saoshi wrote initially :p
568 2012-09-03 21:05:01 <btctrader22> sipa: that's rather good news. I mean the fact that parts of the code are unit-testable.
569 2012-09-03 21:05:01 <sipa> +t
570 2012-09-03 21:05:17 <btctrader22> sipa: :D
571 2012-09-03 21:05:26 <gmaxwell> btctrader22: Well come now, what digital currencies have you invented? Be nice. It's a rather small package, so the fact that it's not a paragon of object oriented design isn't the worst sin in the world.
572 2012-09-03 21:05:45 <sipa> to me it looks like much of this was an experimental version
573 2012-09-03 21:05:57 <sipa> that would get replaced in later stages
574 2012-09-03 21:06:23 <btctrader22> I hate OO design. But that not doing OO design shouldn't prevent one from being an architect. That being said I'm in awe of what was built here. Just the guys wasn't a code, that's all.
575 2012-09-03 21:06:25 <btctrader22> *coder
576 2012-09-03 21:06:42 <sipa> btctrader22: you may want to have a look at libcoin; it's a nice refactoring of the satoshi code (though certainly less tested)
577 2012-09-03 21:07:21 <btctrader22> sipa: yeah, I looked at it. I'm loathe to use a fork of the satoshi code if what I built ever moves real coins around.
578 2012-09-03 21:07:39 <btctrader22> sipa: exactly for the reason you mention.
579 2012-09-03 21:10:56 <sipa> btctrader22: to give you an idea, before satoshi left, we didn't have wallet*, addrman*, sync*, netbase*, protocol*, keystore*, rpc*
580 2012-09-03 21:11:54 <btctrader22> I see. One big source file. The way K&R intended :D
581 2012-09-03 21:11:58 <sipa> wallet, protocol and keystore were done in main; sync in util; netbase in net; address management was less sophisticated but also in net
582 2012-09-03 21:12:52 <btctrader22> yeah, I've noticed that the code was slowly getting in better shape. Great work.
583 2012-09-03 21:13:39 <sipa> and I really hope that after ultraprune, i can split off a coins.*, blktree.* and mempool.* from main
584 2012-09-03 21:14:53 <btctrader22> in an utopian world, the client should be built on top of a bunch of independent libraries ... one for crypto, one for the p2p stuff, one for wallets, etc ...
585 2012-09-03 21:15:00 <btctrader22> but then, utopias are just that.
586 2012-09-03 21:15:20 <sipa> well, they certainly help for seeing a way forward :)
587 2012-09-03 21:15:49 <btctrader22> ok. I gotta get that wallet dumper done. ttyl y'all
588 2012-09-03 21:19:50 <btctrader22> One would have thought main() was in main.cpp. One would be wrong :)
589 2012-09-03 21:20:27 <sipa> Too straightforward.
590 2012-09-03 21:20:42 <btctrader22> lol
591 2012-09-03 21:20:57 <gmaxwell> meh. Well. I don't know that thats utopian either. Excessive indirection reduces reliablity, decreases performance, and increases memory usage. There is some right level of abstraction and isolation.
592 2012-09-03 21:20:58 <sipa> oh, UI update code was already in main.cpp!
593 2012-09-03 21:21:14 <sipa> *also
594 2012-09-03 21:22:07 <btctrader22> I'm getting there. Only code patch to client code thus far is #ifdef'ing out main()
595 2012-09-03 21:22:56 <edcba> main.cpp were where the main things is done :)
596 2012-09-03 21:23:30 <sipa> main.cpp should be called blockchainmanagementandmempoolandblockandtransactionverificationandalertsandsomemiscstuff
597 2012-09-03 21:23:41 <btctrader22> can't wait for the day when the whole notion of files just goes away and code becomes just code.
598 2012-09-03 21:24:05 <edcba> neumann says code is data
599 2012-09-03 21:24:55 <btctrader22> neumann is a dead old geezer.
600 2012-09-03 21:32:19 <btctrader22> it links !!! it's alive an it made a CWallet. OooOo.
601 2012-09-03 22:42:47 <Luke-Jr> btctrader22: moving real coins is pretty safe to do with custom code; it's accepting coins that's risky
602 2012-09-03 22:44:10 <btctrader22> it looks like using the CWallet code directly, even if used on a tmp copy of wallet.dat actually corrupts the original wallet.dat file. ugh.
603 2012-09-03 22:44:24 <Luke-Jr> that being said, one of the biggest services (MtGox) uses completely custom code
604 2012-09-03 22:44:49 <btctrader22> I'm starting to understand why now that I look at the client codebase.
605 2012-09-03 23:00:05 <gmaxwell> btctrader22: you absolutely can not have multiple things concurrently accessing wallet.dat (e.g. your tool plus a running client). Or any of the other database files, but this is due to bdb, not bitcoin's code.
606 2012-09-03 23:01:48 <btctrader22> gmaxwell: All I do is create a CWallet("someotherfile.dat"). One would expect the original wallet.dat to be left alone. Wrong assumption. I'm going to specify a separate, blank datadir. Hopefully this'll behave.
607 2012-09-03 23:02:28 <gmaxwell> btctrader22: it has nothing to do with the wallet.dat file and everything to do with the shared datbase enviroment. Yes, it'll behave with a seperate datadir.
608 2012-09-03 23:03:01 <gmaxwell> you may have also corrupted your blockchain while doing that, so if your node gets stuck then that may be why.
609 2012-09-03 23:03:28 <btctrader22> gmaxwell: not true. I straced my min-program who does nothing except CWallet wallet("somefile.dat"). It opens wallet.dat (the original one) RDWR