1 2013-08-16 02:32:11 <gmaxwell> so ... we're asking for the wallet passphrase every time someone requests a new address in -qt.
2 2013-08-16 02:32:21 <gmaxwell> Do none of the technical people here use QT? :P
3 2013-08-16 02:34:04 <CodeShark> I use it only for development :p
4 2013-08-16 02:34:44 <warren> gmaxwell: same thing for importprivkey ...
5 2013-08-16 02:34:52 <gmaxwell> So??? guesses on preferred behavior: if the pool is empty, rueuse or prompt then?
6 2013-08-16 02:35:04 <gmaxwell> warren: importprivkey needs to. uh but thats not a gui feature.
7 2013-08-16 02:35:19 <warren> importprivkey needs it why?
8 2013-08-16 02:35:30 <CodeShark> pool should never be filled without explicitly prompting the user to make a backup or doing an automated backup :)
9 2013-08-16 02:36:48 <CodeShark> https://github.com/bitcoin/bitcoin/pull/2841
10 2013-08-16 02:37:05 <gmaxwell> warren: because it needs to encrypt the key thats being imported, and we made a intentional design decision to not enable people stuffing in extra keys into an encrypted wallet without a key.
11 2013-08-16 02:37:16 <warren> ahh ok
12 2013-08-16 02:37:18 <CodeShark> not sure why it had been closed
13 2013-08-16 02:37:18 <gmaxwell> (e.g. if they could add to your keypool they could make your change go to an address they control)
14 2013-08-16 02:37:36 <gmaxwell> CodeShark: you closed it!
15 2013-08-16 02:37:44 <CodeShark> yeah, not sure why I closed it :p
16 2013-08-16 02:38:07 <gmaxwell> so, opinions on QT behavior? reuse or prompt on empty pool?
17 2013-08-16 02:38:30 <CodeShark> prompt
18 2013-08-16 02:38:54 <CodeShark> and never autofill without backup
19 2013-08-16 02:39:15 <warren> reuse is only a risk to privacy, and security if bad PRNG?
20 2013-08-16 02:39:44 <CodeShark> if PRNG is bad then the bitcoin-qt wallet is already screwed anyhow
21 2013-08-16 02:42:03 <gmaxwell> warren: it's a risk to security given an attack on ECDSA, and it really impressively craps on privacy (since it leads to all of your addresses in your wallet being linked), more than you might assume.
22 2013-08-16 02:42:13 <gmaxwell> Not the end of the world, but we should prefer to avoid it.
23 2013-08-16 02:42:59 <gmaxwell> privacy has a lot of wrinkles too, like losing your privacy can help thevies better trick you because they can figure out whos rich and who they have to impersonate.
24 2013-08-16 02:43:24 <CodeShark> behavior should be to alert the user when their pool is running low so they can fill it and make a backup (and filling and backup should require the passphrase)
25 2013-08-16 02:43:43 <CodeShark> and disallow transactions when pool is empty
26 2013-08-16 02:43:55 <gmaxwell> CodeShark: for the amount of code required there, we should probably move to determinstic by default wallets.
27 2013-08-16 02:44:10 <CodeShark> that's the only other sane option :)
28 2013-08-16 02:45:32 <CodeShark> even with deterministic wallets, though, we still need to pregenerate
29 2013-08-16 02:45:52 <CodeShark> the difference is that it only takes one master key to recover the sequence
30 2013-08-16 02:45:57 <gmaxwell> well, if they're type 2, we don't... but yea.. I prefer our default wallets be type 1.
31 2013-08-16 02:47:40 <CodeShark> and we should keep the change key sequence separate from the receiving key sequence
32 2013-08-16 02:47:42 <CodeShark> two separate pools
33 2013-08-16 02:48:03 <CodeShark> we don't necessarily need to go full hierarchicl
34 2013-08-16 02:48:06 <CodeShark> *hierarchical
35 2013-08-16 02:50:40 <CodeShark> also, even though the sequence of keys is predetermined, backups are still necessary to keep track of which keys have been issued, account labels, etc...
36 2013-08-16 02:51:17 <gmaxwell> CodeShark: you've read bip32 right?
37 2013-08-16 02:51:27 <gmaxwell> it's designed specifically to enable keeping those sequences seperate.
38 2013-08-16 02:51:29 <CodeShark> I contributed to that document :p
39 2013-08-16 02:51:49 <CodeShark> that's why I said we don't necessarily have to go full hierarchical in bitcoin-qt right now
40 2013-08-16 02:51:59 <CodeShark> it was in reference to the BIP0032 document
41 2013-08-16 02:53:03 <gmaxwell> okay, didn't read completely because I was coding at the same time.
42 2013-08-16 02:55:26 <CodeShark> the change sequence utilization can be derived from the block chain, assuming all transactions are instantly broadcast (and not, say, given to the recipient out-of-band)
43 2013-08-16 02:55:37 <CodeShark> I'm not so sure about this assumption
44 2013-08-16 02:56:03 <CodeShark> but a wallet state could still be fully rendered by just keeping track of the last key used in each sequence
45 2013-08-16 02:56:10 <CodeShark> at least as far as key utilization
46 2013-08-16 02:56:12 <gmaxwell> yea, not a great assumption, see the payment protocol ppull.
47 2013-08-16 02:56:32 <gmaxwell> but if you get mindwiped you could just skip a couple to avoid reuse.
48 2013-08-16 02:56:56 <CodeShark> so with a master key and an index for each sequence you could reconstruct the wallet state (less account labels)
49 2013-08-16 02:56:58 <gmaxwell> also, an implemention could have a wallet id and always skip max_ids.
50 2013-08-16 02:57:20 <gmaxwell> but you'll never get your notes / labels/ recepts/ etc. so you still do need backups.
51 2013-08-16 03:00:04 <CodeShark> those things should probably be possible to store separate from the master key/index information
52 2013-08-16 03:01:59 <CodeShark> with type 2 keys, you could associate labels with sets of indices - with type 1 keys, you'd have to store full addresses
53 2013-08-16 03:03:47 <CodeShark> would be nice to automate replication
54 2013-08-16 03:04:16 <CodeShark> ensure that all the user's data is either saved to two separate devices always - or that it's backed up passphrase-encrypted on some servers somewhere
55 2013-08-16 03:05:02 <gmaxwell> just to make the code cheaper and simpler, might as well just store the actual keys in both cases.
56 2013-08-16 03:06:23 <CodeShark> asymmetric encryption for backups, perhaps
57 2013-08-16 03:07:30 <gmaxwell> yea. having a distinct encrypted backup might be nice... the user factors are a bit tricky.
58 2013-08-16 03:07:43 <gmaxwell> the whole wallet is encrypted.. unless you copy the file on disk instead of a backup.
59 2013-08-16 03:10:44 <CodeShark> the user could point their backup to, say, an ftp server and have it automatically encrypted
60 2013-08-16 03:12:30 <gmaxwell> one issue is that users will, quite reliably, use weak keys. And then they'll point it to a public ftp server and lose the only protection they have (attacker doesn't have the wallet)
61 2013-08-16 03:13:30 <CodeShark> ideally, the recovery key should be of high entropy and stored on some removable device
62 2013-08-16 03:14:06 <gmaxwell> yea, but the notion of a backup that isn't enough to recover your key might really confuse people. (and result in dumb behavior like not keeping the recovery key)
63 2013-08-16 03:14:42 <gmaxwell> armory prompts you to print a recovery key, I think we should do that. and indeed, if they've done that we could make those remote backups secure.
64 2013-08-16 03:14:53 <CodeShark> the idea is that you only need to go through the process of generating this key and storing it securely once
65 2013-08-16 03:15:22 <CodeShark> you still need to make regular backups, but with asymmetric encryption there's no need to expose your master recovery key to the outside world
66 2013-08-16 03:15:34 <gmaxwell> an interesting idea too is to just have it be some anonymous file storage service that you just pay for with bitcoin in your wallet.
67 2013-08-16 03:16:33 <CodeShark> I've had several such ideas for using cryptocurrency for IT resource allocation
68 2013-08-16 03:16:52 <CodeShark> not just storage
69 2013-08-16 03:17:14 <CodeShark> yeah, it's an interesting idea indeed
70 2013-08-16 03:18:20 <CodeShark> that's a whole discussion unto itself (or several of them)
71 2013-08-16 03:19:00 <gmaxwell> yep. In any case, if you think users really can handle a seperate backup from recovery code, then indeed, I think thats a spiffy idea.
72 2013-08-16 03:20:25 <gmaxwell> so, interesting case earlier. User gets the content of his bitcoin-qt wallet stolen.
73 2013-08-16 03:20:34 <gmaxwell> Running windows. Malware infected host.
74 2013-08-16 03:20:53 <gmaxwell> However, his wallet was encrypted. It was encrypted on another host which hasn't been on in months. (OSX host)
75 2013-08-16 03:21:12 <gmaxwell> he insists he never entered the key at all on the windows host, just used it to check the balance here and there.
76 2013-08-16 03:21:58 <gmaxwell> the key he was using seems secure, 28 characters long, and while obviously not randomly generated, it doesn't ??? in my expirence??? seem like something that would at all be easy to crack.
77 2013-08-16 03:22:26 <gmaxwell> I got a pre-compromise copy of his wallet, indeed its encrypted. No evidence of leaked private keys in there.
78 2013-08-16 03:23:19 <gmaxwell> his bitcoin.conf had the rpc setup (just user and password, no server=1 or allow lines). I quizzed him good and can't find anything that would have caused him to set that up.
79 2013-08-16 03:23:33 <gmaxwell> though his debug log doesn't indicate that the coins were taken over the rpc in any case.
80 2013-08-16 03:24:29 <CodeShark> this really happened?
81 2013-08-16 03:24:44 <CodeShark> or hypothetical?
82 2013-08-16 03:24:57 <gmaxwell> yep, it's real... just a bit ago.
83 2013-08-16 03:25:35 <gmaxwell> Obvious answer is that the user is mistaken, that he really did type the key in. Even ninjas such as myself could be mistaken about such a thing.
84 2013-08-16 03:26:03 <gmaxwell> but it seems weird. Other people with similar sounding malware (some java thing) have made similar reports in the past.
85 2013-08-16 03:26:35 <gmaxwell> But none of the prior ones had claimed that they'd never typed the key in on the system in question.
86 2013-08-16 03:27:07 <gmaxwell> I'm contemplating putting a large amount of btc in an encrypted wallet with an impossible key, and posting it on the forum.
87 2013-08-16 03:28:09 <CodeShark> top hypotheses: 1) he did enter the key into the compromised system, 2) his passphrase was guessed or compromised, 3) weak PRNG, 4) the "uncompromised" system was actually compromised as well
88 2013-08-16 03:29:10 <CodeShark> i.e. he might have used the passphrase somewhere else
89 2013-08-16 03:29:19 <gmaxwell> I think (1) is most likely though the only supporting evidence is "even competent users make mistakes", (2) may be the case if his password has structure obvious to an attack with access to his system and not me.
90 2013-08-16 03:29:46 <gmaxwell> He says he used it on nothing else except a litecoin wallet, (whos coins weren't stolen btw) generated and handled in exactly the same manner.
91 2013-08-16 03:30:09 <gmaxwell> (3) perhaps, but if so we would expect massive thefts on windows systems.
92 2013-08-16 03:30:26 <gmaxwell> and (5) is we have a private key information leak. (again)
93 2013-08-16 03:30:41 <gmaxwell> ohh.. lemme try to get his database log files from him, perhaps the private keys are in there.
94 2013-08-16 03:30:42 <CodeShark> (3) seems unlikely because the thefts would tend to be for some keys, not necessarily all, and across many different wallets
95 2013-08-16 03:31:18 <CodeShark> assuming a nondeterministic wallet, that is
96 2013-08-16 03:31:52 <CodeShark> do you know the nature of the malware?
97 2013-08-16 03:31:54 <gmaxwell> point, they did hit several keys. And if it were the case I think we'd see enormous thefts, not just single users.
98 2013-08-16 03:32:15 <CodeShark> wallet stealer? key logger?
99 2013-08-16 03:32:40 <gmaxwell> CodeShark: no, on some prior user is exploited a JVM vulnerability and installed some background java task that did god knows what, this guy said it was "some java program" and had removed it.
100 2013-08-16 03:33:00 <CodeShark> ironic - java is supposed to be "safe" :p
101 2013-08-16 03:34:01 <gmaxwell> darn, he says he copied only the wallet.dat, so the idea of leaking via the database log directory is out.
102 2013-08-16 03:36:36 <gmaxwell> I've asked for the exact file off the mac in case it was from a leaky version of bitcoin... but probably won't get it soon.
103 2013-08-16 03:37:08 <gmaxwell> Its frustrating that users don't have flight recorders on them. I hate that its so easy to cop out on what might be a subtle and serious bug and say "ah, probably user error"
104 2013-08-16 03:38:16 <CodeShark> worst case scenario, I suppose, is a break in the wallet encryption algo
105 2013-08-16 03:39:02 <CodeShark> seems unlikely, but still possible
106 2013-08-16 03:39:11 <CodeShark> perhaps some subtle weakness in OpenSSL's implementation
107 2013-08-16 03:39:12 <CodeShark> lol
108 2013-08-16 03:39:23 <gmaxwell> CodeShark: nah, other people have written decrypters for it.
109 2013-08-16 03:39:37 <CodeShark> ok
110 2013-08-16 03:39:47 <gmaxwell> unless there is something crazy like a data leak in extra space in the field...
111 2013-08-16 03:39:55 <CodeShark> so then barring a side-channel attack, such a break would imply a break in the actual cipher
112 2013-08-16 03:40:46 <CodeShark> where could the data leak possibly be?
113 2013-08-16 03:40:49 <gmaxwell> yea, which is AES.
114 2013-08-16 03:41:12 <gmaxwell> CodeShark: there really isn't room for a leak unless its something like bdb seralizing random bits of the heap into the files.
115 2013-08-16 03:41:37 <gmaxwell> (and, well, I wouldn't be totally shocked if that happened, BDB is not entirely valgrind clean)
116 2013-08-16 03:41:38 <Vinnie_win> Hey guys how's it goin?
117 2013-08-16 03:41:49 <CodeShark> hello, vinnie
118 2013-08-16 03:42:30 <Vinnie_win> Just put up a nice fat update to beast
119 2013-08-16 03:42:37 <Vinnie_win> Got that socket wrapper working!!!!
120 2013-08-16 03:42:52 <CodeShark> I've been doing asio stuff too, lately - let me take a look at your socket wrapper
121 2013-08-16 03:43:17 <Vinnie_win> CodeShark: Its in modules/beast_asio/sockets
122 2013-08-16 03:43:25 <Vinnie_win> although I'm not sure what your use-case would be
123 2013-08-16 03:43:46 <CodeShark> general interest in async I/O and metaprogramming
124 2013-08-16 03:43:57 <Vinnie_win> Well there's plenty of metaprogramming thats for sure
125 2013-08-16 03:44:02 <Vinnie_win> I was swimming in it
126 2013-08-16 03:45:42 <Vinnie_win> There's some handshake detection logic in there too, for building a socket that talks multiple protocols
127 2013-08-16 03:46:19 <CodeShark> you mean it switches at runtime?
128 2013-08-16 03:46:43 <Vinnie_win> CodeShark: The detect logic is in beast. The "MultiSocket" is in ripple
129 2013-08-16 03:47:27 <Vinnie_win> CodeShark: but you could build it yourself from the components in beast, just wrap an asio::socket with an asio::buffered_read_stream and pass the bytes into my detector. once it figures out what you have, then construct the appropriate object
130 2013-08-16 04:19:33 <Luke-Jr> seems everyone is thinking of the "NODE_NETWORK without NODE_BLOOM" use case and usefulness; but "NODE_BLOOM without NODE_NETWORK" might in fact be more relevant O.o
131 2013-08-16 04:21:58 <gmaxwell> Luke-Jr: yes, that indeed might be relevant.
132 2013-08-16 04:28:20 <warren> Luke-Jr: do you plan on backporting the CVE rework and mining node crash to 0.8 stable?
133 2013-08-16 04:28:34 <warren> (you're the backport maintainer right?)
134 2013-08-16 04:28:47 <Luke-Jr> warren: "CVE rework and mining node crash"?
135 2013-08-16 04:28:48 <Luke-Jr> yes
136 2013-08-16 04:30:02 <warren> actually, hwat's the difference between your backport releases and 0.8.3?
137 2013-08-16 04:30:24 <warren> Luke-Jr: why aren't the backport releases on bitcoin.org?
138 2013-08-16 04:32:03 <gmaxwell> Because they're only supported by luke (and whomever cares to help him) and not by the reset of the core team. We don't recommend them, except as a last ditch option to someone who really cannot upgrade.
139 2013-08-16 04:32:20 <warren> ah
140 2013-08-16 04:32:43 <Luke-Jr> even I don't recommend them :p
141 2013-08-16 04:32:48 <warren> so this means no 0.8.4 for those two fixes? it seems 0.9 still is a while away.
142 2013-08-16 04:32:57 <Luke-Jr> there is no excuse for a new user (ie, someone on bitcoin.org) to use a backport
143 2013-08-16 04:33:09 <Luke-Jr> warren: if it's merged to master, it will probably get a backport
144 2013-08-16 04:33:18 <warren> Luke-Jr: both are now merged to master
145 2013-08-16 04:33:22 <Luke-Jr> assuming it is in fact a fix
146 2013-08-16 04:33:49 <warren> if 0.8.4 happens, I have a tree of recommended other backports that we've been using in production for 2+ months
147 2013-08-16 04:33:58 <gmaxwell> I
148 2013-08-16 04:34:10 <Luke-Jr> warren: I have a 0.8.x branch already, just not published at the moment
149 2013-08-16 04:34:14 <gmaxwell> I can't imagine why we'd don that for the mining node crash.
150 2013-08-16 04:34:24 <Luke-Jr> gmaxwell: ?
151 2013-08-16 04:34:28 <gmaxwell> there is a trivial workaround, don't run with debug=1.
152 2013-08-16 04:34:29 <warren> well, if it's happening, let's talk about what backports to include in 0.8.4
153 2013-08-16 04:34:35 <warren> we've been testing them in production
154 2013-08-16 04:34:53 <warren> gmaxwell: eh? none of those nodes had debug=1
155 2013-08-16 04:34:58 <warren> gmaxwell: (that's the confusing part)
156 2013-08-16 04:35:20 <gmaxwell> warren: they did.
157 2013-08-16 04:35:52 <gmaxwell> We confirmed in here everyone that crashed had debugging enabled. One person who said they didn't showed a debug log that clearly did.
158 2013-08-16 04:38:05 <warren> oh
159 2013-08-16 04:38:12 <warren> in that case, do we really need to patch this?
160 2013-08-16 04:38:36 <Luke-Jr> it certainly doesn't sound like a lone justification for 0.8.4
161 2013-08-16 04:39:02 <gmaxwell> it's a buggy case that shouldn't have happened, and with debug enabled it kicks nodes into an assert (in code that otherwise handles the bad case)
162 2013-08-16 04:39:04 <warren> how serious is that CVE rework?
163 2013-08-16 04:39:11 <gmaxwell> but no reason to make a 0.8.4.
164 2013-08-16 04:39:19 <Luke-Jr> warren: define "CVE rework"
165 2013-08-16 04:39:25 <Luke-Jr> CVE is a numbering system
166 2013-08-16 04:39:25 <warren> ACTION finds
167 2013-08-16 04:40:03 <gmaxwell> He's talking about the long tx truncation thing.
168 2013-08-16 04:40:16 <Luke-Jr> the one 0.8.3 fixed?
169 2013-08-16 04:40:20 <warren> Luke-Jr: https://github.com/bitcoin/bitcoin/pull/2871
170 2013-08-16 04:41:57 <gmaxwell> warren: nah, nothing to rerelase there.
171 2013-08-16 04:44:35 <Luke-Jr> warren: do you have someone to do Mac builds btw?
172 2013-08-16 04:49:36 <Luke-Jr> ACTION can't believe his 0.7.2 node hasn't hardforked yet
173 2013-08-16 04:54:05 <warren> Luke-Jr: yes
174 2013-08-16 04:54:10 <warren> Luke-Jr: why?
175 2013-08-16 04:54:20 <Luke-Jr> warren: stable has no Mac builder
176 2013-08-16 04:54:32 <Luke-Jr> as Gavin has better things to do
177 2013-08-16 04:55:08 <warren> Luke-Jr: can you throw some coins at our mac dev? he'll do it, but he's kind of poor
178 2013-08-16 04:55:24 <warren> he dedicated one entire mac for clean builds
179 2013-08-16 04:56:25 <Luke-Jr> warren: I'll think about it; who is it? One problem with Mac builds is that they aren't deterministic, so there's a huge amount of trust needed
180 2013-08-16 04:56:49 <warren> Luke-Jr: yeah, we have that same amount of huge trust on him
181 2013-08-16 04:57:10 <warren> Luke-Jr: he's a personal friend that I've known for 3 years, old school openbsd hacker
182 2013-08-16 04:58:36 <warren> Luke-Jr: He's worked on cross compilers WAY back, and he thinks with funding he can make linux-based deterministic cross-compiled mac build.
183 2013-08-16 04:58:39 <Luke-Jr> warren: no name? ;p
184 2013-08-16 04:58:48 <warren> Luke-Jr: Rama McIntosh (face)
185 2013-08-16 04:59:33 <Luke-Jr> warren: I collected some donations toward gitian Mac target a while back; I could probably forward that
186 2013-08-16 04:59:47 <warren> Luke-Jr: he's in the room here talking about gitian mac, he wants all previous notes on it
187 2013-08-16 05:00:06 <Luke-Jr> I can provide that too
188 2013-08-16 05:00:13 <gmaxwell> Luke-Jr: torproject is gitianing for osx now.
189 2013-08-16 05:00:20 <Luke-Jr> gmaxwell: successfully?
190 2013-08-16 05:00:24 <gmaxwell> I thought so.
191 2013-08-16 05:01:22 <Luke-Jr> warren: notes at http://codepad.org/xieBaqeT
192 2013-08-16 05:01:27 <Luke-Jr> warren: code at https://gitorious.org/cross-osx/cross-osx/trees/master
193 2013-08-16 05:02:05 <Luke-Jr> warren: this worked up until 0.6 ish, when the GCC used no longer supported enough C++11 for Bitcoin
194 2013-08-16 05:03:00 <Luke-Jr> likely needs to be redone with LLVM
195 2013-08-16 05:03:22 <warren> hm
196 2013-08-16 05:03:31 <warren> I'll let him know.
197 2013-08-16 05:03:48 <warren> Luke-Jr: our dev fund will throw some into this goal too, but we can't be the only funder.
198 2013-08-16 05:04:07 <Luke-Jr> warren: I can probably contribute something around 25 BTC
199 2013-08-16 05:05:27 <Luke-Jr> warren: not myself personally, but from previous donations toward that goal
200 2013-08-16 05:05:59 <warren> ok
201 2013-08-16 05:06:37 <warren> Luke-Jr: it seems people want linux-based cross-compile so anyone can build it, but would mac-hosted deterministic be acceptable?
202 2013-08-16 05:07:33 <Luke-Jr> warren: as long as the guest OS is free software, it should work with any host..
203 2013-08-16 05:08:15 <warren> ok, so gitian-like
204 2013-08-16 05:08:44 <Luke-Jr> well, I'd like to assert for-gitian
205 2013-08-16 05:09:13 <warren> ok
206 2013-08-16 05:09:36 <Luke-Jr> if that means redefining gitian to an equivalent software package that does the same thing, that's fine too
207 2013-08-16 05:09:36 <warren> Luke-Jr: can you please send me an e-mail with all relevant details?
208 2013-08-16 05:10:17 <Luke-Jr> warren: email?
209 2013-08-16 05:11:44 <warren> Luke-Jr: PM
210 2013-08-16 05:12:50 <gmaxwell> bitcoin sociopaths make me sad: https://bitcointalk.org/index.php?topic=274749.msg2943846#msg2943846
211 2013-08-16 05:14:42 <warren> I thought we had only rational people.
212 2013-08-16 05:15:55 <gmaxwell> it's so bitcoin that the bad guy hangs around and tells theymos how he's going to dos attack him.
213 2013-08-16 05:28:40 <warren> Luke-Jr: "End binaries should be backward compatible to at least Mac OS X 10.5"
214 2013-08-16 05:29:00 <Luke-Jr> warren: it's my understanding that's what we have today.
215 2013-08-16 05:29:03 <warren> Luke-Jr: our builds worked on 10.5 but mysteriously crashed after a few minutes of uptime
216 2013-08-16 05:29:19 <warren> Luke-Jr: only one user complained about it, indicating how few users are using it
217 2013-08-16 05:29:25 <Luke-Jr> warren: if Gavin says we can drop 10.5 support, that's fine
218 2013-08-16 05:29:57 <warren> Luke-Jr: our builds are supposed to work on 10.5 but mysteriously fail. both 0.6 and 0.8 have the same issue.
219 2013-08-16 05:30:00 <Luke-Jr> in fact, you may want to get Gavin to comment on requirements too
220 2013-08-16 05:30:28 <Luke-Jr> warren: also, just because people don't use scamcoins with 10.5 does not infer they don't use Bitcoin :p
221 2013-08-16 05:31:05 <warren> Luke-Jr: I'm sure this work will be copied by the 300 scamcoins
222 2013-08-16 05:31:12 <Luke-Jr> warren: irrelevant
223 2013-08-16 05:31:23 <Luke-Jr> warren: btw might also check with Freicoin - they have a development fund too
224 2013-08-16 05:31:31 <warren> Luke-Jr: ok
225 2013-08-16 05:31:37 <Luke-Jr> and a completely different userbase than Bitcoin, AIUI
226 2013-08-16 05:32:36 <Luke-Jr> [07:31:01] <warren> Luke-Jr: I'm sure this work will be copied by the 300 scamcoins <-- my point is that you cannot infer that one 10.5 user of Litecoin says ANYTHING about 10.5 users of Bitcoin
227 2013-08-16 05:33:40 <Luke-Jr> personally, if Apple has stopped supporting 10.5, I'd say "upgrade or stfu", but I think that needs Gavin's ok
228 2013-08-16 05:37:32 <warren> Luke-Jr: OTOH, we have had only one 10.5 user complain
229 2013-08-16 05:37:35 <warren> so maybe it's fine
230 2013-08-16 05:37:46 <Luke-Jr> warren: ???
231 2013-08-16 05:38:01 <warren> we're using pretty much the same mac build procedure that gavin uses
232 2013-08-16 05:38:03 <Luke-Jr> warren: "we" being Litecoin. Not Bitcoin.
233 2013-08-16 05:38:36 <warren> Luke-Jr: yeah yeah, I get that you're hostile to Litecoin. Get over it. we have common problems to fix in Bitcoin.
234 2013-08-16 05:38:51 <Luke-Jr> warren: you have to admit that Litecoin's userbase says nothing about Bitcoin's..
235 2013-08-16 05:39:22 <warren> Luke-Jr: I don't have to admit anything. I don't care about ancient mac users. If it can be fixed then fine.
236 2013-08-16 05:39:55 <Luke-Jr> warren: Bitcoin might care. But again that's something to ask Gavin.
237 2013-08-16 05:40:41 <warren> It isn't clear that it is broken, given our statistically significant sample size of 1.
238 2013-08-16 05:41:18 <Luke-Jr> warren: k
239 2013-08-16 05:41:28 <Luke-Jr> it isn't really completed yet either afaik? :p
240 2013-08-16 05:41:55 <warren> what isn't completed?
241 2013-08-16 05:43:04 <Krellan> question about bitcoin-qt: when it puts up the prompt for "Are you sure you want to send X BTC..." has it calculated the TX fee yet? Or is that not known until it's ready to put the TX on the wire? Reason is, I want to try to add the TX fee to that prompt, so user can see the true total they are spending.
242 2013-08-16 05:44:33 <gmaxwell> Krellan: it could be known wherever. the fee can be fixed so long as its holding a lock on the status of the coins in the wallet.
243 2013-08-16 05:47:17 <Luke-Jr> warren: gitian OS X
244 2013-08-16 05:47:39 <Luke-Jr> Krellan: there is a pull request to change that
245 2013-08-16 05:48:50 <Krellan> Nice, there's already a pull request open? Cool, will save duplicate work. Do you have URL for that pull request?
246 2013-08-16 05:49:30 <warren> Luke-Jr: who are the key freicoin people to talk to?
247 2013-08-16 05:50:31 <Luke-Jr> warren: maaku mainly. there are others in #freicoin , but he's the project lead
248 2013-08-16 05:51:02 <Luke-Jr> Krellan: https://github.com/bitcoin/bitcoin/pull/2651 I think
249 2013-08-16 05:52:06 <warren> Luke-Jr: thanks
250 2013-08-16 06:08:33 <Luke-Jr> Is there a reason we're using Ripple's fork of LevelDB in master now? :|
251 2013-08-16 06:09:16 <Luke-Jr> which isn't even merging properly with upstream..
252 2013-08-16 06:09:46 <Luke-Jr> and has a "1.12" release that doesn't include upstream's 1.12 fixes
253 2013-08-16 06:22:46 <petertodd> Luke-Jr: so what do you see NODE_BLOOM w/o NODE_NETWORK meaning?
254 2013-08-16 06:23:12 <Luke-Jr> petertodd: I'm not sure ;)
255 2013-08-16 06:23:26 <Luke-Jr> perhaps relays transactions/blocks, but doesn't have a local store for them?
256 2013-08-16 06:23:45 <petertodd> Luke-Jr: as it is NODE_NETWORK is kinda imprecise, which doesn't help...
257 2013-08-16 06:24:45 <petertodd> For instance it could make sense for an SPV node to relay whole blocks, but then does it make sense for that node to then filter those blocks for SPV peers with even less bandwidth? The security assumption is the same, although practical security is probably weaker.
258 2013-08-16 06:34:03 <Krellan> I read that pull request, seems to make sense to me, it's 3 months old. Curious why it wasn't accepted into mainstream, or is it still in the process of being reviewed?
259 2013-08-16 07:02:19 <Luke-Jr> Krellan: yes
260 2013-08-16 07:46:27 <rubino123> how can I get pending tx data via json rpc
261 2013-08-16 07:51:59 <jouke> what do you mean by pending tx data?
262 2013-08-16 07:52:57 <rubino123> jouke: if an addressin the wallet was sent a sum of bitcoins how do I find how many confirms exist?
263 2013-08-16 07:55:33 <jouke> listreceivedbyaddress
264 2013-08-16 08:16:12 <Luke-Jr> 9bf2a4a Fix multi-block reorg transaction resurrection <-- anyone know what was broken? O.o
265 2013-08-16 08:18:27 <petertodd> Luke-Jr: nothing fancy, we were just adding transactions back to the memory pool in the wrong order
266 2013-08-16 08:19:34 <Luke-Jr> oh ok
267 2013-08-16 12:04:21 <jgarzik> mornin'
268 2013-08-16 12:12:01 <nsh> mornin jgarzik
269 2013-08-16 12:12:06 <nsh> how does the day find you?
270 2013-08-16 12:12:15 <jgarzik> well endowed
271 2013-08-16 12:12:19 <jgarzik> with USB block erupters
272 2013-08-16 12:12:33 <nsh> of the operational variety?
273 2013-08-16 12:12:39 <jgarzik> one hopes!
274 2013-08-16 12:12:45 <jgarzik> haven't plugged them in yet
275 2013-08-16 12:12:52 <jgarzik> but they are well packaged and fun to show off
276 2013-08-16 12:13:04 <nsh> sweet
277 2013-08-16 12:13:18 <nsh> please make a mining robot jgarzik
278 2013-08-16 12:13:29 <sipa> jgarzik: mine are still somewhere in california
279 2013-08-16 12:13:34 <jgarzik> what would a mining robot do? :)
280 2013-08-16 12:13:53 <nsh> it looks like a robot miner and it mines bitcoin and it follows you around telling you statistics
281 2013-08-16 12:13:55 <nsh> in a robot voice
282 2013-08-16 12:14:00 <jgarzik> I have been thinking about writing a pool server, zeromq bitcoind -> stratum-only pool server
283 2013-08-16 12:14:00 <nsh> and maybe it can fetch things
284 2013-08-16 12:14:01 <nsh> i don't know
285 2013-08-16 12:14:08 <jgarzik> like beer
286 2013-08-16 12:14:11 <nsh> yes
287 2013-08-16 12:14:18 <nsh> imagine that, it fetches you beer, while it's paying for your beer
288 2013-08-16 12:14:24 <nsh> that is like the perfect home companion
289 2013-08-16 12:14:27 <jgarzik> hah
290 2013-08-16 12:14:48 <nsh> "MASTER HAS EARNED ONE BEER. MINEBOT FETCH BEER!"
291 2013-08-16 12:15:28 <nsh> ACTION gets the drawing board out. this could be a very successful executive throw-money-at-me toy
292 2013-08-16 12:15:57 <sipa> ;;genrate 330
293 2013-08-16 12:15:58 <gribble> The expected generation output, at 330.0 Mhps, given difficulty of 50810339.0483, is 0.00326624764823 BTC per day and 0.00013609365201 BTC per hour.
294 2013-08-16 12:16:27 <sipa> $0.3 per day, that'd mean like one beer every two weeks here :)
295 2013-08-16 12:16:48 <nsh> lol
296 2013-08-16 12:17:55 <jgarzik> I tried to convince BitPat to group buy 1,000 of them, to give away at shows
297 2013-08-16 12:17:59 <jgarzik> BitPay
298 2013-08-16 12:18:02 <nsh> response?
299 2013-08-16 12:18:06 <jgarzik> laughter
300 2013-08-16 12:18:09 <nsh> lol
301 2013-08-16 12:18:35 <nsh> you need to use hypnosis maybe
302 2013-08-16 12:19:11 <Luke-Jr> jgarzik: why stratum-only? :/
303 2013-08-16 12:19:24 <sipa> just tell them that'll get them 30 beers a day _in switzerland_
304 2013-08-16 12:19:36 <jgarzik> Luke-Jr, alas, that is the direction of momentum
305 2013-08-16 12:19:59 <jgarzik> Luke-Jr, requires much less smarts in the miner -> path of least resistance
306 2013-08-16 12:20:18 <Luke-Jr> jgarzik: path to centralization
307 2013-08-16 12:20:25 <sipa> stratum is fine for the connection between miners and your own trusted server
308 2013-08-16 12:20:43 <jgarzik> ?and that's what we're talking about, indeed
309 2013-08-16 12:20:53 <Luke-Jr> oh, solo mining?
310 2013-08-16 12:21:10 <Luke-Jr> could just use GBT with longpolling in bitcoind
311 2013-08-16 12:23:29 <Luke-Jr> no need to reinvent things with 2 or 3 more layers of indirection :p
312 2013-08-16 12:23:54 <jgarzik> I dislike GBT from bitcoind. Would prefer to receive a full, binary block via zeromq
313 2013-08-16 12:25:07 <jgarzik> bitcoind sends binary block template >>>zeromq>>> pool server >>> stratum >>> miner
314 2013-08-16 12:25:19 <sipa> ok so do GBT over ZeroMQ?
315 2013-08-16 12:25:37 <sipa> instead of over RPC
316 2013-08-16 12:25:49 <jgarzik> sipa, yes, but without all the unneeded GBT bells&whistles
317 2013-08-16 12:25:56 <jgarzik> sipa, just a binary block
318 2013-08-16 12:26:06 <jgarzik> and without all the hex/unhex/hex encoding
319 2013-08-16 12:26:09 <Luke-Jr> ACTION wonders how that's different from GBT, other than skipping JSON nonsense
320 2013-08-16 12:27:10 <Luke-Jr> I looked at ZeroMQ for a possible mining protocol, but it needs some huge library :/
321 2013-08-16 12:27:11 <jgarzik> no need for the GBT metadata
322 2013-08-16 12:27:13 <jgarzik> just a block
323 2013-08-16 12:27:28 <jgarzik> "fee", "sigops", "hash", ? not needed. just send a binary block.
324 2013-08-16 12:27:56 <aspect_> question.. that would be a ZMQ "push", or rather pool server would SUB to the messages from bitcoind, right?
325 2013-08-16 12:28:38 <Luke-Jr> jgarzik: then you're having bitcoind do all the policy decisions ;)
326 2013-08-16 12:28:56 <jgarzik> Luke-Jr, invalid assumption
327 2013-08-16 12:29:11 <Luke-Jr> jgarzik: well, without a blockchain, you can't know the fees from a raw block
328 2013-08-16 12:29:58 <aspect_> jgarzik: pool would subscribe to messages from bitcoind, right?
329 2013-08-16 12:30:06 <jgarzik> aspect_, yes
330 2013-08-16 12:30:16 <jgarzik> Luke-Jr, correct
331 2013-08-16 12:30:21 <aspect_> if bitcoind fails, how does pool know that bitcoind went down?
332 2013-08-16 12:30:28 <Luke-Jr> jgarzik: without fee info, no good way to do policy decisions
333 2013-08-16 12:30:43 <jgarzik> Luke-Jr, no need for much policy in the pool server
334 2013-08-16 12:31:03 <Luke-Jr> perhaps
335 2013-08-16 12:31:05 <sipa> jgarzik: now i'm not following
336 2013-08-16 12:31:29 <sipa> jgarzik: you say Luke-Jr is wrong in saying that raw blocks means having bitcoind do all policy decisions
337 2013-08-16 12:31:38 <jgarzik> aspect_, standard zmq error handling
338 2013-08-16 12:31:38 <warren> aspect_: pool server implementations already detect RPC failure and have facilities to fallback to other defined nodes
339 2013-08-16 12:31:55 <Luke-Jr> warren: the topic is non-RPC stuff
340 2013-08-16 12:32:00 <warren> ok
341 2013-08-16 12:32:03 <sipa> jgarzik: but then confirm that the pool won't be doing any policy
342 2013-08-16 12:32:08 <jgarzik> sipa, I was saying that luke was making an invalid assumption that the policy stuff should be in the pool server, rather than bitcoind
343 2013-08-16 12:32:14 <sipa> ah
344 2013-08-16 12:32:26 <jgarzik> if you trust bitcoind, pool server can be quite dumb
345 2013-08-16 12:32:38 <sipa> sure, the point in GBT is removing that requirement
346 2013-08-16 12:32:47 <sipa> if the bitcoind is yours, you don't need the extra
347 2013-08-16 12:33:01 <jgarzik> nod
348 2013-08-16 12:33:08 <jgarzik> ACTION is not proposing /removing/ anything
349 2013-08-16 12:33:17 <jgarzik> GBT can stay, I'm just not going to use it :)
350 2013-08-16 12:33:33 <Luke-Jr> Policy decisions made in the poolserver have a number of benefits in other use cases :p
351 2013-08-16 12:33:36 <sipa> but it's trivial to implement what you want on top of bitcoind
352 2013-08-16 12:33:40 <sipa> via GBT
353 2013-08-16 12:34:45 <Luke-Jr> perhaps I'm wrong, but now I'm with the impression that jgarzik is doing this for fun/experiment more than improvement :p
354 2013-08-16 12:35:08 <sipa> ha
355 2013-08-16 12:35:26 <jgarzik> sipa, yes -- that describe an incredibly inefficient way to accomplish this
356 2013-08-16 12:35:34 <jgarzik> describes
357 2013-08-16 12:35:55 <sipa> jgarzik: sure, but does that matter?
358 2013-08-16 12:36:07 <jgarzik> sipa, IMO yes
359 2013-08-16 12:36:22 <Luke-Jr> considering that CreateNewBlock is already incredibly inefficient, I'd not be looking to optimize GBT away at this point even if I were trying to optimize
360 2013-08-16 12:37:36 <jgarzik> I would love to find a quick way to multi-thread access to the mempool in a way that does not involve locking the entire damn thing, during block creation
361 2013-08-16 12:37:52 <sipa> that's easy
362 2013-08-16 12:37:57 <sipa> reader/writer locks
363 2013-08-16 12:38:11 <Luke-Jr> jgarzik: petertodd has some changes to precalculate the priority sort
364 2013-08-16 12:40:37 <jgarzik> sipa, still blocks mempool addition (a frequent operation) for a long batch operation (pulling a copy of the entire mempool)
365 2013-08-16 12:42:32 <sipa> jgarzik: hmm, then i guess you'll need something like using individually-allocated mempool objects, with a refcount
366 2013-08-16 12:42:51 <sipa> jgarzik: so you only need to lock shortly, increase the refcounts, unlock, and you can construct the block
367 2013-08-16 12:43:12 <jgarzik> yep
368 2013-08-16 12:44:16 <jgarzik> Standard problem. Container entry/exit need synchronization, and individual items also need synchonization
369 2013-08-16 12:44:39 <jgarzik> Complex, but solutions are known
370 2013-08-16 12:45:05 <jgarzik> Have several data structures like this in the kernel -- so active that the entire container cannot remain locked for long periods.
371 2013-08-16 12:49:00 <Luke-Jr> hmm
372 2013-08-16 12:49:21 <Luke-Jr> is it just me, or did '159bc48 Simplify storage of orphan transactions' fix an unrelated bug accidentally? O.o
373 2013-08-16 12:50:46 <Luke-Jr> looks like the old code relayed the orphan-resolving transaction multiple times, and the new code relays it once PLUS the no-longer-orphaned transaction
374 2013-08-16 12:59:19 <BCB> ahh
375 2013-08-16 12:59:21 <BCB> ok
376 2013-08-16 12:59:33 <Diablo-D3> wouldnt that be funny of it did
377 2013-08-16 13:36:41 <Luke-Jr> ah, I see why I was confused there.
378 2013-08-16 13:36:48 <Luke-Jr> the non-conflicted part renamed stuff
379 2013-08-16 14:18:46 <bmcgee> hey if i want to know when difficulty has changed it's (height % 2016 == 0) ?
380 2013-08-16 14:19:23 <Luke-Jr> bmcgee: depends on how you define "changed"
381 2013-08-16 14:19:55 <Luke-Jr> bmcgee: is it after the last block at the old difficulty, or at the first block of the new? :p
382 2013-08-16 14:21:03 <bmcgee> let me think this through a bit, i've asked a few stupid questions of late lol
383 2013-08-16 14:21:09 <bmcgee> not interested in continuing the streak
384 2013-08-16 14:24:01 <midnightmagic> bmcgee: don't worry so much about that. of the people who would think less of you for asking a stupid question, whose opinion really matters if that's the kind of person they are?
385 2013-08-16 14:24:13 <bmcgee> ok so i poll bitcoind for the block template. Sometimes there's new transactions, other times a new block template. In the block template I see the height. Is it reasonable to simply check the height of each new block template and if it satisfies (height % 2016 == 0) then trigger a getmininginfo request to find out the new network difficulty. Or is it another case of polling getmininginfo
386 2013-08-16 14:24:35 <bmcgee> midnightmagic: don't get me wrong, I wont let it stop me. It would just be nice to get a break is all ;)
387 2013-08-16 14:25:49 <Luke-Jr> warren: #2871 was a pain to backport XD
388 2013-08-16 14:26:48 <Luke-Jr> bmcgee: getblocktemplate gives you the difficulty O.o
389 2013-08-16 14:27:11 <bmcgee> Luke-Jr: see, i just looked up the wiki page again before i asked it, i had a feeling????????????
390 2013-08-16 14:27:20 <bmcgee> couldn't see it
391 2013-08-16 14:27:24 <bmcgee> goddamn it...
392 2013-08-16 14:27:42 <Luke-Jr> bmcgee: bits
393 2013-08-16 14:28:40 <bmcgee> actually i feel retarded?????? of course, the difficulty for the block is the difficulty for the network??????. oh well glad i didn't make a fool of myself in front of a load of peop?????? ha
394 2013-08-16 14:31:41 <pjorrit> if you just acted all sauve while doing that nobody would've noticed
395 2013-08-16 14:34:07 <Luke-Jr> <.<
396 2013-08-16 15:51:00 <Luke-Jr> warren: Raw git changelog for 0.8.4 so far: http://codepad.org/33yv0ZCW
397 2013-08-16 15:51:40 <Luke-Jr> warren: and 0.8.x branch pushed
398 2013-08-16 16:01:48 <jgarzik> Solver() is an annoying name
399 2013-08-16 16:02:04 <jgarzik> it is a matcher, not a solver
400 2013-08-16 16:02:17 <jgarzik> it returns matches, not solutions. It's doing friggin' algebra.
401 2013-08-16 16:02:21 <jgarzik> *not doing
402 2013-08-16 16:03:02 <gmaxwell> hm? It's solving a subset-sum problem. :P
403 2013-08-16 16:05:02 <marcusw> it's solving a computer science problem
404 2013-08-16 16:05:08 <marcusw> so is IRC
405 2013-08-16 16:05:17 <marcusw> we should call it "messaging solution protocol"
406 2013-08-16 16:05:42 <marcusw> or "messaging protocol solution" even
407 2013-08-16 16:06:28 <marcusw> MSVS calls collections of project files "solutions" (for marketing reasons, lol)
408 2013-08-16 16:07:43 <CodeShark> in my signing agent I'm just calling it something like "getPayeeFromScriptPubKey" (or some abbreviated form of this)
409 2013-08-16 16:08:29 <jgarzik> it is solving something in the sense that every function is solving something ;p
410 2013-08-16 16:08:43 <nsh> not f(x) : x
411 2013-08-16 16:09:10 <nsh> well, i suppose that might solve an identity crisis
412 2013-08-16 16:09:12 <jgarzik> CodeShark, yeah, pretty much
413 2013-08-16 16:09:23 <nsh> /hereallweek
414 2013-08-16 16:09:32 <jgarzik> and/or classify-transaction
415 2013-08-16 16:09:38 <gmaxwell> "routeinputs"
416 2013-08-16 16:09:42 <gmaxwell> :P
417 2013-08-16 16:10:28 <jgarzik> it is even more butters because we have two functions named Solver(), that do different things
418 2013-08-16 16:10:30 <CodeShark> it's a three step process: 1) find the payee from scriptPubKey (if the key store has it), 2) construct a scriptSig, 3) sign
419 2013-08-16 16:10:33 <jgarzik> I hate C++
420 2013-08-16 16:10:50 <jgarzik> "let's overload = to mean wacky stuff"
421 2013-08-16 16:10:52 <CodeShark> overloading is great when you're careful not to mix semantics
422 2013-08-16 16:11:11 <jgarzik> every C++ programmer overloads their project with overloading
423 2013-08-16 16:11:40 <jgarzik> surely a distributed SQL database join may be described in C++ as "a + b"
424 2013-08-16 16:12:52 <CodeShark> the problem isn't C++ - the problem is poor design :p
425 2013-08-16 16:13:16 <CodeShark> poor design is a problem regardless of language
426 2013-08-16 16:13:35 <gmaxwell> At some point, when the body count is high enough, you have to consider blaming the tool too.
427 2013-08-16 16:14:03 <CodeShark> the tool is not really to blame - other than that people develop poor habits around certain features
428 2013-08-16 16:14:57 <Ry4an> a gun that you have to hold with two hands or it explodes is a shitty gun even if it never explodes when held w/ two hands.
429 2013-08-16 16:15:17 <CodeShark> C++ gives you great freedom in naming stuff and defining operators - that doesn't mean people are going to pick good names
430 2013-08-16 16:15:36 <gmaxwell> We probably have language arguments in here too often, so I'm going to do my part and not feed it. :P
431 2013-08-16 16:15:38 <CodeShark> if they don't pick good names in C++, chances are they wouldn't have picked good names in any other language either
432 2013-08-16 16:15:47 <handle> Ry4an: I don't understand your analogy
433 2013-08-16 16:16:24 <handle> also, the most popular languages don't have any restrictions on what variable names can be
434 2013-08-16 16:17:29 <CodeShark> overloading is a great feature when used properly because it allows you to resolve types automatically - but it only makes sense when the semantics carry over to all the supported types
435 2013-08-16 16:18:05 <Luke-Jr> handle: I want to name my variables 1 and 2
436 2013-08-16 16:18:55 <CodeShark> in the Solver example, imagine someone calling the second version Solver2 because the language doesn't allow overloading
437 2013-08-16 16:19:03 <CodeShark> we've gotten nowhere - and arguably this is even worse :)
438 2013-08-16 16:20:04 <gmaxwell> solver_impl()
439 2013-08-16 16:20:13 <Luke-Jr> bfgminer has a number of Solver2s
440 2013-08-16 16:20:22 <Luke-Jr> and 3s and 4s <.<
441 2013-08-16 16:20:25 <CodeShark> lol
442 2013-08-16 16:26:00 <sipa> Solver4(x)
443 2013-08-16 16:26:07 <sipa> // Solves for x
444 2013-08-16 16:34:49 <Luke-Jr> would anyone be opposed to adding a #bitcoin-watch equivalent to Bitcoin-Qt?
445 2013-08-16 16:34:59 <Luke-Jr> it's kinda.. too high traffic for freenode/IRC these days ..
446 2013-08-16 16:35:01 <Luke-Jr> :/
447 2013-08-16 16:36:04 <gmaxwell> Luke-Jr: I think that would be neat.
448 2013-08-16 16:36:33 <gmaxwell> But there is no way in hell I'm working on that.
449 2013-08-16 16:36:49 <Luke-Jr> <.<
450 2013-08-16 16:37:19 <gmaxwell> :P
451 2013-08-16 16:37:52 <gmaxwell> Luke-Jr: really that could just be an external program that made a p2p link to your local bitcoin node.
452 2013-08-16 16:38:13 <Luke-Jr> hmm, probably
453 2013-08-16 16:38:31 <Luke-Jr> could even be a CLI app <.<
454 2013-08-16 16:38:48 <sipa> or a webserver
455 2013-08-16 16:39:03 <Luke-Jr> why would it be a webserver? -.-
456 2013-08-16 16:39:20 <sipa> web is the new console
457 2013-08-16 16:39:26 <Luke-Jr> my IRC client doesn't let me open a webpage in it
458 2013-08-16 16:39:29 <Luke-Jr> :p
459 2013-08-16 16:39:30 <gmaxwell> "Why not both"
460 2013-08-16 16:39:50 <gmaxwell> /exec -o ~/watcher
461 2013-08-16 16:39:58 <Luke-Jr> gmaxwell: don't make me host a JS x86 emulator that runs bitcoind and this!
462 2013-08-16 16:40:42 <Luke-Jr> ACTION ponders if anything ever came of that JS x86 emulator
463 2013-08-16 17:25:29 <moarrr> <gribble> Error: You do not meet qualifications for entering ratings.
464 2013-08-16 17:25:34 <moarrr> wtf does this mean>
465 2013-08-16 17:25:35 <moarrr> ?
466 2013-08-16 17:28:11 <Luke-Jr> moarrr: wrong channel
467 2013-08-16 17:48:33 <iwilcox> Anyone know the approximate relative sizes of the things stored in wallet.dat? A p2p trading site I trade at says it has problems with wallet.dat being HUUUGE and apparently being a pain to back up. I was wondering what low-hanging fruit they might want to look at cutting out.
468 2013-08-16 17:49:04 <CodeShark> the bitcoind wallet was not designed for that kind of use
469 2013-08-16 17:49:17 <gmaxwell> if it's p2p why do they have a big wallet? :P
470 2013-08-16 17:49:26 <iwilcox> They give "huge wallet" as a reason to limit how often you can create a new deposit address. I hate not being able to get a fresh address on demand :(
471 2013-08-16 17:49:35 <iwilcox> They escrow the Bitcoin part of all deals
472 2013-08-16 17:49:52 <iwilcox> So I guess p2p is a bit inaccurate.
473 2013-08-16 17:50:06 <CodeShark> hopefully I should have a far more scalable wallet in the very near term
474 2013-08-16 17:50:09 <gmaxwell> iwilcox: thats ... lame. You know we have multisignature transactions so an escrow that can steal your coins doesn't need to exist.
475 2013-08-16 17:50:45 <CodeShark> gmaxwell: the protocol supports it but there aren't very many tools for it
476 2013-08-16 17:50:57 <gmaxwell> in any case, I dunno why they'd need to do that beyond preventing dos attacks (E.g. with a captcha and per user rate limiter). Even with tens of thousands of addresses the wallet is just a few megabytes.
477 2013-08-16 17:51:04 <CodeShark> and most of the people building these sites are script kiddies :p
478 2013-08-16 17:51:10 <iwilcox> gmaxwell: Sure, but escrow isn't all he provides. He works quite hard to vet sellers, attract buyers, make the whole thing easy. He earns his 1% I'd say.
479 2013-08-16 17:51:12 <moarrr> gmaxwell: im having a little problem rating people on gribble...
480 2013-08-16 17:51:13 <gmaxwell> CodeShark: there aren't many tools for a website you deposit money to, and still it exists.
481 2013-08-16 17:51:22 <moarrr> gmaxwell: who can i talk to about this?
482 2013-08-16 17:51:41 <gmaxwell> iwilcox: sure, and you could still give him his 1%. Thats not exclusive with handing your coin over to him.
483 2013-08-16 17:51:55 <gmaxwell> CodeShark: ACK your latter comment. :P
484 2013-08-16 17:54:31 <iwilcox> gmaxwell: OK, how would the multisig thing work if he was merely an introducer?
485 2013-08-16 17:55:34 <iwilcox> CodeShark: The guy has a pretty good track record in security, and I certainly wouldn't put this particular guy in the kiddie category :)
486 2013-08-16 17:57:05 <iwilcox> gmaxwell: You suggesting he just provides one sig on a 2-of-3?
487 2013-08-16 17:57:11 <gmaxwell> iwilcox: Sure.
488 2013-08-16 17:57:18 <gmaxwell> iwilcox: charge for introductions??? send funds, get introductions (and access to a multisig key), charge for resolving disputes.
489 2013-08-16 17:57:25 <Cusipzzz> iwilcox: how large a file? I have one ~100megs, slow but works ok
490 2013-08-16 17:58:04 <iwilcox> ACTION greps logs
491 2013-08-16 17:59:12 <iwilcox> "it sucks that the wallet.dat is hundreds of megs and i have to back it up frequently"
492 2013-08-16 17:59:18 <iwilcox> That was in May.
493 2013-08-16 17:59:41 <Cusipzzz> thanks, was wondering when this one might run into problems
494 2013-08-16 18:01:15 <gmaxwell> iwilcox: sounds like its time to rotate it.
495 2013-08-16 18:01:59 <Cusipzzz> is there some way to bulk importprivkeys to bitcoind in a fresh wallet?
496 2013-08-16 18:02:24 <gmaxwell> Cusipzzz: that shouldn't do you any good.
497 2013-08-16 18:02:29 <iwilcox> I guess he's avoiding that because he'd have to deal with the occasional situation where someone sends to an ancient deposit address that's since been rotated out
498 2013-08-16 18:02:31 <xeroc> if thnkg there is some -norescan switch o so
499 2013-08-16 18:02:37 <gmaxwell> If that helps then just bdb compacting the wallet would have helped.
500 2013-08-16 18:02:49 <Cusipzzz> iwilcox: right, same concern
501 2013-08-16 18:03:05 <gmaxwell> Cusipzzz: import takes an argument to trigger rescan or not.
502 2013-08-16 18:03:27 <gmaxwell> But, what are you trying to do there? importing private keys into a wallet is often a sign that you're doing something misguided.
503 2013-08-16 18:04:05 <helo> who has to back up their wallet frequently?
504 2013-08-16 18:04:16 <Cusipzzz> trying to reduce the size of wallet.dat but keep the active keys
505 2013-08-16 18:04:54 <xeroc> helo: people who use new keys frequently
506 2013-08-16 18:05:16 <gmaxwell> xeroc: you should have your pool size set so that you don't have to backup too frequently.
507 2013-08-16 18:05:37 <Cusipzzz> like if i can identify spent keys that are not assigned to users, they can be removed to save space potentially. I have a wallet.dat ~100meg, so may face the iwilcox issue at some point
508 2013-08-16 18:05:42 <Luke-Jr> iwilcox: my wallet.dat is only 16 MB and I left my bitcoind generating addresses constantly for a while some months ago..
509 2013-08-16 18:05:48 <xeroc> so some thread on btctalk saying to increase it up to ... say 1000
510 2013-08-16 18:06:11 <Luke-Jr> how do you people get huge wallets?
511 2013-08-16 18:06:13 <iwilcox> Luke-Jr: Which is why I ask about the component's sizes
512 2013-08-16 18:06:22 <xeroc> no idea .
513 2013-08-16 18:06:30 <Cusipzzz> Luke-Jr: running a popular service that ad
514 2013-08-16 18:06:38 <Cusipzzz> assigns address often
515 2013-08-16 18:06:38 <gmaxwell> Luke-Jr: yea, I recall having it be something like 10-20mb for 100k addresses or so. But whats probably getting these people are copies of whole paying transactions in the wallet.
516 2013-08-16 18:07:03 <gmaxwell> Luke-Jr: e.g. someone with one of those 100k dust transactions pays 50 of your users. I think we end up with 50 copies of that transaction in the wallet.
517 2013-08-16 18:07:23 <Luke-Jr> O.o
518 2013-08-16 18:07:32 <iwilcox> That's not something that happens at this p2p-ish place.
519 2013-08-16 18:08:08 <iwilcox> I guess, though, perhaps people often withdraw from exchanges to their p2p wallet; perhaps those draw on loads of UTXOs at the exchange.
520 2013-08-16 18:09:26 <gmaxwell> iwilcox: does it keep a balance at all?
521 2013-08-16 18:09:33 <Cusipzzz> right. there are potentially empty/non assigned address from years ago taking up space.
522 2013-08-16 18:09:39 <gmaxwell> besides, people performing deanonymizing transactions pay dust to people who don't want it at all.
523 2013-08-16 18:09:53 <gmaxwell> Cusipzzz: the actual addresses themselves take up ~no space. Transactions involved with them do.
524 2013-08-16 18:10:21 <Cusipzzz> right. best way to clean that up?
525 2013-08-16 18:10:31 <iwilcox> gmaxwell: The p2p site? Yeah, how could it not? I guess you could say it's a web wallet that just happens to be used almost entirely for the p2p stuff.
526 2013-08-16 18:10:36 <xeroc> so why arent txs separated from the wallet (ak keys)?
527 2013-08-16 18:11:02 <gmaxwell> iwilcox: right, so I can promise you some of its users are giving addresses from it to fountains and such.
528 2013-08-16 18:11:42 <iwilcox> Cusipzzz: I guess a really cheap and nasty way might be to do the bulk private key move thing, then just *not* rescan :)
529 2013-08-16 18:11:57 <gmaxwell> xeroc: in seperate files? because then you'd have two files you need to back up instead of one.
530 2013-08-16 18:12:06 <Cusipzzz> iwilcox: yeah :)
531 2013-08-16 18:12:23 <xeroc> wh do i have to backup the txs .. they are stored in the blockchain ..
532 2013-08-16 18:12:34 <gmaxwell> (you need to back up transactions because the network can forget them until they're burried, and because your metadata about them can't be recovered: accounts/labels, timestamps, etc)
533 2013-08-16 18:13:20 <gmaxwell> For transactions paying you the only way to recover them is by doing very expensive scanning or queries against the blockchain, which is also important to avoid.
534 2013-08-16 18:13:20 <xeroc> hmm ... good point .
535 2013-08-16 18:13:41 <gmaxwell> But certantly the data could be stored more efficiently than I think we do now.
536 2013-08-16 18:13:52 <xeroc> yhea . but thats in case you LOOSE them .. which should ne avoided in the first place
537 2013-08-16 18:14:30 <xeroc> isnt the privkey MUCH MUCH more important than some info about txs .. which can BE RECOVERD ... i'm just saying ..
538 2013-08-16 18:14:38 <gmaxwell> sure, but you change some things around and recover a backup... and oh darn, no transactions for a week while you have someone search databases for your transactions.
539 2013-08-16 18:14:56 <gmaxwell> xeroc: the metadata is unrecoverable. Yes, the keying material is more important, but it all ought to be backed up.
540 2013-08-16 18:15:09 <xeroc> ah ..I see .. the big issue is the user ... typical layer-8 problem .. good point ..
541 2013-08-16 18:15:37 <xeroc> ok .. giving up .. you arguemtns are good enough for me ;-)
542 2013-08-16 18:15:44 <xeroc> ACTION hugs gmaxwell
543 2013-08-16 18:16:52 <gmaxwell> xeroc: Your point about key data being more important matters too. There are a couple ways of handling it, none are fantastic.
544 2013-08-16 18:17:18 <xeroc> you mean .. extracting them using pywallet .. or stuff like that?
545 2013-08-16 18:17:25 <xeroc> not so userfriendly
546 2013-08-16 18:17:59 <gmaxwell> No, I mean ways of structuring the wallet to better reflect the differential importance of keying vs other data.
547 2013-08-16 18:18:22 <Cusipzzz> i extract them just in case this 100meg wallet fails completely. but there should be an easier w[Cay to identify unused/spent addresses, copy to new wallet with no rescan
548 2013-08-16 18:18:29 <gmaxwell> E.g. with a determinstic wallet the first backup would be at least sufficient to capture your keying data forever. Or you could do keys only backups.
549 2013-08-16 18:18:58 <gmaxwell> Cusipzzz: a problem with that is that over and over again we see people doing things like that, destroying the original wallet, and then realizing that they have remaining coin.
550 2013-08-16 18:19:44 <Cusipzzz> i'm not destroying the original, will have it, but worried about corruptions as it grows with all this 2 year old unneeded data
551 2013-08-16 18:19:50 <xeroc> at least its a topic with ongoing discussions ..
552 2013-08-16 18:20:20 <xeroc> btw ... are there any plans for deterministic wallets for the original client?
553 2013-08-16 18:20:49 <gmaxwell> Cusipzzz: how would you define in-use for your purpose?
554 2013-08-16 18:21:14 <Cusipzzz> assigned to a user or has a balance (not empty old change addresses, etc
555 2013-08-16 18:21:27 <gmaxwell> what does assigned to a user mean?
556 2013-08-16 18:21:37 <gmaxwell> like in your external system?
557 2013-08-16 18:21:41 <Cusipzzz> users of the service, which they see as a deposit address
558 2013-08-16 18:22:10 <gmaxwell> okay, so exportprivkey on everything in your system, and every address in listunspent. (and perhaps every address that you've recently used if not current)
559 2013-08-16 18:22:17 <gmaxwell> and then importprivkey
560 2013-08-16 18:22:22 <gmaxwell> what else do you need?
561 2013-08-16 18:22:25 <Cusipzzz> with no rescan, right?
562 2013-08-16 18:22:35 <xeroc> preferable ..
563 2013-08-16 18:22:52 <xeroc> unless you want to wait several minutes for each single key
564 2013-08-16 18:22:58 <xeroc> => no rescan ..
565 2013-08-16 18:23:17 <gmaxwell> Cusipzzz: with no rescan.
566 2013-08-16 18:23:18 <xeroc> insert the lat key with rescan to let bitcoind update the unspent list
567 2013-08-16 18:23:26 <xeroc> s/lat/last/
568 2013-08-16 18:23:29 <Cusipzzz> i'm just worried about possible corruption due to size, and delays in querying/backup, like iwilcox mentioned
569 2013-08-16 18:23:31 <gmaxwell> right the last one you rescan on.
570 2013-08-16 18:23:51 <Cusipzzz> if it was tested with no probs at 500megs, i'm safe for some time
571 2013-08-16 18:23:57 <gmaxwell> iwilcox didn't mention any corruption or delays, only that making backups was burdensom.
572 2013-08-16 18:25:01 <Cusipzzz> backups are a little slow, lisstransactions is slow, just trying to head off future problems if there is a clean way to do it
573 2013-08-16 18:25:11 <iwilcox> I don't know the exact details, but I think the guy's issue is with space taken by his regular backups, and time taken to make them (presumably he has to stop the daemon, but the exchange is always running so that means downtime)
574 2013-08-16 18:25:44 <iwilcox> Perhaps I should point him at bup for the backups. Been using it for a couple of years and my backups are still tiny.
575 2013-08-16 18:25:58 <iwilcox> Saved my bacon once too.
576 2013-08-16 18:26:10 <xeroc> isnt there a way to cleanly backup without shutting down?
577 2013-08-16 18:26:23 <Cusipzzz> i have disk space to burn, not too worried on that one
578 2013-08-16 18:26:25 <gmaxwell> sure, backupwallet
579 2013-08-16 18:26:44 <xeroc> thought so .. why not use that @ iwilcox
580 2013-08-16 18:32:06 <iwilcox> Oh, perhaps he is doing that. As I say, I'm just trying to give him no excuse to avoid fixing it, so I don't know the details and my end goal is just: let me create a fresh deposit address whenever I want one, and don't use your backup stuff as an excuse
581 2013-08-16 18:33:21 <gmaxwell> iwilcox: "Gmaxwell from the core development team says that his large wallet problems are almost certantly not caused by having lots of addresses".
582 2013-08-16 18:33:31 <Cusipzzz> iwilcox: i do it by request, but it's not automated so someone doesn't go nuts
583 2013-08-16 18:33:36 <iwilcox> In the long term, in this particular case gmaxwell could have a point about multisig. That said, having the BTC in the site's hotwallet does allow them to ensure that only an offer truly, currently backed by BTC is displayed
584 2013-08-16 18:33:38 <Krellan_> I'd like to have some large wallet problems :)
585 2013-08-16 18:34:01 <iwilcox> Krellan_: We're talking wallet-full-of-receipts problems ;)
586 2013-08-16 18:34:12 <Cusipzzz> Krellan_: disk space size large only, i assure you
587 2013-08-16 18:34:20 <Krellan_> Doh :)
588 2013-08-16 18:34:49 <gmaxwell> iwilcox: interesting point. though even there he could do 2 of 2 locking.
589 2013-08-16 18:36:51 <iwilcox> Think I need to sit down with paper to see how that part works.
590 2013-08-16 18:36:58 <iwilcox> My poor brain.
591 2013-08-16 18:47:17 <deego> In bitcoind listtransactions, is there a way to filter by unixtime? Instead of by count and from? The reason i want that is that the latter changes as more transactions are sent.
592 2013-08-16 18:49:20 <deego> That is, I'm trying to filter all transactions that happened for 2012.
593 2013-08-16 18:49:54 <deego> (2) How do I ask bitcoind to list the total number of transactions?
594 2013-08-16 18:50:08 <handle> iirc it's a bad idea to rely on listtransactions - beyond that I'm no use, sorry
595 2013-08-16 18:50:11 <deego> I could bitcoind listtransactions * and filter, but that seems excessive.
596 2013-08-16 18:50:24 <deego> handle: Oh? :( Why's tat?
597 2013-08-16 18:50:50 <handle> because due to things like orphan blocks, some already existing transactions may become invalid
598 2013-08-16 18:51:03 <handle> and so by just doing a delta of time, you're missing a bunch of changes
599 2013-08-16 18:51:08 <deego> Ah.
600 2013-08-16 18:54:09 <helo> i think listtransactions is ok to rely on, as long as you pay attention to confirmations and reorgs