1 2012-12-20 00:00:03 <sipa> really, no need
  2 2012-12-20 00:00:22 <stealth222> I already did, though - lol
  3 2012-12-20 00:00:25 <sipa> github already knows what your repo is, and will track and show changes to it
  4 2012-12-20 00:00:30 <gmaxwell> stealth222: I changed it to a link to the pull request.
  5 2012-12-20 00:00:54 <stealth222> ah, ok. cool, thanks
  6 2012-12-20 00:02:03 <stealth222> thank you guys so much for your help
  7 2012-12-20 00:02:22 <stealth222> hopefully you won
  8 2012-12-20 00:02:32 <stealth222> you won't need to be holding my hand this much in the future :)
  9 2012-12-20 00:02:53 <gmaxwell> Software development is a team sport.
 10 2012-12-20 00:03:11 <etotheipi_> gmaxwell: not for some of us hermits!
 11 2012-12-20 00:03:29 <gmaxwell> stealth222: hopefully you won't mind when we nitpick your pull and have you change a bunch of stuff. :P
 12 2012-12-20 00:04:03 <stealth222> as long as we're making progress and this stuff is actually going to be useful I have no problem with that :)
 13 2012-12-20 00:04:37 <stealth222> I've been developing on the bitcoin sidelines for far too long - I'm sick of having to maintain a completely separate codebase for everything I do
 14 2012-12-20 00:05:13 <gmaxwell> More contributors is good and important??? the pipeline to merging can be fairly long, but please don't let that put you off.
 15 2012-12-20 00:06:32 <stealth222> these features I'm working on are not just "would be nice to have" functionality but essential for several applications I've written. It would be so nice if the main branch of bitcoind supported much of this out-of-the-box without having to have my own custom builds or entirely separate processes
 16 2012-12-20 00:08:10 <gmaxwell> stealth222: sure. At least for things which are generally useful (e.g. not just interesting to you) thats a good plan. It can just take a while to get things reviewed and tested and api changes hammered out, since once something goes in its harder to take it back out again since people will begin depending on it.
 17 2012-12-20 00:08:30 <stealth222> yes, of course
 18 2012-12-20 00:09:28 <stealth222> anyhow, gotta go - thanks again, guys
 19 2012-12-20 00:35:34 <jgarzik> gmaxwell: is IRC testnet3 still broken for you?  See Laszlo's response?
 20 2012-12-20 00:36:15 <sipa> where?
 21 2012-12-20 00:38:05 <jgarzik> sipa: gmaxwell complained about it here.  I emailed laslzo, and CC'd gmaxwell.  Private email, but I don't mind discussing it
 22 2012-12-20 00:38:22 <jgarzik> <gmaxwell> irc.lfnet.org is now suppressing /who is all the bitcoin
 23 2012-12-20 00:38:27 <jgarzik> sipa: ^ complaint
 24 2012-12-20 00:39:20 <sipa> didn't it do that before?
 25 2012-12-20 00:41:16 <gmaxwell> sipa: no.
 26 2012-12-20 00:42:18 <gmaxwell> jgarzik: /who works in #bitcoinTEST3 for me now but not #bitcoin01
 27 2012-12-20 00:42:28 <gmaxwell> sipa: before it just returned partial /whos.
 28 2012-12-20 00:42:32 <sipa> oh
 29 2012-12-20 00:42:55 <gmaxwell> oh it just worked for me in #bitcoin01
 30 2012-12-20 00:43:19 <gmaxwell> weird. I tried it again an hour or so ago and it wasn't working then.
 31 2012-12-20 00:44:41 <jgarzik> gmaxwell: what lfnet server are you using?
 32 2012-12-20 00:44:56 <jgarzik> gmaxwell: or connection DNS name, actually
 33 2012-12-20 00:46:19 <jgarzik> wfm
 34 2012-12-20 00:48:09 <gmaxwell> as mentioned, WFM now, I've been using the irc.lfnet.org name.
 35 2012-12-20 00:51:13 <jgarzik> hmmm
 36 2012-12-20 00:51:31 <gmaxwell> one of the comments he made was that maybe a server was desynced.
 37 2012-12-20 00:51:34 <jgarzik> xchat did not pull up a list of nicks in the channel... but a manual "/who *" worked
 38 2012-12-20 00:51:41 <jgarzik> for #bitcoin01
 39 2012-12-20 00:51:55 <gmaxwell> yes, it's not whoing on connect but it was like that before I think.
 40 2012-12-20 00:51:55 <jgarzik> unable to reproduce behavior, but definitely saw it
 41 2012-12-20 00:52:05 <jgarzik> ok
 42 2012-12-20 00:54:36 <gmaxwell> sipa: I'm surprised the channel isn't +m...
 43 2012-12-20 00:57:39 <sipa> ha!
 44 2012-12-20 01:08:44 <sipa> ooh, 20/12/2012 !
 45 2012-12-20 01:09:19 <gmaxwell> Whats special about that? oohhh.
 46 2012-12-20 01:09:32 <gmaxwell> okay but what crazy people do day month year? :P
 47 2012-12-20 01:10:12 <sipa> yeah, i agree, year-month-day is the only reasonable order :p
 48 2012-12-20 01:11:27 <gmaxwell> With or without a random letter thrown in? :P  ...
 49 2012-12-20 01:11:40 <gmaxwell> I think the US is gradually changing to Y-M-D.
 50 2012-12-20 01:13:25 <sipa> no reason for a mixed-endian date format
 51 2012-12-20 01:13:31 <sipa> the world is not a PDP-11
 52 2012-12-20 01:14:58 <gmaxwell> too bad for all jeff's dependencies.. would be fun to get picocoin running on a PDP-11... but getting glib on something pre-ansic sounds impossibly painful.
 53 2012-12-20 01:16:04 <sipa> loi
 54 2012-12-20 01:29:41 <stealth222> I hope the entire world changes to YYYY-MM-DD. Makes collation much easier :)
 55 2012-12-20 01:59:38 <Luke-Jr> I still prefer tonal.
 56 2012-12-20 01:59:40 <sipa> stealth222: wallet and block verification are already quite separate, actually
 57 2012-12-20 02:00:15 <sipa> stealth222: but block verification doesn't track transactions, you still need the wallet to do that
 58 2012-12-20 02:00:44 <sipa> for example for unconfirmed transactions, for comments on them, for account info, ...
 59 2012-12-20 02:00:48 <etotheipi_> sipa, gmaxwell: so are you going to use the KDF sipa described a while ago for converting master seed to master extended key?
 60 2012-12-20 02:01:09 <etotheipi_> even if it's not part of the BIP 32 spec, I'd like to follow it
 61 2012-12-20 02:01:19 <sipa> etotheipi_: i certainly wouldn't mind doing that, but there are already several other ideas
 62 2012-12-20 02:01:58 <gmaxwell> etotheipi_: did you ever figure out if bip32 needed to be modified to accomidate your provable in chain addresses?
 63 2012-12-20 02:03:06 <sipa> it's compatible... just provide P_parent and I_L
 64 2012-12-20 02:03:42 <gmaxwell> Thought it was. OK.
 65 2012-12-20 02:03:53 <etotheipi_> I want to double check it for my own sanity, but yes it appears so
 66 2012-12-20 02:03:54 <sipa> etotheipi_ (or roconnor?) came up with something more complex, but i fail to see the necessity
 67 2012-12-20 02:03:55 <gmaxwell> I'm glad I have you guys to think for me.
 68 2012-12-20 02:04:12 <etotheipi_> sipa: are you talking about thanke?
 69 2012-12-20 02:04:15 <sipa> no
 70 2012-12-20 02:04:56 <etotheipi_> what was more complex?  this is pretty damned simple actually (and roconner came up with it, I only asked the question)
 71 2012-12-20 02:05:19 <sipa> it used P*H(chaincode) or so
 72 2012-12-20 02:05:34 <sipa> instead of the normal BIP32 derivation
 73 2012-12-20 02:05:49 <etotheipi_> sipa: that specific derivation wasn't necessary
 74 2012-12-20 02:05:56 <etotheipi_> it was just illustritive
 75 2012-12-20 02:05:58 <sipa> ok
 76 2012-12-20 02:06:25 <etotheipi_> and I wasn't sure that yet that BIP 32 satisified the condition that you can send them the scalar without giving them the chaincode
 77 2012-12-20 02:07:08 <sipa> etotheipi_: also, latest version of gavin's payment protocol proposal has a some future extensibility where the script is derived by the receiver, based on a pubkey-in-cert and hash(paymentrequest) itself
 78 2012-12-20 02:07:32 <sipa> which would accomplish the same thing
 79 2012-12-20 02:07:36 <etotheipi_> cool
 80 2012-12-20 02:07:46 <etotheipi_> I think this is an extremely valuable capability to preserve
 81 2012-12-20 02:07:51 <etotheipi_> even if it's not used right away
 82 2012-12-20 02:07:53 <sipa> it's not defined yet, but there's a comment about it
 83 2012-12-20 02:08:51 <etotheipi_> most definitely a major advantage for Bitcoin compared to other payment systems... the fact that it CAN be done
 84 2012-12-20 02:08:51 <sipa> 
ACTION zZzZ

 85 2012-12-20 02:09:08 <etotheipi_> alright, back to my new wallets
 86 2012-12-20 02:09:22 <etotheipi_> sipa: before you ZzZz
 87 2012-12-20 02:09:55 <etotheipi_> were you considering the more-advanced/flexible version of your KDF?  I can go back and dig apart your email if you are
 88 2012-12-20 02:10:10 <etotheipi_> or were you okay sticking with the simpler, inflexible (N^4) version
 89 2012-12-20 02:10:12 <etotheipi_> ?
 90 2012-12-20 02:48:29 <swulf--> is it possible to determine the originating address in a transaction using the bitcoind json cmdline?
 91 2012-12-20 02:49:27 <Luke-Jr> swulf--: transactions don't have originating addresses
 92 2012-12-20 02:50:38 <swulf--> i know transactions done
 93 2012-12-20 02:50:41 <swulf--> don't*
 94 2012-12-20 02:50:47 <swulf--> but in a transaction, can get the list of inputs?
 95 2012-12-20 02:50:57 <swulf--> from the command-line, specifically
 96 2012-12-20 02:51:11 <Luke-Jr> getrawtransaction
 97 2012-12-20 02:51:45 <swulf--> $ bin/bitcoind getrawtransaction e44cce8f31d2817def07be3c720bdede2eba59837357acaca3c8d631f90e73ba
 98 2012-12-20 02:52:11 <swulf--> does it only accept special txids?
 99 2012-12-20 02:56:08 <gmaxwell> swulf--: are you running 0.8/git?
100 2012-12-20 02:56:34 <gmaxwell> if so it only accepts wallet, mempool, and unspent txids.
101 2012-12-20 02:59:22 <swulf--> i'm running from git, yeah
102 2012-12-20 02:59:42 <swulf--> this txid is actually a txid gained from listtransactions api
103 2012-12-20 02:59:48 <swulf--> so, it *ought* to work, right?
104 2012-12-20 03:04:31 <etotheipi_> sipa, gmaxwell:  additionally, did we agree that 100% of public keys in BIP 32 will be compressed?  at every step?
105 2012-12-20 03:38:40 <swulf--> is there a way to get non-wallet transactions via the command-line?
106 2012-12-20 03:46:57 <gmaxwell> swulf--: not if they're spent, at least not currently in git.  0.7.1 can however.
107 2012-12-20 03:57:14 <swulf--> hmm
108 2012-12-20 05:32:36 <swulf--> Is GetTransaction() special in any way?  I mean, why would rpc call 'gettransaction' return a result with a given txid but getrawtransaction error with {"code":-5,"message":"No information available about transaction"} with the same txid?
109 2012-12-20 05:40:47 <Luke-Jr> http://marylandpirates.com/wp-content/uploads/rsc_policy_brief_--_three_myths_about_copyright_law_and_where_to_start_to_fix_it_--_november_16_2012.pdf
110 2012-12-20 06:20:37 <cande> hi guys
111 2012-12-20 06:21:11 <cande> what is the normal way to run a bitcoin deamon and let others use this to verify transactions?
112 2012-12-20 06:21:27 <cande> is it via the RPC command? or something else?
113 2012-12-20 06:57:12 <Luke-Jr> cande: huh?
114 2012-12-20 06:57:36 <cande> hm
115 2012-12-20 06:57:46 <Luke-Jr> cande: you mean mining? usually you do it cooperatively with other miners on a pool
116 2012-12-20 06:57:52 <Luke-Jr> using a mining program like BFGMiner
117 2012-12-20 06:57:58 <cande> no mining
118 2012-12-20 06:58:32 <cande> it's for eshops to verify payments
119 2012-12-20 07:00:38 <Luke-Jr> cande: oh, you can do that via RPC yeah
120 2012-12-20 07:01:04 <Luke-Jr> cande: use -blocknotify to run a script when there's a new block; the script can call listtransactions to see the status of anything received
121 2012-12-20 07:01:42 <cande> is there any example script of this?
122 2012-12-20 07:01:46 <cande> for linux?
123 2012-12-20 07:05:12 <cande> okej, that looks like a nice solution
124 2012-12-20 09:38:24 <stealth222> I see in coding.text that variable names should begin with the type but this convention has been violated in many places. In adding new code, is it considered more important to stick to these guidelines or to blend in with the style of the surrounding code?
125 2012-12-20 09:39:20 <stealth222> *coding.txt
126 2012-12-20 09:46:48 <luke-jr_> stealth222: it should be possible to do both, but the variable naming rules are mostly ignored these days
127 2012-12-20 09:47:43 <stealth222> I'm looking to start adding multiwallet support - there are a couple issues I'd like to discuss first
128 2012-12-20 09:48:02 <stealth222> like, should we allow the user to specify which wallets to load at startup in the config file and via command line arguments
129 2012-12-20 09:48:06 <stealth222> I think we should
130 2012-12-20 09:48:14 <stealth222> also, whether we should allow dynamic loading and unloading of wallets
131 2012-12-20 09:48:17 <stealth222> again, I think we should
132 2012-12-20 09:48:42 <Jouke> that would be awesome
133 2012-12-20 09:49:26 <sipa> stealth222: there's already a pullreq to specify the wallet location
134 2012-12-20 09:49:52 <stealth222> so this would go beyond that
135 2012-12-20 09:50:48 <stealth222> is there someone working on that feature already, sipa?
136 2012-12-20 09:51:06 <stealth222> I could build this out in like the next couple days
137 2012-12-20 09:52:25 <sipa> swulf--1: gettransaction queries the wallet, getrawtransaction queries the blockchain validation system
138 2012-12-20 09:53:18 <stealth222> the more serious issues pertaining to adding multiwallet support are API changes
139 2012-12-20 09:53:24 <sipa> swulf--1: those are mostly independent, and as of 0.8, no information about spent txn is kept in the validation db
140 2012-12-20 09:53:33 <stealth222> there are a couple possibilities I'm thinking:
141 2012-12-20 09:53:52 <stealth222> 1) RPC calls are context-sensitive...depend on what wallets are currently loaded
142 2012-12-20 09:54:01 <stealth222> 2) RPC calls must also specify a wallet or list of wallets
143 2012-12-20 09:54:32 <stealth222> in principle I tend to prefer (2) but it risks breaking the existing API
144 2012-12-20 09:55:09 <sipa> stealth222: i like having separate username/password per wallet
145 2012-12-20 09:55:25 <stealth222> yes - I like that, too
146 2012-12-20 09:55:46 <sipa> stealth222: and someone is planning to abstract the wallet interface better
147 2012-12-20 09:55:57 <stealth222> who?
148 2012-12-20 09:56:00 <sipa> look through the pullreqs
149 2012-12-20 09:57:16 <stealth222> oh, I just saw someone was doing https://github.com/bitcoin/bitcoin/pull/1974
150 2012-12-20 09:57:21 <stealth222> I was planning on doing that, too :)
151 2012-12-20 09:57:29 <stealth222> nice
152 2012-12-20 10:00:55 <sipa> i was talking about #2075
153 2012-12-20 10:01:07 <stealth222> yeah, I know - just saw that other one, too
154 2012-12-20 10:01:19 <sipa> and #1889
155 2012-12-20 10:03:16 <stealth222> I have a bunch of ideas I could implement very, very quickly - but I'm a little afraid of diverging too much from the existing architecture and making it difficult to get them integrated
156 2012-12-20 10:04:21 <stealth222> how do you browse pull requests by number?
157 2012-12-20 10:04:46 <sipa> ?
158 2012-12-20 10:05:03 <stealth222> I just see submitted, updated, and popularity
159 2012-12-20 10:05:09 <stealth222> is there a way to sort them by number?
160 2012-12-20 10:05:25 <sipa> they are sorted by number
161 2012-12-20 10:05:33 <stealth222> I don't see the numbers
162 2012-12-20 10:05:48 <sipa> yeah, look at the urls
163 2012-12-20 10:06:04 <stealth222> you mean I have to hover over each one to see the number?
164 2012-12-20 10:06:13 <sipa> uhu
165 2012-12-20 10:06:15 <stealth222> lol
166 2012-12-20 10:06:19 <sipa> it's silly
167 2012-12-20 10:06:37 <sipa> you can see the numbers if you go to issues
168 2012-12-20 10:07:01 <sipa> as every pullreq has an associated issue with the same number
169 2012-12-20 10:07:25 <stealth222> I could get dynamic loading/unloading of wallets implemented in a few hours, probably
170 2012-12-20 10:07:26 <stealth222> lol
171 2012-12-20 10:07:40 <stealth222> should I just press on with that?
172 2012-12-20 10:08:24 <sipa> i think that's only useful after multiwallet support
173 2012-12-20 10:08:35 <stealth222> well, I can also add multiwallet support
174 2012-12-20 10:08:59 <stealth222> and even support for loading wallets across a network
175 2012-12-20 10:09:12 <stealth222> and synching them
176 2012-12-20 10:09:20 <sipa> ??
177 2012-12-20 10:09:58 <stealth222> well, for instance, in some applications I've written I've pregenerated a bunch of bitcoin addresses on an offline machine, copied them over to servers that send payment alerts
178 2012-12-20 10:10:23 <stealth222> it would be very nice to be able to just push a "watch-only" wallet onto the server
179 2012-12-20 10:10:31 <luke-jr_> stealth222: HD wallets do that
180 2012-12-20 10:11:24 <stealth222> what do you mean?
181 2012-12-20 10:11:52 <stealth222> I don't want the notification servers storing any private keys at all
182 2012-12-20 10:11:59 <sipa> stealth222: loading a wallet across the network sounds very hard to me
183 2012-12-20 10:12:34 <stealth222> doesn't sound too hard to me
184 2012-12-20 10:12:39 <sipa> and dangerous
185 2012-12-20 10:12:56 <sipa> what will you do with the modified wallet?
186 2012-12-20 10:12:56 <stealth222> we're talking a watch-only wallet
187 2012-12-20 10:13:25 <stealth222> I want to be able to easily tell my notification service nodes to watch a particular set of addresses
188 2012-12-20 10:13:28 <luke-jr_> stealth222: HD wallets enable you to load a single "master" public key once, to match all the derived addresses
189 2012-12-20 10:13:48 <stealth222> I want to decentralize key generation
190 2012-12-20 10:13:58 <sipa> read bip32
191 2012-12-20 10:14:03 <stealth222> not everyone in the organization needs to have access to all acounts
192 2012-12-20 10:14:22 <sipa> really, read bip32 :)
193 2012-12-20 10:15:18 <stealth222> bip32 is interesting indeed...
194 2012-12-20 10:15:38 <stealth222> but surely this isn't something that will be added within the next several days to bitcoind
195 2012-12-20 10:15:44 <stealth222> whereas what I mentioned earlier could be
196 2012-12-20 10:16:23 <sipa> i don't see how to do it in a safe/sane way
197 2012-12-20 10:17:03 <sipa> unless you want nodes to need continuous rescanning, and lose unconfirmed transactions
198 2012-12-20 10:17:22 <stealth222> I was thinking more along the lines of batching
199 2012-12-20 10:17:33 <sipa> batching?
200 2012-12-20 10:18:59 <sipa> etotheipi_: yeah, bip32 is compressedkey-only now
201 2012-12-20 10:20:00 <stealth222> so here's the architecture I'm envisioning - you have key generating/storing nodes which are secured and do not connect to the p2p network. you copy over only addresses to servers which support, say, web-based shopping carts which provide them to users as needed. then you have notification nodes that send alerts to payment processers
202 2012-12-20 10:20:53 <sipa> yeah, that's exactly what bip32 is intended for
203 2012-12-20 10:21:53 <stealth222> ok, nice
204 2012-12-20 10:22:14 <sipa> and it won't be implemented in the next few days, but maybe it will before 0.8 is released
205 2012-12-20 10:23:40 <stealth222> is someone already working on adding this to bitcoind?
206 2012-12-20 10:23:49 <sipa> yes, me
207 2012-12-20 10:23:50 <stealth222> and if not, can I volunteer? :)
208 2012-12-20 10:23:52 <stealth222> ok :)
209 2012-12-20 10:24:08 <stealth222> how far along are you?
210 2012-12-20 10:24:43 <stealth222> and do you need/could you use any help?
211 2012-12-20 10:24:52 <sipa> the key derivation crypto is implemented, and i have an earlier simple but functional deterministic wallet implementation
212 2012-12-20 10:25:21 <sipa> i do have some other priorities now, though
213 2012-12-20 10:25:41 <stealth222> I'd be glad to help if it will speed it up
214 2012-12-20 10:25:50 <stealth222> I'd like to have this implemented like yesterday - lol
215 2012-12-20 10:27:45 <sipa> the hardest part won't be implementation
216 2012-12-20 10:28:07 <sipa> but testing, and convincing others that it is correct :)
217 2012-12-20 10:28:28 <stealth222> it's a heck of a lot easier to convince others that it is correct when you've already built it and it works
218 2012-12-20 10:29:04 <sipa> of course
219 2012-12-20 10:29:36 <stealth222> I'm not a marketer by nature either, sipa - I'm sure marketers would take the opposite view :p
220 2012-12-20 10:30:04 <sipa> i was talking about the correctness of the implememtation
221 2012-12-20 10:30:17 <stealth222> you mean cryptographic soundness?
222 2012-12-20 10:30:23 <sipa> no
223 2012-12-20 10:30:40 <sipa> i mean that the code doesn't crash
224 2012-12-20 10:30:57 <sipa> or does weird things to your balance
225 2012-12-20 10:31:11 <sipa> or has subtile security problems
226 2012-12-20 10:32:01 <sipa> bip32 being correct in terms of cryptography is omething that should be established before implementation
227 2012-12-20 10:32:09 <stealth222> of course
228 2012-12-20 10:32:50 <stealth222> anyhow, point is, the sooner it is implemented the sooner we can get people to try as hard as they can to break it
229 2012-12-20 10:32:58 <sipa> (and we're reasonably sure about that, but it should be looked at by a real cryptographer at some point...)
230 2012-12-20 10:35:30 <stealth222> anyhow, I do think that something like bip32 is the direction in which things should go...but for now, I could at least add multiwallet and dynamic wallet loading to bitcoind
231 2012-12-20 10:35:35 <stealth222> and I could do that fairly quickly
232 2012-12-20 10:35:53 <stealth222> just trying to figure out where I can best focus my efforts
233 2012-12-20 10:36:30 <stealth222> forget the network wallet stuff for now
234 2012-12-20 10:36:39 <stealth222> bip32 seems to be a far better solution to that
235 2012-12-20 10:37:27 <stealth222> the multiwallet and dynamic wallet loading would just reuse existing code and would be fairly easy to test for correctness
236 2012-12-20 10:38:26 <stealth222> the API might be a bit tricky, though
237 2012-12-20 10:38:35 <sipa> i think there's several pitfalls with it
238 2012-12-20 10:38:36 <stealth222> I could make it be fully backwards compatible
239 2012-12-20 10:38:46 <sipa> but certainly doable
240 2012-12-20 10:40:07 <stealth222> it's certainly much less ambitious than bip32...plus, we still need a way to watch arbitrary addresses regardless of scheme used to generate them
241 2012-12-20 10:41:32 <stealth222> I'm wondering whether it might not be better to have two sibling classes, one for signing-enabled wallets and another for watch-only addresses
242 2012-12-20 10:42:05 <stealth222> and to perhaps call the second one something other than "Wallet"
243 2012-12-20 10:43:38 <sipa> maybe
244 2012-12-20 10:44:01 <sipa> the CreateTransaction logic would only be in the second then
245 2012-12-20 10:44:16 <stealth222> you mean the first one
246 2012-12-20 10:44:22 <sipa> yes :D
247 2012-12-20 10:44:24 <sipa> and the encryption logic too
248 2012-12-20 10:44:29 <stealth222> right
249 2012-12-20 10:44:45 <sipa> but CKeyStore would probably be shared, even though several of its feature won't be used for watch-only wallets
250 2012-12-20 10:44:57 <stealth222> right
251 2012-12-20 10:45:29 <stealth222> no point in rewriting something that already exists unless it offers some significant maintenance or performance advantage
252 2012-12-20 10:46:01 <sipa> better modularisation is certainly a goal, but often not a priority
253 2012-12-20 10:46:17 <sipa> though i was it was :)
254 2012-12-20 10:48:02 <stealth222> right now, the wallet is an application singleton object, yes?
255 2012-12-20 10:48:12 <sipa> yes, but it's not that hard to change it
256 2012-12-20 10:48:24 <sipa> you can create a new CWallet, and register it
257 2012-12-20 10:49:19 <sipa> (never tested, but that was the intention ever since i wrote CWallet)
258 2012-12-20 10:49:32 <stealth222> right - so the trickiest part is probably the API
259 2012-12-20 10:49:56 <sipa> indeed, and that is why making it decide based on username/password would be so nice - no API changes at all :)
260 2012-12-20 10:51:10 <stealth222> would the login be per-session? per-connection (REST)?
261 2012-12-20 10:51:40 <stealth222> could multiple clients login under separate accounts simultaneously?
262 2012-12-20 10:52:41 <stealth222> also, with dynamic loading and unloading there's the issue of rescanning
263 2012-12-20 10:52:52 <stealth222> that might be nontrivial
264 2012-12-20 10:53:42 <stealth222> and obviously loading and unloading for each connection is insane
265 2012-12-20 10:56:33 <sipa> stealth222: JSON-RPC isn't persistent, afaik
266 2012-12-20 10:56:47 <sipa> so you can't really have sessions
267 2012-12-20 10:57:13 <sipa> and dynamic loading indeed has the problem of rescanning
268 2012-12-20 10:57:48 <sipa> so at least initially i'd just say specify all your wallets in the config, with associated username/password
269 2012-12-20 10:58:07 <sipa> also, not sure you're aware of how the BDB environment works?
270 2012-12-20 10:58:18 <stealth222> not at a low level
271 2012-12-20 10:59:08 <sipa> no, but you're aware that every database requires a BDB env to work in, and we try to make it look like databases are single files by detaching them from the env at shutdown?
272 2012-12-20 11:00:29 <stealth222> don't really have much experience with BDB - not sure what you mean
273 2012-12-20 11:01:08 <sipa> so in BDB you don't just have databases - every database is part of a database environment (a directory with several files in it, including replay logs)
274 2012-12-20 11:01:20 <sipa> and in case of a crash, you need the environment to be present for recovery
275 2012-12-20 11:01:47 <sipa> you can ask BDB to remove the connection between a database and an env, so it can be moved to a different env
276 2012-12-20 11:01:59 <sipa> that's what we do at shutdown, so wallet files can be moved around
277 2012-12-20 11:02:16 <sipa> it doesn't work in case of an unclean shutdown, however
278 2012-12-20 11:03:03 <stealth222> so if there's an unclean shutdown and I replace wallet.dat, next time I run it will complain?
279 2012-12-20 11:03:43 <sipa> the other way around: if you have an unclean shutdown, and then move that wallet.dat file to another system
280 2012-12-20 11:03:48 <sipa> it will just fail to load your wallet
281 2012-12-20 11:04:04 <stealth222> but if I restart it will fix the wallet?
282 2012-12-20 11:04:06 <sipa> no
283 2012-12-20 11:04:16 <sipa> there
284 2012-12-20 11:04:21 <Jouke> does the satoshiclient reuse change addresses?
285 2012-12-20 11:04:29 <sipa> Jouke: no
286 2012-12-20 11:04:37 <Jouke> ok, thanks.
287 2012-12-20 11:04:46 <sipa> stealth222: there's some hacky recovery code since 0.7 or 0.7.1, but in general, it will corrupt your wallet
288 2012-12-20 11:05:05 <sipa> it's just that BDB databases aren't intended to be just single files, but people want them to be
289 2012-12-20 11:05:45 <stealth222> would LevelDB solve this issue?
290 2012-12-20 11:05:58 <sipa> yes and no - it uses a single directory per database
291 2012-12-20 11:06:11 <sipa> there is no single file to be seen, so people couldn't assume it was
292 2012-12-20 11:06:21 <sipa> but leveldb would be overkill for wallets
293 2012-12-20 11:07:14 <stealth222> I wouldn't have a problem with one wallet per directory - as long as a wallet is easy to move around a file system
294 2012-12-20 11:07:32 <stealth222> if you need to move an entire directory, so be it
295 2012-12-20 11:07:42 <sipa> well i started working on a very simple own format for wallets which does just need a single file
296 2012-12-20 11:07:54 <stealth222> how far did you get?
297 2012-12-20 11:08:28 <sipa> it worked, but with some pitfalls (the file grows too fast, doesn't get compacted, little recovery code if something goes wrong, ...)
298 2012-12-20 11:08:57 <sipa> the solution is changing some wallet semantics first, so objects in it need far less rewriting
299 2012-12-20 11:09:08 <sipa> which isn't hard either, but again not a priority :)
300 2012-12-20 11:09:26 <stealth222> you keep saying "not a priority" but here I am offering to do this
301 2012-12-20 11:09:27 <stealth222> lol
302 2012-12-20 11:09:37 <sipa> why do you think i'm telling you this? :p
303 2012-12-20 11:09:47 <sipa> my priorities are not necessarily the same as yours
304 2012-12-20 11:10:29 <stealth222> anyhow, I
305 2012-12-20 11:10:48 <stealth222> I'm not necessarily interested in reinventing low-level file formats and database systems
306 2012-12-20 11:10:51 <sipa> stealth222: anyway, just so you're aware of this: if you implement multiwallet now, it will mean multiple wallets in the same BDB environment, and allowing wallet files to be anywhere in the filesystem would be letting people shoot them in the foot
307 2012-12-20 11:10:51 <stealth222> lol
308 2012-12-20 11:11:04 <sipa> *themself
309 2012-12-20 11:11:37 <stealth222> so then the limitation is that all the wallets must be in a single directory?
310 2012-12-20 11:11:56 <sipa> not necessarily, but if they're not, people will not know that it has any relation to the env directory
311 2012-12-20 11:12:18 <sipa> and for example having the wallet file on a USB stick but the env on your hard drive would be a recipe for disaster
312 2012-12-20 11:12:43 <sipa> unclean shutdown -> move USB stick to other system -> wallet corrupted
313 2012-12-20 11:12:53 <stealth222> ok, but for practical purposes, it would be less of a headache if we just required the wallets to all reside in the same directory
314 2012-12-20 11:13:01 <sipa> indeed
315 2012-12-20 11:13:12 <stealth222> and if we wanted to mount another file system, we'd have to make sure we first do a clean shutdown
316 2012-12-20 11:13:23 <sipa> so moving away for BDB (for this, and other reasons...) is more important than multiwallet support now
317 2012-12-20 11:13:46 <sipa> the wallet is the only thing still using BDB in the current code
318 2012-12-20 11:14:08 <stealth222> so isn't there any opensource solution to this?
319 2012-12-20 11:14:17 <stealth222> surely there are some decent libraries that take care of this
320 2012-12-20 11:14:38 <stealth222> you say LevelDB is overkill
321 2012-12-20 11:14:50 <sipa> sure there are, but they are all overkill, and added dependencies is not nice
322 2012-12-20 11:15:05 <sipa> we really just need something that maintains a list of key-value pairs :)
323 2012-12-20 11:15:19 <t7> std::map
324 2012-12-20 11:15:23 <sipa> even an append-only text file would be better than what we have now :p
325 2012-12-20 11:15:36 <sipa> t7: on disk, with some guarantees that prevent corruption
326 2012-12-20 11:16:04 <t7> i thought leveldb was a good idea
327 2012-12-20 11:16:26 <stealth222> if we're already using leveldb for everything else, why not just move the wallet to leveldb?
328 2012-12-20 11:16:50 <sipa> meh, an entire directory for a wallet :p
329 2012-12-20 11:17:41 <stealth222> we can tar that sucker if needed - lol
330 2012-12-20 11:18:06 <sipa> haha
331 2012-12-20 11:18:23 <sipa> anyway, gtg
332 2012-12-20 11:18:32 <stealth222> ok, take care
333 2012-12-20 13:50:56 <etotheipi_> stealth, you should really look at Armory :)
334 2012-12-20 13:51:30 <etotheipi_> gah, he's not here
335 2012-12-20 13:56:29 <BTCOxygen> etotheipi_: I would prefer electrum
336 2012-12-20 13:59:59 <sipa> etotheipi_: regarding the KDF... i prefer the more complex one (i just kinda like the math behind it :p), but if that would be difficult to get accepted by the community because of the complexity...
337 2012-12-20 14:00:14 <sipa> it's really just a table with a 16 32-bit tuples
338 2012-12-20 14:04:56 <etotheipi_> sipa: I'm rereading your old email & gist
339 2012-12-20 14:05:02 <etotheipi_> (from may 19)
340 2012-12-20 14:06:07 <oinooob> Usually good engineering is to remove as much as can be removed, but not more, while retaining function.
341 2012-12-20 14:06:36 <oinooob> (read: remove complexity)
342 2012-12-20 14:14:10 <sipa> etotheipi_: here's the math/results, but little description: https://gist.github.com/2731997
343 2012-12-20 14:18:01 <sipa> oinooob: it's not really implementation complexity, more like the explanation why particular numbers are better than the trivial ones is more difficulty
344 2012-12-20 14:18:49 <etotheipi_> sipa: yeah, i actually found that in your May email about it
345 2012-12-20 14:18:57 <etotheipi_> I haven't fully dug through the math, yet
346 2012-12-20 14:19:03 <oinooob> sipa: Q: f(n) is supposed to be in the range 0..1/255 (ish))?
347 2012-12-20 14:20:26 <sipa> oinooob: f(n) is the fraction of seeds that reach a certain iteration for which iteration will stop at that point
348 2012-12-20 14:20:32 <sipa> so it's a number between 0 and 1
349 2012-12-20 14:20:59 <sipa> if you do the math for difficulties that start closer to see, you'll see larger numbers than 1/256
350 2012-12-20 14:21:02 <oinooob> I based my question on the table at the end of that page.
351 2012-12-20 14:22:00 <sipa> for the first step, i(n)/f(n) should be equal to the difficulty
352 2012-12-20 14:22:30 <sipa> so if the first difficulty is 4^N, i(n) is 2^N+1 and f(n) is 1/(2^N-1)
353 2012-12-20 14:22:39 <sipa> so indeed, it starts at 1/255
354 2012-12-20 14:23:17 <sipa> actually at 257/65536, which is very close to 1/255
355 2012-12-20 14:23:27 <oinooob> OK. With n=1, fn(n) is around 1/256 (a tad more) as 2^32*fn(n) is listed to be 2^24+2^16.
356 2012-12-20 14:26:20 <oinooob> Perhaps I'm missing something, but it seems to me 7 out of 32 bits went MIA, and another few bits got diluted along the way?
357 2012-12-20 14:27:10 <oinooob> (for low n)
358 2012-12-20 14:27:11 <sipa> ?
359 2012-12-20 14:27:45 <oinooob> My bad! It's the line for for n=0 I'm referring, not n=1 !
360 2012-12-20 14:28:35 <sipa> oinooob: you understand the purpose of the system? because there's a long private email discussion that preceeded it
361 2012-12-20 14:30:32 <oinooob> sipa: No, sorry to have jumped in like this. I haven't seen any of the previous discussions leading up to this. Perhaps you could just scratch the leading "oi" from my nick. :)
362 2012-12-20 14:32:24 <sipa> oinooob: https://bitcointalk.org/index.php?topic=102349.0
363 2012-12-20 14:33:29 <oinooob> 
ACTION slaps forehead. NOW it sinks in (even without looking at the thread). Thanks for the URL, I'll look at it.

364 2012-12-20 14:35:12 <etotheipi_> so sipa, let me just clarify the terms in your equation, since i'ts rather abstract
365 2012-12-20 14:36:36 <etotheipi_> N(n) is the difficulty of producing the seed, P(n) is the chance of a particular seed not satisfying f() for n-1 iterations, and thus C(n) is the probability that it satisfies f() after exactly n iterations
366 2012-12-20 14:36:56 <sipa> hmm, that may be true - i'll need to look closer at it
367 2012-12-20 14:37:35 <sipa> N(n) is indeed the intended difficulty at step n
368 2012-12-20 14:37:54 <sipa> P(n) is the chance of already having bailed out before reaching step n
369 2012-12-20 14:38:33 <etotheipi_> isn't P(n) actually 1 minus that probability
370 2012-12-20 14:38:40 <sipa> yup!
371 2012-12-20 14:39:10 <oinooob> It's written as such at least. :)
372 2012-12-20 14:39:16 <etotheipi_> so C(n) is the probability of satisfying the condition after exactly n steps
373 2012-12-20 14:39:55 <sipa> indeed
374 2012-12-20 14:40:40 <sipa> H(n) is the number of iterations when limiting to n-1 steps
375 2012-12-20 14:41:10 <sipa> i think
376 2012-12-20 14:41:17 <sipa> it's been a while since i wrote that
377 2012-12-20 14:43:42 <etotheipi_> I'm going ot have to come back to this... I can't fathom "wasting" another day on IRC when I need to work on my new wallets (other aspects of it)
378 2012-12-20 14:44:19 <etotheipi_> sipa: I understand what you're doing there, I just can't follow the math step-by-step yet
379 2012-12-20 14:44:20 <oinooob> Pfft, admit it, you love the math!
380 2012-12-20 14:44:33 <etotheipi_> I am, after all, a statistician
381 2012-12-20 14:44:49 <sipa> etotheipi_: it was really just a braindump at the time, it certainly could use a lot more explanation
382 2012-12-20 14:45:03 <etotheipi_> sipa: understood...
383 2012-12-20 14:46:33 <etotheipi_> sipa: did you ever get a chance to look at my "new wallet ideas"?  https://bitcointalk.org/index.php?topic=128119.0
384 2012-12-20 14:47:11 <etotheipi_> in particular, I'm working on the architecture right now for (2) and trying to predict whether what I'm doing is sane
385 2012-12-20 14:47:21 <sipa> etotheipi_: i saw the thread but kinda decided not to look deeper... i already have way too much i want to work on
386 2012-12-20 14:47:28 <etotheipi_> sipa: haha, okay
387 2012-12-20 14:47:44 <etotheipi_> I just feedback on the "lightly encrypted" comments&P2SH file
388 2012-12-20 14:48:04 <etotheipi_> it sounds a little crazy, but I think it's justified and not too difficult
389 2012-12-20 14:49:49 <sipa> etotheipi_: it's easy to verify empirically the numbers in that table are approximately correct... just run a simulation and measure the average number of iterations :)
390 2012-12-20 14:51:12 <gmaxwell> etotheipi_: why not just use the private key material to encrypt the metadata. If you want to give e.g. a watch wallet access to it, you pass the private data through a one way function first, so then you're able to give out copies of the metadata.
391 2012-12-20 14:51:37 <gmaxwell> and then you're not automatically getting access to it if you don't want it.
392 2012-12-20 14:51:57 <oinooob> I think (2) is the way to got. While me, as a security dude appreciate (1), it's simeply not feasible for wider distribution/usage (as I think you realized).
393 2012-12-20 14:52:40 <etotheipi_> oinooob: I think (1) is perfectly feasible
394 2012-12-20 14:54:02 <oinooob> etotheipi_: Please elaborate, because I see it the completely other way (imagining a user in the future using his wallet on his phone, with no live backup/synch to "safe" storage).
395 2012-12-20 14:54:14 <gmaxwell> etotheipi_: the pairing model sounds fine to me, but why not fully generalize it to N-of-M way.. just allow people to provide up to N-m parters and then set the quarum level.
396 2012-12-20 14:54:20 <etotheipi_> who said there's no backup?
397 2012-12-20 14:54:49 <etotheipi_> you generate wallet A and B, print backups of both, put them in your safe-deposit box, then give your desktop AB' and your phone A'B
398 2012-12-20 14:55:25 <oinooob> gmaxwell: Hmm, now you're starting to remind me of zfs. :) n-way parity and stuff.
399 2012-12-20 14:55:27 <etotheipi_> gmaxwell: that was essentially my intention...
400 2012-12-20 14:55:43 <etotheipi_> "(design will also accommodate 2-of-3 and 3-of-3 wallet combos)"
401 2012-12-20 14:55:55 <etotheipi_> what I meant was "M-of-N"
402 2012-12-20 14:56:06 <gmaxwell> etotheipi_: K. Sure.
403 2012-12-20 14:56:08 <helo> mining on testnet, i keep getting excited, and then disappointed when i see the mining rewards
404 2012-12-20 14:56:46 <gmaxwell> "I'm going to solve a block!!!" "oh... I only got tnbtc. :("
405 2012-12-20 14:56:54 <oinooob> etotheipi_: OK, now I also can join the page and agree (1) sounds more sane compared to how I originally read it.
406 2012-12-20 14:57:11 <etotheipi_> gmaxwell: still better than TBTC
407 2012-12-20 14:57:51 <oinooob> or TBC... (har, har)
408 2012-12-20 14:58:22 <gmaxwell> etotheipi_: so??? on (2)  why not  metakey = H(private data||'metakey') then you store that in your wallet file, optionally hand it out to watchers if you want them having metadata access (or not). And if you lose your data, anyone with the private key access can recover it.
409 2012-12-20 14:58:27 <etotheipi_> oinooob: if the backup aspect wasn't clear, I should amend (1) to make that clear
410 2012-12-20 14:59:04 <gmaxwell> etotheipi_: and I wouldn't call it 'lightly encrypted'??? I think that implies weak encryption. Not sure what I'd call it.
411 2012-12-20 14:59:16 <etotheipi_> gmaxwell: I don't see the necessity of adding extra keys to the mess
412 2012-12-20 14:59:37 <etotheipi_> gmaxwell: if you have access to the chaincodes of the referenced keys, you can decrypt it
413 2012-12-20 14:59:46 <gmaxwell> etotheipi_: there isn't functionally an extra key.
414 2012-12-20 14:59:49 <etotheipi_> if you don't have access to the chaincodes... then you don't need it
415 2012-12-20 15:00:13 <gmaxwell> etotheipi_: but what if you have chaincode access and _shouldn't_ have access to the metadata?
416 2012-12-20 15:00:24 <etotheipi_> (i.e. if you don't have the key&chaincode to decrypt comment for address X, it's because X isn't in your wallet, and thus you don't care what the comment/P2SH is)
417 2012-12-20 15:00:48 <gmaxwell> e.g. I don't want my webserver which generates keys for me to have the keying material needed to read my cloud backups of my metadata.
418 2012-12-20 15:00:58 <oinooob> etotheipi_: What probably threw me off-track was the missing m-n early in the text, and the mention of "script" that (to me) implies the ability to throw the software way off track (read: making it mathematically impossible to prove).
419 2012-12-20 15:01:56 <oinooob> (replace m-n with A, A' and so on)
420 2012-12-20 15:02:27 <gmaxwell> etotheipi_: so instead I propose you H(private) for the metadata key but then leave that value unencrypted in your wallet files.  Then someone has the option of not putting data on their webserver that would give an attacker access to their cloud metadata.
421 2012-12-20 15:02:48 <gmaxwell> etotheipi_: but you still have a single piece of data to backup (the stuff to derrive the private keys)
422 2012-12-20 15:02:57 <etotheipi_> sounds like I need to generalize (1) better... I was focused on the 2-factor-auth use case and wanted to tangentially point out it could easily be expanded for M-of-N, but maybe I should do it the other way arou nd
423 2012-12-20 15:04:19 <gmaxwell> etotheipi_: also, the method of specifying all the watching mates should allow you to put in some kind of description and some kind of URI for instructions on how to gather signatures.
424 2012-12-20 15:04:23 <oinooob> etotheipi_: For a tangential idea it grew to a decently sized paragraph. :-)
425 2012-12-20 15:04:50 <etotheipi_> gmaxwell: I know what you're getting at, but I'm not convinced that what you're saying is a risk... there's still two things that have to be compromised to be able to access that metadata
426 2012-12-20 15:06:19 <etotheipi_> oinooob: the two-factor auth is something that's been in demand for a long time... so I wanted to highlight that...  the tangential idea that it can be expanded to arbitrary M-of-N is really only captured in the parenthetical comments
427 2012-12-20 15:06:38 <etotheipi_> instead I should just talk about "linked wallets" and then add a short list of use-cases, highlighting 2-factor auth
428 2012-12-20 15:06:52 <oinooob> etotheipi_: OK, gotcha.
429 2012-12-20 15:07:14 <gmaxwell> etotheipi_: cloud storage should generally not be considered secure, e.g. in the US it's available without a warrant. Many users will likely use the same cloud accounts to backup their webserver and their 'offline' wallet so it's likely the webserver will already have access.  Plus it costs you nothing if you don't want to support the mode I suggest. Just an extra hash operation and a bit of data in your wallet format, and an extra record in
430 2012-12-20 15:07:44 <oinooob> As for 2-factor, I can only agree. Especially for those use-cases where an (encrypted) walled is stored at a 3rd party.
431 2012-12-20 15:09:19 <gmaxwell> etotheipi_: As far what you should call the security on it perhaps "wallet private metadata" ?  I'd hesitant to call it encrypted when you keep the key for it unencrypted on disk. :P
432 2012-12-20 15:09:51 <etotheipi_> gmaxwell: that's a fair point
433 2012-12-20 15:10:00 <etotheipi_> that's why "lightly encrypted" was in quotes, because I wasn't sure how to phrase it
434 2012-12-20 15:10:31 <etotheipi_> I need to rewrite these points ... there was a (3) that I added later in the thread
435 2012-12-20 15:10:35 <oinooob> "not even ofuscated"? ;-)
436 2012-12-20 15:10:57 <etotheipi_> encrypting the public keys, too
437 2012-12-20 15:11:19 <gmaxwell> etotheipi_: meh. use disk encryption.
438 2012-12-20 15:11:21 <etotheipi_> I'm sure gmaxwell will pick this apart
439 2012-12-20 15:11:38 <etotheipi_> but what I was thinking was that the user will still have one encryption key
440 2012-12-20 15:11:53 <etotheipi_> it will just use 2 different KDFs (it doesn't have to be the same, but I know some users want it)
441 2012-12-20 15:11:55 <gmaxwell> etotheipi_: (the reason that privkey encryption is not a duplicate of disk encryption is that it can stay encrypted the 99.999999% of the time you aren't sending funds)
442 2012-12-20 15:12:02 <gmaxwell> uhg no.
443 2012-12-20 15:12:20 <gmaxwell> You'd remove all advantage of private key encryption over disk encryption by doing that.
444 2012-12-20 15:12:46 <etotheipi_> well, you'd make it slightly easier for keyloggers to grab your passphrase
445 2012-12-20 15:12:55 <etotheipi_> but if they have a keylogger, they'll get it eventually, anyway
446 2012-12-20 15:13:02 <etotheipi_> unless you really never user it
447 2012-12-20 15:13:04 <etotheipi_> *use
448 2012-12-20 15:13:15 <oinooob> If you have a keylogger installed, you're hosed however you view it.
449 2012-12-20 15:13:20 <gmaxwell> much easier??? for example, I have copies of my offline wallet on online machines, I just never decrypt it on them.
450 2012-12-20 15:13:40 <etotheipi_> hold on...
451 2012-12-20 15:14:22 <gmaxwell> etotheipi_: if someone is on your machine your privacy is shot in any case. the marginal value of encrypting the public parts is pretty low, and negative if it makes you type your private-controlling key often.
452 2012-12-20 15:14:45 <etotheipi_> gmaxwell: it would only be typed once, when the app is started, but I know what you're saying
453 2012-12-20 15:15:03 <etotheipi_> the point is that the data stays on disk encrypted, only decrypted in RAM... I'm not sure the disk encryption does the same thing
454 2012-12-20 15:15:10 <gmaxwell> I dunno about you, but I go weeks without spending bitcoin, but seldom go days without lookin at my balances. ::shrugs::
455 2012-12-20 15:15:16 <gmaxwell> etotheipi_: thats what disk encryption does too.
456 2012-12-20 15:15:47 <etotheipi_> gmaxwell: tell me if I'm wrong... during boot I put in my passphrase for my encrypted volume
457 2012-12-20 15:16:00 <etotheipi_> then *any* process can leverage the unlocked volume and read it
458 2012-12-20 15:16:13 <etotheipi_> sure, if the laptop was off to start, it's the same
459 2012-12-20 15:16:24 <etotheipi_> but if it's already on and unlocked, disk access is available to all processes
460 2012-12-20 15:16:31 <etotheipi_> is that correct?
461 2012-12-20 15:16:37 <gmaxwell> (and also closes some the zillion other channels your system leaks data, e.g. bitcoin's logs. provides some robustness to data manipuation)
462 2012-12-20 15:16:55 <oinooob> etotheipi_: Yes, that is correct.
463 2012-12-20 15:16:58 <gmaxwell> etotheipi_: sure. Then again, your process memory is available to all processes as well.
464 2012-12-20 15:17:06 <etotheipi_> gmaxwell: is it?
465 2012-12-20 15:17:31 <etotheipi_> how easy is it to pull data out of another process's RAM?
466 2012-12-20 15:17:36 <etotheipi_> I thought that was rather involved and required root
467 2012-12-20 15:17:40 <gmaxwell> etotheipi_: Yes. without any bugs to any process run by the same user, and in practice with bugs to any process on the whole system because everything is stuffed with local root exploits.
468 2012-12-20 15:17:48 <oinooob> Any process with enough privileges can e.g. ReadProcessMemory on Windows, or kmem, and so on.
469 2012-12-20 15:17:53 <gmaxwell> etotheipi_: hah. no.
470 2012-12-20 15:18:04 <gmaxwell> etotheipi_: otherwise debuggers would be rather annoying!
471 2012-12-20 15:18:14 <sipa> one just uses /dev/exynos-mem
472 2012-12-20 15:18:28 <oinooob> etotheipi_: If you start 2 processes, your process B can read memory from process A.
473 2012-12-20 15:19:00 <oinooob> (generally, at least)
474 2012-12-20 15:19:02 <gmaxwell> in linux it's possible to lock this down??? even within a user, but no one does because it breaks lots of stuff... and even if they did it's likely to be very porous.
475 2012-12-20 15:19:44 <etotheipi_> okay, I am mistaken then
476 2012-12-20 15:20:16 <etotheipi_> but it still seems like an incremental increase in security
477 2012-12-20 15:20:31 <etotheipi_> and I still consider it, because it's not that much extra effort to do this
478 2012-12-20 15:20:56 <gmaxwell> etotheipi_: if it were true it would make what you're suggesting more attractive... but if people only send a fraction of the times they start it still seems clearly negative to me. Also make things like this more effective by far:
479 2012-12-20 15:21:11 <etotheipi_> first of all, the attacker has to be more advanced to dig into the processes memory and find the key
480 2012-12-20 15:21:31 <etotheipi_> whereas any "crawler" that makes it onto your system will just go to your .armory directory and grab your unencrypted file
481 2012-12-20 15:21:37 <gmaxwell> etotheipi_: I get on your system. You use armory. The wallet is encrypted.  I kill your armory process and start a keylogger. 100% chance of success even if six hours later you figure out you're hacked and kick me out.
482 2012-12-20 15:22:15 <etotheipi_> gmaxwell: so that's an argument for *not allowing* the outer encryption passphrase to be the same as the inner passphrase
483 2012-12-20 15:22:30 <etotheipi_> and I agree
484 2012-12-20 15:22:56 <gmaxwell> etotheipi_: oh, well that I could agree with except for the fact that joe user isn't going to get the subtly of this and are going to make them stupidpassword and stupidpassword1
485 2012-12-20 15:23:20 <gmaxwell> etotheipi_: if you do this, make sure you make it so that the _inner_ password is sufficient to recover the wallet, or otherwise you'll increase the risk of data loss.
486 2012-12-20 15:23:39 <etotheipi_> gmaxwell: agreed... I've never trusted users in this way
487 2012-12-20 15:23:41 <gmaxwell> (you can do this by encrypting master key with the inner password too and storing that)
488 2012-12-20 15:24:04 <etotheipi_> gmaxwell: so do you have a recommendation to protect the watching-only wallet?  or is it just "use disk encryption"?
489 2012-12-20 15:24:30 <gmaxwell> etotheipi_: the real benefit of what you're describing is this:  bozo user copies his wallet to his website and is then sad his pubkeys are all public.
490 2012-12-20 15:24:48 <gmaxwell> and disk encryption doesn't save that case.
491 2012-12-20 15:25:33 <oinooob> Call me old-school, but I still think any user having let a keylogger onto his or her system is, as some in security say, "hosed". :-)
492 2012-12-20 15:25:39 <gmaxwell> I do think "use disk encryption" is the starting point though.  If you care about security and you're not using that??? give up.  There are so many ways your system leaks private data to the local disk....
493 2012-12-20 15:25:45 <etotheipi_> gmaxwell: the use case is:  I manage my $1million in BTC using a watching-only wallet on my primary laptop, but when that laptop gets stolen while in hibernate, the theif now knows that I own $1million and he has all my personal info
494 2012-12-20 15:26:06 <gmaxwell> etotheipi_: armory was running when you went into hybernate in any case.
495 2012-12-20 15:26:17 <oinooob> I think disk-encryption to protect your wallet is absurd.
496 2012-12-20 15:26:19 <gmaxwell> well I assume you mean suspend.
497 2012-12-20 15:26:46 <gmaxwell> (hybernate would have required the disk encryption password to come back up again)
498 2012-12-20 15:26:52 <gmaxwell> (normally)
499 2012-12-20 15:26:53 <etotheipi_> gmaxwell: okay, fair enough
500 2012-12-20 15:27:15 <etotheipi_> but the point is that you have to have a *dramatically more advanced* attacker to figure out how to pull that data out of RAM, compared to just copying a file
501 2012-12-20 15:28:12 <etotheipi_> additionally, it protects you if Arrmory isn't running at all
502 2012-12-20 15:28:17 <gmaxwell> etotheipi_: no??? if it's running and you're the evil maid you just see it on the screen. If you're some crawller, it's all just prefab code??? if it was profiltable at all to do this, then it will happen. software already had to be specalized to know it should bother copying wallets in the first place.
503 2012-12-20 15:28:50 <gmaxwell> etotheipi_: yea, but it's not free protection: slightly increased risk of data loss, probably greatly increased risk of private key exposure. (when the user uses related keys for both)
504 2012-12-20 15:28:52 <etotheipi_> okay, maybe I overestimate the difficulty of doing this, because I've never done it myself
505 2012-12-20 15:29:23 <gmaxwell> gdb -p 1234  p global_thing->cryptokey
506 2012-12-20 15:30:03 <gmaxwell> actually writing code to do it isn't that hard either, you attach the memory and just do a search for some common pattern in the struct that stores it.
507 2012-12-20 15:30:19 <oinooob> "If it wasn't difficult, everyone would do it!"
508 2012-12-20 15:30:56 <gmaxwell> it could be as simple as 5 lines of code, assuming there happened to be something nice and matchable a fixed offset from the key.
509 2012-12-20 15:30:58 <etotheipi_> well there's still a gap I want to fill
510 2012-12-20 15:31:15 <etotheipi_> which is... disk encryption is still pretty advanced for users
511 2012-12-20 15:31:46 <oinooob> Agreed, and disk-encryption is slowing systems down.
512 2012-12-20 15:31:48 <etotheipi_> the other reason I wanted to do this is so that it is EASY for users who don't need/want to encrypt their whole disk
513 2012-12-20 15:32:00 <gmaxwell> etotheipi_: you could do better with your time by helping out with that, I expect. It's pretty important generally for people who care about privacy. In modern linux distros its just a checkbox, no biggie.
514 2012-12-20 15:32:14 <gmaxwell> oinooob: it makes pratically no difference in linux.
515 2012-12-20 15:32:18 <oinooob> etotheipi_: I think you are on the right track. Do NOT assume disk encryption.
516 2012-12-20 15:32:54 <etotheipi_> I mean, I could say "if you want to protect this privacy-snafu data, encrypt your disk using one of the following tools..."
517 2012-12-20 15:32:55 <etotheipi_> but I don' tlike that
518 2012-12-20 15:33:01 <etotheipi_> it's not a bad recommendation
519 2012-12-20 15:33:06 <oinooob> gmaxwell: I disagree. I'd say disk encryption is still a rather fringe area for personal use.
520 2012-12-20 15:33:09 <etotheipi_> but I don't think users will go through the effort
521 2012-12-20 15:33:17 <gmaxwell> etotheipi_: but ??? well you need to??? because if they don't then the privacy snafu is still there and they have a false sense of security.
522 2012-12-20 15:33:30 <gmaxwell> oinooob: you disagree with what?
523 2012-12-20 15:33:50 <etotheipi_> haha, maybe make the outer encryption a 4-digit PIN number with a 10 sec KDF :)
524 2012-12-20 15:33:53 <BTCOxygen> Which encryption tool is best to encrypt a bitcoin wallet
525 2012-12-20 15:33:57 <gmaxwell> etotheipi_: hm!.
526 2012-12-20 15:34:26 <etotheipi_> though, the ywould then probably change their inner encryption password after the fact, to be the 4digit PIN + 2 digits
527 2012-12-20 15:35:15 <gmaxwell> etotheipi_: well you can still check the similarity... but they'll make ther outer 1234 then make the inner qwer ::sigh::
528 2012-12-20 15:35:26 <oinooob> gmaxwell: I disagree with your suggestion it's "easy". I agree with my assertion it's a rather fringe are still. I state it's impossible to use if you need high-speed disk access (this disqualifies most laptops, so perhaps we're talking about different scenarios).
529 2012-12-20 15:35:32 <BTCOxygen> Which encryption tool is best to encrypt a bitcoin wallet?, And is it safe to upload wallet.dat file to services like Dropbox wothout encrypting it?
530 2012-12-20 15:35:52 <etotheipi_> oinooob: I'm not sure that it's really a speed drag
531 2012-12-20 15:35:58 <oinooob> s/are still/area still/
532 2012-12-20 15:36:21 <etotheipi_> if it's encrypted with an AES256 key, I'm pretty sure that AES operates much faster on CPUs than the I/O speed of the disk
533 2012-12-20 15:36:31 <etotheipi_> and with very low resources
534 2012-12-20 15:36:31 <gmaxwell> oinooob: You're factually incorrect. It is a checkbox during install for modern linux distros (which was what I limited the easy to) and on current cpus AES is much faster than SATA and there is no latency impact.
535 2012-12-20 15:36:58 <etotheipi_> gmaxwell: but so few users are linux users
536 2012-12-20 15:37:09 <etotheipi_> I agree with you that it *can* be easy, but it rarely is
537 2012-12-20 15:37:19 <gmaxwell> now, you run truecrypt that is another matter, alas. But thats because the software is lame, it's not a problem with crypto itself.
538 2012-12-20 15:37:45 <BTCOxygen> etotheipi_: Yeah, most users use windows
539 2012-12-20 15:37:50 <gmaxwell> etotheipi_: yea, windows is the issue there??? I agree.  But it doesn't change the fact that encrypting your armory public data does almost nothing to deal with the privacy snafu on your disk.
540 2012-12-20 15:37:52 <etotheipi_> BTCOxygen: your Bitcoin wallet already has encryption on the private keys (if you enable it)
541 2012-12-20 15:37:56 <oinooob> If you have an Intel CPU with AES in it, it's quite possible it's fast.  Faster than I/O? Dream on! I can easily read half a GB/s sustained from an SSD. Try that with AES enabled. :-)
542 2012-12-20 15:38:43 <gmaxwell> oinooob: my deskop does 10Gbit/sec of AES per core. It's simply much faster than sata3.
543 2012-12-20 15:38:46 <etotheipi_> BTCOxygen: but I wouldn't go backing it up on wholly insecure channels (which we must assume dropbox is)
544 2012-12-20 15:39:12 <etotheipi_> BTCOxygen: but actually, it's a matter of paranoia and how much you hold
545 2012-12-20 15:39:13 <gavinandresen> 
ACTION wonders if he should mention how approximately nobody will be using traditional PCs in 5 or 10 years

546 2012-12-20 15:39:14 <gmaxwell> oinooob: and I've actually benchmarked this.
547 2012-12-20 15:39:18 <oinooob> gmaxwell: Shitting me? 10Gbps???
548 2012-12-20 15:39:40 <gmaxwell> oinooob: per-core. go go aesni.
549 2012-12-20 15:39:45 <etotheipi_> oinooob: what gmaxwell just said doesn't surprise me... AES256 is ridiculously fast
550 2012-12-20 15:40:25 <oinooob> I know it was designed to be fast, like 24 clocks/byte on IA32 or somthing like that.
551 2012-12-20 15:40:25 <sipa> gmaxwell: but that is *with* hw instructions for AES
552 2012-12-20 15:40:37 <etotheipi_> actually, 10Gbps is kind of high
553 2012-12-20 15:40:50 <gmaxwell> and besides, even if was a slowdown ... it would slows you down by a couple months of moore's law growth. whoptie do. If you care about privacy enough that you'd consider typing in a passphrase every time you start some program, it would be be a good tradeoff.
554 2012-12-20 15:40:50 <sipa> gmaxwell: oinoob wad referring to the case where you don't havr those
555 2012-12-20 15:41:08 <gmaxwell> sipa: "If you have an Intel CPU with AES in it, it's quite possible it's fast.  Faster than I/O? Dream on!" Hm?
556 2012-12-20 15:41:11 <etotheipi_> that's faster than most clocks.... but it wouldn't surprise me if AES is still as fast as the max SATA3 IO
557 2012-12-20 15:41:24 <sipa> oh, misread
558 2012-12-20 15:42:11 <etotheipi_> but I think this is still a relevant feature to address... I think there should be an option for users who don't want to use full-disk/volume encryption
559 2012-12-20 15:42:52 <gmaxwell> etotheipi_: I still think your concern should instead just be copies of the file (E.g. backups)
560 2012-12-20 15:43:23 <gmaxwell> etotheipi_: but still I dunno how to weigh that aganst them making their private key key less secure. :(
561 2012-12-20 15:43:24 <oinooob> etotheipi_: I'm with you 100%. The ones using disk encryption adds another layer to their _data_, but it shouldn't affect BC at all (IMHO).
562 2012-12-20 15:43:41 <etotheipi_> gmaxwell: I think that it's too much to ask to say "wnat to protect this data?  go figure out disk encryption"
563 2012-12-20 15:43:55 <oinooob> Agreed.
564 2012-12-20 15:44:02 <gmaxwell> etotheipi_: I think you're incorrect. You're actually _not_ protecting that data without it.
565 2012-12-20 15:44:13 <etotheipi_> and that it's very important to some users to protect that data, at least between boots
566 2012-12-20 15:44:35 <etotheipi_> but I do agree, I don't want to degrade the security of the private key encryption
567 2012-12-20 15:44:41 <etotheipi_> so is there a solution?
568 2012-12-20 15:44:49 <gmaxwell> You're creating a false sense of security. For example, all the users txn IDs will end up in the bitcoin logs for example. All their browsing history in their caches which will often give away ther spending behavior, etc.
569 2012-12-20 15:44:49 <oinooob> There is "protect" and there is "protect".
570 2012-12-20 15:45:41 <etotheipi_> at least in Armory, I don't think any of that is logged unless you use the --debug flag
571 2012-12-20 15:45:52 <etotheipi_> but it's good point that should be confirmed
572 2012-12-20 15:46:12 <etotheipi_> if you're going to go to the effort to protect the public key, I should make sure the log doesn't record the same information
573 2012-12-20 15:46:33 <etotheipi_> bitcoin-qt logging is something out of my control though
574 2012-12-20 15:47:16 <ThomasV> etotheipi_: does armory implement bip 32?
575 2012-12-20 15:47:24 <etotheipi_> ThomasV: it will, soon
576 2012-12-20 15:47:44 <etotheipi_> ThomasV: I have all the crypto implemented... but I'm working on the new wallet files that will house the BIP 32 data
577 2012-12-20 15:48:08 <ThomasV> etotheipi_: is bip 32 final?
578 2012-12-20 15:48:18 <sipa> i have code for generating test vectors ready; i'll soon publish some
579 2012-12-20 15:48:24 <gmaxwell> etotheipi_: how about this???  if the system has a TPM encrypt the public wallet data with F(private data)=x, and store x in the TPM.  If you stay on the same system, its autodecrypted. If you change systems you must provide the private key to unlock the public data?
580 2012-12-20 15:48:31 <etotheipi_> and by "working on it" I mean, fending off gmaxwell who continues to point out security concerns with every thing I propose :)
581 2012-12-20 15:48:41 <sipa> i suppose i should first request input from the dev mailing list ... and from a cryptographer
582 2012-12-20 15:48:46 <etotheipi_> sipa: please do... I'm ready to run some crypto unit-tests on that code
583 2012-12-20 15:48:53 <sipa> speaking of which, i should just ask Hal...
584 2012-12-20 15:49:11 <gmaxwell> good call on hal, at least he'll understand the problem's were trying to solve.
585 2012-12-20 15:49:31 <gmaxwell> One problem with bip32 as it stands is that its a little opaque to someone who doesn't really know about how bitcoin uses keys.
586 2012-12-20 15:49:33 <sipa> yes, and he seems interested in keeping up with bitcoin as well
587 2012-12-20 15:49:39 <gmaxwell> (I mean the motivation is opaque)
588 2012-12-20 15:49:58 <ThomasV> etotheipi_: I want to use bip 32 in electrum, but I don't want to do it before it is final, because it would double the pain for users if we need to change again
589 2012-12-20 15:50:03 <fuzion24> So, I am working on modifying bitcoinj to store the full block chain, I was curious if anyone here has worked on the satoshi client as to how exactly they implemented it
590 2012-12-20 15:50:03 <sipa> it should carry notices about revealing parent pubkeys
591 2012-12-20 15:50:16 <gmaxwell> So I expect that talking to someone like DJB may first waste an hour of "just use a random value for your private keys!!" :P
592 2012-12-20 15:50:37 <etotheipi_> ThomasV: I'm not going to let users switch to the new wallets using BIP 32 until it's finalized... but there's a lot of work to do, so I can iron that out before it's released
593 2012-12-20 15:51:01 <sipa> if something changes, it's probably just details
594 2012-12-20 15:51:01 <ThomasV> a lot of work?
595 2012-12-20 15:51:22 <gmaxwell> ThomasV: he's redesigning his wallets far beyond just bip32.
596 2012-12-20 15:51:34 <etotheipi_> ThomasV: the crypto implementation is not terribly difficult, but I have a lot of work to do on the new wallet files themselves, some features of which depend on BIP 32
597 2012-12-20 15:52:27 <etotheipi_> for instance, accommodating "linked" wallets, so that BIP32 wallets can have counterparts on other devices/computers, and all addresses generated will be multi-sig/P2SH addresses
598 2012-12-20 15:52:56 <gmaxwell> etotheipi_: In any case, yea, sorry for bludgoning you so much on the public part encryption to be honest I don't really know. I think you were overestimating the benefits before, and I am probably overestmating the risks. I do at least see value in the cases where people foolishly throw their wallets onto websites for backup and such at least.
599 2012-12-20 15:53:25 <etotheipi_> gmaxwell: trust me, I appreciate your input, even if the risks may be overstated
600 2012-12-20 15:53:38 <ThomasV> etotheipi_: when you have "linked" wallets, is there data they need to exchange that is not in the blockchain?
601 2012-12-20 15:54:23 <etotheipi_> ThomasV: yes -- ideally all wallets are created on an offline computer, and distributed to all parties:  computer A get AB'C', B get A'BC' and C gets A'B'C
602 2012-12-20 15:54:41 <etotheipi_> but I also want to accommodate linking if all the wallets were created separately
603 2012-12-20 15:54:45 <gmaxwell> the primes there are 'watch only wallets'.
604 2012-12-20 15:55:20 <etotheipi_> so, there's a bit of work to handle all the use cases
605 2012-12-20 15:55:30 <gmaxwell> The idea is that one computer has  subwallet A and watching-subwallet B.  And the other computer has subwallet B and watching subwallet A.  To sign a transaction you have to pass a partally signed txn between them.
606 2012-12-20 15:55:39 <gmaxwell> (somehow)
607 2012-12-20 15:55:42 <ThomasV> but that's not something they need to exchange; a wallet knows which private keys it has
608 2012-12-20 15:56:05 <gmaxwell> ThomasV: yes, they need to exchange the watching parts.
609 2012-12-20 15:56:09 <etotheipi_> well, until I get feedback on BIP 10 about how other people want to accommodate this, I'm going to do minimal modifications to BIP 10
610 2012-12-20 15:56:16 <gmaxwell> So that it knows what public keys its partner has.
611 2012-12-20 15:57:03 <gmaxwell> and with all sets of public keys you can derrive the p2sh addresses.
612 2012-12-20 15:57:40 <etotheipi_> actually, there is one thing I still need to resolve... which is sending data between devices... you don't want to plug your phone into your computer to do the transfer, and text messages are too small... email is ideal, but it's probably too complicated to link both devices to email addresses
613 2012-12-20 15:57:54 <etotheipi_> (this is for the purposes of passing around partially-signed tx)
614 2012-12-20 15:58:16 <ThomasV> gmaxwell: but they all have all public keys.. do they need to know which private keys each partner does NOT have?
615 2012-12-20 15:58:22 <etotheipi_> I assume that the computer will always start the transfer:  it will sign half the tx... then somehow pass it to the phone which will add the second signature and then broadcast
616 2012-12-20 15:59:00 <gmaxwell> ThomasV: each would ~assume~ only he has his own private keys, and that his partner has the private keys matching the partners public chain.
617 2012-12-20 15:59:34 <gmaxwell> (keeping in mind 2 of 2 is only one case, e.g. I see this being very useful for n of m for organization fund management)
618 2012-12-20 16:00:12 <ThomasV> I mean, each could sign his part of the tx, and pass the partially signed tx around, without worrying about who owns what.. or am I wrong?
619 2012-12-20 16:01:10 <gmaxwell> ThomasV: you can't generate the address to begin with unless you know all the relevant public keys. You don't necessarily need to know who knows what but its useful to know??? e.g. who is the holdout who is offline.
620 2012-12-20 16:01:49 <etotheipi_> ThomasV: that is correct... your device doesn't care who or what owns the other keys, only that it is 1-of-3 potentially signatures, and it agrees to that transfer
621 2012-12-20 16:02:08 <ThomasV> gmaxwell: yes, but I understood everyone knows the public keys of everyone
622 2012-12-20 16:02:11 <etotheipi_> the other devices/parties can decide for themselves whether to add their own signature
623 2012-12-20 16:02:26 <gmaxwell> ThomasV: well thats why you have to exchange watching wallet codes.
624 2012-12-20 16:03:20 <etotheipi_> on that note: I said in my proposal I was sort public keys in the P2SH script based on lexicographic order of wallet IDs... but perhaps it makes sense to sort by individual public key
625 2012-12-20 16:03:49 <etotheipi_> so each transaction:  the wallet creating the transaction gets all N public keys, and sorts them lexicographically... then adds them to the P2SH in that order
626 2012-12-20 16:03:55 <gmaxwell> yea, I think sorting by the public key makes more sense.. simply because its more compatible with pulling some keys out of the chains.
627 2012-12-20 16:11:19 <etotheipi_> gmaxwell: so as I look back at the code, I'm still not entirely sure if there's a reasonable path to getting SOME security on the public-keys in your wallet without sacrifiicing private key security
628 2012-12-20 16:11:55 <gmaxwell> etotheipi_: what did you think of the tpm thought I gave above?
629 2012-12-20 16:12:39 <etotheipi_> gmaxwell: I'm not faimilar with how to detect and use TPMs
630 2012-12-20 16:12:53 <etotheipi_> or who is likely to have it as part of their system
631 2012-12-20 16:12:56 <gmaxwell> I'm not either. There are libraries.
632 2012-12-20 16:13:11 <gmaxwell> it's a standard feature on many laptops, I believe they're less common in desktops.
633 2012-12-20 16:14:00 <etotheipi_> gah... I'm going to defer this by leaving room for an extra encryption layer, but not implement it yet
634 2012-12-20 16:14:07 <etotheipi_> the absence of this feature should not hold up all the othe rfeatures
635 2012-12-20 16:15:10 <rieno> how come he's allowed to do his business here when i am not?
636 2012-12-20 16:16:07 <etotheipi_> someone may come up with a good idea in that time, anyway
637 2012-12-20 16:16:41 <gmaxwell> I've added looking into TPM for wallet security to my todo.
638 2012-12-20 16:16:54 <gmaxwell> should get a chance .. sometime in june .. 2017.
639 2012-12-20 16:16:55 <gmaxwell> :P
640 2012-12-20 16:17:07 <etotheipi_> :) that's how I feel about a lot of things on my todo list
641 2012-12-20 17:20:38 <zooko> Hm, is the weekly meeting not now?
642 2012-12-20 17:20:45 <zooko> 
ACTION reads the email list...

643 2012-12-20 17:21:18 <gmaxwell> zooko: er. "holidays!"
644 2012-12-20 17:21:49 <gmaxwell> (more reasonably, no one sent out reminders that it would be held???  right now I think most activity is in pound on ultraprune mode)
645 2012-12-20 17:26:21 <zooko> gmaxwell: ok!
646 2012-12-20 17:35:19 <fuzion24> Is there any documentation about the architecture of the blockchain datastore for the satoshi client?
647 2012-12-20 17:37:24 <gmaxwell> What do you mean specifically 'blockchain datastore'?
648 2012-12-20 17:37:52 <rieno> fuzion24: read the wiki, and read about libdb (berkeley DB)
649 2012-12-20 17:37:56 <rieno> gmaxwell: go and die
650 2012-12-20 17:38:48 <gmaxwell> Berkley DB is not used to store any of the block data itself.
651 2012-12-20 18:07:48 <ciscoftw> can someone link me to a solid description of what the bitcoin coinbase is... looking for info, regarding insertion into coinbase when block is solved... can this even be done with standard bitcoind client?
652 2012-12-20 18:26:29 <D34TH> sipa: pulling errors making libleveldb.a on leveldb17 branch
653 2012-12-20 18:27:02 <D34TH> sipahttp://pastebin.com/0BjC2Eki
654 2012-12-20 18:28:12 <D34TH> seems like it isnt including port/win/stdint.h
655 2012-12-20 18:29:22 <gmaxwell> "leveldb17 branch"?
656 2012-12-20 18:29:38 <gmaxwell> What awful forgotten code are you running? :P
657 2012-12-20 18:30:51 <gmaxwell> oh, thats the branch with the stock level db 1.7 stuff, ignore me.
658 2012-12-20 18:32:13 <sipa> D34TH: patches welcome - i just didn't touch the windows build code at all, as i have no way to test it
659 2012-12-20 18:36:43 <D34TH> sipa: working on it now
660 2012-12-20 18:39:08 <D34TH> sipa: i figured out the use of LEVELDB_PLATFORM_WINDOWS
661 2012-12-20 18:43:49 <D34TH> sipa: does your build use snappy?
662 2012-12-20 18:43:56 <D34TH> if not why
663 2012-12-20 18:44:04 <sipa> it doesn't
664 2012-12-20 18:44:10 <sipa> i disabled it, because we don't use it
665 2012-12-20 18:44:17 <sipa> and it's an extra dependency problem
666 2012-12-20 18:44:28 <sipa> (and we don't use it because it doesn't help)
667 2012-12-20 18:45:00 <sipa> gmaxwell: do you understand thanke's concerns about tying the chaincode to pubkey and privkey?
668 2012-12-20 18:48:02 <D34TH> sipa: for the life of me, it seems as if the code is correct but it isn't being accepted (with my small c++ knowledge)
669 2012-12-20 18:49:37 <D34TH> oh thats what i did wrong
670 2012-12-20 18:49:40 <D34TH> ok new errors
671 2012-12-20 18:49:41 <D34TH> :D
672 2012-12-20 18:50:17 <gmaxwell> sipa: I'm currently missing the context for your question.
673 2012-12-20 18:51:01 <gmaxwell> sipa: I see it now.
674 2012-12-20 18:52:11 <sipa> i guess it boils down to "the chain code is more secret than the pubkey"
675 2012-12-20 18:52:45 <gmaxwell> I guess I don't see anything wrong with that on its face, though " I am also assuming that every node in the key tree represents a separate entity" suggests strongly different goals.
676 2012-12-20 18:56:55 <D34TH> sipa: seems like i fixed it
677 2012-12-20 18:57:17 <D34TH> going in for testing
678 2012-12-20 18:57:53 <sipa> it built?
679 2012-12-20 18:58:01 <sipa> w00t
680 2012-12-20 18:59:00 <D34TH> new errors
681 2012-12-20 18:59:01 <D34TH> woo
682 2012-12-20 18:59:28 <D34TH> http://pastebin.com/htFxVPg9
683 2012-12-20 19:00:40 <sipa> i can't help you here
684 2012-12-20 19:05:54 <etotheipi_> gmaxwell: there is a context left out of our previous discussion
685 2012-12-20 19:06:15 <gmaxwell> etotheipi_: considering I hadn't read that thread beyond your first message??? I wouldn't be surprised!
686 2012-12-20 19:06:19 <etotheipi_> which is:  sure there's a potential snafu when you are protecting both private and public keys in your wallet and the user picks a similar passphrase for both
687 2012-12-20 19:06:38 <etotheipi_> but most people using this won't even have private key data in their wallet
688 2012-12-20 19:06:57 <etotheipi_> so it's absolutely worth having the "outer" encryption for such wallets...
689 2012-12-20 19:08:01 <etotheipi_> those who are keeping more BTC and more concerned about the privacy & security, are likely to keep them offline and only use watching-only wallets online
690 2012-12-20 19:08:24 <gmaxwell> hm. well then why not just have a switch where you can have inner (default but only available when there is private data) and outer which is only available when there isn't private data?
691 2012-12-20 19:08:37 <etotheipi_> gmaxwell: that's what I was thinking
692 2012-12-20 19:08:46 <etotheipi_> it's a strange dichotomy though
693 2012-12-20 19:08:57 <etotheipi_> as one isn't strictly more permissive than the other
694 2012-12-20 19:08:59 <gmaxwell> Yea, that removes my concerns I think. And if someone wants 'security for both' they should really have the private data offline.
695 2012-12-20 19:09:24 <phantomcircuit> etotheipi_, encryption for the public keys?
696 2012-12-20 19:09:40 <phantomcircuit> why
697 2012-12-20 19:09:41 <gmaxwell> phantomcircuit: public keys, metadata, etc.
698 2012-12-20 19:09:57 <etotheipi_> phantomcircuit: yes... I had a user contact me that he was concerned if he loses his laptop, someone won't get his private keys, but they will have his identity and know he has $100k+ sitting in cold storage at his house
699 2012-12-20 19:10:10 <sipa> etotheipi_: who? :p
700 2012-12-20 19:10:18 <gmaxwell> hahah
701 2012-12-20 19:10:19 <etotheipi_> lol
702 2012-12-20 19:10:35 <phantomcircuit> etotheipi_, tell him to encrypt his laptop...
703 2012-12-20 19:10:37 <gmaxwell> "Hm interesting problem it would help to know where you live .. exactly. while I work on coding a er. fix"
704 2012-12-20 19:10:47 <etotheipi_> I don't know if it was $100k, but it was enough that he was *really* concerned
705 2012-12-20 19:10:58 <etotheipi_> and I think it's a useful feature to have
706 2012-12-20 19:11:21 <gmaxwell> phantomcircuit: and thats the right answer but: (1) doesn't help when he copies the file to a cloud backup place, (2) apparently this is hard to do in windows.
707 2012-12-20 19:11:57 <phantomcircuit> if he has enough that he's worried like that he should buy a separate laptop that is only for the wallet....
708 2012-12-20 19:12:14 <etotheipi_> phantomcircuit: theirs lots of graylevels...
709 2012-12-20 19:12:26 <phantomcircuit> i have a netbook which is only used for plaintext wallets
710 2012-12-20 19:12:31 <etotheipi_> it's easy to say they *should* do something... but a lot of users don't want to do that
711 2012-12-20 19:12:33 <phantomcircuit> i removed the hdd
712 2012-12-20 19:12:35 <etotheipi_> or they don't know how
713 2012-12-20 19:12:38 <etotheipi_> or it's complicated
714 2012-12-20 19:13:01 <etotheipi_> and I've determined it's my job, that if they run Armory, they have useful features that let them compromise on their goals
715 2012-12-20 19:13:20 <etotheipi_> haha, I almost said "compromise their goals"
716 2012-12-20 19:13:30 <phantomcircuit> unless you're locking memory you're wasting time
717 2012-12-20 19:13:38 <phantomcircuit> the page file will contain the info
718 2012-12-20 19:13:40 <gmaxwell> phantomcircuit: generally its our responsibility in making software to make stuff which is not fragile against user carelessness or ignorance when we can??? as we know that users are careless and ignorant (even ourselves???)
719 2012-12-20 19:14:13 <phantomcircuit> gmaxwell, generally speaking i would agree but in this case i think it could give users a false sense of security
720 2012-12-20 19:14:26 <phantomcircuit> which is in many ways worse than not having the feature at all
721 2012-12-20 19:14:55 <etotheipi_> phantomcircuit: it's not a 100% layer of protection... nothing is
722 2012-12-20 19:15:29 <etotheipi_> but it's not terribly difficult to implement, and definitely raises the bar for what an attacker has to do to extract the information
723 2012-12-20 19:15:34 <gmaxwell> phantomcircuit: you're repeating my earlier arguments. :P
724 2012-12-20 19:15:51 <etotheipi_> you're right, users should be aware that it's not 100%
725 2012-12-20 19:17:23 <gmaxwell> phantomcircuit: my biggest concern was that users would make their privkey password and access password the same and then be more likely to lose the former due to typing the latter all the time.
726 2012-12-20 19:17:46 <gmaxwell> phantomcircuit: but then I also pointed out that if your disk isn't encrypted you're leaking info all over the place...
727 2012-12-20 19:18:40 <phantomcircuit> yup same worries
728 2012-12-20 19:20:02 <gmaxwell> I suggested perhaps locking access to the public data to the system's TPM.  (and if you lose the hardware, you'd need the private key to recover the metadata instead) But that seems like a bit more of a science project.
729 2012-12-20 19:21:36 <phantomcircuit> well and the tpm is usually fairly slow/high latency
730 2012-12-20 19:22:05 <phantomcircuit> although that's probably gotten better recently
731 2012-12-20 19:22:46 <gmaxwell> phantomcircuit: well I suppose you'd decrypt and leave it in memory. So you'd only access it at startup. (you're hosed no matter what if the attacker is actively scraping your memory)
732 2012-12-20 19:23:25 <gmaxwell> but the point there being that if you lose control of the file the privacy is protected... while also not making frequently type a private key.
733 2012-12-20 19:23:29 <phantomcircuit> gmaxwell, if there's even relatively minor pressure on memory windows will swap out
734 2012-12-20 19:23:47 <phantomcircuit> without locked memory you'd find your info all over the disk
735 2012-12-20 19:23:51 <gmaxwell> phantomcircuit: uh. does mlock not work in windows?!@#!
736 2012-12-20 19:23:55 <phantomcircuit> although it is a bitch to pull things from page file
737 2012-12-20 19:24:09 <phantomcircuit> gmaxwell, i suspect you'd have to mlock a lot of memory to make ti work reliably
738 2012-12-20 19:24:23 <gmaxwell> but right, since etotheipi_ is talking about more than private keys, you'd have to lock the entire program.
739 2012-12-20 19:24:42 <gmaxwell> which would sure be a lot easier if armory were just a pure wallet. :P
740 2012-12-20 19:24:44 <phantomcircuit> yeah which gets problematic when you're talking a 2+GB memory program
741 2012-12-20 19:37:59 <etotheipi_> speaking of pulling encryption keys out of RAM, this just showed up on slashdot:  http://thenextweb.com/insider/2012/12/20/this-299-tool-is-reportedly-capable-of-cracking-bitlocker-pgp-and-truecrypt-disks-in-real-time/
742 2012-12-20 19:38:25 <phantomcircuit> etotheipi_, open source tools to do that have existed forever
743 2012-12-20 19:40:11 <gmaxwell> it's especially fun with AES because the keyschedule is such that they're easily identifyable in memory and you can even recover an aes key from the schedule with a bunch of bit errors.
744 2012-12-20 19:43:57 <sipa> so, define a crypto standard that uses scrypt in its key schedule :)
745 2012-12-20 19:44:43 <sipa> or even just any kdf
746 2012-12-20 19:44:59 <sipa> doesn't need to be hard
747 2012-12-20 19:47:05 <helo> "capable of cracking bitlocker and truecrypt"
748 2012-12-20 19:47:15 <MC1984> hiberfiel i understand
749 2012-12-20 19:47:23 <MC1984> but how do you get a memory dump
750 2012-12-20 19:49:18 <MC1984> oh theres the firewire thing