1 2014-05-26 01:25:36 <brandond> Anyone know the technical reason why we store blkXXXXX.dat files in a network format and then keep a LevelDB to index the individual block locations within?  I understand the need to index the transactions within the blocks (which is also done in the LevelDB).
  2 2014-05-26 01:32:52 <brandondahler> Anyone know the technical reason why we store blkXXXXX.dat files in a network format and then keep a LevelDB to index the individual block locations within?  I understand the need to index the transactions within the blocks (which is also done in the LevelDB).  [Not sure if this sent a few minutes ago because I was not registered.]
  3 2014-05-26 01:35:06 <scottansan> I suppose the leveldb is just for indexing and caching?
  4 2014-05-26 01:36:55 <brandondahler> It indexes location of the block inside blkXXXXX.dat files and what block transactions are located within, if I remember from the code I looked at.
  5 2014-05-26 01:40:20 <scottansan> I guess that the requirement to perform lookups on transactions was not intended to be solved as part of the wire format
  6 2014-05-26 01:41:49 <scottansan> but if the leveldb stores offsets then its fast enough to seek the .dat file and read the data into memory... thereby saving space
  7 2014-05-26 01:41:53 <scottansan> something like that?
  8 2014-05-26 01:44:34 <brandondahler> I understand that, but I am considering implications behind flipping the storage from network, order-recieved files to a folder-file structure of storing the blocks in a meaningful way.
  9 2014-05-26 01:52:54 <harding> brandondahler: creating separate files for 200,000+ blocks will use up a lot of inodes on Unix-like filesystems.  That's probably not a problem as long as the average block size stays significantly larger than the typical size-to-inode ratio used by modern *nixes.
 10 2014-05-26 02:24:24 <brandondahler> harding: considered that, storing 1:1 blocks to files would be inefficient file-wise, but clustering them in chain order (explicit instead of implicit as it is now) or by hash?
 11 2014-05-26 02:33:51 <harding> brandondahler: that seems potentially fragile and resource intensive to rewrite files if you switch chains during/after a fork.
 12 2014-05-26 02:38:37 <harding> For current mainnet block sizes, one file per block is fine, and I don't mean to imply otherwise.  Only for testnet or regtests would you be majorly wasting inodes and maybe directory overhead.
 13 2014-05-26 02:46:19 <brandondahler> how are forks handled now?
 14 2014-05-26 03:00:47 <justanotheruser> brandondahler: with reorgs?
 15 2014-05-26 03:15:13 <brandondahler> justanotheruser: i mean in the blk files, are the forked blocks left in our blk files forever or are the automatically pruned in some way
 16 2014-05-26 03:16:32 <fun> :)
 17 2014-05-26 03:16:32 <justanotheruser> brandondahler: I'm not 100% sure, but I think they're there forever. I would wait for someone else to answer though.
 18 2014-05-26 03:16:44 <jcorgan> they are there forever
 19 2014-05-26 03:16:56 <fun> when someone solved the block what are the other nodes checking for, before accepting it?
 20 2014-05-26 03:17:10 <jcorgan> the blockchain files are an append only data structure
 21 2014-05-26 03:17:31 <jcorgan> fun: there is a long list of things that are checked by nodes, there is a wiki page on it
 22 2014-05-26 03:17:48 <fun> jcorgan: would u have a link to it?
 23 2014-05-26 03:18:02 <jcorgan> not off the top of my head, but you could google it
 24 2014-05-26 03:18:15 <brandondahler> i sort of figured that, do nodes distribute the forked (orphaned?) blocks anyway?
 25 2014-05-26 03:19:07 <jcorgan> not sure what you are asking
 26 2014-05-26 03:22:04 <brandondahler> So a block is solved twice, causing a fork.  Some nodes get each one (A and B), eventually the next is solved on A, making B the orphan (i understand it is not as straightforward).  After all of this, someone new to bitcoin starts syncing their blockchain.  Would this new person be guarenteed to recieve block B or not?
 27 2014-05-26 03:23:11 <fun> its longest chain wins
 28 2014-05-26 03:23:16 <jcorgan> it's unlikely they'd get B, and if they did, when they received A elsewhere their node would orphan B in the same way the other's did
 29 2014-05-26 03:24:35 <jcorgan> a new node coming up will not get the hisorical forks
 30 2014-05-26 03:24:37 <brandondahler> so part of the change I'm thinking of would aim to make the blocks recieved and the indexes upon those blocks stateless -- everyone (eventually) has the exact same state
 31 2014-05-26 03:24:55 <andytoshi> :D 2014-05-26 03:24:39 receive version message: bitcoinrust v0.1: version 70001, blocks=0, us=127.0.0.1:36128, them=127.0.0.1:15301, peer=127.0.0.1:50491
 32 2014-05-26 03:25:26 <andytoshi> (all it does is send version messages, but it's a start)
 33 2014-05-26 03:25:42 <jcorgan> everyone does converge on the same state in terms of valid blocks and transactions and their order
 34 2014-05-26 03:26:15 <jcorgan> their blockchain files may have differences, but their indexes won't have any entries pointing to those blocks
 35 2014-05-26 03:27:03 <fun> jcorgan: and how is merge mining works, how come btc solution can also be solution on coin b?
 36 2014-05-26 03:27:12 <fun> since they have diff target i think
 37 2014-05-26 03:27:21 <fun> like btc and devcoin for example
 38 2014-05-26 03:27:30 <fun> or they would have same target?
 39 2014-05-26 03:27:39 <fun> nope unless all btc miners merge mine
 40 2014-05-26 03:29:08 <jcorgan> the very first goodle hit for "merge mining" is a Stack Exchange question with an excellent answer to your question
 41 2014-05-26 03:29:13 <jcorgan> *google
 42 2014-05-26 03:29:22 <fun> oki
 43 2014-05-26 03:30:18 <jcorgan> and the bitcoin wiki has a more formal specification
 44 2014-05-26 03:47:34 <jcrubino> andytoshi: you wrote bitcoinrust ???
 45 2014-05-26 03:48:02 <SubCreative> Anyone know if the keys stored in alert.cpp are pubkeys or how they are generated?
 46 2014-05-26 03:48:06 <SubCreative> it's simply an ecdsa key right?
 47 2014-05-26 03:48:19 <gmaxwell> it's just an ECDSA key.
 48 2014-05-26 03:48:52 <andytoshi> jcrubino: i'm working on a rust SPV wallet so that i have a base to bolt -wizards projects onto
 49 2014-05-26 03:49:21 <SubCreative> so are they pubkey's of gavin/satoshi/theymos?
 50 2014-05-26 03:49:28 <SubCreative> bitcoin address pubkey?
 51 2014-05-26 03:49:48 <jcrubino> I am impressed that 1) a mathematician is learning networking stacks 2) your finding the time
 52 2014-05-26 04:05:15 <andytoshi> jcrubino: i'm a first-year phd student in summer, i have no deadlines. and for now at least i'm coding from the protocol spec, i can do it while my brain is too tired to read :)
 53 2014-05-26 09:38:04 <GAit> sipa: is there any bug/issue on github for bip0062?
 54 2014-05-26 09:38:47 <GAit> (i couldn't find one but perhaps my search fu is weak)
 55 2014-05-26 09:44:57 <sipa> nope, no implementation yet
 56 2014-05-26 09:45:12 <sipa> we could create an issue for it
 57 2014-05-26 09:45:47 <sipa> but i want to make some minor adjustments to bip62 before actually proposing it as a softfork
 58 2014-05-26 09:57:30 <GAit> sipa: we got someone at GreenAddress working on 4-7 right now, do you want to give us pointers as to what you would like done first?
 59 2014-05-26 10:00:07 <GAit> (4-7 as in the source of malleability)
 60 2014-05-26 10:09:36 <Luke-Jr> GAit: GreenAddress works on a full node implementation? O.o
 61 2014-05-26 10:10:16 <GAit> Luke-Jr: no, but can't it submit PR and help bitcoind? It won't be limited to malleability
 62 2014-05-26 10:10:33 <Luke-Jr> of course
 63 2014-05-26 10:10:38 <Luke-Jr> I was just curious ☺
 64 2014-05-26 10:11:16 <GAit> a very good friend of mine is helping gait (and bitcoind), one of the guys that contributed heavily to wine
 65 2014-05-26 10:15:45 <GAit> https://github.com/bitcoin/bitcoin/commit/7a0e84dd63ac98a92fd7aafd2d432f1da71eb8da fortunately the ./configure.ac never sets NDEBUG but arguablyy an error in and of itself, will skip reading the block if assertions are compiled out
 66 2014-05-26 10:16:02 <Luke-Jr> GAit: there's a check somewhere
 67 2014-05-26 10:16:10 <Luke-Jr> GAit: it will refuse to build if NDEBUG is defined
 68 2014-05-26 10:17:47 <GAit> nothing quite like changing the meaning of well known standards :)
 69 2014-05-26 10:21:10 <sipa> GAit: cool! contributions very welcome of course
 70 2014-05-26 10:21:50 <sipa> i'd start by just adding checks inside the script execution code, guarded by sigcheck flags that are never enabled
 71 2014-05-26 10:22:22 <GAit> so im wondering what is and isn't acceptable: e.g. a patch to fix all the catches by value to by ref and by non const ref to const ref ?
 72 2014-05-26 10:22:35 <GAit> (was looking at rpc earlier)
 73 2014-05-26 10:24:26 <GAit> sipa: thank you for the pointer
 74 2014-05-26 10:25:53 <GAit> sipa:  I assume you mean adding a flag a-la script VERIFY STRICTENC
 75 2014-05-26 10:26:00 <sipa> yep
 76 2014-05-26 10:26:07 <GAit> kk
 77 2014-05-26 10:31:44 <gmaxwell> I guess its good that I also belt-and-suspendered forcing !NDEBUG when I went and fixed all those goofs before. :P
 78 2014-05-26 10:32:44 <gmaxwell> It's not acceptable in our codebase to do that, that ought to be fixed. But at the same time it should be harmless since we won't build with NDEBUG. (well I hope, it's only a matter of time before some platform/compiler does something more clever there. :) )
 79 2014-05-26 10:34:56 <GAit> gmaxwell: agreed
 80 2014-05-26 10:35:40 <Luke-Jr> gmaxwell: bool i = false; assert( (i = true) ); if (!i) die();
 81 2014-05-26 10:35:42 <Luke-Jr> :P
 82 2014-05-26 14:18:27 <jgarzik> wumpus, gmaxwell: I was thinking that some of the RPCs could move to bitcoin-cli entirely, and removed from bitcoind.  e.g. decoderawtransaction or createrawtransaction.  Some "RPCs" do not interact with the network or the wallet.
 83 2014-05-26 14:19:02 <sipa> that increases the (necessary) dependencies for bitcoin-cli very significanylu
 84 2014-05-26 14:19:15 <sipa> i'd rather see them move a common utility library :)
 85 2014-05-26 14:21:06 <sipa> and i don't like encouraging bitcoin-cli as more than a test tool for RPC
 86 2014-05-26 14:22:00 <sipa> moving actual implementation code there means it becomes inaccessible for RPC users that use it as intended
 87 2014-05-26 14:23:03 <GAit> i agree that bitcoin-cli being as thin as possible is good
 88 2014-05-26 15:22:12 <chichov> does the complete mining subsidy (currently 25BTC) has to be claimed?
 89 2014-05-26 15:22:44 <michagogo> No
 90 2014-05-26 15:22:56 <michagogo> The coinbase can be less than the maximum
 91 2014-05-26 15:23:20 <chichov> michagogo: so I can also just claim 20BTC from the mining subsidy and it'll be accepted?
 92 2014-05-26 15:23:26 <michagogo> yes
 93 2014-05-26 15:23:33 <chichov> same goes for the fees?
 94 2014-05-26 15:24:07 <chichov> note: mining reward = mining subsidy (25BTC) + transaction fees in my lingo
 95 2014-05-26 15:26:20 <sipa> correct
 96 2014-05-26 15:27:06 <harding> chichov: hmm.  Sorry I gave you wrong info; I must've misread the code about this.
 97 2014-05-26 15:27:12 <chichov> just asking because I got an email from a guy stating that the subsidy has to be claimed completely and the fees can be claimed arbitrarily
 98 2014-05-26 15:27:21 <chichov> harding: speaking of the devil :)
 99 2014-05-26 15:27:25 <harding> :-)
100 2014-05-26 15:27:59 <chichov> don't worry, I'm always double-checking everything
101 2014-05-26 15:30:24 <chichov> too bad actually about that
102 2014-05-26 15:30:40 <chichov> then any miner can burn bitcoins once he finds the right block
103 2014-05-26 15:37:45 <harding> chichov: yeah, but burning bitcoins isn't exactly hard.  You can always send them to a script which looks spendable but isn't.  Anyway, thanks again for all the work you put on the paper.  We just added a link to it on bitcoin.org this morning: https://bitcoin.org/en/developer-documentation (near the bottom).
104 2014-05-26 15:38:21 <chichov> oh, that's cool!
105 2014-05-26 15:38:29 <michagogo> chichov: not exactly once he finds the right block
106 2014-05-26 15:38:40 <michagogo> He needs to be mining with the intention of doing that
107 2014-05-26 15:39:09 <michagogo> But yes, it is very easy for anyone to burn bitcoins, not just miners
108 2014-05-26 15:39:10 <chichov> true, he needs to set block in advance
109 2014-05-26 15:40:42 <chichov> harding: I'm also working now on a C++ Bitcoin API for the JSON-RPC communication
110 2014-05-26 15:41:19 <chichov> I've seen one and inspected it, but not everything is implemented and it's a bit buggy
111 2014-05-26 15:41:56 <chichov> it does seem odd that C++, the language of bitcoin, hasn't been covered properly yet, unlike many others
112 2014-05-26 16:01:28 <c4p> Sorry for posting here, but curious... is using the bitcoind client safe to use for commercial applications?  I hear people keep saying it's reference and not production which makes me curious if it's stability or another issue to be aware of?
113 2014-05-26 16:02:12 <Luke-Jr> c4p: *Bitcoin* is not production, basically.
114 2014-05-26 16:02:25 <Luke-Jr> c4p: anyhow, support is in #bitcoin, not here
115 2014-05-26 16:03:55 <c4p> ok, thanks Luke.  will swap to #bitcoin
116 2014-05-26 17:27:05 <cheako> Hello, I was just thinking that cron should be aware of new blocks.  Is there a best practice for new block detection?
117 2014-05-26 17:30:01 <sipa> use -blocknotify instead of cron...
118 2014-05-26 17:30:20 <cheako> sipa: Not multi-user.
119 2014-05-26 17:45:37 <Luke-Jr> cheako: it can be
120 2014-05-26 17:46:21 <Luke-Jr> -blocknotify 'sudo killall -USR1 bitcoin-block-notify-receiver'
121 2014-05-26 17:47:05 <cheako> hmm.
122 2014-05-26 17:48:05 <cheako> Everyone has their fav hammer and any thing else just seems like the wrong tool for the job to them.
123 2014-05-26 17:49:39 <cheako> Though, cron can be made to listen for arbitrary events on a fifo or socket and overall that would be a better addition to cron.
124 2014-05-26 17:57:34 <lechuga_> is implementing a stealth address spv wallet impossible? seems so if i'm understanding the design properly
125 2014-05-26 17:58:35 <lechuga_> i guess it could give a trusted helper full node the scan key
126 2014-05-26 18:11:34 <sipa> lechuga_: which means giving up your privacy to that node
127 2014-05-26 18:13:18 <lechuga_> sipa: yeah i guess it has to be your own secured node
128 2014-05-26 18:14:28 <lechuga_> sipa: what's the name of your experimental block downloading optimization branch?
129 2014-05-26 18:18:47 <gokudev> does bitcoin core support deterministic wallets?
130 2014-05-26 18:19:09 <hearn> no gokudev
131 2014-05-26 18:19:46 <hearn> lechuga_: right, stealth and spv don't really mix
132 2014-05-26 18:19:47 <sipa> lechuga_: a very old and insecure version: https://github.com/sipa/bitcoin/commits/headersfirst
133 2014-05-26 18:19:58 <gokudev> hmm how about hd wallets?
134 2014-05-26 18:20:08 <gokudev> https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki
135 2014-05-26 18:20:09 <hearn> lechuga_: there is a thread on bitcoin-development where i talk about how to integrate with bip70 to try and fix it
136 2014-05-26 18:20:09 <sipa> gokudev: the d stands for deterministic
137 2014-05-26 18:20:30 <lechuga_> hearn: interesting, integrate stealth w/bip70?
138 2014-05-26 18:20:32 <sipa> gokudev: so given that it doesn't have deterministic wallets, it certainly doesn't have hd wallets :)
139 2014-05-26 18:20:41 <sipa> hearn: i don't understand the purpose
140 2014-05-26 18:20:46 <gokudev> sipa i know! there are hierarchical deterministic wallets
141 2014-05-26 18:20:52 <lechuga_> sipa: thx
142 2014-05-26 18:21:11 <sipa> hearn: if you're using bip70, you're communicating with the receiver... why do you need stealth addresses?
143 2014-05-26 18:21:44 <hearn> sipa: for the case when you're not communicating with the receiver (e.g. they are offline)
144 2014-05-26 18:21:54 <sipa> well then you can't use the payment protocol...
145 2014-05-26 18:22:12 <sipa> please don't make the same mistake as satoshi, by providing a backup option around a receiver being offline :)
146 2014-05-26 18:22:42 <gokudev> hmm so is somebody working on deterministic wallets in bitcoin core ?
147 2014-05-26 18:22:48 <sipa> i was
148 2014-05-26 18:22:53 <sipa> but i have too many other things to do
149 2014-05-26 18:23:05 <gokudev> crap
150 2014-05-26 18:23:21 <gokudev> ok does bitcoin core offer offline wallets like armory does ?
151 2014-05-26 18:23:25 <sipa> no
152 2014-05-26 18:23:28 <gokudev> lol
153 2014-05-26 18:23:29 <sipa> you don't have to use bitcoin core...
154 2014-05-26 18:23:33 <gokudev> im told it does offer offline wallets
155 2014-05-26 18:23:38 <hearn> sipa: the idea is, you should be able to. did you read the thread? the point is to have some server store the Payment message for when you come back online
156 2014-05-26 18:23:38 <sipa> it does not
157 2014-05-26 18:23:48 <sipa> you'll shoot yourself in the foot if you try to it for that
158 2014-05-26 18:23:56 <hearn> gokudev: Core has a very basic wallet implementation
159 2014-05-26 18:24:32 <gokudev> im building an freakin exchange and i need to decide to go the armory route or use bitcoin core
160 2014-05-26 18:24:47 <gokudev> the problem with armory is it only works with bitcoin and i want support altcoins
161 2014-05-26 18:25:02 <sipa> hearn: why do you need stealth addresses for that? have a known public key, let the payment server choose a random number to add to it, and store the result?
162 2014-05-26 18:25:03 <gokudev> armory require major work to support altcoins  without even going into details
163 2014-05-26 18:26:26 <sipa> or let the sender choose a random number - that doesn't require a roundtrip time, and is safer
164 2014-05-26 18:26:53 <hearn> sipa: that's almost what my proposal is. the sender picks the nonce and it's combined with the constant public key in the payment request, to produce the new public key via ecdh
165 2014-05-26 18:27:08 <hearn> sipa: the payment server records the Payment message which contains the sender selected nonce. it's forwarded to the users wallet when they become available and download it.
166 2014-05-26 18:27:14 <sipa> oh sure
167 2014-05-26 18:27:28 <sipa> ut that's not a stealth address
168 2014-05-26 18:27:52 <sipa> at least not the existing proposal with that name, which stores the nonce data inside the transaction
169 2014-05-26 18:28:02 <sipa> but if that is not what you mean by it, sure!
170 2014-05-26 18:28:03 <hearn> right. sorry. i was using the term "stealth address" to mean "that proposal where someone can publish a static piece of data and the senders pay to random addresses"
171 2014-05-26 18:28:38 <hearn> because that's how people are now learning about the idea, via this name
172 2014-05-26 18:28:39 <gokudev> armory is posing too much work
173 2014-05-26 18:28:39 <gokudev> maybe electrum ?
174 2014-05-26 18:28:39 <gokudev> sipa what is the alternative to not using bitcoin core
175 2014-05-26 18:28:39 <hearn> 
ACTION was just wanting to point lechuga_ in the right direction rather than being fussy over naming

176 2014-05-26 18:29:48 <hearn> "ecdh in bip70" is not quite as catchy :)
177 2014-05-26 18:29:48 <hearn> gokudev: for running an exchange? bitstamp does use Core directly, actually. they just make regular wallet backups.
178 2014-05-26 18:29:48 <lechuga_> got it just read the intro to the design on thread
179 2014-05-26 18:29:49 <sipa> hearn: i like to keep the terminology separated: stealth addresses (the nonce-in-transaction proposal) are imho only useful for cases where the sender has an anonimity requirement... for almost every other case you're somehow communicating between sender and payment server anyway, and you get all the benefits from that (OOB messaging, ...)
180 2014-05-26 18:30:29 <hearn> right
181 2014-05-26 18:30:29 <sipa> hearn: what you're proposal is basically just a means of delegating key generation in a safe way in BIP70
182 2014-05-26 18:30:57 <hearn> it could equally be specified in the payment request, however, it would need to be added to the outer PaymentRequest message because otherwise the payment server would break the signature
183 2014-05-26 18:30:57 <sipa> sorry, i should read your mails first :)
184 2014-05-26 18:31:12 <hearn> sipa: btw is there any way to permanently adjust the size of the keypool? or do you have to manually use the keypoolresize rpc repeatedly
185 2014-05-26 18:31:30 <hearn> i thought there was an RPC to change the size
186 2014-05-26 18:31:45 <sipa> hearn: set keypoolsize=X in bitcoin.config :)
187 2014-05-26 18:31:54 <sipa> there is an ROPC to topup it once
188 2014-05-26 18:33:46 <hearn> ah ha
189 2014-05-26 18:34:38 <gokudev> sipa/hearn can the bitcoin core do offline signing of transactions like armory?
190 2014-05-26 18:34:48 <sipa> gokudev: if you do things very manually
191 2014-05-26 18:34:59 <sipa> gokudev: like create the entire transaction yourself
192 2014-05-26 18:35:11 <sipa> but the wallet implementation in bitcoin is not state of the art
193 2014-05-26 18:35:27 <hearn> gokudev: not really.
194 2014-05-26 18:35:37 <hearn> gokudev: you're looking for a cold wallet implementation, yes/
195 2014-05-26 18:36:25 <lechuga_> seems liek one danger with stealth-bip70 is only the relevant parties r going to make active requests for related payments so they leak identity info via HTTP and IP layers
196 2014-05-26 18:36:31 <gokudev> hearn yes i am looking for coldstorage/offline storage, i know i can set maxconnections=0
197 2014-05-26 18:36:35 <hearn> gokudev: right now IMHO there is no good tool for this. However it would be easy to make one, at least with bitcoinj.
198 2014-05-26 18:36:38 <lechuga_> related payments / payment requests
199 2014-05-26 18:37:16 <gokudev> i rather use bitcoin core, i can live with the basic bitcoin wallets, but i cant get around offline storage/cold stroage and watchonly wallets
200 2014-05-26 18:37:23 <hearn> lechuga_: bitcoinj has a bundled Tor client at least, and payment servers can use SSL. so it should be possible to scramble things quite effectively, if wanted.
201 2014-05-26 18:38:06 <lechuga_> sure but then you're trusting other systems to maintain the same anonymity properties true stea;th addresses provide
202 2014-05-26 18:38:14 <hearn> gokudev: are you willing to pay for a solution?
203 2014-05-26 18:38:52 <gokudev> maybe how much will it cost?
204 2014-05-26 18:39:00 <hearn> lechuga_: against which adversaries? i don't think the current floodfill system provides much privacy against e.g. the NSA or GCHQ, if that's what you're thinking about. they can quite easily watch for the injection of a tx into the network and record the origin IP.
205 2014-05-26 18:39:51 <gokudev> im looking for 2 things, offline wallets and watchonly wallets
206 2014-05-26 18:40:01 <lechuga_> that's fair i guess
207 2014-05-26 18:40:03 <hearn> gokudev: good question. depends on the exact requirements. however i know some other places have the same issue, with the cold wallet solutions available not being that great. so the cost could potentially be split. if you're interested in this idea, email hearn@vinumeris.com - it's a new company i created to solve problems like this
208 2014-05-26 18:40:17 <hearn> (where multiple bitcoin users/companies have infrastructure they need, but don't want to pay for all of it themselves)
209 2014-05-26 18:40:23 <sipa> gokudev: watchonly wallet support will probably be merged in bitcoin core in the near future
210 2014-05-26 18:40:39 <sipa> cold wallets are really not doably before wallets are deterministic
211 2014-05-26 18:40:44 <gokudev> sipa, im actually a dev testing the build as well :)
212 2014-05-26 18:41:38 <hearn> lechuga_: "stealth addresses" as implemented are not quite as strong as you might imagine, given that the only real users of it today are DarkWallet and they have a almost entirely centralised model. if the DarkWallet private SSL key were to be obtained, the observer would be able to un-stealth all the addresses, is my understanding.
213 2014-05-26 18:42:06 <gokudev> cold wallet is just an offline system  that has an airgap between an online system, it is completely offline
214 2014-05-26 18:42:19 <hearn> gokudev: I think an HD cold wallet system would not be too hard to build with the latest bitcoinj. it has nearly finished support for HD wallets.
215 2014-05-26 18:42:51 <hearn> gokudev: so putting a simple GUI or command line tool on that to create a transaction on the main "online" watching wallet, and then having the keys be in a cold wallet that sign the incomplete transaction, would be not very hard
216 2014-05-26 18:42:53 <gokudev> hmm
217 2014-05-26 18:43:39 <hearn> basically it means definining a file format to hold the unsigned transaction, then having the tool with the keys know how to decrypt those keys, load the unsigned tx, sign for them, and spit the newly finished tx back out to a new file where it can then be shipped back to the online wallet e.g. via USB key for broadcast
218 2014-05-26 18:44:57 <gokudev> yup that makes sense
219 2014-05-26 18:45:16 <gokudev> im a C# programmer actually but i can code in C++ too
220 2014-05-26 18:45:42 <lechuga_> hearn: didn't realize. i thought they were shipping enduser software
221 2014-05-26 18:47:04 <hearn> lechuga_: it's a chrome "app" that relies on a server they run, AFAIK. the server software is open source but it's sort of like electrum, it's basically a block explorer so is rather expensive to run one. it's probably not very hard for governments to penetrate:  they managed to seize the Silk Road servers without DPR even noticing, for example.
222 2014-05-26 18:48:00 <hearn> of course, in fairness, currently SPV wallets don't set very garbled Bloom filters at all, so there the NSA doesn't even have to crack any servers. they can just slurp the filters off the wire.
223 2014-05-26 18:48:21 <hearn> i'm hoping someone will appear and help improve bitcoinj so it garbles the filters as much as possible. right now i'm too busy with HD to do that.
224 2014-05-26 18:48:31 <lechuga_> so the server must have the users' scan keys right?
225 2014-05-26 18:48:43 <hearn> that's my understanding. but you could as well ask them
226 2014-05-26 18:49:38 <lechuga_> this design seems directly opposed to their ideology
227 2014-05-26 18:49:52 <gokudev> hearn ok fair enough bitcoinj i can checkout, but what about watchonly wallets, does it implement them too ?
228 2014-05-26 18:50:49 <hearn> gokudev: yes. the HD wallets support is currently on a branch, hdw-alpha. i'll be merging it into master in the coming 2-3 weeks i expect, just got a few more things to finish off (like figuring out the BIP44 stuff). on that branch Wallet has a static method that creates a watching wallet from an "account key" exported by another wallet (which is basically an xpub BIP32 key).
229 2014-05-26 18:51:06 <hearn> lechuga_: well ........
230 2014-05-26 18:51:19 <lechuga_> heh
231 2014-05-26 18:51:28 <hearn> lechuga_: their ideology is IMHO not really about decentralisation. they care more about anonymity/anarchism than decentralisation
232 2014-05-26 18:52:27 <hearn> i suspect the real reason they have this design though is that many years ago, amir assumed making SPV wallets perform well was impossible. their website for a long time (maybe still does) said something like "block chain servers are obviously the future so that's what we use"
233 2014-05-26 18:52:29 <gokudev> hearn wondering which is smarter way to go either wait for bitcoin core to finish up with their watchonly wallets and then bring in hd wallets from bitcoinj or maybe even get both functionality from bitcoinj!
234 2014-05-26 18:52:51 <hearn> gokudev: don't know! but the watchonly wallets in Core won't solve your cold wallet problem.
235 2014-05-26 18:53:27 <gokudev> hearn right because coldwallets and determistic wallets go hand in hand
236 2014-05-26 18:53:52 <lechuga_> ic
237 2014-05-26 18:54:04 <hearn> lechuga_: ah yes. https://darkwallet.unsystem.net/old_index.html
238 2014-05-26 18:54:11 <hearn> lechuga_: How will you ensure privacy of blockchain queries?
239 2014-05-26 18:54:11 <hearn> New versions of ZeroMQ support encryption between endpoints. We can go further too, but this will require more research. What is clear however, is that blockchain servers are the future of usable Bitcoin wallets where the user owns their keys. We have to innovate around the reality.
240 2014-05-26 18:54:50 <hearn> i would rather disagree with their "reality" given the number of ordinary users who are successfully syncing from the P2P network  without problems
241 2014-05-26 18:54:56 <hearn> but that's why it's designed the way it is
242 2014-05-26 18:55:08 <hearn> gokudev: actually there's no particular reason that has to be the case
243 2014-05-26 18:55:17 <hearn> gokudev: you can have a cold wallet that has, say, 500 keys pre-generated, and then you just keep reusing them
244 2014-05-26 18:55:40 <gokudev> well the keys have to be generated in advance with the non deterministic wallets
245 2014-05-26 18:55:47 <gokudev> which means more trips to the coldstroage
246 2014-05-26 18:56:45 <hearn> hmm, no. it is only a privacy feature. when you set up the cold storage wallet, you just generate the 500 or 1000 keys or whatever. then you make the neutered watching wallet and copy it to your live machine so it can broadcast transactions and see inbound payments from the hot wallet.
247 2014-05-26 18:56:56 <phantomcircuit> hearn, the number of servers isn't growing linearly with the number of users though :P
248 2014-05-26 18:56:59 <hearn> the hot wallet knows the 1000 keys and just sends money to one of them whenever it gets too big
249 2014-05-26 18:57:46 <phantomcircuit> hearn, 1m keys
250 2014-05-26 18:57:47 <hearn> after a while you might end up sending money to keys more than once. but that's how most existing exchanges do it already, AFAIK.
251 2014-05-26 18:57:49 <phantomcircuit> forget about 1k
252 2014-05-26 18:57:50 <phantomcircuit> :P
253 2014-05-26 18:58:02 <lechuga_> their position confuses me i guess. i feel like a true anarchist would almost have to wholly adopt decentralization.
254 2014-05-26 18:58:18 <lechuga_> but maybe anarchism is a spectrum
255 2014-05-26 18:58:25 <lechuga_> we're all a little bit anarchaic
256 2014-05-26 19:00:01 <hearn> lechuga_: principles don't always map easily to code. i'm sure they'd like decentralisation. but the only implementation of SPV+Bloom filtering that's got any real production usage is the one I wrote, and amir thinks I work for the CIA (because he's an idiot). so he'd prefer to use his own code and not use the P2P network rather than build on bitcoinj (and reimplementing it is a lot of work)
257 2014-05-26 19:00:33 <gokudev> hearn so keys  can be resued as well ?
258 2014-05-26 19:00:42 <lechuga_> lol
259 2014-05-26 19:00:47 <hearn> gokudev: of course keys can be reused! it's just not recommended.
260 2014-05-26 19:00:54 <hearn> gokudev: nothing in the protocol stops you doing it though
261 2014-05-26 19:01:24 <gokudev> hearn so that is where deterministic wallets come because they let you create any number of keys without having to worry about reusing them
262 2014-05-26 19:01:30 <hearn> gokudev: right
263 2014-05-26 19:01:34 <lechuga_> i bet deep down it's just java h8
264 2014-05-26 19:01:46 <hearn> lechuga_: i'm sure it's lots of things. regardless, DW is what it is.
265 2014-05-26 19:02:13 <lechuga_> thx for all the info
266 2014-05-26 19:02:24 <hearn> and for all their hating on The Man, what it is, is a Chrome app that delegates all heavy lifting to a server they control. It gives you privacy, except from the DW operators.
267 2014-05-26 19:02:40 <hearn> so we'll see how it evolves over time
268 2014-05-26 19:02:50 <lechuga_> lol
269 2014-05-26 19:03:04 <lechuga_> from the DW operators and from Chrome
270 2014-05-26 19:03:17 <gokudev> hearn i need like a minimalistic implementation to get things going really, lets say i am looking for a coldstroage that works with the non deterministic wallets and want watch only wallets - thats it
271 2014-05-26 19:04:05 <gokudev> and i wont be resuing the keys, just create new ones from hot wallet and then updating my cold storage periodically , that should work albiet lot of leg work to and from coldstorage?
272 2014-05-26 19:04:17 <hearn> gokudev: yes ok well like i said, if you're willing to pay (even if it's not much), email me and we'll see if we can get a bunch of operators to club together. i can write an app that does cold storage without needing Core on all the computers without much difficulty.
273 2014-05-26 19:06:36 <gokudev> uhm can it be general enough to work with altcoins too ?
274 2014-05-26 19:07:06 <hearn> some altcoins have bitcoinj forks. it'd work for them too. for instance dogecoin does.
275 2014-05-26 19:07:10 <hearn> but not all altcoins have that
276 2014-05-26 19:14:53 <gokudev> carbon wallet seems to offer both deterministic wallets as well as  coldstroage solution
277 2014-05-26 19:17:53 <gokudev> does wallet care about the type of coins it holds?  wallets only hold private keys.
278 2014-05-26 19:17:53 <hearn> gokudev: looks nice. though it's yet another web wallet
279 2014-05-26 19:19:18 <hearn> it's a frontend to blockchain.info, it seems
280 2014-05-26 19:20:24 <sipa> gokudev: of course... it needs to kbow how they can be spent
281 2014-05-26 19:20:33 <sipa> which can be arbitrarily complex
282 2014-05-26 19:21:07 <hearn> gokudev: most alt coins don't change the tx format. so they'd probably be compatible in the cold wallet sense. but it's risky and not general. some alt coins are more different than others
283 2014-05-26 19:24:10 <sipa> in particular, the more interesting they are, the more different they are too usually :)
284 2014-05-26 19:25:07 <gokudev> hearn it so sense as long as the tx format is the same as that of bitcoin and litecoin, i am able to store the keys in the same wallet
285 2014-05-26 19:25:46 <gokudev> life would be easier if i could say  BitCoin : ICoinTXType  or LiteCoin : ICoinTxType
286 2014-05-26 19:30:20 <gokudev> here is abetter explaination on hdwallets http://www.reddit.com/r/Bitcoin/comments/1n8yle/finally_someone_will_support_bip32_hierarchical/
287 2014-05-26 19:46:58 <tcatm> Are there any known bugs that would cause Corruption: block checksum mismatch?
288 2014-05-26 19:55:05 <gokudev> https://github.com/NicolasDorier/NBitcoin
289 2014-05-26 19:55:31 <gokudev> guys nicolas has gone and built hd wallets and also ported bitcoin in C#
290 2014-05-26 19:55:36 <sipa> tcatm: crashed in the middle of IBD?
291 2014-05-26 19:56:02 <gokudev> watch only wallets are simply wallets with no private keys right ?
292 2014-05-26 19:56:10 <gokudev> just public keys
293 2014-05-26 19:56:12 <sipa> basically, yes
294 2014-05-26 19:56:17 <sipa> any script even
295 2014-05-26 19:56:24 <sipa> you don't need the key
296 2014-05-26 19:56:30 <tcatm> sipa: I've wiped the datadirectory after it happening for the first time. No software crash.
297 2014-05-26 19:56:45 <sipa> what hardwaree?
298 2014-05-26 19:56:46 <gokudev> so basically all i do is not import the private key in the wallet and just generate a receive address for the hd wallet and im done
299 2014-05-26 19:57:04 <sipa> for hd wallets its a bit more complicated
300 2014-05-26 19:57:18 <tcatm> Server with ECC RAM and checksummed filesystem (ZFS, no errors reported).
301 2014-05-26 19:57:27 <sipa> as you can generate the pubkeys on the fly
302 2014-05-26 19:57:34 <gokudev> well lets assume i only have  1 branch  and one passphrase to generate receive only addresses
303 2014-05-26 19:57:37 <sipa> tcatm: can you trry on not-ZFS?
304 2014-05-26 19:57:50 <gokudev> oh i see
305 2014-05-26 19:58:04 <sipa> gokudev: i would also stay away from reimplementation of the consensus code...
306 2014-05-26 19:58:41 <tcatm> sipa: It's actually on ext4 running in a ZVOL.
307 2014-05-26 19:58:44 <gokudev> because of possible bugs and untested nature ?
308 2014-05-26 19:59:53 <sipa> http://bitcoin.stackexchange.com/questions/25254/why-dont-developers-switch-from-bitcoin-qt-to-a-cleaner-implementation/
309 2014-05-26 20:00:14 <sipa> tcatm: can you try not running on a ZFS stack? :)
310 2014-05-26 20:00:51 <tcatm> Not on the same hardware.
311 2014-05-26 20:02:21 <tcatm> When re-indexing I get a lot of orphaned blocks.
312 2014-05-26 20:02:35 <sipa> ... that's impossible
313 2014-05-26 20:02:41 <sipa> orphans are ignored during reindex
314 2014-05-26 20:03:20 <sipa> oh they may still be logged
315 2014-05-26 20:03:38 <sipa> that means the block data is corrupted... but we already knew that
316 2014-05-26 20:05:18 <tcatm> http://pastebin.com/raw.php?i=1kNKXqF8
317 2014-05-26 20:05:54 <sipa> and after that only more orphan blocls, i guess
318 2014-05-26 20:06:11 <tcatm> yes
319 2014-05-26 20:06:26 <sipa> no known bug that causes that, no
320 2014-05-26 20:08:30 <gokudev> sipa nice writeup! but then that encourages us to stick with bitcoin core and waiting isnt an option to have hd wallets / watchonly wallets or even offline storage
321 2014-05-26 20:09:06 <gokudev> but then again having forks due to implementation problems can be disastrous
322 2014-05-26 20:09:10 <sipa> there is absolutely no reason why core code needs to be even the same program as the wallet
323 2014-05-26 20:09:24 <sipa> satoshi really gave a bad example that many try to copy
324 2014-05-26 20:09:44 <sipa> use a program that is designed as a wallet and nothing more
325 2014-05-26 20:10:01 <gokudev> armory does that i know
326 2014-05-26 20:10:22 <sipa> yes, and bitcoinj, and bitsofproof, and coinvault
327 2014-05-26 20:10:31 <gokudev> what about bitcoinlib ?
328 2014-05-26 20:10:39 <sipa> never heard about that
329 2014-05-26 20:11:33 <gokudev> https://github.com/GeorgeKimionis/BitcoinLib
330 2014-05-26 20:11:47 <gokudev> i think its just a C# wrapper to the bitcoin daemon and makes rpc calls
331 2014-05-26 20:14:10 <sipa> seems like it
332 2014-05-26 20:14:51 <gokudev> so nbitcoin  is a port to bitcoin core itself as well
333 2014-05-26 21:38:33 <ahmed_> hi guys
334 2014-05-26 21:39:01 <ahmed_> does anyone here know how to sort the output of bitcoin's listsinceblock by confirmations?
335 2014-05-26 21:39:08 <ahmed_> im using jgarzik's python bitcoin lib