1 2012-08-02 04:42:26 <finway> I can run bitcoin-qt, but i can't run bitcoind, on ubuntu, what's the problem?
  2 2012-08-02 04:42:44 <finway> Both under root account
  3 2012-08-02 05:04:20 <wumpus> you shouldn't run bitcoin as root
  4 2012-08-02 05:04:39 <bonks> he left
  5 2012-08-02 05:04:56 <wumpus> oh, right, stupid leavers :)
  6 2012-08-02 05:05:07 <bonks> he didn't even give any info
  7 2012-08-02 05:05:30 <bonks> but my usual mistake on different boxes is, running 64bit on 32bit :x
  8 2012-08-02 05:06:35 <wumpus> yeah better do it the other way around
  9 2012-08-02 05:07:04 <wumpus> but at least that will give a kind of clear error message
 10 2012-08-02 05:07:22 <wumpus> or not, some ELF blablalblba
 11 2012-08-02 05:09:43 <gribble> New news from bitcoinrss: Diapolo opened pull request 1649 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1649>
 12 2012-08-02 06:11:24 <finway> sourceforge.net has been WALLED in China.
 13 2012-08-02 06:11:40 <finway> So Chinese people can't download Bitconi Ref Client now.
 14 2012-08-02 06:12:02 <finway> Are there other Download link ?
 15 2012-08-02 06:12:20 <wumpus> is github also wallet? at least you could get the source
 16 2012-08-02 06:12:24 <wumpus> walled*
 17 2012-08-02 06:12:49 <finway> walled -  can't access
 18 2012-08-02 06:13:01 <finway> walled - blocked
 19 2012-08-02 06:14:39 <wumpus> also, sourceforge has many mirrors, I wonde rif they blocked them all
 20 2012-08-02 06:15:36 <gribble> New news from bitcoinrss: Diapolo opened pull request 1650 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1650>
 21 2012-08-02 06:16:30 <wumpus> try this, direct link to the german mirror: http://194.95.248.253/project/bitcoin/Bitcoin/bitcoin-0.6.3/bitcoin-0.6.3-win32-setup.exe
 22 2012-08-02 06:18:25 <finway> thanks
 23 2012-08-02 06:18:27 <wumpus> the czech one: http://193.1.193.66/project/bitcoin/Bitcoin/bitcoin-0.6.3/bitcoin-0.6.3-win32-setup.exe
 24 2012-08-02 06:19:24 <wumpus> replace filename with bitcoin-0.6.3-linux.tar.gz for the linux one, and bitcoin-0.6.3-macosx.dmg for macosx
 25 2012-08-02 06:19:35 <finway> yeah
 26 2012-08-02 06:19:51 <finway> there are only 10K bitcoin fans in China.
 27 2012-08-02 06:19:58 <finway> But it's growing fast.
 28 2012-08-02 06:20:22 <t7> bloody china
 29 2012-08-02 06:20:37 <t7> human rights violators
 30 2012-08-02 06:21:01 <finway> Governments are evil, more or less.
 31 2012-08-02 07:19:53 <yellowhat> are there chinese laws forbidding bitcoin acutally? or is it just censor all-ask-questions-later?
 32 2012-08-02 07:36:38 <Joric> yellowhat, heard in china you may get landed just for receiving a package from ebay
 33 2012-08-02 07:38:17 <c_k> FedEx asked me for an invoice to prove the value of my goods, I gave them one in Bitcoin values and they seemed clarification on what the exchange rate was for btc -> USD hehe
 34 2012-08-02 07:38:57 <MagicalTux> c_k: already had this kind of issue in the past
 35 2012-08-02 07:57:13 <t7> ECC is much more complicated than RSA :(
 36 2012-08-02 08:36:19 <sturles> That's some transaction: http://blockchain.info/tx-index/14303137/9151f499a421583b8c27461e8d5cc8663f241d1a85d46f72748e07e134eb9239
 37 2012-08-02 08:37:10 <sturles> A load of 0.00000001 BTC inputs and a 0.015 BTC fee.
 38 2012-08-02 08:38:38 <sturles> For each few of those 0.00000001 BTC an additional 0.005 BTC fee was needed, so the 0.00000001 BTC ended up costing more BTC to spend than their total value.
 39 2012-08-02 08:39:33 <sipa> well, bitcoin is not fit for microtransactions
 40 2012-08-02 08:39:46 <sipa> and this certainly qualifies as micro
 41 2012-08-02 08:40:39 <etotheipi_> sipa, how much do you know about bitcoind plans to use leveldb?
 42 2012-08-02 08:44:27 <sipa> if it's up to me, we switch in 0.8
 43 2012-08-02 08:45:08 <epscy> from bdb?
 44 2012-08-02 08:45:20 <sipa> yes, for the block/tx index
 45 2012-08-02 08:45:53 <sipa> etotheipi_: why?
 46 2012-08-02 08:47:03 <wumpus> so bdb will be wallet-only?
 47 2012-08-02 08:47:25 <sipa> unless by the time there is some replacement for that too, i guess
 48 2012-08-02 08:47:40 <wumpus> yeah, but we'll have to remain compatible with old wallets likely forever
 49 2012-08-02 08:47:41 <etotheipi_> sipa, I'm wondering what and how leveldb will be used and if there's a way for Armory to share the DB
 50 2012-08-02 08:48:05 <etotheipi_> it looks like no (because it looks like multiple processes cannot share a DB), but I wanted to get the details
 51 2012-08-02 08:48:15 <sipa> etotheipi_: indeed
 52 2012-08-02 08:48:52 <etotheipi_> right now I'm investigating how to use it for myself... it's not only easy, it's damned fast
 53 2012-08-02 08:49:00 <sipa> wumpus: or provide a separate migration tool that has the old bdb wallet code
 54 2012-08-02 08:49:09 <sipa> and which is invoked automatically
 55 2012-08-02 08:49:45 <etotheipi_> why not add old wallet addresses as imported addresses in the new wallets?  it's not like the old ones are deterministic
 56 2012-08-02 08:49:58 <sipa> that's not the problem
 57 2012-08-02 08:50:25 <sipa> but i'd like to rid the codebase of bdb
 58 2012-08-02 08:50:27 <wumpus> right, migration that's just a matter of implementation, but IMO we should remain out-of-the-box compatible with old wallets, not that people have to download some obscure program to port their keys over, that's asking for trouble
 59 2012-08-02 08:50:55 <sipa> anyway, keeping the old bdb code is certainly easier
 60 2012-08-02 08:51:05 <wumpus> even if it's in 10 years and someone kept a wallet in a safe place
 61 2012-08-02 08:52:58 <sipa> etotheipi_: also, i hope to get "my" ultrapruned merged around the same time, which would mean the database layout would change significantly as well (though the blk000*.dat files will remain, have the same format, but be smaller and more numerous)
 62 2012-08-02 08:53:12 <wumpus> which makes bdb a very bad choice, I guess, as the format even changes between BDB versions, it's a heavy legacy to carry
 63 2012-08-02 08:53:22 <sipa> exactly
 64 2012-08-02 08:53:27 <etotheipi_> sipa, I'm sure you already know this, but on Linux 64-bit, I did a raw transfer of blk0001.dat to a leveldb database (txhash->rawtx), and was able to scan the entire thing for balance information in 8.9s
 65 2012-08-02 08:53:39 <etotheipi_> super impressive
 66 2012-08-02 08:53:45 <wumpus> just make sure we don't make the same mistake again, for a new wallet format
 67 2012-08-02 08:54:01 <sipa> etotheipi_: wait, scan?
 68 2012-08-02 08:54:01 <wumpus> make sure all the code to handle it is in in our own control
 69 2012-08-02 08:54:16 <wumpus> (that doesn't we should have written it, just that it's part of the codebase)
 70 2012-08-02 08:54:18 <sipa> without an address->tx index?
 71 2012-08-02 08:54:25 <etotheipi_> sipa, yes and no
 72 2012-08-02 08:54:39 <etotheipi_> I was able to to collect all txouts in 7 seconds
 73 2012-08-02 08:55:23 <etotheipi_> but when I added txins (maintaining a set<OutPoint> to quickly identify txins), I ran into the problem that the DB was iterating out of order
 74 2012-08-02 08:55:34 <sipa> right
 75 2012-08-02 08:55:50 <etotheipi_> so at the moment, it will take about 18s to do two scans
 76 2012-08-02 08:56:20 <etotheipi_> but I'm looking into using a separate, ordered memory map to try accessing in order
 77 2012-08-02 08:56:28 <etotheipi_> to see if I can get something in between
 78 2012-08-02 08:56:44 <etotheipi_> with one scan
 79 2012-08-02 08:56:59 <etotheipi_> either way, 18s is not bad at all for a tremendously simple implementation
 80 2012-08-02 08:57:04 <sipa> i'm pondering about adding two optional indexes to ultraprune; one for finding arbitrary tx's based on txid (yes, it doea not need this!), and one based on address
 81 2012-08-02 08:58:10 <sipa> but first things first :)
 82 2012-08-02 08:58:14 <epscy> sipa: are you working on pruning?
 83 2012-08-02 08:58:28 <sipa> well, ultraprune is nit exactly pruning
 84 2012-08-02 08:58:38 <sipa> but it supports removing old blocks
 85 2012-08-02 08:58:55 <etotheipi_> I'm planning out right now how I want to support my own pruning idea
 86 2012-08-02 08:58:56 <epscy> is it possible to reduce teh size of the blockchain without sacraficing security?
 87 2012-08-02 08:59:02 <sipa> yes
 88 2012-08-02 08:59:07 <epscy> cool
 89 2012-08-02 08:59:10 <etotheipi_> or rather, dual support:  full node, pruned node, or light node... all in one scheme
 90 2012-08-02 08:59:11 <sipa> a lot
 91 2012-08-02 08:59:34 <epscy> i think that would be helpful
 92 2012-08-02 08:59:50 <sipa> the principal idea is that block archiving and normal validation are two separate operations
 93 2012-08-02 09:00:00 <sipa> and you can choose to do either, independently
 94 2012-08-02 09:00:05 <etotheipi_> sipa, has there been any discussion/recommendations about the ideas I proposed?  I haven't been on IRC at all...
 95 2012-08-02 09:00:38 <sipa> etotheipi_: you mean the sidechain with index, that coinbase commits to?
 96 2012-08-02 09:00:53 <etotheipi_> I am wondering if it's something the devs want to support it, even in spirit in anyway
 97 2012-08-02 09:01:07 <etotheipi_> or if I'm on my own... or why it should or should not be done
 98 2012-08-02 09:01:17 <etotheipi_> sipa, yes, the alt-chain
 99 2012-08-02 09:01:32 <sipa> to me, the first, and essential step is to change the mode of operation to one that keeps blocks and the tzout set
100 2012-08-02 09:01:33 <etotheipi_> err.. sidechain
101 2012-08-02 09:01:42 <sipa> instead of one big blop
102 2012-08-02 09:01:55 <etotheipi_> right, that's why I'm looking into maintaining a separate txout store
103 2012-08-02 09:02:08 <sipa> that is what ultraprune is all about
104 2012-08-02 09:02:25 <etotheipi_> sipa, is that using leveldb?
105 2012-08-02 09:02:45 <sipa> it adds an extra index with all txout data in very compact form
106 2012-08-02 09:03:20 <sipa> the backend is not relevant, i already have an std::map backend, bdb backend, and mempool-provided backend
107 2012-08-02 09:03:41 <sipa> and all validation operations just use this set of coins
108 2012-08-02 09:04:05 <etotheipi_> well the backend is slightly relevant, because I'm trying to base the design on something that works well with leveldb access efficiency (or rather, fits well into map<string,string> format)
109 2012-08-02 09:05:03 <sipa> the current bdb backend stores a txid -> very compactly serialized list of txouts map
110 2012-08-02 09:05:59 <etotheipi_> so it stores (key,value) = (txid, serializedtxoutmap) ?
111 2012-08-02 09:06:09 <sipa> yes
112 2012-08-02 09:06:26 <sipa> the next step would be maintaining a merkle tree of this data (or other authenticated data structure), which can be committed to in the coinbase
113 2012-08-02 09:07:02 <sipa> and even as gmaxwell noted, committing against the block undo data
114 2012-08-02 09:07:20 <sipa> (whuch is just a list of txouts removed by a block)
115 2012-08-02 09:07:51 <etotheipi_> so then, are you looking towards similar functionality as I proposed?  allowing trust-less light node access?  or just efficeint storage ?
116 2012-08-02 09:08:06 <sipa> right now, storage and efficiently
117 2012-08-02 09:08:43 <sipa> but as soon as we can commit this data in coinbases, extensions can be provided that allow nodes to get an authenticated lookup in this datastructure
118 2012-08-02 09:09:15 <sipa> at that point, i wouldn't mind some people maintaining another index that is address-based instead of txid-based
119 2012-08-02 09:09:43 <sipa> but i don't think that is essential to operating bitcoin
120 2012-08-02 09:10:11 <etotheipi_> I think it is
121 2012-08-02 09:10:19 <sipa> i know you do :)
122 2012-08-02 09:10:59 <etotheipi_> I can't think of anything better for Bitcoin than new users being able to download a client, and operate with the speed of a lite client, without actually sacrificing much
123 2012-08-02 09:11:20 <etotheipi_> of course, I need a proof-of-concept first...
124 2012-08-02 09:11:26 <sipa> when we get tx filtering, SPV nodes should be as fast
125 2012-08-02 09:11:49 <sipa> with the same security
126 2012-08-02 09:12:03 <etotheipi_> I am not familiar with what you're talking about
127 2012-08-02 09:12:08 <sipa> ih, ok
128 2012-08-02 09:12:47 <sipa> so, an SPV node would send to its peer "i am only interested in transactions that match some pattern X"
129 2012-08-02 09:13:15 <sipa> after that, it would start fetching the chain from the creation point of its own wallet on
130 2012-08-02 09:13:31 <sipa> (for the history before that, only headers suffices)
131 2012-08-02 09:13:58 <etotheipi_> right
132 2012-08-02 09:14:32 <sipa> and it would receive blocks that are filtered, but contain merkle paths to prove that they are indeed in said block
133 2012-08-02 09:15:15 <etotheipi_> so it allows a way for the light node to request the minimal amount of information needed for this, and the peer does the filtering for you
134 2012-08-02 09:15:23 <sipa> indeed
135 2012-08-02 09:16:27 <etotheipi_> my only issue with this is that it feels like it will work well for 90% of users... but any wallet with any older addresses or unknown, have to retrieve from the beginning
136 2012-08-02 09:16:55 <etotheipi_> though, if it's filtered, I guess it's not that bad
137 2012-08-02 09:17:24 <etotheipi_> and you're trusting the peer to give you everything
138 2012-08-02 09:17:25 <sipa> the nice thing is that this only requires a dumb archive node to provide the data
139 2012-08-02 09:17:33 <sipa> no, you don't
140 2012-08-02 09:17:41 <sipa> oh, right, yes!
141 2012-08-02 09:18:17 <sipa> yes, that is the weakness, you cannot be sure a transaction must really be filtered
142 2012-08-02 09:18:30 <etotheipi_> I don't think it's a huge deal... you really only need one honest full node
143 2012-08-02 09:18:40 <sipa> indeed
144 2012-08-02 09:18:57 <sipa> moreover: only an honest archive node
145 2012-08-02 09:19:32 <sipa> (which does not necessarily keep the txout set)
146 2012-08-02 09:20:44 <etotheipi_> wow!  updated timing
147 2012-08-02 09:20:59 <etotheipi_> I just did two full scans of blk0001.dat in leveldb
148 2012-08-02 09:21:24 <sipa> what di you mean by full scan?
149 2012-08-02 09:21:28 <etotheipi_> scanned for all txouts first... stored them in a map<OutPoint,uint64_t>, then rescanned for txins searching that map
150 2012-08-02 09:21:44 <etotheipi_> so it did a full traversal of every blk0001.dat transaction
151 2012-08-02 09:22:00 <etotheipi_> and constructed the balance & UTXO out list for 4 addresses
152 2012-08-02 09:22:07 <etotheipi_> 11.9s
153 2012-08-02 09:22:12 <sipa> nice!
154 2012-08-02 09:23:36 <etotheipi_> that's using just leveldb::Iterator
155 2012-08-02 09:23:44 <etotheipi_> and walking through every stored tx
156 2012-08-02 09:23:49 <etotheipi_> (twice)
157 2012-08-02 09:24:04 <sipa> but your db is kept in RAM, i suppose?
158 2012-08-02 09:24:22 <etotheipi_> umm... kind of
159 2012-08-02 09:24:32 <sipa> or at least cached by the OS
160 2012-08-02 09:24:34 <etotheipi_> that 11.9s is from the time the process starts until it ends
161 2012-08-02 09:25:04 <etotheipi_> so the OS could be caching, but that is from a cold start
162 2012-08-02 09:25:10 <etotheipi_> not just the scan
163 2012-08-02 09:25:18 <sipa> SSD?
164 2012-08-02 09:25:51 <etotheipi_> no SSD
165 2012-08-02 09:26:06 <etotheipi_> that's why I'm so impressed
166 2012-08-02 09:26:28 <etotheipi_> I turned on compression, but of course it wasn't compressed much...
167 2012-08-02 09:26:44 <etotheipi_> it must just do near-optimal forward caching/pre-fetching
168 2012-08-02 09:27:27 <etotheipi_> I'm sure it wouldn't be that fast if I was iterating over all the key-value pair using the keys....
169 2012-08-02 09:27:28 <sipa> btw, not sure you saw the numbers, but the total serialized size of the txout set in ultraprune was 70 MB, a few weeks ago
170 2012-08-02 09:27:30 <etotheipi_> oh, that's pretty good!
171 2012-08-02 09:27:42 <sipa> with BDB overhead, 110
172 2012-08-02 09:28:24 <etotheipi_> leveldb overhead for storing all of blk0001.dat is about 10%
173 2012-08-02 09:28:36 <sipa> that's good
174 2012-08-02 09:28:44 <etotheipi_> blk0001.dat is 2.0 GB,   the leveldb directory holding all the same info is 2.2GB
175 2012-08-02 09:28:46 <sipa> really good, even
176 2012-08-02 09:29:24 <etotheipi_> I'm tremendously impressed
177 2012-08-02 09:29:24 <sipa> looking forward to combining leveldb and ultraprune :)
178 2012-08-02 09:29:39 <etotheipi_> haha, I would be, too
179 2012-08-02 09:30:11 <etotheipi_> I spent a lot of timing trying ti implement my own caching/pre-fetching/memorymapping/etc....
180 2012-08-02 09:30:20 <etotheipi_> and didn't come close to that
181 2012-08-02 09:30:30 <yellowhat> what is the most convinient way to get the hex encoded transaction (manually) for a given tx id? is there a blockexplorer that can give me the hex encoded binary instead of a json?
182 2012-08-02 09:31:07 <sipa> though, full import took (at some point a few weeks ago, without sig checking) 7 minutes on disk, 6 minutes when on tmpfs, and 5 minutes when using an std::map backend
183 2012-08-02 09:31:19 <sipa> so i'm not sure how much there is left to gain
184 2012-08-02 09:31:28 <etotheipi_> what do you mean full import?
185 2012-08-02 09:31:39 <sipa> sync the blockchain from scratch
186 2012-08-02 09:31:41 <lianj> yellowhat: of a json or the real transaction?
187 2012-08-02 09:32:11 <sipa> (the blk0001.dat file being imported from already in OS cache beforehand)
188 2012-08-02 09:32:27 <yellowhat> of a real transaction, e8ac451e7f47d566d4dd822a67fea2181ecfa7c7ba96d9faa63e2f6b26e55fd3
189 2012-08-02 09:33:41 <sipa> etotheipi_: though i used some tricks, like combining multiple small block updates into a single db commit
190 2012-08-02 09:33:53 <lianj> yellowhat: ruby -ropen-uri -e 'puts open("https://coinbase.com/network/tx/e8ac451e7f47d566d4dd822a67fea2181ecfa7c7ba96d9faa63e2f6b26e55fd3.bin").read.unpack("H*")[0]'
191 2012-08-02 09:34:03 <sipa> at least for bdb, that helps a lot
192 2012-08-02 09:34:43 <yellowhat> thanks linaj, binary is fine too :)
193 2012-08-02 09:35:05 <lianj> yellowhat: then skip the .unpack("H*")[0] ;)
194 2012-08-02 09:35:29 <etotheipi_> sipa, do you know a way to get a real "cold start" for my timing?
195 2012-08-02 09:35:35 <etotheipi_> can I somehow clear the cache?
196 2012-08-02 09:35:46 <sipa> etotheipi_: reboot :p
197 2012-08-02 09:35:51 <yellowhat> i don't have ruby installed, but i fetched the file which i can load directly then.
198 2012-08-02 09:35:59 <etotheipi_> maybe mv the directory?
199 2012-08-02 09:35:59 <sipa> (read: no idea)
200 2012-08-02 09:36:12 <sipa> that won't move the data
201 2012-08-02 09:36:18 <lianj> yellowhat: ok, have fun
202 2012-08-02 09:36:24 <sipa> and the cache is at the blockdev level
203 2012-08-02 09:36:29 <etotheipi_> gah...
204 2012-08-02 09:36:30 <sipa> not the fs level
205 2012-08-02 09:37:00 <sipa> put it on a USD disk, and unplug it :p
206 2012-08-02 09:37:19 <etotheipi_> okay, too much effort :)
207 2012-08-02 09:38:35 <etotheipi_> moving on...
208 2012-08-02 10:20:38 <etotheipi_> sipa, indeed caching must be going on... because I'm pretty sure I'm hitting all 2.2 GB of leveldb data, but non-cached HDD reads (hdparm) is only 107 MB/s
209 2012-08-02 10:27:12 <gmaxwell> 03:47 < wumpus> yeah, but we'll have to remain compatible with old wallets likely forever
210 2012-08-02 10:27:25 <gmaxwell> We _could_ make an external wallet converter utility potentially.
211 2012-08-02 10:28:04 <wumpus> yes, we could... but as sipa said, it needs to trigger automatically (to not upset users), so it needs to be packaged anyway
212 2012-08-02 10:28:47 <sipa> gmaxwell: maybe you want to comment: https://bitcointalk.org/index.php?topic=74581.msg1072950#msg1072950
213 2012-08-02 10:30:27 <OneEyed> Anyone from bitmarket.eu here?
214 2012-08-02 10:31:57 <gmaxwell> 04:35 < etotheipi_> can I somehow clear the cache?
215 2012-08-02 10:32:10 <gmaxwell> etotheipi_: /proc/sys/vm/drop_caches
216 2012-08-02 10:32:18 <sipa> ah, good to know
217 2012-08-02 11:08:34 <etotheipi_> gmaxwell, what is drop_caches?  what do I do with it?
218 2012-08-02 11:08:45 <etotheipi_> it doesn't appear to be a script
219 2012-08-02 11:09:07 <gmaxwell> echo 3 > /proc/sys/vm/drop_caches
220 2012-08-02 11:09:37 <gmaxwell> "Does what it says"  the number is how hard to drop the caches, 3 is the hardest IIRC.
221 2012-08-02 11:10:50 <sipa> 1: free page cache
222 2012-08-02 11:10:56 <sipa> 2: dentries and inodes
223 2012-08-02 11:11:03 <sipa> 3: both
224 2012-08-02 11:11:52 <etotheipi_> oh yeah... that just destroyed the scan speed
225 2012-08-02 11:11:56 <etotheipi_> probably 10x
226 2012-08-02 11:52:49 <jgarzik> etotheipi_: shocking ;)
227 2012-08-02 11:58:02 <jgarzik> I wish there was some way to filter out the idiots on bitcointalk.org, so that only useful people showed up in a thread.  MagicalTux still posts there etc,. but it is tough to find sane people through all the trolls.
228 2012-08-02 11:58:14 <jgarzik> clicking 'ignore' on each one is not scalable
229 2012-08-02 11:58:52 <jgarzik> forrestv: ping
230 2012-08-02 11:59:45 <MagicalTux> jgarzik: that'd reduce most of those 50+ pages threads to a couple of pages
231 2012-08-02 12:00:07 <jgarzik> pretty much :)
232 2012-08-02 12:00:46 <epscy> the forums are boring
233 2012-08-02 12:01:24 <MagicalTux> epscy: you don't have enough popcorn
234 2012-08-02 12:01:31 <MagicalTux> I'm pretty sure this could turn into a nice drama
235 2012-08-02 12:01:37 <gmaxwell> "Hey what happened with $X??" "No one knows yet." "Oh. okay." [end]
236 2012-08-02 12:01:40 <epscy> MagicalTux: which thread?
237 2012-08-02 12:03:19 <MagicalTux> epscy: all, more or less
238 2012-08-02 12:03:34 <MagicalTux> right now, especially all about Bitcoinica
239 2012-08-02 12:03:49 <epscy> oh yeah, just saw Tihan's thread
240 2012-08-02 12:46:26 <Ferroh> !seen genjix
241 2012-08-02 12:46:26 <gribble> genjix was last seen in #bitcoin-dev 1 week, 2 days, 19 hours, 55 minutes, and 31 seconds ago: <genjix> are you really such a burnt up bitter person inside?
242 2012-08-02 13:00:40 <Lolcust> Hello! Since ABE site is offline, does anyone know a different place where I could look up average coin age in BTC ?
243 2012-08-02 13:11:36 <etotheipi_> sipa, random thought: is it required to use the smallest var_int possible for blockchain serializations?   It seems like such a small thing, but it seems that if you start storing pieces of blocks/txs and expect to reconstruct them later, someone who used a var_int 0xff0000000000000001 to represent the number 1 would be valid but produce a different hash
244 2012-08-02 13:13:31 <etotheipi_> could cause all sorts of heartache...
245 2012-08-02 13:14:37 <etotheipi_> something very small but significant to consider... someone could start serializing blocks using the 8byte version of every var_int and really throw off all the clients that may accidentally replace that data with smaller var_ints
246 2012-08-02 13:16:27 <t7> nice little DOS ?
247 2012-08-02 13:17:54 <jgarzik> har!
248 2012-08-02 13:18:03 <jgarzik> someone brought SatoshiDice to litecoin
249 2012-08-02 13:18:15 <etotheipi_> it may be nothing, but it also wouldn't surprise me if weird things start happening if I start creating transactions using 8byte var_ints for numTxIn and numTxOut
250 2012-08-02 13:19:01 <jgarzik> etotheipi_: we never re-build from scratch
251 2012-08-02 13:19:06 <sipa> etotheipi_: i have no idea what you're talking about
252 2012-08-02 13:19:30 <etotheipi_> sipa, I was focusing on your comment about storing unspent txouts using maps
253 2012-08-02 13:19:37 <sipa> ok?
254 2012-08-02 13:19:42 <etotheipi_> so you store a tx by its txid, and a list of txouts
255 2012-08-02 13:19:54 <etotheipi_> but you don't store the exact serialization of those txouts
256 2012-08-02 13:19:59 <jgarzik> sipa: if someone sends bitcoin transaction X, constructed sub-optimally, he worries that we internally reconstruct the tx optimally, thereby changing its serialization and thus hash value
257 2012-08-02 13:20:04 <sipa> yes, in a special compact representation
258 2012-08-02 13:20:07 <jgarzik> sipa: but I don't think we ever do that
259 2012-08-02 13:20:10 <etotheipi_> err.. you store the exact txouts, but not the tx itself
260 2012-08-02 13:20:27 <etotheipi_> then if you need to verify the hash, you reconstruct the tx from the TxOuts
261 2012-08-02 13:20:30 <etotheipi_> but you don't get the right hash
262 2012-08-02 13:20:44 <jgarzik> etotheipi_: no, we don't reconstruct
263 2012-08-02 13:20:46 <sipa> the mapping is 1-to-1
264 2012-08-02 13:20:48 <etotheipi_> because the original creator of the tx used an 8-byte var_int
265 2012-08-02 13:21:03 <sipa> it reencodes the serialized data
266 2012-08-02 13:21:31 <sipa> byte for byte
267 2012-08-02 13:21:49 <sipa> not just its semantics
268 2012-08-02 13:23:21 <jgarzik> CastToBigNum() is fun
269 2012-08-02 13:24:01 <jgarzik> its purpose is to make canonical bignums that might have a ton of extra data (leading zeroes), somewhat analagous to what etotheipi_ is worrying about with varint
270 2012-08-02 13:24:33 <jgarzik> bitcoin has its own unique little endian bignum encoding, too, sigh
271 2012-08-02 13:26:08 <etotheipi_> like I said, it's probably nothing... but it wouldn't surprise me to see someone create such a tx in the far future, and something important couldn't deal with it
272 2012-08-02 13:26:57 <etotheipi_> I wasn't sure if it was a possibility with sipa's idea for storing not-exactly-serialized transactions for pruning purposes
273 2012-08-02 13:29:48 <jgarzik> etotheipi_: (hash,n) is pretty static, regardless of how it is passed around
274 2012-08-02 13:32:23 <etotheipi_> jgarzik, what do you mean "(hash,n)"
275 2012-08-02 13:32:24 <sipa> well i do have a special serializer for amounts
276 2012-08-02 13:32:47 <sipa> that gains you 6 bytes per txout
277 2012-08-02 13:32:49 <etotheipi_> sipa, didn't you say that for pruned tx data, you were storing (txid, mapTxOut)
278 2012-08-02 13:32:58 <sipa> yes
279 2012-08-02 13:33:15 <sipa> listUnspentTxOut, really
280 2012-08-02 13:33:22 <etotheipi_> I'm envisioning that for a full node... that map holds *all* of the txOuts
281 2012-08-02 13:33:26 <etotheipi_> for that tx
282 2012-08-02 13:33:28 <sipa> no
283 2012-08-02 13:33:36 <sipa> only unspent ones
284 2012-08-02 13:33:58 <sipa> if you need the full tx, use the one in blk0001.dat
285 2012-08-02 13:34:45 <etotheipi_> okay... I was just trying to highlight a situation where it could be an issue... but it sounds like a non-issue for you
286 2012-08-02 13:36:38 <sipa> i used the entire blockchain as test set
287 2012-08-02 13:37:15 <sipa> serializing every txout list in my format, deserializing it, and comparing it to the original
288 2012-08-02 13:39:09 <etotheipi_> sipa, I don't expect there to be any blocks that do this
289 2012-08-02 13:39:38 <sipa> do what, precisely?
290 2012-08-02 13:39:39 <etotheipi_> I just wanted you to consider the possibility that I start creating all transactions using 8-byte var_ints for script sizes and #txin & #txout
291 2012-08-02 13:40:02 <etotheipi_> make sure this doesn't cause your code to fail
292 2012-08-02 13:40:35 <sipa> how can it? the original tx is never reconstructed
293 2012-08-02 13:40:47 <sipa> only the txouts are used
294 2012-08-02 13:41:15 <sipa> anyway, good that you notice the potential problem
295 2012-08-02 13:41:24 <sipa> i must say i didn't consider it
296 2012-08-02 13:41:46 <etotheipi_> got it
297 2012-08-02 13:43:57 <sipa> (i did pay attention - and i'm quite sure i can guarantee - that ever txout on itself is reconstructed byte-for-byte identically)
298 2012-08-02 13:44:02 <sipa> *every
299 2012-08-02 13:44:55 <sipa> i suppose i should write the specification for the compact serialization down somewhere, as there are quite some special cases (all of which are already tested)
300 2012-08-02 14:00:06 <gribble> New news from bitcoinrss: TheBlueMatt opened pull request 1651 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1651>
301 2012-08-02 14:08:35 <luke-jr> sipa: we seriously just save orphans in RAM? that could explain some IBD issues
302 2012-08-02 14:09:09 <sipa> yes
303 2012-08-02 14:09:47 <luke-jr> in that case, maybe it's semi-important to fix the out-of-order IBD clients get thrown into when their download peer gets a new block&
304 2012-08-02 14:10:20 <sipa> i have recently seen strange long series of blocks being downloaded out-of-order, so it may be becoming an issue indeed
305 2012-08-02 14:12:05 <luke-jr> sipa: in my experience, if your download peer gets a new block, it starts sending you backward from that'
306 2012-08-02 14:12:14 <luke-jr> forgetting about the chronological download
307 2012-08-02 14:14:39 <sipa> that's very strange, as your node should request getdata's in order
308 2012-08-02 14:16:21 <jgarzik> sipa: I see a lot of such strangeness on my public nodes as well
309 2012-08-02 14:16:52 <jgarzik> sipa: you're welcome to the debug.log files now or in the future, if correlation is ever needed.  I've no wallets or anything private on them.  gmaxwell, you too.
310 2012-08-02 14:17:44 <jgarzik> sipa: I also see
311 2012-08-02 14:17:46 <jgarzik> getblocks -1 to 00000000000000000000 limit 500
312 2012-08-02 14:17:57 <jgarzik> sipa: when 'getpeerinfo' shows all nodes with starting height > 190000
313 2012-08-02 14:18:56 <sipa> that's very strange
314 2012-08-02 14:19:07 <sipa> do you happen to know which versions?
315 2012-08-02 14:19:35 <jgarzik> etotheipi_: when in doubt, mine a few transactions on testnet3 with your strange behavior. point people to those tx's.
316 2012-08-02 14:19:48 <jgarzik> etotheipi_: after 30 days, post a couple on mainnet
317 2012-08-02 14:19:48 <sipa> as there have been a few fixes for the stuck block issue that overreacted
318 2012-08-02 14:20:07 <sipa> i can't remember exactly, but maybe some older buggy fix causes this
319 2012-08-02 14:20:22 <jgarzik> sipa: sorry, you know how our logging is...   I know (a) I received that getdata and (b) [$these] are my list of peers
320 2012-08-02 14:21:11 <jgarzik> etotheipi_: the best test for anything like this is putting it on testnet3, and letting people beat on it with their software
321 2012-08-02 14:21:33 <jgarzik> etotheipi_: gavin put as many 'weird script' cases into testnet3 chain as he could
322 2012-08-02 14:21:54 <jgarzik> etotheipi_: please do spam the testnet3 chain with strange ideas and experiments
323 2012-08-02 14:33:12 <gmaxwell> There was someone in here reporting a few particular nodes with high initial heights slamming him with unresonable looking getblocks.
324 2012-08-02 14:47:27 <jgarzik> gmaxwell: might have been me
325 2012-08-02 14:47:35 <OneEyed> Shit, the child-pays-for-parent will ruin my plans of becoming rich by not paying TX fees&
326 2012-08-02 14:48:24 <jgarzik> wumpus: any chance you could make a pass through the pull requests, and take-or-close UI-related ones?
327 2012-08-02 14:49:05 <gmaxwell> jgarzik: No, zevus.
328 2012-08-02 14:49:17 <jgarzik> gmaxwell: interesting
329 2012-08-02 14:49:19 <gmaxwell> (I mean, you mentioned it too, but I was talking about zevus.)
330 2012-08-02 15:19:23 <helo> has there been any discussion to extend the URI to allow transaction payloads (ideally signed or unsigned)?
331 2012-08-02 16:05:31 <jgarzik> Gavin ack'd 0.7-rc1 while he's absent
332 2012-08-02 16:06:51 <luke-jr> what does "Encore on the name." mean? :/
333 2012-08-02 16:07:43 <OneEyed> "Encore" means "Again" in French, if that helps
334 2012-08-02 16:08:03 <gribble> New news from bitcoinrss: jgarzik opened pull request 1652 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1652>
335 2012-08-02 16:10:00 <luke-jr> 1: a demand for repetition or reappearance made by an audience
336 2012-08-02 16:10:01 <luke-jr> 2a : a reappearance or additional performance demanded by an audience  b : a second achievement especially that surpasses the first
337 2012-08-02 16:12:17 <luke-jr> jgarzik: can you interpret?
338 2012-08-02 16:15:53 <jgarzik> luke-jr: I think he means the name is fine as it is.  Regardless, it costs little to proceed with that interpretation, and change the name later.
339 2012-08-02 16:17:26 <luke-jr> "as it is"? :P
340 2012-08-02 16:17:35 <luke-jr> so can we go with "blockmining"? <.<
341 2012-08-02 16:18:43 <jgarzik> luke-jr: getblocktemplate
342 2012-08-02 16:18:56 <luke-jr> meh, whatever
343 2012-08-02 16:23:22 <luke-jr> jgarzik: OK, updated the pullreq metainfo; will rebase longpolling when the first is pulled
344 2012-08-02 16:38:35 <jgarzik> luke-jr:  do you have an updated doc for getblocktemplate anywhere?
345 2012-08-02 16:38:52 <luke-jr> https://en.bitcoin.it/wiki/BIP_0022 ?
346 2012-08-02 16:45:54 <jgarzik> luke-jr: you updated it already?  good.
347 2012-08-02 16:48:07 <luke-jr> yeah
348 2012-08-02 17:41:50 <BlueMatt> does it seem to anyone else that Script.FindAndDelete could be implemented better?...seems like requiring that the script chunk be pushed the same way (same OP_PUSHDATA/etc) could cause a problem somewhere...
349 2012-08-02 17:45:31 <BlueMatt> (or maybe I'm complaining because it makes re-implementing it harder with the way bitcoinj does its script parsing)
350 2012-08-02 17:47:25 <Ferroh> so I know this is a little off topic
351 2012-08-02 17:47:49 <Ferroh> I am implementing an API for this site, and I realized that I could make site.com/account/history the usual page that you visit to see your account history
352 2012-08-02 17:48:08 <Ferroh> but if you POST a var "api" to that page, then it could return JSON data only
353 2012-08-02 17:48:15 <Ferroh> OR I could do what we typically see,
354 2012-08-02 17:48:28 <Ferroh> and make users visit site.com/api/account/history to get that JSON data
355 2012-08-02 17:48:54 <Ferroh> do you guys think it would be bad to just tell users to go to the usual pages and POST "api" to the page to see what api data it will give you?
356 2012-08-02 17:51:11 <Ferroh> whatever i'll just make an API URL, i can always change it
357 2012-08-02 17:56:51 <grondilu> anyone has managed to compile bitcoin on fedora?
358 2012-08-02 17:57:06 <Ferroh> probably a lot of people :P
359 2012-08-02 17:57:23 <Ferroh> you are probably missing a dependency
360 2012-08-02 17:57:26 <Ferroh> what is the error
361 2012-08-02 17:57:47 <Ferroh> the readme says all the dependencies you need to install
362 2012-08-02 17:57:57 <Ferroh> on fedora do you apt-get stuff? just pretend it's ubuntu in that case
363 2012-08-02 17:58:10 <Ferroh> i havent used fedora in like 5 years
364 2012-08-02 17:58:26 <BlueMatt> grondilu: you have to compile your own openssl
365 2012-08-02 17:58:26 <grondilu> at some point I lacked boost/thread.hpp so I installed boost141 and boost141-dev but it seems the header is still not found.
366 2012-08-02 17:58:45 <grondilu> BlueMatt: really?
367 2012-08-02 17:58:47 <BlueMatt> oh, boost, well I dunno
368 2012-08-02 17:58:51 <Ferroh> do you know the location of the boost headers?
369 2012-08-02 17:59:19 <BlueMatt> grondilu: last I heard fedora still removed the ecdsa stuff we use for copyright reasons
370 2012-08-02 17:59:32 <BlueMatt> so you had to compile your own, I wouldnt know the current status though
371 2012-08-02 18:00:32 <grondilu> Ferroh: /usr/include/boost141/boost/thread.hpp
372 2012-08-02 18:00:44 <Ferroh> BOOST_INCLUDE_PATH=/usr/include/boost141/ makefile -f makefile.unix -j2
373 2012-08-02 18:00:49 <Ferroh> try building using that then
374 2012-08-02 18:00:59 <grondilu> ok
375 2012-08-02 18:03:15 <jgarzik> BlueMatt: s/copyright/patent/
376 2012-08-02 18:03:33 <BlueMatt> yea, sorry
377 2012-08-02 18:03:37 <grondilu> I get a lot of errors
378 2012-08-02 18:04:27 <grondilu> lots of "#error "Boost threads unavailable on this platform"
379 2012-08-02 18:04:44 <gmaxwell> grondilu: it's trivial to complile bitcoin on fedora, other than the openssl it works out of the box with the right arguments
380 2012-08-02 18:05:27 <gmaxwell> for bitcoind, BOOST_LIB_SUFFIX='-mt' make -j4 -f makefile.unix bitcoind USE_UPNP=
381 2012-08-02 18:05:43 <gmaxwell> For the GUI, qmake-qt4 USE_UPNP=- BOOST_LIB_SUFFIX=-mt
382 2012-08-02 18:06:16 <gmaxwell> I have openssl builds at http://people.xiph.org/~greg/openssl/ though I'm a version behind and need to update them.
383 2012-08-02 18:06:41 <gmaxwell> (also, if you use my binaries instead of just making your own based on my changes in the spec be sure to validate the signatures)
384 2012-08-02 18:06:55 <grondilu> gmaxwell: src/util.h:21:28: fatal error: boost/thread.hpp: No such file or directory
385 2012-08-02 18:07:27 <grondilu> and I did exactly 'qmake-qt4 USE_UPNP=- BOOST_LIB_SUFFIX=-mt'
386 2012-08-02 18:07:31 <gmaxwell> grondilu: what version of fedora are you on?
387 2012-08-02 18:07:36 <grondilu> F17
388 2012-08-02 18:08:07 <gmaxwell> Sounds like you've screwed up your boost install then.
389 2012-08-02 18:08:27 <gmaxwell> do you have boost-devel-1.48.0-13.fc17 installed?
390 2012-08-02 18:10:00 <grondilu> I did install boost141-devel.  I guess that's my mistake.
391 2012-08-02 18:10:17 <gmaxwell> just yum install boost-devel
392 2012-08-02 18:11:02 <gmaxwell> you'll probably need some qt-devel if you don't already have it.
393 2012-08-02 18:12:21 <grondilu> yeah I installed this one already
394 2012-08-02 18:18:51 <BlueMatt> Im disappointed no one seems to care about win32 auto-update, I would argue thats addresses a decently major issue (people having upgrade apathy) that results in a number of other issues
395 2012-08-02 18:21:21 <gmaxwell> But it brings with it other issues.
396 2012-08-02 18:21:47 <gmaxwell> Also, we are short on windows using technical contributors in general.
397 2012-08-02 18:23:43 <Eliel> auto-update brings more danger than benefits IMO
398 2012-08-02 18:24:18 <gmaxwell> Eliel: there are ways to address the dangers.
399 2012-08-02 18:24:34 <BlueMatt> gmaxwell: which other issues?
400 2012-08-02 18:25:10 <BlueMatt> gmaxwell: ok, but we are heavy windows users in general, which means we should have auto-update for them...
401 2012-08-02 18:26:17 <gmaxwell> BlueMatt: the security problem of potentially being able to push a rooted (private key stealing) bitcoin out to an enormous number of users near instantly.
402 2012-08-02 18:26:45 <gmaxwell> (and the fud connected to people promoting that risk)
403 2012-08-02 18:26:48 <BlueMatt> gmaxwell: gitian?
404 2012-08-02 18:27:00 <gmaxwell> Yes. thus "there are ways to address the dangers"
405 2012-08-02 18:27:02 <BlueMatt> the fud I agree, but the actual risk is pretty low
406 2012-08-02 18:27:07 <BlueMatt> oh, missed that
407 2012-08-02 18:27:34 <gmaxwell> I think it should happen, I don't think you should worry that it hasn't happened yet.
408 2012-08-02 18:27:54 <BlueMatt> Im not worried, Im just commenting on the lack of interest
409 2012-08-02 18:35:26 <Eliel> well, if it's ever done, it should only happen if the new version has signatures from several different developers.
410 2012-08-02 18:36:04 <wumpus> which is what the gitian updater does isn't it...
411 2012-08-02 18:36:18 <BlueMatt> yes, thats how it is in the current implementation
412 2012-08-02 18:36:31 <wumpus> I'm also surprised, many people with comments but no one bothers looking at the actual pull :p
413 2012-08-02 18:36:43 <phantomcircuit> wumpus, sounds like work
414 2012-08-02 18:36:52 <phantomcircuit> that's not allowed around here
415 2012-08-02 18:37:38 <gmaxwell> Eliel: thats exactly what gitian is for... I also asked for the ability to have NAK votes, dunno if that got in yet.
416 2012-08-02 18:37:53 <gmaxwell> Eliel: so e.g. you could give out keys to veto a release a bit more liberally than to approve it.
417 2012-08-02 18:38:31 <BlueMatt> gmaxwell: hmm...how would that be implemented in a mitm scenario?
418 2012-08-02 18:39:14 <wumpus> I also think windows auto updating is important (or at least make it easier to do, like bluematt's pull does), but I'm not a windows developer myself so there's not that much I can do
419 2012-08-02 18:39:37 <gmaxwell> BlueMatt: by pulling signatures from multiple places and not updating until it can get them from more than one.
420 2012-08-02 18:40:09 <wumpus> for linux and mac etc it's unsubstantial, those operating systems have their own sw updating methods, it's just imporant for windows
421 2012-08-02 18:40:12 <BlueMatt> and if the ones which are providing negative sigs are blocked by the mitm?
422 2012-08-02 18:40:39 <gmaxwell> BlueMatt: Then you fail. Thats okay. It can't be perfect and doesn't have to be.
423 2012-08-02 18:41:26 <gmaxwell> BlueMatt: thresholds can be set so that you have to compromise a bunch of things instead of a few to get a bad update out and installed; thats the most we could hope to achieve.
424 2012-08-02 18:41:47 <c_k> any of you guys using OS X 10.8?
425 2012-08-02 18:41:49 <BlueMatt> that seems overly brittle...I agree it would be nice to have a nak method, but it seems like you could accomplish something similar with differing levels of key values
426 2012-08-02 18:42:00 <c_k> bit coin won't run without disabling approved apps in system settings ...
427 2012-08-02 18:42:15 <BlueMatt> (as gitian is, each signer gets a score, and you have to have a min score)
428 2012-08-02 18:42:18 <c_k> there is also something about OK'd developers that will let the app run without having to do that
429 2012-08-02 18:42:40 <c_k> you guys should investigate getting the approved developer thing, what ever it is
430 2012-08-02 18:42:44 <gmaxwell> BlueMatt: you can, but you can't cap a signer to only provide nakness.
431 2012-08-02 18:42:46 <BlueMatt> gmaxwell: seems like for liberal allocation of otherwise nak signers you could give them a low score and get something sort-of similar wihout the mitm ugliness
432 2012-08-02 18:43:15 <BlueMatt> c_k: there has been recent discussion of doing that
433 2012-08-02 18:43:29 <c_k> ah yeah, I just looked at the mailing list after I said it and saw it :S
434 2012-08-02 18:45:37 <gmaxwell> BlueMatt: in any case, indifference to a new release is normal. You don't want to have a high score. And you also don't want a bunch of useless "ran for me +1s" that can be undone by someone a few hours later "-1000 found remote root exploit in code audit"
435 2012-08-02 18:47:07 <BlueMatt> gmaxwell: it seems to me like the major-bug-found-after-release issue isnt really addressed well by a nak system either
436 2012-08-02 18:47:55 <BlueMatt> if you mean maliciously-added-major-bug-found-after-release then I dont know that a nak system necessarily solves that either
437 2012-08-02 18:47:56 <gmaxwell> Fedora's release process has this problem in fact, rpms go into testing first and people can +1 one and enough trigger updates to go out. Sometimes you'll see +1 works for me +1 didn't crash -1 corrupts hdd after 24 hours  and the release pushes.
438 2012-08-02 18:48:26 <gmaxwell> BlueMatt: sure it does, because the updates should quarantine releases in any case.
439 2012-08-02 18:49:03 <BlueMatt> Im not concerned about major-bug-found-after-release, for that a new release can be pushed really quick and the buggy one can be removed immediately
440 2012-08-02 18:49:09 <gmaxwell> Some people connected with tor project folks wrote a very long paper on autoupdates exploring all these issues, it's a little sad that this research is just being ignored.
441 2012-08-02 18:49:37 <gmaxwell> especially because we have all the security concerns they have and then some.
442 2012-08-02 18:49:45 <sipa> luke-jr: i interpret "encore" as +1
443 2012-08-02 18:50:07 <BlueMatt> I agree it would absolutely be nice to do something like naks, but I just dont see a way to implement it well that doesnt make the update system a bit brittle to server issues but also is useful against mitm attacks
444 2012-08-02 18:50:47 <BlueMatt> though Im probably just being dense
445 2012-08-02 18:51:11 <BlueMatt> in any case, adding it to gitian after auto-update is added to bitcoin isnt hard :)
446 2012-08-02 18:51:36 <sipa> just have a tuple (name,key,maxposscore,minnegscore) for all developers
447 2012-08-02 18:51:54 <sipa> and several urls to fetch signatures from
448 2012-08-02 18:52:04 <BlueMatt> and mitm attacks?
449 2012-08-02 18:52:12 <sipa> like?
450 2012-08-02 18:52:28 <BlueMatt> which blocks urls which are providing negative sigs
451 2012-08-02 18:52:36 <BlueMatt> or maybe a ddos against those servers
452 2012-08-02 18:52:38 <gmaxwell> BlueMatt: I think you're overestimating the sever concerns. If N servers can't be kept up we have other problems. Also: Here is a method: Must have enough abs(score), thus making it much harder to filter out naks.
453 2012-08-02 18:52:53 <sipa> that's why there are several urls
454 2012-08-02 18:52:58 <sipa> in multiple legislations
455 2012-08-02 18:53:03 <sipa> on multiple servers
456 2012-08-02 18:53:11 <sipa> by multiple hosts/providers
457 2012-08-02 18:53:20 <gmaxwell> sipa: he's concerned that you'll be not allowed to connect to ANY of the honest ones.
458 2012-08-02 18:53:35 <gmaxwell> And I say just solve that by requring at lest N servers. Which he thinks makes the process brittle.
459 2012-08-02 18:54:17 <sipa> hmm, could be
460 2012-08-02 18:54:25 <gmaxwell> Or instead of abs(score), just require N people where perhaps many people give 0 scores because they didn't review it... they're just signing as evidence that the list is complete and hasn't been nak filtered.
461 2012-08-02 18:54:28 <BlueMatt> meh, maybe Im just being overly risk-sensitive, still Im not a huge fan of the implementation
462 2012-08-02 18:55:02 <BlueMatt> that works a bit better
463 2012-08-02 18:55:32 <sipa> hey let's build a block chain with signatures!
464 2012-08-02 18:55:35 <gmaxwell> hahah.
465 2012-08-02 18:55:42 <BlueMatt> better yet, a DHT
466 2012-08-02 18:55:49 <sipa> oh yes!
467 2012-08-02 18:56:00 <gmaxwell> There is a point here, we could merge mine a gitian signature table.
468 2012-08-02 18:56:12 <gmaxwell> Makes it hard to provide a nak filtered table then.
469 2012-08-02 18:56:31 <sipa> overkill...
470 2012-08-02 18:56:43 <BlueMatt> yea...just 0-value sigs would work too
471 2012-08-02 18:56:47 <BlueMatt> is devrandom on, btw?
472 2012-08-02 18:57:21 <gmaxwell> BlueMatt: well could I filter out naks while leaving 0-value sigs in? (because they're currently independant)?
473 2012-08-02 18:58:11 <BlueMatt> true, but you could merge them in some way without merged-mining (just a sig chain?)
474 2012-08-02 18:58:30 <gmaxwell> Sure.
475 2012-08-02 18:59:02 <gmaxwell> The value of the merged mining is that you can validate that there was a bunch of computation behind the sigset.
476 2012-08-02 18:59:20 <BlueMatt> I dont see how that is entirely relevant here?
477 2012-08-02 18:59:21 <sipa> wow
478 2012-08-02 18:59:38 <BlueMatt> I dont care how big the computers the devs have?
479 2012-08-02 18:59:41 <sipa> btc/usd > 10 :o
480 2012-08-02 18:59:43 <BlueMatt> I care that they have sigs
481 2012-08-02 18:59:50 <BlueMatt> again?
482 2012-08-02 18:59:53 <BlueMatt> dammit
483 2012-08-02 19:01:32 <BlueMatt> what we need is some super-rich billionare who wants to make a very small % to invest in bitcoin and fix the price within a range
484 2012-08-02 19:01:34 <gmaxwell> BlueMatt: it has nothing to do with how much computing power the devs have. It tells you that someone couldn't have maliciously filtered the tally without either great cost or control of significant computing power... but yes, not really necessary.. just lots of 0-value sigs confirming the whole set makes sense.
485 2012-08-02 19:02:28 <jgarzik> sipa gmaxwell: ACK troll for https://github.com/bitcoin/bitcoin/pull/1526 - gavin's height in coinbase
486 2012-08-02 19:02:52 <cande> g'night fokes
487 2012-08-02 19:02:53 <BlueMatt> gmaxwell: I like the idea of making sure the tally isnt maliciously edited, but I dont see how an independent 0-work "blockchain" that requires each block to be from a dev (with a sig) doesnt work just as well?
488 2012-08-02 19:03:18 <BlueMatt> that said, I dont see which problem that is necessarily addressing
489 2012-08-02 19:03:20 <cande> fyi, bitcoin just hit 11$
490 2012-08-02 19:03:21 <gmaxwell> luke-jr: why octet on 1526 and not last _bit_ ?
491 2012-08-02 19:04:00 <Ferroh> jesus:
492 2012-08-02 19:04:02 <Ferroh> " I need to find the highest number from 3 different numbers. The only thing I've found is max() but you can only use 2 numbers. "
493 2012-08-02 19:04:06 <Ferroh> /sigh
494 2012-08-02 19:04:15 <gmaxwell> ...
495 2012-08-02 19:04:19 <BlueMatt> wow
496 2012-08-02 19:04:39 <gmaxwell> Ferroh: send them to this to really blow their minds: http://pages.ripco.net/~jgamble/nw.html
497 2012-08-02 19:05:09 <grondilu> db_cxx.h is provided by three different packages, which one should I install?  db4-devel, libdb-devel or compat-db-headers?
498 2012-08-02 19:06:09 <BlueMatt> if you have malicious dev(s) they just ignore naks and build their chain without them, if you dont, I dont see why a release would be pushed with any naks at all? If a dev is nak'ing a release, chances are the release would be delayed (or the nak is for political/disagreement issues, in which case its all moot)
499 2012-08-02 19:06:13 <gmaxwell> grondilu: db4-devel
500 2012-08-02 19:06:43 <BlueMatt> if the nak comes after release (SECURITY BUG FOUND OMGGG) then the malicious mitm attacker just provides the first part of the chain and poof
501 2012-08-02 19:07:00 <gmaxwell> BlueMatt: because you require the chain to be sufficiently long that it would be hard for devs to get a long enough chain without would-be-nakers participating.
502 2012-08-02 19:07:24 <sipa> right, but what if everyone already positively signed
503 2012-08-02 19:07:28 <BlueMatt> I dont really see that working, are we now expecting all devs to have mining rigs?
504 2012-08-02 19:07:30 <sipa> and then a flaw is found?
505 2012-08-02 19:08:00 <BlueMatt> (otherwise the devs with bigger rigs could be the malicious ones...)
506 2012-08-02 19:08:14 <BlueMatt> and now we are weighting devs sig value by their hash power?
507 2012-08-02 19:08:22 <gmaxwell> sipa: you have people who can add [-100,0]  on good releases they put in 0. You require enough participants that you always have evidence that nakers had a chance.
508 2012-08-02 19:08:30 <gmaxwell> BlueMatt: _STOP_
509 2012-08-02 19:08:44 <gmaxwell> BlueMatt: you're being stupid. no one said anything like "weighting devs sig value by their hash power"
510 2012-08-02 19:09:12 <gmaxwell> Whenever someone you know isn't a complete idiot says something you think is insanely stupid you need to step back and ask if you misunderstood.
511 2012-08-02 19:09:14 <BlueMatt> no, you dont weight them, but now the hash power matters directly in the ability to avoid attack
512 2012-08-02 19:09:47 <gmaxwell> BlueMatt: No, it doesn't at all. I'm trying to figure out where your misunderstanding is.
513 2012-08-02 19:10:15 <gmaxwell> ah.. are you thinking I meant long in terms of computation??  No I mean long in terms of numbers of participants.
514 2012-08-02 19:10:39 <BlueMatt> oh, hey we are saying the same thing
515 2012-08-02 19:10:42 <BlueMatt> sorry
516 2012-08-02 19:11:32 <BlueMatt> wait, didnt you say merged-mined?
517 2012-08-02 19:11:39 <BlueMatt> that requires hash power does it not?
518 2012-08-02 19:11:58 <sipa> yes, but not necessarily your own
519 2012-08-02 19:12:10 <gmaxwell> I did earlier, but I was talking about someting else. And yes, I expect it would just be timestamped by some pool or another.
520 2012-08-02 19:12:11 <sipa> or not necessarily be people who influence the signatures
521 2012-08-02 19:12:31 <BlueMatt> oh, Im not a big fan of that...now we are trusting both devs + big miners
522 2012-08-02 19:12:36 <gmaxwell> Just some evidence that this is the real, complete, list.
523 2012-08-02 19:12:54 <BlueMatt> big miners would likely be much easier to convince to join a malicious group, and now it gets easier for them
524 2012-08-02 19:12:59 <sipa> well, it does at least function as timestamping
525 2012-08-02 19:13:04 <sipa> but why do you need that?
526 2012-08-02 19:13:14 <gmaxwell> It's not needed, I agree. I think using just acks are fine.
527 2012-08-02 19:13:14 <sipa> you dont risk a fake signature
528 2012-08-02 19:13:20 <sipa> you risk an old signature
529 2012-08-02 19:13:34 <sipa> so you want the newest set of signatures always
530 2012-08-02 19:13:35 <BlueMatt> anyway, back to my earlier point, I dont think we are addressing an actual issue with this branch of discussion
531 2012-08-02 19:13:55 <BlueMatt> <BlueMatt> if you have malicious dev(s) they just ignore naks and build their chain without them, if you dont, I dont see why a release would be pushed with any naks at all? If a dev is nak'ing a release, chances are the release would be delayed (or the nak is for political/disagreement issues, in which case its all moot) <-- still stands
532 2012-08-02 19:14:10 <gmaxwell> BlueMatt: No, we're not we're addressing a pointless obsession about not having to poll multiple servers by boiling the oceans.
533 2012-08-02 19:14:14 <BlueMatt> ofc if the nak comes after release, meh
534 2012-08-02 19:14:47 <gmaxwell> Because the release should be quarantined.
535 2012-08-02 19:15:08 <BlueMatt> ok, let me rephrase, Im probably wrong, but afaict, this actually doesnt remove the requirement of multiple servers for address any real attack scenario
536 2012-08-02 19:15:44 <gmaxwell> (You hit critical mass sigs for install (probably at time 0) then you wait another 24+random hours before installing; NAKs come before you install)
537 2012-08-02 19:16:46 <BlueMatt> ok...that addresses the issues with non-malicious devs, but in that case, maybe we should require a release wait 24 hours before pushing to gitian update files?
538 2012-08-02 19:17:01 <gmaxwell> BlueMatt: the multiple signatures do reduce/remove the need for mulple servers. Consider say your required sig length is everyone with an accepted signing key. Clearly a nak could not be filtered.
539 2012-08-02 19:17:31 <BlueMatt> agreed, but that can be implemented with the current gitian
540 2012-08-02 19:17:37 <sipa> surely people can change their sig?
541 2012-08-02 19:17:49 <sipa> i mean, go from positive to negative
542 2012-08-02 19:18:34 <gmaxwell> BlueMatt: it must be enforced by the clients, because you can't just assume devs are non-malicious.
543 2012-08-02 19:20:00 <gmaxwell> Its just an extension of the thinking that holding a key that lets you autoupdate on your own is basically proof that you're not competent to hold one to a group rather than a single person. (because you're too attractive a target)
544 2012-08-02 19:20:10 <BlueMatt> absolutely, but I (am probably being dense) dont see how any of this has solved any real attack scenarios with malicious devs in a way that cant be done with current gitian?
545 2012-08-02 19:20:56 <BlueMatt> sipa: yea, but...that goes back to how to implement it...
546 2012-08-02 19:21:02 <gmaxwell> It can only be fixed in current gitian by requiring a lot of scores and giving a lot of people keys which increases risks.
547 2012-08-02 19:21:36 <gmaxwell> Vs being able to give out negative to zero only keys which only pose the risk of someone obstructing an update.
548 2012-08-02 19:22:01 <gmaxwell> which is a far far less serious problem.
549 2012-08-02 19:22:16 <BlueMatt> yes, I agree naks would be nice, absolutely
550 2012-08-02 19:22:41 <BlueMatt> but, aside from requiring data from N/M servers, I dont think we have a solution to getting those naks in a reliable way
551 2012-08-02 19:22:47 <gmaxwell> but then you pointed out plain naks can be filtered. So I suggested a participants threshold. And somehow we got more complicated from there. :)
552 2012-08-02 19:23:49 <gmaxwell> Soon we'll reinvent addman. :)
553 2012-08-02 19:24:11 <BlueMatt> another idea: sig groups, instead of naks, require n/m sigs from each of x sig groups.  It allows for less risk of a bunch of low-value sigs adding up to push an update?
554 2012-08-02 19:25:26 <gmaxwell> 14:23 < gmaxwell> Soon we'll reinvent addman. :)
555 2012-08-02 19:25:37 <BlueMatt> addman?
556 2012-08-02 19:25:44 <sipa> addrman, i suppose
557 2012-08-02 19:25:47 <gmaxwell> addrman .. lossy r key.
558 2012-08-02 19:25:52 <BlueMatt> ah
559 2012-08-02 19:26:03 <gmaxwell> You can have groups based on where you heard the sig.. groups based on the identy of the sig itself...
560 2012-08-02 19:26:10 <sipa> hehe
561 2012-08-02 19:26:28 <BlueMatt> hah
562 2012-08-02 19:26:31 <BlueMatt> ok, ok
563 2012-08-02 19:26:33 <gmaxwell> but actually.. geo/national groups is an interesting though.
564 2012-08-02 19:27:22 <gmaxwell> naks are still good though. They reflect where open source security comes from: It only takes one person to find a problem to sound an alarm that fixes the world.
565 2012-08-02 19:27:38 <BlueMatt> absolutely
566 2012-08-02 19:27:56 <BlueMatt> but, in the case of non-malicious devs, its easier to address the issue outside of gitian
567 2012-08-02 19:28:02 <BlueMatt> and in the case of malicious devs...
568 2012-08-02 19:28:39 <BlueMatt> anyway, Im not being helpful here, Im gonna go back to finishing off the bitcoinj script execution engine
569 2012-08-02 19:28:39 <gmaxwell> You shouldn't bother seperating the cases. Just assume a subset are malicious. Thats the threat model you ought to take for this.
570 2012-08-02 19:29:15 <gmaxwell> yea.. Later I'll go find that tor related autoupdate paper and bug you to read it(again).
571 2012-08-02 19:29:42 <sipa> let's run bitcoin inside a VM, and distribute update code via signed transactions with a huge txout script
572 2012-08-02 19:29:55 <gmaxwell> 24 hour high: 11
573 2012-08-02 19:30:19 <gmaxwell> sipa: which spends the prior version's txn.
574 2012-08-02 19:30:50 <sipa> yes!!
575 2012-08-02 19:31:39 <gmaxwell> "There can only be one. No really. Our byzantine consensus algorithim makes the possiblity of a conflicting update negligible"
576 2012-08-02 19:38:01 <sipa> oh, and you can force a reorg by spending the code tx, and sending it to 000000
577 2012-08-02 19:38:20 <sipa> then it reorgs back both the code and the chain to an earlier version :)
578 2012-08-02 19:39:09 <gmaxwell> sipa: hush, solidcoin people might be listening, they may implement these terrible ideas. ;)
579 2012-08-02 19:39:38 <copumpkin> I think they take your disapproval as an indicator of viability
580 2012-08-02 19:39:58 <copumpkin> gmaxwell: so pick lots of other shitty things and tell everyone how much you disapprove of them
581 2012-08-02 19:43:03 <jgarzik> IMNSHO Windows auto-update should be a post-0.7 issue
582 2012-08-02 19:43:27 <jgarzik> though I admit I am biased and don't like auto-update in general
583 2012-08-02 19:44:23 <gmaxwell> I think it's a post 0.7 issue now.
584 2012-08-02 19:47:10 <gmaxwell> (that isn't a statement of what it should or shouldn't be it just hasn't had enough attention yet to do it in 0.7; also, I think the problems we have with upgrade reliablity block it anyways)
585 2012-08-02 19:55:53 <MC-Eeepc> syncing to this usb stick isnt going well
586 2012-08-02 19:57:01 <sipa> sync to ram, and then copy it to your usb stick :)
587 2012-08-02 19:57:02 <MC-Eeepc> maybe this was a mistake
588 2012-08-02 19:57:29 <MC-Eeepc> dont have enough rams on this one
589 2012-08-02 19:58:10 <MC-Eeepc> i even took it back out of the truecrypt container to little effect
590 2012-08-02 19:58:15 <MC-Eeepc> anyway
591 2012-08-02 19:58:31 <MC-Eeepc> whats the lastest word of making the chain reasonable
592 2012-08-02 19:58:40 <gmaxwell> MC-Eeepc: writes on most usb sticks are VERY slow.
593 2012-08-02 19:58:46 <Gladamas> blockchain pruning
594 2012-08-02 19:58:52 <MC-Eeepc> im reading buzzwords like ultraprune and archive nodes and stuff
595 2012-08-02 19:59:04 <gmaxwell> MC-Eeepc: You asked this something like 24 hours ago.
596 2012-08-02 19:59:14 <sipa> yes, buzzword compliancy is our #1 priority
597 2012-08-02 19:59:23 <MC-Eeepc> did i?
598 2012-08-02 20:02:28 <gmaxwell> I thought so! wasn't it you that was providing the weekly "DHTs solve everything!" ?
599 2012-08-02 20:02:57 <MC-Eeepc> yeah that sounds like me
600 2012-08-02 20:03:17 <sipa> not *just* you...
601 2012-08-02 20:06:11 <grondilu> I get an error during link edition:  /usr/bin/ld: /usr/local/lib/libcrypto.a(dso_dlfcn.o): undefined reference to symbol 'dladdr@@GLIBC_2.0'
602 2012-08-02 20:08:45 <gmaxwell> grondilu: you've done something horrible and weird.
603 2012-08-02 20:08:55 <MC-Eeepc> what is the write pattern of a chain sync
604 2012-08-02 20:09:01 <MC-Eeepc> lots of tiny writes?
605 2012-08-02 20:10:57 <grondilu> I've just compiled and installed openssl on /usr/local
606 2012-08-02 20:11:11 <gmaxwell> grondilu: and you expected that to work? :)
607 2012-08-02 20:11:32 <grondilu> well... yes
608 2012-08-02 20:11:38 <gmaxwell> grondilu: I expect that something someplace needs -ldl but I don't know what where.
609 2012-08-02 20:12:21 <gmaxwell> Bitcoin complies fine against the fedora stock openssl packages modified to not exclude ecdsa, but openssl needs a bunch of options to get a sane build.
610 2012-08-02 20:15:10 <grondilu> ok so I should search a modified fedora stock openssl package
611 2012-08-02 20:15:31 <gmaxwell> grondilu: like the one I gave you the link to earlier... :)
612 2012-08-02 20:16:17 <grondilu> could you give it to me again please?
613 2012-08-02 20:17:20 <grondilu> yet I get a 'ALL TESTS SUCCESSFUL.
614 2012-08-02 20:17:35 <grondilu> ' in openssl.  So the build looks fine :/
615 2012-08-02 20:18:40 <gmaxwell> grondilu: the argument given in the fedora build are like five lines long. There is a bunch of stuff you need to set to get it to correctly build dynamic libraries and such.
616 2012-08-02 20:19:33 <MagicalTux> grondilu: add -ldl at the end of LIBS
617 2012-08-02 20:19:50 <MagicalTux> basically you're linking openssl statically, which means you need to also add openssl's dependencies
618 2012-08-02 20:20:06 <MagicalTux> (linking dynamically would also solve this)
619 2012-08-02 20:20:29 <grondilu> ahh unfortunately I have to go now.
620 2012-08-02 20:43:06 <galambo> /j #bitcoin
621 2012-08-02 22:04:14 <jgarzik> pynode's CHECKMULTISIG appears to be working... let's see what the next chain validation obstacle is
622 2012-08-02 22:57:08 <luke-jr> gmaxwell: see, it was a waste of my time to resubmit them. jgarzik just closed them without any logical basis
623 2012-08-02 22:58:06 <luke-jr> gmaxwell: because it affects the last 2 bits, and there's no reason to think more than the last octet would be redefined?
624 2012-08-02 22:59:19 <gmaxwell> luke-jr: hm I thought it changed the version from 0 to 1?
625 2012-08-02 22:59:25 <luke-jr> gmaxwell: 1=>2
626 2012-08-02 22:59:39 <gmaxwell> oh. well then.
627 2012-08-02 23:38:50 <jgarzik> luke-jr: AFAIK, gavin nak'd accepting non-standard transactions, even ones from yourself.  If you can find gavin saying otherwise, let's reopen, sure.
628 2012-08-02 23:39:15 <luke-jr> jgarzik: you are asking me to prove a negative?
629 2012-08-02 23:41:13 <luke-jr> Maybe you're thinking of "Accepting transactions you don't understand is a really bad idea, ESPECIALLY if they are to you.", which is the opposite scenario (a misunderstanding which no doubt resulted from my original pullreq having that inversed logic as a bug)
630 2012-08-02 23:41:38 <luke-jr> the intention (and now code) is only to accept ones *from* you, not *to*
631 2012-08-02 23:42:11 <luke-jr> so ones the client itself made ; NOT accepting these results in very weird bugs
632 2012-08-02 23:42:56 <luke-jr> (note: it's not actually possible in theory to trigger this code path except during development of new functionality before making it standard)