1 2012-06-25 00:42:45 <gribble> New news from bitcoinrss: fanquake opened pull request 39 on bitcoin/bitcoin.org <https://github.com/bitcoin/bitcoin.org/pull/39>
  2 2012-06-25 02:01:43 <copumpkin> is there some way to make bitcoin-qt work over an http proxy?
  3 2012-06-25 02:01:59 <copumpkin> probably not, eh
  4 2012-06-25 02:02:09 <copumpkin> I'll see if I can find a socks proxy
  5 2012-06-25 02:09:15 <nanotube> say, i have some socks i'm not using right here in my drawer. >_>
  6 2012-06-25 02:12:27 <copumpkin> :O
  7 2012-06-25 02:12:31 <nanotube> ouch, that key's gotta hurt. :)
  8 2012-06-25 02:12:35 <copumpkin> :(
  9 2012-06-25 02:12:42 <nanotube> ;;hug copumpkin
 10 2012-06-25 02:12:46 <copumpkin> yay
 11 2012-06-25 02:12:57 <nanotube> :)
 12 2012-06-25 02:19:09 <doublec> copumpkin: isn't there a -proxy command line option?
 13 2012-06-25 02:19:27 <copumpkin> isn't that for the socks proxy?
 14 2012-06-25 02:19:50 <doublec> oh right
 15 2012-06-25 02:30:43 <devrandom> ;;later tell BlueMatt yes, probably a simple change.  However, I have only a couple of day between trips, so may be a bit before I get to it.  Could you file an issue on the gitian github issue list?
 16 2012-06-25 02:30:43 <gribble> The operation succeeded.
 17 2012-06-25 02:49:47 <Karmaon> ssss
 18 2012-06-25 02:50:07 <gribble> pong
 19 2012-06-25 02:50:07 <Karmaon> !ping
 20 2012-06-25 02:59:52 <quintopia> is it possible to separate the p2p node (and blockchain storage) from the wallet (and user interface) programmatically in any available client or library?
 21 2012-06-25 03:04:30 <phantomcircuit> yes but not in any stable library
 22 2012-06-25 03:05:11 <quintopia> hmm
 23 2012-06-25 03:05:19 <quintopia> so fallback plan
 24 2012-06-25 03:05:53 <quintopia> is it possible to mix sshfs and the official client to achieve blockchain stored remotely?
 25 2012-06-25 03:06:22 <quintopia> obv a temporary solution until things stabilize
 26 2012-06-25 03:08:55 <weex> quintopia: i'd like to hear how that works out, i think you might see a ton of bandwidth usage but i'm not sure why
 27 2012-06-25 03:09:59 <quintopia> weex: which directory should go on the remote server to ensure the blockchain lands there?
 28 2012-06-25 03:10:08 <phantomcircuit> quintopia, why would you do that?
 29 2012-06-25 03:10:16 <phantomcircuit> local system doesn't have 2GB of space hdd space?
 30 2012-06-25 03:10:45 <quintopia> phantomcircuit: it has just enough to store the current tree and no more. and i've had to do a lot of file moving to achieve that.
 31 2012-06-25 03:10:45 <weex> quintopia: ~/.bitcoin by default
 32 2012-06-25 03:11:04 <weex> the wallet.dat file would move too though with that folder
 33 2012-06-25 03:11:05 <quintopia> weex: i don't want my private keys and stuff on the server. nothing that i can't afford to lose.
 34 2012-06-25 03:11:38 <luke-jr> quintopia: you don't want to do that
 35 2012-06-25 03:12:48 <weex> yeah i think you're screwed, don't see a way to separate wallet.dat from data dir
 36 2012-06-25 03:13:04 <luke-jr> also, the blockchain is accessed for pretty much every packet relayed
 37 2012-06-25 03:13:33 <weex> it is impressive the number of ways people want to seperate the blockchain from wallet though eh?
 38 2012-06-25 03:13:41 <luke-jr> weex: BlueMatt is working on that
 39 2012-06-25 03:13:46 <quintopia> well crap
 40 2012-06-25 03:15:09 <quintopia> weex: if so many people want to do it, then how many people are working on it? and who is making the most progress?
 41 2012-06-25 03:16:14 <weex> quintopia: BlueMatt is working on it...i think there's a thread on it somewhere
 42 2012-06-25 03:16:21 <luke-jr> you can help test it with https://bitcointalk.org/?topic=89099
 43 2012-06-25 03:17:10 <luke-jr> (note: JUST internals are being tested there, it's not exposed to the user)
 44 2012-06-25 03:23:22 <quintopia> looks scary
 45 2012-06-25 03:25:22 <weex> luke-jr: how does that list of pull requests compare in number to previous releases?
 46 2012-06-25 03:25:36 <luke-jr> weex: in what sense?
 47 2012-06-25 03:25:51 <weex> like do releases usually have more or less than that?
 48 2012-06-25 03:26:02 <luke-jr> like full 0.N releases?
 49 2012-06-25 03:26:09 <luke-jr> never really counted those; I'd imagine so
 50 2012-06-25 03:26:44 <weex> yeah or smaller ones...i guess i never really looked too closely at the changelog for any release
 51 2012-06-25 03:29:20 <luke-jr> 0.m.N releases are generally bugfixes only
 52 2012-06-25 03:30:24 <weex> cool, has any thought been given to the issue of keyloggers on windows machines?
 53 2012-06-25 03:30:33 <weex> like alternate password entry methods?
 54 2012-06-25 03:30:38 <luke-jr> yes, that's what P2SH is for
 55 2012-06-25 03:31:49 <luke-jr> so people with smartphones (for example) can require every transaction be approved on their desktop + phone
 56 2012-06-25 03:32:45 <weex> oh, i'm thinking of onscreen keyboards... guess that'd be a bit clunky compared to p2sh
 57 2012-06-25 03:33:09 <weex> i know some banking sites display a number pad you have to click on
 58 2012-06-25 03:33:22 <luke-jr> can keylog screen input just as easily IMO
 59 2012-06-25 03:33:42 <gmaxwell> yea, thats only a pretty weak protection against highly generic attacks.
 60 2012-06-25 03:33:50 <luke-jr> 0.4's wallet encryption was *mostly* just PR
 61 2012-06-25 03:33:54 <gmaxwell> It was made pointless by targeted malware.
 62 2012-06-25 03:34:38 <gmaxwell> Worse that kind of on screen picker discourages the use of long and complex passphrases of the sort that could actually withstand an offline attack, e.g. where the attacker only got a copy of your data after the fact.
 63 2012-06-25 03:35:06 <weex> fair enough...it'll be cool to see a mobile app that integrates well with the desktop client to do p2sh
 64 2012-06-25 03:37:19 <weex> i'm just writing some tips for windows users and it would seem if there's a remote chance a keylogger is installed you had better reinstall or use something else
 65 2012-06-25 03:37:57 <weex> i wouldn't trust it personally with more than the cost of a windows license <--- not a bad line
 66 2012-06-25 03:38:34 <luke-jr> s/a windows license/YOUR windows license/
 67 2012-06-25 03:39:15 <weex> is that a reference to OEM pricing?
 68 2012-06-25 03:39:24 <luke-jr> no, it's a reference to warez
 69 2012-06-25 03:39:25 <luke-jr> :p
 70 2012-06-25 03:39:37 <gmaxwell> I wouldn't be surprised if a lot of people were running pre-rooted windows: Download windows from some illicit copying site, get a rootkit bonus preinstalled for free.
 71 2012-06-25 03:40:03 <luke-jr> ^
 72 2012-06-25 03:40:05 <luke-jr> exactly
 73 2012-06-25 03:40:38 <weex> why does bitcoin have to ruin EVERYTHING!?
 74 2012-06-25 03:41:10 <weex> putting perverse financial incentives everywhere
 75 2012-06-25 03:41:44 <gmaxwell> Pretty sure that particular incentive exists absent bitcoin.
 76 2012-06-25 03:42:10 <luke-jr> gmaxwell: I think Bitcoin brings it to a whole new level
 77 2012-06-25 03:42:21 <luke-jr> or did anyhow
 78 2012-06-25 03:42:30 <luke-jr> ASICs are going to ruin that for botnets
 79 2012-06-25 03:42:48 <luke-jr> P2SH has potential to ruin it for theives
 80 2012-06-25 03:42:57 <weex> how long 'til you see, "no preloaded Bitcoin stealing keyloggers" in a .nfo file?
 81 2012-06-25 03:43:17 <luke-jr> .nfo?
 82 2012-06-25 03:43:48 <weex> it's been a while, and this is probably way ot but .nfo was the file that had all the ascii art and sort of advertised the warez release group
 83 2012-06-25 03:44:31 <luke-jr> o
 84 2012-06-25 03:44:37 <quintopia> i remember
 85 2012-06-25 11:20:32 <TD> sipa: you there?
 86 2012-06-25 11:23:12 <TD> morning, gavinandresen
 87 2012-06-25 11:23:41 <gavinandresen> good morning
 88 2012-06-25 11:23:59 <TD> gavinandresen: do you know offhand what happens if you replay blocks into a wallet, in bitcoin-qt?
 89 2012-06-25 11:24:17 <gavinandresen> no
 90 2012-06-25 11:24:39 <gavinandresen> By replay you mean "CheckBlock/AcceptBlock" as if they were received from the network?
 91 2012-06-25 11:24:46 <TD> yeah, ProcessBlock
 92 2012-06-25 11:24:50 <TD> i'm working on some leveldb migration code
 93 2012-06-25 11:25:05 <TD> i think nothing will happen because replay takes place before pwalletMain is initialized
 94 2012-06-25 11:25:34 <gavinandresen> you're probably right, I'd have to play computer and mentally step through the code
 95 2012-06-25 11:25:36 <sipa> what do you mean by replay? -loadblock ?
 96 2012-06-25 11:26:02 <TD> yeah
 97 2012-06-25 11:26:06 <TD> effectively
 98 2012-06-25 11:26:10 <sipa> that happens before the wallet is loaded, though it may be useful to swap it
 99 2012-06-25 11:26:37 <sipa> if done while the wallet is loaded, i expect the effect to be the same as a -rescan
100 2012-06-25 11:26:54 <TD> a -rescan tells the wallet to rescan a block directly
101 2012-06-25 11:26:59 <TD> a replay calls ProcessBlock on each one
102 2012-06-25 11:27:15 <TD> btw -loadblock happens after wallet is loaded and registered. i'm putting my migration code before
103 2012-06-25 11:27:21 <sipa> yes, but new blocks will cause SyncWithWallet callbakcs as well
104 2012-06-25 11:27:41 <gavinandresen> sipa: I want to tag v0.6.3 (same commit as v0.6.3rc1) and re-gitian build so the v0.6.3 binaries don't report themselves as "rc1"....
105 2012-06-25 11:29:00 <sipa> TD: i'm making some progress on the pruning idea, and refactoring the reorg code somewhat (i believe it will end up being simpler), and it'll only do db writes after verification and calculating all changes
106 2012-06-25 11:29:28 <sipa> TD: so combined with leveld, you wouldn't need the batch inspection
107 2012-06-25 11:29:59 <TD> ok
108 2012-06-25 11:30:03 <TD> that would be good
109 2012-06-25 11:30:06 <TD> that said, i've written the code now
110 2012-06-25 11:30:11 <sipa> gavinandresen: i can rebuild, but my linux binaries differed significatly from all others
111 2012-06-25 11:30:16 <TD> so we can always go in and delete it later once your refactorings are done, if my work gets merged
112 2012-06-25 11:30:33 <TD> i found out today that leveldb needs a bit of massaging to work on windows. people have done ports but they aren't properly integrated upstream.
113 2012-06-25 11:30:43 <sipa> oh
114 2012-06-25 11:30:44 <TD> so it may be a bit longer until i have it working and tested on all platforms
115 2012-06-25 11:30:49 <TD> for now i've made it a conditional compile
116 2012-06-25 11:31:06 <TD> making it work on windows looks easy - somebody sent me an env_boost.cpp that makes it use the boost file IO and thread code
117 2012-06-25 11:31:42 <sipa> right, i read it has a nice OS interaction abstraction layer
118 2012-06-25 11:31:53 <sipa> so that shouldn't be too hard
119 2012-06-25 11:32:04 <TD> oh yes. the "port" is quite easy. my belief it'd be done already because of chrome overlooked the fact that chrome has its own OS abstraction layer
120 2012-06-25 11:32:08 <TD> so they "ported" it to use that
121 2012-06-25 11:32:37 <TD> which pruning idea are you working on?
122 2012-06-25 11:33:18 <sipa> storing unspent txouts directly in database, but keep full blocks and "undo" information for recent blocks only
123 2012-06-25 11:33:39 <TD> hrm
124 2012-06-25 11:33:48 <TD> so you can't serve the chain that way?
125 2012-06-25 11:33:57 <sipa> only the part you keep
126 2012-06-25 11:34:04 <sipa> which is optionally everything
127 2012-06-25 11:34:40 <sipa> even when keeping everything, i expect it to be faster and smaller than what we have now
128 2012-06-25 11:35:48 <TD> would you be able to serve arbitrary stored transactions that way?
129 2012-06-25 11:36:01 <sipa> with an additional index
130 2012-06-25 11:36:30 <sipa> (which would still be smaller than what we have now, and that index becomes optional too)
131 2012-06-25 11:36:51 <sipa> note that bitcoind currently does not serve arbitrary transactions either
132 2012-06-25 11:36:53 <gavinandresen> sipa: RE: pruning: did you see my post about the new gettransaction features and concerns with pruning ?  https://bitcointalk.org/index.php?topic=89725.0
133 2012-06-25 11:36:54 <TD> i know
134 2012-06-25 11:36:57 <TD> but that's an easy fix
135 2012-06-25 11:37:03 <TD> and there are lots of use cases for it
136 2012-06-25 11:37:27 <sipa> gavinandresen: yes
137 2012-06-25 11:37:50 <sipa> gavinandresen: in my current code gettransaction simply doesn't work for non-mempool transactions
138 2012-06-25 11:38:36 <gavinandresen> sipa: ah....  there's still a wallet transaction that stores extra information, I assume?
139 2012-06-25 11:38:52 <sipa> i haven't touched wallet code at all
140 2012-06-25 11:39:21 <TD> it would be very useful for a variety of apps if getdata worked for any transaction and loaded from disk if necessary
141 2012-06-25 11:39:29 <TD> hrm. annoying.
142 2012-06-25 11:39:32 <sipa> i actually mean thet main's GetTrabsaction only works for mempools
143 2012-06-25 11:39:41 <TD> sipa: seems that -loadblock *requires* that the block file be != blk0001.dat
144 2012-06-25 11:39:48 <TD> sipa: which would double disk space usage during migration ....
145 2012-06-25 11:39:58 <sipa> TD: yes, definitely
146 2012-06-25 11:40:10 <sipa> TD: it is for importing, not reindexing
147 2012-06-25 11:40:26 <TD> i guess it's not necessarily a killer problem, but i wonder how many people would care about a temporary spike in usage
148 2012-06-25 11:40:48 <sipa> i believe a reindexing function isn't too hard to write
149 2012-06-25 11:40:50 <gavinandresen> eleven.  And they'll all post the forums and complain loudly.
150 2012-06-25 11:41:17 <sipa> the grttransaction RPC first looks in the wallet anyway
151 2012-06-25 11:41:59 <gavinandresen> sipa: yes, but I worry that if the 0.7 release gives all sorts of spiffy extra information that we'll regret it when we have to take it away in 0.8
152 2012-06-25 11:42:20 <sipa> i was not a fan of all decompositions either
153 2012-06-25 11:42:30 <sipa> but it sure is interesting :)
154 2012-06-25 11:44:04 <TD> sipa: i assume when you say the unspent outs fit in memory, you mean with some kind of cache over the db
155 2012-06-25 11:44:11 <TD> because it won't fit in memory forever (hopefully)
156 2012-06-25 11:44:18 <sipa> by the way: my expectation for the pruned storage requirements: around 150 MB + 1.1*the_size_of_blocks_you_keep
157 2012-06-25 11:44:37 <sipa> TD: of course, that's a short term prognosis
158 2012-06-25 11:46:12 <TD> ok
159 2012-06-25 11:46:54 <sipa> with txid-to-diskpos index it'll be another 50-100 MB
160 2012-06-25 11:51:23 <TD> sipa: ok. leveldb has in-memory caching integrated
161 2012-06-25 11:51:26 <TD> i'm sure the same is true of bdb
162 2012-06-25 11:51:31 <TD> well, i'd imagine it is
163 2012-06-25 11:51:36 <sipa> yes, sure
164 2012-06-25 11:51:52 <sipa> we even have -dbcache to tune it
165 2012-06-25 11:52:03 <sipa> though the default works quite well now
166 2012-06-25 11:52:17 <TD> what size is it by default?
167 2012-06-25 11:52:31 <sipa> 25 MiB iirc
168 2012-06-25 11:53:18 <sipa> i believe benchmarking showed that it hardly improved when set to more, unless you set it to much much more
169 2012-06-25 11:54:00 <sipa> gavinandresen: in pruned mode it is possible to lookup the input amounts/addresses relatively cheaply by the way, as they are stored in blocks' undo files
170 2012-06-25 11:54:16 <sipa> only for the blocks you keep, obviously
171 2012-06-25 11:54:41 <gavinandresen> sipa: amount/address/scriptPubKey ?
172 2012-06-25 11:54:55 <sipa> amount + scriptPubKey are stored
173 2012-06-25 11:54:57 <gavinandresen> never mind,
174 2012-06-25 11:55:08 <gavinandresen> (obviously gotta be the script not the address....)
175 2012-06-25 11:55:13 <sipa> indeed
176 2012-06-25 11:55:39 <gavinandresen> 0.6.3 win gitian sigs pushed, by the way... building linux now
177 2012-06-25 11:57:06 <TD> sipa: how is this scheme different to the simpler one satoshi proposed again? i mean just deleting fully spent transactions.
178 2012-06-25 11:57:23 <TD> it's more efficient?
179 2012-06-25 11:58:05 <sipa> yes
180 2012-06-25 11:58:14 <sipa> txins are the bulk of storage, not txouts
181 2012-06-25 11:58:25 <sipa> here we only keep the txouts
182 2012-06-25 11:58:57 <cande> is the "bitcoind" executeable included in the Windows.exe download from bitcoin.org
183 2012-06-25 11:59:04 <sipa> and there are many partially-spent txn, which would have to be kept entirely
184 2012-06-25 11:59:13 <sipa> cande: yes, in the daemon subdir
185 2012-06-25 11:59:21 <cande> thx
186 2012-06-25 12:01:26 <sipa> TD: this requires 33 bytes per unspent tx, plus 23 bytes per unspent txout (+ overhead per tx)
187 2012-06-25 12:01:36 <sipa> on average
188 2012-06-25 12:02:43 <TD> ok
189 2012-06-25 12:03:10 <sipa> TD: satoshi's pruning supports serving transactions and merkle paths, without having the blocks
190 2012-06-25 12:03:26 <sipa> but it can't serve full blocks either
191 2012-06-25 12:03:37 <TD> yeah
192 2012-06-25 12:03:48 <TD> i'm just trying to think through cases where full transactions might be required in future
193 2012-06-25 12:04:20 <sipa> anywhere they are necessary, they are kept in a wallet of either sender or receiver anyway
194 2012-06-25 12:08:08 <gmaxwell> Did anyone try to reproduce the claims that proxy wasn't working in 0.6.3rc1?
195 2012-06-25 12:08:15 <TD> thinking back to the SPV case where you want to know if a block is invalid, and are willing to do a bunch of work to spot that if it's rare
196 2012-06-25 12:08:26 <TD> you wouldn't be able to check in the presence of pruning
197 2012-06-25 12:08:53 <TD> somebody would say "block N gives itself more money than the inflation formulas allow", but you could not check for yourself without a copy of all previous transactions
198 2012-06-25 12:08:59 <gmaxwell> TD: the invalidity proof would include all the required info, and you'd expect nodes to create them _pre pruning_.
199 2012-06-25 12:09:28 <TD> the proof would require all txns in the block and all input txns too (full copies)
200 2012-06-25 12:09:36 <gmaxwell> TD: though without a tree of unspent txn it might be a lot of data. :(
201 2012-06-25 12:09:41 <gmaxwell> yea.
202 2012-06-25 12:10:00 <TD> i guess some nodes can advertise in their addrs that they are archival nodes
203 2012-06-25 12:10:01 <gmaxwell> With a tree of unspent txn committed in the prior block you wouldn't need their inputs, just all the txn.
204 2012-06-25 12:10:02 <TD> which don't prune
205 2012-06-25 12:10:49 <gmaxwell> yea, thats not elegant, you really want the proof to contain all the required information so a stateless node could check without further help. The reason for this is that you can't assume that such an archival node would be available to you you're already assuming a well funded attack.
206 2012-06-25 12:14:15 <TD> the node that is trying to construct the proof also has to be able to read all the previous transactions
207 2012-06-25 12:14:23 <TD> i'm not saying pruning is bad, it's clearly not
208 2012-06-25 12:14:35 <TD> i'm just wondering if we lose anything by doing it this way vs the older way
209 2012-06-25 12:16:59 <sipa> well, you could chose to store either the list of txids per block, or the merkle path for each unspent tx to the block merkle root
210 2012-06-25 12:17:32 <sipa> oh no, as you still can't serve the transaction itself
211 2012-06-25 12:19:08 <TD> the purpose of this is to save disk space?
212 2012-06-25 12:19:56 <sipa> i think the core improvement is that it reduces the working set size necessary to do full validaton
213 2012-06-25 12:21:15 <gmaxwell> 07:14 < TD> the node that is trying to construct the proof also has to be able to read all the previous transactions
214 2012-06-25 12:21:16 <sipa> additionally, it allows trading in disk space of old blocks, in exchange for removing the ability to serve/rescan/reorg
215 2012-06-25 12:21:26 <gmaxwell> ^ yes, I don't think thats a problem because you can't validate without them.
216 2012-06-25 12:23:04 <sipa> maybe this will lead to the necessity of advertizing not the latest but also the earliest block a node is willing to serve
217 2012-06-25 12:25:58 <TD> sipa: it'd have to be something in the addr packets to let people find archival nodes if they want them
218 2012-06-25 12:26:07 <TD> sipa: so a new service flag might be better, albiet less general
219 2012-06-25 12:27:48 <sipa> maybe a bit for "serves last 100 blocks", one for 1000, 10000, 100000, all
220 2012-06-25 12:28:19 <sipa> or 256, 4096, 65536, all
221 2012-06-25 12:29:12 <TD> are there any cases in which you'd be willing to serve huge chunks of the chain but not all of it?
222 2012-06-25 12:29:15 <TD> it's hard to imagine
223 2012-06-25 12:29:20 <TD> might be simpler to just use one bit
224 2012-06-25 12:29:44 <sipa> i'm sure almost any fully validating node will keep the last 100 blocks around
225 2012-06-25 12:30:10 <sipa> or 1000
226 2012-06-25 12:30:13 <TD> that's less than one days worth
227 2012-06-25 12:30:22 <TD> re-orgs caused by rule changes or attacks could be much larger than that
228 2012-06-25 12:30:24 <TD> disk space is really cheap
229 2012-06-25 12:30:33 <TD> so i'd be generous
230 2012-06-25 12:33:04 <TD> hmm
231 2012-06-25 12:33:15 <TD> where is the code that disables sig checking until the last checkpoint?
232 2012-06-25 12:33:44 <TD> ah
233 2012-06-25 12:33:51 <TD> we don't run the scripts at all
234 2012-06-25 12:34:41 <gmaxwell> A reorg which was greater than a days worth would be horiffic, and would almost certantly result in lots of people getting ripped off... Though I generally agree that keeping more like 1000 would be a lot better.
235 2012-06-25 12:34:49 <TD> well
236 2012-06-25 12:34:59 <TD> if there was a huge re-org that only changed one or two transactions, not necessarily
237 2012-06-25 12:36:56 <gmaxwell> It would be quite hard (and currently impossible) to cooridnate that the reorg would only do that though.
238 2012-06-25 12:37:22 <TD> consider the overflow transaction. huge re-org that affected basically nobody as the fake coins had not yet been spent
239 2012-06-25 12:38:34 <gmaxwell> Right, but almost no txn volume at the time. Thats not a good assumption in the future. In any case, I agree that 1000 would be much better because it would make a fix like that possible.
240 2012-06-25 12:42:13 <sipa> i suppose the standard advice could be keep 2016 blocks
241 2012-06-25 12:43:10 <sipa> or maybe keep X GB worth of blocks, but never less than Y
242 2012-06-25 12:43:20 <TD> keep until the last checkpoint?
243 2012-06-25 12:43:27 <gmaxwell> Sounds fine to me. This is something that could evolve in the (far) future when the system is better understood.
244 2012-06-25 12:43:32 <gmaxwell> TD: Ugh. yuck.
245 2012-06-25 12:45:33 <TD> sipa: why does LoadExternalBlockFile read the pchMessageStart etc by hand instead of just deserializing the markers and sizes to a CDataStream?
246 2012-06-25 12:45:39 <TD> is this old satoshi code or did you write it?
247 2012-06-25 12:46:09 <sipa> TD: i did; reason: dealing with corrupted files
248 2012-06-25 12:47:31 <TD> ok
249 2012-06-25 12:48:21 <sipa> hmm, TD's comment about scripts being disabled entirely... i suppose there is not really any other way
250 2012-06-25 12:48:33 <sipa> what if some idiot script did a OP_CHECKSIG OP_NOT ?
251 2012-06-25 12:50:28 <TD> well, the point is you assume the script outputs were already verified
252 2012-06-25 12:50:33 <TD> so i think it's safe
253 2012-06-25 12:50:40 <TD> no matter what a script contains
254 2012-06-25 12:50:54 <sipa> yes, indeed
255 2012-06-25 12:51:15 <sipa> there's no security difference between not checking the signature vs not checking the script at all, afaics
256 2012-06-25 12:52:04 <sipa> if someone can get you to accept a faulty script, they can as well get you a valid script with a faulty sig
257 2012-06-25 12:52:08 <gmaxwell> right.
258 2012-06-25 12:52:29 <Diapolo> sipa: can you take a look at http://i46.tinypic.com/8wmhye.jpg I'm currently working on the new proxy UI and need some quick input
259 2012-06-25 12:53:04 <Diapolo> sipa: 1. When using the IPv6 proxy, the first entry needs to be IPv4? 2. I add another field similar to the IPv6 one for Tor.
260 2012-06-25 12:54:08 <gmaxwell> Diapolo: dunno if you noticed but hidden service support is now in git-master.
261 2012-06-25 12:54:20 <Diapolo> gmaxwell: I know
262 2012-06-25 12:54:54 <Diapolo> and it needs to be exposed to the GUI, I had a talk about this with sipa 2 days ago
263 2012-06-25 12:55:13 <sipa> Diapolo: i'd say "[ ] Use a SOCKS proxy: [host] [port] [version]", "[ ] Use a separate proxy for IPv6 connections:\n(X) None (X) [host] [port]", "[ ] Use a separate proxy for hidden services:\n(X) None ( ) [host] [port]"
264 2012-06-25 12:55:40 <sipa> if my pseudo-gui code is readable
265 2012-06-25 12:56:07 <Diapolo> sipa: That's what I have in my mind ... just the question, when using IPv6 and the first entry is active this needs to be an IPv4 one?
266 2012-06-25 12:56:08 <sipa> "separate IPv6 SOCKS proxy" sounds like the proxy itself is IPv6 - that's not true
267 2012-06-25 12:56:23 <Diapolo> you are right wrong wording there
268 2012-06-25 12:56:25 <sipa> all host ip addresses can be anything
269 2012-06-25 12:56:35 <Diapolo> and a missunderstanding on my side ^^
270 2012-06-25 12:56:40 <gmaxwell> "separate SOCKS proxy for IPv6 peers"
271 2012-06-25 12:57:18 <sipa> gmaxwell++
272 2012-06-25 12:58:04 <sipa> Diapolo: if one would create a loop (IPv6 proxy is set to onion address, and Tor proxy is set to an IPv6 address, e.g.) there'd be a problem, but don't worry about that now
273 2012-06-25 12:58:43 <gmaxwell> Does the GUI have an interface to addnode?
274 2012-06-25 12:58:48 <sipa> not yet
275 2012-06-25 12:59:29 <Diapolo> I guess the GUI could be far more powerfull and currently I want to add the proxy / Tor stuff.
276 2012-06-25 12:59:56 <sipa> oh wait, that looping is not a problem; connections to proxy servers are always made directly
277 2012-06-25 13:00:07 <sipa> so you cannot use a .onion address as proxy server
278 2012-06-25 13:00:43 <gmaxwell> sipa: hah, I was busily reading code trying to figure out what the #@$# you were talking about.
279 2012-06-25 13:01:24 <Diapolo> sipa: I'm gonna come back, when I have the UI ready and need some help with the network-functions :).
280 2012-06-25 13:02:36 <sipa> Diapolo: feel free
281 2012-06-25 13:06:13 <gavinandresen> sipa BlueMatt luke-jr and anybody else gitian-capable:  please build v0.6.3 and let me know if the builds match.  Should be all ready to announce, assuming we do get matches.
282 2012-06-25 13:08:14 <sipa> gavinandresen: building
283 2012-06-25 13:09:25 <luke-jr> gavinandresen: ^
284 2012-06-25 13:09:39 <gavinandresen> thanks
285 2012-06-25 13:10:47 <luke-jr> sipa BlueMatt et al: if you could build 0.4.7rc3 and 0.5.6rc3 afterward, I'd appreciate it
286 2012-06-25 13:34:12 <luke-jr> gavinandresen: match
287 2012-06-25 13:34:27 <gavinandresen> luke-jr: excellent, thanks
288 2012-06-25 13:38:29 <luke-jr> gavinandresen: any point in me building OSX bitcoind, or shall I just pass until/if I can do it with Bitcoin-Qt?
289 2012-06-25 13:38:43 <gavinandresen> just pass until you can build QT
290 2012-06-25 13:39:37 <gavinandresen> (well, there might be a handful of people who would appreciate a pre-built osx bitcoind, but it is a small enough number I think you should do something better with your time)
291 2012-06-25 13:40:21 <gmaxwell> luke-jr: might want to ask in the p2pool channel.
292 2012-06-25 13:40:46 <luke-jr> gavinandresen: nice thing about gitian is, it doesn't use my time ;p
293 2012-06-25 13:42:06 <gavinandresen> 0.6.3 binaries now up at https://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.6.3/    I'm away for a little bit, will post to forums / etc when I'm back
294 2012-06-25 13:45:44 <gmaxwell> hm. I thought I had gavin's gpg key these are signed with 1FC730C1
295 2012-06-25 13:46:18 <gavinandresen> gmaxwell:  http://pgp.mit.edu:11371/pks/lookup?search=gavinandresen&op=index
296 2012-06-25 13:46:45 <gavinandresen> 1FC730C1 is the right one
297 2012-06-25 13:47:00 <gavinandresen> (both belong to me, and sign each other)
298 2012-06-25 13:47:04 <gmaxwell> Yea, I see its signed by your other one.
299 2012-06-25 13:52:13 <jgarzik> gavinandresen: ACK on the gettransaction RPC change you proposed on the forum
300 2012-06-25 13:53:52 <luke-jr> are we using forum instead of ML now?
301 2012-06-25 13:54:22 <luke-jr> gavinandresen: NACK on removal of much-needed functionality
302 2012-06-25 14:01:59 <jgarzik> luke-jr: NACK on air filled, content-free expressions of feeling
303 2012-06-25 14:03:03 <luke-jr> jgarzik: expression of someone-who-actually-uses-bitcoind-in-the-real-world
304 2012-06-25 14:03:16 <luke-jr> good thing those present when gavinandresen brought it up on IRC originally seem to agree
305 2012-06-25 14:03:38 <luke-jr> didn't know he was going to the forum to ignore the general consensus we had
306 2012-06-25 14:04:43 <jgarzik> I thought I just NACK'd this
307 2012-06-25 14:05:43 <jgarzik> heh, Butterfly Labs has 50,000 bitcoins they need to sell, now that BFL opened up orders: https://bitcointalk.org/index.php?topic=89734.0
308 2012-06-25 14:05:43 <luke-jr> jgarzik: you can't NACK the real world/reality
309 2012-06-25 14:06:04 <jgarzik> luke-jr: yes, sadly, you will continue to post hot air devoid of technical details
310 2012-06-25 14:06:14 <sipa> luke-jr: afaik all who agreed on the decompositions code, preferred a separate call to decompose (except you)
311 2012-06-25 14:06:50 <luke-jr> sipa: didn't seem that way. why would anyone prefer that?
312 2012-06-25 14:07:06 <sipa> it's saner and more flexible
313 2012-06-25 14:07:07 <TD> lol
314 2012-06-25 14:07:15 <TD> i wonder which IP/pool is BFL. eclipse?
315 2012-06-25 14:07:20 <luke-jr> I'd think the getblockhash/getblock combination would have clearly demonstrated what a nusance such behaviour is
316 2012-06-25 14:08:00 <luke-jr> sipa: how so?
317 2012-06-25 14:08:06 <luke-jr> sipa: all it does is reduce the functionality
318 2012-06-25 14:08:20 <jgarzik> TD: ?
319 2012-06-25 14:08:29 <sipa> you need more calls to do the same, yes
320 2012-06-25 14:08:30 <TD> the 50k coins
321 2012-06-25 14:08:46 <sipa> but it doesn't reduce functionality
322 2012-06-25 14:08:47 <jgarzik> TD: Bit-pay is the one sitting on 50k coins.  They're a payment processor, handling payments for bitcoin ASICs.
323 2012-06-25 14:08:54 <jgarzik> not a pool
324 2012-06-25 14:09:11 <TD> yeah, i mean, are the blocks the ASICs mined visible in blockchain.info or somesuch
325 2012-06-25 14:09:16 <TD> iirc their screenshots show them all mining against eclipse
326 2012-06-25 14:09:29 <jgarzik> TD: ASICs are hot air right now
327 2012-06-25 14:09:34 <jgarzik> TD: BFL is only taking pre-orders
328 2012-06-25 14:10:35 <luke-jr> sipa: it reduces usability too, at least
329 2012-06-25 14:14:27 <luke-jr> jgarzik: btw, do you intend to update bitcoinrpc with this "batch" thing?
330 2012-06-25 14:14:32 <TD> right
331 2012-06-25 14:14:46 <TD> leveldb with integrated source, auto migration with progress ui -> done
332 2012-06-25 14:14:55 <TD> next step, testing on windows
333 2012-06-25 14:15:08 <BlueMatt> jgarzik: hot air...artforz was running them a year+ ago?
334 2012-06-25 14:15:12 <jgarzik> luke-jr: I am willing to...  suggestions for an interface (python API) are welcome
335 2012-06-25 14:15:21 <TD> i think he never completed that project? or they were structured asics
336 2012-06-25 14:15:23 <TD> not quite the same thing
337 2012-06-25 14:15:30 <jgarzik> BlueMatt: artforz was doing structured ASIC AFAIK
338 2012-06-25 14:15:36 <BlueMatt> ah, thought he got further, nvm
339 2012-06-25 14:15:43 <jgarzik> dunno, maybe he did
340 2012-06-25 14:16:04 <BlueMatt> na, Im probably dreaming
341 2012-06-25 14:18:10 <drizztbsd> is the last official version 0.6.3?
342 2012-06-25 14:20:13 <jgarzik> drizztbsd: yes
343 2012-06-25 14:20:24 <drizztbsd> so is the topic wrong? :P
344 2012-06-25 14:23:11 <epscy> where is artforz now
345 2012-06-25 14:23:17 <epscy> working for bfl?
346 2012-06-25 14:23:28 <epscy> that would be funny
347 2012-06-25 14:23:40 <epscy> that guy is the puppet master
348 2012-06-25 14:25:20 <sipa> gavinandresen, jgarzik, gmaxwell: i think we may want to reconsider more about the gettransactions and related calls
349 2012-06-25 14:25:49 <sipa> as the call currently mixes wallet info, blockchain info and spentness info
350 2012-06-25 14:26:37 <sipa> for efficiency, when async rpc is merged, we'll certainly want calls that do not lock the wallet if not necessary
351 2012-06-25 14:27:08 <luke-jr> async RPC is already merged
352 2012-06-25 14:27:28 <luke-jr> well, threaded anyhow, which is the relevant bit afaik
353 2012-06-25 14:27:33 <sipa> right, i mean the ability for concurrent calls
354 2012-06-25 14:28:06 <sipa> and some functionality of gettransactions doesn't need any locks
355 2012-06-25 14:28:13 <sipa> some needs a wallet lock
356 2012-06-25 14:28:21 <sipa> some needs blockchain info
357 2012-06-25 14:29:21 <sipa> somehow it'd feel cleaner to me to keep these separated
358 2012-06-25 14:31:36 <sipa> i know i'm changing my mind, as i implemented the querying of the chain in gettransaction myself...
359 2012-06-25 14:38:36 <drizztbsd> luke-jr: v0.6.3.0-g6e0c5e3-beta
360 2012-06-25 14:38:43 <drizztbsd> 0.6.3 wrongly report -beta
361 2012-06-25 14:39:16 <drizztbsd> is it correct?
362 2012-06-25 14:39:34 <luke-jr> drizztbsd: all Bitcoin-Qt is beta
363 2012-06-25 14:39:37 <drizztbsd> oh
364 2012-06-25 14:39:40 <drizztbsd> ok
365 2012-06-25 14:42:47 <drizztbsd> now archlinux have 0.6.3 version :P
366 2012-06-25 14:45:31 <kinlo> who's the gentoo maintainer?
367 2012-06-25 14:45:41 <luke-jr> <--
368 2012-06-25 14:45:45 <kinlo> if archlinux already has 0.6.3... how comes gentoo is still stuck at 0.5 ? :)
369 2012-06-25 14:45:56 <luke-jr> kinlo: it isn't
370 2012-06-25 14:46:10 <kinlo> or perhaps I just am doing it wrong
371 2012-06-25 14:46:12 <luke-jr> 0.6.2 is in the tree
372 2012-06-25 14:46:26 <luke-jr> 0.6.3 is testing in the overlay
373 2012-06-25 14:46:30 <guruvan> but it's masked
374 2012-06-25 14:46:38 <luke-jr> guruvan: no, it's testing keyworded
375 2012-06-25 14:47:00 <kinlo> I'll have a look then
376 2012-06-25 14:47:05 <luke-jr> 0.5.6_rc2 is stable-keyworded
377 2012-06-25 14:47:06 <simple> the version strings on both windows and linux bitcoind are 'Bitcoin version v0.6.3-beta' still
378 2012-06-25 14:47:14 <guruvan> oh ok - I didn't really pay much attention to the messages - I just used the --autonumask-write and went with the latest
379 2012-06-25 14:47:18 <luke-jr> 0.6.3 probably will be promoted to stable in about a month (Gentoo policy on stabilization)
380 2012-06-25 14:47:31 <sipa> simple: is that unexpected?
381 2012-06-25 14:47:39 <simple> yes, since the non-beta was just release
382 2012-06-25 14:47:47 <kinlo> simple: everything is beta
383 2012-06-25 14:47:51 <sipa> every bitcoin release to date has been beta
384 2012-06-25 14:47:55 <simple> oh
385 2012-06-25 14:47:56 <kinlo> simple: there is a difference in "rc" or not
386 2012-06-25 14:47:58 <simple> very good point
387 2012-06-25 14:48:06 <simple> i guess it doesn't say rc1 anymore...
388 2012-06-25 14:48:11 <drizztbsd> well you can request stabilization after a month, but every team have theirs timings :P
389 2012-06-25 14:48:12 <simple> never mind, non-issue lol
390 2012-06-25 14:48:21 <luke-jr> drizztbsd: yeah, true
391 2012-06-25 14:48:36 <kinlo> so I need to upgrade again
392 2012-06-25 14:48:44 <drizztbsd> I was a member of amd64 and powerpc herds
393 2012-06-25 14:48:46 <drizztbsd> lol
394 2012-06-25 14:48:50 <luke-jr> kinlo: only if you want 0.6 features
395 2012-06-25 14:48:55 <sipa> kinlo: when i used gentoo i updated at least once a week :)
396 2012-06-25 14:49:01 <luke-jr> drizztbsd: I was a developer before herds! :p
397 2012-06-25 14:49:09 <drizztbsd> OLD!
398 2012-06-25 14:49:10 <drizztbsd> :D
399 2012-06-25 14:49:14 <kinlo> I'm not using gentoo on servers :)
400 2012-06-25 14:49:23 <kinlo> the gentoo thingy is just something to play with
401 2012-06-25 14:49:24 <sipa> what are herds?
402 2012-06-25 14:49:28 <drizztbsd> "groups"
403 2012-06-25 14:49:39 <kinlo> but I do like to be current with the production bitcoin clients
404 2012-06-25 14:49:39 <sipa> i used gentoo in 2005-2007 or so
405 2012-06-25 14:49:44 <luke-jr> sipa: groups of people with responsibilities
406 2012-06-25 14:49:47 <sipa> ok
407 2012-06-25 14:50:02 <luke-jr> sipa: so the amd64 herd is responsible for makign anything stable on amd64
408 2012-06-25 14:50:39 <sipa> right
409 2012-06-25 14:52:25 <kinlo> I think 0.6.3 is important for pools, coz the propagation is faster, no?
410 2012-06-25 14:52:33 <luke-jr> kinlo: not much
411 2012-06-25 14:53:03 <luke-jr> I guess we should do a group test of that block preview patch or it'll just sit on the back-burner :/
412 2012-06-25 14:53:20 <kinlo> luke-jr: the hub thing?
413 2012-06-25 14:53:29 <luke-jr> kinlo: no, the relaying blocks before validating their txns
414 2012-06-25 14:53:59 <sipa> is there a pullreq for that?
415 2012-06-25 14:54:15 <luke-jr> sipa: no; I figure it should be tested at least a little first
416 2012-06-25 14:54:31 <sipa> i'll test it later today
417 2012-06-25 14:54:39 <luke-jr> my 'fastblockrelay' branch
418 2012-06-25 14:54:56 <luke-jr> anyone else want to join in the testing?
419 2012-06-25 14:59:18 <gmaxwell> 09:25 < sipa> gavinandresen, jgarzik, gmaxwell: i think we may want to reconsider more about the gettransactions and related calls
420 2012-06-25 14:59:32 <gmaxwell> Unfortunately some really important information is not cheap to retrieve in our current setup.
421 2012-06-25 14:59:46 <gmaxwell> E.g. what are the fees on a transaction?  Oops, go fetch the inputs!
422 2012-06-25 15:00:07 <gmaxwell> and a gettransaction that gives you transaction data but no fees? bleh.
423 2012-06-25 15:00:11 <gmaxwell> Thats confusing.
424 2012-06-25 15:01:06 <sipa> if the pruning stuff gets finished and merged, it will be cheap
425 2012-06-25 15:01:59 <sipa> querying the block file will give you the raw transaction data, and input amounts/scripts
426 2012-06-25 15:02:17 <sipa> querying the txout set will give you whether the ouputs are available
427 2012-06-25 15:02:36 <sipa> querying the wallet will give you whatever the sender or receiver cares about
428 2012-06-25 15:02:56 <gmaxwell> ::nods::
429 2012-06-25 15:05:03 <gmaxwell> luke-jr: it's a lot faster when you're not dying on IO already, I don't know why your nodes are so crazy slow.
430 2012-06-25 15:05:12 <gmaxwell> oh. hm. what bdb version are you using?
431 2012-06-25 15:05:37 <gmaxwell> Didn't jgarzik discover that the newever version basically mooted our recent improvements?
432 2012-06-25 15:06:01 <sipa> gavinandresen, luke-jr: my windows build matches; linux again doesn't
433 2012-06-25 15:06:12 <sipa> i wonder if my base image is damaged maybe
434 2012-06-25 15:06:56 <luke-jr> gmaxwell: 4.8
435 2012-06-25 15:07:06 <gmaxwell> hm. k.
436 2012-06-25 15:13:01 <gavinandresen> back.  RE: gettransaction:  I agree that mixing the wallet and non-wallet information is at the very least weird.  Perhaps the get-chain-info should be a new 'getanytransaction' RPC call
437 2012-06-25 15:13:26 <gavinandresen> It is also possible some service will break if they were relying on gettransaction returning nothing for non-wallet transaction
438 2012-06-25 15:14:03 <gmaxwell> It's still goofed up if get$footransaction doesn't tell you the fees. The point on gettransaction there is ... alarming. Yes, that could be bad.
439 2012-06-25 15:14:31 <gmaxwell> Change a url and now all the coins are yours!
440 2012-06-25 15:15:04 <luke-jr> O.o
441 2012-06-25 15:24:23 <jgarzik> luke-jr: FWIW I honestly have no clue what a batch bitcoinrpc API might look like.  if you or someone else can create python client code that imagines using a batch API, I can create the API side for it
442 2012-06-25 15:24:59 <jgarzik> the entire bitcoinrpc API assumes method(args) initiates a json-rpc and returns only after rpc completion
443 2012-06-25 15:25:11 <jgarzik> batching calls and results is entirely disconnected from that flow
444 2012-06-25 15:25:41 <luke-jr> jgarzik: perhaps batch(obj.method(args), obj.method(args), &) ?
445 2012-06-25 15:26:39 <luke-jr> otoh, even if that's possible, it might be difficult to combine with varargs
446 2012-06-25 15:28:27 <luke-jr> seems like this is really something that should be a language feature first - call multiple methods concurrently and return their results in an array
447 2012-06-25 15:28:59 <luke-jr> maybe just an ugly obj._batchcall( (method, args), & ) for now?
448 2012-06-25 15:29:03 <sipa> some high-level client-side batching would be useful
449 2012-06-25 15:29:19 <sipa> maybe some aliases could be defined for high-level methods
450 2012-06-25 15:29:56 <luke-jr> sipa: you mean the CLI use? bitcoinrpc is jgarzik's Python module
451 2012-06-25 15:31:16 <gavinandresen> https://github.com/joshmarshall/jsonrpclib    :  supports Batch submission (via MultiCall)
452 2012-06-25 15:31:46 <sipa> luke-jr: i'm talking about a hypothetical more advanced CLI tool
453 2012-06-25 15:32:09 <sipa> instead of bitcoind being CLI client
454 2012-06-25 15:33:03 <luke-jr> gavinandresen: that does look usable
455 2012-06-25 15:33:12 <gmaxwell> sipa: It's called jgarzik's python module. :)
456 2012-06-25 15:34:32 <jgarzik> >>> batch.add(3, 50)
457 2012-06-25 15:34:32 <jgarzik> >>> batch = jsonrpclib.MultiCall(server)
458 2012-06-25 15:35:16 <gmaxwell> luke-jr: ultimately if speed is really critical the hex encoding is probably as much of a concern as the seperate calls.
459 2012-06-25 15:35:54 <sipa> i doubt that
460 2012-06-25 15:36:22 <jgarzik> gmaxwell: there is more HTTP overhead than data, when talking about a 'getrawmempool' + X gettransactions calls
461 2012-06-25 15:36:25 <jgarzik> *call pattern
462 2012-06-25 15:36:55 <jgarzik> and while pipelining the gettransactions calls is possible, most client libs aren't so smart
463 2012-06-25 15:36:56 <luke-jr> gmaxwell: I'm more concerned about calculating all the extra decompositions I don't need (possibly including disk I/O) etc
464 2012-06-25 15:37:08 <gavinandresen> JSON-RPC is designed to run across a plain old filesystem pipe, raw socket, ....
465 2012-06-25 15:37:41 <gmaxwell> luke-jr: then do your own decompositions?  perhaps that code should get pulled out as a library?
466 2012-06-25 15:38:04 <gavinandresen> disk io is why I think removing the requires-fetching-previous-transaction info from the vin[] output
467 2012-06-25 15:38:14 <gavinandresen> ... is a good idea
468 2012-06-25 15:38:25 <luke-jr> gmaxwell: so we need to have the same code in N different libraries for each language someone might use?
469 2012-06-25 15:38:39 <luke-jr> gmaxwell: and have more problems with upgrades?
470 2012-06-25 15:38:40 <sipa> or one library with bindings in many languages
471 2012-06-25 15:38:47 <sipa> hey, isn't that libbitcoin?
472 2012-06-25 15:38:50 <gmaxwell> luke-jr: If only thought of a way of making one language callable by all others?!
473 2012-06-25 15:38:57 <jgarzik> WRT JSON batches, was already thinking about: batch = JSONBatch() ; batch.add() ... ; results[] = batch.execute()
474 2012-06-25 15:38:59 <luke-jr> sipa: afaik libbitcoin is just C++
475 2012-06-25 15:39:03 <jgarzik> for bitcoinrpc
476 2012-06-25 15:39:13 <sipa> luke-jr: it has python bindings afaik
477 2012-06-25 15:39:41 <luke-jr> so why do I need to reinvent the wheel because bitcoind wants to make life harder? :/
478 2012-06-25 15:39:55 <gmaxwell> Because using RPC calls to decode data is weird?
479 2012-06-25 15:39:57 <luke-jr> I'll just go back to patching in dumpblock - far easier
480 2012-06-25 15:40:16 <jgarzik> ACK/NAK troll -- RPC getnetstats -- https://github.com/bitcoin/bitcoin/pull/1510
481 2012-06-25 15:40:19 <gmaxwell> luke-jr: What do you need decompositions for in programmatic usage?
482 2012-06-25 15:41:21 <gmaxwell> jgarzik: interface looks fine, but getting the actual list of peers and stats per peer would be much more useful.
483 2012-06-25 15:41:27 <luke-jr> gmaxwell: maybe I don't, but I'd need to rewrite a lot of code to avoid using it right now
484 2012-06-25 15:41:52 <jgarzik> gmaxwell: I agree, but that was potential controvery (RE privacy), and warranted discussion
485 2012-06-25 15:41:57 <gmaxwell> luke-jr: this is why the code is pre-release, so we can get this stuff right without having to worry about legacy.
486 2012-06-25 15:42:09 <jgarzik> *controversy
487 2012-06-25 15:42:15 <gmaxwell> jgarzik: considering we log the information&
488 2012-06-25 15:42:28 <jgarzik> gmaxwell: if everyone ACKs per-peer info in there, I will add it
489 2012-06-25 15:42:37 <luke-jr> gmaxwell: I just don't think making the user do *more* work is "getting it right"
490 2012-06-25 15:42:50 <gmaxwell> I think some people will be stupid "omg I'm connected to china!" but it'll be worth it I think.
491 2012-06-25 15:43:13 <gmaxwell> Esp hidden service debugging has been a little annoying for want of an API to see who's currently connected without parsing out the logs.
492 2012-06-25 15:43:50 <sipa> luke-jr: neither is throwing out all information you got, and get people to rely on it at a point where keeping the same interface becomes a nightmare to maintain
493 2012-06-25 15:44:19 <gmaxwell> luke-jr: every bit of api complexity we add is more to support and more work for a fork/clone to deal with so we should try to be somewhat conservative with options. I like getting the decompositions for troubleshooting power user use... but its staisfied by spamming you.
494 2012-06-25 15:44:19 <sipa> if there are 5 ways to get the same information now, we can only add more ways later
495 2012-06-25 15:44:25 <luke-jr> sipa: hence why decompositions allow the user to tell you exactly what they need
496 2012-06-25 15:45:09 <sipa> luke-jr: i don't think you get the point
497 2012-06-25 15:45:10 <luke-jr> gmaxwell: I don't  understand "but its staisfied by spamming you."
498 2012-06-25 15:45:15 <sipa> (neither do i)
499 2012-06-25 15:45:42 <gmaxwell> luke-jr: the option to simply get all the decompositions covers the troubleshooting use case.
500 2012-06-25 15:47:21 <luke-jr> also, object-based parameters *improves* the ability to extend the API in the future, whereas booleans ends up with 10 different commands (send* anyone?) that vary by parameters
501 2012-06-25 15:48:22 <sipa> you have a point there, but i don't disagree about the decomposition itself
502 2012-06-25 15:48:30 <gavinandresen> I have been tempted to teach the RPC interface to take either an array of params OR key/value Object with named params.... but that's just not high priority
503 2012-06-25 15:48:40 <sipa> it is about offering 5 different ways of getting it
504 2012-06-25 15:49:23 <luke-jr> sipa: decomposition is more about offering 5 differnet information, without forcing the user to spend time waiting for other information it doesn't want
505 2012-06-25 15:49:31 <luke-jr> not merely ways of getting the same info
506 2012-06-25 15:49:32 <sipa> i know what it is about
507 2012-06-25 15:49:50 <sipa> and it is useful
508 2012-06-25 15:49:59 <sipa> but it is also an extra burden to maintain
509 2012-06-25 15:50:02 <gmaxwell> Well you cut out half of it by just deciding that it's not going to do the obj type expansion use batch json for that usecase.
510 2012-06-25 15:50:12 <jgarzik> gavinandresen: I thought about that, too
511 2012-06-25 15:50:18 <jgarzik> since JSON-RPC in theory supports it
512 2012-06-25 15:50:19 <gmaxwell> ID, raw data, and decodes all have substantial usecases.
513 2012-06-25 15:50:45 <gmaxwell> ID is needed for the programmatic batch accesses, raw data is needed for tools, and the decodes are needed for troubleshooting and power user use.
514 2012-06-25 15:51:44 <luke-jr> ok, I definitely agree the "obj" decomposition is less necessary than the rest
515 2012-06-25 15:51:51 <luke-jr> for scripts, at least
516 2012-06-25 15:52:11 <gmaxwell> I think it's also much of the unwanted complexity.
517 2012-06-25 15:53:02 <luke-jr> so how about just removing "script":"obj" decomposition, and letting the rest be?
518 2012-06-25 15:53:21 <gmaxwell> And really, I use the decodes almost every day. Getting it into the hands of advanced users (esp via the GUI console interface) will be helpful in fostering solid technical understanding of the bitcoin system.
519 2012-06-25 15:53:31 <luke-jr> that's a simple git revert 7e63dc3
520 2012-06-25 15:55:15 <luke-jr> gavinandresen: jgarzik: does that sound good enough to you?
521 2012-06-25 15:55:45 <gmaxwell> I think there is a general opposition to the choose your flavor arguments.
522 2012-06-25 15:56:25 <gavinandresen> yes, even after reading the code I couldn't figure out how to make gettransaction output what I needed to test the raw transaction API
523 2012-06-25 15:56:47 <gavinandresen> (yes to gmaxwell-- I don't want '{"foo":"bar"}' ....)
524 2012-06-25 15:57:25 <gmaxwell> I wasted about an hour figuring that out even while reading the code, because I was swapping " and ', apparently I'm json stupid.
525 2012-06-25 15:57:56 <luke-jr> [17:48:30] <gavinandresen> I have been tempted to teach the RPC interface to take either an array of params OR key/value Object with named params&. but that's just not high priority
526 2012-06-25 15:58:00 <luke-jr> ^ how is that different?
527 2012-06-25 16:00:12 <gavinandresen> it's not different, you'd be able to do either  ./bitcoind listtransactions 0 10   or   ./bitcoind listtransactions '{"start
528 2012-06-25 16:00:19 <gavinandresen> ":0, "n":10}'
529 2012-06-25 16:00:31 <gavinandresen> The latter is a lot less convenient.
530 2012-06-25 16:00:53 <gavinandresen> (unless you're working in python where it would be listtransactions(size=10, n=1)
531 2012-06-25 16:01:03 <luke-jr> gavinandresen: so& 'gettransaction txid hex/asm/obj' would be better to you?
532 2012-06-25 16:02:51 <gavinandresen> I don't want to spend any more time fence-painting this with you.  I think the general consensus here is there are two cases:  a machine is getting the information and will decode itself (because it is quicker), or you're a geek looking for human-readable output.
533 2012-06-25 16:03:47 <gmaxwell> Is there even really a disagreement anymore?
534 2012-06-25 16:04:43 <luke-jr> I think there's a lack of input from other software developers actually *using* these calls; I seem to be the only one here
535 2012-06-25 16:04:57 <gavinandresen> I think we need to split the 'get any transaction' into it's own routine anyway, because I think gmaxwell is right-- there is a real danger somebody will be hacked because they assumed gettransaction only returns wallet transactions
536 2012-06-25 16:05:13 <sipa> ACK
537 2012-06-25 16:05:20 <gavinandresen> luke-jr: I thought you had given up on the HTTP RPC interface.
538 2012-06-25 16:05:34 <luke-jr> gavinandresen: ?
539 2012-06-25 16:05:45 <PK> I'm using listtransactions for my web ui.
540 2012-06-25 16:06:59 <gavinandresen> luke-jr: maybe I'm misremembering, but I thought you were convinced that JSON could never be fast enough for a front-end <--> bitcoind communication mechanism.  Could have you confused with somebody else.
541 2012-06-25 16:07:00 <PK> But didn't have to bother with gettransactions yet.
542 2012-06-25 16:07:04 <gmaxwell> luke-jr: well, if we burn them they ought to show up here then.
543 2012-06-25 16:07:11 <luke-jr> gavinandresen: I wasn't talking about using it for a GUI
544 2012-06-25 16:07:59 <luke-jr> gmaxwell: except they've never had this functionality to begin with - more likely they'll just continue updating whatever dumpblock patch they're using
545 2012-06-25 16:08:40 <luke-jr> I think PiUK and tcatm's input here is important, to be specific
546 2012-06-25 16:08:47 <luke-jr> (tcatm runs BBE, right?)
547 2012-06-25 16:08:59 <sipa> theymos does, no?
548 2012-06-25 16:09:45 <luke-jr> maybe, I confuse the two of them too easily
549 2012-06-25 16:10:01 <sipa> tcatm runs bitcoincharts
550 2012-06-25 16:10:17 <sipa> i thought
551 2012-06-25 16:11:06 <gmaxwell> luke-jr: right now the extra stuff getting called with the RPCs means that they're somewhat slow. Otherwise the geek usecase is basically the same as that.
552 2012-06-25 16:13:15 <luke-jr> the geek usecase isn't very usable, if he has to iterate over every transaction
553 2012-06-25 16:14:04 <gmaxwell> my bash history indicates that most of the time I'm pulling single txn.
554 2012-06-25 16:14:18 <gmaxwell> where I have iterated it's actually been to walk the txn history.
555 2012-06-25 16:14:56 <luke-jr> gmaxwell: do you think that would be the case if you were analyzing your blocks?
556 2012-06-25 16:15:41 <jgarzik> <gavinandresen> I think the general consensus here is there are two cases:  a machine is getting the information and will decode itself (because it is quicker), or you're a geek looking for human-readable output.  <<-- ACK
557 2012-06-25 16:15:42 <jgarzik> x2
558 2012-06-25 16:15:52 <gmaxwell> luke-jr: well, I've done that (to check p2pool orphans for trouble transactions) but what I'm doing there is dumping the txids to a file and diffing the files then looking at single txn.
559 2012-06-25 16:16:33 <luke-jr> gmaxwell: I've never done that - I look at the transaction asm
560 2012-06-25 16:16:41 <jgarzik> <gavinandresen> I think we need to split the 'get any transaction' into it's own routine anyway  <<-- ACK
561 2012-06-25 16:16:45 <gmaxwell> also if you want to view all of them   for id in `bitcoind getblock | grep ''` ; do bitcoind gettransaction $id ;  done ....
562 2012-06-25 16:17:04 <gmaxwell> (and who cares if thats slow because it forks 1000 times?)
563 2012-06-25 16:17:37 <jgarzik> getrawtransaction <hash> [pretty_print: true/false]
564 2012-06-25 16:17:58 <luke-jr> gmaxwell: is it really that simple?
565 2012-06-25 16:18:13 <sturles> jgarzik: I have been trying to compile your filter branch, but: http://pastebin.com/FbKY3nQy
566 2012-06-25 16:18:20 <gavinandresen> raw transaction API has a decoderawtransaction <hex-encoded-serialized-transaction>  that always pretty-prints
567 2012-06-25 16:18:24 <gmaxwell> luke-jr: you need a bit of sed to strip the quotes, but yet.
568 2012-06-25 16:18:28 <gmaxwell> er yes.
569 2012-06-25 16:18:34 <jgarzik> sturles: #filter doesn't work anymore, due to upstream changes
570 2012-06-25 16:18:39 <gmaxwell> one sec, I'll give you the example.
571 2012-06-25 16:18:51 <sturles> jgarzik: Any plans to update it?
572 2012-06-25 16:19:18 <gavinandresen> ... so the Real(tm) geek way would be    ./bitcoind decoderawtransaction $(./bitcoind getrawtransaction txid)
573 2012-06-25 16:19:20 <jgarzik> sturles: yes... after CBlockStore mess settles out.  the filter-in-P2P seems more urgent
574 2012-06-25 16:19:34 <sturles> OK
575 2012-06-25 16:19:41 <jgarzik> gavinandresen: ACK worthy
576 2012-06-25 16:19:55 <luke-jr> gavinandresen: that gets ugly when you have -rpcuser=& -rpcpassword=& ;p
577 2012-06-25 16:20:06 <jgarzik> getrawtransaction <hash>  is nice and simple
578 2012-06-25 16:20:13 <luke-jr> and perhaps more important: it doesn't work at all in the Bitcoin-Qt Debug console
579 2012-06-25 16:20:44 <gmaxwell> luke-jr: for id in `bd getblock 00000000000001c1d6fc3c73153eb9f0d2ac0fe062ae41faaeab35759f89dd14 | grep '     ' | sed -e 's/[ ",]//g'` ; do bd gettransaction $id ; done
580 2012-06-25 16:21:05 <gmaxwell> luke-jr: and once 0.7.0 is shipping I'll make a great big wiki page with examples like that.
581 2012-06-25 16:21:07 <luke-jr> gmaxwell: I think that demonstrates my original point on usability <.<
582 2012-06-25 16:21:36 <gavinandresen> gmaxwell: I went looking for "JSON tools for Unix geeks" but didn't find any... started to write my own
583 2012-06-25 16:21:40 <jgarzik> gavinandresen: "raw transaction API has" ...  does this imply you already have written the code for this?  :)
584 2012-06-25 16:21:53 <gmaxwell> luke-jr: hard things should be hard, easy things easy. Getting the decodes for all txn in a block isn't exactly the simplest usecase.
585 2012-06-25 16:22:11 <gavinandresen> jgarzik: yes, but I'm iterating over it
586 2012-06-25 16:22:15 <gmaxwell> luke-jr: with my example there you could trivially write exach txn to a file.. or grep it for interesting bits.
587 2012-06-25 16:22:38 <jgarzik> gavinandresen: 'getrawtransaction <hash>' seems easily and immediately upstream-able
588 2012-06-25 16:23:26 <gavinandresen> jgarzik: ACK, as soon as I'm done announcing 0.6.3 I'll switch back to that work
589 2012-06-25 16:24:22 <gmaxwell> luke-jr: e.g. for id in `bd getblock 00000000000001c1d6fc3c73153eb9f0d2ac0fe062ae41faaeab35759f89dd14 | grep '     ' | sed -e 's/[ ",]//g'` ; do bd gettransaction $id '{"script":"asm"}' | sed -e 's/[" ]/\n/g' | grep OP; done) | sort | uniq -c | sort -n
590 2012-06-25 16:24:31 <gmaxwell> (get a histogram of opcodes in a block)
591 2012-06-25 16:32:36 <jgarzik> bleh.  "Update master"
592 2012-06-25 16:32:45 <jgarzik> we really should insist on better commit messages than that
593 2012-06-25 16:37:39 <topi`> luke-jr: why is it so that eligius has only paid me around 0.6 btc during the last 2 days? usually I get steady payment
594 2012-06-25 16:38:23 <gmaxwell> topi`: You might want to ask in #eligius other eligus users might answer you there.. in here you ~only get luke.
595 2012-06-25 16:38:40 <topi`> good point
596 2012-06-25 17:11:25 <sipa> anyone have a fast node i can -connect to for block sync?
597 2012-06-25 17:12:23 <sipa> seems my vps does just fine, nvm :)
598 2012-06-25 17:45:36 <jgarzik> sipa: us[24].exmulti.net, eu3.exmulti.net
599 2012-06-25 17:47:38 <MysteryBanshee> urgh why are blocks always take so long and my transactions never make it in?
600 2012-06-25 17:52:09 <gribble> 186212
601 2012-06-25 17:52:09 <sipa> ;;bc,blocks
602 2012-06-25 17:52:36 <sipa> MysteryBanshee: probably because the network is flooded by tons of transactions that pay more fee than yours
603 2012-06-25 18:15:03 <xorgate> any url for 0.6.3 DOS attack description?
604 2012-06-25 18:21:12 <luke-jr> xorgate: it's not disclosed
605 2012-06-25 18:21:40 <xorgate> right i see now, just found it at https://en.bitcoin.it/wiki/Common_Vulnerabilities_and_Exposures#CVE-2012-3789
606 2012-06-25 18:26:00 <sipa> 203 MB of transactions that send to SD
607 2012-06-25 18:26:20 <sipa> 248 MB of transactions that spend SD coins
608 2012-06-25 18:26:36 <sipa> (and no overlap between both categories)
609 2012-06-25 18:26:38 <gmaxwell> weird. Why are there more of the latter?
610 2012-06-25 18:26:48 <gmaxwell> (well more volume)
611 2012-06-25 18:26:52 <sipa> i suppose non-compressed pubkeys are the cause
612 2012-06-25 18:26:58 <gmaxwell> :(
613 2012-06-25 18:27:19 <sipa> the good news: after pruning, 1.2 MB remains
614 2012-06-25 18:28:32 <sipa> (104 kB of the to-SD txouts, and 1.1 MB of SD-spending-txouts)
615 2012-06-25 18:30:43 <luke-jr> is there a way to trick SD into a loop? <.<
616 2012-06-25 18:31:02 <gribble> New news from bitcoinrss: matthewnealwright opened pull request 40 on bitcoin/bitcoin.org <https://github.com/bitcoin/bitcoin.org/pull/40>
617 2012-06-25 18:32:25 <sipa> luke-jr: hmm?
618 2012-06-25 18:34:36 <nanotube> if .3 is officially released, someone needs to update the website.
619 2012-06-25 18:34:45 <luke-jr> sipa: just pondering if there's a way to get SD placing bets against itself <.<
620 2012-06-25 18:37:28 <sipa> luke-jr: sounds hard; you need to construct a transaction that uses an SD address as source
621 2012-06-25 18:38:55 <gavinandresen> nanotube: thanks, I forgot to pull request changes
622 2012-06-25 18:39:14 <gmaxwell> they have a thing to send txn to alternative outputs.
623 2012-06-25 18:39:21 <gmaxwell> you also pay where you want the coins to go.
624 2012-06-25 18:40:43 <sipa> aha!
625 2012-06-25 18:41:08 <gribble> New news from bitcoinrss: gavinandresen opened pull request 41 on bitcoin/bitcoin.org <https://github.com/bitcoin/bitcoin.org/pull/41> || Diapolo opened pull request 1519 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1519>
626 2012-06-25 18:43:21 <luke-jr> gavinandresen: This is a bug-fix release with no new features.? :p
627 2012-06-25 18:44:10 <luke-jr> normally there's a changelog
628 2012-06-25 18:45:03 <luke-jr> hmm, I forget the URI to trigger bitcoin.org to update
629 2012-06-25 18:45:57 <gmaxwell> I has it
630 2012-06-25 18:46:32 <sipa> gmaxwell: anyone tried setting that extra output to SD itself?
631 2012-06-25 18:46:48 <gmaxwell> Nope.
632 2012-06-25 18:47:02 <luke-jr> gmaxwell: can you post here? I usually grep my IRC logs, but I rotated/archived my older ones >_<
633 2012-06-25 18:47:25 <gmaxwell> I sent you a PM, I don't think it should be public because of some idiot potentially calling it in a loop.
634 2012-06-25 18:47:25 <graingert> luke-jr: bitcoin-dev is a logged channel
635 2012-06-25 18:47:32 <luke-jr> o
636 2012-06-25 18:47:53 <luke-jr> hmm
637 2012-06-25 18:47:56 <luke-jr> is it not working?
638 2012-06-25 18:48:20 <gmaxwell> I got the ack from it when I called it a moment ago. it takes a little bit
639 2012-06-25 18:48:41 <graingert> gmaxwell: shiney a new DOS attack
640 2012-06-25 18:49:11 <sipa> a dos attack against github's updater? good luck
641 2012-06-25 18:49:31 <gmaxwell> well, its some script tcatm has setup to throb the github updater.
642 2012-06-25 18:49:40 <sipa> ah
643 2012-06-25 18:50:11 <gmaxwell> (it's complexified due to the ruby macro replacement stuff, I think)
644 2012-06-25 18:50:31 <Diapolo> it's sometimes so damn weird to idle here ^^
645 2012-06-25 18:53:23 <jgarzik> gmaxwell gavinandresen: what should getpeerinfo return?  address, port.  nservices?  version?  height?  per-command stats as per https://gist.github.com/2980975 ?
646 2012-06-25 18:55:52 <BlueMatt> jgarzik: while you're at it, mind adding a command to get info on connections to -addnode'd peers?
647 2012-06-25 18:55:57 <gmaxwell> network type. inbound vs outbound, uptime.  DOS score.
648 2012-06-25 18:55:58 <gavinandresen> jgarzik: address/port/version, definitely.  current banscore (although that'll almost always be zero).  incoming/outgoing.  and I'd like to know the timestamp when it connected (is that stored?).
649 2012-06-25 18:55:59 <sipa> jgarzik: time since last send/receive, time connected, destination
650 2012-06-25 18:56:16 <sipa> gavinandresen: iirc, it is
651 2012-06-25 18:56:26 <luke-jr> BlueMatt: did you update the PPA yet?
652 2012-06-25 18:56:35 <BlueMatt> luke-jr: no, thanks need to do that
653 2012-06-25 18:56:59 <luke-jr> BlueMatt: also, Launchpad is noting there's a newer db4.8 to port
654 2012-06-25 18:57:05 <sipa> there's probably a few more interesting things in CNode
655 2012-06-25 18:57:16 <gmaxwell> jgarzik: and yes, block/txn/inv counts  (e.g. which node is flooding me?) would be useful
656 2012-06-25 18:57:52 <sipa> gmaxwell: network type isn't stored now
657 2012-06-25 18:58:09 <sipa> i believe we do store the last advertized local address
658 2012-06-25 18:58:54 <gmaxwell> perhaps 'last block advertised to us's hash'
659 2012-06-25 18:59:02 <gmaxwell> rather than / addition to height
660 2012-06-25 18:59:05 <gmaxwell> network time offset.
661 2012-06-25 18:59:41 <gmaxwell> (on the hash, the point is seeing if any peers are on forks)
662 2012-06-25 19:01:12 <jgarzik> client supplies accept-encoding, to trigger
663 2012-06-25 19:02:09 <gmaxwell> (if it's pre-auth, then thats bad)
664 2012-06-25 19:02:22 <jgarzik> -output-, post auth and only generated by bitcoind
665 2012-06-25 19:03:05 <gmaxwell> yea, encode is obviously much safer. :)
666 2012-06-25 19:03:07 <luke-jr> does zlib provide any benefit for localhost?
667 2012-06-25 19:03:13 <jgarzik> compressed input is inconvenient to code, and not as much benefit
668 2012-06-25 19:03:15 <sipa> luke-jr: no
669 2012-06-25 19:03:15 <theodore> guys, i need help and am on a time crunch would appreciate it
670 2012-06-25 19:03:20 <theodore> how do you reinstall Ubuntu?
671 2012-06-25 19:03:27 <theodore> without any external media
672 2012-06-25 19:03:33 <helo> theodore: #ubuntu
673 2012-06-25 19:03:48 <sipa> luke-jr: i don't think it even makes sense for a fast LAN
674 2012-06-25 19:03:48 <theodore> damn
675 2012-06-25 19:03:50 <theodore> thought you guys would know
676 2012-06-25 19:04:01 <gmaxwell> luke-jr: zlib is fast.. but not that fast..
677 2012-06-25 19:04:50 <gmaxwell> Lz4 (>>1gbit/sec encode+decode) or something like that could be a win on a lan, but not locally.
678 2012-06-25 19:05:01 <gmaxwell> and zlib is nowhere near that fast.
679 2012-06-25 19:05:40 <luke-jr> is JSON-RPC still "intended for localhost only"?
680 2012-06-25 19:06:43 <sturles> Against bitcoind?
681 2012-06-25 19:06:52 <gmaxwell> we wouldn't have allowip otherwise, I suppose. there are certantly cases for doing it remotely. esp on coinless nodes.
682 2012-06-25 19:07:02 <graingert> use a caching proxy like nginx
683 2012-06-25 19:07:05 <gmaxwell> personally I ssh -L it, and that does zlib compression.
684 2012-06-25 19:07:16 <graingert> to do the gzip
685 2012-06-25 19:07:26 <graingert> s/caching/revserse/
686 2012-06-25 19:07:37 <graingert> s/revserse/reverse/
687 2012-06-25 19:11:02 <luke-jr> hmm
688 2012-06-25 19:11:09 <luke-jr> does the builtin SSL support do it already, I wonder?
689 2012-06-25 19:21:25 <luke-jr> I get the feeling I'm stressing gavinandresen and jgarzik out today, so I'm going to take a break for a bit. ttyl
690 2012-06-25 19:30:09 <jgarzik> luke-jr: no, that's a useful observation.  if WAN is always SSL, and SSL includes compression, maybe there's the answer
691 2012-06-25 19:35:08 <graingert> hmm
692 2012-06-25 19:35:16 <graingert> I didn't know it had built in SSL
693 2012-06-25 19:35:56 <graingert> seems a bit pointless I thought the standard pattern was to slap nginx or ssh in front of insecure locally hosted daemons
694 2012-06-25 19:44:40 <gmaxwell> graingert: nginx in front of an _rpc_ interface? This seems rather questionable to me. ... ssh yea.. well, we're linking openssl for other stuff anyways. Though the SSL support makes me twitch a little.
695 2012-06-25 19:45:02 <graingert> gmaxwell: yeah what's the issue with that?
696 2012-06-25 19:45:11 <graingert> nginx is used in front of SPARQL boxes
697 2012-06-25 19:45:14 <graingert> and node.js
698 2012-06-25 19:45:53 <graingert> drop the ssl support and recommend people proxy through apache or nginx or some sane place for SSL
699 2012-06-25 19:45:58 <gmaxwell> graingert: it's a lot of extra memcpys and an additional failure point for zero benefit except wrapping with SSL it would all be passthrough.
700 2012-06-25 19:46:25 <graingert> well I use it for adding SSL and caching to node.js
701 2012-06-25 19:46:29 <gmaxwell> The recommendation is not to expose the rpc port to the internet at all. Failure to heed this advice has gotten a bunch of people robbed.
702 2012-06-25 19:46:39 <gmaxwell> Yes, caching makes absolutely no sense in this context.
703 2012-06-25 19:46:46 <graingert> of course
704 2012-06-25 19:47:03 <graingert> but adding SSL I'd rather have it handled by the nginx guys
705 2012-06-25 19:47:08 <graingert> than bitcoin
706 2012-06-25 19:47:32 <graingert> it's also got fancy logging stuff
707 2012-06-25 19:47:48 <graingert> so I can see which tor endpoint robbed me
708 2012-06-25 19:47:52 <graingert> exit node*
709 2012-06-25 19:48:21 <jrmithdobbs> why they're both very simple wrappers around openssl
710 2012-06-25 19:48:29 <graingert> jrmithdobbs: what are?
711 2012-06-25 19:48:38 <jrmithdobbs> ngnix and bitcoin's use of openssl for rpc
712 2012-06-25 19:48:45 <gmaxwell> graingert: we need logging regardless of their being ssl (and we have it).
713 2012-06-25 19:48:45 <graingert> true but fancy logging
714 2012-06-25 19:48:56 <gmaxwell> s/their/there/
715 2012-06-25 19:48:57 <graingert> you have logging?
716 2012-06-25 19:49:01 <jrmithdobbs> yes
717 2012-06-25 19:49:02 <gmaxwell> Yes.
718 2012-06-25 19:49:04 <graingert> fancy
719 2012-06-25 19:49:25 <graingert> yeah but I also want to shard my bitcoin boxes
720 2012-06-25 19:49:25 <jrmithdobbs> graingert: -printtoconsole + svlogd so you can actually keep enough logs for them to be useful but not fill your disks
721 2012-06-25 19:49:44 <gmaxwell> Besides, when it becomes possible to seperate the wallets and the core daemon we'll want to use ssl for that too.
722 2012-06-25 19:49:45 <graingert> with an nginx front end
723 2012-06-25 19:50:06 <graingert> well not ssl on the same box
724 2012-06-25 19:50:10 <gmaxwell> graingert: then do what you want, though that doesn't make a lot of sense.
725 2012-06-25 19:50:22 <graingert> yes but it's web scale
726 2012-06-25 19:50:30 <jrmithdobbs> nginx doesn't really buy you anything here tho, even if you want to do load balancing and stuff there's no reason to strip openssl out of the codebase
727 2012-06-25 19:50:31 <gmaxwell> You're trying to get banned, right? :P
728 2012-06-25 19:50:35 <graingert> >.>
729 2012-06-25 19:50:53 <sipa> gmaxwell: obviously we'll use completely self-written EC code that identifies using Bitcoin keys!
730 2012-06-25 19:51:08 <jrmithdobbs> and load balancing bitcoin rpc could get hairy (and have to be sticky in current form) anywys since you'll be talking to multiple different nodes with different wallets and such
731 2012-06-25 19:51:19 <gmaxwell> sipa: I did half expect us to do that. :(  The ssl cert model sucks even for that.
732 2012-06-25 19:51:30 <graingert> what's wrong with dbus
733 2012-06-25 19:51:37 <gmaxwell> I'd like to be able to create a bitcoin node url that packs the host/port/publickey into it.
734 2012-06-25 19:51:38 <jrmithdobbs> what's not wrong with dbus is a better question
735 2012-06-25 19:51:47 <graingert> dbus rocks stfu
736 2012-06-25 19:52:02 <jrmithdobbs> it's great for letting everything on your box bypass auth, sure
737 2012-06-25 19:52:10 <jrmithdobbs> I don't know how that equates to "rocks" ;p
738 2012-06-25 19:52:12 <gmaxwell> Though I suppose we can still do that with SSL... just put the fingerprint in it and have some validation code know how to pull it from the URL.
739 2012-06-25 19:52:12 <graingert> it makes things faster
740 2012-06-25 19:52:44 <gmaxwell> graingert: dbus is why fedora 18 will require reboots for upgrades because making software that can be updated in place is too hard for the desktop jockies.
741 2012-06-25 19:53:04 <graingert> gmaxwell: kernel upgrades require reboots?
742 2012-06-25 19:53:27 <sipa> yeah, ksplice...
743 2012-06-25 19:53:40 <gmaxwell> graingert: sure, but thats _all_ that does, unless you ksplice as sipa mentions.
744 2012-06-25 19:53:53 <graingert> yeah
745 2012-06-25 19:54:07 <graingert> so what's wrong with fed18 needed reboots?
746 2012-06-25 19:54:13 <graingert> needing*
747 2012-06-25 19:54:26 <jrmithdobbs> ksplice is basically dead, thanks oracle
748 2012-06-25 19:54:32 <graingert> kexec ?
749 2012-06-25 19:54:38 <gmaxwell> ... because it goes from the kernel (which is updated infrequently and can wait until you get around to rebooting) to everything, which is updated more frequently.
750 2012-06-25 19:54:48 <gmaxwell> kexec is a reboot, effectively.
751 2012-06-25 19:54:56 <jrmithdobbs> graingert: kexec doesn't let you replace the running kernel, you still go all the way to init 0 and back up
752 2012-06-25 19:55:02 <graingert> yeah
753 2012-06-25 19:55:08 <jrmithdobbs> you've obviously never actually tried to implement this ;p
754 2012-06-25 19:55:12 <jrmithdobbs> it's a pain in the ass
755 2012-06-25 19:55:18 <sipa> just no BIOS calls to cycle power
756 2012-06-25 19:55:23 <graingert> so what's wrong with needing to reboot?
757 2012-06-25 19:55:26 <graingert> on fed18
758 2012-06-25 19:55:31 <graingert> when every other os did it too
759 2012-06-25 19:55:36 <graingert> does*
760 2012-06-25 19:55:43 <sipa> is that an excuse?
761 2012-06-25 19:55:51 <graingert> yes
762 2012-06-25 19:56:09 <gmaxwell> In any case, OT discussion.
763 2012-06-25 19:56:28 <gmaxwell> No we're not removing SSL from bitcoin even if its arguable in the RPC case we'll have more need for it later.
764 2012-06-25 19:56:53 <graingert> gmaxwell: but no dbus ever
765 2012-06-25 19:57:06 <jrmithdobbs> gmaxwell: did all the BN stuff get stripped? I mean, it's not even feasible really
766 2012-06-25 19:57:11 <jrmithdobbs> without a lot of rework
767 2012-06-25 19:57:12 <gmaxwell> I'm sure we'll have dbus someday too. :-/
768 2012-06-25 19:57:17 <jrmithdobbs> i hope not
769 2012-06-25 19:57:18 <graingert> YAAY
770 2012-06-25 19:57:22 <graingert> DBUUUS
771 2012-06-25 19:57:35 <sipa> we could use gmp for BN instead of OpenSSL
772 2012-06-25 19:58:10 <jrmithdobbs> ya, lets swap out bsd licensed code for gpl for no functional reason, sounds fantastic :(
773 2012-06-25 19:58:12 <gmaxwell> "YOU HAVE NEW BITCOIN. PLEASE WAIT WHILE A BUNCH OF CRAP POPS UP OVER YOUR SCREEN."  or... more likely, the dbus will be needed so that the reboot for update doesn't screw up bitcoin. :)
774 2012-06-25 19:58:32 <graingert> gmaxwell: we already have that
775 2012-06-25 19:58:35 <sipa> gmp is gpl... right
776 2012-06-25 19:58:49 <gmaxwell> <3 GMP.
777 2012-06-25 19:58:52 <graingert> !google is gmp gpl?
778 2012-06-25 19:58:53 <gribble> GNU Multiple Precision Arithmetic Library - Wikipedia, the free ...: <http://en.wikipedia.org/wiki/GNU_Multiple_Precision_Arithmetic_Library>; FW: Is GMP safe for using on multi core processors with openmp or ...: <http://gmplib.org/list-archives/gmp-discuss/2009-July/003844.html>; InstallingGCC - GCC Wiki: <http://gcc.gnu.org/wiki/InstallingGCC>
779 2012-06-25 19:59:00 <graingert> it's GNU
780 2012-06-25 19:59:06 <graingert> of course it's GPL
781 2012-06-25 19:59:15 <sipa> it's LGPL
782 2012-06-25 19:59:21 <graingert> nooooooooooooo
783 2012-06-25 19:59:26 <gmaxwell> graingert: There wasn't any disagreement that it was gpl (though quite a few gnu libs are lgpl)
784 2012-06-25 19:59:31 <jrmithdobbs> still a license downgrade for no functional use, heh
785 2012-06-25 19:59:32 <gmaxwell> oh it's lgpl? cool.
786 2012-06-25 19:59:50 <graingert> bitcoin should totally be AGPL for max trollag
787 2012-06-25 19:59:53 <gmaxwell> jrmithdobbs: we have lgpl things already. ::shrugs::
788 2012-06-25 19:59:53 <graingert> e
789 2012-06-25 20:00:16 <graingert> brb reboot
790 2012-06-25 20:00:22 <jrmithdobbs> gmaxwell: ya, i thought it was gpl, lgpl isn't that bad, but i still don't see an actual reason for openssl/BN -> gmp ;p
791 2012-06-25 20:00:37 <graingert> because why not
792 2012-06-25 20:00:40 <jrmithdobbs> other than if you want to remove openssl
793 2012-06-25 20:00:46 <sipa> well openssl is big, does way too much, and is a problem on distro's that strip EC anyway
794 2012-06-25 20:01:05 <graingert> strip bdb
795 2012-06-25 20:01:08 <graingert> first
796 2012-06-25 20:01:13 <sipa> willdo
797 2012-06-25 20:01:15 <jrmithdobbs> graingert: read the mailling list
798 2012-06-25 20:01:17 <gmaxwell> And openssl also has screwed us over protocol wise several times because it slips in unexpected behavior.
799 2012-06-25 20:01:22 <graingert> jrmithdobbs: link
800 2012-06-25 20:01:26 <gmaxwell> graingert: DONE. basically.
801 2012-06-25 20:01:30 <jrmithdobbs> graingert: sf.net
802 2012-06-25 20:01:31 <graingert> oh rly
803 2012-06-25 20:01:36 <graingert> gmaxwell: how?
804 2012-06-25 20:01:42 <gmaxwell> Magic.
805 2012-06-25 20:02:00 <gmaxwell> TD: has replaced it with an in-tree copy of leveldb.
806 2012-06-25 20:02:10 <sipa> only for the blockchain, btw
807 2012-06-25 20:02:13 <gmaxwell> Dunno if we'll go that route, looks appealing though.
808 2012-06-25 20:02:14 <gribble> New news from bitcoinrss: TheBlueMatt opened pull request 1520 on bitcoin/bitcoin <https://github.com/bitcoin/bitcoin/pull/1520>
809 2012-06-25 20:02:23 <sipa> but the peers db is already custom in git head
810 2012-06-25 20:02:31 <gmaxwell> sipa: I thought he was working on the wallet? (which I thought was silly because we should move to the append only file for that)
811 2012-06-25 20:02:33 <sipa> and i have been working on replacing the wallet
812 2012-06-25 20:02:45 <graingert> jrmithdobbs: that's a domain, now for the protocol and the pathname
813 2012-06-25 20:02:51 <jrmithdobbs> gmaxwell: no just the blockchain index from what i saw
814 2012-06-25 20:03:04 <sipa> gmaxwell: i recall TD saying leveldb for wallet would be a bad idea, and i agreed
815 2012-06-25 20:03:12 <jrmithdobbs> graingert: sorry, not my job to subsidize your laziness
816 2012-06-25 20:03:39 <gmaxwell> sipa: yea, I must have misunderstood what he was saying since I thought otherwise, and also thought it a bad idea.
817 2012-06-25 20:03:42 <graingert> really you want to come up with a wallet serialization format
818 2012-06-25 20:03:53 <jrmithdobbs> yes, there's good reasons for it
819 2012-06-25 20:04:41 <sipa> yes, append-only, cryptographically checksummed, simple key-value store in a single file
820 2012-06-25 20:04:56 <gmaxwell> Ah, I misunderstood "(I only tested
821 2012-06-25 20:04:57 <gmaxwell> with empty wallets)."
822 2012-06-25 20:05:05 <gmaxwell> as meaning he was now doing wallets too. Bad assumption.
823 2012-06-25 20:05:34 <sipa> so it is safe to be moved around, doesn't depend on a db environment, and cannot get corrupted from normal hardware failure
824 2012-06-25 20:05:35 <jrmithdobbs> sipa: prunable though, right so a wallet doesn't have to grow arbitrarily large over time if keeping local copy of all history isn't desirable
825 2012-06-25 20:06:06 <jrmithdobbs> that was supposed to end with a ?
826 2012-06-25 20:06:07 <sipa> jrmithdobbs: yeah, rewrite once everytime X% is overwritten
827 2012-06-25 20:06:21 <jrmithdobbs> ya thought I remembered you mentioning something like that, cool
828 2012-06-25 20:07:10 <gmaxwell> Anyone here seen the bitcoin magazine? Is it reasonable or Bruce Wagner 2.0?
829 2012-06-25 20:07:12 <jgarzik> sipa: block index matches same pattern, though with a larger amount of data