1 2014-08-01 01:05:26 <BlueMatt> RelayNode upgrade testing in progress http://0bin.net/paste/GsNCZu0G87hynM6C#9GlKAkvQHMp6empHfDjyk0rR+xt-ihsbnC3FLZo9XNR
  2 2014-08-01 01:05:58 <BlueMatt> relaying only partial blocks is beating regular p2p relay mechanism and skipping like 1/2 of all transactions
  3 2014-08-01 01:06:18 <BlueMatt> Notably "Relayed 778 txn out of blocks" and "In total, skipped 755"
  4 2014-08-01 01:06:53 <BlueMatt> just means I need a bigger txn cache
  5 2014-08-01 01:11:20 <gmaxwell> BlueMatt: hows this work? run special client code remotely? and it just has a deterministic management of what txn it caches?
  6 2014-08-01 01:18:26 <gmaxwell> BlueMatt: need another tester?
  7 2014-08-01 01:58:38 <rfsoyt> Hi, I'm trying to find a way that I can programmatically get the current balance of any public address. I'd prefer to host my own solution rather than calling blockchain.info or similar. Does anyone have suggestions on a good open source solution that can help me? The closest I've found is bitcoin-abe
  8 2014-08-01 02:00:16 <maaku> rfsoyt: address index
  9 2014-08-01 02:01:27 <rfsoyt> maaku: hmm, care to elaborate? link?
 10 2014-08-01 02:05:06 <sipa> rfsoyt: at the protocol level, there is no such thing as "balance of an address"; there are just outputs to a particular address (script), and some of those may be unspent; their sum is what you call the balance, but you must know the actual outputs to be able to spend them
 11 2014-08-01 02:05:29 <sipa> rfsoyt: bitcoind does not maintain an index for that, only for the actual outputs, so it can't efficiently compute that
 12 2014-08-01 02:05:37 <sipa> as it's not necessary for normal operator
 13 2014-08-01 02:06:06 <sipa> typically, wallet software just watching for transactions that pay to addresses you have the key for, and track those
 14 2014-08-01 02:07:29 <sipa> there are some pieces of software (including a patch for bitcoind called addrindex) which maintain an index for _all_ addresses, but this is expensive in terms of storage and processing overhead
 15 2014-08-01 02:12:13 <rfsoyt> sipa: I see, thanks.
 16 2014-08-01 02:15:11 <cfields> BlueMatt: ping
 17 2014-08-01 02:17:20 <cfields> BlueMatt: mind seeing what's going on with the test here? https://travis-ci.org/theuni/bitcoin/jobs/31387062
 18 2014-08-01 02:17:55 <Luke-Jr> rfsoyt: also note that when those outputs are spent, the transactions spending them have nothing to do with the address they were created receiving to
 19 2014-08-01 02:18:28 <Luke-Jr> so really, to talk about an address balance is nonsensical
 20 2014-08-01 02:20:35 <sipa> cfields: you're running with the large reorg test enabled i think; bitcoin's been failing those for months
 21 2014-08-01 02:20:46 <rfsoyt> Luke-Jr: I see, I'm just looking for a way I can run a service similar to blockchain.info that will give me an API to query public addresses (eg from mobile app), but I'd prefer to run my own rather than relying on a 3rd party
 22 2014-08-01 02:21:29 <Luke-Jr> rfsoyt: I encourage you to run one that doesn't have all the misinformation bc.i has
 23 2014-08-01 02:21:51 <Luke-Jr> rfsoyt: I'd love a site I can link/refer people to without having the concern that they will be confused by all the misinformation on it
 24 2014-08-01 02:23:04 <rfsoyt> Luke-Jr: yeah, I don't care too much for an interface, I just want to make REST calls to get info
 25 2014-08-01 02:23:28 <Luke-Jr> that being said, you'll probably need the address index patch for anything similar
 26 2014-08-01 02:23:51 <Luke-Jr> basically it just means you can search transactions by the scriptPubKey
 27 2014-08-01 02:28:27 <denisx> can I have the link to the bootstrap.dat torrent again pls
 28 2014-08-01 02:29:35 <Luke-Jr> http://bitcointroll.org/?topic=145386.0
 29 2014-08-01 02:30:43 <denisx> Luke-Jr: the new one is still not announced there
 30 2014-08-01 02:30:49 <denisx> I wanted to help seeding
 31 2014-08-01 02:31:05 <Luke-Jr> ah, didn't know there was a new one yet
 32 2014-08-01 02:31:28 <denisx> I have it running on another machine, but it is a magnet link and I dont know how to copy that
 33 2014-08-01 02:32:09 <denisx> ah, I got it
 34 2014-08-01 02:37:40 <dsnrk> rfsoyt: a lot of places use Bitpay's Insight app to do that.
 35 2014-08-01 02:42:19 <cfields> sipa: Ah, didn't realize it was known to be failing
 36 2014-08-01 02:50:43 <randomdev> how do i generate alert keys?
 37 2014-08-01 02:52:15 <randomdev> and how do i send alerts through bitcoind?
 38 2014-08-01 02:52:59 <randomdev> im interested in testing some stuff involving alerts
 39 2014-08-01 02:53:33 <dsnrk> altcoin development is way off topic for -dev.
 40 2014-08-01 02:53:33 <randomdev> is anyone here?
 41 2014-08-01 02:54:23 <dsnrk> if you can't work that little feature out, you shouldn't be even thinking of cloning bitcoin.
 42 2014-08-01 02:54:57 <randomdev> i apologize for bringing it up, but i cant find any documentation about it. I'm not creating an altcoin at the moment, I'm trying to learn how the various parts of Bitcoin work, and this is a part without extensive documentation.
 43 2014-08-01 02:55:30 <Luke-Jr> randomdev: there is only one alert key, and it cannot be generated
 44 2014-08-01 02:56:12 <randomdev> but how was that keypair generated? that is what i am confused about
 45 2014-08-01 02:56:51 <Luke-Jr> randomdev: nobody knows for sure, since only Satoshi ever did it. presumably the same way transaction keypairs are generated
 46 2014-08-01 02:58:15 <dsnrk> randomdev: it's pretty much useless to everybody except the select few with the alert key.
 47 2014-08-01 02:58:16 <dsnrk> it's just a standard keypair like any other.
 48 2014-08-01 02:58:16 <dsnrk> you can even spend coins to it's hash160 if you would like to, though nobody will ever pick them up.
 49 2014-08-01 02:58:41 <dsnrk> from an academic standpoint the alert key is totally uninteresting.
 50 2014-08-01 03:27:37 <nullbyte> ... then Sysrq.
 51 2014-08-01 05:20:23 <BlueMatt> gmaxwell: yes, exactly that, just run local client, deterministic txn cache
 52 2014-08-01 05:20:37 <BlueMatt> gmaxwell: I am absolutely looking for testers, need to iron out one more bug first, though
 53 2014-08-01 05:22:18 <BlueMatt> cfields: looks like you're running with the full test, which will do a very, very large reorg and may take >10 minutes between outputs
 54 2014-08-01 05:22:56 <cfields> BlueMatt: yep, sipa told me. thanks. Didn't realize it was known to be failing atm
 55 2014-08-01 05:23:21 <gmaxwell> BlueMatt: does it inv towards the relay to avoid the relay sending it txn that it learned from the network already?
 56 2014-08-01 05:24:01 <gmaxwell> e.g. realy delay sending txn to the proxy, if it learns from its local node first, it INVs?
 57 2014-08-01 05:24:28 <BlueMatt> gmaxwell: hmm? no nothing so fancy
 58 2014-08-01 05:25:12 <gmaxwell> hm. I wonder how the performance of this could be measured. I guess the metric would be for bitcoind to log on subsiquent block INVs how many seconds ago it accepted the block.
 59 2014-08-01 05:25:47 <BlueMatt> gmaxwell: yes, I've been looking at exactly that
 60 2014-08-01 05:26:01 <BlueMatt> havent had a chance to measure in detail yet due to one strange bug I havent found yet
 61 2014-08-01 05:27:38 <BlueMatt> gmaxwell: the relay nodes will relay blocks as soon as they get them over the regular ports as well, so you can connect bitcoind to both the local client and the regular port and see which wins
 62 2014-08-01 05:28:05 <BlueMatt> but its a rather inconvenient measure (as the log posted indicates) due to biticoind's single-threaded network model
 63 2014-08-01 05:32:10 <gmaxwell> I wonder if you start getting the header of the block if you should inv it towards the node to prevent the node from trying to fetch it from another peer if the other peers inv beats the whole block showing up via your tool?
 64 2014-08-01 05:32:31 <gmaxwell> (it'll getdata towards the first peer to INV it)
 65 2014-08-01 05:33:06 <BlueMatt> hmm, that would seem reasonable
 66 2014-08-01 07:25:28 <gmaxwell> Big (unimportant) news: We have a block with > 80 bits of zeros now. 00000000000000000000b7de9e5c19e52be073156924b7cf235efb27ae8a202a
 67 2014-08-01 07:34:34 <phantomcircuit> gmaxwell, diff 4×10¹⁹
 68 2014-08-01 07:34:39 <phantomcircuit> neat
 69 2014-08-01 07:35:55 <wumpus> woooot
 70 2014-08-01 07:39:34 <gmaxwell> phantomcircuit: more like 1e14 I think.
 71 2014-08-01 07:39:54 <gmaxwell> oh no wrong block
 72 2014-08-01 07:40:46 <gmaxwell> 3.9e14
 73 2014-08-01 08:35:25 <spinza> Should be small news not big...  :)
 74 2014-08-01 10:32:24 <jgarzik> michagogo, new linearize scripts at git://github.com/jgarzik/bitcoin.git 2014_linearize
 75 2014-08-01 10:32:35 <jgarzik> michagogo, creates a clean blocks/blk*.dat
 76 2014-08-01 10:37:44 <wumpus> would it be useful to return (block file, offset) for a block from some RPC call, so that local clients know where to find the data without having to transfer it?
 77 2014-08-01 10:38:20 <cdecker_> I guess that unless you're using that information multiple times things don't change too much
 78 2014-08-01 10:38:31 <wumpus> of even (block file, offset, size)
 79 2014-08-01 10:38:36 <wumpus> well there is a lot of transfer overhead in json-rpc
 80 2014-08-01 10:38:41 <cdecker_> It should not be a huge difference between you doing the seek or the client
 81 2014-08-01 10:39:02 <jgarzik> wumpus, thought about that
 82 2014-08-01 10:39:05 <cdecker_> And my guess is that the seek trumps other operations
 83 2014-08-01 10:39:05 <wumpus> you could do one batched JSON call to get those ranges for a whole slew of blocks
 84 2014-08-01 10:39:12 <jgarzik> wumpus, but no need, I just read through the data on disk
 85 2014-08-01 10:39:18 <jgarzik> wumpus, RPC only used for getting block hashes
 86 2014-08-01 10:39:46 <wumpus> jgarzik: I know, but parsing through all the blocks in python again seems suboptimal
 87 2014-08-01 10:39:56 <wumpus> the daemon already knows where to find the data
 88 2014-08-01 10:40:01 <jgarzik> wumpus, it's far faster than rpc
 89 2014-08-01 10:40:19 <wumpus> ok, never mind...
 90 2014-08-01 10:41:07 <wumpus> at least I don't agree with cdecker_, transferring block data over JSON-RPC is slow, much slower than you'd think with just the seek times
 91 2014-08-01 10:41:37 <jgarzik> wumpus, quite true, that's why I avoided it :)
 92 2014-08-01 10:41:43 <jgarzik> it's achingly slow.
 93 2014-08-01 10:41:59 <wumpus> I know, but your solution is not very general, every client of block data will have to plough through all the block files
 94 2014-08-01 10:42:18 <jgarzik> wumpus, the need is not very general :)
 95 2014-08-01 10:42:37 <wumpus> the need for sourcing raw block data is not very general?!
 96 2014-08-01 10:43:14 <jgarzik> wumpus, the need to scan through lots of block data all at once
 97 2014-08-01 10:43:15 <wumpus> well it has come up for me already at least a few times, in the end I used P2P to get the data instead of JSON-RPC
 98 2014-08-01 10:43:21 <jgarzik> wumpus, HTTP REST is quite fast for simple download
 99 2014-08-01 10:43:24 <wumpus> what about custom indexing and such?
100 2014-08-01 10:43:48 <jgarzik> no decoding w/ REST
101 2014-08-01 10:43:55 <wumpus> right, though I'm not so happy about adding a completely new interface
102 2014-08-01 10:44:45 <wumpus> but indeed it saves the encoding/decoding overhead
103 2014-08-01 10:45:33 <cdecker_> I stand corrected :-)
104 2014-08-01 10:46:35 <wumpus> jgarzik: I'm fine with it if the goal of the HTTP interface is to eventually replace the JSON-RPC interface for 'dumb data queries'
105 2014-08-01 10:52:28 <jgarzik> wumpus, correct
106 2014-08-01 10:54:14 <jgarzik> wumpus, RE (block file, offset)  In general I do not think we want to encourage that.  If you know what you're doing, if you know not to modify these files etc. then it is ok.  Otherwise, it is potentially encouraging corruption of a live dataset by offering backdoor access.
107 2014-08-01 10:54:46 <jgarzik> wumpus, it is usually better to offer an interface, in case underlying data formats change
108 2014-08-01 10:55:21 <wumpus> encouraging corruption? eh okay
109 2014-08-01 10:55:54 <wumpus> and the block format on disk should not change from the network format
110 2014-08-01 10:56:26 <jgarzik> wumpus, it has already changed once, to include zeroes appended after end of file
111 2014-08-01 10:56:34 <jgarzik> end of blocks
112 2014-08-01 10:56:48 <jgarzik> wumpus, that broke two raw block file scanners
113 2014-08-01 10:56:50 <wumpus> will if it just returns the file name, offset and size, what is *outside* those bounds isn't interesting and can change
114 2014-08-01 10:56:53 <wumpus> well*
115 2014-08-01 10:57:19 <wumpus> and changing that wouldn't break block scanners any more in that case
116 2014-08-01 11:00:42 <jgarzik> wumpus, Again, all of what you argue is strictly true.  But it still remains a slightly higher danger level to return (file N, offset).   e.g. when you have to add qualifiers like "what is outside those bounds" that clearly ropes in software with boundary checking bugs :)
117 2014-08-01 11:00:43 <wumpus> the framing could be anything, the only guarantee is that some file will contain the block data in network format at some position, which is a reasonable assumption.... if not, block scanners won't work anyway, and it could return None to signify that we don't have the data anymore
118 2014-08-01 11:00:59 <jgarzik> if there is a strong need, I suppose we can include it
119 2014-08-01 11:01:23 <jgarzik> but it will always get a strong pushback from me, because in general we should avoid offering low level access to on-disk data
120 2014-08-01 11:01:29 <wumpus> well I seem to be the only one to think it's a good idea
121 2014-08-01 11:01:33 <wumpus> so you can be happy
122 2014-08-01 11:01:54 <wumpus> in general I agree, but the block data is different
123 2014-08-01 11:01:58 <jgarzik> sipa described a compressed block format, so even " the block format on disk should not change from the network format" is not true necessarily.
124 2014-08-01 11:02:03 <wumpus> a) it is huge b) it has a well-defined (internal) format
125 2014-08-01 11:02:19 <wumpus> c) you really only want to store one copy, and access it when needed
126 2014-08-01 11:02:34 <jgarzik> (b) looks to change
127 2014-08-01 11:02:39 <wumpus> I wouldn't advocate the same for the database files obviously
128 2014-08-01 11:02:51 <wumpus> compressing the block data makes no sense
129 2014-08-01 11:03:06 <wumpus> many people have tried it and it has almost no win for the CPU time spent on it
130 2014-08-01 11:03:14 <wumpus> and that would *ALSO* break your block scanner
131 2014-08-01 11:03:56 <jgarzik> Yep.  The block format changed once, and according to sipa idea it might change again.
132 2014-08-01 11:04:23 <jgarzik> Certainly a compressed format is much more severe change than zeroes.
133 2014-08-01 11:04:25 <wumpus> sigh, okay
134 2014-08-01 11:05:52 <wumpus> so discourage scanning bitcoind's block files?
135 2014-08-01 11:06:00 <wumpus> I'm pretty sure a certain bitpay project does the same
136 2014-08-01 11:06:40 <wumpus> or does it get the data over P2P now?
137 2014-08-01 11:21:32 <jgarzik> wumpus, a fair question. dunno. it might.
138 2014-08-01 12:47:57 <dsnrk> wumpus: you'd be surprised, just whacking the block files through a compressor today gets you about a 30% decrease in the size of the block files.
139 2014-08-01 12:48:27 <sipa> dsnrk: for a bootstrap that makes sense[A
140 2014-08-01 12:49:08 <sipa> dsnrk: for a bootstrap that makes sense, but for actual usage we need to be able to access the blocks individually
141 2014-08-01 12:49:21 <dsnrk> indeed. not sure if it makes sense for anything else, but it did surprise me that it works.
142 2014-08-01 12:50:22 <wumpus> dsnrk: that's one block at a time or everything at once?
143 2014-08-01 12:51:00 <dsnrk> wumpus: that was just over the block files, the ~100MB hunks. depends a lot on which blocks of course.
144 2014-08-01 12:51:59 <dsnrk> just for my own interest I'm going to try it on a bootstrap.dat, but that's going to take a while.
145 2014-08-01 12:52:17 <sipa> which compression algorithm?
146 2014-08-01 12:52:56 <dsnrk> 7z/LZMA
147 2014-08-01 12:53:01 <sipa> one thing that would be trivial to do in terms of compression is the 'headers' command... it always sends chains of headers, so you don't really need hashprev or nbits or most of ntime in them
148 2014-08-01 12:53:18 <sipa> iirc it could trivially use 39 bytes instead of 81
149 2014-08-01 13:00:00 <dsnrk> blk00158.dat 13412[D[D[D[D[D[D6231 bytes > 89092359 bytes
150 2014-08-01 13:00:22 <wumpus> sipa: I think we're going too far in #4582
151 2014-08-01 13:00:30 <wumpus> my only point there aws to add seeds for onion addresses
152 2014-08-01 13:01:18 <dsnrk> er, sorry. 134MB to 89MB. which is more than I would have guessed.
153 2014-08-01 13:02:09 <sipa> wumpus: sorry, commented
154 2014-08-01 13:07:01 <wumpus> on the next seeds update it makes sense to use dotted addresses (they are supported by the python script), for now having them like this makes it clear that I used the same input data
155 2014-08-01 13:07:30 <sipa> yup
156 2014-08-01 13:38:24 <michagogo> Hm, how were the onion seeds selected?
157 2014-08-01 13:38:47 <michagogo> thfsmmn2jbitcoin.onion is in testnet, but not mainnet
158 2014-08-01 13:39:15 <michagogo> Pretty sure that one is mine, and it's on both nets
159 2014-08-01 13:40:25 <wumpus> michagogo: for mainnet they come from a crawler and the wiki page with stable fallback nodes,  for testnet it's just a random list
160 2014-08-01 13:41:58 <wumpus> (I also posted that to the issue, by the way)
161 2014-08-01 13:50:31 <dsnrk>  /window 5
162 2014-08-01 13:50:34 <dsnrk> mmf.
163 2014-08-01 13:51:59 <payments> Hello. I hope you are all well.
164 2014-08-01 13:52:19 <payments> I have question regarding BIP70, specifically the format of the Certificates field on a Payment Request.
165 2014-08-01 13:52:52 <payments> According to the documentation it is a chain of certificates, which are presented as "DER [ITU.X690.1994] PKIX".
166 2014-08-01 13:53:46 <jgarzik> payments, yes
167 2014-08-01 13:53:47 <payments> I cannot see where this format is explained or documented. Is it basically the -----BEGIN CERTIFICATE----- / END CERTIFICATE data?
168 2014-08-01 13:54:26 <jgarzik> payments, trickles down from X.509
169 2014-08-01 13:54:49 <payments> jgarzik, Okay, so I should be able to find out how to encode my self signed x509 (from openssl) into this format.
170 2014-08-01 13:56:41 <jgarzik> payments, http://www.planetlarg.net/encyclopedia/ssl-secure-sockets-layer/der-distinguished-encoding-rules-certificate-encoding
171 2014-08-01 13:56:41 <payments> According to some the *.pem file I generated is infact ANSI X.690, can I simply include this data in the payment request?
172 2014-08-01 13:56:42 <jgarzik> etc.
173 2014-08-01 13:56:54 <payments> Ah, thank you very much.
174 2014-08-01 13:57:34 <payments> Okay I understand,
175 2014-08-01 13:59:39 <payments> jgarzik,Thank you for your help.
176 2014-08-01 14:01:52 <jgarzik> hum
177 2014-08-01 14:02:06 <jgarzik> how much memory does the block index take up?
178 2014-08-01 14:03:07 <jgarzik> It seems like we need to prune in-memory behavior more than on-disk behavior.  I cannot see needing to query blocks #1-1000 directly from RAM etc.
179 2014-08-01 14:13:27 <wumpus> the block index is not stored in memory, apart from some leveldb caching, or do you mean the block headers?
180 2014-08-01 14:14:28 <jgarzik> wumpus, mapBlockIndex
181 2014-08-01 14:15:02 <jgarzik> and associated CBlockIndex's
182 2014-08-01 14:15:14 <wumpus> right
183 2014-08-01 14:15:45 <jgarzik> block headers+++  :)
184 2014-08-01 14:15:49 <wumpus> that contains everything?
185 2014-08-01 14:16:01 <jgarzik> wumpus, yes
186 2014-08-01 14:18:18 <sipa> maybe some parts of CBlockIndex can be left on disk
187 2014-08-01 14:18:37 <sipa> but not having the actual header tree acessible would complicate things significantly
188 2014-08-01 14:18:41 <sipa> like finding forks
189 2014-08-01 14:19:45 <sipa> CBlockIndex is indeed headers + some metadata
190 2014-08-01 14:20:03 <wumpus> I suppose the most accessed headers are the recent ones
191 2014-08-01 14:20:11 <sipa> jgarzik: it's like 40 MB in memory, iirc
192 2014-08-01 14:20:19 <jgarzik> a forum poster was surprised that a current bitcoind with 30 connections took up 800MB
193 2014-08-01 14:20:19 <sipa> (including malloc overhead)
194 2014-08-01 14:20:21 <wumpus> that's peanuts compared to some other things
195 2014-08-01 14:20:29 <jgarzik> that matched my experience on my recent cloudatcost.com node.
196 2014-08-01 14:20:35 <sipa> 800 sounds right, yes
197 2014-08-01 14:20:41 <jgarzik> I was trying to quantify various major users.
198 2014-08-01 14:20:48 <jgarzik> mempool being largest
199 2014-08-01 14:20:51 <wumpus> connections have a quite high overhead too
200 2014-08-01 14:21:12 <sipa> CTransactions in memory take up many times more memory than serialized
201 2014-08-01 14:21:15 <wumpus> although 30 isn't that much
202 2014-08-01 14:21:16 <sipa> like 8 times, iirc
203 2014-08-01 14:21:30 <sipa> because of the overhead of 2 layers of vectors
204 2014-08-01 14:21:59 <wumpus> my node with 70 connections has 670MB RSS
205 2014-08-01 14:22:03 <jgarzik> IIRC, CNode got reduced.  I used to use 32MB fixed buffer per TCP can, but that got changed.
206 2014-08-01 14:22:11 <jgarzik> wumpus, 32-bit or 64-bti?
207 2014-08-01 14:22:14 <wumpus> 32 bit
208 2014-08-01 14:22:15 <sipa> ok, seems mapBlockIndex is more like 60 MB in total
209 2014-08-01 14:23:28 <sipa> at some point i want to make versions of map/set/vector/... that keep track of how much memory they 'own'
210 2014-08-01 14:23:35 <sipa> to quantify resource usage
211 2014-08-01 14:23:51 <wumpus> now that transactions are immutable, I suppose they could use a more compact format in memory
212 2014-08-01 14:24:28 <sipa> in theory
213 2014-08-01 14:24:52 <sipa> like have a single vector of CScripts, with the different inputs and outputs containing an index into that vector
214 2014-08-01 14:26:11 <wumpus> couldn't a memory profiler be used to quantify memory usage?
215 2014-08-01 14:26:52 <sipa> i meant at run time
216 2014-08-01 14:27:03 <sipa> so we could for example limit how much memory a peer can cause us to use
217 2014-08-01 14:27:12 <sipa> and deprioritize their requests over a certain value etc
218 2014-08-01 14:27:59 <wumpus> oh, more like quotas, yes then it makes sense... I meant just to count usage it shouldn't be needed to make our won version of map/set/vector
219 2014-08-01 14:33:53 <arioBarzan> if someone sends coins to a P2SH address with redeemScript not being one of standard types, are those coins going to be unspendable forever?
220 2014-08-01 14:34:41 <wumpus> isstandard can be bypassed  by finding a miner that will put it in a block for you
221 2014-08-01 14:35:48 <wumpus> if the resulting transaction is not valid, on the other hand, yes they will be unspendable forever
222 2014-08-01 14:44:55 <arioBarzan> wumpus: by isstandard you meant CTransaction::IsStandard() in main.cpp ?
223 2014-08-01 14:45:22 <sipa> standardness generally means "what is relayed"
224 2014-08-01 14:46:05 <sipa> it's stronger than just IsStandard is the source code (there are also limits on size, what features are used inside the script language, whether outputs being spent are standard...)
225 2014-08-01 14:48:54 <arioBarzan> if my redeem script be like:
226 2014-08-01 14:48:55 <arioBarzan> OP_DUP OP_HASH160 <hash_pk1> OP_EQUALVERIFY OP_CHECKSIGVERIFY <pk2> OP_CHECKSIG
227 2014-08-01 14:48:59 <arioBarzan> is it standard or not?
228 2014-08-01 14:49:07 <arioBarzan> (for a p2sh tx)
229 2014-08-01 14:49:16 <sipa> no
230 2014-08-01 14:55:55 <wumpus> not after #4365 (relax isstandard for p2sh transactions) either?
231 2014-08-01 14:56:25 <sipa> ah, maybe
232 2014-08-01 14:57:45 <wumpus> it has less than 15 signature operations, so I'd say so
233 2014-08-01 14:58:25 <sipa> yup, i believe so
234 2014-08-01 15:09:09 <amaclin> arioBarzan, 852e789d2a871dab343b3de24f3fa6109efbd44c02bc1d614342568616980285 will never be mined. you forgot the fee. Luke-Jr does not mine non-standard transactions for free
235 2014-08-01 15:10:23 <wumpus> it indeed would be difficult to make mapBlockIndex not store the entire set of blocks, esp. because CBlockIndex* are used as stable handles for a specific block throughout other data structures
236 2014-08-01 15:11:10 <sipa> we could introduce a BlockId type as a first step
237 2014-08-01 15:11:30 <sipa> and replace CBlockIndex->... calls with GetBlockIndex(BlockId)->
238 2014-08-01 15:11:38 <wumpus> right and make it a typedef at first
239 2014-08-01 15:11:39 <wumpus> sounds good
240 2014-08-01 15:12:27 <sipa> or see if some fields of CBlockIndex can become non-in-ram
241 2014-08-01 15:12:55 <arioBarzan> amaclin: I wonder how you found about 852e789d2a871dab343b3de24f3fa6109efbd44c02bc1d614342568616980285 :) how much fee I had to pay, 0.0001 or more?
242 2014-08-01 15:12:57 <wumpus> later on it could just be an increasing integer handle where the data is loaded from either MRU cache or on miss, the backend database on-demand
243 2014-08-01 15:13:09 <sipa> nFile, nDataPos, nUndoPos for sure - as those are only needed when actually loading data from disk anyway
244 2014-08-01 15:13:21 <wumpus> but yes there may be lower-hanging fruit
245 2014-08-01 15:13:24 <sipa> but that's just 3 MB
246 2014-08-01 15:13:43 <sipa> also, the phashBlock in CBlockIndex is essentially pointless
247 2014-08-01 15:14:03 <sipa> as it always points to the uint256 in the same pair<uint256, CBlockIndex> which is stored in the map
248 2014-08-01 15:14:30 <sipa> just turning it into a set<CBlockIndex>, where CBlockIndex entries hold their own hash would save us 1MB
249 2014-08-01 15:14:44 <wumpus> indeed, I wondered about that one
250 2014-08-01 15:15:28 <amaclin> arioBarzan, I do not think that this chat is good place to discuss non-standard transactions. We are disturbing developers. Ask your questions on bitcointalk.org
251 2014-08-01 15:15:29 <sipa> or we could hack it, and do some ugly pointer arithmetic to recover the position of the uint256 that precedes it
252 2014-08-01 15:15:51 <sipa> which is just a few lines of code changed, but very ugly
253 2014-08-01 15:16:23 <wumpus> using a set sounds easier to understand, and makes no assumptions about a specific in-memory format STL implementation
254 2014-08-01 15:17:02 <sipa> no need for such an assumption; you can use offsetof in map::value_type
255 2014-08-01 15:17:41 <sipa> the disadvantage for using a set is 1) more lines of code changed  2) you need to upcast a uint256 to a CBlockIndex before being able to call find
256 2014-08-01 15:18:15 <sipa> the Really Correct(tm) solution is introducing a CBlockId type that wraps a std::map<uint256, CBlockIndex*>::iterator instead
257 2014-08-01 15:18:27 <sipa> which can be used to both access the CBlockIndex entry, or access the hash
258 2014-08-01 15:18:46 <sipa> instead of just passing the CBlockIndex* entry around
259 2014-08-01 16:52:24 <payments> I have a question about BIP70, and how to debug my payment requests in BitcoinQt
260 2014-08-01 16:55:07 <payments> I have got unsigned requests working, it shows me the correct payment in the Bitcoin client, however my signed payments are coming up as unverified, even when i fill them with a duff signature?
261 2014-08-01 16:55:15 <payments> Shouldn't they not come up at all if the singature is bad?
262 2014-08-01 16:57:38 <payments> Is there anyone lurking who can help me?
263 2014-08-01 16:57:57 <sipa> patience
264 2014-08-01 16:59:48 <payments> sipa, Should I not get debug messages output to the log if there is something wrong with the request singature?
265 2014-08-01 17:00:08 <sipa> i know nothing about the gui stuff
266 2014-08-01 17:00:08 <wumpus> payments: try passing -debug=qt
267 2014-08-01 17:00:25 <payments> wumpus, thanks I shall try that.
268 2014-08-01 17:01:24 <wumpus> sipa: ooh, good point about find
269 2014-08-01 17:02:20 <payments> Thanks wumpus - so it tells me bad signature: PaymentRequestPlus::getMerchant : SSL error:  Bad signature, invalid PaymentRequest. But the payment request still comes up? Aside from me having to fix this, is this not bad behavior? Surely if it is a bad signature it should say so, not come up unverified?
270 2014-08-01 17:02:34 <payments> Otherwise you can still man-in-middle it and the user will not know?
271 2014-08-01 17:02:40 <wumpus> payments: it will show it as 'untrusted' payment request
272 2014-08-01 17:02:56 <wumpus> payments: I hope it doesn't show it in green?
273 2014-08-01 17:03:33 <payments> No it shows Unverified, in yellow, however is the point of the singature not to prevent any tampering? You can now tamper with it and the address will change, they can still pay?
274 2014-08-01 17:03:45 <payments> Would not be thoughtful to deny the request all together?
275 2014-08-01 17:03:49 <wumpus> well if you tamper it becomes unverified - that's the point
276 2014-08-01 17:04:04 <sipa> payments: someone who tampers will just remove the signature
277 2014-08-01 17:04:12 <wumpus> not really, because we want to allow non-signed payment requests as well
278 2014-08-01 17:04:27 <wumpus> right
279 2014-08-01 17:04:33 <payments> Yes but is there a difference between it signed as none vs signed but not verified?
280 2014-08-01 17:04:38 <sipa> no
281 2014-08-01 17:04:50 <sipa> because someone who tampers would just remove the signature
282 2014-08-01 17:04:53 <wumpus> no, there intentionally isn't; it won't show a merchant in either case
283 2014-08-01 17:04:58 <sipa> it is not harder than just invalidating it
284 2014-08-01 17:05:19 <payments> Hmmm, okay. Well I will now try and get my singature working. Thanks for your help guys.
285 2014-08-01 17:09:11 <payments> ???
286 2014-08-01 17:09:49 <payments> wumpus It is working after all. Looks like the trusted certificates are cached in the client, so when I added it to my root store it did not pick it up until i restarted with your recommendation of the debug parameter :)
287 2014-08-01 18:09:44 <wumpus> yes, it loads the list of root certificates at startup
288 2014-08-01 18:11:01 <justusranvier> Is there a command I can use to have bitcoind tell me why it has banned a peer?
289 2014-08-01 18:11:31 <sipa> look at the log; the dos score increases should be logged
290 2014-08-01 18:15:14 <justusranvier> sipa: The problem is the peer that is being banned is my Tor node. Once bitcoind decides to ban that IP address, then I can't receive *any* incoming hidden service connections.
291 2014-08-01 18:19:32 <sipa> ah
292 2014-08-01 18:19:53 <sipa> tor is assumed to be running locally, as localhost is never banned
293 2014-08-01 18:21:05 <justusranvier> I'd never run a tor node on the same machine as a bitcoin node. Actually, I'd never run any to network-facing servers on the same machine.
294 2014-08-01 18:21:10 <justusranvier> any two
295 2014-08-01 18:23:25 <wumpus> the git version has configurable whitelisting functionality
296 2014-08-01 18:23:36 <sipa> you do not want to whitelist tor
297 2014-08-01 18:24:07 <wumpus> so tor on localhost is still a special case? okay
298 2014-08-01 18:24:38 <sipa> whitelisting removes both disconnecting and banning
299 2014-08-01 18:24:49 <sipa> for tor, you want the disconnecting but not banning
300 2014-08-01 18:25:09 <sipa> there is an exception still that localhost is never banned, even if you do disconnect
301 2014-08-01 18:25:42 <justusranvier> Maybe that exception should be configurable to include multiple hosts
302 2014-08-01 18:25:45 <wumpus> ok, understood, I had hoped the whitelisting change would have made this possible
303 2014-08-01 18:26:03 <justusranvier> Or, even better, paramaters like whitelist, max connections, etc should be configurable per-interface
304 2014-08-01 18:26:14 <sipa> i admit i hadn't considered the use case of running tor on non-localhost
305 2014-08-01 18:26:49 <wumpus> meh, no reason to make this too complex
306 2014-08-01 18:26:55 <justusranvier> Given the low overhead of virtualization/containers, there's no reason to run different servers on the same (virtual) machine
307 2014-08-01 18:27:02 <wumpus> I don't want a whole firewall in bitcoind
308 2014-08-01 18:27:27 <justusranvier> I don't want a p0wned bitcoind to compromise everything else, or vice-verse
309 2014-08-01 18:29:15 <wumpus> that's up to you, but at a certain point you're the only person to only ever use certain functionality, and it's just not worth the maintenance and testing overhead
310 2014-08-01 18:29:28 <justusranvier> For that matter, Tor connections reach my bitcoind node via a different network interface than external ipv4 connections, and the lan is yet a different interface. I'd like to be able to configure different connections limits for each interface independently.
311 2014-08-01 18:29:55 <wumpus> you could use a ssh tunnel to work around it
312 2014-08-01 18:30:01 <justusranvier> So that Armory can always connect, even if the external ipv4 quota is full
313 2014-08-01 18:30:49 <wumpus> that is possible with the current whitelisting functionality by adding a specific port for whitelisted connections
314 2014-08-01 18:31:29 <wumpus> armory, in contrast to tor, can be whitelisted
315 2014-08-01 18:32:09 <wumpus> (and yes, the whitelisted port can bind on another interface)
316 2014-08-01 18:34:45 <wumpus> you could add a -torbind that is like -whitebind but does disconnect in misbehavior (but not ban), but it's really obscure
317 2014-08-01 18:36:54 <safinaskar-i> hi
318 2014-08-01 19:06:10 <Luke-Jr> 
ACTION wonders if %0*lx works on Windows

319 2014-08-01 19:27:31 <earlz> So, not to start any flames, but does anyone have an opinion on Etherium? I personally think it's an elaborate scam, but apparently some bitcoin foundation people are involved
320 2014-08-01 19:27:59 <sipa> #bitcoin please
321 2014-08-01 19:28:19 <earlz> #bitcoin is just 1000 people lurking and never saying anything lol
322 2014-08-01 19:28:44 <sipa> i don't care
323 2014-08-01 19:28:51 <earlz> and I mean more from a technical standpoint, like if some of their ideas are even possible
324 2014-08-01 19:29:37 <denisx> sipa: would a change of the dbcache parameter explain my increase of disk IO?
325 2014-08-01 19:32:05 <sipa> denisx: lower cache, more writes, for sure
326 2014-08-01 19:32:27 <denisx> sipa: ok, then this mistery is solved I guess ;)
327 2014-08-01 19:46:53 <iwilcox> sipa: Actually if folks ask that on #bitcoin they're likely to be quickly punted to #ethereum :)
328 2014-08-01 19:50:24 <earlz> I'm really curious to have an unbiased technical discussion about it, without all the marketing buzz words lol
329 2014-08-01 19:55:03 <gmaxwell> earlz: This is not the place for it. Please don't continue here.
330 2014-08-01 19:55:27 <earlz> ok that's understandable
331 2014-08-01 21:10:43 <Forex> is it normal that abe explorer sometimes wont respond to ctrl +c ?
332 2014-08-01 21:10:44 <Forex> :D
333 2014-08-01 21:34:22 <bonks> Anyone know what format the bitcoin wallet backup is stored? I want to extract the private keys
334 2014-08-01 21:35:12 <gmaxwell> bonks: wallet backup is just the wallet format, its not a seperate format.
335 2014-08-01 21:37:07 <Luke-Jr> bonks: I wouldn't advise that btw.
336 2014-08-01 21:37:20 <sipa> use dumpwallet instead of backupwallet
337 2014-08-01 21:37:24 <sipa> it's human readable
338 2014-08-01 21:38:08 <Forex> ;)
339 2014-08-01 21:44:41 <bonks> Oh perfect... Luke-Jr I'm just exploring things with minimum risk :) Thx guys
340 2014-08-01 21:45:00 <Luke-Jr> bonks: managing keys by hand is never minimum risk, but ok ☺
341 2014-08-01 21:46:53 <bonks> Luke-Jr: Backups with few satoshis I want to move, even less than the fee
342 2014-08-01 21:47:30 <bonks> Oh crap I thought I mentioned android wallet
343 2014-08-01 21:47:41 <bonks> The android wallet by andreas is what I'm talking about :x
344 2014-08-01 21:47:46 <Luke-Jr> :/
345 2014-08-01 21:48:48 <bonks> Any idea on that format?
346 2014-08-01 21:49:25 <sipa> yes, the same as bitcoind's dumpwallet
347 2014-08-01 21:49:36 <bonks> Ok cool
348 2014-08-01 21:49:36 <sipa> but encrypted using openssl's aeb
349 2014-08-01 21:49:38 <sipa> *aes
350 2014-08-01 21:50:01 <bonks> ty
351 2014-08-01 22:50:05 <sipa> log2_work=79.999201
352 2014-08-01 23:11:24 <gribble> 1.8736441558310238E10
353 2014-08-01 23:11:24 <sipa> ;;diff