1 2013-07-30 12:12:34 <jgarzik> sipa, are there any obvious to-dos blocking merging, for your watch-only address pull?
2 2013-07-30 12:12:42 <jgarzik> sipa, light testing seems OK here
3 2013-07-30 12:14:07 <sipa> jgarzik: not really, i think
4 2013-07-30 12:16:35 <jgarzik> sipa, does 'listunspent' provide any way to distinguish between watch-only and spendable?
5 2013-07-30 12:17:05 <sipa> jgarzik: no, perhaps it should
6 2013-07-30 12:17:09 <jgarzik> sipa, AvailableCoins() just returns all txouts, spendable or watch, AFAICS
7 2013-07-30 12:17:25 <sipa> jgarzik: it takes a boolean to select only spendable or not
8 2013-07-30 12:18:03 <jgarzik> sipa, related,
9 2013-07-30 12:18:11 <jgarzik> -AvailableCoins(vCoins);
10 2013-07-30 12:18:20 <jgarzik> +AvailableCoins(vCoins, true);
11 2013-07-30 12:18:32 <jgarzik> sipa, that changes onlyconfirmed NOT requireSpendable. Intended?
12 2013-07-30 12:18:39 <jgarzik> sipa, AvailableCoins has two optional bool args
13 2013-07-30 12:19:01 <sipa> jgarzik: nice catch, not intended
14 2013-07-30 12:25:07 <jgarzik> char fYes;
15 2013-07-30 12:25:11 <jgarzik> ssValue >> fYes;
16 2013-07-30 12:25:23 <jgarzik> ACTION could have sworn we support serializing booleans directly
17 2013-07-30 12:27:49 <sipa> jgarzik: the serialization of primitive types is done by just memory-copying them (*DIE*)... and bools do not have well-defined memory representation i think (and it may be much larger than 1 byte if it does)
18 2013-07-30 12:28:45 <edcba> and what about 2 bools...
19 2013-07-30 12:35:44 <jgarzik> sipa, bools have a well-defined memory representation, but not a well-defined size :)
20 2013-07-30 12:35:54 <jgarzik> (zero is false, non zero of any value or size is true)
21 2013-07-30 12:36:13 <jgarzik> (in terms of strict compiler/ABI definitions)
22 2013-07-30 12:36:46 <sipa> jgarzik: sure about that; the semantics for converting to an integer are defined
23 2013-07-30 12:40:42 <Vinnie_win> price is inching up so whatever you guys are doing, keep doing it
24 2013-07-30 12:52:28 <runeks> Anyone have a testnet address that I can test the watch-only functionality with? I can't find any block explorers that support testnet (blockexplorer.com is outdated)
25 2013-07-30 13:05:02 <TD> runeks: there's testnet.btclook.com but it's not that great
26 2013-07-30 13:06:00 <runeks> TD: Thanks. I can't find my address on it though. I'm just gonna move testnet wallet.dat and try to generate some new coins.
27 2013-07-30 13:06:38 <lianj> http://tpfaucet.appspot.com/
28 2013-07-30 13:08:22 <runeks> lianj: Cool!
29 2013-07-30 13:21:33 <jgarzik> Some people focused on tracking the bitcoins used at Silk Road. An interesting blow-by-blow from Krebs, on how some cyber-stalkers tried to set him up as a druggie: http://krebsonsecurity.com/2013/07/mail-from-the-velvet-cybercrime-underground/
30 2013-07-30 13:34:22 <TD> jgarzik: lol. krebs is such a badass. even if he does believe that censoring payments is some silver bullet for fighting crime
31 2013-07-30 13:36:03 <petertodd> jgarzik: nifty, I was wondering when someone would start doing third-party buys on silk road...
32 2013-07-30 13:36:26 <gmaxwell> petertodd: if only the USPS had P2SH^2
33 2013-07-30 13:36:59 <petertodd> <slow clap>
34 2013-07-30 13:37:29 <petertodd> of course, if they did, that'd just make it even harder to deny that the shipment was his...
35 2013-07-30 13:37:30 <gmaxwell> hah. but man, I didn't want to know about that, yet another reason to worry about random people getting my mailing address.
36 2013-07-30 13:37:49 <petertodd> indeed, unfortunately I was stupid about whois ages ago... oh well
37 2013-07-30 13:37:53 <petertodd> I can always move
38 2013-07-30 13:37:56 <petertodd> :P
39 2013-07-30 13:40:02 <jgarzik> Being a property owner, you're just f*cked
40 2013-07-30 13:40:14 <jgarzik> My name and address are listed in public records all over the place
41 2013-07-30 13:40:18 <petertodd> For sure
42 2013-07-30 13:40:49 <petertodd> Hmm... you could always rent from yourself, and claim you live elsewhere :P
43 2013-07-30 13:40:54 <jgarzik> Only way to avoid that is HNW tricks like having corporations own your assets
44 2013-07-30 13:41:00 <jgarzik> yep
45 2013-07-30 13:41:13 <gmaxwell> doesn't prevent someone from sending it there otherwise. The problem with having a corp own your home is crap like not getting the homestead property tax exclusion.
46 2013-07-30 13:41:14 <petertodd> tricks == legal fees...
47 2013-07-30 13:41:29 <jgarzik> best security is to rent for cash from someone else
48 2013-07-30 13:41:39 <jgarzik> and own property as a Nevada etc. corporation if you must
49 2013-07-30 13:41:50 <jgarzik> being a Property Owner makes you findable and assailable, ultimately
50 2013-07-30 13:42:06 <TD> i am thinking it's only a matter of time before some researcher is able to cluster the silk road wallets. co-operation with the major exchanges and rounding up of a bunch of dealers/buyers is surely the next step
51 2013-07-30 13:42:07 <kjj> meh. the homestead credit on my house is like $250 a year
52 2013-07-30 13:42:49 <kjj> the harder part is finding a bank that will write a mortgage to your corporation
53 2013-07-30 13:43:44 <jgarzik> OK, this is a crypto/P2P problem. How to arrange a chain of semi-trusted property ownership, where A owns B's property, B owns C's property, C owns A's property etc. in a way that (a) shrouds ownership but (b) preserves property ownership wealth :)
54 2013-07-30 13:43:44 <petertodd> TD: if you think that's likely you basically think everyone will be snared - easywallet and inputs.io make it dead easy to mix any wallet with their funds anonymously
55 2013-07-30 13:43:59 <petertodd> jgarzik: zerocoin for land titles?
56 2013-07-30 13:44:03 <jgarzik> exactly :)
57 2013-07-30 13:44:23 <jgarzik> maybe if the titles were stored in a chain.....
58 2013-07-30 13:44:26 <TD> easywallet and inputs.io are very new. plus, i doubt it will be used as the only evidence and i doubt most people who show up in the records/SAR dumps would actually be investigated or prosecuted
59 2013-07-30 13:44:35 <TD> but sure. that's one more excellent reason to not use a bitbank
60 2013-07-30 13:44:37 <petertodd> jgarzik: I've got a relative who's a land titles lawyer, I'll tell her that...
61 2013-07-30 13:44:50 <TD> (inputs.io is going to get shut down at some point)
62 2013-07-30 13:44:56 <petertodd> TD: and instawallet before that, and other services
63 2013-07-30 13:45:19 <jgarzik> TD, it drives me crazy when coindesk.com reviews bitcoin apps, and completely fails to mention which ones require centralized websites, and which ones (bitcoinj-based) are fully decentralized.
64 2013-07-30 13:45:29 <jgarzik> Some portions of the community Just Don't Care about being decentralized :(
65 2013-07-30 13:45:29 <petertodd> TD: and just getting a payment from a bitbank puts you in that group, so you're not any safer
66 2013-07-30 13:45:34 <TD> most people don't care/can't tell the difference. that review was weak anyway. it more or less just summarised feature sets
67 2013-07-30 13:46:06 <petertodd> TD: not to mention trust-free mixing... especially if it gets automated
68 2013-07-30 13:46:10 <TD> petertodd: if i were doing that work i'd just look for owners of addresses that came directly from the SR wallet. unless the SR guys are really excellent bitcoin programmers these days, i am willing to bet they leak data like a seive
69 2013-07-30 13:46:22 <TD> we're talking about the status quo, right, not theoretical future code
70 2013-07-30 13:46:26 <TD> anyway, food time
71 2013-07-30 13:46:43 <petertodd> TD: we keep on seeing attacks getting done by people who are likely very good Bitcoin programmers, and we don't have a !@#$ clue who is doing it.
72 2013-07-30 13:47:09 <jgarzik> petertodd, insights can be gained with additional effort? but ultimately that is a problem that cannot be solved
73 2013-07-30 13:47:22 <TD> you are not the police and you're not talking about blockchain analysis
74 2013-07-30 13:47:27 <jgarzik> petertodd, without compromising bitcoin privacy/pseudononimity completely, at least
75 2013-07-30 13:47:58 <kjj> the NSA could probably bust SR wide open. they presumably have copies of all dynamic page loads and would be able to follow addresses around
76 2013-07-30 13:48:03 <petertodd> jgarzik: My point is we are *not* the only people out there who understand Bitcoin well, to assume SR doesn't have good bitcoin programmers is naive.
77 2013-07-30 13:48:27 <petertodd> jgarzik: possibly true too, but we do not know one way or the other
78 2013-07-30 13:48:47 <jgarzik> any large Tor dragnet should be able to home in on SR, IMO
79 2013-07-30 13:49:06 <jgarzik> but sure, there are smart people all over
80 2013-07-30 13:49:08 <petertodd> jgarzik: Well, home in on their servers, which != them.
81 2013-07-30 13:50:21 <petertodd> jgarzik: Mexican drug lords manage to get the engineering talent to build submarines as capable as WWI subs, and increasingly WWII, and they do it for a lot less money too... they also have decentralized packet radio systems - helps when you can kidnap cellphone engineers.
82 2013-07-30 13:51:21 <petertodd> It's the standard Tor argument that criminal enterprises already have lots of ways to get good anonymity... what Tor provides is a level playing field.
83 2013-07-30 13:53:58 <jgarzik> Random: I wondered the other day if anybody has ever done a design for a high latency, high bandwidth network -- like that which could be implemented using flash drives conveyed across the country by long-haul truckers or airplanes
84 2013-07-30 13:54:31 <gwillen> jgarzik: yes, the Deep Space Network has such a design
85 2013-07-30 13:54:32 <jgarzik> you would never observe the network over the public Internet
86 2013-07-30 13:54:50 <gwillen> I mean, it's obviously carried over actual radio waves
87 2013-07-30 13:54:55 <jgarzik> and you can sent/receive huge amounts of data? as long as you're willing to wait a week for it
88 2013-07-30 13:54:58 <gwillen> but it's designed to cope with stupidly large latency
89 2013-07-30 13:55:18 <gwillen> it uses a NAK-based system for bulk data transfer
90 2013-07-30 13:55:20 <gwillen> and retransmission
91 2013-07-30 13:57:27 <gmaxwell> IIRC freenet was talking about some kind of usb relayed mode at some point. I don't recall if it was proposed for freenet proper or something else.
92 2013-07-30 13:57:37 <petertodd> jgarzik: Not only designed, but implemented: FidoNet bbs messaging...
93 2013-07-30 13:57:50 <petertodd> jgarzik: and UUCP
94 2013-07-30 13:58:40 <jgarzik> petertodd, haha, point :)
95 2013-07-30 13:58:41 <petertodd> jgarzik: One laptop per child project has done work in that field too, IIRC at one point they were using UUCP for something
96 2013-07-30 13:58:46 <jgarzik> ACTION -> WWIVnet veteran
97 2013-07-30 13:59:34 <petertodd> ACTION started using BBSes when all the oldies on BBSs were talking about how they were dying off...
98 2013-07-30 13:59:54 <gwillen> ahah, I found it: http://public.ccsds.org/publications/archive/727x0b2s.pdf
99 2013-07-30 13:59:59 <gwillen> the CCSDS File Delivery Protocol
100 2013-07-30 14:00:10 <gwillen> this is how spacecraft (used to?) send each other files
101 2013-07-30 14:00:50 <gwillen> haha
102 2013-07-30 14:00:50 <petertodd> gwillen: guarantee you they still do, and the vendors proudly talk about the "heritage" status of their CCSDS implementations
103 2013-07-30 14:00:51 <gwillen> ACTION nods
104 2013-07-30 14:01:10 <gwillen> I interned at Northrop Grumman Space Technology many years ago, when I was a freshman in college
105 2013-07-30 14:01:14 <gwillen> I had to give a presentation on this protocol
106 2013-07-30 14:01:15 <petertodd> gwillen: their competitors warn that CCSDS is new and still poorly tested technology :P
107 2013-07-30 14:01:19 <gwillen> hahahahhaah
108 2013-07-30 14:01:39 <gwillen> actually, I guess when I was a freshman in college, this protocol _was_ new and poorly tested technology
109 2013-07-30 14:01:43 <gwillen> I wonder how it's held up.
110 2013-07-30 14:02:08 <petertodd> heh
111 2013-07-30 14:17:26 <runeks> sipa, jgarzik: Testing a mixed watch-only wallet showed no bugs. Sending amounts higher than what is available in addresses for which you have the private keys results in a "Signing transaction failed" error.
112 2013-07-30 14:18:44 <jgarzik> runeks, thanks much for the testing. please post these results to the pull req.
113 2013-07-30 14:18:57 <runeks> jgarzik: Done
114 2013-07-30 15:23:33 <sipa> ;;genrate 9800
115 2013-07-30 15:23:34 <gribble> The expected generation output, at 9800.0 Mhps, given difficulty of 31256960.7278, is 0.157676362201 BTC per day and 0.00656984842505 BTC per hour.
116 2013-07-30 15:30:40 <c0rw1n> ;;genrate 2330
117 2013-07-30 15:30:42 <gribble> The expected generation output, at 2330.0 Mhps, given difficulty of 31256960.7278, is 0.0374883595846 BTC per day and 0.00156201498269 BTC per hour.
118 2013-07-30 15:45:30 <bmcgee> So Thailand??? anyone bothered?
119 2013-07-30 15:46:51 <jgarzik> yawn
120 2013-07-30 15:47:00 <bmcgee> not into lady boys then?
121 2013-07-30 15:47:04 <jgarzik> one data point does not a "complete ban in Thailand" make
122 2013-07-30 15:47:10 <bmcgee> lol
123 2013-07-30 15:47:19 <jgarzik> single sourced story getting hyped
124 2013-07-30 15:47:41 <bmcgee> yeah the comments section do a good job of ripping the article to pieces
125 2013-07-30 15:48:46 <edcba> nice we may still be able to do sexual tourism with bitcoins
126 2013-07-30 15:48:46 <sipa> it's also not at all a ban
127 2013-07-30 15:48:56 <sipa> it's the national bank considering it illegal
128 2013-07-30 15:49:08 <sipa> there is no court or legislative body involved, afaik
129 2013-07-30 15:52:39 <midnightmagic> which article's comments are you talking about?
130 2013-07-30 15:52:53 <midnightmagic> the story is in like a dozen places now.
131 2013-07-30 15:53:47 <midnightmagic> bmcgee: ^^
132 2013-07-30 15:54:03 <bmcgee> 2 secs let me open it again
133 2013-07-30 15:54:07 <bmcgee> found it via reddit
134 2013-07-30 15:54:17 <bmcgee> http://www.telegraph.co.uk/finance/currency/10210022/Bitcoins-banned-in-Thailand.html
135 2013-07-30 15:55:51 <edcba> advisement...
136 2013-07-30 15:57:28 <edcba> they advised it was illegal ? :)
137 2013-07-30 15:57:53 <edcba> smells fishy
138 2013-07-30 15:59:42 <Krellan> ;;genrate 3840
139 2013-07-30 15:59:44 <gribble> The expected generation output, at 3840.0 Mhps, given difficulty of 31256960.7278, is 0.0617833909033 BTC per day and 0.00257430795431 BTC per hour.
140 2013-07-30 16:00:14 <edcba> how much is a gpu ?
141 2013-07-30 16:01:29 <sipa> edcba: between 1 and 800 MH/s or so
142 2013-07-30 16:01:40 <edcba> ;;genrate 500
143 2013-07-30 16:01:41 <gribble> The expected generation output, at 500.0 Mhps, given difficulty of 31256960.7278, is 0.00804471235721 BTC per day and 0.000335196348217 BTC per hour.
144 2013-07-30 16:02:29 <edcba> a block every 10 years ?
145 2013-07-30 16:02:51 <edcba> not taking account reward halving...
146 2013-07-30 16:02:53 <edcba> damn lol
147 2013-07-30 16:12:28 <Krellan> yes, the real-world payout is even much less than what ;;genrate will estimate, because it doesn't take into account 1) halving block rewards over time and 2) everybody else adding more hashpower over time.
148 2013-07-30 16:13:09 <arioBarzan> sipa: could fRequireSpendable get avoided by locking watch-only coins in a list similar to CCryptoKeyStore::setLockedCoins, which could be called CCryptoKeyStore::setWatchedCoins?
149 2013-07-30 16:24:08 <sipa> arioBarzan: i find that interface very ugly
150 2013-07-30 16:24:32 <sipa> arioBarzan: as it requires a single context-independent state to be maintained
151 2013-07-30 16:25:16 <sipa> is there a problem with fRequireSpendable, that you'd want to avoid it?
152 2013-07-30 16:27:44 <sipa> also, watched/spendable is not a user-settable property; it's the result of how they were matched
153 2013-07-30 16:29:22 <arioBarzan> sipa: as an adventurous dev, I find pull/2121 plus a lockedCoins list for watch-only addresses less complicated to undertand the code, comparing with your solution. Besides, such a list might cause less overhead.
154 2013-07-30 16:30:15 <sipa> i don't understand how it would work
155 2013-07-30 16:31:21 <arioBarzan> sipa: However, I'm not in a position to see any problem with fRequireSpendable. I tested today, and I got error: {"code":-4,"message":"Signing transaction failed"}
156 2013-07-30 16:31:54 <sipa> who would set this watched coins list?
157 2013-07-30 16:32:12 <arioBarzan> not the user
158 2013-07-30 16:32:27 <arioBarzan> it should be set automatically when wallet loads
159 2013-07-30 16:32:32 <sipa> to what?
160 2013-07-30 16:33:18 <arioBarzan> we just add those coins to a second locklist.
161 2013-07-30 16:33:28 <sipa> so how do you do the matching?
162 2013-07-30 16:33:58 <sipa> when a new transaction is seen, how do you avoid passing information to/from the matching code in script to know if it's spendable or not?
163 2013-07-30 16:34:44 <sipa> imho, that's a very ugly workaround for avoiding a single parameter
164 2013-07-30 16:35:06 <sipa> as you need more state that needs to be kept consistent with the rest of the data structures
165 2013-07-30 16:35:25 <sipa> and you still need to distinguish between spendable and non-spendable when matching
166 2013-07-30 16:37:14 <arioBarzan> we already distinguish spendables with lockunspent.
167 2013-07-30 16:37:33 <sipa> yes, and it requires extra state
168 2013-07-30 16:37:37 <jgarzik> arioBarzan, that is a temporary list, rebuild at each startup
169 2013-07-30 16:37:39 <sipa> which makes sense if it comes from a user
170 2013-07-30 16:37:46 <jgarzik> *rebuilt
171 2013-07-30 16:37:55 <arioBarzan> jgarzik: I noticed that it is tomporary.
172 2013-07-30 16:38:26 <sipa> i personally would rather get rid of the lockedunspent global too, and have it passed as an argument into the code to build a list of spendables
173 2013-07-30 16:38:52 <sipa> (so for example the RPC locked and GUI locked aren't necessarily shared)
174 2013-07-30 16:39:39 <sipa> but please, let's no keep extra state that must be kept consistent with the rest of the data
175 2013-07-30 16:39:51 <sipa> there are just two use cases
176 2013-07-30 16:42:17 <arioBarzan> are the coins saved in wallet.dat?
177 2013-07-30 16:42:26 <sipa> no
178 2013-07-30 16:42:57 <sipa> but you still haven't answered: how would you know whether a new coin should be added to the list, when a new incoming transaction occurs?
179 2013-07-30 16:43:14 <sipa> you need a way for the matching logic in script to distringuish between the two possibilities
180 2013-07-30 16:44:02 <arioBarzan> I'm too confused. I need to learn much more to keep up in such discusions.
181 2013-07-30 16:44:59 <sipa> jgarzik: hmmz, adding a flag in listunspent is more work than i thought
182 2013-07-30 16:45:30 <sipa> one way would be have IsMine (in all its versions) return some enum {NO, YES_SPENDABLE, YES_WATCHONLY}
183 2013-07-30 16:46:00 <sipa> the easier to implement solution is have listunspent take an argument to require spendability or not
184 2013-07-30 16:46:30 <jgarzik> sipa, the latter seems reasonable, and was along the lines of what I was presuming
185 2013-07-30 16:47:08 <jgarzik> sipa, IsMine() would be nice, but for practical reasons you would be touching (or at least auditing) a long of key callsites
186 2013-07-30 16:47:15 <jgarzik> *lot of
187 2013-07-30 16:48:18 <sipa> jgarzik: well, i'm already touching those sites - they now all got a fRequireSpendable flag
188 2013-07-30 16:49:31 <jgarzik> true
189 2013-07-30 16:50:02 <sipa> arioBarzan: if the code is hard to follow, and you have ways to improve that (for example, by adding comments about things you found out while learning), that's very welcome
190 2013-07-30 16:50:17 <sipa> arioBarzan: i don't think anyone will claim the code is very well structured
191 2013-07-30 16:51:20 <kjj> sipa: the code is fiendishly hard for non-C++ programmers to follow, but comments won't help that.
192 2013-07-30 16:52:59 <sipa> my god... http://coinchoose.com/
193 2013-07-30 16:53:44 <sipa> i'm most amazed by the fact that they're all using scrypt
194 2013-07-30 16:53:59 <sipa> i wonder whether they have at least higher memory usage that litecoin's...
195 2013-07-30 16:54:01 <gjs278> using jquery 2 is such aids
196 2013-07-30 16:54:16 <gjs278> I can't believe they couldn't at least put the minimal fallbacks for older browsers so it just doesn't make the page entirely white
197 2013-07-30 16:54:30 <jgarzik> as long as their not hit by botnets (something easy to accomplish once their size reaches a level), it keeps a tiny little moat for a tiny little altcoin
198 2013-07-30 16:54:38 <jgarzik> scrypt, that is
199 2013-07-30 16:56:14 <nsh> ACTION thinks
200 2013-07-30 16:56:34 <gjs278> there's too little cash in any of these unfortunately. you can only gpu mine at best and earn like $3 a day on any of these altcoins
201 2013-07-30 16:57:16 <arioBarzan> sipa: in CWallet::AvailableCoins we have: if (!(pcoin->IsSpent(i)) && IsMine(pcoin->vout[i]) && !IsLockedCoin((*it).first, i) && pcoin->vout[i].nValue > 0) vCoins.push_back(COutput(pcoin, i, pcoin->GetDepthInMainChain()));
202 2013-07-30 16:57:17 <sipa> i don't understand what 'profitability' mean when the hash algorithm is not the same
203 2013-07-30 16:57:43 <sipa> arioBarzan: yes, but that's the wallet side of things
204 2013-07-30 16:57:44 <arioBarzan> sipa: couldn't we there check watch-only locked also?
205 2013-07-30 16:57:56 <sipa> arioBarzan: well not if you don't know what is watch-only
206 2013-07-30 16:58:06 <sipa> and the matching of scripts with the keystore happens in script
207 2013-07-30 16:58:32 <sipa> you need *some* way of communicating - either from or to - the matching logic there that an output is watch-only
208 2013-07-30 16:59:46 <sipa> and at that point, it is a property of the output that it is watch-only
209 2013-07-30 16:59:55 <sipa> it's not something you can check after the fact
210 2013-07-30 16:59:56 <arioBarzan> sipa: when we lock those unspent coins, why we don't face similar problem of matching script ?
211 2013-07-30 17:00:25 <sipa> arioBarzan: because it's not the script matching logic that needs to decide whether a coin is locked or not, but the user!
212 2013-07-30 17:01:45 <sipa> being spendable or not is a property of the coin
213 2013-07-30 17:01:52 <sipa> being locked or not is a user setting
214 2013-07-30 17:02:26 <sipa> and yes, you could deal with it in a similar way, if you had a guaranteed-up-to-date list of unspendable coins in your wallet
215 2013-07-30 17:02:29 <arioBarzan> sipa: but it doesn't have to be a user setting to lock those watch-only coins. the code do that automaticaly.
216 2013-07-30 17:02:35 <sipa> but now you've just moved the problem
217 2013-07-30 17:02:53 <sipa> now you need a way for the script matching logic to inform the wallet that a certain coin is not spendable
218 2013-07-30 17:03:27 <arioBarzan> we already have it, no need for nothing new.
219 2013-07-30 17:03:27 <sipa> which makes no sense, as it's the wallet that should ask things from the matcher, not the other way around
220 2013-07-30 17:03:34 <sipa> ...
221 2013-07-30 17:03:46 <sipa> sorry, i think you need to understand the code better before we can continue this discussion
222 2013-07-30 17:03:53 <arioBarzan> ok
223 2013-07-30 17:03:53 <sipa> it makes no sense
224 2013-07-30 17:04:33 <sipa> < arioBarzan> sipa: but it doesn't have to be a user setting to lock those watch-only coins. the code do that automaticaly.
225 2013-07-30 17:04:58 <sipa> ^- that 'automatically' part now becomes the problem, and i guarantee you that it'll be much uglier than what we have now
226 2013-07-30 17:04:59 <arioBarzan> sipa I meant code (could) do that
227 2013-07-30 17:07:10 <arioBarzan> sipa: I appreciate your clarification. I study the code more and will come back to you if I got a proper hang of it > Thanks
228 2013-07-30 18:13:42 <rdymac> MagicalTux, does the confirmation email for new users in the wiki work?
229 2013-07-30 18:14:07 <sipa> have you checked your spam folder?
230 2013-07-30 18:14:34 <Luke-Jr> rdymac: you're not the only one to have problems
231 2013-07-30 18:14:37 <rdymac> it is not for me, it is for a few new users asking for the email
232 2013-07-30 18:15:03 <rdymac> they had even paid the fee becore receiving the confirmation email
233 2013-07-30 18:15:12 <Luke-Jr> probably someone needs to volunteer to maintain the technical end of the wiki
234 2013-07-30 18:15:24 <Luke-Jr> I think MagicalTux is too busy to give it the attention it needs
235 2013-07-30 18:15:26 <Luke-Jr> :/
236 2013-07-30 18:15:28 <rdymac> Luke-Jr, is MagicalTux up to the problem?
237 2013-07-30 18:15:52 <Luke-Jr> rdymac: usually it's a topic for #bitcoin-wiki, but he's here too
238 2013-07-30 18:16:02 <Luke-Jr> I'm sure he's capable of it, just lacking in time
239 2013-07-30 18:16:15 <rdymac> oh sorry, didn't know about that channel
240 2013-07-30 18:17:06 <rdymac> Luke-Jr: I mean, "does he know about the existance of the problem" not if he is able to fix it
241 2013-07-30 18:18:39 <Luke-Jr> rdymac: I'm pretty sure he's aware
242 2013-07-30 18:19:54 <rdymac> ok! thanks. I'll tell that to the users
243 2013-07-30 18:24:52 <Luke-Jr> rdymac: if they have control over their email server, it may be a SPF issue
244 2013-07-30 20:28:42 <Luke-Jr> first country to ban Bitcoin: https://bitcoin.co.th/trading-suspended-due-to-bank-of-thailand-advisement/
245 2013-07-30 20:29:11 <lianj> good luck to them
246 2013-07-30 20:29:46 <sipa> been mentioned here before; afaik it's just the national bank refusing to give a license because they consider bitcoin illegal
247 2013-07-30 20:30:05 <Luke-Jr> same thing?
248 2013-07-30 20:30:23 <c0rw1n> oh so it's just business as usual : can't run a hosted for-profit exchange legally
249 2013-07-30 20:30:23 <sipa> the bank does not write the law
250 2013-07-30 20:30:59 <sipa> they can stop an exchange, sure, but them saying that trading bitcoin is illegal has no meaning
251 2013-07-30 20:31:15 <sipa> but IANA(thay)L
252 2013-07-30 20:31:23 <Luke-Jr> hm
253 2013-07-30 20:31:54 <c0rw1n> suffices that someone encodes "THAI KING SUCKS LOL" in the blockchain and boom it's illegal
254 2013-07-30 20:32:28 <sipa> (maybe they are right that it is illegal according to the thai law, but at this point, that's just the bank's opinion)
255 2013-07-30 20:42:42 <petertodd> c0rwin: 96eaa9436b05e0cc1932e0a04866656daa11e348d0bdff94989b7aded1f21183
256 2013-07-30 20:44:26 <petertodd> (I don't quite have the heart to put it in the UTXO set permanently)
257 2013-07-30 20:45:10 <c0rw1n> i'm not going to
258 2013-07-30 20:45:31 <petertodd> c0rwin: that was a txid fwiw
259 2013-07-30 20:46:13 <c0rw1n> blockchain doesn't recognize it
260 2013-07-30 20:46:18 <c0rw1n> erm
261 2013-07-30 20:46:20 <c0rw1n> blockchain.info doesn't recognize it
262 2013-07-30 20:46:36 <petertodd> it's a 1-of-2 multisig, blockchain doesn't show them until they get mined
263 2013-07-30 20:46:56 <petertodd> the other key is the compressed version of the correct horse addr
264 2013-07-30 20:47:49 <iwilcox> Heh, so you just *really* made Bitcoin illegal in Thailand?
265 2013-07-30 20:48:27 <petertodd> iwilcox: Not quite - I did give anyone the ability to take it out of the UTXO set. Pretty close though.
266 2013-07-30 20:48:44 <petertodd> iwilcox: Remind me to put "THAI KING SUCKS LOL" in the coinbase of my next alt-coin.
267 2013-07-30 20:49:04 <petertodd> Also remind me not to visit thailand...
268 2013-07-30 20:49:14 <iwilcox> Let the record show that there was a window in which the only basis for the "illegal" claim was a crappy exchange with a domain registered since June and zero substantiation of the claim.
269 2013-07-30 20:49:26 <lianj> its mined
270 2013-07-30 20:49:44 <petertodd> nice!
271 2013-07-30 20:53:13 <c0rw1n> is it worth announcing?
272 2013-07-30 20:54:37 <c0rw1n> hmm
273 2013-07-30 20:54:51 <iwilcox> IMHO it'd be better left until after the Thai gummint make a sensible decision :)
274 2013-07-30 20:55:39 <sipa> we need to start storing block and utxo records xored with some stream cipher output
275 2013-07-30 20:55:42 <c0rw1n> petertodd hey, do kim-jong next
276 2013-07-30 20:56:51 <sipa> the gunmint? :o
277 2013-07-30 20:59:10 <petertodd> sipa: sadly
278 2013-07-30 20:59:40 <petertodd> sipa: and make bare checkmultisig and pay-to-pubkey non-standard
279 2013-07-30 21:00:25 <petertodd> c0rw1n: donate 1mBTC 1JVdnaAbfGTbdatLMtLaXX5QMiHZJXp4DD
280 2013-07-30 21:01:20 <c0rw1n> 'k, wait
281 2013-07-30 21:02:03 <c0rw1n> sent
282 2013-07-30 21:02:17 <petertodd> also, while we're having fun, for those people running dumb brainwallet address scanners, you can make $50 if you add checkmultisig support
283 2013-07-30 21:04:12 <c0rw1n> petertodd 2c6c66ca4f47d672b3298e9f65fe66d02bb3f297c3b2725753ff66182c63e559 <- tx id
284 2013-07-30 21:04:21 <c0rw1n> (not seen in chain yet)
285 2013-07-30 21:04:37 <c0rw1n> tho with a fee of .005 it better get there fast
286 2013-07-30 21:05:07 <petertodd> c0rw1n: weird, still not seeing it, nor do I see it on bc.i
287 2013-07-30 21:06:34 <c0rw1n> bah it'll get there eventually
288 2013-07-30 21:07:14 <petertodd> reminds me: I was thinking that P2Pool shares would make for a good tx propagation metric - p2pool forwards transactions that peers don't have along with shares
289 2013-07-30 21:08:04 <petertodd> has some interesting side-effects too: if you mine on p2pool and accept non-standard transactions, your shares will contain them, and conversely your mempool will get non-std txs from miners that also mine non-std
290 2013-07-30 21:08:24 <petertodd> basically it's a tx distribution mechanism using PoW rather than fees/priority to prevent spam
291 2013-07-30 21:08:57 <petertodd> it'd be a nice way to propagate replace-by-fee doublespends too
292 2013-07-30 21:09:06 <petertodd> up to 50KB of tx's per share
293 2013-07-30 21:09:36 <petertodd> c0rw1n: can you give me getrawtransaction 2c6c66ca4f47d672b3298e9f65fe66d02bb3f297c3b2725753ff66182c63e559?
294 2013-07-30 21:09:52 <c0rw1n> dunno how to do that in multibit :/
295 2013-07-30 21:10:08 <petertodd> c0rw1n: ah, might not be able too :(
296 2013-07-30 21:10:51 <c0rw1n> there's the raw data, but can't copy it to paste
297 2013-07-30 21:11:01 <c0rw1n> i'm _not_ copying that by hand
298 2013-07-30 21:11:16 <c0rw1n> it'll get there eventually
299 2013-07-30 21:11:25 <petertodd> ha, not being able to copy that is a serious misfeature
300 2013-07-30 21:11:28 <c0rw1n> yes
301 2013-07-30 21:11:36 <petertodd> file a bug report
302 2013-07-30 21:12:16 <petertodd> which vout # was my address in?
303 2013-07-30 21:12:23 <petertodd> I can still write the transaction after all
304 2013-07-30 21:13:22 <petertodd> (hmm... probably only #0 to #1 of course...)
305 2013-07-30 21:14:34 <c0rw1n> what's a vout? the "outpoint" ?
306 2013-07-30 21:14:42 <petertodd> yeah
307 2013-07-30 21:14:47 <sipa> likely
308 2013-07-30 21:15:01 <c0rw1n> is that safe to paste in the channel?
309 2013-07-30 21:15:17 <petertodd> yup
310 2013-07-30 21:15:21 <sipa> there may be some weak privacy issues
311 2013-07-30 21:15:45 <c0rw1n> bah i' not hiding with those addresses
312 2013-07-30 21:15:51 <c0rw1n> you wouldn't see my `isp here if i was
313 2013-07-30 21:17:20 <c0rw1n> f5d8fdfd0acc596820bdb70d671133ac6f8872b4d8f6fa34119e1182709f6c1f
314 2013-07-30 21:18:02 <c0rw1n> (if i copied that right)
315 2013-07-30 21:19:17 <petertodd> c0rwin: that's the txid of where the money came from, I'm looking for the index of the entry for where the money was going
316 2013-07-30 21:20:50 <c0rw1n> that's the only thing in there except for your address, my address and the 2c6c66ca4f47d672b3298e9f65fe66d02bb3f297c3b2725753ff66182c63e559
317 2013-07-30 21:21:01 <c0rw1n> multibit says nothing more
318 2013-07-30 21:22:40 <petertodd> pff, that's not very impressive
319 2013-07-30 21:23:22 <sipa> we need spv bitcoin-qt :)
320 2013-07-30 21:28:17 <petertodd> c0rw1n: http://pastebin.com/e03GUESN <- one of these two transations should be valid
321 2013-07-30 21:28:31 <petertodd> c0rw1n: blockchain.info/pushtx will send it
322 2013-07-30 21:29:33 <petertodd> sipa: I know, it's too bad no-one is working on it... :P
323 2013-07-30 21:30:43 <c0rw1n> blockchain.info/pushtx says Unable To find all tx inputs: [2c6c66ca4f47d672b3298e9f65fe66d02bb3f297c3b2725753ff66182c63e559]
324 2013-07-30 21:30:58 <petertodd> yeah, well it won't work until that txid is mined
325 2013-07-30 21:31:13 <petertodd> I'm surprised with that fee it's still not outon the network, very strange
326 2013-07-30 21:31:29 <petertodd> what does your client show?
327 2013-07-30 21:31:31 <c0rw1n> multibit still says "seen by 1 peer"
328 2013-07-30 21:31:41 <c0rw1n> also "not seen in chain"
329 2013-07-30 21:31:58 <petertodd> crazy, sounds like you have a bad peer
330 2013-07-30 21:32:46 <c0rw1n> i have a quantity of bitcoin clients running on my subnet, and some share their addresses ...
331 2013-07-30 21:32:53 <petertodd> maybe a north-korean peer?
332 2013-07-30 21:33:10 <petertodd> hmm... possibly you hit a DoS trigger someone? that's odd
333 2013-07-30 21:33:17 <petertodd> can multibit tell you who your peer is?
334 2013-07-30 21:33:29 <c0rw1n> i suspect the tx was seen by one of my other wallets which then forgot to forward it for any reason
335 2013-07-30 21:33:49 <c0rw1n> seems that no
336 2013-07-30 21:34:31 <petertodd> multibit sucks, even android wallet can show your peer ip's
337 2013-07-30 21:34:36 <petertodd> and you shouldhave more peers than just one
338 2013-07-30 21:35:16 <c0rw1n> previous transaction in my wallet say "seen by 2 peers", it's my mining pay from bitminter
339 2013-07-30 21:35:46 <petertodd> huh, maybe someone's trying to screw with SPV?
340 2013-07-30 21:36:06 <c0rw1n> maybe i screwed something up with my wallets
341 2013-07-30 21:36:13 <petertodd> heck, I myself have two nodes running right now that silently ignore bloom filter anything, although that shouldn't affect tx propagation
342 2013-07-30 21:38:13 <sipa> petertodd: bitcoin-qt doesn't show peer ip'd either...
343 2013-07-30 21:38:24 <sipa> (well, via the rpc console, but that's cheating)
344 2013-07-30 21:38:43 <sipa> we should start calling rpc commands "cheat codes"
345 2013-07-30 21:38:53 <sipa> people would be a lot more excited about them
346 2013-07-30 21:38:55 <petertodd> sipa: so your saying the developers only use cheat codes?
347 2013-07-30 21:39:00 <petertodd> sipa: I can live with that
348 2013-07-30 21:39:10 <sipa> petertodd: no, we use bitcoind :p
349 2013-07-30 21:39:17 <sipa> which is an evil havkers.tool
350 2013-07-30 21:39:25 <sipa> evil hackers tool
351 2013-07-30 21:39:49 <petertodd> no it's not! I added a splashscreen to my bitcoind, which means it's not a hacking tool
352 2013-07-30 21:40:12 <c0rw1n> even metasploit has a splash screen :/
353 2013-07-30 21:40:15 <sipa> with headers-based syncing (probably a better name than headers-first, as it's really simultaneous), spv comes oretty close
354 2013-07-30 21:41:01 <petertodd> sipa: yeah, and headers-first might even wind up being "headers is some bizzare random order controlled by fancy (non-)interactive proof" syncing
355 2013-07-30 21:41:31 <sipa> hmm?
356 2013-07-30 21:42:51 <petertodd> sipa: er, wait, I was thinking headers-first as in newest block header first, got ahead of myself...
357 2013-07-30 21:45:35 <petertodd> WTF: 1C7zdTfnkzmr13HfA2vNm5SJYRK6nEKyq8 compressed correct horse address, and two weeks ago someone put 21BTC in it, moved two hours later...
358 2013-07-30 21:46:26 <sipa> petertodd: read this: https://github.com/sipa/bitcoin/commit/6b1c4dc55c00b88fc8dd394bfd46615c15879d61
359 2013-07-30 21:46:37 <sipa> (yes, there's a typo in the title)
360 2013-07-30 21:48:06 <petertodd> should clarify that orphans aren't relevant *for the purposes of initial synchronization*
361 2013-07-30 21:48:40 <sipa> read on :)
362 2013-07-30 21:49:27 <sipa> wait
363 2013-07-30 21:49:40 <sipa> by orphan, i mean "block without parent"
364 2013-07-30 21:50:13 <petertodd> ah, I really meant... er, what's the accepted term these days?
365 2013-07-30 21:50:46 <sipa> orphan block :P
366 2013-07-30 21:51:11 <sipa> (but the source code has never used orphan block in that meaning, so i don't feel obliged to use it in commit messages either)
367 2013-07-30 21:51:41 <petertodd> yeah, come up with a new term and put it in that commit message if you could - lets get that stupid name out of the collective conciousness
368 2013-07-30 21:52:00 <sipa> extinct block, stale block, inactive block, side chain, ...
369 2013-07-30 21:52:08 <sipa> tons of names have been proposed, but none stick
370 2013-07-30 21:52:58 <petertodd> I never knew the fascination with orphans in English literature would extend so far...
371 2013-07-30 21:53:42 <sipa> yeah, charles dickens started something...
372 2013-07-30 21:53:55 <petertodd> IMO lets go for dead, on the basis that we really really really hope a large number of the dead don't rise again
373 2013-07-30 21:54:24 <sipa> and dead blocks in a chain that grows to be the active chain again... zombie blocks?
374 2013-07-30 21:54:38 <sipa> jesus blocks?
375 2013-07-30 21:54:48 <petertodd> indeed, but we're politically correct and treat zombies like we would any other
376 2013-07-30 21:54:59 <sipa> of course
377 2013-07-30 21:55:45 <sipa> vitacism (discrimination based on aliveness) will not be tolerated
378 2013-07-30 21:56:11 <petertodd> Agreed: blocks are now known to be dead if the live chain bypasses them, and we all fear the zombie apocalypse. But if the zombie apocalypse does happen, we shall treat all zombies with love and respect."
379 2013-07-30 21:56:35 <sipa> in other words: history is written by the victor
380 2013-07-30 21:56:45 <petertodd> I wish I could do a pull-req on your git comment so I could add exactly that. :P
381 2013-07-30 21:56:50 <petertodd> yes, that too
382 2013-07-30 21:56:53 <sipa> (victor hugo?)
383 2013-07-30 21:57:05 <Diablo-D3> _the_ victor hugo
384 2013-07-30 21:58:17 <sipa> petertodd: in any case, executive summary: instead of getblocks, do header syncing, and in a background process, fetch blocks from peers in a moving window along the known best chain
385 2013-07-30 21:58:37 <sipa> (except the background process is really the same thread)
386 2013-07-30 21:59:07 <petertodd> yeah, seems all reasonable to me
387 2013-07-30 21:59:35 <petertodd> so mining code hasn't been touched right, IE it is based on pindexBest only?
388 2013-07-30 22:00:56 <sipa> yes, but see the note about pindexBest during a reorg
389 2013-07-30 22:01:07 <petertodd> hmm... so the logic is still to retrieve a whole block when we get an INV message for that block right?
390 2013-07-30 22:01:20 <sipa> if not in IBD, yes
391 2013-07-30 22:02:00 <petertodd> do we ever adv a header INV?
392 2013-07-30 22:02:09 <sipa> no
393 2013-07-30 22:02:26 <sipa> no such mechanism exists
394 2013-07-30 22:02:29 <sipa> you just INV a block hash
395 2013-07-30 22:02:37 <petertodd> oh right, that makes sense
396 2013-07-30 22:02:41 <sipa> the peer is free to ask for just the header or for the whole block
397 2013-07-30 22:03:07 <sipa> though i think it would make sense to change block invs to just block headers
398 2013-07-30 22:03:08 <petertodd> hmm... we should have a distinction though in some way, so peers can have only headers, rather than whole blocks
399 2013-07-30 22:03:36 <sipa> you can perfectly well do headers-only
400 2013-07-30 22:03:47 <sipa> when you get an inv, ask for headers
401 2013-07-30 22:03:51 <sipa> instead of a getdata
402 2013-07-30 22:04:18 <petertodd> right, point is, how should I advertise that all I have is headers?
403 2013-07-30 22:04:27 <sipa> hmm
404 2013-07-30 22:04:28 <sipa> unsure
405 2013-07-30 22:04:49 <sipa> why should anyone be interested in that, if you can't even prove the block it's for is valid?
406 2013-07-30 22:05:19 <petertodd> SPV nodes for one
407 2013-07-30 22:05:57 <sipa> even SPV nodes will want to ask for the data in it (perhaps just filtered)
408 2013-07-30 22:06:01 <petertodd> not all SPV nodes in the future will even want transactions at all
409 2013-07-30 22:06:08 <sipa> (playing devil's advocate here)
410 2013-07-30 22:06:21 <sipa> do you have a use case for that?
411 2013-07-30 22:06:45 <petertodd> we also should try to distribute headers as fast as possible, because miners will want to know so they can change their mining strategy to produce smaller blocks when they know their block is about to be killed
412 2013-07-30 22:06:49 <petertodd> timestamping
413 2013-07-30 22:06:59 <petertodd> and really anything that needs a random beacon
414 2013-07-30 22:07:02 <sipa> right
415 2013-07-30 22:07:31 <petertodd> also SPV is more secure if you are peering with other SPV nodes - lets you have fewer full nodes, and/or better able to detect if the full nodes are lying
416 2013-07-30 22:08:00 <sipa> i'm not opposed to a p2p way to request "send me headers, even without blocks"
417 2013-07-30 22:08:28 <sipa> and we can probably do without the inv-getdata cycle in between, as a headers packet is hardly larger than an inv
418 2013-07-30 22:08:38 <petertodd> I guess the main thing is how do you say "well I've got a header INV, but not a block yet"
419 2013-07-30 22:08:48 <sipa> you just broadcast the header
420 2013-07-30 22:09:03 <sipa> hmm
421 2013-07-30 22:09:34 <petertodd> oh, right, yes just broadcasting the header is fine
422 2013-07-30 22:09:42 <sipa> i think that could work, but some marker to indicate "not in a block" is perhaps safer and easier to handle
423 2013-07-30 22:10:13 <Luke-Jr> sipa, petertodd: In case either of you are interested, apparently you don't need to be an industry member to run for the industry seat in the BC Foundation (but you do need an industry member to nominate you)
424 2013-07-30 22:10:29 <sipa> i hate politics
425 2013-07-30 22:10:41 <Luke-Jr> who doesn't? :p
426 2013-07-30 22:11:59 <petertodd> sipa: well, lets have a header INV type, or I mean, whatever you produce in response to a getheaders, and add a flag for "I have this block"
427 2013-07-30 22:12:26 <sipa> petertodd: for backward compatibility, rather add a flag for "I don't have this block"
428 2013-07-30 22:13:02 <petertodd> sipa: right, but I thought we didn't already have a header data type at all? or are you sending what an SPV node would otherwise get?
429 2013-07-30 22:13:24 <sipa> you send a "headers" message in response to a "getheaders" message
430 2013-07-30 22:14:11 <sipa> it's the same mechanism that SPV nodes use to sync, yes
431 2013-07-30 22:14:13 <petertodd> ah, ok, I didn't realize we already had getheaders
432 2013-07-30 22:14:19 <jgarzik> yes, "headers" message was added when "getheaders" message was added
433 2013-07-30 22:14:24 <jgarzik> which was >year ago at least
434 2013-07-30 22:14:30 <jgarzik> I think Satoshi added it?
435 2013-07-30 22:14:35 <sipa> jgarzik: it's been there as long as i remember
436 2013-07-30 22:14:38 <petertodd> ACTION hasn't studied the network code enough
437 2013-07-30 22:14:46 <sipa> and i first looked at the code in december 2010 or so
438 2013-07-30 22:14:47 <jgarzik> as part of his half-hearted fClient effort
439 2013-07-30 22:15:17 <sipa> was added in 0.3.18
440 2013-07-30 22:15:57 <jgarzik> has the hard fork happened yet?
441 2013-07-30 22:15:57 <jgarzik> [mildly] related,
442 2013-07-30 22:16:03 <petertodd> I see, so getheaders was added, but of course until your work "headers" is ignored
443 2013-07-30 22:16:06 <sipa> indeed, no handler for 'headers' existed
444 2013-07-30 22:16:17 <sipa> not afaik
445 2013-07-30 22:16:40 <petertodd> sipa: hmm... problematic, because who knows how SPV clients will handle any data added to the vHeaders vector?
446 2013-07-30 22:17:01 <sipa> petertodd: well you don't add anything unless negotiated to do so
447 2013-07-30 22:17:28 <petertodd> sipa: ok, so make it a protocol version thing?
448 2013-07-30 22:17:43 <sipa> that's possible, but i'm not sure i want to enforce such functionality
449 2013-07-30 22:17:47 <petertodd> sipa: bump to 70003? (I have a good use for 70002...
450 2013-07-30 22:18:20 <sipa> (and even it's not strict enforcing, version numbers are a linear sequence, so you can't have a client that says "i don't want the feature added in 70003, but i do want the one added in 70004"
451 2013-07-30 22:18:43 <petertodd> I'm basically thinking, if your peer is >= 70003, then "headers" is defined to have "I have this block", and if that is set, don't expect me to follow it up with an INV
452 2013-07-30 22:18:51 <sipa> that doesn't work
453 2013-07-30 22:18:59 <sipa> it's always min(your_version, peer_version)
454 2013-07-30 22:19:16 <sipa> but there is technically no problem
455 2013-07-30 22:19:17 <petertodd> the alternative would be to just send headers if all we have is headers, and an INV if we have both
456 2013-07-30 22:19:26 <petertodd> which means sometimes will send both in the end
457 2013-07-30 22:19:30 <sipa> you could just declare that 'headers' never indicates the presence of a full block
458 2013-07-30 22:19:34 <petertodd> IMO that's fine, 80 bytes is nothing
459 2013-07-30 22:19:44 <sipa> and 'inv' does
460 2013-07-30 22:20:00 <sipa> as headers are never sent without request (right now)
461 2013-07-30 22:20:01 <petertodd> oh, and the logic works too, because if we have an INV, obviously we have the block, and we'll send them the headers if we don't have it yet
462 2013-07-30 22:20:21 <sipa> i think my current patch would deal perfectly with that
463 2013-07-30 22:20:23 <petertodd> yeah, although it's safe to send our peers headers, because unknown messages are ignored (at least for the ref bitcoind)
464 2013-07-30 22:20:56 <sipa> well, but P2P clients already rely on the headers message, so they have an implementation for it
465 2013-07-30 22:21:06 <sipa> sending it for different reason than they expect may confuse them
466 2013-07-30 22:21:13 <petertodd> yeah, and no-one has written a pure non-SPV alt, I think?
467 2013-07-30 22:21:33 <sipa> altcoin?
468 2013-07-30 22:21:40 <sipa> oh, alternate client?
469 2013-07-30 22:21:42 <petertodd> yeah
470 2013-07-30 22:21:56 <sipa> bitsofproof maybe
471 2013-07-30 22:21:57 <petertodd> probably work figuring out what bitcoinj does in response to an unsolicited header basically
472 2013-07-30 22:23:08 <jgarzik> <petertodd> I see, so getheaders was added, but of course until your work "headers" is ignored <<--- eh? bitcoinj has been using 'getheaders' for ages
473 2013-07-30 22:23:10 <sipa> but there may be just be like a field in version (or another, currently non-existing message) to request "i want these features to be enabled", and have verack (or equivalent) answer with "these features are now enabled! "
474 2013-07-30 22:23:14 <jgarzik> anyway, baby bedtime
475 2013-07-30 22:24:30 <petertodd> jgarzik: I meant in the ref bitcoind
476 2013-07-30 22:26:36 <petertodd> hmm... yeah, just doing the headers + inv eventually thing seems simplier to me over all
477 2013-07-30 22:26:49 <petertodd> I mean, you should be able to handle peers giving you arbitrary bits of information
478 2013-07-30 22:29:53 <petertodd> Here's the other thing: when we receive a new block, and haven't sent headers out yet, what we actually should be doing is sending headers first, followed by the block INV advertisement.
479 2013-07-30 22:30:31 <petertodd> The extra bit of BW is tiny, and it makes sure someone will in fact get the headers out to the P2P network prior to the block at some point.
480 2013-07-30 22:33:51 <sipa> makes sense... send out header when you extended your header-chain, send out inv when you extended your block-chain
481 2013-07-30 22:34:30 <sipa> as processing a block means first processing its header, you'll get both automatically when an unknown block that connects arrives
482 2013-07-30 22:35:00 <petertodd> mind, doesn't the networking code only receive stuff in it's entirely?
483 2013-07-30 22:35:17 <petertodd> I mean, I don't think we can deserialize and process the header first
484 2013-07-30 22:35:57 <sipa> sure
485 2013-07-30 22:36:02 <sipa> i mean the processing logic, not the network side
486 2013-07-30 22:36:05 <petertodd> ah yeah
487 2013-07-30 22:36:11 <sipa> so yes, you'll download the entire block first
488 2013-07-30 22:36:16 <sipa> but its header is still processed first
489 2013-07-30 22:36:41 <petertodd> yeah, so if you were, say, the peer that actually created the block, you could in theory only send the block, not the header
490 2013-07-30 22:36:50 <petertodd> (or if your peer was older)
491 2013-07-30 22:37:20 <petertodd> but you'll always send out the header first because the act of processing a block changes what is the new pindexBestHeader
492 2013-07-30 22:37:33 <sipa> indeed
493 2013-07-30 22:37:43 <sipa> that's what happens, btw
494 2013-07-30 22:37:57 <sipa> if you actually mine a block, it is immediately broadcast without preceding inv
495 2013-07-30 22:38:00 <petertodd> We should also have new headers, and heck, new blocks, "jump the queue" when it comes to messages we're sending to peers - pretty sure we aren't doing that.
496 2013-07-30 22:38:15 <petertodd> Ah good, so that case is being tested.
497 2013-07-30 22:38:23 <petertodd> Heh, that's actually a privacy risk in theory...
498 2013-07-30 22:38:44 <sipa> so run a bitcoind in between :)
499 2013-07-30 22:38:57 <sipa> (and remove whatever advantage the lack of 'inv' had)
500 2013-07-30 22:39:15 <petertodd> Indeed... hmm, we should add code that occasionally drops INVs with, say 1% probability.
501 2013-07-30 22:39:44 <petertodd> Could be good in general really to enforce some level of robustness. :(
502 2013-07-30 22:40:25 <Vinnie_win> Robustness is always good.
503 2013-07-30 22:40:49 <sipa> except for things that need to break :)
504 2013-07-30 22:41:13 <petertodd> well they need to break occasionally, or they'll break...
505 2013-07-30 22:43:19 <sipa> failing to fail can be a failure...
506 2013-07-30 22:43:47 <petertodd> sipa: I notice that you have the code always send a getheaders in response, on the assumption that we're syncing - that should change
507 2013-07-30 22:43:47 <phantomcircuit> sipa, yo dawg
508 2013-07-30 22:45:26 <petertodd> heh, annoying: I see that getheaders only works in one direction
509 2013-07-30 22:46:14 <sipa> petertodd: how so?
510 2013-07-30 22:46:32 <sipa> ah
511 2013-07-30 22:46:33 <petertodd> "for (; pindex; pindex = pindex->pnext)"
512 2013-07-30 22:47:07 <sipa> see, there's a difference!
513 2013-07-30 22:47:16 <petertodd> also, your getheaders implementation adds "&& pindex->nHeight <= nBestHeight", which we actually don't want - we and to get able to getheaders even if we haven't gotten the blocks yet
514 2013-07-30 22:47:54 <sipa> not convinced
515 2013-07-30 22:48:15 <sipa> as there is currently no way to distinguish between having headers and having blocks
516 2013-07-30 22:48:39 <petertodd> consider the SPV headers only peer case - they'll expose getheaders, but not have blocks
517 2013-07-30 22:48:45 <sipa> maybe we need entirely new messages for this
518 2013-07-30 22:48:46 <petertodd> or partial mode for that matter
519 2013-07-30 22:49:06 <petertodd> new messages is a perfectly good solution too, especially since getheaders is only going in one direction
520 2013-07-30 22:49:32 <sipa> you mean you can't ask for ancestry?
521 2013-07-30 22:50:25 <petertodd> oh yeah, locator is an index right?
522 2013-07-30 22:50:29 <petertodd> not a hash?
523 2013-07-30 22:50:35 <sipa> it's a set of hashes
524 2013-07-30 22:50:51 <sipa> look at the code :)
525 2013-07-30 22:50:58 <sipa> it's in main.h afaik
526 2013-07-30 22:51:04 <petertodd> looking
527 2013-07-30 22:51:50 <petertodd> ok, that's more complex than I realized, I thought locator was a hash for some reason
528 2013-07-30 22:55:57 <petertodd> so this means if I want to download the block headers, from the newest first, I actualy can't without checkpoints right? because I don't have block hashes to give the locator?
529 2013-07-30 23:00:08 <sipa> indeed, but why would you want to go backwards?
530 2013-07-30 23:00:21 <sipa> you can't validate later ones without their parents
531 2013-07-30 23:01:21 <petertodd> if I'm really memory/bandwidth constrained I might want to accumulate total work quickly, so my attacker can't send me off into a long low-work chain
532 2013-07-30 23:01:36 <sipa> right, you can check individual PoW
533 2013-07-30 23:02:17 <petertodd> with total work proofs it's even better because they can prove the total work of the chian - but that's a fair bit of code to implement
534 2013-07-30 23:03:21 <sipa> how?
535 2013-07-30 23:03:33 <phantomcircuit> petertodd, part of the problem with initial block sync is that there's no easy way to get all of the headers
536 2013-07-30 23:03:45 <sipa> phantomcircuit: it's not hard...
537 2013-07-30 23:04:02 <petertodd> they commit to a merkle sum tree of all headers, I pick a random one, they prove they have it (and that the sums along the path make sense)
538 2013-07-30 23:04:02 <phantomcircuit> sipa, it is if you're trying to avoid the network latency between blocks of 500
539 2013-07-30 23:04:12 <phantomcircuit> get 500, process, goto 1
540 2013-07-30 23:04:27 <sipa> phantomcircuit: you can get headers in batches of 2000
541 2013-07-30 23:04:46 <phantomcircuit> sipa, well really im talking about full blocks
542 2013-07-30 23:04:58 <phantomcircuit> it's the primary reason the pipelined code is so much faster
543 2013-07-30 23:05:11 <phantomcircuit> you're not sitting around waiting for the next 500 blocks
544 2013-07-30 23:05:15 <petertodd> sipa: if I pick n headers they have a 1/n chance of getting away with any fraud basically
545 2013-07-30 23:05:21 <sipa> phantomcircuit: yes, i noticed :)
546 2013-07-30 23:05:50 <phantomcircuit> sipa, heh im sure you did
547 2013-07-30 23:05:51 <petertodd> sipa: then just weight the tree by work - IE randomly go down from the tip, higher work branches more likely to be requested in proportin to work claimed
548 2013-07-30 23:05:59 <sipa> petertodd: right, though it's not really a proof, in the sense that for 100% safety you still need all headers
549 2013-07-30 23:06:26 <petertodd> sipa: sure, but I can get exponentially more safety for every header picked
550 2013-07-30 23:06:31 <sipa> agree
551 2013-07-30 23:06:52 <sipa> phantomcircuit: though technically, you don't need headers-based syncing for pipelining
552 2013-07-30 23:06:53 <petertodd> sipa: soon the real issue is "are all my peers frauds?"
553 2013-07-30 23:07:16 <phantomcircuit> sipa, right
554 2013-07-30 23:07:24 <petertodd> For now anyway most recent header first is pretty damn good...
555 2013-07-30 23:07:24 <sipa> phantomcircuit: as even if you ask for 500 blocks in one getdata, the actual blocks arrive one by one in 'block' messages
556 2013-07-30 23:07:35 <phantomcircuit> headers based sync just allows you to do hybrid spv operation until you've synced completely
557 2013-07-30 23:07:52 <sipa> no, not really
558 2013-07-30 23:08:04 <sipa> for SPV you need wallet-filtered sync too
559 2013-07-30 23:08:20 <sipa> though headers-based sync is a large step towards SPV
560 2013-07-30 23:08:22 <phantomcircuit> sipa, right but that's gonna be *much* faster
561 2013-07-30 23:08:45 <sipa> sure, but SPV is essentially a way to implement an efficient wallet, using a headers-synced chain
562 2013-07-30 23:08:58 <phantomcircuit> yeah
563 2013-07-30 23:09:02 <sipa> while headers-based syncing is a way to improve IBD for a full node
564 2013-07-30 23:09:11 <sipa> they share the "sync headers" part, but that's it
565 2013-07-30 23:10:05 <phantomcircuit> sipa, well you could more or less operate as an SPV node, quickly get wallet transactions
566 2013-07-30 23:10:14 <phantomcircuit> then more slowly become a full node
567 2013-07-30 23:10:26 <sipa> yeah, they can be combined
568 2013-07-30 23:10:32 <sipa> that's be pretty much optimal :)
569 2013-07-30 23:10:43 <phantomcircuit> yeah
570 2013-07-30 23:10:52 <phantomcircuit> i suspect that'll be kind of difficult to actually implement though
571 2013-07-30 23:11:00 <sipa> i don't think so
572 2013-07-30 23:11:22 <phantomcircuit> sipa, i'll be happy to be proven wrong :)
573 2013-07-30 23:30:52 <CodeShark> do we still need to support any compilers that don't support c++11?
574 2013-07-30 23:31:39 <CodeShark> I guess c++11 is still being worked out
575 2013-07-30 23:31:59 <CodeShark> no compiler supports all the features
576 2013-07-30 23:33:14 <sipa> the gcc in the ubuntu VM used for gitian doesn't support c++11 yet, afaik
577 2013-07-30 23:33:43 <sipa> actually, it seems it does
578 2013-07-30 23:33:57 <sipa> at least, some of the features...
579 2013-07-30 23:34:19 <CodeShark> I'm starting to get into the habit of using some of the features copiously
580 2013-07-30 23:34:22 <CodeShark> and it's hard to go back :p
581 2013-07-30 23:35:43 <CodeShark> it really is a significant improvement of the language on many fronts
582 2013-07-30 23:36:16 <sipa> i know
583 2013-07-30 23:36:41 <sipa> rvalue/move semantics are complex, but really fix an long an inherent problem
584 2013-07-30 23:37:22 <sipa> the bitcoin codebase uses swaps explicitly in some case to get the performance of move semantics
585 2013-07-30 23:39:53 <sipa> ACTION ACPI STANDBY
586 2013-07-30 23:43:05 <Luke-Jr> sipa: ubuntu mingw doesn't
587 2013-07-30 23:43:26 <Luke-Jr> so I should start using anonymous functions in my Bitcoin code?
588 2013-07-30 23:43:36 <Luke-Jr> aka lambdas
589 2013-07-30 23:45:24 <CodeShark> I've been using them a lot in other code - they are quite convenient :)
590 2013-07-30 23:49:03 <CodeShark> the whole function pointer syntax in C has long been one of the language's warts
591 2013-07-30 23:49:43 <CodeShark> functions really should be treated as any other datatype
592 2013-07-30 23:51:07 <CodeShark> in C++ you have the ability to overload the () operator, but that's a hack
593 2013-07-30 23:51:39 <CodeShark> it just hides the ugliness inside a class
594 2013-07-30 23:54:16 <CodeShark> wouldn't be a problem if the semantics were consistent - but they aren't - the () operator is also used for constructors and for operator precedence
595 2013-07-30 23:54:53 <CodeShark> although the latter two can be easily distinguished