1 2014-07-17 00:48:31 <ebfull> is there any 'watchonly' support available in newer versions of bitcoind
  2 2014-07-17 00:49:08 <ebfull> or what is the generally recommended route for watching addresses (without using someone else's API)
  3 2014-07-17 00:49:49 <kazcw> ebfull: watchonly is in the current source on git but isn't in any release yet
  4 2014-07-17 00:50:10 <ebfull> it's in master?
  5 2014-07-17 00:50:17 <kazcw> yeah
  6 2014-07-17 00:50:21 <ebfull> cool thank you :)
  7 2014-07-17 02:08:10 <jrick> can anyone explain how the bitcoind wallet rescan works? I'm looking at it in init.cpp and from what I can tell, it rescans through a fixed range of blocks, ending with the current tip, and when it has finished it marks the wallet synced with the chain tip again
  8 2014-07-17 02:08:35 <jrick> so couldn't any new blocks that were attached, or any rescanned blocks that were reorged out, still be referenced by the wallet?
  9 2014-07-17 02:13:14 <jrick> I guess it may not be attaching stuff to the chain at this point
 10 2014-07-17 02:16:26 <jrick> but importprivkey also has the option to rescan for imported addresses, and that definitely must run while blocks are being attached
 11 2014-07-17 02:19:17 <jgarzik> hrm
 12 2014-07-17 02:19:25 <jgarzik> is pull tester dead?
 13 2014-07-17 02:19:56 <jgarzik> http://jenkins.bluematt.me/pull-tester/p4332_f41b55ece391549cf20ac29cbb69d8692b124cbf/test.log
 14 2014-07-17 02:20:00 <jgarzik> out of space?
 15 2014-07-17 02:40:09 <BlueMatt> jgarzik: yes, but my access to that server is borked, someone else needs to fix it
 16 2014-07-17 06:05:45 <jgarzik> sipa, gmaxwell, BlueMatt: RE pulltester, I am tempted to see if a useful framework within bitcoin/bitcoin.git can be found.  it would be useful to me to run multi-node and chain reorg tests alongside existing in-tree tests, and not have to rely on jenkins.
 17 2014-07-17 06:06:15 <jgarzik> Wonder what is the quickest way to spin up a TNIAB repeatedly
 18 2014-07-17 06:06:55 <jgarzik> sadly cp(1) exists but no cow(1) for building a base snapshot
 19 2014-07-17 06:08:05 <jgarzik> 
ACTION imagines a Fedora-mock-like setup, that can spin up N bitcoind's, and devise some sort of method for describing the more advanced multi-node tests

 20 2014-07-17 06:08:10 <jgarzik> s/method/test language/
 21 2014-07-17 06:09:43 <jgarzik> mock has a "prep" step that builds a .tar.gz containing a configured-and-ready-to-go build setup.  then, unpack that cached filesystem, run tests (in our case), and destroy the entire subtree
 22 2014-07-17 06:10:25 <jgarzik> in our case, the prep step would build N testnet chains including indexes, to enable more rapid startup
 23 2014-07-17 13:08:50 <sipa> jgarzik: we already have comparuson tool integration, and use regtest instead of testnet. also see my latest issue which mentions multi node tests
 24 2014-07-17 13:09:34 <jgarzik> sipa, where is comparison tool integration?
 25 2014-07-17 13:09:51 <jgarzik> certainly regtest must be used.
 26 2014-07-17 13:10:32 <sipa> jgarzik: ./configure --with-comparisoin-tool=jarfile; make check
 27 2014-07-17 13:12:02 <jgarzik> 
ACTION reads

 28 2014-07-17 13:12:12 <jgarzik> sipa, nice.  I just have to hack it to remove evil Java.
 29 2014-07-17 13:12:19 <sipa> lol
 30 2014-07-17 13:12:24 <sipa> see my issue
 31 2014-07-17 13:12:30 <jgarzik> sipa, #?
 32 2014-07-17 13:12:41 <sipa> dunno, i filed it yesterday
 33 2014-07-17 13:13:07 <jgarzik> https://github.com/bitcoin/bitcoin/issues/4545 found it
 34 2014-07-17 13:13:12 <sipa> yup
 35 2014-07-17 13:13:20 <jgarzik>  /issues doesn't give a useful sort by default
 36 2014-07-17 13:13:35 <jgarzik> PRs can be easily sorted by author, but not issues
 37 2014-07-17 13:14:44 <jgarzik> sipa, good point -- multi-version should be tested.  I was thinking primarily about an extended "make check", with a harness built in C++ in-tree [no evil java]
 38 2014-07-17 13:16:18 <sipa> we already have python tests for testing rpc too
 39 2014-07-17 13:16:35 <jgarzik> yes, that was so nice to get added
 40 2014-07-17 13:17:16 <jgarzik> sipa, Related: do you have access to pulltester machine?  pull tester is dead due to some out of space condition or somesuch.
 41 2014-07-17 13:17:25 <sipa> i do
 42 2014-07-17 13:17:30 <sipa> i'll have a look
 43 2014-07-17 13:20:22 <wumpus> jgarzik: the way to get issues by author is to search for author:name in the issues
 44 2014-07-17 13:22:06 <sipa> jgarzik: i'm not a java fan either (though i wouldn't call it evil...), the bitcoinj based comparison tool has been incredibly useful (i've lost count how many bugs it found in my patches...)
 45 2014-07-17 13:22:43 <sipa> but it's a nontrivial thing to make a more flexible one, without full node implementation behind it, and have it regain the same level of trust
 46 2014-07-17 13:23:03 <jgarzik> sipa, a working comparison tool has great value, and the current java version fits that bill, yes.
 47 2014-07-17 13:23:37 <jgarzik> sipa, I've long wanted a domain-specific language that permitted me to describe a test scenario.  find a new scenario?  just use the chain language procedure file.
 48 2014-07-17 13:23:44 <jgarzik> s/use/update/
 49 2014-07-17 13:23:45 <sipa> yup!
 50 2014-07-17 13:28:51 <jgarzik> sipa, I doubt my v1 will be quite that fancy, but that's the goal.  At a minimum, it would need to do what bitcoinj tool does now.
 51 2014-07-17 13:33:50 <BlueMatt> jgarzik: I think cory was working on that
 52 2014-07-17 13:34:07 <BlueMatt> jgarzik: and moving pull-tester to a real service now that those exist
 53 2014-07-17 13:35:41 <jgarzik> BlueMatt, I want something in-tree with zero additional dependencies
 54 2014-07-17 13:35:52 <jgarzik> BlueMatt, though that can be pursued in parallel, sure
 55 2014-07-17 13:35:59 <BlueMatt> jgarzik: you can already run all of pull-tester with make check and a jar file....
 56 2014-07-17 13:36:02 <BlueMatt> (locally)
 57 2014-07-17 13:36:17 <jgarzik> BlueMatt, java+bitcoinj is an additional dependency
 58 2014-07-17 13:36:23 <BlueMatt> ok, sure
 59 2014-07-17 13:43:50 <sipa> jgarzik: cory was working on moving to travis
 60 2014-07-17 13:44:04 <sipa> i think that is orthogonal to what and how the tests are done
 61 2014-07-17 13:44:24 <jgarzik> yes
 62 2014-07-17 13:49:30 <wumpus> it's all relative; java is a good language for interoperability, and I suppose the node tester should be applicable to all node implementations, people testing other implementations may balk at having to compile C++
 63 2014-07-17 13:50:40 <jgarzik> wumpus, I think the overall direction is towards programming-language-independent test descriptions
 64 2014-07-17 13:50:43 <jgarzik> json etc.
 65 2014-07-17 13:53:32 <wumpus> for smaller unit tests json is great, but for higher-level tests having an executable that launches tests at an arbitrary node implementation through the network can also be useful, I thought that was the purpose of the comparison tool
 66 2014-07-17 13:55:35 <wumpus> anyhow, all kinds of different tests can exist in parallel, no problem
 67 2014-07-17 13:55:42 <wumpus> more tests is better
 68 2014-07-17 13:55:43 <jgarzik> wumpus, I think the tests can be described in a domain-specific language
 69 2014-07-17 13:55:56 <jgarzik> java, C++, whomever can parse that and execute tests as they like
 70 2014-07-17 13:56:10 <jgarzik> that's my long term goal anyway.
 71 2014-07-17 13:59:23 <sipa> jgarzik: that's great for unit tests, but not for network interaction tests
 72 2014-07-17 13:59:51 <sipa> jgarzik: you don't want to reimplement the same test behavior, as it relies on reimplementing the exact network behaviour you want to test
 73 2014-07-17 14:00:41 <jgarzik> sure it doesn't cover all test categories
 74 2014-07-17 14:01:01 <wumpus> sipa: right, that's what I was trying to say
 75 2014-07-17 14:01:15 <sipa> bitcoinj based comparison was great _because_ it uses an alternate p2p implementation
 76 2014-07-17 14:21:10 <Guest43411> Will the miners tollerate a transaction with a lock time 72 hours into the future?
 77 2014-07-17 14:21:46 <Guest43411> 48 hours?
 78 2014-07-17 14:22:51 <Guest43411> *tolerate*
 79 2014-07-17 14:23:27 <helo> Guest43411: a good question for #bitcoin
 80 2014-07-17 14:34:23 <jgarzik> 
ACTION kicks github

 81 2014-07-17 14:34:57 <jgarzik> sipa, github is hiding the main.cpp changes in your clang PR.  quite odd.  " 751 additions, 967 deletions not shown" for src/main.cpp at https://github.com/sipa/bitcoin/commit/ee6f59833b99184a267fff3cb6196ffd0603d5cc
 82 2014-07-17 14:35:10 <jgarzik> 
ACTION grabbed it via cmd line, but anyway, odd.

 83 2014-07-17 14:36:09 <sipa> jgarzik: you can click 'view'
 84 2014-07-17 14:36:16 <sipa> which doesn't show the chances, only the result
 85 2014-07-17 14:36:29 <sipa> but for this case, it's useful to just look at the result
 86 2014-07-17 14:36:41 <sipa> and left/right, pfff :)
 87 2014-07-17 14:37:27 <jgarzik> sipa, disagree. thought 'view' was unhelpful, wanted to see diff.
 88 2014-07-17 14:37:30 <jgarzik> sipa, :)
 89 2014-07-17 14:37:54 <jgarzik> sipa, I'm used to ref& foo + void *foo, left & right
 90 2014-07-17 14:38:02 <jgarzik> but thinking about it logically, I think it should cuddle with type.
 91 2014-07-17 14:38:47 <sipa> <bikeshed> at least for *, it should logically stick to the variable
 92 2014-07-17 14:39:00 <sipa> as "int *a" declares *a to be an int, not a to be and int*
 93 2014-07-17 14:39:30 <sipa> for & it's less clear, as it's really just a modifier on the whole argument
 94 2014-07-17 14:40:04 <jgarzik> sipa, nod. My rationale: type modifiers cuddle in the compiler.  it changes the type.  view it as sentence "pointer to int" just like you see "long int" or "unsigned int"
 95 2014-07-17 14:40:13 <wumpus> logically, * should cuddle with the variable, for example in int *a, *b, *c;
 96 2014-07-17 14:40:17 <sipa> yup
 97 2014-07-17 14:40:29 <wumpus> int* a,b,c; would be misleading
 98 2014-07-17 14:40:35 <sipa> yup!
 99 2014-07-17 14:41:03 <jgarzik> for that shorthand, yes
100 2014-07-17 14:41:07 <sipa> but i think this thinking about int *a can be interpreted as "elitist" or something, as it means you're forced to think how the language works rather than the approximation ("int* is the type of pointers to int")
101 2014-07-17 14:41:27 <sipa> so many styles use left alignment
102 2014-07-17 14:41:30 <wumpus> given that C type syntax is already crazy enough (the famous spiral), anything that helps deambiguate it is helpful
103 2014-07-17 14:42:13 <sipa> anyway, i'm in favor of right alignment, but expected that most would be against
104 2014-07-17 14:42:23 <jgarzik> 
ACTION wonders if we can programmatically iterate through all settings, to see which disturbs the least amount of existing code

105 2014-07-17 14:42:23 <wumpus> anyhow, I don't care at all, toss a dice if you can't decide :p
106 2014-07-17 14:42:31 <jgarzik> +1
107 2014-07-17 14:42:37 <sipa> 
ACTION decides: right

108 2014-07-17 14:42:40 <jgarzik> just offering an opinion.  not a strong opinion, here.
109 2014-07-17 14:42:45 <jgarzik> sipa, ok, decided.
110 2014-07-17 14:42:47 <jgarzik> moving on :)
111 2014-07-17 14:42:48 <jrick> var a, b, c *int
112 2014-07-17 14:42:49 <jrick> :p
113 2014-07-17 14:42:55 <wumpus> 
ACTION hereby gives sipa the code-style-decider crown

114 2014-07-17 14:43:50 <jrick> guess since people are around right now.. any ideas about my rescan question from last night? it doesn't look like bitcoind wallet handles additions or reorgs for the chain during a rescan
115 2014-07-17 14:43:59 <sipa> jgarzik: it does
116 2014-07-17 14:44:03 <sipa> eh jrick: it does
117 2014-07-17 14:44:13 <sipa> oh, *during* ?
118 2014-07-17 14:44:18 <jgarzik> looks like clang de-indents inside namespace{}
119 2014-07-17 14:44:21 <jgarzik> I suppose that's ok
120 2014-07-17 14:44:27 <sipa> no, rescan is at startup, before the node is running
121 2014-07-17 14:44:29 <wumpus> jgarzik: yes that's ok, we don't really want to ident namespaces
122 2014-07-17 14:44:41 <sipa> jgarzik: i can turn that on or off
123 2014-07-17 14:44:51 <jrick> sipa: er I meant to say, during block additions/reorgs, rescan doesn't handle the new or removed blocks
124 2014-07-17 14:44:52 <wumpus> sipa: except for on-the-fly rescans like when importing a private key
125 2014-07-17 14:44:58 <jrick> it rescans through what the best chain tip was at the time
126 2014-07-17 14:45:09 <jrick> and then marks the wallet synced with the best chain tip (which may have changed)
127 2014-07-17 14:45:12 <jgarzik> boost style seems to be "do not indent inside namespace{}"
128 2014-07-17 14:45:14 <wumpus> sipa: but we're holding all the locks all that time so not much can happen...
129 2014-07-17 14:45:15 <jgarzik> at least in headers
130 2014-07-17 14:45:26 <jrick> if I understand that code correctly
131 2014-07-17 14:45:28 <wumpus> jgarzik: it makes sense to not indent, at least for the outer namespace() directive
132 2014-07-17 14:45:29 <sipa> jrick: rescan and modification of the chain never happen simultaneously
133 2014-07-17 14:45:47 <wumpus> jgarzik: otherwise *all* the code will be indented a level for no good reason
134 2014-07-17 14:45:54 <jrick> sipa: not from init.cpp, but importprivkey has a rescan parameter, and that should be done with chain modifictions
135 2014-07-17 14:46:05 <sipa> jrick: it locks cs_main
136 2014-07-17 14:46:10 <sipa> i think?
137 2014-07-17 14:46:14 <wumpus> jgarzik: inner namespaces may be different
138 2014-07-17 14:46:19 <jgarzik> sipa, wumpus: pleasantly surprised with how little src/main.cpp code is disturbed
139 2014-07-17 14:46:21 <wumpus> sipa: yes it locks cs_main and the wallet
140 2014-07-17 14:46:27 <sipa> so, no problem
141 2014-07-17 14:46:40 <jrick> hmmm ok
142 2014-07-17 14:46:52 <sipa> we certainly want to push that lock down at some point, and it's a good observation that that means making the rescan code deal with concurrent modification
143 2014-07-17 14:47:23 <wumpus> jgarzik: agreed
144 2014-07-17 14:47:33 <sipa> wumpus, jgarzik: so, rationale: in the future i'd like to see more use of namespaces... probably ones that span an entire file
145 2014-07-17 14:47:42 <jrick> ah I see the lock in import now
146 2014-07-17 14:47:44 <sipa> wumpus, jgarzik: for those cases, you do not want them to indent
147 2014-07-17 14:47:50 <wumpus> sipa: right
148 2014-07-17 14:47:56 <jgarzik> sipa, wumpus: ACK
149 2014-07-17 14:48:02 <jrick> so caller does importprivkey rescan, and now no more blocks are processed?
150 2014-07-17 14:48:07 <jrick> or what exactly gets locked there
151 2014-07-17 14:48:09 <sipa> jrick: indeed
152 2014-07-17 14:48:12 <jrick> :/
153 2014-07-17 14:48:30 <sipa> jrick: it's not a method intended to be regularly used
154 2014-07-17 14:48:44 <jgarzik> sipa, not just anon namespaces.  I like boost's http::server::foo etc.  much more natural class names, inside a sane namespace setup.
155 2014-07-17 14:49:12 <sipa> jgarzik: yes, anon namespaces are a different beast entirely
156 2014-07-17 14:49:18 <sipa> jgarzik: those rarely span much code
157 2014-07-17 14:49:21 <wumpus>  importprivkey with a rescan is more like a 'oh shit' troubleshooting command then something to be used regularly
158 2014-07-17 14:50:26 <sipa> like pretty much all wallet interaction at the individual key level
159 2014-07-17 14:51:20 <wumpus> jgarzik: I'm not a fan of deeply nested namespaces, it reminds me of java's com.bla.org.blabberfactories.blab, which makes it impossible to remember what was where, but use sanely namespaces are a good thiing
160 2014-07-17 14:52:12 <sipa> jrick: on the other hand, we just want get rid of the wallet...
161 2014-07-17 14:52:26 <wumpus> yes the wallet should not run in the same process as the node anyway
162 2014-07-17 14:52:57 <sipa> well yes/get rid of/split off/
163 2014-07-17 14:53:17 <sipa> s/yes/yes \s/
164 2014-07-17 14:53:19 <jrick> right, well for btcd our wallet is separate but I noticed that during a rescan, it could race because of a reorg, and I wanted to see how bitcoind did it
165 2014-07-17 14:53:40 <sipa> if your wallet is separate, why does it matter?
166 2014-07-17 14:53:40 <wumpus> sipa: right - I don't really care which of those, although there must be a good migration path for current bitcoind/qt wallet users
167 2014-07-17 14:54:00 <jrick> it's not spv.. (although in hindsight it probably should be)
168 2014-07-17 14:54:23 <sipa> yes :)
169 2014-07-17 14:54:30 <jrick> we have a rescan rpc extension for websocket connections
170 2014-07-17 14:54:37 <wumpus> wallets should definitely be SPV
171 2014-07-17 14:54:58 <jrick> agree, and when things slow down I want to get it completely switched over
172 2014-07-17 14:55:22 <sipa> which would also mean you can use your wallet with a bitcoind full node, eg
173 2014-07-17 14:55:27 <jrick> yep
174 2014-07-17 14:56:11 <sipa> jgarzik: so, you're going to work on a comparison tool successor?
175 2014-07-17 15:10:29 <andytoshi> it was mentioned on the mailing list that a duplicate coinbase tx has happened in the past. are we supposed to keep the first or the second in the utxo db?
176 2014-07-17 15:12:37 <wumpus> AFAIK the second overrides the first and the first becomes unspendable
177 2014-07-17 15:13:02 <andytoshi> oh, ok, i'm doing the opposite now. will fix that..
178 2014-07-17 15:13:44 <wumpus> andytoshi: but the details should be in https://github.com/bitcoin/bips/blob/master/bip-0030.mediawiki
179 2014-07-17 15:14:00 <andytoshi> thx wumpus
180 2014-07-17 15:14:20 <sipa> bip34 fixed it more thoroughly
181 2014-07-17 15:19:08 <andytoshi> sipa: do you recall whether duplicate txids would overwrite the old utxos before these BIPs?
182 2014-07-17 15:19:38 <sipa> andytoshi: yes
183 2014-07-17 15:23:14 <sipa> andytoshi: as in, yes, utxo's would be rewritten
184 2014-07-17 15:23:27 <andytoshi> ok, great, thx. i will change my code to do this
185 2014-07-17 15:23:49 <sipa> which is exactly the problem, if you have a non-fully-spent txid, which is spent in a side branch which is reorged away
186 2014-07-17 15:24:05 <andytoshi> yeah, i see. if it were the other way around there would not have been a problem
187 2014-07-17 15:24:13 <sipa> you now have a different utxo state for those who have not seen the branhc
188 2014-07-17 15:35:16 <btc111> So, all the core devs are now breaking NY laws. As creators of 'virtual currency' you are now all required to be fingerprinted and pay $3000 for a bitcoin license.  I wonder if the bitcoin foundation will stick up for you guys? probably not.
189 2014-07-17 15:35:24 <btc111> what a joke
190 2014-07-17 15:35:31 <wumpus> btc111: please, #bitcoin
191 2014-07-17 15:35:59 <btc111> wumpus: np, thought i'd let you guys know incase you haven't seen the news.
192 2014-07-17 15:36:08 <btc111> section 200.2n.5 http://www.dfs.ny.gov/about/press2014/pr1407171-vc.pdf
193 2014-07-17 15:36:08 <drizztbsd> FUD?
194 2014-07-17 15:36:19 <btc111> not FUD. read this. http://www.dfs.ny.gov/about/press2014/pr1407171-vc.pdf
195 2014-07-17 15:36:25 <sipa> as far as i know, i'm not subject to NY law
196 2014-07-17 15:36:27 <wumpus> I haven't, but I don't care, I don't leave Europe anyway
197 2014-07-17 15:36:29 <sipa> now take it elsewhere
198 2014-07-17 15:38:12 <btc111> sipa, you should talk to a lawyer. this is not a joke. If you are breaking NY laws, the US can extradidte you from any country with a treaty with US
199 2014-07-17 15:38:38 <btc111> I bet the bitcoin foundation won't even respond to this. what a joke.
200 2014-07-17 15:38:50 <sipa> duly noted.
201 2014-07-17 15:39:47 <drizztbsd> it's still FUD, bitcoin exists before that law :)
202 2014-07-17 15:40:04 <gmaxwell> drizztbsd: please don't respond to OT stuff.
203 2014-07-17 15:40:10 <drizztbsd> ok
204 2014-07-17 15:55:30 <jgarzik> sipa, I would like to
205 2014-07-17 15:55:44 <jgarzik> sipa, it's on the mile long list of other things I would like to do ;p
206 2014-07-17 15:56:12 <jgarzik> sipa, including exploring my cheesy sidechain design & hacking moxie
207 2014-07-17 18:12:59 <andytoshi> i notice that the satoshi client explicitly does not respond to `getdata` requests for invalid blocks, as opposed to other inv tyes where it sends a `notfound`
208 2014-07-17 18:13:03 <andytoshi> why is this?
209 2014-07-17 18:15:26 <andytoshi> the relevant code is `ProcessGetData` in main.cpp
210 2014-07-17 18:44:32 <kazcw> I don't think there is much reason to reply to a getdata for a block we don't have, we wouldn't have sent an inv so why would they getdata it from us, right? headers-first may change that though, since nodes will learn what blocks they want ahead of asking for them.
211 2014-07-17 18:48:40 <andytoshi> maybe we sent an inv, then later there is a reorg, and the node asks for the block again so that it can rewind its UTXO set
212 2014-07-17 18:49:32 <kazcw> wouldn't we still have the block in that case?
213 2014-07-17 18:49:56 <andytoshi> i think so, because bitcoind never prunes its blockchain..but i'd rather not depend on that behaviour
214 2014-07-17 19:00:58 <jgarzik> andytoshi, historically the behavior was "never respond"
215 2014-07-17 19:01:28 <jgarzik> andytoshi, thus the process of adding "reject" messages will cover some areas.  areas not yet covered remain with historical behavior.
216 2014-07-17 19:02:26 <sipa> getdata works for reorged blocks
217 2014-07-17 19:02:35 <sipa> but i don't think we should guarantee that behaviour
218 2014-07-17 19:03:16 <sipa> i guess it would make sense to add notfound for blocks, for consistency, but meh
219 2014-07-17 19:03:36 <sipa> i sort of liked the old asynchronous message processing model, instead of the request-response model
220 2014-07-17 19:03:44 <andytoshi> yeah, i am storing the last hundred blocks to do my own reorgs, but if that fails, i would've tried to bug the bitcoind peer (fingers crossed)
221 2014-07-17 19:03:45 <sipa> it's much more easily cacheable
222 2014-07-17 19:03:49 <andytoshi> but i can't do it because it'll hang the connection
223 2014-07-17 19:20:40 <bassguitarman> where in the code is tx messages implemented?
224 2014-07-17 19:21:58 <sipa> there is no such thing
225 2014-07-17 19:53:22 <blubberbop> BlueMatt: Hello, ae you around?