1 2013-01-02 00:00:23 <phantomcircuit> gmaxwell, im always tempted to buy some bitcoins using paypal from an unfunded account and then dispute the charge
2 2013-01-02 00:00:35 <phantomcircuit> he's just so fucking annoying
3 2013-01-02 00:18:24 <jgarzik> mtgax. guffaw.
4 2013-01-02 00:18:39 <a1111> Have anyone tried BITCOINBERUDECHANGE
5 2013-01-02 00:21:08 <a1111> I do see a review page
6 2013-01-02 00:21:09 <a1111> http://bitcoinberudechange.com/reviews/
7 2013-01-02 00:21:17 <a1111> Is this true?
8 2013-01-02 00:22:03 <jgarzik> a1111: offtopic, please take elsewhere
9 2013-01-02 00:22:23 <a1111> jgarzik: Someone started it, Ok sorry
10 2013-01-02 00:22:37 <phantomcircuit> a1111, it's a scam
11 2013-01-02 00:22:39 <phantomcircuit> </topic>
12 2013-01-02 00:35:33 <gmaxwell> a1111: they have very agressively spammed all the bitcoin IRC channels wih repeated all caps flooding??? evading blocks as they're able??? for some time now, and the general consensus is that its a scam. See also: https://bitcointalk.org/index.php?topic=122319.0
13 2013-01-02 00:37:14 <a1111> gmaxwell: I agree
14 2013-01-02 02:27:48 <nanotube> a1111: as a general rule, i wouldn't put much trust into a page of reviews that's hosted on the very same site that is being reviewed. :)
15 2013-01-02 03:24:39 <TradeFortress> Hi, I'm getting a Didn't receive 200 OK from rmote server when I try to call gettransaction via the RPC.
16 2013-01-02 03:25:53 <TradeFortress> I'm getting this error for other calls like getrecievedbyaddress too
17 2013-01-02 03:31:58 <Luke-Jr> TradeFortress: likely wrong user/pass then
18 2013-01-02 03:32:12 <TradeFortress> correct user/pass through
19 2013-01-02 03:32:33 <TradeFortress> I can call things like getbalance, getaccount
20 2013-01-02 03:35:24 <TradeFortress> getblock works
21 2013-01-02 03:44:52 <TradeFortress> is there anything I should try?
22 2013-01-02 04:04:30 <Luke-Jr> TradeFortress: maybe your parameters are wrong
23 2013-01-02 04:04:54 <TradeFortress> Luke-Jr: I'm just calling gettransaction with the tx hash. That's all.
24 2013-01-02 04:05:12 <Luke-Jr> wireshark it :P
25 2013-01-02 04:12:49 <gmaxwell> Luke-Jr: dude, reporting stupid posts like this https://bitcointalk.org/index.php?topic=133836.msg1429907#msg1429907 just wastes my time. The forum isn't going to do anything about tripe like that, as stupid as it is. :P
26 2013-01-02 04:13:48 <Luke-Jr> gmaxwell: that's why the forum remains troll haven :/
27 2013-01-02 04:14:26 <jgarzik> I love CBitcoin
28 2013-01-02 04:14:29 <jgarzik> "The hamming codes for storage are working. The problem is that they slow down the IO quite considerably. A storage system not using the hamming codes should be used instead for speed. Servers shouldn't need the hamming codes since server hardware is supposed to be highly resistant to data corruption."
29 2013-01-02 04:14:55 <Luke-Jr> O.o
30 2013-01-02 04:15:11 <maaku> please tell me that's not serious
31 2013-01-02 04:15:18 <gmaxwell> ...
32 2013-01-02 04:16:55 <gmaxwell> Would be nice if he'd spend a little of the time he's spent on getting gonzo algorithims into that code on making it actually correctly implement the bitcoin protocol.
33 2013-01-02 04:18:07 <phantomcircuit> jgarzik, hamming codes slow down io considerably?
34 2013-01-02 04:18:09 <jgarzik> spends a lot of time making CBitcoinAdjectiveAdverbVerbNoun APIs
35 2013-01-02 04:19:19 <gmaxwell> has he come up with any more novel macro abuse like NOT ? :P
36 2013-01-02 04:23:36 <jgarzik> ACTION once saw a codebase -- not cbitcoin -- that attempted to replicate (Pascal?) by writing C code with '#define BEGIN {' and the like.
37 2013-01-02 04:23:42 <jgarzik> ACTION shivers
38 2013-01-02 04:26:07 <gmaxwell> or Algol. :P
39 2013-01-02 04:27:34 <Luke-Jr> gmaxwell: I don't get why he didn't just include iso646.h for that
40 2013-01-02 04:28:23 <Luke-Jr> if you're going to insist on plain English operators, might as well use the existing standards for it
41 2013-01-02 04:32:07 <midnightmagic> ACTION is sad that CBitcoin isn't CBitcoin.
42 2013-01-02 04:34:05 <lianj> midnightmagic: well, that would be too easy
43 2013-01-02 04:35:28 <midnightmagic> lianj: I got all excited the first time I heard about it. Pure C! Bliss. And then crushing disappointment. No, no, it was my own fault really. I'm the one with the "Anybody who writes midsize projects in pure C must surely know C" bias.
44 2013-01-02 04:41:26 <midnightmagic> at least he's trying to write testcases. that's nice.
45 2013-01-02 04:41:58 <Luke-Jr> midnightmagic: if you want a C base58 parser, libblkmaker has one :P
46 2013-01-02 04:42:20 <midnightmagic> Luke-Jr: You asked me to review it a while back. :)
47 2013-01-02 04:42:32 <midnightmagic> Luke-Jr: Which I did halfway and then had to change a diaper.
48 2013-01-02 04:42:55 <Luke-Jr> hehe
49 2013-01-02 04:43:07 <Luke-Jr> wizkid057 actually found a bug in it ;)
50 2013-01-02 04:43:13 <midnightmagic> I saw that.
51 2013-01-02 04:43:19 <Luke-Jr> he's using it for his C (CPPSRB) implementation to verify validity
52 2013-01-02 04:43:19 <midnightmagic> That was the leading 1's issue?
53 2013-01-02 04:43:21 <Luke-Jr> yeah
54 2013-01-02 04:43:35 <Luke-Jr> nfc why I put parenthesis there??? O.o
55 2013-01-02 04:49:13 <jgarzik> midnightmagic: I mean, surely he must know C. And Bitcoin.
56 2013-01-02 04:58:36 <midnightmagic> Indubitably!
57 2013-01-02 05:23:06 <gmaxwell> https://lwn.net/Articles/531126/ < I'm happy to see the tux3 people adopting crash only programming. :)
58 2013-01-02 05:37:01 <gmaxwell> kjj: are you on the forums promoting things which make global consistency potentially unachievable? I thought we've talked before and I impressed upon you that making acceptance and reorg have different criteria makes a consensus system potentially non-convergent which is the worst possible failure mode for a currency system.
59 2013-01-02 05:39:47 <andytoshi> link, gmaxwell? i can't see how to separate acceptance and reorg
60 2013-01-02 05:39:53 <andytoshi> so i think there's some concepts crossed in my head
61 2013-01-02 05:40:44 <gmaxwell> e.g. I make two forks A and A', feed half the network A first, half A' first... congrats bitcoin is over. This is a deep result. Any decision system which has _any_ preference for an agents own past state must have a failure mode where consensus can never be achieved again.
62 2013-01-02 05:41:06 <andytoshi> oh, i see now
63 2013-01-02 05:41:21 <andytoshi> i was just eliminating that too early to realize what kjj was suggesting
64 2013-01-02 05:41:51 <andytoshi> that is deep..i've never thought of that before
65 2013-01-02 05:42:10 <gmaxwell> yea, kjj was suggesting that you can solve large hashpower attackers by making it so you don't pick the longest chain. You keep your current chain until some other chain is longer by some exponentially growing factor with the size of the reorg.
66 2013-01-02 05:43:00 <andytoshi> very interesting
67 2013-01-02 05:43:18 <andytoshi> mainly because "chain length" and "time" have the same meaning in bitcoin
68 2013-01-02 05:43:35 <andytoshi> so the obvious thing, "hesitate before trusting an attacker", is provable nonconvergent
69 2013-01-02 05:43:53 <gmaxwell> andytoshi: yea, it applies to things outside of cryptocurrencies and such. It's applicable to social science too, and it undermines things like the rationalists arguing about agents with identicial priors making identical decisions. (or it at least means you have to choose self consistency vs global consistency; you can't have both). It's one of the reasons I think politics is hard, because both self and global consistency are valuable.
70 2013-01-02 05:44:22 <gmaxwell> andytoshi: yea, and an attack can specifically target the non-convergence.
71 2013-01-02 05:45:43 <gmaxwell> E.g. with what kjj wrote on the forum you could literally kill bitcoin??? save rule 'violating' manual intervention??? by mining just a couple blocks faster than the network mines one, and announcing them to half of the hashrate each.
72 2013-01-02 05:46:20 <andytoshi> i understand it with bitcoin, no problem, but i'm having trouble with the "identical priors" scenario you mentioned
73 2013-01-02 05:46:49 <andytoshi> and i also suspect it means something cool in loop quantum gravity (where "time" is discretized in a similar fashion to the blockchain)
74 2013-01-02 05:49:07 <andytoshi> i think i understand the priors thing...suppose we meet in real life, and we both want an apple
75 2013-01-02 05:49:18 <andytoshi> and both thing, "i will take the apple, unless the other guy wants it"
76 2013-01-02 05:49:42 <andytoshi> and somehow we both assign identical weights to taking the apple if/ifnot the other wants it
77 2013-01-02 05:50:10 <gmaxwell> andytoshi: E.g. internet rationalists like to muse on the notion that two parties with identical priors will reach the same rational decisions??? e.g. with equal information we will conclude the same thing "rational decision making is determinstic". But this isn't true unless you give them identical pasts too, or unless they reject any preference for self-consistency. There are a lot of pratical benefits of self consisency (things like ethics
78 2013-01-02 05:50:39 <andytoshi> oh, cool
79 2013-01-02 05:50:52 <andytoshi> not what i was thinking at all
80 2013-01-02 05:52:09 <stealth222> the interesting thing is that even the slightest difference in pasts can lead to a total divergence
81 2013-01-02 05:52:54 <stealth222> to the point where new information is construed in completely opposite ways
82 2013-01-02 05:53:44 <kuzetsa> stealth222: no I wasn't.
83 2013-01-02 05:54:13 <stealth222> lol
84 2013-01-02 05:54:15 <andytoshi> stealth222: right, much like a holonomy measures the curvature of a space
85 2013-01-02 05:54:23 <andytoshi> and -any- nonzero curvature will do something weird
86 2013-01-02 05:54:30 <andytoshi> see the picture on: https://en.wikipedia.org/wiki/Holonomy
87 2013-01-02 05:55:33 <kuzetsa> stealth222: sorry, you said something about "...new information is construed in completely opposite ways..." and I felt the need to troll by saying that thing I said.
88 2013-01-02 05:55:48 <stealth222> kuzetsa: I got it
89 2013-01-02 05:56:15 <andytoshi> ACTION ohhhhh
90 2013-01-02 05:56:16 <kuzetsa> nevermind... I'm a bumbling idiot at times.
91 2013-01-02 05:59:33 <stealth222> http://en.wikipedia.org/wiki/South-pointing_chariot
92 2013-01-02 06:02:38 <andytoshi> ACTION finally knows what a "differential gear" is
93 2013-01-02 06:02:42 <andytoshi> thanks, ancient china!
94 2013-01-02 07:54:54 <rozani> you are saying you are going to make the software stop preferring the longest available chain?
95 2013-01-02 08:44:23 <Luke-Jr> tcatm: ping, I think CampBX's market stream has the timestamps in non-UTC :/
96 2013-01-02 09:35:34 <epscy> does the faucet still work?
97 2013-01-02 09:35:51 <epscy> if it doesn't should the text on the page be updated to reflect that?
98 2013-01-02 10:41:22 <MC1984> Luke-Jr do you have that wikipedia list of gun self defense incidents
99 2013-01-02 10:45:11 <MC1984> http://en.wikipedia.org/w/index.php?title=List_of_defensive_gun_use_incidents&redirect=no oh its been deleted and redirected
100 2013-01-02 10:45:24 <MC1984> is that wikipedias famous liberal bias
101 2013-01-02 10:50:09 <stealth222> listener2 - even cleaner than before: https://github.com/CodeShark/CoinClasses/blob/master/examples/listener2/listener2.cpp
102 2013-01-02 11:26:40 <Jouke> So I downloaded 7.2, I downloaded bootstrap.dat and I copied my old wallet.dat. debug.log keeps saying: "ERROR: ProcessBlock() : AcceptBlock FAILED"
103 2013-01-02 11:28:01 <sipa> you only copied wallet.dat?
104 2013-01-02 11:28:19 <sipa> does it say anything why acceptblock fails?
105 2013-01-02 11:28:40 <Jouke> only wallet.dat yes
106 2013-01-02 11:29:02 <Jouke> ERROR: AcceptBlock() : AddToBlockIndex failed
107 2013-01-02 11:29:08 <Jouke> thath is the only other error
108 2013-01-02 11:29:43 <sipa> read-only filesystem?
109 2013-01-02 11:29:55 <sipa> disk full?
110 2013-01-02 11:30:23 <sipa> is there a file blkindex.dat created?
111 2013-01-02 11:31:33 <Jouke> yes, blk0001.dat is already 2G and 0002.dat is growing
112 2013-01-02 11:31:54 <stealth222> your clock set correctly?
113 2013-01-02 11:32:42 <Jouke> yes
114 2013-01-02 11:34:08 <sipa> you didn't answer my question :)
115 2013-01-02 11:35:26 <Jouke> yes I have a blkindex.dat and blk00001.dat is already 2G and 0002.dat is growing.
116 2013-01-02 11:35:39 <sipa> and that blkindex.dat didn't exist before?
117 2013-01-02 11:35:53 <sipa> and anything else at all in debug.log, even non-errors?
118 2013-01-02 11:36:58 <Jouke> sipa: clean machine install, I created .bitcoin directory, I copied bootstrap.dat and wallet.dat there before I started bitcoind.
119 2013-01-02 11:38:19 <sipa> that sounds like it should work
120 2013-01-02 11:38:26 <Jouke> besides the acceptblock errors I also get "ERROR: FetchInputs() : 6ace17537b mempool Tx prev not found 156333109a" - "stored orphan tx 6ace17537b (mapsz 159)
121 2013-01-02 11:38:32 <stealth222> stick some tracers into main.cpp and rebuild it :)
122 2013-01-02 11:38:35 <sipa> yeah, nothing to worry about
123 2013-01-02 11:39:01 <sipa> Jouke: can you find in debug.log the first few pages after startup?
124 2013-01-02 11:39:03 <Jouke> stealth222: I use the binary from bitcoin.org
125 2013-01-02 11:39:27 <sipa> i use a reverse-search for the string "Bitcoin" to do so
126 2013-01-02 11:42:40 <Jouke> http://pastebin.com/NWhyRPaH
127 2013-01-02 11:43:10 <stealth222> looking at the source (it might not be the same version), it looks like to get that error message, CBlock::ConnectBestBlock() has to return false - and that can only happen either if InvalidChainFound() returns true or SetBestChain() returns false
128 2013-01-02 11:43:34 <stealth222> better exception handling in main and init would be nice :)
129 2013-01-02 11:43:50 <sipa> Jouke: an in db.log?
130 2013-01-02 11:43:52 <sipa> *and
131 2013-01-02 11:45:00 <stealth222> wait, no sorry - SetBestChain has to be returning false
132 2013-01-02 11:45:13 <sipa> stealth222: use the 0.7.2 sourcem not git head
133 2013-01-02 11:45:20 <sipa> it changed quite a bit
134 2013-01-02 11:45:40 <sipa> the only return false from AddToBlockIndex without corresponding error message, is when there's a problem with BDB
135 2013-01-02 11:46:16 <Jouke> hmmm
136 2013-01-02 11:46:21 <Jouke> http://pastebin.com/u91qU13D
137 2013-01-02 11:47:32 <sipa> ok, so it seems related to the wallet file
138 2013-01-02 11:47:40 <stealth222> corrupt wallet file?
139 2013-01-02 11:47:45 <stealth222> :)
140 2013-01-02 11:47:56 <sipa> at least that won't be possible in 0.8 anymore (as the db envs for blockchain and wallet are separate)
141 2013-01-02 11:48:25 <stealth222> that's good news. I tried running the latest master with an empty wallet file and it crashed on me without giving me a nice error message
142 2013-01-02 11:48:51 <sipa> stealth222: anything in debug.log?
143 2013-01-02 11:48:55 <Jouke> hmm, ok, well, I do have some backups from the wallet.dat-file, so that isnt a problemn. Do I need to remove the blockchain-files and start over?
144 2013-01-02 11:49:19 <stealth222> lots, sipa - but it's like days ago. I'd have to search...or replicate the issue again
145 2013-01-02 11:49:49 <stealth222> It didn't even get to wallet loading
146 2013-01-02 11:49:56 <stealth222> just crashed
147 2013-01-02 11:52:10 <stealth222> actually, it does give me a relevant error message
148 2013-01-02 11:52:11 <sipa> most likely it still put something in debug.log
149 2013-01-02 11:52:11 <stealth222> Failed to rename wallet.dat to wallet.1357131112.bak
150 2013-01-02 11:52:48 <sipa> Jouke: probably best
151 2013-01-02 11:54:11 <stealth222> Failed to rename wallet.dat to wallet.1357131112.bak\\n: wallet.dat corrupt, salvage failed
152 2013-01-02 11:55:23 <gmaxwell> stealth222: what do you mean by 'empty wallet'?
153 2013-01-02 11:55:30 <stealth222> touch wallet.dat
154 2013-01-02 11:56:58 <sipa> interesting that BDB fails on zero-length database files
155 2013-01-02 11:57:10 <sipa> and apparently even fails when trying to rename them
156 2013-01-02 11:57:50 <sipa> though a better error message seems useful here, you made the wrong assumption that a 0-length file is a valid wallet :)
157 2013-01-02 11:58:01 <stealth222> no, I'm not assuming that :)
158 2013-01-02 11:58:29 <stealth222> what was strange to me is that it failed before it even reached the part of the code where the wallets are loaded
159 2013-01-02 11:58:48 <gmaxwell> Apparently it didn't since it's giving you the wallet salvage stuff.
160 2013-01-02 11:59:16 <gmaxwell> ACTION pats himself on the back for the excellent user distrust there uncovering the root cause :P
161 2013-01-02 11:59:25 <sipa> well the dbenv.dbrename(...) call fails on your wallet file
162 2013-01-02 11:59:30 <stealth222> it didn't even get to loading the block chain, gmaxwell
163 2013-01-02 11:59:39 <stealth222> wallet loading comes after that in InitApp2
164 2013-01-02 11:59:48 <sipa> wallet salvage is done before, iirc
165 2013-01-02 11:59:54 <stealth222> yes, I see that now, sipa
166 2013-01-02 12:00:14 <stealth222> it
167 2013-01-02 12:00:18 <stealth222> it's obviously done before :)
168 2013-01-02 12:01:58 <stealth222> step 5 :)
169 2013-01-02 12:02:42 <stealth222> should step 5 also try to salvage any other wallets that might be corrupt?
170 2013-01-02 12:03:49 <stealth222> then I'll have to grab the list of wallets in a separate step from the wallet loading
171 2013-01-02 12:05:20 <stealth222> I could easily add that
172 2013-01-02 12:05:29 <Luke-Jr> [12:54:35] <stealth222> Failed to rename wallet.dat to wallet.1357131112.bak\\n: wallet.dat corrupt, salvage failed <-- is there supposed to be a \\n in the filename? O.oi
173 2013-01-02 12:05:43 <stealth222> no, I added it because I didn't want to type two lines
174 2013-01-02 12:07:12 <Luke-Jr> the next line begins with a ":"?
175 2013-01-02 12:07:23 <stealth222> yes
176 2013-01-02 12:07:50 <gmaxwell> Hm. What default key should we use if the keypool is empty and the wallet is locked? Right now we fail with can't initilize keypool on a salvaged locked wallet that has no keypool (due to the salvage) because it can't fetch a key to use as the default key.
177 2013-01-02 12:09:32 <gmaxwell> ugh. https://www.btcfpga.com/forum/index.php?topic=806.msg2436 Another hardware maker goes the consolidation route.
178 2013-01-02 12:09:53 <stealth222> Luke-Jr: walletdb.cpp:607: printf("Failed to rename %s to %s\\n", filename.c_str(), newFilename.c_str());
179 2013-01-02 12:10:10 <stealth222> the \\n is not part of the filename
180 2013-01-02 12:10:32 <Luke-Jr> stealth222: ok, just confused about the colon that shouldn't(?) be there
181 2013-01-02 12:16:57 <stealth222> something to do with uiInterface?
182 2013-01-02 12:29:04 <stealth222> it's not clear on the surface which of all these functions that get called output to debug.log, which output to stdout, which output to GUI, etc...
183 2013-01-02 12:29:11 <stealth222> you really need to dig down
184 2013-01-02 12:29:31 <stealth222> it would be nice to make that all clearer
185 2013-01-02 12:29:50 <stealth222> you sometimes need to dig down the call stack several levels to see the call that generates the output
186 2013-01-02 12:32:55 <stealth222> but the bad part is that it's not clear which way to dig
187 2013-01-02 12:33:26 <stealth222> unless you grep like crazy :)
188 2013-01-02 12:34:23 <stealth222> looks like a string "Error" is getting removed from the output to debug.log
189 2013-01-02 12:34:55 <stealth222> I surrounded all calls to _() by double quotes and now I get this: "wallet.dat corrupt, salvage failed""Error": wallet.dat corrupt, salvage failed
190 2013-01-02 12:36:00 <Diapolo> _() is used to be able to translate strings from the core with Qt
191 2013-01-02 12:36:18 <stealth222> right - so it looks like in the translation, the "Error" is getting removed
192 2013-01-02 12:40:40 <stealth222> at least getting removed from the debug.log output.
193 2013-01-02 12:41:36 <Diapolo> take a look at noui.cpp AFAIK the function _() is in there
194 2013-01-02 12:42:39 <stealth222> the issue seems to be _("Error")
195 2013-01-02 12:43:12 <stealth222> to stdout it outputs "Error: wallet.dat corrupt, salvage failed"
196 2013-01-02 12:43:26 <stealth222> but to debug.log it removes "Error"
197 2013-01-02 12:43:44 <Diapolo> give me the line and file where that message comes from first
198 2013-01-02 12:44:17 <stealth222> noui.cpp: lines 30 and 31. no?
199 2013-01-02 12:44:30 <Diapolo> I had no IDE open :)
200 2013-01-02 12:44:43 <stealth222> I don't either - just using text editors and grep
201 2013-01-02 12:45:39 <stealth222> printf writes to debug.log while fprintf outputs to stderr, yes?
202 2013-01-02 12:46:08 <stealth222> so it looks like the caption string is empty
203 2013-01-02 12:46:29 <Diapolo> printf uses caption and the fprintf strCaption, right
204 2013-01-02 12:46:36 <stealth222> correct
205 2013-01-02 12:47:27 <Diapolo> and caption IS emptry indeed, as uiInterface.ThreadSafeMessageBox(str, "", CClientUIInterface::MSG_ERROR); has "" for it
206 2013-01-02 12:47:42 <stealth222> :)
207 2013-01-02 12:49:58 <stealth222> one thing I've done before in projects to avoid obscuring outputs is to pass function pointers to the output functions down the call stack but only call things like printf, fprintf, cout, etc at one level of the call stack
208 2013-01-02 12:51:20 <stealth222> then it's easy to turn on tracing so you can see where in the call stack the text originated
209 2013-01-02 12:52:27 <stealth222> I guess with the proper debugging tools it's possible to do that without a special framework
210 2013-01-02 12:52:38 <Diapolo> you are right, but that seems hard to do with such a project which grows where people like to work and extend or fix bugs
211 2013-01-02 12:54:03 <stealth222> yeah, it's hard enough to establish consistency in one's own code - let alone a project with no top-down organization involving a bunch of people
212 2013-01-02 12:54:34 <stealth222> every now and then, somebody needs to enforce consistency by refactoring code :)
213 2013-01-02 12:55:13 <Diapolo> I think you are very welcome, if you are able to help with that part
214 2013-01-02 12:55:14 <Diapolo> :)
215 2013-01-02 12:55:17 <gavinandresen> refactors are welcome, as long as you can convince us that you didn't accidently break something when refactoring
216 2013-01-02 12:55:52 <gavinandresen> lack of unit tests and 100% code coverage is a problem, though. So more unit tests that give more code coverage is even more welcome.
217 2013-01-02 13:00:45 <sipa> gavinandresen: what's your opinion about what to do when an inconsistency in the coin database is found?
218 2013-01-02 13:03:32 <stealth222> wouldn't it depend on the kind of inconsistency?
219 2013-01-02 13:03:33 <gavinandresen> sipa: mmmm.... on one hand, I don't like spending lots of time developing for extremely rare events. Which says to me, if coin db is corrupt erase it all and rebuild from blk data. (if blk data is corrupt, then erase and re-download the chain).
220 2013-01-02 13:03:48 <gavinandresen> sipa: on the other hand, doing that might hide real bugs
221 2013-01-02 13:03:53 <sipa> in the GUI something like "Your database seems corrupted. Choose continue to rebuild it now (this may take a while) [Continue] [Quit]"
222 2013-01-02 13:04:51 <sipa> stealth222: the case is "blocks on disk seem ok, but current coin database cannot possible ever result from those blocks"
223 2013-01-02 13:04:54 <gavinandresen> that sounds good for the GUI. Non-gui... always exit with a "coin db corrupt, erase /path/coins/ to recover" ?
224 2013-01-02 13:06:07 <sipa> Diapolo: is something like that possible with the messagebox stuff?
225 2013-01-02 13:06:42 <stealth222> yeah, I tend to agree with gavin - you can really spend a heck of a lot of time developing a beautiful recovery system that you only end up using once a million years
226 2013-01-02 13:07:09 <sipa> well, trying to recover isn't hard
227 2013-01-02 13:07:11 <stealth222> if it takes a minute rather than an hour to recover, was it really worth spending weeks of development time?
228 2013-01-02 13:07:23 <sipa> being sure that recovery was succesfully with hard guarantees is impossible
229 2013-01-02 13:07:44 <stealth222> if you have a good solution to the problem, then by all means use it :)
230 2013-01-02 13:08:00 <sipa> i don't - my question is mostly about how to present it to the user
231 2013-01-02 13:08:10 <gavinandresen> silently recovering is not the right answer, though; if my hard disk is failing I'd like to have some clues
232 2013-01-02 13:08:18 <sipa> indeed
233 2013-01-02 13:08:22 <Diapolo> sipa: sure, but currently not with the uiInterface stuff, as you can't evaluate what the user clicked
234 2013-01-02 13:08:27 <sipa> indeedah
235 2013-01-02 13:08:29 <sipa> ah
236 2013-01-02 13:08:59 <sipa> in case the blocks themselves are corrupted, we can try to reorganize away from the blocks, and drop them from the database
237 2013-01-02 13:09:32 <sipa> if that happens without detecting inconsistencies, it should be fine
238 2013-01-02 13:09:43 <sipa> if it doesn't -> reindex
239 2013-01-02 13:10:41 <gavinandresen> sounds hard to test; will you write a corrupt-blocks tool ?
240 2013-01-02 13:11:32 <gavinandresen> (which gets back to "I don't like spending lots of time....")
241 2013-01-02 13:12:38 <sipa> gavinandresen: i'm talking about -checkblocks
242 2013-01-02 13:12:53 <sipa> which already tests the last 2500 blocks for errors
243 2013-01-02 13:13:12 <sipa> (only standalone checks, of course)
244 2013-01-02 13:13:24 <gavinandresen> sipa: ok, I thought this was going to be a new check
245 2013-01-02 13:14:17 <sipa> the coindb check is new
246 2013-01-02 13:14:29 <gavinandresen> sipa: where are we at with the leveldb17 changes? Still doesn't build for you?
247 2013-01-02 13:14:34 <sipa> but it depends on knowing the blocks it tests against are themself ok
248 2013-01-02 13:14:53 <sipa> gavinandresen: i'll try updating the branch and rebuilding
249 2013-01-02 13:15:47 <stealth222> if you're getting serious data corruption problems, most likely scenarios seem to be 1) faulty hardware/system, 2) user tampering with data files, 3) malicious attack
250 2013-01-02 13:15:58 <stealth222> or 4) bugs in the program
251 2013-01-02 13:16:14 <stealth222> not necessarily in that order :)
252 2013-01-02 13:16:35 <sipa> stealth222: i think in the current (0.7.2) case, the most likely cause see,s "i didn't shutdown properly", which isn't nice, as it's exactly what bdb is supposed to protect against
253 2013-01-02 13:16:41 <sipa> *seems
254 2013-01-02 13:16:50 <stealth222> I guess I would put that in (4)
255 2013-01-02 13:17:34 <gmaxwell> I don't understand why it seems to be suddenly so much more of an issue, however.
256 2013-01-02 13:17:54 <sipa> i suppose the problem worsens with larger blkindex.dat
257 2013-01-02 13:18:32 <stealth222> point is, from the user's perspective, each one of these cases is very distinct
258 2013-01-02 13:19:34 <Diapolo> sipa: I would better call it my PC crashed while the client was running
259 2013-01-02 13:20:02 <sipa> Diapolo: i'm not convinced that that is the main cause
260 2013-01-02 13:22:26 <Diapolo> sipa: if you really think unclean shutdown is a problem, at least for Windows and Bitcoin-Qt I have something that prevents shutdown of the PC, safely quits the client and then allows the shutdown ... dunno if that's worth to get added needs for sure comments from laanwj
261 2013-01-02 13:22:46 <stealth222> but what if the system becomes unstable?
262 2013-01-02 13:23:29 <stealth222> BSOD territory
263 2013-01-02 13:23:49 <stealth222> there's a hardware conflict - or a buffer overrun somewhere
264 2013-01-02 13:25:22 <Diapolo> as I said I think PC crash IS an issue, perhaps much more on Win-PCs than for all other OSes at least with OS <= Vista
265 2013-01-02 13:31:21 <stealth222> btw, on a separate note, how soon do you think before we can integrate multiwallet and watch-only addresses? I'm debating whether to focus more effort on extending native bitcoind functionality or whether cleaning up and documenting CoinClasses better
266 2013-01-02 13:35:36 <sipa> Diapolo: bitcoin should simply be resilient against crashes/unexpected shutdowns whatsoever
267 2013-01-02 13:35:47 <Diapolo> sipa: indeed
268 2013-01-02 13:35:53 <sipa> stealth222: priority now is getting 0.8 ready, and new features typically delay that
269 2013-01-02 13:36:23 <sipa> though both are definitely things i want soon
270 2013-01-02 13:37:40 <stealth222> lots of people seem to be asking for callback functionality
271 2013-01-02 13:37:45 <Diapolo> sipa: what changes are to come for the LevelDB 1.7 branch? I did not succeed in build a static Qt and still suffer from that static switches in the pro-file :-/.
272 2013-01-02 13:38:16 <sipa> ha! pro-file!
273 2013-01-02 13:38:33 <sipa> Diapolo: well gavin was recently able to build and run on XP
274 2013-01-02 13:38:34 <Scrat> stealth222: how does bitcoin-qt check for new transactions? does it poll? how often
275 2013-01-02 13:39:04 <sipa> Scrat: the core notifies the GUI when there are changes to the wallet
276 2013-01-02 13:39:09 <gavinandresen> Diapolo: I didn't build -qt, though.
277 2013-01-02 13:39:35 <Diapolo> gavinandresen: you built bitcoind then?
278 2013-01-02 13:39:59 <Scrat> sipa: sorry for my noobness but where is that in the call list
279 2013-01-02 13:40:31 <sipa> Scrat: Bitcoin-Qt doesn't use the RPC mechanism
280 2013-01-02 13:40:43 <sipa> it's the same binary, it just gets an internal function call
281 2013-01-02 13:40:53 <stealth222> yeah, in-process it's not hard to provide the hooks
282 2013-01-02 13:41:06 <Scrat> so it should be easy to expose that through RPC
283 2013-01-02 13:41:18 <sipa> out of process it isn't hard either, but it's hard to make it a) usable b) safe against DoS attacks
284 2013-01-02 13:42:29 <stealth222> I guess usability depends on your target. what I have is somewhere between the python script stuff and hacking into the core itself
285 2013-01-02 13:42:38 <stealth222> https://github.com/CodeShark/CoinClasses/blob/master/examples/listener2/listener2.cpp
286 2013-01-02 13:42:51 <Scrat> JSON RPC is stateless anyway so it will have to be through polling
287 2013-01-02 13:42:56 <stealth222> still not usable by, say, a typical PHP developer
288 2013-01-02 13:44:02 <stealth222> and JSON-RPC could provide callback mechanisms via subscription
289 2013-01-02 13:44:18 <stealth222> "post to the following URL when the following event occurs:...."
290 2013-01-02 13:44:22 <Scrat> I dont know what is the preferred method for async cross platform IPC nowadays
291 2013-01-02 13:45:16 <stealth222> easiest method for typical web app development is to send a post to a specific URL with event information when an event occurs
292 2013-01-02 13:45:57 <Scrat> yes post will do I guess, with a internal queue for retries in case of failure
293 2013-01-02 13:46:12 <stealth222> if you want to provide live feeds to a browser, then websockets
294 2013-01-02 13:47:25 <stealth222> but the post mechanism is really what is most needed for more sophisticated web app development
295 2013-01-02 13:47:34 <stealth222> urgently :)
296 2013-01-02 13:47:38 <Scrat> then the question is: do JSON callbacks survive restarts?
297 2013-01-02 13:47:48 <Diapolo> sipa: QMAKE_CXXFLAGS *= -flto=4 QMAKE_LFLAGS *= -flto=4 fails with lto-wrapper failed so also not usable for me -_-
298 2013-01-02 13:47:53 <Scrat> (are they saved in the wallet?)
299 2013-01-02 13:48:07 <stealth222> it's not currently implemented, scrat - it could be implemented in a number of ways
300 2013-01-02 13:48:08 <sipa> Scrat: there are no JSON callbacks
301 2013-01-02 13:48:13 <Scrat> I know :p
302 2013-01-02 13:48:33 <stealth222> my implementation will automatically requery for anything it missed since it was shut down
303 2013-01-02 13:48:37 <Scrat> I'm entertaining stealth222's idea
304 2013-01-02 13:49:44 <stealth222> Scrat, callback subscriptions are persisted to disk
305 2013-01-02 13:50:19 <stealth222> and all failed post attempts are logged
306 2013-01-02 13:50:50 <stealth222> I'd love to add these features to the reference client
307 2013-01-02 13:51:48 <Scrat> new dependencies will be needed
308 2013-01-02 13:51:50 <Scrat> prob. libcurl
309 2013-01-02 13:52:13 <stealth222> sure. unless you know of any better HTTP client
310 2013-01-02 13:52:29 <gavinandresen> you can do it all with boost. I had a monitorreceived patch that was never merged because I decided it was feature creep
311 2013-01-02 13:52:47 <Scrat> oh right, boost has a client
312 2013-01-02 13:53:03 <stealth222> feature creep? this was actually one of the most useful things I needed when I first started trying to build bitcoin apps :)
313 2013-01-02 13:53:10 <Scrat> +1
314 2013-01-02 13:53:11 <stealth222> and it seems I'm not alone in this opinion
315 2013-01-02 13:53:18 <Scrat> first think I looked at too
316 2013-01-02 13:53:21 <Scrat> thing*
317 2013-01-02 13:54:02 <gavinandresen> so, you start with plain HTTP POST to a web server, then you decide that you need SSL because a MITM might inject fake transactions and rip off your service...
318 2013-01-02 13:54:02 <stealth222> I needed to be able to send notifications of receipts of payments and take appropriate action - and having to periodically poll via RPC just didn't cut it
319 2013-01-02 13:54:15 <stealth222> SSL is a given
320 2013-01-02 13:54:17 <gavinandresen> ... and if you need SSL then you need client/server certificates and support for that....
321 2013-01-02 13:54:31 <stealth222> yes, but it can be self-signed
322 2013-01-02 13:54:35 <gavinandresen> ... and soon you've added thousands of lines of code to the reference client, and that's a bad idea.
323 2013-01-02 13:54:49 <stealth222> adding SSL certificates doesn't add thousands of lines
324 2013-01-02 13:55:26 <sipa> meh, i'd be in favor of removing all SSL support whatsoever from the client, and use stunnel if you need SSL stuff
325 2013-01-02 13:55:36 <sipa> but i guess i'm alone with that opinion :)
326 2013-01-02 13:55:39 <gmaxwell> stealth222: openssl doesn't really do certificate validation. It's a fair amount of code??? and code you make active from a library is still a kind of addition.
327 2013-01-02 13:55:40 <gavinandresen> sipa: I agree, and I wrote the RPC SSL code.
328 2013-01-02 13:56:15 <gmaxwell> 09:42 < gmaxwell> Rebroad: if you run bitcoin over tor or stunnel you should be fine. :(
329 2013-01-02 13:56:18 <gmaxwell> 12:44 < gmaxwell> well, IMO it's a stupid feature. stunnel ftw. :P
330 2013-01-02 13:56:38 <stealth222> so fine - stunnel. in any case, the features could be supported and the how-to could be documented well enough for your typical web developer to use it
331 2013-01-02 13:57:09 <Scrat> so a plain http client with a big ass disclaimer: use stunnel if you want to use it to post to a different machine
332 2013-01-02 13:57:24 <gavinandresen> we've got a pending pull request on transaction notification, there is a discussion there that is relevant
333 2013-01-02 13:57:44 <gavinandresen> https://github.com/bitcoin/bitcoin/pull/1974
334 2013-01-02 13:58:35 <stealth222> yes, I've seen that - that was actually one of the primary motivations for https://github.com/bitcoin/bitcoin/pull/2121
335 2013-01-02 13:59:23 <stealth222> shell execs are not the smoothest approach but it's better than nothing
336 2013-01-02 14:00:30 <gavinandresen> I still like the idea of writing to a named pipe. Then running whatever you like on the other end of that pipe to do web notifications, jabber notifications, carrier pigeon notifications....
337 2013-01-02 14:00:50 <stealth222> that's essentially what I ended up doing, gavin :)
338 2013-01-02 14:00:57 <sipa> except one popular operating system has no pipes support to date
339 2013-01-02 14:01:01 <sipa> afaik
340 2013-01-02 14:01:19 <stealth222> except no named pipes - just bitcoin protocol
341 2013-01-02 14:01:20 <gavinandresen> windows has something similar, if I recall correctly (always a bad assumption)
342 2013-01-02 14:01:39 <stealth222> tack on a few custom filters and then send your posts to other applications
343 2013-01-02 14:02:07 <gavinandresen> stealth222: I much prefer that approach to making the reference client do everything. Small tools, each designed to do one thing well...
344 2013-01-02 14:02:47 <stealth222> so then perhaps it would be a good idea to continue developing this as a completely separate app from bitcoind
345 2013-01-02 14:03:14 <gavinandresen> sure. We've got a contrib/ directory if it is widely useful
346 2013-01-02 14:03:45 <kjj> windows has something they like to think of as named pipes, but they are not useful to us
347 2013-01-02 14:04:03 <sipa> well, it can be generalized to "write to file"
348 2013-01-02 14:04:16 <stealth222> it's been widely useful to me, gavin. could probably be useful to others, too, with a little more documentation and a few more tools to install and configure it
349 2013-01-02 14:04:18 <sipa> that file may be a named pipe, or a socket, or just a regular file
350 2013-01-02 14:04:22 <kjj> and even UNIX named pipes have lots of issues in this instance
351 2013-01-02 14:05:31 <kjj> there are blocking issues, etc. things get messy
352 2013-01-02 14:05:45 <stealth222> my hope was to build a lightweight daemon that can subscribe event types and provide callbacks
353 2013-01-02 14:05:50 <gavinandresen> kjj: but those are just general "I'm doing IO" issues
354 2013-01-02 14:06:06 <sipa> gavinandresen: /home/ubuntu/build/bitcoin/src/leveldb/libleveldb.a(env_win.o):env_win.cc:(.text+0x19a6): undefined reference to `__imp__PathFileExistsW@4'
355 2013-01-02 14:06:18 <sipa> with my current leveldb17 branch (which has your latest changes)
356 2013-01-02 14:06:28 <gavinandresen> sipa: building with which mingw?
357 2013-01-02 14:06:28 <sipa> strangely enough... only when linking bitcoind.exe
358 2013-01-02 14:06:35 <sipa> bitcoin-qt.exe was done succesfully
359 2013-01-02 14:06:41 <sipa> gitian, mingw-w64
360 2013-01-02 14:06:54 <gavinandresen> mmm. would it be easy to switch back to mingw32 ?
361 2013-01-02 14:06:55 <kjj> well... we have to dump lots and lots of C++ abstraction to do named pipes using the lowest level open calls, and then we have to sliently discard writes if the pipe reports that it would block
362 2013-01-02 14:07:21 <stealth222> what's wrong with regular good old fashioned TCP sockets? :)
363 2013-01-02 14:07:34 <sipa> stealth222: too little overhead :p
364 2013-01-02 14:09:25 <gavinandresen> I don't care what is used, as long as it will scale up to handling hundreds of txns per second
365 2013-01-02 14:10:07 <stealth222> post-filtering, most uses will not require that volume
366 2013-01-02 14:10:08 <kjj> writing to just a dumb file isn't horrible, but at that point we should just put a marker in the log file
367 2013-01-02 14:10:16 <stealth222> pre-filtering, sure
368 2013-01-02 14:11:29 <Scrat> is CWallet::AddToWallet() called on every confirmation?
369 2013-01-02 14:11:34 <sipa> no
370 2013-01-02 14:11:44 <sipa> it's called to add things to the wallet
371 2013-01-02 14:11:52 <stealth222> it
372 2013-01-02 14:12:12 <stealth222> argh, keyboard
373 2013-01-02 14:13:31 <Scrat> what's an updated transaction then?
374 2013-01-02 14:14:14 <sipa> how do you mean?
375 2013-01-02 14:14:24 <kjj> in practice, you get 1 notification when the transaction shows up in your memory pool, 1 notification when a block arrives with the transaction, and then when any transaction redeems from your wallet (mempool or block)
376 2013-01-02 14:16:01 <Scrat> sipa: wallet.cpp L477: // Notify UI of new or updated transaction
377 2013-01-02 14:16:41 <kjj> transactions have sequence numbers and can be replaced (if the sequence number is not MAX)
378 2013-01-02 14:16:54 <stealth222> but that feature is not currently supported
379 2013-01-02 14:17:04 <kjj> sure it is
380 2013-01-02 14:17:08 <stealth222> it is?
381 2013-01-02 14:17:11 <sipa> Scrat: for example if it's found in a block, even when it was found in another block before
382 2013-01-02 14:17:22 <kjj> it just isn't very useful right now
383 2013-01-02 14:17:27 <sipa> transaction replacement has been disabled for a very long time
384 2013-01-02 14:17:41 <sipa> transaction locking works fine though
385 2013-01-02 14:20:06 <Scrat> I like kjj's snippet, very simple. the only drawback is the overhead which might make you easier to DoS
386 2013-01-02 14:20:12 <kjj> ok, well, the code is still there.
387 2013-01-02 14:22:30 <stealth222> I'm all for splitting up the reference client into a few separate applications and having different build options
388 2013-01-02 14:22:31 <stealth222> :)
389 2013-01-02 14:23:16 <kjj> the problem with having a different application connect over p2p for monitoring is that you have to duplicate a big chunk of the system to do that
390 2013-01-02 14:23:52 <stealth222> mostly the network serialization code
391 2013-01-02 14:23:58 <stealth222> but that's about it
392 2013-01-02 14:24:18 <kjj> if you want transaction monitoring, you need to be able to read and understand transactions, which is not trivial
393 2013-01-02 14:24:25 <stealth222> if the listener application trusts the peer, it doesn't have to do its own verification
394 2013-01-02 14:24:50 <stealth222> it doesn't have to have a full implementation of the bitcoin script
395 2013-01-02 14:25:24 <stealth222> the actual data structures are fairly simple
396 2013-01-02 14:25:45 <stealth222> the tricky part is verifying messages from untrusted peers
397 2013-01-02 14:27:56 <stealth222> to do a full verification, not only do you need to have an implementation of the bitcoin script - you also have to have a block chain engine, memory pool, etc...
398 2013-01-02 14:28:39 <stealth222> but assuming you connect to a peer that handles that for you, all you really need to do is pass the messages to applications in a format they can easily understand
399 2013-01-02 14:29:06 <stealth222> deserialize and stick into JSON or something
400 2013-01-02 14:30:00 <Scrat> stealth222: lite / developer versions would make a lot of sense. Why should the normal client include bitcoind? Average joe won't be needing the headless version
401 2013-01-02 14:30:26 <kjj> well, one of your layers is going to need to be able to understand transactions if you want it to be able to notify you of specific ones
402 2013-01-02 14:30:52 <stealth222> my typical use scenario would be a merchant or online service provider that wants to run its own full nodes but have lightweight listener nodes that handle things like sending notifications and alerts
403 2013-01-02 14:31:57 <stealth222> furthermore, someone who wants to separate their wallet functionality entirely from network relay functionality
404 2013-01-02 14:32:17 <stealth222> separate nodes for verification/block chain management and nodes for signing transactions
405 2013-01-02 14:32:19 <kjj> more diverse and specialized software is a good thing, but right now, if you are running a full node, getting transaction notification out of that node is trivial
406 2013-01-02 14:32:52 <stealth222> kjj, not with the reference client as it currently exists
407 2013-01-02 14:33:27 <stealth222> but even assuming you throw in the hooks for the IPC
408 2013-01-02 14:33:41 <stealth222> I certainly don't want to be running my web servers on the same machines as I run my wallets
409 2013-01-02 14:33:48 <stealth222> or my verification nodes
410 2013-01-02 14:34:40 <stealth222> my architecture consists of a few well-protected signing nodes, a few anonymous relay nodes, and a few listener/filter nodes
411 2013-01-02 14:35:47 <stealth222> running a monolithic application that handles block chain management/wallets/and alerts doesn't really make sense
412 2013-01-02 14:36:03 <kjj> true, but the patch is trivial. my stance is that it solves the problem "well enough" for 98% of the people that want it
413 2013-01-02 14:36:27 <stealth222> it's not so trivial to do correctly
414 2013-01-02 14:36:46 <stealth222> most of the apps I've seen out there are horribly hacked together
415 2013-01-02 14:37:07 <stealth222> ready to be exploited in a million ways by someone who cares to attack it - or simply not scalable
416 2013-01-02 14:37:14 <stealth222> or impossible to maintain
417 2013-01-02 14:37:20 <stealth222> and manage
418 2013-01-02 14:38:48 <stealth222> just thing of all the ad hoc tools for moving wallets arund
419 2013-01-02 14:38:50 <stealth222> *around
420 2013-01-02 14:39:04 <kjj> getting from here to there will take some time and effort, but it seems very likely to me that at some point in the future, the "standard bitcoin install" will be three or four different programs
421 2013-01-02 14:39:46 <stealth222> it's obvious that the typical enduser wallet is a very different app from the high volume merchant core transaction processing engine
422 2013-01-02 14:39:55 <kjj> I see a network daemon, a storage and validation daemon, and a wallet/UI application
423 2013-01-02 14:41:06 <stealth222> a core transaction processing engine needs to support high concurrency and access controls
424 2013-01-02 14:42:00 <stealth222> an enduser wallet app doesn't really care about concurrency - and the access controls are very simple
425 2013-01-02 14:43:41 <stealth222> then there's the whole mining side to bitcoin - and unless you are a miner, you probably couldn't care less about running that code
426 2013-01-02 14:44:02 <kjj> meh. mining is cheap if you already have the other parts
427 2013-01-02 14:44:28 <stealth222> point is, a dedicated mining app that's separate from, say, signing nodes makes a lot of sense :)
428 2013-01-02 14:44:53 <stealth222> rarely do you ever want those two things happening within the same process
429 2013-01-02 14:45:23 <kjj> with GBT, all of the necessary block info is available on the storage/validation process
430 2013-01-02 14:46:15 <stealth222> signing could be super extremely lightweight - even implemented on sdcards and such
431 2013-01-02 14:46:26 <kjj> and you need the entire storage/validation system for mining anyway, so presumably that is where the GBT call would reside
432 2013-01-02 14:47:47 <kjj> yeah, signing can be very light, and already works over RPC
433 2013-01-02 14:47:55 <stealth222> yeah, a miner needs to connect to a trusted peer that performs validation
434 2013-01-02 14:49:25 <sipa> gavinandresen: heh, even without -posix i get that error now
435 2013-01-02 14:49:41 <sipa> gavinandresen: i wonder what changed... i've done succesful leveldb17 builds before
436 2013-01-02 14:50:50 <stealth222> getting linker errors, sipa?
437 2013-01-02 14:51:14 <sipa> yes
438 2013-01-02 14:51:20 <sipa> /home/ubuntu/build/bitcoin/src/leveldb/libleveldb.a(env_win.o):env_win.cc:(.text+0x3fdd): undefined reference to `__imp__PathFileExistsW@4'
439 2013-01-02 14:52:16 <gavinandresen> sipa is cross-compiling for windows
440 2013-01-02 14:52:36 <sipa> this makes no sense at all
441 2013-01-02 14:52:48 <stealth222> can you build it in windows?
442 2013-01-02 14:52:50 <sipa> it is the *same* .a file being linked into bitcoin-qt.exe and bitcoind.exe
443 2013-01-02 14:52:52 <gavinandresen> I bet it will be easy when you figure out what is wrong
444 2013-01-02 14:53:03 <sipa> and in bitcoin-qt.exe it works
445 2013-01-02 14:53:05 <stealth222> that's usually the case with these errors, gavin :)
446 2013-01-02 14:53:21 <sipa> stealth222: i have no clue about how to begin trying to build something in windows
447 2013-01-02 14:53:25 <gavinandresen> that's what makes them so frustrating... six hours spent tracking down something trivial....
448 2013-01-02 14:53:29 <sipa> and i prefer not to :)
449 2013-01-02 14:53:54 <stealth222> you spend hours tearing your hair out and then realize it's a typo or spaces where a tab was expected
450 2013-01-02 14:54:26 <sipa> haha
451 2013-01-02 14:55:13 <gavinandresen> sipa: you're starting with a clean build environment? No stray libleveldb.a's in other directories?
452 2013-01-02 14:55:38 <sipa> gavinandresen: gitian is about as clean as it can get
453 2013-01-02 14:55:46 <gavinandresen> yeah, that's what I figured
454 2013-01-02 14:56:26 <sipa> i686-w64-mingw32-g++ -frandom-seed=bitcoin -Wl,--dynamicbase -Wl,--nxcompat -static-libgcc -static-libstdc++ -Wl,-s -lmingwthrd -Wl,-subsystem,windows -o release/bitcoin-qt.exe object_script.bitcoin-qt.Release -L'/home/ubuntu/qt//lib' -lmingwthrd -lmingw32 -lqtmain build/bitcoin-qt_res.o -lqrencode -L/home/ubuntu/build/miniupnpc -lminiupnpc -liphlpapi /home/ubuntu/build/bitcoin/src/leveldb/libleveldb.a...
455 2013-01-02 14:56:33 <sipa> /home/ubuntu/build/bitcoin/src/leveldb/libmemenv.a -L/home/ubuntu/build/boost_1_52_0/stage/lib -L/home/ubuntu/build/db-4.8.30.NC/build_unix -L/home/ubuntu/build/openssl-1.0.1c -L/home/ubuntu/build/qrencode-3.2.0/.libs -lssl -lcrypto -ldb_cxx -lshlwapi -lmswsock -lboost_system-mt-s -lboost_filesystem-mt-s -lboost_program_options-mt-s -lboost_thread_win32-mt-s -L/home/ubuntu/qt//plugins/codecs -lqcncodecs -lqjpcodecs -lqtwcodecs -lqkrcodecs...
456 2013-01-02 14:56:39 <sipa> -L/home/ubuntu/qt//plugins/accessible -lqtaccessiblewidgets -lQtGui -lgdi32 -lcomdlg32 -loleaut32 -limm32 -lwinmm -lwinspool -lmsimg32 -lQtCore -lole32 -luuid -lws2_32 -ladvapi32 -lshell32 -luser32 -lkernel32
457 2013-01-02 14:56:46 <sipa> that's the Qt link command
458 2013-01-02 14:56:59 <sipa> i686-w64-mingw32-g++ -O2 -w -Wall -Wextra -Wformat -Wformat-security -Wno-unused-parameter -frandom-seed=bitcoin -D_MT -DWIN32 -D_WINDOWS -DBOOST_THREAD_USE_LIB -DBOOST_SPIRIT_THREADSAFE -DSTATICLIB -DUSE_UPNP=0 -DUSE_IPV6=1 -I"/home/ubuntu/build/bitcoin/src/leveldb/include" -I"/home/ubuntu/build/bitcoin/src/leveldb/helpers" -DHAVE_BUILD_INFO -I"/home/ubuntu/build/bitcoin/src" -I"/home/ubuntu/build/bitcoin/src"/obj...
459 2013-01-02 14:57:05 <sipa> -I"/home/ubuntu/build/boost_1_52_0" -I"/home/ubuntu/build/db-4.8.30.NC/build_unix" -I"/home/ubuntu/build/openssl-1.0.1c/include" -I"/home/ubuntu/build" -Wl,--dynamicbase -Wl,--nxcompat -static-libgcc -static-libstdc++ -o bitcoind.exe -L"/home/ubuntu/build/boost_1_52_0/stage/lib" -L"/home/ubuntu/build/db-4.8.30.NC/build_unix" -L"/home/ubuntu/build/openssl-1.0.1c" -L"/home/ubuntu/build/miniupnpc" obj/alert.o obj/version.o obj/checkpoints.o...
460 2013-01-02 14:57:11 <sipa> obj/netbase.o obj/addrman.o obj/crypter.o obj/key.o obj/db.o obj/init.o obj/irc.o obj/keystore.o obj/main.o obj/net.o obj/protocol.o obj/bitcoinrpc.o obj/rpcdump.o obj/rpcnet.o obj/rpcmining.o obj/rpcwallet.o obj/rpcblockchain.o obj/rpcrawtransaction.o obj/script.o obj/sync.o obj/util.o obj/wallet.o obj/walletdb.o obj/noui.o obj/leveldb.o obj/txdb.o -l boost_system-mt-s -l boost_filesystem-mt-s -l boost_program_options-mt-s -l...
461 2013-01-02 14:57:17 <sipa> boost_thread_win32-mt-s -l boost_chrono-mt-s -l db_cxx -l ssl -l crypto -l miniupnpc -l iphlpapi -l mingwthrd -l kernel32 -l user32 -l gdi32 -l comdlg32 -l winspool -l winmm -l shell32 -l comctl32 -l ole32 -l oleaut32 -l uuid -l rpcrt4 -l advapi32 -l ws2_32 -l mswsock -l shlwapi /home/ubuntu/build/bitcoin/src/leveldb/libleveldb.a /home/ubuntu/build/bitcoin/src/leveldb/libmemenv.a
462 2013-01-02 14:57:23 <sipa> that's the bitcoind one
463 2013-01-02 14:57:34 <sipa> anyone see something relevant missing?
464 2013-01-02 14:57:44 <gavinandresen> order of libraries?
465 2013-01-02 14:57:59 <gavinandresen> shouldn't libleveldb.a come before the windows/system libs?
466 2013-01-02 14:58:53 <sipa> hmm
467 2013-01-02 14:59:44 <sipa> that should have caused problems long before, imho, if it mattered
468 2013-01-02 14:59:57 <sipa> i'll try
469 2013-01-02 15:02:28 <stealth222> the other day I was getting some nasty linker error and it turned out it was because I had accidentally capitalized a letter in one of my environment variables
470 2013-01-02 15:03:00 <stealth222> took me like three hours to figure out
471 2013-01-02 15:28:47 <sipa> gavinandresen: success!
472 2013-01-02 15:29:10 <gavinandresen> huzzah!
473 2013-01-02 15:29:29 <stealth222> what was it? order?
474 2013-01-02 15:31:43 <jgarzik> chaos
475 2013-01-02 15:31:56 <stealth222> z -> z^2 + c
476 2013-01-02 15:32:09 <sipa> stealth222: yup
477 2013-01-02 15:32:14 <D34TH> jgarzik, chaos is fair
478 2013-01-02 15:32:50 <sipa> there used to be "LIBS += libleveldb.a -lshlwapi", and that -lshlwapi was removed as we linked against that anyway
479 2013-01-02 15:33:15 <stealth222> lol - I had actually run into a link that talked about shlwapi
480 2013-01-02 15:33:17 <sipa> but having -lshlwapi twice on the command line doesn't hurt, and made sure the correct order was there
481 2013-01-02 15:33:40 <stealth222> http://cboard.cprogramming.com/cplusplus-programming/124759-compile-error-shlwapi-library.html
482 2013-01-02 15:33:46 <stealth222> the article isn't particularly helpful
483 2013-01-02 15:33:53 <stealth222> but it does mention shlwapi
484 2013-01-02 15:40:37 <sipa> gavinandresen: http://bitcoin.sipa.be/builds/leveldb17/0.7.1-288-gca17521/
485 2013-01-02 15:40:48 <sipa> if you want to try whether the (null) issue is fixed
486 2013-01-02 15:41:08 <gavinandresen> sipa: sure, I'll fire up my VM...
487 2013-01-02 15:44:03 <gavinandresen> sipa: installed fine, started OK, then crashed...
488 2013-01-02 15:44:58 <sipa> grrrr
489 2013-01-02 15:45:21 <gavinandresen> I'll see if I can figure out where it is crashing
490 2013-01-02 15:45:40 <sipa> i assume in snprintf...
491 2013-01-02 15:46:17 <gavinandresen> mmm, probably....
492 2013-01-02 15:46:52 <gavinandresen> no symbols, but gdb says it is wscanf
493 2013-01-02 15:47:04 <gavinandresen> ... yeah, in vsnprintf
494 2013-01-02 15:47:33 <gavinandresen> vsnprintf from C:\\WINDOWS\\system32\\msvcrt.dll
495 2013-01-02 15:48:18 <gavinandresen> I thought mingw had a vsnprintf replacement... wonder if it just tweaks the formatting and calls the system vsnprintf....
496 2013-01-02 15:49:19 <gavinandresen> sipa: and the filename.(null) problem still exists
497 2013-01-02 15:50:04 <sipa> hmmm, it certainly was compiled with -posix
498 2013-01-02 15:50:29 <sipa> maybe this version of mingw doesn't have that replacement yet?
499 2013-01-02 15:50:37 <gavinandresen> could be-- when did that version fork?
500 2013-01-02 15:51:23 <sipa> no idea
501 2013-01-02 15:52:53 <gavinandresen> FYI: my compile of your leveldb17 branch using mingw32/g++ 4.6.2 works nicely
502 2013-01-02 15:53:26 <gavinandresen> (that's compiled inside the windows VM, not cross-compiled)
503 2013-01-02 15:53:52 <gavinandresen> afk for a bit, lunch
504 2013-01-02 15:54:44 <Luke-Jr> I don't think mingw replaces printf stuff; Windows has a more up-to-date version in another DLL, but I don't know if there's a good way to use it in mingw
505 2013-01-02 15:55:09 <sipa> mingw at least has a workaround
506 2013-01-02 15:55:19 <sipa> which works for gavin but not for me
507 2013-01-02 15:55:32 <sipa> that workaround may be a replacement, or a wrapper
508 2013-01-02 16:04:05 <sipa> gavinandresen: bleh
509 2013-01-02 16:06:28 <sipa> seems there is a 4.6.3 mingw32 g++ as well in ubuntu
510 2013-01-02 16:07:34 <Jouke> Is it normal that RPC doesn't work while it is catching up with the blockchain?
511 2013-01-02 16:07:44 <sipa> Jouke: it should work
512 2013-01-02 16:07:57 <sipa> not while loading block from disk, at startup
513 2013-01-02 16:08:04 <sipa> but blockchain sync is during normal operation
514 2013-01-02 16:08:34 <gmaxwell> etotheipi_: I came up with another use for your provably-part-of-some-chain stuff. It would be nice if gribble (or some tip form on bitcointalk) could issue new addresses for me to people that ask for them, without them having to trust that gribble/bitcointalk isn't lying. Assuming there is some well known key-identity for me, this is possible.
515 2013-01-02 16:08:46 <Jouke> sipa: it is past the bootstrap blocks
516 2013-01-02 16:09:00 <sipa> Jouke: what exactly fails or happens?
517 2013-01-02 16:09:20 <sipa> certain calls are disabled during IBD, like getwork, afaik
518 2013-01-02 16:09:40 <Jouke> getinfo
519 2013-01-02 16:09:50 <sipa> ok, and what happens?
520 2013-01-02 16:10:03 <Jouke> no response from server
521 2013-01-02 16:11:21 <gmaxwell> Jouke: whats going on in the log right now?
522 2013-01-02 16:12:33 <sipa> gavinandresen: i suppose i should redo the change to mingw-w64 to use gcc-mingw32 (which is 4.6.3, while mingw32 itself is 4.2.1...) instead, but bleh i'm really getting tired of this
523 2013-01-02 16:12:51 <Jouke> last few lines has "setbestchain" and "addtowallet"
524 2013-01-02 16:13:11 <sipa> and is it still doing something?
525 2013-01-02 16:13:36 <Jouke> still catching up with the blockchain yes
526 2013-01-02 16:14:14 <gmaxwell> what height is it at?
527 2013-01-02 16:15:02 <Jouke> height=212899
528 2013-01-02 16:21:31 <Jouke> Oh, hmm, I am missing certificate files.
529 2013-01-02 16:24:08 <gmaxwell> Jouke: why are you using SSL RPC?
530 2013-01-02 16:24:24 <sipa> because it's encrypted!!!
531 2013-01-02 16:25:45 <gmaxwell> Well, I'm asking because if the answer is "because I'm going to expose my rpc port to the internet" I wanted to have the chance to say "don't do it!"
532 2013-01-02 16:30:22 <sipa> gavinandresen: i'm going crazy; there are 3 packages that provide a mingw gcc: mingw32, gcc-mingw32 and gcc-mingw-w64-i686
533 2013-01-02 16:30:51 <Luke-Jr> O.o
534 2013-01-02 16:31:18 <sipa> gavinandresen: now, things like ar and ld and windres are provided by mingw32-binutils or binutils-mingw-w64-i686
535 2013-01-02 16:31:25 <Diapolo> sipa: any chance I can help you out with version info from my local MinGW? Well it's installed on Windows, so I guess no.
536 2013-01-02 16:31:51 <gmaxwell> what is mingw32 vs gcc-mingw32? Fedora avoided this insanity by only packaging gcc-mingw-w64-*.
537 2013-01-02 16:31:51 <sipa> gavinandresen: unfortunately mingw32-binutils conflicts with gcc-mingw32
538 2013-01-02 16:32:08 <Jouke> gmaxwell: I am.
539 2013-01-02 16:32:18 <gmaxwell> Jouke: Don't do it!
540 2013-01-02 16:32:29 <epscy> if you are going to do it
541 2013-01-02 16:32:34 <sipa> gmaxwell: afaik the only difference is that mingw32 is a single package that provides an old version of everything
542 2013-01-02 16:32:35 <epscy> can i have your ip?
543 2013-01-02 16:32:59 <sipa> gmaxwell: while gcc-mingw32 is more recent, but depends partially on the w64 toolchain
544 2013-01-02 16:33:34 <sipa> the w64 toolchain seems nicely split out and sane
545 2013-01-02 16:33:44 <sipa> but apparently doesn't have the exact fix we need
546 2013-01-02 16:34:27 <Jouke> gmaxwell: why?
547 2013-01-02 16:36:13 <sipa> gavinandresen: oh, gcc-mingw32 is just a wrapper that actually uses the w64 gcc
548 2013-01-02 16:36:25 <gmaxwell> Jouke: because it's a large, sensitive surface area which has not had as much review as the normally network exposed surface of bitcoin.
549 2013-01-02 16:38:30 <Jouke> well, I only accept one or two IP's.
550 2013-01-02 16:38:40 <sipa> gavinandresen: i think the easiest solution is adding macros instead of %llX stuff to leveldb source *puke*
551 2013-01-02 16:38:57 <Diapolo> Gavin are you finished with updating that LevelDB 1.7 commit yet?
552 2013-01-02 16:39:00 <epscy> Jouke: can you add mine
553 2013-01-02 16:39:17 <Jouke> epscy: sure, would you like the walletpassphrase as well?
554 2013-01-02 16:39:19 <sipa> Diapolo: we're not done updating as long as it doesn't work :)
555 2013-01-02 16:39:42 <Diapolo> what exactly is not working currently?
556 2013-01-02 16:39:53 <Diapolo> that disk space problem in coins?
557 2013-01-02 16:39:53 <Jouke> but, how do I stop the client cleanly?
558 2013-01-02 16:40:05 <epscy> Jouke: on second thoughts, just send me the wallet.dat, i will "back it up" for you
559 2013-01-02 16:40:05 <sipa> Jouke: bitcoind stop
560 2013-01-02 16:40:13 <Luke-Jr> sipa: mingw doesn't have anything to make %ll* magically work, at least
561 2013-01-02 16:40:26 <sipa> Luke-Jr: yes it does
562 2013-01-02 16:40:48 <sipa> Luke-Jr: http://sourceforge.net/project/shownotes.php?release_id=24832
563 2013-01-02 16:41:00 <Jouke> sipa: but that gives me a "no response from server" error
564 2013-01-02 16:41:21 <gavinandresen> Jouke: works for me
565 2013-01-02 16:41:26 <sipa> Jouke: killall bitcoind
566 2013-01-02 16:41:33 <Luke-Jr> sipa: that was back in 2008, I think they got rid of it?
567 2013-01-02 16:41:42 <Jouke> but that will probably corrupt the database?
568 2013-01-02 16:41:49 <Jouke> Oh, wait, that's -9
569 2013-01-02 16:41:53 <sipa> Luke-Jr: well it works verifiably for gain
570 2013-01-02 16:41:55 <sipa> *gavin
571 2013-01-02 16:41:57 <Diapolo> sipa: when I added your repos as remote and did git fetch sipa -p will my local branch leveldb17 be up to date or do I have to rebase now?
572 2013-01-02 16:42:03 <Jouke> stupid me
573 2013-01-02 16:42:37 <sipa> Luke-Jr: specifying -posix results in binaries that work for him, not specifying it produces binaries that show the expected behaviour for an snprintf that doesn't support %ll
574 2013-01-02 16:42:45 <gavinandresen> Well, I can reproduce the %ll not working if I do NOT compile -posix, but does if I do problem straight-compiling with mingw32 inside a Win XP 32-bit VM
575 2013-01-02 16:42:49 <Luke-Jr> hrm
576 2013-01-02 16:43:31 <sipa> Diapolo: after that call, sipa/leveldb17 is the latest branch for you
577 2013-01-02 16:44:06 <sipa> to update leveldb17 itself, either rebase, or just reset (git checkout leveldb17 && git reset --hard sipa/leveldb17)
578 2013-01-02 16:44:07 <gmaxwell> meh. %ll isn't portable. The splicy macros are horrible and are commonly done. Has leveldb upstream indicated an unwillingness to take an I64FORMAT macro?
579 2013-01-02 16:44:26 <sipa> they have %ll all over the source code
580 2013-01-02 16:45:07 <Luke-Jr> sipa: http://www.mingw.org/wiki/FAQ says %ll needs to be replaced with %l64 at least
581 2013-01-02 16:45:12 <gavinandresen> %ll must be part of some standard or another....
582 2013-01-02 16:45:22 <Luke-Jr> gavinandresen: C99, which Microsoft never implemented completely
583 2013-01-02 16:45:24 <sipa> it's C99
584 2013-01-02 16:45:30 <gavinandresen> Luke-Jr: that FAQ is from 2007....
585 2013-01-02 16:46:04 <Luke-Jr> gavinandresen: but updated as recently as 2012-10-24
586 2013-01-02 16:46:14 <gavinandresen> Any mingw experts in the house?
587 2013-01-02 16:46:30 <gmaxwell> (even the requirement to have a 64 bit long long type is C99, but at least its something that even microsoft has)
588 2013-01-02 16:47:29 <gavinandresen> "You should use %I64 instead of %ll when using msvcrt" <--- and I thought compiling -posix told mingw NOT to use msvcrt
589 2013-01-02 16:47:43 <Diapolo> sipa: thanks I just deleted my local leveldb17 and did git checkout sipa/leveldb17 -b leveldb17 ^^
590 2013-01-02 16:47:52 <sipa> it always needs nsvcrt
591 2013-01-02 16:48:00 <Luke-Jr> maybe someone should ask in ##mingw
592 2013-01-02 16:48:07 <gavinandresen> ... not to use msvcrt for the *printf functions....
593 2013-01-02 16:48:12 <gmaxwell> depending on mingw's enhanced posix compatiblity isn't grand here, because it locks us into another narrow toolchainism. If someone ever makes a clang based cross compiler, or people want to compile with MSVC things will be problematic again.
594 2013-01-02 16:48:43 <Diapolo> gmaxwell: I'm sure this is the case anyway ^^
595 2013-01-02 16:48:43 <sipa> yeah, i wanted to avoid patching leveldb code all over the place, but perhaps it's best that way
596 2013-01-02 16:49:12 <Luke-Jr> sipa: possibility upstream LevelDB accepts 64-bit printf macro patches?
597 2013-01-02 16:49:31 <gmaxwell> Luke-Jr: ^ thats what I'm saying. They should.
598 2013-01-02 16:49:54 <gmaxwell> Esp if they want to have any hope of working with the microsoft branded toolchain.
599 2013-01-02 16:50:19 <sipa> well, more recent versions of msvcrt just support %ll
600 2013-01-02 16:50:26 <sipa> it's only a problem on old windowses
601 2013-01-02 16:50:49 <sipa> which is probably the reason why not too many people care about it anymore
602 2013-01-02 16:51:16 <sipa> we could just drop WinXP support in bitcoind :p
603 2013-01-02 16:52:02 <sipa> even the last service pack to that OS is almost 5 years old...
604 2013-01-02 16:52:12 <gavinandresen> I'd be OK with that being the solution.
605 2013-01-02 16:52:21 <gmaxwell> It's still quite widely deployed, though perhaps not among bitcoin users.
606 2013-01-02 16:52:25 <Luke-Jr> sipa: don't we need to link a newer msvcrt explicitly for that though?
607 2013-01-02 16:52:59 <sipa> Luke-Jr: my knowledge about windows DLL versioning is purposefully non-existant
608 2013-01-02 16:53:05 <gavinandresen> A check for minimum windows version in the installer should be done if we decide to drop XP support, though
609 2013-01-02 16:53:15 <gmaxwell> And I'm a little confused to hear it works, I know I made a %ll screwup in one of my packages a couple months ago and instantly got complants from MSVC users.
610 2013-01-02 16:53:24 <Diapolo> I'm NOT okay with dropping XP support, that would be too early IMHO.
611 2013-01-02 16:53:55 <gavinandresen> Diapolo: ok... then pick up the baton from sipa and figure out how to fix it!
612 2013-01-02 16:54:10 <sipa> Diapolo: thank you for demonstrating the largest problem Microsoft has :)
613 2013-01-02 16:54:25 <sipa> Diapolo: (not saying it's their fault or someone else's though)
614 2013-01-02 16:55:09 <Diapolo> to drop a widely used OS is not a solution it's only easy
615 2013-01-02 16:55:42 <Diapolo> If I knew how I could help here I would do that, but I din't even get what the problem is, just that %ll thing
616 2013-01-02 16:56:02 <Luke-Jr> Diapolo: to be fair, even Microsoft has dropped that widely used OS :P
617 2013-01-02 16:56:28 <sipa> Diapolo: MS Visual C++ has traditionally used %I64 in printf format strings for 64-bit integers, while the C99 standard says %ll
618 2013-01-02 16:56:45 <Diapolo> luke-jr: 08/04/2014 extendend support end date
619 2013-01-02 16:56:47 <Diapolo> for WinXP
620 2013-01-02 16:56:51 <sipa> Diapolo: recent versions of microsoft's runtime library support both, but older ones don't
621 2013-01-02 16:57:20 <sipa> Diapolo: we have some macros in util.h to make use we use %I64 on windows systems, and %ll elsewhere
622 2013-01-02 16:57:28 <sipa> Diapolo: but the leveldb source code uses %ll directly
623 2013-01-02 16:57:39 <Diapolo> right, I'm aware of that util.h macros
624 2013-01-02 16:57:42 <sipa> one solution is patching leveldb to also use such a macro
625 2013-01-02 16:58:07 <sipa> but we were trying to avoid that, by using a compiler toolchain which promises a fix/workaround for that problem
626 2013-01-02 16:58:18 <kjj> bitcoin can't possibly be the only leveldb user that has this problem
627 2013-01-02 16:58:19 <Diapolo> would it be as simple as search through the leveldb code and replace with our macro (and adding that to some LevelDB header?)
628 2013-01-02 16:58:29 <sipa> Diapolo: yes
629 2013-01-02 16:58:41 <sipa> Diapolo: and keep doing that every time we update leveldb
630 2013-01-02 16:58:48 <gmaxwell> kjj: considering we had to patch in windows support at all initially, I don't think it's unlikely.
631 2013-01-02 16:58:51 <Luke-Jr> sipa: unless it gets sent upstream???
632 2013-01-02 16:59:02 <sipa> Luke-Jr: yup, certainly will try that
633 2013-01-02 16:59:08 <Diapolo> sipa: could be automated and looks rather elegant? and as luke-jr says perhaps file a bug report?
634 2013-01-02 16:59:09 <kjj> oh. heh, that's different then.
635 2013-01-02 16:59:32 <gmaxwell> Hopefully someone working on leveldb had also dealt with this in other packages- and will just say "sure, of course, send a patch"
636 2013-01-02 16:59:46 <gavinandresen> finding all the %ll's won't be trivial
637 2013-01-02 16:59:51 <kjj> I guess if we are patching in windows support, then we can patch around this wart too
638 2013-01-02 16:59:55 <sipa> gavinandresen: it is
639 2013-01-02 17:00:18 <Diapolo> hasn't TD any knowledge of that bug / problem?
640 2013-01-02 17:00:26 <Luke-Jr> gavinandresen: my mingw throws warnings if there's a %ll
641 2013-01-02 17:00:29 <gavinandresen> sipa: ok. I just noticed a complicated %05ull or something....
642 2013-01-02 17:00:52 <sipa> gavinandresen: i grepped for % + ll
643 2013-01-02 17:00:57 <sipa> there are 24 cases
644 2013-01-02 17:01:35 <TD> Diapolo: we don't run anything on windows internally, hence the lack of support out of the box. sanjay said he'd accept windows patches if it was a native win32 layer, not based on boost or something like it
645 2013-01-02 17:02:08 <sipa> we should try to get that layer we have now get accepted upstream entirely then
646 2013-01-02 17:02:29 <Diapolo> sipa: I thought the same, as it doesn't depend on Boost anymore
647 2013-01-02 17:02:40 <sipa> (perhaps coordinating with the author, who wasn't interested in MinGW support)
648 2013-01-02 17:04:33 <Diapolo> TD: google must really hate MS ^^
649 2013-01-02 17:05:06 <gmaxwell> Diapolo: not bending over backwards for MS's weird toolchains is pretty prudent when you're not in the business of making software for those toolchains!
650 2013-01-02 17:05:41 <TD> sipa: though, it's curious that such problems exist
651 2013-01-02 17:05:46 <TD> because chrome uses leveldb and it runs on windows
652 2013-01-02 17:05:53 <sipa> uhu
653 2013-01-02 17:05:55 <TD> (with its own OS abstraction layer)
654 2013-01-02 17:06:06 <TD> i guess because of the compiler differences or something
655 2013-01-02 17:06:09 <sipa> but not its own printf - indeed, i wondered the same
656 2013-01-02 17:07:01 <TD> well
657 2013-01-02 17:07:05 <TD> i suppose it's possible it uses its own printf
658 2013-01-02 17:07:08 <TD> i have no idea really
659 2013-01-02 17:08:44 <MobTablet> question anyone use a wallet on there phone or tablet ?
660 2013-01-02 17:09:02 <sipa> yes
661 2013-01-02 17:09:15 <Diapolo> sipa: I still don't understand why the change to static-libgcc??-static-libstdc++ was needed ... because you added that after including vanila leveldb 1.7. Is it because of the new compiler suite?
662 2013-01-02 17:09:25 <sipa> Diapolo: yes
663 2013-01-02 17:09:48 <sipa> Diapolo: otherwise the produced binaries depend on some mingw .dll files
664 2013-01-02 17:09:52 <MobTablet> sipa you do
665 2013-01-02 17:09:58 <sipa> yes
666 2013-01-02 17:10:04 <Diapolo> I really wonder, why that didn't happen before...
667 2013-01-02 17:10:07 <MobTablet> on phone or tablet
668 2013-01-02 17:10:16 <sipa> both
669 2013-01-02 17:10:37 <MobTablet> would you mind helping me when i get back home ?
670 2013-01-02 17:29:03 <helo> sipa: bitcoin spinner?
671 2013-01-02 17:32:51 <sipa> helo: i use bitcoin wallet for android (at least it's a real node)
672 2013-01-02 17:34:29 <helo> i should probably revisit it, it's been quite a while since i first ruled it out
673 2013-01-02 17:35:05 <helo> was just so slow to sync
674 2013-01-02 17:35:19 <sipa> well it needs to download blocks
675 2013-01-02 17:35:32 <sipa> (at least until bloom filtering is implemented)
676 2013-01-02 17:36:15 <TD> helo: it's got faster and can sync overnight now. that's how i use it.
677 2013-01-02 17:36:25 <TD> it means in practice when you open it up, it only needs 10-20 seconds to fully catch up
678 2013-01-02 17:36:43 <helo> there's a setting to "sync up each night"?
679 2013-01-02 17:36:51 <helo> pretty good idea
680 2013-01-02 17:36:51 <TD> yes. it's on by default these days, i think
681 2013-01-02 17:36:56 <TD> it actually syncs when you charge
682 2013-01-02 17:37:03 <TD> but most people do that overnight of course
683 2013-01-02 17:42:18 <gavinandresen> sipa: I'm looking at my cross-compiling machine, and /usr/i686-w64-mingw32/include/stdio.h DEFINITELY redefines vsnprintf/etc #if __USE_MINGW_ANSI_STDIO
684 2013-01-02 17:42:47 <gavinandresen> sipa: ... so I'm wondering if maybe somehow in the gitian cross-compile environment another stdio.h is being included instead
685 2013-01-02 17:44:32 <sipa> gavinandresen: gtg now, i'll try finding out later
686 2013-01-02 17:44:53 <gavinandresen> sipa: ok. I'll see if I can cross-compile a binary that works
687 2013-01-02 17:45:47 <gmaxwell> Warning, incoming laughter... On BFL's upcoing miner devices protocol:
688 2013-01-02 17:45:54 <gmaxwell> "We're about to integrate a new command in the firmware. This job-taking command will accept a standard job structure (assuming full-range 4billion nonces) plus a range for the 12 bytes of merkel data. In other words, it will hover all 4 billion nonces, and once finished will increase merkel data (the 12 bytes) and re-calculate the 4billion.
689 2013-01-02 17:45:59 <gmaxwell> It can hold up to 200 nonces in memory until the host asks the device to return the nonces found, or the upper limit of the 12bytes range is reached."
690 2013-01-02 17:46:46 <sipa> i have no clue what they mean by merkel-data
691 2013-01-02 17:47:48 <MC1984> if i deleted an address from my address bok because i labeled it wrong can i get it back
692 2013-01-02 17:47:59 <MC1984> ive still got it in my transactions list under the wrong label
693 2013-01-02 17:48:16 <sipa> how did you delete it?
694 2013-01-02 17:48:40 <gavinandresen> gmaxwell: I hope they mean they've got a 12-byte extranonce in the coinbase and they'll recompute the merkle root in firmware....
695 2013-01-02 17:48:48 <MC1984> just hihglighted it and pressed the delete button in GUI
696 2013-01-02 17:49:05 <MC1984> do i need to receive more money on it to make it reappear in my address book
697 2013-01-02 17:49:19 <sipa> MC1984: you can't delete your own addresses
698 2013-01-02 17:49:26 <sipa> MC1984: only addresses of peers
699 2013-01-02 17:49:43 <MC1984> i just did
700 2013-01-02 17:49:54 <sipa> what tab are you in?
701 2013-01-02 17:49:55 <MC1984> i deleted a receiving address from my address vook
702 2013-01-02 17:50:17 <sipa> it should still show up under receive
703 2013-01-02 17:50:27 <gmaxwell> gavinandresen: there is a struct description in their protcol doc that is char[32] midstate, char[12] merkeldata.
704 2013-01-02 17:50:34 <MC1984> yeah its still there, but labelled wrong
705 2013-01-02 17:50:56 <MC1984> and when i try to change the label, it says its already in my sddress book but its not
706 2013-01-02 17:51:06 <sipa> bleh
707 2013-01-02 17:51:18 <MC1984> is this a bug report?
708 2013-01-02 17:51:23 <sipa> i suppose
709 2013-01-02 17:51:32 <MC1984> oh cool
710 2013-01-02 17:51:51 <gmaxwell> I ... can perhaps guess that it's space after the midstate? 0_o
711 2013-01-02 17:52:09 <MC1984> ive got both sending and receiving addresses in my address book actually, with no clear demarcation between them
712 2013-01-02 17:53:06 <sipa> gmaxwell: ask them?
713 2013-01-02 17:53:17 <MC1984> ok i restarted bitcoin and now it will let me edit the label from the received tab
714 2013-01-02 17:53:29 <MC1984> but its still deleted from the book tab
715 2013-01-02 17:53:51 <sipa> the addressbook is only for others's addresses i think
716 2013-01-02 17:54:27 <MC1984> yeah
717 2013-01-02 17:54:43 <MC1984> but im sure i just had my receive address in it for the 1000 coins someone gave me
718 2013-01-02 17:54:52 <MC1984> something strange going on there
719 2013-01-02 17:54:59 <sipa> my question is how your receive address ended up in the address book
720 2013-01-02 17:59:47 <MC1984> i dont know
721 2013-01-02 17:59:50 <MC1984> maybe im mistaken
722 2013-01-02 18:01:20 <MC1984> bitcoin android wont let you send without a fee
723 2013-01-02 18:01:32 <MC1984> god dammit now my balance is not integer
724 2013-01-02 18:04:30 <sipa> t3h h0rr0r
725 2013-01-02 18:05:01 <jgarzik> let the point float, I say!
726 2013-01-02 18:05:46 <MC1984> my ocd is hurting
727 2013-01-02 18:06:03 <MC1984> someone send me 0.0005 on testnet :<<
728 2013-01-02 18:07:20 <gmaxwell> MC1984: thats 50000 satoshi if that helps.
729 2013-01-02 18:08:21 <MC1984> one day that will be enough to buy a mustang
730 2013-01-02 18:08:42 <gmaxwell> well, not tnsatoshi .. I hope.
731 2013-01-02 18:09:02 <sipa> just set your client to show u(tn)BTC
732 2013-01-02 18:09:56 <MC1984> its like im really in zimbabwe
733 2013-01-02 18:31:11 <nanotube> jgarzik: haha
734 2013-01-02 18:31:26 <gavinandresen> sipa: success! cross-compiling -posix didn't work, but cross-compiling with -D__USE_MINGW_ANSI_STDIO=1 did
735 2013-01-02 19:10:47 <helo> wow, something good has happened since i last did a -loadblock... finished in 1h13m
736 2013-01-02 19:11:58 <helo> probably 60% of my previous best time, and about a third of my last try
737 2013-01-02 19:49:28 <etotheipi_> gmaxwell: true... it enables me to give the full public chain to a "semi-trusted agent" which will distribute addresses for me
738 2013-01-02 20:51:28 <gmaxwell> etotheipi_: yup.
739 2013-01-02 20:57:38 <sipa> gavinandresen: nice!
740 2013-01-02 20:58:26 <gavinandresen> sipa: see, easy when you know how...... (sigh)
741 2013-01-02 20:58:39 <sipa> yeah :p
742 2013-01-02 20:59:46 <gavinandresen> once upon a time I suggested to Satoshi that we drop the GUI and just support bitcoind. He told me I was insane (and was right). I'm still tempted, though....
743 2013-01-02 21:03:18 <TD> heh
744 2013-01-02 21:03:46 <TD> people tend to underestimate the importance of a nice GUI
745 2013-01-02 21:05:05 <gavinandresen> I agree completely that a nice GUI is extremely important. I just have doubts about the ability for open source projects to create and maintain nice GUIs
746 2013-01-02 21:05:10 <TD> a huge part of the brain is devoted to image processing. more than for any other sense, i was once told. so it's not surprising that people respond to graphics and make snap judgements based on how things look. apple understood this more than any other company
747 2013-01-02 21:05:26 <TD> UI design is a skill that can be learned, like any other.
748 2013-01-02 21:05:56 <gavinandresen> my suggestion to Satoshi was back in the wxwidgets days, when we had nobody to maintain it
749 2013-01-02 21:06:02 <TD> ah
750 2013-01-02 21:06:12 <TD> even so, it was still an insane suggestion :-0
751 2013-01-02 21:06:13 <TD> :)
752 2013-01-02 21:06:24 <TD> that smiley came out wrong
753 2013-01-02 21:06:44 <TD> we'll have to do some UI design soon for the payment protocol work, whether we like it or not
754 2013-01-02 21:07:00 <TD> otherwise we're going to see a repeat of the SSL UI fiasco. ui best practices can help avoid it.
755 2013-01-02 21:07:06 <gavinandresen> ... and then more for multi-device wallets.
756 2013-01-02 21:07:34 <TD> could be. i think having "best practices" is only worth doing for UI areas that are very security critical
757 2013-01-02 21:08:11 <gavinandresen> oh, you mean design as in standardized-across-implementations....
758 2013-01-02 21:08:20 <TD> well, at least have some recommendations, yes
759 2013-01-02 21:08:46 <TD> the payment protocol is a security protocol, to some extent. history has several examples of security protocols where the maths worked but they were undermined by poor UI
760 2013-01-02 21:09:10 <gavinandresen> yup.
761 2013-01-02 21:09:28 <TD> you can't force UI like you can with a fixed wire specification, but at least good advice can help
762 2013-01-02 21:09:30 <TD> at some point, if you want to pursue the multi-factor wallet you'll have to decide in which client(s) to implement it, by the way.
763 2013-01-02 21:09:47 <TD> that point is still some way off
764 2013-01-02 21:10:06 <gmaxwell> it's especially relevant here because you can't remove the user from the loop (thats the best way to avoid UI secuirty issues). The UI ultimately has to have the user approve it.
765 2013-01-02 21:10:30 <TD> indeed
766 2013-01-02 21:10:58 <gmaxwell> It can however, make strong recomendations for things like non-user overridable certificate validation??? to head off the creation of a ecosystem where that trap is a pratical necessity.
767 2013-01-02 21:11:15 <TD> one obvious recommendation that springs to mind is to render requests that have broken signing data the same as unverified payments rather than throwing errors
768 2013-01-02 21:11:19 <sipa> gmaxwell: i saw you post on bitcointalk about the ECDSA leak of the bitcoincard... were they informed in any other way about it?
769 2013-01-02 21:12:02 <tcatm> sipa: I was able to get a contact at 29c3 and have exchanged a few mails since then.
770 2013-01-02 21:12:11 <TD> tcatm: you're back!
771 2013-01-02 21:12:15 <gmaxwell> I was about to say I thought tcatm had.
772 2013-01-02 21:12:17 <TD> talking about nice uis ??? :)
773 2013-01-02 21:12:34 <gmaxwell> (though I'd actually thought he'd talked to them prior to 29c3 so I was only accidentally right)
774 2013-01-02 21:13:08 <sipa> tcatm: you did see i found they actually used k=0 ?
775 2013-01-02 21:13:09 <tcatm> Well, I tried their contact form without a response, though.
776 2013-01-02 21:13:21 <tcatm> sipa: yes
777 2013-01-02 21:15:24 <TD> gavinandresen: it sounded like the 2013 conf would have panel discussions but no presentations, beyond a keynote?
778 2013-01-02 21:15:30 <gmaxwell> in unrelated news, earlier this week I figured out how to rearrange the optimal coin selection problem for free transactions so that it's a fully linear (integer) problem. Should be possible to make a much better fee avoiding coin selection.
779 2013-01-02 21:16:32 <TD> gmaxwell: if you can document that in enough detail for people without much ILP experience, i could use it when i implement better fee solving for bitcoinj
780 2013-01-02 21:16:41 <gavinandresen> TD: I dunno, I haven't been paying much attention to conference organization
781 2013-01-02 21:16:45 <TD> ok
782 2013-01-02 21:16:54 <tcatm> sipa: Actually, the k I have discovered is the first output of a pseudo rng. The cards will generate the same ks every time they are turned on.
783 2013-01-02 21:17:24 <TD> that seems like a fairly basic bug. how can you know enough to design a hardware bitcoin wallet but not enough to add a hardware RNG?
784 2013-01-02 21:17:57 <tcatm> It seems like there is a seed where the user is asked to press a few buttons and then hit "OK". Those ks seem to appear when the user hits OK without pressing any buttons.
785 2013-01-02 21:17:59 <sipa> tcatm: yes, i found a few other repeated k's, but only one that was reused many many times (0)
786 2013-01-02 21:18:17 <TD> the point of custom hardware is you can do better than that
787 2013-01-02 21:18:21 <sipa> they can be caused by different offenders
788 2013-01-02 21:18:23 <gmaxwell> TD: I can make it pretty easy for someone who has access to an ILP solver as a black box. I don't know how compactly I can explain using branch&cut and the simplex method. The ILP solver part of it is a fair amount of code. So I'm still contemplating how to get this into production code.
789 2013-01-02 21:18:38 <TD> gmaxwell: i'm sure there are ILP libraries for java that can be re-used
790 2013-01-02 21:18:51 <TD> so i think assuming the existence of an ILP solver is ok. at least to start with.
791 2013-01-02 21:19:22 <sipa> gavinandresen: new "turbo" build: http://bitcoin.sipa.be/builds/pre-0.8/2013-01-02-turbo-ldb17/
792 2013-01-02 21:19:27 <TD> hah
793 2013-01-02 21:19:37 <TD> not only are there implementations but there's even a project to provide a unified API to them
794 2013-01-02 21:19:38 <TD> http://javailp.sourceforge.net/
795 2013-01-02 21:19:39 <sipa> gavinandresen: this one should work on XP...
796 2013-01-02 21:19:46 <gmaxwell> Indeed. Likewise in C. GLPK is the obvious C one, but it's GPL. I was contemplating the idea of forking() in the reference client for the coin solver, would make it pretty easy to make it pluggable.
797 2013-01-02 21:20:11 <gmaxwell> TD: ha, great. Okay, we'll I'll be writing it up in any case.
798 2013-01-02 21:20:15 <TD> thanks
799 2013-01-02 21:20:28 <tcatm> sipa: Weren't there any other ks reused many (>10) times?
800 2013-01-02 21:21:11 <TD> unfortunately most of the solvers and the abstraction library are LGPL licensed
801 2013-01-02 21:21:18 <TD> well, no matter
802 2013-01-02 21:22:41 <gmaxwell> sipa: oh did you get around to actually recovering the K for some of them?
803 2013-01-02 21:23:17 <sipa> yes
804 2013-01-02 21:23:18 <tcatm> btw. did you hear about http://btchip.com/ ?
805 2013-01-02 21:25:13 <helo> oh neat, no more ./bitcoin-qt "${blks[@]/#/-loadblock=}"
806 2013-01-02 21:26:25 <sipa> gmaxwell: i wonder where the code is...
807 2013-01-02 21:26:38 <sipa> is there a way to sort you git branches by date last modified?
808 2013-01-02 21:27:15 <sipa> ok, found
809 2013-01-02 21:28:28 <sipa> gmaxwell: my do a reindex using my 'kreuse' branch (it doesn't do actual sigchecking)
810 2013-01-02 21:28:40 <sipa> it will dump reused R values with corresponding K in debug.log
811 2013-01-02 21:39:15 <tcatm> kreuse could be improved to recover the secret and send all unspent outputs to itself after rescan :)
812 2013-01-02 21:40:13 <sipa> i purposefully decided not to recover private keys
813 2013-01-02 21:40:20 <sipa> read: i was lazy
814 2013-01-02 21:41:04 <sipa> it would be a very good test for the code though, as it can check whether the reused Rs were actually caused by same key + same K
815 2013-01-02 21:47:50 <gmaxwell> thinking about this makes me wonder about the socially productive results of thieves existing. If there were more technically sophicated bitcoin theives then maybe R reuse would have been caught earlier. Fortunately it was found early enough???
816 2013-01-02 22:04:02 <tradefortress> I can call get gettransaction API of TXs I sent
817 2013-01-02 22:04:17 <tradefortress> But if I try to look up a transaction I didn't send, I get a "Didn't receive 200 OK" error.
818 2013-01-02 22:04:33 <sipa> use getrawtransaction for that