1 2014-06-03 00:01:44 <phantomcircuit> er
2 2014-06-03 00:02:04 <phantomcircuit> coryfields, was autotools changed to only look for bdb4.8?
3 2014-06-03 00:02:19 <phantomcircuit> --with-incompatible-bdb doesn't seem to work anymore
4 2014-06-03 00:12:50 <maaku> phantomcircuit: "works for me"
5 2014-06-03 00:13:17 <phantomcircuit> maaku, sigh
6 2014-06-03 00:13:27 <phantomcircuit> i wonder if the search path is screwed up
7 2014-06-03 00:13:47 <maaku> ACTION is doing a autogen to replace the configure script and try again
8 2014-06-03 00:14:29 <maaku> yeah xubuntu 14.04, --with-incompatible-bdb works
9 2014-06-03 00:14:46 <phantomcircuit> sigh
10 2014-06-03 00:14:48 <phantomcircuit> ii libdb5.3++:amd64 5.3.28-3ubuntu3 amd64 Berkeley v5.3 Database Libraries for C++ [runtime]
11 2014-06-03 00:14:53 <phantomcircuit> $ locate db_cxx.h
12 2014-06-03 00:14:55 <phantomcircuit> $
13 2014-06-03 00:14:56 <phantomcircuit> >.>
14 2014-06-03 00:15:32 <maaku> :(
15 2014-06-03 00:17:18 <phantomcircuit> maaku, reinstall and the file magically appears
16 2014-06-03 00:25:35 <maaku> phantomcircuit: of course :)
17 2014-06-03 00:26:00 <maaku> actually i've been exactly there -- interupted upgrade resulted in neither package actually installed
18 2014-06-03 01:03:04 <phantomcircuit> lol
19 2014-06-03 01:03:08 <phantomcircuit> doing a reindex
20 2014-06-03 01:03:10 <phantomcircuit> 2.38% 884765008 bitcoind bitcoind [.] CBlockIndex::IsSuperMajority(int, CBlockIndex const*, unsigned int, unsigned int)
21 2014-06-03 01:03:10 <phantomcircuit> 2.68% 999035319 bitcoind libcrypto.so.1.0.0 [.] 0x00000000000e57d0
22 2014-06-03 01:03:33 <phantomcircuit> that silly function takes only slightly less cpu time than the ECDSA verification stuff
23 2014-06-03 01:05:21 <gmaxwell> seems unlikely, is that also counting its children?
24 2014-06-03 01:06:24 <phantomcircuit> gmaxwell, i've done this a number of times different ways, including putting calls to gettimeoday in there
25 2014-06-03 01:06:35 <phantomcircuit> it's consistently come back as using a comical ridiculous amount of cpu time
26 2014-06-03 01:06:57 <phantomcircuit> i suspect it's something about thrashing the shit out of the cpu cache
27 2014-06-03 01:07:14 <phantomcircuit> obviously that only holds true for the first ~100k blocks
28 2014-06-03 01:07:21 <phantomcircuit> after that the ecdsa stuff dominates by a wide margin
29 2014-06-03 01:07:32 <gmaxwell> well there is no ecdsa in the first 100k blocks.
30 2014-06-03 01:07:37 <gmaxwell> it's distabled.
31 2014-06-03 01:07:48 <phantomcircuit> hmm you're right
32 2014-06-03 01:07:56 <phantomcircuit> i wonder if that's just sha256 then
33 2014-06-03 01:08:10 <phantomcircuit> lets try again wtih a debug build of openssl
34 2014-06-03 01:10:13 <phantomcircuit> gmaxwell, it seems like once IsSuperMajority returns true you can safely cache the result, as downgrade isn't possible
35 2014-06-03 01:11:10 <gmaxwell> sure now that we've passed the point of no return, but it would be good to understand why it's slow
36 2014-06-03 01:12:27 <phantomcircuit> gmaxwell, it's walking a linked list backwards 1000 block headers, then going back to the start and doing it again 100 times
37 2014-06-03 01:12:34 <phantomcircuit> for each block
38 2014-06-03 01:13:08 <phantomcircuit> also the error message is wrong
39 2014-06-03 01:13:13 <gmaxwell> why does it do it 100 times?
40 2014-06-03 01:13:31 <gmaxwell> is it also hashing the header each step? if so that would suddenly explain the profile.
41 2014-06-03 01:14:21 <phantomcircuit> gmaxwell, iirc the actual rule is 950 of the last 1000 blocks OR 75 of the last 100 blocks on testnet
42 2014-06-03 01:14:35 <phantomcircuit> actually i guess this shouldn't do the 100
43 2014-06-03 01:14:38 <phantomcircuit> since TestNet() will be false
44 2014-06-03 01:16:26 <phantomcircuit> gmaxwell, anyways to get to block 100k requires you to do ~100m operations
45 2014-06-03 01:17:12 <phantomcircuit> gmaxwell, actually the uint256 comparison function is also super expensive apparently
46 2014-06-03 01:17:19 <phantomcircuit> (or maybe is just called A LOT)
47 2014-06-03 01:17:56 <gmaxwell> thats also suspicious, I suggest stepping through it in a debugger. :P
48 2014-06-03 01:18:37 <phantomcircuit> gmaxwell, im using perf
49 2014-06-03 01:18:47 <phantomcircuit> i guess i can step through it in gdb also
50 2014-06-03 01:18:57 <phantomcircuit> but i've seen these results enough times that i think they're accurate
51 2014-06-03 01:19:01 <phantomcircuit> if not insane
52 2014-06-03 01:19:37 <phantomcircuit> yeah that openssl function was 3.45% 394140350 bitcoind libcrypto.so.1.0.0 [.] sha256_block_data_order
53 2014-06-03 01:20:27 <gmaxwell> phantomcircuit: the reason I suggest stepping is not because I think the profile is wrong, but because if its right then something moderately surprising is happening that isn't visible in the profile.
54 2014-06-03 01:21:57 <phantomcircuit> 3.03% 8807578250 bitcoind bitcoind [.] base_uint<256u>::CompareTo(base_uint<256u> const&) const
55 2014-06-03 01:21:57 <phantomcircuit> 5.32% 15456911093 bitcoind libcrypto.so.1.0.0 [.] 0x00000000000e80ea
56 2014-06-03 01:22:26 <phantomcircuit> apparently i dont get libcrypto symbols this time?
57 2014-06-03 01:23:23 <phantomcircuit> gmaxwell, the results here are uh
58 2014-06-03 01:23:24 <phantomcircuit> bizarre
59 2014-06-03 01:30:23 <coryfields> phantomcircuit: not afaik
60 2014-06-03 01:30:27 <coryfields> phantomcircuit: get it worked out?
61 2014-06-03 01:33:23 <phantomcircuit> gmaxwell, pastebin.com/raw.php?i=s50M0jyf
62 2014-06-03 01:33:29 <phantomcircuit> http://pastebin.com/raw.php?i=s50M0jyf
63 2014-06-03 01:33:59 <phantomcircuit> that's a reindex upto 140557
64 2014-06-03 01:34:03 <phantomcircuit> weird right
65 2014-06-03 01:34:11 <copumpkin> you clearly have too much RAM
66 2014-06-03 01:34:25 <phantomcircuit> copumpkin, this is a laptop too lol
67 2014-06-03 01:34:36 <copumpkin> is it a "laptop" or a laptop?
68 2014-06-03 01:34:51 <phantomcircuit> it's a 15" ~7lb monster
69 2014-06-03 01:35:23 <phantomcircuit> putting it on your lap is a pretty sure fire way to temporaril'y sterilize yourself
70 2014-06-03 01:52:39 <phantomcircuit> <phantomcircuit> gmaxwell, once again but this time with symbols for libstd++
71 2014-06-03 01:52:39 <phantomcircuit> <phantomcircuit> http://pastebin.com/raw.php?i=LWgxUgu
72 2014-06-03 01:55:33 <phantomcircuit> so it seems like Hash, uint256::CompareTo, and malloc/free, are the most expensive things before we get to checkpoints
73 2014-06-03 01:55:55 <gmaxwell> yea, I knew about the malloc free, the whole coding style used in the codebase heavily abuses the allocator.
74 2014-06-03 01:56:18 <phantomcircuit> 1.10% 9095160715 bitcoind bitcoind [.] std::vector<unsigned char, std::allocator<unsigned char> >::end() const
75 2014-06-03 01:56:19 <phantomcircuit> wat
76 2014-06-03 01:56:29 <gmaxwell> lots and lots of short lived heap allocated things.
77 2014-06-03 01:56:33 <phantomcircuit> returning the end() iterator is that expensive for a vector?
78 2014-06-03 01:56:47 <phantomcircuit> gmaxwell, yeah i noticed that
79 2014-06-03 01:57:45 <phantomcircuit> afaict std::vector::end() should be O(1)
80 2014-06-03 01:57:53 <phantomcircuit> but maybe that 1 is very very expensive
81 2014-06-03 01:58:19 <phantomcircuit> GetOp2
82 2014-06-03 01:58:20 <phantomcircuit> wat
83 2014-06-03 01:59:18 <gmaxwell> I assume you've compiled without optimization?
84 2014-06-03 01:59:39 <gmaxwell> some of this I think you're seing is pure calling overhead for little snippits of code that are otherwise always inlined.
85 2014-06-03 01:59:43 <phantomcircuit> gmaxwell, just ./configure --with-incompatible-bdb --enable-debug --without-miniupnpc
86 2014-06-03 01:59:46 <phantomcircuit> whatever that gets you
87 2014-06-03 01:59:47 <gmaxwell> hm.
88 2014-06-03 02:00:06 <phantomcircuit> that should get me -O2 i think
89 2014-06-03 02:00:59 <phantomcircuit> yeah
90 2014-06-03 02:01:04 <phantomcircuit> ./configure: CXXFLAGS="-g -O2"
91 2014-06-03 02:01:26 <phantomcircuit> oh wait
92 2014-06-03 02:01:27 <phantomcircuit> no
93 2014-06-03 02:01:32 <phantomcircuit> CXXFLAGS = -g3 -O0 -DDEBUG -Wall -Wextra -Wformat -Wformat-security -Wno-unused-parameter -Wstack-protector -fstack-protector-all -fPIE
94 2014-06-03 02:01:40 <phantomcircuit> i guess --enable-debug sets -O0
95 2014-06-03 02:02:10 <phantomcircuit> yeah it does
96 2014-06-03 02:02:18 <phantomcircuit> CXXFLAGS = -g -O2 -Wall -Wextra -Wformat -Wformat-security -Wno-unused-parameter -Wstack-protector -fstack-protector-all -fPIE
97 2014-06-03 02:02:21 <phantomcircuit> well that's annoying
98 2014-06-03 02:08:11 <gmaxwell> oh duh, I missed your enable debug due to a line wrap. Yea, you'll want to profile with O2.
99 2014-06-03 03:12:31 <andytoshi> in PR 4277 i suggested replacing the "pKey != NULL" assertation for checking if the ssl lib supports EC with a nicer error message
100 2014-06-03 03:12:43 <andytoshi> Luke-Jr suggested i use LogPrintf instead of fprintf, what are thoughts on that?
101 2014-06-03 03:12:54 <andytoshi> the current assertation appears on the terminal as a visible failure
102 2014-06-03 03:13:23 <andytoshi> i'd assume the first run of bitcoind would always be on an interactive terminal where that's a better output than debug.log (which a new user might not even know of)
103 2014-06-03 03:13:27 <Luke-Jr> we should use whatever current startup errors use.. eg, dialog for Qt..
104 2014-06-03 03:16:17 <andytoshi> oh, is that the InitWarning() function in init.cpp?
105 2014-06-03 03:16:47 <andytoshi> i mean, uiInterface.ThreadSafeMessageBox
106 2014-06-03 03:18:31 <andytoshi> Luke-Jr: you also suggested i give advice for how to fix this, i'm not sure what to say there. i manage it by compiling my own openssl and overwriting the package manager's files, which is an ugly thing to do
107 2014-06-03 03:28:08 <Luke-Jr> andytoshi: oh, this is your PR? :D
108 2014-06-03 03:28:30 <Luke-Jr> andytoshi: perhaps a wiki page people can list options? :\
109 2014-06-03 03:29:06 <andytoshi> Luke-Jr: oh, yeah, i'm apoelstra everywhere except IRC :}
110 2014-06-03 03:29:31 <andytoshi> except bitcoin-related stuff, i mean
111 2014-06-03 03:29:49 <andytoshi> that's a good idea, i'll look for one or create one
112 2014-06-03 03:30:18 <PRab> Anyone willing to send some testnet coins to 2NG7aR2ovqtsi7Xn6VLxGKUSpkVAjGooub8?
113 2014-06-03 03:31:18 <PRab> And anyone know how big the testnet blockchain is?
114 2014-06-03 03:44:11 <jcrubino> pr 2 gigs using btcd
115 2014-06-03 03:44:19 <jcrubino> PRab,
116 2014-06-03 03:44:31 <jcrubino> maybe 3
117 2014-06-03 03:45:21 <pigeons> no
118 2014-06-03 03:45:28 <pigeons> less
119 2014-06-03 03:51:05 <Luke-Jr> pigeons: you use btcd?
120 2014-06-03 03:54:22 <andytoshi> Luke-Jr, gmaxwell, everyone: can you check https://en.bitcoin.it/wiki/OpenSSL_and_EC_Libraries for correctness?
121 2014-06-03 03:55:54 <Luke-Jr> andytoshi: I'd change 2 things: 1) clearly point out that the patents do NOT affect the EC used by Bitcoin, 2) don't claim Heartbleed's effect on Bitcoin is non-serious (since it can be)
122 2014-06-03 03:56:32 <andytoshi> okey doke
123 2014-06-03 04:00:28 <andytoshi> ok Luke-Jr, i have fixed both
124 2014-06-03 04:04:01 <pigeons> Luke-Jr: oops, no, i dont use btcd sorry my mistake
125 2014-06-03 04:32:28 <justanotheruser> I'm trying to connect to the json rpc and I'm getting an error:
126 2014-06-03 04:32:28 <justanotheruser> IOError: ('http protocol error', 0, 'got a bad status line', None)
127 2014-06-03 04:34:59 <justanotheruser> http://hastebin.com/iwukonigak.hs
128 2014-06-03 04:35:07 <justanotheruser> that is the basic code
129 2014-06-03 07:04:35 <gdm85> wumpus: since I managed to make the docker approach work for gitian builds, do you think we could mention it in the doc/gitian-build.md?
130 2014-06-03 07:05:11 <gdm85> surely can help at having more gitian builds, since it's almost fully automated
131 2014-06-03 07:05:27 <wumpus> gdm85: my goal with gitian-build.md is to have one approach that works for everyone; I'm fine with mentioning it, but don't add multiple equivalent options that are just technically different
132 2014-06-03 07:05:51 <wumpus> (ie, for example, I left the KVM versus LXC choice out of there)
133 2014-06-03 07:09:49 <gdm85> wumpus: I understand. I will temptatively add a hyperlink reference in the PR I am making
134 2014-06-03 07:12:27 <gdm85> wumpus: what do you think about (in future, not now) having the dependencies in a separate list file?
135 2014-06-03 07:13:14 <gdm85> one could streamline wget'ing them, and the docs would need less changes across versions in case of dep changes
136 2014-06-03 07:13:50 <wumpus> I understand the rationale, but I like having everything in one file
137 2014-06-03 07:14:02 <wumpus> a script that fetches al lthe dependencies would be useful of course
138 2014-06-03 07:14:17 <wumpus> (something like: fetch new dependencies if necessary)
139 2014-06-03 07:15:16 <phantomcircuit> gmaxwell, derp http://pastebin.com/raw.php?i=KmwpfVXM
140 2014-06-03 07:15:19 <phantomcircuit> that's more like it
141 2014-06-03 07:15:28 <phantomcircuit> 20% doing sha256 ops
142 2014-06-03 07:15:41 <phantomcircuit> really highlights how retarded it is to make the pow mode expensive
143 2014-06-03 07:18:15 <gdm85> wumpus: yeah indeed. not sure if it's more in the domain of our docs or gitian-builder features
144 2014-06-03 07:19:04 <wumpus> 'build new intermediate dependencies if needed' would be useful too
145 2014-06-03 07:19:08 <wumpus> would be sort of a makefile at the gitian level
146 2014-06-03 07:19:16 <gdm85> yeah
147 2014-06-03 07:19:28 <wumpus> but yes, it's probably outside the scope of the docs
148 2014-06-03 07:20:00 <gdm85> now, on that docs page I would like to add (right before the make-base-vm commands) that user should cleanup the environment from undesired LC_* variables. what do you think about this? it is for sure the fix that would have saved me heaps of time in finding the hashes problem :)
149 2014-06-03 07:20:23 <gdm85> (still, can be seen as a gitian-builder issue..)
150 2014-06-03 07:20:40 <gdm85> but would be helpful to mention that it can give problems
151 2014-06-03 07:21:11 <wumpus> if LC_* variables are a problem, you should update gitian-builder to nuke them
152 2014-06-03 07:21:38 <gdm85> I asked devrandom about that
153 2014-06-03 07:21:38 <Luke-Jr> probably should anyway
154 2014-06-03 07:21:40 <wumpus> let's not make the user do manual steps like that, it's easy to forget them
155 2014-06-03 07:22:06 <wumpus> Luke-Jr: ideally nothing should 'leak into' LXC
156 2014-06-03 07:22:14 <gdm85> wumpus: exactly
157 2014-06-03 07:23:15 <gdm85> but it could be that those LC_* variables are polluting the make-base-vm's generated images only, not the commands executed afterwards. I didn't verify this..but likely they are polluting both
158 2014-06-03 07:23:39 <wumpus> they're even polluting the *making of* the VMs? ouch
159 2014-06-03 07:23:45 <gdm85> only thing I know is that unsetting them and performing all steps from scratch gave me same hashes as everybody else
160 2014-06-03 07:23:52 <gdm85> wumpus: that's my speculation
161 2014-06-03 07:24:03 <Luke-Jr> I'd speculate it's the SSH
162 2014-06-03 07:24:10 <gdm85> Luke-Jr: yep
163 2014-06-03 07:24:16 <wumpus> the environment should definitely be sanitized then
164 2014-06-03 07:24:38 <wumpus> both before creating the vm images and before calling into them
165 2014-06-03 07:25:01 <wumpus> who knows what else other obscure environment settings could mess with
166 2014-06-03 07:25:02 <gdm85> yes. for example in the guide it is mentioned that you can pick any locale, now I am not so sure..
167 2014-06-03 07:25:13 <wumpus> you should be able to pick any locale you want
168 2014-06-03 07:25:19 <poutine> How do you know when someone sends money to you via a multisig address?
169 2014-06-03 07:25:26 <wumpus> the outer OS shouldn't matter that much, it's not part of the determinish
170 2014-06-03 07:25:31 <wumpus> determinism*
171 2014-06-03 07:25:33 <gdm85> yeah, I agree
172 2014-06-03 07:25:51 <Luke-Jr> ssh -o SendEnv=
173 2014-06-03 07:25:54 <Luke-Jr> may do the trick
174 2014-06-03 07:26:43 <gdm85> Luke-Jr, wumpus perhaps we could mention that modified ssh syntax in the docs for the time being until gitian-builder has a proper (acknowledged) fix?
175 2014-06-03 07:27:05 <wumpus> LXC doesn't use ssh AFAIK
176 2014-06-03 07:27:16 <wumpus> but for KVM it makes senes
177 2014-06-03 07:27:31 <wumpus> I don't see what mentioning it in the docs would help, the ssh calls are done internally in gitian, the user has no control over them
178 2014-06-03 07:27:46 <wumpus> the only way to fix this is to fix the gitian builder
179 2014-06-03 07:28:00 <gdm85> wumpus: it helps: if your first SSH is not polluted, the subsequent won't
180 2014-06-03 07:28:33 <gdm85> hold on, will pastebin a docs diff
181 2014-06-03 07:28:40 <wumpus> really, just fix gitian builder, it consists of some simple ruby scripts
182 2014-06-03 07:28:48 <Luke-Jr> env -i <anything> then :p
183 2014-06-03 07:29:50 <gdm85> ACTION <-- not ruby fluent :(
184 2014-06-03 07:30:11 <wumpus> it's almost the same as python
185 2014-06-03 07:30:13 <gdm85> I have been through many languages & dialects, but not much ruby so far..
186 2014-06-03 07:30:35 <Luke-Jr> the relevant code is probably bash
187 2014-06-03 07:30:40 <Luke-Jr> bin/on-target
188 2014-06-03 07:34:07 <gdm85> Luke-Jr: coryfields proposed to add LANG and LC_ALL, but not sure that would cover the (possible, not confirmed) pollution of the base VM creation
189 2014-06-03 07:35:06 <wumpus> it wouldn't; you'd have to do the same in make-base-vm
190 2014-06-03 07:37:43 <gdm85> ACTION nods
191 2014-06-03 07:41:27 <wallet42> anyone doing a swift implementation of bitcoin ? ;)
192 2014-06-03 07:51:10 <wumpus> another programming language, woohoo, let's scatter and fragment the development community even further
193 2014-06-03 07:56:51 <SomeoneWeird> wumpus, hehe, just what i was thinking
194 2014-06-03 07:57:46 <lianj> much room for packagemanager innovations again :D
195 2014-06-03 07:58:18 <wumpus> lianj: yess, time to paint all the old bikesheds again! :)
196 2014-06-03 08:37:24 <kiddouk> yes
197 2014-06-03 09:12:15 <gdm85> wumpus: do you still think that we should use sort -s -n instead of simple sort? If yes I will put a commit for the gitian descriptors in the PR I am going to submit now
198 2014-06-03 09:12:37 <gdm85> because one could say: if the environment is sane you do not need to specify -s -n
199 2014-06-03 09:13:41 <kiddouk> got a question regarding OP_CHECKMULTISIG
200 2014-06-03 09:14:12 <kiddouk> The documentation says : All signatures need to match a public key.
201 2014-06-03 09:14:46 <kiddouk> Meaning that I could repeat the same public key , let's say 2 times. And insert twice the same signature.
202 2014-06-03 09:15:04 <kiddouk> Would that work ? (I guess yes, but would like to be certain).
203 2014-06-03 09:15:57 <kiddouk> And aside, can I have 1 public key and twice the same signature, or MUST they go by pair only ?
204 2014-06-03 09:16:55 <kiddouk> meaning that we have move signature that public key
205 2014-06-03 09:17:06 <kiddouk> (The documentation only explains for the inverse situation â more public key than sigs.)
206 2014-06-03 09:17:50 <kiddouk> we have moRe* signatures thaN* publick key
207 2014-06-03 09:17:54 <kiddouk> (typo)
208 2014-06-03 09:58:30 <gdm85> for all those interested in building deterministically using gitian + docker + LXC: https://github.com/gdm85/tenku/blob/master/docker/gitian-bitcoin-host/README.md
209 2014-06-03 09:58:52 <gdm85> it is almost fully automated, so less pain expected in future :)
210 2014-06-03 10:00:36 <wumpus> gdm85: eh, it makes no sense to add the -o SendEnv= to that ssh command in the guide
211 2014-06-03 10:00:50 <wumpus> it's for connecting to the *outer* VM
212 2014-06-03 10:01:10 <wumpus> you want the environment to be filtered when going to the inner VM (so, LXC in this case)
213 2014-06-03 10:03:11 <gdm85> wumpus: I replied also on github. it is effective because if your initial SSH shell is not poisoned, those coming afterwards can't be either
214 2014-06-03 10:04:33 <wumpus> that's extrememly brittle; the outer VM may also not have the 'right' locale
215 2014-06-03 10:05:11 <kiddouk> wumpus: any chance that you have an answer to my question (read above) ?
216 2014-06-03 10:05:54 <wumpus> or what if the outer VM isn't a VM but a real machine
217 2014-06-03 10:05:58 <wumpus> no ssh needed
218 2014-06-03 10:07:03 <wumpus> kiddouk: yes, you could do a N-out-of-N with N times the same key
219 2014-06-03 10:07:08 <gdm85> wumpus: sure, then one should consider that step also as being part of the VirtualBox-specific section. I will remove that
220 2014-06-03 10:07:34 <gdm85> let me know about other amends, then I can update the commit
221 2014-06-03 10:07:42 <kiddouk> wumpus: and what about N signatures and only 1 time the same key ? Or do they go by pairs only ?
222 2014-06-03 10:07:43 <wumpus> gdm85: as I've said before, I'd like this to be solved either in the gitian builder or in the descriptors
223 2014-06-03 10:07:51 <wumpus> kiddouk: I don't know
224 2014-06-03 10:07:53 <kiddouk> Damn, I have to read that bitcoin code rather sooner than later.
225 2014-06-03 10:07:57 <kiddouk> Ok. Will read then.
226 2014-06-03 10:08:04 <kiddouk> thanks wumpus for the input.
227 2014-06-03 10:08:25 <wumpus> gdm85: I had a patch to the descriptors to sanitize the environment once, but people responded 'this is not needed, gitian does it' .. but apparently not
228 2014-06-03 10:09:09 <gdm85> wumpus: I can add a commit for the sort -s -n fix, but I didn' verify that such change alone will make hashes match. I verified only that cleaning up the shell environment before using gitian-builder gives out correct hashes at the end
229 2014-06-03 10:09:13 <gdm85> ah, that sucks.. :(
230 2014-06-03 10:09:49 <gdm85> probably using sort -s -n will make all hashes change for good (although untie them from locales/env)
231 2014-06-03 10:10:10 <wumpus> and I checked the gitian builder code then, and indeed it set some environment variables, so I have no clue what is happening here
232 2014-06-03 10:10:37 <wumpus> gdm85: I agree with you with regard to sort -s -n ... instead of forcing every single command into no-locale mode, it makes more sense to force a locale for the whole script
233 2014-06-03 10:11:00 <wumpus> otherwise it's too easy to forget one
234 2014-06-03 10:11:31 <gdm85> wumpus: so far my "feeling" is that setting LANG/LC_ALL is not enough, one has to unset the individual vars. but this is counterintuitive for how I know LANG/LC_ALL should work..so needs more testing
235 2014-06-03 10:11:34 <wumpus> I mean, sort is not the only unix command affected by the locale
236 2014-06-03 10:11:41 <gdm85> yep
237 2014-06-03 10:13:11 <wumpus> gdm85: maybe try to patch libexec/on-target first to undefine the environment variables before launching the command on the guest, and if that isn't enough
238 2014-06-03 10:13:21 <wumpus> gdm85: then patch the vmbuilder too
239 2014-06-03 10:13:33 <gdm85> uhm, the vmbuilder..I had forgot about that guy..
240 2014-06-03 10:13:59 <wumpus> but I *think* only on-target should be enough, if you force a locale there, it should take precedence over the system locale defined in vmbuilder (and people won't have to rebuild their images...)
241 2014-06-03 10:14:35 <gdm85> wumpus: ok, will try this. luckily with the stuff I prepared it is easy to iterate
242 2014-06-03 10:14:56 <gdm85> but later, not now
243 2014-06-03 10:16:13 <wumpus> gdm85: thanks a lot for trying to improve the gitian process anyhow :)
244 2014-06-03 10:17:09 <gdm85> wumpus: oh, you're welcome! I thought there would be many more issues to get this working through docker containers, but it was easy in the end. just a locale issue and coryfields pointed that to me (I would *never* have thought of it..)
245 2014-06-03 10:17:47 <gdm85> did you see the diagram? https://github.com/gdm85/tenku/tree/master/docker/gitian-host
246 2014-06-03 10:18:50 <wumpus> gdm85: yes, looks good to me
247 2014-06-03 10:19:58 <gdm85> perhaps I stretched the octagon too much.. :) now I need to go, will try to better narrow down the locale issue later
248 2014-06-03 10:31:17 <wumpus> gdm85: I'm interested though: what locale causes problems?
249 2014-06-03 10:32:21 <gdm85> wumpus: dutch :D but set for almost all e.g. monetary address etc
250 2014-06-03 10:40:24 <wumpus> ok yes I'm dutch but am not using the dutch locale, so that's probably why I haven't noticed
251 2014-06-03 10:41:18 <sipa> kiddouk: every signature must match exactly one public key; there can be excess public keys, but not excess signatures
252 2014-06-03 10:43:08 <kiddouk> sipa: thanks for clarifying this for me.
253 2014-06-03 11:04:42 <gdm85> wumpus: I have removed the -o SendEnv= suggestion from the PR
254 2014-06-03 11:24:51 <gdm85> I have also changed my build script so that it will be straighforward to add support for 0.9.2 rcs
255 2014-06-03 12:03:43 <uiop> int firefox_gc(void) __attribute((noreturn)) {abort(); return SUCCESS;}
256 2014-06-03 12:14:48 <gdm85> uiop: where is that?
257 2014-06-03 12:17:09 <uiop> played out on my laptop daily
258 2014-06-03 12:17:12 <uiop> ;)
259 2014-06-03 12:20:28 <uiop> void abort(void){while(1) printf("\a");} /*it could be worse*/
260 2014-06-03 12:30:39 <wumpus> you should add something like \033[41m\033[2J\033[47m\033[2J in there as well to make the terminal go crazy between the beeping
261 2014-06-03 14:21:34 <crunk-juice> Were there ever any SPV wallets for IOS?
262 2014-06-03 14:29:50 <t7> if you root it
263 2014-06-03 14:30:14 <t7> or get a device that lets you install the software YOU choose
264 2014-06-03 14:31:25 <dsnrk> crunk-juice: not that I am aware of. there's a number of them but they all use central models. there's a SPV library in objective C though, so I don't think there's much stopping it happening.
265 2014-06-03 14:31:26 <crunk-juice> i meant before the apple ban
266 2014-06-03 14:31:52 <crunk-juice> i have an android wallet that I'm interested in putting on the ios platform
267 2014-06-03 14:31:58 <dsnrk> apple never banned bitcoin apps. the CoinJar.io wallet has never been removed.
268 2014-06-03 14:32:21 <crunk-juice> but coinbase, blockchain, etc... have right?
269 2014-06-03 14:32:36 <Hasimir> but a bunch of apps never made it past their vetting process, arguably the same thing
270 2014-06-03 14:32:46 <crunk-juice> there is a reason people were smashing iphones a couple months ago...i hope
271 2014-06-03 14:33:20 <dsnrk> no that was just reddit users being stupid.
272 2014-06-03 14:33:40 <crunk-juice> lol
273 2014-06-03 14:34:20 <wumpus> plaease take this to #bitcoin, this is not development related
274 2014-06-03 14:34:49 <crunk-juice> Anyone got a link to the objective c client?
275 2014-06-03 14:35:22 <dsnrk> wumpus: sorry, didn't realise what channel I was in.
276 2014-06-03 14:44:36 <chichov> is there an off-by-one error in calculating nBits? specifically, that its calculated using timestamps from only 2015 blocks?
277 2014-06-03 14:55:18 <Hasimir> dsnrk, that implies there's a time when reddit users aren't being stupid ...
278 2014-06-03 16:54:32 <jgarzik> wumpus, Has the release announcement actually gone out yet? http://newsbtc.com/2014/06/03/bitcoin-core-version-0-9-2-released/
279 2014-06-03 17:10:47 <Emcy> Per-peer block download tracking and stalled download detection
280 2014-06-03 17:11:01 <Emcy> the second item is great but what is per peer block download about?
281 2014-06-03 17:11:24 <Emcy> bull blocks from all your peers in turn?
282 2014-06-03 17:11:27 <Emcy> pull
283 2014-06-03 17:11:36 <wumpus> jgarzik: no, it hasn't
284 2014-06-03 17:12:12 <wumpus> the announcement was accidentally already posted on the bitcoin foundation blog
285 2014-06-03 17:12:40 <jgarzik> ah
286 2014-06-03 17:13:20 <wumpus> it was removed pretty quickly, but seemingly other sites picked it up
287 2014-06-03 17:13:32 <wumpus> everyone is always too eager to announce a new release
288 2014-06-03 17:13:53 <jgarzik> wumpus, indeed, though I think in this case the linked-to news outlet simply assumed that tag == release
289 2014-06-03 17:14:43 <wumpus> right, and they don't distinguish between the release candidate and the release
290 2014-06-03 17:22:18 <maaku> knotwork: is Mergecoin the most up to date bitcoind with namecoin merged mining patches? People are pointing me to various repos of yours
291 2014-06-03 17:45:24 <Luke-Jr> someone to decide whether RPC is English or American? <.<
292 2014-06-03 17:46:54 <jcrubino> What do they say in Australia?
293 2014-06-03 17:58:01 <phantomcircuit> gmaxwell, lol ok yeah
294 2014-06-03 17:58:03 <phantomcircuit> i was wrong
295 2014-06-03 17:58:17 <phantomcircuit> nearly all of the cpu time is spent on sha256 and malloc/free
296 2014-06-03 17:58:31 <phantomcircuit> but very heavily weighted towards sha256
297 2014-06-03 17:59:45 <devrandom> lastlog devr
298 2014-06-03 17:59:52 <devrandom> urgh
299 2014-06-03 18:01:39 <devrandom> gdm85: thank you for digging into the discrepancy. any chance of a pull request with the LC* stuff? :)
300 2014-06-03 18:52:49 <cfields> ahh, finally
301 2014-06-03 18:53:02 <cfields> sorry if i was spamming dis/reconnects for a while, my ircd was giving me fits
302 2014-06-03 19:02:25 <gmaxwell> phantomcircuit: yea, so the next question is how many of those sha256s are totally redundant.
303 2014-06-03 19:03:11 <gmaxwell> might be kinda fun to just instrument the call to the hash output with something that checks it against a hashmap.
304 2014-06-03 19:07:10 <AndyOfiesh> I believe that I have verified that 3-of-5 multi-sig p2sh (non-compressed) keys is considered non-standard in bitcoind 0.9.1
305 2014-06-03 19:07:15 <AndyOfiesh> 2-of-5 works
306 2014-06-03 19:07:23 <AndyOfiesh> Here is the output from my test http://pastebin.com/5hNVvvY2
307 2014-06-03 19:07:35 <stephenreed> I want a census of full nodes and spv nodes. I know bitnodes.io crawls the network with getaddr. Does this function count connected SPV nodes too?
308 2014-06-03 19:07:46 <AndyOfiesh> Can anyone link me to a 3-of-5 tx spend on the block chain?
309 2014-06-03 19:08:53 <AndyOfiesh> }
310 2014-06-03 19:08:53 <AndyOfiesh> On the 3-of-5 spend I believe it's hitting this condition if (txin.scriptSig.size() > 1650) {
311 2014-06-03 19:08:53 <AndyOfiesh> reason = "scriptsig-size";
312 2014-06-03 19:08:53 <AndyOfiesh> return false;
313 2014-06-03 19:09:12 <AndyOfiesh> The scriptsig-size error shows up in the log
314 2014-06-03 19:09:25 <gmaxwell> stephenreed: no, they can't be counted. also even the spidering only gets a small portion of the full nodes.
315 2014-06-03 19:09:34 <AndyOfiesh> The thing is my script sig is only 570 something bytes
316 2014-06-03 19:09:48 <AndyOfiesh> this is over the limit for the previous version of the code
317 2014-06-03 19:10:18 <stephenreed> gmaxwell: anyones guess then as to the number of SPV clients a full node can support?
318 2014-06-03 19:11:13 <AndyOfiesh> It used to be > 500. That is before march when this was pushed 4d79098ad548874ca6e4c09d873fc2f481e6edb4
319 2014-06-03 19:11:20 <gmaxwell> they cost the same to support as a full node, assuming they aren't catching up with the chain. Nodes normally support a maximum of 125 connections.
320 2014-06-03 19:12:28 <stephenreed> gmaxwell: Thanks.
321 2014-06-03 19:13:13 <AndyOfiesh> The 2-of-5 is considered standard, and it's scriptsig size 480 something.
322 2014-06-03 19:13:24 <AndyOfiesh> It's as if that code change is not there
323 2014-06-03 19:13:38 <AndyOfiesh> getinfo shows that I'm on the latest version
324 2014-06-03 19:13:48 <sipa> which version?
325 2014-06-03 19:13:52 <sipa> 0.9.1 is not master
326 2014-06-03 19:13:58 <AndyOfiesh> 0.9.1
327 2014-06-03 19:14:09 <AndyOfiesh> So maybe thats not in a release yet?
328 2014-06-03 19:14:09 <sipa> that does nit have that change
329 2014-06-03 19:14:13 <sipa> 0.10 will
330 2014-06-03 19:14:42 <phantomcircuit> gmaxwell, actually that's not a bad idea
331 2014-06-03 19:14:44 <AndyOfiesh> Well that's the answer then.
332 2014-06-03 19:14:54 <AndyOfiesh> Thanks
333 2014-06-03 19:15:05 <phantomcircuit> they're all going through the Hash function anyways
334 2014-06-03 19:15:10 <phantomcircuit> so that should be fairly trivial to test
335 2014-06-03 19:15:31 <sipa> i did that a long time ago
336 2014-06-03 19:15:51 <sipa> and removed all duplicates that occur during ibd
337 2014-06-03 19:16:02 <sipa> but there may have been extra ones introduced
338 2014-06-03 19:16:07 <sipa> in fact, i know there have
339 2014-06-03 19:16:35 <Hasimir> ah, sipa, if you ever find yourself in the antipodes, I believe I owe you a beer/drink of your choice :)
340 2014-06-03 19:17:08 <sipa> ... australia?
341 2014-06-03 19:17:15 <Hasimir> yah
342 2014-06-03 19:17:22 <sipa> never been there :(
343 2014-06-03 19:17:36 <Hasimir> a certain samba server was indeed the fault
344 2014-06-03 19:17:46 <sipa> ah :)
345 2014-06-03 19:17:54 <Hasimir> but the vdi sitting on it now is syncing wuite nicely
346 2014-06-03 19:18:01 <Hasimir> *quite
347 2014-06-03 19:18:01 <Luke-Jr> Hasimir: if you give something priority, what is the verb you use? :P
348 2014-06-03 19:18:30 <Hasimir> Luke-Jr, "to the bar!"
349 2014-06-03 19:18:42 <Luke-Jr> lolwut
350 2014-06-03 19:18:48 <Luke-Jr> Hasimir: no, you _______ it <.<
351 2014-06-03 19:19:00 <sipa> "sudo _____"
352 2014-06-03 19:19:12 <gmaxwell> Hasimir: just play along, what word would you use for that?
353 2014-06-03 19:19:13 <Hasimir> inebriate, dilapidate ... escalate
354 2014-06-03 19:19:38 <gmaxwell> Hasimir: he just wants the verb "to give priority" from you
355 2014-06-03 19:19:39 <Hasimir> possibly defenestrate
356 2014-06-03 19:19:44 <Hasimir> depending on mood
357 2014-06-03 19:20:21 <Luke-Jr> Hasimir: say you're mining, and someone wants to pay you to confirm their transaction before others with the same fee..
358 2014-06-03 19:20:25 <Hasimir> prioritise
359 2014-06-03 19:21:32 <gmaxwell> Luke-Jr: told you.
360 2014-06-03 19:21:38 <gmaxwell> (in pm)
361 2014-06-03 19:21:43 <Hasimir> ah
362 2014-06-03 19:21:52 <Luke-Jr> jcrubino: ^
363 2014-06-03 19:22:02 <gmaxwell> I don't know why he was asking, but I assumed he wanted to know if you would use the US or UK spelling.
364 2014-06-03 19:22:23 <Luke-Jr> [17:46:54] <jcrubino> What do they say in Australia?
365 2014-06-03 19:22:28 <jcrubino> yes
366 2014-06-03 19:22:30 <jcrubino> RPC
367 2014-06-03 19:22:31 <jcrubino> ?
368 2014-06-03 19:23:43 <Hasimir> jcrubino, <Hasimir> prioritise
369 2014-06-03 19:23:53 <Hasimir> there you go
370 2014-06-03 19:24:00 <jcrubino> hmm
371 2014-06-03 19:26:03 <sipa> and is that traditional english or simplified english?
372 2014-06-03 19:26:09 <sipa> i always forget which is which
373 2014-06-03 19:26:45 <Hasimir> much of a muchness
374 2014-06-03 19:27:31 <Hasimir> the uk vs. us thing generally boils down to: s vs. z, u vs. no u and some pronunciation
375 2014-06-03 19:27:59 <sipa> ok, prioritise is traditional, prioritize is simplified
376 2014-06-03 19:28:02 <Hasimir> oh, and gaol vs. jail
377 2014-06-03 19:28:03 <Luke-Jr> sipa: the original question, which may be unrelated to Australia, is whether RPC should be English English or American
378 2014-06-03 19:28:30 <sipa> i see
379 2014-06-03 19:28:47 <sipa> let's introduce i18n support for rpc
380 2014-06-03 19:28:58 <sipa> (j/k)
381 2014-06-03 19:29:11 <Hasimir> I think it would depend on how polite the function is meant to be
382 2014-06-03 19:29:44 <sipa> i will leave this discussion to native english speakers
383 2014-06-03 19:30:36 <Hasimir> if it's an imperative, American, if it's a queued request then English
384 2014-06-03 19:31:07 <Hasimir> then you can argue your code has character too, perhaps even a little flair
385 2014-06-03 19:31:29 <sipa> perhaps it shoyld use scittish
386 2014-06-03 19:31:32 <sipa> *scittish
387 2014-06-03 19:31:39 <sipa> *scOttish
388 2014-06-03 19:32:11 <Hasimir> aye, but then only a wee few of us will understand ...
389 2014-06-03 20:32:05 <justanotheruser> Is there a way to set my testnet port? I don't know what it is and 18333 is returning IOError: ('http protocol error', 0, 'got a bad status line', None)
390 2014-06-03 20:32:22 <justanotheruser> or at least I think it may be my port number that's wrong and causing that problem
391 2014-06-03 20:32:40 <justanotheruser> well would it even be returning that if the port no was wrong
392 2014-06-03 20:33:35 <Ry4an> you can check it with something like 'sudo netstat --tcp --listening -p -n' but if something is answering on port 18333 you've probably got the right port and the wrong message
393 2014-06-03 20:36:55 <justanotheruser> Ry4an: yes, it is bitcoind. What would be causing that error? All I do is print getinfo.
394 2014-06-03 20:37:02 <justanotheruser> I started bitcoind with --testnet --server
395 2014-06-03 20:42:32 <Ry4an> from that message it really sounds like it's expecting a HTTP request. What, exactly are you sending to that port?
396 2014-06-03 20:42:53 <justanotheruser> Ry4an: well I connect http://uname:pass:localhost:port
397 2014-06-03 20:43:10 <justanotheruser> It's in python so I do access = ServiceProxy(http://unam...)
398 2014-06-03 20:43:10 <sipa> 18333 is the p2p port
399 2014-06-03 20:43:12 <sipa> use 18332
400 2014-06-03 20:43:15 <justanotheruser> :O
401 2014-06-03 20:43:17 <justanotheruser> ok thanks
402 2014-06-03 20:43:26 <Ry4an> :)
403 2014-06-03 20:45:02 <justanotheruser> thanks sipa Ry4an it worked
404 2014-06-03 21:00:03 <Ry4an> t'was all sipa. I just know netstat well :)
405 2014-06-03 21:02:14 <rvoellmy> help /join
406 2014-06-03 21:02:35 <rvoellmy> asd
407 2014-06-03 21:02:44 <sipa> ?
408 2014-06-03 21:04:22 <rvoellmy> sorry using irssi for the first time. I did not mean to send these messages :)
409 2014-06-03 21:05:31 <snoopstr> irssi rocks :)
410 2014-06-03 21:12:13 <Ry4an> /help join # but I imagine you've figured that out by now :)
411 2014-06-03 21:38:23 <justanotheruser> How is bitcoind able to verify a message if it only has the hashed public key and a message and a signature? Wouldn't it need a public key?
412 2014-06-03 21:38:40 <belcher> you can derive the public key from the message and signature
413 2014-06-03 21:39:03 <sipa> yup
414 2014-06-03 21:39:18 <sipa> unfortunately, this fact was not known to satoshi, so transactions don't use that
415 2014-06-03 21:39:28 <justanotheruser> oh
416 2014-06-03 21:39:38 <justanotheruser> well wouldn't that be a softfork?
417 2014-06-03 21:40:29 <sipa> + changing every wallet on earth
418 2014-06-03 21:41:03 <justanotheruser> oh yeah
419 2014-06-03 21:41:19 <justanotheruser> well it could just make every wallet have the option to send to address 2.0 right?
420 2014-06-03 21:41:30 <sipa> yup
421 2014-06-03 21:41:41 <sipa> but it would be a very invasive softfork
422 2014-06-03 21:41:47 <justanotheruser> okay, thanks guys
423 2014-06-03 21:41:48 <sipa> perhaps we rather want it as a hardfork
424 2014-06-03 21:59:16 <justanotheruser> If someone gives me a pubkey and an address is there an RPC call to verify that that address is derived from the pubkey?
425 2014-06-03 21:59:25 <justanotheruser> Or do I have to construct it myself?
426 2014-06-03 21:59:46 <sipa> afaik no rpc for that exists
427 2014-06-03 22:00:00 <sipa> but it's just hashing + converting to base58
428 2014-06-03 22:00:11 <justanotheruser> indeed
429 2014-06-03 22:00:29 <justanotheruser> I'll probably just do all my crypto using python libs instead of RPC calls then
430 2014-06-03 22:00:57 <justanotheruser> makes more sense anyways I think :)
431 2014-06-03 22:02:05 <sipa> absolutely
432 2014-06-03 22:13:46 <phantomcircuit> gmaxwell, facepalm
433 2014-06-03 22:14:00 <phantomcircuit> the *vast* majority of calls to Hash return duplicate results
434 2014-06-03 22:14:07 <phantomcircuit> total: 1000000 duplicates: 909089 counters.size: 90911
435 2014-06-03 22:15:51 <sipa> phantomcircuit: is that with --disable-wallet ?
436 2014-06-03 22:16:27 <phantomcircuit> sipa, no it's not
437 2014-06-03 22:17:01 <sipa> can you try with --disable-wallet? that will assess whether the duplicates are in wallet code or not
438 2014-06-03 22:19:54 <phantomcircuit> sipa, worth a try, this wallet is mostly empty though so who knows
439 2014-06-03 22:20:17 <sipa> let me fix a few obvious ones first...
440 2014-06-03 22:20:24 <phantomcircuit> i should probably actually go back to work lol
441 2014-06-03 22:22:00 <phantomcircuit> sipa, lol nope it's not
442 2014-06-03 22:22:04 <phantomcircuit> total: 700000 duplicates: 636362 counters.size: 63638
443 2014-06-03 22:22:09 <phantomcircuit> w/ --disable-wallet
444 2014-06-03 22:23:01 <phantomcircuit> sipa, http://pastebin.com/raw.php?i=1P0zhFLT
445 2014-06-03 22:23:06 <phantomcircuit> that does what i think it does right?
446 2014-06-03 22:26:10 <phantomcircuit> yeah it seems to
447 2014-06-03 22:28:49 <sipa> oh my
448 2014-06-03 22:28:58 <sipa> there are many redundant tx hash computations...
449 2014-06-03 22:30:27 <phantomcircuit> sipa, i wonder what that is