1 2013-09-14 11:13:18 <hydromet> interestingly, I was able to run Bitcoin-Qt (which I had built on 27 August 2013) a few minutes ago and the error panel that previously appeared a few days ago (requiring the checklevel=2 workaround) did not appear
2 2013-09-14 11:13:46 <sipa> that's very normal
3 2013-09-14 11:13:56 <sipa> only the last part of the chain is checked at startup
4 2013-09-14 11:14:09 <sipa> since the weird transactions are old enough, they're outside the window
5 2013-09-14 11:14:52 <hydromet> ah! cool
6 2013-09-14 11:15:52 <hydromet> sipa: was that weird transaction an anomaly? likely to occur again in the future?
7 2013-09-14 11:16:59 <jouke> hydromet: it might occur again.
8 2013-09-14 11:47:43 <hydromet> I'd like to build 0.8.4 from Github to avail of the security fix as noted in this article http://thegenesisblock.com/bitcoin-0-8-4-update-provides-security-improvements/
9 2013-09-14 11:48:44 <hydromet> that article was published 4 September 2013 but my git client is showing what appears to be a final date stamp for 0.8.4 of 20 August 2013
10 2013-09-14 11:49:46 <hydromet> is there any way to track down specifically what commit this security update for 0.8.4 came from? I'm reading through the notes and its not yet clear
11 2013-09-14 11:53:23 <sipa> hydromet: we're at 0.8.5 by now
12 2013-09-14 11:53:55 <hydromet> sipa: I know, but I want to try and build 0.8.4 with my existing Qt-5 tools installed (prior to the change over to autotools)
13 2013-09-14 11:54:41 <hydromet> why would the security update article about 0.8.4 be published on 4 September but the last update to 0.8.4 was in August?
14 2013-09-14 11:55:00 <sipa> the commit is authored on august 20
15 2013-09-14 11:55:16 <sipa> doesn't mean it was merged at that point
16 2013-09-14 11:55:25 <hydromet> ah
17 2013-09-14 11:55:51 <sipa> the v0.8.4 git tag has timestamp "Wed Sep 4 06:45:01 2013 +1000"
18 2013-09-14 11:56:58 <sipa> also, i don't understand why 0.8.4 and not 0.8.5
19 2013-09-14 11:58:24 <hydromet> sipa: at this moment I don't want to yet retrograde to Qt-4 (from Qt-5) on my development system ... I will in a few days from now but want to get my friend a version sooner with the security update in 0.8.4 (my friend is paranoid about his wallet)
20 2013-09-14 11:59:05 <hydromet> I'm helping a friend who is not tech savvy
21 2013-09-14 11:59:27 <sipa> the only change between 0.8.4 and 0.8.5 is a bugfix for the incorrect corruption warning
22 2013-09-14 11:59:38 <sipa> if 0.8.4 builds, 0.8.5 will too
23 2013-09-14 11:59:58 <hydromet> sipa: from the timeline, I thought 0.8.4 was prior to the switch over to autotools
24 2013-09-14 12:00:04 <sipa> yes, it is
25 2013-09-14 12:00:06 <sipa> so is 0.8.5
26 2013-09-14 12:00:16 <sipa> only git master is on autotools
27 2013-09-14 12:00:21 <hydromet> really? now I'm confuse
28 2013-09-14 12:00:28 <sipa> but there are no releases yet from the master branch
29 2013-09-14 12:00:36 <sipa> only security backports to the 0.8 line
30 2013-09-14 12:01:55 <hydromet> hmmm ... I thought gavin had mentioned to me several hours ago that 0.8.5 probably wouldn't build on OS X with Qt-5 due to the problem of autotools and Qt-5 on OS X not playing well together (so he reverted to Qt-4 using MacPorts), but maybe he meant for master
31 2013-09-14 12:02:21 <sipa> i suppose so
32 2013-09-14 12:02:36 <sipa> 0.8.5 doesn't have any qt or build changes
33 2013-09-14 12:06:16 <hydromet> sipa: great, then I will try with 0.8.5
34 2013-09-14 12:07:32 <hydromet> sipa: when do you predict autotools might end up being required for branches outside master?
35 2013-09-14 12:07:52 <sipa> 0.9
36 2013-09-14 12:11:18 <warren> hmm, what is an #ifdef to use "is this x86 of any type"?
37 2013-09-14 12:11:53 <sipa> i'd use sizeof(void*), but that doesn't work at preprocessor stage :)
38 2013-09-14 12:12:28 <warren> eh?
39 2013-09-14 12:12:52 <warren> basically I want to #ifdef to copy CPUID only on x86 (but not arm)
40 2013-09-14 12:13:00 <sipa> oh
41 2013-09-14 12:13:19 <sipa> sorry, i thought you meant x86 vs x86_64
42 2013-09-14 12:13:39 <warren> gmaxwell said to save it as an integer global in init.cpp and submit this to bitcoin
43 2013-09-14 12:13:44 <sipa> no idea; perhaps compiler-dependent
44 2013-09-14 12:14:45 <warren> http://git.xiph.org/?p=opus-tools.git;a=blob;f=src/cpusupport.h;h=d68cf0c4f724bd3d35fc03f8cd701d690a5e8c37;hb=HEAD He pointed at this as an example, although it does flag testing instead of saving the integer.
45 2013-09-14 12:17:40 <zeddan81>
46 2013-09-14 12:22:28 <jgarzik> warren, __i386__ || __x86_64__
47 2013-09-14 12:22:59 <sipa> i assume the assembly to fetch cpuid also differs between i386 and x86_64
48 2013-09-14 12:23:09 <jgarzik> it shouldn't
49 2013-09-14 12:23:27 <jgarzik> cpuid in long mode should be the same as cpuid in 32-bit mode
50 2013-09-14 12:25:22 <warren> jgarzik: that works on windows and mac too?
51 2013-09-14 12:25:44 <jgarzik> warren, it should yes
52 2013-09-14 12:26:29 <jgarzik> RE cpuid, I was going between 32-bit mode and 64-bit mode while playing with data storage and registers: http://gtf.org/garzik/misc/mmx-test.c
53 2013-09-14 12:26:50 <warren> ok, so the cpuid global int only gets set #if defined(__i386__) || defined(__x86_64__)
54 2013-09-14 12:26:52 <jgarzik> just a few days ago. cpuid is definitely the same in both modes.
55 2013-09-14 12:27:46 <jgarzik> it's basically a tiny library, that stores a "uint256_t" type in SSE2 or MMX registers
56 2013-09-14 12:27:56 <jgarzik> as a nice alternative for key storage, than RAM
57 2013-09-14 12:28:27 <warren> jgarzik: http://git.xiph.org/?p=opus-tools.git;a=blob_plain;f=src/cpusupport.h;h=d68cf0c4f724bd3d35fc03f8cd701d690a5e8c37;hb=HEAD
58 2013-09-14 12:28:34 <warren> jgarzik: I apparently need this other method for win32?
59 2013-09-14 12:28:49 <jgarzik> for my "security robot" project (a sort of highly secure key storage)
60 2013-09-14 12:29:20 <jgarzik> warren, all that code demonstrates is what is necessary to find a library with cpuid() on various platforms
61 2013-09-14 12:29:40 <jgarzik> warren, that is not a check for processor type
62 2013-09-14 12:29:45 <warren> I know
63 2013-09-14 12:29:51 <warren> I'm doing something different from this
64 2013-09-14 12:31:20 <warren> hm
65 2013-09-14 12:32:35 <jgarzik> ACTION is disappointed at the lack of "data storage" registers, for the purposes just described, on non-x86 platforms
66 2013-09-14 12:33:09 <jgarzik> CISC is wonderfully complex, and inevitably includes some extra register sets, if not a few spare registers untouched in normal operation, such as debug registers
67 2013-09-14 12:34:17 <jgarzik> RISC, OTOH, ...
68 2013-09-14 12:35:50 <warren> jgarzik: is there a way in linux to fake the cpuid in order to test the different codepaths?
69 2013-09-14 12:36:01 <warren> jgarzik: particularly 32bit x86 without SSE2
70 2013-09-14 12:37:20 <jgarzik> warren, I don't understand the question
71 2013-09-14 12:37:31 <jgarzik> warren, if you want to test a codepath stick in a flag and test that codepath
72 2013-09-14 12:37:54 <jgarzik> --clear-sse2-bit-from-cpuid
73 2013-09-14 12:38:31 <warren> jgarzik: I want to be sure that my runtime selecting code runs properly on pre-SSE2 hardware
74 2013-09-14 12:38:52 <jgarzik> --debug-no-sse2
75 2013-09-14 12:39:10 <jgarzik> --debug-disable-all-fancy-features :)
76 2013-09-14 12:39:20 <sipa> --do-what-i-want
77 2013-09-14 12:40:17 <jgarzik> if (dwim) edx &= ~(1U << 26);
78 2013-09-14 12:40:26 <jgarzik> simple
79 2013-09-14 12:40:41 <jgarzik> (26 is the SSE2 bit in cpuid's edx return value)
80 2013-09-14 12:41:33 <jgarzik> On a different note
81 2013-09-14 12:42:09 <jgarzik> I have decided I like JavaScript, but it is too broken to be used as is. It needs: (1) accurate integer math, (2) accurate integer math, (3) ...
82 2013-09-14 12:42:48 <jgarzik> I want JS with optional static typing and python3-like integers
83 2013-09-14 12:43:19 <jgarzik> C-like bit math should work without headaches (it does not in JS)
84 2013-09-14 12:51:38 <sipa> pfff, i really don't feel like digging through bitcoinj to figure out why headers-first fails with it
85 2013-09-14 12:53:25 <jgarzik> sipa, I don't feel like digging through bitcoinj to add P2SH, but it appears that is what we must do
86 2013-09-14 12:53:35 <jgarzik> (we == I, most likely)
87 2013-09-14 12:53:57 <sipa> i expect that to be easy
88 2013-09-14 12:54:02 <jgarzik> TD does not really like P2SH :/
89 2013-09-14 12:54:03 <sipa> but yeah, not feeling like that either
90 2013-09-14 12:54:25 <jgarzik> ACTION -> not a fan of Java
91 2013-09-14 12:54:31 <warren> hmm, we only care about saving the feature information, not the other stuff from cpuid right?
92 2013-09-14 12:54:37 <sipa> ACTION -> neither
93 2013-09-14 12:55:00 <jgarzik> warren, cpuid is not an expensive instruction...
94 2013-09-14 12:55:19 <sipa> as long as you don't run it in a tight loop
95 2013-09-14 12:55:40 <warren> jgarzik: if I understand him correctly, he said save it as a global variable in init.cpp for others to use in whatever way they want
96 2013-09-14 12:55:45 <sipa> (it flushes the execution pipeline, afaik)
97 2013-09-14 12:56:45 <jgarzik> warren, hum, back up a sec :) dumb question: why are you storing cpuid feature bits in init.cpp?
98 2013-09-14 12:57:11 <jgarzik> warren, are we talking about bitcoin-derived codebase?
99 2013-09-14 12:57:14 <warren> jgarzik: it makes no sense to query it many times later during runtime
100 2013-09-14 12:57:48 <warren> jgarzik: we have a SSE2 optimized part of litecoin that's 50% faster. gmaxwell said bitcoin might want something similar in the future, so to put something generic in init.cpp that can be flag tested.
101 2013-09-14 12:57:57 <jgarzik> warren, Yes, almost all cpuid users execute it once upon startup, then refer to returned results
102 2013-09-14 12:58:48 <warren> so I'm going to save only the features that appears to be put into edx as an unsigned integer
103 2013-09-14 12:59:14 <jgarzik> warren, but at least wrap it in a tiny processor-intrinsics class or similar, if this is for long term bitcoin codebase use
104 2013-09-14 12:59:34 <warren> jgarzik: I'd need to see an example of this
105 2013-09-14 12:59:36 <jgarzik> rather that blat "cpuid_edx" as a global var in the global namespace
106 2013-09-14 12:59:40 <jgarzik> *than
107 2013-09-14 13:00:29 <warren> I'm going to make it work for our use, then someone can tell me how to do it properly.
108 2013-09-14 13:07:00 <jgarzik> Awesome! USB condoms: http://venturebeat.com/2013/09/13/phone-charger-condoms/
109 2013-09-14 13:07:18 <jgarzik> makes sense
110 2013-09-14 13:10:28 <warren> scrypt.h:
111 2013-09-14 13:10:29 <warren> void scrypt_1024_1_1_256_sp(const char *input, char *output, char *scratchpad);
112 2013-09-14 13:10:29 <warren> void scrypt_1024_1_1_256_sp_sse2(const char *input, char *output, char *scratchpad);
113 2013-09-14 13:10:30 <hydromet> sipa: I just started from scratch, git cloning the 0.8.5 branch into a fresh new directory on my Mac development machine (with Qt-5) and when I tried to make it (qmake etc.) I got the same error as I did several hours ago
114 2013-09-14 13:10:38 <warren> hmm
115 2013-09-14 13:10:49 <hydromet> sipa: so apparently Gavin was correct that 0.8.5 doesn't like Qt-5 on OS X
116 2013-09-14 13:10:56 <hydromet> sipa: master branch or no master branch
117 2013-09-14 13:11:15 <hydromet> sipa: so now I will try 0.8.4
118 2013-09-14 13:11:27 <warren> hydromet: that didn't change
119 2013-09-14 13:12:43 <hydromet> warren: sigh, well something must have changed because on 27 August 2013 I had no problem building on OS X (with Gavin's suggestions) with qmake etc. with Qt-5 on my Mac
120 2013-09-14 13:12:58 <hydromet> warren: Gavin has since reverted to Qt-4 via MacPorts and dumped Qt-5
121 2013-09-14 13:13:13 <hydromet> warren: apparently autotools and Qt-5 do not like each other on OS X
122 2013-09-14 13:13:32 <hydromet> warren: but apparently autotools are not in the 0.8.5 branch?
123 2013-09-14 13:14:27 <warren> hydromet: 0.8.4 and 0.8.5 are nearly the same thing, and pre auto tools
124 2013-09-14 13:14:29 <jgarzik> hydromet, autotools are not in the 0.8.5 branch
125 2013-09-14 13:14:35 <jgarzik> nor 0.8.4
126 2013-09-14 13:15:09 <sipa> https://github.com/bitcoin/bitcoin/compare/v0.8.4...v0.8.5
127 2013-09-14 13:15:18 <hydromet> sigh
128 2013-09-14 13:15:19 <sipa> that's the diff between 0.8.4 and 0.8.5
129 2013-09-14 13:15:34 <sipa> are you sure you chedk out 0.8.5 and not master?
130 2013-09-14 13:16:24 <hydromet> maybe I screwed that up but using my git client I specifically asked to clone from the 0.8.5 branch
131 2013-09-14 13:17:03 <sipa> you don't close a branch, you lone a repository
132 2013-09-14 13:17:14 <sipa> within a repository, you checkout a branch
133 2013-09-14 13:17:16 <jgarzik> hydromet, git clone git://github.com/bitcoin/bitcoin.git && git checkout -b tmp v0.8.5
134 2013-09-14 13:17:26 <jgarzik> hydromet, git clone git://github.com/bitcoin/bitcoin.git && cd bitcoin && git checkout -b tmp v0.8.5
135 2013-09-14 13:18:58 <hydromet> actually I used the wrong language sorry, I just looked at screen capture of my git client and I cloned https://github.com/bitcoin/bitcoin.git but also checked out branch 0.8.5
136 2013-09-14 13:19:21 <sipa> ok
137 2013-09-14 13:19:29 <sipa> what is in clientversion.h?
138 2013-09-14 13:19:49 <sipa> CLIENT_VERSION_REVISION in particular
139 2013-09-14 13:20:36 <hydromet> #define CLIENT_VERSION_REVISION 5
140 2013-09-14 13:20:58 <warren> hmm, if I want to set a function pointer during startup to use either scrypt_1024_1_1_256_sp_sse2() or scrypt_1024_1_1_256_sp(), do I want to do it in init.cpp?
141 2013-09-14 13:21:23 <hydromet> perhaps I should not use a git client with a GUI (I'm using a git client named SourceTree - I like having a GUI but maybe I should just go CLI)?
142 2013-09-14 13:21:44 <imton> guys , how can multisig txs no-completed signed be broadcasted?
143 2013-09-14 13:21:54 <jgarzik> sigh
144 2013-09-14 13:22:06 <jgarzik> this auction project makes me want watch-only addresses
145 2013-09-14 13:22:21 <jgarzik> ACTION tries to recall Gavin's point about watch-only wallets versus watch-only addresses
146 2013-09-14 13:22:22 <sipa> imton: you can only broadcast fully-signed transactions
147 2013-09-14 13:22:54 <jgarzik> imton, what sipa said. Partially signed multisig transactions must be passed around via email or some other means.
148 2013-09-14 13:23:41 <sipa> jgarzik: i tried to come up with watch-only addresses that can be integrated with spendable addresses within the same wallet, but the semantics are a bit fuzzy
149 2013-09-14 13:23:55 <sipa> gavin prefers whole wallets to be spendable or not, and i think i agree
150 2013-09-14 13:23:56 <imton> sipa jgarzik I see. Is there any way in the future to implement a solution for this?
151 2013-09-14 13:24:02 <sipa> imton: absolutely not
152 2013-09-14 13:24:18 <sipa> imton: there is no point in requiring the world to mirror your invalid transaction
153 2013-09-14 13:24:42 <sipa> jgarzik: apart from the fact that multiwallet seems stalled
154 2013-09-14 13:25:15 <jgarzik> sipa, reviewing gavin's comments on https://github.com/bitcoin/bitcoin/pull/2861 it sounds like the PR can move forward with minor RPC additions
155 2013-09-14 13:25:21 <imton> sipa: I agree with that, but maybe there could be a messaging system on top of bitcoin-network for this kind of situations...
156 2013-09-14 13:25:21 <jgarzik> I could probably do that
157 2013-09-14 13:25:38 <jgarzik> imton, like bitmessage?
158 2013-09-14 13:25:52 <jgarzik> or irc? or AIM? or email? or...
159 2013-09-14 13:25:53 <imton> jgarzik I have no idea what bit message is ..
160 2013-09-14 13:26:01 <jgarzik> imton, I bet Google knows ;p
161 2013-09-14 13:26:06 <sipa> imton: sure, just have some protocol to negotiate transactions between participating parties to build up transactions
162 2013-09-14 13:26:13 <jgarzik> or http
163 2013-09-14 13:26:16 <sipa> and when they're done, broadcast on the bitcoin p2p network
164 2013-09-14 13:27:12 <imton> sipa: but that protocol should be implemented separately right?
165 2013-09-14 13:27:19 <sipa> yes
166 2013-09-14 13:27:36 <sipa> it doesn't belong in the p2p sysem
167 2013-09-14 13:29:31 <imton> how will the proposed payment protocol send the request to from the merchant to the buyer ?
168 2013-09-14 13:29:38 <imton> (may be I understood it wrong)
169 2013-09-14 13:29:47 <sipa> the buyer fetches it
170 2013-09-14 13:29:49 <sipa> https
171 2013-09-14 13:30:26 <sipa> and then sends a transaction + metadata back
172 2013-09-14 13:30:36 <imton> fetches from?
173 2013-09-14 13:31:07 <sipa> the merchant
174 2013-09-14 13:31:34 <imton> oh, so the request doesn't come from the bitcoin network, right?
175 2013-09-14 13:31:40 <sipa> no, not at all
176 2013-09-14 13:31:50 <sipa> the network is only involved when the deal is done
177 2013-09-14 13:31:59 <sipa> and the transaction is broadcast
178 2013-09-14 13:32:38 <imton> ok. so there is no way to send anything that isn't a properly signed tx right?
179 2013-09-14 13:32:54 <sipa> no, why would you?
180 2013-09-14 13:33:14 <sipa> the only reason to broadcast it is to get it into a block
181 2013-09-14 13:33:37 <imton> my idea is to rely *on* the bitcoin network to send a message (whatever it is)
182 2013-09-14 13:33:53 <sipa> DO NOT
183 2013-09-14 13:34:05 <sipa> it's not designed for that, it's expensive and unreliable
184 2013-09-14 13:34:16 <sipa> it may seem easy because of existing infrastructure
185 2013-09-14 13:34:21 <sipa> but it's really a bad solution
186 2013-09-14 13:34:42 <imton> if I properly sign a tx that is send to my self, and do not include any fee.... couldn't I send a message and everybody connected will get it?
187 2013-09-14 13:35:01 <sipa> why would anyone want to get it?
188 2013-09-14 13:35:17 <imton> I won't do it. I am just trying to understand all parts of bitcoin.
189 2013-09-14 13:35:18 <sipa> you're forcing people to download data that is irrelevant to maintaining the system
190 2013-09-14 13:35:34 <sipa> you're just abusing it as a horribly inefficient communication mechnaism
191 2013-09-14 13:35:53 <imton> because that message could be a partially signed tx
192 2013-09-14 13:35:54 <sipa> ok, see it this way
193 2013-09-14 13:36:07 <sipa> bitcoin-the-network is an incredibly expensive system
194 2013-09-14 13:36:16 <sipa> if you count the cost of all full nodes that are running
195 2013-09-14 13:36:20 <sipa> and bandwidth it uses
196 2013-09-14 13:36:47 <sipa> people run nodes for free because they believe it helps building something that is valuable: a worldwide payment system with many advantages
197 2013-09-14 13:37:10 <sipa> but if you're going to abuse that network for other purposes, you're increasing everyone's costs, without making it more useful
198 2013-09-14 13:37:37 <sipa> the only data that belongs in the network is whatever is necessary for the world to validate that nobody is cheating
199 2013-09-14 13:38:01 <imton> ok, as I told you, I get that principle and I agree. I would be not ethical at all.
200 2013-09-14 13:38:08 <sipa> ok
201 2013-09-14 13:38:39 <imton> I am just trying to understand.
202 2013-09-14 13:38:59 <sipa> i don't understand why you would even want to do that, even if it were ethical and cheap
203 2013-09-14 13:39:15 <sipa> it's slow and unreliable
204 2013-09-14 13:39:22 <sipa> and has very bad privacy properties
205 2013-09-14 13:39:36 <sipa> just keep the data to whomever still needs to sign
206 2013-09-14 13:40:30 <imton> I mean, it would be awesome if in your client appears a kind of a "push notification"/popup and ask you "do you accept?" something like that
207 2013-09-14 13:40:47 <sipa> sure
208 2013-09-14 13:40:55 <sipa> but we have TCP/IP
209 2013-09-14 13:40:57 <sipa> and email
210 2013-09-14 13:41:02 <sipa> and IM
211 2013-09-14 13:41:05 <imton> the idea of a notification, that it comes from the bitcoin-network it self, would be much more user friendly that passing weird hexs
212 2013-09-14 13:41:14 <imton> or base-wahetever
213 2013-09-14 13:41:21 <sipa> what does user-friendlyness have to do with it
214 2013-09-14 13:41:30 <sipa> you just need a good interface for that
215 2013-09-14 13:41:46 <sipa> the protocol used behind the scenes doesn't need to be visible to users
216 2013-09-14 13:41:57 <sipa> oh, and i agree, human shouldn't ever see transactions or addresses
217 2013-09-14 13:42:24 <sipa> i don't mean passing transaction in hex manually around in emails or IM
218 2013-09-14 13:42:31 <imton> how would you implement that for bitcoin-qt?
219 2013-09-14 13:42:32 <sipa> i just mean using similar technology
220 2013-09-14 13:43:06 <sipa> i'm no user experience designer :D
221 2013-09-14 13:43:37 <imton> my requirement would be that a user partially signed the tx and the other 2 (2-of-3) receive a notification that they may redeem/spend if they want that tx.
222 2013-09-14 13:43:55 <sipa> sounds good
223 2013-09-14 13:43:58 <imton> no problem about the UX (i am a ui/ux designer btw)
224 2013-09-14 13:44:17 <imton> I mean the "notification" system/protocol behind?
225 2013-09-14 13:44:20 <imton> how would you do it?
226 2013-09-14 13:44:29 <sipa> send email-s with an attachment that you can open using the client?
227 2013-09-14 13:44:38 <sipa> or have an own broadcast network
228 2013-09-14 13:44:55 <imton> no email... yeah.. 2nd one I think.
229 2013-09-14 13:45:05 <sipa> or have a server that listens for incoming connections, like the payment protocol
230 2013-09-14 13:45:27 <sipa> if the internet weren't crippled by NAT, that'd be a perfect solution
231 2013-09-14 13:45:38 <sipa> now you perhaps need a proxy like an email server is now
232 2013-09-14 13:45:51 <sipa> where you have an account at some payment provider that runs the software
233 2013-09-14 13:45:57 <sipa> though you can just run it yourself too
234 2013-09-14 13:46:13 <imton> yeah
235 2013-09-14 13:46:18 <sipa> the payment protocol can just be extended to support things like that
236 2013-09-14 13:46:29 <imton> I would love to understand more about that
237 2013-09-14 13:46:33 <imton> I still don't get it
238 2013-09-14 13:46:36 <imton> specially the "have a server that listens for incoming connections, like the payment protocol" part
239 2013-09-14 13:46:53 <imton> i haven't read much about it tbh
240 2013-09-14 13:46:53 <sipa> it's a webserver
241 2013-09-14 13:47:02 <sipa> you request a payment descriptor
242 2013-09-14 13:47:06 <sipa> and then send a payment back
243 2013-09-14 13:47:30 <warren> hmm.... is it possible to assign a function pointer dynamically in C++? (I might not be calling it the right thing...)
244 2013-09-14 13:47:43 <sipa> warren: how do you mean?>
245 2013-09-14 13:48:03 <sipa> imton: now look at that webserver as something like a mailserver, that remembers incoming messages
246 2013-09-14 13:48:37 <sipa> imton: and you can connect to it with something like a mailclient, to receive incomi ng (partial) payments
247 2013-09-14 13:48:53 <warren> void scrypt_1024_1_1_256_sp_sse2(const char *input, char *output, char *scratchpad);
248 2013-09-14 13:48:54 <warren> void scrypt_1024_1_1_256_sp(const char *input, char *output, char *scratchpad);
249 2013-09-14 13:49:09 <imton> sipa: who will run that server?
250 2013-09-14 13:49:41 <sipa> imton: anyone who needs to
251 2013-09-14 13:49:51 <warren> sipa: these two functions do the same thing, just built separately. If SSE2 is detected, I want a function to be the former instead of the latter, so dynamic assignment during runtime.
252 2013-09-14 13:50:04 <sipa> warren: ok, so define a function pointer variable
253 2013-09-14 13:50:09 <sipa> and set it to one or the other?
254 2013-09-14 13:50:50 <jgarzik> or just a simple branch, which is more friendly to static linking and some branch prediction hardware. if (sse) foo else bar
255 2013-09-14 13:51:04 <jgarzik> function pointer is cleaner, from the code's perspective
256 2013-09-14 13:51:20 <jgarzik> but ever so slightly less performant
257 2013-09-14 13:51:30 <warren> huh!
258 2013-09-14 13:51:54 <warren> I would have thought a function pointer performs better than a conditional.
259 2013-09-14 13:52:32 <sipa> no, worse
260 2013-09-14 13:52:45 <jgarzik> actually , assuming sse is fastest: if (!sse) foo else bar
261 2013-09-14 13:52:45 <jgarzik> standard assumption is branch-not-taken
262 2013-09-14 13:53:03 <sipa> because you need to know the contents of the variable before you can start executing
263 2013-09-14 13:53:04 <jgarzik> scrypt could also benefit from prefetch instructions
264 2013-09-14 13:53:52 <warren> ok, I understand the branch prediction hardware standard assumption, but I don't get why a function pointer would be slower than a static function?
265 2013-09-14 13:54:06 <jgarzik> though I haven't really studied scrypt I think litecoin's scrypt fits entirely within L1 anyway
266 2013-09-14 13:54:14 <jgarzik> and surely L2 if not L1
267 2013-09-14 13:54:20 <warren> L2 yes
268 2013-09-14 13:55:43 <jgarzik> warren, a static function call is known at compile time. the compiler can, for example, know that it is making a short jump within the current code segment. there are several micro-optimizations that become available if you know the -exact- address of your functions at compile time.
269 2013-09-14 13:56:53 <imton> sipa: oh, and btw, thank you for the watch-only pull :) I kind of love you. I now need to change all my code from the searchrawtransaction pull but it will be worth it
270 2013-09-14 13:57:18 <imton> *searchaddress sorry, or something like that
271 2013-09-14 13:59:04 <warren> jgarzik: "some branch prediction hardware" meaning all modern x86 processors, where standard assumption is branch-not-taken?
272 2013-09-14 13:59:41 <sipa> conditional jumps forward are assumed to be not taken, backards are assumed to be taken
273 2013-09-14 13:59:44 <sipa> afaik
274 2013-09-14 13:59:56 <sipa> backward ones
275 2013-09-14 14:00:43 <jgarzik> sipa, it's mainly a matter of compiler behavior, as the compiler rearranges the code sections depending on what it thinks is right
276 2013-09-14 14:00:51 <warren> are bitcoin conditionals coded with this in mind? do we have a LIKELY() or UNLIKELY() macro available to us?
277 2013-09-14 14:00:55 <jgarzik> rearranges to match the target CPU, I mean
278 2013-09-14 14:01:11 <swulf--> aren't there ways of telling hte compiler which branch is likely to be taken?
279 2013-09-14 14:01:15 <jgarzik> warren, "premature optimization is the root of all..."
280 2013-09-14 14:01:15 <swulf--> ah, what warren said ;)
281 2013-09-14 14:01:34 <warren> jgarzik: in this case I know SSE is faster, so I want to get the conditional correct.
282 2013-09-14 14:01:38 <jgarzik> warren, the kernel had a lot of likely() and unlikely(), but those are quite static, and programmers tend to get them wrong.
283 2013-09-14 14:01:50 <jgarzik> warren, and this is an excellent example :)
284 2013-09-14 14:02:09 <jgarzik> warren, for !sse cases, you would not want to cause an unlikely() statement that trips every single time
285 2013-09-14 14:02:59 <warren> crap, I'll just benchmark the function pointer for now.
286 2013-09-14 14:03:27 <jgarzik> likely() and unlikely() are for situations that apply to all your users. for cases where certain user populations have feature X and other populations do not, unlikely() can hurt if used naively.
287 2013-09-14 14:03:38 <warren> I see what you mean.
288 2013-09-14 14:03:57 <warren> function pointer would punish both populations equally, I guess?
289 2013-09-14 14:04:05 <jgarzik> for users lacking SSE, the "if (!sse)" branch is 100% likely
290 2013-09-14 14:04:30 <warren> might be easier to just declare !SSE2 == unsupported, go away
291 2013-09-14 14:04:36 <warren> less code
292 2013-09-14 14:04:42 <jgarzik> warren, like I said, "if (!sse) scrypt() else scrypt_sse()" should do what you want
293 2013-09-14 14:04:58 <jgarzik> warren, but function pointer is cleaner and easier
294 2013-09-14 14:05:14 <jgarzik> it's all compiled code, so we're just talking little wasted cycles in any case
295 2013-09-14 14:05:20 <jgarzik> "premature optimization..."
296 2013-09-14 14:05:36 <jgarzik> though with mining code, I guess people probably do care a bit more
297 2013-09-14 14:05:53 <warren> this isn't mining code, this is PoW validation
298 2013-09-14 14:07:28 <warren> ok ... I don't even know the syntax to assign a function pointer.
299 2013-09-14 14:07:30 <jgarzik> After doing CPU-intensive heavy lifting in python and javascript, wasting a few dozen or hundred CPU cycles here and there is just fine to me, now. Heresy in Linux kernel land that's what script languages will do to you.
300 2013-09-14 14:08:07 <sipa> void (*sse_code)(const char *inut, char *output, char *scratchpad);
301 2013-09-14 14:08:28 <sipa> if (see) sse_code = script_1024_1_1_256_sse2;
302 2013-09-14 14:08:46 <jgarzik> then "sse_code = myfunc" or "see_code = &myfunc" if ampersands give you extra warm fuzzies
303 2013-09-14 14:08:49 <sipa> and then call sse_code(foo, bar, blah);
304 2013-09-14 14:11:08 <warren> jgarzik: I'll do it both ways and compare the performance. I don't care about the old hardware failing a branch prediction. the vast majority are in the SSE2 population.
305 2013-09-14 14:13:17 <warren> sipa: jgarzik: thank you both
306 2013-09-14 14:14:52 <jgarzik> warren, well the nice thing about branch prediction is that repeatedly taking a branch is a big hint to h/w
307 2013-09-14 14:15:24 <jgarzik> But yes, measurement is best.
308 2013-09-14 14:15:51 <jgarzik> Who knows, maybe a commonly hit indirect function pointer is a case worth optimizing in the CPU these days.
309 2013-09-14 14:16:15 <jgarzik> Seems like such a feature would help many languages, if it could be implemented efficiently in h/w
310 2013-09-14 14:16:36 <jgarzik> i.e. tag a bit of memory in L1 with prediction metadata
311 2013-09-14 14:22:10 <swulf--> jgarzik: can't you just lock the cache line ?
312 2013-09-14 14:24:39 <swulf--> maybe not, from my searches. best you could do is prefetch?
313 2013-09-14 14:27:33 <warren> 4;30am and I'm seeing weird linker errors hat I've never seen before. time to sleep on this.
314 2013-09-14 14:46:46 <sipa> ;;blocks
315 2013-09-14 14:46:47 <gribble> 257970
316 2013-09-14 14:54:37 <hydromet> fanquake: I decided to install Qt-4 on OS X and then I started cleanly from master (instead of checking out branch 0.8.5) and had no problem with
317 2013-09-14 14:54:42 <hydromet> ./autogen.sh && ./configure && make
318 2013-09-14 14:54:51 <hydromet> per your and sipa suggestion
319 2013-09-14 14:57:05 <hydromet> the qualm is that it did not build out a Bitcoin-Qt.app for OS X, but I did get an executable built -> src/qt/bitcoin-qt
320 2013-09-14 14:57:11 <sipa> make deploy
321 2013-09-14 14:57:12 <sipa> i think
322 2013-09-14 14:57:26 <hydromet> I also got src/bitcoind built
323 2013-09-14 14:57:34 <sipa> and the unit tests too
324 2013-09-14 14:58:33 <hydromet> sipa: do you mean Making all in test as in ...
325 2013-09-14 14:58:34 <hydromet> CXX test_bitcoin-bloom_tests.o
326 2013-09-14 14:58:40 <hydromet> if so, then yep I got them
327 2013-09-14 14:59:17 <hydromet> I'll try make deploy now
328 2013-09-14 15:01:48 <hydromet> quite close but make deploy ended in this error
329 2013-09-14 15:01:51 <hydromet> Error: Could not import appscript which is required for fancy disk images.
330 2013-09-14 15:02:00 <hydromet> Please install it e.g. with "sudo easy_install appscript".
331 2013-09-14 15:02:11 <hydromet> make: *** [Bitcoin-Qt.dmg] Error 1
332 2013-09-14 15:02:15 <hydromet> interesting
333 2013-09-14 15:02:21 <warren> /home/warren/gitian/litecoin/src/db.cpp:493: multiple definition of `scrypt_1024_1_1_256_sp'
334 2013-09-14 15:02:21 <warren> obj/db.o: In function `__gnu_cxx::new_allocator<int>::deallocate(int*, unsigned long)':
335 2013-09-14 15:02:22 <warren> obj/checkpoints.o:/home/warren/gitian/litecoin/src/checkpoints.cpp:72: first defined here
336 2013-09-14 15:03:06 <warren> sipa: followed your suggested syntax, at final link it fails with a bunch of errors like this. There is no `scrypt_1024_1_1_256_sp' in checkpoints.cpp ...
337 2013-09-14 15:07:33 <hydromet> hmm ... http://appscript.sourceforge.net/
338 2013-09-14 15:07:42 <hydromet> > Please note that appscript is no longer developed or supported, and its use is not recommended for new projects.
339 2013-09-14 15:10:20 <sipa> warren: is your code in a header file, and included in multiple places?
340 2013-09-14 15:10:56 <hydromet> gavinanderson: did you install appscript on OS X so make deploy will create Bitcoin-Qt.app?
341 2013-09-14 15:11:31 <warren> sipa: I guess I shouldn't have put the void (*sse_code) ... in there... moving
342 2013-09-14 15:12:56 <warren> then I need to tackle the annoyingly different makefile.unix and bitcoin-qt.pro
343 2013-09-14 15:14:59 <warren> grr... scope errors now
344 2013-09-14 15:22:11 <warren> sipa: https://github.com/litecoin-project/litecoin/compare/7d1cd46208d4%5E...4a026b83dbe8
345 2013-09-14 15:22:24 <warren> sipa: I am extremely close ...
346 2013-09-14 15:27:42 <warren> jgarzik: your earlier question about L1 or L2: static const int SCRYPT_SCRATCHPAD_SIZE = 131072 + 63;
347 2013-09-14 16:21:55 <BenderCoin> Hello is there a site or a script that is available right now to show the number of bitcoin addresses with a non-zero balance? I have seen some calcs, but it would be nice to see someone publishing this or a script where I can run it myself. Thanks.
348 2013-09-14 16:33:45 <greBit> Hello, any kind souls out there who can help me with libBitcoin and Genjix's SX tools? How do I actually get a leveldb blockchain?
349 2013-09-14 17:03:08 <Vinnie_win> I'm still a little confused about the difference between the interpretation of the bits in the byte sequences used as input to both EncodeBase64 and EncodeBase58. It seems that for base 64 encoding, the bytes are treated as a bit stream where the most significant bits come "first". They are separated into groups of 6 and then converted to the proper alphabet character.
350 2013-09-14 17:03:46 <Vinnie_win> But for base 58 encoding, the byte sequence is considered a big endian multi precision unsigned integer. The code first transforms it into little endian and then performs a straightforward radix conversion.
351 2013-09-14 17:05:20 <Vinnie_win> This method used for interpreting the byte sequence is different between base64 and base58. The first character output from the base58 encoder is formed from the low order 6 bits of the first data byte. The first character output from the base64 encoder is formed from the high order 6 bits of the first data byte. Is this analysis correct?
352 2013-09-14 17:10:16 <jgarzik> Vinnie_win, base58 is quite different from base64, yes
353 2013-09-14 18:26:37 <Vinnie_win> so my analysis is right
354 2013-09-14 18:27:28 <Vinnie_win> Or does BIGNUM treat the most significant bit of the first byte in the mpi representation as the lowest order bit in the little endian mpi?
355 2013-09-14 18:27:43 <Vinnie_win> (excluding the 4 byte length prefix)
356 2013-09-14 19:15:38 <greBit> Anyone know how to create a leveldb blockchain for libbitcoin?
357 2013-09-14 22:18:10 <gavinandresen> hydromet: make deploy creates the .dmg file. You should get a Bitcoin-Qt.app from the default autotools build.
358 2013-09-14 22:20:17 <gavinandresen> hydromet: RE: Qt5 building: 0.8.0/1/2/3/4/5 do not support Qt5 (I believe). The 0.9 master branch did, until autotools. And autotools probably does, I just don't know how to tell it to useQt5 instead of Qt4â¦.
359 2013-09-14 22:23:03 <warren> gavinandresen: I got autotools master to build qt5
360 2013-09-14 22:23:53 <warren> gavinandresen: I had to hardcode lrelease-qt5 and othertools-qt5, which allowed bitcoin-qt to build, but fails build during tests unless I remove qt4 headers from the system
361 2013-09-14 22:26:54 <Luke-Jr> warren: sounds like we just need to get it to detect header directories smarter - how does your OS install both header-sets concurrently?
362 2013-09-14 22:43:29 <warren> Luke-Jr: this will be different on other distros
363 2013-09-14 22:43:51 <Luke-Jr> warren: perhaps, but might as well make it work on ones we know
364 2013-09-14 22:45:13 <warren> Luke-Jr: there seems to be a reluctance for people to review autotools patches, and the obviously good -O2 PR got stuck in minutiae
365 2013-09-14 22:45:31 <Luke-Jr> warren: it's only been a day or two
366 2013-09-14 22:45:52 <warren> and 27 comments...
367 2013-09-14 22:45:55 <Luke-Jr> I like cfields/jgarzik's proposal to just use autotools default for optimisations
368 2013-09-14 22:46:11 <Luke-Jr> saves a lot of fence painting
369 2013-09-14 22:46:21 <warren> autotools default is -O2?
370 2013-09-14 22:46:36 <Luke-Jr> I think -g -O1, but not 100% certain
371 2013-09-14 22:46:43 <warren> fail
372 2013-09-14 22:47:13 <Luke-Jr> completely overridable
373 2013-09-14 22:47:34 <warren> make the default what users actually use
374 2013-09-14 22:47:41 <warren> make debug not default
375 2013-09-14 22:48:28 <Luke-Jr> there is no debug in autotools default
376 2013-09-14 22:50:09 <warren> just accept this: https://github.com/bitcoin/bitcoin/pull/2996
377 2013-09-14 22:50:25 <warren> it's an impediment to other people getting things done
378 2013-09-14 22:50:37 <warren> if people want to clean it up more later, do so later
379 2013-09-14 22:50:42 <warren> for now just make it work
380 2013-09-14 22:54:56 <jcorgan> has there been any empirical studies showing that bitcoin tx and block arrival times really do follow a Poisson distribution?
381 2013-09-14 22:56:00 <sipa> transaction frequencies are quite correlated
382 2013-09-14 22:56:04 <jcorgan> well, that really should be "can be modeled as a poisson process"
383 2013-09-14 22:56:05 <sipa> i doubt those are poisson
384 2013-09-14 22:56:15 <sipa> block generation probably is
385 2013-09-14 22:56:30 <sipa> assuming constant hashrate
386 2013-09-14 22:57:44 <jcorgan> where do you think the tx arrival times become correlated
387 2013-09-14 22:58:54 <sipa> they are often broadcast in batches
388 2013-09-14 22:59:00 <jcorgan> ah ok
389 2013-09-14 23:03:06 <michagogo> Broadcast in batches? o_O
390 2013-09-14 23:03:47 <jcorgan> that would make the mean arrival time less than expected, but i wonder if it changes the distribution shape much given the "dispersion" that would happen while relaying
391 2013-09-14 23:09:13 <jcorgan> i'm instrumenting bitcoind to collect inv arrival times across connections
392 2013-09-14 23:09:42 <jcorgan> i could store in ram all the individual data points and give that to the user over rpc, or
393 2013-09-14 23:09:59 <jcorgan> i could accumulate them into a histogram of some appropriate bin size and present that, or
394 2013-09-14 23:10:29 <jcorgan> i could make some assumptions about the distribution and do an online parameter estimation and provide the distribution parameters
395 2013-09-14 23:11:19 <michagogo> ipconfig
396 2013-09-14 23:11:20 <michagogo> bah
397 2013-09-14 23:11:54 <jcorgan> laziness dictates the histogram approach
398 2013-09-14 23:19:14 <hydromet> gavinandresen: with Qt-4 now installed on my Mac (via MacPorts), I was successful building the latest from Master with
399 2013-09-14 23:19:15 <hydromet> ./autogen.sh && ./configure && make
400 2013-09-14 23:19:32 <hydromet> after this I then tried make deploy
401 2013-09-14 23:20:04 <hydromet> but ended up with this error
402 2013-09-14 23:20:23 <hydromet> Error: Could not import appscript which is required for fancy disk images.
403 2013-09-14 23:20:28 <hydromet> Please install it e.g. with "sudo easy_install appscript".
404 2013-09-14 23:20:33 <hydromet> make: *** [Bitcoin-Qt.dmg] Error 1
405 2013-09-14 23:20:57 <hydromet> gavinandresen: so I was not able to get Bitcoin-Qt.app
406 2013-09-14 23:21:57 <hydromet> I'm wondering if, on your OS X development system, did you install a MacPort such as py27-appscript ?
407 2013-09-14 23:22:38 <hydromet> I'd prefer to use a MacPort for the appscript requirement rather than "sudo easy_install appscript"
408 2013-09-14 23:23:57 <gavinandresen> hydromet: I didn't use the macports version, but that might just be because the macports version wasn't available before.
409 2013-09-14 23:24:17 <gavinandresen> I'm using: /Library/Python/2.7/site-packages/appscript-1.0.1-py2.7-macosx-10.7-intel.egg
410 2013-09-14 23:24:23 <gavinandresen> I don't remember how I installed it.
411 2013-09-14 23:24:34 <hydromet> ok
412 2013-09-14 23:24:47 <gavinandresen> ⦠but again, unless you're going to be creating .dmg pacakges, you don't need it.
413 2013-09-14 23:24:56 <hydromet> my guess is that you probably did "sudo easy_install appscript"
414 2013-09-14 23:25:38 <hydromet> I don't need .dmg files but I wonder why I'm not able to arrive at a Bitcoin-Qt. app file after make deploy?
415 2013-09-14 23:26:53 <hydromet> gavinandresen: when you build with autotools do you do -> ./autogen.sh && ./configure && make deploy ?
416 2013-09-14 23:28:52 <gavinandresen> I will only make deploy if I want a final release build.
417 2013-09-14 23:29:03 <midnightmagic> do the makefiles know how to do gitian builds yet?
418 2013-09-14 23:29:40 <gavinandresen> hydromet: during development, I just 'make' . And that creates Bitcoin-Qt.app.
419 2013-09-14 23:29:59 <hydromet> gavinandresen: it turns out I am getting Bitcoin-Qt.app to be created in my source directory but in the OS X Finder it has a circle with a slash in the middle of it (and it is greyed out)
420 2013-09-14 23:31:04 <gavinandresen> hydromet: what version of OSX?
421 2013-09-14 23:31:20 <hydromet> gavinandresen: 10.8.4
422 2013-09-14 23:31:29 <gavinandresen> works for me on 10.7
423 2013-09-14 23:31:40 <hydromet> gavinandresen: do you pass anything to configure?
424 2013-09-14 23:31:54 <gavinandresen> hydromet: nope
425 2013-09-14 23:32:35 <hydromet> gavinandresen: hmm ... I wonder if this is going to be problematic for people given that OS X 10.9 is soon to be released?
426 2013-09-14 23:32:40 <gavinandresen> hydromet: (well, I've got a couple different trees, some are --enable-debug for debugging)
427 2013-09-14 23:33:45 <hydromet> gavinandresen: its hard to say when Apple will stop supporting OS X Lion (10.7)
428 2013-09-14 23:33:58 <gavinandresen> hydromet: okey dokey
429 2013-09-14 23:34:49 <gavinandresen> hydromet: I don't have an OSX 10.8 machine here in Australia to test, so either find somebody else who does and work with them to figure out what is wrong, or figure it out yourself.
430 2013-09-14 23:34:50 <hydromet> gavinandresen: sorry to be a pain, its a bummer I can't get this to work ... I'll dig into it some more and see what I can come up with
431 2013-09-14 23:51:48 <kuzetsa> gmaxwell: re: commentary 29 hours ago about peer 16952 seconds off ---> http://www.timeanddate.com/time/time-zones-interesting.html <-- that's a valid offset for nepal --> (quote) "When it is midnight in Greenwich, it is 5:45 AM in Kathmandu and the rest of Nepal. Nepal is 15 minutes ahead of India."
432 2013-09-14 23:53:01 <kuzetsa> (assuming an hour difference due to summer savings time going on or coming off somewhere)
433 2013-09-14 23:55:09 <kuzetsa> it's naive to pretend there are no timezones set plus or minus 15 or 30 from the more common "off by an hour east to west" timezones
434 2013-09-14 23:56:54 <sipa> kuzetsa: where are such assumptions made?
435 2013-09-14 23:59:44 <kuzetsa> [14:33:08] <@gmaxwell> I've been seeing a lot of people reporting the time wrong warning. Doublec mentioned it a while back, it seems to me that there is some insane misbehavior on the network, and I'm wondering what the motivation is?
436 2013-09-14 23:59:45 <kuzetsa> [14:43:31] <@gmaxwell> e.g. one user just gave me a getpeerinfo that has a spread of 1379096587 - 1379079635 = 16952 seconds.
437 2013-09-14 23:59:52 <kuzetsa> like I said, 29 hours ago