1 2014-10-14 01:03:27 <jgarzik> cfields, sipa, BlueMatt: Who read the example code?
2 2014-10-14 01:03:52 <jgarzik> It seems like many are making assumptions without reading code...
3 2014-10-14 01:04:40 <sipa> this code defines no separation at all; any internal change will break what depends on it
4 2014-10-14 01:04:51 <sipa> so what is the advantage of including it?
5 2014-10-14 01:05:17 <sipa> i'm not actually opposed to it, but i think the value is extremely small
6 2014-10-14 01:06:11 <sipa> i mean: there is little downside to keeping the code to build such a library external, together with the code that is using it
7 2014-10-14 01:07:09 <sipa> the code changes are sane (... there are nearly none), but i don't like exposing a library that pretty cannot be used unless you're willing to commit to high-maintenance work to keep it up to date
8 2014-10-14 01:07:19 <cfields> jgarzik: i read the example test-program, yes. The fact that it starts the app-life-cycle told me that the lib inherited bitcoind's runtime behavior
9 2014-10-14 01:08:03 <sipa> if bitpay wants to use something like this, please do - but as a service to the community we're much better spending time on actually modularizing thinhgs
10 2014-10-14 01:09:09 <jgarzik> sipa, 1) The maintenance burden is -theoretically- high, but given what's exported, not so much. Doesn't impact Bitcoin Core anyway.
11 2014-10-14 01:09:38 <jgarzik> sipa, 2) it is a false dichotomy to suggest this prevents further modularization
12 2014-10-14 01:09:48 <sipa> i never suggested 2
13 2014-10-14 01:09:58 <sipa> and i apologize if i gave that impression
14 2014-10-14 01:10:02 <gmaxwell> sipa: https://bitcointalk.org/index.php?topic=816199.msg9182114#msg9182114
15 2014-10-14 01:11:03 <jgarzik> sipa, "we're much better spending time on actually modularizing thinhgs" I don't see the PR has such a burdensome cost that it impacts that.
16 2014-10-14 01:11:30 <sipa> jgarzik: the cost of that library will be poking people to upgrade their shit
17 2014-10-14 01:11:35 <jgarzik> The main complaint seems to be "JJ's plugin must keep pace with internals"
18 2014-10-14 01:11:41 <sipa> no
19 2014-10-14 01:11:42 <jgarzik> which is normal for plugins
20 2014-10-14 01:11:59 <sipa> my complaint is that every application that uses it must keep pace with internals...
21 2014-10-14 01:12:08 <gmaxwell> which they'll refuse to do, esp since the modulatization changes will gratitiouslly breaking things. But that already exists, since people patch things.
22 2014-10-14 01:12:21 <jgarzik> sipa, Um, that's not true.
23 2014-10-14 01:12:33 <jgarzik> sipa, the JS API doesn't directly export internals or anything of the sort.
24 2014-10-14 01:12:47 <jgarzik> very much marshalled for JS
25 2014-10-14 01:12:50 <sipa> any user of libbitcoind.so
26 2014-10-14 01:13:00 <sipa> which bitcoind.js is one of
27 2014-10-14 01:13:25 <sipa> if bitcoind.js is intended to be the only user of libbitcoind.so, i think the code for building it belongs there and not here
28 2014-10-14 01:14:13 <lechuga_> which PR r u guys discussing?
29 2014-10-14 01:14:19 <sipa> 5084
30 2014-10-14 01:14:22 <lechuga_> thx
31 2014-10-14 01:16:10 <lechuga_> yikes
32 2014-10-14 01:17:41 <jgarzik> sipa, That's impossible, the code for building the lib is bitcoin.git
33 2014-10-14 01:18:04 <jgarzik> You cannot stick autoconf gunk into a JS project
34 2014-10-14 01:18:12 <BlueMatt> jgarzik: if the only user would be bitcoind.js, then bitcoind.js should maintain the patch
35 2014-10-14 01:18:15 <BlueMatt> not bitcoin core
36 2014-10-14 01:19:29 <gmaxwell> jgarzik: you can already see this butting against the interfaces, last hunk in the patch is taking the private SelectCoins function and making it public. Providing good encapsulated interfaces will likely involve moving in the other direction.
37 2014-10-14 01:19:57 <lechuga_> fwiw i doubt we'd use this in toshi
38 2014-10-14 01:20:22 <BlueMatt> jgarzik: more specifically, I'm still looking for someone to answer my question...what, specifically, are you looking to access from this?
39 2014-10-14 01:20:29 <BlueMatt> blocks you should be pretty well covered from the rpc
40 2014-10-14 01:20:34 <jgarzik> BlueMatt, false
41 2014-10-14 01:20:43 <jgarzik> BlueMatt, Read my github comment, please.
42 2014-10-14 01:20:45 <BlueMatt> ok, what is missing (and should it be added to the rpc?)
43 2014-10-14 01:20:52 <gmaxwell> the RPC is slow as shit, among other issues.
44 2014-10-14 01:21:13 <sipa> RPC is certainly not appropriate for every use
45 2014-10-14 01:21:20 <sipa> and i fully understand the use case
46 2014-10-14 01:21:40 <sipa> or rather, that there would be use for something like this
47 2014-10-14 01:21:52 <gmaxwell> In any case, certantly a good move to get people out of the reimplementation game. Just need to make sure that doing so won't give us something worse (e.g. being unable to actually finish the much needed code restructuring; or resulting in more criticial infrastructure stuck forever on old versions)
48 2014-10-14 01:22:19 <sipa> yes, that's what i'm worried about: people complaining that they are using this library, and need a ton of work to keep up
49 2014-10-14 01:22:37 <jgarzik> sipa, that's solved with a one line readme note
50 2014-10-14 01:22:54 <sipa> yup, and people don't download illegal mp3's either
51 2014-10-14 01:23:08 <gmaxwell> sipa: it already exists today, e.g. we know of popular services which are constantly going down because they're on old busted code and get stuck on reorgs...
52 2014-10-14 01:23:34 <sipa> yes, and having a stable C++ api that we can maintain would improve that a lot
53 2014-10-14 01:23:49 <jgarzik> a stable C++ api does not address this use case at all
54 2014-10-14 01:24:01 <gmaxwell> What we don't have today is people complaining that we can't refactor or move something internally (say to move things towards presenting a stable API) because it will break them.
55 2014-10-14 01:24:04 <jgarzik> that's what so many on github seem to be missing
56 2014-10-14 01:24:59 <cfields> jgarzik: i hastily nacked because i see no api defined, and the visibility control turned off, implying that the entire lib is exported. in that case, every code change is a potential abi breakage. Have I misread?
57 2014-10-14 01:25:29 <sipa> cfields: no; they do the encapsulation in a JS library that uses this
58 2014-10-14 01:25:47 <sipa> which will need to keep up with bitcoin core development as a result
59 2014-10-14 01:25:53 <sipa> jgarzik: correct me if i'm wrong there, please
60 2014-10-14 01:26:57 <cfields> regardless of how they handle it, is the above a correct assessment?
61 2014-10-14 01:27:15 <sipa> as far as i understand it: yes
62 2014-10-14 01:27:16 <lechuga_> seems about right
63 2014-10-14 01:28:08 <cfields> ok, thanks.
64 2014-10-14 01:28:09 <jgarzik> cfields, sipa: mostly
65 2014-10-14 01:28:40 <jgarzik> It is notable that bitcoind is actually run inside the process. It is an interface to a live, running bitcoind.
66 2014-10-14 01:28:52 <jgarzik> It is quite a bit different from a random lib.
67 2014-10-14 01:29:11 <jgarzik> thus the comparison to ZMQ
68 2014-10-14 01:29:18 <cfields> jgarzik: in that case, bitpay may only need a few functions, but there's no telling what some other user/company is depending on. So with that lib added, our external api must essentially freeze
69 2014-10-14 01:29:33 <jgarzik> cfields, bad assumption
70 2014-10-14 01:29:35 <cfields> the alternative being to define a slim set of api's, which we're working towards with other efforts
71 2014-10-14 01:29:44 <cfields> how so?
72 2014-10-14 01:29:54 <gmaxwell> I think a patch that ZMQ wrapped every function would get an equally concerned response.
73 2014-10-14 01:30:01 <BlueMatt> jgarzik: a bitcoin library that exposes "AddBlock(block, ValidationState)", "returns EVERYTHING GetInfoAboutConsensus" and maybe a pluggable db backend should satisfy your requirements, no?
74 2014-10-14 01:30:25 <sipa> BlueMatt: i doubt that
75 2014-10-14 01:30:38 <BlueMatt> sipa: what else is missing
76 2014-10-14 01:30:40 <BlueMatt> oh, and some callbacks
77 2014-10-14 01:30:41 <jgarzik> cfields, Because you just listed a whole bunch of unnecessary process
78 2014-10-14 01:31:00 <cfields> hmm? what did i list?
79 2014-10-14 01:31:23 <sipa> BlueMatt: it interfaces with the wallet
80 2014-10-14 01:31:35 <BlueMatt> sipa: oh, well thats just a bad idea period
81 2014-10-14 01:31:37 <jgarzik> cfields, "freeze external api", "define slim set of apis", alternative #3: don't do that.
82 2014-10-14 01:31:47 <jgarzik> in this context
83 2014-10-14 01:31:49 <gmaxwell> a significant amount of recent activity has been drastically changing internal interfaces in order to modularize things to actually be usable as a sensible library... every one of those changes will likely break any user of this library. I don't think anyone wants to send a message that any interface is remotely stable right now.
84 2014-10-14 01:31:51 <jgarzik> because it's not needed
85 2014-10-14 01:32:06 <jgarzik> gmaxwell, ...and nobody is doing so
86 2014-10-14 01:32:07 <cfields> jgarzik: nm -CD $(LIBBITCOIND)
87 2014-10-14 01:32:17 <sipa> running a full node + wallet inside a JS process seems just very dangerous from a security POW too
88 2014-10-14 01:32:19 <cfields> that's what's exposed, so that must be supported
89 2014-10-14 01:32:21 <sipa> *POV
90 2014-10-14 01:32:37 <sipa> i want a security POW, though!
91 2014-10-14 01:32:38 <Luke-Jr> cfields: it doesn't "must be" supported
92 2014-10-14 01:32:44 <jgarzik> cfields, what must be supported is what you say will be supported.
93 2014-10-14 01:32:46 <jgarzik> exactly.
94 2014-10-14 01:32:56 <Luke-Jr> cfields: if you want to change it every version, you can just bump the version every release
95 2014-10-14 01:33:00 <Luke-Jr> that's what libtool is for
96 2014-10-14 01:33:13 <cfields> jgarzik: "what you say is supported" is the definition of an api, and there's not one provided...
97 2014-10-14 01:33:15 <gmaxwell> jgarzik: so you're never going to NAK a refactoring change when it makes some internal infrastructure completely unavilable to this API? Why is it already making a private wallet function public?
98 2014-10-14 01:34:01 <jgarzik> gmaxwell, (1) no, (2) access coin control, AFAICS
99 2014-10-14 01:34:08 <Luke-Jr> cfields: libraries don't *need* to have APIs
100 2014-10-14 01:34:16 <Luke-Jr> and the API *can* change
101 2014-10-14 01:35:10 <jgarzik> Again, (1) one line release note "daemonlib API changes at a whim", (2) enforce release note with human actions. Not rocket science. Keep changing internals just like they planned to change yesterday.
102 2014-10-14 01:35:20 <jgarzik> If you keep changing the internals,
103 2014-10-14 01:35:26 <jgarzik> the natural result ensues regardless.
104 2014-10-14 01:35:36 <cfields> Luke-Jr: of course they do, otherwise you can't use them. as-implemented, the api is defined as "bitcoind's headers"
105 2014-10-14 01:35:49 <cfields> *as this is implemented
106 2014-10-14 01:36:00 <Luke-Jr> cfields: you just gave two definitions of API, which aren't necessarily equivalent
107 2014-10-14 01:36:36 <Luke-Jr> cfields: even if your library-using code is tied to a single version, you're still able to use it
108 2014-10-14 01:36:44 <gmaxwell> jgarzik: yea... so I follow why there would be a desire to use that externally... , but in the current codebase that function really should not be exposed outside of the class. So as the patch is it's already screwing up internal interfaces. (I admit, in pretty harmless way) But yea, I don't see a reason that it couldn't be included with at "this isn't supported, and you're crazy if you use it and don't expect to commit significant ...
109 2014-10-14 01:36:50 <gmaxwell> ... work to maintaining it at this time".
110 2014-10-14 01:37:04 <jgarzik> If you embed bitcoind and something breaks, you get to keep both pieces.
111 2014-10-14 01:37:19 <Luke-Jr> (FWIW, I don't think we should do libbitcoind simply because it's duplicated effort, but if someone actually wants to use it, it's harmless)
112 2014-10-14 01:37:55 <jgarzik> gmaxwell, I'm fine with changing SelectCoins back FWIW
113 2014-10-14 01:38:00 <jgarzik> to me that's a small detail
114 2014-10-14 01:38:03 <cfields> ok, maybe i'm misunderstanding...
115 2014-10-14 01:38:19 <sipa> jgarzik: so you wouldn't NAK removing the accounts code either?
116 2014-10-14 01:38:25 <cfields> is what's available in bitcoind.h meant to be all that's used?
117 2014-10-14 01:38:25 <jgarzik> sipa, no
118 2014-10-14 01:38:46 <jgarzik> sipa, keep making all changes that were planned, as if you had not seen this PR
119 2014-10-14 01:38:59 <jgarzik> sipa, break stuff
120 2014-10-14 01:39:13 <cfields> jgarzik: ^^ ?
121 2014-10-14 01:39:20 <sipa> if we should behave as if the code introduced by this PR is not there, why is it in our repo?
122 2014-10-14 01:39:34 <Luke-Jr> sipa: so people don't need to patch it to use it as a library..
123 2014-10-14 01:40:05 <jgarzik> Sigh, it's a metaphor. The general principle is: it's an internal API, keep changing it, do not concern with breaking libbitcoind.so users.
124 2014-10-14 01:40:31 <jgarzik> A one line release note is sufficient
125 2014-10-14 01:40:53 <sipa> and not building it by default, and not including it in releases... ok
126 2014-10-14 01:41:15 <gmaxwell> sounds fine to me, just don't want it getting in the way of an actually sustainable solution. :)
127 2014-10-14 01:41:15 <sipa> i'd still much more prefer to see the functions provided by bitcoind.js in a libbitcoind.so
128 2014-10-14 01:41:20 <sipa> seems trivial...
129 2014-10-14 01:41:46 <lechuga_> seems like a hack
130 2014-10-14 01:41:57 <gmaxwell> (kinda sad if it means people are building functionality in external, non-integratable, propritary systems that could equally be done internally)
131 2014-10-14 01:42:10 <jgarzik> IMO it is a fair question "why in the repo?"
132 2014-10-14 01:42:21 <sipa> that would be equally useful, but reusable and maintainable
133 2014-10-14 01:42:31 <gmaxwell> lechuga_: sure, but so is a reimplementation which is virtually guarenteed to be consensus-inconsistent. This is a hack from a maintance perspective instead of a network safty one.
134 2014-10-14 01:42:35 <jgarzik> alongside #4657, it gets integrated with other open source projects like Insight, creating sort of a bitcoin supernode
135 2014-10-14 01:42:43 <Luke-Jr> gmaxwell: I think BitPay just wants a quick hack to use until the better solution is available/done
136 2014-10-14 01:42:47 <BlueMatt> jgarzik: yes, and in that case, just put the build patch in the libbitcoind.js repo
137 2014-10-14 01:42:58 <BlueMatt> there is absolutely no reason to have it in bitcoin.git if thats your approach
138 2014-10-14 01:43:00 <jgarzik> Insight-like apps benenfit from direct integration like that
139 2014-10-14 01:43:15 <sipa> yeah, if the alternative is people reimplementing things... this seems definitely preferable
140 2014-10-14 01:43:17 <lechuga_> gmaxwell: nod, but i'm in favor of things like libbitcoinscript to address that problem
141 2014-10-14 01:43:20 <jgarzik> Basically it's a default-off "embed bitcoind" switch.
142 2014-10-14 01:43:21 <BlueMatt> yes, and if you want to integrate internal apis, and are prepared for things to break, then integrate patches too
143 2014-10-14 01:43:22 <gmaxwell> jgarzik: if we've done the interfaces right we should be able to accomidate them ... uh, without having no interface. :)
144 2014-10-14 01:43:23 <BlueMatt> its the same thing
145 2014-10-14 01:43:28 <sipa> but really... why is the abstraction at the JS level?
146 2014-10-14 01:43:42 <phantomcircuit> jgarzik, you're basically trying to run a bitcoind node in the same process space?
147 2014-10-14 01:43:49 <phantomcircuit> minus the wallet and stuff
148 2014-10-14 01:43:58 <BlueMatt> sipa: that is a false dychotomy
149 2014-10-14 01:43:59 <jgarzik> phantomcircuit, process family
150 2014-10-14 01:44:03 <jgarzik> yes
151 2014-10-14 01:44:13 <BlueMatt> sipa: the alternative is libbitcoinconsensus and similar libraries
152 2014-10-14 01:44:13 <sipa> phantomcircuit: no, WITH the wallet
153 2014-10-14 01:44:16 <gmaxwell> lechuga_: Oh I certantly agree. Jeff is arguing that its in no way exclusive. As you can see, libbitcoinscript is taking a bit of time, and only has a couple people seriously working on it though...
154 2014-10-14 01:44:39 <lechuga_> so this is intended as a temporary stop-gap?
155 2014-10-14 01:44:40 <phantomcircuit> sipa, with? jgarzik why with the wallet
156 2014-10-14 01:44:43 <jgarzik> sipa, phantomcircuit: with or without wallet. Honestly I think the wallet was included by JJ as a demo.
157 2014-10-14 01:44:46 <lechuga_> if so y not just maintain a fork until then
158 2014-10-14 01:44:47 <BlueMatt> the problem is, with something like this where bitpay is working on it, its gonna be the biggest use of this crap
159 2014-10-14 01:44:47 <jgarzik> ^
160 2014-10-14 01:44:48 <cfields> i plan on having the rest of the PR's for that up this week
161 2014-10-14 01:44:50 <phantomcircuit> ah
162 2014-10-14 01:44:53 <phantomcircuit> fair enough
163 2014-10-14 01:44:59 <sipa> jgarzik: ok, good
164 2014-10-14 01:45:02 <jgarzik> if wallet is a blocker, I'm OK with removing
165 2014-10-14 01:45:07 <gmaxwell> lechuga_: perhaps. One less set of patches to constantly get broken though.
166 2014-10-14 01:45:43 <gmaxwell> jgarzik: I don't consider it a blocker; I'm just pointing out that the patch so far was evidence that it wouldn't have no-effect.
167 2014-10-14 01:45:43 <phantomcircuit> jgarzik, can you explain what the advantage is over simply running a bitcoind as a child process and connecting using p2p protocol ?
168 2014-10-14 01:45:46 <cfields> er.. doesn't that constant breakage imply that an outside app would have trouble keeping up with its abi?
169 2014-10-14 01:45:58 <BlueMatt> jgarzik: if bitpay is so serious about maintaining this for their bitcoind.js library, then they should maintain a fork, specifically to discourage people from using this in other projects due to the unstable api issues
170 2014-10-14 01:46:06 <gmaxwell> bleh
171 2014-10-14 01:47:58 <phantomcircuit> gmaxwell, btw as for opencl secp256k1, i doubt the latency will be a sane tradeoff until nearly all blocks are full
172 2014-10-14 01:48:20 <sipa> what latency are we talking about?
173 2014-10-14 01:48:21 <gmaxwell> phantomcircuit: I already told the person that... might still be interesting in IBD.
174 2014-10-14 01:48:31 <gmaxwell> distpatch latency to the gpu, it's pretty awful.
175 2014-10-14 01:48:32 <jgarzik> gmaxwell, sipa: Time and language intersect. BitPay, Blockchain and several other shops are JS. libbitcoin{script,...} are not moving anywhere near the necessary speed to embed consensus compatible code as a complete solution "soon", and they love to hire JS devs and build a JS universe, with the predictable result on the C++ side
176 2014-10-14 01:48:58 <jgarzik> I say this being _not_ a JS fan, in a JS shop ;p
177 2014-10-14 01:49:25 <BlueMatt> jgarzik: the other point, is that with /only/ script in libbitcoinconsensus, we are already 95% of where we need to be wrt people reimplementing
178 2014-10-14 01:49:26 <jgarzik> JS works on browser + multiple mobile platforms
179 2014-10-14 01:49:36 <BlueMatt> by far the majority of the issues we've seen are in script, not other block handling things
180 2014-10-14 01:49:37 <sipa> jgarzik: matt had a working libbitcoinscript is no time; all work being done is just to reduce the dependencies and code size
181 2014-10-14 01:49:44 <gmaxwell> so some of these parties have already done instrumentation into bitcoind forks for their business logic and as a result are not upgrading their nodes.
182 2014-10-14 01:49:50 <BlueMatt> and, yea, libbitcoinscript was a few hours
183 2014-10-14 01:49:53 <cfields> and portability
184 2014-10-14 01:49:56 <gmaxwell> sipa: libbitcoinscript is not bitcoin-node-in-a-box
185 2014-10-14 01:50:16 <kanzure> they have forks and they aren't maintaining their forks? wow
186 2014-10-14 01:50:23 <midnightmagic> awesome
187 2014-10-14 01:50:23 <sipa> gmaxwell: and bitcoin-node-in-a-box is imho even easier, as there is absolutely no requirement that it has few dependencies
188 2014-10-14 01:50:35 <sipa> jgarzik: a lib that does exactly what bitcoind.js does seems trivial
189 2014-10-14 01:50:43 <Luke-Jr> kanzure: AFAIK they haven't even merged in the bugfix branches I [used to?] maintain
190 2014-10-14 01:50:49 <BlueMatt> gmaxwell/jgarzik: and I'm really not sure we want people to use anything more than the consensus of bitcoind
191 2014-10-14 01:51:02 <sipa> query blocks, query the mempool, query the utxo set
192 2014-10-14 01:51:03 <BlueMatt> the wallet is not so great...
193 2014-10-14 01:51:11 <BlueMatt> yea
194 2014-10-14 01:51:23 <jgarzik> sipa, it does far more than query
195 2014-10-14 01:51:31 <sipa> yes, i'm looking at the api
196 2014-10-14 01:51:59 <gmaxwell> To the extent that this change just enables parties already doing unwise, unmaintainable things (instead of putting in resources to help get the interfaces they need exposed), to do them in a less one-off manner thats probably good. If it expands the practice thats probably bad. And of course nothing is in a bubble, so "you get to keep the pieces" is good, but only goes so far.
197 2014-10-14 01:52:29 <gmaxwell> Though really we do not want to create a public api for the system internal parts under commercial time pressure.
198 2014-10-14 01:52:49 <BlueMatt> this rieks of such a bad idea
199 2014-10-14 01:53:02 <BlueMatt> bitpay needs something, writes up a dirty hack, and pushes it in bitcoin core
200 2014-10-14 01:53:13 <gmaxwell> (or with primarily-JS developers making it their my-first-C++/systems-language-project)
201 2014-10-14 01:53:20 <jgarzik> BlueMatt, what precisely about the JS API is dirty?
202 2014-10-14 01:53:29 <BlueMatt> not bitpay needs something, writes up a hack and moves towards a more sustainable solution, merging some fork into bitcoin core slowly with proper review
203 2014-10-14 01:53:35 <BlueMatt> jgarzik: the js part isnt the dirty part
204 2014-10-14 01:53:41 <BlueMatt> its the actual library
205 2014-10-14 01:53:49 <gmaxwell> jgarzik: exposing the whole of bitcoind's internals to things outside of the project is a pretty dirty hack, come on. :)
206 2014-10-14 01:54:04 <jgarzik> BlueMatt, so then it is rather like SQL DB drivers, which present a useful interface, but under the hood, speak internal protocols and APIs...
207 2014-10-14 01:54:15 <gmaxwell> If instead it was some SWIG binding script or whatever that was submitted it would get an even more negative response.
208 2014-10-14 01:54:29 <BlueMatt> jgarzik: yes, exactly, but in those cases those things are built by the same project
209 2014-10-14 01:54:35 <BlueMatt> if bitcoind.js were a part of bitcoin core, ok, fine
210 2014-10-14 01:54:37 <BlueMatt> but its not, and wont be
211 2014-10-14 01:54:46 <jgarzik> lovely roadblock logic, that
212 2014-10-14 01:54:51 <gmaxwell> SQL databases have on the wire formats that are much more stable than bitcoind's internals right now.
213 2014-10-14 01:54:53 <sipa> or a libbitcoind.so that does exactly what bitcoind.js does now...
214 2014-10-14 01:54:56 <BlueMatt> not at all, there is an alternate way forward
215 2014-10-14 01:55:02 <BlueMatt> that is much cleaner, and is already being worked on
216 2014-10-14 01:55:13 <gmaxwell> sipa: what it does now isn't the concern. Jeff explained above.
217 2014-10-14 01:55:18 <BlueMatt> if bitpay needs something so badly, it should devote some resources to making it happen, not in rewriting it in some hacky way
218 2014-10-14 01:55:27 <moa> very enlightening discussion
219 2014-10-14 01:55:32 <gmaxwell> Users who are JS shops filled with JS coders will want to extend bitcoind.js.
220 2014-10-14 01:55:43 <jgarzik> We devoted resources to making a solution and we made a solution.
221 2014-10-14 01:55:57 <jgarzik> Not some blue sky maybe-need for a guessed-for future.
222 2014-10-14 01:56:06 <BlueMatt> no, you made a hack in bitcoin core
223 2014-10-14 01:56:27 <kanzure> oh, you want node bindings?
224 2014-10-14 01:56:29 <BlueMatt> and a solution in some private thing
225 2014-10-14 01:56:33 <gmaxwell> "commercial time puressure" is the polite presentation. :)
226 2014-10-14 01:56:38 <kanzure> i mean nodejs bindings
227 2014-10-14 01:56:47 <BlueMatt> ACTION -> out
228 2014-10-14 01:57:17 <crescendo> Bitcoind.js is not private; it is public: https://github.com/bitpay/bitcoind.js
229 2014-10-14 01:58:42 <jgarzik> gmaxwell, Slightly disagree -- it's relative. No expected guarantee of any PR getting in in any sort of deadline. But waiting for stable C++ API (a) doesn't really address the solution sought, and (b) seems a common Wait For The Imaginary Monkey Coming In Version 2.0 pattern often found in open source projects.
230 2014-10-14 01:59:10 <jgarzik> so there's no deadline, but they are separate efforts I think
231 2014-10-14 01:59:20 <jgarzik> and move at separate pace
232 2014-10-14 01:59:48 <gmaxwell> jgarzik: not sure if you've been following the codebase, but there is really significant work on the modulization parts; you've mostly not seemed to comment on any of the design. So (b) is overstated, but a reasonable concern.
233 2014-10-14 01:59:56 <gmaxwell> I agree these are seperate issues.
234 2014-10-14 02:00:42 <crescendo> If the concern is maintainability through future updates, then the feedback should be centered around making the existing body of work more maintainable, imho â the two bodies of work solve distinct problems, and as such I see no clear reason why they shouldn't continue independently.
235 2014-10-14 02:01:30 <jgarzik> BlueMatt, Indeed this is a not a "private thing", everything is open source, proposed in parallel
236 2014-10-14 02:02:56 <jgarzik> Why upstream this, versus maintain a patch?
237 2014-10-14 02:03:05 <kanzure> i wonder how much worse hammering v8's event loop into boost::thread_group would be
238 2014-10-14 02:03:13 <gmaxwell> crescendo: The approach in this pull is completely unmaintainable (though we won't accept it unless it's clear that bitcoin core will make no effort to maintain it). There is already large amounts of work underway to offer stable library interfaces with probably hundreds of commits by coryfields, BlueMatt, jtimon, and others in the last couple months. Part of the negative reaction, I think, is people who are concerned that exposing the ...
239 2014-10-14 02:03:13 <kanzure> ACTION ducks
240 2014-10-14 02:03:14 <lechuga_> bitcoindjs.cc is pretty cool effort tho
241 2014-10-14 02:03:19 <gmaxwell> ... existing interfaces will create pressure against the much needed work underway.
242 2014-10-14 02:05:38 <jgarzik> (1) Direct bitcoind bindings like this will always need a set of features like this, no matter the "stable C++ API" This is an API to a running daemon, not abstract execution functions. (2) Ideally the binding will not be short lived, so a long lived patch in a popular open source project (Insight, other JS users) should just upstream the patch.
243 2014-10-14 02:06:47 <jgarzik> Embedding-bitcoind is the feature, and that's not a closed source or BitPay-only thing.
244 2014-10-14 02:07:40 <gmaxwell> jgarzik: I think the comment on the closed thing, is that it's embedding bitcoin in order to make closed extensions to it. That might not actually be the goal there, but its easy to see why someone might assume it that way.
245 2014-10-14 02:08:45 <jgarzik> Yes, making the internals available is ugly, but that's not expected to be a stable or supported API for external consumption. --enable-daemonlib can print out a configure warning or somesuch, about non-stable API, if people think warranted.
246 2014-10-14 02:09:43 <Luke-Jr> gmaxwell: they can already make closed extensions, it's MIT license..
247 2014-10-14 02:09:55 <jgarzik> gmaxwell, Never thought about it that way, honestly, since we open source all that stuff
248 2014-10-14 02:10:23 <jgarzik> our backend payment engine is the only closed source component afaik
249 2014-10-14 02:11:16 <jgarzik> This, bitcore, insight, copay, all OSS.
250 2014-10-14 02:11:26 <Luke-Jr> gmaxwell: arguably, mainlining the lib hack makes it more accessible for open extensions on it
251 2014-10-14 02:11:29 <gmaxwell> Yea, I was presenting the potential concern there as I saw it... not trying to argue it myself. For better or worse bitcoind is mit licensed anyways.
252 2014-10-14 02:12:26 <crescendo> gmaxwell: understandable on the maintenance point. perhaps we can come with a far better way of exposing these calls, in a low or even zero-maintenance method?
253 2014-10-14 02:12:28 <gmaxwell> Luke-Jr: Kinda prefer they not be open. lol... really do not want to deal with anyone opposing a really useful internal interface change simply because it breaks some popular app which is used by people who haven't made the required maintance commitments. :)
254 2014-10-14 02:13:01 <Luke-Jr> gmaxwell: sure, just pointing out the proprietary argument doesn't make sense :p
255 2014-10-14 02:13:28 <gmaxwell> crescendo: There is an ongoing project to break the software into a set of libraries which will eventually offer a stable interface to the functionality.
256 2014-10-14 02:14:28 <jgarzik> OK, I really need to be doing Real Work instead of getting myself distracted ;p
257 2014-10-14 02:14:42 <jgarzik> In general, I did not think any of the early respondents even understood what the PR did
258 2014-10-14 02:14:52 <jgarzik> that seems to be corrected
259 2014-10-14 02:15:16 <jgarzik> I think fair points that remain are: maintenance trickle down, and why not distribute a patch
260 2014-10-14 02:15:18 <gmaxwell> Ultimately I hope that covers it. But absent a customer the priority order has been around the other way... mostly starting at script and growing out. E.g. libscript -> libbitcoinconsensus -> ... which means that it will be some time (and probably a complete wallet rewrite) before it would likely be a replacement for this.
261 2014-10-14 02:15:38 <jgarzik> *poof*
262 2014-10-14 02:16:35 <gmaxwell> jgarzik: thanks for hanging around and clarifying some things.
263 2014-10-14 02:16:38 <Luke-Jr> looking at the PR, I would expect much smaller diff for simply libifying bitcoind
264 2014-10-14 02:17:18 <gmaxwell> I think it's larger than it needs to be, seems to have a bunch of surprising build system interaction.
265 2014-10-14 02:17:41 <cfields> Luke-Jr: looks like someone just got lost in the autotools maze. it simplifies down to very little.
266 2014-10-14 02:18:32 <gmaxwell> cfields: that was my read on it too.
267 2014-10-14 02:22:54 <crescendo> gmaxwell: I'm aware; what would an alternative be to having one of our engineers dedicated to adding the functionality we needed? Trying to understand how we can better make shared improvements available for comment.
268 2014-10-14 02:24:46 <sipa> crescendo: would a .cpp file that does pretty much what bitcoind.js does at the C++ level be possible?
269 2014-10-14 02:25:13 <sipa> what can be maintained in-repository, wouldn't break compatibility when things are refactored, and would be more generally useful
270 2014-10-14 02:25:33 <cfields> er, can't linux progs be tricked into acting as shared libs with -rdynamic
271 2014-10-14 02:25:34 <cfields> ?
272 2014-10-14 02:26:00 <sipa> ha!
273 2014-10-14 02:26:01 <cfields> if so, i'd just do that and call it a day
274 2014-10-14 02:26:06 <sipa> never knew about that
275 2014-10-14 02:26:10 <cfields> (not joking, btw)
276 2014-10-14 02:26:17 <sipa> yes, i just looked it up
277 2014-10-14 02:27:13 <Luke-Jr> cfields: -rdynamic to the linker? O.o
278 2014-10-14 02:27:38 <Luke-Jr> oh, cute
279 2014-10-14 02:27:48 <Luke-Jr> ./configure LDFLAGS='-rdynamic'
280 2014-10-14 02:27:50 <Luke-Jr> problem solved :P
281 2014-10-14 02:28:07 <cfields> it used to work, then there was some debate, then drepper got involved iirc. Not sure what the result was.
282 2014-10-14 02:28:21 <cfields> memory is very hazy though. that could've been wrt dlopen().
283 2014-10-14 02:28:37 <crescendo> yes, dynamic would be appropriate in this context and likely would reduce the maintenance overhead â maybe we can move towards that if we can determine how to make it work (again)
284 2014-10-14 02:29:02 <cfields> there would be no overhead (on our side). and Luke-Jr just told you how to make it work :)
285 2014-10-14 02:29:04 <Luke-Jr> ./configure LDFLAGS='-rdynamic' && make BITCOIND_BIN=$PWD/libbitcoind.so
286 2014-10-14 02:29:07 <cfields> or try it, anyway
287 2014-10-14 02:29:31 <Luke-Jr> maybe even
288 2014-10-14 02:29:35 <Luke-Jr> ./configure LDFLAGS='-shared -rdynamic' && make BITCOIND_BIN=$PWD/libbitcoind.so
289 2014-10-14 02:30:19 <cfields> mm, libtool will eat those. it'd probably take a bit of hackery
290 2014-10-14 02:30:28 <moa> might need LT_INIT([dlopen]) instead of LT_INIT([disable-shared]) in configure.ac?
291 2014-10-14 02:31:03 <cfields> heh, need to see if it actually works before making any changes
292 2014-10-14 02:33:22 <cfields> heh, works :)
293 2014-10-14 02:33:53 <crescendo> alright; @Luke-Jr â can you add that to the PR, and link to any relevant (potentially old) code that would be helpful to our dev?
294 2014-10-14 02:34:13 <Luke-Jr> crescendo: I didn't actually test it..
295 2014-10-14 02:34:19 <Luke-Jr> I guess I can easily enough
296 2014-10-14 02:34:58 <crescendo> we can do the testing, if you just help guide us in the right direction :)
297 2014-10-14 02:35:58 <cfields> crescendo: http://pastebin.com/raw.php?i=zya3hdyh
298 2014-10-14 02:36:08 <cfields> that's the whole shebang
299 2014-10-14 02:36:39 <cfields> er.. reverse those build commands. stupid c/p.
300 2014-10-14 02:37:48 <cfields> and ignore the contents of main() in the first file, that's part of whatever crud i had around in test.c at the moment
301 2014-10-14 02:38:59 <cfields> there: http://pastebin.com/raw.php?i=g5TmTCps
302 2014-10-14 02:40:02 <crescendo> great, thanks â we'll take a look at this and test with bitcoind.js
303 2014-10-14 02:41:22 <cfields> rdynamic doesn't seem to be needed if symbols are exported properly. And you'll need to watch out for certain build options, some of hardening for example you might want disabled
304 2014-10-14 02:50:49 <luke-jr_> dyn.cppâ¹.text+0x35): undefined reference to `AppInit(int, charâ)'
305 2014-10-14 02:50:51 <luke-jr_> not sure why
306 2014-10-14 02:53:32 <coryfields_> er, did freenode just have a stroke? or was that me?
307 2014-10-14 02:54:18 <moa> freenode
308 2014-10-14 02:54:22 <crescendo> freenode netsplit.
309 2014-10-14 02:57:01 <luke-jr_> dyn.cpp:(.text+0x35): undefined reference to `AppInit(int, char**)'
310 2014-10-14 02:57:04 <luke-jr_> not sure why
311 2014-10-14 02:58:15 <coryfields_> luke-jr_: c++ linkage
312 2014-10-14 02:59:01 <coryfields_> (assuming you used --enable-reduce-exports as well)
313 2014-10-14 02:59:07 <coryfields_> er, disable
314 2014-10-14 02:59:18 <luke-jr_> coryfields_: nope, but I saw it in nm
315 2014-10-14 02:59:46 <coryfields_> Luke-Jr: nm -CD ?
316 2014-10-14 03:00:40 <coryfields_> doesn't help to list all globals, that'll show hidden stuff as well until they're stripped
317 2014-10-14 03:01:51 <coryfields_> Luke-Jr: also, you'll need to mind whether you link with gcc or g++, as they imply different external linkages
318 2014-10-14 03:01:59 <Luke-Jr> I was using g++
319 2014-10-14 03:02:11 <coryfields_> for both?
320 2014-10-14 03:02:31 <Luke-Jr> both what?
321 2014-10-14 03:02:44 <coryfields_> er.. are those your ski's? both of em?
322 2014-10-14 03:02:55 <Luke-Jr> O.o?
323 2014-10-14 03:02:58 <coryfields_> i realize that was a dumb question now, you couldn't have built bitcoind with gcc :)
324 2014-10-14 03:03:06 <Luke-Jr> :p
325 2014-10-14 03:04:29 <coryfields_> Luke-Jr: there's no way it will link if you didn't build with --disable-reduce-exports, though
326 2014-10-14 03:04:58 <Luke-Jr> trying that now
327 2014-10-14 03:05:41 <coryfields_> see the note about hardening as well. I'm not sure what effect building a PIE binary has when trying to use it as a shared lib
328 2014-10-14 03:06:10 <Luke-Jr> PIE = PIC
329 2014-10-14 03:06:23 <Luke-Jr> just PIE isn't usually a lib
330 2014-10-14 03:06:40 <coryfields_> no, PIE is always an executable :)
331 2014-10-14 03:07:27 <moa> might not work unless the linked deps are built with the same PIC flag
332 2014-10-14 03:08:18 <coryfields_> has no effect on my test lib though, so maybe not a problem
333 2014-10-14 03:11:00 <Luke-Jr> ./configure LDFLAGS='-rdynamic' --disable-reduce-exports --without-utils --without-gui --disable-tests
334 2014-10-14 03:11:03 <Luke-Jr> seems to mostly work
335 2014-10-14 03:11:10 <Luke-Jr> http://codepad.org/VIurrwBe
336 2014-10-14 03:11:36 <Luke-Jr> crash at exit though (double free)
337 2014-10-14 03:15:10 <coryfields_> Luke-Jr: that'll likely be the atexit() (and boost's versions of those) not working correctly
338 2014-10-14 03:15:31 <coryfields_> another reason against the shared lib approach, btw
339 2014-10-14 03:15:41 <coryfields_> those will probably need to be manually called
340 2014-10-14 03:15:48 <Luke-Jr> why?
341 2014-10-14 03:16:13 <Luke-Jr> I mean, why are we using atexit instead of a destructor?
342 2014-10-14 03:16:34 <Luke-Jr> also, the manpage says atexit is fine in libs
343 2014-10-14 03:16:47 <coryfields_> static destruct order. gcc has hooks into its teardown, presumably boost uses em
344 2014-10-14 03:18:10 <coryfields_> Luke-Jr: sorry, i didn't mean atexit() in particular (though it may be a problem as well). I meant the internal teardown hooks.
345 2014-10-14 03:18:25 <Luke-Jr> i c
346 2014-10-14 03:18:34 <Luke-Jr> dunno why the order would change from being a lib *shrug*
347 2014-10-14 03:18:35 <coryfields_> sec, i'll find gcc's
348 2014-10-14 03:19:41 <Luke-Jr> we're probably spending more time than it's worth at this point, especially since crescendo inferred they have paid staff to work on it in BitPay (and AFAIK nobody else plans to use it)
349 2014-10-14 03:21:53 <coryfields_> indeed
350 2014-10-14 03:24:53 <coryfields_> Luke-Jr: iirc you can essentially re-implement __cxa_atexit or so, but i'm not remembering where i've done that before
351 2014-10-14 03:25:04 <coryfields_> maybe an android dyloader, or a jni trick
352 2014-10-14 03:25:05 <Luke-Jr> that sounds awful
353 2014-10-14 03:25:29 <coryfields_> well, boost does something similar i believe
354 2014-10-14 03:25:57 <coryfields_> anyway, my point was that if tricks like that are being used, they're likely not playing nice with running from a shared lib
355 2014-10-14 03:26:15 <coryfields_> so, i'd suspect that bitpay was probably seeing that anyway
356 2014-10-14 03:45:09 <crescendo> in theory, we'll see competing open source node.js libraries competing with ours... or perhaps other bindings for other toolchains. the point is to make sure we open source such changes, and work to make them useful for everyone.
357 2014-10-14 03:48:54 <coryfields_> crescendo: so why not work to create a set of bindings in core, with a real api?
358 2014-10-14 03:49:26 <kanzure_> doesn't boost do bindings somewhere?
359 2014-10-14 03:50:25 <coryfields_> i'd go on a rampage if we pulled in _more_ boost :)
360 2014-10-14 03:50:37 <kanzure_> fair enough
361 2014-10-14 04:14:52 <Luke-Jr> coryfields_: meh, it'd be fine if it's an optional requirement for bindings (assuming it doesn't require designing symbols specifically for it)
362 2014-10-14 04:18:56 <coryfields_> optional requirement?
363 2014-10-14 04:21:39 <Luke-Jr> sipa_: btw, that stackexchange you linked me like a week ago - is 9 months old (3.8 when current is 4.9)
364 2014-10-14 04:21:56 <Luke-Jr> coryfields_: well, obviously we need boost anyway, but it's not like using it for bindings makes it worse
365 2014-10-14 04:23:30 <coryfields_> Luke-Jr: boost is removed from libbitcoinconsensus, and i'd hope to do the same for anything else we expose
366 2014-10-14 04:23:47 <Luke-Jr> coryfields_: does boost require changing libbitcoinconsensus to get bindings?
367 2014-10-14 04:24:47 <coryfields_> well i assume it'd have to be used on both sides...
368 2014-10-14 04:25:16 <coryfields_> i dunno why i'm arguing such an abstract topic though, it was just an offhand comment :)
369 2014-10-14 04:35:32 <Luke-Jr> coryfields_: a sane binding-generator wouldn't require changing the source code :p
370 2014-10-14 04:36:08 <Luke-Jr> err, origin code? "source code" was two separate words there
371 2014-10-14 04:37:06 <coryfields_> Luke-Jr: sure, but it may be necessary at client runtime
372 2014-10-14 04:37:16 <coryfields_> see genproto + libprotobuf
373 2014-10-14 06:39:05 <coryfields_> BlueMatt: ping. i ended up digging into some stupid libtool stuff today and burning some time. dll's and so's build now and pass all tests. should have it all pushed up tomorrow no problem.
374 2014-10-14 06:39:18 <BlueMatt> nice!
375 2014-10-14 06:39:28 <BlueMatt> no rush...you saw sipa's ml post on bip62?
376 2014-10-14 06:39:31 <coryfields_> sorry it's not up today as i intended
377 2014-10-14 06:39:37 <BlueMatt> (ie it needs extended...again...)
378 2014-10-14 06:39:51 <coryfields_> yea. i responded here, not sure if it was dropped in the freenode split or not
379 2014-10-14 06:40:14 <BlueMatt> I stopped reading everything between the bitpay stuff blowing up, me leaving and the splits
380 2014-10-14 06:40:31 <BlueMatt> I think everyone should go home and read script.cpp and try to find more malleability outside of bip62 this week
381 2014-10-14 06:40:49 <coryfields_> i noticed the bool malleability when reviewing yesterday as well, and iirc there were a few others
382 2014-10-14 06:40:58 <coryfields_> but i wanted to test before bringing it up
383 2014-10-14 06:41:15 <coryfields_> so i'd like to spend some time going through it again
384 2014-10-14 06:41:24 <BlueMatt> better to bring them up and make everyone think about them vs ignoring them...that kind of thinking often leads to other related issues :)
385 2014-10-14 06:41:36 <BlueMatt> or going off and thinking alone
386 2014-10-14 06:42:18 <gmaxwell> The bool thing isn't something that hits standard transactions.. (though more reason to do checklocktimeverify...)
387 2014-10-14 06:43:47 <BlueMatt> indeed its not a huge deal
388 2014-10-14 06:43:56 <BlueMatt> its more of a well, while we're at it with bip62.....
389 2014-10-14 06:57:18 <wumpus> <Luke-Jr> coryfields_: a sane binding-generator wouldn't require changing the source code :p <- agreed; if you make bindings that are accessible from C, making specific language bindings is a piece of cake anyway
390 2014-10-14 06:57:52 <wumpus> but the specific bindings are out of scope for bitcoin core, and belong in the appropriate language repositories
391 2014-10-14 06:58:19 <coryfields_> wumpus: tbh i have no clue what that discussion was about. I think my kneejerk is to argue with Luke-Jr and see where it leads :)
392 2014-10-14 06:58:31 <Luke-Jr> lol
393 2014-10-14 07:00:22 <wumpus> sometimes even differents kinds of bindings are possible per language; for example in Python it is currently preferred to make bindings in python w/ ctypes instead of boost::python, because that works with pypy as well
394 2014-10-14 07:00:31 <wumpus> so we shouldn't concern ourselves there
395 2014-10-14 07:00:33 <coryfields_> wumpus: i do agree with you though. C bindings are a good and realistic goal for anything we expose. client libs can wrap them however they like
396 2014-10-14 07:01:03 <wumpus> coryfields_: +1
397 2014-10-14 07:01:04 <Luke-Jr> +2 for C bindings
398 2014-10-14 07:01:22 <BlueMatt> +9k for C bindings
399 2014-10-14 07:01:29 <BlueMatt> ITS NOW OVER 9000!!!
400 2014-10-14 07:01:47 <Luke-Jr> k, BlueMatt gets to implement
401 2014-10-14 07:01:49 <Luke-Jr> :P
402 2014-10-14 07:01:52 <BlueMatt> shit
403 2014-10-14 07:02:25 <coryfields_> BlueMatt: btw, you don't have your libconsensus stuff deployed anywhere, do you?
404 2014-10-14 07:02:30 <BlueMatt> no
405 2014-10-14 07:02:42 <BlueMatt> I wrote a patch to make bitcoinj use it, but it was like 20 minutes of work
406 2014-10-14 07:02:44 <BlueMatt> why?
407 2014-10-14 07:02:50 <coryfields_> ok
408 2014-10-14 07:03:02 <BlueMatt> I was gonna wait till you pushed something and then write a quick commit to tweak the api a tiny bit
409 2014-10-14 07:03:05 <BlueMatt> unless you want me to do that first
410 2014-10-14 07:03:10 <coryfields_> there's a tiny bug that caused a few failed tests, but nothing significant
411 2014-10-14 07:03:15 <BlueMatt> oh, psh
412 2014-10-14 07:03:22 <coryfields_> yep
413 2014-10-14 07:03:33 <BlueMatt> tests are boring
414 2014-10-14 07:03:44 <Luke-Jr> ACTION hopes it's at least libbcconsensus. libconsensus is too vague/ambiguous.
415 2014-10-14 07:03:54 <BlueMatt> its libbitcoinconsensus
416 2014-10-14 07:04:08 <Luke-Jr> lbc
417 2014-10-14 07:04:09 <wumpus> libconsensusexceptforlukejr :)
418 2014-10-14 07:04:12 <Luke-Jr> almost looks like lhc
419 2014-10-14 07:04:18 <BlueMatt> YES
420 2014-10-14 07:04:28 <BlueMatt> 0.10 must release with LHC mode, just so that sipa is happy
421 2014-10-14 07:04:42 <coryfields_> Luke-Jr: disagreeing on libconsensus is either brilliant irony, or the ultimate troll.
422 2014-10-14 07:04:49 <wumpus> LHC mode does sound bad-ass
423 2014-10-14 07:04:56 <Luke-Jr> coryfields_: who's disagreeing?
424 2014-10-14 07:05:13 <coryfields_> heh, nm
425 2014-10-14 07:05:18 <BlueMatt> should we rename headers-first something as buzzword-compatible as HD wallets, before its too late?
426 2014-10-14 07:05:20 <Luke-Jr> (yes, I got it)
427 2014-10-14 07:05:23 <wumpus> coryfields_: hehe
428 2014-10-14 07:05:27 <gmaxwell> libcoinsync
429 2014-10-14 07:05:41 <coryfields_> BlueMatt: yes, actually
430 2014-10-14 07:05:51 <wumpus> BlueMatt: it's too late for that already I think
431 2014-10-14 07:05:59 <wumpus> but for the next thing...
432 2014-10-14 07:06:11 <coryfields_> quicksync is too obvious?
433 2014-10-14 07:06:12 <Luke-Jr> something to imply we peek into the future to see the outcome, before we get there?
434 2014-10-14 07:06:21 <BlueMatt> votes on int bitcoinconsensus_version_check(int script_flags_I_know_how_to_use) -> -1 (you are missing something), 0 (all good in the 'hood), 1 (FLYING DELORIAN ALERT)
435 2014-10-14 07:06:23 <BlueMatt> ?
436 2014-10-14 07:06:49 <coryfields_> s/DELORIAN/DORIAN/
437 2014-10-14 07:06:49 <wumpus> satoshi's flying delorean is back?
438 2014-10-14 07:07:04 <Luke-Jr> BlueMatt: can we abbreviate the functions bcagree_*?
439 2014-10-14 07:07:10 <BlueMatt> Luke-Jr: ehhhh
440 2014-10-14 07:07:17 <BlueMatt> I would like them abbreviated, but bcagree???
441 2014-10-14 07:07:17 <Luke-Jr> coryfields_: careful, he might sue you <.<
442 2014-10-14 07:07:34 <Luke-Jr> BlueMatt: what else is consensus but agreement? :P
443 2014-10-14 07:07:46 <BlueMatt> true, but I'm not sure it sends the right message?
444 2014-10-14 07:07:58 <gmaxwell> libbitcoinconform
445 2014-10-14 07:08:07 <BlueMatt> how is that much shorter/
446 2014-10-14 07:08:08 <BlueMatt> ?
447 2014-10-14 07:08:10 <Luke-Jr> bccnsnss
448 2014-10-14 07:08:20 <Luke-Jr> please not that
449 2014-10-14 07:08:27 <BlueMatt> yay, startup letter play!
450 2014-10-14 07:08:32 <BlueMatt> I vote werahgjksethrkljgar
451 2014-10-14 07:09:10 <Luke-Jr> bcunity
452 2014-10-14 07:09:37 <BlueMatt> heh, I kinda like that
453 2014-10-14 07:09:45 <coryfields_> wumpus: i have a few more things in mind for .10, am i in danger if i don't have 'em PR'd asap?
454 2014-10-14 07:09:47 <BlueMatt> btcconsensus is always an option
455 2014-10-14 07:09:52 <gmaxwell> just name it the sha256 sum of the code.
456 2014-10-14 07:10:02 <coryfields_> wumpus: namely the gitian descriptors and gitian osx signing
457 2014-10-14 07:10:03 <Luke-Jr> bcamity
458 2014-10-14 07:10:11 <Luke-Jr> gmaxwell: we're past names, just doing func prefix now
459 2014-10-14 07:10:20 <gmaxwell> bcc_
460 2014-10-14 07:10:25 <wumpus> Luke-Jr: I like it, though I hope it doesn't get confused with ubuntu's unity
461 2014-10-14 07:10:29 <sipa_> coryfields_: i have a shell script i used for the HF builds
462 2014-10-14 07:10:31 <wumpus> or the unity 3d engine
463 2014-10-14 07:10:33 <Luke-Jr> bcgoodvibes_*
464 2014-10-14 07:10:36 <wumpus> naah too overused already
465 2014-10-14 07:10:53 <sipa_> coryfields_: it could pretty much directly become gitian
466 2014-10-14 07:10:54 <BlueMatt> Luke-Jr: you know thats a sex shop?
467 2014-10-14 07:10:56 <coryfields_> HF builds?
468 2014-10-14 07:11:03 <sipa_> headersfirst
469 2014-10-14 07:11:08 <Luke-Jr> BlueMatt: no, I was just going through a thesaurus
470 2014-10-14 07:11:24 <BlueMatt> we should name it after famous sex stores!
471 2014-10-14 07:11:26 <Luke-Jr> â¦â¦â¦..
472 2014-10-14 07:11:36 <BlueMatt> I thought Luke-Jr might like that
473 2014-10-14 07:11:37 <wumpus> coryfields_: that depends; if it's in line with what we already agreed on needs to be in 0.10, there is no special hurry
474 2014-10-14 07:11:40 <coryfields_> sipa_: it's all done and PR'd, just needs to be revisited and get some ACKs
475 2014-10-14 07:11:41 <Luke-Jr> more seriously, bcc_ was ok
476 2014-10-14 07:11:47 <sipa_> oh ok!
477 2014-10-14 07:11:53 <BlueMatt> bcc seems too short?
478 2014-10-14 07:11:59 <BlueMatt> naming collisions very possible?
479 2014-10-14 07:12:03 <coryfields_> sipa_: mind having a look and seeing if your stuff would be any better though?
480 2014-10-14 07:12:06 <BlueMatt> btcc_ at least?
481 2014-10-14 07:12:10 <Luke-Jr> BlueMatt: how many other bcc_* are going to have the same functions?
482 2014-10-14 07:12:15 <Luke-Jr> no way, btc is the unit
483 2014-10-14 07:12:27 <BlueMatt> it is?
484 2014-10-14 07:12:30 <coryfields_> wumpus: i think it's all agreed upon, i just haven't pushed anybody to test/verify yet
485 2014-10-14 07:12:30 <Luke-Jr> ⦠yes?
486 2014-10-14 07:12:36 <wumpus> wasn't it xtc? I mean xbt
487 2014-10-14 07:12:48 <Luke-Jr> wumpus: that's another unit, one that was left undefined XD
488 2014-10-14 07:12:54 <coryfields_> for the gitian descriptors, it basically shoves the travis builds into gitian, so those have some miles on em now
489 2014-10-14 07:13:02 <Luke-Jr> XBT could be the same as either BTC or mBTC by popular opinion <.<
490 2014-10-14 07:13:32 <coryfields_> sipa_: #4727 is what i'm referencing
491 2014-10-14 07:14:15 <wumpus> coryfields_: yes, the gitian overhaul
492 2014-10-14 07:15:38 <wumpus> coryfields_: I don't remember why it was closed unmerged last time
493 2014-10-14 07:16:08 <wumpus> but in principle it is the way forward
494 2014-10-14 07:16:19 <coryfields_> wumpus: the qt4/qt5 debate
495 2014-10-14 07:16:36 <wumpus> wasn't that resolved?
496 2014-10-14 07:17:13 <wumpus> ie - keep using qt4.old on Linux for now for compatibility
497 2014-10-14 07:17:21 <coryfields_> yea, i just didn't push for merge because travis wasn't up and running at the time, and i wanted that to happen first to knock out the major kinks
498 2014-10-14 07:17:45 <wumpus> ok, so anyhow if you PR that again I'm sure we can merge it this time
499 2014-10-14 07:18:01 <coryfields_> i haven't revisted it since travis. just need a day or so to get it up to date and cleaned up
500 2014-10-14 07:18:21 <coryfields_> ok, great. will do once i can hand off the consensus lib stuff
501 2014-10-14 07:18:40 <wumpus> okay
502 2014-10-14 07:19:51 <sipa_> coryfields_: http://0bin.net/paste/MzKNWTezq-kytXp0#xcaYbgTi-V45o3bHDXBQlbhYHBA/ZKiWHCb5ade2few
503 2014-10-14 07:22:59 <BlueMatt> coryfields: it looks like several of these drop boost dep changes are not runtime deps?
504 2014-10-14 07:23:13 <BlueMatt> seems like your attacking both runtime and compile-time deps all at once?
505 2014-10-14 07:23:15 <BlueMatt> coryfields_:
506 2014-10-14 07:23:50 <coryfields_> sipa_: ah, that combines all the builds. for the gitian descriptors, they need to be split up
507 2014-10-14 07:23:57 <wumpus> sipa: to avoid confusion we use `gpg --digest-algo sha256 --clearsign SHA256SUMS` to make the SHA256SUMS.asc
508 2014-10-14 07:23:58 <sipa_> sure sure
509 2014-10-14 07:24:15 <wumpus> otherwise you get a Hash: SHA512 header or such at the top
510 2014-10-14 07:24:27 <sipa_> wumpus: good idea
511 2014-10-14 07:24:44 <sipa_> coryfields_: yeah, just showing what i quickly put together
512 2014-10-14 07:24:56 <sipa_> coryfields_: i was surprised to see that the gitian builds were not using depends yet
513 2014-10-14 07:25:10 <sipa_> (also, not going to check the PR now; too drunk)
514 2014-10-14 07:25:46 <sipa_> it is convenient to just run one script, and end up with one set of releaseable files
515 2014-10-14 07:26:05 <coryfields_> BlueMatt: yes, easiest that way. if you can build it without boost, you can be 100% certain you're free of its quirks
516 2014-10-14 07:26:16 <sipa_> agree
517 2014-10-14 07:26:26 <coryfields_> sipa_: haha, sure you're not at the Ballmer peak?
518 2014-10-14 07:26:34 <coryfields_> that's prime PR reviewing time ;)
519 2014-10-14 07:26:34 <sipa_> coryfields_: how would i test?
520 2014-10-14 07:26:38 <BlueMatt> coryfields_: sure, ok
521 2014-10-14 07:27:37 <coryfields_> sipa_: if you can get deep into templated templates, you've reached the peak.
522 2014-10-14 07:28:16 <coryfields_> (not sure if you wanted to test your drunken coding abilities or the PR, but the first was more fun to consider)
523 2014-10-14 07:30:29 <coryfields_> if you meant the PR, don't worry about it yet. It needs a rebase and a few updates.
524 2014-10-14 07:31:47 <BlueMatt> can I disagree with sipa and suggest that coryfields_ undos the moving of the constant string out of CAccEntryMeta, etc in https://github.com/theuni/bitcoin/commit/293dfb8e282071104f7b59e4eba478d60d3b89a9 ?
525 2014-10-14 07:32:01 <BlueMatt> its in the same cpp, and clearly designed for use only in those functions.....
526 2014-10-14 07:32:18 <sipa_> NO YOU CANNOT DISAGREE
527 2014-10-14 07:32:20 <BlueMatt> (maybe actually even define right before WriteAccountingEntry?)
528 2014-10-14 07:32:20 <sipa_> duh
529 2014-10-14 07:32:23 <BlueMatt> bye
530 2014-10-14 07:32:23 <BlueMatt> ok
531 2014-10-14 07:32:35 <sipa_> personal opinion is outlawed here
532 2014-10-14 07:32:38 <BlueMatt> (pronounced byeeeeeee)
533 2014-10-14 07:32:50 <sipa_> it should be "kthxbye"
534 2014-10-14 07:33:42 <BlueMatt> coryfields_: ehhh to https://github.com/theuni/bitcoin/commit/21f37856f40c308dbd5dab8d15556328569d01e0 can we make it more explicit what is going on there?
535 2014-10-14 07:33:55 <BlueMatt> coryfields_: otherwise #5082 lgtm
536 2014-10-14 07:34:52 <coryfields_> sipa: that script is pretty convenient, btw. especially if you're working on something long-lived like headers-first and don't want to rely on travis
537 2014-10-14 07:35:01 <BlueMatt> coryfields_: sorry, I'll leave comments to github
538 2014-10-14 07:37:29 <coryfields_> BlueMatt: np. that change is kinda clunky. i was kinda hoping someone better with templates (and not spoiled by c++11) would come up with something cleaner and make me look dumb.
539 2014-10-14 07:38:05 <BlueMatt> well, just some comments to say what it is doing would be nice :)
540 2014-10-14 07:40:27 <coryfields_> BlueMatt: if it's a vector of classes, each one needs to be serialized individually. in the special case where it's a vector of unsigned char's, it's understood that it's a single serialize operation.
541 2014-10-14 07:40:42 <coryfields_> i'll something along those lines as a comment
542 2014-10-14 07:40:47 <coryfields_> *i'll add
543 2014-10-14 09:06:33 <kindoge> kryptokit doesnt support p2sh address? "malformed address"
544 2014-10-14 09:06:36 <kindoge> way to go
545 2014-10-14 12:32:17 <Taek> I'm trying to understand the difference between the implementations of p2pkh and p2sh.
546 2014-10-14 12:33:06 <Taek> In both cases, it looks like you submit some script as the pubkey script and some script as the signature script
547 2014-10-14 12:34:07 <Taek> and it looks like you wouldn't need a fork of the client to do both, except that now the client recognizes a certain prefix on the hash of a wallet (3) as different instructions on how to build the pubkey hash
548 2014-10-14 12:34:46 <Taek> meaning that p2sh was always possible, it's just that when making the transaction you also had to tell the sender what the whole pubkey script was
549 2014-10-14 12:34:50 <Taek> is this correct?
550 2014-10-14 13:20:36 <Taek> ;
551 2014-10-14 13:42:00 <petertodd> Taek: I'm not quite sure what you're saying there; have you seen the developer docs at https://bitcoin.org/en/developer-documentation?
552 2014-10-14 13:42:22 <Taek> that's what I'm reading
553 2014-10-14 13:42:37 <petertodd> Taek: these low level examples of creating a P2PKH and P2SH signature/transaction may help you too if you follow the logic through closely: https://github.com/petertodd/python-bitcoinlib/blob/master/examples/spend-p2pkh-txout.py https://github.com/petertodd/python-bitcoinlib/blob/master/examples/spend-p2sh-txout.py
554 2014-10-14 13:42:58 <Taek> Thanks, I will check them out
555 2014-10-14 13:44:55 <petertodd> Taek: it's a bit of an investment of time, but I'd *strongly* suggest you read the actual script evaluation code in Bitcoin Core to get a better handle on what's actually going on at the low-level. That or read the script handling code in my python-bitcoinlib, which is basically a 1:1 translation of it to Python.
556 2014-10-14 14:30:05 <aschildbach> I wonder if there any effort going on for bitcoin-qt to support cameras? I'm peu a peu removing Bitcoin addresses from Bitcoin Wallet but users complain they can't type the addresses into bitcoin-qt any more.
557 2014-10-14 14:30:24 <aschildbach> petertodd: I think you have been a proponent of removing addresses, right?
558 2014-10-14 14:32:24 <wumpus> I'm not opposed to QR code reading, but it sounds like that would pull in crazy deependencies; I doubt we'll merge something like that as long as the wallet is still in the same repository as the node implementation
559 2014-10-14 14:35:52 <wumpus> ACTION already sees the "bitcoin-qt uses your webcam, big brother is watching you make transactions through bitcoin-qt!" articles on coindesk :p
560 2014-10-14 14:36:55 <dabura667> ACTION runs
561 2014-10-14 14:37:02 <wumpus> what would already be possible is to use an external QR-code reading program, which dispatches URIs through the system; bitcoin: URIs end up at the right place
562 2014-10-14 14:39:28 <wumpus> if there is an open source QR code reader we could even encourage people to use it
563 2014-10-14 15:01:16 <aschildbach> wumpus: QtQR is available for Ubuntu, but it doesn't throw any "Intents".
564 2014-10-14 15:05:32 <wumpus> I suppose that wouldn't be too hard to add, the decoding is the hard part
565 2014-10-14 15:07:25 <wumpus> removing addresses completely from a Bitcoin Wallet doesn't sound like a good idea to me, if you'd rather not show them by default you could make it an extra 'click' to display them
566 2014-10-14 17:16:37 <sipa> Taek: arbitrary redeem scripts were always possible at the protocol level, but before p2sh they had to go in full in the txouts... with p2sh the output can just contain a hash, and the full redeemscript is revealed by the receiver when spending the coins