1 2014-10-20 00:32:34 <CodeShark> backackiz: "Because it can produce a few notes, tho they are very flat; and it is never put with the wrong end in front!"
2 2014-10-20 00:58:33 <sipa> Luke-Jr: whenever state is not clean, there should be a chain of return falses
3 2014-10-20 00:59:42 <Luke-Jr> sipa: so, I can safely reinitialise state before ConnectBlock, right?
4 2014-10-20 00:59:49 <Luke-Jr> ConnectTip*
5 2014-10-20 00:59:53 <sipa> why would you?
6 2014-10-20 00:59:57 <sipa> it must be clean
7 2014-10-20 01:01:26 <Luke-Jr> sipa: "clean" is less well-defined in bugfix_processblock now - ConnectTip sets it to "conclusive", so we need to reinitialise it to inconclusive for the next connect
8 2014-10-20 01:01:37 <sipa> i really don't like that
9 2014-10-20 01:01:47 <Luke-Jr> why?
10 2014-10-20 01:02:06 <sipa> imho we should just do something where you can install a handler to listen for block validation result events
11 2014-10-20 01:02:14 <sipa> rather than add more state that is passed along everywhere
12 2014-10-20 01:03:36 <Luke-Jr> seems overengineered for this case IMO
13 2014-10-20 01:04:14 <Luke-Jr> how would we block on it?
14 2014-10-20 01:04:18 <Luke-Jr> what if it doesn't get reorg'd to?
15 2014-10-20 01:04:23 <Luke-Jr> then the block never ends
16 2014-10-20 01:04:29 <Luke-Jr> blocking*
17 2014-10-20 01:04:36 <sipa> just wait until ProcessBlock returns
18 2014-10-20 01:05:53 <sipa> if the block never causes validation because it never becomes the best chain, it's inconclusive
19 2014-10-20 01:06:43 <Luke-Jr> eck, kindof a roundable hack that suddenly breaks if someone innocently moves the reorg to a new thread and lets ProcessBlock return early
20 2014-10-20 01:07:52 <sipa> fair enough
21 2014-10-20 01:07:57 <Luke-Jr> seems much simpler to restore ProcessBlock's expected behaviour
22 2014-10-20 01:08:24 <sipa> your way will also break if we move things to another thread
23 2014-10-20 01:08:28 <Luke-Jr> I mean, right now it returns true even if the block is invalid if it ended up using another tip
24 2014-10-20 01:08:47 <Luke-Jr> this way forces the person moving it to another thread, to consider this case
25 2014-10-20 01:08:51 <sipa> I think ProcessBlock should not return anything at all, because it can't guarantee anything
26 2014-10-20 01:09:24 <sipa> and in fact the only thing it returns right now is the potential system error case
27 2014-10-20 01:09:44 <Luke-Jr> even if the only return is the CValidationState, we need *some* kind of idea what it was
28 2014-10-20 01:10:04 <Luke-Jr> for example, right now the p2p protocol "reject" message is broken too
29 2014-10-20 01:10:08 <sipa> no
30 2014-10-20 01:10:37 <Luke-Jr> for any invalidity detected in ConnectTip, it is
31 2014-10-20 01:10:37 <sipa> failed block connection triggers a reject message to be sent
32 2014-10-20 01:10:46 <Luke-Jr> ah
33 2014-10-20 01:10:50 <sipa> in InvalidBlockFound
34 2014-10-20 01:10:51 <Luke-Jr> ok, so it uses a different code path
35 2014-10-20 01:11:01 <sipa> which is where i've been saying that this handler should be put
36 2014-10-20 01:11:12 <sipa> so we can move the reject stuff also into that handler
37 2014-10-20 01:11:23 <sipa> and not duplicate that logic
38 2014-10-20 01:11:36 <sipa> you want me to try writing something quickly?
39 2014-10-20 01:12:02 <Luke-Jr> I suppose. Having ProcessBlock return with the correct state set still seems simpler and less prone to breakage in the future.
40 2014-10-20 01:12:17 <sipa> my point is that processblock shouldn't be able to tell you much
41 2014-10-20 01:12:29 <Luke-Jr> right now, it's expected to in multiple places
42 2014-10-20 01:13:01 <sipa> it only promises "we accepted the block into the tree, and afterwards reorganized to the resulting best tip"
43 2014-10-20 01:13:43 <sipa> as a single block processing can trigger validation of any number of blocks (including 0 or 100000), it seems silly to to try to tie the validation of a block to its invokation of ProcessBlock
44 2014-10-20 01:15:43 <Luke-Jr> sipa: bugfix_processblock does correctly handle all cases, at least
45 2014-10-20 01:16:01 <sipa> with more technical debt, imho
46 2014-10-20 01:16:25 <Luke-Jr> positive technical debt? ;)
47 2014-10-20 01:16:33 <Luke-Jr> the technical debt in this case is what prevents accidental breaking it
48 2014-10-20 01:17:13 <sipa> it's already ugly that we need handlers for bad validation in several layers of validation
49 2014-10-20 01:17:31 <Luke-Jr> ?
50 2014-10-20 01:17:33 <sipa> this makes it worse, by introducing more states to reason about
51 2014-10-20 01:17:48 <sipa> there's code for sending reject messages in several places afaik
52 2014-10-20 01:18:05 <Luke-Jr> it's unavoidable to reason about all possible states..?
53 2014-10-20 01:18:18 <sipa> i want to reduce the number of states
54 2014-10-20 01:18:32 <sipa> ideally, ProcessBlock _can_ only return some structure that can return system failure
55 2014-10-20 01:18:55 <sipa> and every per-block action that needs to be takes as the result of the outcome of validation of particular blocks is moved to some signal handling
56 2014-10-20 01:18:58 <Luke-Jr> the other states (invalid and inconclusive) still exist though
57 2014-10-20 01:19:26 <Luke-Jr> well, I suppose there is some refactoring that could improve here
58 2014-10-20 01:19:38 <Luke-Jr> to track the state each block has ever been validated on the CBlockIndex or such
59 2014-10-20 01:19:46 <sipa> that's already the case
60 2014-10-20 01:20:10 <sipa> well, the level of validation passed is stored
61 2014-10-20 01:20:13 <sipa> not the actual message
62 2014-10-20 01:21:07 <sipa> sorry, i'm mostly responsible for the current mess (and i consider it a mess)
63 2014-10-20 01:21:23 <sipa> so i would rather make things clean, than add more cases to it
64 2014-10-20 01:21:43 <Luke-Jr> not in a case generally usable I think: pindex->RaiseValidity(BLOCK_VALID_SCRIPTS); is only called after writing to disk, which is too late for proposals (including the existing miner tests, not necessarily RPC)
65 2014-10-20 01:22:00 <Luke-Jr> is it safe to move that before if (fJustCheck)?
66 2014-10-20 01:22:29 <sipa> you shouldn't be modifying anything in justcheck mode
67 2014-10-20 01:22:33 <sipa> i think?
68 2014-10-20 01:22:53 <Luke-Jr> JustCheck should modify the CBlockIndex validity flags, since it does the checks
69 2014-10-20 01:23:19 <Luke-Jr> if we want to use the CBlockIndex validity flags for differentiating between inconclusive, invalid, and valid, it needs to?
70 2014-10-20 01:23:37 <sipa> ugh
71 2014-10-20 01:23:57 <sipa> it would be so much simpler if you could just call ConnectTip
72 2014-10-20 01:25:25 <Luke-Jr> that would be a mess for both proposals and submitblock RPC I think
73 2014-10-20 01:25:37 <Luke-Jr> since the former shouldn't write to disk, and the latter might reorg
74 2014-10-20 01:26:19 <sipa> it could do connectblock on a reversible cache of the chainstate
75 2014-10-20 01:26:30 <sipa> that would be a lot cleaner too than having justcheck flags everywhere
76 2014-10-20 01:26:41 <sipa> but that will require a lot more refactorings
77 2014-10-20 01:27:37 <Luke-Jr> and use more CPU time for submitblock as it connects it twice (one to determine validity, then again for the real reorg) - unless I misunderstand what you mean by that
78 2014-10-20 01:28:15 <sipa> connecttip would actually connect
79 2014-10-20 01:28:23 <sipa> not for proposals, though
80 2014-10-20 01:29:07 <Luke-Jr> sipa: so what if the parent to the submitted block is not in the main chain?
81 2014-10-20 01:29:29 <sipa> well, then it's a stale block anyway, no?
82 2014-10-20 01:29:40 <Luke-Jr> no, it could be causing a reorg
83 2014-10-20 01:29:46 <Luke-Jr> current main chain*
84 2014-10-20 01:29:51 <sipa> ah, agree
85 2014-10-20 01:31:13 <sipa> but for a proposal you absolutely shouldn't even get to the point of having a CBlockIndex entry - at least not one in the mapBlockIndex
86 2014-10-20 01:31:46 <sipa> and i don't see how to do that
87 2014-10-20 01:32:16 <sipa> so i would indeed something like using a copy the cache, rollback to the latest common block, and try to connect
88 2014-10-20 01:32:49 <Luke-Jr> if we don't use a CBlockIndex for the state, then we need all 4 states on CValidationCheck..
89 2014-10-20 01:33:16 <sipa> wait, are trying to solve how to do proposals too, or not?
90 2014-10-20 01:33:21 <Luke-Jr> at least not one in the mapBlockIndex <-- right, it doesn't have to be
91 2014-10-20 01:33:29 <sipa> i don't see how it can not be
92 2014-10-20 01:33:38 <Luke-Jr> sipa: we have proposals already, just not exposed to RPC
93 2014-10-20 01:33:44 <sipa> eh?
94 2014-10-20 01:33:59 <Luke-Jr> sipa: CreateNewBlock checks everything it produces
95 2014-10-20 01:34:15 <Luke-Jr> it creates a dummy CBlockIndex to do so
96 2014-10-20 01:34:28 <sipa> right, but that's easy as we know it's always on the top
97 2014-10-20 01:34:42 <sipa> with proposals that is not the case, so you need to full reorg logic
98 2014-10-20 01:35:19 <Luke-Jr> sipa: I'd like to solve this problem in a way that doesn't require rewriting an hour later to implement GBT proposals ;)
99 2014-10-20 01:35:29 <sipa> yes, i understand
100 2014-10-20 01:36:00 <sipa> but for proposals i see no other way than creating a copy of the utxo set, and doing a full reorg in it
101 2014-10-20 01:36:08 <sipa> unless you see simplifications
102 2014-10-20 01:36:10 <Luke-Jr> I don't think we need full reorg logic though - it'd be nice to have, but it'd be okay to return "inconclusive" rejection for stale blocks
103 2014-10-20 01:36:15 <sipa> ok
104 2014-10-20 01:36:36 <sipa> but you still want reorg logic
105 2014-10-20 01:36:59 <Luke-Jr> we probably *should* return a better response if it would trigger a reorg, I suppose
106 2014-10-20 01:37:33 <sipa> ok, if you can keep the validation like CreateNewBlock does, fine - using a dummy CBlockIndex
107 2014-10-20 01:37:53 <sipa> and just say "whenever the parent hash is not the current tip, return some error"
108 2014-10-20 01:38:20 <Luke-Jr> note CreateNewBlock's check is currently broken
109 2014-10-20 01:38:33 <sipa> it doesn't do the checks in AcceptBlock
110 2014-10-20 01:38:37 <Luke-Jr> and because ProcessBlock fails to return the correct state
111 2014-10-20 01:38:41 <Luke-Jr> err
112 2014-10-20 01:38:48 <Luke-Jr> nm
113 2014-10-20 01:39:07 <sipa> CreateNewBlock and proposal don't use ProcessBlock, and shouldn't
114 2014-10-20 01:39:11 <Luke-Jr> ProcessBlock only affects submitblock & similar right now
115 2014-10-20 01:39:15 <Luke-Jr> right
116 2014-10-20 01:39:35 <sipa> we can later think about a solution that is more generic and perhaps involves dealing with reorgs for those
117 2014-10-20 01:40:01 <sipa> so the real problem now is submitblock, and getting the result out
118 2014-10-20 01:40:22 <Luke-Jr> yes
119 2014-10-20 01:40:35 <Luke-Jr> we should also probably deal with the other cases that expect ProcessBlock to return a valid state too\
120 2014-10-20 01:41:44 <sipa> well ProcessBlock returns whether the block was succesfully processed :)
121 2014-10-20 01:42:05 <sipa> but i agree it's confusing now
122 2014-10-20 01:43:14 <Luke-Jr> historically, ProcessBlock has returned failure in all cases where the block was invalid - I think
123 2014-10-20 01:43:31 <Luke-Jr> if we're going to go the error-only route, we should at least rename it and document the difference
124 2014-10-20 01:43:45 <sipa> agree
125 2014-10-20 01:44:52 <Luke-Jr> hm, do we really want to store the rejection reason for invalidity on every invalid CBlockIndex?
126 2014-10-20 01:49:23 <Luke-Jr> sipa: were you going to throw together these new ideas in code, or should I go ahead and do that?
127 2014-10-20 01:59:37 <sipa> Luke-Jr: hell no
128 2014-10-20 01:59:54 <sipa> i don't want to use CblockIndex, I want a signal handler that reports it to whatever cares :)
129 2014-10-20 02:01:02 <Luke-Jr> sipa: ok, were you going to draft that, or should I ponder how exactly such a thing would work? :p
130 2014-10-20 02:01:15 <sipa> i'll write it, but give me a few days time
131 2014-10-20 02:01:46 <Luke-Jr> ok (I'm not sure what wumpus wants in terms of schedule though)
132 2014-10-20 02:01:55 <sipa> bip62 isn't in yet
133 2014-10-20 02:02:25 <sipa> (then again, i shouldn't delay working on bip62 for this, if you get a faster correct and easy to review solution)
134 2014-10-20 02:03:43 <Luke-Jr> I guess I can look into it more.. were you thinking we pass the callback as an argument?
135 2014-10-20 02:04:27 <sipa> no, something in CWalletInterface (which should really be renamed to CValidationInterface)
136 2014-10-20 02:07:46 <sipa> #5105 for that
137 2014-10-20 02:14:26 <Luke-Jr> sipa: so I'll add a callback for invalid block found, a callback for valid block found, and a callback for error processing block?
138 2014-10-20 02:14:45 <Luke-Jr> (all 3 cases need handling..)
139 2014-10-20 02:16:04 <sipa> well, at which level? :)
140 2014-10-20 02:16:31 <sipa> invalid it easy - you can have a handler for any type of invalid result, which just passes the CValidationState
141 2014-10-20 02:16:42 <Luke-Jr> sipa: submitblock should return a reject reason, not return "inconclusive", or throw an exception, respectively
142 2014-10-20 02:17:17 <Luke-Jr> hm, I guess I could have a single callback for "validation completed" that just gets CValidationState
143 2014-10-20 02:17:29 <Luke-Jr> ?
144 2014-10-20 02:17:29 <sipa> what is "validation completed" ?
145 2014-10-20 02:17:43 <Luke-Jr> sipa: ConnectBlock returns
146 2014-10-20 02:17:45 <sipa> ok
147 2014-10-20 02:17:54 <Luke-Jr> there's nothing more to check in the block itself at that point, correct?
148 2014-10-20 02:18:00 <sipa> indeed
149 2014-10-20 02:18:16 <sipa> though it is still not guaranteed to be become part of the tip at that point
150 2014-10-20 02:18:25 <Luke-Jr> sure, that's not what I care about here
151 2014-10-20 02:20:08 <phantomcircuit> Luke-Jr, watcha doin
152 2014-10-20 02:20:10 <Luke-Jr> (and there's already a callback for new tip)
153 2014-10-20 02:20:15 <Luke-Jr> phantomcircuit: fixing submitblock, you? :P
154 2014-10-20 02:20:32 <phantomcircuit> nothing much
155 2014-10-20 02:20:45 <phantomcircuit> figured out why tftp loading the kernel wasn't working on some machines
156 2014-10-20 02:20:53 <phantomcircuit> (timeouts weren't set high enough)
157 2014-10-20 02:21:17 <phantomcircuit> Luke-Jr, making submitblock actually return a meaningful reject reason?
158 2014-10-20 02:21:50 <Luke-Jr> sipa: IMO for the rename, bundle noop virtuals so I don't need to implement every possible one ;p
159 2014-10-20 02:22:01 <Luke-Jr> phantomcircuit: making it actually return "rejected" for rejected blocks, for starters
160 2014-10-20 02:22:29 <Luke-Jr> phantomcircuit: it's fine in 0.9.3, but broken in git from some refactoring
161 2014-10-20 02:22:47 <Luke-Jr> (and only broken in weird cases, like invalid scripts)
162 2014-10-20 02:29:05 <lechuga_> gmaxwell: re: script VM, is perf really going 2 b too terrible?
163 2014-10-20 03:02:22 <BlueMatt> re: me signign pull requests....travelling this week: http://0bin.net/paste/52fes0z4deNZoJ92#xureVUwmM8eaczOgyUh5EQnw4h6rFbKUbHUCqkxwGXx
164 2014-10-20 03:02:26 <BlueMatt> no pgp access
165 2014-10-20 03:02:32 <BlueMatt> until friday/saturday
166 2014-10-20 03:04:32 <Luke-Jr> BlueMatt: why not make a temporary key to sign with while away? :P
167 2014-10-20 03:05:36 <BlueMatt> Luke-Jr: because....effort (and I was very tired when I left this morning and didnt think that far ahead)
168 2014-10-20 03:05:54 <BlueMatt> I was walking out the door and was like "I should sign something that said I was gone"
169 2014-10-20 03:05:55 <Luke-Jr> XD
170 2014-10-20 04:19:25 <Luke-Jr> sipa: how's this? https://github.com/bitcoin/bitcoin/pull/5106/files
171 2014-10-20 04:20:14 <Luke-Jr> (I'll do a reject-reason improvement when this is merged)
172 2014-10-20 04:32:25 <Luke-Jr> wow, Gentoo wants to rebuild all my [cross-]compilers to add sanitize support :P
173 2014-10-20 04:47:08 <ewrwer> Hi Everybody
174 2014-10-20 05:41:16 <CodeShark> removal of main.h dependency from rpcserver is complete: https://github.com/bitcoin/bitcoin/pull/5107
175 2014-10-20 06:02:29 <mrebola> thanks for the update CodeShark , nice job
176 2014-10-20 06:02:39 <CodeShark> thx
177 2014-10-20 06:03:40 <mrebola> CodeShark, what is the idea behind moving the rpc server to a separate package ? Further security?
178 2014-10-20 06:04:17 <sipa> ...?
179 2014-10-20 06:04:33 <sipa> reusability, i guess
180 2014-10-20 06:05:41 <CodeShark> just on general architectural principles, it is desirable that any RPC server functionality which is generally useful should reside in a reusable unit. But there are also other issues - such as that we don't want the RPC server module to have to be rebuilt each time we make a change in main.hâ¦and we want to make it easier for developers to improve units independently
181 2014-10-20 06:06:37 <CodeShark> it will also make it far simpler to extend bitcoind to support more commands in the future
182 2014-10-20 06:07:36 <CodeShark> or if not far simpler, at least far cleaner :)
183 2014-10-20 06:07:36 <mrebola> go it , CodeShark
184 2014-10-20 06:07:37 <mrebola> cool !
185 2014-10-20 06:08:07 <mrebola> Best coding practices
186 2014-10-20 06:08:07 <mrebola> :P
187 2014-10-20 10:10:54 <hearn> good day
188 2014-10-20 10:15:50 <wumpus> hello
189 2014-10-20 11:03:06 <CodeShark> hi wumpus
190 2014-10-20 11:03:18 <CodeShark> hi mike
191 2014-10-20 11:04:05 <CodeShark> wumpus: I finished doing what we had talked about yesterday
192 2014-10-20 11:04:12 <CodeShark> pull #5107
193 2014-10-20 11:05:01 <CodeShark> once reviewed, tested, and merged, I'm ready to start working on the abstract callable interface
194 2014-10-20 11:05:40 <wumpus> CodeShark: ok great
195 2014-10-20 11:08:03 <wumpus> CodeShark: do we need any changes to the src/qt/rpcconsole.cpp?
196 2014-10-20 11:08:15 <wumpus> (due to the locking move)
197 2014-10-20 11:08:42 <CodeShark> not sure - I didn't touch that
198 2014-10-20 11:09:41 <wumpus> I don't actually think so, as it just calls tableRPC.execute
199 2014-10-20 11:10:14 <wumpus> although: you need to connect the RPC signals always
200 2014-10-20 11:11:03 <wumpus> the precommand handler is also used when a RPC command is executed through the GUI
201 2014-10-20 11:11:06 <CodeShark> as far as I know, the locking move just moves the locks one level down the call stack
202 2014-10-20 11:11:19 <CodeShark> there are no other calls in between as far as I know
203 2014-10-20 11:11:33 <wumpus> agreed, there is no concern about the locking
204 2014-10-20 11:12:02 <CodeShark> does the GUI call execute()?
205 2014-10-20 11:12:09 <wumpus> yes
206 2014-10-20 11:12:13 <CodeShark> then it should be fine
207 2014-10-20 11:12:24 <CodeShark> no?
208 2014-10-20 11:12:30 <wumpus> well it needs a change to init
209 2014-10-20 11:12:49 <wumpus> you only call RPCServer::OnPreCommand(&OnRPCPreCommand); if fServer
210 2014-10-20 11:12:58 <CodeShark> ah, right
211 2014-10-20 11:13:00 <wumpus> UI doesn't have fServer by default
212 2014-10-20 11:13:27 <CodeShark> so should we just subscribe the handlers no matter what but only start the RPC threads if fServer?
213 2014-10-20 11:13:35 <wumpus> yep
214 2014-10-20 11:14:13 <CodeShark> and it's safe to subscribe them just prior to the if (fServer) ?
215 2014-10-20 11:14:30 <CodeShark> in other words, the GUI won't call the RPC before that?
216 2014-10-20 11:14:33 <wumpus> sure - why not? the UI has its own entry point to start a dummy RPC thread, StartDummyRPCThread(), which is executed after AppInit2
217 2014-10-20 11:14:43 <CodeShark> ok
218 2014-10-20 11:14:47 <wumpus> it certainly won't execute RPC before finishing AppInit2
219 2014-10-20 11:19:16 <CodeShark> ok, just rebased
220 2014-10-20 11:27:23 <CodeShark> hmm, where are the category-specific LogPrints written?
221 2014-10-20 11:47:48 <wumpus> to debug.log if you provide -debug=<category>
222 2014-10-20 11:49:34 <CodeShark> so should we add an rpc category?
223 2014-10-20 11:50:17 <wumpus> if you think that's useful, sure; I'm not sure (for example) logging every prerpccommand signal invocation is useful tho
224 2014-10-20 11:50:53 <wumpus> eh
225 2014-10-20 11:50:58 <wumpus> there *IS* a rpc debug category
226 2014-10-20 11:51:06 <CodeShark> oh, right :)
227 2014-10-20 11:51:21 <CodeShark> well, it's certainly useful for testing and debugging - and logging every rpc command is absolutely essential for auditing
228 2014-10-20 11:51:42 <wumpus> that already happens LogPrint("rpc", "ThreadRPCServer method=%s\n", strMethod);
229 2014-10-20 11:52:03 <CodeShark> I just added the tracers to make sure the signal mechanism was working
230 2014-10-20 11:52:24 <wumpus> I'd do the same, I'd just take them out after I was assured of that :)
231 2014-10-20 11:53:20 <CodeShark> ok, not a problem - just kill the last commit :)
232 2014-10-20 11:53:37 <CodeShark> err, the penultimate one
233 2014-10-20 11:54:23 <wumpus> if you do want to keep at least move them to the rpc category,but as said I personally don't think multiple log entries per RPC command are useful
234 2014-10-20 11:54:58 <wumpus> if you're paranoid about the signal mechanism failing, add a unit test for it :)
235 2014-10-20 11:55:26 <CodeShark> nah, not paranoid about the signal mechanism itself failing - that only really needs to be tested once. more afraid of issues in handlers
236 2014-10-20 11:56:00 <CodeShark> in my own code I usually err on the side of having too many tracers rather than too few
237 2014-10-20 11:56:14 <wumpus> the thing is, debugging and troubleshooting is incredibly personal
238 2014-10-20 11:56:18 <CodeShark> it helps diagnose things like server crashes
239 2014-10-20 11:56:27 <CodeShark> when you don't happen to have a debugger running
240 2014-10-20 11:56:28 <wumpus> one person likes trace calls all over the place, others use gdb and a breakpoint/tracepoint, etc
241 2014-10-20 11:57:12 <wumpus> the logging mechanism is not meant or that, though
242 2014-10-20 11:57:31 <CodeShark> anyhow, it isn't really a big deal - I can take them out
243 2014-10-20 11:57:56 <wumpus> especially log messages without category should be used extremely sparsely
244 2014-10-20 11:58:59 <CodeShark> that's fine - I'll remove all of them once we've tested
245 2014-10-20 12:00:25 <CodeShark> in any case, the OnPreCommand thing will be taken out once we have the abstract callable interface
246 2014-10-20 12:00:42 <wumpus> I don't have any problems with the signal itself
247 2014-10-20 12:00:59 <CodeShark> the callable objects will have a pre and post method
248 2014-10-20 12:01:03 <CodeShark> virtual methods
249 2014-10-20 12:01:30 <CodeShark> and will implement operator()
250 2014-10-20 12:02:00 <CodeShark> the constructor can take a functional - or a lambda (are we using C++11 yet?)
251 2014-10-20 12:02:22 <wumpus> isn't just operator () enough? pre- and post handlers can be implicit
252 2014-10-20 12:02:37 <wumpus> no, we're not using C++11
253 2014-10-20 12:02:59 <CodeShark> pre- and post- handlers mean we can inherit them and only overload operator ()
254 2014-10-20 12:03:41 <wumpus> I don't know - I generally prefer RAII to explicit context setup/teardown, and pre and post handlers force that
255 2014-10-20 12:04:06 <wumpus> also it means having to handle exceptions specially? (will post() be called in case of exception?)
256 2014-10-20 12:04:17 <CodeShark> currently no
257 2014-10-20 12:04:33 <wumpus> so that means any cleanup will not be done in case of an exception
258 2014-10-20 12:04:34 <CodeShark> post() is only called if the call returns normally
259 2014-10-20 12:05:10 <CodeShark> I guess it depends on what we want to be able to do
260 2014-10-20 12:05:34 <CodeShark> I wasn't looking at pre and post as setting up and cleaning up a context
261 2014-10-20 12:06:39 <CodeShark> if we don't have explicit pre and post methods, does this mean we have to call a base class operator () explicitly?
262 2014-10-20 12:07:04 <CodeShark> I guess the style I was after was something along the lines of:
263 2014-10-20 12:08:01 <CodeShark> rpcServer.addCommand(CRPCCommand("getinfo", [&](const params_t& params) { // do something }));
264 2014-10-20 12:08:27 <wumpus> dunno - the most flexibility you'd get if you could wrap a callable around another callable
265 2014-10-20 12:08:32 <CodeShark> we could also pass help info and param boundaries into the constructor
266 2014-10-20 12:09:05 <wumpus> anyhow, let's use what makes sense
267 2014-10-20 12:09:35 <CodeShark> I guess we could do something like rpcServer.addCommand(SafeModeEnabledRPCCommand(â¦.))
268 2014-10-20 12:09:45 <CodeShark> or hmm - a callable in another callable
269 2014-10-20 12:10:52 <wumpus> or a BitcoinRPCCommand(), that is a subclass of RPCCommand() that takes a boolean whether to allow the command in safe mode, that would be the simplest transition from the table
270 2014-10-20 12:11:10 <CodeShark> right
271 2014-10-20 12:12:09 <CodeShark> at what level does it make sense to do parameter type-checking and validation?
272 2014-10-20 12:12:36 <CodeShark> I mean, in general, some validation obviously cannot be done outside the method itself
273 2014-10-20 12:13:14 <wumpus> I wouldn't change too much there
274 2014-10-20 12:13:33 <wumpus> let's keep that the responsibility of the called function itself
275 2014-10-20 12:14:58 <CodeShark> this is something along these lines I put together a while back: https://github.com/ciphrex/CoinVault/blob/master/deps/cli/src/cli.hpp
276 2014-10-20 12:15:52 <CodeShark> example usage: https://github.com/ciphrex/CoinVault/blob/master/deps/CoinDB/tools/coindb/src/coindb.cpp#L1057
277 2014-10-20 12:17:04 <CodeShark> it handles help messages with parameter names and checks parameter number (but not type)
278 2014-10-20 12:18:50 <wumpus> I still think it makes sense to do that in the functions themselves, to have everything in one place
279 2014-10-20 12:19:34 <CodeShark> yeah, it's the most flexible approach - just requires rewriting the same type of conditionals to check parameters for each function - but I don't really mind
280 2014-10-20 12:20:13 <CodeShark> the help messages are intended for humans, not for machines
281 2014-10-20 12:20:21 <CodeShark> it's not really an introspection mechanism
282 2014-10-20 12:20:33 <wumpus> otherwise you end up with some description language, which is always incomplete anyway, and is another thing contributors need to learn... I prefer code that is direct, and doesn't put obvious things behind too many layers of abstraction
283 2014-10-20 12:20:42 <CodeShark> yeah, agreed
284 2014-10-20 12:21:19 <wumpus> I mean there would be an obvious advantage to describing the parameters in some form: JSON RPC introspection, there is actually a sort of standard for that
285 2014-10-20 12:21:31 <wumpus> but I think it's quite a detour
286 2014-10-20 12:22:02 <CodeShark> if we don't need it now and our architecture in principle allows us to add something like that later I'm all for skipping that for now :)
287 2014-10-20 12:22:09 <wumpus> better to spend our limited energy on things we actually need
288 2014-10-20 12:22:10 <wumpus> right :)
289 2014-10-20 12:26:47 <wumpus> does anyone perhaps have code to assure UTXO set integrity? ie, hash the whole UTXO set, as to compare with another node at the same height?
290 2014-10-20 12:27:46 <Luke-Jr> wumpus: bitcoind does..?
291 2014-10-20 12:27:56 <Luke-Jr> at least, it will give you a hash to compare..
292 2014-10-20 12:27:56 <wumpus> Luke-Jr: how?
293 2014-10-20 12:28:12 <Luke-Jr> I don't recall, but we have a RPC to get a UTXO hash
294 2014-10-20 12:29:33 <wumpus> ah, gettxoutsetinfo has a hash
295 2014-10-20 12:29:38 <wumpus> thanks :)
296 2014-10-20 12:30:07 <Luke-Jr> âº
297 2014-10-20 12:33:16 <wumpus> curiously, it computes the statistics on the backing database, but doesn't first Flush pcoinsTip
298 2014-10-20 12:34:30 <wumpus> (not sure this is actually an issue in practice, but it looks like it could result in problems determining *at which height* we're really computing it)
299 2014-10-20 12:37:55 <CodeShark> when are we going to be moving to C++11? or C++14?
300 2014-10-20 12:38:39 <wumpus> (ah, no, the RPC conveniently returns a height, which is actually the height of the backing store, great)
301 2014-10-20 12:39:24 <wumpus> I'd like to start using at least some C++11 functionality for 0.11, this is in practice restricted by the compilers we're using
302 2014-10-20 12:40:58 <CodeShark> which compilers? I only see gcc
303 2014-10-20 12:41:04 <wumpus> gcc and clang
304 2014-10-20 12:41:16 <wumpus> but it's not so much about which compilers, but the versions
305 2014-10-20 12:41:20 <CodeShark> they both support C++11, at least gcc 4.8 and above
306 2014-10-20 12:41:55 <CodeShark> and clang with llvm v3.3
307 2014-10-20 12:42:04 <wumpus> that's not the point though; does the gitian environment that we use to build releases support it, and can it be done keeping binary compatibility with older linux releases
308 2014-10-20 12:42:59 <CodeShark> it requires a different runtime library - but you should be able to statically link it. I guess there could be some issues for really old OSes
309 2014-10-20 12:43:22 <wumpus> statically linking libc++/libc is potentially dangerous, we don't do that
310 2014-10-20 12:43:31 <t7> wumpus: how so?
311 2014-10-20 12:43:36 <wumpus> we do have some compatibility framework in place though
312 2014-10-20 12:43:51 <wumpus> t7: has to do with name resolution/nss, and some other things, I suggest google
313 2014-10-20 12:44:33 <t7> ah ok
314 2014-10-20 12:46:08 <CodeShark> we don't even statically link it for windows?
315 2014-10-20 12:46:34 <wumpus> on windows you can't even fully statically link everything, you always need to use dlls
316 2014-10-20 12:47:22 <CodeShark> you mean dlls that we distribute? or dlls that we can safely assume to be preinstalled?
317 2014-10-20 12:47:24 <wumpus> anyhow, windows is a completely different case from linux here
318 2014-10-20 12:47:31 <wumpus> operating system dlls
319 2014-10-20 12:47:41 <wumpus> *everthing* is a dll on windows, even the kernel
320 2014-10-20 12:47:56 <CodeShark> well, of course - I'm specifically talking about the gcc runtime libraries
321 2014-10-20 12:48:02 <CodeShark> mingw stuff
322 2014-10-20 12:48:13 <wumpus> mingw can be safely statically linked on windows
323 2014-10-20 12:49:55 <CodeShark> I haven't been trying to build for ancient linux versions - but the only OS I've really had at least a little bit of a struggle with so far (other than archaic windows xp and the like) is OS X v10.6 or before
324 2014-10-20 12:50:10 <wumpus> on linux you should regard the interface against libc as an operating system interface, for better or worse, I'm not 100% sure about libstdc++ though, but I know cfields looked into it at some point and decided it would be either dangerous or too much hassle to link that statically
325 2014-10-20 12:50:35 <CodeShark> libstdc++ and libc++ are mutually incompatible
326 2014-10-20 12:50:58 <wumpus> I don't know what's the difference between those
327 2014-10-20 12:51:22 <wumpus> i mean libstdc++.so.6
328 2014-10-20 12:51:27 <CodeShark> completely different implementations - they use different underlying representations for core datatypes
329 2014-10-20 12:52:00 <wumpus> ok but what's your point,that we could use another c++ standard lib?
330 2014-10-20 12:52:35 <CodeShark> no, just saying that mixing these two libraries on a single OS can be dangerous
331 2014-10-20 12:52:54 <wumpus> right - so that's what makes it dangerous to static link, understood
332 2014-10-20 12:53:49 <wumpus> at least if you statically link the c++ library you should make sure that you statically link *every* other c++ library as well, to avoid interaction with the system there
333 2014-10-20 12:53:57 <CodeShark> OS X 10.7 is a little annoying because some lib distributions use one library and some use the other
334 2014-10-20 12:54:23 <CodeShark> so you have to consistently use only one or the other for the entire project
335 2014-10-20 12:54:26 <wumpus> (and it rules out any kind of dynamic loading, not that we use that)
336 2014-10-20 12:54:58 <wumpus> yes that sounds very annoying
337 2014-10-20 12:55:44 <wumpus> sounds like the situation on windows, where MSVC is abi-incompatible with MINGW, so a c++ library compiled for one is useless for the other
338 2014-10-20 12:59:14 <wumpus> no such issue on linux, but on linux the most annoying issue is to ship a GUI that can run on various linux distributions and still uses the system theme/desktop integration
339 2014-10-20 12:59:37 <CodeShark> I've all but given up on that one :)
340 2014-10-20 12:59:54 <CodeShark> I figure if you use linux desktop, I'll give you the source code and you can figure that out for yourself :p
341 2014-10-20 13:00:14 <CodeShark> it's the windows and mac users I'm more concerned about, regarding GUI cleanliness
342 2014-10-20 13:00:35 <wumpus> well the problem is that people already have an expectation, you cannot just drop something in a new version
343 2014-10-20 13:00:49 <wumpus> if you're making new software it isn't that much of an issue :)
344 2014-10-20 13:01:06 <hearn> matching system themes is so 2001 :)
345 2014-10-20 13:01:28 <hearn> lighthouse takes the same approach web apps do - screw the users theme, let's just make it visually attractive with its own style.
346 2014-10-20 13:01:52 <wumpus> I don't want 'a style', I just want a functional GUI with as little hassle as possible
347 2014-10-20 13:02:22 <wumpus> the better if it integrates into the system and stays on the background
348 2014-10-20 13:04:55 <CodeShark> it's more about consistency in look and feel - for the sake of usability (and not only aesthetics)
349 2014-10-20 13:05:28 <wumpus> consistency to me means that different applications on an OS look and behave similarly
350 2014-10-20 13:05:39 <CodeShark> a well-designed GUI should be practically self-explanatory to a user accustomed to the environment
351 2014-10-20 13:07:10 <wumpus> right, so if people know a certain widget it should ideally behave in a certain way across the OS
352 2014-10-20 13:07:42 <wumpus> I'm personally not really happy about webapps because of that, all try to invent their own, be as flashy as possible, etc
353 2014-10-20 13:09:22 <CodeShark> flash is sooo 2003 :p
354 2014-10-20 13:09:56 <wumpus> alas, it's just as possible to be 'flashy' with html5 :p
355 2014-10-20 13:10:04 <michagogo> Ooh, headers-first is in?
356 2014-10-20 13:10:09 <wumpus> michagogo: yes
357 2014-10-20 13:10:26 <michagogo> ACTION just pulled on the machine running his master node
358 2014-10-20 13:11:01 <michagogo> master/tri-stack/dual-network
359 2014-10-20 13:11:14 <CodeShark> there's nothing wrong with eye candy in a UIâ¦as long as it communicates something important and isn't just there to make you forget how horrible the app really is
360 2014-10-20 13:11:53 <michagogo> Hm, maybe I should nuke the datadir blocks/ / chainstate/
361 2014-10-20 13:12:03 <wumpus> michagogo: why?
362 2014-10-20 13:12:18 <wumpus> it should just work
363 2014-10-20 13:12:21 <michagogo> See how it actually does
364 2014-10-20 13:12:27 <wumpus> oh, okay
365 2014-10-20 13:12:31 <michagogo> wumpus: well, just to test it :)
366 2014-10-20 13:12:40 <michagogo> Yeah, I know it's not needed :D
367 2014-10-20 13:12:53 <CodeShark> it's the only way to really benchmark it
368 2014-10-20 13:13:02 <Luke-Jr> CodeShark: probably more important than the gitian versions, we need to support compiling on stable versions of at least Debian, RedHat, Gentoo, etc⦠it's usually RedHat that holds us back
369 2014-10-20 13:14:24 <CodeShark> why do people still use RedHat?
370 2014-10-20 13:14:37 <wumpus> commercial support
371 2014-10-20 13:15:02 <CodeShark> in my experience, the commercial support for RedHat has been inferior to just google searches I do on debian, gentoo, or ubuntu :p
372 2014-10-20 13:15:15 <wumpus> agree on that
373 2014-10-20 13:15:42 <wumpus> but if you're a big company, I guess you can get things done from them
374 2014-10-20 13:16:55 <CodeShark> I guess once you get hooked into them you're stuck with all the legacy crap
375 2014-10-20 13:17:17 <CodeShark> and big companies have too much inertia to retrofit their IT infrastructure
376 2014-10-20 13:18:49 <michagogo> (also, ccache ftw...)
377 2014-10-20 13:18:53 <CodeShark> in any case, huge bureaucracies have other sources of grave inefficiencies that probably dwarf their IT department inefficiencies
378 2014-10-20 13:19:20 <wumpus> right
379 2014-10-20 13:24:21 <ilovebitcoinfr> hi
380 2014-10-20 13:24:40 <ilovebitcoinfr> Which PBKDF2 library does Bitcoin-core Qt app use?
381 2014-10-20 13:25:18 <wumpus> openssl
382 2014-10-20 13:25:51 <ilovebitcoinfr> ok
383 2014-10-20 13:28:50 <michagogo> So all the 2014-10-20 13:28:30 GetNextWorkRequired RETARGET lines that I'm seeing, those are as it fills in its header chain?
384 2014-10-20 13:29:10 <ilovebitcoinfr> wumpus where do I find the code that deals with password storage of the bitcoin reference Qt client?
385 2014-10-20 13:29:14 <michagogo> And then the 2014-10-20 13:28:51 UpdateTip: new best=00000000ce532079f1135377e7b1b4e7f1ae60f793ff5427e8b2510af911b391 height=1237 log2_work=42.273818 tx=1261 09:13:22 progress=0.000013 cache=1212...
386 2014-10-20 13:29:29 <michagogo> lines are as it fetches those blocks from peers?
387 2014-10-20 13:29:43 <ilovebitcoinfr> i'm searching the github repo but it's full of "password" keyword
388 2014-10-20 13:36:00 <ilovebitcoinfr> i guess it's in src/crypter.cpp/.h
389 2014-10-20 13:37:36 <michagogo> What does the "UpdateTip: new best" logline mean now with headers-first?
390 2014-10-20 13:37:59 <blast_> man supressing those boost warnings
391 2014-10-20 13:38:43 <michagogo> That a block has been processed, validated, and applied to chainstate?
392 2014-10-20 13:39:12 <gribble> 326167
393 2014-10-20 13:39:12 <michagogo> ;;blocks
394 2014-10-20 13:41:13 <michagogo> And what's the cache= item? I first thought it might have been blocks fetched from peers and waiting to be processed, but then I realized that it was climbing too quickly for that to make sense...
395 2014-10-20 13:41:26 <michagogo> (and then it passed the length of the blockchain)
396 2014-10-20 13:51:55 <michagogo> Hm, what are your thoughts on putting out an alert for versions prior to getheaders, to notify them that they will actually (finally) be completely unable to connect to 0.10?
397 2014-10-20 13:52:15 <michagogo> (I'm only half-serious there :P )
398 2014-10-20 13:53:19 <michagogo> Hm, impressive -- testnet is at progress=0.765438 after just about 10 minutes
399 2014-10-20 14:07:02 <wumpus> cache= is the number of entries in the utxo (ccoinsview) cache in memory
400 2014-10-20 14:07:25 <ekafeman42> can someone help me if they have a min to spare? Trying to send a raw txn using testnet; I'm I've concatenated the version, inputs etc and sha(sha256(hex_data)) ... how do I go about signing this with private key. I can share the privkey if req since it's testnet
401 2014-10-20 14:27:07 <blast_> signrawtransaction
402 2014-10-20 14:41:41 <JonCharge> HeHellodsd
403 2014-10-20 14:42:31 <JonCharge> Hello, I would like to have 100 BTC (no value) of testnet coins. Is there someone who can perhaps sign that message? I don't expect to return them after product devo.
404 2014-10-20 14:42:41 <JonCharge> Thank you in advance.
405 2014-10-20 14:44:58 <JonCharge> I have to check some things, I will return in a bit to follow up.
406 2014-10-20 17:21:39 <Zarlboro_> Can someone help me to read the leveldb index : blocks/index. I try to get the position of a transaction in blok0000$.dat. I managed to read the value with something like leveldb.get('t' + tx_hash), but I don't know how to deserialize this value to get the block offset and the tx offset..
407 2014-10-20 17:31:08 <michagogo> Zarlboro_: do you have txindex on?
408 2014-10-20 17:31:44 <michagogo> (I can't help you, but I figure I may as well get the first thing that many people get wrong out of the way to avoid confusion if someone does try to help)
409 2014-10-20 17:32:23 <Zarlboro_> @michagogo: yes! I managed to get the value from the index, but don't know how to read it.
410 2014-10-20 17:32:38 <Zarlboro_> Seems the first byte is the file num
411 2014-10-20 17:32:43 <michagogo> Zarlboro_: Okay, great. Good luck!
412 2014-10-20 17:32:58 <Zarlboro_> thank you!
413 2014-10-20 17:33:02 <michagogo> I assume you already tried source-diving and seeing how Bitcoin Core does it?
414 2014-10-20 17:34:20 <Zarlboro_> yeah I try to find the information in source, find this https://github.com/bitcoin/bitcoin/blob/84d13eef883769451ba9f77b56d9738d24474d5c/src/main.h#L209
415 2014-10-20 17:35:04 <Zarlboro_> but I'am not familiar at all with C and don't know how to translate this structure in Python
416 2014-10-20 17:43:26 <michagogo> C++*
417 2014-10-20 17:44:21 <michagogo> That's the header file. I don't know exactly what headers are for, but AIUI each header file has a corresponding source file, ending in .cpp rather than .h
418 2014-10-20 17:53:00 <Zarlboro_> @michagogo: yes I know I also search in main.cpp and just find this: https://github.com/bitcoin/bitcoin/blob/84d13eef883769451ba9f77b56d9738d24474d5c/src/main.cpp#L1637 (but don't help mor :-(
419 2014-10-20 18:03:34 <sipa> Zarlboro_: why do you need to read the transaction index from another program?
420 2014-10-20 18:03:42 <sipa> leveldb is not intended to be shared with other application
421 2014-10-20 18:04:14 <Zarlboro_> because RPC are too much slow..
422 2014-10-20 18:06:48 <Zarlboro_> @sipa: I need to parse 50000 blocks as fast is possible and with RPC this take several days
423 2014-10-20 18:07:42 <sipa> use the P2P protocol
424 2014-10-20 18:08:01 <Zarlboro_> I would to stop bitcoind, parse what I need, then restart bitcoind..
425 2014-10-20 18:08:29 <sipa> there's also linearize.py, which just reads the blocks from disk directly, without using the index
426 2014-10-20 18:08:40 <sipa> you can reuse much of that code probably
427 2014-10-20 18:09:06 <Zarlboro_> @sipa: network operations will take also a lot of time! like RPC
428 2014-10-20 18:09:17 <sipa> not at all
429 2014-10-20 18:09:24 <sipa> the p2p code is way way more optimized than rpc
430 2014-10-20 18:09:49 <sipa> i mean, you can downloads blocks at several MB/s over p2p...
431 2014-10-20 18:10:47 <Zarlboro_> I need a direct very fast access, I doubt than p2p can be fast like direct file access, even if the code it's optimized
432 2014-10-20 18:11:08 <sipa> ok
433 2014-10-20 18:11:27 <sipa> you don't need the index really, if you just need the blocks
434 2014-10-20 18:11:34 <Zarlboro_> but I don't have an internet connection with several MB/s ;-)
435 2014-10-20 18:11:50 <sipa> well you can access your local bitcoind over p2p
436 2014-10-20 18:11:56 <Zarlboro_> I don't need the block, I need the transactions
437 2014-10-20 18:12:06 <sipa> the transactions are in the blocks
438 2014-10-20 18:12:17 <sipa> or you need to access individual transactions by txid?
439 2014-10-20 18:12:34 <Zarlboro_> What I need is to make a lot of getrawtransaction
440 2014-10-20 18:12:42 <sipa> ok
441 2014-10-20 18:12:50 <Zarlboro_> yep with the tx hash
442 2014-10-20 18:13:04 <sipa> have a look at txdb.h/.cpp
443 2014-10-20 18:13:50 <sipa> the keypairs are 't' + txid -> cdisktxpos entries
444 2014-10-20 18:15:51 <sipa> a cdisktxpos entry is encoded as 3 varints: the file number, the byte position in that file where the block starts, and the byte posiiton in that block where that transaction starts (after the header)
445 2014-10-20 18:16:09 <Zarlboro_> yepn I manage to read the value with key 't' + txid
446 2014-10-20 18:16:36 <sipa> the varints use a special encoding, see GetSizeOfVarint, ReadVarint, WriteVarInt in serialize.h
447 2014-10-20 18:19:50 <Zarlboro_> ReadVarint that was the code I was searching! :-)) thank you very much sipa!!
448 2014-10-20 19:51:57 <coryfields_> jtimon: ping
449 2014-10-20 19:52:12 <jtimon> pong
450 2014-10-20 19:53:12 <coryfields_> jtimon: reading #5100, I'm a bit confused about your intentions with core/*
451 2014-10-20 19:53:24 <coryfields_> i was with you up until the core_read/core_write movement into there
452 2014-10-20 19:53:55 <jtimon> well, the initial intention was to have a minimal core/transaction and core/block
453 2014-10-20 19:54:25 <coryfields_> yes, that makes sense for sure
454 2014-10-20 19:54:32 <jtimon> but I thought putting the rest of the core stuff in the folder would be good, I can leave that for another PR
455 2014-10-20 19:55:33 <sipa> i guess part is how to organize the sourcre tree: by layer, or by "knowledge domain"
456 2014-10-20 19:55:37 <coryfields_> jtimon: i like the idea of having base strucures in core/, that makes for a very clear distinction. But adding utility stuff there defeats the purpose imo
457 2014-10-20 19:56:03 <sipa> if you want it by layer, script/script should move to core as well
458 2014-10-20 19:56:12 <sipa> and core/io doesn't belong there
459 2014-10-20 19:56:18 <jtimon> I'll leave io.h read.cpp and write.cpp out, no problem
460 2014-10-20 19:56:43 <coryfields_> sipa: i'd agree with moving script/script there, yea
461 2014-10-20 19:56:54 <sipa> coryfields_: so you follow the layer approach - yes, i like that more too
462 2014-10-20 19:56:57 <jtimon> core/script/script ?
463 2014-10-20 19:57:10 <sipa> just core/script
464 2014-10-20 19:57:19 <sipa> uint should also move to core then
465 2014-10-20 19:57:25 <sipa> and serialize
466 2014-10-20 19:57:37 <jtimon> yeah, I mean, core/script/script.h, core/script/interpreter ...
467 2014-10-20 19:57:46 <sipa> no, interpreter is not part of core in that model
468 2014-10-20 19:57:57 <coryfields_> sipa: agree on serialize. uint and friends is where it starts getting hazy to me
469 2014-10-20 19:58:11 <jtimon> mhmm, I think I prefer script.h with intepreter then
470 2014-10-20 19:58:35 <sipa> coryfields_: well, i still want to separate uint256 for the math operations on it
471 2014-10-20 19:58:43 <jtimon> anyway, I'll just move transaction and block to core in that PR
472 2014-10-20 19:58:46 <sipa> jtimon reasons more per knowledge domain than per layer :)
473 2014-10-20 19:58:56 <coryfields_> sipa: because then hash starts getting involved as well. if we could split that as we discussed a while back, i'd easily agree
474 2014-10-20 19:59:42 <coryfields_> sipa: ok good, sounds like we're aligned then. structures in one domain, functionality above it
475 2014-10-20 19:59:55 <sipa> it's crosscutting
476 2014-10-20 20:01:55 <coryfields_> jtimon: sounds good
477 2014-10-20 20:02:50 <coryfields_> jtimon: amount seems to make sense as well, but i haven't looked at that in detail yet
478 2014-10-20 20:04:14 <jtimon> it's just moving amount constants (COIN, MAX_MONEY, etc) and CFeeRate there
479 2014-10-20 20:04:31 <jtimon> I didn't like it in core/transaction
480 2014-10-20 20:05:08 <jtimon> though script/interpreter will still be importing it unnecessarily
481 2014-10-20 20:05:31 <jtimon> but I thought CFeeRate was small enough to not care
482 2014-10-20 20:06:02 <sipa> yeah, let's not bother with those for now
483 2014-10-20 20:15:32 <michagogo> I wish ccache had some concept of "older versions", basically letting you say "if I compile a newer version of this file discard the old"
484 2014-10-20 20:18:07 <coryfields_> michagogo: agreed, i'd like that as well. X versions, though.
485 2014-10-20 20:18:32 <michagogo> Yeah, makes sense
486 2014-10-20 20:18:41 <coryfields_> michagogo: at one point i started on a script to try to tie the current cache dir to my current git branch
487 2014-10-20 20:18:51 <michagogo> (also, damn, I was kinda hoping for a "Well, actually..." :-P )
488 2014-10-20 20:18:53 <coryfields_> got distracted though
489 2014-10-20 20:19:11 <michagogo> coryfields_: hm?
490 2014-10-20 20:19:16 <michagogo> Ah, just to be more organized?
491 2014-10-20 20:19:40 <michagogo> I was thinking more in the name of saving disk space without completely losing the benefits of ccache
492 2014-10-20 20:19:49 <coryfields_> michagogo: git checkout master && make && git checkout topic && make && git checkout master && make
493 2014-10-20 20:20:11 <coryfields_> that way master would have its own cache, and topic would have its own. after switching, both would remake instantly
494 2014-10-20 20:20:20 <michagogo> coryfields_: Doesn't that work right now?
495 2014-10-20 20:20:37 <michagogo> I mean, wouldn't master and topic's build results both be in the cache?
496 2014-10-20 20:20:41 <coryfields_> yea, but if the cache fills up, it'll discard at random
497 2014-10-20 20:20:44 <michagogo> ah
498 2014-10-20 20:21:05 <michagogo> (do you have CCACHE_COMPRESS or whatever it's called set?)
499 2014-10-20 20:21:44 <coryfields_> pretty sure that's default in recent versions
500 2014-10-20 20:21:49 <coryfields_> not 100, though
501 2014-10-20 20:21:54 <michagogo> coryfields_: also, shouldn't that be s|make|./autogen.sh && ./configure && make|g ?
502 2014-10-20 20:22:27 <michagogo> (maybe not, I honestly don't really have any idea what each of those three things does)
503 2014-10-20 20:22:28 <coryfields_> sure, as necessary. when i'm working on topics, i try to rebase to a recent common ancestor, to avoid the need for that
504 2014-10-20 20:24:20 <michagogo> Well, I mean, that's not completely true -- I know autogen, given a certain set of code, gets configure prepared, and I know that configure looks at the system and prepares the makefile
505 2014-10-20 20:25:00 <michagogo> and that the makefile is sort of like a build script
506 2014-10-20 20:25:14 <michagogo> But beyond that, it's ~all blackbox magic to me
507 2014-10-20 20:25:17 <coryfields_> autogen builds configure. configure builds make, and makes substitutions in make as necessary.
508 2014-10-20 20:25:55 <coryfields_> a makefile is just a series of targets and instructions for building each one. if the dependencies are satisfied for a target, it's not rebuilt
509 2014-10-20 20:26:00 <coryfields_> that's all there is to it :)
510 2014-10-20 20:26:11 <michagogo> Well, I mean, what causes things to change?
511 2014-10-20 20:26:21 <michagogo> What does autogen look at to create configure?
512 2014-10-20 20:26:32 <michagogo> What does configure look at to create the makefile?
513 2014-10-20 20:26:34 <coryfields_> if you change a .h, there are .cpp's that depend on it. so those rebuild
514 2014-10-20 20:26:49 <michagogo> What needs to be regenerated when stuff changes?
515 2014-10-20 20:27:11 <sipa> well you need to rerun autogen whenever configure.ac changes, i guess?
516 2014-10-20 20:27:16 <coryfields_> autogen looks at configure.ac, and the m4's in build-aux. the makefiles are built from the .am and the Makefile.foo.include's
517 2014-10-20 20:27:17 <michagogo> (for varying definitions of stuff)
518 2014-10-20 20:27:38 <michagogo> Okay, and then my next question would be what are those files
519 2014-10-20 20:27:47 <michagogo> And so on
520 2014-10-20 20:27:59 <sipa> configure.ac is the human-readable form of the configure script
521 2014-10-20 20:28:12 <coryfields_> michagogo: the m4's are just split off chunks of configure that many projects can share
522 2014-10-20 20:28:12 <sipa> like the .cpp files are the human-readable form of the produced binaries
523 2014-10-20 20:28:28 <coryfields_> reusable modules
524 2014-10-20 20:28:30 <michagogo> FSVO human :P
525 2014-10-20 20:28:45 <sipa> well, coryfields readable form of the configure script
526 2014-10-20 20:29:02 <coryfields_> michagogo: fyi, the end-result is suppose to be a configure and makefile that require no other tools
527 2014-10-20 20:29:19 <sipa> which is what is packaged in releases
528 2014-10-20 20:29:27 <coryfields_> the autogen stuff is for git builders only. that's why "./configure && make" is pretty universal in the linux world
529 2014-10-20 20:29:37 <michagogo> coryfields_: Right, that's how I guessed that "autogen creates a configure given a certain set of code"
530 2014-10-20 20:29:55 <michagogo> Because I do know that tarballs come with configure
531 2014-10-20 20:29:56 <sipa> autogen does not look at the code
532 2014-10-20 20:30:09 <coryfields_> correct. so end-users never see a changing configure. we rebuild it as it changes in source.
533 2014-10-20 20:30:20 <sipa> well, unless you consider m4 & configure.ac to be 'code' (which it is for some interpretation)
534 2014-10-20 20:30:25 <coryfields_> sipa: i think he meant the configure.ac and m4's
535 2014-10-20 20:30:38 <michagogo> well, I didn't know that
536 2014-10-20 20:30:49 <michagogo> So wait, why is only autogen in git?
537 2014-10-20 20:31:20 <coryfields_> michagogo: run "make dist". it gives you a tarball where autogen has already been run.
538 2014-10-20 20:31:29 <michagogo> coryfields_: that rings a bell
539 2014-10-20 20:31:40 <coryfields_> that can be used by a user who has no auto* stuff installed
540 2014-10-20 20:32:04 <coryfields_> it condenses down to just shell scripts and makefiles
541 2014-10-20 20:32:04 <michagogo> Right
542 2014-10-20 20:32:20 <michagogo> So why do we not have configure in git?
543 2014-10-20 20:32:40 <coryfields_> because configure changes. some projects do that, but it's the wrong solution imo
544 2014-10-20 20:32:42 <sipa> why don't we have bitcoind.exe in git?
545 2014-10-20 20:33:04 <coryfields_> if we did that, then someone with the same version of auto* would have to regenerate the script every time configure.ac was touched, then push a massive set of unreadable changes
546 2014-10-20 20:33:12 <michagogo> Ah
547 2014-10-20 20:33:13 <coryfields_> gcc does that, for example
548 2014-10-20 20:33:23 <coryfields_> iirc they have a buildbot for nothing but that purpose
549 2014-10-20 20:33:26 <michagogo> So configure can be considered "compiled"?
550 2014-10-20 20:33:38 <coryfields_> yep
551 2014-10-20 20:33:40 <michagogo> and so can the makefile configure makes?
552 2014-10-20 20:33:48 <coryfields_> yep
553 2014-10-20 20:33:50 <michagogo> s/makes/creates/ to avoid confusion
554 2014-10-20 20:33:56 <coryfields_> you can edit both by hand, but it really really sucks
555 2014-10-20 20:33:57 <michagogo> Ah