1 2015-06-17 00:52:25 <sipa> phantomcircuit: it will show -1 confirms if a tx cannot enter the mempool afaik
  2 2015-06-17 00:52:48 <sipa> the malleability stuff has some extra conflict tracking too
  3 2015-06-17 00:52:50 <sipa> afaik
  4 2015-06-17 01:02:33 <phantomcircuit> sipa, hmm
  5 2015-06-17 01:02:54 <phantomcircuit> that doesn't seem too bad since this is by defualt going to be 48 hours worth of blocks
  6 2015-06-17 01:03:27 <phantomcircuit> so if you attempt to broadcast a transaction that is linked to your wallet but which the mempool doesn't think is going to be mined for 48 hours it doesn't appear
  7 2015-06-17 01:03:33 <phantomcircuit> or rather appears as -1
  8 2015-06-17 01:07:32 <Diablo-D3> http://autoblopnik.com/2015/06/11/stockholders-demand-child-friendly-tesla/
  9 2015-06-17 01:08:30 <phantomcircuit> Diablo-D3, wildly offtopic
 10 2015-06-17 01:08:39 <Diablo-D3> yeah that was meant for #bitcoin
 11 2015-06-17 03:37:13 <Luke-Jr> phantomcircuit: -1 is supposed to indicate conflicted
 12 2015-06-17 03:54:57 <phantomcircuit> Luke-Jr, but why do we even care?
 13 2015-06-17 03:55:01 <phantomcircuit> it's unconfirmed right?
 14 2015-06-17 03:56:09 <gmaxwell> phantomcircuit: we shouldn't author transactions we won't relay.
 15 2015-06-17 04:02:15 <Luke-Jr> phantomcircuit: it's unconfirmABLE
 16 2015-06-17 04:02:43 <Luke-Jr> gmaxwell: could be left in wallet from other sw..
 17 2015-06-17 04:03:49 <phantomcircuit> Luke-Jr, if it's in conflict with a transaction that is actually in the blockchain then we should handle that
 18 2015-06-17 04:04:29 <phantomcircuit> gmaxwell, right so we need to guess what kind of fees we would need before we were willing to relay the transaction
 19 2015-06-17 04:04:36 <Luke-Jr> phantomcircuit: that is what -1 is supposed to indicate
 20 2015-06-17 04:13:39 <phantomcircuit> Luke-Jr, so the mempool interaction is really only to determine what the fee rate should be
 21 2015-06-17 06:23:18 <wumpus> also to find out if a transaction conflicts with one in the mempool, e.g. due to mutability, double spends, that's why there is the "Conflicted" transaction state
 22 2015-06-17 06:25:13 <jonasschnelli> fanquake: non, did not test #6284 yet.
 23 2015-06-17 06:25:27 <Luke-Jr> imo if it shows confirmations -1 when the tx is valid, it is a bug
 24 2015-06-17 06:29:26 <wumpus> Luke-Jr: oh? what if e.g. another variant or a double spend of the transaction ends up in the block chain? the other transactions that spend its inputs will get confirms=-1
 25 2015-06-17 06:47:34 <Luke-Jr> wumpus: if another is in the blockchain it is invalid and -n makes sense
 26 2015-06-17 06:55:07 <wumpus> right
 27 2015-06-17 06:55:22 <wumpus> yes, you mean 'valid' in that way, then I agree
 28 2015-06-17 07:12:05 <wumpus> I'm going to tag rc2 in a jiffy, working on updating the release note
 29 2015-06-17 07:18:51 <jonasschnelli> wumpus: the release notes are missing full retina/HiDPI support for mac as well as some icon changes. But not sure if this is required. And because my english poor i not gonna try to add a PR. :)
 30 2015-06-17 07:19:40 <wumpus> jonasschnelli: ok, that's no reason for holding up rc2, if you change your mind (or you can get someone else to write it) you still have until -final to submit that
 31 2015-06-17 07:20:27 <jonasschnelli> yeah. Sure. Let me ask fanquake.
 32 2015-06-17 07:43:55 <jonasschnelli> Would it really be a stupid idea to add a flag (returnraw=<boolean> or similar) for sendtoaddress / sendmany?
 33 2015-06-17 07:44:21 <jonasschnelli> I see a lot of complaining that there is no way to get the fee before sending.
 34 2015-06-17 07:44:53 <jonasschnelli> funraw would be a solution, but its more for experienced users IMO.
 35 2015-06-17 07:45:27 <wumpus> isn't that the purpose of fundrawtransaction?
 36 2015-06-17 07:45:37 <wumpus> let's not complicate this further
 37 2015-06-17 07:45:40 <jonasschnelli> with the returnhex flag for send* it would be very easy to see the fee (return {hex: "<txhex>", fee: <value>}) and send it over sendraw*
 38 2015-06-17 07:46:11 <wumpus> well there is the flag to not broadcast transactions (but it's not meant for that)
 39 2015-06-17 07:46:28 <wumpus> anyhow - the problem has always been that the inputs need to remain locked, or released
 40 2015-06-17 07:46:56 <wumpus> a whole discussion came after that, and it was decided to do fundrawtransaction intead
 41 2015-06-17 07:46:58 <jonasschnelli> arg. Right. The locking. Didn't tought about that.
 42 2015-06-17 07:47:01 <wumpus> so let's focus on getting that merged.
 43 2015-06-17 07:47:15 <jonasschnelli> wumpus: right.
 44 2015-06-17 07:47:30 <jonasschnelli> agreed that this is the better way.
 45 2015-06-17 07:49:15 <wumpus> so the other alternative was indeed to make send* transactions lock the inputs and return an information tuple, then have sendrawtransaction as well as a canceltransaction
 46 2015-06-17 07:50:14 <wumpus> a "two phase commit" send... but managing the cancels can get thorny, if people forget the funds will remain locked, that needs extra micro-management calls etc... (or expiration, which comes with other reliability issues)
 47 2015-06-17 07:55:50 <wumpus> the advantage would be that this is more like the GUI (with a confirmation box with info), but for the GUI the flow is more constricted ie the user cannot do anything else but click OK for Cancel, and if they shut down the client the transaction is cancelled.  For remote control this there is more flexibility, but also more scope for error.
 48 2015-06-17 08:00:22 <wumpus> e.g one option there would be to have in the wallet a persistent outbox, like in mail programs, with still-unsent transactions that have to be confirmed, edited or rejected
 49 2015-06-17 08:00:34 <wumpus> ...but fundrawtransaction is more realistic for now :)
 50 2015-06-17 09:32:29 <sipa> Luke-Jr, wumpus, phantomcircuit: i think we also show -1 confirmations if a tx can't enter the mempool because it conflicts with another mempool transaction
 51 2015-06-17 09:33:39 <fanquake1> jonasschnelli Ask me what?
 52 2015-06-17 09:34:19 <jonasschnelli> fanquake1: if you could write something for the releasnotes about the HiDPI/retina support and the new icons.
 53 2015-06-17 09:35:02 <jonasschnelli> fanquake1: 0.11 feels different in terms of using the UI and therefore i think would be cool to mention that in the release-notes
 54 2015-06-17 09:35:52 <fanquake1> ah no worries. I can do, just not right this second.
 55 2015-06-17 09:40:26 <jonasschnelli> fanquake1: no hurry.
 56 2015-06-17 09:51:36 <wumpus> sipa: yes, indeed
 57 2015-06-17 09:54:52 <jonasschnelli> github down?
 58 2015-06-17 09:55:14 <fanquake1> Not from here.
 59 2015-06-17 09:56:43 <jonasschnelli> strange. Down (ping timeout) from switzerland. Successfull ping if i take my german line.
 60 2015-06-17 10:00:05 <wumpus> loads here, although it feels slower than normal
 61 2015-06-17 10:01:16 <fanquake1> wumpus what else do you plan on merging before rc2? Can test a few more pulls now.
 62 2015-06-17 10:01:37 <wumpus> fanquake1: nothing, I'm doing a final verification gitian build now
 63 2015-06-17 10:03:34 <wumpus> fanquake1: but you're always welcome to test pulls of course,for further on
 64 2015-06-17 10:04:47 <wumpus> for example fundrawtransaction (#6088)
 65 2015-06-17 10:08:23 <wumpus> unless there's some very important fix that I overlooked and needs to be backported to 0.11, then let me know asap
 66 2015-06-17 10:09:57 <fanquake> wumpus will do, after a gitian building tonight.
 67 2015-06-17 10:13:40 <cadenadelabloque> Anyone home?
 68 2015-06-17 10:15:39 <fanquake> cadenadelabloque There is always people idling. If you have a question just ask.
 69 2015-06-17 10:16:01 <cadenadelabloque> I've been doing some reading up on zero knowledge contingent payments,  computational complexity theory, PSPACE(interactive proof systems), secure multiparty computations, and bitcoin contracts in general and I'm in need of some help from someone with working knowledge of them.
 70 2015-06-17 10:16:26 <cadenadelabloque> From what I can tell, gmaxwell (who proposed CoinSwap) is likely to not only understand these concepts but have actually written code for them, but I can't seem to find any examples or samples online anywhere from him or anyone else, only theories and discussions.
 71 2015-06-17 10:16:54 <fanquake> You might be better off asking in #bitcoin-wizards
 72 2015-06-17 10:17:28 <cadenadelabloque> I didn't realize that was a thing...I don't know why I would be surprised though.
 73 2015-06-17 10:17:39 <cadenadelabloque> Thank you
 74 2015-06-17 10:18:53 <sipa> wumpus: cool, i have been a bit away; i'll have a look at the diff between rc1 and rc2
 75 2015-06-17 10:22:58 <wumpus> sipa: thanks
 76 2015-06-17 10:26:02 <midnightmagic> :-)
 77 2015-06-17 10:26:23 <midnightmagic> cadenadelabloque: Apologies, I didn't know you were directed to -wizards
 78 2015-06-17 10:28:10 <cadenadelabloque> No worries
 79 2015-06-17 10:28:44 <sipa> cadenadelabloque: also, gmaxwell is likely asleep now
 80 2015-06-17 10:28:50 <cadenadelabloque> I'm quite desperate as this is quite a difficult topic to grasp (for most I bet!) much less code around, and I feel absolutely helpless.
 81 2015-06-17 10:30:57 <cadenadelabloque> To be honest, I also feel like people who understand this topic are probably all too smart and too busy to help!
 82 2015-06-17 10:32:33 <cadenadelabloque> I'll try to idle as much and as often as possible from here on, but it would help if I knew who I was waiting for. I guess gmaxwell is the best place to start. Is he someone as busy as I think he is, and probably doesn't see PMs on IRC? Should I email his github email address or just PM and idle here?
 83 2015-06-17 11:08:54 <xabbix> Is there a way to tell bitcoin-core to show IP addresses instead of peer ids when logging new txs?
 84 2015-06-17 11:11:41 <paveljanik> ./bitcoind --help | grep ips
 85 2015-06-17 11:12:08 <paveljanik> --logips
 86 2015-06-17 11:12:33 <paveljanik> Please ask on general channel - #bitcoin.
 87 2015-06-17 11:15:10 <xabbix> paveljanik, not sure why that's not a bitcoin dev question, but thanks for the reply
 88 2015-06-17 11:16:25 <wumpus> it doesn't relate to development but to usage, so it's a user question (not that I mind you asking it here, but it would be more appropriate in #bitcoin)
 89 2015-06-17 12:27:11 <wumpus> * [new tag]         v0.11.0rc2 -> v0.11.0rc2
 90 2015-06-17 13:13:16 <Luke-Jr> wumpus: you wont *find* it, its nonredistributable ☹
 91 2015-06-17 13:14:08 <thrasher`> yeah :(
 92 2015-06-17 13:14:32 <sipa> wumpus: does the version of the SDK matter?
 93 2015-06-17 13:14:39 <sipa> i assume it does
 94 2015-06-17 13:14:42 <sipa> but did it change for 0.11?
 95 2015-06-17 13:15:44 <sipa> i have 10.7
 96 2015-06-17 13:16:04 <thrasher`> yeah it requires MacOSX10.9.sdk.tar.gz
 97 2015-06-17 13:16:37 <sipa> ok, no OSX build from me then
 98 2015-06-17 13:16:49 <thrasher`> before you could download Xcode DMG, mount it and get it from that
 99 2015-06-17 13:17:21 <Diablo-D3> what, is that the xcode cli tools?
100 2015-06-17 13:17:32 <Diablo-D3> homebrew knows how to install those without installing xcode
101 2015-06-17 13:20:29 <thrasher`> maybe you can do it that way, I just followed the instructions in the release-process documentation
102 2015-06-17 13:20:36 <midnightmagic> holy 2.5GB download batman
103 2015-06-17 13:21:14 <wumpus> thrasher`: many people have tried, no one succeeded AFAIK
104 2015-06-17 13:21:58 <wumpus> so unless you have some special extraction tool no one else knows of...
105 2015-06-17 13:22:07 <jonasschnelli> '4c1ad2d088771c3b7f0d2d3abe74bc34562109c0d5c9fb577922d613ba333ec9  bitcoin-0.11.0-linux32.tar.gz
106 2015-06-17 13:22:09 <jonasschnelli> match?
107 2015-06-17 13:22:39 <wumpus> yes matches 0.11.0rc2-linux/laanwj/bitcoin-linux-0.11-build.assert
108 2015-06-17 13:22:59 <sipa> i get lxc-start errors
109 2015-06-17 13:23:01 <wumpus> midnightmagic: hey, it's smaller than the block chain :)
110 2015-06-17 13:23:31 <sipa> hmm it's doing something at least
111 2015-06-17 13:23:44 <midnightmagic> wumpus: internally, everything I download for the last three years or so I compare with a blockchain download :)
112 2015-06-17 13:23:49 <wumpus> sipa: strange. it worked for rc1? gitian-wise, nothing should have changed
113 2015-06-17 13:24:47 <wumpus> midnightmagic: but it's a bit sad, yes, having to download 2.5GB of applecrap just to extract, I don't know, 100kB of headers and libraries
114 2015-06-17 13:24:50 <sipa> i didn't build rc1
115 2015-06-17 13:26:28 <midnightmagic> wumpus: it's cool, I use xcode anyway, I'm just a bit ahead on 6.3.2.
116 2015-06-17 13:28:16 <thrasher`> wumpus: to obtain the 10.9 SDK, I installed Xcode 6.1.1 and updated it, then found the required SDK in /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/
117 2015-06-17 13:40:18 <wumpus> thrasher`: yes, but that only works if you have a mac
118 2015-06-17 13:51:46 <wumpus> sipa: any luck now?
119 2015-06-17 13:53:00 <sipa> no
120 2015-06-17 13:53:08 <sipa> i downgraded gitian, and it gets further
121 2015-06-17 13:53:18 <sipa> at least to the building phase
122 2015-06-17 13:53:25 <sipa> but run on-target fails now
123 2015-06-17 13:53:35 <wumpus> hmm
124 2015-06-17 13:53:44 <wumpus> what error?
125 2015-06-17 13:54:11 <sipa> ./bin/gbuild:21:in `system!': failed to run on-target -u root bash < target-bin/upgrade-system.sh > var/install.log 2>&1 (RuntimeError)
126 2015-06-17 13:54:17 <sipa> oh, no it didn't get to building
127 2015-06-17 13:54:49 <wumpus> how does install.log end?
128 2015-06-17 13:55:41 <sipa> hold on, retrying it
129 2015-06-17 13:57:02 <sipa> it succeeding in upgrading some backages, but now install.log only contains "stdin: not a tty"
130 2015-06-17 13:57:32 <h3xc0d3r> /msg chanserv access #CHANNEL list
131 2015-06-17 13:58:28 <sipa> wumpus: and there it ends
132 2015-06-17 13:59:00 <wumpus> hm, the not a tty should not be fatal
133 2015-06-17 13:59:22 <sipa> maybe it isn't
134 2015-06-17 13:59:47 <wumpus> but the upgrades are always a fragile part, I've had to regenerate my image at least once to avoid problems with some package upgrade :/ but that doesn't always work either.
135 2015-06-17 14:00:25 <sipa> packages upgrades succeeded
136 2015-06-17 14:00:39 <sipa> but suddenly the install.log was overwritten with that error
137 2015-06-17 14:00:51 <wumpus> right. So that's all install.log should do. After that it continues in build.log :)
138 2015-06-17 14:05:32 <helo> corrupted db on reindex twice in a row reported in #bitcoin by gjanssens - log http://pastebin.ca/3030085
139 2015-06-17 14:05:58 <helo> err block checksum mismatch
140 2015-06-17 14:13:18 <wumpus> that's a leveldb error, looks like either the block database or utxo set db gets corrupted on disk
141 2015-06-17 14:42:38 <michagogo> Hm
142 2015-06-17 14:43:18 <michagogo> cfields: is there now a way to check in a script whether the sigs for a given version are available?
143 2015-06-17 14:44:21 <cfields> michagogo: you can just watch the git repo
144 2015-06-17 14:45:10 <michagogo> cfields: I want a command that will return true or false depending on whether they're available
145 2015-06-17 14:45:21 <michagogo> Er, non-/zero I guess
146 2015-06-17 14:45:28 <michagogo> Before, that command was ever
147 2015-06-17 14:45:32 <michagogo> Wget*
148 2015-06-17 14:46:09 <cfields> michagogo: you can still use wget, then. just try to grab the tag from github and see if it 404's
149 2015-06-17 14:46:27 <michagogo> Specifically, I want the ability to automatically skip the signing if they're not there
150 2015-06-17 14:46:40 <michagogo> Ah, really? Cool. What's the URL?
151 2015-06-17 14:46:45 <michagogo> (On my phone ATM...)
152 2015-06-17 14:47:17 <cfields> not sure yet, this is the first time building this way. I'll give it to you after the fact
153 2015-06-17 15:03:51 <michagogo> Okay
154 2015-06-17 15:03:59 <michagogo> So can someone test this for me?
155 2015-06-17 15:04:35 <michagogo> $ wget -O /dev/null https://github.com/bitcoin/bitcoin/archive/v0.11.0rc2.tar.gz && echo true || echo false
156 2015-06-17 15:04:45 <michagogo> That should be true
157 2015-06-17 15:04:55 <michagogo> And this should be false: $ wget -O /dev/null https://github.com/bitcoin/bitcoin/archive/v0.11.0rc3.tar.gz && echo true || echo false
158 2015-06-17 15:05:04 <wumpus> it outputs true
159 2015-06-17 15:05:15 <michagogo> (Or is there a way to tell wget not to actually download?)
160 2015-06-17 15:05:16 <wumpus> works
161 2015-06-17 15:05:33 <michagogo> wumpus: thanks
162 2015-06-17 15:05:47 <wumpus> maybe you can make it do a HEAD request? I don't know
163 2015-06-17 15:05:55 <michagogo> On the bus now, should be home in 15-20 mins and I can start building
164 2015-06-17 15:06:50 <morcos> jonasschnelli: did you see my issue regarding JSON?  i'm not sure how to make the smartfees.py test work again, and was hoping someone who knew more about python and JSON could point me to how to work around the issue
165 2015-06-17 15:07:40 <wumpus> morcos: what's wrong with it?
166 2015-06-17 15:08:01 <morcos> the python code automatically turns 0.00001 into 1e-5
167 2015-06-17 15:08:12 <morcos> which used to be accepted, but now won't be
168 2015-06-17 15:08:23 <morcos> or anything with 4 or more leading 0's
169 2015-06-17 15:09:17 <wumpus> whoa it shouldn't do that, looks like the JSON parser is no longer using Decimal?
170 2015-06-17 15:09:28 <morcos> i assume its intentional to not have 1e-5 be valid JSON any more, and the fix is to make the python code not output it
171 2015-06-17 15:09:32 <wumpus> (at the Python side)
172 2015-06-17 15:09:33 <wumpus> right.
173 2015-06-17 15:09:44 <morcos> the python code always outputted scientific notation for values below .0001
174 2015-06-17 15:10:00 <wumpus> oh, that's a bug :)
175 2015-06-17 15:10:18 <morcos> btw, .0001 is also not valid, has to be 0.0001 now
176 2015-06-17 15:10:28 <wumpus> yes
177 2015-06-17 15:11:34 <morcos> yeah ajweiss took a quick look at the python code last night and it was immediately apparent i don't think how to change it to give us the right thing
178 2015-06-17 15:11:36 <wumpus> well if it is allowed JSON syntax, then it needs to be fixed server side too
179 2015-06-17 15:11:54 <morcos> wasn't
180 2015-06-17 15:12:12 <michagogo> wumpus: looks like --spider?
181 2015-06-17 15:12:14 <wumpus> I mean python's json implementation has existed for almost a decade I wouldn't expect it to generate invalid JSON
182 2015-06-17 15:12:19 <michagogo> Does that work?
183 2015-06-17 15:13:12 <wumpus> the problem is that ParseMoney(), which we now use to handle fixed-point values, doesn't understand either scientific notation or .XXX without trailing zero
184 2015-06-17 15:14:04 <morcos> and do we want it to?
185 2015-06-17 15:14:24 <michagogo> Exit status manpage section doesn't indicate that it wouldn't work, but someone mentioned somewhere somewhen that it doesn't work
186 2015-06-17 15:14:25 <wumpus> michagogo: yes, it works
187 2015-06-17 15:14:36 <wumpus> morcos: well if the JSON standard allows it, we want it to
188 2015-06-17 15:14:40 <michagogo> Ah, great
189 2015-06-17 15:14:55 <wumpus> morcos: otherwise other JSON implementations besides python's may get into trouble too
190 2015-06-17 15:15:30 <morcos> ok, yeah it looks pretty valid json to me
191 2015-06-17 15:15:44 <sipa> wait, how is our json code using parsemoney?
192 2015-06-17 15:15:44 <wumpus> I'll open an issue
193 2015-06-17 15:15:57 <sipa> thenjson code cannot know it is a numeric amount
194 2015-06-17 15:15:57 <wumpus> sipa: AmountFromValue
195 2015-06-17 15:16:10 <sipa> hmm
196 2015-06-17 15:16:21 <michagogo> From json.org https://usercontent.irccloud-cdn.com/file/Y2aSQLis/IMG_2694.GIF
197 2015-06-17 15:16:24 <sipa> well AmountFromValue should support whatever json supports
198 2015-06-17 15:16:36 <sipa> ParseMoney not necessarily
199 2015-06-17 15:16:36 <wumpus> yes, it should, but right now it doesn't, that's the issue
200 2015-06-17 15:16:44 <sipa> got it
201 2015-06-17 15:17:11 <wumpus> it used to go through double, was changed to go to fixed-point directly, but this parser apparently doesn't support the full json syntax
202 2015-06-17 15:17:18 <michagogo> So e notation is valid, no leading 0 isn't
203 2015-06-17 15:17:24 <michagogo> If I understand that correctly
204 2015-06-17 15:17:26 <wumpus> thanks michagogo
205 2015-06-17 15:17:51 <wumpus> morcos: how did you find out? does python's JSON generate values without leading 0? or just e-21x?
206 2015-06-17 15:17:57 <morcos> ok so i was wrong on it being the univalue stuff then, sorry jonasschnelli
207 2015-06-17 15:18:06 <michagogo> "Very much like a C or Java number, except that the octal and hexadecimal formats are not used"
208 2015-06-17 15:18:08 <wumpus> univalue just parcels up the JSON into strings
209 2015-06-17 15:18:29 <morcos> it ended up with a small amount, which got converted into scientific notation like 8.775e-5
210 2015-06-17 15:19:06 <morcos> the leading 0 stuff was just something i discovered in trying to pin this down
211 2015-06-17 15:19:06 <sipa> the change was done simultaneously with the univalue conversion
212 2015-06-17 15:19:53 <wumpus> conversion to actual types (with checking) is done at a higher level with get_bool, get_int, and yes ValueToAmount
213 2015-06-17 15:20:36 <wumpus> morcos: ok, clear then
214 2015-06-17 15:23:14 <morcos> yeah ok this makes sense to me now.. the error without the leading 0 was "Error parsing JSON" and the error with sci notation was "Invalid amount"
215 2015-06-17 15:24:27 <wumpus> well this is why we merge large changes like univalue into the branch after a release, so that there's plenty of time to find and fix problems like this
216 2015-06-17 15:25:11 <michagogo> morcos: so that was correct
217 2015-06-17 15:25:22 <morcos> ok well while you and sipa are paying attention, reindex is also broken because of 5975.  sdaftuar pinged jtimon b/c of that.  it might be an easy fix.
218 2015-06-17 15:25:29 <michagogo> A fractional number in json must start with 0., it looks like
219 2015-06-17 15:25:46 <morcos> michagogo: yep, odd that it worked with the old code though
220 2015-06-17 15:25:49 <michagogo> Leading zeroes are not allowed, but in a fraction, the 0 is required
221 2015-06-17 15:26:14 <michagogo> http://stackoverflow.com/questions/26898431/are-decimals-without-leading-zeros-valid-json for more explanation, if anyone wants to understand better
222 2015-06-17 15:27:05 <wumpus> morcos: https://github.com/bitcoin/bitcoin/issues/6297
223 2015-06-17 15:27:49 <cfields> jonasschnelli: osx match
224 2015-06-17 15:28:07 <morcos> wumpus: a slightly more informative version of 6296.  :)  i'll close that one
225 2015-06-17 15:28:17 <wumpus> oh you had opened one too? ok :)
226 2015-06-17 15:29:07 <jtimon> why is the genesis block validated? it is the first rule of the chain and valid by definition
227 2015-06-17 15:29:46 <sipa> there is no need to validate it technically, but it does serve as a minor sanity check
228 2015-06-17 15:29:54 <jtimon> it should be accepted without validation
229 2015-06-17 15:30:33 <wumpus> internal consistency check
230 2015-06-17 15:30:44 <jtimon> I think it serves to spread a lot of consensusParams.hashGenesisBlock to avoid checks in many places
231 2015-06-17 15:31:20 <wumpus> the overhead to validate it should be about zero?
232 2015-06-17 15:31:23 <jtimon> I think avoiding it would help simplify the code
233 2015-06-17 15:31:38 <sipa> that would be a good argument
234 2015-06-17 15:32:02 <wumpus> it would be a good argument, but as I understand you're adding an exception to not validate it, not removing one
235 2015-06-17 15:32:21 <jtimon> it's more complexity for no good reason, no internal consistency is gained by validating it, you're already comparing its hash with a hardcoded value
236 2015-06-17 15:32:41 <jtimon> you're adding one to remove like 8 exceptions
237 2015-06-17 15:33:13 <wumpus> are you sure none of the downstream code needs to be able to handle the genesis block anyway?
238 2015-06-17 15:33:14 <sipa> jtimon: feel free to PR it, if it simplifies the code, i'm in favor
239 2015-06-17 15:33:43 <sipa> wumpus: the genesis block does not reallybexist as part of the chain for consensus reasons
240 2015-06-17 15:33:49 <wumpus> removing exceptions from e.g. consensus code could be dangerous
241 2015-06-17 15:33:55 <sipa> it has 0 impact on thebutxo ser
242 2015-06-17 15:34:06 <wumpus> I know, but I'm very scared of introducing edge cases
243 2015-06-17 15:34:08 <sipa> and in satoshi's code, it was not validated
244 2015-06-17 15:34:17 <jtimon> there's 6 comparisons with hashGenesisBlock in main, there should only be one
245 2015-06-17 15:34:23 <sipa> and afaik, the code i wrote did not either
246 2015-06-17 15:34:35 <sipa> so i don't know where this validation comes from
247 2015-06-17 15:34:43 <wumpus> well it's fine with me, just don't introduce any horrible bugs :)
248 2015-06-17 15:34:55 <sipa> in connectblock there is a big if at the top for specifically avoiding validating it
249 2015-06-17 15:35:10 <morcos> wumpus: words to live by
250 2015-06-17 15:35:46 <sipa> so maybe i should stop judging based on an irc discussion, and check what validation is actually being done, as imho there should be none
251 2015-06-17 15:35:59 <IngCr3at1on> bugs are fun though, tasty too
252 2015-06-17 15:36:06 <jtimon> ok, I think it's the best way to fix what I apparently broke with 5975
253 2015-06-17 15:36:53 <sdaftuar> my recollection is that we only validate the genesis block on reindexing
254 2015-06-17 15:37:21 <sdaftuar> ie starting with a fresh datadir works fine
255 2015-06-17 15:42:02 <jtimon> sdaftuar why validate the genesis block on reindexing? accept it without validating it in reindexing is enough, no? just make sure the first block it's actually the genesis block hardcoded in Consensus::Params::hashGenesisBlock
256 2015-06-17 15:43:05 <sipa> we do need some processing when reindexing, because we do need to create a CBlockIndex entry for it with the actual disk position
257 2015-06-17 15:43:39 <sipa> not validation, though
258 2015-06-17 15:43:41 <sdaftuar> jtimon: i agree.  just explaining the code: InitBlockIndex will add the genesis block without validating it, only if we're not reindexing.  if we're reindexing, we'll reprocess everything on disk
259 2015-06-17 15:45:24 <jtimon> sdaftuar: I see, thanks, that is useful
260 2015-06-17 15:48:24 <jtimon> I see https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L3399
261 2015-06-17 15:49:48 <wumpus> #5975 broke reindexing? okay, that's new to me
262 2015-06-17 15:52:08 <sipa> acceptblockheader was previously never called for the genesis block, and now is, it seems
263 2015-06-17 15:52:47 <wumpus> another example of apparantly trivial refactors introducing bugs
264 2015-06-17 15:53:36 <sdaftuar> sipa: what version of the code are you comparing to?
265 2015-06-17 15:53:53 <wumpus> + we don't have a test for reindexing, probably should have one
266 2015-06-17 15:54:02 <sdaftuar> we do have reindex.py -- we need to enable it in pulltester
267 2015-06-17 15:54:11 <sdaftuar> it's in the "extended" list for some reason
268 2015-06-17 15:54:26 <sdaftuar> (that was how i discovered this issue)
269 2015-06-17 15:54:32 <morcos> wumpus: do you keep up with GitHub via emails?  I've noticed I'm missing lots and lots of comments on the issues/PR's recently
270 2015-06-17 15:55:05 <wumpus> sdaftuar: oh! we do have a test for reindexing, that's interesting, maybe it takes a long time?
271 2015-06-17 15:55:11 <sdaftuar> no it's quite fast :)
272 2015-06-17 15:55:35 <wumpus> sdaftuar: ok just overlooked then, going to add it to the default list
273 2015-06-17 15:56:43 <wumpus> morcos: yes I use email for that - haven't noticed missing anything myself yet, the flood of messages keeps coming in unabated
274 2015-06-17 15:57:30 <morcos> i was just asking b/c both of these were reported yesterday as issues and/or PR comments, but yes of course..  i can't imagine actually keeping up with them all
275 2015-06-17 16:02:02 <wumpus> oh if the question is whether I keep up and read everything... eh, I try to, but have trouble with that :) these days I usually make lists of pulls and issues "merge queues" to keep it somewhat manageable
276 2015-06-17 16:18:43 <michagogo> cfields: will the tag be identically named to the bitcoin tag?
277 2015-06-17 16:18:51 <cfields> yes
278 2015-06-17 16:19:10 <michagogo> Okay, so the canary link should be able to be https://github.com/bitcoin/bitcoin-detached-sigs/archive/v0.11.0rc2.tar.gz
279 2015-06-17 16:20:43 <michagogo> Hm, question is how quickly it goes live
280 2015-06-17 16:20:53 <michagogo> I would hope it happens instantly, but I don't know
281 2015-06-17 16:21:21 <cfields> michagogo: it'd probably be more reliable to just use git directly
282 2015-06-17 16:21:27 <cfields> git fetch origin v0.11.0rc2
283 2015-06-17 16:21:51 <michagogo> cfields: I'm not going to have the repo cloned, though
284 2015-06-17 16:22:08 <cfields> it's cloned in your gitian dir
285 2015-06-17 16:22:31 <michagogo> Hm
286 2015-06-17 16:23:55 <michagogo> Looks like it's not exactly cloned: https://github.com/devrandom/gitian-builder/blob/master/bin/gbuild#L235
287 2015-06-17 16:24:26 <michagogo> It fetches by URL each time
288 2015-06-17 16:24:54 <michagogo> Would it work to do this:
289 2015-06-17 16:25:20 <cfields> michagogo: it fetches
290 2015-06-17 16:25:31 <michagogo> git fetch --update-head-ok https://github.com/bitcoin/bitcoin-detached-sigs/ +refs/tags/v${VERSION}:refs/tags/v${VERSION}
291 2015-06-17 16:25:55 <cfields> something like: GIT_DIR=inputs/bitcoin-detached-sigs/.git/ git fetch origin v0.11.0rc2
292 2015-06-17 16:26:23 <michagogo> cfields: origin doesn't exist, it seems
293 2015-06-17 16:27:09 <michagogo> Would this work? pushd inputs/bitcoin-detached-sigs; git fetch https://github.com/bitcoin/bitcoin-detached-sigs/ +refs/tags/v${VERSION}:refs/tags/v${VERSION}; popd
294 2015-06-17 16:27:29 <cfields> GIT_DIR=inputs/bitcoin-detached-sigs/.git/ git fetch https://github.com/bitcoin/bitcoin.git v0.11.0rc2
295 2015-06-17 16:27:33 <cfields> that one works for me.
296 2015-06-17 16:27:48 <michagogo> Okay, cool
297 2015-06-17 16:27:58 <cfields> need to fix the url obviously
298 2015-06-17 16:27:58 <michagogo> Where does that do with an if !?
299 2015-06-17 16:28:25 <michagogo> if ! GIT_DIR=inputs/bitcoin-detached-sigs/.git/ git fetch https://github.com/bitcoin-detached-sigs/bitcoin.git v${VERSION}
300 2015-06-17 16:28:26 <michagogo> ?
301 2015-06-17 16:28:59 <michagogo> Or would it be: GIT_DIR=inputs/bitcoin-detached-sigs/.git/ if ! git fetch https://github.com/bitcoin-detached-sigs/bitcoin.git v${VERSION}
302 2015-06-17 16:29:02 <michagogo> ?
303 2015-06-17 16:30:03 <cfields> you can just set GIT_DIR separately, or use --git-dir instead
304 2015-06-17 16:39:36 <michagogo> heh, got the url wrong
305 2015-06-17 16:49:24 <michagogo> cfields: hm, just noticed that release-process.md refers to it as an App Store key
306 2015-06-17 16:49:33 <michagogo> When it's not, it's just Developer ID
307 2015-06-17 16:59:06 <immah> list
308 2015-06-17 16:59:44 <immah> hello . . i made an exchange and i was given a hash and password.
309 2015-06-17 17:00:03 <immah> i waited for days and it hasn't arrived and its not in the block chain
310 2015-06-17 17:00:11 <immah> what do i do or need to know
311 2015-06-17 17:00:17 <immah> thanks in advance
312 2015-06-17 17:01:01 <cfields> * [new tag]         v0.11.0rc2 -> v0.11.0rc2
313 2015-06-17 17:01:01 <cfields> To git@github.com:bitcoin/bitcoin-detached-sigs.git
314 2015-06-17 17:01:45 <sipa> immah: #bitcoin
315 2015-06-17 17:02:06 <cfields> michagogo: Developer ID is needed to sign releases for the App Store, so that seems kinda nitpicky
316 2015-06-17 17:02:11 <wumpus> cfields: woohoo, going to test the new macosx release process for the first time
317 2015-06-17 17:02:16 <michagogo> cfields: No, pretty sure it's different
318 2015-06-17 17:02:33 <michagogo> Developer ID, IIRC, is free and is used to sign to make Gatekeeper happy
319 2015-06-17 17:02:53 <michagogo> Signing for the MAS is a different cert, which requires membership in the $99/y dev program
320 2015-06-17 17:04:34 <cfields> michagogo: ah
321 2015-06-17 17:05:49 <michagogo> Does this look right, btw? https://www.irccloud.com/pastebin/F6rObq3V/
322 2015-06-17 17:08:39 <wumpus> 6eb0a118824bfbf3fc37b72d0e4afcf16d5a0b7d092e944cd1247d7ff93e0ebb  bitcoin-osx-signed.dmg
323 2015-06-17 17:09:49 <cfields> wumpus: match
324 2015-06-17 17:10:12 <wumpus> cfields: works great!
325 2015-06-17 17:10:56 <cfields> woohoo, no snags :)
326 2015-06-17 17:14:32 <michagogo> Okay, I have the script adapted (and improved), I think
327 2015-06-17 17:14:51 <michagogo> Now I just need to reboot for these updates, then I can get the build started
328 2015-06-17 17:26:03 <michagogo> uh, what?
329 2015-06-17 17:26:05 <michagogo> Could not download some packages, please run gbuild --upgrade
330 2015-06-17 17:26:15 <michagogo> But looking at the log, nothing failed...
331 2015-06-17 17:48:50 <michagogo> Looks like I fixed it by redoing the upgrade thing
332 2015-06-17 17:48:55 <michagogo> (https://github.com/devrandom/gitian-builder/issues/91#issuecomment-109021479)
333 2015-06-17 17:49:23 <michagogo> Seems to be building properly now
334 2015-06-17 17:55:52 <sipa> wumpus: he did not
335 2015-06-17 18:03:16 <wumpus> cfields: sipa is having weird troubles with lxc
336 2015-06-17 18:03:45 <cfields> oh? sipa: what's the issue?
337 2015-06-17 18:15:26 <michagogo> Okay, going to head towards sleeping now since I need to get up very early tomorrow morning
338 2015-06-17 18:16:17 <michagogo> But the script is running and it just finished building for linux, so hopefully it'll work properly, and the 4 sigs should be created and pushed and PR'd shortly
339 2015-06-17 18:16:27 <devrandom> sipa: I revamped the LXC support in the past month or so
340 2015-06-17 18:17:26 <sipa> devrandom: well it complains about lxc-start now
341 2015-06-17 18:17:41 <sipa> devrandom: lxc-start prints its help message out because of incorrect arguments
342 2015-06-17 18:21:15 <devrandom> uh oh
343 2015-06-17 18:22:03 <devrandom> sipa: I was trying to get correctly choose lxc-start vs lxc-execute based on lxc version, but might have gotten the version threshold wrong
344 2015-06-17 18:22:15 <devrandom> what version do you have?
345 2015-06-17 18:22:31 <devrandom> and, does it work better if you force lxc-execute?  (there's an env var for that)
346 2015-06-17 18:22:36 <sipa> 1.0.6
347 2015-06-17 18:23:29 <devrandom> it should have chosen lxc-execute on the latest
348 2015-06-17 18:23:31 <devrandom> https://github.com/devrandom/gitian-builder/blob/master/libexec/gconfig#L4
349 2015-06-17 18:28:07 <jonasschnelli> wumpus: how did json spirit handle the scientific number notation?
350 2015-06-17 18:28:43 <jonasschnelli> morcos: just tried './src/bitcoin-cli --regtest sendtoaddress mnyc6XVxp238yvV47EkJpuetMvPuiekVaU 0.00001'  worked fine? Is it a python framework issue? does it need a Decimal() wrapping there?
351 2015-06-17 18:29:02 <sipa> 0.00001 will work
352 2015-06-17 18:29:27 <morcos> the problem is python packages 0.00001 up as 1e-5 which should be valid JSON
353 2015-06-17 18:29:42 <jonasschnelli> ah. Okay. I see.
354 2015-06-17 18:29:45 <morcos> but that now won't work from your commandline (and would previously)
355 2015-06-17 18:30:51 <jonasschnelli> so it's just the <num>-<exp><num> issue, like 10e-5?
356 2015-06-17 18:31:15 <morcos> well 10e-5 isn't valid JSON, but yes
357 2015-06-17 18:31:28 <morcos> oh wait, maybe it is
358 2015-06-17 18:31:28 <morcos> sorry
359 2015-06-17 18:31:30 <sipa> it is
360 2015-06-17 18:31:44 <sipa> .10e-3 isn't
361 2015-06-17 18:32:41 <michagogo> ....
362 2015-06-17 18:33:03 <michagogo> that's invalid on multiple levels
363 2015-06-17 18:33:33 <michagogo> First, scientific notation is supposed to be 1 digit before the decimal point
364 2015-06-17 18:33:39 <morcos> see wumpus's comment in 6297.
365 2015-06-17 18:34:00 <michagogo> And also, even without e notation, you still need to have a leading 0 for fractions
366 2015-06-17 18:34:41 <jonasschnelli> 0.10e-3 but .10e-3 isn't?
367 2015-06-17 18:34:52 <sipa> indeed
368 2015-06-17 18:34:57 <gmaxwell> bleh
369 2015-06-17 18:34:59 <michagogo> I don't know if 0.10e-3 is valid json
370 2015-06-17 18:35:06 <michagogo> it's not valid scientific notation, though
371 2015-06-17 18:35:14 <michagogo> should be 1e-2, I think
372 2015-06-17 18:35:16 <jonasschnelli> thanks to wumpus we have a dummy save gfx (https://github.com/bitcoin/bitcoin/issues/6297)
373 2015-06-17 18:36:36 <sipa> michagogo: it is valid
374 2015-06-17 18:36:38 <morcos> well, i wouldnt go that far, who wants 100E-00007
375 2015-06-17 18:36:41 <jonasschnelli> morcos: and json spirit, or lets say before we had univalue, did handle 10e-3 fine?
376 2015-06-17 18:36:54 <michagogo> morcos: erm, again, not canonical scientific notation
377 2015-06-17 18:37:03 <michagogo> Should be 1e-5
378 2015-06-17 18:37:04 <morcos> yes, but appears to be legit json
379 2015-06-17 18:37:32 <sipa> michagogo: who cares about scientific notation? the json standard is what matters
380 2015-06-17 18:37:39 <michagogo> sipa: yeah, I know
381 2015-06-17 18:37:47 <sipa> and 100e-00000 seems valid
382 2015-06-17 18:38:34 <michagogo> jsonlint does say [100e-00000] is valid json
383 2015-06-17 18:45:02 <jonasschnelli> hmmm... look at: https://github.com/bitcoin/bitcoin/blob/master/src/univalue/univalue_read.cpp#L146
384 2015-06-17 18:45:15 <jonasschnelli> our univalue parser should handle exp.s
385 2015-06-17 18:45:18 <sipa> yes
386 2015-06-17 18:45:31 <sipa> but we don't use the univalue parser in AmountFromValue
387 2015-06-17 18:46:17 <morcos> also looks like we balk at handling trailing 0's
388 2015-06-17 18:46:20 <sipa> that's the awesome feature of UniValue, that it maintains the original string representation
389 2015-06-17 18:46:27 <jonasschnelli> why the hell do we have a AmountFromValue() function?!
390 2015-06-17 18:46:29 <sipa> but we should use it
391 2015-06-17 18:46:42 <sipa> jonasschnelli: because we don't want to pass by floating point representation
392 2015-06-17 18:46:54 <sipa> when going from a JSON encoded amount to a CAmount
393 2015-06-17 18:47:03 <morcos> i could imagine someone writing something that always delivered 8 decimal points, 0.00001000 which doesn't work now and used to
394 2015-06-17 18:47:23 <jonasschnelli> sipa: okay. I see.
395 2015-06-17 18:47:25 <morcos> the question is how many trailing 0's is too many
396 2015-06-17 18:47:43 <sipa> i wonder if we should have AmountFromValue croak if more than 8 (nonzero) decimals are specified
397 2015-06-17 18:47:50 <sipa> because it cannot do exact conversion
398 2015-06-17 18:47:54 <wumpus> sipa: yes
399 2015-06-17 18:48:02 <wumpus> it does, I think
400 2015-06-17 18:48:27 <wumpus> trailing 0 should be no problem?
401 2015-06-17 18:48:59 <wumpus> well, up until 8 digits
402 2015-06-17 18:49:04 <morcos> sorry
403 2015-06-17 18:49:09 <morcos> yeah i forgot the leading 0
404 2015-06-17 18:49:10 <morcos> ha ha
405 2015-06-17 18:49:15 <morcos> it works until 8 and then breaks
406 2015-06-17 18:49:22 <wumpus> it's a very precise parser
407 2015-06-17 18:49:23 <wumpus> right
408 2015-06-17 18:50:15 <sipa> the JSON one should support additional digits as long as they are 0
409 2015-06-17 18:50:16 <wumpus> in principle, 0's after the 8th shouldn't be a problem, but other numbers should be
410 2015-06-17 18:51:01 <gmaxwell> I hate the brittleness of this.
411 2015-06-17 18:51:07 <gmaxwell> not anyone here's fault.
412 2015-06-17 18:51:29 <wumpus> what brittleness? we're finally parsing strings, instead of weird doubles
413 2015-06-17 18:51:42 <wumpus> we can decide exactly what input is accepted
414 2015-06-17 18:51:49 <morcos> yes, i think the JSON is fine with the extra trailing 0's, it doesn't error parsing JSON, it says invalid amount
415 2015-06-17 18:51:59 <gmaxwell> I know, it's just hard to make definitions that don't result in surprising brittleness. :)
416 2015-06-17 18:52:09 <wumpus> just a matter of following the standard
417 2015-06-17 18:52:26 <wumpus> which I posted a diagram of, in that issue
418 2015-06-17 18:52:48 <gmaxwell> e.g. if you reject more than 9 places, then some stupid implementation that takes an integer and appeneds ".00000000" will break for 10. etc.
419 2015-06-17 18:53:12 <wumpus> yes, as sipa says it should accept extra zeros
420 2015-06-17 18:53:15 <wumpus> just not other numbers
421 2015-06-17 18:53:37 <wumpus> eg  0.000000010 but not 0.000000011
422 2015-06-17 18:54:08 <wumpus> as long as it can be represented exactly in 8-digit fixed point, it should be accepted
423 2015-06-17 18:54:22 <jonasschnelli> so would it make more sense to un-exp the scientific notation while parsing or in AmountFromValue()? later probably?
424 2015-06-17 18:54:35 <sipa> right, implicitly we convert the json imput to an infinite-precision decimal, and then conversion to a CAmount may fail
425 2015-06-17 18:54:36 <wumpus> in AmountFromValue
426 2015-06-17 18:54:48 <wumpus> the *parsing* just verifies strings, and is perfectly ok
427 2015-06-17 18:54:59 <wumpus> the problem is converting the string (which has been parsed) to a fixed-point value
428 2015-06-17 18:55:17 <wumpus> that's why you get 'invalid amount' not a JSON parsing error
429 2015-06-17 18:55:22 <sipa> indeed
430 2015-06-17 18:56:37 <wumpus> no need to change anything in univalue, it does what it should do and accepts numbers according to the JSON standard. The challenge is converting them correctly, I'm willing to do that but not today.
431 2015-06-17 18:56:59 <sipa> same
432 2015-06-17 18:57:13 <jonasschnelli> check if we have a 'e|E' in the string, signsubscale=-1 if e-, while (*num>='0' && *num<='9') subscale=(subscale*10)+(*num++ - '0'), n=sign*n*pow(10.0,(scale+subscale*signsubscale));
433 2015-06-17 18:57:23 <jonasschnelli> and add a lot of tests
434 2015-06-17 18:57:30 <wumpus> no don't use pow()
435 2015-06-17 18:57:34 <wumpus> that's a floating point function
436 2015-06-17 18:57:37 <sipa> well implement that with only integer arithmetic
437 2015-06-17 18:58:03 <wumpus> yes, use only integer arithmethic, and make sure you avoid overflows/underlfows too :)
438 2015-06-17 18:58:33 <wumpus> but the idea is correct
439 2015-06-17 19:03:31 <jonasschnelli> I think the code should go into ParseMoney()
440 2015-06-17 19:04:10 <sipa> i think there is no reason why bitcoin's money parser should be adapted to support JSON's quirks
441 2015-06-17 19:04:11 <jonasschnelli> which then also affects -minrelaytxfee, etc. and bitcoin-tx
442 2015-06-17 19:04:21 <wumpus> I think that's a good idea yes, to have only one function for parsing money. As long as the other places where it's used it's not a problem
443 2015-06-17 19:04:28 <sipa> i disagree :)
444 2015-06-17 19:04:37 <wumpus> for testing it's good to have only once
445 2015-06-17 19:04:40 <wumpus> but ok, I don't care.
446 2015-06-17 19:04:41 <jonasschnelli> 10e-5 is not a json only thing.
447 2015-06-17 19:04:57 <sipa> jonasschnelli: i don't think we should encourage scientific notation for numbers, period
448 2015-06-17 19:05:00 <wumpus> right, it doesn't hurt to be able to specify -maxtxfee=1e-5
449 2015-06-17 19:05:01 <sipa> but for JSON we have no choice
450 2015-06-17 19:05:20 <wumpus> it's not super nice, but normally no one would even try it
451 2015-06-17 19:05:44 <jonasschnelli> but bitcoin-tx would be untreated then.
452 2015-06-17 19:05:54 <sipa> what is wrong with bitcoin-tx now?
453 2015-06-17 19:05:56 <wumpus> I don't want to end up with 20 money parsing functions for different purposes
454 2015-06-17 19:06:13 <wumpus> much likelier that one is broken in some way
455 2015-06-17 19:06:18 <sipa> i think it's a layer violation that our need for a money parser in json changes how we parse money elsewhere
456 2015-06-17 19:06:32 <sipa> but i don't care stronger :)
457 2015-06-17 19:06:36 <sipa> *strongly
458 2015-06-17 19:06:47 <jonasschnelli> i just say maybe someone comes up with adding scientific numbers in bitcoin-tx . But right. We should focus on the json transmition / rpc
459 2015-06-17 19:07:12 <jonasschnelli> then it have to go into AmountFromValue()
460 2015-06-17 19:07:17 <wumpus> bitcoin-tx is besides the focus here, as long as you don't break it (I think it uses some strange ways of parsing, and still needs to use the strict get_XXX type-checking functions)
461 2015-06-17 19:07:44 <wumpus> it never uses AmountFromValue nor ParseMoney AFAIK
462 2015-06-17 19:08:08 <jonasschnelli> bitcoin-tx uses ParseMoney
463 2015-06-17 19:08:15 <jonasschnelli> L215
464 2015-06-17 19:08:47 <wumpus> oh it does, good!
465 2015-06-17 19:08:48 <jonasschnelli> what about adding a function to UniValue (convertScientificNum()) which would unwrap/un-exp the string?
466 2015-06-17 19:08:58 <wumpus> no, don't add it to univalue
467 2015-06-17 19:09:03 <wumpus> please define a seperate functino for this
468 2015-06-17 19:09:04 <sipa> jonasschnelli: what would it return?
469 2015-06-17 19:09:12 <sipa> it cannot assume the result is a number
470 2015-06-17 19:09:20 <wumpus> ParseMoneyJSON or such
471 2015-06-17 19:09:24 <sipa> eh, cannot assume it's an amount
472 2015-06-17 19:09:37 <jonasschnelli> sipa: it would convert the internal UniValue string from "10e-3" to "0.01"
473 2015-06-17 19:09:40 <sipa> oh
474 2015-06-17 19:09:42 <sipa> hmm
475 2015-06-17 19:09:47 <wumpus> don't burden univalue with this
476 2015-06-17 19:10:00 <sipa> not opposed to it, but i prefer a separate parser indeed
477 2015-06-17 19:10:10 <wumpus> this is an external utility and should be an external utility
478 2015-06-17 19:10:23 <wumpus> what about 1e-1000? convert it to 0.00000000000000000000000000000000000000000xxx1?
479 2015-06-17 19:10:29 <wumpus> no need to do that intermediate step
480 2015-06-17 19:10:32 <wumpus> just parse it at once
481 2015-06-17 19:10:50 <wumpus> it's not rocket science, just let me do it, but tomorrow
482 2015-06-17 19:11:25 <jonasschnelli> okay.. cool.
483 2015-06-17 19:11:33 <wumpus> if you want to work on this, write test cases :)
484 2015-06-17 19:12:15 <jonasschnelli> yes. Let me search after existing test cases for this. We probably won't be the first guys who like to trust in that kind of parsing.
485 2015-06-17 19:12:35 <sipa> well the test cases will need to be specific for us
486 2015-06-17 19:12:49 <sipa> as post-8-position nonzeroes will cause a problem
487 2015-06-17 19:13:07 <jonasschnelli> hmm... right.
488 2015-06-17 19:13:11 <wumpus> right, it's kind of specific. eg 1e-8 should work 1e-9 shouldn't, 1.1e-8 shoudln't
489 2015-06-17 19:14:01 <jonasschnelli> Yes. Let me individually write some tests.
490 2015-06-17 19:14:29 <jonasschnelli> i aim for ParseMoneyJSON() in util_tests.cpp
491 2015-06-17 19:14:35 <wumpus> +1 jonasschnelli
492 2015-06-17 19:15:49 <jonasschnelli> <joke>i'm always fascinated by the formatting of L153-L169 in util_tests.cpp</joke>
493 2015-06-17 19:16:37 <wumpus> also overflow tests make sense, it's easy to overflow the max amount with 1e8, for example, or even max 63 bit int with 1e11
494 2015-06-17 19:17:02 <sipa> but 0.0000000000000000001e10 is valid :)
495 2015-06-17 19:17:18 <wumpus> lol yes
496 2015-06-17 19:17:27 <wumpus> jonasschnelli: yes it's almost art
497 2015-06-17 19:18:13 <jonasschnelli> but where do we stop the sci. support? 0.00000000000000000000000000000000000001e30 valid?
498 2015-06-17 19:18:26 <sipa> imho yes
499 2015-06-17 19:18:30 <sipa> if we do it, do it properly
500 2015-06-17 19:18:37 <wumpus> yes I don't see why not
501 2015-06-17 19:18:38 <sipa> it's not hard
502 2015-06-17 19:19:05 <jonasschnelli> so it won't be a DOS vector?
503 2015-06-17 19:19:34 <wumpus> well apart from exhausting memory with a very long string, which doesn't have anything to do with the convertion to fixed point
504 2015-06-17 19:19:39 <sipa> first substring select the before-comma and after-comma part, compute its exponents, add 8 to those exponents, pop of trailing zeroes
505 2015-06-17 19:19:55 <sipa> and if the result is anything with non-zero value with below-0 exponent, fail
506 2015-06-17 19:20:08 <wumpus> e.g. you'd already be DoSing the parser way before it reaches this function
507 2015-06-17 19:20:38 <jonasschnelli> I tough could be some heavy math operation, but indeed. It's not a problem (or not more than normal strings).
508 2015-06-17 19:20:43 <jonasschnelli> *thought
509 2015-06-17 19:21:57 <wumpus> yeah, well if we really used arbitrary-digit arithmetic that'd be true, but we can take a shortcut (as sipa says)
510 2015-06-17 19:23:47 <sipa> no heavy math necessary
511 2015-06-17 19:23:53 <sipa> no biginteger necessary
512 2015-06-17 19:24:06 <wumpus> right
513 2015-06-17 19:30:29 <jonasschnelli> 10e5 is invalid, 10e+5 is not (after json specs)
514 2015-06-17 19:30:36 <jonasschnelli> the sign must be there
515 2015-06-17 19:32:09 <jtimon> sdaftuar wumpus: https://github.com/bitcoin/bitcoin/pull/6299
516 2015-06-17 19:32:16 <wumpus> jonasschnelli: seems so, yes
517 2015-06-17 19:32:31 <morcos> jonasschnelli: huh? the sign is not necessary
518 2015-06-17 19:32:42 <wumpus> ah no
519 2015-06-17 19:32:43 <sdaftuar> jtimon: thanks, taking a look
520 2015-06-17 19:32:46 <morcos> there is a nice shortcut in between the + and - on the diagram
521 2015-06-17 19:33:06 <wumpus> morcos: yes :) easy to overlook
522 2015-06-17 19:33:08 <jonasschnelli> ah. Right.
523 2015-06-17 19:33:52 <morcos> michagogo pointed me to jsonlint.com.  untrusted sanity checking
524 2015-06-17 19:36:45 <jonasschnelli> wumpus: https://github.com/jonasschnelli/bitcoin/commit/a101af5f6a928b294286d276e21296e32d6890d4 (starting point for your work plans for tmr)
525 2015-06-17 19:38:51 <wumpus> jonasschnelli: thanks! the bottom few should probably be ParseMoneyJSON too
526 2015-06-17 19:39:57 <jonasschnelli> wumpus: oh. Right. Fixed, new commit hash, better start here https://github.com/jonasschnelli/bitcoin/commits/2015/06/json_scientific_notation
527 2015-06-17 19:43:12 <wumpus> jonasschnelli: looks good
528 2015-06-17 19:44:55 <jonasschnelli> CLevelDBWrapper::Read() is thread safe?
529 2015-06-17 19:45:12 <jonasschnelli> (regarding #6290)
530 2015-06-17 19:45:30 <michagogo> morcos: not vouching for that site or anything, just first Google result for json validate
531 2015-06-17 19:45:35 <michagogo> Validation
532 2015-06-17 19:45:39 <michagogo> Validator
533 2015-06-17 19:45:47 <michagogo> Shut up, autocorrect.
534 2015-06-17 19:48:38 <Shogun__> Hi guys
535 2015-06-17 19:48:44 <Shogun__> someone here?
536 2015-06-17 19:49:26 <Shogun__> Guys?
537 2015-06-17 20:00:37 <jtimon> can someone please verify https://github.com/bitcoin/bitcoin/pull/6051 is moveonly? sipa ? cfields ?
538 2015-06-17 20:01:02 <cfields> jtimon: sure thing. Will do when i finish up what i'm working on atm
539 2015-06-17 20:01:19 <jtimon> cfields great thanks!
540 2015-06-17 20:05:16 <CodeShark> I am generally not too thrilled about doing floating point stuff over JSON :p
541 2015-06-17 20:05:32 <CodeShark> not because it's difficult to do...but because it's easy for people to screw up
542 2015-06-17 20:06:17 <jonasschnelli> CodeShark: i think the issue is more between encode/decode <input><json-encode><----><json-decode><ouput>.
543 2015-06-17 20:06:36 <jonasschnelli> Some encoders (like python) start to convert 0.00000001 into 1e-8.
544 2015-06-17 20:06:55 <jonasschnelli> it's valid json and therefore we need to deal with it.
545 2015-06-17 20:07:16 <CodeShark> right - the issue is that encoders and decoders don't always comply with a particular spec...and you have malleability issues, as well - no unique representation
546 2015-06-17 20:07:23 <jonasschnelli> that's why i originally though it would be good to have this solved out within the UniValue space.
547 2015-06-17 20:08:11 <jonasschnelli> CodeShark: Agreed, json has some interpretation space (which could lead to significant bugs)
548 2015-06-17 20:08:42 <jonasschnelli> need to get some rest. good night.
549 2015-06-17 20:08:48 <CodeShark> night
550 2015-06-17 20:09:33 <CodeShark> there's also the precision issue - and rounding errors
551 2015-06-17 20:09:39 <CodeShark> if you use floating point types
552 2015-06-17 20:09:56 <CodeShark> which gets REALLY ugly...especially when dealing with financial maths in base 10...represented as base-2 floating points
553 2015-06-17 20:09:58 <CodeShark> ugh
554 2015-06-17 20:10:40 <CodeShark> so even if the decoder is compliant, if the underlying datatype it gets mapped to isn't you still have errors
555 2015-06-17 20:11:39 <CodeShark> by only using integers for data interchange, it forces both ends to make a conscious choice of datatype before using any floating point types
556 2015-06-17 20:11:56 <CodeShark> rather than just using defaults for a particular language/library
557 2015-06-17 20:13:20 <CodeShark> which in most cases is base-2 floating point
558 2015-06-17 20:13:55 <zooko> Just don't use floating point when implementing json decimals.
559 2015-06-17 20:15:25 <CodeShark> zooko: agreed - however, using stuff like 1e-5 in json encourages lazy habits from poor developers :p
560 2015-06-17 20:15:35 <ajweiss> codeshark: +1
561 2015-06-17 20:15:55 <ajweiss> also bitcoind rules don't have to follow json rules
562 2015-06-17 20:16:30 <ajweiss> -x is valid json, but certainly a bogus amount
563 2015-06-17 20:22:44 <jtimon> yep wumpus, it seems travis is happy with #6299 + #6298
564 2015-06-17 20:27:01 <dgenr8> what is the target to wipe out everything that 'make depends' made?
565 2015-06-17 20:28:51 <jtimon> btw, cfields, do you think the following should go in gitignore? would that be a good candidate for the trivial branch ?
566 2015-06-17 20:28:51 <jtimon> src/build-aux/compile
567 2015-06-17 20:28:51 <jtimon> src/build-aux/config.guess
568 2015-06-17 20:28:51 <jtimon> src/build-aux/config.sub
569 2015-06-17 20:28:51 <jtimon> src/build-aux/depcomp
570 2015-06-17 20:28:52 <jtimon> src/build-aux/install-sh
571 2015-06-17 20:28:54 <jtimon> src/build-aux/missing
572 2015-06-17 20:28:56 <jtimon> src/build-aux/test-driver
573 2015-06-17 20:28:59 <jtimon> cache/
574 2015-06-17 20:30:10 <cfields> jtimon: i don't think build-aux is supposed to live in src/ ?
575 2015-06-17 20:30:50 <jtimon> mhmm, is it possible that it's caused by running make check from src/ ?
576 2015-06-17 20:32:06 <cfields> nope, i do everything from src/
577 2015-06-17 21:56:12 <ahmed_> BlueMatt: you around? ive got a bitcoinj question
578 2015-06-17 22:01:48 <BlueMatt> ahmed_: -> #bitcoinj ?
579 2015-06-17 22:01:58 <ahmed_> ah awesome didnt know that existed