1 2015-09-23 00:32:54 <esso> I would like to build a simple program to parse through blk*.dat files where can I find technical documentation on the block file data?
  2 2015-09-23 00:33:16 <dcousens> esso: https://github.com/dcousens/blkdat-stream might be helpful
  3 2015-09-23 00:33:47 <dcousens> namely https://github.com/dcousens/blkdat-stream/blob/master/index.js should hopefully be very clear :S,  YMMV though
  4 2015-09-23 00:36:56 <esso> dcousens: thanks that looks very useful!
  5 2015-09-23 05:57:47 <nwilcox> Hm...  I'm in gdb within VerifyScript OP_CHECKSIG* and when entering the line [*] which removes the signature from the script, the signature is not present in the script.
  6 2015-09-23 05:57:55 <nwilcox> [*] https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L841
  7 2015-09-23 05:58:03 <nwilcox> -yet the transaction verifies just fine.
  8 2015-09-23 06:40:43 <CodeShark> nwilcox: the signature cannot be part of what gets signed
  9 2015-09-23 06:41:08 <CodeShark> so the op code must remove the signature from the script before checking whether the signature is valid
 10 2015-09-23 06:41:11 <CodeShark> err
 11 2015-09-23 06:41:17 <CodeShark> yes
 12 2015-09-23 06:43:21 <CodeShark> notice that if the signature were not to be present at the time the op code executes, garbage gets popped off the stack and it fails
 13 2015-09-23 06:49:07 <CodeShark> so you're saying that before that line executes the signature is already missing from the script?
 14 2015-09-23 06:50:03 <Luke-Jr> nwilcox: SIGHASH_SINGLE is .. fun? :P
 15 2015-09-23 06:55:44 <nwilcox> CodeShark: This is a call to EvalScript passing scriptPubKey here: https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L1180
 16 2015-09-23 06:55:53 <nwilcox> scriptPubKey doesn't contain the signature.
 17 2015-09-23 06:57:12 <nwilcox> One way line 841 would actually remove something is if the signature and the OP_CHECKSIG* appear in the same "script half" together (either both in scriptPubKey or both in scriptSig).
 18 2015-09-23 06:58:06 <nwilcox> It almost feels like the two scripts used to be concatenated, so this removal was necessary, and then they were separated in distinct calls to EvalScript later, but the removal logic remained...
 19 2015-09-23 06:58:22 <nwilcox> But I don't know enough about the code or its history to conclude that.
 20 2015-09-23 06:58:33 <nwilcox> Luke-Jr: This is SIGHASH_ALL.
 21 2015-09-23 06:59:27 <CodeShark> did you try a breakpoint before 834?
 22 2015-09-23 07:00:11 <nwilcox> No, I didn't.
 23 2015-09-23 07:00:55 <nwilcox> I set the breakpoint at 841...
 24 2015-09-23 07:01:23 <nwilcox> Anyway I'm super tired, so I think I need to sleep and come back to this.
 25 2015-09-23 07:02:00 <CodeShark> notice that the stack is not the same thing as scriptCode
 26 2015-09-23 07:03:09 <nwilcox> Right...  The signature is indeed on the stack, but it's not in the scriptCode variable (for this invocation of EvalScript)...
 27 2015-09-23 07:04:30 <nwilcox> Oh, just noticed: CodeShark> so you're saying that before that line executes the signature is already missing from the script?
 28 2015-09-23 07:04:33 <nwilcox> Yes.
 29 2015-09-23 07:04:58 <nwilcox> EvalScript is called twice sharing the same stack: first with scriptSig, and next with scriptPubKey.
 30 2015-09-23 07:05:48 <nwilcox> I expect for a normal payment to a pubkey, that scriptSig contains a signature, but not the OP_CHECKSIG* opcode, and scriptPubKey contains no signature but does OP_CHECKSIG*.
 31 2015-09-23 07:06:25 <CodeShark> yes, indeed
 32 2015-09-23 07:06:29 <nwilcox> (The two calls are in VerifyScript...)
 33 2015-09-23 07:07:22 <CodeShark> but that would not be the case for p2sh
 34 2015-09-23 07:07:49 <nwilcox> Ah...  Ok.  I don't know p2sh well yet.
 35 2015-09-23 07:08:57 <nwilcox> Does p2sh involve the "code separator" stuff or different hash flags besides SIGHASH_ALL?
 36 2015-09-23 07:09:20 <CodeShark> not necessarily
 37 2015-09-23 07:09:38 <nwilcox> -because if so, that involves code paths I've deliberately excluded so far in my reading so I can get a handle on script verification for a "base case".
 38 2015-09-23 07:10:42 <CodeShark> in p2sh, you generally have scriptSig = {signatures}{redeemscript} and scriptPubKey = OP_HASH160 {script hash} OP_EQUAL
 39 2015-09-23 07:11:19 <CodeShark> where script hash is HASH160 of the redeemscript
 40 2015-09-23 07:12:12 <CodeShark> and the redeemscript is where the OP_CHECKSIG would be
 41 2015-09-23 07:12:45 <nwilcox> Ah, got it.  Thanks.
 42 2015-09-23 07:13:42 <nwilcox> Alright, off to bed.  Thanks for the help!
 43 2015-09-23 07:23:31 <CodeShark> it's a bit unfortunate we didn't do BIP17... :p
 44 2015-09-23 07:24:46 <CodeShark> although I suppose we still could :)
 45 2015-09-23 07:31:17 <Luke-Jr> CodeShark: that'd make a mess of P2SH addresses I think
 46 2015-09-23 08:24:51 <CodeShark> Luke-Jr: we could just use a new version byte...but meh, it's hardly the kind of battle worth fighting right now given everything else going on right now :p
 47 2015-09-23 08:25:18 <Luke-Jr> merklised stuff for example
 48 2015-09-23 08:25:47 <CodeShark> agreed - if we're going to fix the script we might as well go big :)
 49 2015-09-23 08:46:49 <StormDev> hello, Is python-bitcoinlib the "standard" library to make some stuff with python?
 50 2015-09-23 08:56:40 <StormDev> thanks btcdrak
 51 2015-09-23 13:32:49 <jgarzik> jonasschnelli, does univalue PR #6 fix the -fPIC problem when embedded as git subtree?
 52 2015-09-23 13:32:58 <jonasschnelli> jgarzik: yes.
 53 2015-09-23 13:32:59 <jgarzik> jonasschnelli, looks merge-ready
 54 2015-09-23 13:33:05 <jonasschnelli> IMO it's merge ready
 55 2015-09-23 13:33:22 <jonasschnelli> libtool will enable -fPIC, etc.
 56 2015-09-23 13:33:51 <jonasschnelli> i think after PR #6 we have "stable" ABI and bitcoin/subtree should be possible
 57 2015-09-23 13:34:39 <jgarzik> agreed
 58 2015-09-23 13:47:24 <jgarzik> jonasschnelli, PR merged, tested, univalue v1.0.0 GPG-signed tag pushed to repo.
 59 2015-09-23 13:47:35 <jgarzik> (aka 1.0 release)
 60 2015-09-23 13:48:01 <jgarzik> travis is happy on clang/gcc linux/osx matrix
 61 2015-09-23 13:48:03 <jonasschnelli> jgarzik: Nice. Will rebase the bitcoin/univalue subtree PR.
 62 2015-09-23 13:48:22 <jonasschnelli> Maybe we should pimp README(.md) soon.
 63 2015-09-23 13:50:47 <jgarzik> jonasschnelli, cfields, wumpus: FYI got native OSX Travis build working for bitcoind: https://github.com/jgarzik/bitcoin/tree/2015_travis_osx + https://travis-ci.org/jgarzik/bitcoin/builds/81106635
 64 2015-09-23 13:51:00 <jgarzik> should just be a bit more work for Qt
 65 2015-09-23 13:51:09 <btcdrak> nice
 66 2015-09-23 13:51:19 <jonasschnelli> jgarzik: +1 for native osx builds...
 67 2015-09-23 13:52:01 <jgarzik> ref https://travis-ci.org/jgarzik/bitcoin/builds/81106635 ...
 68 2015-09-23 13:52:18 <jgarzik> I haven't created a PR because "32 bit + dash" is inexplicably broken
 69 2015-09-23 13:52:26 <jgarzik> even though everything else, including native OSX, works
 70 2015-09-23 13:52:36 <jonasschnelli> IIRC: travis does not build QT at all?
 71 2015-09-23 13:52:55 <jgarzik> jonasschnelli, it does.  the "deploy" target
 72 2015-09-23 13:53:09 <jgarzik> er, GOAL
 73 2015-09-23 13:54:07 <jgarzik> jonasschnelli, it explicitly does --enable-gui on some targets
 74 2015-09-23 13:54:45 <jonasschnelli> A. Right. Just two targets have the NO_QT=1 flag..
 75 2015-09-23 13:54:51 <jgarzik> jonasschnelli, in addition, the cross-mac option literally builds the Qt library, before building the bitcoin Qt GUI
 76 2015-09-23 13:55:03 <jgarzik> native OSX is actually easier - "brew install qt"
 77 2015-09-23 13:55:52 <jonasschnelli> jgarzik: Right. Although i think the cross built QT dependency will be cached.
 78 2015-09-23 13:58:59 <jgarzik> yes
 79 2015-09-23 14:16:16 <jonasschnelli> jgarzik: univalue: maybe it would be good to add a ./include dir next to ./lib for the exposed headers.
 80 2015-09-23 14:17:09 <jgarzik> jonasschnelli, I thought about that... but do not really see the value other than some consistency with unrelated OSS projects
 81 2015-09-23 14:17:15 <jgarzik> jonasschnelli, happy to be convinced otherwise
 82 2015-09-23 14:17:17 <jonasschnelli> UniValue subtree works/compiles,... https://github.com/bitcoin/bitcoin/pull/6637, lets see what travis thinks about it.
 83 2015-09-23 14:17:35 <jgarzik> jonasschnelli, the main bit is include_HEADERS in *.am
 84 2015-09-23 14:17:48 <jonasschnelli> jgarzik: Agreed.
 85 2015-09-23 14:18:12 <jonasschnelli> jgarzik: I just stumbled over "make[2]: Circular univalue/lib/libunivalue.la <- univalue/lib/libunivalue.la dependency dropped.", could be because of https://github.com/bitcoin/bitcoin/pull/6637/files#diff-480477e89f9b6ddafb30c4383dcdd705R24
 86 2015-09-23 14:18:46 <jgarzik> jonasschnelli, in which directory is that warning?
 87 2015-09-23 14:19:11 <jonasschnelli> got the warning when executed "make check"
 88 2015-09-23 14:20:02 <jgarzik> jonasschnelli, right... in which sub-directory?  lib? test?
 89 2015-09-23 14:20:10 <jonasschnelli> test
 90 2015-09-23 14:22:02 <wumpus> jgarzik: nice
 91 2015-09-23 14:23:13 <jgarzik> jonasschnelli, can you re-run following "git clean -dfx" ?
 92 2015-09-23 14:23:22 <jgarzik> (make sure no unsaved work first...)
 93 2015-09-23 14:24:18 <jonasschnelli> jgarzik: okay... autogen-ing/configure-ing/make-ing now
 94 2015-09-23 14:25:07 <jgarzik> jonasschnelli, also, what is your build platform where the problem occurs?
 95 2015-09-23 14:25:14 <jgarzik> os version
 96 2015-09-23 14:25:16 <jonasschnelli> osx 10.10
 97 2015-09-23 14:25:38 <jonasschnelli> but it's just a warning *duck*
 98 2015-09-23 14:26:23 <jonasschnelli> Making check in src \n make[2]: Circular univalue/lib/libunivalue.la <- univalue/lib/libunivalue.la dependency dropped.
 99 2015-09-23 14:29:08 <jgarzik> jonasschnelli, hrm, so XCode make and not GNU make
100 2015-09-23 14:29:38 <jonasschnelli> jgarzik: no. GNU make: GNU Make 3.81
101 2015-09-23 14:29:57 <jonasschnelli> xcode make is clang llvm, i guess.
102 2015-09-23 14:30:14 <jgarzik> yeah. ok.  surprised it is not BSD make.
103 2015-09-23 14:30:32 <jonasschnelli> jgarzik: I'll double check on Debian 8.1
104 2015-09-23 14:31:00 <jgarzik> jonasschnelli, FWIW, it is worth keeping an Ubuntu 12 VM around for testing - that's what Travis uses
105 2015-09-23 14:32:42 <jonasschnelli> jgarzik: Hmm.. only have 14.04 and 15.04 vm's around. But right, a 12er vm would make sense. But starting VMWare = fan starts kicking in. Mostly i build on a remote root server (less local CPU wasting)
106 2015-09-23 14:33:09 <jgarzik> jonasschnelli, virtual box works :)
107 2015-09-23 14:33:34 <jgarzik> but yes a remote cloud server works just as well
108 2015-09-23 14:33:44 <helo> does a chroot not work just as well?
109 2015-09-23 14:34:18 <jonasschnelli> helo: chroot as nothing todo with building? hm?
110 2015-09-23 14:35:10 <jgarzik> well it does to a limited extent... if and only if a userland for OS version X will work on kernel for OS version Y
111 2015-09-23 14:35:25 <jgarzik> much cleaner to boot a VM with the right kernel and right OS version match
112 2015-09-23 14:35:58 <jonasschnelli> jgarzik: windows build fails: https://travis-ci.org/bitcoin/bitcoin/jobs/81780740#L2636
113 2015-09-23 14:36:26 <jonasschnelli> maybe add a mingw build jgarzik/univalue travis
114 2015-09-23 14:36:38 <jgarzik> jonasschnelli,  heh -agreed- was just typing the same thing myself :)
115 2015-09-23 14:37:11 <jgarzik> jonasschnelli, looks like missing C/C++ runtime DLLs
116 2015-09-23 14:37:34 <jgarzik> jonasschnelli, will tests even work on cross-windows?  is it running WINE or something for bitcoin?
117 2015-09-23 14:37:49 <jonasschnelli> jgarzik: WINE yes. Should work...
118 2015-09-23 14:39:27 <jonasschnelli> jgarzik: mingw can't find libstdc++-6.dll and libgcc_s_sjlj-1.dll.. hmm..
119 2015-09-23 14:40:07 <jgarzik> jonasschnelli, probably just a matter of missing apt packages?
120 2015-09-23 14:40:09 <jonasschnelli> jgarzik: i think it requires a -static-libstdc++
121 2015-09-23 14:40:24 <jgarzik> that works too :)
122 2015-09-23 14:42:41 <jonasschnelli> jgarzik: need to add something like this (https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L224) to univalue/configure.ac
123 2015-09-23 15:21:47 <slimjim_> Could someone describe the status of the bitcoin script at the moment... The opcodes that are disabled and such stuf
124 2015-09-23 15:23:16 <wumpus> slimjim_: https://bitcoin.org/en/developer-reference#op-codes
125 2015-09-23 17:15:05 <bitcoin-dev653> hey all
126 2015-09-23 17:16:14 <bitcoin-dev653> was wondering if anyone knew about the bitcoin CI builds? specifically, is gitian actually being used to produce the binaries downloadable on bitcoin.org? or just the travis-ci build's binaries?
127 2015-09-23 17:18:28 <xenog> does anybody know if there are some Haskell bindings for bitcoin/secp256k1?
128 2015-09-23 17:19:06 <xenog> I would like to change Haskoin to use this library
129 2015-09-23 17:20:33 <teward> bitcoin-dev653: patience is a virtue
130 2015-09-23 17:20:37 <teward> you haven't been here for 4 minutes
131 2015-09-23 17:20:41 <teward> so have patience, and WAIT for an answer
132 2015-09-23 17:25:40 <wumpus> but yes, gitian is used to build the releases on bitcoin.org, gitian is not used in travis CI
133 2015-09-23 17:28:09 <bitcoin-dev653> Thanks wumpus. Do you know if there is a public server that runs the gitian builds? or is that done privately by someone specific?
134 2015-09-23 17:29:10 <kanzure> multiple people run the gitian build so that they can verify the output
135 2015-09-23 17:30:21 <sipa> maaku: it's non-obvious to me that the new logic is equivalent
136 2015-09-23 17:31:12 <wumpus> bitcoin-dev653: multiple people build using gitian for a release then commit signatures, to assure everyone is building the same (unmodified) source code, see https://github.com/bitcoin/bitcoin/blob/master/doc/gitian-building.md https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md for details
137 2015-09-23 17:31:19 <sipa> did other reviewers look at that specifically (i've been out of the loop, so i'm certainly missing things)
138 2015-09-23 17:31:30 <maaku> sipa: did you look at the individual commits? I split out one commit that rearranges code without any changes, then another that adds the BIP 68 stuff
139 2015-09-23 17:32:04 <sipa> maaku: i did not!
140 2015-09-23 17:35:41 <bitcoin-dev653> Thanks. wumpus and kanzure
141 2015-09-23 17:37:37 <sipa> maaku: ok, that indeed clarifies things... sorry for not noticing earlier
142 2015-09-23 17:38:04 <maaku> sipa: no worries. I'm still going to comment on github in case others have the same concern
143 2015-09-23 17:38:50 <sipa> (i'm still not convinced that the logic is equivalent, but it shouldn't be hard to review)
144 2015-09-23 17:44:40 <maaku> sipa: hrm. the "Add rules" commit could probably be further split to minimize review difficulty
145 2015-09-23 17:44:49 <maaku> I can do that if you think it is desireable
146 2015-09-23 17:44:56 <drian> why doesn't gitian use vagrant?
147 2015-09-23 17:46:26 <maaku> drian: I had a patch once to do that.
148 2015-09-23 17:48:22 <drian> ok cool. thank you.
149 2015-09-23 17:49:02 <btcdrak> maaku: I can see sipa's point, an "add rules" commit would help.
150 2015-09-23 17:49:13 <Luke-Jr> drian: what purpose would that serve?
151 2015-09-23 17:49:14 <wumpus> it already has the choice of various virtualization methods (LXC/KVM qemu/virtualbox), not sure what vagrant would add
152 2015-09-23 17:49:47 <Luke-Jr> vagrant seemed to me to be gitian without the deterministic build stuff
153 2015-09-23 17:49:55 <sipa> longer term, i think the idea is to have the build environment itself built, so no deterministic host is necessary anymore
154 2015-09-23 17:50:06 <maaku> wumpus: vagrant is a script on top of virtualbox. it lets you 'cd contrib/vagrant && vagrant up' and have that build out a gitian vm
155 2015-09-23 17:50:06 <wumpus> yes
156 2015-09-23 17:50:55 <maaku> it would be quite useful for helping new people get their gitian vm setup
157 2015-09-23 17:51:17 <wumpus> ok, great then, why did you never upstream your patch?
158 2015-09-23 17:51:18 <drian> Luke-Jr: vagrant makes things easier.  no need to mess around with a UI when creating your image.
159 2015-09-23 17:51:42 <Luke-Jr> maaku: so it does the same thing as vmbuilder?
160 2015-09-23 17:51:54 <Luke-Jr> drian: uh? already gitian does not use anything UI
161 2015-09-23 17:52:08 <Luke-Jr> wumpus: maaku's patch was a PR for a while
162 2015-09-23 17:52:15 <kanzure> vagrant/chroot/docker could all serve similar purposes there
163 2015-09-23 17:52:17 <wumpus> drian: the gitian build guide is really the lowest common denominator; there's tons of ways to speed up stuff if you don't mind
164 2015-09-23 17:52:30 <drian> Luke-Jr: reading the inscructions that wumpus posted, you have to use a UI when setting up the image the first time.
165 2015-09-23 17:52:33 <maaku> wumpus: https://github.com/bitcoin/bitcoin/pull/1597
166 2015-09-23 17:52:40 <wumpus> drian: you don't have to, it's just one way
167 2015-09-23 17:52:50 <drian> wumpus: understood.  fewer dependencies, works better in windows, I agree.
168 2015-09-23 17:52:53 <wumpus> I just run gitian on my Ubuntu 14.04 host
169 2015-09-23 17:53:02 <wumpus> (without creating another level of VM)
170 2015-09-23 17:53:30 <Luke-Jr> wumpus: eh? does that actually work?
171 2015-09-23 17:53:59 <wumpus> Luke-Jr: well it still creates the inner VM
172 2015-09-23 17:54:32 <Luke-Jr> ah
173 2015-09-23 17:54:46 <Luke-Jr> I thought I was the only one ever nesting VMs :P
174 2015-09-23 17:56:42 <btcdrak> That would be excellent. Manually setting up gitian VM is a slow pain.
175 2015-09-23 17:57:40 <maaku> the last time I tried getting vagrant builds upstream it died a slow death because everyone else already had gitian setup and didn't want to maintain two systems
176 2015-09-23 17:57:57 <sipa> would the resulting binaries be identica
177 2015-09-23 17:58:00 <sipa> ?
178 2015-09-23 17:58:10 <maaku> sipa: it's still gitian
179 2015-09-23 17:58:53 <maaku> so you build a gitian host using vagrant scripts, and then do a normal gitian build there
180 2015-09-23 17:59:46 <wumpus> but it's infinite regress, you could have a script to build a host with vagrant scripts to build a gitian host... etc :-) the idea was that if you already have a linux-like host you can run gitian as-is
181 2015-09-23 18:00:15 <wumpus> in practice it seems that gitian mostly only works on ubuntu and debian, but ok...
182 2015-09-23 18:01:31 <drian> maaku: as kanzure mentioned, docker and chroot can do this job too.  vagrant is the only one that works in windows.
183 2015-09-23 18:01:33 <maaku> wumpus: well that latter point is why vagrant is useful. no matter what platform you are on, make a standard ubuntu 14.04 (or debian stable, whatever) vm image, update, install toolchain, and be ready to run gitian
184 2015-09-23 18:02:56 <kanzure> vagrant boxes are often built using packer, https://www.packer.io/
185 2015-09-23 18:03:09 <kanzure> packer has various builders such as ec2/ami, docker, chroot, virtualbox, vagrant, whatever
186 2015-09-23 18:03:21 <kanzure> (perhaps one day there will be a gitian target)
187 2015-09-23 18:03:23 <maaku> drian: my original point for doing the pull request was to allow bitcoin to be built out of the box on all supported platforms -- I believe you still can't build bitcoind on windows
188 2015-09-23 18:03:33 <kanzure> (this is instead of "attach some random provisioning scripts to the Vagrantfile file")
189 2015-09-23 18:03:35 <wumpus> maaku: yes, my point is that I'd prefer patching gitian so that the intermediate VM is not necessary, especially as gitian already creates a VM. But anyhow.
190 2015-09-23 18:04:21 <wumpus> maaku: in any case, looks like people are interested in vagrant instructions, if you commit to maintaining them I'm fine with including them
191 2015-09-23 18:04:53 <kanzure> vagrantfile should be in bitcoin.git repo, vagrant .box file should be hosted somewhere on bitcoin.org or otherwise
192 2015-09-23 18:04:55 <maaku> wumpus: well that's why I haven't pushed the issue in the intervening 2 years, but it's a decent stopgap solution between the long term goal of deterministic linux distros, and where we are now
193 2015-09-23 18:04:59 <wumpus> otherwise it tends to go out of date very soon, and as no one of us uses it... that's why the original pull was closed
194 2015-09-23 18:05:20 <kanzure> vagrant .box files have tendency to fall out-of-date due to lack of maintenance or regular updates. but distributing vagrant .box file with a release for gitian verification would be pretty awesome.
195 2015-09-23 18:05:22 <wumpus> kanzure: uhm we're not going to host a VM image, that defeats the purpose
196 2015-09-23 18:05:36 <kanzure> there's no reason to have everyone build a vagrant box :-)
197 2015-09-23 18:06:08 <kanzure> the purpose was "be ready to run gitian"
198 2015-09-23 18:06:10 <wumpus> (I mean, it's easy enough to create a VM that will secretly add a backdoor to the source before building)
199 2015-09-23 18:06:37 <wumpus> the point of gitian is that you set it up yourself, on your own hardware, or it's at least a list of steps that is easy to verify. Hence, no VM images.
200 2015-09-23 18:07:04 <kanzure> i think that could be trivially fixed by telling vagrant users that they should run gitian and confirm that the output matches another gitian build... hmm no, backdoors can defeat this.
201 2015-09-23 18:07:08 <kanzure> ok i see your point.
202 2015-09-23 18:07:52 <wumpus> that wouldn't work for the case if *everyone* used it (which is unlikely, but cannot be excluded, say for an emergency release when most of the devs are offline)
203 2015-09-23 18:07:54 <kanzure> why isn't debian completely deterministic?
204 2015-09-23 18:08:07 <maaku> kanzure: working on it
205 2015-09-23 18:08:16 <maaku> https://wiki.debian.org/ReproducibleBuilds
206 2015-09-23 18:08:31 <kanzure> yes i was looking at that page a few days ago, was surprised that bitcoin was catalyst for this :-(
207 2015-09-23 18:08:50 <maaku> but note that same problem exists with or without vagrant -- how do you trust your debian CDs?
208 2015-09-23 18:08:56 <kanzure> or maybe it was gitian page that i was looking at
209 2015-09-23 18:09:10 <kanzure> well i don't trust my debian disks, i do pgp things, right?
210 2015-09-23 18:10:56 <kanzure> "i only trust bootloaders signed by the private key of linus torvalds himself~~"
211 2015-09-23 18:13:56 <wumpus> maaku: sure, but a VM image hosted by some bitcoin developer would be easier to compromise for a bticoin-specific backdoor than the entire debian project
212 2015-09-23 18:14:15 <wumpus> security is never absolute, that's true...
213 2015-09-23 18:16:47 <kanzure> you could do a deterministic build of the vagrant box, heh.
214 2015-09-23 18:16:57 <maaku> wumpus: well hopefully this motivates repreducable linux distirbutions. it should not be so complicated to get a bit-for-bit identical build of a linux distro
215 2015-09-23 18:16:59 <kanzure> and then repeat quorum process there.
216 2015-09-23 18:17:03 <maaku> kanzure: that would be the goal ;)
217 2015-09-23 18:17:34 <kanzure> problem with vagrant boxes is that often the dependencies are shifting (new stuff to add etc) so requires frequent rebuilds, oh well
218 2015-09-23 18:18:22 <maaku> kanzure: we've had that same problem with gitian. determinism occationally broke due to toolchain package updates
219 2015-09-23 18:19:23 <wumpus> sure, you could. As I said, you can nest this infinitely deep :) But I"m not going to bother.
220 2015-09-23 18:26:21 <Luke-Jr> can I do a deterministic build of the CPU? :x
221 2015-09-23 18:28:24 <wumpus> sure Luke-Jr, build the source using a determinsitic HDL compiler and place and route tools, then look at your CPU using an electron microscope and compare :p
222 2015-09-23 18:31:38 <btcdrak> wumpus: Luke-Jr: tldr; we're all screwed
223 2015-09-23 18:31:54 <Luke-Jr> maybe some day we can 3D-print CPUs
224 2015-09-23 18:32:07 <kanzure> bitlbox is one way to 3d-print CPUs at the moment
225 2015-09-23 18:32:37 <wumpus> a less hacky way may be to use a softcore that runs on a FPGA, but that assumes that the bitstream doesn't somehow get compromised
226 2015-09-23 18:32:39 <Luke-Jr> kanzure: link?
227 2015-09-23 18:32:54 <kanzure> Luke-Jr: http://creativemachines.cornell.edu/sites/default/files/IJRR14_MacCurdy.pdf
228 2015-09-23 18:32:54 <Luke-Jr> wumpus: then we need deterministic builds of the FPGA :/
229 2015-09-23 18:33:03 <kanzure> see figure 11 and figure 12
230 2015-09-23 18:33:53 <kanzure> (figure 11 is on page 14)
231 2015-09-23 18:34:04 <Luke-Jr> wumpus: although even an open source bitstream compiler would be a nice step forward..
232 2015-09-23 18:36:42 <btcdrak> It's no joke, a Foundry can compromise your designed easily and undetectably using doping techniques http://sharps.org/wp-content/uploads/BECKER-CHES.pdf
233 2015-09-23 18:36:45 <wumpus> Luke-Jr: there is, for some smaller Lattice FPGAs: cliffordwolf's yosys+arachne-pnr+icestorm. The largest variant can run a small 32 bit core: https://github.com/cliffordwolf/picorv32
234 2015-09-23 18:37:15 <kanzure> btcdrak: not entirely true, see http://diyhpl.us/~bryan/papers2/security/Reversing%20Stealthy%20Dopant-Level%20Circuits.pdf
235 2015-09-23 18:38:22 <jtimon> about spliting the mailing list, I think bitcoin-discuss will probably be abandoned (again), I would prefer bitcoin-dev for things specific to Bitcoin Core and bitcoin-consensus for the consensus rules and BIPs that affect all implementations
236 2015-09-23 18:38:49 <kanzure> was it previously abandoned?
237 2015-09-23 18:40:18 <kanzure> were people directed to use -discuss when posting off-topic things?
238 2015-09-23 18:40:19 <jtimon> kanzure: http://sourceforge.net/p/bitcoin/mailman/?source=navbar
239 2015-09-23 18:40:51 <jtimon> kanzure: I don't really know much of the history, but I believe that was the intention
240 2015-09-23 18:40:51 <sipa> kanzure: until a few months ago, i wasn't aware of the discuss list. And I'm and admin for it.
241 2015-09-23 18:41:06 <kanzure> sipa: awesome
242 2015-09-23 18:41:27 <btcdrak> jtimons: read the full discussion that was had a few days ago.
243 2015-09-23 18:42:00 <jtimon> btcdrak: sure, do you have a link to the relevant part of the log?
244 2015-09-23 19:25:49 <maaku> sipa: I've split up the "add rules" commit so it should be *much* easier to review
245 2015-09-23 19:25:58 <sipa> maaku: thanks!
246 2015-09-23 19:27:38 <maaku> sipa: unforuntately github orders them by commit date, but i'm sure you can figure that out
247 2015-09-23 19:57:27 <Luke-Jr> do people actually review on GitHub? seems like a security weakness..
248 2015-09-23 20:03:08 <btcdrak> maaku: try a force push for good luck
249 2015-09-23 20:03:39 <maaku> btcdrak: yeah force pushed to master and back again and that seemed to work
250 2015-09-23 20:03:48 <maaku> travis is building onw
251 2015-09-23 20:16:12 <btcdrak> maaku: travis borked
252 2015-09-23 20:17:47 <maaku> life hates me today, aparantly
253 2015-09-23 20:22:00 <btcdrak> maaku: heh, travis has been really unreliable lately.
254 2015-09-23 20:22:19 <btcdrak> maaku: on another note did you see a PR to your fork for OP_CSV https://github.com/maaku/bitcoin/pull/1 ?
255 2015-09-23 20:22:33 <maaku> yeah, I did
256 2015-09-23 20:23:06 <maaku> trying to get BIP 62 / #6312 fixed up first
257 2015-09-23 20:24:42 <btcdrak> Can someone manually restart https://travis-ci.org/bitcoin/bitcoin/jobs/81845332?
258 2015-09-23 20:28:20 <maaku> I don't know if you can .. I made a meaningless change to a commit message and force purshed
259 2015-09-23 20:31:42 <sipa> i can restart in 15 min
260 2015-09-23 20:33:01 <btcdrak> maaku: admins can restart individual jobs
261 2015-09-23 20:37:43 <sipa> ok
262 2015-09-23 21:03:32 <btcdrak> maaku: travis love you again.
263 2015-09-23 21:22:21 <nwilcox> In interpreter.* what's the motivation to separate out BaseSignatureChecker from TransactionSignatureChecker?
264 2015-09-23 21:22:31 <nwilcox> Are there any other subclasses of BSC?
265 2015-09-23 21:22:44 <nwilcox> Is it ever possible to check a signature without reference to the txTo?
266 2015-09-23 21:23:41 <sipa> nwilcox: the reason was the ability to disentangle the signature cache from the script evaluator
267 2015-09-23 21:23:54 <sipa> but there are more useful possibilities
268 2015-09-23 21:26:12 <sipa> nwilcox: for example, we could have alert messages or messages signed by the script system, rather than individual keys
269 2015-09-23 21:26:19 <sipa> giving instant multisig support to those
270 2015-09-23 22:18:40 <bonks> What could be causing this besides a bad network connection? My node currently has 123 connections:
271 2015-09-23 22:18:43 <bonks> "errors" : "WARNING: check your network connection, 3 blocks received in the last 4 hours (24 expected)"
272 2015-09-23 22:21:10 <bonks> Nevermind, probably related to https://github.com/bitcoin/bitcoin/issues/6251
273 2015-09-23 23:03:54 <maaku> nwilcox: or more generally, libscript could be used to verify signatures over arbitrary data, e.g. a hypothetical script-signature replacement of PGP
274 2015-09-23 23:06:49 <gmaxwell> bonks: that issue is closed...
275 2015-09-23 23:10:39 <bonks> gmaxwell: I'm not familiar with bitcoin dev process. I figured it was on your guys' radar. Are you implying the cause of problem still isn't identified?
276 2015-09-23 23:11:07 <gmaxwell> bonks: AFAIK any issues with that bogusly triggering were fixed. :(
277 2015-09-23 23:13:05 <bonks> Hmm okay then. Thx
278 2015-09-23 23:14:13 <sipa> maybe his network connection was down for a while?
279 2015-09-23 23:49:22 <rusty> gmaxwell, sipa: is there any genuine benefit to the 75% bit-activation step?  I dropped it from the versionbits as a simplification, but jtimon likes it.
280 2015-09-23 23:49:31 <rusty> Seems to be the only remaining issue.
281 2015-09-23 23:49:46 <rusty> (Other than implementation, looking at CodeShark__)
282 2015-09-23 23:50:11 <jtimon> rusty I said the opposite: I don't like it, you can activate the new rule in your own blocks from the start
283 2015-09-23 23:50:21 <gmaxwell> I think its limited benefits are largely mooted by mempool enforcement.
284 2015-09-23 23:50:47 <gmaxwell> Ah, that sounds like what jtimon just said.
285 2015-09-23 23:51:17 <jtimon> rusty I still have to answer to the medianTime vs height thing though
286 2015-09-23 23:52:02 <rusty> jtimon: My argument is that it's the "never happens" case where a soft fork doesn't reach consensus.  We need *some* timeout, and recommending 3 years from drafting seems reasonable.
287 2015-09-23 23:52:30 <jtimon> that seems completely orthogonal
288 2015-09-23 23:53:39 <jtimon> oh, maybe related, you mean you want to use "exactly" (median time) 3 years instead of 3 years worth of blocks (regulated by difficulty through median time)?
289 2015-09-23 23:54:44 <rusty> jtimon: I mean exactly what the draft says?  If mean time >= <timeout date>, the soft fork has failed and stop setting the bit please.
290 2015-09-23 23:55:28 <sipa> not "please", "should"
291 2015-09-23 23:55:40 <sipa> the warning mechanism will trigger if you don't :)
292 2015-09-23 23:55:43 <jtimon> I would replace that with timeout height
293 2015-09-23 23:55:57 <rusty> sipa: if we used RFC-language, it'd be MUST.
294 2015-09-23 23:56:27 <sipa> jtimon: why is height better?
295 2015-09-23 23:56:31 <rusty> jtimon: the implementation would be slightly simpler if it was "if block height >= timeout height", but it's significantly less useful.  In practice it will mean 6 year timeouts, not 3 year, because you have to set tumeout height conservatively.
296 2015-09-23 23:57:25 <jtimon> I disagree with "significantly less useful"
297 2015-09-23 23:58:08 <gmaxwell> If you intend to use RFC definitions please cite RFC 2119 specifically. :)  Citing 6919 is optional. :P
298 2015-09-23 23:58:34 <rusty> jtimon: deployment speed is related to wallclock time, not hashing rate.
299 2015-09-23 23:59:31 <jtimon> sipa: it's simpler to use height for everything activation-related instead of block.time or mediantime some times but height when it is buried in the chain (or aparently irght after activation in this case)