1 2015-04-13 00:57:41 <IRCplus> selam
2 2015-04-13 00:59:01 <IRCplus> kimse yokmu
3 2015-04-13 02:24:15 <StephenM347> Hi all, I emailed about this a few days back: https://github.com/scmorse/bitcoin-misc/blob/master/sighash_proposal.md. I'm thinking of either dropping SIGHASH_WITHOUT_TX_VERSION or merging SIGHASH_WITHOUT_PREV_VALUE and SIGHASH_WITHOUT_PREV_SCRIPTPUBKEY together to get it down to 16 flags, for a 2 byte nHashType. Does anyone have any suggestions on which would be better?
4 2015-04-13 02:24:27 <StephenM347> Or if it would be better to just leave the sighash flag at a full 4 byte int
5 2015-04-13 04:10:54 <sipa> StephenM347: how do you deal with recursion? if your sighash includes tx hashes of previous transactions, those must also exclude the fields you are omitting, or you get the same malleability problem one level down
6 2015-04-13 04:12:13 <StephenM347> The TXID flags don't change at all, but you can sign with SIGHASH_WITHOUT_INPUT_TXID and then only have to update the current transaction if the transaction it relies upon changes due to malleability
7 2015-04-13 04:12:39 <StephenM347> sipa: that's my take on how it would work at least
8 2015-04-13 04:12:52 <StephenM347> I'm working on an updated proposal now
9 2015-04-13 04:13:04 <sipa> if you drop the input txid entirely, your transactions become susceptible to replay
10 2015-04-13 04:13:56 <sipa> pay to the same address twice: spend the resulting inputs twice using the same transaction (with only the txid modified)
11 2015-04-13 04:14:43 <StephenM347> True, but since the prev-value is now specified (or can be), you can protect yourself from this being used to give miners big fees.
12 2015-04-13 04:14:55 <StephenM347> In an ideal world, no one would send you coins to the same address more than once anyway
13 2015-04-13 04:14:57 <sipa> agree
14 2015-04-13 04:15:26 <StephenM347> https://github.com/scmorse/bitcoin-misc/blob/master/sighash_proposal_v2.md if you want to read the more updated stuff
15 2015-04-13 04:16:03 <StephenM347> I'm re-doing the code now, making a separate set of params CNewHashTypeParams that are similar to RegTest params to test it out
16 2015-04-13 04:17:12 <sipa> minor nit: Schnorr instead of ECDSA
17 2015-04-13 04:17:41 <sipa> not "standard secp256k1"; Schnorr uses secp256k1 exactly as much as ECDSA does
18 2015-04-13 04:18:27 <StephenM347> sipa: Oh, thakns
19 2015-04-13 04:19:34 <StephenM347> sipa: With Mike hearn's suggestions about how the biggest gain that could be made here would be to limit the amount of re-hashing that needs to be done per input, I'm going to try to run some tests to see how much faster it is
20 2015-04-13 04:25:05 <sipa> well the signature validation is around 2 orders of magnitude slower than the sighashing, but if you can optimize it, sure
21 2015-04-13 04:34:42 <sipa> but i guess you want to optimize for the worst case
22 2015-04-13 04:35:21 <sipa> making something that is slightly worse thsn the current code for normal transactions is perhaps accetable, if youcan make the very slow cases better
23 2015-04-13 04:40:45 <StephenM347> sipa: Yeah, it would probably be good to test it on some average block data and on some specially crafted hash-intensive block data
24 2015-04-13 05:24:32 <Luke-Jr> sipa: replay is not a real issue assuming no key reuse afaik
25 2015-04-13 05:24:53 <sipa> uhu
26 2015-04-13 07:18:03 <jonasschnelli> sipa: CExtKey/Bip32Wallet: Is calling RandAddSeedPerfmon(); followed by "GetRandBytes(vch, sizeof(vch));" sufficient for master key generation? I saw that vch is 32byte. I think this is recommended for Bip32 Master Keys as seed?
27 2015-04-13 07:18:27 <sipa> yes
28 2015-04-13 07:18:46 <jonasschnelli> Okay.
29 2015-04-13 07:23:54 <jonasschnelli> sipa: how about persisting CExtKey? Persist CExtKey's->CKey as a normal key and add corresponding Bip32 data (nChild, depth) as metadata OR adding a serialization method to CExtKey?
30 2015-04-13 07:26:51 <jonasschnelli> i think it would be nice if you could get a Bip32 key from the wallet with a chain argument like "m/0/0/1" and derive subkeys from it. Maybe a rpc call like this "getnewaddress m/0/0/1/x" where x will the next possible key at this depth.
31 2015-04-13 07:27:02 <jonasschnelli> or a bad idea?
32 2015-04-13 07:27:39 <sipa> jonasschnelli: you can store the key's parent and depth as metadata for the key
33 2015-04-13 07:27:43 <jonasschnelli> Of course there should be support for Bip43/44 as standard key path
34 2015-04-13 07:27:49 <sipa> that way you can reconstruct everything
35 2015-04-13 07:27:58 <sipa> and you get the wallet encryption for free
36 2015-04-13 07:28:43 <jonasschnelli> sipa: i also though about the wallet encryption and i'm not sure, if metadata should also end up in a secured space. Metadata (labels!) could harm privacy...
37 2015-04-13 07:29:02 <sipa> yes they could
38 2015-04-13 07:29:03 <jonasschnelli> If one thinks his wallet "is encrypted" he probably expect that everything is encrypted
39 2015-04-13 07:29:19 <sipa> well you would want two levels thrn
40 2015-04-13 07:29:36 <sipa> as you would not be able to do anything without unlocking
41 2015-04-13 07:29:57 <sipa> but you would not want to have the signing keys accessible all the time
42 2015-04-13 07:30:20 <jonasschnelli> sipa: Right. There must be two levels somehow...
43 2015-04-13 07:45:51 <jonasschnelli> was there already a discussion about namespaces? Is it okay when i use namespaces for the whole new wallet implementation?
44 2015-04-13 07:46:06 <jonasschnelli> */use a namespace/
45 2015-04-13 08:05:49 <wump> putting it in a namespace is useful, it means you don't have to name every type *Wallet* but it is implicit in the namespace
46 2015-04-13 08:06:38 <jonasschnelli> wumpus: Okay.
47 2015-04-13 08:07:16 <jonasschnelli> wumpus: btw: Diapolo is pressing about: https://github.com/bitcoin/bitcoin/pull/5665 Looks good IMO but has lack of other ACKs
48 2015-04-13 08:07:17 <wumpus> as always, don't overdo it, but namespaces are a useful code organization concept
49 2015-04-13 08:07:46 <jonasschnelli> wumpus namespace makes sense here because it could interfere with the exiting wallet implementation
50 2015-04-13 08:07:56 <wumpus> yes, that's exactly where they're useful
51 2015-04-13 08:09:45 <wumpus> jonasschnelli: I see. The problem with diapolo's changes is usually that they're hard to ACK because they contain all kinds of auxilary changes, in this case it removes the test for readPaymentRequestFromFile
52 2015-04-13 08:09:56 <wumpus> (which I've also commented on)
53 2015-04-13 08:11:36 <jonasschnelli> Indeed... :)
54 2015-04-13 08:15:37 <wumpus> Peiter Wiulle, lol
55 2015-04-13 08:22:05 <buZz> lol
56 2015-04-13 08:22:22 <buZz> vanillacoin is a full new codesource, and ppl were asking if it was externally auditted
57 2015-04-13 08:22:32 <buZz> the dev is now going 'where is the bitcoin audit report!'
58 2015-04-13 08:22:37 <buZz> to shift the blame :P
59 2015-04-13 08:23:24 <wumpus> the code has been audited various times, usually privately
60 2015-04-13 08:25:46 <wumpus> "where is the audit report" is not generally how it works in open source, the assumption is that many eyes will be on the code because it is published, there may be paid external companies reviewing the code but this is usually at the behest of corporate users, not the project itself
61 2015-04-13 08:28:12 <wumpus> ok - vanillacoin seems to be an altcoin, not a reimplementation of bitcoin, so is off topic here
62 2015-04-13 08:28:45 <buZz> yeah sorry, i was just commenting on the hilarity of reference
63 2015-04-13 08:30:41 <wumpus> yeah
64 2015-04-13 11:52:11 <BW^-> anyone has knowledge of the BitFinex exchange's API?
65 2015-04-13 11:59:59 <wumpus> not me; also, vendor APIs are off topic here
66 2015-04-13 12:17:44 <BW^-> wumpus: any channel for that? #bitfinex asleep.
67 2015-04-13 12:19:32 <Ukra-ine> Beware of scams! Scammers are sending users private messages with bitcoin-stealing malware and offers to trade. We are unable to stop them, so you must protect yourself. NEVER download or run programs from strangers! When in doubt, ask the ops.
68 2015-04-13 13:11:45 <jonasschnelli> wumpus: sry. comment was a bit short: https://github.com/bitcoin/bitcoin/pull/5992#discussion_r28235927
69 2015-04-13 13:12:10 <jonasschnelli> Help message will not be displayed when wallet is disabled
70 2015-04-13 13:12:27 <jonasschnelli> because of https://github.com/bitcoin/bitcoin/pull/5992/files#diff-df7d84ff2f53fcb2a0dc15a3a51e55ceR42
71 2015-04-13 13:12:45 <jonasschnelli> src/wallet/rpcwallet.cpp L42
72 2015-04-13 13:17:29 <wumpus> jonasschnelli: ok, I wasn't sure about that, but if you say so
73 2015-04-13 13:18:23 <wumpus> then I understand the return null
74 2015-04-13 14:20:41 <priidu> does anyone know what is done internally during the "Rescanning" stage of starting the bitcoin daemon?
75 2015-04-13 14:21:11 <priidu> just wondering
76 2015-04-13 14:22:12 <priidu> i.e what's being rescanned
77 2015-04-13 14:24:27 <wumpus> the wallet keys comb through the block chain for transactions
78 2015-04-13 14:31:00 <priidu> wumpus: ok, thanks
79 2015-04-13 14:31:25 <priidu> was trying out the watch-only address functionality about a month ago
80 2015-04-13 14:31:53 <priidu> and shot myself in the foot be choosing a random address from bc.i
81 2015-04-13 14:32:56 <priidu> which happenes to be used by some gambling site, as i found out later :P
82 2015-04-13 14:33:09 <priidu> so boatloads of micropayments, lol
83 2015-04-13 14:35:28 <rgenito> bc.i ?
84 2015-04-13 14:35:33 <priidu> good think i used a random junk wallet :P
85 2015-04-13 14:35:39 <priidu> blockchain.info
86 2015-04-13 14:36:06 <gdm85> a value of -1 for synced_blocks/synced_headers means the node is not fully synchronized with net?
87 2015-04-13 14:37:43 <gmaxwell> gdm85: no, it just means we have no information.
88 2015-04-13 14:38:09 <gdm85> gmaxwell: ok, thanks.
89 2015-04-13 14:38:28 <gmaxwell> gdm85: we infer that value from advertisements from the node; if there have been no blocks we just don't know.
90 2015-04-13 14:39:16 <gdm85> gmaxwell: right now I am considering the node "not yet ready" when it has synced_blocks == -1
91 2015-04-13 14:39:20 <gdm85> perhaps I should refine this condition check.
92 2015-04-13 14:39:37 <gmaxwell> gdm85: not ready for what?
93 2015-04-13 14:40:02 <gdm85> gmaxwell: to accept other RPC calls. as in: busy checking blocks, for example
94 2015-04-13 14:41:03 <gdm85> use case: a (testnet) node starting up with pre-existing chainstate + blocks files
95 2015-04-13 14:41:33 <gmaxwell> gdm85: ah, well thats incorrect.
96 2015-04-13 14:42:04 <gmaxwell> No node ever knows when its synced it's always syncing (though it can potentially have evidence that its not synced).
97 2015-04-13 14:43:45 <gdm85> sure. I had erroneously inferred that synced_blocks == -1 is always indicator of some problematic situation.
98 2015-04-13 14:45:09 <gdm85> basically this is an integration test where I want to know that node has finished its homework on blockchain and is connected to a (specific) peer.
99 2015-04-13 14:45:38 <gdm85> gmaxwell: any advice about which information to poll (via API) to know when all the existing blocks have been checked?
100 2015-04-13 14:46:03 <gdm85> (existing as on filesystem)
101 2015-04-13 14:47:11 <wumpus> ask the node for getblockchaininfo height ?
102 2015-04-13 14:48:49 <gdm85> wumpus: you mean verificationprogress? I don't see height there
103 2015-04-13 14:49:23 <wumpus> no, not progress, simply the block height of the node itself
104 2015-04-13 14:49:32 <wumpus> but I'm probably misunderstanding what you want to do
105 2015-04-13 14:50:10 <wumpus> 'verification progress' is a very rough estimate, don't use it for anything important
106 2015-04-13 14:50:20 <gdm85> I see.
107 2015-04-13 14:50:54 <gmaxwell> gdm85: he's just saying to read the blocks field of getblockchaininfo.
108 2015-04-13 14:51:37 <gdm85> it's a new testnet node, but it is provided with pre-existing 'blocks' and 'chainstate' directories, thus when bitcoind starts it will check all of them (as expected). However I would like to avoid start using the node until that is completed.
109 2015-04-13 14:52:11 <gdm85> gmaxwell: if I read the blocks field, then I need a term of comparison right?
110 2015-04-13 14:52:14 <wumpus> if you copy blocks and chainstate (including the databases) then it has to do nothing
111 2015-04-13 14:52:32 <gmaxwell> gdm85: headers, if you're just talking about locally.
112 2015-04-13 14:52:43 <gmaxwell> (and if it doesn't know the header yet, then you just cannot know if you're done yet)
113 2015-04-13 14:53:02 <wumpus> you know how many blocks there were before copying
114 2015-04-13 14:53:33 <gdm85> wumpus: you're right. I think that sometimes the provided blocks are invalid/incomplete, thus for some reason I get the node in an unusable status (at least temporarily).
115 2015-04-13 14:53:49 <gdm85> gmaxwell: makes sense. thanks.
116 2015-04-13 14:53:52 <wumpus> gdm85: make sure that the node is shut down before copying its data
117 2015-04-13 14:54:21 <gdm85> wumpus: I think that's what is happening: 2015-04-13 14:49:09 ERROR: ContextualCheckBlockHeader : rejected nVersion=2 block
118 2015-04-13 14:54:35 <gdm85> either the node was not shutdown before copying, or something more fishy :)
119 2015-04-13 14:55:11 <wumpus> well if you copy the database while the node is running the outcome is entirely unpredictable
120 2015-04-13 14:55:59 <gdm85> I can understand that. The automation should indeed stop the node before providing those files.
121 2015-04-13 14:56:06 <gdm85> ACTION wears miner helmet and starts digging
122 2015-04-13 15:09:20 <priidu> lol
123 2015-04-13 15:09:36 <priidu> a total of about ~50,000 transactions from the address I mentioned earlier
124 2015-04-13 15:09:44 <priidu> talk about address reuse
125 2015-04-13 15:09:45 <priidu> :p
126 2015-04-13 15:14:54 <tjopper> priidu thats nothing, look at satoshi dice :P https://www.blocktrail.com/BTC/address/1dice8EMZmqKvrGE4Qc9bUFf9PX3xaYDp
127 2015-04-13 15:15:46 <tjopper> ~3193000
128 2015-04-13 15:19:16 <priidu> lol
129 2015-04-13 15:19:50 <priidu> at least I didn't blindly pick the worst address for testing watch-only addresses then
130 2015-04-13 15:20:25 <priidu> the 50k one bloated the wallet to about 30MiB
131 2015-04-13 15:20:28 <gmaxwell> priidu: 50k should work fine.
132 2015-04-13 15:20:47 <priidu> yeah, everything worked fine
133 2015-04-13 15:21:24 <priidu> i'm currently on a pretty slow box so it just took some time
134 2015-04-13 15:21:39 <gmaxwell> priidu: sometime in ~2012 I tried 'watchwallet' importing the whole blockchain into the wallet; worked okay, was kinda slow.. made the wallet a couple gigabytes.
135 2015-04-13 15:21:49 <priidu> lol
136 2015-04-13 15:22:52 <gdm85> wumpus: it might be the case that it's not an issue of corrupted/partially copied files.
137 2015-04-13 15:23:02 <gdm85> on one of the reference nodes I see: Reject block code 0: badprevblk: hash 000000000bf5984e31f2dcaba37092678a361cbcb024ee26196a03c395b0d018
138 2015-04-13 15:35:03 <jonasschnelli> sipa: "[09:27:58] and you get the wallet encryption for free": For a m/44/0/0/<key> generation; don't store the masterkey-privkey? Only give out the masterseed once when creating the wallet (so user can create a backup or something)?
139 2015-04-13 15:35:46 <jonasschnelli> * backup = write down the seed on a paper (non mnemonic)
140 2015-04-13 15:35:53 <jonasschnelli> * (as hex)
141 2015-04-13 15:36:17 <BlueMatt> hmmmm....why does submitblock run unlocked and, as the first thing it does, touch mapBlockIndex?
142 2015-04-13 15:36:49 <gmaxwell> BlueMatt: 0.10rc3 time I guess. :-/
143 2015-04-13 15:38:13 <gmaxwell> is there some tidy stl magic we can do to get lock asserts in some of these datastructures in lockorder builds?
144 2015-04-13 15:40:18 <BlueMatt> gmaxwell: #ifdef DEBUG_LOCKORDER typedef LockCheckingMap BlockMap #else typedef boost::unordered_map BlockMap #endif
145 2015-04-13 15:40:19 <BlueMatt> I guess
146 2015-04-13 15:41:04 <gmaxwell> yea, I guess my question is that if we can do a LockCheckingMap that will typecheck without going around chaging how we use the unordered map throughout the code.
147 2015-04-13 15:41:46 <wumpus> anything that AssertLockHeld won't work for?
148 2015-04-13 15:42:04 <gmaxwell> wumpus: only that we have to remember to add one every new place we access the object.
149 2015-04-13 15:42:16 <gmaxwell> vs putting it inside the access methods of the object itself.
150 2015-04-13 15:42:33 <BlueMatt> gmaxwell: I was just suggesting making a thin wrapper object that AssertLockHelds and then passes it on to the boost map
151 2015-04-13 15:42:37 <wumpus> there's also some capability in modern compilers to check data guarded by locks
152 2015-04-13 15:42:51 <BlueMatt> should by small enough, though it would be nice to do something fancier that is fewer loc
153 2015-04-13 15:43:03 <wumpus> see threadsafety.h, we don't actually use it anywhere though
154 2015-04-13 15:43:13 <gmaxwell> BlueMatt: thats what I was suggesting to, but my question was if we just do this if it'll actually typecheck.
155 2015-04-13 15:44:20 <BlueMatt> oh, I misread typecheck somehow...though it should be doable to make it typecheck, Id think
156 2015-04-13 15:46:59 <wumpus> looks like the mapblockindex check was introduced in 60755dbf762cd36bb01859b5e9b0bd4447f23229
157 2015-04-13 15:48:44 <gmaxwell> ACTION hands Luke-Jr and sipa the brown paper bags. :)