1 2012-12-26 00:30:40 <Lefont> Hey there merry christmas everyone!... was hoping someone could help me with a question i have? I am trying to install bitcoinD on an amazon server server ...any one know of any guides that can help me do this
  2 2012-12-26 00:31:48 <pigeons> only thing i can think is amazon uses ecdsa-less openssl i think so roll your own openssl for bitcoin
  3 2012-12-26 00:32:35 <Scrat> pigeons: or not use AMI
  4 2012-12-26 00:32:48 <pigeons> yeah or that
  5 2012-12-26 00:35:28 <da2ce7> https://github.com/bitcoin/bitcoin/pull/2128
  6 2012-12-26 00:39:18 <Lefont> not sure what you mean pigeon? ..also i am working on a web project and we decided to host on amazon..so for right now i dont have a choice...we orginaly had the headless bitcoind cient running on another amazon instance but we had to upgrade it...it was set up by someone else =(
  7 2012-12-26 00:40:32 <pigeons> well you have a choice cause you can run debian on aws
  8 2012-12-26 00:41:02 <pigeons> but yeah it runs on amazon linux, but you need to compile an openssl that has ecdsa support
  9 2012-12-26 01:34:46 <gmaxwell> wumpus: Can you explain ExitTimeout to me? (in particular why do we have it and why only on win32?)  The commit that added it wasn't super informative. :P
 10 2012-12-26 01:52:52 <etotheipi_> gmaxwell: how much work would it be to make a variant of the satoshi client that forwards all transactions it receives?  it is just as simple as changing MIN_TX_FEE and MIN_RELAY_TX_FEE to zero?
 11 2012-12-26 01:54:11 <etotheipi_> gmaxwell: (err, not ALL tx, just the low-fee tx that wouldn't normally be forwarded)
 12 2012-12-26 01:57:16 <Luke-Jr> etotheipi_: that's not how I do it, at least
 13 2012-12-26 01:57:36 <etotheipi_> well, how *do* you do it?
 14 2012-12-26 01:58:00 <Luke-Jr> http://gitorious.org/~Luke-Jr/bitcoin/luke-jr-bitcoin/commits/free_relay
 15 2012-12-26 01:58:18 <gmaxwell> etotheipi_: define all?  That would avoid the fee related filtering. But not non-standardness checks. Watcha trying to accomplish?
 16 2012-12-26 01:58:29 <Luke-Jr> etotheipi_: actually, you also need to bypass the non-standard checks
 17 2012-12-26 01:58:51 <gmaxwell> Relaying by itself isn't going to do you any good if your peers won't relay either.
 18 2012-12-26 01:59:55 <etotheipi_> gmaxwell, I'm writing a monitoring script that will run (as usual) with bitcoind as my gateway to the network
 19 2012-12-26 02:00:15 <etotheipi_> and I just want to make sure that any zero-fee tx that the bitcoind instance sees make it to my script
 20 2012-12-26 02:00:32 <Luke-Jr> etotheipi_: like #Bitcoin-watch ?
 21 2012-12-26 02:00:58 <etotheipi_> Luke-Jr: I don't know what that is
 22 2012-12-26 02:01:10 <Luke-Jr> /j #bitcoin-watch
 23 2012-12-26 02:01:35 <etotheipi_> but yeah, I'm not trying to send zero-fee tx, I just want to make sure that bitcoind forwards them to me (and I guess all the peers)
 24 2012-12-26 02:03:11 <gmaxwell> yea, you don't want what you think you want.
 25 2012-12-26 02:03:27 <gmaxwell> Presumably for monitoring you actually want to see invalid transactions, right?
 26 2012-12-26 02:03:36 <gmaxwell> But if you relay those to your peers they will ban your IP.
 27 2012-12-26 02:04:26 <Luke-Jr> 
ACTION ponders if 0.8 is going to ban blockchain.info O.o

 28 2012-12-26 02:04:28 <gmaxwell> (not to mention that you'll be a DOS multiplier??? which is why nodes get banned for black and white invalidity where it can be easily detected)
 29 2012-12-26 02:04:59 <etotheipi_> gmaxwell: what do you mean by "invalid"?
 30 2012-12-26 02:05:08 <gmaxwell> Luke-Jr: what, because of noncanonical signatures? no??? we wouldn't DoS for that, and they've mostly fixed it.
 31 2012-12-26 02:05:20 <Luke-Jr> gmaxwell: because master is DoS-banning for invalid P2SH
 32 2012-12-26 02:05:42 <gmaxwell> Luke-Jr: are they forwarding invalid p2sh?
 33 2012-12-26 02:05:50 <etotheipi_> I don't want "invalid" in the strictest sense
 34 2012-12-26 02:05:51 <Luke-Jr> gmaxwell: I imagine so, since they index them
 35 2012-12-26 02:06:05 <etotheipi_> I just want transactions that normally wouldn't meet the fee requirement
 36 2012-12-26 02:06:16 <etotheipi_> and I know stock bitcoind will not forward such transactions
 37 2012-12-26 02:06:22 <gmaxwell> etotheipi_: okay, then turning down your fee requirements would work.
 38 2012-12-26 02:06:34 <gmaxwell> If you start opening up other things you may risk your node getting banned.
 39 2012-12-26 02:06:48 <Luke-Jr> I think even my return 0 from GetMinFee will still implicate the free-flooding code
 40 2012-12-26 02:07:00 <etotheipi_> it sounds like there's a subnetwork of nodes that have also turned down the fee reqt
 41 2012-12-26 02:07:10 <gmaxwell> etotheipi_: no, not really.
 42 2012-12-26 02:07:24 <Luke-Jr> etotheipi_: *mostly* just me/Eligius
 43 2012-12-26 02:07:42 <gmaxwell> (there were some vague efforts, and at times even luke's own nodes haven't managed to do it)
 44 2012-12-26 02:08:06 <gmaxwell> and luke's don't set them at zero.
 45 2012-12-26 02:08:11 <gmaxwell> IIRC.
 46 2012-12-26 02:08:27 <etotheipi_> I seem to remember hearing in the past that lots of tx that normally wouldn't be free... will still get mined , usually
 47 2012-12-26 02:08:52 <etotheipi_> that means there must be at least one significant miner, and some critical threshold of nodes that forward them (I guess it wouldn't have to be a large proportion)
 48 2012-12-26 02:08:52 <gmaxwell> etotheipi_: Nope.
 49 2012-12-26 02:09:36 <gmaxwell> etotheipi_: There is a lot of confusion that arises because the free criteria changes, and because big pools process their own transactions. (also, luke has his backdoor agreements with varrious folks)
 50 2012-12-26 02:09:36 <Luke-Jr> gmaxwell: I do zero for relaying
 51 2012-12-26 02:09:51 <etotheipi_> well... I guess I'll find out
 52 2012-12-26 02:09:56 <gmaxwell> etotheipi_: er, changes was ambigious, I mean its time dependant.
 53 2012-12-26 02:10:24 <Luke-Jr> hmm, valgrind is behaving very oddly
 54 2012-12-26 02:10:57 <Luke-Jr> realloc(p, 0x7fffffff) is returning NULL, yet it claims ==6394==   total heap usage: 2 allocs, 1 frees, 2,147,483,648 bytes allocated
 55 2012-12-26 02:10:58 <gmaxwell> etotheipi_: there is, for example that crazy ShadowHerringbone guy on the forum who is utterly convinced that the fee rules are purely local because he has had successful txn "violating" them??? but whats really happening is that he forms a txn that violates them and then just retransmits them until they actually do get relayed.
 56 2012-12-26 02:14:57 <etotheipi_> you don't get banned for that, correct?
 57 2012-12-26 02:15:39 <gmaxwell> Nope.
 58 2012-12-26 02:16:34 <gmaxwell> I'm having a hard time being clear tonight, I see???  "until they actually do get relayed" because the criteria depends on how many confirmations a transaction has, so a couple more blocks is often enough to get a transaction over the hump.
 59 2012-12-26 02:16:45 <stealth222> I'm working on a listener architecture that can plug in to filter any messages from peers and pass them through to another component
 60 2012-12-26 02:17:06 <stealth222> so you can do that without relaying invalid stuff to peers
 61 2012-12-26 02:18:15 <stealth222> I'd also love to be able to receive notifications on strange transactions and doublespend attempts and stuff like that
 62 2012-12-26 02:18:26 <gmaxwell> Yea, that what you'd want for a monitoring thing.
 63 2012-12-26 02:18:41 <gmaxwell> stealth222: keep in mind that it's super duper lossy for doublespend attempts since they aren't relayed.
 64 2012-12-26 02:19:27 <stealth222> gmaxwell: to really do doublespend attempt detection, you'd have to be connected to the right set of peers
 65 2012-12-26 02:19:38 <stealth222> probably miners are the best bet
 66 2012-12-26 02:19:46 <SomeoneWeird> does the satoshi client actually ban nodes ?
 67 2012-12-26 02:19:52 <gmaxwell> SomeoneWeird: sure.
 68 2012-12-26 02:20:03 <SomeoneWeird> ic
 69 2012-12-26 02:20:04 <etotheipi_> well, I'm trying to detect something fairly specific and unexciting, so I don't need anything flexible or customizable
 70 2012-12-26 02:20:04 <SomeoneWeird> TIL
 71 2012-12-26 02:20:22 <etotheipi_> I just want to make sure I see all valid tx, not just the ones with the correct fee
 72 2012-12-26 02:20:30 <Luke-Jr> SomeoneWeird: only if they blatently violate immovable rules
 73 2012-12-26 02:20:34 <stealth222> the simplest approach, etotheipi, is probobably to stick in your own custom code into ProcessMessage and send that stuff over to another program
 74 2012-12-26 02:20:45 <Luke-Jr> SomeoneWeird: like flooding blocks with low hashes
 75 2012-12-26 02:21:23 <gmaxwell> stealth222: "to really do doublespend" even thats really lossy. You can't identify most of the miners in any case, nothing obligates them to realay any particular transactions, etc.
 76 2012-12-26 02:21:50 <stealth222> gmaxwell, I'm saying hypothetically - if you could
 77 2012-12-26 02:22:00 <etotheipi_> stealth222: actually I think the minfee() returns zero thing is perfect... I've already got an armory-based listener connect to my bitcoind hearing the relayed tx
 78 2012-12-26 02:22:00 <stealth222> if you could know the mempool of a good set of miners
 79 2012-12-26 02:22:09 <gmaxwell> stealth222: We could extend the protocol to make doublespend detection pretty reliable absent a Finney attack, but until we do no matter how hard you try it will be pretty flaky.
 80 2012-12-26 02:22:25 <SomeoneWeird> ah, makes sense Luke-Jr
 81 2012-12-26 02:22:47 <stealth222> gmaxwell: I suppose you're right. I still think a listener architecture that supports custom filters is a good idea :)
 82 2012-12-26 02:23:08 <gmaxwell> stealth222: sure sure, I agree. useful for stuff, but not guarding against conflicting transactions.
 83 2012-12-26 02:24:07 <gmaxwell> stealth222: if you feel like doing more wallet stuff with the reference client??? we really ought be be able to return negative confirmations (e.g. for transactions which are conflicted and are expected to never confirm??? e.g. they should show -N confirmations where N is the depth of the earliest conflict in the chain)
 84 2012-12-26 02:25:30 <Luke-Jr> ^ once this is done, allowing users to delete transactions with -6 confirms would probably be trivial
 85 2012-12-26 02:25:32 <stealth222> ah, yes. I could certainly do that
 86 2012-12-26 02:27:41 <gmaxwell> Yea, we should stop relaying at -1 (for one, that identifies us as the source/dest of the txn!).  And allow switching a transaction directly into an 'aborted' state directly at -6.
 87 2012-12-26 02:27:50 <etotheipi_> when's the last time the fee rules were changed?  is it possible that there's a sufficient subset of nodes still running an older version that would essentially have different forwarding/acceptance rules?
 88 2012-12-26 02:28:11 <gmaxwell> etotheipi_: over a year ago, and the prior rules required higher fees.
 89 2012-12-26 02:28:15 <Luke-Jr> etotheipi_: pretty likely
 90 2012-12-26 02:28:21 <Luke-Jr> etotheipi_: Deepbit still runs 0.3.x
 91 2012-12-26 02:28:37 <etotheipi_> I don't really care which direction.. the important part is being able to exploit the split
 92 2012-12-26 02:28:43 <gmaxwell> Luke-Jr: you're missing the word 'sufficient'
 93 2012-12-26 02:28:49 <gmaxwell> etotheipi_: exploit for what purpose?
 94 2012-12-26 02:29:31 <gmaxwell> etotheipi_: if you want to doublespend sdice, you can just use luke's pool as he'll mine txn with much lower fees than most nodes will relay.
 95 2012-12-26 02:30:22 <etotheipi_> gmaxwell: my only concern was whether transactions can be created that will be accepted by one subnetwork but not the others
 96 2012-12-26 02:30:47 <gmaxwell> There should also probably be an abort-requested state that you can put a transaction in so long as it has <=0 confirms, which stops relaying and converts into an abort state after 144 blocks or something.
 97 2012-12-26 02:30:58 <gmaxwell> etotheipi_: uh. define accepted?
 98 2012-12-26 02:31:21 <etotheipi_> gmaxwell: forwarded and/or mined
 99 2012-12-26 02:31:39 <gmaxwell> etotheipi_: in the chain? No that would be horrible. For relay? sure. well??? some nodes at least. No reason to think that the oddball nodes form a connected partition.
100 2012-12-26 02:32:07 <stealth222> it would also be nice to know how long the dependency chain for an unconfirmed transaction is. That's to say, how many transactions would need to get confirmed until it can get confirmed
101 2012-12-26 02:32:18 <etotheipi_> gmaxwell: didn't you just say that Luke's pull will mine such tx?
102 2012-12-26 02:32:32 <stealth222> and perhaps also get a fee total for all these transactions
103 2012-12-26 02:32:37 <stealth222> and size
104 2012-12-26 02:32:42 <stealth222> so you can assess double-spend risk
105 2012-12-26 02:32:48 <gmaxwell> the last fee change was over a year ago, it decreased the basefee for relay/mining from 0.01. Any node running code that old will tend to mine invalid blocks. If there are miners with different criteria they've been modified by their operators.
106 2012-12-26 02:33:04 <gmaxwell> etotheipi_: luke's _pool_.
107 2012-12-26 02:33:47 <etotheipi_> er... yeah, "pool"
108 2012-12-26 02:33:50 <Luke-Jr> I think there's a disconnect on meanings here. All nodes will accept any valid transaction in blocks, that is consistent across ~all clients.
109 2012-12-26 02:33:57 <gmaxwell> stealth222: what you really want to know is the _size_ of a transaction including all its unconfirmed parents, or better??? the group priority.. though no one yet uses that for txn decisions.
110 2012-12-26 02:34:01 <Luke-Jr> But they vary on what they will put in their own
111 2012-12-26 02:34:12 <etotheipi_> Luke-Jr: agreed...
112 2012-12-26 02:35:00 <gmaxwell> etotheipi_: Yes, luke runs non-standard rules but in may ways he is unique in that. (though there are other pools with some differences in their behavior)
113 2012-12-26 02:36:28 <gmaxwell> etotheipi_: unfortunately it's basically impossible to sniff out the behavior because you can't see any private dealings. :(
114 2012-12-26 02:37:25 <gmaxwell> e.g. luke's pool mines txn's selected by parties he has business relationships with even if they wouldn't meet even his rules. And all major pools mine their own payouts even if they wouldn't meet their own inclusion rules.
115 2012-12-26 02:37:43 <gmaxwell> what you can try to do is observe when things don't get mined.
116 2012-12-26 02:38:24 <gmaxwell> But even that is lossy, ??? some parties block all dice transactions now. so you might guess that txn with those fees aren't enough to get them mined, when in fact they are but they're being selectively blocked.
117 2012-12-26 02:41:07 <gmaxwell> so on my public node with 98 peers currently I see that I have one peer that is old enough that it wouldn't fail to relay a low priority txn.
118 2012-12-26 02:41:17 <gmaxwell> 0.3.19 will, IIRC.
119 2012-12-26 02:41:48 <gmaxwell> but ...         "startingheight" : 178622,
120 2012-12-26 02:41:50 <gmaxwell> it's stuck.
121 2012-12-26 02:46:59 <etotheipi_> one other clarification:  it's true that if bitcoind relays a tx, that bitcoind instance also added it to its memory pool, so any conflicting tx will be DOA?
122 2012-12-26 02:47:20 <etotheipi_> are there instances where bitcoind forwards but doesn't add it to its own pool?
123 2012-12-26 02:51:43 <gmaxwell> etotheipi_: all actions with a txn recieved from the network get conditioned on the node accepting it into its mempool.
124 2012-12-26 02:52:30 <gmaxwell> That said, I believe a node with a local wallet will rebroadcast its own _conflicted_ transactions.
125 2012-12-26 02:52:36 <stealth222> gmaxwell: so the scenario you're considering presumably only deals with transactions in the mempool, right?
126 2012-12-26 02:52:44 <stealth222> for the negative confirmation thing
127 2012-12-26 02:53:18 <stealth222> I'm still getting acquainted with the details of all the reference client structures
128 2012-12-26 02:53:27 <gmaxwell> stealth222: the negative confirmations would be in the _chain_. Yea, not entirely trivial.
129 2012-12-26 02:53:55 <stealth222> is it the case that all transactions stored in the reference client are either in the main chain or are in the mempool?
130 2012-12-26 02:54:08 <gmaxwell> e.g. you'd have a unconfirmed txn in the wallet (and presumably the mempool) and then it gets conflicted by a new block. This ejects it from the mempool and the wallet should read -1.
131 2012-12-26 02:54:22 <gmaxwell> stealth222: No, the wallet has transactions in it too.
132 2012-12-26 02:55:14 <stealth222> oh, right. so then the scenario is you receive bitcoins, the transaction is stored in your wallet (it goes through the mempool first), then a block comes in that conflicts, it gets removed from the mempool but stays in your wallet?
133 2012-12-26 02:55:17 <gmaxwell> So for example, if someone pays you, and then that payment is conflicted by a block the payment will only exist in your wallet??? it's not permitted in the mempool because its invalid relative to the current chain.  If there is a reorg and the conflict falls out you'll rebroadcast and potentially get it back in.
134 2012-12-26 02:55:24 <gmaxwell> Bingo.
135 2012-12-26 02:55:31 <gmaxwell> And right now it just shows confirms forever.
136 2012-12-26 02:55:45 <gmaxwell> This is especially annoying if you're the sender and only some of the inputs it consumes are conflicted.
137 2012-12-26 02:56:04 <gmaxwell> As that basically locks up some of your funds and makes then forever unspendable (until you get out a hex editor)
138 2012-12-26 02:56:07 <stealth222> what if both transactions are to that wallet? will the wallet ever see the second one?
139 2012-12-26 02:56:17 <gmaxwell> And as a reciever you don't discover you've been robbed until much later.
140 2012-12-26 02:56:37 <gmaxwell> stealth222: It will.
141 2012-12-26 02:57:04 <stealth222> so the second one will be treated correctly by the wallet but not the first
142 2012-12-26 02:57:15 <stealth222> that's to say, the one that does get confirmed
143 2012-12-26 02:57:39 <gmaxwell> Right. Well 'correctly' ... it'll just show as an unconfirmed txn forever.
144 2012-12-26 02:58:39 <gmaxwell> So it would be good if we could show negative confirmations??? the reciever would learn they got robbed faster.. and we could allow the sender to eventually 'abort' a hopeless transaction. (if we just allow random aborting people would footgun themselves by aborting transactions which aren't at all hopeless)
145 2012-12-26 02:59:47 <stealth222> so this is then *only* an issue with wallet transactions. not mempool transactions, right?
146 2012-12-26 02:59:52 <gmaxwell> Right.
147 2012-12-26 03:00:11 <stealth222> getrawtransaction won't keep returning the killed transaction
148 2012-12-26 03:00:18 <stealth222> but gettransaction will
149 2012-12-26 03:00:25 <stealth222> correct?
150 2012-12-26 03:00:36 <gmaxwell> the fact that getrawtransaction doesn't consult the wallet seems a bit buggy to me. :P
151 2012-12-26 03:00:43 <gmaxwell> but yes.
152 2012-12-26 03:01:30 <stealth222> I'm trying to work with what's there but with an eye towards an eventual split between the block chain/relay engine and the wallet
153 2012-12-26 03:01:44 <gmaxwell> Hm! point.
154 2012-12-26 03:02:49 <gmaxwell> The data you need for this is 'txn X got consumed at height Y'  the blockchain can easily tell us if its currently available for spending or not??? but getting the height is problematic.
155 2012-12-26 03:03:33 <gmaxwell> I wonder if actually counting the negative height is worth it then. :-/ Since you can't just query the utxo set for that information.
156 2012-12-26 03:04:32 <stealth222> yeah, we'd need to store another index to make retrieval fast
157 2012-12-26 03:04:42 <gmaxwell> yea, thats not worth it.
158 2012-12-26 03:04:54 <gmaxwell> Luke-Jr: thoughts on this?
159 2012-12-26 03:07:53 <stealth222> the behavior you're describing is very problematic, though - it's begging for a solution
160 2012-12-26 03:09:16 <stealth222> if we don't index the block chain to make arbitrary retrieval fast, at least we can check the wallet for conflicts when adding a tx to the mempool
161 2012-12-26 03:09:35 <stealth222> or when synching, rather
162 2012-12-26 03:10:01 <stealth222> and conflicted transactions can be flagged
163 2012-12-26 03:10:09 <gmaxwell> Right but that breaks the wallet chain abstraction barrier.
164 2012-12-26 03:10:19 <stealth222> yeah, it's far from ideal
165 2012-12-26 03:10:22 <gmaxwell> (esp because it needs to handle reorgs)
166 2012-12-26 03:10:47 <gmaxwell> The alternative is to just note that its parent transactions appear to not exist.
167 2012-12-26 03:11:00 <gmaxwell> E.g. the confirms is negative, we just don't know how negative it is.
168 2012-12-26 03:12:13 <stealth222> or yet another alternative is to add an RPC call to do a slow recursive search
169 2012-12-26 03:12:59 <stealth222> presumably this isn't something someone would be doing very frequently
170 2012-12-26 03:13:27 <gmaxwell> uh. it should be displayed in the UI in a very clear way. Is every 20ms not frequently? :P
171 2012-12-26 03:13:47 <stealth222> the wallet shows 0 conf. the recipient starts to suspect they might have gotten cheated, they can then invoke a check
172 2012-12-26 03:13:52 <gmaxwell> (e.g. a conflicted txn should turn red and display unhappy faces and skulls and crossbones and such)
173 2012-12-26 03:14:00 <stealth222> or we can have a timer
174 2012-12-26 03:14:11 <stealth222> if a transaction remains at 0 confirmations for a certain amount of time, we do the check
175 2012-12-26 03:14:37 <gmaxwell> Rescaning the blockchain isn't viable long term no matter how infrequently you hope to poll it. For one, the node may not even have the data.
176 2012-12-26 03:15:04 <stealth222> the long term solution is to have a way of querying block by transaction hash
177 2012-12-26 03:15:10 <gmaxwell> but the most important data??? is the transaction potentially viable from my perspective??? is always available and cheaply so.
178 2012-12-26 03:15:25 <gmaxwell> stealth222: no, thats not viable for a general node.
179 2012-12-26 03:15:50 <gmaxwell> we just took out the ability to do that in git. 0.7.x can.
180 2012-12-26 03:16:54 <gmaxwell> having an full index over past transactions means you can't forget them, and it's a rather large and growing index (>>1gb now) which is expensive to maintain and increases the cost of running a node.
181 2012-12-26 03:17:47 <stealth222> what about a bloom filter over spent outputs?
182 2012-12-26 03:18:12 <gmaxwell> stealth222: you need to be able to reorg.
183 2012-12-26 03:18:52 <gmaxwell> subtraction from a bloom filter requires a full reconstruction. Unless you have a counting bloom filter, which greatly increases the size, and still leaves it lossy.
184 2012-12-26 03:18:58 <gmaxwell> I also don't see what that accomplishes.
185 2012-12-26 03:19:10 <gmaxwell> You can tell when an input is not spendable by the unspent txoutset returning nothing.
186 2012-12-26 03:19:49 <gmaxwell> You just can't tell why it's not spendable. e.g. where it was conflicted, by whom, or if the parent never existed in the first place.
187 2012-12-26 03:20:05 <gmaxwell> (though I don't think we'll ever accept an orphan txn into the wallet.)
188 2012-12-26 03:24:36 <gmaxwell> I suppose that this is better in any case. How would a Confirmations: -conflict_depth   work when a txn itself wasn't conflicted, but one of its inputs was? I suppose you could state that height, but it gets messy.
189 2012-12-26 03:26:07 <stealth222> you'd have to recurse the inputs, no?
190 2012-12-26 03:26:53 <gmaxwell> Yes, but then the idea of tracking the height of conflicts with just your wallet txn becomes messy and you end up needing to potentially track it for all parents
191 2012-12-26 03:30:19 <stealth222> there are two conflicting objectives here: 1) the ability to perform fast queries on arbitrary bitcoin data. 2) a lightweight client that provides a wallet which doesn't require connecting to a trusted server
192 2012-12-26 03:30:50 <gmaxwell> nope.
193 2012-12-26 03:30:52 <gmaxwell> :P
194 2012-12-26 03:31:12 <stealth222> if I'm wrong, please enlighten me :)
195 2012-12-26 03:32:04 <gmaxwell> It's not a question of running a 'a lightweight client', its a question keeping it viable to run a bitcoin node on commodity hardware at all.
196 2012-12-26 03:32:40 <stealth222> a bitcoin node shouldn't have to store all the data locally - but the bitcoin protocol should still allow queries of it
197 2012-12-26 03:32:44 <gmaxwell> And, sure, support for fast queries on arbitrary bitcoin data is fine as an optional thing??? for local block explorer stuff.. but it's not something we can make wallet functionality depend on.
198 2012-12-26 03:33:03 <gmaxwell> stealth222: yes/no.
199 2012-12-26 03:33:22 <gmaxwell> It's not acceptable to ask nodes to process queries for data which is unrelated to the current operation of the network.
200 2012-12-26 03:34:32 <gmaxwell> The problem is that if you require a 1 tb index of historic data to run a bitcoin node (so that it can answer queries for your 'lite wallets') then basically no one will do it??? except perhaps people who have a financial or economic-political interest in giving dishonest replies.
201 2012-12-26 03:35:06 <stealth222> or in charging fees for the service
202 2012-12-26 03:35:26 <gmaxwell> So we're unwilling to add any query capability facing the public network which isn't ~free once you're taking the cost of enforcing the bitcoin rules.
203 2012-12-26 03:35:55 <gmaxwell> Sure. Thats possible too??? but it rapidly diverges from your "which doesn't require connecting to a trusted server" if you expect a wallet to use it.
204 2012-12-26 03:36:26 <stealth222> right - that's my point
205 2012-12-26 03:37:44 <gmaxwell> Well I'm not following you then.  A wallet's basic functionality doesn't actually need queries which are expensive for a node which is doing the minimum required to validate the chain. It does if you want fancy block explorerish functionality.
206 2012-12-26 03:38:31 <gmaxwell> And my answer to that is??? connect to someone willing to provide that, because if we try to force everyone to providing it by making it part of the protocol and increasing the cost of running a full node tending-to-infinity-fold we'll just doom the whole system.
207 2012-12-26 03:39:00 <stealth222> a lightweight wallet would still need to go through new blocks to find transactions belonging to it even if it can just store portions of the merkle tree, right?
208 2012-12-26 03:39:29 <kiba> 
ACTION works on a game that will be a tactical simulator like no others!

209 2012-12-26 03:39:37 <kiba> compelete with cat officiers!
210 2012-12-26 03:39:39 <stealth222> I haven't been up on the latest developments of the pruning stuff
211 2012-12-26 03:39:40 <gmaxwell> stealth222: Thats what the bloom filter stuff created by TD and Bluematt addresses.
212 2012-12-26 03:40:06 <gmaxwell> stealth222: basically you can tell your peers to filter their block advertisments to you.
213 2012-12-26 03:41:20 <gmaxwell> Although since the maximum long term average datarate is only about 14kbytes/sec... meh.. its mostly only really interesting for mobile clients.
214 2012-12-26 03:43:05 <stealth222> isn't that sort of contrary to the objective you stated earlier? "It's not acceptable to ask nodes to process queries for data which is unrelated to the current operation of the network."
215 2012-12-26 03:44:02 <gmaxwell> stealth222: Relaying blocks is essential to the operation of the network. They're free to ignore your bloom filters. Though they're nearly free to apply, by design.
216 2012-12-26 03:47:44 <stealth222> if you can set up a bloom filter on txouts, why can't you set one up on txins?
217 2012-12-26 03:48:19 <stealth222> "only relay blocks that contain transactions which claim the following outputs..."
218 2012-12-26 03:49:14 <stealth222> perhaps I don't really understand the TD and BlueMatt stuff well enough
219 2012-12-26 03:49:21 <gmaxwell> stealth222: Sure, thats not a problem??? but you didn't really save yourself anything there (as mentioned the savings from filtering blocks is kinda boring).
220 2012-12-26 03:51:10 <stealth222> so this bloom filter thing isn't some amazing optimization?
221 2012-12-26 03:51:35 <gmaxwell> It just reduces the data you'd send to a mobile SPV client that isn't interested in anything but its own transactions.
222 2012-12-26 03:52:27 <gmaxwell> Which is a good thing and all. But ::meh::.
223 2012-12-26 03:53:05 <stealth222> but a mobile client presumably only wants block headers up to the point where it started sending transactions. as long as it syncs sufficiently frequently, catching up shouldn't be an issue.
224 2012-12-26 03:53:31 <gmaxwell> started recieving transactions*
225 2012-12-26 03:54:00 <gmaxwell> In any case, so sure you can use the bloom filters to also grab spends of the txins you depend on.
226 2012-12-26 03:54:00 <stealth222> well, doesn't it want to know whether transactions it sent are confirmed?
227 2012-12-26 03:54:28 <stealth222> so sending or receiving
228 2012-12-26 03:54:31 <gmaxwell> stealth222: Fair enough.
229 2012-12-26 03:54:57 <gmaxwell> the way the filter is setup is flexible enough to do that??? it tests both the txid itself and the txouts against the filter.
230 2012-12-26 03:55:03 <stealth222> point is, it saves itself the trouble of downloading 200k+ full blocks the first time it's run
231 2012-12-26 03:55:33 <gmaxwell> stealth222: it does that in any case.
232 2012-12-26 03:55:39 <gmaxwell> Doesn't need the filter for that.
233 2012-12-26 03:55:53 <stealth222> right, point is the filter is only used passed that point
234 2012-12-26 03:55:59 <gmaxwell> Right.
235 2012-12-26 03:55:59 <stealth222> further back, it only cares about headers
236 2012-12-26 03:59:05 <gmaxwell> but yea, so I was perhaps too quickly negative before: You could have a wallet tracking it's own whole swarm of unconfirmed transactions... which it just picks up as it goes; so no arbritary txn index required... Though thats a lot of complexity just to give an accurate negative confirmation count.
237 2012-12-26 04:00:12 <stealth222> right now, every time that a new transaction is added to the mempool or seen in a newly connected block, SyncWithWallets is called
238 2012-12-26 04:00:28 <stealth222> it wouldn't be too hard to have the wallet check for conflicts
239 2012-12-26 04:00:36 <stealth222> I understand that eventually it won't work that way
240 2012-12-26 04:01:45 <stealth222> reorgs could be dealt with by passing back a block hash if the conflicting transaction is found in a block
241 2012-12-26 04:02:28 <gmaxwell> well the harder part is this.. a bunch of txn go into the mempool... then a txn goes into the wallet that depends on those transactions. The wallet was ignoring them because it didn't care about them, and now it cares.
242 2012-12-26 04:05:42 <stealth222> so the wallet only sees unconfirmed transactions 1-ply?
243 2012-12-26 04:06:11 <stealth222> let me see...
244 2012-12-26 04:11:11 <stealth222> are you talking about if (wtxIn.hashBlock != 0) in main.cpp line 381?
245 2012-12-26 04:12:57 <stealth222> and if (wtxIn.hashBlock != 0 && wtxIn.hashBlock != wtx.hashBlock)
246 2012-12-26 04:13:11 <gmaxwell> not sure what version you're looking at??? here that code is in wallet.cpp.
247 2012-12-26 04:13:25 <stealth222> I'm looking at bitcoin/bitcoin master, I believe
248 2012-12-26 04:13:30 <stealth222> err
249 2012-12-26 04:13:32 <stealth222> sorry, yeah
250 2012-12-26 04:13:35 <stealth222> I meant wallet.cpp
251 2012-12-26 04:13:42 <stealth222> sorry :p
252 2012-12-26 04:14:34 <stealth222> so if any of the transaction inputs are not in a block, the wallet ignores it?
253 2012-12-26 04:15:58 <gmaxwell> I just mean that the wallet currently only tracks transactions which are IsMine (or IsFromMe). No, it doesn't ignore those.
254 2012-12-26 04:16:11 <gmaxwell> (the code you're pointing at is the timestamp tracking stuff)
255 2012-12-26 04:17:04 <gmaxwell> stealth222: when a txn is accepted into the mempool it means that the parents are either spendable in the chain, or in the mempool already.  Those transactions get handed to the wallet for inspection, and if they are from/to the wallet it takes them.
256 2012-12-26 04:22:19 <stealth222> the wallet will always care about transactions involving it, even if orphaned, no?
257 2012-12-26 04:22:51 <stealth222> still studying the code
258 2012-12-26 04:23:30 <gmaxwell> stealth222: We need more/different words. I don't know which meaning of orphan you're using there.   Once the wallet knows about a transaction it always cares and will not forget.
259 2012-12-26 04:23:57 <stealth222> I'm using orphaned in the way that the metaphor actually makes sense - inputs cannot be connected
260 2012-12-26 04:24:03 <stealth222> not in the way some use it - which means sidechain
261 2012-12-26 04:24:17 <gmaxwell> If a transaction is an orphan from the perspective of the node (e.g. its parents are currently unknown) then AFAIR (without looking at the code) it will never be passed onto the wallet. So while the wallet might want to care, it will never know.
262 2012-12-26 04:25:07 <stealth222> look at the block on lines 3171 - 3179 in main.cpp
263 2012-12-26 04:25:10 <gmaxwell> If a transaction was at once point valid and not an orphan, then the wallet will be told.  If the inputs are later conflicted, nothing will bother telling the wallet about this fact.. and it will go on expecting the transaction to someday confirm.
264 2012-12-26 04:25:40 <gmaxwell> give me a code fragment, I don't have any checkouts on trunk right now.
265 2012-12-26 04:26:55 <stealth222> http://pastebin.com/W7svF1Mu
266 2012-12-26 04:27:08 <stealth222> SyncWithWallets
267 2012-12-26 04:27:43 <stealth222> btw, I moved SyncWithWallets into TxMemPool::accept in the multiwallet branch and removed the explicit calls from ProcessMessages
268 2012-12-26 04:28:02 <stealth222> because the way it was, a transaction sent from one wallet to another wouldn't be picked up by the recipient wallet
269 2012-12-26 04:28:23 <stealth222> s/ProcessMessages/ProcessMessage
270 2012-12-26 04:28:41 <stealth222> anyhow, point is, SyncWithWallets is always called whenever a transaction is either added to the mempool or when seen in a connected block
271 2012-12-26 04:28:48 <gmaxwell> stealth222: sure thats what pulls orphans out of the orphan pool when their parents show up.
272 2012-12-26 04:29:14 <gmaxwell> stealth222: yea, I think we're failing to communicate here. I give up. :P
273 2012-12-26 04:29:38 <stealth222> wait, don't give up yet :(
274 2012-12-26 04:29:54 <gmaxwell> hah. You'll get a lot more out per minute spent with the code than with me.
275 2012-12-26 04:31:16 <stealth222> so what I'm trying to say is that we could not only have IsMine() and IsFromMe() but also ConflictsWithMine() in CWallet
276 2012-12-26 04:32:00 <gmaxwell> stealth222: Yes. We could. However, you'd also need a ParentOfMine and a time machine in order to implement ConflictsWithMine.
277 2012-12-26 04:33:24 <gmaxwell> E.g. unconfirmed transactions A->B->C   C is IsMine.  A/B fail all IsFoo() tests.  Later D shows up that conflicts with A. How does ConflictsWithMine trigger on D if the wallet ignored A because it didn't (1) have a IsAncestorOfMine  (2) didn't know about C at the time of A.
278 2012-12-26 04:34:01 <gmaxwell> this is all resolvable. but it ends up where the wallet would have its own mempool, I think. (maybe there is an easier way, to resolve it without breaking isolation between the wallet and the rest that I'm missing)
279 2012-12-26 04:34:41 <stealth222> couldn't we have a structure containing the earliest unconfirmed output dependencies?
280 2012-12-26 04:34:59 <stealth222> I guess we need the entire tree
281 2012-12-26 04:35:19 <gmaxwell> Right, Unless you have a time machine. With a time machine you can have just the ones you need. :P
282 2012-12-26 04:35:25 <stealth222> lol
283 2012-12-26 04:36:08 <gmaxwell> Once you get a working FutureAncestorOfMine() test, we should forget this bitcoin stuff and go into futures trading. :P
284 2012-12-26 04:36:17 <stealth222> hah
285 2012-12-26 04:36:48 <stealth222> if (ThisProgramWillHalt()) while (true);
286 2012-12-26 04:37:09 <gmaxwell> perhaps the interesting bit is that the wallet really only cares about the graph. So it could be much smaller than the real mempool to track this stuff.
287 2012-12-26 04:37:29 <stealth222> yeah, that's exactly what I was thinking
288 2012-12-26 04:37:52 <stealth222> I meant graph, not tree :p
289 2012-12-26 04:38:37 <stealth222> that's exactly correct - it doesn't care about the actual scripts and what not
290 2012-12-26 04:42:14 <stealth222> then we could not only detect negative confirmations - we could also compute the metrics we had discussed earlier regarding the size of all unconfirmed transactions that it depends on and for how many generations
291 2012-12-26 04:42:32 <stealth222> and whether or not an unconfirmed transaction is orphaned
292 2012-12-26 04:42:42 <stealth222> (in the "we don't know the parents" sense)
293 2012-12-26 04:45:01 <gmaxwell> stealth222: now you just have to solve the problem of that data set growing without bound (so you have to remove stuff when its confirmed) and being correct after a reorg.
294 2012-12-26 04:45:06 <gmaxwell> 
ACTION out for the night &

295 2012-12-26 04:45:27 <stealth222> cheers :D
296 2012-12-26 05:33:09 <forgot> how does increasing job.time per second affect mining rate?
297 2012-12-26 05:35:21 <Godzilla123> Hi I have a question that I am hoping to get the answer for without looking into the source :P
298 2012-12-26 05:36:07 <Godzilla123> When is a unspent output removed from the pool of unspent outputs? Does the client wait for the output to be included in a mined block or immediately as it is referenced?
299 2012-12-26 05:39:44 <gmaxwell> Godzilla123: which pool of unspent outputs? The database used for validating blocks or the wallets unspent outputs which are kept for spending?
300 2012-12-26 05:40:25 <gmaxwell> The former is updated when there is a new block, the latter is updated when the wallet creates or sees a transaction spending them.
301 2012-12-26 05:41:11 <gmaxwell> New transactions are also explicitly checked against conflicts in the mempool, so you can think of it as a temporary overlay on the set of spendable transactions for the purpose of validating currently unconfirmed transactions.
302 2012-12-26 05:41:32 <gmaxwell> (but not blocks??? since there is no promise that your mempool is consistent with anyone elses)
303 2012-12-26 05:44:14 <forgot> should I set noncebase to a random number after everytime I get a new job or modify job.time to get expected mining rate?
304 2012-12-26 05:49:03 <Godzilla123> gmaxwell: The database used for validating blocks (i,e., unspent outputs of other's wallets)
305 2012-12-26 05:49:27 <Godzilla123> gmaxwell: thanks :)
306 2012-12-26 05:54:26 <kiba> anyone want to join the BAM development army?
307 2012-12-26 05:55:13 <kiba> pew pew pew bam! bam! bam! trububububububub
308 2012-12-26 05:56:10 <kiba> I guess I should keep developing my game until it's compelling enough to attract fans and developers
309 2012-12-26 06:58:02 <Luke-Jr> http://imgur.com/gallery/F1yGP
310 2012-12-26 07:00:08 <gmaxwell> "Features:  * Impervious to antenna damage."
311 2012-12-26 07:08:38 <kiba> so I got invisible combat working pretty ok
312 2012-12-26 07:09:03 <kiba> 
ACTION only observes the combat activity with debug logs

313 2012-12-26 08:17:25 <wumpus> gmaxwell: I did not introduce it, now looking at v0.3.22rc1 and it's already there
314 2012-12-26 08:18:23 <wumpus> maybe it was a workaround for some wx on windows issue?
315 2012-12-26 08:18:53 <wumpus> I doubt it's needed anymore
316 2012-12-26 10:08:24 <boin> suck it up gmaxwell
317 2012-12-26 10:09:33 <boin> what a sucker you are, gmaxwell
318 2012-12-26 10:10:00 <boin> learn to lose
319 2012-12-26 10:13:15 <boin> keep playing "master of bitcoin"
320 2012-12-26 10:13:31 <boin> as if it's your property
321 2012-12-26 10:14:19 <boin> you don't own us. you dont make the rules
322 2012-12-26 10:57:21 <MC1984> wow i would be really quite angry if i had something with an aerial that wasnt even connected to anything
323 2012-12-26 11:01:19 <Scrat> GOD DAMN CHINESE
324 2012-12-26 11:01:22 <Scrat> anger unleashed
325 2012-12-26 11:02:17 <MC1984> the hokey mem cards coing out of chinese factories are the worst
326 2012-12-26 11:02:45 <MC1984> theyre like 2gbyte chips with a 64gbyte controller, you shit is NOT SAFE on such a card
327 2012-12-26 11:03:03 <MC1984> but they show up as what you paid for
328 2012-12-26 11:15:23 <sipa> gmaxwell, wumpus: iirc it's always been there
329 2012-12-26 12:23:53 <boin> gmaxwell is a ratings troll. he is posting inflammatory, extraneous and off-topic ratings on the WoT database. i think he should be rating-banned from gribble
330 2012-12-26 12:24:10 <boin> but is he? nope. he owns it!
331 2012-12-26 12:29:46 <boin> you're faking it, gmaxwell
332 2012-12-26 12:37:54 <MC1984> the fact that youre being such a baby about it says to me his bad rating of you is on the money
333 2012-12-26 12:53:15 <boin> no, gmaxwell is the baby
334 2012-12-26 12:53:18 <boin> can't admit defeat
335 2012-12-26 12:53:24 <boin> he won't apologize
336 2012-12-26 12:53:54 <boin> my bad rating of him is spot on, i'd say
337 2012-12-26 12:54:22 <boin> you know you are wrong, gmaxwell
338 2012-12-26 12:54:34 <boin> be the better man
339 2012-12-26 12:54:41 <boin> (or what's the expression)
340 2012-12-26 12:55:03 <boin> take it like a man, gmaxwell!
341 2012-12-26 12:55:25 <boin> you can't have everything, you know
342 2012-12-26 12:55:42 <boin> can't have everything you want in life, gmaxwell
343 2012-12-26 12:57:44 <boin> this is what you owe me, gmaxwell: 617.28 EUR  426.23 USD  4.6765 BTC
344 2012-12-26 12:59:01 <boin> gmaxwell is such a troll, and also, such a "douche" and "dipshit"
345 2012-12-26 12:59:21 <boin> you KNOW what you did wrong, gmaxwell!
346 2012-12-26 12:59:38 <boin> explain me, gmaxwell. i won't explain myself. YOU will explain me
347 2012-12-26 13:00:15 <boin> you see, i am always right, and if i do something weird, it's perfectly justified
348 2012-12-26 13:06:10 <boin> who wants to pick up the debt?
349 2012-12-26 13:07:39 <kjj> you are off topic.  stfu
350 2012-12-26 13:07:58 <boin> kick gmaxwell out of here so he can't do his business. that's on-topic
351 2012-12-26 13:08:17 <kjj> no, dumbass, this isn't a business channel.  this is the development channel
352 2012-12-26 13:08:43 <boin> i used to work on the business channel, until he started banning me
353 2012-12-26 13:09:16 <boin> man, gmaxwell, you are such a dumbass
354 2012-12-26 13:09:16 <kjj> well, I've spent a good part of the last half hour hoping that an op would wake up and ban you from here, so I guess I'm not surprised by that
355 2012-12-26 13:09:44 <boin> i learn new words every day
356 2012-12-26 13:10:13 <kjj> put "ignore" on your list
357 2012-12-26 13:10:46 <boin> everybody, ignore gmaxwell!
358 2012-12-26 13:45:39 <boin> i think everybody should just stop talking to this gmaxwell character
359 2012-12-26 13:45:45 <boin> don't feed the trolls
360 2012-12-26 13:46:36 <boin> hey, i hear he used to hit on women. what a creep!
361 2012-12-26 13:47:09 <boin> i think he is mentally deranged
362 2012-12-26 13:47:50 <boin> you don't want people like that near you
363 2012-12-26 13:49:11 <boin> find some other channel, gmaxwell, or some other network
364 2012-12-26 13:49:19 <boin> this place isn't for you
365 2012-12-26 13:50:10 <boin> you know what, i'll give you a nice encouragement talk, like i'm your idol or something, before i shun you too (because it's supposed to happen)
366 2012-12-26 13:52:29 <boin> also, who deals with this guy? how can someone do business of any kind with a person like him?
367 2012-12-26 13:52:41 <boin> is he serious?
368 2012-12-26 13:53:19 <boin> man, i pity this guy
369 2012-12-26 13:55:37 <boin> heh
370 2012-12-26 13:56:41 <kjj> finally!
371 2012-12-26 13:57:06 <gmaxwell> Sorry. I have to sleep sometimes???
372 2012-12-26 13:57:21 <kinlo> perhaps we need a few more chanops :)
373 2012-12-26 13:57:42 <MC1984> you really pissed in that guys cornflakes greg
374 2012-12-26 14:44:18 <etotheipi_> gmaxwell: does this sound right to you?  my crappy little listening script only watches for zero-conf tx that are forwarded by bitcoind and new blocks... on new blocks it removes any zero-conf tx sitting in memory... after running for 12 hours it had about 1000 zero-conf tx
375 2012-12-26 14:44:41 <etotheipi_> *remove any zero-conf tx _in that block_ from memory
376 2012-12-26 14:57:12 <sipa> etotheipi_: you will probably accumulate conflicting transactions over.te thwn
377 2012-12-26 14:57:18 <sipa> over time
378 2012-12-26 14:57:20 <sipa> then
379 2012-12-26 14:58:53 <etotheipi_> sipa: so there's that many?
380 2012-12-26 14:59:07 <sipa> apparently
381 2012-12-26 14:59:38 <gmaxwell> It's now become popular to try to double spend zero conf accepting things.
382 2012-12-26 14:59:52 <etotheipi_> ahh, that's what I'm missing
383 2012-12-26 14:59:59 <etotheipi_> checking for zero-conf tx that were invalidated
384 2012-12-26 15:00:20 <jgarzik> 
ACTION might look into mempool TX expiration, as an early 2013 project

385 2012-12-26 15:01:09 <etotheipi_> right now, I'm really just trying to put together a super-minimal python/armoryengine watcher as a template to put into my examples directory
386 2012-12-26 15:01:23 <etotheipi_> and trying to figure out if it's useful at all
387 2012-12-26 15:01:45 <etotheipi_> (or buggy)
388 2012-12-26 15:01:49 <gmaxwell> jgarzik: It's sort of interesting to ponder how tightly related that is to replacement. After all, if you'd allow expiration under some criteria than you should be willing to accept a replacement under at least those criteria.
389 2012-12-26 15:02:51 <gmaxwell> (and probably less, since replacement can be permitted which has strictly less risk of resulting in people getting robbed)
390 2012-12-26 15:03:33 <etotheipi_> that's what always bothered me about it (though I couldn't determine if it was avoidable), was that any deterministic guarantee that a tx will be invalidated makes for deterministic exploits
391 2012-12-26 15:04:05 <etotheipi_> but at the same time... you might as well get it out there and let people start abusing it and the economy will adapt to it
392 2012-12-26 15:05:33 <housh> etotheipi_: don't talk to him! don't feed him!
393 2012-12-26 15:05:53 <gmaxwell> etotheipi_: well non-determinstic falure can only be avoided by prior determinstic failure, and non-determinstic failure may be harder to defend against.
394 2012-12-26 15:06:22 <etotheipi_> gmaxwell: you've really been pissing some people off, haven't you?
395 2012-12-26 15:07:02 <etotheipi_> or is this all the same person re-logging in under different names?
396 2012-12-26 15:07:05 <gmaxwell> Yes.
397 2012-12-26 15:14:42 <sipa> jgarzik: can you op me here?
398 2012-12-26 15:15:08 <sipa> not sure about the chanserv command to do so, though
399 2012-12-26 15:15:15 <jgarzik> sipa: can you auth with nickserv?
400 2012-12-26 15:15:25 <gmaxwell> sipa: I thought you were?
401 2012-12-26 15:15:25 <sipa> i'm not? sec
402 2012-12-26 15:16:01 <gmaxwell> Ah, I see you're not.
403 2012-12-26 15:16:11 <sipa> only on #bitcoin
404 2012-12-26 15:16:17 <sipa> i'm authed
405 2012-12-26 15:16:50 <sipa> hmm, and via chanserv?
406 2012-12-26 15:17:23 <jgarzik> I think there might be forms to fill out or something
407 2012-12-26 15:17:26 <jgarzik> I dunno
408 2012-12-26 15:17:44 <jgarzik> I am the "owner" of #bitcoin-*, so in theory you could tell me how to do it :)
409 2012-12-26 15:18:07 <jgarzik> or do it via gribble
410 2012-12-26 15:18:35 <sipa> /msg ChanServ ACCESS #bitcoin-dev ADD sipa OP
411 2012-12-26 15:19:10 <jgarzik> sipa: done
412 2012-12-26 15:19:22 <D34TH> or instead of +o set +O
413 2012-12-26 15:20:01 <sipa> thanks!
414 2012-12-26 15:33:24 <etotheipi_> are there going to be any changes to the peer finding/connection behavior in 0.8?  There's been this problem in the past with bitcoind/-qt maxing out connections and now allowing incoming connection from localhost Armory
415 2012-12-26 15:33:46 <etotheipi_> but there appears to be no problem with stock bitcoin-qt/d
416 2012-12-26 15:34:19 <etotheipi_> but now someone is reporting that Armory can't connect to their stock "turbo" build, though it works fine with 0.7
417 2012-12-26 15:38:26 <maaku> imho connection from 127.0.0.1 should take priority over all else...
418 2012-12-26 15:39:34 <etotheipi_> maaku: I thought so, too, but gmaxwell pointed out that if you use tor, it appears that all connections come from localhost
419 2012-12-26 15:39:53 <maaku> hrm. that is troublesome
420 2012-12-26 15:40:08 <etotheipi_> but what do I know about tor?  (never used it)
421 2012-12-26 15:40:53 <maaku> i've thought for a while that we've needed a addnode json-rpc command
422 2012-12-26 15:41:03 <maaku> which would probably solve your use case
423 2012-12-26 15:41:21 <gmaxwell> etotheipi_: "turbo" ? ... that old network abusive patch that opens up thousands of connections? Tell them not to run that.
424 2012-12-26 15:41:56 <jgarzik> etotheipi_: there are definitely peer finding changes planned
425 2012-12-26 15:41:57 <etotheipi_> gmaxwell: I'm talking about sipa's build
426 2012-12-26 15:41:57 <gmaxwell> maaku: tools like p2pool and armory connect to bitcoind??? making them listen for connections from bitcoind would be kind of odd.
427 2012-12-26 15:42:08 <jgarzik> for I.B.D.
428 2012-12-26 15:42:12 <gmaxwell> etotheipi_: ah.
429 2012-12-26 15:42:26 <etotheipi_> gmaxwell: he labeled it the "turbo" branch
430 2012-12-26 15:46:26 <gmaxwell> I wonder how people are getting filled up nodes, my public node seems pretty stable at around 100 connections.
431 2012-12-26 15:52:34 <sipa> etotheipi_, maaku: one idea is to add sort of 'whitelisted' peers, which would always get in, and whose transactions would always be relayed (even if already known to the mempool)
432 2012-12-26 15:52:59 <sipa> an idea was adding a second P2P listening port, by default only bound to localhost, which would get this behaviour
433 2012-12-26 15:53:53 <sipa> another idea is peer rotation (i've been doing some simulations even for that), where sometimes a new connection would be attempted, and if it succeeds, have it replace another connection, so the network becomes more dynamic
434 2012-12-26 15:55:26 <sipa> i have 80 connections on my public node
435 2012-12-26 15:56:12 <kinlo> that's a lot, my public node never goes above 15
436 2012-12-26 15:56:16 <kinlo> I wonder why
437 2012-12-26 16:05:56 <gmaxwell> kinlo: you may be on a /16 with many other public nodes.
438 2012-12-26 16:17:53 <kinlo> mmmz, it's PI space so 256 netblock owners
439 2012-12-26 16:17:56 <kinlo> difficult to tell
440 2012-12-26 16:22:43 <sipa> PI?
441 2012-12-26 16:30:28 <jgarzik> exmulti public node connection counts:  86, 89, 99
442 2012-12-26 16:30:36 <jgarzik> testnet node cxn ct: 19
443 2012-12-26 16:32:08 <jgarzik> sipa: for picocoin, I was pondering (1) headers download, then (2) requesting block hashes in parallel from all peers
444 2012-12-26 16:32:36 <jgarzik> push parallel to the max :)
445 2012-12-26 16:32:42 <sipa> jgarzik: that's exactly what i'd want the reference client to do
446 2012-12-26 17:49:58 <tcatm> hey :)
447 2012-12-26 17:50:07 <sipa> hi there
448 2012-12-26 17:50:20 <tcatm> was there ever a patch for bitcoin that made it send the change back to the same address?
449 2012-12-26 17:50:44 <sipa> why would you want that?
450 2012-12-26 17:50:59 <sipa> can't be hard to write, but sounds like a bad thing
451 2012-12-26 17:51:08 <etotheipi_> tcatm: Armory has that option in "Expert" user mode :)
452 2012-12-26 17:51:11 <tcatm> there are some transactions like that in the chain
453 2012-12-26 17:51:29 <tcatm> etotheipi_: Great. When was armory first released with that option?
454 2012-12-26 17:51:30 <sipa> multibit does that, afaik
455 2012-12-26 17:51:59 <etotheipi_> tcatm: it's been there for a while, but I started advertising it a couple weeks when I added coin-control
456 2012-12-26 17:52:15 <ThomasV_> electrum has it too
457 2012-12-26 17:53:40 <tcatm> mkay, so at least 3 clients to look at :/
458 2012-12-26 17:53:46 <sipa> why?
459 2012-12-26 17:54:07 <sipa> ThomasV_: by default, or not?
460 2012-12-26 17:54:21 <ThomasV_> not by default, it's a setting
461 2012-12-26 17:55:09 <sipa> tcatm: first guess would then be multibit, as it does it by default
462 2012-12-26 17:55:25 <gmaxwell> tcatm: blockchain.info apparently does that.
463 2012-12-26 17:55:49 <tcatm> doesn't look like the client does it by default. only a few transactions are affected by the bug I might have discovered.
464 2012-12-26 17:56:17 <ThomasV_> are you trying to track a particular transaction?
465 2012-12-26 17:56:21 <tcatm> nope
466 2012-12-26 17:57:23 <etotheipi_> tcatm:  I added the option back in July
467 2012-12-26 17:58:08 <etotheipi_> 
ACTION just discovered how awesome "git blame" is

468 2012-12-26 17:58:46 <gmaxwell> etotheipi_: kind of a crappy option, since it harms the privacy of people other than the ones who select it. (As _I_ get deanonymized when you send me coins you've deanonymized) ... especially since it doesn't do anything useful in the context of a determinstic wallet except better resonate with the misunderstanding of the blockchain as a set of balances.
469 2012-12-26 17:59:27 <gmaxwell> etotheipi_: yea, though careful??? you can blame the wrong commits (as I did with asking wumpus about the exit timeout on windows this morning)
470 2012-12-26 17:59:40 <sipa> 341 non-canonical transactions on 150000
471 2012-12-26 17:59:49 <etotheipi_> gmaxwell: I was only responding to requests for the feature
472 2012-12-26 18:00:00 <etotheipi_> gmaxwell: that's why it only exists in Expert usermode
473 2012-12-26 18:00:25 <sipa> 5 ERROR: Non-canonical public key: compressed nor uncompressed
474 2012-12-26 18:00:26 <sipa> 59 ERROR: Non-canonical signature: R value excessively padded
475 2012-12-26 18:00:58 <sipa> interesting distribution
476 2012-12-26 18:01:05 <tcatm> anyone got a nice implementation of op_checksig? maybe in python?
477 2012-12-26 18:01:20 <etotheipi_> tcatm: I do
478 2012-12-26 18:01:21 <sipa> tcatm: i suppose pynode has that
479 2012-12-26 18:01:40 <etotheipi_> though it's not very isolateable without grabbing everything else in "armoryengine"
480 2012-12-26 18:02:59 <etotheipi_> it starts here:  https://github.com/etotheipi/BitcoinArmory/blob/master/armoryengine.py#L4024
481 2012-12-26 18:06:18 <tcatm> hrm. Pretty complicated. Can you help me getting armoryengine to verify a specific transaction?
482 2012-12-26 18:06:45 <etotheipi_> tcatm: there is some examples in unittest.py
483 2012-12-26 18:06:54 <etotheipi_> hold on, let me find it for you
484 2012-12-26 18:07:17 <etotheipi_> just be aware it requires not only the transaction you are verifying, but the scripts (or full tx) of the outputs you are spending
485 2012-12-26 18:07:55 <etotheipi_> https://github.com/etotheipi/BitcoinArmory/blob/master/unittest.py#L586
486 2012-12-26 18:09:27 <phantomcircuit> tcatm, it turns the DAG that transactions can be into a DG
487 2012-12-26 18:09:47 <etotheipi_> the inputs are:  verifyTransactionValid(fullPrevTx, txToVerify, txInputIndexToVerify)
488 2012-12-26 18:09:54 <etotheipi_> so it actually only verifies one input
489 2012-12-26 18:10:21 <tcatm> that's fine for what I want to do
490 2012-12-26 18:10:28 <etotheipi_> put it in a loop to verify each
491 2012-12-26 18:11:05 <etotheipi_> if you want to understand the process better:  check out hte "On-the-wire byte map" for OP_CHECKSIG (about half way down): https://bitcointalk.org/index.php?topic=29416.0
492 2012-12-26 18:11:51 <gmaxwell> tcatm: perhaps you're noticing transactions with negative r? IIRC blockchain.info was creating those and sending their change back to themselves.
493 2012-12-26 18:14:32 <tcatm> etotheipi_: Is there an API documentation for armoryengine anywhere?
494 2012-12-26 18:14:55 <etotheipi_> tcatm: not really... it's in the plans, but I was focused on GUI users first, developers later
495 2012-12-26 18:15:09 <etotheipi_> for now, unittest.py and extras/sample_armory_code.py is the documentation
496 2012-12-26 18:15:32 <etotheipi_> at least it's working examples of just about everything armoryengine can do
497 2012-12-26 18:16:07 <jgarzik> tcatm: pynode
498 2012-12-26 18:16:30 <jgarzik> tcatm: https://github.com/jgarzik/pynode/blob/master/bitcoin/scripteval.py#L77
499 2012-12-26 18:16:48 <etotheipi_> tcatm: my guess is that pynode is better suited for you needs
500 2012-12-26 18:16:58 <jgarzik> tcatm: the /bitcoin/ modules are reasonably independent, and may be used independently of the node.py softwares
501 2012-12-26 18:20:20 <sipa> https://bitcointalk.org/index.php?topic=133402.0
502 2012-12-26 18:22:49 <gmaxwell> sipa: You might want to resolve the input addresses for these transactions and post that list as links to blockchain info. Then people can see if the links show up blue for them.
503 2012-12-26 18:23:25 <sipa> good idea
504 2012-12-26 18:25:30 <etotheipi_> any idea how many might be related to Armory?  do you know that Armory has resolved all of those?  should I go ahead and try to figure out the odd-only S-values?
505 2012-12-26 18:25:59 <sipa> etotheipi_: i don't think armory is producing any non-DER ones anymore
506 2012-12-26 18:26:30 <sipa> and we don't even have code in the satoshi client to produce odd-only (or even-only; we'll have to decide soon :p) S values
507 2012-12-26 18:27:21 <etotheipi_> is it just a matter of taking whatever S value was selected, and if it's wrong, S = S^(n-1) where n is the number of elts on the EC?
508 2012-12-26 18:27:40 <etotheipi_> (mod N)
509 2012-12-26 18:27:55 <sipa> no, S = -S (mod N)
510 2012-12-26 18:28:08 <sipa> or, easier: S = N - S
511 2012-12-26 18:28:11 <etotheipi_> oh
512 2012-12-26 18:28:18 <etotheipi_> it's not even multiplicative inverse
513 2012-12-26 18:28:20 <tcatm> etotheipi_: Is there a TheBDM.getTransactionByHash()?
514 2012-12-26 18:28:34 <etotheipi_> tcatm: yes:  TheBDM.getTxByHash()
515 2012-12-26 18:28:37 <etotheipi_> you were close :)
516 2012-12-26 18:28:38 <tcatm> ah :)
517 2012-12-26 18:29:31 <sipa> gmaxwell: you have a 0.7 node with getrawtransaction running still?
518 2012-12-26 18:29:44 <gmaxwell> Yes.
519 2012-12-26 18:30:09 <gmaxwell> ah, shall I resolve those txids then?
520 2012-12-26 18:30:35 <etotheipi_> tcatm, just be aware that you have to load the blockchain before that will return anything useful:  as seen https://github.com/etotheipi/BitcoinArmory/blob/master/extras/sample_armory_code.py#L65
521 2012-12-26 18:30:41 <sipa> gmaxwell: if possible :)
522 2012-12-26 18:30:46 <sipa> gmaxwell: txids are here: http://bitcoin.sipa.be/re.txt
523 2012-12-26 18:31:44 <gmaxwell> Yea, no problem.
524 2012-12-26 18:32:10 <gmaxwell> sipa: odd txnid "a390235d83"
525 2012-12-26 18:32:42 <sipa> ah, yes, that's from before i patched my node to dump full txids
526 2012-12-26 18:32:45 <sipa> ignore it
527 2012-12-26 18:33:15 <gmaxwell> Do you want me to grab the addresses from the txins?
528 2012-12-26 18:33:24 <gmaxwell> or do you want the transactions?
529 2012-12-26 18:34:01 <sipa> the txin addresses is probably more useful
530 2012-12-26 18:34:34 <gmaxwell> k.
531 2012-12-26 18:34:51 <tcatm> etotheipi_: How does getTxByHash work? Doesn't seem to like integers nor hexstrings.
532 2012-12-26 18:35:38 <etotheipi_> tcatm: it's takes binary
533 2012-12-26 18:35:46 <etotheipi_> use hex_to_binary()
534 2012-12-26 18:36:13 <etotheipi_> all hash-indexed objects need that conversion
535 2012-12-26 18:37:24 <etotheipi_> there is binary_to_hex() as well, and if the endianness is wrong, you can use binary_switchEndian()
536 2012-12-26 18:37:50 <etotheipi_> (or hex_switchEndian(), depending on what form it's in)
537 2012-12-26 18:40:15 <tcatm> Hrm. I get an Tx object but it looks empty.
538 2012-12-26 18:40:31 <tcatm> (blockchain is loaded)
539 2012-12-26 18:40:38 <etotheipi_> try switching endian and pass that in
540 2012-12-26 18:41:44 <etotheipi_> I have to admit, very early in development, I was confused about how to deal with endianness, and I never went back through and fixed it
541 2012-12-26 18:42:12 <etotheipi_> that was one of my first things to do when I start focusing on dev tools, is clean all that up
542 2012-12-26 18:43:03 <tcatm> Can't get it to work: TheBDM.getTxByHash(hex_to_binary("999e1c837c76a1b7fbb7e57baf87b309960f5ffefbf2a9b95dd890602272f644")).getNumTxIn() => 4294967295. Same with hex_switchEndian
543 2012-12-26 18:45:04 <gmaxwell> sipa: which group are those?
544 2012-12-26 18:45:19 <etotheipi_> tcatm, give me a sec...
545 2012-12-26 18:45:27 <sipa> gmaxwell: group?
546 2012-12-26 18:45:36 <sipa> gmaxwell: oh, all of them
547 2012-12-26 18:45:40 <sipa> i can separate if you like
548 2012-12-26 18:47:21 <sipa> gmaxwell: http://bitcoin.sipa.be/nc-[rs][ne].txt
549 2012-12-26 18:53:11 <gmaxwell> sipa: for your reference: http://pastebin.com/aMa2ZjF9
550 2012-12-26 18:53:55 <gmaxwell> http://bitcoin.sipa.be/nc-rs.txt < 404
551 2012-12-26 18:54:28 <sipa> it's a shell pattern :)
552 2012-12-26 18:54:33 <gmaxwell> duur
553 2012-12-26 18:54:52 <sipa> that's dutch for "expensive", but i doubt that's what you mean...
554 2012-12-26 18:55:03 <etotheipi_> lol, sorry tcatm... I broke my whole setup just before you asked that question... give me another min
555 2012-12-26 18:58:43 <gmaxwell> haha  The message exceeds the maximum allowed length (64000 characters).
556 2012-12-26 19:01:12 <etotheipi_> tcatm:  "txHash = binary_switchEndian(hex_to_binary("999e1c837c76a1b7fbb7e57baf87b309960f5ffefbf2a9b95dd890602272f644")); tx = TheBDM.getTxByHash(txHash).pprint()"
557 2012-12-26 19:01:49 <etotheipi_> maybe you accidentally used hex_switchEndian() on binary data, or vice versa (using binary_switchEndian on hex data)
558 2012-12-26 19:03:27 <sipa> gmaxwell: your post looks kinda short
559 2012-12-26 19:03:44 <sipa> ah, the posts are yet to come perhaps
560 2012-12-26 19:04:50 <gmaxwell> sipa: I redid it in order to deduplicate it, otherwise they were all hitting the maximum post length due to duplciated txin addresses.
561 2012-12-26 19:05:03 <sipa> | sort | uniq
562 2012-12-26 19:07:03 <gmaxwell> sure sure, but I had to redo the post to do that. :P
563 2012-12-26 19:07:26 <tcatm> etotheipi_: Thanks! :)
564 2012-12-26 19:08:46 <gmaxwell> sipa: well crap, the colors on bitcoin talk don't seem to indicate visited or not!
565 2012-12-26 19:09:34 <sipa> post on an own html page, and like to that?
566 2012-12-26 19:09:49 <gmaxwell> sure, one sec.
567 2012-12-26 19:10:54 <sipa> etotheipi_: preference for even or odd S? :p
568 2012-12-26 19:11:10 <etotheipi_> sipa: hold on, let me find a coin
569 2012-12-26 19:11:30 <etotheipi_> err.... I mean: "no preference"
570 2012-12-26 19:11:33 <sipa> haha
571 2012-12-26 19:11:48 <sipa> for a second i thought you were referring to a txout when saying "coin" ...
572 2012-12-26 19:20:58 <sipa> gmaxwell, etotheipi_: https://github.com/sipa/bitcoin/commit/7a0dac94424ab7a0a54b1a8edf59c396c89a1a70
573 2012-12-26 19:26:17 <gmaxwell> sipa: http://people.xiph.org/~greg/non-standard-signatures.html
574 2012-12-26 19:29:13 <sipa> gmaxwell: thank god, none were colored for me
575 2012-12-26 19:29:27 <etotheipi_> none for me
576 2012-12-26 19:31:44 <gmaxwell> one was for me, but it was my test one.
577 2012-12-26 19:35:35 <gmaxwell> sipa: I've got one
578 2012-12-26 19:35:51 <gmaxwell> TheButterZone who hangs on on IRC and on the forums has  1TBZYXjrGjXCEN1SprpF66Jzy5uN3GiLS
579 2012-12-26 19:36:22 <gmaxwell> which is one of the addresses, TuxBlackEdo had a colored link on that one. :P
580 2012-12-26 19:37:23 <gmaxwell> (thats a  Sources with (at least) a signature with an "negative R value": and  Sources with (at least) a signature with an "excessively padded S value": )
581 2012-12-26 19:43:29 <gmaxwell> http://blockchain.info/address/15DjyGdFAdGrvK3XY4EyScnfz2feMRCYMs < is getting paid by deepbit, perhaps ask [tycho] to prod his user for us.
582 2012-12-26 19:45:11 <gmaxwell> (its an excessively padded R value source)
583 2012-12-26 19:50:31 <gmaxwell> 18ix3ikc51sKTwmfho5fjmfqe2miJBQR2c was a forums signature address of user weefan.
584 2012-12-26 19:53:57 <gmaxwell> Luke-Jr: can you grep #eligius channel logs to you see if you can tell which of your users this is: http://eligius.st/~wizkid057/newstats/userstats.php/1MMc227TR2ropsB7CaHaYGqPZNNdfs1yH
585 2012-12-26 19:54:53 <gmaxwell> 1J13NBTKiV8xrAo2dwaD4LhWs3zPobhh5S is "punningclan" on the forums. (I'm sending messages to these people)
586 2012-12-26 19:55:56 <sipa> great, thanks
587 2012-12-26 19:56:08 <wizkid057> whats this for anyways?
588 2012-12-26 19:56:52 <sipa> see https://bitcointalk.org/index.php?topic=133402.0
589 2012-12-26 19:58:41 <wizkid057> hm
590 2012-12-26 20:10:00 <tcatm> Mkay, at least one client does have a serious security problem.
591 2012-12-26 20:10:50 <etotheipi_> tcatm: how so?
592 2012-12-26 20:12:06 <gmaxwell> 
ACTION goes to check the entropy of R on the network.

593 2012-12-26 20:12:41 <tcatm> gmaxwell: that's the problem. There are some transactions with R duplicates.
594 2012-12-26 20:12:53 <tcatm> I've successfully recovered the private key from two of them.
595 2012-12-26 20:12:55 <gmaxwell> I've warned people about this many times???
596 2012-12-26 20:13:23 <sipa> same R implies same pubkey + same k (normally generated at random)
597 2012-12-26 20:13:40 <etotheipi_> oh wow
598 2012-12-26 20:13:44 <gmaxwell> tcatm: if you give me a list of txid's you think are naughty, I'll resove their previns and start googling.
599 2012-12-26 20:14:07 <etotheipi_> major snafu
600 2012-12-26 20:14:54 <gmaxwell> sorry to give it away with that guess. but it's basically the _one_ thing you must do right when constructing a signature, so it wasn't much of a guess.
601 2012-12-26 20:15:20 <tcatm> This one is pretty bad. Same R in both TxIns: 9ec4bc49e828d924af1d1029cacf709431abbde46d59554b62bc270e3b29c4b1
602 2012-12-26 20:15:23 <etotheipi_> tcatm: are these old?  or newer ones?
603 2012-12-26 20:15:36 <tcatm> Haven't looked at most of them yet.
604 2012-12-26 20:16:11 <gmaxwell> 1BFhrfTTZP3Nw4BNy4eX4KFLsn9ZeijcMm
605 2012-12-26 20:16:39 <gmaxwell> haha, I just got captchaed by google because I've been searching too many bitcoin addresses.
606 2012-12-26 20:18:20 <etotheipi_> how do you get that wrong?
607 2012-12-26 20:18:53 <etotheipi_> I guess, really custom ECDSA ...?  but if you're good enough to do that, I'd think you're good enough to use any kind of PRNG even a bad one
608 2012-12-26 20:19:00 <etotheipi_> or maybe you just don't read
609 2012-12-26 20:19:05 <blinky> gmaxwell: you've seen this post? https://bitcointalk.org/index.php?topic=133122.0
610 2012-12-26 20:19:19 <gmaxwell> etotheipi_: Lots of ways. (1) trusting some system random source to be random, (2) not actually knowing that there is a security implication ('my unit tests fail with this made random!'), typos, etc.
611 2012-12-26 20:19:52 <tcatm> I've counted 312 affected transactions so far.
612 2012-12-26 20:19:55 <gmaxwell> Using randval instead of *randval as your random.
613 2012-12-26 20:20:14 <sipa> tcatm: total value of UTXO's involved?
614 2012-12-26 20:20:23 <tcatm> UTXO?
615 2012-12-26 20:20:29 <sipa> unspent transaction outputs
616 2012-12-26 20:20:32 <andytoshi> etotheipi_, why generate your own addresses? probably for some brain-wallety idea
617 2012-12-26 20:20:50 <tcatm> sipa: I'll try to check that.
618 2012-12-26 20:21:24 <sipa> tcatm: 0.8 (git head) has a gettxout RPC to query the value if you have txid + output#
619 2012-12-26 20:21:35 <gmaxwell> etotheipi_: Using randval instead of *randval as your random.  Calling seed(time) right before your invoation.
620 2012-12-26 20:21:43 <andytoshi> or is the non-randomness only in the DSA algo, not the address itself?
621 2012-12-26 20:21:45 <andytoshi> 
ACTION just woke up

622 2012-12-26 20:21:45 <etotheipi_> gmaxwell: heh, okay
623 2012-12-26 20:21:55 <sipa> andytoshi: in DSA
624 2012-12-26 20:22:11 <etotheipi_> andytoshi: the random number is selected for each signature... you should pick a different one every time
625 2012-12-26 20:22:14 <sipa> that's why deterministic signatures are so nice (go Ed25519)
626 2012-12-26 20:22:22 <etotheipi_> so even signing the same thing twice should produce diff sigs
627 2012-12-26 20:22:28 <sipa> though we could add it to ECDSA as well
628 2012-12-26 20:23:23 <etotheipi_> the wiki page suggests that the PS3 developers made the same mistake
629 2012-12-26 20:23:35 <etotheipi_> and that's how the PS3 signing key was leaked
630 2012-12-26 20:23:49 <gmaxwell> 13:23 <+TheButterZone> gmaxwell: a copy of brainwallet.org saved to usb key for offline use
631 2012-12-26 20:24:01 <gmaxwell> ^ thats how the 1TBZ* addresses are getting signed.
632 2012-12-26 20:24:07 <gmaxwell> I believe brainwallet.org has been fixed.
633 2012-12-26 20:24:31 <theymos> gmaxwell: Didn't he say on the forum that the 1TBZ address was not a brainwallet.org address?
634 2012-12-26 20:25:10 <gmaxwell> theymos: I believe so, but he's telling me here that he's _signing_ with it. (sign vs create)
635 2012-12-26 20:25:22 <theymos> Oh, OK.
636 2012-12-26 20:26:30 <gmaxwell> theymos: would be neat if there was an API where I could give bitcointalk a list of addresses and it returns a list of matching urls. :P
637 2012-12-26 20:28:04 <theymos> gmaxwell: Just searching the text of posts?
638 2012-12-26 20:28:10 <tcatm> At least one address used the same R 67 times
639 2012-12-26 20:28:12 <gmaxwell> yea. and signatures. :P
640 2012-12-26 20:29:01 <sipa> brainwallet.org is by Joric, right?
641 2012-12-26 20:29:11 <gmaxwell> yes.
642 2012-12-26 20:29:12 <gmaxwell> 13:28 <+TheButterZone> gmaxwell: october 1, 2012
643 2012-12-26 20:29:26 <theymos> gmaxwell: Wouldn't be too difficult. I'll put it on my to-do list.
644 2012-12-26 20:29:57 <gmaxwell> theymos: several times now I've gone sluthing for weird behavior and searching the forum for addresses is surprisingly productive.
645 2012-12-26 20:38:21 <tcatm> There's less than 1 BTC unspent in the affected keys.
646 2012-12-26 20:40:26 <gmaxwell> "hardest $10 I ever made!"
647 2012-12-26 20:44:55 <gmaxwell> tcatm: I especially like the mixture with the repeated change addresses, the address reuse makes the insecure k really obvious.
648 2012-12-26 20:45:33 <tcatm> yep
649 2012-12-26 20:45:44 <tcatm> I wonder how many weak Rs there are that aren't that obvious.
650 2012-12-26 20:45:58 <tcatm> Maybe from nodes signing lots of tx per second?
651 2012-12-26 20:47:07 <tcatm> I guess that's the first successful crypto attack on bitcoin, isn't it?
652 2012-12-26 20:48:00 <etotheipi_> I'm not sure it qualifies if there was gross errors in the crypto implementation
653 2012-12-26 20:48:36 <etotheipi_> I could implement ROT13 encryption in Armory somewhere and it would be responsible for a security breach... should that count against the network?
654 2012-12-26 20:56:36 <gmaxwell> etotheipi_: otoh, ECDSA is needlessly fragile.
655 2012-12-26 20:57:49 <gmaxwell> would be funny if picocoin were responsible for these txns.
656 2012-12-26 20:57:55 <etotheipi_> is it?
657 2012-12-26 20:58:44 <gmaxwell> ECDSA fragile? Yes. Picocoin? dunno. checking.
658 2012-12-26 20:59:10 <gmaxwell> ed25519 solves this particular fragility in an official way and makes the signatures determinstic.
659 2012-12-26 20:59:28 <sipa> does ECDSa require different k overall? afaik only a different k is needed when the key is identical
660 2012-12-26 20:59:48 <etotheipi_> my point was simply that an implementation that clearly violates a known security problem should not be counted against that crypto algorithm
661 2012-12-26 20:59:52 <sipa> so there shouldn't be any non-obvious but vulnerable weak k
662 2012-12-26 21:00:18 <etotheipi_> gmaxwell: if I wasn't leaving in 30 seconds, I'd like to discuss fragility
663 2012-12-26 21:01:15 <gmaxwell> etotheipi_: maybe. I could argue it either way. If it were actually hard to solve, well thats life. But I'm generally of the view that the quality of something must realistically be considered in the context of the vulnerabilities it will expose in other thins.
664 2012-12-26 21:02:10 <gmaxwell> E.g. I think people are right to say that bitcoin is "flawed" because personal computer security sucks, and users can't keep online money safe.  (although, we can answer those claims with hardware multisign tokens??? we we should, rather than just sitting back and saying your-windows-pc-non-security-is-not-our-problem)
665 2012-12-26 21:02:46 <gmaxwell> So likewise, ECDSA is flawed because it requires a @#!@#! random value and it's well established that everyone gets that wrong. .. and worse, when you get it wrong the vulnerablity is largely invisible.
666 2012-12-26 21:03:03 <gmaxwell> Your k could always be 1 but you won't even possibly notice until you reuse a public key for the first time.
667 2012-12-26 21:03:07 <gmaxwell> And only then if you're looking.
668 2012-12-26 21:04:52 <sipa> i wonder if there's any reason not to use k=SHA256(privkey || message)
669 2012-12-26 21:05:17 <sipa> Ed25519 uses a separate half of the secret key for its nonce generation - i wonder why
670 2012-12-26 21:05:23 <sipa> maybe just because it was available
671 2012-12-26 21:06:10 <tcatm> that should be fine
672 2012-12-26 21:06:59 <andytoshi> sipa, what if the message and privkey are different lengths? would there be a way for two different messages to give the same "privkey || message"?
673 2012-12-26 21:07:26 <jrmithdobbs> not unless sha256 is broken in ways that make much more import parts fall apart
674 2012-12-26 21:07:29 <andytoshi> that would be my concern
675 2012-12-26 21:07:35 <jrmithdobbs> s/import/important/
676 2012-12-26 21:07:41 <sipa> andytoshi: since the privkey is constant length, that shouldn't be an issue
677 2012-12-26 21:07:50 <tcatm> though, entropy shouldn't be a problem. there's software like haveged that works pretty well on embedded devices.
678 2012-12-26 21:07:55 <sipa> if it was, you could use HMAC-SHA256(key=privkey, msg=message)
679 2012-12-26 21:08:08 <jrmithdobbs> sipa: ya i was going to say, might want to use hmac
680 2012-12-26 21:08:19 <sipa> Ed25519 uses SHA512 IIRC
681 2012-12-26 21:10:13 <gmaxwell> sipa: IIRC there are ways to recover the private key if K is only partially known. so I'd guess they were just being double safe?
682 2012-12-26 21:13:37 <jrmithdobbs> sipa: it uses sha512 but not like that, it hashes then manipulates the hash, it doesn't concat strings and then hash them jfyi
683 2012-12-26 21:14:11 <jrmithdobbs> ( H() in http://ed25519.cr.yp.to/python/ed25519.py )
684 2012-12-26 21:16:29 <gmaxwell> tcatm: it's not that it's fundimentally a problem, it's that it's practically a problem... even little things like not being able to create normal unit tests, to people simply getting it wrong. "What happens if some genius kills rngd?" "All your private keys leak" (well, not on linux??? but just generally that idea)
685 2012-12-26 21:19:14 <sipa> well, implementing something like that in the satoshi client isn't hard - but would it gain us anything?
686 2012-12-26 21:19:41 <gmaxwell> I don't think so.
687 2012-12-26 21:19:41 <jrmithdobbs> eh? it's pretty hard to do well and portably
688 2012-12-26 21:19:43 <gmaxwell> :(
689 2012-12-26 21:19:58 <gmaxwell> jrmithdobbs: he means ed25519 like determinstic k.
690 2012-12-26 21:20:01 <jrmithdobbs> o
691 2012-12-26 21:20:10 <jrmithdobbs> i thought he meant an inbuilt rngd ;p
692 2012-12-26 21:20:19 <sipa> the advantage is not depending on your RNG for signing; the disadvantage is code review, testing, and risk of exposing other cryptographic flaws
693 2012-12-26 21:20:56 <gmaxwell> well if you don't already have a ECDSA implementation that comes with a secure prng then the risks favor the determinstic one
694 2012-12-26 21:20:58 <sipa> too bad we can't make it a network rule :D
695 2012-12-26 21:21:01 <gmaxwell> esp as you can use standard test vectors.
696 2012-12-26 21:21:20 <Luke-Jr> Merry Christmas, all!
697 2012-12-26 21:21:37 <gmaxwell> Is it tonal christmas now?
698 2012-12-26 21:21:49 <Luke-Jr> ???
699 2012-12-26 21:21:53 <jrmithdobbs> yes, started f minutes ago
700 2012-12-26 21:21:53 <Luke-Jr> Christmas is Christmas
701 2012-12-26 21:22:25 <sipa> April 1st: the protocol will be changed; txins must now contain your private key, so nodes can validate you used a deterministic signature!
702 2012-12-26 21:22:40 <wizkid057> ...
703 2012-12-26 21:22:41 <gmaxwell> 'badcoin'
704 2012-12-26 21:23:02 <gmaxwell> 'an alt. chain with as many bad ideas as possible included'
705 2012-12-26 21:23:09 <jrmithdobbs> merry christmas to me, someone broken the damned ghc port in freebsd
706 2012-12-26 21:23:24 <jrmithdobbs> s/someone/someone's/
707 2012-12-26 21:23:40 <gmaxwell> actually running programs isn't very pure.
708 2012-12-26 21:24:37 <Luke-Jr> gmaxwell: web-only?
709 2012-12-26 21:24:43 <jrmithdobbs> gmaxwell: actually, this box uses two (well, three but radiusd isn't really supposed to live there, it's temporary) whole things outside of base
710 2012-12-26 21:24:59 <jrmithdobbs> gmaxwell: netatalk and git annex (which is haskell hence ghc)
711 2012-12-26 21:25:29 <jrmithdobbs> gmaxwell: the rest of what it does is nfs/zfs stuff ;p
712 2012-12-26 21:25:45 <Luke-Jr> gmaxwell: and the merkletree algorithm is secret, so miners must use a single centralized getwork pool <.<
713 2012-12-26 21:26:16 <sipa> what about using NTP to decide transaction order, instead of this bulky blockchain thing?
714 2012-12-26 21:26:35 <andytoshi> Luke-Jr, it should be hosted on a filesharing site
715 2012-12-26 21:26:52 <andytoshi> dropbox
716 2012-12-26 21:26:52 <wizkid057> meagbitupload
717 2012-12-26 21:26:56 <andytoshi> haha
718 2012-12-26 21:26:58 <wizkid057> *mega
719 2012-12-26 21:27:02 <jrmithdobbs> sipa: too decentralized
720 2012-12-26 21:27:11 <jrmithdobbs> sipa: tlsdate off of https://facebook.com
721 2012-12-26 21:29:13 <wizkid057> should just email the coins
722 2012-12-26 21:30:43 <wizkid057> we'll call them spamcoins
723 2012-12-26 21:31:04 <sipa> how about using proof-of-work to prevent spam?
724 2012-12-26 21:31:27 <theymos> Instead of crypto signatures, pass around scans of legal documents with handwritten signatures.
725 2012-12-26 21:33:28 <upb> you mean digital signatures right ?:P
726 2012-12-26 21:33:29 <jgarzik> store it in a DHT
727 2012-12-26 21:33:45 <wizkid057> jgarzik: thats an awesome idea :D
728 2012-12-26 21:34:46 <wizkid057> i still have this in my favorite IRC quotes: [12:15] <gmaxwell> Someday I'm going to get myself invited to some conference with the president, and while he's talking about some middle east conflict thing??? I'm going to ask if they've considered using a DHT.
729 2012-12-26 21:39:44 <andytoshi> the recommended client should be written in BASIC
730 2012-12-26 21:40:48 <andytoshi> and have an easy-to-pun name like 'TrucksYallCoin'
731 2012-12-26 21:50:51 <wizkid057> probably needs to be 8 chars or less for BASIC.... and we can store the code on cassette tapes
732 2012-12-26 21:56:01 <andytoshi> BADCOI~1.EXE
733 2012-12-26 21:56:08 <sipa> LOL
734 2012-12-26 22:09:28 <gmaxwell> hahaha
735 2012-12-26 22:10:17 <gmaxwell> 14:33 < upb> you mean digital signatures right ?:P
736 2012-12-26 22:11:10 <gmaxwell> Yea, there were these laws passed in the US to allow 'digital signatures' to be used for varrous things, but no one defined digital signature... so there are now all these official things that ask you to provide "a digital signature" on official docments (you type your name).
737 2012-12-26 22:13:00 <freewil> lol yeah
738 2012-12-26 22:13:21 <freewil> pin codes too sometimes act as a digital signature
739 2012-12-26 22:13:40 <freewil> thats how student loans work with FAFSA
740 2012-12-26 22:13:48 <gmaxwell> sipa: I got one forum message reply, but I don't think he quite understands that I'm talking about _his_ transactions, as he's offering me abstract ideas on what programs might make non-standard signatures.
741 2012-12-26 22:14:02 <maaku> When I switched insurance a year ago, I was asked to "digitally sign" the document by typing my initials in a little form.
742 2012-12-26 22:14:02 <sipa> ha
743 2012-12-26 22:14:07 <maaku> I felt very secure.
744 2012-12-26 22:15:20 <sipa> And rightfully so.
745 2012-12-26 22:17:53 <gmaxwell> To be fair, most of this stuff exists in a framework where _everything_ is reversable. Different elements of security matter.
746 2012-12-26 22:37:02 <gmaxwell> one of the non-canonical signer addresses is mentioned in a tweet:
747 2012-12-26 22:37:03 <gmaxwell> @NGOBuzz ?? 13 Sep '12, 10am
748 2012-12-26 22:38:59 <sipa> are you on twitter? I can send a DM otherwise
749 2012-12-26 22:39:50 <andytoshi> gmaxwell, a certain fan fiction site is using "Lawrence G. Walters, Esq. (2004). BirthDateVerifierTM Version 4.0"
750 2012-12-26 22:39:54 <andytoshi> which involves a digital signature
751 2012-12-26 22:40:13 <andytoshi> "Access to this electronic record requires a simple browser program such as Internet Explorer??? or Netscape??? and a computer. You adopt the mark typed in the form below as your electronic signature."
752 2012-12-26 22:42:17 <andytoshi> http://www.birthdateverifier.com/
753 2012-12-26 22:43:08 <gmaxwell> On the plus side I've now PMed 6 more people on the forums about their weird signatures. On the minus side, google is now convinced that I'm a robot and is giving me captachas on almost every search... and I can't solve half the captchas.
754 2012-12-26 22:44:25 <andytoshi> ouch gmaxwell, i get often get those using tor
755 2012-12-26 22:44:31 <andytoshi> so i switch exit nodes, because i can't solve any of them
756 2012-12-26 22:46:44 <gmaxwell> I wonder if you can buy programattic access to google. ... maybe thats what that google+ thing is about.
757 2012-12-26 22:47:03 <sipa> the search API?
758 2012-12-26 22:48:38 <gmaxwell> Yea. :P
759 2012-12-26 22:49:25 <stealth222> yes, you can
760 2012-12-26 22:51:20 <stealth222> 100 queries a day for free
761 2012-12-26 22:51:23 <stealth222> any more, you need to pay
762 2012-12-26 23:17:26 <epscy> anyway to limit how much memory bitcoind can use
763 2012-12-26 23:17:58 <sipa> not really, but most of it is send/receive buffers
764 2012-12-26 23:18:04 <sipa> if you have many connections
765 2012-12-26 23:18:20 <gmaxwell> epscy: sure! if you use ulimit it will be limited and crash when it hits the limit.
766 2012-12-26 23:18:21 <epscy> ahh so i could limit connections
767 2012-12-26 23:18:49 <gmaxwell> ohh.. you want it to be limited _AND_ keep working? yea, disable listening and perhaps reduce your connection count further.
768 2012-12-26 23:18:49 <sipa> or reduce the size of the buffer per connection
769 2012-12-26 23:19:08 <epscy> hmmm, how do i do all that
770 2012-12-26 23:20:09 <sipa> -maxreceivebuffer (mine is at 1200), -maxsendbuffer (mine is at 1600), -maxconnections
771 2012-12-26 23:20:30 <sipa> and -nolisten perhaps
772 2012-12-26 23:22:16 <epscy> gonna try setting connections to 10
773 2012-12-26 23:22:20 <epscy> it had 63
774 2012-12-26 23:22:38 <epscy> this is on an openvz vm with 1 gig of ram
775 2012-12-26 23:22:49 <epscy> which it was using almost all of
776 2012-12-26 23:23:06 <epscy> and giving me occasional bad malloc errors
777 2012-12-26 23:23:14 <epscy> thanks for your help
778 2012-12-26 23:23:15 <sipa> what code version?
779 2012-12-26 23:23:43 <epscy> 0.7.2
780 2012-12-26 23:24:10 <gmaxwell> epscy: I wonder if your openvz is limiting vm instead of rss.
781 2012-12-26 23:24:22 <epscy> rss?
782 2012-12-26 23:24:39 <gmaxwell> s/rss/res/
783 2012-12-26 23:24:39 <sipa> resident set size
784 2012-12-26 23:24:45 <epscy> oh
785 2012-12-26 23:24:45 <gmaxwell> yea, or that.
786 2012-12-26 23:24:49 <epscy> i don't know
787 2012-12-26 23:24:56 <gmaxwell> actual used memory verses address space.
788 2012-12-26 23:25:03 <epscy> i don't run the the host
789 2012-12-26 23:25:06 <gmaxwell> 1gb sounds a bit high for actual used memory for bitcoin.
790 2012-12-26 23:25:46 <gmaxwell> e.g. I have a node right now with 25 connections using 200mb.
791 2012-12-26 23:25:59 <gmaxwell> ah, actually 291.
792 2012-12-26 23:26:19 <gmaxwell> but virtual is 1857m
793 2012-12-26 23:26:22 <gmaxwell> (uhhh)
794 2012-12-26 23:26:58 <sipa> mine has 454/1334, with 86 connections
795 2012-12-26 23:27:06 <epscy> i see
796 2012-12-26 23:27:16 <epscy> it's a pretty cheap vm
797 2012-12-26 23:27:38 <sipa> i see no reason why it would limit VM
798 2012-12-26 23:27:54 <sipa> (virtual memory, not virtual machine)
799 2012-12-26 23:27:55 <gmaxwell> sipa: there is no good reason, but I know that at least once before someone ran into that.
800 2012-12-26 23:28:21 <stealth222> the virtual machine might be running on a very small HD partition?
801 2012-12-26 23:28:36 <gmaxwell> I only suggested it because as you point out, 86/454... I'd expect max connections can be reached with 1gb.
802 2012-12-26 23:28:43 <gmaxwell> stealth222: vm has nothing to do with swap space.
803 2012-12-26 23:29:08 <epscy> huh only 1.3gb of disk space available
804 2012-12-26 23:29:12 <gmaxwell> stealth222: it's basically measuring address space fragmentation there.
805 2012-12-26 23:29:15 <epscy> could that be a problem?
806 2012-12-26 23:29:24 <sipa> not yet
807 2012-12-26 23:30:14 <stealth222> don't you need like 20gb of disk space just for the block chain?
808 2012-12-26 23:30:24 <epscy> more like 6gb
809 2012-12-26 23:31:04 <sipa> for the block chain, 4.5 GB or so
810 2012-12-26 23:31:25 <sipa> the 0.7 blkindex.dat requires another 1.5 GB
811 2012-12-26 23:31:25 <stealth222> it used to take a lot more with BDB
812 2012-12-26 23:31:41 <lianj> our mongodb is 18 oO
813 2012-12-26 23:32:12 <sipa> 0.8 undo data takes 600 MB
814 2012-12-26 23:35:18 <gmaxwell> my 0.8 directories are about a gig smaller than the 0.7 directories right now.
815 2012-12-26 23:35:44 <gmaxwell> I guess that difference should increase with time.
816 2012-12-26 23:39:07 <stealth222> about how much RAM does the mempool typically take? and how long before orphaned transactions are removed?