1 2011-11-19 00:06:34 <gavinandresen> dooglus: could be... what operating system?
  2 2011-11-19 00:06:55 <dooglus> I made a bug report with the details: https://github.com/bitcoin/bitcoin/issues/644
  3 2011-11-19 00:07:00 <dooglus> but it's archlinux
  4 2011-11-19 00:11:33 <gavinandresen> dooglus: dunno what the issue is...
  5 2011-11-19 00:12:29 <dooglus> gavinandresen: is there some test I can run to help locate it?  maybe turn on debugging in 'db'?
  6 2011-11-19 00:13:03 <gavinandresen> bdb isn't hard to compile, you could compile and link against a -g version of it and see where/why it is crashing
  7 2011-11-19 00:13:43 <dooglus> I'm building 4.8.30 at the moment
  8 2011-11-19 00:13:45 <Folklore> does bitcoin use epoll
  9 2011-11-19 00:13:48 <dooglus> will see if that fixes the problem
 10 2011-11-19 00:14:17 <gavinandresen> Folklore: nope
 11 2011-11-19 00:14:34 <gavinandresen> Folklore: does epoll work on Windows?
 12 2011-11-19 00:15:11 <phantomcircuit> gavinandresen, uh no
 13 2011-11-19 00:15:21 <Folklore> gavinandresen no but why does that matter?
 14 2011-11-19 00:15:27 <gavinandresen> .... well, then, that'd be a good reason bitcoin doesn't use it...
 15 2011-11-19 00:15:35 <Folklore> windows has IOCP
 16 2011-11-19 00:15:38 <gavinandresen> bitcoin is windows/linux/mac
 17 2011-11-19 00:15:47 <phantomcircuit> you use libevent (or similar)
 18 2011-11-19 00:15:48 <Folklore> no that's not a good reason
 19 2011-11-19 00:15:57 <luke-jr> phantomcircuit: UGH NO
 20 2011-11-19 00:15:59 <Folklore> ever heard of OS macros?
 21 2011-11-19 00:16:00 <Folklore> #IF MACOS
 22 2011-11-19 00:16:03 <phantomcircuit> luke-jr, or similar
 23 2011-11-19 00:16:14 <Folklore> so what's it use select?
 24 2011-11-19 00:16:20 <gavinandresen> yes, select.
 25 2011-11-19 00:16:27 <phantomcircuit> it kind of uses select
 26 2011-11-19 00:16:37 <gavinandresen> If you want to rewrite it, seems like boost::asio would be the way to go.
 27 2011-11-19 00:16:39 <Folklore> ugh, I thought the server side handled a lot of clients, no?
 28 2011-11-19 00:16:40 <phantomcircuit> i kind of forget
 29 2011-11-19 00:16:43 <phantomcircuit> but it's something like
 30 2011-11-19 00:16:59 <phantomcircuit> 1 thread does network io, 1 thread does logic, or something like that
 31 2011-11-19 00:17:43 <Folklore> alrighty, guess that makes sense, since select is cross platform, i'm not sure how much the connection weight is
 32 2011-11-19 00:18:06 <gavinandresen> not an issue at this point.
 33 2011-11-19 00:18:09 <Folklore> for high end server should consider epoll for linux, and IOCP for win
 34 2011-11-19 00:18:15 <Folklore> maybe in the future
 35 2011-11-19 00:18:23 <Folklore> idk what unix uses
 36 2011-11-19 00:18:23 <phantomcircuit> uh
 37 2011-11-19 00:18:32 <phantomcircuit> the main problem is that bitcoin uses an internal buffer
 38 2011-11-19 00:18:41 <phantomcircuit> instead of trusting the os network buffer
 39 2011-11-19 00:19:03 <Folklore> network level proven to be faster, that's the whole point
 40 2011-11-19 00:19:10 <Folklore> kernel rather
 41 2011-11-19 00:19:19 <Folklore> kernel level, ugh I need coffee
 42 2011-11-19 00:20:31 <luke-jr> Folklore: I think the general consensus is "yeah. got a branch to merge?"
 43 2011-11-19 00:20:48 <Folklore> lol fair enough
 44 2011-11-19 00:21:03 <Folklore> heres a good epoll example though, IOCP alil bit more tricky
 45 2011-11-19 00:21:08 <phantomcircuit> part of the problem is that the individual bitcoin messages can be huge
 46 2011-11-19 00:21:21 <luke-jr> Folklore: no no, it's not that anyone doesn't know how, or needs examples
 47 2011-11-19 00:21:23 <Folklore> https://banu.com/blog/2/how-to-use-epoll-a-complete-example-in-c/
 48 2011-11-19 00:21:24 <gavinandresen> "Patches welcome"    Just remember we're WAY WAY WAY more uptight about security than the typical open source project.  And even then we screw up.
 49 2011-11-19 00:21:25 <luke-jr> Folklore: someone just has to actually *do it*
 50 2011-11-19 00:21:41 <Folklore> luke-jr I get that, but examples don't hurt
 51 2011-11-19 00:21:48 <luke-jr> Folklore: they don't help either :p
 52 2011-11-19 00:22:17 <Folklore> can give ideas of a slightly different way to do somethin :p
 53 2011-11-19 00:22:59 <luke-jr> redoing the network stack in the codebase is one of those "eeek, easier to rewrite it all" things
 54 2011-11-19 00:23:01 <Folklore> phantom what does it matter how large the bitcoin is? i'm misunderstanding you
 55 2011-11-19 00:24:05 <phantomcircuit> a single getdata can result in 49968 individual blocks being requested
 56 2011-11-19 00:24:26 <Folklore> things are broken up in smaller packets anyway, trying to send something at a megabyte a time, for all machines isn't ideal, much better to lower it to a reasonable number, 512/1024 bytes
 57 2011-11-19 00:25:21 <phantomcircuit> the problem there being that the single getdata packet is processed all in 1 go
 58 2011-11-19 00:25:50 <phantomcircuit> gavinandresen, btw that reminds me you can cause memory to spike if you send getdata packets simultaneously and rapidly reconnect
 59 2011-11-19 00:26:02 <Folklore> by design or are you saying it needs to be process as a whole? cause you can just append the data as it comes in, or am I still misunderstanding?
 60 2011-11-19 00:26:26 <phantomcircuit> Folklore, because of the way the current client is engineered
 61 2011-11-19 00:26:54 <gavinandresen> phantomcircuit: thanks
 62 2011-11-19 00:26:58 <Folklore> sounds really broken if it's trying to read packets at megabytes at a time
 63 2011-11-19 00:27:16 <Folklore> if you test lower request sizes you'll get better throughoput on most machines
 64 2011-11-19 00:28:04 <gavinandresen> optimizing for throughput is so far down on the priority list I can't even see it....
 65 2011-11-19 00:28:06 <phantomcircuit> Folklore, yeah there is an internal send buffer which kills the client at 1MB (10MB??) i forget exactly
 66 2011-11-19 00:28:46 <phantomcircuit> i mean kills the connection
 67 2011-11-19 00:29:16 <gribble> 153893
 68 2011-11-19 00:29:16 <phantomcircuit> ;;bc,blocks
 69 2011-11-19 00:30:07 <Folklore> sounds kinda funny if you're being serious
 70 2011-11-19 00:30:49 <gavinandresen> what sounds funny?  If a peer trys to flood you with data you drop its connection?
 71 2011-11-19 00:31:37 <phantomcircuit> hmm the other way
 72 2011-11-19 00:32:00 <dooglus> gavinandresen: rebuilding with bdb 4.8.30 didn't help
 73 2011-11-19 00:32:07 <Folklore> yeah he's saying is a client is trying to recieve data too fast he just kills that connections, and lol,that is funny
 74 2011-11-19 00:32:07 <gavinandresen> Ok, if a peer requests a bunch of data and then refuses to process it in a timely fashion you drop the connection?
 75 2011-11-19 00:33:04 <gavinandresen> patches welcome... rewriting that code to be smarter about data flow control would be great.
 76 2011-11-19 00:33:42 <phantomcircuit> to do that you basically have to rewrite the logic to queue requests for information instead of the actual information
 77 2011-11-19 00:33:52 <gavinandresen> dooglus:  weird, haven't seen anybody else having that particular problem.
 78 2011-11-19 00:33:57 <Folklore> if it's contigent on the other side processing it fast enough between requests thats one thing, freenode does that with channel lists, and is a good method, but that's not what he said, at least what I read
 79 2011-11-19 00:33:59 <phantomcircuit> then select for write on sockets with queue'd requests
 80 2011-11-19 00:34:12 <phantomcircuit> but then you could stall the networking thread waiting for db stuff
 81 2011-11-19 00:34:28 <phantomcircuit> so
 82 2011-11-19 00:34:32 <phantomcircuit> yeah screw that
 83 2011-11-19 00:34:52 <Folklore> or you just skip that users socket and poll the rest? :p
 84 2011-11-19 00:34:52 <phantomcircuit> which is probably the reason it's built the way it is anyways
 85 2011-11-19 00:35:17 <phantomcircuit> Folklore, uh no
 86 2011-11-19 00:35:31 <Folklore> if they haven't done what they needed within a timeframe then you can close, but to block on a individual users socket is silly
 87 2011-11-19 00:35:33 <phantomcircuit> <- request
 88 2011-11-19 00:35:38 <Folklore> for large scale anyway
 89 2011-11-19 00:35:49 <phantomcircuit> queue response identifiers ->
 90 2011-11-19 00:36:03 <phantomcircuit> wait for socket write
 91 2011-11-19 00:36:14 <phantomcircuit> pull actual data
 92 2011-11-19 00:36:16 <phantomcircuit> send data
 93 2011-11-19 00:36:23 <phantomcircuit> only problem
 94 2011-11-19 00:36:39 <phantomcircuit> pulling the actual data can break
 95 2011-11-19 00:36:40 <phantomcircuit> er
 96 2011-11-19 00:36:42 <phantomcircuit> block
 97 2011-11-19 00:37:03 <Folklore> what are you pulling it from
 98 2011-11-19 00:37:11 <Folklore> remote or a local file or something
 99 2011-11-19 00:37:12 <phantomcircuit> possibly disk
100 2011-11-19 00:37:20 <phantomcircuit> possibly nfs
101 2011-11-19 00:37:24 <phantomcircuit> you dont know
102 2011-11-19 00:37:29 <phantomcircuit> it could block
103 2011-11-19 00:38:23 <Folklore> if the bitcoin database doesn't suck, it should be pretty fast? IDK how large it is but could keep in memory too, for even quicker access
104 2011-11-19 00:39:20 <Folklore> from the sounds of it though, sounds like it works fine, the main thread polling sockets, and the other actually doing the work, only big improvement you should add is a threadpool
105 2011-11-19 00:39:29 <Folklore> for the workload, and have them go at it
106 2011-11-19 00:39:35 <gavinandresen> what workload?
107 2011-11-19 00:39:42 <Folklore> recv/sending
108 2011-11-19 00:39:46 <Folklore> and processing data
109 2011-11-19 00:40:04 <gavinandresen> Have you looked at bitcoin network bandwidth?  We're nowhere near close to having that be the issue.
110 2011-11-19 00:40:13 <Folklore> for instance recv and the socket can be added to a que, and a thread in the pool could andle it
111 2011-11-19 00:40:15 <gavinandresen> ... and before that is the issue, CPU usage will be the issue.
112 2011-11-19 00:40:29 <gavinandresen> (validating transactions requires ECDSA signature operations, which are slow)
113 2011-11-19 00:41:50 <Folklore> anyway it could catch questions? cache something
114 2011-11-19 00:42:07 <Folklore> it's the same block data being shared to everyone, and theres no encryption used right?
115 2011-11-19 00:43:29 <Folklore> cach'n in mem is an awesome thing, would probably speed things up a lot too, I remember when I had to download all the blocks or whatever, took forever
116 2011-11-19 00:44:02 <gavinandresen> sigh....
117 2011-11-19 00:44:11 <Folklore> could also implement a simple speed test feature, and perhaps have the bitcoin client utilize that as its max, obviously giving the user the ability to rate limit
118 2011-11-19 00:44:29 <gavinandresen> I can assure you epoll or caching on the sending side won't make downloading the block chain faster.
119 2011-11-19 00:45:02 <gavinandresen> (well, not noticeably faster)
120 2011-11-19 00:45:27 <gavinandresen> Speeding up initial block-chain download is near the top of my TODO list
121 2011-11-19 00:45:59 <Folklore> it makes http servers run noticeably fasters and they handle a great deal of requests, your complaint seems to be some form of verification, hashing or something that takes a lot of cpu cycles, why would not keeping the fruit of that labor in a cache not speed up things?
122 2011-11-19 00:46:53 <gavinandresen> No two hashes in the block chain are the same, so there is nothing to cache.  I am actually sitting on a patch to speed up ECDSA verification, but need to write more unit tests to make sure it doesn't break something
123 2011-11-19 00:47:11 <gavinandresen> I'll catch y'all later.
124 2011-11-19 00:47:35 <Folklore> awesome :D faster bitcoin good for everyone
125 2011-11-19 00:54:03 <coventry> I'm trying to understand the bitcoin transaction protocol.  The TxIn table in the Protocol Specification page mentions a "sequence" field, described as "Transaction version as defined by the sender. Intended for "replacement" of transactions when information is updated before inclusion into a block."  Can you use this to invalidate a transaction before it's included in a block?  I assume not, because I have heard that a
126 2011-11-19 00:57:13 <coventry> Are such questions unwelcome here?  (Is this only for discussion of development of the official bitcoin client?)
127 2011-11-19 01:05:31 <coventry> Oh, I bet most of the time it's set to MAX_UINT, so you can't post a later version...
128 2011-11-19 01:05:52 <devrandom> coventry: tx replacement is currently disabled, AFAIK
129 2011-11-19 01:07:42 <coventry> Thanks.
130 2011-11-19 03:32:31 <Vontux> hello, I was wondering if anyone could explain why bitcoin.conf isn't present on my system after installing the bitcoin client
131 2011-11-19 10:34:36 <[Tycho]> Hello.
132 2011-11-19 10:49:31 <Eliel> Hello
133 2011-11-19 10:50:30 <[Tycho]> Hello, Eliel. Who are you ?
134 2011-11-19 10:53:35 <Eliel> Not famous for anything. at least, not yet :)
135 2011-11-19 10:54:51 <[Tycho]> Oh, ok.
136 2011-11-19 11:01:49 <SomeoneWeird> lol..
137 2011-11-19 11:42:55 <[Tycho]> Did someone tried to use "sendmany" ?
138 2011-11-19 13:02:48 <Diablo-D3> ;;bc,diff
139 2011-11-19 13:02:50 <gribble> 1192497.7500895
140 2011-11-19 13:02:51 <Diablo-D3> ;;ticker
141 2011-11-19 13:02:52 <gribble> Best bid: 2.2007, Best ask: 2.2097, Bid-ask spread: 0.009, Last trade: 2.19437, 24 hour volume: 140844, 24 hour low: 1.99998, 24 hour high: 2.30498
142 2011-11-19 14:06:51 <[Tycho]> Why so quiet ?
143 2011-11-19 14:36:10 <Litt> Everyone is busy enjoying the new p2p forum
144 2011-11-19 15:20:12 <roconnor> devrandom: While tx replacement is disabled in the standard client, it is still part of the protocol; ie other clients could support tx replacement.
145 2011-11-19 16:43:47 <gmaxwell> Sort of evidence that people are unable to reason about security bugs& the litecoin fork removed encrypted wallet support as a result of the current issue.
146 2011-11-19 16:47:26 <roconnor> gmaxwell: arugably broken encrypted wallet is worse than no encrypted wallet
147 2011-11-19 16:48:02 <gmaxwell> roconnor: Not if you warn the user.
148 2011-11-19 16:48:34 <roconnor> gmaxwell: I still think it is a little worse
149 2011-11-19 16:49:12 <gmaxwell> roconnor: You'd have to weigh it against the fact that all new keys are totally secure.
150 2011-11-19 16:49:43 <roconnor> true
151 2011-11-19 16:50:26 <roconnor> I wish we had (your?) scheme for generating keys from one key
152 2011-11-19 16:51:05 <gmaxwell> I don't know that it would really help that much.. and plus, it would make it harder to recover from the kind of partial leaks this created.
153 2011-11-19 16:52:05 <[Tycho]> Why transactions with size less than 100 bytes are not accepted ?
154 2011-11-19 16:53:53 <gmaxwell> It's actually possible to write a useful transaction that small?
155 2011-11-19 16:54:25 <[Tycho]> The question is: it's actually possible to write a malicious transaction that small ?
156 2011-11-19 16:55:06 <roconnor> it does seem silly
157 2011-11-19 16:55:17 <roconnor> It is pretty easy to pad transactions
158 2011-11-19 16:55:21 <gmaxwell> well, at least its a computationally cheap check.
159 2011-11-19 16:55:33 <roconnor> right
160 2011-11-19 16:55:42 <[Tycho]> if (GetSigOpCount() > nSize / 34 || nSize < 100)
161 2011-11-19 16:55:59 <roconnor> [Tycho]: I think it is used as a quick way to drop non-standard transactions that the standard client will drop anyways
162 2011-11-19 16:56:32 <gmaxwell> [Tycho]: There are a lot of "redundant" tests on transactions because DOS resistance implies dropping things that are bad as quickly and easily as possible,  also, belt+suspenders is good security.
163 2011-11-19 16:56:38 <[Tycho]> Ok, I'm just being careful about breaking something :)
164 2011-11-19 16:57:01 <roconnor> [Tycho]: this check is specific to the standard client; It is not part of the protocol AFAIU
165 2011-11-19 16:59:45 <iddo> roconnor: but the protocol was derived from the standard client?
166 2011-11-19 17:00:04 <roconnor> iddo: only a fragment of the standard client
167 2011-11-19 17:00:40 <iddo> what's considered the official protocol? the parts written in the wiki?
168 2011-11-19 17:00:42 <gmaxwell> iddo: The distinction here is that checks which will cause the software to reject a longer chain that fails them are part of the dirtributed algorithim.
169 2011-11-19 17:00:46 <[Tycho]> Also transactions without outputs are effectively prohibited by network :(
170 2011-11-19 17:00:52 <roconnor> iddo: what transactions the standard client chooses to rebroadcast has nothing to do with what transactions are accepted in the blockchain
171 2011-11-19 17:01:28 <gmaxwell> [Tycho]: .. What are you upto?
172 2011-11-19 17:01:32 <roconnor> iddo: The code in the standard client dealing with the blockchain (directly and indirectly) is the official protocol.
173 2011-11-19 17:01:51 <[Tycho]> gmaxwell: this week i'm playing with non-standart TXes
174 2011-11-19 17:02:09 <iddo> roconnor: ok thanks
175 2011-11-19 17:02:21 <roconnor> iddo: actually the word protocol I am using is a bit ambigous
176 2011-11-19 17:02:40 <roconnor> 'cause there is also the bitcoin P2P network protocol in addition to the bitcoin blockchain protocol
177 2011-11-19 17:02:56 <gmaxwell> roconnor: I like to call the blockchain rules "the distributed algorithim"
178 2011-11-19 17:02:58 <[Tycho]> It's like blockchain rules.
179 2011-11-19 17:03:21 <roconnor> gmaxwell: Hmm.
180 2011-11-19 17:03:44 <gmaxwell> Because if you change yours _at all_ you'll become a different currency eventually.
181 2011-11-19 17:04:04 <roconnor> yep
182 2011-11-19 17:04:16 <roconnor> I don't like the word "algorithm" in your name
183 2011-11-19 17:05:06 <roconnor> I do like "blockchian rules"
184 2011-11-19 17:05:14 <gmaxwell> roconnor: its a set of well defined steps for reaching a conclusion. (and perhaps its not narrow enough?)
185 2011-11-19 17:05:15 <roconnor> naming things is so hard
186 2011-11-19 17:05:25 <roconnor> gmaxwell: what is the conclusion?
187 2011-11-19 17:05:37 <gmaxwell> roconnor: the identity of the unique longest chain.
188 2011-11-19 17:05:56 <roconnor> hmm
189 2011-11-19 17:55:18 <[Tycho]> Why OP_CAT is disabled ? It's so useful.
190 2011-11-19 17:56:07 <lupine_85> kunwon1, it works for other channels too
191 2011-11-19 18:03:07 <gmaxwell> [Tycho]: presumably because it looks a bit scarry in terms of overflow risk. All the disabled opcodes make me sad, but people have still not written tests and independant implementations of all of the,
192 2011-11-19 18:03:49 <sipa> roconnor even discovered an error on the op list on the wiki
193 2011-11-19 18:04:06 <sipa> so you can see how "well tested" the documentation is
194 2011-11-19 18:04:26 <gmaxwell> oh the NEQ ? yea...
195 2011-11-19 19:15:40 <roconnor> [Tycho]: OP_CAT being disabled *is* part of the bitcoin rules :)
196 2011-11-19 19:16:00 <roconnor> well probably
197 2011-11-19 19:16:01 <[Tycho]> Why it's disabled ?
198 2011-11-19 19:16:18 <roconnor> I'm not entirely sure
199 2011-11-19 19:16:56 <roconnor> Does doing a sequence of OP_DUP, OP_CAT's cause expontial growth in memory use?
200 2011-11-19 19:17:26 <makomk> Yeah, should do.
201 2011-11-19 19:18:25 <makomk> You'd need to document and enforce some kind of memory limit and it'd have to be the same on all implementations...
202 2011-11-19 19:20:33 <roconnor> I'd be interested in a theorem that says that the space and/or time use of the current set of operation is polynomial in the size of the script
203 2011-11-19 19:20:59 <roconnor> and wield that against this scary OP_EVAL proposal
204 2011-11-19 19:21:24 <sipa> that depends entirely on which operations are allowed
205 2011-11-19 19:22:59 <[Tycho]> Why exponential ?
206 2011-11-19 19:23:21 <sipa> and as the size of const operations is limited, the number of stack elements is limited, no looping constructs exist, and no operations exist that increase the size of an element more that with a constant term
207 2011-11-19 19:23:28 <sipa> i'd say it's all nicely O(1) :)
208 2011-11-19 19:25:23 <roconnor> sipa: you lost me
209 2011-11-19 19:26:47 <roconnor> sipa: who limits the number of stack items?
210 2011-11-19 19:27:03 <roconnor> sipa: OP_EVAL is effectively a looping construct
211 2011-11-19 19:27:55 <roconnor> sipa: OP_CAT could increase the size of an element by more than the constant term.
212 2011-11-19 19:30:38 <roconnor> sipa: and if you are right it makes the memory use linear in the size of the script, not constant :)
213 2011-11-19 19:33:31 <roconnor> sipa: anyhow OP_DUP clearly doesn't cause a bounded amount of memory increase (well in Haskell I have to do something like OP_DUP, OP_1ADD to unshare the duplication).
214 2011-11-19 19:34:39 <sipa> roconnor: with the current limitations, you cannot push values larger than 520 bytes
215 2011-11-19 19:35:00 <roconnor> (and I realize now that OP_CAT actually only affect memory use by a bounded amount)
216 2011-11-19 19:35:02 <sipa> and the size of the script seems limited to 10k
217 2011-11-19 19:35:42 <sipa> i claim that this limits the amount of memory to 5.2M :)
218 2011-11-19 19:36:03 <roconnor> sipa: but OP_DUP duplicates whatever is one the stack, and using arithmetic you can increase the size of objects on the stack beyond 520 bytes.
219 2011-11-19 19:36:23 <sipa> i was all assuming the current limitations
220 2011-11-19 19:36:37 <roconnor> Can't OP_PUSHDATA push 2^32 bytes onto a the stack item?
221 2011-11-19 19:36:52 <sipa> the script language can, the limitations don't allow it
222 2011-11-19 19:37:17 <roconnor> sipa: how does the limitations stop OP_PUSHDATA4?
223 2011-11-19 19:38:00 <sipa> as the script can't be larger itself (in bytes) than 10000, how can an OP_PUSHDATA4 with 4k size ever be parsed correctly?
224 2011-11-19 19:38:14 <roconnor> sipa: but you said the limit was 520 bytes
225 2011-11-19 19:38:39 <sipa> oh, right, there's an additional limitation on the size of data being pushed
226 2011-11-19 19:39:27 <[Tycho]> Actually OP_CAT checks for op sizr
227 2011-11-19 19:39:31 <[Tycho]> *sizee
228 2011-11-19 19:39:46 <sipa> OP_CAT is just disabled
229 2011-11-19 19:39:49 <the__batman> anyone figure out how to make bitcoin marketable yet?
230 2011-11-19 19:39:57 <the__batman> and not just a sweet money laundering tool
231 2011-11-19 19:39:58 <the__batman> :)
232 2011-11-19 19:40:02 <sipa> the__batman: with patience
233 2011-11-19 19:40:10 <the__batman> I prefer aggressive science
234 2011-11-19 19:40:28 <luke-jr> the__batman: make it easy for merchants
235 2011-11-19 19:40:30 <roconnor> sipa: Oh I see the limit now
236 2011-11-19 19:40:36 <roconnor> heh
237 2011-11-19 19:40:49 <roconnor> I'm a bit abivalent about implementing these arbitrary limits
238 2011-11-19 19:40:50 <sipa> roconnor: and there are afaik no operations that increase the size of data itself
239 2011-11-19 19:40:51 <the__batman> luke-jr: suppose it were perfectly easy. then what? why would they?
240 2011-11-19 19:41:02 <roconnor> sipa: OP_1ADD
241 2011-11-19 19:41:12 <luke-jr> the__batman: cuz USD is untrustworthy
242 2011-11-19 19:41:32 <roconnor> OP_ADD even
243 2011-11-19 19:42:35 <roconnor> funny how OP_2MUL is disabled when it can be implemented by OP_DUP OP_ADD
244 2011-11-19 19:42:35 <sipa> roconnor: oh yes, but N OP_1ADD operations can at most increase the max size of an element with log2(N)
245 2011-11-19 19:42:49 <sipa> hmm, OP_ADD is enabled still?
246 2011-11-19 19:42:59 <sipa> i misremembered
247 2011-11-19 19:43:03 <roconnor> I believe so
248 2011-11-19 19:43:09 <sipa> you're right
249 2011-11-19 19:43:10 <roconnor> it might even be used
250 2011-11-19 19:43:22 <roconnor> sipa: I'm impressed that I was so close to being wrong
251 2011-11-19 19:44:03 <roconnor> OP_ADD and OP_SUB should be disabled
252 2011-11-19 19:44:08 <sipa> still, N OP_ADD operations can still at most increase the max size of an element with log2(N)
253 2011-11-19 19:44:15 <sipa> eh
254 2011-11-19 19:44:16 <sipa> N
255 2011-11-19 19:44:23 <roconnor> sipa: no
256 2011-11-19 19:44:44 <sipa> with N bits, not with log2(N) bits
257 2011-11-19 19:44:52 <roconnor> 8n*(OP_DUP OP_ADD) increases the size of an element by n bytes
258 2011-11-19 19:45:09 <sipa> ye
259 2011-11-19 19:45:10 <sipa> s
260 2011-11-19 19:45:25 <roconnor> I guess that is still linear in the script size
261 2011-11-19 19:45:34 <roconnor> so the memory use appears to be linear in the script size
262 2011-11-19 19:45:57 <sipa> and the number of stack elements is limited to 1000
263 2011-11-19 19:46:51 <sipa> a 10000k script can at most do 625 times 8n(OP_DUP OP_ADD), so add 625 bytes to the size of the largest element
264 2011-11-19 19:46:57 <sipa> eh, a 10k script
265 2011-11-19 19:49:46 <sipa> push 520 bytes, 529x (OP_DUP OP_ADD), 1000x OP_DUP
266 2011-11-19 19:50:03 <sipa> -> 1000 elements of 1049 bytes on stack
267 2011-11-19 19:50:33 <sipa> 529*8 times, i mean
268 2011-11-19 19:50:47 <sipa> beat me
269 2011-11-19 19:51:08 <roconnor> sipa: do we have room for an OP_TRUE at the end so we can embed this in an accepted transaction?
270 2011-11-19 19:51:18 <sipa> just don't push zero
271 2011-11-19 19:51:24 <roconnor> oh right
272 2011-11-19 19:51:31 <roconnor> almost every number is true
273 2011-11-19 21:36:50 <roconnor> waik, how does Eligius pay out from the coinbase immediately if coinbase funds cannot be used for 100 blocks?
274 2011-11-19 21:36:52 <roconnor> *wait
275 2011-11-19 21:37:15 <cjdelisle> the coinbase contains your address
276 2011-11-19 21:37:27 <cjdelisle> the same way as solo mining pays out to the address of the miner
277 2011-11-19 21:37:33 <roconnor> oh
278 2011-11-19 21:37:40 <roconnor> good point
279 2011-11-19 21:37:58 <cjdelisle> One of those things that sounds wrong till you relize it..
280 2011-11-19 21:38:02 <roconnor> heh
281 2011-11-19 21:38:12 <roconnor> cjdelisle: thanks for making me feel better
282 2011-11-19 21:38:53 <cjdelisle> how's your script validator going?
283 2011-11-19 21:39:35 <roconnor> cjdelisle: I need to implement OP_IF et al., properly implement OP_CODESEPARATOR, and implement OP_SHA1
284 2011-11-19 21:39:44 <roconnor> also I need to implement orphan blocks
285 2011-11-19 21:39:52 <cjdelisle> you have CHECKSIG?
286 2011-11-19 21:39:56 <roconnor> oh yes
287 2011-11-19 21:39:58 <cjdelisle> and CHECKMULTISIG?
288 2011-11-19 21:40:01 <roconnor> yes
289 2011-11-19 21:40:03 <cjdelisle> nice
290 2011-11-19 21:40:21 <cjdelisle> then you're on the home stretch
291 2011-11-19 21:40:38 <roconnor> oh then I need to fix the million of tiny incompatabilities
292 2011-11-19 21:40:46 <cjdelisle> heh
293 2011-11-19 21:41:14 <cjdelisle> Something I was sort of thinking would be fun is to rip the satoshi script validator out of bitcoin and make an executable
294 2011-11-19 21:41:27 <roconnor> :)
295 2011-11-19 21:41:35 <cjdelisle> not really useful but you could play with it
296 2011-11-19 21:41:58 <roconnor> I have a custom build of bitcoin that puts random numbers for version numbers.
297 2011-11-19 21:42:07 <cjdelisle> echo 'hsuahlkjkr83IFHKJSsfhl' | openssl enc -d -base64 | validate
298 2011-11-19 21:42:13 <cjdelisle> hehe
299 2011-11-19 21:42:31 <cjdelisle> what does the default impl do?
300 2011-11-19 21:42:38 <cjdelisle> when it sees your weird numbers
301 2011-11-19 21:42:44 <roconnor> ignores them
302 2011-11-19 21:43:26 <cjdelisle> Actually something that would benefit bitcoin IMO is breaking it up into libraries
303 2011-11-19 21:43:49 <cjdelisle> so you could call libsatoshi_validateScript()
304 2011-11-19 21:43:50 <roconnor> heh, my impelemention has a lot more modules :)
305 2011-11-19 21:44:22 <cjdelisle> Also easy reuse of uint256 and bignum would be cool
306 2011-11-19 21:44:48 <roconnor> bignum is just a wrapper around OpenSSL's BIGNUM
307 2011-11-19 21:45:23 <cjdelisle> Yea but the operator overloading makes it candy to use.
308 2011-11-19 21:45:44 <roconnor> true
309 2011-11-19 21:46:12 <roconnor> though there must be much better implementations out ther
310 2011-11-19 21:46:17 <roconnor> that use GMP
311 2011-11-19 21:46:29 <cjdelisle> GMP?
312 2011-11-19 21:46:43 <roconnor> http://gmplib.org/
313 2011-11-19 21:46:56 <roconnor> GMP is a free library for arbitrary precision arithmetic, operating on signed integers, rational numbers, and floating point numbers. There is no practical limit to the precision except the ones implied by the available memory in the machine GMP runs on. GMP has a rich set of functions, and the functions have a regular interface.
314 2011-11-19 21:47:10 <cjdelisle> nice
315 2011-11-19 21:47:18 <cjdelisle> hopefully not as horrible as openssl
316 2011-11-19 21:47:27 <roconnor> I don't think so
317 2011-11-19 21:47:30 <roconnor> Haskell uses GMP
318 2011-11-19 21:47:35 <cjdelisle> although I looked at parts of openssl and it was okish
319 2011-11-19 21:47:37 <roconnor> rather GHC uses GMP
320 2011-11-19 21:47:56 <Diablo-D3> gmp is awesome
321 2011-11-19 21:48:06 <Diablo-D3> slow as fuck though
322 2011-11-19 21:48:12 <cjdelisle> ahh yes
323 2011-11-19 21:48:24 <cjdelisle> openssl people probably threw a ton of optimizations at it
324 2011-11-19 21:48:43 <Diablo-D3> well, its a software fp impl that allows unlimited precision
325 2011-11-19 21:48:49 <Diablo-D3> its going to be slow
326 2011-11-19 21:49:11 <roconnor> Diablo-D3: what about the GMP integer implementation?
327 2011-11-19 21:49:22 <cjdelisle> fp and integer are the same except for the extra number
328 2011-11-19 21:49:22 <Diablo-D3> bigint?
329 2011-11-19 21:49:27 <Diablo-D3> almost the same thing
330 2011-11-19 21:50:22 <copumpkin> Diablo-D3: what's faster than it?
331 2011-11-19 21:50:31 <copumpkin> and nobody uses GMP's multiprecision floating-point numbers
332 2011-11-19 21:50:48 <copumpkin> they were effectively moved to MPFR
333 2011-11-19 21:51:04 <copumpkin> isn't openssl trying to be slower?
334 2011-11-19 21:51:15 <copumpkin> to avoid timing information leaking from big integer operations?
335 2011-11-19 21:51:30 <copumpkin> and GMP is fairly active and mainstream
336 2011-11-19 21:51:56 <cjdelisle> fairly active and mainstream != universal
337 2011-11-19 21:51:58 <copumpkin> http://lists.virus.org/cryptography-0601/msg00002.html
338 2011-11-19 21:52:04 <cjdelisle> openssl is the way things work
339 2011-11-19 21:52:10 <copumpkin> my anecdotal evidence wins!
340 2011-11-19 21:52:34 <cjdelisle> cool
341 2011-11-19 21:52:34 <copumpkin> http://gmplib.org/list-archives/gmp-discuss/2005-March/001575.html moar
342 2011-11-19 21:52:51 <copumpkin> but as it says, for timing-sensitive operations, you don't want speed
343 2011-11-19 21:52:58 <copumpkin> you want constant-time arithmetic
344 2011-11-19 21:53:08 <copumpkin> or rather, no-shortcut arithmetic :)
345 2011-11-19 21:53:34 <cjdelisle> well that's RSA
346 2011-11-19 21:54:06 <cjdelisle> A very good argument can be made for constant time RSA ops
347 2011-11-19 21:54:14 <copumpkin> a huge amount of effort goes into a crapload of architecture-specific assembly for the tight inner loops in gmp
348 2011-11-19 21:54:35 <cjdelisle> it would be wastefull to foist that upon plain multiplication or whatnot
349 2011-11-19 21:54:39 <copumpkin> cjdelisle: true
350 2011-11-19 21:54:52 <copumpkin> it'd be nice for gmp to have no-shortcut versions of all its operations
351 2011-11-19 21:55:02 <copumpkin> but it's never really sold itself as a cryptographic library
352 2011-11-19 21:55:48 <cjdelisle> hmm you would need extra crap for doing point multiplication
353 2011-11-19 21:56:28 <copumpkin> roconnor: you're using GHC's GMP Integer for your implementation, right?
354 2011-11-19 21:58:47 <roconnor> copumpkin: when compiled with GHC I am
355 2011-11-19 21:59:19 <roconnor> copumpkin: but verification, by its nature, doesn't need to worry about leaking secrets
356 2011-11-19 21:59:27 <copumpkin> yeah
357 2011-11-19 21:59:30 <roconnor> copumpkin: I have code to generate signatures, but it isn't exported
358 2011-11-19 22:00:00 <roconnor> (it is also called unsafeMkSignature since it doesn't enforce that a random nonce is used)
359 2011-11-19 22:00:14 <roconnor> (I guess I should warn about timing attacks)
360 2011-11-19 22:01:11 <roconnor> maybe OpenSSL bindings should be used to create signatures.
361 2011-11-19 22:01:21 <roconnor> when/if I get to that point
362 2011-11-19 22:01:46 <roconnor> and frankly if people are timing my processes on my laptop; I have much bigger problems.
363 2011-11-19 22:02:13 <copumpkin> yep, but you could imagine people timing stuff remotely, if you provide any sort of remote interface to it
364 2011-11-19 22:02:32 <copumpkin> (and unrestricted access to operations that use your private key)
365 2011-11-19 22:10:10 <cjdelisle> Major changes between OpenSSL 0.9.7g and OpenSSL 0.9.8:
366 2011-11-19 22:10:15 <cjdelisle> New engine module: GMP (performs private key exponentiation).
367 2011-11-19 22:17:59 <roconnor> copumpkin: I've put in motion my nefarious plan to fork the blockchain
368 2011-11-19 22:18:37 <copumpkin> oh?
369 2011-11-19 22:18:49 <cjdelisle> what do you need a fork for?
370 2011-11-19 22:19:31 <roconnor> cjdelisle: to either destroy bitcoin or make it stronger
371 2011-11-19 22:19:53 <cjdelisle> oh, you're attacking to see what happens?
372 2011-11-19 22:19:56 <cjdelisle> sounds like fun
373 2011-11-19 22:20:00 <roconnor> :)
374 2011-11-19 22:20:08 <cjdelisle> how does it work?
375 2011-11-19 22:20:29 <roconnor> My weapon is a blog post: http://r6.ca/blog/20111119T211504Z.html
376 2011-11-19 22:21:01 <roconnor> it is designed to entice a change in the way that OpenSSL decodes ECDSA signatures.
377 2011-11-19 22:21:30 <roconnor> thus bitcoin clients linking to different versions of openssl will implement different bitcoin rules
378 2011-11-19 22:21:40 <roconnor> making the conditions ripe for a fork
379 2011-11-19 22:22:02 <roconnor> all without touching the bitcoin source files
380 2011-11-19 22:22:05 <cjdelisle> hehe
381 2011-11-19 22:22:10 <cjdelisle> not likely
382 2011-11-19 22:22:21 <cjdelisle> more likely the openssl people will fix it like this:
383 2011-11-19 22:22:48 <cjdelisle> /* We could fix this but it would mean the new version would be different than the old version so we won't. Problem? */
384 2011-11-19 22:23:08 <cjdelisle> same as their DER encoding goofieness
385 2011-11-19 22:23:11 <roconnor> right, that is why I need someone to find an exploit; preferably several
386 2011-11-19 22:23:57 <cjdelisle> which will no doubt be met with a dirty if statement
387 2011-11-19 22:24:41 <cjdelisle> if (this_looks_like_johns_exploit()) { oh_no_you_dont(); }
388 2011-11-19 22:25:01 <roconnor> ya, the solution, in the unlikely event that openssl changes would be that bitcoin would massage the signatures before sending them off to openssl to maintain compatibility.
389 2011-11-19 22:25:53 <roconnor> bitcoin could even implement this today, just to be safe.
390 2011-11-19 22:27:33 <cjdelisle> have you crafted a tx which should not validate?
391 2011-11-19 22:28:07 <roconnor> cjdelisle: not, me; I found it in testnet
392 2011-11-19 22:29:06 <cjdelisle> oh, ic
393 2011-11-19 22:30:36 <cjdelisle> it sounds a bit like the way xulrunner parses xml
394 2011-11-19 22:30:44 <cjdelisle> it tries to guess what you meant...
395 2011-11-19 22:32:28 <roconnor> cjdelisle: this is the transaction in question: http://blockexplorer.com/testnet/tx/95038c3155de45fc7753f90b35c04b494ff1379e665dbbd9d013496a2531b7a7
396 2011-11-19 22:32:35 <cjdelisle> hmm if bignum uses this, all of satoshi's work on negative numbers is for naut.
397 2011-11-19 22:32:54 <roconnor> cjdelisle:  nope, it is specifically the DER decoding of openssl
398 2011-11-19 22:33:21 <roconnor> bignum supports negative numbers AFAIK
399 2011-11-19 22:33:25 <roconnor> (I haven't tested it)
400 2011-11-19 22:33:31 <cjdelisle> yes, it has negative stuff in it
401 2011-11-19 22:33:53 <roconnor> basically the openssl developers are very very lazy
402 2011-11-19 22:34:57 <cjdelisle> that and very intent on keeping up their APIs
403 2011-11-19 22:35:12 <cjdelisle> broken apis suck
404 2011-11-19 22:35:17 <cjdelisle> insecure apis are worse
405 2011-11-19 22:35:53 <cjdelisle> like read_file8() /* We'll get it right this time. */
406 2011-11-19 22:36:32 <roconnor> cjdelisle: is that real?
407 2011-11-19 22:36:49 <cjdelisle> that's an example but microsoft is full of stuff like that
408 2011-11-19 22:36:53 <roconnor> :)
409 2011-11-19 22:37:07 <cjdelisle> You have an api which can be used to root the box:
410 2011-11-19 22:37:07 <roconnor> more like readFileExExExExExExExEx
411 2011-11-19 22:37:22 <cjdelisle> 1. remove it and break thousands of programs and make people really mad
412 2011-11-19 22:37:45 <cjdelisle> 2. add some detection to see if the user is likely to get root by using it and stop them
413 2011-11-19 22:37:58 <cjdelisle> everyone always chooses 2
414 2011-11-19 23:10:22 <CIA-89> bitcoin: Luke Dashjr * r5549a7bcbd81 gentoo/net-p2p/bitcoind/ (5 files): net-p2p/bitcoind: bump to rc6
415 2011-11-19 23:10:24 <CIA-89> bitcoin: Luke Dashjr * r070bde62287f gentoo/net-p2p/bitcoind/ (Manifest bitcoind-0.4.0.ebuild): net-p2p/bitcoind: remove 0.4.0 due to security issue
416 2011-11-19 23:15:00 <roconnor> I guess it is a good thing that I haven't pushed 0.4.0 into nix yet
417 2011-11-19 23:15:45 <cjdelisle> what was the issue?
418 2011-11-19 23:16:40 <roconnor> wallet encryption leaks ... something
419 2011-11-19 23:16:46 <cjdelisle> oh right, that
420 2011-11-19 23:16:54 <cjdelisle> I derp'd it
421 2011-11-19 23:17:05 <cjdelisle> Not much of an issue to me
422 2011-11-19 23:18:30 <roconnor> ya, but when you are putting something in a distro, you have to tell everyone when things go wrong :(
423 2011-11-19 23:19:25 <cjdelisle> yea, I just saw it and was like "wha?" because I use the bleeding edge git pull ofc
424 2011-11-19 23:20:32 <CIA-89> bitcoin: Luke Dashjr * r1a69b568f374 gentoo/net-p2p/wxbitcoin/ (Manifest wxbitcoin-0.4.1_rc6.ebuild): net-p2p/wxbitcoin: bump to 0.4.1_rc6
425 2011-11-19 23:20:33 <CIA-89> bitcoin: Luke Dashjr * rf1bf3bdb0a3b gentoo/net-p2p/wxbitcoin/ (Manifest wxbitcoin-0.4.0.ebuild): net-p2p/wxbitcoin: remove 0.4.0 due to security issue
426 2011-11-19 23:20:41 <CIA-89> bitcoin: Luke Dashjr * rc1894aed2811 gentoo/net-p2p/bitcoin-qt/ (3 files): net-p2p/bitcoin-qt: bump to rc6
427 2011-11-19 23:20:43 <CIA-89> bitcoin: Luke Dashjr * r4ab39e0820b6 gentoo/net-p2p/wxbitcoin/wxbitcoin-0.4.1_rc5.ebuild: net-p2p/wxbitcoin: forgot to remove rc5
428 2011-11-19 23:20:43 <luke-jr> roconnor: nix?
429 2011-11-19 23:20:58 <roconnor> luke-jr: http://nixos.org/
430 2011-11-19 23:21:07 <roconnor> ``Nix is a purely functional package manager. This means that it can ensure that an upgrade to one package cannot break others, that you can always roll back to previous version, that multiple versions of a package can coexist on the same system, and much more.
431 2011-11-19 23:21:35 <luke-jr> sounds similar to what I had planned for Utopios
432 2011-11-19 23:21:50 <roconnor> Nix has been my distro on my laptop for the last 2 years
433 2011-11-19 23:22:18 <roconnor> I guess I'm the maintainer for the bitcoin package
434 2011-11-19 23:22:25 <luke-jr> Moving to GitHub
435 2011-11-19 23:22:45 <roconnor> still, it is an improvement over svn
436 2011-11-19 23:22:51 <luke-jr> in some areas.
437 2011-11-19 23:48:37 <roconnor> ``The protocol will never support transactions that change/expire automatically after some period of time, as this causes problems.
438 2011-11-19 23:48:46 <roconnor> http://bitcoin.stackexchange.com/questions/1986/contracts-will-this-be-possible
439 2011-11-19 23:48:52 <roconnor> this isn't entirely true
440 2011-11-19 23:49:46 <roconnor> bitcoin supports transactions time locked transactions that can be placed into the block chain unless superceeded
441 2011-11-19 23:50:17 <CIA-89> bitcoin: Luke Dashjr * r10affcd3bbd2 gentoo/net-p2p/namecoin/ (Manifest namecoin-0.3.24.64.ebuild namecoin-9999.ebuild): net-p2p/namecoin: still needs crypto++