1 2014-12-10 00:25:03 <dgenr8> tdlfbx: a lognormal distribution has the right properties and models the 90th percentiles pretty well when plugging in the different medians, but that's only one point prediction, it will be nice when cdecker shares more data
  2 2014-12-10 00:55:18 <rusty> BlueMatt: Do you have any data on txs missing/received for analyzing iblt performance?  Kalle Rosenbaum and I have been toying with variations, but I'm not confident of the results without a real corpus.
  3 2014-12-10 00:56:06 <gmaxwell> rusty: I have logs.
  4 2014-12-10 00:56:17 <BlueMatt> rusty: best I can do is dig out logs for real block size and block size - txn already seen
  5 2014-12-10 00:56:19 <BlueMatt> oh, ok
  6 2014-12-10 00:58:44 <gmaxwell> BlueMatt: hm your daemons' current logs are not so useful.
  7 2014-12-10 00:59:26 <BlueMatt> yes
  8 2014-12-10 01:00:05 <BlueMatt> rusty: if you want to help with that instead of running your own measurement, feel free to do so at https://github.com/TheBlueMatt/RelayNode/tree/master/c%2B%2B :)
  9 2014-12-10 01:03:27 <gmaxwell> rusty: alas, I can only tell you the bytes saved.. matt's stuff used to log number of transactions.
 10 2014-12-10 01:03:48 <gmaxwell> e.g.
 11 2014-12-10 01:03:49 <gmaxwell> [2014-12-09 16:18:54+00] 00000000000000001b751799ade96a60a02ffb05d142708c636c0029ec28aa8f recv'd, size 966417 with 7177 bytes on the wire
 12 2014-12-10 01:03:52 <gmaxwell> [2014-12-09 16:35:19+00] 000000000000000003069f8412357d649a5a7554b2f32142c047e3ca4be1bb01 recv'd, size 910093 with 2538 bytes on the wire
 13 2014-12-10 01:03:55 <gmaxwell> [2014-12-09 16:46:50+00] 00000000000000000fddea3a11c95f2a113b60cadf9f401eadeacba1d8fe0403 recv'd, size 533651 with 1904 bytes on the wire
 14 2014-12-10 01:03:58 <gmaxwell> [2014-12-09 16:49:53+00] 00000000000000000f6dd29f4be1003751bdd2172b26325ab2fa1da28e7b788b recv'd, size 128657 with 611 bytes on the wire
 15 2014-12-10 01:04:04 <rusty> gmaxwell: ah, I really want the actual txids, since it matters for this.
 16 2014-12-10 01:04:27 <rusty> BlueMatt: OK, what's your preference, a new --log-txids flag?
 17 2014-12-10 01:04:28 <gmaxwell> yea, it could be made to log this easily enough.
 18 2014-12-10 01:04:47 <BlueMatt> rusty: or just print to stderr and not out and let people redirect at will
 19 2014-12-10 01:04:48 <BlueMatt> I'm not picky
 20 2014-12-10 01:05:39 <rusty> gmaxwell: my aim is to produce a corpus of txs to put up on github.  Kalle and I are getting different results (I think I know why) but it'd be nice to have real data than our separate randomly-chosen txs.
 21 2014-12-10 01:05:46 <gmaxwell> in any case, if someone adds logs I'll run them and I expect a few other folks would be glad to as well.
 22 2014-12-10 01:06:12 <rusty> gmaxwell: OK, I'll start hacking now.  Thanks BlueMatt...
 23 2014-12-10 01:24:28 <rusty> BlueMatt: hmm, does this relay code chew 100% cpu?  It looks like it sets the socket to nonblocking, then spins reading...
 24 2014-12-10 01:24:54 <BlueMatt> rusty: sockets are blocking
 25 2014-12-10 01:25:05 <BlueMatt> and overly-threaded
 26 2014-12-10 01:25:13 <BlueMatt> no one ever said it was good code
 27 2014-12-10 01:27:59 <rusty> BlueMatt: ah, do_connect() is setting nonblock on some other socket, maybe.  Weird.
 28 2014-12-10 01:28:26 <rusty> BlueMatt: and I'm guessing I'm taking the shortest path by reading the code to figure out the protocol?  Seems pretty simple, though.
 29 2014-12-10 01:30:44 <uiop> separation of abstract from concrete is the last limit of experience and the last effort of genius
 30 2014-12-10 01:30:54 <uiop> (while still achieving both)
 31 2014-12-10 01:58:35 <BlueMatt> rusty: yea, pretty much
 32 2014-12-10 01:58:39 <BlueMatt> its dead-simple
 33 2014-12-10 02:01:26 <sipa> it's really tempting to go implement something similar as optional protocol extension in bitcoin core, given the amount of savings
 34 2014-12-10 02:05:35 <sipa> we already have significant per-peer memory usage... just say keep the last X mb of sent tx messages, in the order they are sent, around, and allow the peer to refer to entries in it inside block messages
 35 2014-12-10 02:05:42 <sipa> it seems trivial
 36 2014-12-10 02:06:49 <sipa> (which means you do not get the round trip savings that the hub topology offers, but you definitely reduce the bandwidth spikes)
 37 2014-12-10 02:09:44 <gmaxwell> could also use my my permutation-prefix IDing rather than txids. You can get some pretty impressive compression on existing transactions by doing pubkey recovery.
 38 2014-12-10 02:09:54 <gmaxwell> In any case, I'd still want that stuff to mature externally.
 39 2014-12-10 02:17:27 <Diablo-D3> gmaxwell: did conman get his problem fixed?
 40 2014-12-10 02:19:56 <BlueMatt> sipa: there is separate advantage to being a separate protocol entirely
 41 2014-12-10 02:20:02 <BlueMatt> sipa: not to say its mutually exclusive
 42 2014-12-10 02:20:28 <BlueMatt> sipa: but, eg, gfw has apparently randomly blocked some bitcoin p2p at times, so...
 43 2014-12-10 02:21:12 <sipa> fully agree
 44 2014-12-10 02:21:48 <sipa> just saying it is tempting to just use the technique in the core p2p protocol too
 45 2014-12-10 02:23:59 <sipa> thinking about that... being bidirectional complicates things, as we for example do not inv transactions we know the peer already has
 46 2014-12-10 02:25:30 <sipa> so you would want some message saying "pretend that i just relayed tx with hash X to you... i assume you already have it", which likely requires confirmation... bleh
 47 2014-12-10 02:26:26 <TheButterZone> does anyone know the dev "humanfractal" possibly real name alexandr romanov (per twitter)? their help is needed with https://bitcointalk.org/index.php?topic=882162.0
 48 2014-12-10 02:35:29 <gmaxwell> sipa: I was being a bit daft earlier, there are applications for controlling the nonce. You need to do so for a single show signature, for example.  Perhaps advanced enough that it need not be exposed until someone actually wants to use it... but earlier I wasn't coming up with anything.
 49 2014-12-10 03:35:51 <rusty> BlueMatt: wow, running the relay client was shockingly straightforward.  Nice job!
 50 2014-12-10 03:36:58 <BlueMatt> rusty: awesome, thanks
 51 2014-12-10 04:57:54 <rusty> BlueMatt: Hmm, dumb q.  I'm dumping recv_tx_cache after a block has been received, but am seeing txs which were (eg.) included in a block 11 blocks ago.
 52 2014-12-10 05:01:45 <rusty> BlueMatt: hmm, that may be a startup problem...
 53 2014-12-10 05:30:08 <BlueMatt> rusty: I imagine finding bugs in this shit /should/ be easy...its never been particularly stable, though its worked well enough to get by
 54 2014-12-10 05:32:38 <algesten> System Info: Model: MacBook Pro (15-inch Retina Late 2013) · CPU Speed: 4 @ 2.50 GHz · RAM: 16 GB · Diskspace: 262,38 GB/464,8 GB · GPU: Intel Iris Pro · Screen Resolution: 1440x900 · OS X: Version 10.10.1 (Build 14B25) · Uptime: 7 days · Load Averages: 1.58 1.69 1.58 · Client: Lingo 2.0.4/47734
 55 2014-12-10 05:32:45 <algesten> hm. sorry
 56 2014-12-10 05:36:41 <paveljanik> gen=1? 8)
 57 2014-12-10 05:56:07 <rusty> BlueMatt: why do I get a crapload of txs from the server whe n I connect?  Does it send everything since last block?
 58 2014-12-10 05:56:28 <BlueMatt> rusty: the server doesnt keep per-client state, so it has to sync you up to the latest txn set when you connect
 59 2014-12-10 05:56:28 <rusty> BlueMatt: (BTW my previous problem may be related to the fact that my bitcoind was way behind)
 60 2014-12-10 05:56:35 <BlueMatt> otherwise it would have to compress each block for each user
 61 2014-12-10 05:56:41 <BlueMatt> and its latency-limited, so it avoids that
 62 2014-12-10 05:56:52 <rusty> BlueMatt: right, so that
 63 2014-12-10 05:57:01 <rusty> 's basically the server mempool dump.
 64 2014-12-10 05:58:11 <BlueMatt> yea
 65 2014-12-10 06:25:04 <rusty> BlueMatt: yes, it definitely seems to have some already-included transactions.  Over time they'll fall out of the recv_tx_cache I guess.  Will add more logging to try to get to the bottom of it.
 66 2014-12-10 06:26:10 <rusty> BlueMatt: Am I understanding correctly, that recv_tx_cache would be expected to be almost empty after a block?
 67 2014-12-10 06:26:19 <BlueMatt> rusty: that seems strange...it is supposed to remove them when a block hits, though its entirely possible they somehow get re-added afterwards
 68 2014-12-10 06:26:37 <BlueMatt> rusty: it should remove ones in the block itself, which should be far from empty (probably, I'd bet)
 69 2014-12-10 06:27:11 <rusty> BlueMatt: it does remove them... I'm still seeing lots of confirmed txs in there though.  More debugging reqd...
 70 2014-12-10 06:27:49 <BlueMatt> probably my bitcoind is fucked up
 71 2014-12-10 06:33:13 <rusty> BlueMatt: the initial dump from server looks sane...
 72 2014-12-10 06:33:26 <BlueMatt> really? thats...strange
 73 2014-12-10 06:33:42 <BlueMatt> oh, does recv_tx_cache actually remove them? or just lazily do something?
 74 2014-12-10 06:33:46 <BlueMatt> I dont at all recall, its been a while
 75 2014-12-10 06:35:39 <rusty> BlueMatt: I lie... your server sent 45f66c6b994d320b425c856291a39230369476233d826ad820136c950ec3e3ea, which is already 8 deep
 76 2014-12-10 06:36:35 <rusty> BlueMatt: it seems to actually delete them, AFAICT.
 77 2014-12-10 06:38:08 <BlueMatt> that sounds strange...I'd think this is a bug in the bitcoind-handling-logic, then
 78 2014-12-10 06:38:19 <BlueMatt> it relies on a central bitcoind to filter/anti-dos transactions
 79 2014-12-10 06:38:27 <BlueMatt> since those arent so latency-sensitive
 80 2014-12-10 06:38:36 <rusty> BlueMatt, anyway, you're sending about 5 txs with 8 confirms, 20 with 7, and another bunch with 3 (maybe 60?)
 81 2014-12-10 06:38:44 <BlueMatt> fuck, thats damn high
 82 2014-12-10 06:38:55 <BlueMatt> I'm digging into other bugs now, but I'll see if I can find that today
 83 2014-12-10 06:39:22 <rusty> BlueMatt: OK, I'll push my tree with debugging up to github if you're feeling lazy.  My logging stuff seems to work, thougj.
 84 2014-12-10 06:41:45 <rusty> BlueMatt: https://github.com/rustyrussell/RelayNode FYI
 85 2014-12-10 06:42:53 <rusty> BlueMatt: and I was using "./relaynetworkclient --log-block-txids public.au.relay.mattcorallo.com localhost 8333 | tee log" and "for f in `grep '^Received tx ' log | cut -d\  -f3-`; do echo $f; bitcoind getrawtransaction $f 1 | grep confirm; done" to see the txstatus (bitcoind is running with -txindex, so it has the info)
 86 2014-12-10 06:44:00 <BlueMatt> yes, I'm also looking into why connections are dying without the code picking up on it - the au node being worse for that due to latency and packet loss
 87 2014-12-10 06:44:14 <BlueMatt> though I think packet loss to the new-york-based bitcoind from singapore is actually the worst
 88 2014-12-10 06:45:31 <BlueMatt> rusty: do you get different results in the us-west (just restarted for unrelated reasons) or us-east?
 89 2014-12-10 06:51:21 <anton000> hey guys
 90 2014-12-10 06:51:29 <anton000> bitoind on centos suddenyl crashed
 91 2014-12-10 06:51:32 <anton000> LevelDB read failure: Invalid argument: not an sstable (bad magic number)
 92 2014-12-10 06:51:36 <anton000> got this from th log
 93 2014-12-10 07:01:40 <paveljanik> anton000: what OS?
 94 2014-12-10 07:01:41 <paveljanik> https://github.com/bitcoin/bitcoin/issues/4362
 95 2014-12-10 07:02:40 <gmaxwell> anton000: any chance you ran out of disk space?
 96 2014-12-10 07:03:31 <paveljanik> how the hexdump -C look on the file?
 97 2014-12-10 07:07:00 <paveljanik> ah linux
 98 2014-12-10 07:09:06 <paveljanik> anton000: ?
 99 2014-12-10 07:16:21 <anton000> still 5gb avail
100 2014-12-10 07:16:27 <anton000> mustve run into bad mem
101 2014-12-10 07:16:34 <anton000> checking
102 2014-12-10 07:23:03 <BlueMatt> gmaxwell: I have recently noted that our out-of-diskspace safe-warn-shutdown stuff is not working properly
103 2014-12-10 07:23:17 <b_lumenkraft> sipa: http://bitcoin.sipa.be/speed-lin-2k.png is your site, right?
104 2014-12-10 07:23:25 <BlueMatt> b_lumenkraft: yes
105 2014-12-10 07:23:48 <gmaxwell> BlueMatt: oh, really?
106 2014-12-10 07:24:10 <b_lumenkraft> looks like 400 PH is no more enough as the max… :(
107 2014-12-10 07:24:25 <BlueMatt> gmaxwell: IIRC I saw it once and was like "should fix", but forgot until just now
108 2014-12-10 07:25:17 <BlueMatt> gmaxwell: got a leveldb error or something instead of "OUT OF DISK SPACE"
109 2014-12-10 07:26:58 <gmaxwell> IIRC we cannot actually guarentee that it will work, esp since disk usage can change out from under us (I don't know if there is a way to make leveldb prealloc so it will never run out by surprise)
110 2014-12-10 07:27:23 <BlueMatt> yes, but it used to successfully warn if bitcoin is the thing that runs you out of disk space
111 2014-12-10 07:28:20 <gmaxwell> oh you're saying its totally busted.
112 2014-12-10 07:28:28 <BlueMatt> well, seemed to be
113 2014-12-10 07:28:32 <phantomcircuit> BlueMatt, leveldb uses like 2x disk space at random
114 2014-12-10 07:28:45 <BlueMatt> again, I just had a node crash and was like "whaaa", and then promptly forgot to debug until now
115 2014-12-10 07:29:01 <BlueMatt> phantomcircuit: great, so we should warn and fail if you arent capable of reaching that
116 2014-12-10 07:29:25 <gmaxwell> BlueMatt: we should have some commandline todo program. too many times I've encountered something weird and forgotten about it before I could look into it more. :(
117 2014-12-10 07:29:37 <gmaxwell> A number of bugs I've seen reported and went "oh yea, I'd seen that too." :(
118 2014-12-10 07:29:51 <BlueMatt> gmaxwell: I had a todo tracker txt file with this crap....and then forgot to read it for weeks and it quickly became useless
119 2014-12-10 07:30:06 <BlueMatt> I suppose I have the screen real-estate to spare for these kinds of things...hmmm
120 2014-12-10 07:38:40 <BlueMatt> wowwwwwwwwww GITHU CODE HIGHLIGHTING????????
121 2014-12-10 07:38:46 <BlueMatt> when did this happen?
122 2014-12-10 07:40:01 <gmaxwell> minutes ago, I guess. I reloaded and it appeared.
123 2014-12-10 07:40:25 <BlueMatt> its...beutiful...*sob*
124 2014-12-10 08:10:25 <antanst> has anyone here managed to successfully sync Insight.js on testnet?
125 2014-12-10 08:10:57 <antanst> it always bails out with the "Block differs by <...>, delete and resync" error message
126 2014-12-10 08:20:34 <jonasschnelli> antanst i assume Insight.js is off-topic here. Did you you try to create a issue at https://github.com/bitpay/insight/?
127 2014-12-10 08:21:45 <antanst> yes. Apologies for the off-topic message.
128 2014-12-10 08:49:20 <midnightmagic> hey, found a bug in the gitian builder for LXC builds for those of us running ubuntu 12.04.x (or with multipath-tools 0.4.9 installed, I haven't tested multipath-tools 0.5.0)
129 2014-12-10 08:53:50 <midnightmagic> there *appears* to be a bug in the gitian builder that's been haunting me for maybe a week. in gitian-builder/bin/make-base-vm kpartx is used to configure a loopback device which allows us to copy a single raw partition into a build image for use with LXC. During cleanup, kpartx -d /dev/loop${loop} is used, but in multipath-tools 0.4.9 (which is on ubuntu 12.04.x) kpartx fails when using the /dev/loop${loop} syntax, it has to be
130 2014-12-10 08:53:56 <midnightmagic> given the file-side mapping. losetup -d /dev/loop${loop} will work, as will kpartx -d /path/to/image.raw. test results:  http://0bin.net/paste/eNQLYPhnc2199yAu#uEwKw4ZhiUQzMbwtfcLcIscndEC1bhouINvRGX6kuRb This means that multiple build runs will eventually fill loop devices, holding open old images (and thus eating disk space) and in particular *old builds will still hang around and potentially interfere with future builds because
131 2014-12-10 08:54:02 <midnightmagic> it's possible for on-target to execute against the old, failed-to-delete mapping*
132 2014-12-10 08:59:57 <iwilcox> Nice catch.
133 2014-12-10 09:17:07 <midnightmagic> iwilcox: hopefully i'm not crazy
134 2014-12-10 11:11:08 <huma> i get this error repeatedly in my debug.log when starting bitcoin-qt client
135 2014-12-10 11:11:16 <huma> ERROR: CheckBlock() : hashMerkleRoot mismatch
136 2014-12-10 11:11:16 <huma> ERROR: ConnectTip() : ConnectBlock 000000000000000003cd9048dfba61889e4305832094c6c7683e090d6032c4ad failed
137 2014-12-10 11:11:35 <huma> any tips on how to solve this?
138 2014-12-10 11:15:39 <sipa> huma: seems that block on disk is corrupted
139 2014-12-10 11:16:07 <sipa> starting with -reindex will help, but will take a while (it needs to reprocess all old blocks)
140 2014-12-10 11:18:11 <huma> sipa: thank you. i guess it's still faster than downloading the whole chain anew. could it be due to disk issues as in bad blocks?
141 2014-12-10 11:18:28 <sipa> yes
142 2014-12-10 11:20:46 <huma> sipa: ok, i'll move the datadir to a different disk then run -reindex. thank you.
143 2014-12-10 11:35:28 <huma> sipa: is it an unrecoverable error? should it not be fatal and stop execution?
144 2014-12-10 11:36:09 <sipa> huma: yes, what version are you running?
145 2014-12-10 11:37:05 <huma> latest release 0.9.3
146 2014-12-10 11:37:58 <sipa> that code changed a lot in master (which will become 0.10), so it may be fixed
147 2014-12-10 11:38:06 <sipa> but no, it shouldn't repeatedly try the same thing
148 2014-12-10 11:38:38 <huma> my debug.log grew to 400 mb :)
149 2014-12-10 11:47:23 <huma> sipa: oh. i copied the datadir and ran -reindex on it. in debug.log it seems to have started reindexing, but very early on crashed with exception code c0000005 (access violation). these are the last 2 lines in the log:
150 2014-12-10 11:47:33 <huma> ProcessBlock: ACCEPTED
151 2014-12-10 11:47:33 <huma> UpdateTip: new best=00000000dbe99fe90915218f74be250fffd6140041b1e8cf139543ae88af0795  height=2378  log2_work=43.216162  tx=2410  date=2009-01-30 17:46:56 progress=0.000022
152 2014-12-10 11:47:56 <sipa> i would suggest running memtest on your system :)
153 2014-12-10 11:48:55 <huma> hmm
154 2014-12-10 11:51:56 <huma> sipa: maybe i should try the -cli version
155 2014-12-10 11:52:42 <sipa> bitcoin-cli is an RPC client
156 2014-12-10 11:52:58 <sipa> and bitcoind and bitcoin-qt will behave the same way
157 2014-12-10 11:53:11 <huma> ah
158 2014-12-10 11:57:19 <huma> sipa: do i need to get rid of any leftover files maybe (from the last run on the "bad" disk)? could it be an issue?
159 2014-12-10 11:57:42 <huma> as it didn't shut down properly
160 2014-12-10 11:59:39 <huma> sipa: ah, there. it's started. i might really need to check that memory, i guess. thank you.
161 2014-12-10 12:37:08 <huma> sipa: it's going quite slow. still at 2012. a bit surprising given it's already on disk.
162 2014-12-10 12:37:18 <sipa> increasing -dbcache can help
163 2014-12-10 12:37:27 <sipa> if you have ram
164 2014-12-10 12:47:59 <huma> sipa: will i lose the indexing progress if i kill the program now?
165 2014-12-10 12:48:04 <sipa> no
166 2014-12-10 12:51:22 <huma> sipa: ok, will stop now and try -dbcache 1024. thank you.
167 2014-12-10 12:56:01 <huma> sipa: damn. it started reindexing from scratch :(
168 2014-12-10 12:56:10 <sipa> huma: don't pass -reindex the second time
169 2014-12-10 12:56:21 <sipa> it will start over if you do; if you don't, it will continue where it left off
170 2014-12-10 12:56:28 <huma> arg, too late :/
171 2014-12-10 13:00:53 <michagogo> Hm, interesting idea for a trolly thing to do to someone with a Bitcoin Core node
172 2014-12-10 13:00:59 <michagogo> add reindex=1 to the config
173 2014-12-10 13:01:09 <huma> and it seems to be going even slower :(
174 2014-12-10 13:01:30 <huma> memory usage 98 mb
175 2014-12-10 13:02:06 <huma> cpu utilization 5%, disk 33%
176 2014-12-10 13:19:35 <huma> sipa: it does help indeed. cpu and memory utilizations are up and crunching. thank you!
177 2014-12-10 13:19:59 <huma> i suppose i could also disable debug output to speed it up a little
178 2014-12-10 13:42:50 <jonasschnelli> Am i right that the bitcoind's HTTP/RPC Server does not have a connection timeout? It looks like it uses sync/blocking getline?
179 2014-12-10 17:31:40 <RixiM> Any tips on the best way to force bitcoind to be nice on a single core VPS?
180 2014-12-10 17:35:34 <sipa> RixiM: define 'nice'
181 2014-12-10 17:35:41 <sipa> low CPU? low RAM? low bandwidth?
182 2014-12-10 17:35:42 <op_null> RixiM: you won't have much luck if you have less than 1GB of memory. single core will work, it'll just be painfully slow to sync.
183 2014-12-10 17:35:58 <sipa> RixiM: limit the number of connections (-maxconnections), reduce rpc threads (-rpcthreads=1), disable parallel verification (-par=1), reduce coin cache size (-dbcache=20), run git master if you can
184 2014-12-10 17:36:04 <sipa> ^- to limit memory usage
185 2014-12-10 17:37:01 <RixiM> I am running on aws. what size of machine is a recommended minimum? I don't want to mine, I just don't want my service to rely on another service.
186 2014-12-10 17:37:21 <op_null> super don't run a bitcoin node on AES.
187 2014-12-10 17:37:32 <RixiM> why?
188 2014-12-10 17:37:35 <op_null> AWS, rather. it's a terrible place to run one.
189 2014-12-10 17:37:46 <op_null> dog slow and very expensive.
190 2014-12-10 17:38:05 <RixiM> op_null: expensive compared to?
191 2014-12-10 17:38:15 <RixiM> rather, what would you suggest?
192 2014-12-10 17:39:40 <op_null> more traditional VPS hosts do a better job for considerably less money, even a cheap OVH instance will way out perform most AWS ones
193 2014-12-10 17:40:48 <RixiM> op_null: ok, what are the minimum requirements to run a bitcoin node?
194 2014-12-10 17:40:58 <op_null> of course if it's mission critical that's probably not the best idea, but people have found in the past that AWS bottlenecks bitcoin nodes something terrible
195 2014-12-10 17:41:51 <op_null> if you want a wallet, probably >40GB of space, 1GB of memory.
196 2014-12-10 17:42:18 <RixiM> It should, which is fine. I am not trying to create a full node, i just need to read transactions, and don't want to rely on a third party, i somewhat naively assumed this wouldn't be exceptionally difficult.
197 2014-12-10 17:42:55 <sipa> define 'read transactions'
198 2014-12-10 17:43:35 <RixiM> I am writing a service that sells encrypted timestamped random numbers.
199 2014-12-10 17:44:02 <RixiM> and then writes them decrpyted to the blockchain (somehow, haven't figured that part out yet)
200 2014-12-10 17:44:11 <op_null> please don't abuse the block chain.
201 2014-12-10 17:44:23 <op_null> (that's abusing the block chain)
202 2014-12-10 17:44:33 <RixiM> How? it's a service and a reciept.
203 2014-12-10 17:44:44 <kanzure> couldn't you just send them that information by email instead of using the bitcoin blockchain?
204 2014-12-10 17:44:53 <op_null> the block chain is for transactions, not data storage.
205 2014-12-10 17:45:10 <RixiM> i guess i don't understand the definition of transaction.
206 2014-12-10 17:45:40 <op_null> if it's not moving money around, it doesn't belong in the block chain.
207 2014-12-10 17:45:58 <RixiM> so how do you associate those transactions to actual events?
208 2014-12-10 17:46:05 <RixiM> how do people do escrow, for example?
209 2014-12-10 17:46:35 <RixiM> I can't say in the blockchain what a transaction was for?
210 2014-12-10 17:46:44 <op_null> you use addresses to associate payments. you, the service gives me a new address, I pay to that address, only I could have known that address, so you assume the payment was from me.
211 2014-12-10 17:46:48 <op_null> nope.
212 2014-12-10 17:47:08 <RixiM> ugh, that's stupid.
213 2014-12-10 17:47:09 <RixiM> sorry.
214 2014-12-10 17:47:11 <RixiM> but it is.
215 2014-12-10 17:47:22 <op_null> why?
216 2014-12-10 17:47:32 <RixiM> there is a whole secondary set of systems missing from btc.
217 2014-12-10 17:47:49 <op_null> no, you're just trying to shove your square head in a round hole.
218 2014-12-10 17:47:56 <kanzure> square problem
219 2014-12-10 17:47:58 <RixiM> the logic that you gave is sound if we assume that all the other dozens of things required to conduct business are done someplace else... securely.
220 2014-12-10 17:48:24 <op_null> huh? you just have to know what addresses are linked to which customer. that's kilobytes of data, if that.
221 2014-12-10 17:48:43 <RixiM> op_null: do you run a business?
222 2014-12-10 17:48:51 <harding> Maybe this conversation should move to #bitcoin
223 2014-12-10 17:49:08 <RixiM> harding: yeah, i am done with my dev questions :D
224 2014-12-10 18:29:56 <huma> sipa: reindexing is 5h in, still running (-dbcache=1500). 40 weeks behind. performance is down to 1 block in 1/0.5 sec. is it normal?
225 2014-12-10 21:29:44 <BlueMatt> yay travis log tab eatil >1gb of memory by itself...
226 2014-12-10 22:24:48 <huma_> i've got this error during reindexing: "LevelDB read failure: Corruption: block checksum mismatch". does it mean the actual blockchain data on disk is currupt? is it recommended to download the blockchain from scratch?
227 2014-12-10 22:25:19 <op_null> no, blocks on disk aren't stored in a leveldb, just the unspent outputs database.
228 2014-12-10 22:25:50 <op_null> a reindex agressively tries to recover from bad blocks on disk.
229 2014-12-10 22:26:27 <phantomcircuit> op_null, that specific error might indicate a corrupt block file
230 2014-12-10 22:26:30 <iwilcox> huma_: Did you memtest yet?
231 2014-12-10 22:27:02 <phantomcircuit> nope that's just a leveldb failure
232 2014-12-10 22:27:13 <phantomcircuit> which indicates a hw issue almost certainly
233 2014-12-10 22:27:29 <phantomcircuit> note leveldb does not provide strong guarantees with failing hw
234 2014-12-10 22:27:41 <phantomcircuit> so using that system would probably be ill advised
235 2014-12-10 22:27:42 <huma_> iwilcox: not yet. odd, this system has been smooth for a few years.
236 2014-12-10 22:28:59 <iwilcox> huma_: Those things can be surprisingly subtle.
237 2014-12-10 22:29:09 <huma_> eh, only 35 weeks behind left
238 2014-12-10 22:30:03 <op_null> phantomcircuit: if it's during index though, isn't that unlikely?
239 2014-12-10 22:31:00 <phantomcircuit> op_null, leveldb only correctly identifies very specific classes of hw errors
240 2014-12-10 22:31:27 <phantomcircuit> specifically afaict it only corrects mutations which do not change the record size (since that's silently ignored!)
241 2014-12-10 22:43:07 <gmaxwell> huma_: you're almost certantly suffering hardware induced data corruption.
242 2014-12-10 22:43:51 <Diablo-D3> aaaand I bet its his power supply
243 2014-12-10 22:55:34 <op_null> that could be a fun project, have thousands of computers over the world do a deterministic action and send back the results. see what sort of junk people's computers make.
244 2014-12-10 22:56:53 <phantomcircuit> op_null, i know for certain my workstation would fail
245 2014-12-10 22:57:07 <phantomcircuit> there's about 512MB of high mem which random does bit flips
246 2014-12-10 22:57:21 <phantomcircuit> i cant get the sodimm out to replace it though
247 2014-12-10 22:58:21 <op_null> phantomcircuit: bet it would make better random numbers than blockchain.info :)
248 2014-12-10 23:00:05 <phantomcircuit> op_null, possibly
249 2014-12-10 23:00:35 <op_null> phantomcircuit: if you have more than 8 bits of entropy you'd win that shoot out.
250 2014-12-10 23:00:39 <moa> cosmic rays
251 2014-12-10 23:00:50 <wyager> op_null: IIRC, Google did a study that showed that, at sea level, you expect something like .25 random bit flips per hour per gig of non-error-corrected RAM. So you just need to use a few GB*hours to have a reasonable expectation of a bit flip
252 2014-12-10 23:01:08 <op_null> wyager: woah, that high?
253 2014-12-10 23:01:11 <wyager> Yeah
254 2014-12-10 23:01:15 <wyager> even higher at high altitudes
255 2014-12-10 23:01:33 <op_null> I'm stunned we don't have more problems with that in Bitcoin then
256 2014-12-10 23:02:07 <wyager> In what respect? Some wallets do software RAM error correction, I think. Namely Armory
257 2014-12-10 23:04:21 <wyager> Oops, it was .025-.07, not .25. Still pretty high
258 2014-12-10 23:04:49 <op_null> that's still way way above what I was expecting.
259 2014-12-10 23:05:02 <op_null> I guess for 99% of the time it doesn't matter
260 2014-12-10 23:30:54 <huma_> the first time i got "hashMerkleRoot mismatch" error (during regular blockchain update, on another disk) i also got this error repeatedly along with the merkle problem: "Misbehaving: 108.170.37.114:8333 (0 -> 100) BAN THRESHOLD EXCEEDED". is it of any significance to my problem?
261 2014-12-10 23:57:20 <ryan-c> How does one determine which output is covered by SIGHASH_SINGLE?
262 2014-12-10 23:57:40 <gmaxwell> the output at the same index as the signature.
263 2014-12-10 23:59:58 <ryan-c> gmaxwell: That's what I thought, thanks. I don't suppose you can clarify why SIGHASH_SINGLE can't have more inputs than outputs? (does SIGHASH_SINGLE|SIGHASH_ANYONECANPAY change this?)