1 2015-06-23 00:05:37 <phantomcircuit> CodeShark, for sure
  2 2015-06-23 00:05:48 <phantomcircuit> there's essentially no computational improvements left to be implemented
  3 2015-06-23 00:06:02 <phantomcircuit> the only way to scale up is as compute time becomes cheaper
  4 2015-06-23 00:07:05 <phantomcircuit> which is certainly not a given
  5 2015-06-23 00:16:26 <phantomcircuit> Emcy, i would be pretty surprised if intel was holding back significant improvements
  6 2015-06-23 00:16:46 <phantomcircuit> if they had them releasing them would be the optimal strategy
  7 2015-06-23 00:16:52 <phantomcircuit> it would very likely bankrupt amd
  8 2015-06-23 00:17:01 <phantomcircuit> forever cementing their monopoly position
  9 2015-06-23 00:29:42 <nwilcox> Is the ./depends system primarily for deterministic builds via gitian or is it convenient to use on dev machines?
 10 2015-06-23 00:30:15 <nwilcox> I've tried this approach on a linux machine: make -C ./depends/ && ./autogen.sh && ./configure --prefix=./depends/${my_platform}/
 11 2015-06-23 00:30:26 <nwilcox> -but that bakes funky .so paths into the binaries.
 12 2015-06-23 00:30:42 <nwilcox> (Is this the wrong channel for these questions?  No one responded in #bitcoin.)
 13 2015-06-23 00:31:30 <gmaxwell> nwilcox: yes it's useful for both. The main target is gitian but lots of people also use it locally for testing.
 14 2015-06-23 00:31:43 <cfields> nwilcox: it shouldn't bake weird paths in, it should be built with all static libs
 15 2015-06-23 00:31:56 <gmaxwell> I didn't respond because I'm not that clueful for it.  coryfields_ is the main expert on it.
 16 2015-06-23 00:32:16 <nwilcox> cfields: Thanks...  then my problem is of my own creation by tweaking ./depends files...
 17 2015-06-23 00:33:00 <cfields> hm... coryfields_ is a ghost. need to figure out how to kill that off
 18 2015-06-23 00:33:11 <nwilcox> cfields: Oh, wait... Did you see my command above, where I pass ./configure --prefix=...  How does that know to link static libs?
 19 2015-06-23 00:33:13 <cfields> nwilcox: sounds more likely, yes
 20 2015-06-23 00:33:20 <nwilcox> -or does make always link statically?
 21 2015-06-23 00:33:52 <cfields> nwilcox: when you pass the --prefix=/path/to/depends, it picks up the config.site from depends/host/share
 22 2015-06-23 00:34:04 <cfields> that loads up all vars automatically
 23 2015-06-23 00:36:35 <nwilcox> Gotcha. So is config.site a standard autoconf feature?
 24 2015-06-23 00:37:44 <cfields> yes
 25 2015-06-23 00:42:12 <Emcy> phantomcircuit iirc theyre sticking with barely +10% ipc per generation now. They could do better surely
 26 2015-06-23 00:42:22 <Emcy> and i dont think theyd be allowed to bankrupt amd somehow
 27 2015-06-23 00:43:38 <gmaxwell> Emcy: offtopic.
 28 2015-06-23 05:44:06 <paveljanik> Debug window should have a Memory pool tab with a three-in-one graphs - mempool size/the number of mempool transactions/estimatefee 2...
 29 2015-06-23 05:51:22 <wumpus> paveljanik: yes, good idea, go add it :)
 30 2015-06-23 06:02:56 <zveda> What if I run a 'lazy' full node; I check every transaction, but for some transactions, I simply act like a middle-man, passing it through to one other peer. And for some transactions, I send them on to many of my peers. This way I contribute some of my outbound bandwidth and save some bandwidth for the full nodes connected to me.
 31 2015-06-23 06:05:12 <phantomcircuit> zveda, not sybil resistant
 32 2015-06-23 06:05:14 <phantomcircuit> (at all!)
 33 2015-06-23 06:06:02 <zveda> phantomcircuit: what do you mean
 34 2015-06-23 06:06:33 <zveda> in the case where I simply act as a middle-man, I don't contribute or take away anything from the network at all
 35 2015-06-23 06:06:50 <zveda> well i add a bit of lag making the transactions propagate slightly slower
 36 2015-06-23 06:07:38 <zveda> so how are there any new avenues for sybil attack to take advantage of ?
 37 2015-06-23 06:07:46 <phantomcircuit> zveda, you would likely effectively end up being used as a relay for garbage
 38 2015-06-23 06:07:56 <phantomcircuit> ie you would be relaying invalid transactions and that would be bad
 39 2015-06-23 06:07:59 <paveljanik> wumpus, ;-)
 40 2015-06-23 06:08:06 <zveda> well I am still checking every single transaction
 41 2015-06-23 06:08:13 <zveda> why would it be garbage?
 42 2015-06-23 06:08:35 <zveda> I don't forward invalid transactions
 43 2015-06-23 06:08:47 <justanotherusr> zveda: you don't know its invalid if you don't check whether its already spent in a block
 44 2015-06-23 06:08:51 <phantomcircuit> zveda, oh you're validating and then forgetting?
 45 2015-06-23 06:09:08 <phantomcircuit> that seems largely pointless
 46 2015-06-23 06:09:18 <zveda> I am validating every single transaction. but for some I only forward to a single peer, and others I forward to all peers
 47 2015-06-23 06:09:21 <phantomcircuit> you want to keep all of the available transactions in your mempool upto some limit
 48 2015-06-23 06:09:36 <phantomcircuit> and then remove the least useful explicitly
 49 2015-06-23 06:10:11 <wumpus> run a full node for yourself, not 'to help the network'
 50 2015-06-23 06:10:11 <zveda> my goal is to save outbound bandwidth to some full nodes because instead of them sending the transaction to all their peers, they send one copy to me and I send it to everyone
 51 2015-06-23 06:10:31 <justanotherusr> wumpus: then why do we relay?
 52 2015-06-23 06:10:31 <zveda> it's a way to let people even with modest connection speed still contribute to the network
 53 2015-06-23 06:10:52 <wumpus> zveda: stopping serving historical blocks helps a lot
 54 2015-06-23 06:11:23 <wumpus> besides that, yes you could stop relaying some transactions, but I think that's more confusing than useful
 55 2015-06-23 06:11:54 <zveda> wumpus: I can already run a full node but with connections = 0. I want to be able to give some back to the network without killing my router
 56 2015-06-23 06:11:55 <wumpus> e.g. it removes any kind of predictability of acceptance versus relaying
 57 2015-06-23 06:12:26 <wumpus> zveda: but you're sure it is not the serving of old blocks that is taking most of your outbound bandwidth?
 58 2015-06-23 06:12:42 <wumpus> zveda: just wondering why this focus on relaying
 59 2015-06-23 06:13:34 <zveda> wumpus: well the relaying bandwidth seems to be the limit for a lot of people
 60 2015-06-23 06:13:39 <zveda> that stops them running a full node
 61 2015-06-23 06:13:41 <zveda> (eg me)
 62 2015-06-23 06:13:50 <wumpus> have you measured that?
 63 2015-06-23 06:13:58 <zveda> I see your point about stopping serving of old blocks
 64 2015-06-23 06:14:17 <wumpus> what part of your traffic is a) relaying transactions b) relaying blocks c) serving up old blocks ?
 65 2015-06-23 06:14:20 <zveda> wumpus: yes. When I run a full node and forward ports, my router and internet completely freezes up
 66 2015-06-23 06:14:30 <wumpus> zveda: that doesn't answer my question at all
 67 2015-06-23 06:14:42 <zveda> no that part I haven't measured
 68 2015-06-23 06:14:57 <zveda> but anyway we're not discussing my idea now :D
 69 2015-06-23 06:15:04 <wumpus> zveda: if you haven't measured what part of the traffic is problematic, it doesn't make sense to worry about a certain bottleneck conribution
 70 2015-06-23 06:15:21 <chmod755> zveda, you could add luke-jrs patch to stop relaying gambling spam
 71 2015-06-23 06:15:22 <wumpus> zveda: we're working on adding traffic limits to P2P, that's a more robust solution anyhow
 72 2015-06-23 06:16:31 <zveda> wumpus: I think that'll be more complicated. plus you will likely be uploading some transactions very slowly if the traffic is limited
 73 2015-06-23 06:16:46 <zveda> better to upload some transactions fast than to upload them all but at 1/10th the speed
 74 2015-06-23 06:17:28 <wumpus> but it's also more predictable. you could just say I want to dedicate say 25kB/s upload to bitcoin, instead of making it dependent on other factors yet
 75 2015-06-23 06:17:40 <paveljanik> wumpus, my Qt knowledge is ~zero, I'll wait for jonasschnelli ;-)
 76 2015-06-23 06:17:47 <wumpus> anyhow - it's open source, if you want to add a rand()%N to relaying transactions, by all means do that in your local repo
 77 2015-06-23 06:18:10 <zveda> wumpus: yes I know there's that option lol
 78 2015-06-23 06:18:14 <zveda> just wanted to run it by you guys first
 79 2015-06-23 06:18:26 <zveda> thanks for the input
 80 2015-06-23 06:18:28 <wumpus> my gut feeling is that it doesn't helps very little in the overall bandwidth usage, but go experiemnt and measure
 81 2015-06-23 06:18:35 <wumpus> -doesn't
 82 2015-06-23 06:19:51 <wumpus> btw: pruning disables serving and relaying of blocks right now, but not of transactions, so it could be used as a measurement point
 83 2015-06-23 06:20:08 <zveda> I see
 84 2015-06-23 06:35:06 <cfields> wumpus: I waffled around ~5 responses to #6322. Ultimately, i decided not to say anything. Clearly your response was the correct one :)
 85 2015-06-23 06:36:59 <wumpus> cfields: it's completely uneffective, but sums up the current atmosphere perfectly, so it's kind of funny
 86 2015-06-23 06:38:44 <cfields> yep, exactly. I'm still not sure if it's a clever troll or not, but probably best not to find out.
 87 2015-06-23 06:56:56 <Luke-Jr> wumpus: 0.11 should get https://github.com/bitcoin/bitcoin/pull/6246 ?
 88 2015-06-23 06:57:46 <wumpus> Luke-Jr: ok
 89 2015-06-23 07:03:12 <Luke-Jr> wumpus: https://github.com/bitcoin/bitcoin/pull/6282 looks like maybe 0.11 material also, but I'm not familiar with the bug
 90 2015-06-23 07:14:03 <bedeho> Is the purpose of the keypool in the bitcoin core wallet to hold a number of cleartext unused keys, so that they can used without being decrypted quickly?
 91 2015-06-23 07:16:22 <wumpus> bedeho: the keypool holds encrypted keys - but yes, they can be dealt out without needing to generate and encrypt a new key
 92 2015-06-23 07:17:26 <bedeho> wumpus: ok, and these are unused, correct?
 93 2015-06-23 07:17:31 <wumpus> this has the advantage that not every getnewaddress or generation of new receiving address through the GUI needs to ask for a passphrase
 94 2015-06-23 07:17:50 <wumpus> (that's not the original intention of the keypool: that is to extend the life of backups somewhat)
 95 2015-06-23 07:18:11 <wumpus> the keys in the keypool are unused
 96 2015-06-23 07:18:19 <bedeho> ok
 97 2015-06-23 07:18:27 <wumpus> it is more a 'key fifo queue' than a key pool
 98 2015-06-23 07:19:05 <bedeho> why would one need to keep asking for the passphrase, isnt that kept cleartext in memory after first being supplied? It is only occasionally flushed right?
 99 2015-06-23 07:19:50 <wumpus> the passphrase is nuked from memory as soon as possible (in GUI), in RPC that's decided by the user by providing the walletpassphrase timeout
100 2015-06-23 07:21:33 <bedeho> ok
101 2015-06-23 07:33:21 <bedeho> wumpus: when a private key is handled in different parts of the wallet and ui, then CKey is not used, correct? as it seems to be just a raw key, but rather CPrivKey is used, which makes sure the cleartext key it is never paged to disk?
102 2015-06-23 07:35:19 <wumpus> no private key is ever used in the UI, in normal usage they remain within the CWallet (exception could be dumpprivkey, but that's not normal usage)
103 2015-06-23 07:37:44 <bedeho> right, ok, but I guess you could imagine the ui wanting to show WIF version of key to user to write down or something... I dont know. Just trying to get clear on how to manage keys in a safe way within the wallet, but also allow use of it.
104 2015-06-23 07:38:38 <wumpus> as for avoiding paging to disk, that's what the secure_allocator is for, everything that uses that will try, on best effort basis, to try out of swap
105 2015-06-23 07:39:02 <CodeShark> there is no safe way to manage keys in a bitcoind wallet :p
106 2015-06-23 07:39:24 <wumpus> in the absolute, no
107 2015-06-23 07:45:57 <bedeho> :D
108 2015-06-23 07:48:14 <Luke-Jr> bedeho: WIF is Wallet *Import* Format for a reason.. :p
109 2015-06-23 07:56:51 <priidu> is the unit used for Bitcoin fee calculations kB or kiB?
110 2015-06-23 07:57:19 <priidu> as in "0.0001BTC per kB" etc
111 2015-06-23 07:58:02 <Luke-Jr> priidu: it's kB, 1000
112 2015-06-23 07:58:19 <priidu> sorry for the dumb question but am too lazy to look for it in the code myself right now :P
113 2015-06-23 07:58:23 <priidu> Luke-Jr: great, thanks
114 2015-06-23 07:58:47 <Luke-Jr> the only place in Bitcoin Core using KB (1024) is the debug window's network traffic
115 2015-06-23 07:58:57 <priidu> ok, great to know
116 2015-06-23 08:00:00 <wumpus> we aim to use proper SI units
117 2015-06-23 08:00:06 <priidu> yes, that's great
118 2015-06-23 08:00:17 <priidu> hate the whole kB/kiB confusion
119 2015-06-23 08:00:56 <priidu> have had several less-techsavvy people try to convince me I'm retarded and don't know how much bytes a kB is etc
120 2015-06-23 08:02:21 <priidu> mostly due to them not knowing about the IEC *iB units
121 2015-06-23 08:03:06 <Diablo-D3> 1024. duh.
122 2015-06-23 08:03:12 <Diablo-D3> and there is no kb/kib confusion
123 2015-06-23 08:03:12 <priidu> :P
124 2015-06-23 08:03:21 <wumpus> kiB really only makes sense for memory chips, which are necessarily produced in such packages, for bandwidth etc it makes no sense to use it
125 2015-06-23 08:03:22 <Diablo-D3> kb as 1000, and the existence of kib simply isnt valid
126 2015-06-23 08:03:26 <Diablo-D3> and it will never be accepted
127 2015-06-23 08:03:28 <Diablo-D3> thats just the way it is
128 2015-06-23 08:03:36 <Diablo-D3> people need to quit trying to make it a thing
129 2015-06-23 08:03:47 <Diablo-D3> wumpus: oh, with bandwidth, its not
130 2015-06-23 08:03:49 <Diablo-D3> bits are 1000
131 2015-06-23 08:03:50 <priidu> lol @ Luke-Jr
132 2015-06-23 08:03:52 <wumpus> anyhow, let's stop that discussion
133 2015-06-23 08:04:02 <priidu> gotta love them feet and pounds :p
134 2015-06-23 08:04:04 <Diablo-D3> and bits being 1000 is _utterly fucked_
135 2015-06-23 08:04:07 <wumpus> this is not #bitcoin-dev relevant
136 2015-06-23 08:04:45 <priidu> whoops, soz about that, didnt even realize i was in dev
137 2015-06-23 08:05:08 <Diablo-D3> I dont care where you are, I will happily tell you you are wrong
138 2015-06-23 08:11:51 <chmod755> Luke-Jr, do you use tonal bytes in your client?
139 2015-06-23 08:12:00 <wumpus> #bitcoin please
140 2015-06-23 08:12:13 <chmod755> :x
141 2015-06-23 08:20:17 <Luke-Jr> wumpus: what do you think re 6282 ?
142 2015-06-23 08:21:29 <wumpus> Luke-Jr: backporting it
143 2015-06-23 08:24:33 <jonasschnelli> Luke-Jr: re the netmask format. The UI uses the same CSubNet::ToString() function. Would adding a ToShortString() function make sense (where CIDR notation is used)?
144 2015-06-23 08:25:03 <Luke-Jr> jonasschnelli: hm, do we use ToString elsewhere?
145 2015-06-23 08:25:22 <wumpus> jonasschnelli: I'm fine with making ToString smarter
146 2015-06-23 08:25:30 <jonasschnelli> RPC
147 2015-06-23 08:25:40 <wumpus> the requirement is that it results in a string that can be parsed to the same value again
148 2015-06-23 08:26:22 <jonasschnelli> wumpus: i just fear by changing the ToString() function that it could break API listeners
149 2015-06-23 08:26:26 <wumpus> if you can represent a subnet more compactly, e.g. /24 instead of /fff:fff:ffff:ffff:ffff:ffff:ffff:0000, by all means implement that
150 2015-06-23 08:26:38 <wumpus> (eh that example is wrong, but you get the gist)
151 2015-06-23 08:26:48 <jonasschnelli> That would by trivial. Let me first check where we use the ToString in RPC
152 2015-06-23 08:27:02 <jonasschnelli> sure
153 2015-06-23 08:29:51 <jonasschnelli> I think we only use the CSubNet::ToString in the new ban functions. So it's fine changing that.
154 2015-06-23 08:31:19 <jonasschnelli> paveljanik: Yes. A mempool view would be cool in the UI. Need to take up my work for the "core-puls" screen.
155 2015-06-23 08:33:25 <jtimon> in qa/rpc_tests/smartfees.py there's a few while (len(self.nodes[0].getrawmempool()) > 0)
156 2015-06-23 08:34:00 <jtimon> can't that cause infinite loops ?
157 2015-06-23 08:34:11 <paveljanik> jonasschnelli, yes
158 2015-06-23 08:34:32 <jtimon> can I replace those with a forthat caps it to 100 iterations or something?
159 2015-06-23 08:48:43 <wumpus> jtimon: it makes sense to add upper bounds, but be careful; putting too low limits has a tendency to result in random failures when travis server is overloaded
160 2015-06-23 08:49:35 <jtimon> wumpus I see, what can be "too low" there?
161 2015-06-23 08:49:51 <wumpus> jtimon: that's the difficult thing, and that's why no bounds have been added
162 2015-06-23 08:50:30 <jtimon> wumpus would, say, 1000 be sane?
163 2015-06-23 08:50:36 <wumpus> jtimon: no idea
164 2015-06-23 08:50:47 <jtimon> ok
165 2015-06-23 08:53:53 <jtimon> mhmm anyway my infinite loops seems to be in sync_mempools...
166 2015-06-23 08:54:18 <jonasschnelli> jtimon: yes. I also encountered this from time to time
167 2015-06-23 08:54:40 <jonasschnelli> jtimon: do you attached a gdb, non-rpc-test regtest node?
168 2015-06-23 08:54:51 <wumpus> jtimon: that's certainly a test failure
169 2015-06-23 08:55:23 <jonasschnelli> sync_mempools loops until all mempools have the same amount (and ids) of transactions
170 2015-06-23 08:55:45 <wumpus> jtimon: then it would be more constructive to look at fixing the problem, instead of the symptom, no matter if it gets in an eternal loop or times out, it's stil a failure
171 2015-06-23 08:56:33 <jonasschnelli> This could be becuase you have different codebases/chains for your regtest nodes or you managed to create a tx that only stays in one nodes local mempool
172 2015-06-23 08:56:36 <wumpus> all you can accomplish by changing the loop itself is better test error reporting (which is welcome, of course)
173 2015-06-23 08:57:09 <jonasschnelli> This would be good. If sync_mempool don't succeed after 10 seconds it's probably a error
174 2015-06-23 08:59:08 <jtimon> jonasschnelli I'm trying to fix #5180, for some reason the correct policy is not set when AcceptToMemoryPool is called from the python tests, it's weird because params should be set to regtest and therefore policy to test...so I'm directly using a "null policy" (permissive one) in sendrawtransaction (which results in the infinite loop)
175 2015-06-23 09:00:50 <jtimon> jonasschnelli do you have a suggestion to replace seconds with iterations? 1000 iterations should be safe?
176 2015-06-23 09:01:43 <jonasschnelli> jtimon: i think your problem is, that your mempools do no sync and that is not supported over the rpc py testframework IMO
177 2015-06-23 09:01:55 <jonasschnelli> jtimon: try to print some stuff in def sync_mempools(rpc_connections, wait=1): (L45/util.py)
178 2015-06-23 09:02:33 <jonasschnelli> maybe insert after L53 a "print rpc_connections[i].getrawmempool()"
179 2015-06-23 09:03:45 <jonasschnelli> jtimon: if you need mempools that are not in sync for a test, don't call sync_mempools or change it that it supports your test/usecase
180 2015-06-23 09:05:08 <bedeho> jonasschnelli: hi again. Was wondering, is there any reason one should not use something like sqlite for a wallet? it is acid compliant, and I quite like having some type constraints on my data, rather than just key->blob?
181 2015-06-23 09:05:18 <jtimon> I mean, what's happening is that some transactions get rejected by the standard policy but not by the test policy, what I don't know is why the testing policy is not being correctly set (I'm setting it every time after SelectParamsFromCommandLine or SelectParams are called)
182 2015-06-23 09:06:18 <jonasschnelli> jtimon: no idea. :) Happy debugging. Of course it could be that the testframework itself has a bug somewhere
183 2015-06-23 09:07:14 <jonasschnelli> bedeho: i also thought about using sqlite for a wallet... it's a dependency, but probably not a bad one. Yes. Why not.
184 2015-06-23 09:07:33 <bedeho> thanks :)
185 2015-06-23 09:07:34 <jtimon> jonasschnelli yes, that's what I think...
186 2015-06-23 09:07:49 <CodeShark> sqlite is a hell of a lot more pleasant to use than bdb... :p
187 2015-06-23 09:08:13 <jonasschnelli> jtimon: but i'm to far away from #5180 to judge that...
188 2015-06-23 09:08:30 <wumpus> CodeShark: how is that relevant here?
189 2015-06-23 09:08:45 <jonasschnelli> sqlite would also give you the opportunity to handle large wallets (many transactions)
190 2015-06-23 09:08:58 <CodeShark> I've written wallets that use sqlite
191 2015-06-23 09:09:14 <CodeShark> wumpus: it isn't particularly relevant
192 2015-06-23 09:09:15 <jonasschnelli> CodeShark: what is your experience (+ & -)?
193 2015-06-23 09:09:16 <wumpus> (I agree with you, sqlite is very pleasant to use, but it's not #bitcoin-dev discission)
194 2015-06-23 09:09:49 <phantomcircuit> CodeShark, the wallet should be using a log
195 2015-06-23 09:09:55 <jonasschnelli> wumpus: but what do you think about using sqlite for a new wallet insted of bdb or logdb?
196 2015-06-23 09:09:56 <phantomcircuit> not a database
197 2015-06-23 09:10:18 <jonasschnelli> phantomcircuit: why?
198 2015-06-23 09:10:29 <wumpus> jonasschnelli: yet another proposal for a wallet?
199 2015-06-23 09:10:39 <CodeShark> well, we're confounding several things here, perhaps - are we talking about a transaction history? or a key manager/signer?
200 2015-06-23 09:10:45 <jonasschnelli> :) ... just thinking loud.
201 2015-06-23 09:11:10 <phantomcircuit> jonasschnelli, it's much harder to screw up a log than a database
202 2015-06-23 09:11:46 <wumpus> although sqlite is superior to berkeleydb, I wouldn't want to be dependent on yet another off the shelf database for the wallet
203 2015-06-23 09:11:50 <jonasschnelli> phantomcircuit: agreed. What about using it for a transaction history, ability to perform fast queries?
204 2015-06-23 09:12:57 <phantomcircuit> jonasschnelli, at start you walk the log and construct an index in memory
205 2015-06-23 09:13:03 <jonasschnelli> wumpus: Agreed. The compatibility issues would probably worse!
206 2015-06-23 09:13:52 <CodeShark> if you use odb you can easily support several SQL databases (sqlite, posgresql, mysql, oracle, etc...) but its license is not compatible with MIT
207 2015-06-23 09:13:58 <wumpus> if you want to build a super-scalable wallet, storing some things like transaction history in a SQL database makes sense especially if you can make the backend pluggable. But for the reference wallet in bitcoin core I don't think we should go that far.
208 2015-06-23 09:14:30 <phantomcircuit> wumpus, i dont believe that's necessary actually
209 2015-06-23 09:14:41 <jonasschnelli> phantomcircuit: Yes. I think sqlite makes sense if you have to manage a staoshi-dice-like transaction list. But for bitcoin-core i also see a hashed log of frames.
210 2015-06-23 09:15:26 <phantomcircuit> jonasschnelli, consider that the tools used in bitcoin core to maintain the utxo are identical to the tools needed to track unspent outputs
211 2015-06-23 09:16:06 <jonasschnelli> right
212 2015-06-23 09:16:07 <Luke-Jr> jonasschnelli: let's not pretend there is anything reasonable about what DP does? :p
213 2015-06-23 09:16:47 <jonasschnelli> DP?
214 2015-06-23 09:16:58 <CodeShark> I don't think bitcoin core should have a wallet :p
215 2015-06-23 09:17:00 <CodeShark> lol
216 2015-06-23 09:17:13 <wumpus> you're not being constructive CodeShark
217 2015-06-23 09:17:22 <phantomcircuit> CodeShark, i think we're all agreed that it should, but that it should be separated from the consensus daemon
218 2015-06-23 09:17:32 <CodeShark> ok, ok...
219 2015-06-23 09:17:34 <Luke-Jr> phantomcircuit: same thing
220 2015-06-23 09:17:48 <phantomcircuit> Luke-Jr, not exactly
221 2015-06-23 09:17:57 <chmod755> CodeShark, it should have a wallet, but why does it still have a miner....
222 2015-06-23 09:18:15 <phantomcircuit> chmod755, for testing purposes
223 2015-06-23 09:18:25 <CodeShark> they could both be packaged together in the same distribution bundle...but I really like thinking of bitcoin core as a runtime service for other applications
224 2015-06-23 09:18:27 <wumpus> IMO at this point there's not really any better alternative for a wallet besides the one in bitcoin core
225 2015-06-23 09:18:38 <Luke-Jr> +1
226 2015-06-23 09:18:43 <phantomcircuit> +1
227 2015-06-23 09:19:04 <Luke-Jr> (disclaimer: I haven't tried CodeShark's recently)
228 2015-06-23 09:19:07 <phantomcircuit> the state of the wallet ecosystem is pretty sad
229 2015-06-23 09:19:08 <chmod755> phantomcircuit, the user doesn't need it tho - i don't think it's needed in the main branch
230 2015-06-23 09:19:11 <wumpus> that's what counts, not idealism about what should be where, I agreed with you maybe a year ago but other implementations failed to materialize as expeted
231 2015-06-23 09:19:27 <wumpus> phantomcircuit: exactly
232 2015-06-23 09:19:29 <phantomcircuit> chmod755, it's part of the automated test suite
233 2015-06-23 09:19:37 <phantomcircuit> it's absolutely necessary in the main branch because of that
234 2015-06-23 09:19:37 <wumpus> too much talk, too many proposals, too little actual safe and usable systems
235 2015-06-23 09:20:52 <wumpus> CodeShark: how is your wallet project doing?
236 2015-06-23 09:21:01 <CodeShark> it's going very well
237 2015-06-23 09:21:36 <phantomcircuit> CodeShark, spv client?
238 2015-06-23 09:22:38 <CodeShark> we continue to have a bunch of downloads - it's been used for several major crowdsales - and we're working on an application development platform
239 2015-06-23 09:22:45 <CodeShark> yes, phantomcircuit
240 2015-06-23 09:23:13 <bedeho> CodeShark: is it on github? link?
241 2015-06-23 09:23:31 <CodeShark> https://github.com/ciphrex/mSIGNA
242 2015-06-23 09:24:21 <CodeShark> it's designed around using parallel BIP32 trees to construct m-of-n p2sh
243 2015-06-23 09:24:52 <CodeShark> but we're working on more advanced scripts
244 2015-06-23 09:25:15 <bedeho> CodeShark: nice, Im am also trying to get a minimal C++/Qt wallet going. Do you use some standard bitcoin lib?
245 2015-06-23 09:25:42 <CodeShark> no, bedeho - I wrote my own libraries for core bitcoin structures and networking
246 2015-06-23 09:26:29 <bedeho> CodeShark: I see, I can see William has worked on it with you, I know he uses libbitcoin with AirBitz, why did you not just use that?
247 2015-06-23 09:26:33 <Luke-Jr> CodeShark: but there *are* libraries?
248 2015-06-23 09:27:04 <CodeShark> Luke-Jr: what do you mean?
249 2015-06-23 09:27:25 <CodeShark> bedeho: I started working on this WAY before there was libbitcoin or anything even remotely like that
250 2015-06-23 09:27:44 <Luke-Jr> CodeShark: you have independent libraries for the wallet?
251 2015-06-23 09:27:54 <Luke-Jr> CodeShark: uh, libbitcoin is pretty old :p
252 2015-06-23 09:28:10 <Luke-Jr> as in, 2011 May
253 2015-06-23 09:28:22 <CodeShark> well, it certainly wasn't stable back then...
254 2015-06-23 09:28:28 <CodeShark> and didn't support all the stuff I wanted
255 2015-06-23 09:29:03 <CodeShark> and yes, Luke-Jr - the github repo builds three independent libraries that I've reused in multiple projects
256 2015-06-23 09:29:14 <Luke-Jr> CodeShark: neat
257 2015-06-23 09:30:38 <CodeShark> thanks :)
258 2015-06-23 09:32:47 <bedeho> CodeShark: that is very cool. I cant see a license here, can people use it for commercial projects?
259 2015-06-23 09:33:18 <CodeShark> one of the libraries is MIT, two are AGPLv3
260 2015-06-23 09:33:42 <jtimon> jonaschenlli why not just use levelDB for the wallet too?
261 2015-06-23 09:34:01 <CodeShark> the GUI app has an eula which permits usage for commercial purposes...but the library licenses still apply...as do those of other dependencies
262 2015-06-23 09:34:27 <CodeShark> in other words, if you want to customize it for your business you have to publish your source code
263 2015-06-23 09:34:30 <CodeShark> and include copyrights
264 2015-06-23 09:34:52 <Luke-Jr> CodeShark: AGPL doesn't require publishing source code FYI
265 2015-06-23 09:35:36 <wumpus> jtimon: there's not even a guarantee we'll stick with leveldb for the block index and utxo databases
266 2015-06-23 09:35:57 <wumpus> jtimon: in that case, relying on leveldb will make sure we have to lug another database library around for legacy
267 2015-06-23 09:36:18 <jtimon> wumpus is anybody talking about replacing it in the near medium-term?
268 2015-06-23 09:36:25 <wumpus> the leveldb databases are transient; the wallet is supposed to live for a long time
269 2015-06-23 09:36:53 <jtimon> mhmm, transient in what sense?
270 2015-06-23 09:36:55 <wumpus> (which is also the problem with depending on berkeleydb :( )
271 2015-06-23 09:37:11 <wumpus> that it can be rebuilt from scratch at any time, there's no need to resurrect them in ~20 years
272 2015-06-23 09:37:46 <bitcoin-dev146> Hi guys ... i am looking to develop an exchange with a domain name that i have and i am looking for a partner who can work the system out as i am new to bitcoins
273 2015-06-23 09:37:48 <jtimon> I'm not sure I'm following
274 2015-06-23 09:37:50 <wumpus> jtimon: lmdb would be a potential alternative
275 2015-06-23 09:38:05 <wumpus> (for the indexes, not for the wallet, obviously)
276 2015-06-23 09:38:44 <wumpus> bitcoin-dev146: not here
277 2015-06-23 09:38:52 <bitcoin-dev146> ok
278 2015-06-23 09:39:11 <bitcoin-dev146> where could i talk about it
279 2015-06-23 09:39:19 <bedeho> CodeShark: is it an SPV wallet?
280 2015-06-23 09:39:26 <jtimon> wumpus I didn't understood the problem with levleDB and the need for 2 different databases...
281 2015-06-23 09:39:42 <wumpus> jtimon: wallet should have a custom, well-documented format, instead of a database library
282 2015-06-23 09:40:03 <bitcoin-dev146> anyone interested in aquiring a domain name sharkdate.com for 1 btc ?
283 2015-06-23 09:40:10 <CodeShark> bedeho: yes - but it's highly recommended you run a full validator node for maximal security. I've somewhat given up on trying to optimize SPV because I think it's fundamentally broken :)
284 2015-06-23 09:40:21 <bitcoin-dev146> also pokerviper.com a poker site for an exchange of 1btc
285 2015-06-23 09:40:42 <CodeShark> but yes, bedeho - it uses filtered blocks and can connect to any node
286 2015-06-23 09:41:44 <CodeShark> the truth of the matter is that there are only really two markets right now - those who want to run full validators...and those who are willing to let others provide validation :)
287 2015-06-23 09:42:44 <jtimon> wumpus what's the problem with a database? how is that undocumented? you're proposing other databases  anyway, so...
288 2015-06-23 09:42:45 <CodeShark> trying to make SPV work requires essentially providing some level of validation services
289 2015-06-23 09:42:52 <CodeShark> so it's not really worth optimizing, IMHO
290 2015-06-23 09:42:58 <wumpus> jtimon: the wallet needs a stable format, databases aren't optimized for that
291 2015-06-23 09:43:14 <wumpus> jtimon: as seen with berkeleydb there are lots of problems with forward and backward compatibility
292 2015-06-23 09:43:39 <wumpus> jtimon: for the next wallet format we'd like to avoid that, and have our own control over the format
293 2015-06-23 09:43:41 <jtimon> wumpus then no database is acceptable then?
294 2015-06-23 09:44:28 <CodeShark> I want O(log n) validation for all nodes :)
295 2015-06-23 09:44:47 <wumpus> I'd say no third-party database library is acceptable, no
296 2015-06-23 09:45:44 <CodeShark> the bdb compatibility thing is starting to get a little annoying :p
297 2015-06-23 09:46:02 <CodeShark> I always override it...because let's face it, I'm never going to be sharing my wallet.dat with anyone else
298 2015-06-23 09:46:08 <jtimon> wumpus "lmdb would be a potential alternative" you seem to be contradicting yourself?
299 2015-06-23 09:46:09 <wumpus> CodeShark: you wouldn't say...
300 2015-06-23 09:46:21 <wumpus> jtimon: FOR THE INDEXES AND UTXO SET
301 2015-06-23 09:46:39 <jtimon> oh, you mean alternative to levelDB, sorry
302 2015-06-23 09:46:43 <wumpus> yes.
303 2015-06-23 09:47:38 <jtimon> wumpus: then... json on a file for the wallet or something?
304 2015-06-23 09:47:53 <wumpus> if someone is interesting in doing performance benchmarks, it would be interesting how lmdb would perform in place of leveldb. lmdb does have some practical issues, e.g. it's optimized for 64-bit, there is a specific 32-bit fork but it's less well maintained
305 2015-06-23 09:48:12 <wumpus> but that may change, who knows
306 2015-06-23 09:49:02 <wumpus> jtimon: something like that, see e.g. https://github.com/bitcoin/bitcoin/pull/5686
307 2015-06-23 09:49:27 <wumpus> also lmdb doesn't have the paranoid integrity checking that leveldb has
308 2015-06-23 09:49:28 <CodeShark> if you want a stable format, a standard import/export format might suffice
309 2015-06-23 09:50:16 <CodeShark> it then frees you to use whatever storage engine you want
310 2015-06-23 09:50:16 <wumpus> CodeShark: we have a standard import/export format
311 2015-06-23 09:50:23 <jtimon> wumpus oh, I thought logDB was "another DB" and therefore unacceptable
312 2015-06-23 09:50:30 <wumpus> CodeShark: still, it hasn't made people any less worried about wallet.dat files
313 2015-06-23 09:50:39 <wumpus> jtimon: no, it's an internal solution
314 2015-06-23 09:51:36 <wumpus> CodeShark: I do agree that e.g. an external converter script to dump a berkeleydb wallet.dat and produce a standard import/export format wallet may help, but it shouldn't be dependent on the application itself
315 2015-06-23 09:52:19 <CodeShark> not sure what you mean by that it shouldn't be dependent on the application itself
316 2015-06-23 09:52:31 <CodeShark> what shouldn't be dependent on the application itself?
317 2015-06-23 09:52:35 <wumpus> never mind
318 2015-06-23 09:53:22 <CodeShark> what can get a little tricky is when your schema changes...and you need to migrate data
319 2015-06-23 09:53:27 <wumpus> I'm tired of eternal discussions, going back working on software that actually exists
320 2015-06-23 09:53:32 <CodeShark> hehe
321 2015-06-23 09:54:01 <CodeShark> doesn't all this exist?
322 2015-06-23 09:55:57 <CodeShark> Luke-Jr: regarding AGPLv3, how do you interpret section 6? http://www.gnu.org/licenses/agpl-3.0.en.html
323 2015-06-23 09:56:26 <CodeShark> "You may convey a covered work in object code form...provided that you also convey the machine-readable Corresponding Source under the terms of this License..."
324 2015-06-23 09:57:05 <bedeho> CodeShark: what does CoinQ do vs CoinCore?
325 2015-06-23 09:57:06 <CodeShark> where Corresponding Source is defined as: "The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities....
326 2015-06-23 09:57:41 <CodeShark> bedeho: CoinCore are the core message structures and serialization classes for them
327 2015-06-23 09:58:29 <CodeShark> essentially, all the stuff defined here: https://en.bitcoin.it/wiki/Protocol_documentation
328 2015-06-23 09:58:50 <CodeShark> CoinQ is the networking library
329 2015-06-23 09:59:13 <CodeShark> it manages peer sessions and provides for message subscriptions to other parts of the application
330 2015-06-23 10:00:55 <CodeShark> CoinQ also manages some other things like script parsing and chain parameters
331 2015-06-23 10:02:15 <CodeShark> I've been meaning to work on a library just for scripting...but perhaps the satoshi script will ultimately get replaced :p
332 2015-06-23 10:05:34 <CodeShark> I've also been meaning to separate the crypto stuff into an entirely separate library
333 2015-06-23 10:10:19 <phantomcircuit> CodeShark, you missed the midmarket
334 2015-06-23 10:10:31 <CodeShark> what's the midmarket?
335 2015-06-23 10:10:37 <phantomcircuit> CodeShark, people who can accept other people doing the validation but would like more security if they can get it for free
336 2015-06-23 10:10:55 <phantomcircuit> SPV w/ continuous upgrade attempts
337 2015-06-23 10:11:07 <phantomcircuit> unfortunately that's non trivial to implement :P
338 2015-06-23 10:11:21 <CodeShark> the validation is the only part vis security that is a problem :)
339 2015-06-23 10:11:46 <CodeShark> the rest of security stuff - key generation/management, signing, etc...can be offered 100% regardless of whether they validate themselves
340 2015-06-23 10:13:44 <CodeShark> SPV is too broken to be fixed at this point, IMHO...the only thing that could perhaps help is txout commitments
341 2015-06-23 10:14:01 <CodeShark> but even then there are a bunch of problems with general queries
342 2015-06-23 10:14:26 <CodeShark> at least with txout commitments you'd be able to validate proofs of txout sets
343 2015-06-23 10:14:33 <phantomcircuit> CodeShark, utxo commitments are too expensive to actually use
344 2015-06-23 10:15:18 <CodeShark> I keep hearing this but I have not seen hard evidence (nor hard models) for this
345 2015-06-23 10:16:26 <CodeShark> tree operations are generally O(log n) no?
346 2015-06-23 10:16:31 <phantomcircuit> CodeShark, the validation for such a commitment is O(n)
347 2015-06-23 10:16:48 <phantomcircuit> n being the number of entries in the utxo
348 2015-06-23 10:16:51 <phantomcircuit> proceed to cry
349 2015-06-23 10:17:55 <CodeShark> the validation for the commitment is O(n), I agree...if you start from genesis. but we already have O(n) for validation
350 2015-06-23 10:18:29 <phantomcircuit> CodeShark, different n values
351 2015-06-23 10:18:42 <CodeShark> O(n) for generating the utxo set on a validator
352 2015-06-23 10:18:53 <CodeShark> or more like Omega(n) :p
353 2015-06-23 10:19:39 <CodeShark> applying a txout state transition is O(log n) per output so O(n log n) per block, yes?
354 2015-06-23 10:20:20 <CodeShark> but the validator already has to insert the txout into an indexed fast memory
355 2015-06-23 10:20:27 <CodeShark> so you already have the log n factor
356 2015-06-23 10:20:57 <CodeShark> regardless of whether you commit the db hash or not
357 2015-06-23 10:21:48 <CodeShark> basically, validators already must track the utxo set regardless of whether they commit or not
358 2015-06-23 10:21:56 <jtimon> CodeShark does CoinQ use libbitcoinconsensus?
359 2015-06-23 10:22:27 <phantomcircuit> CodeShark, huh
360 2015-06-23 10:22:39 <phantomcircuit> CodeShark, core doesn't calculate the utxo for normal validation
361 2015-06-23 10:22:58 <CodeShark> jtimon: no, CoinQ does not do very many consensus checks other than basic stuff - it is not a full validator
362 2015-06-23 10:23:01 <phantomcircuit> it only does lookups of the transactions referenced
363 2015-06-23 10:23:24 <phantomcircuit> CodeShark, also be careful with n there
364 2015-06-23 10:24:04 <CodeShark> phantomcircuit: true, the index is txhash:txindex
365 2015-06-23 10:24:12 <CodeShark> another annoying thing about the protocol :p
366 2015-06-23 10:24:27 <phantomcircuit> CodeShark, insertion/lookup in leveldb is amortized O(log n) where n is the size of the utxo, so total checking is O(m log n) m = transactions referenced in block and n = utxo entries
367 2015-06-23 10:26:33 <phantomcircuit> CodeShark, this is what makes utxo expansion such a big problem btw
368 2015-06-23 10:26:49 <phantomcircuit> this is the fun part where bitcoin really truly does not scale
369 2015-06-23 10:27:01 <CodeShark> yep
370 2015-06-23 10:27:07 <CodeShark> and why SPV is fundamentally broken
371 2015-06-23 10:27:24 <phantomcircuit> unfortunately sentences that start out with "the runtime complexity" end with people not understanding
372 2015-06-23 10:27:42 <phantomcircuit> (also note that there's probably huge constant time factors missing in that analysis)
373 2015-06-23 10:32:06 <CodeShark> phantomcircuit: they must be starting to notice that synching a new node seems to take longer and longer and longer (headers-first sync being the only substantial recent optimization)
374 2015-06-23 10:32:46 <CodeShark> and that the problem will only continue to get worse and worse and worse
375 2015-06-23 10:33:24 <phantomcircuit> CodeShark, most people who cant understand the runtime complexity explanation simply switch to using an spv client
376 2015-06-23 10:33:38 <phantomcircuit> and assume that someone will run a client on some crazy server or something
377 2015-06-23 10:34:24 <CodeShark> but SPV provides such weak validation and such weak privacy, you might as well just serve txout queries directly
378 2015-06-23 10:34:53 <CodeShark> makes the client logic far simpler...and isn't really all that much weaker securitywise
379 2015-06-23 10:36:22 <wumpus> committing to bloom filters in blocks was an interesting idea re: SPV
380 2015-06-23 10:36:54 <wumpus> at least the client can then check the filters locally, instead of revealing them to others
381 2015-06-23 10:39:39 <CodeShark> yes - that would help with privacy. but I'm not really sure it's worth trying to fix this too much because SPV has too many other problems to be a long-term solution
382 2015-06-23 10:40:41 <phantomcircuit> CodeShark, huh
383 2015-06-23 10:40:44 <phantomcircuit> running the numbers
384 2015-06-23 10:40:48 <phantomcircuit> yeah that's bad
385 2015-06-23 10:40:51 <wumpus> I don't even want to have an opinion on whethe rit's worth to do, if people want to work on it they do
386 2015-06-23 10:41:16 <phantomcircuit> utxo lookups and inserts need to be amortized O(1) for this to actually scale right
387 2015-06-23 10:41:24 <CodeShark> if I were in your spot I'd probably say something similar, wumpus :)
388 2015-06-23 10:42:04 <CodeShark> or perhaps I wouldn't - which is why it's better for you to be in that spot than me, I suppose
389 2015-06-23 10:42:10 <CodeShark> :p
390 2015-06-23 10:42:55 <CodeShark> wumpus really knows how to get along with everyone - it's pretty amazing :)
391 2015-06-23 10:45:01 <CodeShark> it's just too bad I couldn't find you in Amsterdam so we could meet in person
392 2015-06-23 10:47:13 <wumpus> heh :)
393 2015-06-23 10:48:37 <wumpus> yes, heard lots of people were looking for me then, apparantly I'm good in hiding
394 2015-06-23 10:48:44 <CodeShark> you are!
395 2015-06-23 10:48:53 <CodeShark> I looked for you for like two or three straight days
396 2015-06-23 10:49:04 <CodeShark> even sipa had a hard time finding you :p
397 2015-06-23 10:54:36 <CodeShark> it's been a tough decision what to do regarding SPV support - but I just don't see how to really improve upon it in a way that really is worthwhile, unfortunately
398 2015-06-23 10:55:44 <CodeShark> at the same time, I don't want to just run a bunch of validation nodes for my users
399 2015-06-23 10:56:21 <phantomcircuit> CodeShark, plug the hole with trust
400 2015-06-23 10:56:24 <phantomcircuit> fun right?
401 2015-06-23 10:56:28 <CodeShark> heh
402 2015-06-23 10:58:14 <CodeShark> on the positive side, there's still a lot that can be done on the key management/account policy/script stuff :)
403 2015-06-23 10:59:08 <CodeShark> but for now I'm basically recommending that my users run a full node of their own for validation
404 2015-06-23 11:00:50 <CodeShark> and could provide lists of "trusted" nodes and attempt connecting to them in a way that's fairly transparent to the user...but argh
405 2015-06-23 11:26:03 <Belxjander> v bbbbbbbbbbbbbb8o;122222222222222222222222222222222222222222222222222222q
406 2015-06-23 11:36:14 <michagogo> Hm, did gmaxwell assign BIP100? Not seeing it on the ML (might have missed it)
407 2015-06-23 12:14:59 <volante> how can I execute just a specific test?  I see that test_bitcoin has a -t <testname> option, but i'm having trouble working on how to format the name (eg. ./src/test/test_bitcoin -t MempoolRemoveTest doesn't work)
408 2015-06-23 12:16:24 <wumpus> I know you can execute a certain testsuite, I don't know if you can execute a certain specific test
409 2015-06-23 12:18:00 <volante> ahh i see.  -t mempool_tests works.  thanks.
410 2015-06-23 12:41:48 <btcdrak> michagogo: I dont believe a number was assigned, looks like jgarzik just picked a number.
411 2015-06-23 12:42:01 <michagogo> Isn't that strongly discouraged?
412 2015-06-23 12:47:38 <wumpus> it also wasn't submitted as pull to the bips repository
413 2015-06-23 12:48:12 <wumpus> not sure if it's intended to be part of the normal bip process at all
414 2015-06-23 13:09:07 <MrTratta> michagogo, yeah it usually is
415 2015-06-23 13:16:00 <michagogo> wumpus: meh, it's misleading to call it that if not
416 2015-06-23 13:51:18 <c25> I'm redownloading the block chain, stuck at block 361573 with version 100200, "errors" field is empty
417 2015-06-23 13:51:46 <c25> is there something i should try?
418 2015-06-23 14:25:37 <jonasschnelli> is there a reason why ./bitcoin-cli's error messages must be flat unformatted? example: https://gist.github.com/jonasschnelli/97c2d1c373182e067410
419 2015-06-23 14:25:57 <jonasschnelli> error messages should be treated like the help message, nice and formatted
420 2015-06-23 15:22:08 <jonasschnelli> wumpus: do you see a more optimized way for this: https://github.com/bitcoin/bitcoin/commit/8d50ab3d4ec08a9dd0a3b5eb411bf42b03fb0542#diff-4942713bcd2752bb531bf76c9a0ba38eR1307
421 2015-06-23 15:22:31 <jonasschnelli> my brain is not capable for such binary operations...
422 2015-06-23 16:49:58 <wumpus> I created a P2P transaction submission tool, don't remember who was asking for this a while ago: https://github.com/laanwj/bitcoin-submittx
423 2015-06-23 17:01:32 <wallet42> can a pruned node be asked for blocks that is still in its blockstore (e.g. not yet pruned)
424 2015-06-23 17:02:51 <gmaxwell> not currently. Why?
425 2015-06-23 17:03:13 <wumpus> I think it can be? it just won't inv them
426 2015-06-23 17:03:32 <wallet42> so pruned nodes only inv txs
427 2015-06-23 17:03:44 <s7r> what data is pruned, actually? and how much in disk space usage does it save via this operation?
428 2015-06-23 17:04:28 <wallet42> s7r: block data, up to a minimum of ~ 550 MB worth of blocks (~ 4 days)
429 2015-06-23 17:04:30 <wumpus> block data is pruned, please refer to the release notes of 0.11 https://github.com/bitcoin/bitcoin/blob/0.11/doc/release-notes.md#privacy-disable-wallet-transaction-broadcast
430 2015-06-23 17:04:39 <gmaxwell> wumpus: IIRC we actually detect someone attempting to getblock and reject.
431 2015-06-23 17:04:50 <wumpus> eh https://github.com/bitcoin/bitcoin/blob/0.11/doc/release-notes.md#block-file-pruning
432 2015-06-23 17:05:04 <wumpus> gmaxwell: ok, I thought that just added a HAVE_BLOCK check
433 2015-06-23 17:05:33 <wallet42> so the pruned node is actually slowing down block propagation?
434 2015-06-23 17:05:42 <wumpus> no, it just doesn't have an effect on it
435 2015-06-23 17:06:15 <wumpus> anyhow there are some pulls that attempt to re-enable block relaying with pruning, I suggest checking them out if you're interested in that
436 2015-06-23 17:06:29 <wallet42> okay thx
437 2015-06-23 17:06:32 <wallet42> last q
438 2015-06-23 17:06:47 <wallet42> getheaders is still available on these nodes?
439 2015-06-23 17:06:56 <wumpus> yes, headers are unaffected.
440 2015-06-23 17:07:53 <gmaxwell> wumpus: indeed, we won't inv, not even new blocks. though we will respond to getdata if we've got it.
441 2015-06-23 17:07:59 <s7r> oh, yeah
442 2015-06-23 17:08:16 <s7r> a pruned node will not relay, after that
443 2015-06-23 17:08:20 <s7r> except the last 2 days of 288 blocks?
444 2015-06-23 17:08:22 <gmaxwell> I note this is a fingerprinting vector, fwiw.
445 2015-06-23 17:08:31 <gmaxwell> s7r: no, no blocks right now.
446 2015-06-23 17:09:10 <gmaxwell> s7r: there is a potential issue where relaying blocks can cause a peer to try to fetch the history; which it cannot provide.
447 2015-06-23 19:29:38 <jgarzik> gmaxwell, req a bip #
448 2015-06-23 19:30:33 <Luke-Jr> CodeShark: source code needs to be available to anyone using the software, but not the general public
449 2015-06-23 20:08:19 <jonasschnelli> is there a reason why CRPCTable tableRPC has no locking?
450 2015-06-23 20:09:07 <jonasschnelli> it's accessed over multiple threads: https://github.com/bitcoin/bitcoin/blob/master/src/rpcserver.cpp#L878
451 2015-06-23 20:11:40 <hearn> because it's read only, no?
452 2015-06-23 20:12:12 <hearn> and on x86 it's OK to not have the synchronisation in this case, due to the strong memory model
453 2015-06-23 20:13:29 <Luke-Jr> jonasschnelli: why are forward slashes escaped?
454 2015-06-23 20:14:00 <jonasschnelli> Luke-Jr: you mean UniValue json output?
455 2015-06-23 20:14:11 <Luke-Jr> yes
456 2015-06-23 20:14:59 <jonasschnelli> Luke-Jr: good question. Was written by jgarzik.. :)
457 2015-06-23 20:15:17 <jonasschnelli> it's not required by json. But it's convenient in parsing.
458 2015-06-23 20:15:30 <jonasschnelli> hearn: yes. I see. Need to adapt locking for a dynamic table.
459 2015-06-23 20:16:29 <jonasschnelli> Luke-Jr: i assume if you parse it with a standard json parser you won't have problems with \/.
460 2015-06-23 20:16:58 <jonasschnelli> But i agree. It looks silly. We should patch bitcoin-cli for this.
461 2015-06-23 20:18:42 <Luke-Jr> rather patch UniValue <.<
462 2015-06-23 20:19:35 <Luke-Jr> jonasschnelli: the active/ban node lists - is there a sizer thing in between them?
463 2015-06-23 20:22:01 <jonasschnelli> Luke-Jr: patching UniValue: nah. The escaping is useful if you embed the json. On transport level i thin the \/ makes sense. But bitcoin-cli should def. un-parse it.
464 2015-06-23 20:22:26 <jonasschnelli> jonasschnelli: ban/window:no, they have fix sized. I'm not sure if a such element exists out of the box in QT
465 2015-06-23 20:22:28 <Luke-Jr> jonasschnelli: if you embed the JSON? what? how?
466 2015-06-23 20:23:22 <jonasschnelli> Luke-Jr: in html... (now don't laugh)
467 2015-06-23 20:23:22 <Luke-Jr> / is not a special character in any programming language I know of
468 2015-06-23 20:23:48 <Luke-Jr> HTML only gives special meaning to / inside <>, and doesn't care about \ for escaping..
469 2015-06-23 20:25:08 <jonasschnelli> Luke-Jr: Agreed. Hmmm... need to inspect the code
470 2015-06-23 20:26:30 <gmaxwell> hearn: "benign" races are _not safe_ on x86, regardless of what the hardware does the compiler is only obligated to apply the language's memory model, and the compiler can and will optimize as if no race is possible.
471 2015-06-23 20:27:10 <Diablo-D3> gokay so I have a question
472 2015-06-23 20:27:16 <hearn> regardless, in this case the compiler won't move the construction of CRPCTable to after the reads
473 2015-06-23 20:27:31 <Diablo-D3> are we or are we not doing the block size increase
474 2015-06-23 20:27:46 <Diablo-D3> because I agree with the school of thought that we should wait so it pushes miner fees up higher
475 2015-06-23 20:28:32 <jonasschnelli> Luke-Jr: you could change univalue_escapes.h
476 2015-06-23 20:28:48 <jonasschnelli> L39
477 2015-06-23 20:29:04 <Luke-Jr> jgarzik: ^ ok?
478 2015-06-23 20:31:47 <gmaxwell> hearn: yea, didn't look at the specific case; just was making sure the point that you can't assume the x86 memory model even running on x86 wasn't lost.
479 2015-06-23 20:31:56 <jonasschnelli> Luke-Jr: it would be still valid json, i think jgarzik won't mind.
480 2015-06-23 20:32:07 <hearn> gmaxwell: indeed
481 2015-06-23 20:32:16 <hearn> i think there's work to define a memory model for C++, or i vaguely recall reading about it
482 2015-06-23 20:32:26 <hearn> jvm has had one for a long time. it's useful.
483 2015-06-23 20:33:51 <jonasschnelli> But locking that line (lets say LOCK(cs_tableRPC) one line above): https://github.com/bitcoin/bitcoin/blob/master/src/rpcserver.cpp#L878 would lock a thread until the whole RPC action is finished?!
484 2015-06-23 20:34:30 <hearn> yeah
485 2015-06-23 20:34:47 <jonasschnelli> i kinda dislike this rpc dispatching.
486 2015-06-23 20:35:25 <hearn> why?
487 2015-06-23 20:37:31 <jonasschnelli> It's too static. I like to rebase a PR which makes the table flexible so that other "modules" can extend the table and add new commands to the rpc.
488 2015-06-23 20:38:26 <jonasschnelli> hearn: i think the signaling approach makes more sense (https://github.com/bitcoin/bitcoin/pull/6006)
489 2015-06-23 20:39:15 <hearn> i see
490 2015-06-23 20:41:38 <jonasschnelli> time for bed. Nite.
491 2015-06-23 20:44:50 <BlueMatt> wumpus: ummm...why did you merge https://github.com/bitcoin/bitcoin/pull/6088 ????????
492 2015-06-23 20:45:20 <BlueMatt> wumpus: I see 0 acks on the latest code? can you revert that?
493 2015-06-23 20:46:29 <BlueMatt> or can i get some posthumous acks on 6088?
494 2015-06-23 20:51:58 <BlueMatt> gmaxwell/sipa/jgarzik: ^
495 2015-06-23 21:13:54 <jgarzik> gmaxwell, a new block size proposal bio
496 2015-06-23 21:14:17 <jgarzik> gmaxwell, a new block size proposal bip
497 2015-06-23 21:18:42 <gmaxwell> jgarzik: 102
498 2015-06-23 21:29:27 <jgarzik> jonasschnelli, luke-jr:  Any deviation from JSON standard in UniValue is a bug
499 2015-06-23 22:17:26 <jgarzik> gmaxwell, thanks
500 2015-06-23 22:36:32 <gribble> Error: "nextdiff" is not a valid command.
501 2015-06-23 22:36:32 <jgarzik> ;;nextdiff