1 2016-05-06 09:10:26 <xenog> I am looking for a patch that was mentioned to me by afk11 that avoids doing signature validation on transactions included in old blocks. I have been browsing GitHub looking for that pull request, but I cannot seem to find it. Would somebody here know what I am talking about and point me in the right direction?
  2 2016-05-06 09:18:52 <jonasschnelli> xenog: bitcoin-core does that already
  3 2016-05-06 09:19:51 <jonasschnelli> Or you can merge classics PR https://github.com/bitcoinclassic/bitcoinclassic/pull/143/files
  4 2016-05-06 09:20:10 <jonasschnelli> Which essential is a drastic change in the security model (and not recommended)
  5 2016-05-06 09:29:39 <xenog> jonasschnelli: I am using btcd at the moment, and I have been syncing the blockchain for a day and a half, and I am thinking why should it take so long if most of the blockchain is buried under so much proof-of-work that it isn't even funny.
  6 2016-05-06 09:30:20 <jonasschnelli> xenog: verifying everything is bitcoin non-trust trust model
  7 2016-05-06 09:31:17 <xenog> You tell me that Bitcoin Core does that already? by default?
  8 2016-05-06 09:32:55 <phantomcircuit> xenog, btcd is just slow
  9 2016-05-06 09:33:36 <jonasschnelli> xenog: no. Bitcoin-Core does verify (almost) everything.
 10 2016-05-06 09:33:54 <jonasschnelli> Bitcoin-Core can do a full sync on a modern system <3h.
 11 2016-05-06 09:34:06 <jonasschnelli> xenog: how many RAM do you have?
 12 2016-05-06 09:34:12 <xenog> jonasschnelli: 8 GiB.
 13 2016-05-06 09:34:20 <jonasschnelli> CPU?
 14 2016-05-06 09:34:25 <jonasschnelli> SSD?
 15 2016-05-06 09:34:47 <xenog> jonasschnelli: SSD, Core i7-5600U.
 16 2016-05-06 09:34:51 <jonasschnelli> you can pass `-dbcache=4000`
 17 2016-05-06 09:34:58 <jonasschnelli> And you should be able to sync in about 4h.
 18 2016-05-06 09:35:07 <jonasschnelli> (your system will essentially be unusable during this time)
 19 2016-05-06 09:35:35 <jonasschnelli> make sure you use bitcoin-core > 0.12
 20 2016-05-06 09:35:37 <xenog> jonasschnelli: thanks, I'm on that system right now.
 21 2016-05-06 09:35:38 <jonasschnelli> >=
 22 2016-05-06 09:35:50 <xenog> I'm looking for an address index.
 23 2016-05-06 09:36:07 <jonasschnelli> I think there is an old PR for core.
 24 2016-05-06 09:36:09 <xenog> I know there is btcdrak's implementation.
 25 2016-05-06 09:36:28 <jonasschnelli> Right. Do you need an index for all addresses or just some you are interested in?
 26 2016-05-06 09:36:31 <xenog> But in btcd it is integrated in the main codebase. I will test both.
 27 2016-05-06 09:36:36 <xenog> All addresses.
 28 2016-05-06 09:36:38 <jonasschnelli> address index requires a lot of space I guess.
 29 2016-05-06 09:36:45 <xenog> I already developed something that allows me to index some.
 30 2016-05-06 09:37:00 <jonasschnelli> Because you can use the wallet for that by using importaddress
 31 2016-05-06 09:37:01 <xenog> I don't mind much about space, I can build the index then move it to external HDD.
 32 2016-05-06 09:37:13 <jonasschnelli> sure
 33 2016-05-06 09:37:51 <xenog> I am considering adding the option to Haskoin to use a full address index from either Bitcoin Core or btcd.
 34 2016-05-06 09:39:27 <xenog> Since Bitcoin Core does not have one yet, then btcdrak's patch is something I will be looking at, but I started with btcd because I like the API.
 35 2016-05-06 09:41:34 <jonasschnelli> xenog: hows the API different?
 36 2016-05-06 09:42:40 <xenog> One WebSocket connection and JSON-RPC requests traveling back-and forth AFAIK.
 37 2016-05-06 09:43:29 <xenog> (It was my friend who checked this for me, I haven't looked at the details yet)
 38 2016-05-06 09:43:31 <jonasschnelli> xenog: traveling back-and forth?
 39 2016-05-06 09:44:15 <jonasschnelli> WebSockets would make it difficult to use reverse proxies if you want to access your RPC server from a network with different security policies.
 40 2016-05-06 09:44:20 <xenog> Well, if you have a websocket open with a server, you get notifications through that same socket.
 41 2016-05-06 09:44:42 <jonasschnelli> Yes. This is nice.
 42 2016-05-06 09:44:48 <xenog> I know Bitcoin Core has ZeroMQ these days, which is even better.
 43 2016-05-06 09:44:50 <jonasschnelli> I have written a similar interface for that: https://github.com/bitcoin/bitcoin/pull/7949
 44 2016-05-06 09:45:04 <jonasschnelli> Long Poll. You have the benefits of a WebSockets but not the downsided. :)
 45 2016-05-06 09:45:35 <xenog> Yeah, long poll is kind of a hack though, I like it because it traverses proxies well.
 46 2016-05-06 09:45:39 <jonasschnelli> ZMQ is nice. But one channel that solves both directions makes stuff much easier.
 47 2016-05-06 09:45:59 <jonasschnelli> Long poll is not that hacky. Its also in use for Apples Push Notification Server.
 48 2016-05-06 09:46:13 <jonasschnelli> IMO its very robust and surives tons of network connections.
 49 2016-05-06 09:46:45 <xenog> I should take a look at that again.
 50 2016-05-06 09:47:00 <xenog> ZeroMQ is great, but it isn't web-friendly.
 51 2016-05-06 09:47:29 <xenog> AFAIK you cannot open ZeroMQ sockets from a JS app running on a browser (only from Node.js and such).
 52 2016-05-06 09:48:41 <jonasschnelli> ZeroMQ is more for local or LAN notifications.
 53 2016-05-06 09:49:05 <jonasschnelli> The long poll PR would allow to connect a remote client (Smartphone or detatched GUI) over just one channel.
 54 2016-05-06 09:49:09 <xenog> It works great on the Internet too, as long as you are not running on the web.
 55 2016-05-06 09:49:22 <jonasschnelli> One could add a apache/nging reverse proxy with cert or http-digest auth.. done!
 56 2016-05-06 09:49:39 <jonasschnelli> *nginx
 57 2016-05-06 09:50:20 <xenog> I'll look at it, I hope it gets merged. ;)
 58 2016-05-06 09:50:36 <xenog> Then it will make it to btcdrak's addrindex branch eventually.
 59 2016-05-06 09:52:15 <xenog> I would love to see address indices make it to Bitcoin Core. It's pretty essential to build wallets.
 60 2016-05-06 09:53:29 <jonasschnelli> xenog: why is it essential?
 61 2016-05-06 09:53:46 <jonasschnelli> You can build a wallet by filtering blocks
 62 2016-05-06 09:53:58 <xenog> It takes forever, the server has to scan the whole blockchain.
 63 2016-05-06 09:54:01 <jonasschnelli> xenog: it would only be essential if you want to build a wallet centralized API
 64 2016-05-06 09:54:02 <xenog> It is quite impractical.
 65 2016-05-06 09:54:24 <xenog> Sure you can optimize by only scanning the blocks from which point your wallet was created, but that isn't helping when you have had a wallet for a year.
 66 2016-05-06 09:54:43 <xenog> I like the idea of Bitcoin Core as a border router for the Bitcoin network.
 67 2016-05-06 09:54:52 <jonasschnelli> If you havent used your SPV wallet for a year, it migh require 1-2 mins syncing. Right.
 68 2016-05-06 09:55:05 <xenog> I install and sync it in some server, then I build clients that use it.
 69 2016-05-06 09:55:18 <jonasschnelli> But the address index would introduce serval security risks.
 70 2016-05-06 09:55:29 <xenog> Maybe my Haskoin code is too slow, because it requires a tad more than 1-2 minutes.
 71 2016-05-06 09:55:56 <jonasschnelli> It depends on the node you are connected to and your network connection
 72 2016-05-06 09:56:15 <jonasschnelli> But you essentially filter 6*24*365 blocks...
 73 2016-05-06 09:56:31 <jonasschnelli> which is mostly CPU/disk work for you node you requesting the filtered blocks from
 74 2016-05-06 09:56:54 <xenog> We are talking 50 GiB of block data for a year of full blocks, which is an upper-bound.
 75 2016-05-06 09:57:23 <jonasschnelli> Yes. I agree it's not an ideal modal.
 76 2016-05-06 09:57:32 <xenog> It is CPU/disk work (not much CPU since murmur3 is quite lightweight) for the node, but it still takes time.
 77 2016-05-06 09:57:41 <xenog> It doesn't help if you want to run a stateless wallet for example.
 78 2016-05-06 09:58:26 <xenog> Maybe something over the web that you may want to sync as quickly as possible.
 79 2016-05-06 09:58:30 <jonasschnelli> xenog: right. But an address index would require to fully trust the node you are connected to. Similar then electrums/stratums model
 80 2016-05-06 09:58:47 <xenog> Well, Stratum gives you merkle proofs if you want them, same as Bitcoin Core.
 81 2016-05-06 09:59:13 <jonasschnelli> Right. But still you should connect to serval nodes to make sure your not censored
 82 2016-05-06 09:59:32 <xenog> So say you sync the header chain, which doesn't take up much space, and then you get the transactions with corresponding merkle proofs, you are at the SPV level at that point.
 83 2016-05-06 10:00:07 <jonasschnelli> What if your nodes doesnt pass you a specific transaction (hold back / censor)?
 84 2016-05-06 10:00:13 <xenog> Sure, there is still the possibility of getting isolated.
 85 2016-05-06 10:00:33 <xenog> Yeah, that has happened to me in testnet. I have missed confirmations for transactions.
 86 2016-05-06 10:00:46 <jonasschnelli> censor resistance is important IMO. This is why I don't like SPV that much.
 87 2016-05-06 10:00:51 <xenog> Some nodes did not update their internal merkle filter fast enough or something.
 88 2016-05-06 10:01:54 <jonasschnelli> My favorite setup is: 1) trusted Core node with wallet, 2) remote client connected over RPC/TLS, 3) Core runs watch only wallet, 4) remote clients gets transactions over RPC, has privkeys locally and does signing there.
 89 2016-05-06 10:02:21 <jonasschnelli> The trusted core node could be a 50$ device next to your home WiFi router.
 90 2016-05-06 10:04:55 <xenog> I prefer: indexing node without wallet, clients connect over RPC. Node is not concerned with state pertaining to wallets that use it.
 91 2016-05-06 10:07:38 <jonasschnelli> xenog: Address index would require tons of space and your indexing probably 95% of data that will never be retrived.
 92 2016-05-06 10:07:54 <jonasschnelli> The 50$ device could run in pruning mode and require <3GB space
 93 2016-05-06 10:08:20 <jonasschnelli> But right,.. it would track state.
 94 2016-05-06 10:08:30 <jonasschnelli> It's like your "bank at home".
 95 2016-05-06 10:09:05 <xenog> I can make a node public over a "safe" RPC interface creating a network of nodes a-la Stratum that provide indexing services. Those are the border routers to the Bitcoin network.
 96 2016-05-06 10:10:52 <xenog> Another alternative is exposing an optional address index via Bitcoin protocol, and optionally being able to connect to it via WebSocket. That would even allow web wallets to become P2P capable.
 97 2016-05-06 10:18:59 <xenog> Anyway jonasschnelli, thank you for the info!
 98 2016-05-06 10:19:39 <jonasschnelli> xenog: Thanks for the chat and you infos! Seems you are also interested in the "perfect" wallet use-case.
 99 2016-05-06 10:20:04 <xenog> jonasschnelli: I'm one of the authors of Haskoin, the Haskell wallet.
100 2016-05-06 10:20:30 <xenog> jonasschnelli: I'm always looking for the best way to do it.
101 2016-05-06 10:21:31 <xenog> jonasschnelli: all security, usability, speed, scalability, realibility. Somehow it seems that today one has to choose some among those. I would like to have everything.
102 2016-05-06 10:21:46 <jonasschnelli> yes. Indeed!
103 2016-05-06 19:28:12 <wallet42> jonasschnelli: i saw you were looking at encrypting p2p traffic. did you know abotu https://curvecp.org/ ? not sure if it's still developed