1 2015-08-11 01:48:51 <dcousens> What is the overall consensus on breaking bitcoin-qt away from consensus code and bitcoind code? (repo wise)
  2 2015-08-11 01:57:44 <gmaxwell> splitting things into seperate repositories is a disaster for ordinary users of the software who go and build it; other projects who've done that (or started that way) have gone and merged things later.
  3 2015-08-11 01:58:07 <gmaxwell> It's a great way to ensure no one uses your software; and by that I mean it soulds like something that bitcoin core is likely to do.
  4 2015-08-11 02:00:37 <kanzure> yes while we're at it let's get rid of libraries and dependencies
  5 2015-08-11 02:00:41 <kanzure> since nobody understands dependencies
  6 2015-08-11 02:01:27 <kanzure> also git submodules. i've received lots of complaints about those...
  7 2015-08-11 02:02:26 <gmaxwell> There is a real trade-off in taking another dependency esp one that isn't bundled by common systems by default; in that you n-fold increase the number of places where someone trying to grab and build the package can get stuck.
  8 2015-08-11 02:24:24 <dcousens> gmaxwell: the issues you just outlined seem more in line with the difficulties of package management in C++ than any real problem?   Modularity has long been a touted proponent in making software more maintainable.
  9 2015-08-11 02:24:24 <dcousens> So you're opinion is that you'd be against moving the wallet functionality to another repository?   Where do you draw that line in terms of related components?  Obviously libsecp256k1 is over the line,  but not the GUI vs bitcoind?
 10 2015-08-11 02:24:55 <dcousens> (Apologies if I'm misunderstanding the above)
 11 2015-08-11 02:25:42 <dcousens> " and by that I mean it sounds like something that bitcoin core is likely to do."  - sarcasm? Haha
 12 2015-08-11 02:42:56 <warren> dcousens: wallet in another repo might be appropriate, the concern about dependencies is for things like leveldb or openssl that have proven in the past to be hazards to consensus.
 13 2015-08-11 02:43:45 <dcousens> I wasn't talking about dependencies though,  I'm not sure why they came up other than the fact I guess the idea of another repo might mean something like bitcoin-qt would have dependencies on the the consensus code and bitcoind.
 14 2015-08-11 02:44:07 <warren> dcousens: (if someone goes through the necessary work of splitting the wallet into a SPV GUI + command line client that operates in a separate process from the core bitcoind which merely is a validation relay) ]
 15 2015-08-11 02:44:56 <warren> The wallet doesn't need to use the consensus code, it could rely on the bitcoind for that over some RPC, just nobody did the work to split it off yet.
 16 2015-08-11 02:46:02 <dcousens> Is that a commonly held opinion?  I'm just wondering if that kind of effort is worthwhile in light of it being rejected for reasons or dogma that I'm not aware of.
 17 2015-08-11 02:46:39 <warren> I think it's been desired for a long time.
 18 2015-08-11 02:47:05 <warren> It would improve security to have the private keys in a separate process from the thing that connects to other nodes.
 19 2015-08-11 03:02:25 <gmaxwell> dcousens: do not confuse putting software in another repository with modularity.
 20 2015-08-11 03:06:26 <dcousens> Its not entirely synonymous,  no,  but if it lowers the cognitive load required upon coming across a repository which has a source directory with over 800 files,   then,  I see no issue with modularity being as simple as a separation of concerns.
 21 2015-08-11 03:06:43 <dcousens> s/source/src\//
 22 2015-08-11 03:08:32 <dcousens> >770*
 23 2015-08-11 03:39:10 <Luke-Jr> dcousens: FWIW, I agree with you that we should make the software more modular, including moving the different logical components to their own repositories
 24 2015-08-11 03:40:22 <Luke-Jr> I've tried to encourage jonasschnelli to write his new wallet as a library, but alas, he is not it seems
 25 2015-08-11 03:42:10 <Luke-Jr> gmaxwell:  it's trivial to write scripts to download and build uncommon dependencies for your project, and we already even have that for Core
 26 2015-08-11 05:40:54 <jonasschnelli> Luke-Jr: the idea of having a library (wallet/core) is still in my mind.
 27 2015-08-11 05:41:24 <Luke-Jr> good to hear
 28 2015-08-11 05:41:28 <jonasschnelli> But changing all at once is hard to achieve.
 29 2015-08-11 05:41:50 <dcousens> jonasschnelli: why would it need to all be done at once?
 30 2015-08-11 05:42:03 <dcousens> Surely you can start with a single function and just add the build system, make sure htat works etc
 31 2015-08-11 05:42:09 <dcousens> Then *progressively* extract code.
 32 2015-08-11 05:42:15 <jonasschnelli> I already struggle with keeping everything rebased with the master and still find time to implement new stuff. :-)
 33 2015-08-11 05:42:35 <dcousens> jonasschnelli: the point is you wouldn't need to rebase that in line with other code
 34 2015-08-11 05:43:44 <dcousens> What would it take to get a new repo created now,  that we could at least start structuring out?  Even if its a while until we can actually start merging code into bitcoin/bitcoin that extracts anything
 35 2015-08-11 05:44:08 <jonasschnelli> Yeah... Sounds good in theory. But in practice is not so easy. I'd like to reuse the whole core base classes like key, hash, base58, etc... So even if I would write a library from the scratch, it would require bitcoin-core as dependency
 36 2015-08-11 05:45:03 <jonasschnelli> The wallet is really coupled with the core. Height / conflict calculations, etc.
 37 2015-08-11 05:45:23 <jonasschnelli> I don't want to end up with another weak SPV implementation.
 38 2015-08-11 05:45:36 <dcousens> jonasschnelli: is requiring bitcoin-core as a dependency a bad thing though? to start?
 39 2015-08-11 05:45:54 <dcousens> We can at least _play_ with the idea right?
 40 2015-08-11 05:46:05 <jonasschnelli> Why is this bad? Bitcoin core has been proven to be stable.
 41 2015-08-11 05:47:11 <jonasschnelli> If you compile Bitcoin-Core you automatically get libbitcoincore-common, etc.
 42 2015-08-11 05:47:31 <jonasschnelli> It just lacks some nice documentation and some config.h freedom.
 43 2015-08-11 05:47:54 <dcousens> To clarify,  I'm talking about extracting the wallet from bitcoin/bitcoin to say,  bitcoin/wallet   -  if bitcoin/wallet  needed bitcoin/bitcoin entirely to start, and consisted *solely* as a the build system with `-enable-wallet`,  that is a start, IMHO
 44 2015-08-11 05:48:52 <dcousens> and from that point,  you could start removing code and deconstructing bitcoin/bitcoin as a wallet
 45 2015-08-11 05:48:58 <jonasschnelli> My primary goal is not to extract the wallet (but it one of the goals, sure). My primary goal is to have a good user experience while using a full node wallet
 46 2015-08-11 05:49:38 <jonasschnelli> I just don't want to end up maintaining my own set of base classes (net, key, etc.)
 47 2015-08-11 05:49:40 <dcousens> jonasschnelli: granted,  but that goal could achieved much faster if this repo wasn't so monolithic and a large technical burden for people to help with tasks which maybe aren't as critical as consensus
 48 2015-08-11 05:50:25 <jonasschnelli> Yes. That's why corewallet is a fork https://github.com/jonasschnelli/bitcoin
 49 2015-08-11 05:50:49 <jonasschnelli> I don't need to keep it rebase with the master... But I do.
 50 2015-08-11 05:50:49 <phantomcircuit> lots of stuff would be easier if the core data structures were in their own library
 51 2015-08-11 05:51:32 <jonasschnelli> phantomcircuit: the core stuff is already available as library (libbitcoincore-common)
 52 2015-08-11 05:51:47 <dcousens> jonasschnelli: granted that is great to see (wasn't aware of your work),  however,  if its not in line with simultaneously removing code from bitcoin/bitcoin,  its not really helping the situation
 53 2015-08-11 05:53:20 <jonasschnelli> dcousens: my idea is to go along with the fork, convince users to use it, maybe add Bitcoin-core as git submodule, remove wallet from Bitcoin-core (could take serval years).
 54 2015-08-11 05:53:51 <jonasschnelli> There is no hurry in removing the current wallet from bitcoin-core IMO
 55 2015-08-11 05:55:01 <jonasschnelli> I still think if we all can download 10GB 1080p movies and play 3D games, we should really be able to run a full node in background (of a standard PC).
 56 2015-08-11 05:55:36 <jonasschnelli> If there would be a SPV mode during throttled catchup,...
 57 2015-08-11 05:55:56 <phantomcircuit> oh i see it's just not the default
 58 2015-08-11 05:56:12 <dcousens> IMHO anything that could be considered a block on allowing technical innovation is very important to bitcoin-core,  and the monolithic structure of bitcoin-core as it stands means innovating in the space comes down to the choice of rolling your own (or using non-core) sub-optimal node to do 1 or 2 things only,  or forking core with a few features which
 59 2015-08-11 05:56:12 <dcousens> quickly becomes a huge maintenance burden.
 60 2015-08-11 05:56:32 <phantomcircuit> jonasschnelli, an spv wallet that either explicitly trusted a full node
 61 2015-08-11 05:56:39 <phantomcircuit> or which over time ran the fullnode validation
 62 2015-08-11 05:56:43 <phantomcircuit> would be very nice
 63 2015-08-11 05:57:22 <jonasschnelli> phantomcircuit: yes. People come up with this Idea often. The problem there is, how do you secure the channel between the full node and your SPV client?
 64 2015-08-11 05:57:23 <phantomcircuit> jonasschnelli, ha i hadn't noticed --enable-shared
 65 2015-08-11 05:57:51 <jonasschnelli> We can use SSH tun. But can you non experience friend do this also?
 66 2015-08-11 05:58:30 <jonasschnelli> SPV over 8333 p2p (even to a trusted full node) is in practice hard to achieve.
 67 2015-08-11 05:58:54 <jonasschnelli> With a goal to avoid risks of MITM
 68 2015-08-11 05:59:23 <phantomcircuit> something tells me we can come up with a sane protocol for authenticating (note not encrypting) the link
 69 2015-08-11 06:00:32 <phantomcircuit> a slowly upgrading wallet is more interested tbh
 70 2015-08-11 08:42:15 <wumpus> authenticating the protocol has been discussed may times. The rough idea would be to give each node a ECDSA key (or multiple), use that to negotiate a secret right after connection, add a MAC too each packet
 71 2015-08-11 08:42:53 <wumpus> of course, without encrypting, this does not give any privacy in the wallet->trusted node case
 72 2015-08-11 08:43:32 <wumpus> but it with help with whitelisting/trusted peering to make the network more resistant to partition attacks
 73 2015-08-11 08:44:03 <wumpus> and other DoS
 74 2015-08-11 08:45:44 <dcousens> @wumpus what is a partition attack for those unaware of the term :)
 75 2015-08-11 08:46:07 <wumpus> dcousens: e.g an ISP isolating you
 76 2015-08-11 08:46:46 <dcousens> how would whitelisting help in that case?  (assuming by isolation, you mean, the ISP dropping peers/stopping discovery?)
 77 2015-08-11 08:47:12 <wumpus> if you know you're connected to a friend on the other side of the world, you know you can't be isolated to a silly little network
 78 2015-08-11 08:47:57 <wumpus> though right now you can do the same through tor and hidden services of course, but it could be done with less overhead I guess if it's directly in the protocol
 79 2015-08-11 08:48:25 <dcousens> so,  essentially just adding the equivalent of a known_hosts file?
 80 2015-08-11 08:48:52 <dcousens> IP's change, most peoples nodes aren't static?
 81 2015-08-11 08:49:01 <dcousens> (or are they?)
 82 2015-08-11 08:49:38 <dcousens> or is this just whitelisting for the case where they have a static ip, and you want to auth it?
 83 2015-08-11 08:50:13 <wumpus> sort of a known_hosts file, yes. It would introduce a F2F component. I don't think this needs any requirements with regard to static IPs - nodes would recognize each other by e.g. public key hash.
 84 2015-08-11 08:50:43 <wumpus> whitelisting with static IPs is already possible but doesn't protect you from someone messing with the network, who has full control over IPs
 85 2015-08-11 08:51:10 <dcousens> right, so this suggested BIP would help stop that,  but it would be hard to detect if that node is simply unreachable vs being blocked though? no?
 86 2015-08-11 08:51:44 <wumpus> no need to distinguish between those cases, if you can't reach any of your "friends", something is wrong and you need to be very wary
 87 2015-08-11 08:52:10 <dcousens> so I guess,  its a matter of having lots of friends? :P
 88 2015-08-11 08:52:31 <wumpus> yes, or a few very reliable ones :p that part is politics, I don't care :)
 89 2015-08-11 08:53:26 <wumpus> I mean 'friends' in this case could also be a company that you're doing business with
 90 2015-08-11 08:54:27 <dcousens> is this worth it at the bitcoin protocol level?
 91 2015-08-11 08:54:29 <wumpus> then if, say,  the network comes under attack you can keep the connections to those and e.g. drop other incoming connections
 92 2015-08-11 08:54:58 <wumpus> I don't know?
 93 2015-08-11 08:55:08 <dcousens> if whitelisting is optional,  per say,  it doesn't really add strength except to those that might have been able to verify each other in other ways?
 94 2015-08-11 08:55:44 <dcousens> though MACs would probably be useful to ensure your never being MITM?  But,  SSL then?
 95 2015-08-11 08:57:19 <wumpus> I'm not actually trying to argue for this idea, just that it has been discussed quite a few time, and would provide bidirectional authentication between nodes , when that is desired
 96 2015-08-11 08:58:45 <warren> wumpus: I'm personally in favor of optional trusted peering
 97 2015-08-11 08:59:05 <warren> as long as untrusted remains very common
 98 2015-08-11 08:59:19 <wumpus> warren: yes - there are quite some users that do
 99 2015-08-11 09:00:11 <wumpus> it's potentially irrelevant whether it's part of the P2P protocol or say, a layer on top, I think the only issue is that there is no standard way to do it
100 2015-08-11 09:01:22 <wumpus> warren: yes, what they use is up to the user... a lot of people likely won't bother setting this up, but in the case that you're a high-profile target it may be worth it
101 2015-08-11 09:03:43 <avocado_> dcousens: SSL is undesirable, simply due to the amount of weight an implementation carries. ideally bitcoin nodes would have no dependency on the massive OpenSSL library at all. you can approximate this setup if you want today using the whitelist port and stunnel.
102 2015-08-11 09:04:59 <wumpus> and in the case encryption is added (which isn't too hard if you already have authentication, I guess), it could be used for private connection from SPV node to trusted full node
103 2015-08-11 09:05:03 <avocado_> stunnel just lets you pipe arbitrary TCP connections through SSL, with pinning and whatever else if you fancy. it's cludgy and needs a lot of setup time but it does work fine with Bitcoind.
104 2015-08-11 09:05:41 <wumpus> yes, stunnel, ssh tunnels, tor, it's already possible to layer the P2P protocol over quite a few things accomplishing the same
105 2015-08-11 09:06:02 <wumpus> I have a bitcoin node on cjdns
106 2015-08-11 09:07:15 <avocado_> would bitcoin core accept a patch which adds a hyperboria DNS seed, if it were by default disabled at compile time?
107 2015-08-11 09:07:17 <wumpus> but as long as there is no standard for doing so, it will remain kludgy to set up, and it hampers the network effect
108 2015-08-11 09:07:48 <wumpus> avocado_: sure, I don't see why that would be rejected
109 2015-08-11 09:09:45 <wumpus> maybe even some hardcoded  seeds like we have for ipv4,ipv6,tor
110 2015-08-11 09:38:17 <moa> wumpus: I have a bitcoin node on cjdns
111 2015-08-11 09:38:31 <moa> interesting have you documented this anywhere?
112 2015-08-11 09:38:50 <avocado_> wumpus: I thought it might be a bit niche.
113 2015-08-11 09:41:34 <wumpus> moa: no, I don't usually get around to documenting my experiments :) the address is fcbd:1867:d1dc:1a3e:6fd1:d071:b62c:30fb
114 2015-08-11 09:43:07 <moa> thnx
115 2015-08-11 09:47:37 <wumpus> Adlai: it's a bit niche. But highly relevant, and sometimes it helps if niches cooporate to form a larger niche :)
116 2015-08-11 09:52:55 <wumpus> if anyone else is running a bitcoin node on cjdns or wants to peer up there, let me know
117 2015-08-11 12:55:44 <kanzure> jonasschnelli: if full validation takes a few weeks then should we really recommend users to make transactions in the mean time?
118 2015-08-11 12:56:29 <jonasschnelli> kanzure: full validation can take a few weeks if you would throttle it and run it in background... WHILE you use SPV during that time...
119 2015-08-11 12:56:41 <jonasschnelli> imo still be better than running a standard SPV client
120 2015-08-11 12:57:06 <kanzure> ah okay, temporary spv sounds okay to me
121 2015-08-11 12:59:01 <jonasschnelli> One usability problem of bitcoin-core is that you can't use it immediately. If one presses the button "I WHISH FULL SECURITY/PRIVACY", he might need to wait until the client is fully in sync...
122 2015-08-11 13:00:14 <jonasschnelli> otherwise there could be a warning ("you blockchain is not in sync, you are using SPV, warning blabla...")
123 2015-08-11 13:59:24 <andytoshi> i don't know that there's any user-comprehendable way to say "warning, you have only SPV security"
124 2015-08-11 14:03:34 <harding> andytoshi: "confirmations less reliable"?
125 2015-08-11 14:04:20 <harding> andytoshi: or maybe, "the bitcoins you received may not actually exist"
126 2015-08-11 14:04:34 <morcos> Warning: You have not validated the blockchain, you are trusting the peers you are currently connected to.
127 2015-08-11 14:05:23 <kanzure> perhaps include a "trust mode" where it silently migrates you to paypal
128 2015-08-11 14:06:05 <morcos> Thats a good idea for the low fee mode
129 2015-08-11 14:07:13 <andytoshi> i don't wanna bikeshed, but any of those sound fine except "your money might not exist". you just need something that sounds non-threatening because 99% of your users will not want to take the time understand its implications
130 2015-08-11 14:08:51 <kanzure> oh right, the other provision was throttling, i forgot this
131 2015-08-11 14:14:05 <jonasschnelli> andytoshi: SPV / full-node could be like the https lock in the browser. People also doesn't know what really happens,... but it looks save.
132 2015-08-11 14:16:55 <mn3monic> hello, I am working on a blockchain parser, and I've encountered old fashion multisig transactions. I am far from understanding how the scriptPubKey.addresses array (given the json object) is determined.
133 2015-08-11 14:17:11 <mn3monic> Three samples I am unable to understand: http://paste.ofcode.org/Aq4t8Cj4Wp8pFGAfKB7akS
134 2015-08-11 14:17:24 <jgarzik> Well SPV should at least test the block headers it receives...  that is not trusting connected peers 100%... those peers also must provide valid block PoW, which cannot simply be faked by an attacker without real hash power
135 2015-08-11 14:18:10 <mn3monic> In the linked vouts the script looks the same to me, but the result of the rawtransactions decoder is different :(
136 2015-08-11 14:25:43 <jonasschnelli> jgarzik: Agreed. SPV also offers a form of validation.
137 2015-08-11 14:50:33 <morcos> wumpus: i'm going to rebase 6470 on top of 6519 so i get the debug logging right in it.  Let me know if you disagree?
138 2015-08-11 14:51:24 <andytoshi> mn3monic: https://en.bitcoin.it/wiki/Script ... you've got a bit of reading to do. i think that also links to the Transactions page which you should read
139 2015-08-11 15:23:38 <wumpus> morcos: sounds good to me
140 2015-08-11 15:25:41 <mn3monic> thank you for your time andytoshi, but the wiki didn't helped me understanding how the address array is populated. a more precise reference would help me a lot, thank you !
141 2015-08-11 15:26:04 <wumpus> morcos: or, maybe delay it a bit - going to fix the comment and merge it
142 2015-08-11 15:58:29 <andytoshi> mn3monic: there is the bitcoind source code, or possibly the Address wiki page. if you ask me in #bitcoin i'll say a bit more
143 2015-08-11 16:03:13 <mn3monic> anytoshi: asked in #bitcoin, thanks
144 2015-08-11 19:42:28 <instagibbs> Does anyone know of a list of hidden service nodes for testnet?
145 2015-08-11 19:43:34 <instagibbs> or just a single one? Just need to connect to one for a bit of testing
146 2015-08-11 19:46:07 <phantomcircuit> instagibbs, one sec
147 2015-08-11 19:46:53 <helo> instagibbs: b6fr7dlbu2kpiysf.onion
148 2015-08-11 19:47:03 <helo> oh, testnet -.-
149 2015-08-11 19:47:20 <instagibbs> yeah sorry :) I have a mainnet one running, but need to try testnet for a bit more safety
150 2015-08-11 19:47:22 <helo> i always forget to restart it
151 2015-08-11 19:47:26 <phantomcircuit> huh i only configured these with elements alpha on a hidden service
152 2015-08-11 19:47:35 <phantomcircuit> instagibbs, how about just a regular ip?
153 2015-08-11 19:47:45 <instagibbs> Oh true... that will work nevermind I'm good
154 2015-08-11 19:48:46 <instagibbs> Actually I'd still take one if anyone stumbles upon one
155 2015-08-11 19:49:51 <helo> well, mine should be up now but behind
156 2015-08-11 19:50:06 <phantomcircuit> instagibbs, bbgw7gpjpko5yvuh.onion
157 2015-08-11 19:50:23 <instagibbs> thanks, behind is fine
158 2015-08-11 19:50:49 <phantomcircuit> bonus elements alpha node included!
159 2015-08-11 19:51:29 <instagibbs> Wow! (I actually have no idea how the alpha stuff interacts with regular bitcoin stuff but I'm sure it'll make me feel taller or handsome or something)
160 2015-08-11 19:53:06 <phantomcircuit> instagibbs, elements alpha is a sidechain for testnet
161 2015-08-11 19:53:15 <phantomcircuit> all the coins are 1:!
162 2015-08-11 19:53:17 <phantomcircuit> 1:1
163 2015-08-11 19:54:07 <instagibbs> Right I more didn't realize that they appear as regular full nodes to testnet nodes, but I guess that makes sense.
164 2015-08-11 19:54:31 <phantomcircuit> instagibbs, oh no i was just being silly
165 2015-08-11 19:54:43 <phantomcircuit> there's an alpha node on port 4242 as well as a testnet node on 18333
166 2015-08-11 19:55:10 <instagibbs> Oh thanks for confusing me heh
167 2015-08-11 19:55:22 <drazisil> phantomcircuit: you completely lost me. what's an alpha node?
168 2015-08-11 19:56:57 <phantomcircuit> this is probably getting offtopic now but
169 2015-08-11 19:56:59 <phantomcircuit> https://github.com/ElementsProject/elementsproject.github.io
170 2015-08-11 20:00:01 <drazisil> thanks *takes one look and files for later reading* :)
171 2015-08-11 22:21:13 <zxcvbnmasd> There is a bug
172 2015-08-11 22:21:46 <zxcvbnmasd> can send bitcoin without having bitcoin
173 2015-08-11 22:22:11 <warren> zxcvbnmasd: ok, send me bitcoin to prove it?
174 2015-08-11 22:22:21 <zxcvbnmasd> give me address
175 2015-08-11 22:22:28 <zxcvbnmasd> give me your address
176 2015-08-11 22:22:33 <zxcvbnmasd> ill send you sample
177 2015-08-11 23:03:11 <BlueMatt> sfhousing@mattcorallo.como%NGz{gt+qH_|#_|:`_H.4m/h
178 2015-08-11 23:05:48 <warren> BlueMatt: damn, that's my password too
179 2015-08-11 23:06:00 <BlueMatt> damn password managers and their auto-typing
180 2015-08-11 23:06:47 <andytoshi> BlueMatt: what manager do you use? i have keepassx and i don't know how to make it autotype
181 2015-08-11 23:07:16 <BlueMatt> andytoshi: keepassx: use control-v
182 2015-08-11 23:07:51 <andytoshi> oh, cool, thx
183 2015-08-11 23:18:03 <gmaxwell> Cory: You were working on the 'bursty' sync speed (phantomcircuit had also observed this issue)-- did you get anywhere on it?