1 2015-05-03 00:40:08 <jgarzik> jonasschnelli, joshc: just pushed out most of what is needed. OSX needs a better configure.ac check for the argp library - required on OSX, not required on linux
2 2015-05-03 00:40:22 <jgarzik> it should build, assuming you have argp, or fail mysteriously if not
3 2015-05-03 00:48:46 <phantomcircuit> jgarzik, lol fail mysteriously
4 2015-05-03 00:49:09 <phantomcircuit> sadly not that far off from state of the art in build systems...
5 2015-05-03 02:16:04 <goosoodude> Hey
6 2015-05-03 02:40:14 <jgarzik> jonasschnelli, joshc: just pushed out most of what is needed. OSX needs a better configure.ac check for the argp library - required on OSX, not required on linux
7 2015-05-03 02:40:29 <jgarzik> it should build, assuming you have argp, or fail mysteriously if not
8 2015-05-03 02:48:53 <phantomcircuit> jgarzik, lol fail mysteriously
9 2015-05-03 02:49:15 <phantomcircuit> sadly not that far off from state of the art in build systems...
10 2015-05-03 02:51:02 <xperia> hi all. i just pulled the latest code from github and compiled it on ubuntu linux but it looks like the source code is broken and has some bugs. whenever i try to start bitcoind i get a segmentation fault.
11 2015-05-03 02:53:22 <Luke-Jr> xperia: the master branch (and this channel) are intended for developers, not really support. if you want something to use, download a release..
12 2015-05-03 02:56:09 <xperia> Luke-Jr: i just wanted to report it. I find it strange that people upload code to the git repo that breaks the Programm itself. Would have expected that they at least tested it localy before they pushed it to the public.
13 2015-05-03 02:56:40 <Luke-Jr> everything on git master has passed testing, so it is probably something unusual with your build
14 2015-05-03 02:56:49 <Luke-Jr> xperia: if you can identify the cause, that might be useful to know
15 2015-05-03 02:57:03 <Luke-Jr> does test_bitcoin work for you?
16 2015-05-03 02:57:45 <xperia> okey will try to rebuild it full clean and report it then.
17 2015-05-03 03:00:51 <xperia> Luke-Jr: yrs test_bitcoin works ! This is the Output of it => unknown location(0): fatal error in "AlertApplies": memory access violation at address: 0x00000001: no mapping at fault address
18 2015-05-03 03:01:25 <Luke-Jr> curious
19 2015-05-03 03:06:28 <leakypat> In the 0.10.0 > after an unexpected shutdown, need an entire reindex?
20 2015-05-03 03:18:13 <xperia> have to go. see you all later. maybe in the meantime the bug that caouses the master branch to crash will be fixed. will check it out later again.
21 2015-05-03 03:18:19 <leakypat> Or is this just a one time thing after upgrade and the first time it has needed to reindex ?
22 2015-05-03 03:19:24 <Luke-Jr> leakypat: 0.10.x in general doesn't seem to survive unclean shutdowns
23 2015-05-03 03:19:34 <Luke-Jr> it's not a planned reindex, no
24 2015-05-03 03:39:25 <gmaxwell> Luke-Jr: wait; what are you talking about 0.10.1 should be absolutely fine across unclean shutdowns _generally_; though there are some exceptions and 0.10.1 should be strictly better than 0.8.x and 0.9x in that regard. This is not true for 0.10.0 however.
25 2015-05-03 03:41:14 <gmaxwell> Unfortunately, google leveldb is not very durable in general.
26 2015-05-03 03:41:31 <gmaxwell> See also this paper: https://www.usenix.org/conference/osdi14/technical-sessions/presentation/pillai
27 2015-05-03 03:45:00 <Luke-Jr> gmaxwell: I don't think I have ever seen an unclean shutdown that 0.10.x or master managed to recover from.
28 2015-05-03 03:45:36 <Luke-Jr> the ARM USB stick is at 90bef6638fc689c13924fc17d115d5866b425e6c, with at least 2 or 3 of those failures
29 2015-05-03 03:46:14 <Luke-Jr> actually, I take it back. I did see one: it recovered after an OOM killer attack
30 2015-05-03 03:46:56 <Luke-Jr> but that is an unusual failure mode IMO
31 2015-05-03 04:10:57 <phantomcircuit> Luke-Jr, i regularly killall -9 bitcoind
32 2015-05-03 04:11:05 <phantomcircuit> i've only had one failure to restart
33 2015-05-03 04:12:02 <Luke-Jr> hm, I guess this suggests the kernel/disk play some role in the non-recovery scenario? ie, out-of-order writing or something
34 2015-05-03 04:14:01 <phantomcircuit> Luke-Jr, hmm i haven't tried pulling the power actually
35 2015-05-03 04:14:25 <phantomcircuit> so maybe out of order writes combined with low memory results in weird failures
36 2015-05-03 04:15:58 <Luke-Jr> it would be interesting if hard drive controllers behave in a way that makes things recover nicer. in the ARM case I've mostly experienced this with, it's just a microSD card
37 2015-05-03 04:16:10 <goosoodude> Hey
38 2015-05-03 04:17:17 <gmaxwell> your failures there are all out of memory ones.
39 2015-05-03 04:17:45 <gmaxwell> I left a recent RC in a kill -9 loop for a week without corruption.
40 2015-05-03 04:18:04 <gmaxwell> Luke-Jr: wrt kernel/disk; go see the paper I cited. :)
41 2015-05-03 04:19:24 <Luke-Jr> not all, the USB stick came out at least a few times, one of which had the uSD card lost on my floor <.<
42 2015-05-03 04:21:59 <Luke-Jr> gmaxwell: that paper sounds scary XD
43 2015-05-03 04:27:30 <gmaxwell> Yea, its an everythign is broken paper.
44 2015-05-03 04:28:11 <gmaxwell> I mean, we knew that leveldb wasn't very durable; phantomcircuit has pointed out a bunch of things it does stupidly that mean it can't be; that they found silent misbehavior is news to me and more concerning.
45 2015-05-03 04:28:32 <gmaxwell> I guess its concrete evidence that LMDB has better durrability.
46 2015-05-03 04:35:27 <phantomcircuit> gmaxwell, so on that front, i need a database for.. stuff.. that has strong durability and consistency properties
47 2015-05-03 04:35:55 <phantomcircuit> it's entirely append only so it's basically a journal
48 2015-05-03 04:36:19 <phantomcircuit> i was thinking of defining the format and writing something which would be about half of writing a trivial k/v db
49 2015-05-03 04:36:44 <gmaxwell> phantomcircuit: extfs3 in data journaling mode (see paper) and append data. :) ... I mean do you actually need random access? if not...
50 2015-05-03 04:38:22 <phantomcircuit> gmaxwell, nope
51 2015-05-03 04:38:33 <phantomcircuit> (note leveldb style databases dont either)
52 2015-05-03 04:39:02 <esso> What would you say is the target audience of Bitcoin core? Like what kind of people should be using Bitcoin core instead of a web wallet or some other type of wallet? The techies, brand new users, etc.
53 2015-05-03 04:39:26 <phantomcircuit> esso, everybody using bitcoin should be using bitcoin core...
54 2015-05-03 04:39:54 <kadoban> esso: Well, nobody should be using a web wallet, aka an IOU wallet.
55 2015-05-03 04:41:07 <hulkhogan_> alternatively, there is no 'audience', and the core is simply the reference implementation for the bitcoin protocol
56 2015-05-03 04:41:16 <esso> kadoban: What about a light weight client like electrum?
57 2015-05-03 04:41:39 <kadoban> esso: That's not a web wallet. I personally like electrum, especially for new people.
58 2015-05-03 04:42:08 <esso> hulkhogan_: So you would push for a wallet that every single user should be comfortable with?
59 2015-05-03 04:42:44 <hulkhogan_> esso: i wouldn't push for anything, there are plenty of incentives for someone else to provide that service already
60 2015-05-03 04:43:03 <hulkhogan_> a stable core is hard enough..
61 2015-05-03 04:43:13 <esso> kadoban: I'm aware. I'm looking for opinions on all types of wallets compared to Bitcoin core. I'm doing some UX revision and defining an audience is important.
62 2015-05-03 04:43:39 <kadoban> Probably can't help you there, personally.
63 2015-05-03 04:43:43 <esso> hulkhogan_: haha very true.
64 2015-05-03 04:43:59 <esso> kadoban: thanks for answer
65 2015-05-03 04:44:36 <gmaxwell> esso: Bitcoin Core's GUI wallet should be usable by all users; though it should not shy away from power user features at the expense of being being a little easier for total newbies.
66 2015-05-03 04:44:44 <esso> kadoban: I'm thinking the best bet is to appeal to everyone whether it's an experience user or brand new beginner
67 2015-05-03 04:45:46 <gmaxwell> esso: There isn't a fundimental conflict; the basic applications for a GUI wallet are the same for all users. It's okay to have more advanced features hidden a bit; but in general we should perfer security and good practices slightly to the utmost ease, and we shouldn't eschew a powerful feature because it might confuse a newbie (but its okay to hide it behind a setting)
68 2015-05-03 04:46:03 <esso> gmaxwell: I completely agree. It's also important to make sure the standard features that everyone uses do not get overshadowed by the more powerful features.
69 2015-05-03 04:46:30 <gmaxwell> yea, I agree. I think it's possible to balance that; mostly by tucking powerful features away where they won't bother people who don't need them.
70 2015-05-03 04:47:00 <gmaxwell> E.g. the most powerful interface to the GUI is the "CLI"-like debug console, but you'll never hear of a random user accidentally ending up with it and getting confused. :)
71 2015-05-03 04:47:26 <hulkhogan_> iirc there was some kind of checkbox setting for 'advanced mode' features in -qt ui, isnt there?
72 2015-05-03 04:47:45 <gmaxwell> hulkhogan_: there is coin-control behind a switch
73 2015-05-03 04:47:52 <esso> Yea that's definitely looking like the best bet. I've been writing up a review of the UX of Bitcoin core, do you think this is something that mailing list should discuss once my review is completed?
74 2015-05-03 04:47:57 <gmaxwell> I don't really like "advanced mode" over more specific options.
75 2015-05-03 04:48:49 <hulkhogan_> hmm, but its important to define- is an advanced user a 'gui-type' advanced user or a 'cli-type', if there is that dichotomy, i say focus on features for the 'gui-type' and let the latter use the rpc/bitcoin-cli
76 2015-05-03 04:49:18 <gmaxwell> esso: might be interesting; on this subject I think many people would be interested in the result of some user behavior study, e.g. set up situations and ask inexpirenced users to accomplish them, while recording their screens, and determine where they get lost.
77 2015-05-03 04:49:37 <gmaxwell> I think there is perhaps less interest in raw opinion; since everyone has opinions. :)
78 2015-05-03 04:50:04 <hulkhogan_> i think it goes back a bit to esso's Q about having defined an audience, as well hm
79 2015-05-03 04:50:23 <gmaxwell> hulkhogan_: I don't think there is any real conflict. Some tasks are more easily accomplished via different interfaces; it's fine for things to be done via the best interface for them.
80 2015-05-03 04:50:42 <gmaxwell> hulkhogan_: And every more advanced user still spends most of their time doing simple tasks.
81 2015-05-03 04:51:08 <xperia> hi all. i just pulled the latest code from github and compiled it on ubuntu linux but it looks like the source code is broken and has some bugs. whenever i try to start bitcoind i get a segmentation fault.
82 2015-05-03 04:51:15 <gmaxwell> The affordance for advanced users is mostly to not add a lot of additional time wasting steps; which is good for security and reliablity too, since extra steps fatigue all users and increase errors in general.
83 2015-05-03 04:51:15 <hulkhogan_> no conflict, just separation of concerns- an example is coin control, a power user may want this, a simple user might not, and a console user might not care at all, maybe
84 2015-05-03 04:51:39 <esso> I definitely think that some studies would be very useful. Once I get the review done and seen where opinions are conflicting I could set up a mock wallet, either online or downloadable, and the results of a survey after have
85 2015-05-03 04:51:57 <esso> Having some users test them out could be helpful
86 2015-05-03 04:52:03 <gmaxwell> hulkhogan_: well thats why it's behind a setting. Because if you want it you probably know you want it; and if you don't it will likely just confuse you.
87 2015-05-03 04:52:24 <hulkhogan_> yea agreed on that
88 2015-05-03 04:53:00 <hulkhogan_> i'm not very good at reasoning about ux, so perhaps those dichotomies dont really help much.
89 2015-05-03 04:53:12 <gmaxwell> so yea, I do think our audience for the GUI is "power users" but keeping in mind that 99% of the time a power user is just an ordinary user. :)
90 2015-05-03 04:53:28 <Luke-Jr> xperia: the master branch (and this channel) are intended for developers, not really support. if you want something to use, download a release..
91 2015-05-03 04:53:50 <esso> It's always necessary to design for the worst of end users :)
92 2015-05-03 04:54:02 <hulkhogan_> haha thats a narrow definition for an audience, i'd say
93 2015-05-03 04:54:17 <hulkhogan_> s/narrow/specific/
94 2015-05-03 04:54:35 <esso> Haha maybe not the worst of the worst
95 2015-05-03 04:55:21 <hulkhogan_> well, i think the simple case, of users just 'wanting to use bitcoin', probably would flock to a web wallet first, but also that might be a mischaracterization of bitcoin's simple users, as gmaxwell has pointed out
96 2015-05-03 04:55:28 <gmaxwell> esso: Well thats why I said that "not shy"; if it was really targeted at the most foolish of ordinary users it would eschew any remotely advanced feature on the slim chance Mr.WorstCaseFool might shoot his foot off. So I think we should abandon some of the least able users in order to satisify the powerusers who will not use software with a paperclip popping up to talk to them every 3 seconds. :)
97 2015-05-03 04:56:15 <gmaxwell> (maybe a money clip instead? :) )
98 2015-05-03 04:56:15 <xperia> Luke-Jr: i just wanted to report it. I find it strange that people upload code to the git repo that breaks the Programm itself. Would have expected that they at least tested it localy before they pushed it to the public.
99 2015-05-03 04:56:18 <esso> gmaxwell: yea there's definitely a middle ground to be found here.
100 2015-05-03 04:56:23 <hulkhogan_> haha i do agree
101 2015-05-03 04:56:45 <esso> Thanks for all the input guys it's been helpful :)
102 2015-05-03 04:56:47 <Luke-Jr> everything on git master has passed testing, so it is probably something unusual with your build
103 2015-05-03 04:56:56 <Luke-Jr> xperia: if you can identify the cause, that might be useful to know
104 2015-05-03 04:57:09 <Luke-Jr> does test_bitcoin work for you?
105 2015-05-03 04:57:30 <gmaxwell> Another goal for bitcoin core is for it to be a learning ground for people to learn more about the system and to become application developers and such; thats actually part of the motivation of coin control because it gives people a useful avenue to learn more about how the system actually works.
106 2015-05-03 04:57:52 <xperia> okey will try to rebuild it full clean and report it then.
107 2015-05-03 04:58:18 <hulkhogan_> that makes sense- actually the console being exposed in the ui seems almost reasonable now within that context
108 2015-05-03 04:58:39 <gmaxwell> Yes, exactly!
109 2015-05-03 04:58:52 <hulkhogan_> because it just helps expose developers to those internal functions you wouldn't normally see , or be exposed to in the 'regular' Ui
110 2015-05-03 04:59:08 <hulkhogan_> hehe, nice
111 2015-05-03 04:59:27 <gmaxwell> thats also why I am a big fan of having the CLI map 1:1 to the RPC interface. So that anyone who knows how to use the CLI already knows how to use the RPC. And when you're debugging your RPC using applications you can just try out your commands at the cli or debug console.
112 2015-05-03 05:00:21 <hulkhogan_> i wonder what would happen if the ui were to completely tear down into basically a scriptable environment for that kind of thing
113 2015-05-03 05:00:41 <gmaxwell> Basically any user should be able to use the wallet, and as they have more complex needs the wallet should accomidate those too; and if you want to grow in expirence you shouldn't have to graduate to totally different software.... or if you just want to stay a basic user you should be happy too.
114 2015-05-03 05:00:58 <xperia> Luke-Jr: yrs test_bitcoin works ! This is the Output of it => unknown location(0): fatal error in "AlertApplies": memory access violation at address: 0x00000001: no mapping at fault address
115 2015-05-03 05:01:24 <gmaxwell> hulkhogan_: might be neat; hard to say... one of the big unsolved problems in this space is providing safe interfaces for advanced functionality. Like, how do you make sure the UI captures the users intent and doesn't mislead them and help them shoot their foot off?
116 2015-05-03 05:01:31 <Luke-Jr> curious
117 2015-05-03 05:01:43 <hulkhogan_> hmm, i just can't conceptualize an interface (right now at least) that successfully caters to both an advanced and !advanced crowd...
118 2015-05-03 05:02:11 <gmaxwell> This is why we've never provided a GUI click to create your own bitcoin scriptpubkey... 99% of the users (myself included) would probably blackhole funds 99% of the time with it. :)
119 2015-05-03 05:02:26 <hulkhogan_> yes. i definitely see the need for that wrt: safe/usable interfaces. usability is key with crypto, its one of the main constraints for adoption and success
120 2015-05-03 05:02:59 <hulkhogan_> hahaha
121 2015-05-03 05:03:05 <hulkhogan_> right, myself included
122 2015-05-03 05:03:18 <gmaxwell> hulkhogan_: well I think we do today in bitcoin core in a dual way; I mean you can jointly use both the debug console AND the gui. Do your basic tasks in the gui, and when you need to do something weird, the debug console awaits. Though thats like a step function in the learning curve. Coin control is an example of a GUI middle ground.
123 2015-05-03 05:04:18 <hulkhogan_> I think there ought to be some kind of way to graduate users from the pure Ui to the console, maybe via tooltips, maybe through some tutorial-like walkthrough within the interface... something.
124 2015-05-03 05:05:10 <hulkhogan_> i think the coolest stuff happens in the console, and probably (maybe?) most users never get to the actual parts where they can say, create new transactions strictly from the CLI (would they want that, ever?)
125 2015-05-03 05:05:27 <gmaxwell> yea, so one way would be to have a log of the 'equivient commands' the gui does.
126 2015-05-03 05:06:01 <hulkhogan_> ohh nice, ie side by side
127 2015-05-03 05:06:33 <hulkhogan_> yeah i think it would help disambiguate what some of those (possibly cryptic) rpc calls actually mean and do
128 2015-05-03 05:06:34 <leakypat> In the 0.10.0 > after an unexpected shutdown, need an entire reindex?
129 2015-05-03 05:06:54 <hulkhogan_> while providing a accessible way for users to grok what would actually happen when they called said rpc commands
130 2015-05-03 05:07:05 <gmaxwell> another thing; on the learning front; is that we should have an easier way to get to a regtestish mode. where the whole UI turns bright green and you can try out whatever safely.
131 2015-05-03 05:07:53 <hulkhogan_> (sort of implies setting up a practice network on-the-fly?)
132 2015-05-03 05:08:16 <gmaxwell> perhaps but surely any complexity could be hidden from joe-user there.
133 2015-05-03 05:08:42 <gmaxwell> the challenge is that bitcoin is irreverable; so you really don't want to be trying anything that actually moves funds on the production network with real money.
134 2015-05-03 05:09:24 <esso> I like these ideas a lot and I think I have a few solutions that I'll work to incorporate
135 2015-05-03 05:09:28 <hulkhogan_> yeah, i'm just trying to think, if that were to take place i think they would need even more explainers to help them understand why we dropped them into a special green mode (is there a simplification im missing)
136 2015-05-03 05:09:44 <hulkhogan_> because you have this special mode, lets say
137 2015-05-03 05:09:52 <gmaxwell> Yea, its tricky; the basic functionality should all be safe to use without first using safemode.
138 2015-05-03 05:10:10 <hulkhogan_> and you need some way for Joe/Jill Bitcoin to send funds around a artifical network to get the 'feel' of using it
139 2015-05-03 05:10:30 <hulkhogan_> yea, safemode would be nice though, it could be a killer feature if done right
140 2015-05-03 05:11:17 <esso> so safemode would be the wallet connected to the testnet, correct?
141 2015-05-03 05:11:22 <hulkhogan_> i dont know how flexible Qt is, but technically i think its pretty flexible (no word on how much of that is actually enabled in master, though)
142 2015-05-03 05:11:58 <hulkhogan_> esso: regtest is a special mode like testnet but its just yourself on the network, mining blocks and sending txes
143 2015-05-03 05:12:29 <hulkhogan_> you would be able to simulate other nodes and toplogies, but basically it allows for limitless playgroundy fun
144 2015-05-03 05:12:44 <esso> ah that sounds like it could be a really nice feature to incorporate.
145 2015-05-03 05:13:08 <esso> So these look the best ideas so far:
146 2015-05-03 05:13:45 <esso> safemode, rpc interface, and ways to learn more about the network (maybe tutorials integrated)
147 2015-05-03 05:14:34 <hulkhogan_> yea
148 2015-05-03 05:15:06 <hulkhogan_> i think if you found a way to kind of unlock the other parts of qt that could help as well
149 2015-05-03 05:15:27 <hulkhogan_> i dont know for sure but i think bitcoin is only using a subset of what Qt actually allows, but could be off there
150 2015-05-03 05:15:45 <phantomcircuit> gmaxwell, that study probably doesn't take into account how common bizarro hw failures are either
151 2015-05-03 05:15:46 <phantomcircuit> :/
152 2015-05-03 05:16:36 <hulkhogan_> (i may be thinking of qt5 also)
153 2015-05-03 05:18:19 <xperia> have to go. see you all later. maybe in the meantime the bug that caouses the master branch to crash will be fixed. will check it out later again.
154 2015-05-03 05:18:26 <leakypat> Or is this just a one time thing after upgrade and the first time it has needed to reindex ?
155 2015-05-03 05:19:30 <Luke-Jr> leakypat: 0.10.x in general doesn't seem to survive unclean shutdowns
156 2015-05-03 05:19:40 <Luke-Jr> it's not a planned reindex, no
157 2015-05-03 05:20:52 <hulkhogan_> hm weird, i see qt5 referenced in the makefile, so my guess is the fancy things qt can do are just disabled for compatibiliy
158 2015-05-03 05:21:08 <hulkhogan_> probably not worth digging too deeply there
159 2015-05-03 05:21:57 <esso> Yea there are a lot of qt features the client doesn't seem to take advantage of. Compatibility seems like the most practical reason.
160 2015-05-03 05:23:18 <hulkhogan_> (on a broader note, i want to believe the ui stuff ought to have its own codebase (ie libsecp256k1) outside of the core to faciilitate cool features like esso wants to work on, but i really dont know what kind of work that would take, or if its reasonable)
161 2015-05-03 05:26:10 <esso> hulkhogan_: That seems very reasonable from my perspective. I think I'll bring that up in the report I'm doing.
162 2015-05-03 05:26:42 <esso> It's a little too easy for most devs to skip over design and user experience
163 2015-05-03 05:27:00 <hulkhogan_> esso: yea, but i can't have been the first person to bring that up. there likely is a good reason for it being the way it is, but cool :)
164 2015-05-03 05:27:25 <hulkhogan_> all too true..
165 2015-05-03 05:28:53 <esso> It'd be interesting to hear what the general consensus among most devs are concerning this topic
166 2015-05-03 05:29:08 <esso> It should make for a good discussion
167 2015-05-03 05:31:19 <kadoban> If I had to guess, it'd be: great, who's going to do all the work?
168 2015-05-03 05:32:18 <hulkhogan_> heh, if i were inclined i might insinuate bringing up the idea relatively soon, while all the modularization talk is still hot..
169 2015-05-03 05:39:31 <gmaxwell> Luke-Jr: wait; what are you talking about 0.10.1 should be absolutely fine across unclean shutdowns _generally_; though there are some exceptions and 0.10.1 should be strictly better than 0.8.x and 0.9x in that regard. This is not true for 0.10.0 however.
170 2015-05-03 05:39:55 <esso> kadoban: I think I might dive into some code here
171 2015-05-03 05:41:08 <esso> hulkhogan_: I'm going to try to finish up my report on the usability of Bitcoin core by this weekend and send an email to the list by monday.
172 2015-05-03 05:41:21 <gmaxwell> Unfortunately, google leveldb is not very durable in general.
173 2015-05-03 05:41:38 <gmaxwell> See also this paper: https://www.usenix.org/conference/osdi14/technical-sessions/presentation/pillai
174 2015-05-03 05:45:07 <Luke-Jr> gmaxwell: I don't think I have ever seen an unclean shutdown that 0.10.x or master managed to recover from.
175 2015-05-03 05:45:42 <Luke-Jr> the ARM USB stick is at 90bef6638fc689c13924fc17d115d5866b425e6c, with at least 2 or 3 of those failures
176 2015-05-03 05:45:56 <hulkhogan_> esso: that sounds really cool, are you doing that independent of any third party (if you dont mind me asking)?
177 2015-05-03 05:46:21 <Luke-Jr> actually, I take it back. I did see one: it recovered after an OOM killer attack
178 2015-05-03 05:46:50 <esso> hulkhogan_: Yep, completely independent. This is what I do for fun in my free time :)
179 2015-05-03 05:47:02 <Luke-Jr> but that is an unusual failure mode IMO
180 2015-05-03 05:47:22 <hulkhogan_> :D
181 2015-05-03 05:47:51 <hulkhogan_> thats awesome... and i'm definitely looking forwrad to checking it out on the list later !
182 2015-05-03 05:48:17 <esso> Awesome thanks man!
183 2015-05-03 05:50:50 <hulkhogan_> Good luck!
184 2015-05-03 06:11:03 <phantomcircuit> Luke-Jr, i regularly killall -9 bitcoind
185 2015-05-03 06:11:11 <phantomcircuit> i've only had one failure to restart
186 2015-05-03 06:12:08 <Luke-Jr> hm, I guess this suggests the kernel/disk play some role in the non-recovery scenario? ie, out-of-order writing or something
187 2015-05-03 06:14:07 <phantomcircuit> Luke-Jr, hmm i haven't tried pulling the power actually
188 2015-05-03 06:14:31 <phantomcircuit> so maybe out of order writes combined with low memory results in weird failures
189 2015-05-03 06:16:04 <Luke-Jr> it would be interesting if hard drive controllers behave in a way that makes things recover nicer. in the ARM case I've mostly experienced this with, it's just a microSD card
190 2015-05-03 06:17:24 <gmaxwell> your failures there are all out of memory ones.
191 2015-05-03 06:17:51 <gmaxwell> I left a recent RC in a kill -9 loop for a week without corruption.
192 2015-05-03 06:18:11 <gmaxwell> Luke-Jr: wrt kernel/disk; go see the paper I cited. :)
193 2015-05-03 06:19:30 <Luke-Jr> not all, the USB stick came out at least a few times, one of which had the uSD card lost on my floor <.<
194 2015-05-03 06:22:06 <Luke-Jr> gmaxwell: that paper sounds scary XD
195 2015-05-03 06:27:36 <gmaxwell> Yea, its an everythign is broken paper.
196 2015-05-03 06:28:17 <gmaxwell> I mean, we knew that leveldb wasn't very durable; phantomcircuit has pointed out a bunch of things it does stupidly that mean it can't be; that they found silent misbehavior is news to me and more concerning.
197 2015-05-03 06:28:38 <gmaxwell> I guess its concrete evidence that LMDB has better durrability.
198 2015-05-03 06:35:34 <phantomcircuit> gmaxwell, so on that front, i need a database for.. stuff.. that has strong durability and consistency properties
199 2015-05-03 06:36:01 <phantomcircuit> it's entirely append only so it's basically a journal
200 2015-05-03 06:36:26 <phantomcircuit> i was thinking of defining the format and writing something which would be about half of writing a trivial k/v db
201 2015-05-03 06:36:50 <gmaxwell> phantomcircuit: extfs3 in data journaling mode (see paper) and append data. :) ... I mean do you actually need random access? if not...
202 2015-05-03 06:38:29 <phantomcircuit> gmaxwell, nope
203 2015-05-03 06:38:39 <phantomcircuit> (note leveldb style databases dont either)
204 2015-05-03 06:39:08 <esso> What would you say is the target audience of Bitcoin core? Like what kind of people should be using Bitcoin core instead of a web wallet or some other type of wallet? The techies, brand new users, etc.
205 2015-05-03 06:39:32 <phantomcircuit> esso, everybody using bitcoin should be using bitcoin core...
206 2015-05-03 06:40:01 <kadoban> esso: Well, nobody should be using a web wallet, aka an IOU wallet.
207 2015-05-03 06:41:14 <hulkhogan_> alternatively, there is no 'audience', and the core is simply the reference implementation for the bitcoin protocol
208 2015-05-03 06:41:22 <esso> kadoban: What about a light weight client like electrum?
209 2015-05-03 06:41:46 <kadoban> esso: That's not a web wallet. I personally like electrum, especially for new people.
210 2015-05-03 06:42:14 <esso> hulkhogan_: So you would push for a wallet that every single user should be comfortable with?
211 2015-05-03 06:42:51 <hulkhogan_> esso: i wouldn't push for anything, there are plenty of incentives for someone else to provide that service already
212 2015-05-03 06:43:09 <hulkhogan_> a stable core is hard enough..
213 2015-05-03 06:43:20 <esso> kadoban: I'm aware. I'm looking for opinions on all types of wallets compared to Bitcoin core. I'm doing some UX revision and defining an audience is important.
214 2015-05-03 06:43:45 <kadoban> Probably can't help you there, personally.
215 2015-05-03 06:43:49 <esso> hulkhogan_: haha very true.
216 2015-05-03 06:44:05 <esso> kadoban: thanks for answer
217 2015-05-03 06:44:43 <gmaxwell> esso: Bitcoin Core's GUI wallet should be usable by all users; though it should not shy away from power user features at the expense of being being a little easier for total newbies.
218 2015-05-03 06:44:50 <esso> kadoban: I'm thinking the best bet is to appeal to everyone whether it's an experience user or brand new beginner
219 2015-05-03 06:45:52 <gmaxwell> esso: There isn't a fundimental conflict; the basic applications for a GUI wallet are the same for all users. It's okay to have more advanced features hidden a bit; but in general we should perfer security and good practices slightly to the utmost ease, and we shouldn't eschew a powerful feature because it might confuse a newbie (but its okay to hide it behind a setting)
220 2015-05-03 06:46:09 <esso> gmaxwell: I completely agree. It's also important to make sure the standard features that everyone uses do not get overshadowed by the more powerful features.
221 2015-05-03 06:46:37 <gmaxwell> yea, I agree. I think it's possible to balance that; mostly by tucking powerful features away where they won't bother people who don't need them.
222 2015-05-03 06:47:06 <gmaxwell> E.g. the most powerful interface to the GUI is the "CLI"-like debug console, but you'll never hear of a random user accidentally ending up with it and getting confused. :)
223 2015-05-03 06:47:33 <hulkhogan_> iirc there was some kind of checkbox setting for 'advanced mode' features in -qt ui, isnt there?
224 2015-05-03 06:47:52 <gmaxwell> hulkhogan_: there is coin-control behind a switch
225 2015-05-03 06:47:58 <esso> Yea that's definitely looking like the best bet. I've been writing up a review of the UX of Bitcoin core, do you think this is something that mailing list should discuss once my review is completed?
226 2015-05-03 06:48:04 <gmaxwell> I don't really like "advanced mode" over more specific options.
227 2015-05-03 06:48:55 <hulkhogan_> hmm, but its important to define- is an advanced user a 'gui-type' advanced user or a 'cli-type', if there is that dichotomy, i say focus on features for the 'gui-type' and let the latter use the rpc/bitcoin-cli
228 2015-05-03 06:49:24 <gmaxwell> esso: might be interesting; on this subject I think many people would be interested in the result of some user behavior study, e.g. set up situations and ask inexpirenced users to accomplish them, while recording their screens, and determine where they get lost.
229 2015-05-03 06:49:43 <gmaxwell> I think there is perhaps less interest in raw opinion; since everyone has opinions. :)
230 2015-05-03 06:50:11 <hulkhogan_> i think it goes back a bit to esso's Q about having defined an audience, as well hm
231 2015-05-03 06:50:30 <gmaxwell> hulkhogan_: I don't think there is any real conflict. Some tasks are more easily accomplished via different interfaces; it's fine for things to be done via the best interface for them.
232 2015-05-03 06:50:48 <gmaxwell> hulkhogan_: And every more advanced user still spends most of their time doing simple tasks.
233 2015-05-03 06:51:21 <gmaxwell> The affordance for advanced users is mostly to not add a lot of additional time wasting steps; which is good for security and reliablity too, since extra steps fatigue all users and increase errors in general.
234 2015-05-03 06:51:22 <hulkhogan_> no conflict, just separation of concerns- an example is coin control, a power user may want this, a simple user might not, and a console user might not care at all, maybe
235 2015-05-03 06:51:45 <esso> I definitely think that some studies would be very useful. Once I get the review done and seen where opinions are conflicting I could set up a mock wallet, either online or downloadable, and the results of a survey after have
236 2015-05-03 06:52:03 <esso> Having some users test them out could be helpful
237 2015-05-03 06:52:10 <gmaxwell> hulkhogan_: well thats why it's behind a setting. Because if you want it you probably know you want it; and if you don't it will likely just confuse you.
238 2015-05-03 06:52:31 <hulkhogan_> yea agreed on that
239 2015-05-03 06:53:06 <hulkhogan_> i'm not very good at reasoning about ux, so perhaps those dichotomies dont really help much.
240 2015-05-03 06:53:18 <gmaxwell> so yea, I do think our audience for the GUI is "power users" but keeping in mind that 99% of the time a power user is just an ordinary user. :)
241 2015-05-03 06:53:56 <esso> It's always necessary to design for the worst of end users :)
242 2015-05-03 06:54:09 <hulkhogan_> haha thats a narrow definition for an audience, i'd say
243 2015-05-03 06:54:24 <hulkhogan_> s/narrow/specific/
244 2015-05-03 06:54:42 <esso> Haha maybe not the worst of the worst
245 2015-05-03 06:55:27 <hulkhogan_> well, i think the simple case, of users just 'wanting to use bitcoin', probably would flock to a web wallet first, but also that might be a mischaracterization of bitcoin's simple users, as gmaxwell has pointed out
246 2015-05-03 06:55:34 <gmaxwell> esso: Well thats why I said that "not shy"; if it was really targeted at the most foolish of ordinary users it would eschew any remotely advanced feature on the slim chance Mr.WorstCaseFool might shoot his foot off. So I think we should abandon some of the least able users in order to satisify the powerusers who will not use software with a paperclip popping up to talk to them every 3 seconds. :)
247 2015-05-03 06:56:21 <gmaxwell> (maybe a money clip instead? :) )
248 2015-05-03 06:56:24 <esso> gmaxwell: yea there's definitely a middle ground to be found here.
249 2015-05-03 06:56:30 <hulkhogan_> haha i do agree
250 2015-05-03 06:56:51 <esso> Thanks for all the input guys it's been helpful :)
251 2015-05-03 06:57:36 <gmaxwell> Another goal for bitcoin core is for it to be a learning ground for people to learn more about the system and to become application developers and such; thats actually part of the motivation of coin control because it gives people a useful avenue to learn more about how the system actually works.
252 2015-05-03 06:58:25 <hulkhogan_> that makes sense- actually the console being exposed in the ui seems almost reasonable now within that context
253 2015-05-03 06:58:45 <gmaxwell> Yes, exactly!
254 2015-05-03 06:58:59 <hulkhogan_> because it just helps expose developers to those internal functions you wouldn't normally see , or be exposed to in the 'regular' Ui
255 2015-05-03 06:59:14 <hulkhogan_> hehe, nice
256 2015-05-03 06:59:33 <gmaxwell> thats also why I am a big fan of having the CLI map 1:1 to the RPC interface. So that anyone who knows how to use the CLI already knows how to use the RPC. And when you're debugging your RPC using applications you can just try out your commands at the cli or debug console.
257 2015-05-03 07:00:27 <hulkhogan_> i wonder what would happen if the ui were to completely tear down into basically a scriptable environment for that kind of thing
258 2015-05-03 07:00:47 <gmaxwell> Basically any user should be able to use the wallet, and as they have more complex needs the wallet should accomidate those too; and if you want to grow in expirence you shouldn't have to graduate to totally different software.... or if you just want to stay a basic user you should be happy too.
259 2015-05-03 07:01:30 <gmaxwell> hulkhogan_: might be neat; hard to say... one of the big unsolved problems in this space is providing safe interfaces for advanced functionality. Like, how do you make sure the UI captures the users intent and doesn't mislead them and help them shoot their foot off?
260 2015-05-03 07:01:49 <hulkhogan_> hmm, i just can't conceptualize an interface (right now at least) that successfully caters to both an advanced and !advanced crowd...
261 2015-05-03 07:02:18 <gmaxwell> This is why we've never provided a GUI click to create your own bitcoin scriptpubkey... 99% of the users (myself included) would probably blackhole funds 99% of the time with it. :)
262 2015-05-03 07:02:33 <hulkhogan_> yes. i definitely see the need for that wrt: safe/usable interfaces. usability is key with crypto, its one of the main constraints for adoption and success
263 2015-05-03 07:03:05 <hulkhogan_> hahaha
264 2015-05-03 07:03:11 <hulkhogan_> right, myself included
265 2015-05-03 07:03:24 <gmaxwell> hulkhogan_: well I think we do today in bitcoin core in a dual way; I mean you can jointly use both the debug console AND the gui. Do your basic tasks in the gui, and when you need to do something weird, the debug console awaits. Though thats like a step function in the learning curve. Coin control is an example of a GUI middle ground.
266 2015-05-03 07:04:25 <hulkhogan_> I think there ought to be some kind of way to graduate users from the pure Ui to the console, maybe via tooltips, maybe through some tutorial-like walkthrough within the interface... something.
267 2015-05-03 07:05:17 <hulkhogan_> i think the coolest stuff happens in the console, and probably (maybe?) most users never get to the actual parts where they can say, create new transactions strictly from the CLI (would they want that, ever?)
268 2015-05-03 07:05:34 <gmaxwell> yea, so one way would be to have a log of the 'equivient commands' the gui does.
269 2015-05-03 07:06:07 <hulkhogan_> ohh nice, ie side by side
270 2015-05-03 07:06:40 <hulkhogan_> yeah i think it would help disambiguate what some of those (possibly cryptic) rpc calls actually mean and do
271 2015-05-03 07:07:01 <hulkhogan_> while providing a accessible way for users to grok what would actually happen when they called said rpc commands
272 2015-05-03 07:07:12 <gmaxwell> another thing; on the learning front; is that we should have an easier way to get to a regtestish mode. where the whole UI turns bright green and you can try out whatever safely.
273 2015-05-03 07:07:29 <jonasschnelli> esso: if you need any assistance for writing you report just ask. I think UI/UX improvements are highly welcome
274 2015-05-03 07:08:00 <hulkhogan_> (sort of implies setting up a practice network on-the-fly?)
275 2015-05-03 07:08:21 <jonasschnelli> esso: we need to make sure that we try to use QT and avoid platform depending calls.
276 2015-05-03 07:08:23 <gmaxwell> perhaps but surely any complexity could be hidden from joe-user there.
277 2015-05-03 07:08:49 <gmaxwell> the challenge is that bitcoin is irreverable; so you really don't want to be trying anything that actually moves funds on the production network with real money.
278 2015-05-03 07:09:20 <jonasschnelli> If you provide some mockups or improvement ideas in textual form I'm ready to do the implementation.
279 2015-05-03 07:09:30 <esso> I like these ideas a lot and I think I have a few solutions that I'll work to incorporate
280 2015-05-03 07:09:34 <hulkhogan_> yeah, i'm just trying to think, if that were to take place i think they would need even more explainers to help them understand why we dropped them into a special green mode (is there a simplification im missing)
281 2015-05-03 07:09:51 <hulkhogan_> because you have this special mode, lets say
282 2015-05-03 07:09:58 <gmaxwell> Yea, its tricky; the basic functionality should all be safe to use without first using safemode.
283 2015-05-03 07:10:16 <hulkhogan_> and you need some way for Joe/Jill Bitcoin to send funds around a artifical network to get the 'feel' of using it
284 2015-05-03 07:10:36 <hulkhogan_> yea, safemode would be nice though, it could be a killer feature if done right
285 2015-05-03 07:11:24 <esso> so safemode would be the wallet connected to the testnet, correct?
286 2015-05-03 07:11:28 <hulkhogan_> i dont know how flexible Qt is, but technically i think its pretty flexible (no word on how much of that is actually enabled in master, though)
287 2015-05-03 07:12:04 <hulkhogan_> esso: regtest is a special mode like testnet but its just yourself on the network, mining blocks and sending txes
288 2015-05-03 07:12:36 <hulkhogan_> you would be able to simulate other nodes and toplogies, but basically it allows for limitless playgroundy fun
289 2015-05-03 07:12:51 <esso> ah that sounds like it could be a really nice feature to incorporate.
290 2015-05-03 07:13:15 <esso> So these look the best ideas so far:
291 2015-05-03 07:13:52 <esso> safemode, rpc interface, and ways to learn more about the network (maybe tutorials integrated)
292 2015-05-03 07:14:40 <hulkhogan_> yea
293 2015-05-03 07:15:12 <hulkhogan_> i think if you found a way to kind of unlock the other parts of qt that could help as well
294 2015-05-03 07:15:34 <hulkhogan_> i dont know for sure but i think bitcoin is only using a subset of what Qt actually allows, but could be off there
295 2015-05-03 07:15:51 <phantomcircuit> gmaxwell, that study probably doesn't take into account how common bizarro hw failures are either
296 2015-05-03 07:15:52 <phantomcircuit> :/
297 2015-05-03 07:16:42 <hulkhogan_> (i may be thinking of qt5 also)
298 2015-05-03 07:20:58 <hulkhogan_> hm weird, i see qt5 referenced in the makefile, so my guess is the fancy things qt can do are just disabled for compatibiliy
299 2015-05-03 07:21:14 <hulkhogan_> probably not worth digging too deeply there
300 2015-05-03 07:22:03 <esso> Yea there are a lot of qt features the client doesn't seem to take advantage of. Compatibility seems like the most practical reason.
301 2015-05-03 07:23:24 <hulkhogan_> (on a broader note, i want to believe the ui stuff ought to have its own codebase (ie libsecp256k1) outside of the core to faciilitate cool features like esso wants to work on, but i really dont know what kind of work that would take, or if its reasonable)
302 2015-05-03 07:26:17 <esso> hulkhogan_: That seems very reasonable from my perspective. I think I'll bring that up in the report I'm doing.
303 2015-05-03 07:26:48 <esso> It's a little too easy for most devs to skip over design and user experience
304 2015-05-03 07:27:06 <hulkhogan_> esso: yea, but i can't have been the first person to bring that up. there likely is a good reason for it being the way it is, but cool :)
305 2015-05-03 07:27:31 <hulkhogan_> all too true..
306 2015-05-03 07:28:59 <esso> It'd be interesting to hear what the general consensus among most devs are concerning this topic
307 2015-05-03 07:29:14 <esso> It should make for a good discussion
308 2015-05-03 07:31:25 <kadoban> If I had to guess, it'd be: great, who's going to do all the work?
309 2015-05-03 07:32:24 <hulkhogan_> heh, if i were inclined i might insinuate bringing up the idea relatively soon, while all the modularization talk is still hot..
310 2015-05-03 07:40:02 <esso> kadoban: I think I might dive into some code here
311 2015-05-03 07:41:14 <esso> hulkhogan_: I'm going to try to finish up my report on the usability of Bitcoin core by this weekend and send an email to the list by monday.
312 2015-05-03 07:46:02 <hulkhogan_> esso: that sounds really cool, are you doing that independent of any third party (if you dont mind me asking)?
313 2015-05-03 07:46:57 <esso> hulkhogan_: Yep, completely independent. This is what I do for fun in my free time :)
314 2015-05-03 07:47:29 <hulkhogan_> :D
315 2015-05-03 07:47:57 <hulkhogan_> thats awesome... and i'm definitely looking forwrad to checking it out on the list later !
316 2015-05-03 07:48:24 <esso> Awesome thanks man!
317 2015-05-03 07:50:56 <hulkhogan_> Good luck!
318 2015-05-03 09:04:32 <fanquake> ;;blocks
319 2015-05-03 09:04:33 <gribble> 354763
320 2015-05-03 09:07:35 <jonasschnelli> esso: if you need any assistance for writing you report just ask. I think UI/UX improvements are highly welcome
321 2015-05-03 09:08:27 <jonasschnelli> esso: we need to make sure that we try to use QT and avoid platform depending calls.
322 2015-05-03 09:09:26 <jonasschnelli> If you provide some mockups or improvement ideas in textual form I'm ready to do the implementation.
323 2015-05-03 11:04:39 <fanquake> ;;blocks
324 2015-05-03 11:04:39 <gribble> 354763
325 2015-05-03 13:25:23 <nkuttler> somebody in #bitcoin has an error on mac EXCEPTION: St13runtime_error CWalletDB::ListAccountCreditDebit() : cannot create DB cursor bitcoin in Runaway exception
326 2015-05-03 13:25:30 <nkuttler> feel free to drop in and help
327 2015-05-03 14:38:22 <esso> jonasschnelli: thanks for the offer, I'm may take you up on that!
328 2015-05-03 15:03:42 <Raccoon> Does anyone want the channel #bitcoin-coders before I drop it (and then it becomes blocked forever)
329 2015-05-03 15:25:29 <michagogo> Raccoon: eh, what?
330 2015-05-03 15:25:31 <nkuttler> somebody in #bitcoin has an error on mac EXCEPTION: St13runtime_error CWalletDB::ListAccountCreditDebit() : cannot create DB cursor bitcoin in Runaway exception
331 2015-05-03 15:25:37 <nkuttler> feel free to drop in and help
332 2015-05-03 15:25:46 <michagogo> If you use ChanServ's DROP command, anyone can register it
333 2015-05-03 15:26:13 <michagogo> (any channel op, which the next person to join will be after you leave it)
334 2015-05-03 15:26:37 <michagogo> Also, it's under the #bitcoin namespace, and IIRC that's a registered group, with jgarzik as the GC
335 2015-05-03 15:26:50 <michagogo> So whether or not you drop it, he could claim it at any point
336 2015-05-03 15:26:52 <Raccoon> michagogo: yeah, freenode is weird about that.
337 2015-05-03 15:27:09 <michagogo> ?
338 2015-05-03 15:27:09 <michagogo> Raccoon: Eh? How is that weird
339 2015-05-03 15:27:39 <Raccoon> yes, it would fall under the bitcoin namespace, but someone would have to spend 2 months going through arbitration and hunting down the actual #bitcoin- namespace group contact (who is that?) to kick off negotiations to reclaim the name.
340 2015-05-03 15:27:41 <michagogo> I don't see anything weird in "when someone drops a channel, anyone else can claim it"
341 2015-05-03 15:28:05 <Raccoon> Until that time, all DROP'd names with a single '#' are held by "freenode-staff" placeholder.
342 2015-05-03 15:28:08 <michagogo> Raccoon: Hm?
343 2015-05-03 15:28:19 <michagogo> No, that's if your account drops when you're the founder
344 2015-05-03 15:28:20 <Raccoon> try it on a name like #sflialwtjiawf
345 2015-05-03 15:28:28 <Raccoon> register then drop it
346 2015-05-03 15:28:58 <Raccoon> it will become occupied by the owner "freenode-staff"
347 2015-05-03 15:29:18 <Raccoon> and requires a multi-month intervention of tracking down the group contact and starting negotiations for reclaiming that name
348 2015-05-03 15:29:19 <michagogo> Raccoon: no, just tried it
349 2015-05-03 15:29:24 <michagogo> Registered and then dropped
350 2015-05-03 15:29:28 <michagogo> and then reregistered
351 2015-05-03 15:29:36 <Raccoon> #sflialwtjiawf ?
352 2015-05-03 15:29:41 <michagogo> No, #testchannell
353 2015-05-03 15:29:51 <michagogo> Channels transferring to freenode-staff happens when the founder's account drops
354 2015-05-03 15:30:05 <Raccoon> hmmmmm
355 2015-05-03 15:30:12 <Raccoon> that's not what I'm seeing in #freenode
356 2015-05-03 15:30:17 <Raccoon> maybe a change?
357 2015-05-03 15:30:32 <michagogo> No, AFAIK it's always been like this
358 2015-05-03 15:30:39 <Raccoon> interesting
359 2015-05-03 15:32:43 <Raccoon> well, I just stuck #bitcoin-coders on midnightmagic and #bitcoin-twitter on gribble ;)
360 2015-05-03 15:32:51 <Raccoon> they'll have to figure it out now, heh
361 2015-05-03 15:36:14 <michagogo> uhhh
362 2015-05-03 15:36:39 <michagogo> ...why?
363 2015-05-03 15:36:51 <michagogo> You should have just dropped them
364 2015-05-03 15:37:00 <michagogo> (not that it'll matter, in all likelyhood...)
365 2015-05-03 15:41:17 <priidu> yeah there's already like a bunch of bitcoin channels on here :p
366 2015-05-03 15:41:36 <priidu> compared to some other open-source projects, for example
367 2015-05-03 16:32:09 <joshc> jgarzik: sorry if I rebased out from under you; github's PR request mechanism is terrible
368 2015-05-03 16:34:28 <jgarzik> joshc, I'll just need you to grab master, rebase on top of that, and push --force to the PR's branch
369 2015-05-03 16:34:29 <jgarzik> joshc, I'll just need you to grab master, rebase on top of that, and push --force to the PR's branch
370 2015-05-03 16:34:42 <jgarzik> joshc, github figures it out from there
371 2015-05-03 16:34:50 <joshc> yep, can do.
372 2015-05-03 16:35:06 <jgarzik> joshc, grabbed the first commit, as I noted in the github comment just posted
373 2015-05-03 16:35:36 <joshc> great, thanks; let's not split conversations between github and here, I'll respond there.
374 2015-05-03 16:36:54 <jgarzik> joshc, indeed - github is preferred as it's a bit more public & archived & catalogued
375 2015-05-03 16:36:55 <jgarzik> joshc, indeed - github is preferred as it's a bit more public & archived & catalogued
376 2015-05-03 16:38:29 <esso> jonasschnelli: thanks for the offer, I'm may take you up on that!
377 2015-05-03 16:48:52 <michagogo> Well, this is also archived
378 2015-05-03 16:52:19 <jgarzik> michagogo, not nicely sorted into conversations though ;p
379 2015-05-03 17:03:49 <Raccoon> Does anyone want the channel #bitcoin-coders before I drop it (and then it becomes blocked forever)
380 2015-05-03 17:25:37 <michagogo> Raccoon: eh, what?
381 2015-05-03 17:25:54 <michagogo> If you use ChanServ's DROP command, anyone can register it
382 2015-05-03 17:26:20 <michagogo> (any channel op, which the next person to join will be after you leave it)
383 2015-05-03 17:26:44 <michagogo> Also, it's under the #bitcoin namespace, and IIRC that's a registered group, with jgarzik as the GC
384 2015-05-03 17:26:57 <michagogo> So whether or not you drop it, he could claim it at any point
385 2015-05-03 17:26:59 <Raccoon> michagogo: yeah, freenode is weird about that.
386 2015-05-03 17:27:16 <michagogo> Raccoon: Eh? How is that weird
387 2015-05-03 17:27:17 <michagogo> ?
388 2015-05-03 17:27:46 <Raccoon> yes, it would fall under the bitcoin namespace, but someone would have to spend 2 months going through arbitration and hunting down the actual #bitcoin- namespace group contact (who is that?) to kick off negotiations to reclaim the name.
389 2015-05-03 17:27:49 <michagogo> I don't see anything weird in "when someone drops a channel, anyone else can claim it"
390 2015-05-03 17:28:13 <Raccoon> Until that time, all DROP'd names with a single '#' are held by "freenode-staff" placeholder.
391 2015-05-03 17:28:16 <michagogo> Raccoon: Hm?
392 2015-05-03 17:28:26 <michagogo> No, that's if your account drops when you're the founder
393 2015-05-03 17:28:27 <Raccoon> try it on a name like #sflialwtjiawf
394 2015-05-03 17:28:36 <Raccoon> register then drop it
395 2015-05-03 17:29:05 <Raccoon> it will become occupied by the owner "freenode-staff"
396 2015-05-03 17:29:26 <michagogo> Raccoon: no, just tried it
397 2015-05-03 17:29:26 <Raccoon> and requires a multi-month intervention of tracking down the group contact and starting negotiations for reclaiming that name
398 2015-05-03 17:29:31 <michagogo> Registered and then dropped
399 2015-05-03 17:29:35 <michagogo> and then reregistered
400 2015-05-03 17:29:43 <Raccoon> #sflialwtjiawf ?
401 2015-05-03 17:29:49 <michagogo> No, #testchannell
402 2015-05-03 17:29:58 <michagogo> Channels transferring to freenode-staff happens when the founder's account drops
403 2015-05-03 17:30:13 <Raccoon> hmmmmm
404 2015-05-03 17:30:19 <Raccoon> that's not what I'm seeing in #freenode
405 2015-05-03 17:30:24 <Raccoon> maybe a change?
406 2015-05-03 17:30:39 <michagogo> No, AFAIK it's always been like this
407 2015-05-03 17:30:46 <Raccoon> interesting
408 2015-05-03 17:31:36 <jgarzik> joshc, jonasschnelli: also, adding basic HD derivations is another piece of low hanging fruit for libccoin
409 2015-05-03 17:31:50 <jgarzik> and produces an even more efficient SPV wallet
410 2015-05-03 17:31:51 <jgarzik> and produces an even more efficient SPV wallet
411 2015-05-03 17:32:50 <Raccoon> well, I just stuck #bitcoin-coders on midnightmagic and #bitcoin-twitter on gribble ;)
412 2015-05-03 17:32:58 <Raccoon> they'll have to figure it out now, heh
413 2015-05-03 17:36:21 <michagogo> uhhh
414 2015-05-03 17:36:47 <michagogo> ...why?
415 2015-05-03 17:36:58 <michagogo> You should have just dropped them
416 2015-05-03 17:37:07 <michagogo> (not that it'll matter, in all likelyhood...)
417 2015-05-03 17:41:24 <priidu> yeah there's already like a bunch of bitcoin channels on here :p
418 2015-05-03 17:41:43 <priidu> compared to some other open-source projects, for example
419 2015-05-03 18:09:23 <jgarzik> joshc, jonasschnelli: To be specific, with regards to key birthday, HD permutes that into seed birthday + some note of when derivations are used
420 2015-05-03 18:09:24 <jgarzik> joshc, jonasschnelli: To be specific, with regards to key birthday, HD permutes that into seed birthday + some note of when derivations are used
421 2015-05-03 18:16:43 <joshc> I'm assuming this is what's specified in BIP32?
422 2015-05-03 18:40:12 <jonasschnelli> jgarzik, joshc: i'm not sure if you (only) need seed birthday. What if you like to use a non-masterkey wallet? Like m/44/0/y/0/x (where y is your local root key and y are the child-keys you like to generate in your wallet)? An besides use-cases: IMO a key-birthday is something you really should habe in your wallet-metadata, whether you use bip32 or not
423 2015-05-03 18:40:46 <jonasschnelli> * x are the child keys, * And besides
424 2015-05-03 18:42:14 <jonasschnelli> jgarzik: i didn't had a close look at picocoins "netsync". But how complete is the SPV wallet feature? Can i do a netsync and i'll get wtx/balance in my local wallet? Will a 2nd netsync start from wallets known bestblock?
425 2015-05-03 18:43:37 <joshc> ACTION does some reading
426 2015-05-03 18:44:36 <jonasschnelli> another thing on the top (so don't stop reading. :): for unit-tests. Would it be appropriate to use Check as C Unit Test Framework? http://check.sourceforge.net/web/install.html
427 2015-05-03 19:18:15 <gmaxwell> jonasschnelli: why a "framework" ... it adds so little, just another irritating dependency. Someone else had that idea on another project I worked on, and we tore check out after three months. It made the tests take several times longer-- I'd written code that performed millions of small tests and check was actually slower than the code, and users kept complaining about the dependency.
428 2015-05-03 19:20:19 <jonasschnelli> gmaxwell: Right. It was meant to be for jgarzik's picocoin project (not for bitcoin core). But i just saw that there are already unittests, so just follow the current paradigm.
429 2015-05-03 19:22:36 <gmaxwell> I was aware.
430 2015-05-03 19:27:46 <jonasschnelli> Okay. Sorry. Though you have just read that last line. But your right. Especially for a small c library (or binary), a frameworks adds little and produces probably some overhead.
431 2015-05-03 19:45:36 <jgarzik> jonasschnelli, RE netsync - very incomplete. the pieces are there, but "putting it all together" is a WIP
432 2015-05-03 19:46:07 <jonasschnelli> jgarzik: okay. Thanks.
433 2015-05-03 19:46:31 <jgarzik> jonasschnelli, RE framework, tend to agree w/ gmaxwell, it adds little
434 2015-05-03 19:46:44 <jgarzik> jonasschnelli, we can engineer the unit tests to be very small, with a test-only library for common code
435 2015-05-03 19:46:46 <jonasschnelli> Agreed.
436 2015-05-03 19:47:40 <jonasschnelli> jgarzik: key-metadata: i also though it could be a separation of security layer. Don't keep a structure with metadata and privkeys.
437 2015-05-03 19:47:56 <jgarzik> jonasschnelli, that is a point, yes
438 2015-05-03 19:49:09 <jgarzik> jonasschnelli, RE "netsync" #2 -- part of this comes down to user experience. picocoin is designed to be an offline-by-default command line wallet. "netsync" is the magic command that goes online, gets new tx's, sends tx's, and then disconnects.
439 2015-05-03 19:49:44 <jgarzik> jonasschnelli, if there is a better way (read: less LOC :)), I'm keeping an open mind
440 2015-05-03 19:50:26 <jonasschnelli> jgarzik: that's fine. It should just start syncing from the last validated block. The wallet must keep track of "his bestblock".
441 2015-05-03 19:50:34 <jgarzik> yes
442 2015-05-03 19:50:49 <jonasschnelli> and for sure it would be nice if there would be bloomfilters once so you could use the library for embedded stuff with low bandwith.
443 2015-05-03 19:52:05 <jgarzik> jonasschnelli, yep. libccoin already has bloom filtering... just another piece that Just Needs To Be Hooked Up
444 2015-05-03 19:55:20 <jgarzik> jonasschnelli, general pattern of development was: implement it in lib/, write a lib/ unit test, use it in a src/ client app.
445 2015-05-03 19:55:56 <jgarzik> jonasschnelli, lib/ has a ton of stuff that is just waiting to be used. BIP32 would appear in lib/ first.
446 2015-05-03 19:56:06 <jonasschnelli> when i encountered picocoin in 2012, i was thinking this could be a great lightwight SPV library.
447 2015-05-03 19:57:20 <jgarzik> yes
448 2015-05-03 19:57:21 <jgarzik> yes
449 2015-05-03 19:59:11 <jonasschnelli> jgarzik: maybe a basic wallet support should be within the lib section (as bitcoinj has)?
450 2015-05-03 20:01:01 <jgarzik> jonasschnelli, sure
451 2015-05-03 20:01:22 <jgarzik> jonasschnelli, as long as wallet is a separate layer / modules that are optional
452 2015-05-03 20:01:35 <jonasschnelli> agreed
453 2015-05-03 21:57:04 <hulkhogan_> cfields_: curious, did you have any chance to go over those jni updates ?
454 2015-05-03 21:57:05 <hulkhogan_> cfields_: curious, did you have any chance to go over those jni updates ?
455 2015-05-03 21:58:33 <cfields_> hulkhogan_: yes, they look great. the main issue for rebasing is getting the context stuff passed through jni. I had hoped to work on that on friday but ran out of time...
456 2015-05-03 21:59:02 <cfields_> hulkhogan_: if you're tired of waiting on me, you're welcome to rebase on master and add the context handling
457 2015-05-03 21:59:40 <cfields_> it should be as simple as storing a jlong on the java side
458 2015-05-03 22:00:08 <hulkhogan_> yeah i see what you mean, can i ask what the motivation is to expose that to java?
459 2015-05-03 22:00:35 <hulkhogan_> i rebased, but i sidestepped the exposure by just returning 0/1 instead of letting the user deal with setting that up
460 2015-05-03 22:00:48 <sipa> returning 0/1 for what>
461 2015-05-03 22:00:49 <sipa> returning 0/1 for what>
462 2015-05-03 22:01:05 <hulkhogan_> just ecdsa_verify
463 2015-05-03 22:01:20 <hulkhogan_> so for the developer, its basically, pass the params and get a bool back
464 2015-05-03 22:01:33 <hulkhogan_> on the actual interface code, we just auto-set up a VERIFY context
465 2015-05-03 22:01:55 <hulkhogan_> but dont pass any of that through for the developer to interact with
466 2015-05-03 22:01:58 <sipa> sure
467 2015-05-03 22:02:05 <sipa> but the question is where do you keep the context?
468 2015-05-03 22:02:06 <sipa> but the question is where do you keep the context?
469 2015-05-03 22:02:20 <hulkhogan_> within JNI only
470 2015-05-03 22:02:23 <sipa> in the JNI-C layer, int he JNI-Java wrapper, or in the user code?
471 2015-05-03 22:02:24 <sipa> in the JNI-C layer, int he JNI-Java wrapper, or in the user code?
472 2015-05-03 22:02:30 <sipa> i would say in the JNI-Java wrapper
473 2015-05-03 22:02:43 <sipa> as you have access to Java's synchronization primitives there
474 2015-05-03 22:02:47 <cfields_> hulkhogan_: right, i didn't mean to have it exposed, only a matter of creating it
475 2015-05-03 22:03:03 <hulkhogan_> hm, yea then ill need to refactor a bit if we want some way to set up contexts in the wrapper
476 2015-05-03 22:03:35 <sipa> if you do it in C, is the context created at startup?
477 2015-05-03 22:03:38 <hulkhogan_> is there a reason? i guess to be faithful to the interface secp256k1 exposes?
478 2015-05-03 22:03:57 <hulkhogan_> er, i should explain it better, its just
479 2015-05-03 22:04:07 <sipa> well, why do you think libsecp256k1 exposes it? :)
480 2015-05-03 22:04:29 <hulkhogan_> to set up the context depending on the type of operation (?) being done, i thought
481 2015-05-03 22:04:49 <hulkhogan_> but i was trying to abstract the low level details a bit, probably not a good idea in retrospect
482 2015-05-03 22:04:57 <sipa> no, that's a very good idea
483 2015-05-03 22:05:09 <hulkhogan_> it didnt seem like giving the dev the ability to shoot themselves was something i wanted
484 2015-05-03 22:05:12 <sipa> but you should know why they weren't abstracted at the lower level :)
485 2015-05-03 22:05:13 <hulkhogan_> but can change that heh
486 2015-05-03 22:05:13 <sipa> but you should know why they weren't abstracted at the lower level :)
487 2015-05-03 22:05:23 <sipa> so, there are several reasons
488 2015-05-03 22:05:36 <hulkhogan_> oh yea, definitely, lots of '|' OR action going on there :)
489 2015-05-03 22:05:50 <sipa> one is that the contexts are large (~megabyte) and slow to create, so you don't just want to create them for every operation separately
490 2015-05-03 22:05:51 <sipa> one is that the contexts are large (~megabyte) and slow to create, so you don't just want to create them for every operation separately
491 2015-05-03 22:06:02 <hulkhogan_> hmm
492 2015-05-03 22:06:22 <hulkhogan_> i see, that explains some of the original code a bit more
493 2015-05-03 22:06:30 <sipa> the older interface just had a function secp256k1_start(flags)
494 2015-05-03 22:06:35 <sipa> but did not have contexts
495 2015-05-03 22:06:43 <hulkhogan_> which created one at the start and another to destory at the end
496 2015-05-03 22:06:46 <hulkhogan_> yea
497 2015-05-03 22:07:02 <hulkhogan_> so proper way: regbase to context on init, and then (i guess no cleanup is needed) ?
498 2015-05-03 22:07:06 <sipa> that's a problem, because what if you have different users of the library, and both call start with different flags, what do you want?
499 2015-05-03 22:07:30 <hulkhogan_> right, well just an init method that passes flags back i guess, but then its trickier
500 2015-05-03 22:07:37 <sipa> no no
501 2015-05-03 22:07:42 <hulkhogan_> you have to init for every op or something, which is kind of what is happening now
502 2015-05-03 22:07:49 <sipa> the problem is: what if different threads call secp256k1_start simultaneously
503 2015-05-03 22:07:50 <hulkhogan_> init/destroy loop
504 2015-05-03 22:07:55 <sipa> the C library would likely crash
505 2015-05-03 22:08:00 <hulkhogan_> ah
506 2015-05-03 22:08:03 <sipa> as it assumes the caller takes care of synchronization
507 2015-05-03 22:08:04 <sipa> as it assumes the caller takes care of synchronization
508 2015-05-03 22:08:15 <hulkhogan_> i thought the context was thread safe though?
509 2015-05-03 22:08:20 <sipa> once it is created
510 2015-05-03 22:08:37 <hulkhogan_> yea
511 2015-05-03 22:08:43 <sipa> but before we had explicit contexts, there was one implicit shared context among all users
512 2015-05-03 22:09:02 <hulkhogan_> i see , so
513 2015-05-03 22:09:03 <sipa> and that is not thread safe, if different users can call start at different times
514 2015-05-03 22:09:09 <hulkhogan_> yea, makes sense
515 2015-05-03 22:09:16 <sipa> now, there is another reason
516 2015-05-03 22:09:20 <sipa> which was only added recently
517 2015-05-03 22:09:27 <sipa> and that's the blinding randomization
518 2015-05-03 22:09:34 <sipa> you can add randomness to a context
519 2015-05-03 22:09:39 <hulkhogan_> i didnt see that :/
520 2015-05-03 22:09:43 <sipa> which randomizes the internal tables
521 2015-05-03 22:10:00 <hulkhogan_> oh, is this the patch gmaxwell submitted?
522 2015-05-03 22:10:00 <sipa> and protects against some obscure attacks
523 2015-05-03 22:10:16 <sipa> yes
524 2015-05-03 22:10:18 <hulkhogan_> i did see that but i didnt think it effected anything the developer should need to think about (?)
525 2015-05-03 22:10:28 <hulkhogan_> ie for the wrapper code, jni code, that stuff
526 2015-05-03 22:10:39 <hulkhogan_> it was a good optimization, secuirity
527 2015-05-03 22:10:41 <hulkhogan_> n
528 2015-05-03 22:10:47 <sipa> it's absolutely not an optimization
529 2015-05-03 22:10:48 <sipa> it's absolutely not an optimization
530 2015-05-03 22:10:56 <hulkhogan_> well, right
531 2015-05-03 22:10:56 <sipa> it slows things down (very slightly)
532 2015-05-03 22:11:10 <sipa> no, Java has synchronization primitives
533 2015-05-03 22:11:12 <sipa> C does not
534 2015-05-03 22:11:13 <hulkhogan_> but you get protection from specific timing attacks
535 2015-05-03 22:11:17 <sipa> so handle the contexts in Java
536 2015-05-03 22:11:18 <sipa> so handle the contexts in Java
537 2015-05-03 22:11:32 <hulkhogan_> gotcah
538 2015-05-03 22:11:44 <sipa> have a class global with the context in Java
539 2015-05-03 22:11:47 <hulkhogan_> gotcha, ill have to look at the blinding code, i didnt think about it
540 2015-05-03 22:11:54 <sipa> use synchronized blocks to initialize it on load
541 2015-05-03 22:12:14 <sipa> you may want a read-write lock, if you want to allow multithreaded access
542 2015-05-03 22:12:28 <sipa> (where the randomize function uses a write lock, and all other things use a read lock)
543 2015-05-03 22:12:32 <hulkhogan_> but so how can the user switch context types per operation? using that same synch block?
544 2015-05-03 22:12:40 <sipa> i don't know java
545 2015-05-03 22:12:43 <hulkhogan_> lol
546 2015-05-03 22:12:43 <sipa> :p
547 2015-05-03 22:12:45 <hulkhogan_> well
548 2015-05-03 22:12:51 <hulkhogan_> im asking because i implemented the other functions
549 2015-05-03 22:12:53 <sipa> oh
550 2015-05-03 22:12:57 <hulkhogan_> ie verify sign etcd
551 2015-05-03 22:12:57 <sipa> there would just be one context
552 2015-05-03 22:13:03 <hulkhogan_> oh ok
553 2015-05-03 22:13:03 <sipa> no need to have several ones
554 2015-05-03 22:13:11 <hulkhogan_> right, the | OR trick
555 2015-05-03 22:13:17 <sipa> eh?
556 2015-05-03 22:13:21 <hulkhogan_> well, yea ill look at that then
557 2015-05-03 22:13:36 <sipa> what is the or trick?
558 2015-05-03 22:13:37 <hulkhogan_> nothing, just saw the context initialized with SIGN|VERIFY instead of one or the other, in places
559 2015-05-03 22:13:43 <sipa> oh yes
560 2015-05-03 22:13:44 <sipa> oh yes
561 2015-05-03 22:13:49 <sipa> yes absolutely, that's what you would do
562 2015-05-03 22:13:50 <sipa> yes absolutely, that's what you would do
563 2015-05-03 22:13:55 <hulkhogan_> it just didnt make sense but kind of does now i suppose
564 2015-05-03 22:14:01 <hulkhogan_> robustness
565 2015-05-03 22:14:16 <sipa> i guess this all means the contexts in the header file need a lot more explanation
566 2015-05-03 22:14:23 <hulkhogan_> well il have to look again so perhaps something updated on that middle of the week or so
567 2015-05-03 22:14:47 <hulkhogan_> i mean not particularly, the flags are very well named, their use is a little convoluted at best
568 2015-05-03 22:14:58 <hulkhogan_> but nothing to worry about imo..
569 2015-05-03 22:27:37 <hulkhogan_> sipa: a question about multithreading, so you're saying two users should not be able to call _verify() using the same context at the same time? this is the reason for a need on read/write locks on the context obj?
570 2015-05-03 22:28:28 <hulkhogan_> i might just disallow multithreaded access then..
571 2015-05-03 22:28:47 <sipa> hulkhogan_: no, two users should not be able to call randomize at the same time
572 2015-05-03 22:29:00 <sipa> hulkhogan_: this is actually in the comments in secp256k1.h :)
573 2015-05-03 22:29:18 <hulkhogan_> ah ok, thanks i will double check that!
574 2015-05-03 22:29:19 <sipa> functions that take a pointer to a const context object can run in parallel
575 2015-05-03 22:29:31 <sipa> functions that take a mutable pointer need exclusive access
576 2015-05-03 22:29:42 <hulkhogan_> yea i see
577 2015-05-03 22:29:43 <hulkhogan_> yea i see
578 2015-05-03 22:29:58 <sipa> if you don't expose the randomize function at all, it becomes much simpler
579 2015-05-03 22:30:02 <hulkhogan_> its not so trivial to fix, just need to think a bit before committing
580 2015-05-03 22:30:07 <sipa> as it means you need no synchronization whatsoever
581 2015-05-03 22:30:20 <sipa> i'm fine with a version of the JNI code that doesn't expose randomize at first :)
582 2015-05-03 22:30:30 <hulkhogan_> well, is randomize part of the initialization of the context? i thought this was what was happening, but no, we dont expose that (yet)
583 2015-05-03 22:30:34 <sipa> no
584 2015-05-03 22:30:43 <sipa> it's an explicit call
585 2015-05-03 22:30:45 <hulkhogan_> yea, its very simple atm, just pub/sec verify, sign/verify
586 2015-05-03 22:30:52 <hulkhogan_> i left the rest as TODOs...
587 2015-05-03 22:30:53 <sipa> the user uses it to "add" randomness to the context
588 2015-05-03 22:30:56 <hulkhogan_> ok
589 2015-05-03 22:31:10 <hulkhogan_> well, i think it should be safe, but it does leave a future consideration in mind
590 2015-05-03 22:31:14 <sipa> you can do it at startup of course, if you have access to a good RNG in java
591 2015-05-03 22:31:38 <hulkhogan_> well, prefer to let the lib do its thing over untested java code
592 2015-05-03 22:31:45 <hulkhogan_> relatively untested, anyway ;p
593 2015-05-03 22:31:47 <sipa> ?
594 2015-05-03 22:32:11 <hulkhogan_> using a RNG instead of randomize, are you saying randomize needs to be called before the timing protections work?
595 2015-05-03 22:32:13 <sipa> libsecp256k1 does not have access to a random number generator, so it leaves it up the the user to call it
596 2015-05-03 22:32:35 <sipa> no, you would use a RNG, and pass the generated random number to randomize!
597 2015-05-03 22:32:37 <hulkhogan_> well, let me go back and look at the header, i have a good idea of what has to be done
598 2015-05-03 22:32:54 <sipa> if your JNI wrapper can hide that from the user, awesome
599 2015-05-03 22:33:10 <sipa> yes, some of the blinding protections only work after you've called randomize on the context
600 2015-05-03 22:33:11 <hulkhogan_> i see yeah, i didnt read any of the comments for randomize yet because i didnt think about its implementation, et al
601 2015-05-03 22:33:37 <hulkhogan_> i just- i just didnt know how useful exposing the rest of the lib would be, some of those low level things i just figured would not get much use
602 2015-05-03 22:33:54 <hulkhogan_> im happy to be worong and implement those of course, but i kind of wanted to solve the simple case, first
603 2015-05-03 22:34:05 <hulkhogan_> (in this case is just sign + verify)