1 2012-12-10 00:00:10 <jgarzik> then further connections were labelled "banned"
2 2012-12-10 00:01:02 <jgarzik> receive version message: version 60001, blocks=211140, us=127.0.0.1:8333, them=1
3 2012-12-10 00:04:54 <vazakl> nice1
4 2012-12-10 00:13:23 <sta> i think you should apologize, gmaxwell
5 2012-12-10 00:15:09 <NaruFGT> would this be an appropriate place to ask questions about the reference client?
6 2012-12-10 00:15:19 <gmaxwell> NaruFGT: it's a good place.
7 2012-12-10 00:15:50 <gmaxwell> jgarzik: interesting, that hash doesn't show up in my logs here.
8 2012-12-10 00:17:36 <midnightmagic> :-/
9 2012-12-10 01:54:41 <slush> does exist something like tor2web, but tunelling any traffic (not just HTTP)?
10 2012-12-10 01:55:21 <slush> tor2web even cannot be used for anything serious, because it adds banners to the payload :-/
11 2012-12-10 01:56:25 <jgarzik> yeah it's called "tor"
12 2012-12-10 01:56:26 <jgarzik> ;p
13 2012-12-10 01:57:17 <jgarzik> a non-HTTP protocol has no opportunity for being accessed through a normal web browser
14 2012-12-10 01:57:35 <jgarzik> so that doesn't make much sense
15 2012-12-10 01:59:00 <slush> jgarzik: maybe you didn't noticed, but there are more protocols than HTTP :-P
16 2012-12-10 01:59:39 <slush> it would enable hidden jabber/mail/torrent servers etc
17 2012-12-10 01:59:47 <slush> but accessible from normal internet
18 2012-12-10 02:00:51 <jgarzik> slush: sure. that's basically setting up an exit node, but with more visibility, and no way (like banner ads) to generate revenue to pay for the internet connection and lawyers
19 2012-12-10 02:01:06 <slush> jgarzik: I can imagine another business model
20 2012-12-10 02:01:11 <jgarzik> in the end it's easier and cheaper to just run Tor ;p
21 2012-12-10 02:01:24 <slush> jgarzik: that proxy can receive bitcoins for tunelling some specific hidden service
22 2012-12-10 02:02:34 <jgarzik> slush: or just point people to the Tor Browser Bundle, and there is no need for intermediaries... https://www.torproject.org/projects/torbrowser.html.en
23 2012-12-10 02:03:04 <slush> this has very limited use
24 2012-12-10 02:03:15 <slush> you cannot easily run hidden mail server
25 2012-12-10 02:03:21 <slush> because normal mail server cannot deliver you emails
26 2012-12-10 02:04:00 <slush> but with such tunelling tool, somebody can pay for tunelling somedomain.slush.cz:25 -> his_onion_address:25
27 2012-12-10 02:04:47 <phantomcircuit> slush, it's actually very easy to purchase totally anonymous servers in various parts of the world
28 2012-12-10 02:05:07 <phantomcircuit> you just have to learn how to navigate sites in other languages (they're never in english)
29 2012-12-10 02:05:18 <slush> phantomcircuit: link? :-)
30 2012-12-10 02:05:21 <jgarzik> slush: Sounds like a perfect task for an agent: https://en.bitcoin.it/wiki/Agents
31 2012-12-10 02:05:56 <slush> jgarzik: except that there's no standard API for buying VPSes :-/
32 2012-12-10 02:06:26 <slush> but yes, with providers accepint bitcoins and having automatic API for creating machines, lot of completely autonomous services would be possible
33 2012-12-10 02:06:27 <upb> so theres an api for buying a prepaid credit card for cash?
34 2012-12-10 02:06:44 <phantomcircuit> slush, https://trac.torproject.org/projects/tor/wiki/doc/GoodBadISPs
35 2012-12-10 02:06:59 <slush> phantomcircuit: thanks!
36 2012-12-10 02:07:12 <slush> few years ago this list was almost empty
37 2012-12-10 03:25:24 <jgarzik> slush: read the whole thing, especially the StorJ linked example
38 2012-12-10 03:25:32 <jgarzik> slush: that's OK that there is no standard API
39 2012-12-10 03:25:38 <jgarzik> slush: the agent hires and tests
40 2012-12-10 03:26:02 <jgarzik> slush: or, honestly, maybe just ignore the Agent wiki page and focus on gmaxwell's StorJ post
41 2012-12-10 03:33:41 <slush> jgarzik: not possible with current state of AI
42 2012-12-10 03:33:52 <jgarzik> slush: sure it is. it is narrow AI.
43 2012-12-10 03:34:55 <jgarzik> no reasoning involved
44 2012-12-10 03:35:43 <amiller> the agent page is misleading
45 2012-12-10 03:36:04 <gmaxwell> I was quite careful to not mention anything that requires blind inference. It probably requires software engineering more reliable than we know how to build but no AI improvements at all.
46 2012-12-10 03:37:00 <slush> currently software cannot work untouched by the human even in deterministic environments like server rooms.
47 2012-12-10 03:37:43 <slush> and it is even not trying to improve itself, thankfully ;)
48 2012-12-10 03:38:20 <amiller> there are some tricks with proof of work puzzles that sort of come close, like this one especially http://www.cs.columbia.edu/~roxana/teaching/COMS-E6998-7-Fall-2011/papers/bowers-iacr10.pdf
49 2012-12-10 03:38:41 <amiller> where the proof of work is public verifiable and so it could let people compete to prove that they're storing your cipher text
50 2012-12-10 03:38:57 <gmaxwell> slush: the software I propose doesn't improve itself.
51 2012-12-10 03:39:12 <amiller> the storj itself is pretty conservative and cool
52 2012-12-10 03:39:33 <jgarzik> slush: Just read it :) https://bitcointalk.org/index.php?topic=53855.msg642768#msg642768
53 2012-12-10 03:41:15 <jgarzik> amiller: Agree that the agent page isn't the best. It is equal parts derivative and summary. The Agent page is sorta TD's summary of the general concept. I think it helps people to read specific details like the forum post.
54 2012-12-10 03:42:10 <slush> jgarzik: I already read it
55 2012-12-10 03:42:23 <gmaxwell> I do think that basic software quality is a major challenge there, bitcoin has just as critical a requirement for software quality and its required quite a few emergency fixes.
56 2012-12-10 03:42:48 <gmaxwell> And it would be rather brittle even with excellent software quality.
57 2012-12-10 03:43:30 <gmaxwell> e.g. the host rearranges their payment api and it dies. :P
58 2012-12-10 03:44:07 <holosdfg> ugh i am hating the reddit today
59 2012-12-10 03:50:23 <jgarzik> gmaxwell: yep. that happens all the time even in absence of any agent bugs :/
60 2012-12-10 03:51:05 <jgarzik> there are so few published APIs versus unpublished (i.e. needing to be reverse engineered by watching browser behavior, then emulating it)
61 2012-12-10 03:52:29 <gmaxwell> There is also the (mostly reasonable) assumption that things can be changed with adequate notice... but thats only true if there is a really powerful AI in charge. :P
62 2012-12-10 03:57:08 <muhoo> ok, i'm trying to build 0.7.1 released version, on debian wheezy, and this happens: https://www.refheap.com/paste/7467
63 2012-12-10 03:57:29 <muhoo> the link command made more readable is this https://www.refheap.com/paste/7466
64 2012-12-10 03:57:59 <muhoo> looks like boost is pissed off about no std::__detail::_List_node_base::_M_unhook(), but i confess to having NFI what that is
65 2012-12-10 03:58:11 <muhoo> it looks like it's not in the library, for sure.
66 2012-12-10 03:58:35 <muhoo> so, um, help?
67 2012-12-10 04:04:35 <jgarzik> Another related project I had was a distributed version of Siri. Essentially a distributed digital assistant that could communicate with its owner through various means (sms, voice, IM, ...). It would act as personal assistant, much like Siri does now, and also communicate with third parties on your behalf (including other humans)
68 2012-12-10 04:06:28 <jgarzik> not a botnet, but your own, personal, distributed bot.
69 2012-12-10 04:07:15 <jgarzik> if you are ever caught in an Action Movie Scenario, you can call it collect from a pay phone in the middle of the desert, etc. ;p
70 2012-12-10 04:09:11 <gmaxwell> jgarzik: I wrote an email on intelligence amplfication that had some of the storj tone of building incremental solutions from current tech: http://people.xiph.org/~greg/ia.txt that you might find interesting.
71 2012-12-10 04:13:24 <phantomcircuit> gmaxwell, that is a terrible name
72 2012-12-10 04:13:25 <phantomcircuit> lol
73 2012-12-10 04:14:48 <jgarzik> Google is evolving in a direction towards Wolfram Alpha... being able to ask natural question questions, and get reasonable answers
74 2012-12-10 04:14:59 <jgarzik> *natural language questions
75 2012-12-10 04:16:32 <jgarzik> and Google definitely already functions as IA for a lot of us :) I make less effort to remember easily google-able details.
76 2012-12-10 04:19:24 <gmaxwell> meh. the WA natural language stuff is ... more impressive than useful.
77 2012-12-10 04:19:54 <gmaxwell> It figures out all kinds of stuff, but then makes it absolutely impossible to ask some questions. You want dx instead of dy? screw you. Not possible!
78 2012-12-10 04:24:04 <jgarzik> It's a dependency of the solution I want, and I'm happy someone else is doing the heavy lifting :)
79 2012-12-10 04:24:34 <jgarzik> Have thought about direct IA but that always seemed less interesting than a robot (or robots) that I could converse with directly.
80 2012-12-10 04:24:55 <jgarzik> Mainly the realm of personal preference :)
81 2012-12-10 04:27:11 <gmaxwell> Maninly the notion I was driving at there that there are two hard problems: interface and AI. So why not seperate them? Meat bast brains are not _that_ costly, esp if you multiplex them well. And they're way more powerful than any AI available.
82 2012-12-10 04:31:27 <amiller> mechanical turk is the first platform that lets you tune on the interface side the way people are used to tuning on the AI side
83 2012-12-10 04:31:32 <jgarzik> true
84 2012-12-10 04:32:01 <jgarzik> I'm surprised nobody has yet done a viable MT running on bitcoins
85 2012-12-10 04:32:03 <amiller> if you can break up a task so that humans can do it without much attention, then it can be effective - if you can improve the interface so that they're marginally more productive, you get the direct savings
86 2012-12-10 04:32:24 <amiller> "brain mining"
87 2012-12-10 04:32:24 <jgarzik> I think there was one amateurish attempt (or maybe that was just forum talk)
88 2012-12-10 04:32:31 <amiller> earn bitcoins using just your dumb face
89 2012-12-10 04:33:07 <weex> coinworker was one, they used a service called crowdflower i think
90 2012-12-10 04:33:22 <weex> and mainly you'd look at pictures to see if they were obscene
91 2012-12-10 04:37:52 <ThomasV_> sounds like a fascinating work where you can truly develop yourself
92 2012-12-10 04:39:17 <weex> there IS a zen to that, you're 100% correct
93 2012-12-10 04:40:05 <weex> but still, a native version of that would be good, rather than depending on the operator of the service to do exchange to keep it funded
94 2012-12-10 04:40:27 <weex> i'd also like to see a pure bitcoin version of an advertising CAPTCHA system like Solve Media
95 2012-12-10 09:07:43 <sta> nothing replaces a nice human. no digital assistant. especially a nice human who will answer calls instead of you while you are right there and tell people that you are "in a meeting" when you don't feel like talking to them
96 2012-12-10 09:10:05 <sta> no answering machine can do that
97 2012-12-10 09:22:28 <Eliel> yes... not yet
98 2012-12-10 10:11:15 <t7> some kinda chat bot could really annoy them :)
99 2012-12-10 10:43:00 <sta> t7: has to pass some version of the turing test
100 2012-12-10 10:43:02 <sta> t7: has to pass some version of the turing test
101 2012-12-10 10:43:22 <sta> everybody likes a human slave
102 2012-12-10 10:43:25 <sta> everybody likes a human slave
103 2012-12-10 10:44:52 <sta> especially one that they can unload their frustrations on, when the phone conversation didn't go well for some reason
104 2012-12-10 10:44:53 <sta> especially one that they can unload their frustrations on, when the phone conversation didn't go well for some reason
105 2012-12-10 10:48:18 <t7> i want a good looking female secretary
106 2012-12-10 10:48:38 <t7> who needs the day off every now and then but she has no holiday time left
107 2012-12-10 11:09:23 <jine> It seriously takes forever to sync the chain nowdays :( Haven't done it in quite some time, but this is a pain in the ...
108 2012-12-10 11:09:24 <jine> It seriously takes forever to sync the chain nowdays :( Haven't done it in quite some time, but this is a pain in the ...
109 2012-12-10 11:25:26 <TD> good day
110 2012-12-10 13:17:18 <gavinandresen> sipa: RE: new checkpoint for 0.7.2: good idea.
111 2012-12-10 13:17:21 <gavinandresen> sipa: RE: new checkpoint for 0.7.2: good idea.
112 2012-12-10 13:35:41 <helo> could miners ever let some users send transactions for free in exchange for the user is running a full node and relaying to the miner?
113 2012-12-10 13:36:17 <helo> s/is //
114 2012-12-10 13:36:44 <t7> helo: I dont see why not
115 2012-12-10 13:36:57 <helo> s/is //
116 2012-12-10 13:36:57 <t7> helo: I dont see why not
117 2012-12-10 13:37:01 <t7> but why would the miner be so grateful ?
118 2012-12-10 13:37:13 <t7> but why would the miner be so grateful ?
119 2012-12-10 13:46:01 <helo> dunno... maybe people become apathetic about running full nodes, and miners want a healthy network?
120 2012-12-10 13:57:17 <helo> maybe just a "Node Day" holiday where the goal is to get as many people running the reference client as is possible
121 2012-12-10 14:00:26 <sipa> ;;bc,blcoks
122 2012-12-10 14:00:27 <gribble> Error: "bc,blcoks" is not a valid command.
123 2012-12-10 14:00:28 <gribble> Error: "bc,blcoks" is not a valid command.
124 2012-12-10 14:00:28 <sipa> ;;bc,blcoks
125 2012-12-10 14:00:29 <sipa> ;;bc,blocks
126 2012-12-10 14:00:35 <gribble> timed out
127 2012-12-10 14:00:37 <gribble> timed out
128 2012-12-10 14:00:38 <sipa> ;;bc,blocks
129 2012-12-10 14:00:40 <sipa> ;;bc,blocks
130 2012-12-10 14:00:43 <gribble> timed out
131 2012-12-10 16:45:38 <jgarzik> chain-verf: spend block @ 180000
132 2012-12-10 16:45:49 <jgarzik> still on 180k, up to 1.6g RAM now
133 2012-12-10 16:46:15 <jgarzik> still on 180k, up to 1.6g RAM now
134 2012-12-10 17:40:26 <novusordo> is it possible to run bitcoin on 2 networks at the same time?
135 2012-12-10 17:40:33 <novusordo> as in, a full node
136 2012-12-10 17:41:40 <novusordo> let's say that I have a network that has no access to the internet whatsoever
137 2012-12-10 17:41:42 <novusordo> let's say that I have a network that has no access to the internet whatsoever
138 2012-12-10 17:41:53 <novusordo> but my internet-connected computer is also connected to that network
139 2012-12-10 17:42:19 <gmaxwell> novusordo: thats mostly a host network stack question, not a bitcoin one??? bitcoin listens on all addresses, so you can connect to it from your other network.
140 2012-12-10 17:42:59 <TD> hmm
141 2012-12-10 17:43:04 <novusordo> could i run a bitcoin node on my computer that connects to the main bitcoin network on the internet, and act as a bridge that other computers on the internet-less network could connect to?
142 2012-12-10 17:43:06 <TD> bitsofproof has bits of code copied from bitcoinj
143 2012-12-10 17:43:12 <novusordo> could i run a bitcoin node on my computer that connects to the main bitcoin network on the internet, and act as a bridge that other computers on the internet-less network could connect to?
144 2012-12-10 17:43:12 <TD> bitsofproof has bits of code copied from bitcoinj
145 2012-12-10 17:43:26 <novusordo> (also, with one network being ipv4 and one being ipv6)
146 2012-12-10 17:43:29 <TD> novusordo: sure
147 2012-12-10 17:43:36 <novusordo> interesting
148 2012-12-10 17:44:46 <novusordo> i'm attempting to get a bitcoin node running on the CJDNS network that bridges to the main network
149 2012-12-10 17:44:48 <novusordo> i'm attempting to get a bitcoin node running on the CJDNS network that bridges to the main network
150 2012-12-10 17:45:15 <TD> huh
151 2012-12-10 17:45:17 <TD> it's using IRC too
152 2012-12-10 17:45:21 <TD> novusordo: oh sweet
153 2012-12-10 17:45:27 <TD> novusordo: let us know if you want help. cjdns is cool
154 2012-12-10 17:45:47 <TD> oh, i see. it's using the testnet
155 2012-12-10 17:45:48 <TD> ok
156 2012-12-10 17:45:56 <novusordo> thanks! i agree, it's pretty amazing
157 2012-12-10 17:46:13 <novusordo> thanks! i agree, it's pretty amazing
158 2012-12-10 17:46:13 <TD> oh, i see. it's using the testnet
159 2012-12-10 17:48:59 <novusordo> oh, in the process, i'm going to be installing bitcoin for the 1st time on my raspberry pi
160 2012-12-10 17:49:02 <novusordo> oh, in the process, i'm going to be installing bitcoin for the 1st time on my raspberry pi
161 2012-12-10 17:49:07 <novusordo> planning on using 0.8
162 2012-12-10 17:49:10 <TD> er
163 2012-12-10 17:49:16 <TD> ACTION is skeptical a pi can keep up in any reasonable time
164 2012-12-10 17:49:17 <TD> ACTION is skeptical a pi can keep up in any reasonable time
165 2012-12-10 17:49:25 <TD> even with 0.8
166 2012-12-10 17:49:26 <TD> even with 0.8
167 2012-12-10 17:49:29 <TD> why do you want to do that?
168 2012-12-10 17:50:00 <novusordo> well, i've been using my pi as an i2p and cjdns node, figured i'd use it
169 2012-12-10 17:50:08 <TD> bitcoin is a lot heavier than most software
170 2012-12-10 17:50:12 <TD> especially on disk IO
171 2012-12-10 17:50:23 <TD> feel free to try it, but don't be surprised if you get bored waiting for it to catch up to the chain head
172 2012-12-10 17:50:24 <TD> feel free to try it, but don't be surprised if you get bored waiting for it to catch up to the chain head
173 2012-12-10 17:50:36 <TD> i suppose, you could sync a database on a faster machine, copy it across and just have it keep up
174 2012-12-10 17:50:45 <novusordo> yeah that's what my next question was about
175 2012-12-10 17:50:52 <novusordo> yeah that's what my next question was about
176 2012-12-10 17:51:15 <novusordo> i have the db synced on my laptop, should i just copy that over? or is there anything with 0.8 that would make it better to simply download from scratch?
177 2012-12-10 17:51:28 <TD> 0.8 uses a totally different db format to 0.7
178 2012-12-10 17:51:34 <maaku> novusordo: copy that over and use the command-line import
179 2012-12-10 17:51:37 <maaku> novusordo: copy that over and use the command-line import
180 2012-12-10 17:51:38 <novusordo> ok, laptop is using 0.7
181 2012-12-10 17:51:40 <TD> your best bet, i think, would be to sync 0.8 on a fast desktop
182 2012-12-10 17:51:41 <novusordo> ok, laptop is using 0.7
183 2012-12-10 17:51:41 <TD> your best bet, i think, would be to sync 0.8 on a fast desktop
184 2012-12-10 17:51:42 <TD> or laptop
185 2012-12-10 17:51:44 <TD> or laptop
186 2012-12-10 17:51:48 <TD> and then copy it over and run 0.8 on the pi, and see how it fairs
187 2012-12-10 17:52:07 <novusordo> hmm, ok
188 2012-12-10 17:52:09 <novusordo> hmm, ok
189 2012-12-10 17:55:44 <maaku> CBitcoinAddress("1???").Get() returns a different script in fTestnet vs main line, why is that?
190 2012-12-10 17:55:51 <maaku> CBitcoinAddress("1???").Get() returns a different script in fTestnet vs main line, why is that?
191 2012-12-10 18:55:21 <jgarzik> gmaxwell: RE last night's discussion, the US military predicts neuro-enhancements in their Global Trends 2030 reports: http://www.wired.com/dangerroom/2012/12/superhumans-instant-cities/
192 2012-12-10 18:55:42 <jgarzik> gmaxwell: RE last night's discussion, the US military predicts neuro-enhancements in their Global Trends 2030 reports: http://www.wired.com/dangerroom/2012/12/superhumans-instant-cities/
193 2012-12-10 19:00:49 <MC1984> cool
194 2012-12-10 19:00:54 <MC1984> cool
195 2012-12-10 19:02:19 <jgarzik> Lots of good papers/panels/links from http://gt2030.com/ apparently
196 2012-12-10 19:14:28 <jgarzik> chain-verf: spend block @ 190000
197 2012-12-10 19:14:33 <jgarzik> 5820 jgarzik 20 0 1811m 1.6g 948 R 98.6 42.7 3644:32 memcheck-amd64-
198 2012-12-10 19:14:34 <jgarzik> 5820 jgarzik 20 0 1811m 1.6g 948 R 98.6 42.7 3644:32 memcheck-amd64-
199 2012-12-10 19:14:38 <jgarzik> ACTION gives up, and hits Ctrl-C
200 2012-12-10 19:14:46 <jgarzik> time to implement checkpoints for a realistic test
201 2012-12-10 19:14:49 <jgarzik> time to implement checkpoints for a realistic test
202 2012-12-10 19:52:33 <Diapolo> sipa: gmaxwell: I tried "bitcoin-qt.exe -Proxy=127.0.0.1:9050 -externalip=yubxrmbxwbcrs5gy.onion -onlynet="Tor" -bind=127.0.0.1" and I get incoming connections with my patch applied... currently I have none outbound though.
203 2012-12-10 19:57:26 <gmaxwell> Diapolo: I can't fathom why! do you still if you run onlynet ipv4?
204 2012-12-10 19:57:43 <Diapolo> I'll try, just a sec
205 2012-12-10 19:59:46 <Diapolo> gmaxwell: seems like no
206 2012-12-10 19:59:49 <Diapolo> gmaxwell: seems like no
207 2012-12-10 20:00:13 <sipa> this doesn't make sense
208 2012-12-10 20:01:06 <sipa> any reason to capitalize Proxy and Tor?
209 2012-12-10 20:01:13 <Diapolo> sipa: bitcoin-qt.exe -Proxy=127.0.0.1:9050 -externalip=yubxrmbxwbcrs5gy.onion -onlynet="IPv4" -bind=127.0.0.1" that's what we talk about now?
210 2012-12-10 20:01:19 <Diapolo> not really
211 2012-12-10 20:01:20 <sipa> i doubt that has anything to do with it
212 2012-12-10 20:02:09 <sipa> yes
213 2012-12-10 20:02:10 <sipa> yes
214 2012-12-10 20:02:21 <sipa> and you say this block connections?
215 2012-12-10 20:02:47 <Diapolo> yeah, no out- or inbound connection is opened
216 2012-12-10 20:02:54 <Diapolo> wait.... now
217 2012-12-10 20:03:29 <sipa> not having an incoming connection is not the same as having incoming connections blocked by tjat code :)
218 2012-12-10 20:03:34 <sipa> *that
219 2012-12-10 20:03:48 <Diapolo> took quite some time, now I have 2 via IPv4
220 2012-12-10 20:03:57 <Diapolo> outbound
221 2012-12-10 20:03:57 <sipa> not having an incoming connection is not the same as having incoming connections blocked by tjat code :)
222 2012-12-10 20:04:39 <sipa> so, with -onlynet=tor, which address does it report in the "accepted connection from" message?
223 2012-12-10 20:04:45 <sipa> so, with -onlynet=tor, which address does it report in the "accepted connection from" message?
224 2012-12-10 20:04:54 <Diapolo> connections to the Tor HS I'm running
225 2012-12-10 20:05:06 <Diapolo> connections to the Tor HS I'm running
226 2012-12-10 20:05:33 <gavinandresen> gmaxwell: re checkpoints: do you really care if we release 0.7.2 at 1,600 blocks past the checkpoint versus 2,016? Both seem equally arbitrary to me....
227 2012-12-10 20:05:41 <sipa> _incoming_ connections
228 2012-12-10 20:05:57 <sipa> gavinandresen: seems like a slippery slope
229 2012-12-10 20:06:04 <sipa> gavinandresen: seems like a slippery slope
230 2012-12-10 20:06:37 <gavinandresen> sipa: Sure. I'd vote "checkpoint is OK if we're 144 blocks (1 day)" past it, because even a 6-block re-org would be a major disaster.
231 2012-12-10 20:06:41 <Diapolo> "addr" : "127.0.0.1:57457",
232 2012-12-10 20:06:43 <gavinandresen> sipa: Sure. I'd vote "checkpoint is OK if we're 144 blocks (1 day)" past it, because even a 6-block re-org would be a major disaster.
233 2012-12-10 20:06:49 <Diapolo> "addr" : "127.0.0.1:57457",
234 2012-12-10 20:07:26 <gavinandresen> ... but I bow to consensus. And waiting until we're past a difficulty adjustment at least has some rhyme or reason.
235 2012-12-10 20:07:54 <sipa> gavinandresen: seems reasonable, given that it is above 120 (which probably the point at which reorgs become very painful anyway), but still... i prefer checkpoints only as a last resort solution to cope with efficiency
236 2012-12-10 20:09:18 <sipa> perhaps we should set a rule once and for all
237 2012-12-10 20:09:26 <sipa> perhaps we should set a rule once and for all
238 2012-12-10 20:09:58 <sipa> but on the other hand, 0.7.2 will be several thousand blocks after checkpoint by the time 0.8 is released anyway
239 2012-12-10 20:11:04 <Diapolo> sipa: incoming == inbound: true, no?
240 2012-12-10 20:11:33 <sipa> yes
241 2012-12-10 20:14:06 <sipa> ;;bc,estimate
242 2012-12-10 20:14:07 <gribble> 3459402.09975206
243 2012-12-10 20:14:12 <Diapolo> sipa: accepted connection 127.0.0.1:57457
244 2012-12-10 20:14:45 <sipa> Diapolo: oh, i think localhost is considered to be a separate network
245 2012-12-10 20:14:57 <sipa> which perhaps doesn't get blocked in -onlynet
246 2012-12-10 20:15:21 <sipa> though that will cause unexpected behavior for people using a remote tor proxy
247 2012-12-10 20:15:26 <Diapolo> remember I used -bind=127.0.0.1
248 2012-12-10 20:15:32 <sipa> irrelevant
249 2012-12-10 20:15:55 <sipa> irrelevant
250 2012-12-10 20:22:04 <Diapolo> sipa: but why do I get 0 connections with -bind=127.0.0.1 removed then?
251 2012-12-10 20:22:47 <sipa> because having a proxy disables listening by default
252 2012-12-10 20:23:07 <sipa> gavinandresen: what do you think about #2034
253 2012-12-10 20:23:08 <sipa> gavinandresen: what do you think about #2034
254 2012-12-10 20:23:30 <sipa> gavinandresen: imho, that's the next low-hanging fruit for a good user experience
255 2012-12-10 20:23:32 <sipa> gavinandresen: imho, that's the next low-hanging fruit for a good user experience
256 2012-12-10 20:24:07 <sipa> (issue, not pull)
257 2012-12-10 20:24:09 <sipa> (issue, not pull)
258 2012-12-10 20:24:13 <gavinandresen> sipa: I tend to trust you and jgarzik and gmaxwell to Do The Right Thing(tm) for network-related stuff.
259 2012-12-10 20:24:16 <gavinandresen> sipa: I tend to trust you and jgarzik and gmaxwell to Do The Right Thing(tm) for network-related stuff.
260 2012-12-10 20:24:29 <sipa> ok
261 2012-12-10 20:24:55 <sipa> gmaxwell, jgarzik: what do you think about #2034 ?
262 2012-12-10 20:25:05 <sipa> oh, we lost jeff
263 2012-12-10 20:25:25 <gavinandresen> IBD experience is still not high on my personal priority list; until we get to "ready to send and receive bitcoins in less than a minute" for new users I don't think we solved the problem.
264 2012-12-10 20:25:59 <Diapolo> lost jeff?
265 2012-12-10 20:26:41 <sipa> well, together with ultraprune, leveldb, parallel sig checking and a few other tgings, IBD mechanism is the differemce between 2 hours and 12 hours of sync time
266 2012-12-10 20:26:49 <sipa> well, together with ultraprune, leveldb, parallel sig checking and a few other tgings, IBD mechanism is the differemce between 2 hours and 12 hours of sync time
267 2012-12-10 20:28:19 <Diapolo> I think that block handling stuff during IBD is worth working on or at least prevent "ERROR: ProcessBlock() : already have block 198076" as that doesn't seem to be a real error then
268 2012-12-10 20:28:25 <Diapolo> I think that block handling stuff during IBD is worth working on or at least prevent "ERROR: ProcessBlock() : already have block 198076" as that doesn't seem to be a real error then
269 2012-12-10 20:28:30 <sipa> Diapolo: that too
270 2012-12-10 20:28:34 <sipa> Diapolo: that too
271 2012-12-10 20:29:04 <sipa> gavinandresen: do you find the hal code refactor an improvement at least, already?
272 2012-12-10 20:29:57 <gavinandresen> sipa: yes
273 2012-12-10 20:32:24 <sipa> some more code to be able to test whether the output of both EC_POINT_mul operations is identical would be nice
274 2012-12-10 20:45:27 <sipa> hmm, easiest way would be some VERIFY_OPTIMIZED_SECP256K1 #define, which is only set when building test code, which runs both versions
275 2012-12-10 20:45:40 <sipa> but it'd be nicer if the test code could be localized in the test code
276 2012-12-10 20:51:46 <thermoman> ETA for 0.7.2?
277 2012-12-10 20:52:40 <sipa> very soon
278 2012-12-10 21:45:43 <etotheipi_> sipa: I have an example for you about leveldb optimization ... you could put all the headers and tx into a single DB, since they all have unique hashes
279 2012-12-10 21:45:58 <etotheipi_> and you're rarely looking for a hash without knowing what it would be
280 2012-12-10 21:46:25 <etotheipi_> but if I want to iterate over all headers, even if I have a list of headers to iterate over, LevelDB is not optimized for random access like that
281 2012-12-10 21:46:54 <etotheipi_> but if instead if use "HD"+Hash256 for headers and "TX"+Hash256 for tx keys... then the DB is "presorted"
282 2012-12-10 21:47:07 <sipa> well of course, that's what we do
283 2012-12-10 21:47:26 <sipa> if you have any useful information about the key, make sure it gets sorted on
284 2012-12-10 21:47:32 <sipa> but you could go further than that
285 2012-12-10 21:48:05 <etotheipi_> sipa: my point was not that the sorting alone is preferred, but that leveldb is optimized for that sequential access
286 2012-12-10 21:48:16 <etotheipi_> and I was wondering if there were other things that you were doing like that
287 2012-12-10 21:48:19 <sipa> etotheipi_: the nice thing is that the sstable format even removes that redundancy
288 2012-12-10 21:48:24 <zapsoda> Hey everyone
289 2012-12-10 21:48:57 <zapsoda> So im reading about accounts here https://en.bitcoin.it/wiki/Accounts_explained how do i make a new account?
290 2012-12-10 21:49:24 <etotheipi_> sipa: I started out by separating the headers from the Tx, but then I realized not only would be it be plenty efficient to keep them in the same DB, but it makes it possible to do transactions on both sub-DBs at the same time
291 2012-12-10 21:49:33 <etotheipi_> *transactions==batchwrite?
292 2012-12-10 21:50:09 <sipa> etotheipi_: but you could split the data into a txid->num and a num->txdata, which would make the actual transaction data at higher locality
293 2012-12-10 21:50:36 <sipa> in case the txdata was very big, and you often needed access to transactions that were created close to eachother, that would make sense
294 2012-12-10 21:50:38 <etotheipi_> sipa: num==blockehight?
295 2012-12-10 21:51:01 <sipa> just some ever-increasing number you locally use to identify transactions
296 2012-12-10 21:51:06 <etotheipi_> sipa: exactly
297 2012-12-10 21:51:13 <etotheipi_> I had already considered doing that
298 2012-12-10 21:51:21 <sipa> right now, however, my txdata is on average smaller than a single txid
299 2012-12-10 21:51:24 <etotheipi_> it just added a little extra complexity for handling reorgs since blockheight may not be constant
300 2012-12-10 21:51:32 <sipa> so it would definitely not be worth it
301 2012-12-10 21:52:07 <sipa> etotheipi_: also, headers and transactions are in separate files here, because they belong to a fundamentally different dataset
302 2012-12-10 21:52:23 <sipa> one is metadata about the blocks you have on disk, the other is a representation of the UTXO data
303 2012-12-10 21:52:39 <sipa> so you can download one, and have the other one rebuilt from scratch
304 2012-12-10 21:52:54 <etotheipi_> sipa: that's exaclty what I originally planned to do with headers
305 2012-12-10 21:52:55 <etotheipi_> for that reason
306 2012-12-10 21:53:08 <etotheipi_> but then I liked the idea of putting things into consolidated databases
307 2012-12-10 21:53:14 <etotheipi_> because I like the fact that I can batch-update the DB
308 2012-12-10 21:53:21 <etotheipi_> and there's no chance for desynchronization
309 2012-12-10 21:53:27 <sipa> well, you don't actually need consistency between them
310 2012-12-10 21:53:45 <etotheipi_> i.e. -- put in header and tx... and make sure they're all there or none of them
311 2012-12-10 21:53:46 <sipa> but in practice, it's probably easer
312 2012-12-10 21:53:52 <etotheipi_> header&tx isn't the best example
313 2012-12-10 21:54:09 <etotheipi_> but I like the synchronization potential for keeping things in the same DB
314 2012-12-10 21:55:06 <sipa> yeah, it certainly has advantage wrt sync
315 2012-12-10 21:56:41 <etotheipi_> sipa: I'm not ready to do a full LevelDB upgrade (I'm just doing a partial upgrade to replace some map<> objects in RAM), but I've already started working out how I want to do the full upgrade -- such that the DB accommodates full nodes, pruned nodes, pruned lite nodes, etc... and the only distinction is the level of data available
316 2012-12-10 21:57:39 <sipa> well there's two databases here now: one with data about the block tree, one with data about the current state
317 2012-12-10 21:57:42 <etotheipi_> sipa: I might look to see what you've done and try to collaborate....
318 2012-12-10 21:58:18 <etotheipi_> sipa: for instance, the tx store would store raw tx without the TxOuts, there would be a separate OutPoint entry for it to go look up the TxOuts
319 2012-12-10 21:58:19 <sipa> if i add indexes, if they are about the entire history they go in in the first, if they're about unspent coins, they go in the second
320 2012-12-10 21:58:41 <sipa> etotheipi_: i thougt long about that, but txid's are very large compared to txouts
321 2012-12-10 21:58:52 <etotheipi_> sipa: yes, that's my concern, too
322 2012-12-10 21:59:03 <sipa> so i store UTXO's per txid, and not per outpoint
323 2012-12-10 21:59:20 <sipa> though leveldb is actually very efficient about keys that share a prefix
324 2012-12-10 21:59:27 <sipa> i didn't know that at the time
325 2012-12-10 22:00:24 <etotheipi_> sipa: I had thought that the outpoint store would simply be {txID --> a+TxOut(a) + b+TxOut(b) + c+TxOut(c)}
326 2012-12-10 22:00:42 <etotheipi_> that way, it can hold partial tx or full tx
327 2012-12-10 22:00:55 <etotheipi_> but updating it would be slow
328 2012-12-10 22:01:02 <etotheipi_> (*slower)
329 2012-12-10 22:01:17 <sipa> that's why you try to bundle multiple updates to the same txid in one batch )
330 2012-12-10 22:01:43 <etotheipi_> sipa: sure... but how often are your operations actually hitting the same tx?
331 2012-12-10 22:01:43 <sipa> have a look at how i encode utxo data... it has 0 redundancy :p
332 2012-12-10 22:02:05 <etotheipi_> sipa: I'll take you up on that... hopefully in the next month when I get a chance to do it
333 2012-12-10 22:02:15 <etotheipi_> I want to get the new wallets first
334 2012-12-10 22:02:37 <sipa> etotheipi_: depends on your settings, but with "larger" cache settings, it easily bundles millions of updated tx's in one batch
335 2012-12-10 22:02:41 <etotheipi_> then maybe I'll have some time to rewriting how Armory handles all that data
336 2012-12-10 22:03:00 <sipa> so yes, i hit the same tx many times - most often, it's spent entirely before ever hitting disk
337 2012-12-10 22:03:23 <sipa> during IBD, that is
338 2012-12-10 22:03:37 <etotheipi_> ahh
339 2012-12-10 22:16:01 <etotheipi_> sipa: do you have any understanding (or "feel") for how resistant leveldb will be to corruption? i.e. how many times do I have to cut the power during a write operation for a non-tangible chance to be corrupted?
340 2012-12-10 22:16:21 <etotheipi_> theoretically, if it's done right, it should never corrupt... correct?
341 2012-12-10 22:16:53 <sipa> etotheipi_: i haven't ever seen it myself (including tests where i make my system lose power during writes), but there have been reports of windows users with corrupted db's
342 2012-12-10 22:17:17 <etotheipi_> ...stupid windows...
343 2012-12-10 22:17:21 <gmaxwell> etotheipi_: what you're describing is achievable. I don't know if leveldb achieves it.
344 2012-12-10 22:18:19 <etotheipi_> maybe I can RAID5 all the data into 4 different databases
345 2012-12-10 22:18:24 <sipa> haha
346 2012-12-10 22:18:33 <sipa> that won't give you atomicity
347 2012-12-10 22:18:40 <sipa> afaict, it would make it even harder
348 2012-12-10 22:18:56 <etotheipi_> gah
349 2012-12-10 22:19:00 <sipa> as you don't have inter-database guarantees
350 2012-12-10 22:19:04 <etotheipi_> right
351 2012-12-10 22:19:16 <sipa> well, you have some, if you use sync writes
352 2012-12-10 22:19:19 <etotheipi_> I guess... I'm curious how much extra headache I'm going to have from this...
353 2012-12-10 22:19:45 <sipa> there are alternatives
354 2012-12-10 22:19:48 <etotheipi_> one major benefit of the fact that Armory rescans every load, is that it's impossible to corrupt (unless blk000X.dat is corrupted)
355 2012-12-10 22:19:52 <sipa> i guess sqlite has had more testing
356 2012-12-10 22:20:15 <etotheipi_> or rathre, any corruption in my indexing is fixed by an Armory restart
357 2012-12-10 22:20:19 <sipa> the windows leveldb port seems badly maintained
358 2012-12-10 22:21:30 <etotheipi_> I'm not looking forward to having corruptible, persistent storage, and dealing with different bad states... but it has to be done sometime
359 2012-12-10 22:25:51 <etotheipi_> one other thing... if I want to dump a *list* of serialized objects into the same DB, as long as they are unique, would I just put them all in the key and leave the value empty?
360 2012-12-10 22:26:00 <etotheipi_> is there a convention for this?
361 2012-12-10 22:27:13 <sipa> if you have a list, you probably care about the order?
362 2012-12-10 22:27:18 <sipa> or do you have a set?
363 2012-12-10 22:31:26 <etotheipi_> oh yeah, this is a set
364 2012-12-10 22:31:41 <sipa> then there's no reason not to use the key
365 2012-12-10 22:31:49 <etotheipi_> if it's an ordered list, I guess I just use keyprefix+4byteIndex-->data
366 2012-12-10 22:32:25 <sipa> as that 1) guarantees the set property and 2) check for presence in the set quickly
367 2012-12-10 22:32:46 <etotheipi_> sipa: have you enabled bloom filtering in leveldb?
368 2012-12-10 22:32:50 <sipa> yes
369 2012-12-10 22:32:52 <etotheipi_> I couldn't figure out how to set it up
370 2012-12-10 22:33:01 <sipa> TD did that part
371 2012-12-10 22:33:02 <etotheipi_> do you have a chunk of C++ code that enables it
372 2012-12-10 22:33:17 <etotheipi_> it looks like the interface changed recently, and I can't find one that works
373 2012-12-10 22:33:49 <sipa> https://github.com/bitcoin/bitcoin/blob/master/src/leveldb.cpp
374 2012-12-10 22:35:17 <etotheipi_> well that's easy... I don't know why I couldn't get that to work
375 2012-12-10 23:06:13 <phantomcircuit> etotheipi_, it's funny the on disk storage is really just a cache of the network state
376 2012-12-10 23:06:23 <phantomcircuit> but even then it's hard to get the performance to be good enough
377 2012-12-10 23:07:19 <sipa> what are you talking about?
378 2012-12-10 23:13:46 <phantomcircuit> sipa, he was discussing leveldb consistency guarantees before