1 2012-08-26 00:37:19 <jaxtr> broooo
2 2012-08-26 01:19:08 <gmaxwell> Gah Gavin! Tho shall not make the txout bloat problem worse! (RE: https://bitcointalk.org/index.php?topic=103258.msg1132776#msg1132776 )
3 2012-08-26 07:38:38 <wumpus> <Luke-Jr> sipa: beyond the documented split, 7f3ccb5 also replaces a boost::interprocess::interprocess_mutex with a CCriticalSection - what was the purpose of that? <- because an interproces mutex was a terribly inefficient thing to use within a process
4 2012-08-26 07:38:52 <wumpus> especially on mac it resulted in busy loops
5 2012-08-26 07:39:36 <Guest58083> hi any one pls help me how to connect bit coin API through Vb
6 2012-08-26 09:49:48 <sipa> Luke-Jr: that ppmutexOpenSSL? because it was duplication
7 2012-08-26 09:50:10 <sipa> wumpus, Luke-Jr: the change from interprocess thread locks was done later
8 2012-08-26 16:24:37 <Luke-Jr> sipa: what duplication? :o
9 2012-08-26 16:41:17 <sipa> Luke-Jr: now there is one mechanism for lockin, easily changable with another implementation
10 2012-08-26 16:41:33 <sipa> and it's used for everything
11 2012-08-26 18:57:31 <Luke-Jr> hrm, how was height-in-coinbase tested? :x
12 2012-08-26 19:05:44 <Luke-Jr> hmm, it seems like DoSing nodes relaying block-height-not-in-coinbase is a bad idea until a higher criteria at least
13 2012-08-26 19:14:45 <Luke-Jr> forrestv: ping
14 2012-08-26 19:15:09 <forrestv> pong
15 2012-08-26 19:15:37 <Luke-Jr> How should getmemorypool behave (in backports)? Simply refuse to work when version==1 becomes invalid (95%), or start adding "height" at 75%?
16 2012-08-26 19:16:26 <forrestv> is bip 0034 being backported?
17 2012-08-26 19:16:49 <Luke-Jr> yes, it's a mandatory protocol change, so creates bugs in clients not implementing it
18 2012-08-26 19:16:56 <forrestv> ah, ok
19 2012-08-26 19:17:21 <forrestv> you can add height without any sort of check..
20 2012-08-26 19:17:30 <gmaxwell> forrestv: have you updated p2pool yet for the 'last minute' change from getmemorypool to getblocktemplate?
21 2012-08-26 19:17:31 <forrestv> it being there before the 75% point shouldn't hurt anything
22 2012-08-26 19:17:31 <Luke-Jr> forrestv: keep in mind that there is at least a chance Debian stable will stick to 0.6.x
23 2012-08-26 19:17:33 <Luke-Jr> true
24 2012-08-26 19:18:11 <forrestv> gmaxwell, yeah, p2pool works with whatever was in bitcoin git as of two days ago
25 2012-08-26 19:18:19 <gmaxwell> great
26 2012-08-26 19:18:58 <Luke-Jr> forrestv: so I guess the only other concern for GMP backport is, whether/when to express version==2, since it requires block-maker cooperation
27 2012-08-26 19:19:26 <Luke-Jr> which IMO the same question applies to: throw an error, or do it at 75%? :p
28 2012-08-26 19:19:44 <Luke-Jr> (or maybe 95%?)
29 2012-08-26 19:20:38 <Luke-Jr> another option might be to make a minimal clone called getmemorypool2 to express version==2 support
30 2012-08-26 19:20:42 <forrestv> do most GMP clients simply pass version through to blocks? and do any of them have BIP0034 support?
31 2012-08-26 19:21:31 <Luke-Jr> AFAIK yes and no
32 2012-08-26 19:21:50 <forrestv> yeah, that's definitely wrong behavior, as i discovered with p2pool
33 2012-08-26 19:21:59 <forrestv> clients should use min(version from getblocktemplate, VERSION_I_KNOW_ABOUT) in blocks
34 2012-08-26 19:22:37 <gmaxwell> sipa: in ultraprune, do you have an opinion on storing blocks in different directories? I think it would be useful to be able to store blocks/undo logs on slow spinning media while the UTXO database is on a SSD or in ram.
35 2012-08-26 19:23:12 <forrestv> if they did that.. we could just make getblocktemplate pass the correct version through at all times
36 2012-08-26 19:23:12 <gmaxwell> I'm kinda bummed about blk0002.dat being >650MiB already.
37 2012-08-26 19:23:18 <forrestv> but they don't so we need some kind of workaround
38 2012-08-26 19:23:21 <forrestv> is that correct?
39 2012-08-26 19:23:29 <Luke-Jr> forrestv: getblocktemplate is not being backported.
40 2012-08-26 19:23:36 <forrestv> ah
41 2012-08-26 19:23:58 <gmaxwell> I'm thinking that the BIP34 backport should just disable getmemorypool.
42 2012-08-26 19:24:20 <gmaxwell> If you're running an out of date bitcoin backport on debian stable you're a hazard to the network.
43 2012-08-26 19:25:08 <Luke-Jr> backports aren't out of date or a hazard
44 2012-08-26 19:25:17 <gmaxwell> they are if they are out of date.
45 2012-08-26 19:25:26 <Luke-Jr> I'm not talking about out of date ones though.
46 2012-08-26 19:25:32 <Luke-Jr> those can't be changed.
47 2012-08-26 19:25:35 <gmaxwell> well, then backport getblocktemplate.
48 2012-08-26 19:25:44 <Luke-Jr> that's a new feature, and breaks compatibility.
49 2012-08-26 19:26:01 <gmaxwell> No miner that will create correct blocks post change works without it.
50 2012-08-26 19:26:13 <Luke-Jr> hmm
51 2012-08-26 19:27:16 <gmaxwell> I don't think there is anything too controversial about not supporting mining in sufficiently old backports.
52 2012-08-26 19:27:57 <Luke-Jr> gmaxwell: I'm thinking forrestv may want to support distro stable versions for p2pool, otherwise sure
53 2012-08-26 19:27:58 <gmaxwell> (in particular, staying on an old client mostly just harms yourself, but mining on an old one potentially hurts other people)
54 2012-08-26 19:28:27 <Luke-Jr> we're not talking about out-of-date-harming, just not-the-latest-features-but-still-maintained..
55 2012-08-26 19:28:48 <gmaxwell> Considering the performance changes coming up (e.g. changing to ultraprune and leveldb; async block validation, etc), I doubt it.
56 2012-08-26 19:29:01 <sipa> gmaxwell: the block database and utxo database are (almost) totally independent, so they can be separated
57 2012-08-26 19:29:22 <sipa> there are no atomic transactions to both combined
58 2012-08-26 19:30:50 <gmaxwell> If the filesystem layout is setup right it would be possible to move them simply by symlinking the right subdirectory. That would be handy.
59 2012-08-26 19:31:20 <sipa> gmaxwell: when we have leveldb, both databases will be a directory each, and nothing outside those directories
60 2012-08-26 19:31:37 <forrestv> Luke-Jr, for me, always inserting the height, and keeping version at 1 until version==1 is invalid, and then setting version to 2 seems best
61 2012-08-26 19:32:20 <sipa> gmaxwell: so i guess that means you can put blocks, the block db, and the utxo db on three different devices if need be
62 2012-08-26 19:32:23 <forrestv> that will prevent anybody from creating invalid version==2 blocks up until version==2 is necessary
63 2012-08-26 19:32:48 <forrestv> and by that point, the p2pool update i'm pushing should be adopted, so blocks will have the height in coinbase
64 2012-08-26 19:32:49 <Luke-Jr> forrestv: what do you think of gmaxwell's proposal? leave getmemorypool as-is, but error when v=1 is dead, and backport BIP22 minimally?
65 2012-08-26 19:36:46 <forrestv> i don't think it's really necessary.. p2pool, as it is in git, will produce correct BIP 0034 blocks even if only getmemorypool is present (if it's returning version=2)
66 2012-08-26 19:37:31 <forrestv> it requests the block height using "getblock" so the height key isn't even necessary
67 2012-08-26 19:37:54 <Luke-Jr> gmaxwell: are you ok with forrestv's solution then? <.<
68 2012-08-26 19:52:50 <Luke-Jr> http://codepad.org/j8fALwsa <-- look ok? gmaxwell forrestv
69 2012-08-26 19:54:45 <forrestv> looks good to me
70 2012-08-26 20:58:59 <MC-Eeepc> doing about 10,000 blocks per 24 hrs here
71 2012-08-26 20:59:01 <MC-Eeepc> lel
72 2012-08-26 21:00:00 <MC-Eeepc> when did SD start
73 2012-08-26 21:00:07 <MC-Eeepc> i think it was more than 164 days ago
74 2012-08-26 21:00:16 <MC-Eeepc> oh lawdy.......
75 2012-08-26 21:00:40 <MC-Eeepc> less than*
76 2012-08-26 21:02:30 <sipa> what hardware?
77 2012-08-26 21:08:37 <MC-Eeepc> intel atom 1.6
78 2012-08-26 21:08:55 <MC-Eeepc> but its not pegged on 100%, and the disk is an SSD
79 2012-08-26 21:09:16 <MC-Eeepc> well its a USB drive
80 2012-08-26 21:09:21 <sipa> ah :)
81 2012-08-26 21:09:33 <sipa> interested in trying my ultraprune branch?
82 2012-08-26 21:09:50 <MC-Eeepc> for windows?
83 2012-08-26 21:10:03 <sipa> you need bitcoind or bitcoin-qt?
84 2012-08-26 21:10:15 <MC-Eeepc> qt
85 2012-08-26 21:11:40 <gmaxwell> MC-Eeepc: don't call a usb disk a ssd. :)
86 2012-08-26 21:12:08 <MC-Eeepc> i corrected the error lol
87 2012-08-26 21:12:18 <MC-Eeepc> but its not much faster using the actual internal SSD
88 2012-08-26 21:12:55 <MC-Eeepc> im using a usb drive in order to not rape all my write cycles away on this expensive SSD
89 2012-08-26 21:12:56 <Luke-Jr> gmaxwell: USB drives are usually NAND, so are SSD
90 2012-08-26 21:13:29 <sipa> if we're going into pedantic mode, SSD's are usually not disks at all
91 2012-08-26 21:13:39 <Luke-Jr> disk != disc :P
92 2012-08-26 21:14:21 <sipa> anyway
93 2012-08-26 21:14:32 <sipa> MC-Eeepc: what did you benchmark to claim that there is little speed difference?
94 2012-08-26 21:15:37 <MC-Eeepc> i tried to sync the chain to the SSD and gave up after 2 days ish
95 2012-08-26 21:15:52 <MC-Eeepc> so i got this USB drive instead
96 2012-08-26 21:16:04 <MC-Eeepc> its been longer than two days so i suppose there is a difference
97 2012-08-26 21:16:13 <MC-Eeepc> however both are slow
98 2012-08-26 21:16:33 <Luke-Jr> tmpfs is nice
99 2012-08-26 21:17:21 <MC-Eeepc> well i cant see the bottleneck, unless the database ops consist of innumerable tiny little writes
100 2012-08-26 21:17:30 <sipa> they do
101 2012-08-26 21:17:33 <MC-Eeepc> because flash storage is terrible at that
102 2012-08-26 21:17:45 <MC-Eeepc> oh well theres my answer
103 2012-08-26 21:18:04 <midnightmagic> don't use SSD for databases.
104 2012-08-26 21:18:23 <gmaxwell> Luke-Jr: sure sure. But performance wise almost all usb storage looks more like drum memory from a pdp-11.
105 2012-08-26 21:18:31 <MC-Eeepc> this is an eepc so i have no choice
106 2012-08-26 21:18:39 <midnightmagic> MC-Eeepc: What generation?
107 2012-08-26 21:18:41 <MC-Eeepc> besides SSD is the future
108 2012-08-26 21:18:47 <midnightmagic> MC-Eeepc: No it's not.
109 2012-08-26 21:18:55 <Luke-Jr> gmaxwell: :P
110 2012-08-26 21:19:13 <Luke-Jr> so let's talk about FSSD or ESSD :P
111 2012-08-26 21:19:16 <sipa> raw storage in spinning disks, with some SSDs in front for caching maybe :)
112 2012-08-26 21:19:17 <Luke-Jr> (Fast/Expensive)
113 2012-08-26 21:19:17 <MC-Eeepc> this is a 901 but with 2gb rams and 70 ish GB SSD
114 2012-08-26 21:20:02 <MC-Eeepc> the SSD is worth more than the rest of the machine but the ebay seller didnt appear to know that lol
115 2012-08-26 21:20:25 <Luke-Jr> or the ebay seller knows the SSD is dead <.<
116 2012-08-26 21:20:48 <midnightmagic> MC-Eeepc: I run bitcoind on a 701. :)
117 2012-08-26 21:20:51 <MC-Eeepc> well its been trucking for 6 months so
118 2012-08-26 21:21:05 <MC-Eeepc> i think smart reported power on hours of 600 so its not old
119 2012-08-26 21:21:53 <MC-Eeepc> midnightmagic how on earth
120 2012-08-26 21:22:18 <midnightmagic> MC-Eeepc: I jumpstart with a copy of blk*, but it works just fine after that.
121 2012-08-26 21:22:41 <midnightmagic> MC-Eeepc: Oh, external HDD btw.
122 2012-08-26 21:22:48 <midnightmagic> (But it's over USB..)
123 2012-08-26 21:22:54 <MC-Eeepc> oh yeah you can run bitcoin on an average phone for the foreseeable prob, as long as you skip the full sync
124 2012-08-26 21:23:18 <midnightmagic> the 701 is basically useless for a full sync from even a LAN node.
125 2012-08-26 21:24:30 <MC-Eeepc> its a shame because it so obvious that those in africa etc will not be able to participate in bitcoin on thier own volition, but only by trusting some people running full nods in the glorious west
126 2012-08-26 21:24:40 <MC-Eeepc> as if that has worked out for them before
127 2012-08-26 21:25:29 <gmaxwell> MC-Eeepc: how about you test ultraprune and spare us the histronics? :)
128 2012-08-26 21:25:56 <MC-Eeepc> ok then
129 2012-08-26 21:29:52 <gmaxwell> and fwiw, its worth I successfully synced up bitcoin about a month ago on a >10 year old PIII. I just ran it over night. The problem you're hitting is almost certantly a product of usb storage that can only do a few write transations per second.
130 2012-08-26 21:30:12 <MC-Eeepc> most certainly
131 2012-08-26 21:30:17 <gmaxwell> That hardware will probably also have poor write endurance and will fail after a fair bit of use.
132 2012-08-26 21:30:50 <MC-Eeepc> however remember i tried a full sync some months ago on a 1.6 first gen centrino, with a HDD and it took like 24 hrs
133 2012-08-26 21:30:54 <MC-Eeepc> i sent you the log
134 2012-08-26 21:31:05 <MC-Eeepc> and that was BEFORE satoshidice
135 2012-08-26 21:31:45 <gmaxwell> Yes, and we've made it substantially faster in the more recent months; and taking 24 hours to full sync doesn't support your woe-is-the-poor-people-of-africa.
136 2012-08-26 21:31:48 <MC-Eeepc> if this USB drive fails then meh, it was ?20 for 32gb
137 2012-08-26 21:32:03 <gmaxwell> MC-Eeepc: hope you'll have good wallet backups. :)
138 2012-08-26 21:32:24 <MC-Eeepc> i dont have any btc on this machine and never will
139 2012-08-26 21:32:35 <MC-Eeepc> i just want to store and serve the full chain
140 2012-08-26 21:34:35 <gmaxwell> MC-Eeepc: watcha gonna serve it to?
141 2012-08-26 21:34:41 <MC-Eeepc> yeah i said the 24 hours was before the dice nonsense, and also there is a lot of africa that could not have a continuous net connection for 24hrs + anyway
142 2012-08-26 21:35:05 <MC-Eeepc> i will serve it to anyone that wants it
143 2012-08-26 21:35:15 <gmaxwell> MC-Eeepc: you don't have to have a continual net connection though.
144 2012-08-26 21:35:44 <MC-Eeepc> on first sync
145 2012-08-26 21:37:21 <gmaxwell> MC-Eeepc: you don't have to have a continual net connection on first sync either.
146 2012-08-26 21:37:57 <gmaxwell> MC-Eeepc: if you only care about serviing the chain, perhaps you should just run your whole node in tmpfs.
147 2012-08-26 21:38:30 <MC-Eeepc> 2gb ram here
148 2012-08-26 21:38:45 <gmaxwell> Sure. Add several gigs of swap.
149 2012-08-26 21:38:56 <MC-Eeepc> bitcoin asks for blocks one after another though, how do you not need a constant connection
150 2012-08-26 21:39:07 <MC-Eeepc> swap is disabled for SSD longevity and speed etc
151 2012-08-26 21:39:28 <gmaxwell> MC-Eeepc: using tmp fs can only _decrease_ the writes you're doing.
152 2012-08-26 21:39:46 <gmaxwell> MC-Eeepc: because it can happily wait until the connectivity comes back to continue.
153 2012-08-26 21:40:04 <MC-Eeepc> yeah but that takes even longer then
154 2012-08-26 21:40:43 <gmaxwell> MC-Eeepc: sure, but how long did it take you to get the computer in the first place? Seriously... not everything has to be instant gratification.
155 2012-08-26 21:41:24 <MC-Eeepc> it has to be rather close to it if you want people to actually use your shit these days
156 2012-08-26 21:42:03 <MC-Eeepc> comapnies build billion dollar datacenters on every continent to shave off 70ms for a couple tens of millions of people you know
157 2012-08-26 21:42:29 <gmaxwell> Then they can use some SPV client then. You don't get to define the parameters so that there are no solutions and then complain that there are no solutions and cause me to care.
158 2012-08-26 21:45:09 <MC-Eeepc> wat
159 2012-08-26 21:59:29 <MC-Eeepc> you seem rather perturbed at my offhand africa comment
160 2012-08-26 21:59:55 <MC-Eeepc> no ones criticising anyone really, otherwise i wouldnt even be here
161 2012-08-26 22:00:11 <MC-Eeepc> anyway how do i do this ultraprune shit then
162 2012-08-26 22:11:08 <sipa> MC-Eeepc: my gitian environment is currently broken; i don't think i can do a windows build now
163 2012-08-26 22:11:46 <MC-Eeepc> oh
164 2012-08-26 22:11:58 <sipa> BlueMatt: can you?
165 2012-08-26 22:12:04 <Luke-Jr> ACTION wonders if we have 3 people for 0.7 then O.o
166 2012-08-26 22:12:19 <sipa> Luke-Jr: i'll get it up one of the next days, no worries
167 2012-08-26 22:16:14 <gmaxwell> sipa: won't the pulltester do it if you update the pull?
168 2012-08-26 22:17:31 <sipa> gmaxwell: yup, doing that; had to fix unit tests
169 2012-08-26 22:24:10 <sipa> gmaxwell: know of any way to speed the pull tester up? :)
170 2012-08-26 22:50:25 <genjix> gmaxwell: you're on bash.org http://bash.org/?latest
171 2012-08-26 22:50:33 <genjix> http://bash.org/?952948
172 2012-08-26 22:51:17 <Luke-Jr> sipa: should be good to pull now
173 2012-08-26 22:51:28 <Luke-Jr> #1716*
174 2012-08-26 22:51:52 <sipa> gmaxwell: context...?
175 2012-08-26 23:12:14 <devrandom> sipa: have you tried gitian + lxc? should be easier to work with than kvm
176 2012-08-26 23:13:08 <sipa> devrandom: last time i tried, it didn't work, but that was some time ago
177 2012-08-26 23:14:05 <genjix> ;;seen phantomcircuit
178 2012-08-26 23:14:05 <gribble> phantomcircuit was last seen in #bitcoin-dev 3 days, 17 hours, 4 minutes, and 27 seconds ago: <phantomcircuit> sad panda
179 2012-08-26 23:14:45 <devrandom> sipa: the latest also works around the vmbuilder issue (hang on unmount)
180 2012-08-26 23:15:13 <sipa> devrandom: let me try it
181 2012-08-26 23:16:39 <devrandom> I'll be around for about an hour if you run into issues, or later-tell me. I'd like to make it more pain free to use...
182 2012-08-26 23:17:17 <Luke-Jr> devrandom: it'd be pain-free if it worked on other distros, and without Ruby??? :P
183 2012-08-26 23:17:53 <devrandom> Luke-Jr: do the "other distros" at least have debootstrap?
184 2012-08-26 23:18:02 <Luke-Jr> devrandom: no? :p
185 2012-08-26 23:18:13 <devrandom> what do they have?
186 2012-08-26 23:18:32 <devrandom> (for building an image)
187 2012-08-26 23:18:38 <Luke-Jr> tar? <.<
188 2012-08-26 23:19:33 <devrandom> heh... but we do need to build an image, so need to install packages
189 2012-08-26 23:19:42 <Luke-Jr> or use a template? :P
190 2012-08-26 23:20:00 <devrandom> where would the template come from?
191 2012-08-26 23:20:17 <Luke-Jr> http://en.gentoo-wiki.com/wiki/LXC#Template_scripts
192 2012-08-26 23:20:20 <devrandom> and how would it be verified not to have been tampered with?
193 2012-08-26 23:20:40 <Luke-Jr> well, to do that, you need to build everything from source <.<
194 2012-08-26 23:21:16 <devrandom> Luke-Jr: well, that may be a good medium term goal...
195 2012-08-26 23:21:48 <Luke-Jr> you'd end up with a Gentoo chroot I think lol
196 2012-08-26 23:22:00 <devrandom> that might not be a bad idea
197 2012-08-26 23:22:18 <devrandom> I notice that lxc has a debian template, which calls debootstrap
198 2012-08-26 23:22:39 <Luke-Jr> hmm
199 2012-08-26 23:22:42 <devrandom> Luke-Jr: what distro do you use? gentoo?
200 2012-08-26 23:22:46 <Luke-Jr> ACTION ponders if Gentoo can be made fully deterministic
201 2012-08-26 23:22:48 <Luke-Jr> yes
202 2012-08-26 23:24:28 <devrandom> Luke-Jr: is there an 'emerge' (or whatever packages are called) for bootstrap in gentoo?
203 2012-08-26 23:24:57 <devrandom> oh yes, there is
204 2012-08-26 23:25:37 <Luke-Jr> devrandom: emerge -e world <-- rebuilds everything
205 2012-08-26 23:25:38 <devrandom> so if I create a debootstrap VM build option, it should work?
206 2012-08-26 23:25:49 <Luke-Jr> I suppose
207 2012-08-26 23:26:03 <Luke-Jr> using lxc-create might be better tho
208 2012-08-26 23:26:22 <devrandom> it will just in turn call debootstrap?
209 2012-08-26 23:27:09 <devrandom> but yes, could just create a suitable lxc template that calls debootstrap for now
210 2012-08-26 23:27:55 <devrandom> I do think that gentoo is more suitable in the long term because it builds everything from source
211 2012-08-26 23:28:28 <sipa> well, obviously someone could rig the 1st stage compiler you compile the rest with ;0
212 2012-08-26 23:29:36 <devrandom> sipa: yes, a recursive compiler trojan is a weak point of all this
213 2012-08-26 23:30:09 <devrandom> there was some paper about using two compilers to detect such a thing
214 2012-08-26 23:30:51 <devrandom> there it is: http://www.dwheeler.com/trusting-trust/
215 2012-08-26 23:32:38 <sipa> http://scienceblogs.com/goodmath/2007/04/15/strange-loops-dennis-ritchie-a/
216 2012-08-26 23:33:36 <devrandom> yup
217 2012-08-26 23:34:36 <devrandom> fun :)
218 2012-08-26 23:36:19 <[Tycho]> Somehow this reminds me of "Primer" movie.
219 2012-08-26 23:39:30 <devrandom> hm.. the obfuscator idea sounds like a good countermeasure
220 2012-08-26 23:54:51 <sipa> devrandom: http://pastebin.com/arKA37xr
221 2012-08-26 23:55:27 <sipa> ubuntu 12.04
222 2012-08-26 23:57:19 <gribble> The operation succeeded.
223 2012-08-26 23:57:19 <sipa> ;;later tell devrandom http://pastebin.com/arKA37xr
224 2012-08-26 23:58:07 <ne0futur> http://www.grsecurity.net/~spender/exploits/exploit2.txt
225 2012-08-26 23:58:58 <ne0futur> was a good example of a bug ( backdoor ) appearing only if you use the good compiler + glib +kernel + selinux
226 2012-08-26 23:59:28 <ne0futur> well since redhat manages gcc and glib and more . . .
227 2012-08-26 23:59:32 <gmaxwell> ne0futur: meh, it's an example of a bug which is caused by code obviously depending on undefined behavior in C.