1 2013-09-16 00:03:04 <Krellan> bfgminer question: some of my Erupters go into ERR state, with "unable to open /dev/ttyUSB99" or whatever their number is
2 2013-09-16 00:03:24 <Krellan> I have a script that tells the USB hub to reinitialize, it works, but requires manually running it.
3 2013-09-16 00:04:14 <Krellan> What's the best way to automate that?
4 2013-09-16 00:07:13 <Luke-Jr> Krellan: --cmd-dead maybe?
5 2013-09-16 00:07:19 <Luke-Jr> not sure if ERR triggers that
6 2013-09-16 00:09:28 <Krellan> Luke-Jr: Thanks. It's not SICK or DEAD.
7 2013-09-16 00:09:44 <Luke-Jr> --cmd-idle perhaps
8 2013-09-16 00:09:45 <Krellan> I looked for an ERR callout but didn't find. What triggers ERR? I noticed it attempts to retry several times (going by dmesg output).
9 2013-09-16 00:10:09 <Krellan> Does that command take arguments, such as the device that died?
10 2013-09-16 00:13:44 <Luke-Jr> no
11 2013-09-16 00:15:57 <Krellan> Ah.
12 2013-09-16 00:16:19 <Krellan> Maybe an idea for some future work. I don't want to disrupt all my USB hubs, just the offending Erupter.
13 2013-09-16 00:28:07 <cfields> BlueMatt: ping
14 2013-09-16 00:34:14 <Krellan> Luke-Jr: Here's my USB device reset script https://gist.github.com/Krellan/6575538
15 2013-09-16 00:34:39 <Krellan> I'm investigating ways to make bfgminer call that automatically instead of just giving up on a device that is in ERR state.
16 2013-09-16 00:40:45 <BlueMatt> cfields: pong
17 2013-09-16 00:41:45 <cfields> BlueMatt: what's up with the build failure on your recent PR? Looks like pull-tester is messing with leveldb somehow?
18 2013-09-16 00:43:10 <BlueMatt> cfields: looks like autotools fucked something up...or so
19 2013-09-16 00:43:32 <cfields> BlueMatt: so there are no local patches being used for pulltester?
20 2013-09-16 00:43:38 <BlueMatt> shouldnt be, no
21 2013-09-16 00:45:50 <cfields> any way to verify? I'm really not sure how/why that file would be deleted halfway through a build
22 2013-09-16 00:46:51 <BlueMatt> no, it definitely just runs the scripts in qa/pull-tester
23 2013-09-16 00:48:11 <cfields> ok, thanks. looking into it
24 2013-09-16 00:48:32 <BlueMatt> just reset that pull to see if it was arbitrary, it'l test again
25 2013-09-16 01:05:12 <warren> gmaxwell: cfields said he is not aware of build slowdowns from autotools, has anyone quantified the slowdown?
26 2013-09-16 01:16:31 <cfields> BlueMatt: i think i see it. If so, it's a pretty freakish race. It'd be nice to be able to reproduce. Have you seen that problem with any other builds?
27 2013-09-16 01:17:07 <BlueMatt> warren: I noticed autotools builds seem to be slower, but that may be unrelated here
28 2013-09-16 01:17:30 <warren> BlueMatt: it's WAYYYY slower here, and gmaxwell commented as such too
29 2013-09-16 01:17:37 <BlueMatt> cfields: I havent been notified of any such failures, but then again most people ignore the "Please ping BlueMatt" message
30 2013-09-16 01:17:50 <cfields> heh, ok
31 2013-09-16 01:17:51 <BlueMatt> warren: well, its significantly slower here, but not nearly waayyyy, so probably unrelated
32 2013-09-16 01:18:05 <cfields> warren: please explain which part is significantly slower?
33 2013-09-16 01:18:08 <cfields> it really shouldn't be
34 2013-09-16 01:18:24 <warren> cfields: I'm not working much on 0.9 yet, ask the devs who are.
35 2013-09-16 01:19:19 <cfields> warren: you said above that it's way slower for you. was that second-hand info?
36 2013-09-16 01:19:31 <warren> cfields: no, I witnessed it
37 2013-09-16 01:19:50 <warren> cfields: I participated in 0.9 only to the extent that it builds on fedora again
38 2013-09-16 01:20:31 <cfields> warren: are you saying that the process start to finish is slower? or the make itself?
39 2013-09-16 01:20:57 <warren> I'll measure pre-autotools and post-autotools now.
40 2013-09-16 01:20:59 <cfields> if it's the former, sure, that's the price to be paid. if it's the latter, that's a bug
41 2013-09-16 01:21:14 <warren> cfields: make itself
42 2013-09-16 01:47:10 <jgarzik> "Intel shows 14nm Broadwell Consuming 30% Less Power Than 22nm Haswell"
43 2013-09-16 01:47:25 <jgarzik> time for another leap in ASIC mining efficiency...
44 2013-09-16 01:47:47 <jgarzik> (referencing the companies that are terribly proud of their 28nm stuff)
45 2013-09-16 01:50:15 <jgarzik> Police catch cyber-thieves. "Cyber" twist: evil KVM switch involved, providing remote access. http://www.techweekeurope.co.uk/news/police-arrest-12-santander-cyber-scam-127038
46 2013-09-16 01:56:51 <BlueMatt> why do pcie fpga boards cost so damn much :(
47 2013-09-16 02:00:27 <Luke-Jr> BlueMatt: just get a bitcoin USB one and retrofit it :P
48 2013-09-16 02:01:05 <BlueMatt> Luke-Jr: the point was to program a pcie device...
49 2013-09-16 02:07:22 <petertodd> BlueMatt
50 2013-09-16 02:07:30 <petertodd> BlueMatt: Niche market is part of it.
51 2013-09-16 02:07:32 <BlueMatt> petertodd:
52 2013-09-16 02:07:49 <BlueMatt> well, yes...
53 2013-09-16 02:08:23 <petertodd> BlueMatt: The other half is that the PCIE standard is genuinely pretty complex to implement at the PCB level due to impedance-controlled traces and all sorts of other stuff, which makes the fixed cost of the design + production run high.
54 2013-09-16 02:09:31 <BlueMatt> petertodd: ahh, well yea thats kinda what I was thinking...now to see if I can find some lying around cs labs here...
55 2013-09-16 02:10:13 <petertodd> BlueMatt: Oh right, you're in uni, surely you can find something. :P
56 2013-09-16 02:10:24 <petertodd> BlueMatt: What's the project anyway?
57 2013-09-16 02:10:57 <BlueMatt> looking at some of the dma stuff wrt vt-d's protection and actually implementing some theoretical attacks that afaik havent been implemented
58 2013-09-16 02:11:07 <BlueMatt> dma attack stuff, that is
59 2013-09-16 02:12:47 <petertodd> BlueMatt: Oh, attacking TPM then? Like Intels north-bridge integrated stuff?
60 2013-09-16 02:13:22 <BlueMatt> wasnt looking at attacking tpm stuff (yet), but, yea the vt-d stuff is northbridge-integrated IOMMU, essentially
61 2013-09-16 02:14:20 <BlueMatt> ACTION ponders pcie-over-thunderbolt as being an easier target
62 2013-09-16 02:14:32 <gmaxwell> 10:10 < sipa> jgarzik: i'd really like some explanation about that
63 2013-09-16 02:14:40 <gmaxwell> are there really any open parameters?
64 2013-09-16 02:15:15 <gmaxwell> The field size is a prime with the word divisible structure, it's probably the largest 256 bit such number.
65 2013-09-16 02:15:47 <gmaxwell> A is the lowest a that gets you a prime order group, and I think b ends up fixed.
66 2013-09-16 02:15:47 <petertodd> BlueMatt: Ah, well if you have a need for TPM stuff send me an email; I've done a fair bit of research there.
67 2013-09-16 02:16:34 <BlueMatt> petertodd: pm, this is far ot
68 2013-09-16 02:16:47 <gmaxwell> BlueMatt: #bitcoin-wizards for that stuff probably.
69 2013-09-16 02:41:15 <cfields> warren: any result data?
70 2013-09-16 02:41:48 <warren> cfields: sorry, busy with RL thing
71 2013-09-16 02:41:55 <cfields> ok
72 2013-09-16 02:50:15 <hydromet> fanquake: how's your system coming along?
73 2013-09-16 02:52:35 <fanquake> hydromet, haven't had a chance to do much since last night. However, work got rained off today, so I'll have a chance to look at it. Probably finish getting gitian working though, rather than play with Qt5.
74 2013-09-16 03:08:24 <gmaxwell> ...
75 2013-09-16 03:08:24 <gmaxwell> so
76 2013-09-16 03:08:26 <gmaxwell> OSX
77 2013-09-16 03:08:36 <jgarzik> ?
78 2013-09-16 03:08:40 <gmaxwell> util.cpp:1160
79 2013-09-16 03:08:41 <SomeoneWeird> is crap
80 2013-09-16 03:08:41 <TheLordOfTime> ... so, OS X closed source wonderland of non-community supportable software.
81 2013-09-16 03:08:48 <gmaxwell> We're still using fsync() for the block files.
82 2013-09-16 03:09:06 <gmaxwell> Note that all the reports of osx data corruption are now one that look like the blockfiles missed writes.
83 2013-09-16 03:09:20 <gmaxwell> (all the reports since 0.8.4)
84 2013-09-16 03:10:23 <gmaxwell> Do we need to be making that use fdatasync (which on OSX should be fcntl(fd, F_FULLFSYNC, 0)) too?
85 2013-09-16 03:13:27 <jgarzik> gmaxwell, yes
86 2013-09-16 03:13:35 <gmaxwell> k. will make pull. doh.
87 2013-09-16 03:14:45 <cfields> gmaxwell: would you like me to abstract that with proper autoconf checks?
88 2013-09-16 03:14:50 <cfields> for .9 ofc
89 2013-09-16 03:15:37 <gmaxwell> cfields: How can you? Whats the "test" for "FSYNC claims to work but really does nothing"?
90 2013-09-16 03:17:33 <jgarzik> gmaxwell, if (os==osx), of course, when all else fails
91 2013-09-16 03:18:08 <gmaxwell> sure, thats what I did: https://github.com/bitcoin/bitcoin/pull/3000
92 2013-09-16 03:18:43 <jgarzik> gmaxwell, are you sure OS_MACOSX is not a leveldb-specific symbol?
93 2013-09-16 03:19:01 <gmaxwell> indeed I was just thinking that.
94 2013-09-16 03:19:08 <cfields> hmm
95 2013-09-16 03:19:31 <gmaxwell> jgarzik: and now I have no idea what you're talking about. ;)
96 2013-09-16 03:19:54 <cfields> F_FULLFSYNC is darwin-only, no?
97 2013-09-16 03:20:06 <jgarzik> gmaxwell, and in general, the autotools-ish philosophy would be to detect OSX in configure, and then do similar to what leveldb does: define fdatasync() in OS-specific manner, then end-line code is very simple: no ifdefs, just fdatasync(fd) for all cases.
98 2013-09-16 03:20:51 <jgarzik> autoconf and the obvious header file hackery creates illusion of fdatasync()
99 2013-09-16 03:21:03 <gmaxwell> jgarzik: I'm not superduper keen on the macro being called fdatasync()... since if you manage to not have it defined it'll still work on places that just have one.
100 2013-09-16 03:21:38 <cfields> jgarzik: i nearly agree. imo checking "if host == osx" is asking for trouble. I'd rather do something like: if F_FULLFSYNC is available, use it
101 2013-09-16 03:22:04 <cfields> (with the assumption that the above would always be the preference)
102 2013-09-16 03:22:10 <gmaxwell> cfields: and what happens when something else with working fdatasync adds that flag?
103 2013-09-16 03:22:24 <gmaxwell> It's not the preference its a crappy replacement for fdatasync. :(
104 2013-09-16 03:22:38 <jgarzik> gmaxwell, then you are not superduper keen on the autoconf philosophy in general ;p There are many cases where the pattern is applied of "provide foo(), where platform does not, such that end line code may simply assume foo() exists and works as expected"
105 2013-09-16 03:22:45 <cfields> gmaxwell: i mean if the preference is to use it on c platform where it's found
106 2013-09-16 03:22:46 <gmaxwell> it's basically fsync() by not sabotaged by an evil OS vendor who wants to cheat at benchmarks. :P
107 2013-09-16 03:23:01 <cfields> *on a
108 2013-09-16 03:23:03 <jgarzik> gmaxwell, for that and many other reasons, PROJECT-config.h should be included in any file where portability matters
109 2013-09-16 03:23:16 <jgarzik> that's just how autotools is designed to provide portability, in part
110 2013-09-16 03:23:20 <gmaxwell> cfields: but it is not our preference, our preference is fdatasync().
111 2013-09-16 03:24:09 <cfields> gmaxwell: i don't understand. on osx, it's required to use F_FULLFSYNC in order to get a proper sync, no?
112 2013-09-16 03:24:41 <gmaxwell> jgarzik: I think for most things the problem isn't someone provided a non-working version of a thing, it's just not providing one at all, enh?
113 2013-09-16 03:25:38 <gmaxwell> cfields: yes, my understanding is that to get fdatasync() or stronger on OSX you must use fcntl(F_FULLFSYNC) which does what fsync is specified to do.
114 2013-09-16 03:26:06 <jgarzik> gmaxwell, nod. but that is not really relevant to understanding the model. ("gnulib" was created to supply such missing bits, though, FWIW)
115 2013-09-16 03:26:33 <jgarzik> to be clear: on OSX, F_FULLFSYNC == fsync() + flush to disk
116 2013-09-16 03:26:39 <cfields> gmaxwell: right. so in terms of autotools, your preference becomes: use F_FULLFSYNC when possible. when not possible, fall back to fdatasync
117 2013-09-16 03:26:43 <gmaxwell> cfields: but my point there is that on platforms that have a working fdatasync() we want to use that, even if they have fcntl(F_FULLFSYNC), since a fsync() is performance intrusive.
118 2013-09-16 03:26:57 <cfields> as that variable will only be found on darwin
119 2013-09-16 03:27:12 <gmaxwell> cfields: today, but then you're just using it as a poor proxy to detect darwin.
120 2013-09-16 03:27:27 <gmaxwell> perhaps in the future freebsd adds it for better portability for mac software.
121 2013-09-16 03:28:03 <gmaxwell> it's kind of an odd case where a call is undetectably bad. :)
122 2013-09-16 03:28:22 <cfields> gmaxwell: imo it follows that it'd only be defined where it's needed as an alternative to fsync.
123 2013-09-16 03:29:18 <cfields> yea, i'm surprised to see that gnulib doesn't address it as well
124 2013-09-16 03:29:43 <jgarzik> cfields, just check for darwin in configure.ac and be done with it
125 2013-09-16 03:30:15 <cfields> ok
126 2013-09-16 03:30:24 <jgarzik> cfields, there is no point and only danger in trying a very OSX-specific F_FULLFSYNC test that might magically and unexpectedly start working on another OS in the future
127 2013-09-16 03:30:48 <gmaxwell> there is a netbsd mailing list thread where they debated adding it, though it looks like people were very opposed to it.
128 2013-09-16 03:31:37 <gmaxwell> libuv does
129 2013-09-16 03:31:38 <gmaxwell> #elif defined(__APPLE__) && defined(F_FULLFSYNC)
130 2013-09-16 03:33:37 <jgarzik> gmaxwell, I think that's fine for the purposes of your PR
131 2013-09-16 03:33:52 <jgarzik> gmaxwell, Doing It Properly in configure.ac can happen in parallel
132 2013-09-16 03:33:59 <jgarzik> gmaxwell, your patch will also backport nicely
133 2013-09-16 03:34:57 <jgarzik> gmaxwell, defined(F_FULLFSYNC) test success is predicted on fcntl.h header inclusion, note.
134 2013-09-16 03:35:05 <jgarzik> predicated
135 2013-09-16 03:36:15 <gmaxwell> Yep, so is fnctl() itself. And indeed, backporting is a point... this is certantly something we'll want in an update for OSX.
136 2013-09-16 03:38:52 <Diablo-D3> gmaxwell: I swear we already had this conversation 2-3 weeks ago
137 2013-09-16 03:39:27 <gmaxwell> We did, for leveldb.
138 2013-09-16 03:39:39 <gmaxwell> And it was fixed in leveldb, and not elsewhere. Because good is dumb.
139 2013-09-16 03:40:17 <gmaxwell> (And, surprise surprise, after 0.8.4 which included the fix some OSX users started reporting that they no longer had leveldb errors but block-reading errors)
140 2013-09-16 03:41:25 <jgarzik> rofl
141 2013-09-16 03:41:42 <jgarzik> (because good is dumb)
142 2013-09-16 03:50:32 <Diablo-D3> gmaxwell: lol
143 2013-09-16 05:54:11 <jgarzik> ASICminer sure has "fallen" far + fast
144 2013-09-16 05:55:02 <warren> cfields: ok, I actually measured it, building both bitcoin-qt and bitcoind is faster with autotools than pre-autotools by about 10%. No ccache and nothing prebuilt.
145 2013-09-16 05:55:34 <jgarzik> http://www.marketwatch.com/story/most-secure-bitcoin-wallet-armory-raises-600k-led-by-trace-mayer-2013-09-16
146 2013-09-16 05:55:42 <jgarzik> congrats to etothepi are in order