1 2015-06-06 03:44:38 <gmaxwell> http://www.reddit.com/r/Bitcoin/comments/38l3jc/the_bitcoin_core_011_release_cycle_has_begun_with/crx2g63?context=3 :(
2 2015-06-06 06:01:22 <Diablo-D3> gmaxwell: http://imgur.com/gallery/MfFKGP4
3 2015-06-06 06:10:16 <wumpus> Diablo-D3: lol
4 2015-06-06 06:10:37 <Diablo-D3> nuke.gif
5 2015-06-06 06:53:28 <midnightmagic> Diablo-D3: hah
6 2015-06-06 06:53:57 <Diablo-D3> :D
7 2015-06-06 08:28:33 <sipa> jonasschnelli: http://bitcoin.stackexchange.com/questions/37815/how-to-know-if-bitcoind-synced -> can you please not suggest to compare against blockchain.info (they've been out of sync multiple times in the past...), and if you must do so anyway at least add the .info name :)
8 2015-06-06 08:29:56 <jonasschnelli> sipa: right. I had a bad feeling when i mention blockchain (.info). Thanks for the edit!
9 2015-06-06 08:30:45 <sipa> there is a bit of a systemic risk if everyone uses the same source for authoritative information... what if there was an actual network split caused by a bug, and the bug affects the site as well
10 2015-06-06 08:32:14 <jonasschnelli> sipa: totally agreed. Does bitcoind nowhere expose the internally calculated "% synced" over RPC?
11 2015-06-06 08:32:41 <sipa> it doesn't, because it isn't something you should rely on :)
12 2015-06-06 08:33:02 <sipa> though there are various tests in different places in the code, and exposing something may be useful
13 2015-06-06 08:33:35 <sipa> timestamp of the latest block is a very reliable way to know whether you're synced up to a few hours, assuming no fork has occurred
14 2015-06-06 08:34:44 <jonasschnelli> sipa: Right. Timestamp is probably the best way to do a rough check of the sync progress.
15 2015-06-06 09:22:55 <wumpus> jonasschnelli: yes, the GUI uses that too
16 2015-06-06 09:24:16 <jonasschnelli> wumpus: hmm... lost the context. :)
17 2015-06-06 09:27:12 <wumpus> <jonasschnelli> sipa: Right. Timestamp is probably the best way to do a rough check of the sync progress.
18 2015-06-06 09:29:20 <jonasschnelli> wumpus: ah. Right!
19 2015-06-06 09:43:48 <wumpus> https://github.com/bitcoin/bitcoin/issues/6211 so ok, time to open the floodgates to c++11 then?
20 2015-06-06 09:45:56 <Diablo-D3> wumpus: meh
21 2015-06-06 09:45:58 <wumpus> if we want it for 0.12, it's best to decide soon, now that development on 0.12 just started
22 2015-06-06 09:46:07 <Diablo-D3> do we even have c99 support in most compilers yet =/
23 2015-06-06 09:46:17 <wumpus> if not, I'm going to close all the c++11-transition-ish pulls until the next release window
24 2015-06-06 09:46:45 <wumpus> c99 support is not relevant to this
25 2015-06-06 09:46:59 <wumpus> c99 was rejected by microsoft, c++11 was adopted very quickly
26 2015-06-06 09:47:34 <wumpus> it's not even controversial, *everyone* seems to want it
27 2015-06-06 09:51:04 <mjerr> when creating a transaction (e.g. with bitcoinj) with multiple inputs, I have to add all inputs unsigned first, and only after that I can start signing (in any order), right?
28 2015-06-06 09:51:11 <sipa> mjerr: correct
29 2015-06-06 09:51:25 <sipa> mjerr: as the signatures (usually) cover the inputs
30 2015-06-06 09:52:05 <mjerr> and on signing, the sigScript of that input is replaced with the script of the output I am spending?
31 2015-06-06 09:52:27 <mjerr> and the sigScript of all other inputs are replaced with 0x00
32 2015-06-06 09:52:51 <buZz> aww c++11
33 2015-06-06 09:52:55 <buZz> :(
34 2015-06-06 09:53:08 <sipa> wumpus: assuming no blockers, i'm all for that
35 2015-06-06 09:53:12 <sipa> buZz: what's the objection?
36 2015-06-06 09:53:14 <buZz> gonna take me a while to patch that back to being able to compile it on debian stable
37 2015-06-06 09:53:26 <wumpus> sipa: no blockers imo
38 2015-06-06 09:53:31 <sipa> what g++ does debian stable have?
39 2015-06-06 09:53:40 <wumpus> stable is 5.0 now right?
40 2015-06-06 09:53:59 <buZz> wheezy has 4.7.2
41 2015-06-06 09:54:02 <wumpus> which has.. . gcc 4.9.2
42 2015-06-06 09:54:08 <buZz> jessie has 4.9.2
43 2015-06-06 09:54:32 <buZz> c++11 in 4.9.2 is not finalized, nor complete
44 2015-06-06 09:54:45 <sipa> there are no complete c++11 implementations at all, afaik
45 2015-06-06 09:54:45 <wumpus> 4.7 already has most c++11 features afaik
46 2015-06-06 09:54:51 <buZz> its just a drag, which doesnt really add much
47 2015-06-06 09:54:59 <buZz> just some codestyles
48 2015-06-06 09:55:02 <sipa> we can restrict to "whatever g++ 4.8 supports" or so
49 2015-06-06 09:55:14 <buZz> my vote is on that, yes
50 2015-06-06 09:55:14 <wumpus> well, the developers seem to disagree on that buZz and that's what matters here
51 2015-06-06 09:55:27 <sipa> buZz: rvalues, move semantics, auto types, atomics, threading
52 2015-06-06 09:55:30 <buZz> wumpus: from the MLs , seems devs disagree on most :P
53 2015-06-06 09:55:30 <sipa> i want all those
54 2015-06-06 09:55:44 <wumpus> people have been pushing for c++11'ish constructs for years, I want to put this behind us
55 2015-06-06 09:56:01 <wumpus> I'm tired of this eternal discussion
56 2015-06-06 09:56:10 <sipa> what does gitian have?
57 2015-06-06 09:56:41 <wumpus> ubuntu 12.04, with gcc 4.6. But for 0.12 we're going to give that an upgrade anyway.
58 2015-06-06 09:56:46 <sipa> ok
59 2015-06-06 09:57:20 <wumpus> even without c++11, new compilers generate better code
60 2015-06-06 09:57:29 <sipa> more importantly: what does mingw-w64 support in gitian?
61 2015-06-06 09:58:28 <wumpus> should be the same version
62 2015-06-06 09:59:01 <sipa> 4.8.2 in ubuntu 14.04, it seems
63 2015-06-06 09:59:39 <wumpus> yes, and going from 12.04 to 14.04 for gitian makes sense, it's the new(ish) ubuntu stable
64 2015-06-06 09:59:46 <sipa> seems that as of gcc 4.8.1, only garbage collection support of c++11 is missing
65 2015-06-06 10:00:17 <wumpus> yep. Good enough for us
66 2015-06-06 10:02:37 <sipa> 4.9 adds stdatomic.h and TLS support
67 2015-06-06 10:03:48 <sipa> wait, that's C11, not c++11
68 2015-06-06 10:04:09 <Diablo-D3> [05:46:45] <wumpus> c99 support is not relevant to this
69 2015-06-06 10:04:14 <Diablo-D3> [05:46:59] <wumpus> c99 was rejected by microsoft, c++11 was adopted very quickly
70 2015-06-06 10:04:25 <Diablo-D3> I thought c++11 inherited all the important c99 things?
71 2015-06-06 10:04:50 <sipa> 4.8 has c++11 atomics support
72 2015-06-06 10:04:51 <wumpus> Diablo-D3: that's possible, even likely, but that still doesn't make compilers implementing full c99 relevant
73 2015-06-06 10:05:08 <wumpus> MSVC will *never* implement full C99. They're fairly far along with c++11.
74 2015-06-06 10:05:50 <wachub> Hi anyone here!!
75 2015-06-06 10:05:53 <wachub> ??
76 2015-06-06 10:06:18 <wumpus> sipa: nice! (although we neither use atomics nor TLS at this point, well leveldb uses some atomics)
77 2015-06-06 10:07:26 <sipa> atomics are nice because we have quite a few very simple variables protected by a mutex (or not protected at all...)
78 2015-06-06 10:07:35 <wumpus> true, using them would be great
79 2015-06-06 10:07:38 <sipa> and for those use cases, atomics are much faster
80 2015-06-06 10:08:07 <wumpus> wachub: never ask that question on IRC :)
81 2015-06-06 10:08:36 <Diablo-D3> sipa: re: atomics
82 2015-06-06 10:08:41 <Diablo-D3> sipa: msvc always had them
83 2015-06-06 10:08:44 <Diablo-D3> for useful values of always
84 2015-06-06 10:08:58 <Diablo-D3> c and c++ just grew unified apis that basically cloned the existing gcc and msvc apis
85 2015-06-06 10:09:38 <wumpus> Diablo-D3: yes, all platforms have had them for a while, the point here is that there is a sensible, cross-platform way to do them now that doesn't introduce dependencies
86 2015-06-06 10:09:48 <Diablo-D3> wumpus: eh, questionable on that
87 2015-06-06 10:09:55 <Diablo-D3> remember, Ive actually used atomics in my code
88 2015-06-06 10:10:07 <wumpus> Diablo-D3: ok, whatever makes you happy, I don't feel like arguing
89 2015-06-06 10:10:17 <Diablo-D3> Ive seen the c and c++ apis, they're not as useful
90 2015-06-06 10:10:25 <wachub> wumpus: I am a beginner trying to understand the C++ version of the source code. Is this the place for me..??
91 2015-06-06 10:10:47 <Diablo-D3> they should mock up what gcc and msvc do directly, but its closer to what some embedded C compilers do
92 2015-06-06 10:10:49 <sipa> wachub: well, start by reading the code, and the developer documentation on bitcoin.org
93 2015-06-06 10:10:59 <sipa> source code is at github.com/bitcoin/bitcoin
94 2015-06-06 10:11:15 <Diablo-D3> wumpus: theres no argument.
95 2015-06-06 10:11:26 <sipa> feel free to ask specific questions here, but if you're unclear about how bitcoin itself works at a conceptual level, #bitcoin is a better place
96 2015-06-06 10:11:36 <Diablo-D3> but sometimes I feel like people don't know you can #ifdef based on compilers.
97 2015-06-06 10:27:56 <wumpus> c++11 propels c++'s standard library into the 21th century, there is only one glaring hole IMO, networking
98 2015-06-06 10:30:01 <wumpus> it's so important these days there is hardly an excuse to leave it out. Hopefully (async) networking will be addressed inthe next c++XX
99 2015-06-06 10:36:27 <Diablo-D3> wumpus: so c++ is turning into java now? =/
100 2015-06-06 13:35:41 <PRab> I think there is something weird with gitian for 0.11.0rc1. It appears to have frozen "Running build script".
101 2015-06-06 13:36:27 <PRab> I did the gitian build for linux yesterday and ended up having to establish a new SSH connection to restart the build.
102 2015-06-06 13:36:58 <PRab> Today I'm doing windows and it appears to have the same symptoms (I'm giving it a few more minutes)
103 2015-06-06 13:37:43 <PRab> Has anyone else seen this behavior?
104 2015-06-06 13:45:35 <wumpus> nope, worked fine for me. Have you looked at build.log? Any clue where in the build it gets stuck?
105 2015-06-06 13:46:17 <PRab> I'll let you know if this one gets stuck. I just double checked my CPU and it appears to still be doing real work.
106 2015-06-06 13:46:45 <PRab> The one last night wasn't using any cpu, but still wasn't responding to anything (not even CTRL + C)
107 2015-06-06 13:49:00 <PRab> wumpus: Where is build.log?
108 2015-06-06 13:50:41 <PRab> wumpus: nm, found it. (/gitian-builder/var/build.log)
109 2015-06-06 14:04:43 <PRab> windows build ended up finishing. It just took a really long time (maybe ~1 hour).
110 2015-06-06 15:46:17 <Luke-Jr> 3 years of seeds.txt pushed to https://gitlab.com/Luke-Jr/my-bitcoin-dnsseed-historical-seeds.txt with daily updates
111 2015-06-06 20:09:46 <Luke-Jr> finally, after nearly 2 months, my USB stick has finished IBD
112 2015-06-06 20:10:23 <Luke-Jr> would be nice if there was a way to save its state <.<
113 2015-06-06 20:10:48 <Diablo-D3> ibd?
114 2015-06-06 20:59:41 <sturles> Initial Block Download?
115 2015-06-06 21:54:12 <dhill> testnet3 - 105000 in mempool :)
116 2015-06-06 22:44:45 <Luke-Jr> do we have a way to turn off out-of-order block storage? this is messing up btrfs's deduplication.. :/
117 2015-06-06 22:52:52 <phantomcircuit> Luke-Jr, it's easy to disable but not easy to disable efficiently
118 2015-06-06 22:56:21 <Luke-Jr> I wonder if we should sparse-align blocks to 1 MB or something
119 2015-06-06 22:56:55 <gmaxwell> and add about 200000 MB of storage required on random file systems?
120 2015-06-06 22:57:44 <Luke-Jr> gmaxwell: what filesystems don't support sparse files? O.o could be optional
121 2015-06-06 22:58:32 <gmaxwell> (also even where supported users will freak out about the bitcoin having broken their system by using many gigabytes more space than they have. :) )
122 2015-06-06 22:58:50 <gmaxwell> See also how awfully we sufferd with that hardlinked blockfile with windows users.
123 2015-06-06 22:59:44 <Luke-Jr> disabled by default unless on btrfs? :/
124 2015-06-06 23:05:51 <Luke-Jr> hm, #btrfs seems to suggest it may be unnecessary, and that perhaps I can deduplicate arbitrary byte ranges O.o
125 2015-06-06 23:06:54 <gmaxwell> Luke-Jr: just might need to make the right ioctls.
126 2015-06-06 23:07:24 <Luke-Jr> with that knowledge, I can just forget about it and put it off until I need free space <.<
127 2015-06-06 23:08:08 <gmaxwell> Luke-Jr: the alternative thing to do would be to make utility that packs blockfiles. E.g. opens up the blockindex and rebuilds the block files to be fully linear and such.
128 2015-06-06 23:24:33 <phantomcircuit> Luke-Jr, a general solution would be to store the blocks in memory for flush to disk until you have a linear set of blocks
129 2015-06-06 23:24:41 <phantomcircuit> even then though you're going to end up with reorgs
130 2015-06-06 23:25:18 <Luke-Jr> phantomcircuit: IBD wouldn't have reorgs. I hate to go back to the orphan block stuff though :|
131 2015-06-06 23:25:20 <phantomcircuit> actually i dont see why bitcoin core couldn't rewrite the blockfiles when it restarts to remove very old forks and linearize the blocks
132 2015-06-06 23:25:34 <Luke-Jr> phantomcircuit: unnecessary I/O is ugly :P
133 2015-06-06 23:25:56 <phantomcircuit> Luke-Jr, it should basically never run though
134 2015-06-06 23:26:08 <phantomcircuit> you run it once and then you're only fixing up the later blocks with small reorgs
135 2015-06-06 23:26:50 <Luke-Jr> phantomcircuit: eh? no, one small reorg with 20 weeks continuous uptime after it = move ALL the data
136 2015-06-06 23:27:04 <Luke-Jr> gmaxwell: maybe we should be truncating the block files when we declare them full? :P
137 2015-06-06 23:27:17 <phantomcircuit> Luke-Jr, 20 weeks of data is currently what like
138 2015-06-06 23:27:19 <phantomcircuit> a few GB?
139 2015-06-06 23:27:35 <phantomcircuit> (20 * 7 * 24 * 6) MB max
140 2015-06-06 23:27:48 <phantomcircuit> 20GB max
141 2015-06-06 23:27:51 <phantomcircuit> ok maybe a bit much
142 2015-06-06 23:28:19 <phantomcircuit> well you could do this in the background with a huge amount of effort
143 2015-06-06 23:28:22 <phantomcircuit> yeah not worth it
144 2015-06-06 23:28:58 <phantomcircuit> Luke-Jr, keeping the queued blocks in memory until you can flush them in series is a good 80/10 measure though
145 2015-06-06 23:30:02 <Luke-Jr> phantomcircuit: might be able to DoS nodes by withholding the first block they ask for
146 2015-06-06 23:31:06 <phantomcircuit> Luke-Jr, there's already some shenanigans that can be pulled there though
147 2015-06-06 23:31:19 <phantomcircuit> it takes like 10 minutes for a request for a block to timeout
148 2015-06-06 23:31:30 <phantomcircuit> oh it's 20 now gmaxwell changed it
149 2015-06-06 23:31:53 <phantomcircuit> the behavior is close to optimal if you assume that none of the peers are intentionally messing with you
150 2015-06-06 23:32:14 <phantomcircuit> if you keep it all in memory your worst case is 10MB more memory
151 2015-06-06 23:32:20 <phantomcircuit> which seems like a reasonable tradeoff
152 2015-06-06 23:33:19 <Luke-Jr> phantomcircuit: except we fetch 1024 blocks at a time..
153 2015-06-06 23:33:36 <gmaxwell> phantomcircuit: that isn't a general solution. In order to get good concurrency during download with differently fast peers the reorder window must be quite large. Right now its 1024 blocks.
154 2015-06-06 23:35:12 <phantomcircuit> ah
155 2015-06-06 23:36:04 <phantomcircuit> replace in memory with "in temporary blockfile"
156 2015-06-06 23:36:11 <phantomcircuit> but that doubles the io
157 2015-06-06 23:36:40 <Luke-Jr> too bad headers don't include sizes
158 2015-06-06 23:44:49 <agorecki> satoshi where are you/?
159 2015-06-06 23:45:45 <sipa> Luke-Jr: just an option to have 1 block per file would help :)
160 2015-06-06 23:54:36 <Luke-Jr> sipa: that may ⦠eat ⦠inodes
161 2015-06-06 23:57:23 <sipa> Luke-Jr: i'm btrfs can deal with that
162 2015-06-06 23:57:26 <sipa> *sure
163 2015-06-06 23:58:12 <Luke-Jr> sipa: btrfs probably, but not ext4 :P
164 2015-06-06 23:58:15 <sipa> and if you prune anyway, it's only a few 1000 blocks
165 2015-06-06 23:58:21 <Luke-Jr> oh, option, right
166 2015-06-06 23:58:45 <Luke-Jr> that just kills my niche case where I'm backing up another system which is itself ext4 :P