1 2015-12-04 01:43:55 <Lightsword> would the cs_main lock https://github.com/bitcoin/bitcoin/blob/master/src/rpcmining.cpp#L385 for getblocktemplate automatically get unlocked when getblocktemplate returns?
  2 2015-12-04 01:44:06 <phantomcircuit> Lightsword, yes
  3 2015-12-04 01:44:23 <phantomcircuit> Lightsword, if you want to run TBV in a background thread you need to acquire the lock there also
  4 2015-12-04 01:45:44 <Lightsword> can two threads hold the same lock a the same time?
  5 2015-12-04 01:46:54 <phantomcircuit> no they're exclusive locks
  6 2015-12-04 01:47:16 <phantomcircuit> (we should probably have read locks also but that's a significant complexity increase and we would definitely fuck it up)
  7 2015-12-04 01:51:16 <Lightsword> is there a way to handover a lock between threads?
  8 2015-12-04 01:53:52 <Lightsword> hmm, does GBT need the lock after CNB?
  9 2015-12-04 01:53:53 <Lightsword> hmm, does GBT need the lock after CNB?
 10 2015-12-04 01:54:46 <gmaxwell> Lightsword: Do you have a rig setup to test latency of responses?
 11 2015-12-04 01:54:47 <gmaxwell> Lightsword: Do you have a rig setup to test latency of responses?
 12 2015-12-04 01:54:58 <Lightsword> gmaxwell yes
 13 2015-12-04 01:55:15 <gmaxwell> Good. (I was going to remind you of the first principle of optimization: don't optimize what you can't test.)
 14 2015-12-04 01:55:18 <Lightsword> I was testing locally on my mac though using some debug statemens
 15 2015-12-04 01:55:59 <Lightsword> seems if I can get this to work it’s an easy ~100ms
 16 2015-12-04 01:56:05 <Lightsword> maybe more
 17 2015-12-04 01:56:48 <Lightsword> main test server I have on poolbench as well
 18 2015-12-04 01:57:43 <gmaxwell> oh right backgrounding the validation.
 19 2015-12-04 01:57:49 <gmaxwell> Indeed. I'm glad you're trying.
 20 2015-12-04 01:58:59 <Lightsword> gmaxwell, it looks like getblocktemplate is losing the cs_main lock before the TBV thread finishes
 21 2015-12-04 01:59:53 <gmaxwell> Lightsword: the locks are scoped, meaning that when the go out of scope (like a stack variable) they are freed.
 22 2015-12-04 02:02:02 <Lightsword> gmaxwell, I’m thinking I need to hand off the lock held by GBT to CNB and then to the TBV thread, does that sound right?
 23 2015-12-04 02:05:27 <phantomcircuit> Lightsword, i think you'll find that the majority of the latency now is hex encode and json encode/decode
 24 2015-12-04 02:06:43 <Lightsword> phantomcircuit, I did notice that was also a big source of latency, I think it may have been a little more than TBV but both were significant
 25 2015-12-04 02:06:56 <morcos> Lightsword: I think you should just be able to acquire it separately in the new thread and it'll block until it's freed up from GBT (would be nice to make sure that happens before the json crap though)
 26 2015-12-04 02:07:41 <dgenr8> the p2p request to list the whole mempool is probably unnecessary, UNLESS a bloom filter is in force.  what's the problem in that case?
 27 2015-12-04 02:08:16 <Lightsword> morcos, can I have GBT unlock before CNB and then have CNB grab the lock so it can be handed off to TBV?
 28 2015-12-04 02:09:25 <morcos> Lightsword: are you worried about something else grabbing the lock in between
 29 2015-12-04 02:09:44 <Lightsword> morcos, that or GBT needing the lock after CNB returns
 30 2015-12-04 02:09:45 <Lightsword> morcos, that or GBT needing the lock after CNB returns
 31 2015-12-04 02:13:56 <morcos> Lightsword: it looks to me from a quick check that you could rescope the lock or manually change to release after UpdateTime in GBT
 32 2015-12-04 02:14:21 <morcos> i don't think it should be a problem if some other code grabs the lock before TBV as you're not really waiting on TBV at that point anyway
 33 2015-12-04 02:14:38 <morcos> my bigger concern is what happens when TBV fails (commented on the PR)
 34 2015-12-04 02:15:48 <gmaxwell> One thought is that you stop mining transactions and set stop setting the verify flag ( BIP I plan on posting to the list as soon as people working more on version bits tell me they won't strangle me for stealing a bit out from under them: https://people.xiph.org/~greg/bip.draft-maxwell-flagverify.mediawiki )
 35 2015-12-04 02:17:52 <Lightsword> morcos, I thought the throw would shut down the node entierely in the event of a TBV failure
 36 2015-12-04 02:20:01 <phantomcircuit> Lightsword, oh and fyi easiest way to get a rough idea of what's causing latency is to call gbt in an infinte loop with perf record -ag running
 37 2015-12-04 02:20:52 <Lightsword> I was adding logging to the code and just looking at the timestamps
 38 2015-12-04 02:20:53 <Lightsword> I was adding logging to the code and just looking at the timestamps
 39 2015-12-04 02:21:24 <phantomcircuit> Lightsword, the logging adds latency :P
 40 2015-12-04 02:21:33 <morcos> Lightsword: the existing code does not shut down the node
 41 2015-12-04 02:22:11 <Lightsword> what should I call to do that?
 42 2015-12-04 02:22:12 <Lightsword> what should I call to do that?
 43 2015-12-04 02:22:52 <phantomcircuit> Lightsword, assert(TestBlockValidity())
 44 2015-12-04 02:23:47 <morcos> Lightsword: I meant the code before your change doesn't shut down the node.  I'm not sure what your change causes
 45 2015-12-04 02:23:49 <Lightsword> should I just remove the throw?
 46 2015-12-04 02:24:02 <morcos> gmaxwell: interesting, but seems risky...
 47 2015-12-04 02:24:03 <Lightsword> and use assert instead?
 48 2015-12-04 02:24:04 <Lightsword> and use assert instead?
 49 2015-12-04 02:24:30 <gmaxwell> I thought we did assert on failure there already. Did that get removed recently?
 50 2015-12-04 02:24:48 <phantomcircuit> gmaxwell, it simply returns Null
 51 2015-12-04 02:24:54 <gmaxwell> does the caller assert on null?
 52 2015-12-04 02:24:55 <gmaxwell> does the caller assert on null?
 53 2015-12-04 02:25:18 <morcos> 0.11 and 0.12/master both throw a runtime_error, but it is caught by the rpc handling code and returns a JSON error (hopefully an informative one)
 54 2015-12-04 02:25:19 <morcos> 0.11 and 0.12/master both throw a runtime_error, but it is caught by the rpc handling code and returns a JSON error (hopefully an informative one)
 55 2015-12-04 02:25:22 <gmaxwell> morcos: consider: say we don't do this. Many miners probably will. They'll do it less safely, and they'll have an advantage from doing so.
 56 2015-12-04 02:25:41 <phantomcircuit> gmaxwell, it just returns an error
 57 2015-12-04 02:25:53 <phantomcircuit> which is pretty opaque throw JSONRPCError(RPC_OUT_OF_MEMORY, "Out of memory");
 58 2015-12-04 02:26:13 <morcos> gmaxwell: yeah i see the motivation.  but i worry about things which don't have any enforceable meaning, but people might come to rely on unwisely.  i don't have a specific objection, i'm just wary
 59 2015-12-04 02:26:31 <gmaxwell> I don't know why I remember otherwise. (not that it's weird for me to be wrong; but I thought I remembered fixing a CNB bug that was causing it to assert)
 60 2015-12-04 02:26:39 <morcos> phantomcircuit: thats what it does on returning NULL, it doesn't return NULL unless you've got a SERIOUS problem
 61 2015-12-04 02:26:45 <morcos> a failed block doesn't cause that
 62 2015-12-04 02:27:31 <gmaxwell> morcos: well we know right now that a significant fraction of hashpower extends chains without verifying them. (maybe even a majority). We don't hav any clue how fast they switch back to verifying if that even works, etc.
 63 2015-12-04 02:28:38 <phantomcircuit> gmaxwell, i dont think they even can switch back because of "protections" in the mining clients
 64 2015-12-04 02:28:39 <Lightsword> gmaxwell, I think they have a ~75 second validation timeout
 65 2015-12-04 02:28:53 <gmaxwell> Lightsword: if it works.
 66 2015-12-04 02:28:56 <phantomcircuit> they can only switch to another longer chain
 67 2015-12-04 02:29:27 <gmaxwell> phantomcircuit: bfgminer iirc will reorg one block, due to deepbit frequently loadbalancing across daemons which were racing themselves
 68 2015-12-04 02:29:30 <Lightsword> yeah, I think they shut off the stratum servers….I should check on that
 69 2015-12-04 02:29:31 <morcos> gmaxwell: yes i suppose i only quickly skimmed it, so if you are a full node or a miner doing things right, you're just going to ignore that bit entirely anyway right
 70 2015-12-04 02:29:48 <Luke-Jr> gmaxwell: no
 71 2015-12-04 02:31:05 <gmaxwell> morcos: right; it's a hint to lite wallets and other SPV clients that some blocks shouldn't count in their risk assesment.
 72 2015-12-04 02:31:06 <gmaxwell> morcos: right; it's a hint to lite wallets and other SPV clients that some blocks shouldn't count in their risk assesment.
 73 2015-12-04 02:32:48 <Luke-Jr> gmaxwell: makes so much more sense to get the SPV miners to not publish invalid blocks, even if they search for them
 74 2015-12-04 02:33:53 <gmaxwell> of course, miners can be outright malicious; ... but thats the risk SPV clients take. The issue is that the SPV assumption assumes that validation is economically rational for miners; but thats not really true in the first second after a new block.
 75 2015-12-04 02:33:54 <gmaxwell> of course, miners can be outright malicious; ... but thats the risk SPV clients take. The issue is that the SPV assumption assumes that validation is economically rational for miners; but thats not really true in the first second after a new block.
 76 2015-12-04 02:35:01 <gmaxwell> Luke-Jr: thats not operationally pratical: getting your block out fast means sending it lots of ways; you'll leak and you won't even know it (until someone gets screwed over), and the more reasonable ways to implement it end up adding a block-validation worth of latency.
 77 2015-12-04 02:35:02 <gmaxwell> Luke-Jr: thats not operationally pratical: getting your block out fast means sending it lots of ways; you'll leak and you won't even know it (until someone gets screwed over), and the more reasonable ways to implement it end up adding a block-validation worth of latency.
 78 2015-12-04 02:36:15 <gmaxwell> and every piece of latency is progress that makes larger miners are more profitable than smaller ones; or if you define not validating as unscruplous its a reason that cheaters are more profitable: not exactly the ecosystem we want the system to encourage. :)
 79 2015-12-04 02:36:16 <gmaxwell> and every piece of latency is progress that makes larger miners are more profitable than smaller ones; or if you define not validating as unscruplous its a reason that cheaters are more profitable: not exactly the ecosystem we want the system to encourage. :)
 80 2015-12-04 02:36:52 <morcos> gmaxwell: how do they publish blocks without validating them?
 81 2015-12-04 02:37:23 <gmaxwell> morcos: send directly from their mining pool servers to everything they can connect them to. (including things like the relay node client)
 82 2015-12-04 02:38:10 <morcos> so the mining pool servers talk p2p directly to all these other things?  yeah i guess that makes sense
 83 2015-12-04 02:38:19 <gmaxwell> now, hopefully none of these blocks are making it to an SPV client without being filtered by something, but thats hard to be sure of.
 84 2015-12-04 02:42:12 <Lightsword> how do I manually unlock a lock set by LOCK(cs_main)?
 85 2015-12-04 02:52:17 <Lightsword> LEAVE_CRITICAL_SECTION?
 86 2015-12-04 02:57:34 <gmaxwell> without the code in front of me: adjust the scoping so the lock goes out of scope.
 87 2015-12-04 02:57:44 <gmaxwell> Explicitly leaving should be avoided if at all possible.
 88 2015-12-04 03:24:57 <morcos> is it safe to assume BIP 68 and whichever one MedianTimePast is will go in together, or in particular we will never have 68 without MTP
 89 2015-12-04 04:15:25 <Prattler> Hello, I believe I have discovered a critical socket bug
 90 2015-12-04 04:15:26 <Prattler> Hello, I believe I have discovered a critical socket bug
 91 2015-12-04 04:15:27 <Prattler> https://github.com/bitcoin/bitcoin/pull/7168
 92 2015-12-04 04:15:30 <Prattler> please comment
 93 2015-12-04 04:16:14 <Prattler> gmaxwell
 94 2015-12-04 04:16:37 <Prattler> ATM, receiving is done in 8 byte chunks
 95 2015-12-04 04:16:41 <Prattler> that's very inefficient
 96 2015-12-04 04:16:42 <Prattler> that's very inefficient
 97 2015-12-04 04:17:14 <Lightsword> Prattler, is that an issue only with 0.12?
 98 2015-12-04 04:17:15 <Lightsword> Prattler, is that an issue only with 0.12?
 99 2015-12-04 04:17:27 <Prattler> affects all code from year 2011
100 2015-12-04 04:17:31 <Prattler> BOOM!
101 2015-12-04 04:20:17 <Lightsword> Prattler, is that used for all p2p connections?
102 2015-12-04 04:20:36 <Prattler> yes, block receive, tx receive, everything
103 2015-12-04 04:20:38 <tulip> the line is modified in 9cbae55a6ed6fcc46e636b4ae670816aab3746ec but it wasn't added then.
104 2015-12-04 04:20:39 <tulip> the line is modified in 9cbae55a6ed6fcc46e636b4ae670816aab3746ec but it wasn't added then.
105 2015-12-04 04:22:01 <Prattler> noticed this as my LAN node setup was not going over 3 MB/s. Now it's max available speed as expected.
106 2015-12-04 04:22:05 <tulip> it's present in wxBitcoin 0.1 actually, along with the comment above it.
107 2015-12-04 04:22:14 <Prattler> haha!
108 2015-12-04 04:22:15 <Prattler> haha!
109 2015-12-04 04:22:35 <Prattler> sizeof(ptrToArray) = 8, not the size of the array!
110 2015-12-04 04:22:39 <Prattler> == 8
111 2015-12-04 04:24:40 <Prattler> on BIP 101 testnet 8-9 MB blocks I could actually see my CPU go up as a new block was being downloaded in these 8 byte chunks
112 2015-12-04 04:25:10 <Prattler> and speed only reached ~3 MB/s
113 2015-12-04 04:26:46 <Prattler> by first pull request, so please comment if anything can be done better
114 2015-12-04 04:26:50 <Prattler> my first pull request
115 2015-12-04 04:28:18 <Luke-Jr> Prattler: … you're wrong
116 2015-12-04 04:28:25 <Prattler> ah?..
117 2015-12-04 04:28:26 <Prattler> ah?..
118 2015-12-04 04:28:27 <Luke-Jr> pchBuf is not a pointer at all
119 2015-12-04 04:28:29 <Luke-Jr> it's an array
120 2015-12-04 04:29:14 <Prattler> Firstly, the sizeof() operator does not give you the number of elements in an array, it gives you the number of bytes a thing occupies in memory. Hence:
121 2015-12-04 04:29:59 <Luke-Jr> which in this case is 0x10000
122 2015-12-04 04:30:21 <Luke-Jr> 0x10000 * sizeof(char) = 0x10000
123 2015-12-04 04:30:46 <Prattler> well I did run the changes with debug info... size went up from 8 to 1440 or smth
124 2015-12-04 04:32:04 <Prattler> someone will soon confirm
125 2015-12-04 04:33:53 <Prattler> oh damn! You're probably right! It's an XT-only bug
126 2015-12-04 04:40:03 <Prattler> so it sneaked into XT, it's a bit different there:
127 2015-12-04 04:40:04 <Prattler> char *pchBuf = new char[amt];
128 2015-12-04 04:40:04 <Prattler> int nBytes = recv(pnode->hSocket, pchBuf, sizeof(pchBuf), MSG_DONTWAIT);
129 2015-12-04 04:41:32 <Lightsword> Prattler, yeah I checked that code and those do come out the same
130 2015-12-04 04:41:44 <Prattler> Thanks!
131 2015-12-04 04:42:49 <Prattler> it's some stupid bandwidth throttling patch
132 2015-12-04 04:43:31 <Prattler> probably a bad idea anyway, since it would degrade block propagation
133 2015-12-04 04:57:03 <Prattler> damn, wanted to contribute so much!
134 2015-12-04 05:01:41 <gmaxwell> Prattler: thats for the effort though!
135 2015-12-04 05:01:52 <gmaxwell> er thanks!
136 2015-12-04 05:02:35 <Prattler> :)
137 2015-12-04 05:14:01 <Luke-Jr> Prattler: hey, at least you weren't imagining it ☺ next time you'll find a real one..
138 2015-12-04 05:14:27 <Prattler> yeah, it's real.. XT is seriously messed up ATM
139 2015-12-04 05:14:46 <Prattler> and it's a decent amount of the total network full nodes!
140 2015-12-04 05:14:55 <gmaxwell> s'not the only way; if you want to be a hero you can go review, you'd be like the only person doing it there. :)
141 2015-12-04 05:14:56 <gmaxwell> s'not the only way; if you want to be a hero you can go review, you'd be like the only person doing it there. :)
142 2015-12-04 07:01:20 <Lightsword> do boost::threads created by a function that has a LOCK(cs_main); retain the lock?
143 2015-12-04 07:52:26 <BW^-> by connecting to an IP, is there any byte sequence or other trick, that can help you distinguish between a dummy port, and a Bitcoin node?
144 2015-12-04 07:52:40 <BW^-> just to verify that the connection works
145 2015-12-04 07:53:27 <tulip> you can handshake with it, but that doesn't prove it's a node, it just proves it knows how to handshake.
146 2015-12-04 07:54:14 <BW^-> tulip: the handshake is a binary sequence, so i'd need to have it in a binary file and pipe it to telnet or ncat right?
147 2015-12-04 07:55:18 <BW^-> aha there's a checker at https://bitnodes.21.co/ also
148 2015-12-04 07:55:19 <BW^-> aha there's a checker at https://bitnodes.21.co/ also
149 2015-12-04 07:55:32 <tulip> not attempting to talk to a Bitcoin node using Bash would probably be better.
150 2015-12-04 07:59:12 <BW^-> tulip: do you have any favourite testing tool, or only bitcoind -connect ?
151 2015-12-04 08:07:54 <BW^-> tulip,*: http://pastebin.com/kXUnQtqP , my instance is not listening to port 8333 at all
152 2015-12-04 08:08:30 <tulip> telnet is enough for diagnosing that.
153 2015-12-04 08:09:58 <BW^-> tulip: righ t- maybe i must add "-bind=*:8333" because i use "-whitebind" ??
154 2015-12-04 08:12:15 <tulip> you want 0.0.0.0:8333, not an asterisk. that's the default so you don't need to explicitly configure it.
155 2015-12-04 08:12:50 <BW^-> aha
156 2015-12-04 08:12:58 <BW^-> tulip: but mine is not listening to 8333??
157 2015-12-04 08:13:03 <BW^-> 0.11.2
158 2015-12-04 08:13:16 <BW^-> tulip: clearly "telnet localhost 8333" and "telnet myexternalip 8333" do not work,
159 2015-12-04 08:13:33 <BW^-> and netstat -a | grep LISTEN | grep 8333 shows nothing
160 2015-12-04 08:13:43 <tulip> check your debug.log to find out why, binding the listening port happens early on.
161 2015-12-04 08:14:17 <BW^-> brb
162 2015-12-04 08:14:17 <BW^-> mhm
163 2015-12-04 08:16:46 <BW^-> tulip: when I specified -whitebind=... -listen=1 but no -bind, there was only a binding formy whitebind port listed
164 2015-12-04 08:16:50 <BW^-> and indeed it did not listen on 8333
165 2015-12-04 08:16:54 <BW^-> so I must add -bind=0.0.0.0:8333 .
166 2015-12-04 08:16:55 <BW^-> so I must add -bind=0.0.0.0:8333 .
167 2015-12-04 08:22:51 <BW^-> here's my tmux.conf: https://pastebin.mozilla.org/8853816
168 2015-12-04 08:22:56 <BW^-> gribble: = informal bug report!
169 2015-12-04 08:23:19 <BW^-> also, switching terminals tends to stop working and it's passed through to the guest console, but, the D option to detach always works!
170 2015-12-04 08:24:37 <BW^-> top says: 15521 MYUSER       64    0  218M  217M onproc    -       904:17 99.02% tmux: server (/tmp/tmux-1000/default)
171 2015-12-04 08:24:38 <BW^-> top says: 15521 MYUSER       64    0  218M  217M onproc    -       904:17 99.02% tmux: server (/tmp/tmux-1000/default)
172 2015-12-04 08:25:37 <BW^-> right, and now it dissolved magically and allw orks again
173 2015-12-04 08:27:39 <BW^-> sigh, wrong chan, sorry
174 2015-12-04 09:33:46 <CubicEarth> Multiple Choice Question:  Bitcoin is not GPU accelerated because:  A) It is  B) It can't be C) It can and should be, but there are more important things to do D) Eh, more complexity, minimal gain, so probably will not happen
175 2015-12-04 09:33:59 <CubicEarth> (I don't know the answer)
176 2015-12-04 09:34:00 <CubicEarth> (I don't know the answer)
177 2015-12-04 09:44:29 <tulip> b basically.
178 2015-12-04 09:45:34 <tulip> you don't want to do script verification on a GPU because it would be a complete reimplementation, and not necessarily consensus compatible.
179 2015-12-04 09:46:55 <tulip> well c I guess.
180 2015-12-04 09:51:09 <CubicEarth> I was forgetting about the consensus-critical part for a moment, so yeah, that makes sense.
181 2015-12-04 09:51:10 <CubicEarth> I was forgetting about the consensus-critical part for a moment, so yeah, that makes sense.
182 2015-12-04 09:57:20 <CubicEarth> But otherwise, validating txs and blocks is very parallelizable, right?
183 2015-12-04 09:58:54 <CubicEarth> But a large tx with many inputs, does that need to be handled in a serial way?
184 2015-12-04 09:58:55 <CubicEarth> But a large tx with many inputs, does that need to be handled in a serial way?
185 2015-12-04 10:07:33 <Lightsword> CubicEarth, right now even multiple CPU core optimization has a long way to go for bitcoin
186 2015-12-04 10:08:30 <Luke-Jr> Lightsword: eh? not sure how it can do much better
187 2015-12-04 10:10:34 <Lightsword> Luke-Jr, I’m not as familiar with the codebase as you are obviously but I get the idea that it doesn’t scale out to multiple cores that well overall, although I assume some of that is due to certain operations being difficult to thread
188 2015-12-04 10:11:09 <Luke-Jr> s/difficult/impossible/
189 2015-12-04 10:11:20 <Luke-Jr> where possible, it is already done multi-threaded
190 2015-12-04 10:13:06 <Lightsword> Luke-Jr, well yeah obviously there is quite a bit of stuff that is impossible but I think there is still a decent amount is, I’ve been trying to thread TestBlockValidity for instance
191 2015-12-04 10:14:21 <Luke-Jr> oh, I was thinking just the sync
192 2015-12-04 10:14:32 <midnightmagic> parallelizing merkel tree building won't win you much
193 2015-12-04 10:17:45 <Lightsword> Luke-Jr, I’m talking more about the application side stuff, networking stack etc, btw does it look like I have a locking issue here? https://github.com/bitcoin/bitcoin/pull/7167/files
194 2015-12-04 10:18:56 <Lightsword> would ValidateBlock get covered by the lock from getblocktemplate or do I need to somehow have the ValidateBlock hold the lock itself?
195 2015-12-04 10:18:57 <Lightsword> would ValidateBlock get covered by the lock from getblocktemplate or do I need to somehow have the ValidateBlock hold the lock itself?
196 2015-12-04 10:19:46 <Luke-Jr> Lightsword: the lock held by thread A cannot cover thread B :/
197 2015-12-04 10:20:39 <Luke-Jr> Lightsword: and block seems to still be in a race there
198 2015-12-04 10:23:59 <Lightsword> Luke-Jr, yeah that’s what I thought, I’m having trouble figuring out how to handle this though, would something along the lines of handing off the lock between threads work?
199 2015-12-04 10:24:41 <Luke-Jr> no, you probably need to wait for the lock in the new thread
200 2015-12-04 10:24:47 <Luke-Jr> the block is going to be a bigger issue for you :p
201 2015-12-04 10:24:51 <Luke-Jr> basically you need to copy it
202 2015-12-04 10:25:12 <Lightsword> ah, ok I was thinking I might
203 2015-12-04 10:25:12 <Luke-Jr> or do some smart pointer magic that probably also needs a lock :/
204 2015-12-04 10:25:13 <Lightsword> ah, ok I was thinking I might
205 2015-12-04 10:25:43 <Lightsword> btw why does TBV need a cs_main lock?
206 2015-12-04 10:25:44 <Lightsword> btw why does TBV need a cs_main lock?
207 2015-12-04 10:28:11 <Lightsword> Luke-Jr, the other thing I was thinking about what doing a join right before GBT returns, that way cs_main would be locked by GBT until TBV finishes right?
208 2015-12-04 10:28:12 <Lightsword> Luke-Jr, the other thing I was thinking about what doing a join right before GBT returns, that way cs_main would be locked by GBT until TBV finishes right?
209 2015-12-04 10:31:02 <Luke-Jr> Lightsword: that'll block your GBT reply
210 2015-12-04 10:31:19 <Luke-Jr> and probably impossible to prove correct
211 2015-12-04 10:31:28 <Lightsword> Luke-Jr, yeah I know but at least it won’t block the serialization parts of GBT right?
212 2015-12-04 10:32:07 <Luke-Jr> it will
213 2015-12-04 10:34:13 <Lightsword> Luke-Jr, so TBV wouldn’t be able to run in parallel to the serialization part?
214 2015-12-04 10:35:32 <Luke-Jr> it might, but not this way
215 2015-12-04 10:35:33 <Luke-Jr> it might, but not this way
216 2015-12-04 10:37:31 <CubicEarth> midnightmagic: wouldn't the merkle tree construction be speeded up by log2(number of txs)
217 2015-12-04 10:39:28 <CubicEarth> (assuming more available threads than txs)
218 2015-12-04 12:35:20 <cypherblock> Is there any estimate (or way to find out) how many different sets of transactions are being mined at any one time?
219 2015-12-04 12:50:42 <tulip> cypherblock: no.
220 2015-12-04 12:51:52 <tulip> memory pools are not contiguous, we don't even know how many unconfirmed transactions exist, it's basically an infinite amount. any given transaction has practically limitless permutations.
221 2015-12-04 12:54:08 <cypherblock> well certainly not an infinite amount. At anyone time there are a finite number of sets of transactions (what will go into blocks), being mined. There ought to be a way to estimate how many sets.
222 2015-12-04 12:56:07 <cypherblock> Do you mean that every miner has a different set of transactions? If so then the answer is just the total number of miners running at any one time. Do we have estimates for that?
223 2015-12-04 12:56:20 <tulip> it's for all intents unlimited. if you count each TXID as a transaction, there's 2**256 permutations for every signature in it. if you count every combination of inputs, well there's lots of them too.
224 2015-12-04 12:56:34 <tulip> nope.
225 2015-12-04 12:57:36 <cypherblock> I don't think you are understanding me. How many different blocks are being mined at any one time (disregarding coinbase, nonce, etc).?
226 2015-12-04 12:58:56 <tulip> what do you call different blocks?
227 2015-12-04 12:59:56 <tulip> for different merkle roots, again it's limitless.
228 2015-12-04 13:00:03 <cypherblock> Mining: Get a set of transactions from a node, add some stuff to it (coinbase, etc), start hashing.  Right? So that first step "get a set of transactions from a node" is what I'm talking about. How many different sets are being worked on?
229 2015-12-04 13:00:20 <sipa> cypherblock: likely as many as there are miners
230 2015-12-04 13:00:37 <sipa> cypherblock: as they all include a different coinbase transaction at least, that oays to themselves
231 2015-12-04 13:00:46 <tulip> no two nodes are going to have the same set of transactions, even if they are they are unlikely to put them in the same order.
232 2015-12-04 13:01:20 <cypherblock> Right, but ignoring coinbase and other non-transaction data.
233 2015-12-04 13:01:42 <tulip> I am ignoring the coinbase. the other transactions will be different.
234 2015-12-04 13:01:43 <sipa> very likely still all different
235 2015-12-04 13:01:44 <sipa> very likely still all different
236 2015-12-04 13:02:12 <sipa> they hear about transactions in a different order, miss some dependencies, have different policies, software versions, ...
237 2015-12-04 13:02:58 <cypherblock> Ok, so is there an estimate then on how many total miners are currently active at any one time?
238 2015-12-04 13:03:06 <tulip> nope.
239 2015-12-04 13:03:48 <tulip> could be one, could be millions, the design doesn't differentiate between one or the other. pools muddy things further, is someone connected to a pool a "miner"? probably not by this sort of definition.
240 2015-12-04 13:06:03 <cypherblock> Yes, a miner, in my sense is any process trying to solve a block. So a pool might have 2000 miners connected to it, all hashing away. But what is the real number (2000 was an example). If we had numbers from the largest pools, some estimate could be made.
241 2015-12-04 13:06:24 <sipa> why does it matter?
242 2015-12-04 13:06:40 <sipa> for all intents and purposes, all miners in a pool behave as one
243 2015-12-04 13:06:41 <sipa> for all intents and purposes, all miners in a pool behave as one
244 2015-12-04 13:07:31 <sipa> two hashing boxes owned by one person, is that one or two miners? they hash independently, trying different blocks
245 2015-12-04 13:09:39 <cypherblock> Yes trying different blocks = different miner. I started this discussion on "how many different sets of txs are being mined?". Answer was, as many as there are "miners". So how many miners. Different blocks being worked on at any one time?
246 2015-12-04 13:10:10 <sipa> Every asic tries different ranges of nonces
247 2015-12-04 13:10:27 <sipa> may i ask why that number matters?
248 2015-12-04 13:10:37 <afk11> they often use different merkle roots (ie, include different sets of transactions) so each header is distinct, and no duplication of work.
249 2015-12-04 13:11:36 <Lightsword> think they mainly just increment extranonce2
250 2015-12-04 13:11:37 <Lightsword> think they mainly just increment extranonce2
251 2015-12-04 13:11:54 <cypherblock> Well, was just trying to understand the issue of block propagation, and solutions like IBLT.
252 2015-12-04 13:12:13 <afk11> Lightsword: how long do you think it takes an asic to clear 2^32? it's pretty fast. they compute with a different merkle tree, meaning they can try another range of 2^32
253 2015-12-04 13:12:14 <Lightsword> well that’s more pools you are talking about then
254 2015-12-04 13:12:15 <Lightsword> well that’s more pools you are talking about then
255 2015-12-04 13:12:45 <Lightsword> yeah, they do internal work generation based off of stratum templates
256 2015-12-04 13:13:03 <tulip> Lightsword: some mining software rolls the sequence number.
257 2015-12-04 13:13:12 <Lightsword> yep
258 2015-12-04 13:32:57 <cypherblock> In stratum mining, the jobs are broadcast right, not requested by miners? Is that right. So in that case, everyone who gets the broadcast is working on the same set of transactions (merkle branch)? Right?
259 2015-12-04 13:35:51 <cypherblock> And then I guess it would be good to know how often (for a given mining pool) that merkle branch is changed in the broadcast.
260 2015-12-04 13:52:03 <paveljanik> One hour between blocks...
261 2015-12-04 13:54:19 <cypherblock> Ouch, yeah, slush pool just solved though.
262 2015-12-04 13:54:35 <cypherblock> 6 min ago.
263 2015-12-04 13:54:36 <cypherblock> 6 min ago.
264 2015-12-04 15:38:34 <elguido> .
265 2015-12-04 15:38:35 <elguido> .
266 2015-12-04 17:46:56 <BW^-> why does BitcoinD take 180% CPU when "-par=1" is set, is it hardwired to always have at least two running threads (one central and one for verifications), working concurrently?
267 2015-12-04 19:28:32 <jl2012> 73.83% of recent blocks are v4, and AntPool has around 20%. As soon as they upgrade, the softfork will complete. Beware of SPV mining.