1 2018-03-31 00:17:45 <nmnkgl> Hi. I'm doing some research and I was wondering if http://luke.dashjr.org/programs/bitcoin/files/charts/software.html is still supported? Also to doublecheck what I've found on Internet: does it attempts to find *all* the nodes (both accessible for in connections and not)?
  2 2018-03-31 01:31:51 <ProfMac> Does the sequence number of coinbase have to be any particular value?
  3 2018-03-31 01:36:19 <midnightmagic> ProfMac: there's a script/asm disassembler thing which feeds output to the "asm" output of e.g. decoderawtransaction which can be helpful in doing it yourself with an external tool.
  4 2018-03-31 01:37:22 <midnightmagic> ProfMac: that would be in the isvalid() chain of checks. *not* isstandard, but in the isvalid. If there isn't a limitation in the isvalid() then there isn't a sequence number requirement.
  5 2018-03-31 02:16:52 <echeveria> ProfMac: no.
  6 2018-03-31 06:30:38 <ProfMac> hex(301) == 0x12d;  "fd2d01" is a sequence of bytes that head a 301 byte long CDataStream..
  7 2018-03-31 06:48:07 <arubi> that's varint for you :)
  8 2018-03-31 06:49:42 <ProfMac> Well, I'm getting pretty close.  Plopped a string I obtained from getrawtransaction into my code, pushed a JSON interpretation out, it matches up with what I know separately.
  9 2018-03-31 06:50:40 <ProfMac> There is a quirk when I initialize a CDataStream from a hex stream, there are some extra length bytes.  I just used strm.ignore(count), but nope, won't be that simple.
 10 2018-03-31 06:51:09 <ProfMac> where are the varint's encoded, what do I grep for to read even more code?
 11 2018-03-31 06:52:09 <arubi> "Variable-length integers: bytes are a MSB base-128 encoding of the number."
 12 2018-03-31 06:52:12 <arubi> in serialize.h
 13 2018-03-31 06:55:27 <ProfMac> I've been reading serialize.h all day.  At least it's familiar territory.
 14 2018-03-31 06:57:44 <arubi> it's a simple encoding really.  the comments are very good
 15 2018-03-31 07:00:24 <ProfMac> Yes, I just read it.  I really have to read deeper in serialize.  I've captured the behavior I want in a new RPC command, I'm cleaning it up before I GitHub it for comment.
 16 2018-03-31 07:10:15 <ProfMac> anyone know the lowest block height that has stale blocks.
 17 2018-03-31 07:13:13 <wumpus> I don't think anyone has a total picture of that. Stale blocks are very easy to generate at low block height, due to the historical low difficulty, so clients will ignore them to avoid DoS potential.
 18 2018-03-31 07:14:27 <ProfMac> Yeah.  My approach may cause some problems, but I'm gonna code it up and see how it works in my lab.
 19 2018-03-31 07:23:52 <ProfMac> DoS now, by sending blocks to low block height?
 20 2018-03-31 07:24:17 <midnightmagic> I (hopefully) have an early view of stale/extinct/whatever blocks from my nodes' perspectives, assuming i managed to hang on to the block data properly.
 21 2018-03-31 07:26:09 <ProfMac> I mined for about a year.  I think I put my node online in early 2013, just when the takeoff from $20/BTE was starting.  There may be lots of stales when the Batch 2 Avalons hit the network in about June '14.
 22 2018-03-31 07:26:55 <midnightmagic> my nodes went online (originally) in late 2010..  but while I've been rabid about keeping old block data around, I did end up deleting much of it.
 23 2018-03-31 07:27:15 <midnightmagic> I remember watching huge forks happening it was awesome
 24 2018-03-31 07:27:26 <ProfMac> I seem to delete / loose data about a month before radical price drops in storage.
 25 2018-03-31 07:28:31 <ProfMac> Any clue to the timeframe.  I'm looking at blockchain.info now, they may be a resource, I don't know.  I'm looking at block 292,532 on their site at the moment.
 26 2018-03-31 17:57:52 <ProfMac> Is there a well known Public ECDSA key that is used in a coinbase on the main chain?
 27 2018-03-31 17:59:49 <arubi> I'd be surprised if there was.. but why?
 28 2018-03-31 17:59:50 <ProfMac> Oh, duh.  Nevermind, I'm looking right at the one I want.
 29 2018-03-31 18:01:03 <arubi> oh public key..  yea maybe
 30 2018-03-31 18:03:15 <ProfMac> For various reasons, I am reading a coinbase in hex from getrawtransaction, recovering a txn into memory from that hex, then manipulating it.  I was using Block 1 as my example, and it pays to address  12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX.  The PublicECDSA Key is right there in the getrawtransaction output,  it starts with 0496B5
 31 2018-03-31 18:03:54 <ProfMac> Oh, and a useful web page is at http://gobittest.appspot.com/Address
 32 2018-03-31 18:03:57 <arubi> basically almost all coinbase txs pay to pubkey
 33 2018-03-31 18:04:23 <arubi> the one at block 1 is satoshi's right :)
 34 2018-03-31 18:04:36 <arubi> well, at least it's likely
 35 2018-03-31 18:04:47 <ProfMac> Well, this is why I am working through the client code line-by-line.  It's surprising how many misconceptions hit one in the head when doing that.
 36 2018-03-31 18:05:19 <arubi> oh yes.  I know exactly what you mean
 37 2018-03-31 18:05:56 <ProfMac> I assume so.  Actually, I looked at the 1st 10,008 block and did some statistics on (blocktime-MedianPrevTime) and I assume all those blocks were mined in the same lab.
 38 2018-03-31 18:06:23 <arubi> I'd be very surprised if that were true
 39 2018-03-31 18:07:01 <arubi> we probably have folks right here in this channel who mined some of those
 40 2018-03-31 18:10:13 <ProfMac> Here are some of the plots that I made related to those 10,008 blocks.
 41 2018-03-31 18:10:15 <ProfMac> https://plus.google.com/u/0/photos/115426745065196075335/album/6519521612516431713/6519521612265200578?authkey=CK2wi7Gd3uaRwwE
 42 2018-03-31 18:11:22 <arubi> yea, but what makes you think they were all mined by the same party?
 43 2018-03-31 18:12:19 <arubi> there was only one mining software back then, the same one everybody used - bitcoind
 44 2018-03-31 18:13:00 <ProfMac> 1.  The distribution changes very soon after that block.  2.  There are dead times, that show up as spikes in this graph, where no mining takes places for hours to days, 3.  A person who has signed a recent message with a key used in a moderately early block claims to be the 2nd person to mine with Satoshi.
 45 2018-03-31 18:13:56 <satwo> Hello all. Is there a way to query the orphan pool in bitcoind? And does ZMQ fire a notification when an orphaned tx (or orphaned/stale block) is received from a peer?
 46 2018-03-31 18:14:01 <ProfMac> I'm not on a quest to prove or disprove this, it is just a thought in the back of my mind as I look at the data.
 47 2018-03-31 18:14:40 <ProfMac> I do wish people would call them stale and not orphan.  Orphans don't have parents.
 48 2018-03-31 18:14:56 <arubi> the only orphan is the genesis block :)
 49 2018-03-31 18:14:59 <ProfMac> marooned would have been so much better nomenclature.
 50 2018-03-31 18:15:14 <ProfMac> arubi, exactly.
 51 2018-03-31 18:15:51 <satwo> I agree; it just seems that people commonly say "orphan" to mean "stale"
 52 2018-03-31 18:15:56 <ProfMac> the code does call block which don't have their immediate ancestor on location at that node "orphans.."
 53 2018-03-31 18:16:24 <ProfMac> They do.  But I'm a professor and it is pretty innate to be really conservative about language.
 54 2018-03-31 18:16:37 <arubi> satwo, you can use 'getchaintips' to see some blocks that are stales
 55 2018-03-31 18:17:03 <ProfMac> oh yeah, we could be helpful.  Welcome aboard.
 56 2018-03-31 18:17:45 <ProfMac> I think that even blockchain.info calls them orphans.  Barren or childless would also work.
 57 2018-03-31 18:19:27 <ProfMac> So, to make some lab code that I am working on read cleaner, it would be really helpful if I could make "reservedkey" be the public key from block 1.  My mind is numb from reading and grepping and awking.  Anyone have any clue how to do this with minimal damage to the code?
 58 2018-03-31 18:19:36 <satwo> ProfMac Indeed! Lack of semantic precision/accuracy is the cause of too much unnecessary confusion.
 59 2018-03-31 18:20:10 <arubi> ProfMac, well there's generatetoaddress, but I'm not sure if it takes a pubkey as argument
 60 2018-03-31 18:20:24 <ProfMac> I'm on it.
 61 2018-03-31 18:20:37 <satwo> arubi thanks! I have used that a couple of times. Too bad it doesn't have all historical stale blocks but that's a minor issue.
 62 2018-03-31 18:20:54 <arubi> it has the blocks your node knows about
 63 2018-03-31 18:20:58 <arubi> the ones that were relayed to it
 64 2018-03-31 18:21:01 <ProfMac> After decsdes of avoidance, I dived into awk this week.  Boy I wish I had sooner.
 65 2018-03-31 18:21:11 <arubi> obviously we can't just tell everybody about every stale block
 66 2018-03-31 18:21:14 <satwo> What about orphaned tx? I know there is a separate orphan tx pool from the "regular" mempool, but don't know if it is queryable
 67 2018-03-31 18:21:28 <arubi> there's no such thing as orphan tx
 68 2018-03-31 18:21:36 <arubi> it would be invalid
 69 2018-03-31 18:21:53 <arubi> you can enable debugging in your node and listen to invalid txs coming in if they do
 70 2018-03-31 18:22:07 <arubi> debug=mempoolrej I think
 71 2018-03-31 18:22:20 <ProfMac> lol.  Ima gonna create a random block with a silly prev-block-hash and call it the stale-orphan.
 72 2018-03-31 18:22:27 <arubi> haha
 73 2018-03-31 18:22:49 <ProfMac> Actually, I have one at hand now...
 74 2018-03-31 18:23:18 <arubi> satwo, by orphan tx I do mean a tx with missing inputs.  it's completely invalid and won't be relayed
 75 2018-03-31 18:23:58 <arubi> if you mean double spends, these are not kept at all anywhere except for if they're in your wallet
 76 2018-03-31 18:24:19 <arubi> (or you can listen to them in debug)
 77 2018-03-31 18:24:44 <arubi> ProfMac, well not a mainnet difficulty block right :)
 78 2018-03-31 18:25:02 <satwo> Thanks arubi, I'll try using that bitcoind param
 79 2018-03-31 18:25:12 <arubi> alright
 80 2018-03-31 18:25:25 <satwo> Here's the reference to "orphan transactions" that got me curious: https://bitcoin.stackexchange.com/questions/54314/do-we-put-all-the-transactions-in-the-memory-pool/54322#54322
 81 2018-03-31 18:26:03 <ProfMac> I was looking at stales and thinking "winner take all" is kinda harsh.  Then I thought "I could take all the stales at the same height, and generate a block with a weighted payont to all the original coinbase.  Then I thought "but why, that's dumb."    Then I thought, actually I have probably just re-invented the way P2Pool works.
 82 2018-03-31 18:26:04 <satwo> Murch references a "separate orphan pool" for "orphaned transactions"
 83 2018-03-31 18:26:10 <arubi> oh well I stand corrected, but they're still invalid in our mempool without a parent
 84 2018-03-31 18:26:57 <arubi> it's probably an internal thing.  there's no reason for it to be query-able
 85 2018-03-31 18:28:41 <ProfMac> So, here are my thoughts on how to use "stales" to form a "campaign" as the start of a distributed mining pool
 86 2018-03-31 18:28:44 <ProfMac> https://docs.google.com/document/d/1oy5neCZMH8FGbc0KA6dGfl9caPqeegh-rUGMwNJOiOI/edit
 87 2018-03-31 18:28:49 <arubi> well p2pool works by letting folks mine with a lower difficulty to get shares, eventually one share is below the actual diff
 88 2018-03-31 18:29:06 <arubi> ethereum has something called "uncle blocks" that uses stales
 89 2018-03-31 18:29:37 <ProfMac> Yes, I assumed that.  Uncle blocks.  LOL.  That's actually a really good name.
 90 2018-03-31 18:29:59 <arubi> not sure how it's working out for them
 91 2018-03-31 18:30:27 <arubi> there's been a good presentation about exploiting that mechanism..  I forget what the name was
 92 2018-03-31 18:31:00 <ProfMac> Well, I'm coding up my idea, testing it on a quarantine net.  My progress meter is somewhere between "gee, this is harder than I thought" and "why did I ever think it was a good idea"
 93 2018-03-31 18:31:16 <arubi> sounds like coding for bitcoin :)
 94 2018-03-31 18:32:04 <ProfMac> I'm stuck back in time (for reasons that won't easily budge) and am implimenting this in the 0.8.1 client.
 95 2018-03-31 18:32:36 <arubi> the longer you continue in your 0.8.1, the harder it'll be to advance to a recent version
 96 2018-03-31 18:35:06 <arubi> bite the bullet.  I had to do it a bunch of times.
 97 2018-03-31 18:40:44 <arubi> I remember having to migrate from 0.13.1 to 0.15, it sucked as I did want to continue working on my stuff but eventually it was for the best and it gave me the chance to refractor.  now moving from 0.15 to 0.16 will be easier once I get around to it
 98 2018-03-31 18:43:27 <ProfMac> I'll move, in time.  But my karate instructor always told us to master one style first, don't keep hopping around.
 99 2018-03-31 18:54:44 <arubi> :)
100 2018-03-31 18:56:04 <ProfMac> lol.  always too late for the conversation.  I should have said "professors work in all three areas, the past, present, and future"
101 2018-03-31 19:03:50 <arubi> hopefully not a too distant future
102 2018-03-31 19:04:51 <ProfMac> I usually try to plan ahead 10-15 years.  I'm also trying to look 50 years ahead and plant some trees for my grandsons that will produce food as the climate changes.
103 2018-03-31 19:07:13 <arubi> 10-15 is fine, 50 years is only viable if we manage to make humans live longer than they would on average now.  I think we would
104 2018-03-31 19:07:52 <ProfMac> One energetically does things for grandkids that seems foolish as a young parent.
105 2018-03-31 19:09:54 <arubi> well as a non parent, I just hope to live long enough to be around for when we invent immortality :)
106 2018-03-31 19:13:00 <arubi> we'll see.  eternity is a pretty long time to be filling buckets
107 2018-03-31 19:13:00 <ProfMac> I think of interesting new things frequently enough.
108 2018-03-31 19:13:14 <ProfMac> I interviewed to teach 9-12th grade mathematics on the reservation at a school dedicated to revitalizing the Ojibwe language.  Cool, huh?
109 2018-03-31 19:14:08 <arubi> that's pretty exotic yea
110 2018-03-31 19:16:00 <ProfMac> I am strongly considering a project to make a blockchain based auction system.  I did run an auction on the blockchain back in 2013 when I bought my Avalon.
111 2018-03-31 19:16:27 <arubi> what's the blockchain used for?
112 2018-03-31 19:16:47 <ProfMac> Well, I'm glad you asked.  Let me find a link ...
113 2018-03-31 19:18:32 <ProfMac> Here is a link to one of the threads.  They are dreary long and tedious, as business things are when they are not hot & angry.
114 2018-03-31 19:18:36 <ProfMac> https://bitcointalk.org/index.php?topic=159214.0
115 2018-03-31 19:20:29 <ProfMac> Basically, you pay to an address to bid.  Top n bidders win n items.  The blockchain provides timestamps for end of auction, public transparency of bid and pay process, guarantee of interested, funded bidders(no shilling)
116 2018-03-31 19:21:29 <ProfMac> I might add the ability to store documents, or at least links with sha-256 to documents, and introduce a kickoff transaction.
117 2018-03-31 19:22:28 <ProfMac> lol.  Back when I could own 62.781 BTC.
118 2018-03-31 19:25:07 <arubi> I'll read through it.  personally I think that by now with advancements in bitcoin the on chain stuff is mostly obsolete, so it might be interesting to move the mechanism off chain
119 2018-03-31 19:28:19 <ProfMac> ruptively) submitted.
120 2018-03-31 19:28:19 <ProfMac> Yes.  Even then, there was the mechanism to put time limits on transactions, and "microchannels."  I tested some of that stuff on a private network and it seemd to be working in 0.8.1 (which was the new & trendy release then).  The bidders all said that I make things way too complicated, so I did it on chain.  One danger with microchannels back then was that they were live transactions traded between parties, but, ahem,  not (dis
121 2018-03-31 19:30:03 <arubi> right, back then script couldn't handle proper timelocks and things had to be done interactively.  these days we have timelocks in script and contract hash signing
122 2018-03-31 19:30:38 <arubi> sign to contract rather
123 2018-03-31 19:33:06 <ProfMac> um...  in which version did the timelocks get stable?
124 2018-03-31 19:35:44 <arubi> we have two opcodes, csv and ctlv.  cltv is earlier, csv was enabled on mainnet in may 2016
125 2018-03-31 19:36:26 <arubi> oh sorry, the soft fork count was started in may
126 2018-03-31 19:38:01 <arubi> I think it was at 0.12~
127 2018-03-31 19:39:04 <ProfMac> ah, maybe that's my next version to deep-study.
128 2018-03-31 19:40:51 <arubi> I think you should just go to something after 0.13.1 that has segwit too
129 2018-03-31 19:41:00 <arubi> well, 0.13.1 and on
130 2018-03-31 19:41:47 <ProfMac> I'm kind of avoiding segwit at the moment.  I can't seem to get answers about it that aren't hot to discuss.
131 2018-03-31 19:42:00 <arubi> like what?  it's really simple
132 2018-03-31 19:42:25 <arubi> it's genius level simple, that's what's so great about it
133 2018-03-31 19:42:42 <ProfMac> Can you give the 50 word summary?
134 2018-03-31 19:44:37 <arubi> well, it does two major things.  first is adding a new scriptpubkey type that is versioned, so soft forks become easier, and second is that it moves scriptsig stuff into a new construct in a transaction called a witness stack.  the items in the witness stack are not hashed as part of the txid and the items are not treated as a script (like scriptsig) but rather as a pure stack
135 2018-03-31 19:45:28 <arubi> that's it mostly
136 2018-03-31 19:46:03 <ProfMac> I assume the first question raised was about not hashing the witness stack.
137 2018-03-31 19:46:43 <arubi> it's hashed, just not part of the txid.  it's hash is just called a "hash" and is committed to by a miner in the coinbase tx
138 2018-03-31 19:47:48 <arubi> so a transaction is still referred to by its txid, but for validation in a block the hash is used.  signing is completely overhauled and is much simpler than legacy signing
139 2018-03-31 19:50:26 <ProfMac> I fear to ask.  Why was this so divisive?
140 2018-03-31 19:50:50 <arubi> it really wasn't.  the fud was all manufactured by altcoin pumpers
141 2018-03-31 19:51:22 <arubi> in reality, 0.13.1 (the first version to add segwit) was adopted very quickly by the network
142 2018-03-31 19:52:10 <arubi> there's this other thing where segwit block validation disables covert asicboost.  some miners were not happy with that
143 2018-03-31 19:55:52 <ProfMac> ah.  Bitcoin does seem to attract certain personality types.
144 2018-03-31 19:56:50 <arubi> for better or for worse right :)
145 2018-03-31 19:58:12 <ProfMac> I've stopped bringing dates to bitcoin meetups.
146 2018-03-31 19:59:06 <arubi> haha, I see my friends are getting tired of me talking about bitcoin so I can assume a date would go crazy in a meetup
147 2018-03-31 19:59:53 <arubi> I did mention it on dates, but it didn't seem to be interesting for her
148 2018-03-31 20:01:17 <arubi> eventually I'll have to stop mentioning it to people altogether.  you wouldn't tell people that you keep gold bars at home
149 2018-03-31 20:01:39 <ProfMac> She just doesn't want me to wear my tin-foil hat to dinners with her friends.
150 2018-03-31 20:02:03 <ProfMac> Yeah, the really rich are quite about it.
151 2018-03-31 20:04:11 <ProfMac> I think my project is ready for GitHub.  I'm going to eat and let my thoughts percolate before I do that.
152 2018-03-31 20:04:24 <arubi> cool, I'm off to sleep o/
153 2018-03-31 21:49:41 <ProfMac> I have pushed git@github.com:a-mcintosh/expt-block1.git  This is a new rpc command that creates a coinbase transaction, serializes it, recovers the hex, recovers a txn, shows that the txn can be manipulate under program control, emits a JSON object with verification.