1 2015-10-17 18:50:11 <Dr-G2> Where is Aquentin the core dev?
  2 2015-10-17 19:57:41 <maaku> Dr-G2: ?
  3 2015-10-17 20:32:23 <skyzer> gentlemen, something super strange happened with malleability
  4 2015-10-17 20:33:19 <skyzer> be408a5a8cb17191d6684f61dd0a4d3bdcdcd787f18cdaeb6a42fd29a4f1f343 <- this double-spend transaction did hit my bitcoin core, which is now just -1 confirms, but the other malleated transaction that got confirmed never hit my bitcoin core callback :( 51c402a48c7394336e9038e2fc31e92c06f4fb6805ff0151e9946bb1a9c1e6fc
  5 2015-10-17 20:34:02 <skyzer> there is a footprint of the first transaction hash, but not the second one that got confirmed. this is frightening, because of that person wasn't credited
  6 2015-10-17 20:34:36 <skyzer> I'm not sure how to reproduce that or give additional information to verify it.
  7 2015-10-17 20:35:38 <belcher> why is it frightening?
  8 2015-10-17 20:36:13 <belcher> one cause might be that miners are malleating transactions now and not broadcasting them first, only in the completed blocks
  9 2015-10-17 20:37:03 <skyzer> it's been already more than 12 blocks
 10 2015-10-17 20:37:47 <skyzer> frightening is since malleated confirmed transaction never hit my bitcoin core, so my app didn't credit that user at all
 11 2015-10-17 20:38:30 <belcher> yes thats worrying
 12 2015-10-17 20:38:45 <belcher> so your node only notified you for the unconfirmed but not the confirmed?
 13 2015-10-17 20:39:05 <sturles> This may happen if there is a reorg.
 14 2015-10-17 20:39:16 <sturles> Assuming you use walletnotify.
 15 2015-10-17 20:39:26 <skyzer> belcher: yes
 16 2015-10-17 20:39:39 <skyzer> yes, using walletnotify exactly for callback to my script
 17 2015-10-17 20:39:53 <belcher> what happens so that walletnotify doesnt fire if theres a reorg ?
 18 2015-10-17 20:40:02 <sturles> But there should be a walletconflicts key in gettransaction for the tx with -1 confirmations.
 19 2015-10-17 20:40:55 <skyzer> well, now if it's double spent transaction i just drop it after 6 hours if it's -1 confirm. i still want to figure out why bitcoin core didn't notify about confirmed tx
 20 2015-10-17 20:41:01 <sturles> walletnotify triggers the first time bitcoind see the transaction and on the first confirm.  If there is a reorg, the transaction may suddenly have two confirmations.
 21 2015-10-17 20:41:46 <sturles> In that case walletnotify don't trigger.  It has happened to me a few times, which is the main reason why I use accounts.
 22 2015-10-17 20:42:01 <belcher> is there a workaround ?
 23 2015-10-17 20:42:05 <skyzer> yes, sometimes walletnotify shoorts 2-3 confirms notification, but that's okay, now i know the answer to that why not always on 0 and 1.
 24 2015-10-17 20:42:29 <skyzer> sturles, wondering same, how to get that malleated confirmed transaction by bitcoin core and that it notifies
 25 2015-10-17 20:42:30 <belcher> its odd that walletnotify wont fire if the tx has 2 confirms instaed of 1
 26 2015-10-17 20:42:31 <Luke-Jr> also, it should be considered the same transaction even if the txid changes
 27 2015-10-17 20:42:42 <sturles> Receiving addresses are tied to accounts, and if a transaction arrives without trigging walletnotify, or if the walletnotify script should fail for some reason, the account will have a positive balance.
 28 2015-10-17 20:43:04 <Luke-Jr> but yes, I'd expect to see it fire for a confirmation (at 6 blocks, not 1!)
 29 2015-10-17 20:43:14 <belcher> sturles does that mean you have to poll account balances ?
 30 2015-10-17 20:43:18 <Luke-Jr> sturles: accounts are dead
 31 2015-10-17 20:43:46 <sturles> belcher: Not really, I check them a few times a day just to make sure everything is working.
 32 2015-10-17 20:44:13 <sturles> Luke-Jr: So I've heard, but there isn't a replacement yet.
 33 2015-10-17 20:44:25 <Luke-Jr> sturles: external accounts you implement yourself
 34 2015-10-17 20:44:34 <Luke-Jr> sturles: afaik nobody plans to replace them in bitcoind
 35 2015-10-17 20:44:40 <sturles> Luke-Jr: How would that catch this type of errors?
 36 2015-10-17 20:44:49 <Luke-Jr> sturles: by not having this type of errors?
 37 2015-10-17 20:45:29 <sturles> Luke-Jr: OK, so this walletnotify problem will get fixed?
 38 2015-10-17 20:45:52 <Luke-Jr> sturles: depends if someone fixes it
 39 2015-10-17 20:46:05 <Luke-Jr> so far it hasn't been reported yet ;)
 40 2015-10-17 20:46:12 <skyzer> it's still a mystery why bitcoin core never fired callback for malleated confirmed tx 51c402a48c7394336e9038e2fc31e92c06f4fb6805ff0151e9946bb1a9c1e6fc and how to solve it before user complains about not receiving payment and crediting them manually
 41 2015-10-17 20:46:37 <Luke-Jr> skyzer: open an issue if you're sure it never did
 42 2015-10-17 20:46:44 <skyzer> all the malleated transaction last week and before were fired by bitcoin core
 43 2015-10-17 20:46:52 <skyzer> transactions*
 44 2015-10-17 20:46:58 <Luke-Jr> could your notify script have failed somehow?
 45 2015-10-17 20:47:01 <sturles> skyzer: I already explained that.  A reorg made it get two confirmations instantly, skipping the first.
 46 2015-10-17 20:47:06 <skyzer> okay, thanks, i'd need to provide as much details as possible, can you recommend what I can provide extra?
 47 2015-10-17 20:47:14 <Luke-Jr> sturles: you've confirmed that behaviour is there?
 48 2015-10-17 20:47:33 <sturles> Luke-Jr: Yes.  Happened to me many times.
 49 2015-10-17 20:47:41 <sturles> Reported on #bitcoin-dev as well.
 50 2015-10-17 20:47:54 <Luke-Jr> sturles: GitHub issues is the place to report stuff if you don't want it forgotten
 51 2015-10-17 20:48:10 <sturles> Requires a GitHub account, I think.
 52 2015-10-17 20:48:19 <Luke-Jr> skyzer: if you are able to write a test case reproducing it, that would be very handy
 53 2015-10-17 20:48:39 <Luke-Jr> skyzer: it doesn't necessarily need to do malleation, just reproduce not getting the 1-block notify in a reorg
 54 2015-10-17 20:48:58 <Luke-Jr> sturles: indeed; if you don't want one, pastebin the details to me and I can open it for oyu
 55 2015-10-17 20:49:01 <skyzer> so it didn't fire callback since that malleated confirmed tx had already 2 confirms on reorg. sometimes I do see in my database transactions with not 1 confirms but over it.
 56 2015-10-17 20:49:10 <Luke-Jr> sturles: although you won't get notified for comments if we do that
 57 2015-10-17 20:49:13 <skyzer> how can re-org be reproduced?
 58 2015-10-17 20:49:45 <Luke-Jr> skyzer: some unit tests generate blocks; so make block A->B->C, then C'->D'
 59 2015-10-17 20:50:23 <sturles> Luke-Jr: Last time I reported it here, I got the understanding it wasn't considered a bug, and I'm happy as long as I can work around it with accounts.  Could write up something of course.
 60 2015-10-17 20:50:48 <Luke-Jr> sturles: then maybe there's more to it than I know
 61 2015-10-17 20:50:55 <Luke-Jr> actually, as you describe it, it sounds impossible
 62 2015-10-17 20:50:59 <Luke-Jr> because *all* reorgs would trigger it
 63 2015-10-17 20:51:02 <Luke-Jr> and clearly that is not the case
 64 2015-10-17 20:51:30 <Luke-Jr> (based on skyzer's description)
 65 2015-10-17 20:51:37 <sturles> No, usually transactions will be confirmed in both chains.
 66 2015-10-17 20:51:53 <Luke-Jr> ah, possible
 67 2015-10-17 20:52:02 <sturles> So you got one confirmation already, and now you just get an alternate confirmation.
 68 2015-10-17 20:52:24 <kanzure> sturles: "reported on #bitcoin-dev as well" (this is #bitcoin-dev)
 69 2015-10-17 20:52:31 <sturles> I know. :-)
 70 2015-10-17 20:53:06 <sturles> A long time ago.  Could be during the previous malleability storm in January or February 2014.
 71 2015-10-17 20:53:41 <sturles> I have been using walletnotify since it arrived in git.
 72 2015-10-17 20:58:17 <sturles> To fix this properly, walletnotify should probably trigger on unconfirms as well, if there is a reorg where the tx isn't confirmed, or a tx got gonfirmed with a new txid.
 73 2015-10-17 20:59:54 <skyzer> walletnotify should trigger on any new transaction on 0 and 1 confirm, so no transactions are missed. let your software handle the user balances, since you are waiting anyway for couple of confirmations and not confirmed transactions will be dropped
 74 2015-10-17 21:00:23 <sturles> Looks like I reported this first here on Mon Aug 26 2013:
 75 2015-10-17 21:00:24 <sturles> 18:02 < sturles> Strange..  I use walletnotify for notification of new transactions to my wallet.  I just got a transaction where I got a notify at 0 confirmations (first spotted) and then at 2 confirmations!  No notify for first confirmation.  How can this happen?
 76 2015-10-17 21:00:28 <sturles> 18:02 < Luke-Jr> sturles: reorg
 77 2015-10-17 21:00:37 <sturles> From my log. :-)
 78 2015-10-17 21:00:48 <sturles> Luke-Jr: You knew about it then. :-)
 79 2015-10-17 21:00:53 <Luke-Jr> sturles: but it did notify you!
 80 2015-10-17 21:01:27 <skyzer> sturles, it notified, I see walletnotifys on 1+ confirms all the time. my case was that malleated confirmed transaction was never notified
 81 2015-10-17 21:02:13 <sturles> Same thing.  You didn't see the malleated transaction until it suddenly got two confirmations due to a reorg.
 82 2015-10-17 21:02:32 <sturles> So it wouldn't trigger at all.
 83 2015-10-17 21:02:33 <belcher> okay so it will notify you
 84 2015-10-17 21:02:43 <belcher> and you can use gettxout to check how many blocks deep it is
 85 2015-10-17 21:03:31 <Luke-Jr> skyzer's problem is that he was not notified..
 86 2015-10-17 21:03:43 <skyzer> so in 1 year out of 10k incoming transactions, that has happened now first time, that bitcoin core never notified of malleated confirmed tx, at least first time user is reporting missing incoming tx and unconfirmed -1 confirm transactions.
 87 2015-10-17 21:04:23 <skyzer> so just to repeat: the uncofirmed -1 transaction was notified, but not the malleated confirmed
 88 2015-10-17 21:04:36 <skyzer> i guess that's it, need to make some actions to find out the prob
 89 2015-10-17 21:07:14 <sturles> skyzer: Does the -1 transaction have a walletconflicts key?
 90 2015-10-17 21:07:28 <sturles> Something in the array?
 91 2015-10-17 21:08:24 <skyzer> i'm not sure what you mean, the -1 tx is that one I posted above: be408a5a8cb17191d6684f61dd0a4d3bdcdcd787f18cdaeb6a42fd29a4f1f343
 92 2015-10-17 21:12:34 <sturles> No notifications is possible as well, but very rare.  Usually you see a transaction before it has been confirmed, but new transactions may show up in a new block as well.  Already confirmed.  If it first shows up twice confirmed in a reorg, you won't get a notification.
 93 2015-10-17 21:14:39 <skyzer> actualy send out happened in bitcoin core, the web server just didn't get response. It happens like once, twice a day, I think it happens while bitcoin core is doing some heavy internal things like block found, reindex etc? Once I ran 'listaddressgroupings' and whole bitcoin core froze, had to kill process and restart it
 94 2015-10-17 21:14:39 <skyzer> Also while now running a heavily used wallet there are other issues I see from time to time, one that bugs the most is that when 'sendmany' happens, my web server request times out, because for bitcoin core takes too much time to execute, but it still sends out the transactions! So there is no response from bitcoin core and on my app side it's shown that transaction failed, even though
 95 2015-10-17 21:18:34 <skyzer> sturles: so your only workaround is some script that is verifying balances through accounts not to miss any transaction?
 96 2015-10-17 21:22:40 <skyzer> I wonder if any of block explorers show to for each transaction on how many confirmations it was first seen
 97 2015-10-17 21:24:23 <skyzer> sturles: I see solution here without accounts is to scan on every new block if some of your addresses have received transaction that is not yet recorded in your internal db
 98 2015-10-17 21:28:22 <gmaxwell> skyzer: the problem you're encoutering is likely that you have http keepalive sessions still up and you're exausting your RPC threads.
 99 2015-10-17 21:28:52 <gmaxwell> Turning off keepalive in bitcoin core should make the issue go away. (or fix your calling library to close connections more agressively when you're not actually pipelining)
100 2015-10-17 21:29:41 <skyzer> gmaxwell, I've actually redone everything into queue, and problem lessened alot, only once twice a day. So basically all requests from my app to bitcoin core are going 1 by 1 through queue
101 2015-10-17 21:30:11 <skyzer> Thanks for that alot! I have tuned my nginx alot, timeouts to 300sec, and error is still popping out
102 2015-10-17 21:30:16 <gmaxwell> skyzer: some jsonrpc/http client libraries leak kept alive connections, unfortunately.
103 2015-10-17 21:30:46 <gmaxwell> you an also increase the rpc thread count in bitcoin core (though if your client is leaking kept alive connections even higher limits will run out eventually)
104 2015-10-17 21:31:51 <skyzer> thanks! wonderful, will be testing it tmrw!
105 2015-10-17 21:32:23 <skyzer> never would have guessed myself
106 2015-10-17 21:33:01 <gmaxwell> yea, thats the irritating thing about this behavior, anyone who suffers from it never knows where to look.
107 2015-10-17 21:33:09 <gmaxwell> 0.12 should fix this conclusively.
108 2015-10-17 21:34:07 <skyzer> great! first thing iäll check my json-rpc php wrapper :D
109 2015-10-17 21:35:23 <sturles> skyzer: It will always be possible to work around with horrible kludges.  Accounts work well for me.
110 2015-10-17 23:28:11 <trinity_help> hey, I was wondering, did anyone reputable ever vette the Vanitygen codebase?