1 2014-01-27 04:56:23 <jspilman> asking dev questions in #bitcoin - facepalm
2 2014-01-27 05:14:05 <andytoshi> jspilman: what do you mean by "less priv'd"? to even do createrawtransaction you need rpc
3 2014-01-27 05:15:31 <andytoshi> well, createrawtransaction i could do with my own code. but for usability reasons i need listunspent and getnewaddress to work.
4 2014-01-27 05:24:40 <jspilman> andytoshi: of course not, you can just broadcast a transaction, but for example no way to get a public key that way
5 2014-01-27 05:24:50 <jspilman> preceisely
6 2014-01-27 05:25:37 <jspilman> those are basically the minimum required commands to make an app of significance without touching the wallet
7 2014-01-27 05:26:09 <jspilman> or I should say, owning not touching
8 2014-01-27 05:27:33 <jspilman> you could do some other interesting things, like send a payment request to bitcoind, but as you say, usability
9 2014-01-27 05:30:48 <jspilman> would you ever have / want a situation where you can query all the public data, list unspent and create receive points, you build the transaction but signing happens in bitcoind? almost like adding plugin support
10 2014-01-27 07:47:51 <aynstein> is disablewallet production ready? Or is it still considered at all experimental?
11 2014-01-27 07:48:52 <brisque> aynstein: I've been using it for a while with no problems. it breaks p2pool, even if your payout address isn't in the associated node's wallet. easy fix though.
12 2014-01-27 07:49:52 <aynstein> brisque: Thats what I was going to ask next, I want to use it with a p2pool node
13 2014-01-27 07:50:21 <aynstein> brisque: whats the fix?
14 2014-01-27 07:50:40 <brisque> aynstein: https://github.com/forrestv/p2pool/pull/148
15 2014-01-27 07:51:24 <aynstein> Thank you!
16 2014-01-27 07:53:08 <aynstein> Now I am very confused. Is there a simple answer to why using the genesis block fixes this?
17 2014-01-27 07:54:18 <brisque> it's just a sanity check to see if bitcoind is alive. the patch just changes it to something which exists when the wallet is disabled.
18 2014-01-27 07:54:52 <aynstein> Ahh, now that is far less scary then I was expecting :)
19 2014-01-27 07:55:30 <aynstein> BTW, thanks for the info on the block difficulty calculations, gave me hours of frustration and entertainment all rolled into one!!
20 2014-01-27 08:04:40 <skruffy[]> Hi folks
21 2014-01-27 08:05:05 <skruffy[]> Do you care about proper ASLR for window binaries?
22 2014-01-27 08:09:09 <SomeoneWeird> skruffy[], I'm sure they probably do, is there something broken?
23 2014-01-27 08:10:09 <skruffy[]> ld skips reloc section during linking for exe, windows loader can't to change base addr if no relocations.
24 2014-01-27 08:12:27 <SomeoneWeird> hrm, right
25 2014-01-27 08:12:29 <SomeoneWeird> open an issue on github
26 2014-01-27 08:22:24 <wumpus> skruffy[]: some flag needs to be passed to the linker to avoid that?
27 2014-01-27 08:23:48 <wumpus> I had to add -fPIC for some of the dependencies to make pie work on x64 for linux
28 2014-01-27 08:24:11 <wumpus> but the idea of PIC is that no relocations are needed and the code is just relocatable, but windows may be different
29 2014-01-27 08:27:33 <skruffy[]> -fPIC not support by pe target for gcc
30 2014-01-27 08:27:48 <wumpus> then what should be provided?
31 2014-01-27 08:28:39 <skruffy[]> -pie creates reloc section but broken exe with wrong entry point and waste export table
32 2014-01-27 08:29:11 <wumpus> right, it's only worth doing if it doesn't break the output
33 2014-01-27 08:29:21 <skruffy[]> --export-all-symbols creates working exe with reloc section but export ALL symbols.
34 2014-01-27 08:29:26 <wumpus> bleh
35 2014-01-27 08:29:30 <skruffy[]> ls patch needs.
36 2014-01-27 08:29:34 <skruffy[]> *ld
37 2014-01-27 08:31:51 <wumpus> is there a patch for upstream mingw-w64?
38 2014-01-27 08:32:13 <skruffy[]> mingw not writes binutils
39 2014-01-27 08:32:23 <skruffy[]> why to patch mingw?
40 2014-01-27 08:32:52 <wumpus> I don't want to argue semantics, it's part of the mingw-w64 cross compile chain
41 2014-01-27 08:38:41 <wumpus> likely in the future we will switch gitian to build the entire toolchain from scratch, so in that case we could apply patches to the compiler or binutils as well, but that's not currently possible
42 2014-01-27 08:39:08 <wumpus> you're still welcome to file an issue though with a link to the patch, so it can be picked up then
43 2014-01-27 08:39:09 <skruffy[]> http://pastebin.com/HaVCuH2D some version of patch that adds new option to create reloc section.
44 2014-01-27 12:34:19 <wallet42> is a transaction that has 2 outputs to the same address valid?
45 2014-01-27 12:36:15 <sipa> yes
46 2014-01-27 12:36:20 <wallet42> crafting such a tx with bitcoind is not possible afaik since the destinations is a json key-value. but would it be a valid transaction? and if yes how will it be listet in listtransactions.
47 2014-01-27 12:39:23 <sipa> you can try it with createrawtransaction :)
48 2014-01-27 12:39:57 <wallet42> no you cant since createrawtransaction takes a {"address": 1} json hashmap
49 2014-01-27 12:40:17 <wallet42> so the keys would overwright each other (imo)
50 2014-01-27 12:41:00 <gavinandresen> wallet42: yes. Why do you want to have 2 outputs to the same address? What are you trying to accomplish? Or just curious?
51 2014-01-27 12:41:10 <wallet42> i dont want to ^^
52 2014-01-27 12:42:11 <wallet42> it would complicate things for my project where i try to have a unique compound pk on txid-account
53 2014-01-27 12:42:27 <wallet42> for the received funds
54 2014-01-27 12:42:47 <sipa> oh, right, you'll need to construct the transaction by hand
55 2014-01-27 12:42:57 <sipa> but you can still use signrawtransaction to sign it, iirc
56 2014-01-27 12:43:05 <gavinandresen> if you want to test, you could createrawtransaction with two addresses, and then just replace the 20 bytes of the second address hash with the first. signrawtransaction will (I think-- not sure it was ever tested) sign it
57 2014-01-27 12:43:21 <wallet42> okay so it is very unlikly this will be an issue
58 2014-01-27 12:43:42 <wallet42> and the user only hurt himself if i only credit 1 output
59 2014-01-27 12:43:54 <freewil> wallet42, thats what i would do
60 2014-01-27 12:44:13 <freewil> worst case scenario you have to do some sort of manual adjustment to increase the credit
61 2014-01-27 12:44:59 <wallet42> thx!
62 2014-01-27 12:46:01 <gavinandresen> wallet42: that feels like the kind of decision you might very much regret if your project is successful
63 2014-01-27 12:46:47 <gavinandresen> wallet42: if coinjoin or some other merge-transactions-together privacy technique gets popular you might find your customers complaining if they lose money because you can't handle one transaction paying to an address twice
64 2014-01-27 12:48:01 <freewil> that's a good point
65 2014-01-27 12:48:14 <freewil> alternatively you could add the tx output index to the compound pk
66 2014-01-27 12:48:39 <wallet42> gavinandresen: how would it look like in listtransactions ? i think i simply sum the recieve funds group by txid-account and still have 1 row in the db
67 2014-01-27 12:50:39 <sipa> wallet42: you should see it as separate entries in listtransactions
68 2014-01-27 12:52:12 <wallet42> ok then i sum these and credit in 1 row. it seems to me to be a very unlikely case since you cannot even create it without tricks like replacing a placeholder address
69 2014-01-27 12:54:12 <wallet42> another question is: i would like to use listsinceblock in a cronjob that credits all the receive txs, then saves the "lastblock" in cache and runs every minute or so. but what happens if there is a reorg and "lastblock" becomes an orphan?
70 2014-01-27 14:15:39 <gavinandresen> wallet42: if you are crediting 1-confirmation transactions, then you must realize that a re-org has happened and possibly un-credit them. You could notice by calling getblockhash <height> and seeing if that is still the block you cached. If not⦠then you've got work to do to figure out if the re-org invalidated any transactions that were confirmed before.
71 2014-01-27 14:17:29 <TD> hey gavin
72 2014-01-27 14:17:35 <gavinandresen> hey mike
73 2014-01-27 18:29:40 <nsokl> hi guys, forgive me if this is the wrong channel for this. I have a raw multisig transaction (BBE dump is at http://pastebin.com/qZMNng02) that I need to sign in bitcoinjs-lib (the unsigned txn was produced on the server side and I need to sign it with the privkey on the webclient). It's a bit of an odd txn (actually it encodes metadata ito the 2nd pubkey in the multisig txout)... unfortunately the libbitcoin-js code and functions i'
74 2014-01-27 18:29:40 <nsokl> ve found assume a txin with a script set.. I am a bitcoin dev newbie, so in this odd case, I'm not sure the process. bitcoind's signrawtransaction will sign it (and it's ready and valid for broadcast if the privkey were in the wallet), but I'm not sure what the process is here, given I need to sign it in JS client side. any pointers would be more than welcome!!
75 2014-01-27 18:33:55 <kuzetsa> I think this is a strange error to get while compiling the source tarball from https://github.com/bitcoin/bitcoin/archive/v0.8.6.tar.gz ---> (quote / error message during compile) [" fatal: Not a git repository (or any parent up to mount point /var/tmp) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). "]
76 2014-01-27 18:34:07 <nsokl> moreover, if I pass it through bitcoind's signrawtransaction (where the privkey is not in the wallet, of course), I get back a modified output that has a "script" field in txout 0. I'm not sure if I should be signing and working with that, or with the original raw txn
77 2014-01-27 18:34:56 <kuzetsa> it works fine with the v0.8.5 tarball located in the same folder onthe github repo for bitcoin
78 2014-01-27 18:38:31 <midnightmagic> kuzetsa: It's simpler to grab the entire repository and then checkout the tag.
79 2014-01-27 18:41:26 <kuzetsa> midnightmagic: all I did to get this error was this:
80 2014-01-27 18:41:32 <kuzetsa> emerge bitcoind --verbose --ask
81 2014-01-27 18:43:23 <kuzetsa> and /usr/portage/net-p2p/bitcoind/bitcoind-0.8.6.ebuild (( the official package on gentoo )) pulls from https://github.com/bitcoin/bitcoin/archive/v0.8.6.tar.gz
82 2014-01-27 18:43:48 <kuzetsa> because gentoo doesn't force users to have git when installing packages if a source tarball for a stable, known version is available
83 2014-01-27 18:44:01 <midnightmagic> kuzetsa: Ah, I see. Sorry I thought you were just doing it manually.
84 2014-01-27 18:44:33 <kuzetsa> no worries
85 2014-01-27 18:45:38 <kuzetsa> I wonder if this is related to why gentoo still has bitcoind version 0.8.6 still marked as unstable / testing and 0.8.5 marked as stable
86 2014-01-27 18:47:36 <kuzetsa> I'm running on gentoo stable, only reason 0.8.6 was building is that I manually unmasked it to see if it would upgrade me from 0.8.5 to 0.8.6 with my otherwise stable system (very few exceptions from gentoo's stable branch, and none of the exceptions are core libraries or build dependancies)
87 2014-01-27 18:52:17 <kuzetsa> removed the line: [" =net-p2p/bitcoind-0.8.6 ~x86 "] from my /etc/portage/package.keywords and it reverted to the known stable (bitcoin 0.8.5) so all is well I guess
88 2014-01-27 18:52:55 <kuzetsa> that GIT_DISCOVERY_ACROSS_FILESYSTEM thing looked like potentially a red herring / false positive for the source of the error though
89 2014-01-27 18:53:48 <kuzetsa> I did some digging, and it looks like smtp suggested thought that error message could be misleading and mess up the debug process ---> https://bitcointalk.org/index.php?topic=129861.msg1453060#msg1453060
90 2014-01-27 18:54:21 <kuzetsa> the wording "fatal" implies that a condition is blocking a successful build
91 2014-01-27 18:54:40 <kuzetsa> so I assumed that it was related to my being unable to build
92 2014-01-27 18:59:35 <kuzetsa> https://gist.github.com/kuzetsa/8655171#file-gistfile1-txt-L31 <--- build log, error in question on or around line #31
93 2014-01-27 19:01:52 <sipa> kuzetsa: that's just git describe failing because you're not building from within a git repo
94 2014-01-27 19:01:59 <sipa> it shouldn't prevent building
95 2014-01-27 19:06:21 <kuzetsa> sipa: yeah well... like I said, all I did was type: "emerge bitcoind --verbose --ask" and that was the result *shrugs*
96 2014-01-27 19:06:36 <sipa> well, i have no clue what the gentoo ebuild is doing
97 2014-01-27 19:06:47 <kuzetsa> uhh
98 2014-01-27 19:06:57 <sipa> go tell the person who's doing that
99 2014-01-27 19:07:01 <kuzetsa> heh
100 2014-01-27 19:07:10 <kuzetsa> the ebuild does the exact same thing as 0.8.5 ebuild
101 2014-01-27 19:07:28 <sipa> strange, very little changed
102 2014-01-27 19:07:29 <kuzetsa> only difference is pulling the 0.8.6 tarball instead
103 2014-01-27 19:07:32 <kuzetsa> indeed
104 2014-01-27 19:10:52 <phantomcircuit> Luke-Jr, ^
105 2014-01-27 19:11:54 <kuzetsa> I have git installed on this system, but only because I use it for building windows binaries (mingw target / gentoo crossdev) for cgminer
106 2014-01-27 19:12:43 <kuzetsa> it just doesn't seem like it should be neccessary to manually use git to pull the source and build and install a package on gentoo
107 2014-01-27 19:22:09 <gjs278> kuzetsa: it isn't
108 2014-01-27 19:22:18 <gjs278> I emerge bitcoind without those errors all the time
109 2014-01-27 19:22:30 <gjs278> can't tell you what's wrong with yours though, but it should work with nothing more than emerge bitcoind
110 2014-01-27 19:22:30 <sipa> kuzetsa: git is not necessary
111 2014-01-27 19:22:48 <sipa> kuzetsa: but the build process *tries* to use git to find what version the current code is
112 2014-01-27 19:24:00 <gjs278> kuzetsa: I tried it just now and I get all the same warnings and messages you do, but my compile completed
113 2014-01-27 19:24:17 <gjs278> that fatal git error message was there as well, but it still finished fine
114 2014-01-27 19:24:54 <sipa> oh, internal compile error
115 2014-01-27 19:25:00 <sipa> buggy compiler
116 2014-01-27 19:25:02 <sipa> or perhaps OOM
117 2014-01-27 19:25:05 <gjs278> indeed
118 2014-01-27 19:25:12 <sipa> the git warning has nothing to do sith it
119 2014-01-27 19:25:31 <gjs278> gcc version 4.8.2 was able to finish the compile on gentoo here, but this is x64 not x86
120 2014-01-27 19:46:05 <kuzetsa> [14:25:30] <gjs278> gcc version 4.8.2 was able to finish the compile on gentoo here, but this is x64 not x86
121 2014-01-27 19:46:13 <kuzetsa> sorry, was AFK figuring out brunch
122 2014-01-27 19:46:22 <kuzetsa> my build system is x86
123 2014-01-27 19:46:49 <sipa> kuzetsa: how much RAM do you have free?
124 2014-01-27 19:48:06 <kuzetsa> "cached" is at: 598304
125 2014-01-27 19:48:24 <kuzetsa> free swap is at 2609480
126 2014-01-27 19:48:49 <kuzetsa> (total swap is 2621436, total physical ram is 1030068)
127 2014-01-27 19:48:55 <sipa> and how much actually free :)
128 2014-01-27 19:48:59 <kuzetsa> heh
129 2014-01-27 19:49:11 <sipa> i think your compiler just OOMed
130 2014-01-27 19:49:33 <kuzetsa> 18908 "free" and but counting buffers / cache is @ 625144
131 2014-01-27 19:50:26 <kuzetsa> it seems strange that I can compile the linux kernel itself, rebuild new versions of gcc (for new cross compile targets or otherwise) but bitcoind can't compile because OOM
132 2014-01-27 19:51:20 <sipa> heh, C++ compilation - especially with tons of stuff in headers as bitcoin does - requires orders of magnitude more memory than C compilation
133 2014-01-27 19:51:48 <kuzetsa> :(
134 2014-01-27 19:52:08 <kuzetsa> wait... gcc 4.8.2 uses c++ as the implementatino language though
135 2014-01-27 19:53:02 <sipa> all depends on how templated the code is
136 2014-01-27 19:53:04 <kuzetsa> and I can compile gcc itself just fine and the source packages and binary excecutable outputs for gcc are much larger than bitcoin's
137 2014-01-27 19:53:16 <sipa> the executable size isn't relevant
138 2014-01-27 19:53:39 <sipa> bitcoin really just isn't written for efficient compilation
139 2014-01-27 19:54:51 <phantomcircuit> sipa, quite possibly the understatement of the month
140 2014-01-27 19:59:25 <kuzetsa> hmm
141 2014-01-27 19:59:50 <sipa> my gcc processes often reach 400-600 MB
142 2014-01-27 20:00:08 <kuzetsa> so free ram + swap in excess of 2 GB isn't enough to compile bitcoin?
143 2014-01-27 20:00:34 <jgarzik> well RAM matters, not swap
144 2014-01-27 20:00:45 <kuzetsa> weird
145 2014-01-27 20:00:47 <sipa> technically, virtual memory is all that matters
146 2014-01-27 20:00:48 <jcorgan> lol. try gnuradio; until we took steps to lower it, we once had a working set of 1.7G RAM needed
147 2014-01-27 20:00:49 <jgarzik> you can get into swap death, where progress is so slow it takes days
148 2014-01-27 20:01:10 <sipa> jgarzik: yes, but it dies; it doesn't trash
149 2014-01-27 20:01:19 <jgarzik> ACTION kicks OSX XChat
150 2014-01-27 20:01:24 <kuzetsa> jgarzik: 1 gig ram, but the storage layer is SSD-backed
151 2014-01-27 20:01:30 <sipa> kuzetsa: are you building in parallel?
152 2014-01-27 20:01:46 <kuzetsa> sipa: oh, reduce the jobs to like 1 or something?
153 2014-01-27 20:02:12 <jgarzik> yeah, don't build in parallel, for one
154 2014-01-27 20:02:35 <kuzetsa> make -j12 works fine for me for most things
155 2014-01-27 20:02:38 <sipa> lol
156 2014-01-27 20:02:50 <kuzetsa> I'll reduce it to -j3
157 2014-01-27 20:02:52 <sipa> bitcoin with -j6 kills my 8 GiB RAM laptop
158 2014-01-27 20:02:57 <kuzetsa> whaaat?
159 2014-01-27 20:03:10 <kuzetsa> buh
160 2014-01-27 20:03:21 <sipa> with 1 GiB ram, i suggest -j1 or -j2 maybe
161 2014-01-27 20:03:31 <kuzetsa> I'll try -j1
162 2014-01-27 20:10:17 <kuzetsa> there, I set up an environment variable override thingy to disable makeopts -j12 in favor of -j1 for certain packages, and applied it for bitcoind since it's the first thing I've ever had trouble building at -j12
163 2014-01-27 20:13:16 <gjs278> j12 how many cores do you even have
164 2014-01-27 20:13:33 <gjs278> I have 6 cores and I only j12 if I'm not using ck patches
165 2014-01-27 20:13:41 <gjs278> but I also have 24gb of ram
166 2014-01-27 20:15:00 <jgarzik> a few other packages -- libreoffice, maybe? trying to remember from RH days -- have huge C++ codebases, and they regularly killed the standard MAKEOPTS parallel build settings the RPM build bots did
167 2014-01-27 20:15:38 <jgarzik> all these templates, N copies of the same damn algorithm
168 2014-01-27 20:15:57 <jgarzik> C++ was secretly funded by DRAM manufacturers, I'm convinced
169 2014-01-27 20:16:07 <jcorgan> lol
170 2014-01-27 20:17:01 <phantomcircuit> jgarzik, swap matters if you're using -j
171 2014-01-27 20:18:06 <phantomcircuit> kuzetsa, you'll get a much better improvement by using ccache
172 2014-01-27 20:18:07 <kuzetsa> gjs278: the system I use for buidling has 2x intel xeon E5-2670 (so 16 physical cores, 32 threads since hyperthreading really does do what it was designed for and improve concurrency & utilization)
173 2014-01-27 20:18:27 <gjs278> but it only has 2gb of ram?
174 2014-01-27 20:18:56 <sipa> ACTION guesses: a VM
175 2014-01-27 20:19:06 <gjs278> yeah that's what I thought when I read 2670
176 2014-01-27 20:19:33 <kuzetsa> the xen instance I'm using for compiling only has 1 gig ram, but the storage array is SSD backed & most disk IO actually hits the ram on the dom0 hypervisor instead of physical disk anyway when I use swap
177 2014-01-27 20:20:05 <gjs278> you can't "share" memory between xen instances with pure ram?
178 2014-01-27 20:20:09 <sipa> well, -j12 will certainly need more than a few GB for compiling
179 2014-01-27 20:20:20 <kuzetsa> heh
180 2014-01-27 20:20:27 <kuzetsa> whatever
181 2014-01-27 20:20:43 <gjs278> if your swap is high enough I think it would work
182 2014-01-27 20:20:44 <kuzetsa> I'm cpu bottlenecked / limited not ram most of the time no matter what I build
183 2014-01-27 20:20:54 <kuzetsa> bitcoin is the only thing I ever had issues with before today
184 2014-01-27 20:21:10 <kuzetsa> (it built successfully now BTW, thanks for suggesting -j1 for debugging my issue)
185 2014-01-27 20:21:19 <gjs278> oh nice
186 2014-01-27 20:21:24 <sipa> good to hear
187 2014-01-27 20:21:50 <kuzetsa> getinfo now says "version" : 80600
188 2014-01-27 20:21:59 <kuzetsa> I'm assuming that's right :)
189 2014-01-27 20:22:29 <gjs278> it is
190 2014-01-27 20:27:14 <kuzetsa> http://libusbx.org/ <--- (quote) "...As of 2014.01.26, this project has been fully merged back into libusb and is being discontinued."
191 2014-01-27 20:27:36 <kuzetsa> err... nevermind that, it was meant for a different channel (related to development of mining stuff)
192 2014-01-27 20:30:54 <kuzetsa> WRT the "getinfo" thing I just did, I still find it frustrating that ( "subver" : "/Satoshi:0.8.6/", ) shows up on getpeerinfo on a remote, attached node (and is human readable) but getinfo only reports the less nicely formatted ( "version" : 80600 )
193 2014-01-27 20:31:25 <kuzetsa> (I felt the need just now to test using getpeerinfo from a remote node as a sanity test)
194 2014-01-27 20:32:13 <kuzetsa> waaaait a sec
195 2014-01-27 20:32:16 <kuzetsa> this is a mismatch
196 2014-01-27 20:32:27 <kuzetsa> getinfo on the local bitcoind said 80600
197 2014-01-27 20:32:43 <kuzetsa> but getpeerinfo to check remotely said ( "version" : 70001, )
198 2014-01-27 20:32:57 <sipa> that's the protocol version
199 2014-01-27 20:33:00 <sipa> not the client version
200 2014-01-27 20:33:08 <kuzetsa> ...
201 2014-01-27 20:33:12 <sipa> since somewhere 0.6 or so they are not linked anymore
202 2014-01-27 20:33:24 <sipa> getinfo should also report protocolversion 70001
203 2014-01-27 20:33:50 <kuzetsa> ah, so it does
204 2014-01-27 20:33:55 <kuzetsa> thanks for clearing that up
205 2014-01-27 20:36:10 <kuzetsa> ACTION nudges some used chewing gum out of the way and sticks some more duct tape and twine on the local notes about bitcoin's internal structure and codebase to try and tidy things up a bit, and accidentally misplaces and drops in last week's obsolete grocery list and a few copper coins
206 2014-01-27 20:36:25 <kuzetsa> (I can't keep track of it all anymore)
207 2014-01-27 20:37:37 <maaku> current bitcoin source tree is pretty clean imho
208 2014-01-27 20:37:45 <maaku> didn't used to be
209 2014-01-27 20:40:37 <sipa> main/net are still a mess with unclear abstractions and responsibilities
210 2014-01-27 20:41:47 <jcorgan> it's better, but there are well-known code organization practices that could make things cleaner and faster to compile, yet i'm afraid it would be considered "bike shedding"
211 2014-01-27 20:42:18 <gavinandresen> every change has risk associated, if only the risk that the time it takes us to code review means we don't do something more important
212 2014-01-27 20:42:36 <gavinandresen> ⦠so do cleanups as you FIX something, please, don't cleanup for cleanups sake
213 2014-01-27 20:43:10 <jgarzik> current codebase is a _lot_ better than it used to be
214 2014-01-27 20:43:17 <jcorgan> agree
215 2014-01-27 20:43:21 <jgarzik> sipa, agree
216 2014-01-27 20:44:37 <sipa> well, i'm sure we all remember the time when wallet/protocol/core/keystore/mining/version/alert/addrman/txmempool were inside main...
217 2014-01-27 20:45:11 <sipa> then again, i'm not sure main is actually shorter in number of lines of code than then...
218 2014-01-27 20:45:18 <gavinandresen> lol
219 2014-01-27 20:51:41 <sipa> ok, did a build from scratch while monitoring the size of the forked cc1plus processes
220 2014-01-27 20:51:50 <sipa> a total of 108 (?!) were run
221 2014-01-27 20:52:27 <sipa> and they require over 4 GiB of RAM all together
222 2014-01-27 20:52:30 <phantomcircuit> gavinandresen, right which is why i haven't pushed for fixing IsConfirmed
223 2014-01-27 20:52:43 <phantomcircuit> afaict there is some behaviour that only works correctly when it's broken
224 2014-01-27 20:52:43 <sipa> phantomcircuit: i really want to get vtxPrev fixed
225 2014-01-27 20:53:05 <phantomcircuit> sipa, i can submit a patch that removes vtxPrev without trying to fix the IsConfirmed logic
226 2014-01-27 20:53:39 <phantomcircuit> im just not sure if there is anything else that works because that doesn't
227 2014-01-27 20:53:44 <sipa> i am unconvinced there is a problem with it
228 2014-01-27 20:54:08 <phantomcircuit> the behavior of the getbalance rpc call is bizarre with edge cases
229 2014-01-27 20:54:15 <sipa> it is
230 2014-01-27 20:54:20 <phantomcircuit> it can actually display a negative balance if you send payments to yourself
231 2014-01-27 20:56:16 <sipa> phantomcircuit: given that the only things that *can* end up in vtxPrev, are things that are in your own wallet
232 2014-01-27 20:56:45 <phantomcircuit> sipa, right
233 2014-01-27 20:56:46 <sipa> would it make sense to keep the current processing in IsConfirmed, but have it pull from mapWallet instead of from vtxPrev?
234 2014-01-27 20:57:00 <sipa> i am unconvinced it makes a difference
235 2014-01-27 20:57:03 <phantomcircuit> that's actually what my patch currently does
236 2014-01-27 20:57:54 <sipa> oh!
237 2014-01-27 20:57:55 <phantomcircuit> sipa, https://github.com/bitcoin/bitcoin/pull/2966
238 2014-01-27 20:59:32 <phantomcircuit> sipa, it also fixes the behavior of relaytransaction
239 2014-01-27 21:00:00 <sipa> ewww, a CWalletTx in main
240 2014-01-27 21:00:07 <sipa> that looks wrong
241 2014-01-27 21:00:29 <phantomcircuit> sipa, yeah im sure it is
242 2014-01-27 21:05:47 <phantomcircuit> sipa, the ListUnconfirmedSupportingTransactions function could be improved significantly by pulling from the mempool
243 2014-01-27 21:06:00 <phantomcircuit> it would then be a fairly complete snapshot of what the client knows
244 2014-01-27 21:06:28 <phantomcircuit> either way the code needs to be reviewed and probably CWalletTx parts of it moved into the proper file
245 2014-01-27 21:06:43 <phantomcircuit> and i just dont have time to do that
246 2014-01-27 21:09:43 <sipa> phantomcircuit: as only wallet transactions can enter vtxPrev, and wallet transactions are never forgotten, just replacing every lookup of vtxPrev by finding inputs in the wallet itself, should have 0 impact
247 2014-01-27 21:09:46 <gfawkes__> so now that man-in-the-middle attacks have been proven to be viable at least by the state level, is there now any importance on fixing that for bitcoin? probably not.
248 2014-01-27 21:10:01 <sipa> gfawkes__: what MitM are you talking about?
249 2014-01-27 21:10:17 <sipa> gfawkes__: bitcoin p2p is not encrypted or authenticated, there is nothing to mitm
250 2014-01-27 21:10:28 <jgarzik> gfawkes__, Don't be ridiculous. Users are targeted far more often by non-state-level bad guys.
251 2014-01-27 21:10:46 <phantomcircuit> sipa, right, but IsConfirmed's insane fractal whatever run time effectively limits how screwed up getbalance can be relative to outputs
252 2014-01-27 21:10:56 <phantomcircuit> it's still wrong but not quite as bad
253 2014-01-27 21:11:06 <sipa> phantomcircuit: adding the mempool into view would improve certain decisions perhaps, but it shouldn't be necessary to retain current behaviour
254 2014-01-27 21:11:25 <gfawkes__> i privately informed folks back in 2011, id prefer not to publicly announce it, and no there have been no release notes about it
255 2014-01-27 21:11:42 <phantomcircuit> sipa, im not sure if there is anything else where the logic breaks if IsConfirmed starts working properly for unconfirmed txs
256 2014-01-27 21:11:49 <phantomcircuit> as it stands it basically never works
257 2014-01-27 21:12:27 <gavinandresen> gfawkes__: we have no idea what you're talking about, unless you're talking about MITM'ing bitcoin addresses sent over insecure channels like email
258 2014-01-27 21:12:44 <gavinandresen> gfawkes__: ⦠in which case the payment protocol will fix those attacks.
259 2014-01-27 21:13:22 <sipa> phantomcircuit: IsConfirmed can already return true for unconfirmed transactions, namely if they are from yourself
260 2014-01-27 21:14:09 <phantomcircuit> sipa, right but the runtime performance for IsConfirmed with unconfirmed transactions of any complexity is so attrocious that i doubt that happens very frequently
261 2014-01-27 21:14:23 <phantomcircuit> the getbalance logic only breaks when you're spending your own unconfirmed transactions
262 2014-01-27 21:14:38 <phantomcircuit> which is exactly the case where IsConfirmed takes approximately forever to run
263 2014-01-27 21:14:53 <sipa> phantomcircuit: plan: 1) remove vtxPrev and replace it with wallet lookups (just save memory and wallet size)
264 2014-01-27 21:15:28 <sipa> phantomcircuit: 2) make IsConfirmed use a cache so every transaction is at most evaluated once
265 2014-01-27 21:15:44 <phantomcircuit> sipa, i believe it would be safer to remove the unconfirmed tx logic in IsConfirmed
266 2014-01-27 21:15:47 <gfawkes__> anyway, thats the last time i report it privately, we'll see what happens
267 2014-01-27 21:15:59 <phantomcircuit> then we could add it back as a separate pull request
268 2014-01-27 21:17:01 <sipa> phantomcircuit: 3) (potentially) make it use mempool data to form a better conclusion
269 2014-01-27 21:17:30 <sipa> phantomcircuit: being able to spend your own unconfirmed balance is a pretty important feature
270 2014-01-27 21:19:25 <phantomcircuit> sipa, right, but is that dependent on IsConfirmed working?
271 2014-01-27 21:19:37 <sipa> i believe so
272 2014-01-27 21:35:56 <Subo1977> wieviel connections habt iihr bei euren hidden service?
273 2014-01-27 21:36:25 <sipa> english please
274 2014-01-27 21:36:54 <sipa> and it seems i have 85 (WTF??) onion connections
275 2014-01-27 21:40:37 <Subo1977> sorry sipa. oh i got only 20 after 24 hours
276 2014-01-27 21:41:11 <Subo1977> i would post the question in -de queue
277 2014-01-27 22:39:30 <Luke-Jr> kuzetsa: the build error there is a GCC crash. broken GCC is not something we can deal with here
278 2014-01-27 22:39:47 <Luke-Jr> i686-pc-linux-gnu-g++: internal compiler error: Killed (program cc1plus)
279 2014-01-27 22:39:54 <Luke-Jr> otoh, that might just be OOM killer too
280 2014-01-27 22:39:58 <gjs278> it was
281 2014-01-27 22:40:06 <Luke-Jr> k
282 2014-01-27 22:40:07 <gjs278> he solved it by lowering his jobs
283 2014-01-27 22:40:15 <Ry4an> heh
284 2014-01-27 23:00:37 <phantomcircuit> sipa, hmm you're right
285 2014-01-27 23:00:46 <phantomcircuit> ok well in that case my current solution is optimal
286 2014-01-27 23:00:57 <phantomcircuit> getbalance should be fixed with another pull request
287 2014-01-27 23:01:10 <sipa> phantomcircuit: mind if i try implementing it?
288 2014-01-27 23:04:37 <sipa> phantomcircuit: just getting rid of vtxPrev seems very simple
289 2014-01-27 23:05:38 <phantomcircuit> sipa, no of course not
290 2014-01-27 23:05:41 <phantomcircuit> have at it
291 2014-01-27 23:06:46 <sipa> there are 3 places where vtxPrev is used: IsConfirmed, Relaying and Reaccepting
292 2014-01-27 23:07:48 <phantomcircuit> sipa, reaccepting?
293 2014-01-27 23:09:04 <sipa> phantomcircuit: push missing wallet transactions to mempool
294 2014-01-27 23:09:26 <phantomcircuit> CWalletTx::AcceptWalletTransaction ?
295 2014-01-27 23:09:27 <sipa> and finding spent coins
296 2014-01-27 23:09:40 <sipa> yeah, it's only called from ReacceptWalletTransactions
297 2014-01-27 23:10:12 <Luke-Jr> ACTION was pleasantly surprised to see CodeShark and BlueMatt in Miami. :>
298 2014-01-27 23:10:18 <Luke-Jr> phantomcircuit: where were youâ :P
299 2014-01-27 23:11:26 <phantomcircuit> Luke-Jr, working :(
300 2014-01-27 23:11:38 <Luke-Jr> phantomcircuit: all your coworkers made it <.<
301 2014-01-27 23:11:48 <phantomcircuit> yeah they dont actually do any real work
302 2014-01-27 23:11:52 <phantomcircuit> (i didn't say that)
303 2014-01-27 23:12:04 <Luke-Jr> lol
304 2014-01-27 23:19:54 <sipa> phantomcircuit: ummm... this code has not worked since 0.8.0
305 2014-01-27 23:19:59 <sipa> typo by me
306 2014-01-27 23:20:33 <phantomcircuit> sipa, you mean the patch i have?
307 2014-01-27 23:20:34 <sipa> if (!mempool.exists(hash) && pcoinsTip->HaveCoins(hash)) tx.AcceptToMemoryPool(false);
308 2014-01-27 23:20:40 <sipa> no, current master
309 2014-01-27 23:20:44 <phantomcircuit> oh
310 2014-01-27 23:20:49 <phantomcircuit> where is that?
311 2014-01-27 23:21:05 <sipa> so if a transaction is NOT in the memory, but IS in the blockchain, then we attempt to add it to the memory pool...
312 2014-01-27 23:21:14 <phantomcircuit> lol
313 2014-01-27 23:21:16 <sipa> 0.7.2 had an extra !
314 2014-01-27 23:22:15 <sipa> ACTION doubts: delete or fix
315 2014-01-27 23:22:29 <phantomcircuit> 722fa283 (Pieter Wuille 2013-10-19 18:42:14 +0200 807) if (!mempool.exists(hash) && pcoinsTip->HaveCoins(hash))
316 2014-01-27 23:22:32 <phantomcircuit> :X
317 2014-01-27 23:22:42 <sipa> 00:19:57 < sipa> typo by me
318 2014-01-27 23:23:29 <phantomcircuit> i believe my current patch handles that
319 2014-01-27 23:23:37 <phantomcircuit> it gets a list of the unconfirmed supporting transaction
320 2014-01-27 23:23:40 <phantomcircuit> and tries to add those
321 2014-01-27 23:24:09 <phantomcircuit> ListUnconfirmedSupportingTransactions is actually fairly useful in a bunch of places
322 2014-01-27 23:24:18 <sipa> yup, those 3