text:
Bitcoin Address Workflow on Spend Leftovers
I need to understand the correct Bitcoin address workflow for transactions. I created an address a while back with a common wallet and sent some coin there. The address is at segwit m/49'/0'/0'/0/0. I then spent some coin from that. When I looked up the address on the blockchain, I noticed that the balance was 0. The transaction that occurred had two output addresses. The funds were sent to two addresses. The second is the non-segwit address of the person of who I sent the funds to. The first was segwit m/49'/0'/0'/1/0 (address from my wallet). It appears that the wallet moved my funds to a new address in my wallet. I.e. m/49'/0'/0'/0/0 -> m/49'/0'/0'/1/0.
Should a wallet always move the left over from m/49'/0'/0'/0/x to m/49'/0'/0'/x/0? Is this logic standard practice? Or is this specific to the wallet I used? Is there a recommended practice? I need to know because I am developing a wallet, and I need to follow standard practice.
What should happen when I transfer the funds from m/49'/0'/0'/x/0 to another address, and there are leftovers? Where should the new address for the leftovers be moved to? m/49'/0'/x'/0/0?

It appears that the wallet moved my funds to a new address in my wallet.

That is normal and desirable behavior.  See How does change work in a bitcoin transaction?

Should a wallet always move the left over from m/49'/0'/0'/0/x to m/49'/0'/0'/x/0?

This wallet is using BIP 49, which extends the hierarchical wallet organization described in BIP 44.  General standards for hierarchical deterministic wallets are described in BIP 32.  You'll want to read these carefully if you are implementing your own wallet.
Under BIP 44, the fourth level of the derivation path is to be 0 for "external" keypairs (those presented to the user as a receive address which she may send to a payer), and 1 for "internal" keypairs (to be used as change addresses).

Constant 0 is used for external chain and constant 1 for internal chain (also known as change addresses). External chain is used for addresses that are meant to be visible outside of the wallet (e.g. for receiving payments). Internal chain is used for addresses which are not meant to be visible outside of the wallet and is used for return transaction change.

BIP 32 specifies something similar:

Each account is composed of two keypair chains: an internal and an external one. The external keychain is used to generate new public addresses, while the internal keychain is used for all other operations (change addresses, generation addresses, ..., anything that doesn't need to be communicated). Clients that do not support separate keychains for these should use the external one for everything.

So it is in keeping with this standard that an external keychain address .../0/x was used to receive your coins initially, while the internal keychain address .../1/y was used for the change from your outgoing transaction.  (It's just a coincidence that x and y were both 0 in this case; it would normally just be the next unused value on each chain, but since you hadn't done anything else with the wallet, you got the first keypair on each chain.)

Thanks. I'll have a read through this and check to see if it answers my questions completely.

It seems that the spec covers a lot of the cases, but not necessarily all cases. It seems that there are some cases that the spec does not dictate best practice. And, aside from that, a wallet app is able to do whatever it wants really. It could transact funds in to wherever it wants. This seems like a general problem, but it's hard to say whether this will cause issues down the line or not. I can only try to follow the spec the best I can.

I think this is mostly correct, but the wallet implements BIP0044, and BIP0049 so BIP0032 doesn't seem to be enough to answer the full question at this point.

Ah, yes, that's a better reference in this case.  Thanks.

, if you modify your answer, I will mark it as the answer.

Particularly this point: https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki#account-discovery

@MelbourneDeveloper: See what you think now.

Awesome. Good work. I think it's settled my issue and I can move forward with the functionality.


text:
What are conflicting transactions? Does it mean BTC lost permanently?
An online store recently received a payment via a wallet-to-wallet BTC payment. But after a while the total_received value went down to zero on blockchain.info. Now the address cannot be sweeped as there is no balance so the merchant assumed that the money is lost/stolen.
Tradeblock seems to have more details for this transaction:
https://tradeblock.com/bitcoin/address/1L1MSa36vaSeo8myBcGfGA3UddGDitZXai
Here's another similar one "suspected" from the same user:
https://tradeblock.com/bitcoin/address/1KsVJ1ySWVmVD5uci4mSuT98Uj65VpEzD
(icon at the rightmost column of each transaction has more details)
Can someone pls explain the anatomy of this - how to interpret and how to rectify (if at all possible).

Conflicting transactions are simply two or more transactions that spent the same UTXO in its inputs (i.e. doublespending).
The only remedy is to not treat a transaction as successful until it has a number of confirmation that satisfies you.
It is impossible to recover from a successful doublespend attack should the conflicting transaction be included in the blockchain.

Looking a the inputs and outputs, does it look like the transaction was initiated with zero fees? If so it seems strange as most wallets commonly do not initiate transactions with low/no fees.

I don't see how does that help. The point is you can create transactions that pay zero or more fees. Whether or not they confirm is another matter.

Thanks for the responses. Due to mostly smaller sized transactions the merchant cannot wait for confirmations. Given this primary constraint I'm trying to find out how to take precautions to minimize double spend attacks. That's why I was trying to identify some parameters that can be used for defence - e.g. not accepting zero fee Txs. A reasonable fee having a high chance of being propagated quickly (I understand that the same inputs can be sent in two txs one with higher fees)

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
Why is randomness needed in Proof-of-Work?
In Proof-of-Work (PoW) mining, a miner that contributes 10% of the network hash rate will on average receive 10% of the mining rewards. "On average" means that 10% of blocks are mined by this miner, and 90% of blocks are mined by other miners.
In practice, basically all miners join mining pools which removes this randomness, i.e., pools pay miners deterministically based on the hash rate they supply.
My question is: Why is randomness needed in the PoW protocol? Specifically, if a miner supplies 10% of the network hash rate, why cannot this miner simply receive 10% of the mining reward for each block?

How would you calculate what percentage of hash rate the miner supplies and who would do the calculation?

I see. The entire network would basically have to operate as a large mining pool, estimating the hash rate of miners based on "shares".

This is arguably what systems like p2pool used to do: it was its own lower-work blockchain system whose "blocks" were shares of a global, distributed mining pool, which then distributed winnings proportional to the shares. This works, but it doesn't avoid the need for randomness/variance - it just does so by substituting in another blockchain with more rapid blocks in order to compute the relative hashrate of each participant.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Compiling Bitcoin Core v0.10.0 on Ubuntu 20.04
I'm trying to compile Bitcoin Core v0.10.0 on Ubuntu 20.04, but then I get the following error:

rpcserver.cpp: In function ‘void StartRPCThreads()’:
rpcserver.cpp:594:77: error: no matching function for call to ‘boost::asio::ssl::context::context(boost::asio::io_service&, boost::asio::ssl::context_base::method)’
  594 |     rpc_ssl_context = new ssl::context(*rpc_io_service, ssl::context::sslv23);
      |                                                                             ^
In file included from /usr/include/boost/asio/ssl/context.hpp:757,
                 from /usr/include/boost/asio/ssl.hpp:18,
                 from rpcprotocol.h:16,
                 from rpcserver.h:10,
                 from rpcserver.cpp:6:
/usr/include/boost/asio/ssl/impl/context.ipp:371:1: note: candidate: ‘boost::asio::ssl::context::context(boost::asio::ssl::context&&)’
  371 | context::context(context&& other)
      | ^~~~~~~
/usr/include/boost/asio/ssl/impl/context.ipp:371:1: note:   candidate expects 1 argument, 2 provided
/usr/include/boost/asio/ssl/impl/context.ipp:63:1: note: candidate: ‘boost::asio::ssl::context::context(boost::asio::ssl::context_base::method)’
   63 | context::context(context::method m)
      | ^~~~~~~
/usr/include/boost/asio/ssl/impl/context.ipp:63:1: note:   candidate expects 1 argument, 2 provided
rpcserver.cpp:613:50: error: ‘class boost::asio::ssl::context’ has no member named ‘impl’
  613 |         SSL_CTX_set_cipher_list(rpc_ssl_context->impl(), strCiphers.c_str());
      |                                                  ^~~~
rpcserver.cpp: In instantiation of ‘void RPCListen(boost::shared_ptr >, boost::asio::ssl::context&, bool) [with Protocol = boost::asio::ip::tcp; SocketAcceptorService = boost::asio::executor]’:
rpcserver.cpp:673:58:   required from here
rpcserver.cpp:482:111: error: ‘class boost::asio::basic_socket_acceptor’ has no member named ‘get_io_service’
  482 |     boost::shared_ptr > conn(new AcceptedConnectionImpl(acceptor->get_io_service(), context, fUseSSL));
      |                                                                                ~~~~~~~~~~^~~~~~~~~~~~~~
In file included from primitives/block.h:9,
                 from chainparams.h:11,
                 from base58.h:17,
                 from rpcserver.cpp:8:
./primitives/transaction.h: In instantiation of ‘static _OI std::__copy_move::__copy_m(_II, _II, _OI) [with _II = const CTxOut*; _OI = CTxOut*]’:
/usr/include/c++/9/bits/stl_algobase.h:404:30:   required from ‘_OI std::__copy_move_a(_II, _II, _OI) [with bool _IsMove = false; _II = const CTxOut*; _OI = CTxOut*]’
/usr/include/c++/9/bits/stl_algobase.h:441:30:   required from ‘_OI std::__copy_move_a2(_II, _II, _OI) [with bool _IsMove = false; _II = __gnu_cxx::__normal_iterator >; _OI = __gnu_cxx::__normal_iterator >]’
/usr/include/c++/9/bits/stl_algobase.h:474:7:   required from ‘_OI std::copy(_II, _II, _OI) [with _II = __gnu_cxx::__normal_iterator >; _OI = __gnu_cxx::__normal_iterator >]’
/usr/include/c++/9/bits/vector.tcc:238:31:   required from ‘std::vector& std::vector::operator=(const std::vector&) [with _Tp = CTxOut; _Alloc = std::allocator]’
coins.h:90:19:   required from here
./primitives/transaction.h:103:7: warning: implicitly-declared ‘CScript& CScript::operator=(const CScript&)’ is deprecated [-Wdeprecated-copy]
  103 | class CTxOut
      |       ^~~~~~
In file included from ./primitives/transaction.h:10,
                 from primitives/block.h:9,
                 from chainparams.h:11,
                 from base58.h:17,
                 from rpcserver.cpp:8:
./script/script.h:365:5: note: because ‘CScript’ has user-provided ‘CScript::CScript(const CScript&)’
  365 |     CScript(const CScript& b) : std::vector(b.begin(), b.end()) { }
      |     ^~~~~~~
In file included from /usr/include/c++/9/algorithm:61,
                 from serialize.h:9,
                 from amount.h:9,
                 from rpcserver.h:9,
                 from rpcserver.cpp:6:
/usr/include/c++/9/bits/stl_algobase.h:342:18: note: synthesized method ‘CTxOut& CTxOut::operator=(const CTxOut&)’ first required here
  342 |        *__result = *__first;
      |        ~~~~~~~~~~^~~~~~~~~~
In file included from chain.h:11,
                 from main.h:14,
                 from rpcserver.cpp:10:
tinyformat.h: In static member function ‘static const char* tinyformat::detail::FormatIterator::streamStateFromFormat(std::ostream&, unsigned int&, const char*, int, int)’:
tinyformat.h:785:21: warning: this statement may fall through [-Wimplicit-fallthrough=]
  785 |             out.setf(std::ios::uppercase);
      |             ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
tinyformat.h:786:9: note: here
  786 |         case 'x': case 'p':
      |         ^~~~
tinyformat.h:791:21: warning: this statement may fall through [-Wimplicit-fallthrough=]
  791 |             out.setf(std::ios::uppercase);
      |             ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
tinyformat.h:792:9: note: here
  792 |         case 'e':
      |         ^~~~
tinyformat.h:797:21: warning: this statement may fall through [-Wimplicit-fallthrough=]
  797 |             out.setf(std::ios::uppercase);
      |             ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
tinyformat.h:798:9: note: here
  798 |         case 'f':
      |         ^~~~
tinyformat.h:802:21: warning: this statement may fall through [-Wimplicit-fallthrough=]
  802 |             out.setf(std::ios::uppercase);
      |             ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
tinyformat.h:803:9: note: here
  803 |         case 'g':
      |         ^~~~
make[2]: *** [Makefile:3910: libbitcoin_server_a-rpcserver.o] Error 1
make[2]: Leaving directory '/usr/local/src/bitcoin-0.10.0/src'
make[1]: *** [Makefile:6475: all-recursive] Error 1
make[1]: Leaving directory '/usr/local/src/bitcoin-0.10.0/src'
make: *** [Makefile:569: all-recursive] Error 1

I followed the instructions on the build-unix.md file, and I believe the problem is related to the required dependencies, especially the libboost-all-dev package. Interestingly, I was able to compile every released version from v0.20.1 to 0.16.0, but nothing older than that, using the same instructions on build-unix.md. If the problem really is libboost-all-dev , how can I solve it?
Update:
I've installed boost version 1.57 as Pieter suggested with:

tar --bzip2 -xf /path/to/boost_1_57_0.tar.bz2
cd boost_1_57_0/
./bootstrap.sh --prefix=/usr/
./b2
sudo ./b2 install

And then, on the bitcoin root, used:

./configure --with-boost-liddir=LIB_DIR

But as result I got:

configure: error: Could not find a version of the boost_system library!

And I've tried different paths but always got the same result. Is this just path problem or is there something else?
Update 2:
Apparently I fixed the path with:

./configure --with-boost-libdir=/usr/lib/

But when I run make I got this error.
What is the problem?

You may need an older version of Boost.

Do you know which version? Or even if is there any place where I can find information about dependencie's version for specific bitcoin core's version?

For recent versions that information is in doc/dependencies.md, but that didn't exist at the time of 0.10.0. I'd just start with whatever version of Boost was most recent at the time. Bitcoin Core 0.10.0 was released in Feb 2015, so maybe 1.57 or a bit earlier?

Another aspect: Bitcoin Core was not using C++11 until 0.13.0, but your compilation output suggests you are using C++11 here. With a modern compiler you may want to explicitly disable that (-std=c++98).

Thanks Pieter. I will try it and update with the results.

The two points raised in the comments are necessary to compile bitcoin core v0.10.0. First the best way to install Boost libraries in unix is described in build_unix.md, where in the Boost section we have

sudo su
./bootstrap.sh
./bjam install

but for v0.10.0 we need at least version 1.57 of Boost that can be found here.
We will also need older versions of OpenSSL libraries, in this case I needed the packages libssl-dev version 1.0.1f, libssl1.0.0 version 1.0.1f and multiarch-support version 2.19.
After installing the packages we can compile bitcoin core using

./autogen.sh
./configure --without-gui CXX="g++ -std=c++98"
make
make install

The flags in configure follow the comments for the C++ version and the absence of GUI is due to the fact that v0.10.0 will request Qt 4 packages for graphical interface which I couldn't find. Once those step are followed bitcoin core will be successfully compiled.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Compiling a statically linked binary for bitcoin core v0.12.0
I'm trying to compile a statically linked binary following https://github.com/bitcoin/bitcoin/tree/master/depends. But when I run make I got this error:

fcmatch.c:284:63: error: 'PRI_CHAR_WIDTH_STRONG' undeclared here (not in a function); did you mean 'PRI_CHAR_HEIGHT_STRONG'?
  284 | #define FC_OBJECT(NAME, Type, Cmp) { FC_##NAME##_OBJECT, Cmp, PRI_##NAME##_STRONG, PRI_##NAME##_WEAK },
      |                                                               ^~~~
fcobjs.h:54:1: note: in expansion of macro 'FC_OBJECT'
   54 | FC_OBJECT (CHAR_WIDTH,  FcTypeInteger, NULL)
      | ^~~~~~~~~
fcmatch.c:284:84: error: 'PRI_CHAR_WIDTH_WEAK' undeclared here (not in a function); did you mean 'PRI_CHAR_HEIGHT_WEAK'?
  284 | #define FC_OBJECT(NAME, Type, Cmp) { FC_##NAME##_OBJECT, Cmp, PRI_##NAME##_STRONG, PRI_##NAME##_WEAK },
      |                                                                                ^~~~
fcobjs.h:54:1: note: in expansion of macro 'FC_OBJECT'
   54 | FC_OBJECT (CHAR_WIDTH,  FcTypeInteger, NULL)
      | ^~~~~~~~~
make[4]: *** [Makefile:613: fcmatch.lo] Error 1
make[4]: Leaving directory '/usr/local/src/bitcoin-0.12.0/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.11.1-14f43f3a52a/src'
make[3]: *** [Makefile:474: all] Error 2
make[3]: Leaving directory '/usr/local/src/bitcoin-0.12.0/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.11.1-14f43f3a52a/src'
make[2]: *** [Makefile:561: all-recursive] Error 1
make[2]: Leaving directory '/usr/local/src/bitcoin-0.12.0/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.11.1-14f43f3a52a'
make[1]: *** [Makefile:444: all] Error 2
make[1]: Leaving directory '/usr/local/src/bitcoin-0.12.0/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.11.1-14f43f3a52a'
make: *** [funcs.mk:240: /usr/local/src/bitcoin-0.12.0/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.11.1-14f43f3a52a/./.stamp_built] Error 2

I'm doing this on ubuntu 20.04 LTS using the source code of bitcoin core v0.12.0. I already compiled the source code and because it is an old version it was necessary to install some old libraries. But I believe the problem is related to the C++ version and I don't know how to set the right C++ version. How can I fix that?

Apply these two commits in order to update fontconfig.
https://github.com/bitcoin/bitcoin/commit/86d410d91b2188405a56bba5f1a1f74694be886a#diff-7cec5704a9b80e0db0bdc569395a7e8f2584356138b37b6a70ff30ba38224516
https://github.com/bitcoin/bitcoin/commit/3498a8d018fa02de17e1f9c4cda49f4c7edb3c6c#diff-7cec5704a9b80e0db0bdc569395a7e8f2584356138b37b6a70ff30ba38224516


text:
My Bitcoin Qt is behind 7 years and 32 weeks
My Bitcoin Qt is behind 7 years and 32 weeks... I'm new to all this so I wonder:

if I shut down the pc, is the sync gonna continue or will it restart from the beginning?
And.. how can I get my transaction without waiting for 7 years?
and also, does the sync need bigger ssd storage?

If you shut down your PC and then restart, the Bitcoin sync will pick up where it left off.
The latest transaction your Bitcoin currently knows about is 7 years old, but it won't take 7 years to catch up. It should be all caught up in a matter of days.
You will need a lot of storage for Bitcoin Qt. The current size of the blockchain is over 80 GB, and grows at about 4 GB per month. The blockchain is always growing so you will continue to need more space indefinitely.
If you want to use Bitcoin but don't want to store the entire blockchain forever, consider using an SPV client. See the question What is an SPV client? for more information.

many thanks greg, if it really need bigger storage then can i just import my backup wallet from bitcoin core to another wallet .. btw .. i still behind 5 yrs and 8 weeks to sync.  and i really need to use some BTC from my core wallet . so.. is there any easy way to export my wallet.dat so that i can use some of my BTC to pay my debt as soon as possible.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How to create raw transaction
I've started creating my own wallet in bitcoinj and I have serious problem. I want to create raw transaction without wallet object. So I wrote this piece of code:
    //String to a private key
    DumpedPrivateKey dumpedPrivateKey = DumpedPrivateKey.fromBase58(params,
            privKey);
    ECKey key = dumpedPrivateKey.getKey();

    //String to an address
    Address address2 = Address.fromBase58(params, address);

    Transaction tx = new Transaction(params);
    //value is a sum of all inputs, fee is 4013
    tx.addOutput(Coin.valueOf(amount-4013), address2);

    //utxos is an array of inputs from my wallet
    for(UTXO utxo : utxos)
    {
        TransactionOutPoint outPoint = new TransactionOutPoint(params, utxo.getIndex(), utxo.getHash());
        tx.addSignedInput(outPoint, utxo.getScript(), key);
    }

    tx.getConfidence().setSource(TransactionConfidence.Source.SELF);
    tx.setPurpose(Transaction.Purpose.USER_PAYMENT);

    System.out.println(tx.getHashAsString());
    b_peerGroup.GetPeerGroup().broadcastTransaction(tx);

But this ends with:

mandatory-script-verify-flag-failed (Script evaluated without error but finished with a false/empty top stack e

What is wrong with my code?
EDIT:
I found a solution. I've changed only this line: tx.addSignedInput(outPoint, utxo.getScript(), key, Transaction.SigHash.ALL, true);

http://bitcoin.stackexchange.com/questions/47928/how-can-i-create-a-raw-transaction-in-hex-format-with-bitcoinj This might help?

Unfortunately I've already used this code :(

I believe this means that you provided the right public key, but the signature does not match the transaction. Is the transaction being modified between when it is signed and when it is broadcast?

Already found the solution :)
All is under edit

Could you please describe your solution in an answer post rather instead of putting it in the question? Thanks.

Sure :)
My solution was pretty simple:
//String to a private key
DumpedPrivateKey dumpedPrivateKey = DumpedPrivateKey.fromBase58(params,
        privKey);
ECKey key = dumpedPrivateKey.getKey();

//String to an address
Address address2 = Address.fromBase58(params, address);

Transaction tx = new Transaction(params);
//value is a sum of all inputs, fee is 4013
tx.addOutput(Coin.valueOf(amount-4013), address2);

//utxos is an array of inputs from my wallet
for(UTXO utxo : utxos)
{
    TransactionOutPoint outPoint = new TransactionOutPoint(params, utxo.getIndex(), utxo.getHash());
    //YOU HAVE TO CHANGE THIS
    tx.addSignedInput(outPoint, utxo.getScript(), key, Transaction.SigHash.ALL, true);
}

tx.getConfidence().setSource(TransactionConfidence.Source.SELF);
tx.setPurpose(Transaction.Purpose.USER_PAYMENT);

System.out.println(tx.getHashAsString());
b_peerGroup.GetPeerGroup().broadcastTransaction(tx);

How you create UTXO?

Pawan, you need to fetch it before by listunspent api request


text:
When The Unlocking Script Is Created, Who Verifies The Script?
I am familiar with the many ways an unlocking script is created. But in this scenario, lets use P2PKH. When we generate the unlocking script (excuse the simplicity):
<dup> <hash160> <wallet address> <equalverify> <checksig> |<signature> <Public Key>

Does a node such as a mining node push this data into the stack and verify it using the opcodes or is it the wallet software?

To be clear, what you list is both the locking and unlocking script:

The locking script: DUP HASH160 <recipient address> EQUALVERIFY CHECKSIG
The unlocking script: <signature> <pubkey>

The locking script (scriptPubKey) is part of the transaction where the sender S pays the recipient R. The unlocking script is part of any transaction where R uses their received money to send it further to a destination D.
So to be clear, in the example I'm describing there are two transactions: one from S to R, and later one from R to D.

In the first step, R gives their address to S. S constructs a transaction with some of their own inputs, along with an output to R, in the form of the locking script above.
In the second step, D gives their address to R. R constructs a transaction with as input the funds sent by S and the unlocking script above to authorize spending it, and an output which has a locking script based on D's address.

That R->D transaction (the one with <signature> <pubkey> in) undergoes the following steps:

It is created by R's wallet which likely does some basic safety check on it
It is sent out by R's wallet, and then relayed further to other nodes. If those are full nodes, they will validate the script, by executing the scriptSig, yielding [sig pubkey], and feeding that as input to the previous transaction's scriptPubKey (which will, if the signature and public key are valid, evaluate to true). If this evaluation returns failure, the transaction is ignored. Not all nodes are full nodes, and some may skip the full checking.
At some point the transaction reaches its destination D, which is a node like any other, and may or may not do full validation on it, depending on the software used.
At some point the transaction reaches one or more miners, who will hopefully include it in their next attempt to construct a block. Miners ought to perform full validation on the transaction, because if they don't, and include the transaction in a block despite being invalid, then that entire block will be invalid, and ignored by the network's full nodes.

Does a node such as a mining node push this data into the stack and verify it using the opcodes or is it the wallet software?

The wallet will construct a transaction that provides the unlocking script and transfers to the new locking condition. Then this transaction will be propagated to the rest of the network typically by the full node associated with that wallet. When other nodes receive this transaction they verify that the unlocking script meets the conditions needed to transfer those Bitcoin (and it is a valid transaction). If it does they accept into their mempool. Miners should also verify that the transaction is valid before considering whether to include that transaction in a block that they are attempting to mine.


text:
Really strange issue when trying to connect Sparrow Wallet to Bitcoin Core
I am really loosing my mind on this. I am actually not a technical noob but also not the very best technican. But things like this should actually be very easy for me.
What I want to do: I just want to connect my freshly set up Sparrow Wallet to my freshly set up Bitcoin Core.
First of all, my setup:

Bitcoin Core 23.0 on Windows 10, 64Bit (PC, local network)
Sparrow Wallet 1.7.1 on Linux (Laptop, same local network)

I've set up Bitcoin Core on my Windows PC as usual and blocks are up to date.
These lines were added to the "bitcoin.conf"-file on my Bitcoin Core Windows node:
rpcuser=user
rpcpassword=password

[main]
rpcbind=127.0.0.1
rpcbind=192.168.178.111
rpcallowip=127.0.0.1
rpcallowip=192.168.178.199/24

Where:

"rpcbind" - is the local IP address of the Bitcoin Core node on Windows 10
"rpcallowip" - is the local IP address of the laptop where Sparrow Wallet is running on
The subnet mask is "255.255.255.0" so "/24" should be fine.

I've set up a few things like firewall rules and so on. I've pressed on "Test connection" hundreds of times and it did not work at all. Somehow, it suddenly worked and I got successfully connected to Bitcoin Core! I was really happy, BUT: After closing the wallet and opening again, it again won't connect at all.
Again, I am getting the following error message:
Could not connect:
Check if Bitcoin Core is running, and the authentication details are correct.
See https://sparrowwallet.com/docs/connect-node.html

I noticed something really strange as well:
When I click "Test connection" on the laptop in Sparrow Wallet, even when Sparrow Wallet shows the above error, the Bitcoin Core screen changes from "You have no wallet loaded" to a wallet that is named "sparrow" (in Bitcoin Core!) and shows "0.00000000 BTC" (I did not set up any wallet on Sparrow Wallet yet!).
I tried to do some research on this issue but it looks like I am alone with this and feel kinda stupid since I don't get what I am doing wrong ..
I also tried disabling the Windows firewall before connecting Sparrow Wallet to it without any success.
Then I've randomly found this issue on Github.
I've tried the following workaround stated on the Github issue page:

Shutting down Bitcoin Core on Windows
Shutting down Sparrow Wallet on Linux
Deleting a folder called "sparrow" in the Bitcoin Core directory
Starting up Bitcoin Core again
Starting up Sparrow Wallet again
And TADA: It works! It says "Connected to "192.168.178.111:8332". But only one time! When I close Bitcoin Core or Sparrow Wallet again, it won't connect afterwards. It gives me the above mentioned error again. BUT: When I follow the workaround that I've stated above again, it works again (until I close Bitcoin Core or Sparrow Wallet and open it again). So it only works for a while when I delete the "sparrow" folder in the Bitcoin Core directory.
But only until none of both applications get closed. If I do so, I have to re-apply the stated workaround.

Other strange things I've noticed:

When I do the workaround and are able to connect Sparrow Wallet to Bitcoin Core, it does NOT show any inbound connection (but Sparrow Wallet says "Connected to..")
When I add "disablewallet=1" to bitcoin.conf, it does NOT even work with the mentioned workaround (even if it should as far as my understanding is correct)
When entering wrong authentication credentials on purpose, it takes some time until I get the error message "Could not connect" - when I enter them correctly, it shows this error message INSTANTLY! I got the feeling that it actually CAN connect (since the time until the error message shows up is less than a second, compared to when I enter credentials wrong which takes 10-15 seconds until the error message shows up)

Can anyone tell me what the actual heck is happening here? That's one of the strangest thing I've ever seen in my 20 year computer carreer. Sure I could just to this workaround every time but I feel kinda uncomfortable to connect/create any wallet with this strange issue being present.
I should mention that a VPN software is running on both machines. Anyways, it works when applying the workaround even with VPNs on (I've enabled the "Local Connections" options within both VPN GUIs). But it also does not work without following the workaround when both VPN softwares are completely closed and shut down. So the VPN can't be the cause of this issue I think.
I really appreciate any kind of help!
Is there any possibility that this is a serious issue by Sparrow Wallet and not my fault at all?
Thanks for any help in advance!

tl;dr: Make sure to add a wallet in Sparrow when first connecting to Bitcoin Core.
The cause of this issue is due to BWT requiring the optional keypoololdest field from the RPC call getwalletinfo, which causes it to fail to start when loading a legacy Bitcoin Core wallet that has no keys added to it. I should add that in fact it is the rust-bitcoincore-rpc project that is at fault (out of date) here, which BWT depends on.
Interestingly Bitcoin Core does return the field when a wallet is first created, but does not if the wallet is unloaded and then reloaded (as would happen when restarting Sparrow/Bitcoin Core).
The current solution is to stop both Bitcoin Core and Sparrow, delete the Bitcoin Core wallet, restart both and then make sure to create a wallet in Sparrow before restarting either application (to ensure some keys are added to the Bitcoin Core wallet first).


text:
What is difference between hmac_sha256, hkdf_sha256 and hmac_sha_512?
In the open source (https://github.com/bitcoin/bitcoin/tree/master/src/crypto) there are these algrithms that as I know connected with gerenation the new keys and master key. So I try to understand what exactly each algorithm does and how they related in Bitcoin Core.

They implement 3 distinct cryptographic schemes:

HMAC-SHA256 is a MAC (message authentication code, or a keyed hash function) based on SHA-256. It is used in the RFC6979 nonce generation function for deterministic ECDSA signing.
HMAC-SHA512 is a MAC based on SHA-512. It is used in the BIP32 key derivation logic.
HKDF-SHA256 is a (symmetric) key derivation function (KDF) based on HMAC-256. It is currently unused, but was added in anticipation of being used in the BIP324 (v2 P2P transport layer) implementation.

HMAC in the name stands for "Hash-based Message Authentication Code" and the hmac_sha_512 is used to generate keys in hierarchic-deterministic BIP-0032 wallets, and to generate a master key from the seed phrase (BIP-0039), for example here.
HKDF in the name stands for "Hash-based Key-Derivation Function", and there's a comment in the code which describes it:

A rfc5869 HKDF implementation with HMAC_SHA256 and fixed key output length of 32 bytes (L=32)

If you look at rfc5869 specification, it uses some HMAC as a primitive in the key derivation scheme, and in hkdf_sha256 the hmac_sha_256 is the primitive i.e. hmac_sha_256 is used for hkdf_sha256. The hmac_sha_256 is also used in httprpc.cpp and torcontrol.cpp.
But what is hkdf_sha256 used for? It doesn't seem to be used anywhere.


text:
Where is chacha20 and chacha_poly_aead used in Bitcoin Core?
In the source code of Bitcoin Core there are these functions but no explanation of how and where they are used. I need to make a description for all crypto-functions used in Bitcoin Core (https://github.com/bitcoin/bitcoin/tree/master/src/crypto) but it's difficult to find information, so asking you here.

You can open the git history for each file and see when and why they were added.
chacha20.cpp was added in PR #9792, apparently to improve Bitcoin Core's random number generator. chacha_poly_aead.cpp was added in PR #15649 as a preliminary step for implementing BIP324 (the P2P encrypted transport protocol).


text:
How do I set up a time-decay multisig?
I want to set up a multisig that requires fewer signatures as time goes by.
Let T be some time in the future (e.g. a year from now).
Before T, 2/3 signatures are required to unlock the UTXO (send coins).
After T, 1/3 signatures is required to unlock the UTXO.
The idea is to provide a recovery mechanism in case I lose 2 out of 3 keys. If I do not lose 2 keys, then my multisig would be just as secure as a regular 2/3 as long as I move the coins before T. If I lose 2 keys, then I have a recovery mechanism as long as I still have 1 key and none of the 3 keys were leaked.
Is this possible in Bitcoin, and if so, how? Is there a user-friendly tool that lets us do this? Thanks in advance!

This is technically possible but the user friendly tools are a work in progress depending on what you want to use. You've added the label "Ledger Hardware Wallet" to your question (since removed) so you may be interested in the latest state of Miniscript support in Ledger devices.
How I would do this currently (there may be better ways today or in future) is to use one of these sites (sipa's Miniscript, Bitcoin Dev Kit playground, shesek's min.sc) to generate your Miniscript from a Policy. sipa's site has an example Policy of "A 3-of-3 that turns into a 2-of-3 after 90 days":
thresh(3,pk(key_1),pk(key_2),pk(key_3),older(12960))

that compiles down to Miniscript of:
thresh(3,pk(key_1),s:pk(key_2),s:pk(key_3),sln:older(12960))

To change the Policy into what you want you could do (12960 x 4 = 51840):
thresh(2,pk(key_1),pk(key_2),pk(key_3),older(51840))

and run that through the compiler to get the Miniscript. Once you have the Miniscript you can import it into a Miniscript supporting wallet like Bitcoin Core, Specter or possibly in future Ledger.
I'd advise you practice with signet or testnet coins a few times first as the spending from this Miniscript will require PSBT and HWI if you are using hardware signers.
TL;DR technically possible but requires a lot more work to make it user friendly. It isn't using Taproot either so it would be a SegWit v0 address.


text:
Is there anyway to find a lighting payment request preimage after it is paid if you aren’t the sending or receiving node?
From what I understand, the preimage is publicly revealed when the lightning payment request is paid as a part of the transaction. Is there any way to find that transaction and therefore the preimage fro the details included in the decoded payment request? The reason for this question is that I am using an app to pay lightning payment requests that does not show me the preimage in the UI, so I’m wondering if there’s any other way to retrieve it through some kind of RPC call for example. Thanks in advance.

From what I understand, the preimage is publicly revealed when the lightning payment request is paid as a part of the transaction.

The preimage is revealed across the route from the destination back to the source but isn't announced publicly to say all nodes on the network.

Is there any way to find that transaction and therefore the preimage fro the details included in the decoded payment request?

If you are running a Lightning node then you can query the past transactions of your node but the payment request doesn't include the preimage. The preimage is only revealed once the payment is completed.

The reason for this question is that I am using an app to pay lightning payment requests that does not show me the preimage in the UI, so I’m wondering if there’s any other way to retrieve it through some kind of RPC call for example.

If you are running a Lightning node then the preimage should be returned on payment completion. With Core Lightning the pay RPC and keysend RPC both return the preimage on a completed payment. If you aren't running a Lightning node then it depends on the app you are using and whether you even have a Lightning node running (it may be that the app doesn't allocate you a specific Lightning channel).

Thank you for your answer. You said that "the preimage is revealed across the route from the destination back to the source"--how exactly does this work? How is the preimage is communicated from the node that received the payment back to the node that sent it?

@cool171: The same way all messages are communicated between Lightning nodes. The Lightning protocol relies on many messages being sent back and forth between Lightning nodes. They are typically communicated via TCP. Messaging is covered in BOLT 1: https://github.com/lightning/bolts/blob/master/01-messaging.md. Transport is covered in BOLT 8: https://github.com/lightning/bolts/blob/master/08-transport.md


text:
Converting raw scriptPubKey to readable format
I am debugging Bitcoin cpp code. I wanted to see what exactly come for scriptPubKey property of output. Theoretically, I know this but I want to see in the code how does it looks like.
Now while debugging I see  scriptPubKey object which is of type CScript and hence the format is different which is unreadable (see the screen shot below, it looks like char[]). How to read this value?
It looks like this :

\200\002\0\0à\210¼XUU\0\0\220»ÿÿÿ\177\0\0à¡·XUU\0

Why do you think there is a human readable form of a bitcoin script tape?

Try using hex encoding, then you can see what each byte value represents: https://en.bitcoin.it/wiki/Script

@JBaczuk thanks man, you are right I had to get the HEX of it and then call rpc decodescript to get in human readable format. :)

I figured it out.
setps:

Get the transactionID
RPCCall getRawTransaction transactionID - copy the rawTransaction
RPCCall decoderawtransaction rawTransaction - copy the scriptPubKey
RPCCall decodescript scriptPubKey


text:
Error: Prune mode is incompatible with -txindex
How do i remove this prune mode??

Do you know how to find your bitcoin.conf?

I want to add more details on this problem, with the actual hype of Bitcoin there are a lot of people that make bad tutorials with copy and paste on the web, and this causes a lot of confusion.
My additional explanation that I want to add to the answer is the motivation that this error happens.
The pruning mode is the unique mode to store only the partial data of Bitcoin blockchain, you can find the data stored from Bitcoin core on the directory path/bitcoin/blocks by default the path/bitcoin si in the home directory and the name of it is .bitcoin.
When you use the -txindex you are telling the Bitcoin core to be ready to receive any type of query with RPC commands. If you want to use Bitcoin core as a normal wallet you can avoid the option -txindex.
The options -txindex and prune=XXX are mutually exclusive, this means that can not stay together in the same configuration, because you are telling the Bitcoin core to be ready for any type of query but you are also telling that you don't want that  Bitcoin core store a lot of data on you PC. With partial data -txindex can not work.
P.S.: Noted also that if you remove the option prune=XXX, you need to re-scan the entire blockchain, and also the space on the disk required is very large.

I don't agree, I might want to see every tx-es (not only the ones meant for me), but I don't need the old ones. Like a detailed short history. So there is definitely a usecase for it.

add this to bitcoin.conf
prune=0 txindex=1 reindex=1 ( this will start to download the entire blockchain again if you have already downloaded it before). Ignore this line if you're going to run the core for the first time. If you use this code to reindex the blockchain then make sure to remove it later on when the reindexing is done.
Good luck
p.s.: i copied that from another forum. but it worked :)
after it started you can already remove reindex, as the change starts after reboot

How do i remove this prune mode?

There is documentation which says

To enable block pruning set prune=<N> on the command line or in bitcoin.conf, where N is > the number of MiB to allot for raw block & undo data.
A value of 0 disables pruning. The minimal value above 0 is 550.

Related

How can I run bitcoind in pruning mode?


text:
How to view addresses previously shown by Bitcoin QT console
I want to be able to compare receive and change addresses that are provided by watch-only wallet providers like Specter and Electrum with Bitcoin Core (not something like Ian Coleman's tool). I already used the "getnewaddress" RPC a bunch of times when playing around, and I would like to start over so that there's not a huge gap between unused addresses. I would love to be able to either 1) start over at index 0, 2) get an address for a specific index, or 3) get a list of, say, 10 addresses for a wallet even if the address has no transactions associated with it yet.
I'm guessing that one solution could be related to the "deriveaddresses" command. I've had troubles with this command though; so if there's an answer to avoid this method, I'd prefer to go that route. If "deriveaddresses" is the only way, then I'll open up a separate question and close this one.
Thank you


text:
Plot of distribution of all bitcoin transaction fees paid?
Is there a way to find and plot every transaction fee ever paid on the blockchain/bitcoin?
Background
I have read about transaction fees:

Though the average Bitcoin transaction may be around $25, for example, the median is closer to what most users will likely need to pay.

At that rate, if you want to send a transaction right away, the fastest transaction fee is currently around 102 satoshis/byte, according to bitcoinfees.earn. For a median transaction size of 224 bytes, this results in a fee of 22,848 satoshis, or $11. If you don’t mind waiting roughly a half hour, the price would be 83 satoshis/byte, or about $9. The price continues to slide down from there if you’re willing to wait even longer.

So this gives very useful info, but if I want to see the distribution of all historical transaction fees on the blockchain, how could I do that?
Also, because the value of bitcoin has changed a great deal over the past several years, it would be sensible to measure the transaction fees in USD equivalent (like the article does).
Is this easy to do (or even possible at all)?

Transactions vary a lot in data size, so it's usually more useful to compare the feerates of transactions rather than absolute fees. For example, a transaction that batches fifty payments may pay the same feerate, but a significantly higher fee, while still having a lower fee per payment than a transaction that performs a single payment.
You may find this block feerate composition graph (via Bitcoin Optech Group) interesting to get a sense of how feerates changed historically. While the data is a bit noisy for single blocks beyond a day or two, the 12-block average gives a sense of how fee rates change over the course of a day and the week, while the 144-block average may be interesting for bigger timeframes:

There is also this graph by Anduck that splits out blocks more precisely, but only for the recent past:

I think we have exactly the plot you are looking for in our paper Full Bitcoin Blockchain Data Made Easy (Figure 7):

This is a cumulative distribution: for each amount on the horizontal axis, we plot the fraction of transactions that have fees lower than or equal to this value.
As suggested by Joseph, we downloaded the whole blockchain and computed the fees (difference between output and input amounts) of each transaction. The figure also displays the fee distribution for the last million transactions, since putting together all fees actually makes little sense, as discussed in comments above.
Code and data are publicly available. In particular, the 12GB compressed text file blockchain.distilled_amounts.gz contains a line per transaction, with its input and output amounts (see the paper for details).

Bravo for sharing your hard work and making it available to other interested parties. Great idea to include 'most recent million'. Of those, looks like the vast majority of the fees are between 0.00001 and 0.01 BTC. I'd be extremely curious to see the plot without log x axis and a non-cumulative distribution. Also curious, what is the file size of the 12gb file when uncompressed (e.g. in RAM)?

I uploaded the raw distributions for you at http://latapy.complexnetworks.fr/fees_d and http://latapy.complexnetworks.fr/last_fees_d so you may play with them and try all sorts of scales :) The fees in satoshis are in the first field, then comes the number of occurrences of this fee amount.

Regarding your question on the uncompressed data, the 12GB become 42GB. It is plain text format, though, so not optimized. It lists the amount of each input and each output of each of the 623,483,734 transactions we have, which leads to a total of 3,225,135,853 values; with 32 bit integers, you may store this in slightly more than 12GB RAM, if I am correct.

Amazing. Did you use vanilla python on a large computer (i.e. >42GB ram), or something like spark, or some other tool (e.g. BigQuery) to analyse the uncompressed file? It would be pretty great to place in something like a public BigQuery dataset.

Thanks also for making the raw fee data available. When I get a chance, I'll try to plot them using ggplot or similar, I will provide the plot/code if I do.

:) Our goal was to show that one may quite easily and efficiently handle the full blockchain data with basic tools (small python scripts and command lines). This actually is the most effective way for various processing tasks. Using more advanced technologies for more advanced tasks is a perspective, and we release the data for this. I would love to see what others do with it.

Thanks for the bounty! Please consider accepting this answer too, if you think it does answer your question :)

The transaction fees can be calculated by subtracting each transaction's inputs from its outputs – that difference is the fee. I would guess your best bet is to clone Bitcoin Core, download the entire blockchain, and start parsing the data yourself.
If that sounds like too much, I would start here and then ask another question once you get a full node up and running.
Best of luck!

+1 Great to know it's possible! I think DIYing it would be marvellous learning, however, do you know if someone would already have attempted it and stored the resulting data somewhere?


text:
When does client decide to stop/cancel proof of work algorithm?
I have been reading about bitcoin and blockchain in the last few days, but I have something I don't clearly understand.
The "proof of work" may take a long time for a simple node to guess, but if we combine all nodes, they guess it in 10 minutes.
Now lets say another node guessed it faster than me, how do I know when I should stop my proof of work algorithm?
Is there any point on keep trying to guess it?
When do you start trying to guess the next proof of work?
Hope the question is clear.

Now lets say another node guessed it faster than me, how do I know when I should stop my proof of work algorithm?
When do you start trying to guess the next proof of work?

Whenever a node successfully solves a proof-of-work it immediately broadcasts it to the whole network. So whenever you hear a block that you are trying to solve you should stop and start mining a fresh block on top of the newly received block

NOTE: There is an attack called "Selfish mining strategy" where a node does not immediately broadcast a newly mined block. (You may leave it until you become clear with the basics)


text:
Is there a way to retrieve my LTC that did not go into my wallet?
Last night I tried to send LTC from my Binance account to my desktop wallet. I found out later that my wallet doesn't accept LTC so the transfer is lost. Is there any way I can retrieve it? I have the transaction ID and Blockcypher ledger with all the inputs consumed and outputs created. Is there any way I can recover my funds using that info? Green Address wallet said they can't help me, and the response from Binance is generic and non helpful. Please help?

You'll have to explain in more detail what exactly you did.  What desktop wallet software are you using?  Are you saying that you generated some address with that software which happened to work as an LTC address?  What did you do to accomplish that?

Thanks Nate, I bought the LTC on Bianace using Bitcoin. A few days later, I tried to tranfer LTC from my Binance account to my Green Address wallet. Apparently, Green Address wallets only accept Bitcoin. I simply used my Green 34 digit "receive" address and tried to send the LTC from Binance to the green wallet.

What was the first character of this address?  Would you be willing to share the transaction id?

The first character of the receive address is "3". I would prefer to send the transaction id to your email. I trust you based on your posts but I would prefer not to post it here. I have no idea if that id can be used by someone else to transfer my coins?

Nate, please see my comments below to "Evil". Is my passphrase the same as the private key?

can I use the QR code associated with that receive address in any way to recover my coins?

Ok, starting with a 3 means it's pay-to-script-hash for both coins.  The question would be whether the associated script is usable without change in a Litecoin transaction.  I don't know the answer to that, but perhaps some expert here will.  I would suggest you edit this info into your question.

thanks Nate. I'm in the process of composing a more detailed post of everything that happened and will post it later tonight. I've tried to resolve this issue with binance for days but they are unhelpful and unwilling to try and retrieve my coins. And their website has been down for over a day.

There is nothing Binance can do about this, so no need to spend any more time in that direction.  They did exactly what was asked of them by sending coins to the address you specified.  The only possible solution is to get the private key for that address (either directly from Green Address or by calculating it from your wallet seed) and see if it can be used in a Litecoin wallet to create a transaction to spend the coins.   Whether that is possible depends on whether Litecoin and Bitcoin handle P2SH scripts in the same way, which I don't know.

Nate, I heard back from Green Address, they don't store users private keys. Can you tell me how to "calculate it from my wallet seed", as you mentioned? From there I will create a Litecoin wallet and try to create a transaction. Any guidance on how to do that would be much appreciated.

No it is lost.
Basically because the transaction is now in the LTC blockchain with a send to address that no one has the private key for.

Depending on how the address was created, there is a chance that OP actually does have the private key.  But we need more information in order to know for sure.

@NateEldredge its possible if the desktop wallet exposes the private key, to make a new LTC transaction back to the originating account. I would assume it was LTC sent to a BTCor other addess. So yeah it may be possible.

Exactly.  So I think it's premature to answer before we know more.  I voted to place the question on hold as "unclear" for this reason.

Thanks Evil and Nate, Is there a way for me to retrieve the private key? I didn't know I needed one before I tried to make the transfer. Is the private key the same as my mnemonic passphrase?

I have the original "receiving wallet" address, and the Txid, Will that be helpful?

@Kaa see if your wallet has the option to convert to a cold wallet, this should display the private and public key. You can then use those recieving addresses as originating addresses on the LTC network. If the hash is treated the same.

thanks Evil, I just emailed Green Address wallet and asked them if it's possible to convert to a cold wallet, awaiting their reply


text:
Copay does not go past splash page ios
I am running IOS 10.3 on an Iphone 6s and can't get past the copay splash screen no matter how long I wait.  Everything worked just fine until two software updates ago.
Anyone else seeing this?

Exactly the duplicate of copay does not work in iphone5 ios10
Don't worry, it has been reported: https://github.com/bitpay/copay/issues/8382
It's not only you, other people seem to have the same problem.
Just wait until they release a fix for it.
EDIT: It has been fixed. Please update the app!


text:
How do I run my own testnet? What's testnet-in-a-box?
If I want better control of my test environment while developing a bitcoin-related application it seems useful to run my own testnet rather than use the public testnet. How can I do this?
What is testnet-in-a-box? Can I use that? What are the advantages and limitations, if any? Are there other options?

Checkout the updated Testnet in a box.  You will need bitcoind already installed in your system.
The "Testnet in a box" linked above is just like the world-wide bitcoin network except the only members are the 2 running on your computer. There should be no difference except that it's easier to generate coins.


text:
Is there an alternative to salvagewallet that keeps account info?
I am using the accounts feature of bitcoind , if a wallet gets corrupted and bitcoind is started with -salvagewallet then the account information is lost.
So, is there any built in feature that will rebuilt also my accounts ?

have you tried to use pywallet to restore the file?

Haven't tried but I have read this : https://en.bitcoin.it/wiki/Pywallet , and there is no mention about accounts support for this utility.

well there's a lot more to it than just that one liner. there's an entire thread on it (https://bitcointalk.org/index.php?topic=34028.0). you can do wallet surgery and remove offending errors and then set the rest back in place. if you are a skilled enough wallet surgeon you can leave everything in place. Honestly though: I'm not a huge fan of the accounting feature it causes too many headaches when we should be using other means rather than accounting  (personal opinion though nothing more) :)

This could be a solution once released:
"Add -zapwallettx function, a diagnostic tool to assist in wallet repair" - https://github.com/bitcoin/bitcoin/pull/3659

Will surely take a look when it is released. Thanks for the heads up.


text:
What are valid BIP32 version bytes in descriptors & optional key origin defaults?
BIP49 & BIP84 defined different BIP32 version bytes so that serialized xpubs start with ypub/zpub/upub/vpub.
This led to some incompatibility in the past with wallets and libraries that used (or didn't) those version bytes.

Do descriptors finally get rid of different version bytes?

For example, is wpkh([d34db33f/84'/0'/0']zpub...) not valid? (note the zpub string)

Also, since key origin information is optional in descriptors, are there default derivation paths or are they supposed to be undefined?
For example, in an extended descriptor without optional key origin such as this one: wpkh(xpub...), can we assume its derivation path starts from 84'/...?
Or for wsh(pkh(xpub...)) is it 49'/...? Or should we not assume that?


text:
Is writing unit tests a good way to start contributing to Bitcoin core?
I'm considering writing unit tests to Bitcoin core as a first step to C++ development and Bitcoin contribution. Anyone have any suggestions about the journey I'm about to begin and possible files that are needing tests?

If you are learning C++ from scratch you may struggle to start writing unit tests to begin with even if you identify a particular unit test that is missing. So that is one challenge.
In terms of discovering a missing unit test to write there are various ways to do this. It is worth monitoring the good first issues on the GitHub repo though I think that would be relying on a PR author deliberately not writing a unit test and leaving it as a good first issue.
Alternatively you could check out the unit test coverage on Marco Falke's site to see what files have low unit test coverage.
A useful case study is Fabian Jahr's first contribution to Core which was a simple code change that meant MacOS users skipped a functional test that was flaky on MacOS. He did a couple of excellent talks at Bitcoin Edge Dev++ 2019 on the Bitcoin Core functional test framework and on debugging Bitcoin Core.
Probably the most productive use of your time though would be to attend the weekly Bitcoin Core PR review club set up by John Newbery. Every week a different PR is chosen and during that you can ask about the tests written for a particular PR and whether there are any unit tests that are missing. You can attend on IRC at ##bitcoin-core-pr-reviews and you can ask whatever you want. New contributors are generally the primary target audience.

Jumping straight to writing unit test can be hard, I suggest You check the official github page for how to contribute to the bitcoin core:
https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md
If I wanted to contribute I would personally start by trying to solve some of the problems on their issues page because it is fresh, there is a lot of work, some can be easy and help is always needed:
https://github.com/bitcoin/bitcoin/issues


text:
how to convert WIF key return to HEX format
any one knows how to convert WIF key return to hex format please also let me know if the private key is out side range of ecc then how to import it to some wallet ?

Using this python library (I am the author)
>>> from cryptotools import PrivateKey

>>> prv = PrivateKey.from_wif('5KUkQNKHA9cVmhSyKuNmop7r83DN2fMhPtNUW843Q6JxqRddEVz')
>>> prv.hex()
'db07a0c6c98c7a2b5cdc19cd4431a05908a40424d6d1ff20eac9cb2f9488e12d'

If the key is too large
>>> PrivateKey.from_int(1 << 260)

it will throw an error
AssertionError: Key larger than Curve Order

thank you mike is there any way to import the private key which fall out of range of elliptical curve

No. If you have a number that's too large you should first find it's congruent modulo N and then import that

There is also a bash (or zsh) shell script to convert from WIF to hexadecimal format: https://gist.github.com/GregTonoski/fbe2740391a366add4bfcefded00c5c4
Simply run the script like in the example:
$ bash Bitcoin_WIF_into_hex.bash "L3wB8ytuxNS3SPX2CJnHqK48Zzqj1AnayDTrJomvNPDxuKvHyvpT" > private_key_bitcoin.hex


text:
Bitcoin blockchain in mongodb
How bad would it be to use MongoDB as a Bitcoin database instead of LevelDB? I know that LevelDB is faster but I've also read that MongoDB is better for doing queries.
It seems cleaner to store the information in JSON. And also easier to query.
Any reason why LevelDB was chosen?

On the subject of why LevelDB is used, core developer Greg Maxwell stated the following to the ref:

I think people are falling into a trap of thinking "It's a <database>,
  I know a <black box> for that!"; but the application and needs are
  very specialized here. . . It just so happens that on the back of the
  very bitcoin specific cryptographic consensus algorithim there was a
  slot where a pre-existing high performance key-value store fit; and so
  we're using one and saving ourselves some effort...


text:
How to debug bitcoin core in "real time"
I am new to bitcoin core and C++
I downloaded already the project from: https://github.com/bitcoin/bitcoin
and run "makefile" command.
Now I can make all transaction against the blockchain.
I want to have the option to run "getnewaddress" in the command line
and it will stop in a break point in the source code.
but i cann't run the commands "bitcoind" and "getnewaddress", without first building the project.
How can I debug this project?

In order to debug, you need debug symbols. You can only generate them if you build the source code on your machine with the debug flag on.

@CinCout how to build the project with the debug flag on ? Is this something I need to change in the code or in the terminal with the "make" command? thanks.

https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#compiling-for-debugging

There is no "makefile" command.
If you don't build the project, there is no way for you to debug it. The released binaries do not contain any debugging symbols so it will be very difficult for you to set breakpoints and see what is going on. So in order to do that, you will need to build the project from the source code that you downloaded. There are instructions for how to do this in the docs folder of the downloaded source code.


text:
Discovering Bitcoin network topology
For a research I am doing I would love to have a graph with the network of reachable Bitcoin nodes. Active reachable nodes can be extracted by using getaddr messages (e.g. bitnodes project). However direct inbound/outbounds connections cannot be discovered at least you have access to the node and you can run the RPC command getpeerinfo.
Is there any way to get a detailed graph of the network?

No, due to different reasons.
First, the network is always in flux, nodes are appearing and disappearing, and connections are even more dynamic, so even if there were an API you could use to ask a node for its connections, that information would be outdated quickly.
Second, privacy is part of the reason why crypto currencies exist. Divulging the list of peers of every node would run counter to this goal, so node software would be ill-designed if it provided such an interface.


text:
How to create raw bitcoin transaction with multisig type of output scriptPubKey
I need to send money in one transaction to 2 addresses so that any of them can spend it.
In other words - I need to create same transaction as here
https://www.blocktrail.com/BTC/tx/6e2c5b6b2a926a7da2415ae55c701e83d6e7bc46bad9bc23b5f9c7ef1ae26998
(
hex of raw transaction is
010000000227ed8477b4d0ab4d52bc31e0639f2f58665d9f7aff0a2aab229bba51af28ea5b000000008a4730440220279b45f812ebd1004ee041eb75a3f42657dce19bdba06b30b2d1b70f45e2590602201b06ecbbbf6fae0a3455d98ac2924cd6dc3022425a2f08c60ffb46e96dbdc8e9014104a84d304aa8963fbd36287e674f109827b6d6ea60d57a7d9357df03be1fcedb2c47475ca128b1a50408b7f584041ffd52d6b19aba5256e99dcdbbe2ed7373775dfffffffff302a856316c80cfbb5735c7a1fd5caad5b856206e2855e6f98bad5959778620010000008a47304402203b163fb30bf53c15e1b7d4ab8af17c3e79193f12d29b17d0a68afac8dc07f07b0220364d304ee4d3fcba82729ba5b011f99636e5633f66f5af224fd8ec5cd1850a44014104a84d304aa8963fbd36287e674f109827b6d6ea60d57a7d9357df03be1fcedb2c47475ca128b1a50408b7f584041ffd52d6b19aba5256e99dcdbbe2ed7373775dffffffff028096980000000000c9524104478f498fe3f6872a9559ae0fd5975bc44f500eed955a835027962099c333536f60b4e60383e6e1081efa0a76df1ef0aefb4da87ff0c8f12dab5da2969fc7b24e41044f09a164267c635c6991f7a96bc7901d035c07161a0074d719be723f6a9c50bc72b900092cffec5f3c3484dae35d04a5a2fa2e75f3a99e17577537c1227b44ba4104971e5b8b222fe47f742fa07d3327d36a6cd37088656ce29842ed82e1dc8bffaa81848b3219359fdf9d8b590d3af85cfff2d06d4b19fde5ed560b2c9caa5dd65653ae40196604000000001976a9148c1b30a7b5f8bd87c1cd8b8c4cb4f74345f1978a88ac00000000

)
with multisig type of scriptPubKey of output and array of destination addresses with 2 P2PKH (no 1 P2SH!) items.
curl --user USERNAME:PASSWORD --data-binary '{"method":"getrawtransaction","id":"a2e8f6b0-5c03-4315-a067-050a9adb6dfc","params":["6e2c5b6b2a926a7da2415ae55c701e83d6e7bc46bad9bc23b5f9c7ef1ae26998",1]}' "http://127.0.0.1:18833"

{"result":{"hex":"010000000227ed8477b4d0ab4d52bc31e0639f2f58665d9f7aff0a2aab229bba51af28ea5b000000008a4730440220279b45f812ebd1004ee041eb75a3f42657dce19bdba06b30b2d1b70f45e2590602201b06ecbbbf6fae0a3455d98ac2924cd6dc3022425a2f08c60ffb46e96dbdc8e9014104a84d304aa8963fbd36287e674f109827b6d6ea60d57a7d9357df03be1fcedb2c47475ca128b1a50408b7f584041ffd52d6b19aba5256e99dcdbbe2ed7373775dfffffffff302a856316c80cfbb5735c7a1fd5caad5b856206e2855e6f98bad5959778620010000008a47304402203b163fb30bf53c15e1b7d4ab8af17c3e79193f12d29b17d0a68afac8dc07f07b0220364d304ee4d3fcba82729ba5b011f99636e5633f66f5af224fd8ec5cd1850a44014104a84d304aa8963fbd36287e674f109827b6d6ea60d57a7d9357df03be1fcedb2c47475ca128b1a50408b7f584041ffd52d6b19aba5256e99dcdbbe2ed7373775dffffffff028096980000000000c9524104478f498fe3f6872a9559ae0fd5975bc44f500eed955a835027962099c333536f60b4e60383e6e1081efa0a76df1ef0aefb4da87ff0c8f12dab5da2969fc7b24e41044f09a164267c635c6991f7a96bc7901d035c07161a0074d719be723f6a9c50bc72b900092cffec5f3c3484dae35d04a5a2fa2e75f3a99e17577537c1227b44ba4104971e5b8b222fe47f742fa07d3327d36a6cd37088656ce29842ed82e1dc8bffaa81848b3219359fdf9d8b590d3af85cfff2d06d4b19fde5ed560b2c9caa5dd65653ae40196604000000001976a9148c1b30a7b5f8bd87c1cd8b8c4cb4f74345f1978a88ac00000000","txid":"6e2c5b6b2a926a7da2415ae55c701e83d6e7bc46bad9bc23b5f9c7ef1ae26998","hash":"6e2c5b6b2a926a7da2415ae55c701e83d6e7bc46bad9bc23b5f9c7ef1ae26998","size":612,"vsize":612,"version":1,"locktime":0,"vin":[{"txid":"5bea28af51ba9b22ab2a0aff7a9f5d66582f9f63e031bc524dabd0b47784ed27","vout":0,"scriptSig":{"asm":"30440220279b45f812ebd1004ee041eb75a3f42657dce19bdba06b30b2d1b70f45e2590602201b06ecbbbf6fae0a3455d98ac2924cd6dc3022425a2f08c60ffb46e96dbdc8e9[ALL]
  04a84d304aa8963fbd36287e674f109827b6d6ea60d57a7d9357df03be1fcedb2c47475ca128b1a50408b7f584041ffd52d6b19aba5256e99dcdbbe2ed7373775d","hex":"4730440220279b45f812ebd1004ee041eb75a3f42657dce19bdba06b30b2d1b70f45e2590602201b06ecbbbf6fae0a3455d98ac2924cd6dc3022425a2f08c60ffb46e96dbdc8e9014104a84d304aa8963fbd36287e674f109827b6d6ea60d57a7d9357df03be1fcedb2c47475ca128b1a50408b7f584041ffd52d6b19aba5256e99dcdbbe2ed7373775d"},"sequence":4294967295},{"txid":"2086775959ad8bf9e655286e2056b8d5aa5cfda1c73557bbcf806c3156a802f3","vout":1,"scriptSig":{"asm":"304402203b163fb30bf53c15e1b7d4ab8af17c3e79193f12d29b17d0a68afac8dc07f07b0220364d304ee4d3fcba82729ba5b011f99636e5633f66f5af224fd8ec5cd1850a44[ALL]
  04a84d304aa8963fbd36287e674f109827b6d6ea60d57a7d9357df03be1fcedb2c47475ca128b1a50408b7f584041ffd52d6b19aba5256e99dcdbbe2ed7373775d","hex":"47304402203b163fb30bf53c15e1b7d4ab8af17c3e79193f12d29b17d0a68afac8dc07f07b0220364d304ee4d3fcba82729ba5b011f99636e5633f66f5af224fd8ec5cd1850a44014104a84d304aa8963fbd36287e674f109827b6d6ea60d57a7d9357df03be1fcedb2c47475ca128b1a50408b7f584041ffd52d6b19aba5256e99dcdbbe2ed7373775d"},"sequence":4294967295}],"vout":[{"value":0.10000000,"n":0,"scriptPubKey":{"asm":"2
  04478f498fe3f6872a9559ae0fd5975bc44f500eed955a835027962099c333536f60b4e60383e6e1081efa0a76df1ef0aefb4da87ff0c8f12dab5da2969fc7b24e
  044f09a164267c635c6991f7a96bc7901d035c07161a0074d719be723f6a9c50bc72b900092cffec5f3c3484dae35d04a5a2fa2e75f3a99e17577537c1227b44ba
  04971e5b8b222fe47f742fa07d3327d36a6cd37088656ce29842ed82e1dc8bffaa81848b3219359fdf9d8b590d3af85cfff2d06d4b19fde5ed560b2c9caa5dd656
  3
  OP_CHECKMULTISIG","hex":"524104478f498fe3f6872a9559ae0fd5975bc44f500eed955a835027962099c333536f60b4e60383e6e1081efa0a76df1ef0aefb4da87ff0c8f12dab5da2969fc7b24e41044f09a164267c635c6991f7a96bc7901d035c07161a0074d719be723f6a9c50bc72b900092cffec5f3c3484dae35d04a5a2fa2e75f3a99e17577537c1227b44ba4104971e5b8b222fe47f742fa07d3327d36a6cd37088656ce29842ed82e1dc8bffaa81848b3219359fdf9d8b590d3af85cfff2d06d4b19fde5ed560b2c9caa5dd65653ae","reqSigs":2,"type":"multisig","addresses":["1NiA6V8Ges2vEkSx11X5oo2aCyTsCv3XH3","18am8jUnBqru2jtQpQbE4LCywBWUPUooP1","1Fr1wwdwoNH3F7zFAvcWJte5vsacto3EXC"]}},{"value":0.73800000,"n":1,"scriptPubKey":{"asm":"OP_DUP
  OP_HASH160 8c1b30a7b5f8bd87c1cd8b8c4cb4f74345f1978a OP_EQUALVERIFY
  OP_CHECKSIG","hex":"76a9148c1b30a7b5f8bd87c1cd8b8c4cb4f74345f1978a88ac","reqSigs":1,"type":"pubkeyhash","addresses":["1DmpB4AxC8tee5cMNhEjxg4xcaXw9cmAen"]}}],"blockhash":"0000000000000a96fe9d4d61b19127548bfeb40f2aa972c41b7bf8dc09e2967b","confirmations":326412,"time":1329692179,"blocktime":1329692179},"error":null,"id":"a2e8f6b0-5c03-4315-a067-050a9adb6dfc"}

How we can see:
"type":"multisig","addresses":["1NiA6V8Ges2vEkSx11X5oo2aCyTsCv3XH3","18am8jUnBqru2jtQpQbE4LCywBWUPUooP1","1Fr1wwdwoNH3F7zFAvcWJte5vsacto3EXC"]
I need same result - array of several P2PKH addresses and multisig type.
But https://bitcoin.org/en/developer-reference#createrawtransaction method does not allow directly set array of addresses for 1 output
in transaction - only 1 address.

I created transactions with P2SH address, which was created with createmultisig function:
user@workstation:/home/username$ bitcoin-cli createmultisig 1 '''["1BdxBor4JG76RKLAwJZfHC58fWbgidYukz","1MN3cT9Ro927h4kgpSZ5V7SfYjrwTysXv7"]'''
{
  "address": "3Jc4v7eo9rptPj4277c9cCjgJmWSQmBiwj",
  "redeemScript": "512102f2221ec0e4baa45633a1f10d41cb199d943ec448368b90d9c7982095fd926ffa2102a9132ba9b227c62cf02deaff00e974fd9bfbe96ed1947d9bf8c7af40c5c22dd752ae"
}
and after
bitcoin-cli createrawtransaction '''[{"txid" : "4989efcec28faedc86ee309a0cb1ba1319987d5c214fcb39d709538576e79342","vout" : 1,"sequence":1}]''' '{ "3Jc4v7eo9rptPj4277c9cCjgJmWSQmBiwj": 0.00005500,"12S42ZEw2741DHrivgZHLfX8M58mxb7bFy": 0.01102678}'
after signed transaction and sent into the network
hash of transaction:
d1789305f1ae4a39c045e0c67a024048cda0990297dc863e4caac546c6c32b86
raw:
02000000014293e776855309d739cb4f215c7d981913bab10c9a30ee86dcae8fc2ceef8949010000006a47304402200c7a7e89dc37a457c9070b928619b0ebed9b9a8fcbe1ec0ae9a06ecd695279eb0220015f1898c527e38319818f9939822b0ade1c1c508cf7d1215bf2b99905b1fcff01210267af6c6bf4ae6e37f019fbfbc7df70acf48663adbf19161bd874f3babd6bf15c01000000027c1500000000000017a914b98a8e7f0375003d5537ff8489fcf9c4ad98f2358756d31000000000001976a9140fb50d2ec6bb62bd690bb55142101ca28a678be188ac00000000

But in the result of json-rpc method getrawtransaction I see
"scriptPubKey":
{
"asm":"OP_HASH160 b98a8e7f0375003d5537ff8489fcf9c4ad98f235 OP_EQUAL",
"hex":"a914b98a8e7f0375003d5537ff8489fcf9c4ad98f23587",
"reqSigs":1,
"type":"scripthash",
"addresses":["3Jc4v7eo9rptPj4277c9cCjgJmWSQmBiwj"]
}
meanwhile I need type: "multisig" and array of several P2PKH addresses, no 1 P2SH address.

Another attemption: I used addmultisigaddress function
user@workstation:/home/username$ bitcoin-cli addmultisigaddress 2 '["1BdxBor4JG76RKLAwJZfHC58fWbgidYukz","1MN3cT9Ro927h4kgpSZ5V7SfYjrwTysXv7"]'
3P6sVMdtiqSwPuruSMguJQmmvFba3groVh
user@workstation:/home/username$ bitcoin-cli validateaddress 3P6sVMdtiqSwPuruSMguJQmmvFba3groVh
{
  "isvalid": true,
  "address": "3P6sVMdtiqSwPuruSMguJQmmvFba3groVh",
  "scriptPubKey": "a914eadd957f6e1c6f4e71f9bd0a8632d8e3a61af43f87",
  "ismine": true,
  "iswatchonly": false,
  "isscript": true,
  "script": "multisig",
  "hex": "522102f2221ec0e4baa45633a1f10d41cb199d943ec448368b90d9c7982095fd926ffa2102a9132ba9b227c62cf02deaff00e974fd9bfbe96ed1947d9bf8c7af40c5c22dd752ae",
  "addresses": [
    "1BdxBor4JG76RKLAwJZfHC58fWbgidYukz",
    "1MN3cT9Ro927h4kgpSZ5V7SfYjrwTysXv7"
  ],
  "sigsrequired": 2,
  "account": ""
}
bitcoin-cli createrawtransaction '''[{"txid" : "d1789305f1ae4a39c045e0c67a024048cda0990297dc863e4caac546c6c32b86","vout" : 1,"sequence":1}]''' '{ "3P6sVMdtiqSwPuruSMguJQmmvFba3groVh": 0.00005500,"12S42ZEw2741DHrivgZHLfX8M58mxb7bFy": 0.01}'
after signed transaction and sent into the network
hash of transaction:
cbf1bba98082f23889f79364ae146b6cbbfb1767a20c0e2bc210fee43561ee1d
raw:

0200000001862bc3c646c5aa4c3e86dc970299a0cd4840027ac6e045c0394aaef1059378d1010000006a47304402205ad43bb4c6e07387a62d5c1859f48869b21cd069fe9bda93b05466e0da42228102202d3d894fae7a11b3369c5616eebbc6b3ae00001ed71f0b0d8e3324ef2d8e5c9101210267af6c6bf4ae6e37f019fbfbc7df70acf48663adbf19161bd874f3babd6bf15c01000000027c1500000000000017a914eadd957f6e1c6f4e71f9bd0a8632d8e3a61af43f8740420f00000000001976a9140fb50d2ec6bb62bd690bb55142101ca28a678be188ac00000000

But in the result of json-rpc method getrawtransaction I see again
scriptPubKey":
{
"asm":"OP_HASH160 eadd957f6e1c6f4e71f9bd0a8632d8e3a61af43f OP_EQUAL",
"hex":"a914eadd957f6e1c6f4e71f9bd0a8632d8e3a61af43f87",
"reqSigs":1,
"type":"scripthash",
"addresses":["3P6sVMdtiqSwPuruSMguJQmmvFba3groVh"]
}
meanwhile I need type: "multisig" and array of several P2PKH addresses, no 1 P2SH address.

What I am doing wrong?

I try to provide a (partial) answer: I thin you are not doing anything wrong... there are two ways of creating multisig tx. The "old" way as with your link from blocktrail, and the "new" way as per your following text. The old way had two or three disadvantages: it bloats the blockchain from the beginning, the fees are high at the beginning, and it discloses the possible signer's pubkeys. In this case the 3 non compressed pubkeys have each 65 bytes, so 200 bytes on the blockchain...
The new multisig via a P2SH construct basically hashes the spending logic (also called the redeem script: 2    3 CHECKMULTISIG), and base58check encoded provides a "P2SH address" (beginning with a "3"). This hash is then only 20 Bytes long, non-reversible (you can't see which are the pubkeys), and the fee for bringing this tx to the blockchain remain low. Of course, at the very end, the spending this tx requires to disclose and pay the higher fees.
Unless you don't have a very, very specific use case, do you think it is impossible to use the newer logic?
I found further description in Andreas book "Mastering Bitcoin" in Chapter 7, page 149ff - it is also online available.

pebwindkraft, thank you so much for the answer. I have mentioned very very specific use case, so if you can navigate me how to create bitcoin transaction with multisig-output (with array of P2PKH addresses) using
createrawtransaction-method (or with php/js/.../language) - please do it ).
Unfortunately I did not found this info in Mastering Bitcoin.

this looks more difficult than I thought. Keyword is probably "raw multisig", quickly searching through the forum reveals an answer from Pieter (https://bitcoin.stackexchange.com/questions/46447/with-what-number-do-multisig-addresses-non-p2sh-start/46448#46448), which discourages the usage of raw multisig. Probably BIP11 provide more info. This post https://bitcoin.stackexchange.com/questions/3712/how-can-i-create-a-multi-signature-2-of-3-transaction/10593#10593 mentions the use of an "SX Library", but I have never checked any libraries for this...

and here is the link to the first edition from Andreas: http://chimera.labs.oreilly.com/books/1234000001802/ch05.html#multisig and bitcoin has this link: https://bitcoinj.github.io/working-with-contracts


text:
bitcoin-cli getbalance 0 after restart
I am working on a private chain using regtest mode. I have a strange behaviour if i restart the bitcoind process the balance becomes 0.?
i generate 101 new block to get the reward 50BTC. I repeat this operation to be sure but always i get the same result the balance is 0.
before
C:\WINDOWS\system32>bitcoin-cli -regtest getbalance
8800.00000000
after
C:\WINDOWS\system32>bitcoin-cli -regtest getbalance
0.00000000

update : the problem was the use of CTRL+C to stop the bitcoind deamon

Can you please update your post and show us the commands you use to start and restart bitcoind in regtest mode?

You probably haven't gracefully shutdown your node during the first run.
Make sure you always stop your node(s) with ./bitcoin-cli -regtest stop


text:
What Do I Do? Unconfirmed Transaction For Almost 7 Hours :(
Sorry if I'm acting like a noob. Never had a transaction take this long and it's worrying :/ It's been about 6-7 hours and still no confirmations. Do I just wait it out? Please help! :(
The transaction ID is
c3cfa18f21ab69dce87b269643cc841f0c16b7e69683afaad14a25d4b0740277
EDIT: My wallet will not synchronize now either...
Thanks

It looks like the transaction confirmed shortly after you posted this. https://blockchain.info/tx/c3cfa18f21ab69dce87b269643cc841f0c16b7e69683afaad14a25d4b0740277 Should I delete this?

You can thank the China government and internet censorship for the recent blockchain issues considering the two largest bitcoin miners are in China.
I have payments from yesterday almost 18 hours that have not finalized so you are not alone.
It is the beginning of the end for bitcoin brotha better start learning Monero!

This issue is unlikely to be caused by miners directly.


text:
How can I use Bitcoin privately with the Electrum wallet and Tor?
I do not want to run a full node like Bitcoin Core but still want privacy running an SPV node like Electrum wallet to use Bitcoin.
How can I setup to run Electrum privately with Tor?

Electrum v3.1.3 and earlier includes a setting to use Tor
When Tor is correctly setup on your system, it is possible to directly configure Electrum to connect via Tor for all connections, both to Electrum servers and, to third-party services.
Using these steps you can be anonymous in only five minutes.
Setting up Electrum and Tor
These instructions work on Fedora 27 and assume a default installation of Electrum v3.1.3 and Tor v0.3.1.10 or newer. Fedora is a modern operating system that will run on most standard modern hardware. The configuration is the same on Windows, but the instructions are different. There are some instructions for setting up Tor on Windows here.
Further instructions for other *nix based systems are available here. NOTE: You do not need to configure your Tor client as a relay or exit node for Tor to operate, so you can skip the step for 'Put the configuration file /etc/tor/torrc place:' in that guide. You will still need to use all of the following steps in this guide.

Setup Electrum

Install the Electrum package:
sudo dnf install electrum

Run Electrum. If you start Electrum with the -o offline switch the first time you run it your details don't leak out.

Choose Automatically Connect and click on Next.
Name your wallet if you like or accept the default and click on Next.
Select Standard Wallet and click on Next.
Select Create a new seed and click on Next.
Select Segwit and click on Next.
Securely, privately and accurately store your seed. This is your wallet backup. If it is lost all your BTC is gone permanently. Click on Next.
Type your seed in the box and click on Next to verify you have it correctly.
Set a good password that you will never forget. This must be secure to ensure your wallet cannot be stolen. Click on Next.
Your wallet is setup.

Setup Tor

Install the tor package:
sudo dnf install tor

Start the tor daemon and make sure it starts at boot:
sudo systemctl enable tor
sudo systemctl start tor

With Electrum setup and running:

Go to Tools -> Network
Click on the Proxy tab
Place a tick for Use Tor proxy at port 9050
Note that Electrum will momentarily disconnect and reconnect via Tor
Click on Close
Exit and restart Electrum normally (to go online)
Check that your password works and that you remember it before you use your wallet.
Look for the icon to turn turquoise indicating that you are online. You can click it to check status.

Done! Enjoy being anonymous!
Appendix 1 - Monitoring Tor
You can monitor (and further tweak/break) Tor using nyx.
There are several installation methods available. On Fedora 27:
sudo dnf install nyx

To start nyx simply type nyx in the console and it will connect to Tor if it is running. Requires control port 9051 to have been uncommented in /etc/tor/torrc.
Footnotes
There are more configuration options available, and additional ways you can support the Tor network. Please see the several pages available here for information.
*by default, Tor will participate in the Tor network.

probably best to run electrum with the -o offline switch the first time you run it so that your details don't leak out. that's also the time when you create your wallet and configure tor proxy.

@Abdussamad Yes, but once I run it with -o switch and I'm offline then the network settings in "Tools - >  Network" are not active and I cannot access them while being offline.


text:
How can I setup Bitcoin to be anonymous with Tor?
Bitcoin is billed as many things, among them its anonymity is highly regarded. While it is true that a transaction does not identify a user or wallet, recent news shows that there is the potential for privacy to be partially undermined, by an 'educated guess' method that may identify the IP address of a transaction originator.
How do I setup to use Bitcoin completely anonymously using Tor?

This is not a thorough schooling on Tor and only shows how to configure it to work together with Bitcoin Core.
Bitcoin Core includes Tor integration
When Tor is correctly setup on your system, Bitcoin Core automatically identifies Tor and creates an anonymous service. Little configuration is required to be 'off the grid' and, just a tiny bit more to be completely anonymous if that is important to you, with none of your Bitcoin traffic reaching out onto the public internet.
Using these steps you can be anonymous in only five minutes.
With the full privacy setup, transactions will of course still be broadcast but will only be broadcast actually onto the public internet by other Bitcoin nodes. With the standard 'off-the-grid' Tor setup, your Bitcoin traffic will be routed through the anonymous Tor network before reaching the public internet and other Bitcoin nodes on and off the Tor network to be effectively untraceable.
Setting Up Bitcoin Core and Tor
These instructions work on Fedora 23>29 and assume a default setup of Bitcoin Core v0.15.1 and Tor v0.2.7.1 or newer (and have been tested to work with Bitcoin Core v0.16.0 on Fedora 27 with Tor v0.3.1.9). Fedora is a modern operating system that will run on most standard modern hardware. The configuration is the same on Windows, but the instructions are different. There are some instructions for setting up Tor on Windows here.
Further instructions for other *nix based systems are available here. NOTE: You do not need to configure your Tor client as a relay or exit node for Tor to operate, so you can skip the step for 'Put the configuration file /etc/tor/torrc place:' in that guide. You will still need to use all of the following steps in this guide.

Setup Tor

Install the tor package:
   sudo dnf install tor

Start the tor daemon and make sure it starts at boot:
   sudo systemctl enable tor
   sudo systemctl start tor

Figure out where your torrc file is (/etc/tor/torrc is one possibility).

Open the torrc file to edit:
 xhost +local: ## skip if earlier than v29 only needed for Wayland
 sudo gedit /etc/tor/torrc

or
 sudo nano /etc/tor/torrc

Add these lines to your torrc (or ensure that they are uncommented):
 ControlPort 9051
 CookieAuthentication 1
 CookieAuthFileGroupReadable 1

You need to figure out what group tor is using. On Fedora 23 it is toranon. Run the following command:
 ps -eo user,group,comm |egrep 'tor' |awk '{print "tor group: " $2}'

You need to figure out what user bitcoind or bitcoin-qt is running as. Run the following command while Bitcoin is running:
 ps -eo user,group,comm |egrep 'bitcoind|bitcoin-qt' |awk '{print "Bitcoin user: " $1}'

Run the following command as root, which adds your Bitcoin user to the tor group. Replace TOR_GROUP and BITCOIN_USER with the actual information found above:
 sudo usermod -a -G TOR_GROUP BITCOIN_USER

If you don't modify any other settings, Bitcoin Core will usually connect over the regular Internet, but will also allow connections to and from the hidden Tor service.

So that Bitcoin Core wil only connect via Tor (for standard 'off-the-grid' setup), add these lines to bitcoin.conf. In Bitcoin Core, go to Settings -> Options -> Open Configuration File. Bitcoin Core uses Tor stream isolation by default:
 proxy=127.0.0.1:9050 #If you use Windows, this could possibly be 127.0.0.1:9150 in some cases.
 listen=1
 bind=127.0.0.1

(optional) If you like, you can add some onion service peer nodes to connect to. This will help especially if you do all of the following optional configurations. Add the following lines to your bitcoin.conf file. Bitcoin Core will only connect to a maximum of eight of these at any one time randomly, depending which ones are online (Starting with version 22.0, Bitcoin Core only supports Tor version 3 hidden services (Tor v3). Tor v2 addresses are ignored by Bitcoin Core and neither relayed nor stored. A list of Tor v3 addresses can be found here):
 #Add seed nodes
 seednode=wxvp2d4rspn7tqyu.onion
 seednode=bk5ejfe56xakvtkk.onion
 seednode=bpdlwholl7rnkrkw.onion
 seednode=hhiv5pnxenvbf4am.onion
 seednode=4iuf2zac6aq3ndrb.onion
 seednode=nkf5e6b7pl4jfd4a.onion
 seednode=xqzfakpeuvrobvpj.onion
 seednode=tsyvzsqwa2kkf6b2.onion

 #And/or add some nodes
 addnode=gyn2vguc35viks2b.onion
 addnode=kvd44sw7skb5folw.onion
 addnode=nkf5e6b7pl4jfd4a.onion
 addnode=yu7sezmixhmyljn4.onion
 addnode=3ffk7iumtx3cegbi.onion
 addnode=3nmbbakinewlgdln.onion
 addnode=4j77gihpokxu2kj4.onion
 addnode=546esc6botbjfbxb.onion
 addnode=5at7sq5nm76xijkd.onion
 addnode=77mx2jsxaoyesz2p.onion
 addnode=7g7j54btiaxhtsiy.onion
 addnode=a6obdgzn67l7exu3.onion
 addnode=ab64h7olpl7qpxci.onion
 addnode=am2a4rahltfuxz6l.onion
 addnode=azuxls4ihrr2mep7.onion
 addnode=bitcoin7bi4op7wb.onion
 addnode=bitcoinostk4e4re.onion
 addnode=bk7yp6epnmcllq72.onion
 addnode=bmutjfrj5btseddb.onion
 addnode=ceeji4qpfs3ms3zc.onion
 addnode=clexmzqio7yhdao4.onion
 addnode=gb5ypqt63du3wfhn.onion
 addnode=h2vlpudzphzqxutd.onion
 addnode=n42h7r6oumcfsbrs.onion:4176
 addnode=ncwk3lutemffcpc4.onion
 addnode=okdzjarwekbshnof.onion
 addnode=pjghcivzkoersesd.onion
 addnode=rw7ocjltix26mefn.onion
 addnode=uws7itep7o3yinxo.onion
 addnode=vk3qjdehyy4dwcxw.onion
 addnode=vqpye2k5rcqvj5mq.onion
 addnode=wpi7rpvhnndl52ee.onion

If you additionally want Bitcoin Core to only connect out to Tor hidden services and not even to connect to IPv4/IPv6 nodes on the public internet via the Tor network proxy:

(optional) Also add this to bitcoin.conf for full anonymity (not particularly recommended)*:
onlynet=onion

*Note: Bitcoin Core will still query for peer addresses via DNS lookup if low on addresses. This also can be disabled using the next option. However, it is possible your node may not be able to find any other nodes to connect to.
*Note: Bitcoin Core v0.15.1 currently seems to make some outbound IPv4 connections at node startup even when onlynet=onion, none have been observed after initial startup. These connections should be made via your onion proxy, however, using the next option has been observed to prevent them.

(optional) (advanced) If you also want to disable DNS lookup to query for peer addresses then also add the following to bitcoin.conf (not particularly recommended) note: if you use this option your node may be unable to find peers until you add some good peers with the addnode= parameter.:
dnsseed=0
dns=0

Restart tor:
sudo systemctl stop tor
sudo systemctl start tor

Log out of your user, log back in (this is so that your new user group permissions are effective, I do not know what user you are running Bitcoin Core on).

Restart Bitcoin Core. Since Tor version 0.2.7.1 and newer the Bitcoin Core GUI version called bitcoin-qt automatically registers your Tor hidden service and makes it reachable on the onion network. For the command line version of Bitcoin Core, bitcoind, add the following parameter to your command line:
>bitcoind -listenonion

No port forwarding is necessary for everything to work with Tor including incoming connections via the Tor hidden service, you do not need to forward any ports for Bitcoin Core or Tor for this.
If you want your Bitcoin node still publicly reachable via the public internet for incoming connections you will still need to forward port 8333 for Bitcoin Core.
Checking everything is working
There are only two things to check that all is working. Checking peer info in the debug window of bitcoin-qt, you should see that connections to IPv4/IPv6 peers now have some extra connected 'via' info along with the peer address when you click on a peer. Onion addresses only route via Tor.
Checking the same thing via console or CLI for getnetworkinfo, you should see for each network type the proxy info and, checking with getpeerinfo you should see that the addrlocal info is a remote address for each peer. Onion peers do not have addrlocal and just have their onion service name for addr.
The second thing to check is that your onion service for inbound Tor connections is up and all configuration is in place. Have a look in your debug.log file, you should see a few entries after the most recent node restart that match the following:
2018-02-10 06:31:48 InitParameterInteraction: parameter interaction: -proxy set -> setting -upnp=0
2018-02-10 06:31:48 InitParameterInteraction: parameter interaction: -proxy set -> setting -discover=0
...
2018-02-10 06:32:13 Bound to 127.0.0.1:8333
...
2018-02-10 06:32:13 torcontrol thread start
2018-02-10 06:32:13 tor: Got service ID {onion}, advertising service {onion}.onion:8333
2018-02-10 06:32:13 AddLocal({onion}.onion:8333,4)

The advertising service information is your onion service address.
In the debug.log, connections to onion peers will only look like the following but still show up in the peers tab of the debug window on bitcoin-qt:
2018-02-10 06:34:07 receive version message: /Satoshi:0.15.1/: version 70015, blocks=508469, us=[::]:0, peer=7

It is not necessary to configure port forwarding on your modem/router for Tor to operate. If you are behind a restrictive firewall it may be necessary to configure outbound connections to allow Tor to connect out to other Tor nodes. Tor can be configured to only connect out using port 80/443 if that helps. See Appendix 1 - Monitoring Tor for nyx and access to full Tor configuration options.
It is difficult to be completely anonymous since the sender and the receiver know, however, you can obfuscate your transaction origin so that your data cannot be traced by IP address without breaching the Tor network. Do some research, onlynet=onion is more secure.
Additionally, there has been research[1][2] done on graphing the blockchain in an attempt to trace all BTC to their origin, potentially identifying source<-wallet<-purchase and depending on the combination of UTXOs potentially identifying wallet balances or wallet balance subsets. Data linkage is a privacy issue we may all be concerned about, this article discusses the use of bitcoin mixers, and this series of tweets.
Done! Enjoy being anonymous!
Appendix 1 - Monitoring Tor
You can monitor (and further tweak/break) Tor using nyx.
There are several installation methods available. On Fedora 27:
sudo dnf install nyx

To start nyx simply type nyx in the console and it will connect to Tor if it is running.
Footnotes
Once correctly configured, most synchronisation issues are to do with your hardware. See this answer for more information.
There are more configuration options available, and additional ways you can support the Tor network. Please see the several pages available here for information.
Thanks to en.bitcoin.it for your excellent guide that got me started on this.
Additional information is available from the bitcoin project here.
For an even higher level of anonymity, it is possible to configure Tor as a DNS resolver and, configure your system network configuration to use Tor to resolve DNS queries.
*by default, Tor will participate in the Tor network.

Waiting for peer review before marking as accepted.

If you aren't extremely careful with dns configuration, it can betray the fact that your wallet is communicating with tor-based wallet nodes.  https://tor.stackexchange.com/questions/8/how-does-tor-route-dns-requests

@MaxVernon Okay, appreciate the risk. This answer seems to indicate that Tor handles DNS resolution for requests through the exit node. However, if the client directly resolves DNS I doubt this is handled.  I will need drag out my old network log box to test what is actually happening. Do you think I should re-word the answer? Still, all transaction information is only relayed by Tor so the objective is achieved. Even if it is possible to identify a particular IP is running a Bitcoin wallet and/or Tor the data traffic is unidentifiable.

@MaxVernon added the option to disable DNS lookups.

Where did you get this list of tor nodes?

Great guide, thanks! Note that bitcoind saves a file onion_private_key in its home folder (default ~/.bitcoin/). As long as that file is in the bitcoind data folder your *.onion address will remain the same, across restarts and reboots. Deleting that file will force bitcoind (or tor) to generate a new onion_private_key and thus a new *.onion

@MrHash That list comes from the FBI ;-)  You can just grab your own list from bitnodes if you like.  The ones at the end have the best uptime.

On ubuntu the tor group command returns 'root' and 'debian-+', but neither of these are groups on the machine, so I can't add the user to these groups.  Not sure what to do.

@wordsforthewise you should be able to list groups with something like cut -d: -f1 /etc/group | grep toranon if group toranon exists use that. It may be the group is debian-tor?

Could someone elaborate on the error I get: Issue: Authentication failed: unable to read '/run/tor/control.authcookie' ([Errno 13] Permission denied: '/run/tor/control.authcookie'). This happens when I try to run it as a normal user. It works fine with sudo but then I get: Nyx is currently running with root permissions. This isn't a good idea, nor hould it be necessary.

@Ben If you have completed step 7 correctly it should work, unless something is changed on a newer version of Tor I will check it eventually.

There has been some discussion today before the Bitcoin Core PR review club session on the subject of Tor v3 support. I haven't been through the above answers to comment on whether any of it needs updating.
But Jon Atack pointed out on IRC some good additional resources on getting started with Tor. There is the Bitcoin wiki and the Bitcoin Core documentation.
He also explained how you can build and run a Tor v3 service using this draft PR 19954 (draft at the time of the writing, September 2020).
If you have Tor configured and running, Bitcoin will automatically create a Tor service even with the default settings. You can get debug.log info on your Tor status by setting debug=tor in your bitcoin.conf file or by launching bitcoind -debug=tor
You can observe your node with:
MacOS
./src/bitcoin-cli -netinfo 4

Linux
watch ./src/bitcoin-cli -netinfo 4

Here is a simple guide on how to build and run your own home node behind a tor.
https://github.com/bitembassy/home-node

There is a pretty straight forward way to get system-wide TOR for arch linux. It might be worth to setup an encrypted arch linux VM with TOR and then proceed to use bitcoin-core from the AUR.