text:
Why keys generated in Bitcoin Core mismatch that from online generators despite using the same seed?
I have the following problem in Bitcoin Core. With this command:
echo carpet begin bacon master draft fortune food cherry cage axis vault clown | \
bx mnemonic-to-seed | \
bx hd-new -v 70615956 | \
bx hd-to-ec -c bx-testnet.cfg | \
sed 's/$/01/' | \
bx base58check-encode -v 239

What I am doing is from the mnemonic carpet begin bacon master draft fortune food cherry cage axis vault clown, I am generating an hdseed for Bitcoin Core in regtest. Results:
cNQQqyR81GZXU3Pa1pK1gQQp6jgni5HyLYM99nL2JpzVRmdCyrFE

Then I use the command:
bitcoin-cli -regtest sethdseed true cNQQqyR81GZXU3Pa1pK1gQQp6jgni5HyLYM99nL2JpzVRmdCyrFE

Where I change the hdseed for which I just got you and Bitcoin Core generates a key pool. When I put that same mnemonic on the page BIP39 it gives me as root key the same that gives me the command up to this part:
echo carpet begin bacon master draft fortune food cherry cage axis vault clown | \
bx mnemonic-to-seed | \
bx hd-new -v 70615956

which is: tprv8ZgxMBicQKsPd5cY6ZZzsRzkcfDANp2y3jFtYNQZsp953sXiFNP9ZFod1fAhkgtCJZGvTcLJVdhFCj7VDNu68nuP4m2QGLcd4JpkteG8Ccc
However, the derivations of private keys generated by Bitcoin Core do not coincide with those of BIP39, nor does the BIP32 root key of BIP39 coincide with the private masterkey of Bitcoin Core, while the hdseed if it matches the one that happens in the command. Both the derivations, hdseed and the Bitcoin Core master key are obtained with the command bitcoin-cli -regtest dumpwallet test_wallet
I get two big unknowns:

Is the way in which I am generating the hdseed from the mnemonic correct?
Bitcoin Core generates its private master key from hdseed? And if so, is that operation reversible?

But in general I want to know why they are giving different!!!
Thanks in advance.

I want to know why the two tools are giving different results

Bitcoin Core uses hardened derivation to derive child private keys from the master private key. You can check the embedded link for more details, but in summary it means that the software uses parent private keys within the hash function to generate child private keys. This compares to non-hardened derivation which uses parent public keys in the hash function. Since Bitcoin Core uses hardened derivation while the online tool that you mentioned uses non-hardened derivation (and also different paths), there is a natural mismatch in the keys generated by the two of them.
Bitcoin Core generates private keys using the hierarchical deterministic key generation using the path m/0'/0'/k' where k is the kth key generated from the master private key. Most wallets use the path m/44'/0'/0'/0/k for generating the kth receiving key.

Bitcoin Core generates its private master key from hdseed? And if so, is that operation reversible?

HD seed is used to generate the master private key using a hash function. So the operation is only one way and cannot work the other way.

Is the way in which I am generating the hdseed from the mnemonic correct?

It looks to be correct but on other aspect, the mnemonic that you provided in the question is not valid (I'm unsure if it was intentional) as the checksum does not match up with the entropy.

BIP-39 works with hardened derivation. Calculate the derived private key with m / 0 '/ 0'  and then derive the keys

@wic it can, but the paths are different.

I use bx mnemonic-to-seed command to verify mnemonic and it is correct, because if it is not correct, it return exception


text:
How 1 cpu 1 vote works?
How 1 cpu 1 vote works? What is its validity? Does it use IP address or mac address or similar to that? If yes, then it can be changed and can make fool to the network. If it can't, then how is it preventing any malicious activity?

one-cpu one-vote is about Proof of Work. Except it really is not a vote and "votes" aren't being recorded as to which CPU they came from.
What it really means is that if you participate in mining (so you are performing Bitcoin's Proof of Work), you have the ability to decide which blockchain is "the blockchain" in the even that there are multiple valid blockchains. How this works is that a miner chooses a block to build their block on top of, and they dedicate their computing power to extend that blockchain. The blockchain that "wins the vote" is the blockchain that has the most cumulative Proof of Work, which is generally the one that the most miners have decided to mine on top of.
There is no way to "fool the network" because it isn't a vote. Nothing is broadcast to the network except complete and valid blocks.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Does merkle root contain hashes of transactions from previous blocks?
Every block in a blockchain contains merkle root value. So for any block, there is some list of transactions and you create a merkle tree based on these transaction.
For other blocks, same process will repeated.
My question is:
Let us assume that A, B,C,D are transactions and we create a merkle tree and we add the merkle root in block. Now later we got the transactions E,F,G,H so:

whether merkle tree is based on E,F,G,H only
or it is based on A,B,C,D,E,F,G,H .

I want to know that for some block, we created a merkle root value based on A,B,C,D. For others block whether we include previous transaction with new transactions  and create a merkle tree. Or with only new transactions, merkle tree is created.

The blockchain is the mechanism for Bitcoin to converge on one shared state. Participants submit payment orders in the form of unconfirmed transactions to the network from which miners then select a set of transactions into their block candidates. The Merkle root hereby represents a commitment to one specific list of previously unconfirmed, valid transactions. When a new block gets authored, all network participants validate the block and then adopt the new set of balances as the current network state.
Beside the Merkle root as a commitment to this block's transactions, the block header also commits to exactly one preceding block as its parent by including the hash of that predecessor block. As the predecessor in turn committed to it's own predecessor, a new block therefore inductively commits to the full history since the Genesis Block.
Given that each preceding block in turn contains a Merkle root with which it committed to its own set of transactions, any new block also commits indirectly to the full history of all transactions.
Coming back to your example, this means that if Block#1 committed to {A, B, C, D}, the Merkle root in Block#2 would only commit to {E, F, G, H}. However, Block#2 commits to {A, B, C, D} indirectly by declaring Block#1 it's predecessor.

Thanks for this valuable information. It is really helpful for me.

For every new block there is a new Merkle tree of transactions. Once the block is included in the blockchain the Merkle tree (and hence the Merkle root) for that block is not changed. In your example, if you are a miner trying to mine a block you can choose to continue with transactions A, B, C and D or form a new Merkle tree with A, B, C, D, E, F, G and H. The larger the transaction fees on E, F, G and H the more likely the miner will switch from trying to mine a block with A, B, C and D to trying to mine a block with A, B, C, D, E, F, G and H.
The key point here is that Merkle trees of transactions cannot be changed once included in the global blockchain but prior to that they can be changed locally as frequently as you wish.

Merkle root is block specific, so each block has a different merkle root. If transactions A-H belong to the same block, then you will have to calculate the merkle root using all the transactions from A-H, if not you will have merkle root containing transaction A-D in Block N and merkle root containing transactions E-H in block N+1.
Blocks commit to the previous block using previousblockhash, which is the double SHA-256 hash of the header of the previous block. Header of a block includes merkle root, previous block hash, timestamp amongst other fields. So by using the hash of the previous block's header in the current block's header you are in essence indirectly committing to the transactions that were included in the previous block thus forming a chain.

It means that if we collect some transactions at some time, let say  T(1) than Merkle root based on these transactions. After some time let say T(2) if we collect some other transaction then Merkle root based on The transactions  which we collect at time T(2). Previous transactions are not included in the current Merkle tree to get the Merkle root value.

@SanjeevDwivedi Yes that is correct, if you meant transactions in a block. There are unconfirmed transactions that are not included in the blocks (fee used may be low etc.) that will not form the part of the merkle root in any block. The merkle root does not contain transactions from previous blocks. But the use of previous block header hash in the header of the current block indirectly commits to the transactions in the previous block.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to prevent double spending Bitcoins?
If I create a transaction, my node will check my UTXO's and if i have enough BTC to do this transaction it will be mined.

So let's say I want to spend 12 BTC to Anna.
The System will check if i have enough BTC and i do.
But the UTXO list will not be updated until my transaction is mined.
So what will happen if i do another transaction with 15 BTC to anna.
Would the System check the UTXO's again and accept the new transaction?
Or is there something like a second UTXO list with the spended but not mined Transactions?

When a transaction is created by your node and relayed to full nodes, they will check to ensure that the transaction is valid (no double spending is one condition amongst them). Full node maintains a set of UTXO list in the chainstate folder and is aggressively cached in memory. If a transaction is valid, the full nodes will remove the UTXOs that are consumed in the transaction and add the outputs of that transaction to the UTXO set. Then the node will add that transaction to the mempool and relay the transaction to other full nodes connected to it.
Most full nodes/mining nodes operate on first seen basis. That is if they see another transaction spending the same UTXO as to the first one, they will discard it as invalid and will not relay it further (unless it is RBF). So if you send two transactions consuming the same UTXO then it might be the case that half of the nodes will use a different transaction in the mempool as compared to the other half. This issue will be resolved when a block is mined containing one of the two transactions. The miner will only include first transaction that it has seen in the block it mines. When the full nodes receive the mined block, the ones that had seen the other transaction will roll back the changes to the UTXO set and converge on the transaction mined in the block.
If you want to include the other transaction, you will now have to mine two blocks: one at the height that was just received (by removing the mined transaction and including the other) and another block incrementing that height so that nodes accept that version of the chain (since nodes accept the longest PoW chain). That is the reason PoW is very important in resolving conflicts in a decentralized P2P system that does not have a central authority to dictate which transaction is the valid one.

Thanks for that awnser! I thought a UTXO would only been changed if a block is mined. I'm trying to create a visualization of my own programmed Bitcoin Blockchain in Java. So i have to update the UTXO list everytime a transaction is added.
And if another Block uses other UTXO's for that transaction and mined it faster i have to revert the UTXO's and use these of the mined Block.
I cant give u a upvote cause im to new, but thank you very much.

Yes, it is better/faster to update the UTXO list on the fly. You can reject double spending transactions outright. But when a block is received with almost 2000 transactions, you don't have to download and verify each of them all at once. You must have seen a majority of the transactions and must have applied the changes. There might be a few that you haven't seen so you can just update the UTXO set those newly seen transactions and the other conflicting ones.

One last Example: I have 13 BTC and send 5 btc to Anna, Node 0 will update his UTXO list. Anna Spends this 5 UTXO and sends them to Tim and Simon. And they spend them too, but the block is not mined yet. Node 100 got a transaction where i spend 10 BTC and send them to Peter. Node 100 mines the Block with the transaction immediately and sends the block to Node 0. So Node 0 has to undo every affected transaction untill i have my 13 BTC to send them to peter?

@TomS That is correct. One UTXO can only be once. When the block got mined with you sending the 10 BTC UTXO to Peter, it was considered final. So node 0 discarded all transactions and its children transactions that consumed that UTXO.


text:
Could any one show me how to get pubkey in Bitcoin Core?
I am new in Bitcoin Core, and I want to get my public key. Could anyone please show me how to get the pubkey?

Try:
$ bitcoin-cli getaddressinfo '<your address>'
{
  "address" : "address",        (string) The bitcoin address validated
  "scriptPubKey" : "hex",       (string) The hex-encoded scriptPubKey generated by the address
  "ismine" : true|false,        (boolean) If the address is yours or not
  "iswatchonly" : true|false,   (boolean) If the address is watchonly
  "solvable" : true|false,      (boolean) Whether we know how to spend coins sent to this address, ignoring the possible lack of private keys
  "desc" : "desc",            (string, optional) A descriptor for spending coins sent to this address (only when solvable)
  "isscript" : true|false,      (boolean) If the key is a script
  "ischange" : true|false,      (boolean) If the address was used for change output
  "iswitness" : true|false,     (boolean) If the address is a witness address
  "witness_version" : version   (numeric, optional) The version number of the witness program
  "witness_program" : "hex"     (string, optional) The hex value of the witness program
  "script" : "type"             (string, optional) The output script type. Only if "isscript" is true and the redeemscript is known. Possible types: nonstandard, pubkey, pubkeyhash, scripthash, multisig, nulldata, witness_v0_keyhash, witness_v0_scripthash, witness_unknown
  "hex" : "hex",                (string, optional) The redeemscript for the p2sh address
  "pubkeys"                     (string, optional) Array of pubkeys associated with the known redeemscript (only if "script" is "multisig")
    [
      "pubkey"
      ,...
    ]
  "sigsrequired" : xxxxx        (numeric, optional) Number of signatures required to spend multisig output (only if "script" is "multisig")
  "pubkey" : "publickeyhex",    (string, optional) The hex value of the raw public key, for single-key addresses (possibly embedded in P2SH or P2WSH)
  "embedded" : {...},           (object, optional) Information about the address embedded in P2SH or P2WSH, if relevant and known. It includes all getaddressinfo output fields for the embedded address, excluding metadata ("timestamp", "hdkeypath", "hdseedid") and relation to the wallet ("ismine", "iswatchonly").
  "iscompressed" : true|false,  (boolean, optional) If the pubkey is compressed
  "label" :  "label"         (string) The label associated with the address, "" is the default label
  "timestamp" : timestamp,      (number, optional) The creation time of the key if available in seconds since epoch (Jan 1 1970 GMT)
  "hdkeypath" : "keypath"       (string, optional) The HD keypath if the key is HD and available
  "hdseedid" : "<hash160>"      (string, optional) The Hash160 of the HD seed
  "hdmasterfingerprint" : "<hash160>" (string, optional) The fingperint of the master key.
  "labels"                      (object) Array of labels associated with the address.
    [
      { (json object of label data)
        "name": "labelname" (string) The label
        "purpose": "string" (string) Purpose of address ("send" for sending address, "receive" for receiving address)
      },...
    ]
}

See https://bitcoincore.org/en/doc/0.18.0/rpc/wallet/getaddressinfo/


text:
Double Spend Behavior on First Network Propagation
Supposing there is a double spend, call the transactions T1 and T2. If someone propagated both transactions within a short space of time to the network, of course only one of them can make it into the next mined block. Do we know anything about the probabilities of which of T1 and T2 will make it, or is it 50/50? Obviously from block 2 onward the longest chain is most likely to remain. But for the very next block, do we know anything?

Bitcoin is a gossip network and relays transaction or blocks on a best effort basis. Hence you cannot determine the probability of one transaction being seen by the network versus the other.
Now in terms of mining the transactions in a block there are a couple of cases that needs to be considered. For assumption sake let us assume that we are considering a miner's node that first receives T1 and then after a few seconds receives T2.

Default case: Most miners still manage their mempool on a first seen basis. That means if a miner sees T1 first, it might reject T2 even if the fees provided by T2 are higher than that of T1.
Replace by Fee (RBF) is signaled: If T1 had opt-in RBF signaled (nSequence in any one its input is less than 0xFFFFFFFE) then the miner may replace T1 with T2 from its mempool.

Now the only remaining conflict can be if there are two blocks mined at the same time and at the same height, one containing T1 and the other containing T2. The chain version on which the next block gets mined will finally determine which transaction will attain practical finality.


text:
What are the consequences from the leak of xpub and child private key?
Let's say the following keys are leaked:

Extended public key derived from path 44'/60'/4'
Leaked child private key from path 44'/60'/4'/0

Does that imply all the child private keys of the root 44'/60'/4' can be computed? Looking and reading at the BIP32 and BIP44 - that doesn't seem to be the case. But wanted to confirm.
Also, if we know that it doesn't cause the sibling keys to be derived, then can I go and safely use the following public key of path 44'/60'/4'/1 without any worries?

If the above two keys are leaked, then your master private key at the account level (m/44'/60'/4') can be back-calculated. That means, any private key (both receiving and change) that you derive from that account is compromised.
To elaborate, BIP 44 uses non-hardened derivation to derive private keys after the account level. The child private key derivation equation is kchild = kpar + hash(Kpar, cpar, i). Here k represents private key, K represents public key, c is chain code and i is the index. Now, since your xpub at m/44'/60'/4' is leaked, the attacker has Kpar (first 256 bits of xpub) and chain code, c (latter 256 bits of the xpub). Since your child private key (m/44'/60'/4'/0) is also leaked, that means the attacker has kchild. Using the above equation the attacker can simply back-calculate kpar. Even if he doesn't know the index number (0 in your case), the attacker can simply run an iteration (just a space of 231 to exhaust).
Since your parent private key at the account level is compromised, you cannot use any addresses derived from the m/44'/60'/4' account unless you want to lose your bitcoins. However, you can safely use m/44'/60'/5' or other accounts.


text:
How public-key cryptography secures bitcoin transactions from 51% attacks and Mining Attacks
I wanted to better understand how public-key cryptography avoids 51% attacks and mining attacks, also if this question is wrong please correct so that i can improve my research and provide me with right resources

As Ugam said these two are not related. You may want to check out the tag [majority-attack]: https://bitcoin.stackexchange.com/questions/tagged/majority-attack?tab=Frequent

Public key cryptography has nothing to do with 51% attacks. The only thing a miner can do with >50% hash power is double spend bitcoins for which he controls the private key or censor transactions by not including them in the blocks he creates. He cannot spend bitcoins for which he doesn't control the private keys.


text:
What makes Bitcoin transaction secure against attackers/hackers?
Wanted to know what is it that makes bitcoin transaction secure and make it less vulnerable to attackers and hackers

Bitcoin transaction works on low-level primitive constructs of scripting languages and cryptography. In simple terms, you lock your funds to an equation and anyone who can provide a solution to that equation can spend the money. So there are no account level data, just unspent transaction outputs (UTXOs). Every time you need to spend your bitcoins, you just include these UTXOs as inputs to your transaction, provide a solution to the locking equation and send your funds to another locking equation (which can then be spent by providing solution to that equation). Most of this unlocking equations involve asymmetric cryptography of providing signature from your private key.
The most important aspect of Bitcoin transactions is that every information related to a transaction is entirely public. From the moment, you sign your transaction with your private key and broadcast it, the entire data of that transaction becomes a part of the public record (be it in the mempool before confirmation or in the blocks after confirmation). Since the signing process of Bitcoin transactions involve signing the entire transaction data as the message, any person in between cannot change an iota of information as it will render the transaction as invalid (let's leave signature malleability alone for now). This makes transactions secure against attackers who might want to modify the data.
The only thing that needs to be kept secure are your private keys. As the old adage goes, "Not your keys not your funds". Thus, care has to be taken to ensure that you keep your keys safe. Most of the attacks that you read about involving lost bitcoins is because of careless management of keys. You can use cold storage like hardware wallets, wherein your private keys will never enter a network connected device and as such will remain immune to network attacks.
The private key associated with signing the transaction is a random number in the 2256 bit key space. From the private key, you generate a public key using elliptic curve multiplication. This process is one-way - that is you cannot get private key if you know the public key - unless you solve the discrete log math problem or you brute force (try every combination of private key to get the resulting public key). No one has found a solution to the first case, and the second case is impossible due to the energy requirements. Moreover, the public key is hashed using RIPEMD160 and SHA256 hashing functions to generate a bitcoin address to which you send the bitcoins. These hashing functions are also one way. Now, the solution to the equation (in many cases) I was talking to is to provide the public key that hashes to the address and a private key whose signature would be verified using that public key. So if you do not re-use your address, you have a 2-layer security from private key to the address.


text:
Find sender's public key in SegWit Transaction
How can I retrieve the sender's public key in a Segwit transaction?
For example: https://www.blockchain.com/btc/tx/3179d18d6f36fa77b88909496535485d8188d96b79d295843789a9e0ff6c3e6a
I can happily parse the witness part as in
02 : 2 items to follow
47 : 0x47 = 71 bytes (signature)
304402201cf8db0c4afc164970ec4397327fe9b1dd9b7ff4a9093f94e554d624b5ffdcb702202848072c17f2bcce16b8d3d7cb2efdad87bc6c942b79f44b79f1c52f817d81ea01 : the actual signature (last byte is the sighash code)
21 : 0x21 = 33 bytes (the pubkey)
02384052a5ecde83bf8ee7ed77f378edb58aa65de22c4e91af87eee68015b9d509 : the actual pubkey
But this pubkey is not the public key of the sender. It's probably the pubkey of a segwit (or something like that)...
How exactly is this signature made up and what pubkey is that? And more importantly, how can I retrieve the sender's public key (the one belonging to 35yfMa3CRBiWny8DFdb4tUu9fn7fcdvVp9 in this example).

“public key of a segwit”?

You have no idea who the sender is (in the sense that you can't just send coins back to that key and expect them to arrive at the sender's wallet). But it is the public key corresponding to the private key which previously controlled the funds. Segwit has nothing to do with this.

The public key that you mentioned is indeed the key that was used to derive the P2SH(P2WPKH) address: 35yfMa3CRBiWny8DFdb4tUu9fn7fcdvVp9. The way in which a P2SH(P2WPKH) address is derived is as follows:
1. witness_script = hash160(pub_key) #this is equal to '4b9d2d3dd1174ad656754a0c664e7a129b131f3b'
2. witness_version = 0x00  #current SegWit version
3. scriptSig = witness_version + OP_DATA(0x14) + witness_script
#Above is equal to '00144b9d2d3dd1174ad656754a0c664e7a129b131f3b' and is the one you are seeing in the scriptSig in the explorer.
#0x14 tell the script to push the next 20 bytes to the stack
4. public_address = hash160(scriptSig)
5. bitcoin_address = base58check(public_address) #with prefix of 0x05
result = 35yfMa3CRBiWny8DFdb4tUu9fn7fcdvVp9

Native SegWit transactions works by locking an output with scriptPubKey: version + OP_DATA (bytes to push) + witness_script. For older clients, this looks as if it is any one can spend transaction, as it does not have an opcode verifying anything. So in order to take advantage of lower fees of spending SegWit transaction, but at the same time be compatible with wallet software that are not SegWit aware, we use the scriptPubKey as the script and create a P2SH locking script out of it. So the locking script is: OP_HASH160 <public_address> OP_EQUAL. (public_address convention is same as the one I mentioned when deriving the address above). So a customer can send you bitcoins without upgrading his wallet software to be SegWit aware, but when spending the bitcoins, you can take advantage of lower fees of SegWit.
Now, in terms of verification when this output is spent in a transaction: Older clients verifying the transaction will just look at the scriptSig, take its has160, verify with the public_address and consider this transaction to be valid. Newer clients will realize that the scriptSig is in itself a SegWit and will look for signatures in the Witness part of the transaction. There the clients will verify that the hash160 of the public key is equal to the witness_script and then the signature matches the public key when signed with the transaction as a message. The signature message digest is described in BIP 143.


text:
Is the "recovery phrase" generated by a wallet app platform independent?
I recently started a wallet with Coinbase Wallet, and I was asked to backup a 12 word "recovery phrase" that is supposed to be able to recover my wallet in the event that I delete the app from my phone. I have previously used other wallet apps, which often give "recovery phrases" of varying word count. I read online that the phrase is passed through an algorithm and used to recover a key or a group of keys.
I am now curious if these recovery phrases are platform independent (meaning you can recover your keys just by way of knowing the phrase), or if the app must exist with a specific algorithm in order to recover keys (your keys are lost if the recovery implementation is destroyed, e.g. Coinbase shuts down operation and removes their app).

The mnemonic words that is given to you by the wallets is a representation of entropy. This entropy is then passed through PBKDF2 hashing function to generate seed. The industry standard for this conversion of mnemonic to seed is documented in BIP 39 and most wallets follow that method. The seed is then passed to HMAC-SHA512 to get a master extended private key.
Until this point the implementations are standard and you are unlikely to face any issue with different wallet implementations. But after this step, the derivation gets a bit trickier. There are a number of methods to go from here. You could use BIP 32, BIP 44, BIP 49 or the BIP 84 derivation methods to generate individual private keys from the master private key. Then there will be distinction between hardened and non-hardened derivation between different wallets. Each derivation path will give you a completely different set of keys as compared to other paths.
There is a spreadsheet tracking these derivation paths of numerous wallets here (please verify the data yourself as it may be dated or incorrect). I have pasted the derivation paths of different wallets below from the linked spreadsheet.

So, when using a wallet you should note the derivation path the wallet followed in order to recover it later. Also note that use of a passphrase along with the mnemonic words will generate a completely different set of keys as compared to only using the mnemonic words.


text:
Question regarding LN node backups
I have LN node running on c-lightning. I am performing ~/.lightning dir backups time-to-time.
Let's say after backup i open few new channels and withdraw rest of onchain funds to my electrum. Then i restore a backup.
What will happen to my node and its channels?
P.S. i have tried such scenario once. Onchain funds status after backup restore become 'spent'. i had to launch 'lightning-cli dev-rescan-outputs' to refresh onchain funds and get actual status. While my node lists channels in 'listfunds', i see that those channels become inactive. What is happening under the hood? what is the correct way of making and restoring node backups?

As pointed out in similar questions backups are tricky with lightning. In the case of lightning each payment channel is a 2 of 2 multisig wallet. One of the keys is in your possession and it differs for each channel but in generally is derived from your master key.
The masterkey in c Lightning is 32 byte and stored in binary in ./lightning/hsm_secret
Unluckily it is not sufficient to backup your masterkey. Every channel has a so called state and an old state of the channel will yield problems (potentially a forceful close of the channel partner and them being able to claim your funds.)
So here is what should happen :
If you turn off your node and make a backup of your lightning dir you will be able to restore that backup on this or a different machine and after turning on your lightning node everything should work as before.
If however you change channel state before restori g your backup your channel partner and you will most likely see a mismatch and fail the channel which means they publish their latest state. If you are lucky it is your partner and you can claim your outputs. If not it is your node and your partner will probably throw in their penalty tx to claim all coins of the channel due to the channel breach.
Generally under the hood after restart of a lightning node all peers are being connected again. If that does not work (because the other node might be offline) there will be inactive channels.

It's worth pointing out that state changes happen automatically, e.g., fee changes and HTLC forwarding, so it should never be considered safe to restore if the node had been running inbetween. Shutting down, migrating, the restarting is ok, put one must be absolutely sure the node hadn't been running in the interim.
With option_dataloss_protection, now enabled with all up to date implementations, no node should ever inadvertently cheat, but at that point you're at the mercy of the counterparty (see criticism of LNDs SCB scheme).

After restore i have now about 20 inactive channels. I can connect to almost every node (some nodes went offline) but this doesnt restore channel state to active even after 15 minutes. i have a bunch of inactive channels but what to do now? i cannot close these channels, i am getting "this channel is inactive". Both sides are online. as i understand there is some timeout which will be reached and then channel will auto-close? am i right?

It seems that i got expensive lesson about backups :) channels now are closing, i see that onchain funds appear, but it looks that i loosed already about 15% of funds that was on channels. We'll see what happens next, i have still about 50% offchain on inactive channels. What keywords should be searched in logs to see was there claim for all funds on other side or not?

Interesting fact, but almost all my funds appeared few days later after running dev-rescan-outputs (lasted quite long). Anyway, i think this is lucky case and you should not run 'old backup' version in online mode if you have chances to recover the last one node state. In my case i was doing exactly wrong way and ran 'last_state" then 'prev_state' then again 'last_stait'. On second step i had many messages in logfile about 'Recovering funds!' events, so you should not allow to happen this for best experience. anyway, i am happy that protocol itself is pretty fail-proof.

Could be that your node detected problems, failed the channel with force closing newest state. In that case there was a timelock on your funds. Your lightning node keeps track of them and redeems them after the timelock this could be the reason why funds did not appear directly


text:
Bitcoin-Core re-synchronizing after updating to v0.18.0
I recently updated Bitcoin-Core to 0.18.0 and then it had to resync the blocks which has taken 8 days so far and is only on 67% completed (using windows 7). I have the files saved in D:/.. I noticed the config file was blank. What should be written in there in a standard case?
I did try to install node launcher but was to technical for me, but after I tried this I noticed I had another bitcoin folder in F:/ which I have since deleted.

What is a "node launcher"?

What did you update from?  A blank configuration file is fine.

By default, Bitcoin Core does not create a configuration file. If you want to change configuration options, you can either pass startup parameters or add the bitcoin.conf yourself.

I updated from the previous version 0.17, I have just realised that my IP address I used for port 8333 in router had reassigned its self,so this is why I had no incoming connections,  but it’s still processing v slowly, increase per hour is 0.02%. The number of blocks left is unknown and says is syncing headers?

@MarkRichards: Bitcoin Core will first download all headers of the blocks to determine the best chain, then download and verify the complete best chain. It sounds like you're still in the first phase. Is this a microcomputer?

Ah ok thanks, so would say I still have a way to go, I  am just using an old laptop around 10 years old

v0.18.0 release notes include:

systemd init file
The systemd init file (contrib/init/bitcoind.service) has been
  changed to use /var/lib/bitcoind as the data directory instead of
  ~bitcoin/.bitcoin. This change makes Bitcoin Core more consistent
  with other services, and makes the systemd init config more consistent
  with existing Upstart and OpenRC configs.

So, to prevent a slow re-sync, you'd need to move your old data directory to the new location.  Although the release notes specify the data directory used to be ~bitcoin/.bitcoin, mine on v0.17.1 was in /var/lib/bitcoin/.bitcoin, which I just needed to move to /var/lib/bitcoind to prevent a re-sync.


text:
Address change confusion
I registered to a client with a Bitcoin address on Blockchain.info and after some time that address I registered with was changed. I need to pay a fee to the client. Will it reflect to them that even though the address they are receiving from is different but it is the same wallet?

No, they would not. In Bitcoin, wallet is an abstract concept as it is just a software that manages your private keys and addresses. The addresses generated from the private key in your wallet bear no resemblance to others. Unless you explicitly mention it to them that it is your address, there is no way of knowing whether it is you who has sent the bitcoins to your client. From their perspective it might be as they received a 'donation' of bitcoins from some random person.


text:
How to send bitcoin from an offline Bitcoin Core that doesn't have a synced balance?
I am setting up a cold wallet. How can I generate a signed raw transaction from bitcoin core wallet that controls some bitcoins?
PS: I generated the wallet using getnewaddress in console from the bitcoin-qt GUI

The easiest way to do this by importing the address you generated on the offline Bitcoin Core client into an online fully synced Bitcoin-Core. The command is: bitcoin-cli importaddress <address>. The benefit of importing this address into the synced Bitcoin-Core machine is that you can use it as a watch-only address which will allow you to see all the UTXOs related to it. You can then use the command bitcoin-cli listunspent to show you all the UTXOs and then you can use the ones related to the address in question.
Note: Below txids and amounts are made up.
Now, using the UTXOs that were shown in the listunspent output you can create a transaction on the online client using the createrawtransaction command. I deliberately used two UTXOs to show how this can be done for multiple UTXOs and also to show how other commands will look like if one of the outpoints in the inputs belong to a legacy address and the other outpoint belongs to a P2SH-P2WPKH address.
bitcoin-cli createrawtransaction '[{"txid":"27893ad827ddd94c442c561b1d3bf3d0f60571d6518e4941698e5778b25e1991","vout":0},{"txid":"776be22ff789a86283b247f99f7b2c6f7f53185cb765570bc65c6e6bb7ed8558","vout":1}]' '{"12WJykFa1hqJUJ65QbFB8dS5XNJaR22w2S":0.1}'

This would give you the below hexstring that you can copy over to your offline machine.
output: 020000000291195eb278578e6941498e51d67105f6d0f33b1d1b562c444cd9dd27d83a89270000000000ffffffff5885edb76b6e5cc60b5765b75c18537f6f2c7b9ff947b28362a889f72fe26b770100000000ffffffff0180969800000000001976a9141083323e89f0f734c448cd16d5a5513afb841b1788ac00000000

Copy over that output on the offline machine in order to sign the transaction. Now signrawtransaction command was deprecated in v0.17 and removed since v0.18. In place of that we now have signrawtransactionwithwallet and signrawtransactionwithkey. You will use the former when you want to sign the transaction from a key belonging to your wallet in Bitcoin Core and use the latter when you want to sign a transaction with a key not in your wallet by passing the private key along with the command.
Since your address was generated from a key in the wallet, we will use the first command. After copying over the output of the createrawtransaction command we had got on the online client, we also need to provide the sciptPubKey and redeemScript. This is done so that the offline client knows what the locking script is for the outpoints and which key it needs to use as it does not have access to the transactions (since it is offline).
Now, use the command:
bitcoin-cli signrawtransactionwithwallet "020000000291195eb278578e6941498e51d67105f6d0f33b1d1b562c444cd9dd27d83a89270000000000ffffffff5885edb76b6e5cc60b5765b75c18537f6f2c7b9ff947b28362a889f72fe26b770100000000ffffffff0180969800000000001976a9141083323e89f0f734c448cd16d5a5513afb841b1788ac00000000"
'[{"txid":"27893ad827ddd94c442c561b1d3bf3d0f60571d6518e4941698e5778b25e1991","vout":0,"scriptPubKey":"76a914623ff66fb88ff802fe1b8eaff666ba52307eecb088ac","amount":0.09},
{"txid":"776be22ff789a86283b247f99f7b2c6f7f53185cb765570bc65c6e6bb7ed8558","vout":1, "scriptPubKey":"a914de7dad83bb2d6e9638d347dcb3c53fe238773c9387","redeemScript":"0014df7bcc159ade813e6bd6290d97ed0bc8987dc084", "amount":0.011}]'
"ALL"

This would give you a signed transaction as an output. Let's call it hexoutput. A couple of things to note. If you pay close attention you will see that the second outpoint used in the input contains redeemScript. This is because it belonged to a P2SH-P2WPKH transaction and hence you need to provide the redeemScript next to the scriptPubKey field. For more info you can refer the documentation here.
Now copy this hexoutput over to your online Bitcoin-core client and use the below command to relay it over the network.
bitcoin-cli sendrawtransaction hexoutput

This will give you a txid of the transaction as an output that you can use to monitor your transaction.

Why initiate the raw transaction on the online machine? Why not just make and sign the transaction on the offline machine, and copy the signed transaction over just for the broadcast?

It can be done offline and with no synced data.
Start:
.\bitcoind.exe -datadir=c:\bitcoin -prune=550 -noconnect

Create transaction:
.\bitcoin-cli.exe -datadir=c:\bitcoin createrawtransaction '[{\"txid\":\"some taxid\",\"vout\":0}]' '{\"some target address\":a_number}'

To not spend all the sum, of course, add a returning address:
,\"a returning address\":a_number

vout is the output in the input taxid, for example 1 input, 2 outputs(first for someone else, second is for returning coins at owner) then:
\"vout\":1
, etc.
Count the returning number of characters: ~190 for 1 input 1 output, ~230 1 input 2 outputs,etc.
Then calculate the fee, for example, 10 satoshis per byte will be ~2300 for 1 input and 2 outputs. Then: sum - to_address - returning_address = fee
Sign the transaction:
.\bitcoin-cli.exe -datadir=c:\bitcoin signrawtransactionwithkey "from create" '[\"the private key\"]' [{\"txid\":\"some taxid\",\"vout\":1,\"scriptPubKey\":\"explained below\",\"redeemScript\":\"\",\"amount\":taxid_exact_input}]'

Get scriptPubKey for example from here: https://live.blockcypher.com/btc/tx/here_is_the_taxid/ then "Advanced Details", then "API Call", then at the source output (1 in this case), the "script" value (76a9148a598937f5ffb018fd71520b7c9459391744e28688ac at my case).
Push the signed transaction, for example, from here:
https://live.blockcypher.com/btc/pushtx/


text:
Multisig refund transaction flaw in Lightning Network talk
I was watching Tadje Dryja's talk at SF Bitcoin Devs conference on transaction malleability, which formed part of the basis behind the Lightning Network.

At 17'45", I saw this example. Prior to SegWit and even before considering malleability, how is it possible to even sign a refund txn without knowing the TX-fund signatures? The funding signatures from both parties are required to create the txid that is referenced in the TX-refund input. However, if this information is known, one of the parties can send the funding txn to the network before a refund txn is ever signed, thus forcing the other party's funds into the multi-sig contract. Am I missing something?
(I think SIGHASH_NOINPUT was introduced to get around this exact limitation.)

Yes, you are correct. Joseph Poon and Tadje Dryja in their Lightning Network whitepaper proposed the SIGHASH_NOINPUT flag to prevent transaction malleability flaw hampering the creation of commitment transaction prior to funding transaction. However, with the deployment of SegWit in 2017, NOINPUT was not longer needed as signatures are not part of the txid construction in a SegWit transaction. Since signatures did not form the part of the txid calculation, it solved the ECDSA signature malleability issue that NOINPUT was trying to solve. Thus SegWit deployment allowed us to create the commitment transaction even before signing the funding transaction.
Lightning Network specs have gone a sea change since that paper was published way back in early 2016. I have tried to highlight below how the parties communicate the signing of the transactions as is today and proposals of the future when opening up a channel with each other.
In the current specs (BOLT 1.0), only one party is allowed to contribute bitcoins to the channel. This simplifies the process of exchanging signatures and creating the commitment transaction prior to the funding transaction. Let us say Alice and Bob wants to open a channel. Alice will broadcast the funding transaction only when she has received Bob's signature for the commitment transaction that she holds. Since Alice is the sole funder over here, she has all ammunition to create the funding transaction after Bob exchanges his funding pub key. She creates and signs the transaction, calculates the txid and then shares it with Bob along with her commitment transaction signature that Bob is holding.
Currently there is a proposal (BOLT 1.1) for allowing both parties to contribute funds to the channel. Both parties help each other to build the funding transaction by sharing the inputs and outputs of the funding transaction with each other that they plan to use to fund the channel. It is important to note that they share all information apart from the signatures for those inputs. Once both of them have shared all the inputs and outputs, the parties have enough information to calculate the funding txid. This is because SegWit transactions signatures are not included in calculation of the txid (as mentioned before). This allows us to build the first commitment transaction even before the funding transaction has been created.
Note: Currently, as is, Lightning Network implementation does not require the NOINPUT flag. But to implement the Eltoo payment construction there is a proposal to introduce SIGHASH_NOINPUT. In a nutshell, Eltoo allows us to get rid of the penalty transactions and reduce a lot of overhead on maintaining the channel states. If you are interested you can read the discussion of this proposal on the Lightning mailing list.

Fantastic response, thanks! I know Tad & Joseph are very capable so I wanted to make sure the errata was reported correctly. Prior to NOINPUT and SegWit, the example I've posted is Dead on Conception because the refund/commitment cannot be guaranteed in a trustless manner. I'm hopping into the discussion archive now. What is the best discussion forum to actively contribute feedback and proposals to LN?

@GViz You can propose and give feedback on the Lightning Dev email list. You can register in the mailing list here. For full info on the lightning network specs you can refer to BOLTs on Github Page. Open source github pages for lightning network implementations are: (1) C-Lightning; (2) LND; (3) Eclair


text:
bitcoind bulk import private keys to different wallets and update with one rescan
I was wondering if someone already has this thoughts or problems:
I generate bitcoin addresses and private keys by a script - to not depend on the bitcoin daemon and manage them in a database. By generating the bitcoin addresses, I generate one address and also select a wallet name by some criterias.
BITCOIN ADDRESS | PRIVATE KEY | WALLETNAME
After a specific time i want to import the private keys to the given wallets ("WALLET NAMES") (to the wallets defined in the database before.)
I do this import with importprivkey without a rescan. I was wondering if it's possible to import the private keys to the wallets and to rescan one time and update all wallets with this one rescan to avoid for every wallet a explicit rescans each one and a half hour time.
Is it possible to import the private keys rescan=false and after this to run bitcoind with all wallets loaded and do the rescan which should be affecting all the wallets and update all of them in one rescan?
Has anyone thought about this before and has a solution for this?

You really should use the newer importmulti RPC.
It lets you import multiple private keys at once (and since 0.18, also supports BIP32 derivation etc), and will do the rescan (if desired) once for all keys simultaneously.
If you have accurate birthday information for your keys (a lower bound on the time when the corresponding addresses might be first used on the network), importmulti will even only rescan the relevant part of the chain.

thanks for your fast answer. a rescan with many loaded wallets / a update of all wallets with just one rescan is not possible or? before i did it with import key and rescan=false and the last import i did with rescan=true.. this was working for one wallet, but i still have 20 wallets to update per day.. that would mean 20x one and a half hour..

@Jack Yes, that works too for now, but importmulti is preferable for other reasons over importprivkey in any case.

Is there a script that will generate the JSON given a list of private keys?


text:
Does the pool miners in mining pool receive the same block in case of a fork?
For example there are 2 mining pools A and B and they mined a block in same time.
Then, A mining pool got a block that A mining pool mined?
Or nodes in mining pool receive independently so they select block regardless of mining pool?
I have known nodes choose a block which are propagated early to them and block is propagated not geographically close node but randomly (I mean block is propagated close nodes but this ‘close’ do not mean geographically close but close in network).
When miner is mining or propagating, miner can recognize which node is mining in same mining pool?

Most of the mining pools that are run are managed pools, which means it is operated by an individual or a company. The pool server runs specialized software and a pool-mining protocol that coordinate the activities of the pool miners. The pool miners need not run a full Bitcoin node as the pool server runs one and validates blocks and transactions on behalf of the pool miners.
The pool server constructs a candidate block by aggregating transactions, adding a coinbase transaction (with extra nonce space), calculating the merkle root, and linking to the previous block hash. The header of the candidate block is then sent to each of the pool miners as a template. Each pool miner then mines using the block template, at an easier difficulty than the bitcoin network target, and sends any successful results back to the pool server to earn shares.
Since, the pool miners receives the block template directly from the pool operator, all the pool miners are building the candidate block on the same version of the blockchain.

To drive this answer home: Yes, if the pool server is smart it sends it's own block as the last block in the new block template in case of a fork. What's also interested to look into is is 'Selfish Mining' in which a mining pool already starts mining the second block,  before broadcasting the found valid block. Sirer and Eyal proved in 2013 that this increases the overall revenue of the mining pool. It's also a theoretical danger for the decentralized nature of the blockchain. Selfish mining by definition means that the mining pool (secretly) shares the latest block between them.


text:
Bitcoin fork resolution - Fork losers download previous blocks 'til common block?
When resolving a naturally occurring fork in Bitcoin, do the miners who were working on the orphaned fork download previous blocks from the miner who won the block reward that resolved the fork until they find a common block?
My understanding is that miners accept a new block as long as it is valid. As long as it's one block, they can just add it to their local version and start on the next block. However, if they receive a new block that is the head of a different fork that split off from their fork 10 blocks previously, how do the miners of the orphaned fork get the blocks from the other fork? How do they know to?

As soon as miners find a solution to the block header, they relay the block to the full nodes in their network. These full nodes will validate the block and will add the block that reaches to them first on top of the existing blockchain. Later, when they receive the other block at the same height, they will not discard it but maintain a copy of it. However, the nodes do not relay the block that was received later. Although the nodes assume that the block they received later must have lost the 'race', it keeps the “losing” block so that it has the information needed to reconverge if the “losing” chain ends up “winning.” So here you have a blockchain fork, where around half the nodes have a different version of the blockchain at the best height. When the fork is ultimately resolved when the next block is mined, the nodes can converge back because of the copy or the other block that they keep.
Chain reorganization with more than two blocks is rare unless there is a bug in the software like the one that caused one of the major fork back in March, 2013 that lasted 31 blocks.


text:
Checking if a receiver has enough coins
How do miners verify the blockchain transaction history to check if the sender has enough money if the address can be changed at any time?

Bitcoin does not have the concept of account balance. Anyone who can provide a valid signature to the locking script of a particular transaction output can spend those bitcoins. As a result when you spend your bitcoins in a transaction, you have to include the transaction id and the output no. from which you earned those bitcoins. The nodes maintain a set of unspent transaction outputs (UTXOs) in the memory to check that this transaction output that has been referenced in the input has not been spent previously (this prevents double spending.)
If you want to send X BTC to a particular address, you will need to reference enough transactions in the input so that sum of all those unspent bitcoins >= X. This referenced transactions can come from different addresses. You just need to provide valid signature with your private key to show that you are the owner of those bitcoins.

A node will receive many transactions from other nodes through network propagation. If it determines that a transaction it has received is not valid, it will not relay the transaction to other nodes on the network (essentially preventing it from "spreading").
A node individually verifies each transaction by comparing its attributes against a set of defined protocol rules.
In your case, rule #14 states:

Reject if the sum of input values < sum of output values

The code for this behavior can be found in tx_verify.cpp
const CAmount value_out = tx.GetValueOut();

if (nValueIn < value_out) {
    return state.DoS(100, false, REJECT_INVALID, "bad-txns-in-belowout", false,
        strprintf("value in (%s) < value out (%s)", FormatMoney(nValueIn), FormatMoney(value_out)));
}

While new Bitcoin addresses can be generated at any time, only an address capable of unlocking the output's scriptPubKey can be used to spend these funds.


text:
What does the sequence in a transaction input mean?
Following a transaction input:
What is the meaning of the sequence number? What does the 4294967295 tell me?

Original meaning of nSequence in transactions
nSequence is a 4 byte input level feature. The original meaning of nSequence was to allow modification of transactions in the mempool. So, if the nSequence value of the input was less than 0xFFFFFFFF (4294967295 in decimal), it indicated a transaction that was not yet finalized. Such a transaction would be held in the mempool until it was replaced by another transaction spending the same inputs with a higher nSequence value. Once a transaction was received whose inputs had an nSequence value of 0xFFFFFFFF it would be considered finalized and mined.
In that implementation, replacement transactions did not have to pay additional fees, so there was no direct incentive for miners to include the replacement. Also, no built-in rate limiting prevented overuse of relay node bandwidth. Hence, this feature was disabled later as it was possible for an attacker to use up all the bandwidth among full nodes at only a small cost to themselves, creating a denial-of-service vulnerability. This made it difficult for spenders to adjust their previously sent transactions to deal with unexpected confirmation delays or to perform other useful replacements.
Opt-in Full Replace-by-Fee Signaling
Then came the proposal for Opt-in Full Replace-by-Fee Signaling. It considered a transaction to have opted in to allow replacement of itself, if any of its inputs have an nSequence number less than 0xFFFFFFFE. It also required a higher fee than the previous relayed transaction.
Absolute and Relative Locktime
Transaction with nLockTime is used to prevent the mining of a transaction until a certain date (specified in block no. or time). BIP 68 activation defined the meaning of sequence numbers for transactions with an nVersion >=2. nSequence was repurposed as a relative locktime. That means it prevents mining of a transaction until the input that the transaction is spending has reached a certain age in blocks or timespan.  If nSequence > 0xEFFFFFFF then there us no consensus meaning to the sequence number and it can be included in any block under all currently possible circumstances. If nSequence <= 0xEFFFFFFF, it implies relative locktime.
If nSequence <=0xEFFFFFFF, then the sequence's 22nd bit determines if the relative lock-time is time-based or block based. If the bit is 1, the relative lock-time specifies a timespan in units of 512 seconds granularity. If the bit is not set (i.e. it is 0), the relative lock-time specifies a number of blocks. The value of the relative locktime, is then interpreted from the first 16 bits of the nSequence. The below figure shows how nSequence is evaluated.

Use of nSequence with CHECKSEQUENCEVERIFY (CSV) opcode
CSV is a script level opcode that is used to lock the spending of a particular output of a transaction until certain blocks/time from when the transaction containing the output was mined. When the user is spending this UTXO in a transaction input, he must set the nSequence value in that input greater than or equal to the CSV parameter. The value in CSV must match the format in the corresponding nSequence value. If CSV is specified in terms of blocks, then so must nSequence.
Other miscellaneous points

Currently, if nSequence = 0xFFFFFFFF, the sequence number has no effect on the transaction.
If nSequence = 0xFFFFFFFE, then it implies that the transaction uses locktime but does not opt-in replace-by-fee.
Plain vanilla transactions created from Bitcoin Core uses nSequence=0xFFFFFFFE, because they want to use nLocktime to prevent miners from pulling the transactions forward. The nLocktime used in such transaction is set to the last mined block.

For more information on Locktime refer here.
Future potential for nSequence
As can be seen from the description above, a number of bits in nSequence remain undefined. This allows the possibility to increase its granularity by a soft-fork, or to increase the maximum possible relative lock-time in the future. For example:

If nSequence > 0xEFFFFFFF then the remaining 31 bits have no meaning as this condition can be satisfied by just setting bit (1<<31).
If nSequence <= 0xEFFFFFFF then the bits 23 to 30 have no meaning.
Also, only first 16 bits are used to actually encode relative lock-time meaning a further 6 are unused (1 << 16 through 1 << 21 inclusive).

The old nSequence semantics (where non-final transactions were held in the mempool until finalization) were not only a bandwidth issue, but also a memory problem. I believe that was the original reason for removing it, long before the mempool got an actual bounded size.

@PieterWuille I always had the opinion that it was to prevent DoS attacks. Earlier, no minrelayfee was required and hence that allowed nodes to spam the network at no cost by just incrementing nSequence. BIP-125 actually brought the protocol that the fees should be higher than the previously relayed transaction.

I think that discussing a relationship between a transaction and an opcode ("CSV is a script level opcode that is used to lock the spending of a particular output of a transaction...") is misleading.  I thought the script has no awareness of transactions, only of the specific UTXO that it's return value, if true/1, unlocks and which therefore makes the transaction to which it's an input valid.  Ben Carman suggested nSequence apply to "at least one" rather than "the first" (input), and I thought "Why not use a bit or two in nSequence to identify what it applies to?"

@UgamKamat Are you sure about you answer? https://bitcoin.stackexchange.com/questions/120298/what-are-the-correct-threshold-values-of-the-nsequence-in-order-to-activate-re


text:
Recover Bitcoin Wallet info
I sent some (worth around 1000usd) bitcoins to one of my wallets and doesn't remember which wallet it was. Now I'm checking wallet and seeing balance is still there. Can somebody tell me how do I see wallet information using wallet address (3CGphwQHf5QvK6dMrLcLQvQNCuBsc85T5k) or wallet name(1fac700f695d9d63).

You cannot determine the software used to create an address wallet, and this is by design. Anything else would be a serious security flaw: everyone you transact with would be given information about how you store your bitcoin.

wallet name(1fac700f695d9d63)

I assume this comes from an explorer site like OXT. If so, then it is simply an index created by their system to group transactions according to certain heuristics, it does not provide any info about the wallet software used in the first place.
I hope you are able to locate your coins, but the information you’ve provided is not sufficient to do so. Best of luck.


text:
Mining Pools Transactions Fee
I am learning about mining pools and trying to understand their reward structure and transactions fees.
Looking at the two largest pools, Antpool & BTC.com, as far as I understand AntPool uses PPS and BTC.com FPPS.
The main difference i guess is that in FPPS also the transaction fees are incorporated in the reward structure.
First of all, what exactly is the transaction fee? The fee of finding a block? The fee of transferring the btc reward to another account?
and how can the transaction fee be incorporated in the reward structure as in FPPS?

What exactly is the transaction fee? The fee of finding a block? The fee of transferring the btc reward to another account?

Transaction fee is a method by which anyone who creates a transaction incentivizes the miners to include that transaction in the block that the miner will mine. Bitcoin blocks can have a maximum 4MB of block weight, which means there are only certain transactions that can be included in the block that is being mined. So you include a fee so that miner picks your transaction over others to include in the block. Transaction fees also serve another purpose of preventing DoS attacks. If there were no fees, people in order to spam the network would send between between addresses they control so as to deny the service to legitimate people. Transaction fees ensure that there is a cost associated for launching such attacks.
To be more thorough, you do not send a transaction fee to a particular address, but it is the difference between the input and output amounts. So consider a transaction that consumes 1BTC and sends 0.8BTC to one address and 0.15 BTC to another address, then the transaction fee is 0.05BTC. When the miner creates the coinbase transaction for the block he is mining, he will pay out the block reward (12.5 BTC currently) and sum of all the fees of the transactions that are included in the block to the pool participants.
As to your other question about how they pay transaction fees vs block reward, that is more of a service related question and is more suited to the customer relationship department of each pool.


text:
How chaining the blocks protect against altering data?
I am confused with the block chaining, as i know a block header contains a reference to the previous block header hash with the merkle root and other fields. This header hash is related (computed) to the header and not including the block of transactions, so if the header is tampered the chain will fail. I understood that, but if the transactions in the block are edited, it will not change the header hash only if the merkle root is re-calculated.
My question is when a node receive a block, how it verifies it (just rehashing the header, or recalculating merkle root first then rehashing the header)?

Merkle root is a part of the block header and is calculated based on the transactions included in the block. If transactions in the block are changed, it would change the merkle root in the header. If the merkle root is changed, the block header will change and the header hash which was calculated initially will no longer be valid.

When a node receive a block, how it verifies it (just rehashing the header, or recalculating merkle root first then rehashing the header)?

When a full node receives a block it performs various checks to ensure the block conforms to the consensus rules. This includes checking that all the transactions in the block are valid. If any particular transaction is not valid, then the entire block will be rejected by the full node. After verification of the transactions, the full node would verify the merkle hash. Then full node will look at the block header and verify the timestamp rule, previous block hash, and other components and then calculate the block header hash and check if it is less than the target. If one of the verification rules fail, the node will reject the block. You can find a complete list of block verification rules here.

ok i got it now, i was missing these verification rules. You were talking about full nodes, what about other simple nodes, do they do these verifications too ?

Simple Payment Verification (SPV) nodes that many lightweight wallet uses, downloads only the block headers and not the transactions included in each block. SPV nodes rely on a slightly different method of verification and banks on peers to provide partial views of the relevant parts of the blockchain on demand. SPV nodes link the transaction and the block containing it using Merkle path. Thus it does not have full visibility of the blockchain, but however, it cannot be fooled into assuming that a transaction is included in a block when in fact it isn't.


text:
What system should I implement to make use of accounts with Bitcoin Core 'accounts' now deprecated?
I am designing one of the Bitcoin based app on Bitcoin-Core using account system.
Accounts I have used to store users wallet balance in different accounts.
But since I know it is deprecated, then how can i store users individual balance in separate identifiable addresses.
Is there any database we need to implement?

Accounts were replaced by labels, a more flexible tool which can probably be made to fit your requirement. Alternatively, if you have really been using the now-obsolete "accounts" feature to maintain separate wallets within a wallet, there is now an option to use separate wallets in separate wallet files

The best way to implement accounts is to insource

Address database
Private key database
Transaction signing

and leave only transaction relaying and checking for transactions to Bitcoin Core.


text:
Why do we need revocation in both the offered/received HTLC outputs and HTLC timeout/success outputs?
The witness script for the output in the HTLC success/timeout transaction is:
OP_IF
    # Penalty transaction
    <revocationpubkey>
OP_ELSE
    `to_self_delay`
    OP_CHECKSEQUENCEVERIFY
    OP_DROP
    <local_delayedpubkey>
OP_ENDIF
OP_CHECKSIG

Which already exists a penalty transaction. While in the offered HTLC and received HTLC outputs in the commitment transaction, we have,
# To remote node with revocation key
OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
...

Why don't we put the revocation in the HTLC-Timeout/Success outputs only?

TL;DR
Because it would create a nasty dependency of the CLTV deltas over the CSV delay or a reduced security for HTLCs.
Having the revocation path in the second-stage transaction allows to be able to timeout an HTLC (ie your peer cannot redeem it via pre-image anymore) while still leaving the revocation period (CSV) intact for you to be punished if you cheated.
Two-stage HTLCs
The Lightning Network protocol makes use of two-stage HTLCs.
Why: allow to use absolute timelock delta shorter than the relative timelock delta used for the fundamental security model (punishment), without degrading the security model for HTLC outputs.
How: make (basically) HTLC outputs pay to either the revocation path, the [timeout/success] transactions only broadcastable after block B, or the receiver with the preimage. Make the [timeout/success] transaction pay only after full expiration of the revocation period (no preimage path anymore here, if the receiver did not cheat they will get paid).
More about two stages HTLCs in this answer.
Why put the revocation path at both stages ?
Because the first stage expires at the end of the absolute timelock period, which is likely to be less than the revocation period.
If the revocation path was not present in the second transaction (which is actually the point of using two stages) then you would either drastically reduce the revocation period for HTLC outputs, or require using insanely high CLTV deltas.
An example
I've a channel with you, and we agreed on a to_self_delay of 144 blocks (1 day to punish the broadcast of a revoked state) and a (reckless) cltv_expiry_delta of 6 blocks.
I just attempted to make a payment through our channel. We appended an offered HTLC output to my commitment transaction and a received HTLC output to yours.
Let's say someone found funny to not take its negligible fees and instead stuck the HTLC to mess with us. After 4 blocks, we fail the HTLC, sign a new pair of commitment transactions and continue operations --I'd later learn that you actually stuck the HTLC just after gathering the preimage.
We don't create new states to our channel and the same night I suffer a power outage, my node comes offline and I don't notice it until I'm awake (5 hours later).
At the exact moment I got offline you took the absolutely irrational decision to broadcast your ancient commitment transaction in which you still have a signed received HTLC output.
Your transaction gets confirmed, and passed the 6 blocks (still 4 hours until I wake up) you can claim the output with your HTLC success transactions.
4 hours later, I wake up, put my continuously backed up node back online and notice the breach.
Before I could even have the time to feel disillusioned, disappointed and frustrated enough to throw swear words to a random internet colored alias, my node stole your coins from your commitment transaction's to_self output, claimed my to_remote output and spent your HTLC success transaction.
Without the revocation path in the second stage my coins would be lost.

HTLCs have a different timeout (cltv_expiry) than the regular timeout that we use for penalty transactions (to_self_delay). That is the reason why we use a separate transaction stage (HTLC success/HTLC timeout) to allow our counterparty sufficient time to exercise their right to get the entire balance of the channel in case we broadcast the previous state of the channel.
To elaborate further, when your counterparty tells you to add a HTLC, your version of the commitment transaction will have 3 outputs:

Paying to your counterparty immediately
Paying to self, guarded by the revocation key for to_self_delay time
Received HTLC output like below

# To remote node with revocation key
OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
OP_IF
    OP_CHECKSIG
OP_ELSE
    <remote_htlcpubkey> OP_SWAP OP_SIZE 32 OP_EQUAL
    OP_IF
        # To local node via HTLC-success transaction.
        OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY
        2 OP_SWAP <local_htlcpubkey> 2 OP_CHECKMULTISIG
    OP_ELSE
        # To remote node after timeout.
        OP_DROP <cltv_expiry> OP_CHECKLOCKTIMEVERIFY OP_DROP
        OP_CHECKSIG
    OP_ENDIF
OP_ENDIF

When you supply the HTLC pre-image to your counterparty the transaction gets resolved. However, even after successful resolution of the HTLC with your counterparty, let us say you decide to cheat them by force closing the earlier state of the commitment transaction containing the HTLC. As soon as you publish this commitment transaction, you publish the HTLC success transaction that consumes the OP_IF part inside the OP_ELSE. Since the HTLC success transaction is consuming an output that is not timelocked, you are not providing enough time for your counterparty to use their right of the revocation key which is in the received HTLC output. That is the reason there is a second stage, where your spending from the offered HTLC output to yourself is locked until to_self_delay which allows your counterparty to use the revocation secret.
Now you might wonder, why the revocation exists in the commitment transaction received HTLC output if it is provided in the HTLC success/timeout transactions. Let us take the above case to explain it. You added the HTLC and signed the commitment transaction. That HTLC was settled by you providing the pre-image to your counterparty and the commitment transaction containing that HTLC was revoked. Despite that, assume you try to broadcast the previous commitment transaction containing the HTLC and refrains from publishing the HTLC success transaction. If revocation did not exist in the commitment transaction, the counterparty might not be able to get their funds back until the cltv_expiry which might extend to hundreds of blocks (couple of days) if there are a number of hops. This is cumbersome since that HTLC was already settled (especially if there are number of previously satisfied HTLCs), and the provision of revocation in the commitment transaction allows the counterparty to settle them immediately.
The revocation part is designed so that your counterparty never suffers because of the actions you decide to take to cheat them. The presence of revocation in commitment transaction and the HTLC success/HTLC timeout transaction help protect the Lightning Network participants from frauds committed their peers.

If you publish an old commitment tx with 1 received HTLC, the counterparty can enter the timeout process as specified in cltv_expiry and then collect the money via HTLC-timeout anyway? So your point is that we don't want the counterparty to wait that long if you cheat? You don't seem to have a good reason to publish this tx though pure harm may be wanted.

@yyforyongyu  (1) Yes, you are correct that the counterparty can collect after the time-out process, but the local node (who is cheating) can spend it immediately using the HTLC success transaction (since it has the pre-image). (2) it is not just pure harm but you are definitely economically incentivized to publish it. You got the funds from your counterparty by providing the pre-image for the HTLC. Suppose after that you provided some funds to your counterparty for some good you bought from them. The previous state that included the HTLC is more beneficial to you so you publish that,.

Let's say I'm the local node who is cheating. I have to spend it via a HTLC-success tx, which has a revocation inside it. Once I spend the commitment tx, I have to wait to_self_delay time before I can spend the HTLC-success tx, meanwhile, the counterparty can take the money via the revocation?

@yyforyongyu Yes. Since you can spend the commitment transaction HTLC output immediately (for HTLC success) you will not be providing any time for your counterparty to react. As a result the to_self_delay within the HTLC success output allows the counterparty to use the revocation and then spend the money.

interesting. This goes back to the original question, if the counterparty can take the money via revocation inside the HTLC-success tx, what's the point of having a revocation inside the commitment tx?

@yyforyongyu That's answered in the second last paragraph of my answer (i tried to make it more clearer). What if the cheater never publishes the HTLC success transaction? Even though that HTLC is settled, the counterparty might have to wait couple hundred blocks to get it settled (if there are number of hops). Imagine if it contained multiple such HTLCs and you have to wait for each of them to time out despite the fact that all of them have been settled.


text:
How a SegWit transaction is accepted without witness?
I would like ask 2 questions.
First, as perspective of legacy node(doesn't upgrade software):
When SegWit transactions are transferred, witness data is stripped. It means that there are no signatures to proof the tx inputs in transaction propagated because there are no witness data as described above. How does the legacy node verify these transactions without the witness?
Lastly, as perspective of fresh node (did upgrade):
Regardless of SegWit, the block size can't exceed 1MB. But fresh nodes make a block which is consist of only 1MB transactions and NByte of witness(signature) data. This is very strange point because the block is already full with only transactions but there are some witness data.
Can you give point on my two questions?
I'd like to find which points are wrong in my knowledge.

How legacy node can verify them without the witness?

An example of a SegWit output is: scriptPubKey: 0 ab68025513c3dbd2f7b92a94e0581f5d50f654e7. For legacy nodes, this output looks like an anyone can spend output as there are no opcodes or verification. Such outputs do not require any signatures to spend. Thus when a legacy node sees a transaction spending a SegWit output, it does not go looking for any signatures and considers the transaction to be perfectly valid.

Block size of 1MB

With the SegWit soft-fork, block size got redefined to 4,000,000 Weight Units (WU). In the WU calculation, transaction data (excluding segregated signatures) are multiplied by 4 while segregated signature data are multiplied by just 1. For example, if transaction data (excluding signatures) is 200 bytes and the witness data is 144 bytes, then the transaction has a WU of (200*4) + (144*1) = 944 WU. Blocks cannot breach this 4M WU criteria without getting invalidated. If there is a block with 1MB of non-signature data then it's size is 4M WU (1MB*4). Adding signature data on top of that will breach that threshold.
For legacy nodes, the block size continues to remain at 1 MB as these nodes do not query for the witness data for a SegWit transaction. If a block size exceeded 1MB with transaction data excluding signatures, then it would also exceed the 4M WU criteria as well.

Thanks for awesome replying :) I'd like to clarify your answer in another post. can you join too? (https://bitcoin.stackexchange.com/questions/91761/is-segwit-really-safe-and-reduced)


text:
Can someone help me find the private key for my bitcoin wallet?
I have a Bitcoin Wallet on Blockchain and i have a imported wallet there where my bitcoin is stuck please help.

By design, it is not possible to determine the private key for arbitrary addresses.
If you do not have a wallet file, seed phrase, or other form of keys that lead to that address, no one can help you.
If someone asked you to transfer Bitcoin to a watch only address claiming it will be accessible by you, you have been scammed.

where can i find my wallet file

@Praveen: https://bitcoin.stackexchange.com/search?q=wallet+file


text:
How does Bitcoin makes sure that nobody can duplicate a transaction?
I understand that when creating a transaction, you send in data, the hash of that data, and a signature that is created using the hash of the data and your private key. However, what is stopping someone, for example the recipient of the original transaction, to simply send in the same exact transaction with the same data and signature. Wouldn't this come up as being valid, and therefore make the system insecure? How does Bitcoin solve this problem?
If the answer is a timestamp, how does Bitcoin deal with different transactions coming in at various times, often out of order?

However, what is stopping someone, for example the recipient of the original transaction, to simply send in the same exact transaction with the same data and signature?

Bitcoin does not have the concept of account balance but works on the concept of unspent transaction outputs (UTXOs). Each output of a transaction (except OP_RETURN ones) lead to the creation of a separate UTXO. When you create transactions in Bitcoin, you consume these UTXOs in entirety and create separate UTXOs. This is done by referencing the outpoint (txid and n) from which you 'earnt' these UTXO. For example, if you control 2 UTXOs (1 BTC and 0.5 BTC), and you want to send your friend 1.25 BTC, you would have to consume both your UTXOs and send the remaining change of 0.25 BTC to yourself (neglecting tx fees). So if the recipient of a transaction output tries to broadcast the same transaction again, the Bitcoin nodes when verifying the transaction would find that these utxos do not exist and hence the transaction is invalid.
When full nodes start syncing from the Genesis block, they start to build the database of all these UTXOs. Every transaction involves removing spent UTXOs and adding new ones. These database is stores in chainstate directory and aggressively cached in memory.

Technically OP_RETURN outputs are still outputs, they can just be ignored because there is no condition which can make them spendable.

@Anonymous yeah true. they are outputs but not unspent outputs. So no need to add them to the UTXO database


text:
How do block explorer sites show information that is not available on Bitcoin Core?
As I understood in the following post How to get unspents for a specified bitcoin address?, we should use listunspent only on addresses that belong to our own wallet otherwise will return blank.
But let's take an example of blockchain.info or block.io. Once we type in an address - any address, even if we generate it manually (paper wallet, or algorithmically) - and transfer some BTCs, a search of the address in one of those explorers shows me the funds.

How can they possibly have information from this newly generated address?
How can they list the unspent transactions and the current amount of an address which "is not in their wallet"?

How can they list the unspent transactions and the current amount of an address which "is not in their wallet"?

Block explorer sites maintain a separate database of information which is built by continuously scanning the Bitcoin blockchain. If block explorer sites see a standard locking script in the outputs of the transactions, they scrape the addresses out of those locking script and the amount associated with that output. This information is added to their database. Also, when a user spends bitcoins, he has to reference the txid and the output index in the input of the transaction. The block explorer sites uses this information to locate that output and check the output locking script, from which they derive the 'input address'. All of this information and continuous lookups allows block explorer sites to associate funds with addresses and show whether the bitcoins sent from a transaction have been spent or not. However, if you lock a transaction in a custom script, the block explorer sites won't be able to decode this information.

How can they possibly have information from this newly generated address?

They continuously scan the blockchain from which they can get transactions included in blocks. Also block explorer sites run full nodes that helps them collect transactions relayed by other nodes in the mempool that haven't been confirmed in blocks yet.

So you mean that they have a new database (probably a NoSQL like Mongo) and they update infos like spent/unspent so we can have this info on screen?

@AllanRomanato I don't know the exact database that they use, but yes they maintain a separate one that can be easily queried when an end-user enters an address to check the transactions/addresses.

Makes sense since using the full node is easy to query transactions but not addresses. We can get all info as well but always starting from transactions not address.


text:
How does BIP 39 mnemonic work?
As I already understood the mnemonics are the private key separated in 12 words which joined together in the very same order produces the private key.

How are mnemonics generated?
Why Ledger mnemonics have 24 words?
How those words are converted to private keys?

Mnemonics are not private keys. A mnemonic represents an entropy (along with checksum) that is used to generate a 512 bits seed. This seed then acts like a keychain that is used to generate different private keys. Check the last section of the answer to see how private keys are generated from seed.

How are mnemonics generated?

Mnemonics are representation of entropy. First step involves making a decision as to how much entropy you consider safe for your operations. Assume, for now that you have decided on 128 bits of entropy. Below are the steps that you will follow to convert this entropy to mnemonic.

Use some cryptographically secure entropy generator to generate 128 bits of entropy.
Calculate the SHA256 of the entropy.
Append the first entropy_length_in_bits/32 bits of the SHA256 of the entropy at the end of the entropy (this will be your checksum). For example, in our case we will append the first 4 bits of the SHA256(entropy) to the entropy since our entropy is 128 bits.
Each word of the mnemonic represents 11 bits. Hence, if you check the wordlist you will find 2048 unique words. Now, divide the entropy + checksum into parts of 11 bits each.
Match this 11 bit fragments to the words in the lookup table in the wordlist. Since we used 128 bits of entropy our checksum was 4 bits. So our entropy along with checksum represented a total of 132 bits. Thus our mnemonic will be 12 words.

If you had used 256 bits of entropy, your checksum would have been (256/32 =) 8 bits. That would represent (264/11) = 24 words.
One thing to note is that any 12/24 words cannot be used as a mnemonic. Some 'portion' of the last word generally contains the checksum of the words chosen and hence has to be calculated. It is also discouraged to generate words directly from thought and use a secure cryptographic function to do so.

Why Ledger Mnemonics have 24 words?

That is a design choice of security. More the number of words higher the entropy. 24 words will provide 256 bits of entropy. It is also important to note you cannot convert a 24 word representation to 12 words and vice versa.

How those words are converted to a private key?

The mnemonic is passed to key-stretching function PBKDF2 with 2048 rounds of hashing. The PBKDF2 function also has the ability to take a 'salt' that can be an optional passphrase. This passphrase can be taken from user and serves as an additional layer of security to prevents brute-force attack with look-up tables or side-channel attacks. The output of this function is a 512 bit seed.
This seed is then passed to HMAC-SHA512 with key "Bitcoin seed". The resulting hash is used to create the master private key (m) and master chain code (c). The left 256 bits of that resulting hash represents m while the right 256 bits represents c. The master private key m is then used to generate master public key M (M = m*G).
From here a number of derivation paths existing for different wallets. The most common one is a hardened derivation method as specified in BIP 44. Essentially, hardened keys use the parent private key in the hash function to generate child private key, while non-hardened uses parent public key in the hash function. This improves the security in the generation of child keys. In the below derivation, k and K represents private key and the associated public key respectively.
We would first need to show that we have used BIP 44 derivation path. That can be done with an index number and generate a private key one level deeper from the master private key. The child private key one level deeper is generated by: kchild = kpar + hash(kpar, cpar, i) where i is the index number. For hardened derivation of BIP 44, i will be 0x80000044 (we use the latter 231 half of the index number for hardened derivation). This result will give us a 512 bit number. The left 256 bits will represent the child private key and the right 256 bits will represent the child chain code.
The next level represents the coin. For Bitcoin, that is 0 or 0x80000000 in hardened derivation. You then calculate the child private key and child chain code  one level deeper using the formula above.
The next level represents account. You can use multiple accounts to represent different functions and help manage your funds better. You can use the above logic to generate the account private key and chain code. Again, this is hardened derivation so the first account will have index number as 0x80000000.
From here onward we do not use the hardened derivation. The next level represents receiving address vs change. This allows you to have different bunch for receiving private keys and different key bunch for change private keys. The function we will use to generate the child private from parent will be: kchild = kpar + hash(Kpar, cpar, i). Now i will be 0x00000000 for receiving and 0x00000001 for change. Also note, now we have public key in the hash function rather than private key which shows this is not hardened derivation.
Now, at the next level we use these receiving and change key bunch to generate individual private keys. Use the above generate private keys and chain code and pass them to the above mentioned function kchild = kpar + hash(Kpar, cpar, i) to generate individual keys. Every increment of i will give you a different private key.
Now use these private keys to generate bitcoin addresses.

Do you have any code references on how to derive private key from mnemonic for BIP44 path, such as "m/44'/60'/0'/0/0"?

I encountered the same situation where I needed to dig further into the process of generating an Ethereum wallet. I have read through the material I could search for but none is giving me an answer I need to complete my programming task of creating a wallet. Here is a brief summary on what I have learnt so far:

generate mnemonic
generate a seed from mnemonic. For example: bip39.mnemonicToSeed(mnemonic).
derive a master key from this seed.
derive child key from a path such as: "m/44'/60'/0'/0/0".
this child key is your wallet's private key (for Ethereum).
you can extract the address for the wallet from this  private key.

Even though the above process is not code you can directly use, but at least you may have a better understanding on the steps and what's involved.
In dart, the whole process can be achieved using the following libraries:
bip39, bip32, dart_bip32_bip44, dart crypto etc..


text:
How does Lightning Network over TOR work?
I'm interested to understand in detail how the lightning network works over Tor. Implementations like lnd, c-lightning etc. offer an extension which allow running TOR with Lightning. I would like to understand how running Lightning with TOR works in practice. In particular I'm interested in the following cases:
1) I am the sender, but I enabled Tor in my node. How does sending to a node R outside Tor network looks like?
2) I am the recipient node, and I do not advertise an IP address, but an onion address. How do I receive payments?
3) The sender node S is a regular LN node, without any Tor connections. As a sender, I want to send a payment to node R and my LN node finds the best path to send my payment to R. Is it possible that this path will at any hop go thought Tor or go through onion node, because any of the nodes in the selected path happens to be onion or has access to Tor? And if yes, how the routing looks then?

Running lightning node over TOR is no different than running it over normal IP connection. Sending payment, fulfilling incoming payment, sending error messages etc. would happen in the exact same way in both cases. The only difference is that the above messages that you send to your peer will now happen over TOR network rather than a direct IP package.
If you are using only TOR without any public IP address then to route your payment to a node that is using only public IP address you will need to have a node in your path to the receiver that is (1) running TOR and public IP or (2) or running public IP and can connect to TOR nodes using the socks5 proxy. If you do not have this node in between you would not be able to send the payment.
When Tor service starts it creates a socks5 proxy which is by default at address 127.0.0.1:9050. If a node with public IP is started with the option --proxy=127.0.0.1:9050 (or including it in the config file) the node will be able to connect to nodes running TOR (like yours).
If you are running TOR and have a public IP address then you can directly connect with nodes that run tor or public IP nodes via the tor service socks5 proxy.

I am the sender, but I enabled TOR in my node. How does sending to a node R outside TOR network looks like?

The network routing happens according to what I mentioned above. However, the path calculation for sending the payment to the receiver happens on your node so it does not involve what network routing you are using. You would construct the onion routing packet with the path to the receiver (the channels you will use to send the payment), and try to send this onion and the payment_hash to your peer via the update_add_htlc message. This message will then go over TOR nodes before reaching your peer, instead of a directly reaching your peer.

I am the recipient node, and I do not advertise an IP address, but an onion address. How do I receive payments?

You can receive payments from nodes directly that are running TOR. If you want to receive payments from nodes that have only public IP, then you would need to have a node in your path that has the proxy option set so that it can connect to TOR nodes via socks5 proxy.

The sender node S is a regular LN node, without any Tor connections. As a sender, I want to send a payment to node R and my LN node finds the best path to send my payment to R. Is it possible that this path will at any hop go thought Tor or go through onion node, because any of the nodes in the selected path happens to be onion or has access to Tor? And if yes, how the routing looks then?

Assume you the path from S to R looks like this: S -> T -> U -> V -> R. Number of cases can arise:

S and R do not run TOR: It depends

All the nodes could be on public IP and your payment goes through.
T could be node running public IP and TOR. It has a public IP channel with you, and TOR channel with U. U can then have a proxy option set that allow it to have TOR based channels with T and public IP channel with U. V is a public IP node and U routes payment to V in normal way.

R runs TOR: At least one node in between should run/understand TOR

T/U/V has a public IP, and have TOR so that they can make channels with TOR nodes and public IP nodes
T/U/V are all public IP nodes but V has a proxy option set which allows it to have a tor based channel connection with R.

Thanks for the great reply, highly appreciate it. Here are just a couple of detailed questions:
Is there any way to know which nodes can support TOR connections beside the ones that actually publish an onion address? The majority of the nodes publish only a public IP but that doesn't mean they don't have TOR enabled, right? On the other hand, if we don't know which nodes enable TOR how we can correctly pick the path from source to destination of the payment?

Part 1: (1) I think it has to work the opposite way. Using only TOR node you cannot connect to a public IP address node (outbound connection), but can accept connection from a public IP node that can connect to TOR through proxy. (2) Nodes announce the services that they run in the node_announcement message. You can just run the command lightning-cli listnodes <node_id> and check what network that node supports.

Part 2: (3) you pick the path based on channels. If there is a path with channels in between you and the receiver than at least one node in between must support both the services. If it didn't the nodes wouldn't have been able to connect, let alone set up a channel.

Thanks for the replies, that's super helpful! The information about the supported networks, is that the one information under the key 'addresses' or something else?

@AnnMP yes, you will see the supported address types in the key addresses. For example, you could query 03a503d8e30f2ff407096d235b5db63b4fcf3f89a653acb6f43d3fc492a7674019 node and find that it runs three types of addresses: ipv4, ipv6, and torv3.

"If there is a path with channels in between you and the receiver than at least one node in between must support both the services." That's wrong, you can of course connect to a public ip node over tor (unless the other node is blocking connections from tor exits).

Also: you can receive payments from a public ip node if you're running an only-tor node by connecting to that node and then opening a channel, the other way around (opening a connection from the public ip node to your tor node) wouldn't work because of your node not having a public ip. A channel can be opened in both directions as long as there is a connection between nodes.

@chpio 1/2 You might be right that the public IP can accept incoming connection, but the Lightning implementations does not support that. Check out the c-lightning docs related to what they support over tor. In case #1 you will see that public IP address without TOR service supports only outgoing.

@chpio 2/2 As I said above related to the implementations, public IP nodes can only send outbound connections to the nodes running TOR and TOR only nodes can only receive inbound connection. However, after the connection is established, they can open the channel in any direction.