text:
How to decode a coinbase transaction?
Below is block result of block height 200,000
{
   'hash':'000000000000034a7dedef4a161fa058a2d67a173a90155f3a2fe6fc132e0ebf',
   'confirmations':396774,
   'strippedsize':247533,
   'size':247533,
   'weight':990132,
   'height':200000,
   'version':2,
   'versionHex':'00000002',
   'merkleroot':'a08f8101f50fd9c9b3e5252aff4c1c1bd668f878fffaf3d0dbddeb029c307e88',
   'tx':[
      'dbaf14e1c476e76ea05a8b71921a46d6b06f0a950f17c5f9f1a03b8fae467f10',
      'ee475443f1fbfff84ffba43ba092a70d291df233bd1428f3d09f7bd1a6054a1f',
      'e03a9a4b5c557f6ee3400a29ff1475d1df73e9cddb48c2391abdc391d8c1504a',
      'ffa0267c8f2af736858894d6f3e5081a05e2ec16dc98f78a80f376ce35077491',
...
      'd22795f708706cbcd67aaf2bb63e7defd9e0c6c13f4f263dfc36592c047c981a'
   ],
   'time':1348310759,
   'mediantime':1348307828,
   'nonce':4158183488,
   'bits':'1a05db8b',
   'difficulty':Decimal('2864140.507810974'),
   'chainwork':'00000000000000000000000000000000000000000000001ac073536b8dbae81c',
   'nTx':388,
   'previousblockhash':'00000000000003a20def7a05a77361b9657ff954b2f2080e135ea6f5970da215',
   'nextblockhash':'00000000000002e3269b8a00caf315115297c626f954770e8398470d7f387e1c'
}

Below is coinbase transaction of 200000 block
{
   'txid':'dbaf14e1c476e76ea05a8b71921a46d6b06f0a950f17c5f9f1a03b8fae467f10',
   'hash':'dbaf14e1c476e76ea05a8b71921a46d6b06f0a950f17c5f9f1a03b8fae467f10',
   'version':1,
   'size':192,
   'vsize':192,
   'weight':768,
   'locktime':0,
   'vin':[
      {
         'coinbase':'03400d0302ef02062f503253482f522cfabe6d6dd90d39663d10f8fd25ec88338295d4c6ce1c90d4aeb368d8bdbadcc1da3b635801000000000000000474073e03',
         'sequence':4294967295
      }
   ],
   'vout':[
      {
         'value':Decimal('50.63517500'),
         'n':0,
         'scriptPubKey':{
            'asm':'04b0bd634234abbb1ba1e986e884185c61cf43e001f9137f23c2c409273eb16e6537a576782eba668a7ef8bd3b3cfb1edb7117ab65129b8a2e681f3c1e0908ef7b OP_CHECKSIG',
            'hex':'4104b0bd634234abbb1ba1e986e884185c61cf43e001f9137f23c2c409273eb16e6537a576782eba668a7ef8bd3b3cfb1edb7117ab65129b8a2e681f3c1e0908ef7bac',
            'reqSigs':1,
            'type':'pubkey',
            'addresses':[
               '1MdYC22Gmjp2ejVPCxyYjFyWbQCYTGhGq8'
            ]
         }
      }
   ]
}

What I want to is decoding vin.coinbaise
I searched out some docs, but I failed to find out exact way of decoding this part:
coinbase: 03400d0302ef02062f503253482f522cfabe6d6dd90d39663d10f8fd25ec88338295d4c6ce1c90d4aeb368d8bdbadcc1da3b635801000000000000000474073e03

Do you have any idea how to decode it?

In the original source code there were little restrictions on what could be in the coinbase data of the coinbase transaction. I think the only restriction was that the coinbase data should be between 2 to 100 bytes. However, this lack of restriction on what can be within the coinbase data brought up a vulnerability. It was perfectly legal for miners to create a coinbase transaction identical to previous one and mine it in a block at a height higher than where the "original" coinbase transaction. When this happened it led to the invalidation of the previous transaction.
This was fixed later when couple of blocks were mined with identical coinbase data. As per BIP-34, version-2 blocks must contain the block height index as a script push operation in the beginning of the coinbase field. For example, in your coinbase transaction this data is represented by the bytes 0x03400d03. The first byte 0x03 indicates the script should push the next 3 bytes. The next three bytes 0x400d03 represent the block height 200,000 in little endian.
The rest of the coinbase data is free for the miners to use as per their will. This is mostly used as a space for extra-nonce. In the early days of Bitcoin, a miner could find a block by iterating through the nonce until the resulting hash was below the target. As difficulty increased, miners often cycled through all 4 billion values of the nonce (232) without finding a valid block header hash. However, this was easily resolved by updating the block timestamp to account for the elapsed time. Because the timestamp is part of the header, the change would allow miners to iterate through the values of the nonce again with different results.
Once mining hardware became really efficient, however, this approach became increasingly difficult because the nonce values were exhausted in less than a second. Hence, the mining software needed more space for nonce values in order to find valid blocks. The timestamp could be stretched a bit, but moving it too far into the future would cause the block to become invalid. The solution was to use the coinbase transaction as a source of extra nonce values. Because the coinbase script can store between 2 and 100 bytes of data (~first 4 bytes reserved for block identification), miners started using that 96 byte space as extra nonce, allowing them to explore a much larger range of block header values to find valid blocks.
The mining pools also use the coinbase data as an identification of who mined the block. For example if you look at block 594,041, you will see the coinbase data contain the bytes 0x4d696e656420627920416e74506f6f6c. It represents "'Mined by AntPool" in ASCII. (Unfortunately the block you mentioned did not have any identification hence I had to use a different block that did have it). However, it is important to note that this data cannot be relied upon, as any miner can mine a block tomorrow with a data claiming it to be mined by a different pool, may be to hide its identity or frame others.

Thank you for your comment :)
It will be really helpful.
The reason why I want to decode coinbase tx is I want get who mines a block. But according to your answer, my approach is not reliable.
Then is there no reliable way to find out who mines a block?

@dingyo777 it is not possible to determine it with certainty who mines the block. You can refer to the answer here for more information.


text:
How to make a transaction with locktime on testnet?
I tried to make transaction with nLocktime on bitcoin testnet. I am not able to broadcast it.
Can someone please help me to make a basic transaction with 1 input and 1 output, so that I can broadcast on the testnet. (p2pkh -> p2pkh)
Usecase:
mvvz9TNZ37fFnUHXZPUJYV3MDutbkb9ipA -> mgGUnb6BhgxzrjWau4X4WD9Lfp9YS4kMy7
Transfering: 0.017 BTC at block height 1567415 (now 1567406)

Raw transaction (before signing):
020000000140c96a4c43a5db3c57be3fd92f491a5d40ccf00afe349c596bc35fac7bbee59a000000001976a914a9156a64ea643e7c4db772c96c6a570712e0cc1988acfeffffff01a0f01900000000001976a914083b56fb8b320a6373d4738bb81496b096e8034c88acb7ea170001000000

After Signature:
020000000140c96a4c43a5db3c57be3fd92f491a5d40ccf00afe349c596bc35fac7bbee59a000000008a4730440220271919b389ed3b635522167d3e2a0201cd85d67941816af4c1638664054c078202203b93ddf929692c2a6d081ef422de9f375183fd595e8ca9e8135af86e47e47c580141043db79f05b69d34dfd3d8658a557ddb6b1a8ebca9070968b1e3833ca6b581d8a30073638dafca0bf6a5a2a08ff89ce91186fefa5a46d2a3349eb47904ab981114feffffff01a0f01900000000001976a914083b56fb8b320a6373d4738bb81496b096e8034c88acb7ea1700

How to use CLTV and CSV?

Note: Although this answer is written for testnet, it is absolutely valid for the mainnet as well. The script will be similar, just the address encoding will be different (P2PKH start with 1 vs m for testnet as we use 0x00 as prefix in address base58check in mainnet).

I tried to make transaction with Locktime on bitcoin testnet. I am not able to broadcast it.

When nLocktime is specified, you cannot broadcast the transaction until that nLocktime value is reached. In your case, when writing this answer, the testnet height was at 1567414 while nLocktime is set as 1567415. Other than that your transaction is correct from a script point of view.

And how to use CLTV and CSV?

CLTV and CSV are script level timelocks, unlike nLocktime and nSequence which are transaction and input level timelocks. CLTV or CSV are specified in the output locking script, and when spending those outputs, the user must set the nLocktime or the nSequence value to satisfy the locking condition.
CLTV
Using your example, below will be the script for CLTV in the outputs and the hex of the outputs. The desired block height is 1567415 which is 0x17eab7 in hexadecimal and 0xb7ea17 in little-endian. nLocktime value should be equal or greater to this number in order to spend the outputs that are locked with CLTV. So, for this transaction to go through nLocktime value should be equal to or lower than the current block height which is equal to 0x0017eaae (1567406) or 0xaeea1700 in little-endian. However, do note that when spending this output that is locked with CLTV as an input in a transaction your nsequence should be between 0xf0000000 to 0xfffffffe. Using 0xffffffff disables timelock.
"vout": [
            {
                "value": 0.01700000,
                "n": 0,
                "scriptPubKey": {
                    "asm": "b7ea17 OP_CHECKLOCKTIMEVERIFY OP_DROP OP_DUP OP_HASH160 083b56fb8b320a6373d4738bb81496b096e8034c OP_EQUALVERIFY OP_CHECKSIG",
                    "hex": "03b7ea17b17576a914083b56fb8b320a6373d4738bb81496b096e8034c88ac",
                    "reqSigs": 1,
                    "type": "pubkeyhash",
                    "addresses": [
                        "mgGUnb6BhgxzrjWau4X4WD9Lfp9YS4kMy7 "
                    ]
                }
            }
        ]

Using this output to your unsigned transaction hex, it gives the following script:
Raw transaction before signing:
020000000140c96a4c43a5db3c57be3fd92f491a5d40ccf00afe349c596bc35fac7bbee59a000000001976a914a9156a64ea643e7c4db772c96c6a570712e0cc1988acfeffffff01a0f01900000000001f03b7ea17b17576a914083b56fb8b320a6373d4738bb81496b096e8034c88acaeea170001000000

CSV
CSV is a relative timelock, so the height of the blocks in relative sense is provided. For example, in your case the testnet height was at 1567406 while you wanted the transaction to be spent at height 1567415. So your relative height is 9 blocks. We are using nLocktime for the current transaction to be the current block height which is 0xaeea1700 in little-endian.
"vout": [
            {
                "value": 0.01700000,
                "n": 0,
                "scriptPubKey": {
                    "asm": "09 OP_CHECKSEQUENCEVERIFY OP_DROP OP_DUP OP_HASH160 083b56fb8b320a6373d4738bb81496b096e8034c OP_EQUALVERIFY OP_CHECKSIG",
                    "hex": "0109b27576a914083b56fb8b320a6373d4738bb81496b096e8034c88ac",
                    "reqSigs": 1,
                    "type": "pubkeyhash",
                    "addresses": [
                        "mgGUnb6BhgxzrjWau4X4WD9Lfp9YS4kMy7 "
                    ]
                }
            }
        ]

Using this output script in the unsigned transaction hex you specified gives us:
Raw transaction before signing:
020000000140c96a4c43a5db3c57be3fd92f491a5d40ccf00afe349c596bc35fac7bbee59a000000001976a914a9156a64ea643e7c4db772c96c6a570712e0cc1988acfeffffff01a0f01900000000001d0109b27576a914083b56fb8b320a6373d4738bb81496b096e8034c88acaeea170001000000

When spending this CSV locked outputs, you will need to keep a few things in mind: (1) nSequence <= 0xEFFFFFFF; (2) nSequence's 22nd bit determines if the relative lock-time is time-based or block based. This bit should be set to 0 since your CSV is locked in blocks; (3) The value of the relative locktime, is then interpreted from the first 16 bits of the nSequence.
Although I have shown the locking scripts in its raw form (using CLTV/CSV) in scriptPubKey, it is recommended to create a P2SH locking script out of this raw script. This not only ensures that you pay lower fees for making this transaction, but also ensures your privacy (nobody will know that they transaction is timelocked) until you spend that transaction.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How do I delete a Bitcoin full node from my hard drive?
I was using Bitcoin Core as my wallet (recently installed), but a couple of hours of syncronizing, all of a sudden, I have no space at all in my hard drive. I deleted bitcoin Core, but that did nothing. What files do I need to delete?

Before you delete you Bitcoin data from your hard drive, ensure that you have backed up the wallet.dat file, or you risk losing your bitcoins in case you received them on the addresses generated by Bitcoin-Core.
Ubuntu
The most easy way to remove Bitcoin-Core from Ubuntu is by running:
sudo apt-get remove bitcoind.
When you run bitcoind, it typically (by default) creates a .bitcoin directory in the home directory of the running user unless you have explicitly specified the datadir to where the software will download the blocks and other data. If you're not sure where this path is, you could run the following command to help find it:
sudo locate .bitcoin.
After finding the path you can purge the entire directory by running the command: sudo rm -rf /path/to/.bitcoin.
Windows
Navigate to Start --> control panel --> programs and features and scroll down the list to where bitcoin core/bitcoin-qt is listed, right click and click "uninstall or change a program" and then try uninstalling it that way.
In Windows, by default the block data is stored in the directory C:\Users\your_username\AppData\Roaming\Bitcoin. If you want to delete the entire existence of Bitcoin blockchain from your machine, you can delete the entire folder.
Finally, run regedit (win + r -> regedit). In the search, enter bitcoin and delete all entries matching the list.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Fundamentals: Addresses don't hold coins, wallets do?
In the beginning it was my understanding that:

Every address has a private key
Every address has a balance
Every address may send and receive coins.

This is simple.
But now I am using more proper wallets (because you shouldn't reuse addresses) and got a little confused about not having one central bitcoin address. In specific - how does an airdrop work if my holdings are split between all these addresses.
But more generally, what I read was:

Bitcoin transactions can have many inputs.
  You shouldn't think in terms of address balances. The protocol doesn't work in balances internally. If you receive 0.3 BTC, then you have a "0.3 BTC coin" in your wallet, not so much an address with a balance of 0.3.

There are many implications.
What if I add a private key of an address to two different wallets?
Is a wallet not just a keychain of all these addresses and private keys?
I guess not, if so, how would airdrops work, how would you send more coins than one address holds with just one transaction?

Bitcoin does not work on an account model but operates on an UTXO (unspent transaction output) model. When you send bitcoins to an address, what you are essentially doing is locking those bitcoins in a mathematical equation. Spending those coins requires that you provide the correct unlocking condition (most often signature and public key associated with that address). "Address balance" is a higher level abstraction that can be derived from this basic construct although that information does not explicitly exist in the blockchain.

How would you send more coins than one address holds with just one transaction?

When you spend your bitcoins, you have to reference the transaction outpoint (txid and output number) from which you earned those coins and then provide the satisfactory unlocking condition. If you intend to spend more bitcoins than you earned from a single transaction, you will have to reference multiple transactions from where you control those bitcoins in the input field of your transaction until input value > output value.

Is a wallet not just a keychain of all these addresses and private keys?

Yes, a wallet is just a bunch of keys. There are two types of wallets: (1) non-deterministic or random wallets and (2) deterministic or seeded wallets. The first wallet is just a collection of bunch of randomly generated private keys, while the second type of wallet follows a deterministic path of private key generation. The latter approach helps in better management in terms of backup and import. You can read more about HD wallets in BIP 32 and how mnemonics can be used to generate HD wallets here.

What if I add a private key of an address to two different wallets?

Just consider that you have a normal key that open a lock. You create a duplicate of that key. Now, both of those keys will be able to open the lock. Similarly, both the wallets will be able to control the bitcoins locked in that address. You can spend your bitcoins from either one of the wallets.

Thank you! That was quite the comprehensive answer and left me with no questions so far!

This is such a beautifully clear answer that I refrained from marking the question as a duplicate of Where are bitcoins stored


text:
How does Bitcoin utilize cryptography and secure hash algorithm (SHA) to link records in the blockchain?
I am wondering how Bitcoin uses cryptography and secure hash algorithm such as SHA256 to link the records in the blockchain. And if possible I think it will give me a better understanding if there is a demonstration of a figure of blockchains.

A Bitcoin block is a container data structure that aggregates all the transactions. The block is made of a header (containing metadata), followed by a long list of transactions that are included in the block. The block header consists of merkle tree root, previous block header hash, timestamp amongst other metadata.
The Merkle tree root in simpler terms includes the summary of all the transactions in the block using the merkle tree. Every transaction in block is serialized and hashed twice with SHA256 to get the transaction id or txid. These txids of all the transactions in the block is then double-hashed pairwise, with SHA256, until there is only one node at the top called a merkle root. Since this merkle root is built from all the transactions in the block themselves, the root can be thought as containing the summary of all the transactions included. An illustrative example of Merkle root calculation is shown below:

The header portion of the block is hashed twice with SHA256 to get the block header hash (in the mining process). This block header hash is always referenced in the next block in the previousblockhash field. Thus every block is linked to the previous block using its header hash forming a link that goes all the way back to the genesis block like the illustration shown below:

Since SHA256 is a one way mathematical function, there is no way to reverse engineer the data by just looking at the hash. SHA256 is also deterministic, in the sense that a particular data will always give a particular hash. Slightly changing the data will completely change the hash output.
So, if a party wants to change some data in a transaction that is included in a block somewhere in the blockchain, it will change the merkle root of that block. Since merkle root forms the part of the block header, changing the merkle root will completely change the block header hash. Since this hash is referenced in the next block, the current hash will be incompatible with the hash that was referenced and hence this block will not be accepted.
This is where proof of work in mining plays a key role. As I have said above that changing a data in a block completely changes the block header. So if you want to change a data in a past block, you will have to re-mine all subsequent blocks. So you will have to spend the electricity and compute cost to re-mine all the block AND outpace the network by forming the longest chain. The cost of doing such an action is very high, and may not outweigh the benefits that you derive from it, thus providing inherent security to Bitcoin.
Also, I would like to point out that you cannot change transaction data structure at your whims and fancies. Every transaction is signed with a private key that includes the entire transaction data as a message. So, slightly changing the transaction data will render the transaction invalid and including such invalid transactions in a block will render the block invalid. So, in a nutshell all that you can do is include/exclude certain transaction from previous blocks, or make slight cosmetic changes (like ECDSA signature malleability).


text:
Chain Race: How to know chain-length of competing chain?
If a fork happens, miners mine on the block they receive at first.
Further, they only forward the block they received first (so the block they will use to mine)!
So it can happen that 10% of the network will mine on the chain A. The other 90% will mine on chain B. So only 10% know about chain A and 90% know about chain B. And some edge node will know about chain A and chain B... Depends on how the overlay structure looks like.
What happens if now the 10% fork (chain A) will mine a block and send it to the network? Why won't the other nodes discard the block from chain A? How do they know that this chain is longer? How do they know that the block is not just some part of DDOS attack?
If someone could give me a detailed describtion how the conflict is solved, I would be very happy.

I think this is a good lecture on blockchain fork

The Paper "Majority is not Enough: Bitcoin Mining is Vulnerable*" says "The Bitcoin protocol prescribes that when a miner knows of multiple branches of the same length, it mines and propagates only the first branch it received."
In your tutorial it is described that every node receives every block? So they all have the same knowledge about all forks?
Is the paper wrong?

What happens if now the 10% fork (chain A) will mine a block and send it to the network?

When a miner building on top of chain A finds a valid block, it will relay the block to the rest of the network. For convention let us assume the fork (chain A and chain B) has happened at block height N. This is how different nodes on the network will react when they receive a block at height N+1 whose parent is on chain A.

Nodes already using chain A: For them, this block N + 1 is a natural progression of the blockchain and as a result they will just add this block to their blockchain.
Nodes using chain B but has chain A block: There might be some nodes who might have received the chain A block at height N after receiving block at height N from chain B. Since these node received block from chain B first, they will use this version of the chain as their primary one. However, block from chain A that is received later is not discarded but is processed and kept on the disk. So, when block at height N + 1 is received from chain A version, these nodes just revert the chain at height N and use the other block that was kept on disk.
Nodes only having chain B: When they receive block at height N+1 from chain A, these nodes will see that they do not have its parent. As a result, these nodes will query the parent block using the previousblockhash and see if it exists. The nodes having this block will then transmit that block to the nodes not having it. When that block is received, these nodes will see that this chain is the longest proof of work chain and as a result they will now operate on chain A. The block at height N from chain B becomes stale but it is not discarded and is still kept on disk.

How do they know that this chain is longer?

Nodes accept the longest proof of work chain (in most cases it is same as the longest chain). Nodes sync headers first and verify the proof of work of the chain before asking the actual blocks from other nodes. When nodes on chain B version of chain receive the header of block N+1 on chain A version, they will see that the chainwork of the block received is more than that of the chain they already have and as a result will use it to query the block associated with it.

How do they know that the block is not just some part of DDoS attack?

When they receive the block header, the proof of work is verified first. So, if the attacker wants to spam the network with invalid blocks, he/she would have to do so with valid proof of work. That means spending immense amount of energy as other miners but not getting the reward (as the block does not have any ancestor, it will be rejected). If the node receives a block header with invalid proof of work, then the node that has sent this block header will be blacklisted after certain attempts of this mischief. This was actually the basis of PoW in 1990s when it was used to prevent spam emails.

Thanks a lot! This is exactly I was looking for.
What about combining old blocks with sibling attacks? :P

What do you mean? Can you elaborate on it?

I'm not sure how bitcoin is blacklisting nodes. If blacklisting is IP-based, spawning infinitely new identities won't work. If it is based on bitcoin addresses, it may work.
To circumvent calculating always new PoWs for wrong blocks, I just use old ones there the PoW is correct. Then nodes would have to always findout that this is an old block... So they need efficient lookup data structures. Further, the UTXO check would probably crash. I think this attack would not work... ^^ Just an idea...
But if blacklisting is IP-basd I can spoof IP addresses and take down the network?!

@user674907 Bitcoin blacklists nodes from IP. Address is not a node identifier, it is just some mathematical equation to lock bitcoins. I'm unsure as to how the lookup process actually works, but if a node sees some header it has already received then it will not query the block associated with it. I think it would make for a good new question as to how the lookup process works, if it has not been asked on this forum.


text:
Why is my lighting node not routing any transaction?
I have set up a lighting node months ago using LND under the tor network.
My node (Caffeine) looks correctly configured and connected to several channels but it is not routing any transaction. In the beginning, I thought it would simply take some time but after months still nothing!
What could be the problem?

Obviously for questions like this I can't give a 100% certain answer as I would have to look at your nodes logfiles and configuration / state. However I can make some (hopefully) educated guesses and we can see if that helps you.
First of all it seems like you kept the standard base fee and fee rate. Following my analysis and video it seems pretty unlikely that payments are routed over your channels as there are many people currently offering cheaper routing.
Secondly if I read the data from 1ml correctly besides 1 channel with lnbig your channels are all channels that you have created in the past. If you have never paid someone the entire balance of the channel is on your side meaning they can only be used for routing if you also have inbound capacity (which seems to be the case only with the lnbig channel)
Last but not least: it is well known that lightning is still not very wide spread. There are currently only a few services and experimental projects using the technology so that many people don't use it for payments yet. For example my node has optimized routing fees and just earned a couple satoshis from a few hundred routing attempts over the last 6 months. Roughly half of the routing attsmps actually fail because I have way more inbound than outbound liquidity (the opposite situation from you)
There could be other errors which would need a close look at your logfiles / configuration.

Thanks for the insights. I will try lowering the fees although I am using the AUTOPILOT function of LND and I though it would do that by default. As soon I can I will post the configuration file.

The autopilot of lnd does only suggest nodes to open a channel with. Maybe the current Verdun also suggests amounts. It does not adjust your routing fees for each channel which would be a nice feature. I wrote the autopilot for clightning and I might update the pilot to also adjust / optimize fees

That would be great!

I have lowered the fee substantially but I can’t see any effect yet. I will wait more. Given your experience how long should it take to route at least one transaction?

The update channel messages can take a couple minutes to hours to propagate through the network. Also keep in mind that you also need some inbound liquidity. As I mentioned it looks like you only have inbound capacity from lnbig who provide inbound capacity to many other nodes. So you could use a service like loop to rebalance your channels

Can you please post the link to this "loop"? I still not get the "inbound liquidity" problem. I thought It would be enough to link a wallet with some funds, which I did. Apparently, it is not the case.

Mir info on this site at https://bitcoin.stackexchange.com/questions/87537/ive-been-reading-about-loop-in-and-it-seems-to-be-like-a-one-way-offline-chan and at https://bitcoin.stackexchange.com/questions/85912/how-can-lightning-loop-out-help-me-to-receive-more-payments the second link also refers to background about inbound capacity and a link to lightning labs article explaining loop


text:
When does miner insert the coinbase transaction?
When a miner creates the candidate block with transactions from the mempool, does he insert the coinbase transaction after winning the PoW or during PoW?

If you add the coinbase transaction later, that will change the merkle root. If merkle root is changed, the block header is changed and hence the header hash you calculated earlier is no longer valid. This is why the coinbase transaction needs to be entered before you start calculating the hash of the header.


text:
How to extract raw script from tx messages when the output script starts with 0xA9?
I'm working on extracting output addresses from tx messages in BITCOIN packets.
Currently, I extract address from output scripts which starts with 0x76(OP_DUP)
like this :
76 a9 14 6d 1d 74 58 95 6e 80 cd b4 c3 3f 1e d5 8e 91 c4 92 1a 85 d0 88

but i don't have any idea about some output scripts which start with 0xA9(OP_HASH160).
As an example, i don't know in below example, what is the raw script, which is used to generate output address?
a9 14 16 52 33 02 f2 ee d0 e0 aa 6a 4c 1d 0a 41 39 6f c2 6f 53 6e 87

How i can extract output address from output scripts which start with -xA9?

The first example that you have shown is a P2PKH address. The 0x14 indicates how many bytes to push onto the stack. 0x14 is 20 in decimal so the script pushes the next 20 bytes on the stack which is 6d 1d 74 58 95 6e 80 cd b4 c3 3f 1e d5 8e 91 c4 92 1a 85 d0.
The second case is a Pay to Script Hash example (P2SH). The script is HASH160 Push 20 bytes <redeem_script> OP_EQUAL>. Hence your redeem script in the second case is 16523302f2eed0e0aa6a4c1d0a41396fc26f536e. You would need to then Base58Check it with prefix of 0x05, this gives you the address as 33j3G4xMEn4CtKL5iYnsr97ww67wYh8TPY.
Below is the python script that gives you the address:
import base58
from hashlib import sha256

redeem_script = '16523302f2eed0e0aa6a4c1d0a41396fc26f536e'
version = '05'
payload = version + redeem_script
checksum = sha256(sha256(bytes.fromhex(payload)).digest()).hexdigest()[:8]

pre_encoded_address = payload + checksum
address = base58.b58encode(bytes.fromhex(pre_encoded_address)).decode()

thanks, can you splain how you calculate "33j3G4xMEn4CtKL5iYnsr97ww67wYh8TPY" from this raw script : "16523302f2eed0e0aa6a4c1d0a41396fc26f536e" ??

That's just Base58Check. You first append 0x05 to the redeem script. Then you double hash it with sha256 and then append the first 4 bytes at the end to 0x05+redeemscript. Those first 4 bytes is the checksum. Then you base58 encode it.

Thank you a lot !
you mean that
redeem script is "16523302f2eed0e0aa6a4c1d0a41396fc26f536e"
then  A1 is equal to "0516523302f2eed0e0aa6a4c1d0a41396fc26f536e"
then A2 is  sha256(sha256(first))
and then checksum is first 4 bytes of A2.
then answer is base58encode(first+checksum)
.
but answer is not equal to "33j3G4xMEn4CtKL5iYnsr97ww67wYh8TPY" !

@Saeed I have edited my answer with a python script that shows you how to get the address from the redeem script. I think the common mistake most people do is hashing the plain script rather than using the bytes from the hexadecimal.

Thanks a lot. it is correct. but, what about the other op codes?
it seems that each op code has a different method to extract an output address.
how i can extract address from other types of op_code like "0x00" and "0x40" and etc?

This seems to be a classical P2PKH script which start with op_dup flowed by op_hash160 if you look at the next byte there is the hex value 14 which is 20 in decimal and means read 20 byte of data and push it to the stack.
I have created a video in the past showing step by step how to dissect a P2PKH tx. Look: https://youtu.be/1n4g3eYX1UI

yes, it is correct about my frist example.
but, what about my second example? which starts with 0xA9 and then 14 apperes?
i assumed it line the first example and i read 20 byte of data to push it to stack, but the result output address is incorrect

I think the general rule you have been missing is that if a hex value does not translate to an op code you should read that many bytes of data and push it to the stack.


text:
Why a specific bitcoin value of a tx message appears in source IP address and then comes to same IP address as the destination address?
Sometime back, I created two wallets on two distinct cellphones. I sent a specific bitcoin value (A bitcoin) from one cellphone to another one. During this process, I captured the entire traffic.
After that, I extracted all tx values and output addresses. I saw same value (A bitcoin) and its output address (Address B) in the output which was sent from Source IP address (my cellphone) to destination IP address. Also this value and address was sent to some other IPs.
I also saw that this value (A bitcoin) and its output address (address B) is in some tx messages which is sent from another IP address to my IP address (my cellphone). In another words, I see that the specific value and its address return to my address again. It should be noted that this is a spent value and is not an UTXO. Why does my Bitcoin transaction value and its address in some tx messages gets returned back to me?

Since you using wallets on the cell phone, I presume you are running a SPV node. SPV nodes verify transactions using a slightly different method that relies on peers to provide partial views of relevant parts of the blockchain on demand. In order to ensure that the transaction that you have sent has been confirmed in a block, your SPV nodes will use bloom filters and send that to the full nodes connected to it. Those full nodes will then test each transaction’s outputs against the bloom filter. Only transactions that match the filter are sent to the node.
In response to a getdata message from the node, peers will send a merkleblock message that contains only block headers for blocks matching the filter, a merkle path for each matching transaction and then also send tx messages containing the transactions matched by the filter.
It is this transaction that is sent again to the node is what you are seeing in the packet traffic.

Thank you, I understand somewhat, although I'm a little confused because of some terms like SPV node and bloom filter.

@Saeed In simple words its basically your node asking the data back to ensure that transaction is included in the blockchain. For more info, read answers on this forum related to SPV Node and Bloom filters (I did not want to repeat that in my answer) or read this section from Andreas' Mastering bitcoin book. https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch08.asciidoc#simplified-payment-verification-spv-nodes


text:
How to calculate the size of multisig transaction?
Can you help or suggest the formula for calculation of simple multi-sig transactions?
I have an issue to calculate the size of multi signature  2-2 transaction.
I tried to follow this:
Predict multi signature transaction size answer, however it is vague and unclear.
Here are my calculations:
 pubKeySize = 33;
 sigSize = 72;
 n = (m = 2);
 sizeOfRedeemScript = 1 + n * pubKeySize + 2;
 sizeOfScriptSig = 1 + m * (1 + sigSize) + 2 + sizeOfRedeemScript;
 sizeOfInput = 32 + 4 + 2 + sizeOfScriptSig + 4;
 sizeOfScriptPubKey = 2 + /** what is the length of script??  */ len(script);
 sizeOfOutput = 8 + sizeOfScriptPubKey;
 sizeOfTx = 4 + 2 + sizeOfInput + sizeOfOutput + 4;

If to be more precise I didn't get what is SizeOfVarIntFor(lengthOfScript) and what is the  len(script) (how to calculate it?)

Let's re-write and calculate from scratch as the calculations that you have posted might not be self-explanatory in itself and some of them are not entirely accurate. From the question, I presume you are trying to find the size of transaction for spending bitcoins from a multi-sig address.
Redeem Script Size
I think there is an error in the size of redeem script that you posted. You need to include the size of OP_DATA as well which comes prior to each public key. It tells the program how many bytes needs to be pushed to the stack. Let's assume we use compressed public keys which are 33 bytes in size. So the total size of redeem_script is 1 + N + 33*N + 1 + 1 = 3+34*N
OP_M: 1 byte
OP_DATA: 1 byte (pub_key length) * N
pub_key: 33 bytes * N
OP_N: 1 byte
OP_CHECKMULTISIG: 1 byte

scriptSig size
I think there is also inaccuracy in the data you posted for this section. ECDSA signatures (including SIGHASH flag) can be 71, 72 or 73 bytes (73 bytes is non-standard as standardness requires low-S value but still flies by consensus). So from a conservative perspective you need to assume 73 bytes for each signature required. Below is the calculation. So you need 1+M+73*M+1+3+34*N = 5 + 74*M + 34*N
nil_length: 1 byte
OP_DATA: 1 byte (len of signature to follow) * M
signature: 73*M
redeem_script_length: 1 byte
redeem_script_size: 3 + 34*N

Input Size Calculation
This section assumes there is only one input that consumes the bitcoins from the multi-sig address. Total input size is 46 + 74*M + 34*N
previous_out_point: 36 bytes
    txid: 32 bytes
    vout_index: 4 bytes
var_int: 1 byte (script_sig length)
script_sig: 5+74*M+34*N bytes
sequence: 4 bytes

Locking Script Size Calculation
P2PKH: 25 bytes
    OP_DUP: 1byte
    OP_HASH160: 1 byte
    OP_DATA (size of scriptPubKey to push): 1 byte
    scriptPubKey: 20 bytes
    OP_EQUALVERIFY: 1 byte
    OP_CHECKSIG: 1 byte

P2SH: 23 bytes
    OP_HASH160: 1 byte
    OP_DATA: 1 byte
    scriptPubKey: 20 byte
    OP_EQUAL: 1 byte

P2WSH: 34 bytes
    - OP_0: 1 byte
    - OP_DATA: 1 byte (witness_script_SHA256 length)
    - witness_script_SHA256: 32 bytes

P2WPKH: 22 bytes
    - OP_0: 1 byte
    - OP_DATA: 1 byte (public_key_HASH160 length)
    - public_key_HASH160: 20 bytes

TX_out_size
value: 8 bytes
len of script: 1 byte
script length: variable as defined in previous section

Total Transaction Size
Let's first calculate the size assuming no SegWit. Then we will look at how SegWit helps in reducing transaction size.
version: 4 bytes
count_tx_in: 1 byte (variable but since we have one input can be represented in 1 byte)
inputsize: 46 +74*M + 34*N
count_tx_out: 1 byte (variable again)
tx_out (assuming one input to P2PKH): 34 bytes
locktime: 4 bytes

So your total size is 90 + 74*M + 34*N. Assuming M=N (as your question dictates) the size is 90+108*N.
Size with SegWit
With the implementation of Segwit, we now count transaction size in weight units and not bytes. Normal transaction size is multiplied by 4 and witness data is multiplied by one. In the previous example our WU will be 360+432*N.
In segwit, we will discount the scriptSig by 4 and add one weight unit that signifies the  number of witness elements. So our transaction size is
traditonal tx: 360 + 432*N WU
- scriptSig: (5 + 108*N)*4 = 20 + 432*N WU
= witness_stripped: = 340 WU
+ number of witness elements: 1WU
+ scriptSig: 5 + 108*N WU
= total: 346 + 108*N WU

Thank you for such detailed response!  You helped me a lot

what if I have multiple unspent outputs in my multi-sig wallet? Then I have to multiply locking script * K (K - is the number of unspent outputs)
and the total Tx size will be: 4 + 1 + (46 +74M + 34N) + 1 + 34*K + 4  ?

@newbie What do you mean you have multiple unspent outputs? You are creating outputs from the multi-sig or using these ourputs as inputs in the transactions?

second. if I use a few outputs of previous tx as inputs in the future transaction

@newbie No your formula isn't correct then. The way you have defined is using the multi-sig input and creating K outputs from the transaction. If you are consuming the P2PKH output your redeem_script is the DER encoded pubkey which is 33 bytes if compressed and 65 bytes if uncompressed. So use the above formulas I gave and calculate the size.


text:
What is transaction "finality"?
bitcoind IsFinalTx() as seen here checks the properties of a transaction in this order:

Is nLockTime exactly 0? That's final. return.
If it's not 0, is it below the current block height / time? If so, it's final. return.
Ensure that every tx input is exactly 0xffffffff. If so, it's final.

Because these are checked in order with return statements, nSequence won't even be checked if nLockTime == 0. Furthermore, a transaction that is NOT past its nLockTime value could still be "final" as long as all the nSequence values are 0xffffffff
Non-final transactions are not valid in blocks, so my questions are:

Why is a "bad" locktime ok as long as all sequences are final? The tx won't be mineable anyway, right?
What are the implications for replace-by-fee and Check Sequence Verify transactions (when a non-final nSequence value is used)? Does this mean that to use those features, nLockTime has to be set a certain way?
Or, maybe, because nLockTime has to be below the current time/height anyway, this is all just moot and really, nSequence doesn't really impact transaction finality at all.

Why is a "bad" locktime ok as long as all sequences are final? The tx won't be mineable anyway, right?

According to BIP 68, setting nSequence value to 0xFFFFFFFF for every input in the transaction disables nLocktime. So, whatever may be the value of nLocktime, it will have no effect on the transaction as far as nSequence value is 0xFFFFFFFF.

What are the implications for replace-by-fee and Check Sequence Verify transactions (when a non-final nSequence value is used)? Does this mean that to use those features, nLockTime has to be set a certain way?

It considered a transaction has opted in to allow replacement of itself, if any of its inputs have an nSequence number LESS than 0xFFFFFFFE. To use CSV your nSequence value has to be <= 0xEFFFFFFF. If nSequence value > 0xEFFFFFFF, then there us no consensus meaning to the sequence number and it can be included in any block under all currently possible circumstances. So for using nLocktime along with Opt-in RBF, your nSequence value has to be between 0xF0000000 to 0xFFFFFFFD. Refer this for more info on nSequence and how it has evolved in the last few years.

Or, maybe, because nLockTime has to be below the current time/height anyway, this is all just moot and really, nSequence doesn't really impact transaction finality at all..

nLocktime is a transaction level feature while nSequence is an input level feature. So, the program basically is optimized so that individual inputs are not checked if the transaction itself is invalid due to nLocktime. If you check the function below IsFinalTx, it checks the validity of each transaction inputs by verifying that the nSequence numbers are valid.

So a value of 0xF0000000 has "no consensus meaning" -- but it is still signalling RBF right? So it still has mempool/policy meaning. This would be true for all transactions, version 1 or 2?

And, does this mean that ALL CSV txs are necessarily "replace-by-fee" as well?

@pinhead as I said in the answer there is no consensus meaning to the sequence number but Other rules apply. Yes, it will be signalling RBF. This changed meaning to nSequence was done as a part of BIP 68 soft fork and work for version>=2.

@pinhead Yes, all CSV tx signal RBF due to the nature in which the nSequence value is set

Why is a "bad" locktime ok as long as all sequences are final? The tx won't be mineable anyway, right?

With a final nSequence, the transaction will be mineable no matter the nLockTime.

What are the implications for replace-by-fee and Check Sequence Verify transactions (when a non-final nSequence value is used)? Does this mean that to use those features, nLockTime has to be set a certain way?

No, it's not required.
Regarding the implications, if the nSequence is:

= 0xffffffff, then the transaction is final no matter the nLockTime.
< 0xfffffffe, then the transaction signals for RBF.
< 0xefffffff, then the transaction signals BIP68 relative locktime. More here.

Or, maybe, because nLockTime has to be below the current time/height anyway, this is all just moot and really, nSequence doesn't really impact transaction finality at all.

It does. It impacts finality by deactivating the check for nLockTime if it is set to the maximum value, and since BIP68 (consensus soft fork) was activated on the network it may impact finality by specifying a relative lock time.


text:
Do full-nodes process all transactions before broadcasting the new block header?
If so, why not do it after (re)broadcast? I think it would cut down on lag time.

Since the node would still validate the PoW and the previousblockhash, it would not be possible to DoS the network with invalid headers
Miners could begin processing the candidate block from its header and the list of transaction hashes since they already likely have > 90% of those txs.
Miners and full-nodes would still only update their head once all the txs are validated.
Miners could determine how much time (=money) they are losing by comparing the time they receive the header and the last tx.

Do full-nodes process all txs before broadcasting the new blockheader?

It depends. For nodes having protocol version 70015 and higher, the peer will announce the compact block even before full validation of the transactions contained in the block. However, the header has to be conform with the difficulty requirement. For nodes using protocol version 70015 and lower, the blocks are fully processed before broadcast.

Since the node would still validate the PoW and the previousblockhash, it would not be possible to DoS the network with invalid headers

Yes, that is exactly what Bitcoin full nodes do. Whenever you come online, or receive a new block header, the block headers will be verified first to see that it has the required PoW. As you mentioned this does help prevent DoS attacks.

Miners could begin processing the candidate block from its header and the list of transaction hashes since they already likely have > 90% of those txs.

Miners actually do that. Most of them will build and start processing the new block before even verifying the old one so as to not waste any time mining. However, there will be some transactions in that previous block that the miner hadn't seen in the past. Although those transactions will be very few, the miner would not know which UTXOs have been consumed by those transactions.
Thus miners would start building an empty block (containing only the coinbase transaction) on top of the block that they have just received which hasn't been fully verified. After the previous block gets verified in parallel, the miners will then add the unconfirmed valid transactions from the mempool to the block that they are currently mining.

Miners could determine how much time (=money) they are losing by comparing the time they receive the header and the last tx.

As I mentioned above, it is not only the time but also the uncertainty related to what UTXOs have been consumed by the block that had transactions that the miner did not see in the past. As a result miners have to start building an empty block before fully verifying its parent.
However, it is important to note an incident that happened when blocks does not get fully verified. When BIP 66 (strict DER encoding of signatures) was implemented, some miners running the BIP 66 enabled Bitcoin Core versions were listening to pre-BIP 66 pools. One of the pool running a pre-BIP 66 version produced a block that had an incorrect block version number. The miner with BIP 66 started working on top of that unverified block by processing an empty block.
However, their bitcoind never accepted the full block as it was invalid. The result was building a series of empty blocks on top of the other that were ultimately not accepted by the network.


text:
Under what circumstances will a full node disconnect a peer?
I know a full node will disconnect from a node if that node "misbehaves". For example, a node's connection will be dropped if it sends too many messages that don't follow the rules (what rules?) and nodes that send a lot of expired, duplicate, or otherwise useless messages. What other reasons do full nodes disconnect from a peer?
For example, do full nodes disconnect from connections that don't send enough data?

For a node to be disconnected the default threshold of banscore is 100. In versions prior to v0.18, the default time to keep the misbehaving peer from reconnecting is 86,400 seconds or 1 day. However, this was circumvented by attackers with multiple IP addresses.
v0.18 release, allows peers that your node automatically disconnected for misbehavior (e.g. sending invalid data) to reconnect to your node if you have unused incoming connection slots. If your slots fill up, a misbehaving node will be disconnected to make room for nodes without a history of problems (unless the misbehaving node helps your node in some other way, such as by connecting to a part of the Internet from which you don't have many other peers).
Following are the reasons for disconnection, or increasing the banscore.

Peers which provide a chain with less work than the minimum-chain-work during IBD are  disconnected
If the node is providing invalid data like mutated block or data not meeting consensus requirements
If the compact block had a valid header, but contained invalid txs, the peer should not be punished
Ban outbound (but not inbound) peers if on an invalid chain.
Invalid header, invalid block checkpoint, and invalid previous block will result in outright ban
If previous block is missing, the banscore is incremented by 10. This is done so as to prevent DoS attacks
If the peer sends a getblocktxn with out-of-bounds tx indices it will result in a ban
There are other items related to bloom version and bloom size which might result in ban
First message should be a version message and each connection can send version message only once. If not, the banscore is incremented by 1
If the node is sending addr messages with more than 1000 addresses, then banscore is incremented by 20
Sending inv messages that exceeds the max size, then banscore is incremented by 20.
Sending header messages with more than 2000 elements, resulting in incrementing banscore by 20.
Nodes must NEVER send a data item > 520 bytes or misbehaving score is incremented by 100

There are other couple reason which might have been ommited. You can search "misbehaving" in the netprocessing.cpp file for all the reasons.

Do full nodes disconnect from connections that don't send enough data?

Not particularly but somewhat yes. When the chain tip does not advance for over 30 minutes, the full node assumes that the tip may be stale and hence will try to connect to an additional outbound peer. There is a periodic check that ensures that if the extra peer connection is in use, the node will disconnect the peer that least recently announced a new block.

Thanks for the detailed answer! I'm curious why certain things aren't punished, like sending a block header for a block that contains invalid transactions, or forwarding an invalid chain from an inbound connection. Shouldn't we expect honest nodes to fully validate the transactions and blocks before forwarding?

@BT you can read more about it in BIP 152. But, the essence is that currently nodes sync header first and CMPCTBLOCK messages are forwarded by validating the the block header hash without full block validation. This is to promote faster propagation of blocks.


text:
Does Bitcoin script support nested IFs (OP_IF or OP_NOTIF inside one another)?
For example can we convert the following code to a script?
if(bool_1)
{
    FOO1();
    if(!bool_2)
    {
        FOO2();
    }
    else
    {
        BAR1();
        if(bool_3)
        {
            BAR2();
        }
        BAR3();
    }
}
else
{
    FOOBAR();
}

<push> <push> <push> OP_IF OP_FOO1 OP_NOTIF OP_FOO2 OP_ELSE OP_BAR1 OP_IF OP_BAR2 OP_ENDIF OP_BAR3 OP_ENDIF OP_FOOBAR OP_ENDIF(correct?)
If yes then:
1. Is there any limit on how deep a nested IFs can go?
2. Is there any examples (raw bytes/hex) I could look at?

Do bitcoin scripts support nested ifs (OP_IF OP_NOTIF inside one another)?

Yes, Bitcoin support nested IF/ELSE. In fact we use the nested IFs in Lightning Network all the time when adding HTLCs to forward a payment.

Is there any limit on how deep a nested if can go?

There is no explicit rules dictating how far deep the nested IFs can go but there are some constrains on the script itself:

Maximum number of bytes pushable to the stack should be less than 520 bytes
Maximum operations per script is limited to 201
Maximum size of the script is 10,000 bytes
Maximum number of values on script interpreter stack is 1,000 bytes

Is there any examples (raw bytes/hex) I could look at?

As I mentioned earlier, we use nested IFs all the time while adding HTLCs in the Lightning Network implementation. Below is an output script of offered HTLC outputs for a peer.
# To remote node with revocation key
OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
OP_IF
    OP_CHECKSIG
OP_ELSE
    <remote_htlcpubkey> OP_SWAP OP_SIZE 32 OP_EQUAL
    OP_NOTIF
        # To local node via HTLC-timeout transaction (timelocked).
        OP_DROP 2 OP_SWAP <local_htlcpubkey> 2 OP_CHECKMULTISIG
    OP_ELSE
        # To remote node with preimage.
        OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY
        OP_CHECKSIG
    OP_ENDIF
OP_ENDIF

IF/ELSE loops Is "loop" the correct term to use?

@CodingEnthusiast thanks for correcting. I wrote the answer from my phone.


text:
Can we construct a UTXO which is spendable only via proof of work?
Do the current Bitcoin opcodes and transaction size limit allow constructing a UTXO which is only spendable if a specified amount of work has been performed?
So the spender, would provide a nonce as input to the script. Then the script would essentially perform a highly simplified version of how bitcoin's block headers are verified (the nonce is combined with some other data, hashed, and the hash is compared with a "target.").
Without OP_CAT and with arithmetic limited to 4 byte inputs, do I understand correctly that the above would be impossible? Or is there some workaround?

You could add an op return to your tx with the first 200 bits of the private key. A spender would have to grind the last 56 bits until they find the right private key to sign with. Combine with multisig or Schnorr tweak to target a single recipient!

@pinhead thanks! That's a clever way to do it. Indeed it would require proof of work to uncover the key. However, in this case everyone would be guaranteed to ultimately end up with essentially the same "nonce" (which is just the full private key), so wouldn't a mainchain miner just be able to steal the reward?
Or, put another way, can you clarify what you mean about using, for example, multisig so that it targets a single recipient? Namely whoever put in the work but also in a manner that the original UTXO didn't need to know who the set of potential workers is. Is this making sense?

Hm that's hard. Miners can steal anything that's not signed. My thought was you could send to a 2of2 multisig where one key is a known recipient and the other key must be brute forced (by that recipient) A miner wouldn't be able to steal that.

Again, thank you very much for your answer. This is extremely helpful. So the challenge have with that construction (the 2of2 method you proposed) is that I would need to know the key of the recipient ahead of time which is not what I want. Is there a way we can sort of chain a series of these types of transactions together to accomplish that?

What is the application of this? I’m curious to know what you’re trying to accomplish

hi @chytrik, thanks for your interest. There are a couple applications I can think of. The primary one I'm most interested in though is eluded to in my other question here https://bitcoin.stackexchange.com/questions/91069/are-there-any-slow-sidechains-what-are-the-tradeoffs  Basically I want to determine if there is any way whatsoever that bitcoin as-is could verify even a single block header of a purely proof-of-work (as opposed to federated) sidechain. The question here then is just a reduced form of that question.

The answer depends on what you are trying to accomplish with this construction. I have three guesses in this regard:

a set of determined recipients can spend if they perform the PoW: you can have a transaction that can be spent if someone provides: a value whose hash is less than a specified amount (that determines difficulty) AND a signature that describes the set of allowed entities (potentially any multisig).
anybody that performs the proof of work can spend (AND you can spend without PoW): this is trickier since a simple hash based PoW would not work, a miner can just change the destination address of the spending transaction to one under his control. You need to prove that you know a secret (the result of the PoW) without actually including it in the tx so that miners are not able to take the money without repeating the PoW: this is what digital signatures accomplish. As suggested in the comments you can contsruct a utxo that can be spent by signing with a certain private key known by you alone and that contains part of it under an op_return. So one has to try any private key in the set of the allowed ones until he can produce a valid signature. This does not leak the private key itself so the miner cannot sign the same tx with a different destination address.
anybody that performs the proof of work can spend (AND you can NOT spend without PoW): I thought about this a while and I can't find an easy construction for this, it might be hard even with stronger scripting capabilities than bitcoin's. My best guess is that you have a provably random public key (for instance using a VDF on the nonce of the last block) and everybody has to brute-force the private key to spend. I see two major issues: miners can have some advantage in selecting nonces to influence the randomness of the PK and it's hard to tune the difficulty, you need a custom signature scheme that has the security level you want and that's not trivial with bitcoin script.

This is NOT a complete list, it's just the first ideas that came to my mind. Especially the second case, which looks the most likely to me, can be constructed in several different ways: for instance you could also publish the hash of the private key in an op_return (so the private key doesn't have to be included in the transaction) and you would turn it into a hash-based PoW since hashing is a lot faster than signing.

thanks for your answer. For the first bullet point, you mention comparing if the hash is less than a certain value. However, the hash is a 256 bit number, yet the numerical comparison operations in bitcoin script are limited to 32 bits. So I am not sure that this option would work at all?
Of your scenarios though, the third one is the one I am trying to do -- anybody performs the work AND I cannot spend without also performing work. So far I am stumped on it.

I had imagined the desirable case is the third one. Not an easy task at all.
You get me unprepared as for the numerical comparison matter, but at the cost of writing an insanely convoluted script I guess you can workaround it. If you have enough of 32 security bits then you can just take the last 4 bytes of the hash and compare them to the target. Otherwise you can take the last two 4 bytes blocks and combine two checks.

This is an interesting approach (though, as you suggested, it sounds like the script may end up quite bloated/convoluted). However, I think the "take the last 4 bytes of the hash" is where it may break down. Pushing 32 bytes (the hash) on to the stack seems possible. However, pulling 4 bytes off at a time and comparing them seems not possible, unless I am mistaken? Again, thank you for your thoughts on it.

I'm definitely no bitcoin script expert, but from a quick look it actually seems not possible, I don't think I can help further. It might become easier with something like scriptless script, but I really know too little about it.

Yes, with the coming (eventual) upgrades, something like this may be possible it seems. Until then, I am not sure either. Thank you for your thoughts so far on it!

To build the proof of work in a way that is also secure to miners' attacks you can still do what was suggested in the first comment. You can add to a normal tx an output containing part of the private key. This way one has to bruteforce the remaining bits to be able to produce a valid signature and miners have no significant advantage. In this case the creator knows the private key for sure, no way around it.
Why are you not considering this direction?

Sorry for my delay. The primary reason why that direction is less interesting to me is precisely because the creator would know the private key. So it would not be a 'fair competition' among those who are trying to complete the work.

Maybe there could be a pay to script hash in which the requirement to solve the difficulty required mining on the most recently produced block. You could have the competing miners hash the most recent block and need to come up with a nonce that creates the level of share difficulty needed to spend reward. That way there could only be one winner after each new block, difficulty can be adjusted per block, and no one could "fake" a proof of work that met these requirements. If someone provides a copy of the last block with an extra nonce attached that meets the competition difficulty return true.

To elaborate, using a whole btc block is not realistic (unless you hash a compressed version?), but a merkle proof of data from the block should be enough( which is basically a compressed checksum of sorts). Although the size of this data may add to the difficulty of solving a share which needs to be considered.

@Poseidon I don't think that mechanism can be used to lock funds though, as it would require  something like "hash the previous block header with this nonce and count leading zeros" to be encoded into the bitcoin script locking funds.
Actually, the hash of blocks is already computed on the blockheader, which has some metadata plus the merkle root of transactions. This way we can build compact (O(log n)) proofs of inclusion of a transaction in a block without sharing the whole block.

I'm not fully following why proof of work cannot be used to lock funds. Lets use the bitcoin proof of work as an example. when someone solves a proof the network receives their block and compares it against blocks with proofs from other miners. the one with the largest chain wins meaning the one with more transactions and more work. This same principle can be applied to the spending of script locked coins, you create a challenge so that the proof meets the target set with bits and that it also includes a signature, add the pubkey of the signer to the proof and you have an unbreakable claim.

To manage competition you need rules so that the provers can maybe add more data to their message that they are hashing, or have the script logic pick the most golden nonce so the one with the highest amount of proceeding zeroes. because of the logic in proof of work as long as you get users to commit to signing new messages each block no one could reuse or use someone elses work. (unless people form pools at which point they are forfeiting their work)

To be specific the directions of hash the previous block header with this nonce and count leading zeros are not what happens in proof of work. only the message which in this instance is the block header would be shared amongst the parties participating in the challenge, the nonces are selected by each individual otherwise it would be completely insecure. My idea of using previous block header was just to get a chain of data relating to the underlying chain that would be persistent, deterministic, consensus based, and relatively compact. This chain can be used to set the message automatically

Effectively each block there is a competition to submit the transaction that claims the funds with the highest amount of difficulty and claims their work via their own signature. Its pretty much just how bitcoin proof of work works, consensus and competition.

Exactly. Bitcoin full nodes contain some consensus logic so that each block showing enough proof of work can contain an inputless (coinbase) transaction giving block reward to some addresses.
I don't see any other place where proof of work can lock funds since all non-coinbase funds can only be locked by fulfilling a bitcoin script and I see no way to implement a proof-of-work verifier in bitcoin script.


text:
How to search for TXs using only the PublicKey linked to those TXs?
The quest is to define a method for searching TXs on the Bitcoin blockchain using only a PublicKey in HEX format - as opposed to searching TXs by popular terms like  wallet ID, txid, hash, etc.
I haven't found any such method. Does it exist?
Scenario:
Bob holds a compressed PublicKey of the form
02af31b067645ffe4eb534d609da80332e5c5747543fd97385e832c6fdf2d4b4bf
Hypothetically, Bob has no other info at his disposal and he wants to query the Blockchain to find all TXs associated with the above PublicKey.
If Bob succeeds, he finds this Transaction
Is this type of searching the blockchain possible?

In fact, the usual way to consult blockchains is using the public key (purportedly the public key of the searcher) as the result will be all the transactions in which the key is involved:
https://www.blockchain.com/btc/address/1EwpnNBdFJykwxp6X8v9AfZnup9bgmrLE1
I suppose you're puzzled about the way to convert the public key to the public address, as the latter is the one used in blockchain explorers (I don't know if there's some one that can use the public key as input) :

You can use online tools like this one: https://iancoleman.io/bitcoin-key-compression/
Or some code, for example using my very simple Python scripts bitcoin-in-tiny-pieces, in this case, as the prefix of the compressed pub key is 02 we know that y is even, and as y is not necessary in compressed form we can use 0x00 as y coordinate:

:
$ ./bitcoin-address-from-public-key.py af31b067645ffe4eb534d609da80332e5c5747543fd97385e832c6fdf2d4b4bf 00
pubkey = 04af31b067645ffe4eb534d609da80332e5c5747543fd97385e832c6fdf2d4b4bf00

Compressed public key:
key_hash =      98f7fbd15e7b30a4ed55aec7baae40729967ba67
checksum =      c199dbe6fc87793152351aaa25b4ce28e395225dd3abcb4606b159e1ac26b346
key_hash + checksum =   0098f7fbd15e7b30a4ed55aec7baae40729967ba67 c199dbe6
bitcoin address =       1EwpnNBdFJykwxp6X8v9AfZnup9bgmrLE1

In case you want to ask directly blockchain.info from command line it is possible to combine various scripts:
C:\> bitcoin-address-from-public-key af31b067645ffe4eb534d609da80332e5c5747543fd97385e832c6fdf2d4b4bf 00 | bitcoin-get-address-balance

address         = 1EwpnNBdFJykwxp6X8v9AfZnup9bgmrLE1
total_received  = 1.57390781 Bitcoin
final_balance   = 0 Bitcoin

In these examples I've modified two lines in bitcoin-address-from-public-key.py with COMPRESS_PUBKEY = 1 and VERBOSE = False.

Thanks but I am aware of the facts mentioned in your reply. What I want to know is how to query the blockchain (if possible) using the raw HEX public key as a single search term. As you said, blockchain explorers only allow searching for the hashed public key / address ID, TX id, etc. Perhaps some code is out there but cannot find anything on GitHub or Google.

The proposed Python script code doesn't fit your requirements? You could combinebitcoin-address-from-public-key.pywith bitcoin-get-address-balance.py to directly ask blockchain.info from command line ! I'm going to add this to the answer.

Yes, thank you so much. Your script is great and it does the job.It asks the blockchain directly and you are right this can only be done from command line. I will mark your answer :) The only issues I have is 1) when I enter the public key above it returns a diff address 18eCccERvefbyrgL15eK2exczQge9gQit5. And 2) how do I combine your two scripts on Ubuntu, in python so I can reproduce your  results
total_received  = 1.57390781 Bitcoin final_balance   = 0 Bitcoin I can see you used Windows in your example?

final edit: All done now, I managed to combine your scripts and also it now returns the correct result. Thanks again.

If the transaction outputs associated to the public key has not been spent, then finding the funds associated with that public key will mostly be a trial and error method.
Searching in outputs
Most outputs in Bitcoin send funds to an address that is the hash of the public key. Pay-to-public-key (P2PK) is the only standard output that sends funds directly to the public key, but is rarely used. Now, I said finding funds from the outputs is a trial and error method, because you do not know what type of address is being used by the person who controls that public key. He may be using P2PKH (addresses starting with 1), P2WPKH (addresses starting with bc1) or P2SH-P2WPKH (addresses starting with 3). You would need to generate those addresses from the public key and see if it matches to that using any explorers. Also, the user might have used the uncompressed public key version to generate the P2PKH address, so you would need to search for that as well.
Moreover, if funds are controlled in multi-sig addresses, where one of the public key involved is the one you know, it is impossible to calculate the address without knowing the other public keys AND the spending condition (m-of-n).
Searching in inputs
However, if the funds are spent, the task is 'easier'. You just have to go through all the inputs of all the transactions, and check that the public key was used to satisfy unlocking condition. Most inputs will have their scriptSig (witness data in case of SegWit) as <sig><pub_key>. You can match the public key against the one that is mentioned in the scriptSig.
This is also helpful with analyzing the public key for multi-sig transaction, if that public key was involved in the spending. However, there is still a gap involved. If the public key was say used in 2-of-3 multi-sig, and the transaction was spent with the other two keys, then there is no way of knowing, whether that public key was involved in that transaction at all.


text:
How to re-connect to `bitcoind -printtoconsole` after socket timeout?
I started bitcoind with the option -printtoconsole to check for errors and my terminal ssh connection was timed out during the night - probably because my computer was in hibernation.
I see that the process is still running under top.
Is there an option to reconnect so that it will print the output to a new ssh connected shell?

I don't think you can reconnect to it easily, but for next time, you could create a screen session and start bitcoind inside of it. That way you could reconnect to it with screen -r

You cannot reconnect, but everything that is printed with -printtoconsole will also be written to the debug.log file. So you can just do tail -f debug.log to get the same output.
If you wanted to monitor for errors in this way, I would suggest that you always do tail -f debug.log instead of using -printtoconsole. The debug.log will always be written to so you can look at previous logs while you are offline/away. And it is still written to when Bitcoin Core is daemonized (with -daemon) or otherwise running disconnected from your shell (as in your case).


text:
How to request a list of blocks using bitcoin-cli?
I'm trying to figure out how to use bitcoin-cli effectively to request a list of 50 blocks.
As far as I am aware, bitcoin-cli getblock list  does not work. Also how to best extract information out of the getblock output.

You cannot as there currently no way in Bitcoin Core's RPC to get multiple blocks with one command. You will have to issue multiple getblock commands. If you use a JSON-RPC client programmatically, you can use JSON-RPC's batched requests in order to have to only send one JSON-RPC request to execute multiple RPCs and get the results back in one response.
As for how to extract the information, you will need a JSON parser to extract the information. You can use verbosity level 2 in order to get the transactions decoded as well instead of just their txids.


text:
Clarification on the script P2SH, how is executed to Bitcoin core?
This question is a continuation of the my post yesterday.
Today I have a question on P2SH execution so, an example of the script should be this
OP_0 <A Signature> <B Signature> OP_2 <Public key A> <Public key B>
<Public key C> OP_3 OP_CHECKMULTISIG
OP_HASH160 <ScriptSig Hash> OP_EQUAL

or if used the P2SH key should be is this (I think in this script left-hand something to convert the hash160 inside the P2SH key, right?)
OP_0 <A Signature> <B Signature> OP_2 <Public key A> <Public key B>
<Public key C> OP_3 OP_CHECKMULTISIG
OP_HASH160 <P2SH key> OP_EQUAL

Now if I have understood well the simulation of the execution is divided into two phases

The hash scriptSig is equal to hash inside the scriptPubKey

Simulation

Put on the scriptSig on the stack, so the stack now is <A sig> <B sig> <A pubkey> <B pubkey> <C pubkey>
Calculate the hash160 with the data inside the stack, so now the stack status is <scriptSig hash>
Put on the scriptSig hash expected, now the stack status is <hash scriptsig> <hash scriptsigexpected>
return the result of the operator OP_EQUAL (true or false)

the scriptSig will execute such as the script multi-signature

Simulation

put on the stack the scriptSig, the stack status is <A Signature> <B Signature>
push the public keys, that stack status is <A Signature> <B Signature> <Public key A> <Public key B> <Public key C>
apply the OP_CHECKMULTISIG operator and get the result

My questions are:

With the P2SH key, the execution is the same? if not, what is the form of the P2SH with the P2SH key?

Also, I read this post but I don't think contains this answer.
If my example script is wrong, I'm sorry to give me a corrections

Let's go over this step by step. Suppose say you want to lock some bitcoins in a 2-of-3 multi-sig.
Locking steps

Create a multi-sig script: OP_2 <pub_key 1> <pub_key2> <pub_key3> OP_3 OP_CHECKMULTISIG.
Hash the multi-sig script with HASH160 which gives you multi-sig_scripthash.
Base58Check the multi-sig_scripthash with version 0x05 giving you an address starting with 3....
The scriptPubKey will be OP_HASH160 <multi-sig_scripthash> OP_EQUAL.

Unlocking Steps

Your stack will start at <signatures><redeemScript>. The signature is OP_0 <sig_A> <sig_B> and redeemScript is OP_2 <pub_key 1> <pub_key2> <pub_key3> OP_3 OP_CHECKMULTISIG.
Since v0.3.7, Bitcoin Core executes the unlocking script first and looks at the resulting stack. So your <signatures> and <redeemScript> are evaluated together first. This would result in a multi-signature evaluation.
In normal cases the above step should just return 1 as signatures satisfy the redeemScript. But with P2SH there is a caveat added by BIP-16. Whenever you have a OP_HASH160 <hash> OP_EQUAL locking script, it serves as a P2SH identifier. So before the evaluation of <signatures> with <redeemScript> the redeemScript is copied from the stack to another one. Then the evaluation is done with what is on stack (<signatures><redeemScript> in our case). If successful, replace the current stack with the just redeemScript. Hence, instead of 1 you have <redeemScript> at the top of the stack.
You run OP_HASH160 on the redeemScript. Stack is now <hash160_of_redeemScript>
You push <expectedredeemScripthash>. Stack is now <hash160_of_redeemScript><expectedredeemScripthash>
You run OP_EQUAL resulting in evaluation and returning 1 to the top of the stack.

Thanks for your answer, I have a question, exist some operator inside bitcoin script for calculate the Base58Check?

@vincenzopalazzo Base58check encoding is for human readability. It is for humans to distinguish whether the address generated/being sent to is P2PKH or P2SH. It does not have any meaning inside the software.

My fault, I have jump this note "Note: If you just insert the address starting with 3, the software will automatically calculate the scriptPubKey as above."

@vincenzopalazzo What I meant was, as a user you just enter an address starting with 3. The wallet software then transitions it to the scriptPubKey.

@vincenzopalazzo as further clarification, addresses are not sent over the wire nor stored in the blockchain. Addresses are only for human readability. So when a user enters an address starting with 3.., the wallet software decodes the base58check, extracts the redeem_script_hash and uses OP_HASH160 <redeem_scripthash> OP_EQUAL as the scriptPubKey. If your address began with 1, the wallet software will use OP_DUP OP_HASH160 <redeem_scripthash> OP_EQUALVERIFY OP_CHECKSIG as scriptPubKey.

Yes thanks for your clarification, you have help me :)


text:
Clarification on execution of complex script with OP_CHECKLOCKTIMEVERIFY
I'm quoting this example of BIP65
BIP 65 has this bitcoin script code
 IF
    <now + 3 months> CHECKLOCKTIMEVERIFY DROP
    <Lenny's pubkey> CHECKSIGVERIFY
    1
 ELSE
    2
 ENDIF
 <Alice's pubkey> <Bob's pubkey> 2 CHECKMULTISIG

And with this bitcoin scriptSig is possible unlock the transaction

Now with this ScriptSig 0 <Alice's signature> <Bob's signature> 0
after 3 months with this ScirpSig 0 <Alice/Bob's signature> <Lenny's signature> 1

But I don't understand this part of If-then-else
ELSE
    2

When is it executed and what I don't understand inside the if-then-else condition?

Your ELSE loop was indented inside the IF condition, which is not the case in BIP 65. I have edited it to reflect the correct script since you have quoted that example.

Bitcoin uses a stack based language due to which the logical condition that will execute the IF or ELSE statements comes prior to the operator IF or ELSE. Let's take both your cases one by one and see how it gets executed on the stack.
Case 1: ScriptSig: 0 <Alice's signature> <Bob's signature> 0

The stack starts at 0 <Alice's signature> <Bob's signature> 0
Since 0 is on top of the stack, the ELSE condition will execute and drop the 0
number 2 is pushed on the stack
Stack is now 0 <Alice's signature> <Bob's signature> 2
script outside the IF/ELSE statement is pushed to the stack
Stack is now 0 <Alice's signature> <Bob's signature> 2 <Alice's pubkey> <Bob's pubkey> 2 CHECKMULTISIG
This is a normal 2-of-2 Multi-sig operation that will execute by verifying both Alice's and Bob's signature to the public keys

Case 2: ScriptSig: 0 <Alice/Bob's signature> <Lenny's signature> 1

The stack starts at 0 <Alice/Bob's signature> <Lenny's signature> 1
Since 1 is on top of the stack, the IF statement will execute and drop the 1
<now + 3 months> CHECKLOCKTIMEVERIFY is pushed to the stack and verified whether 3 months have passed. If yes, the execution will continue.
DROP operator drops the TRUE on top of the stack if verification is successful
<Lenny's pubkey> CHECKSIGVERIFY is pushed to the stack
Stack is now 0 <Alice/Bob's signature> <Lenny's signature> <Lenny's pubkey> CHECKSIGVERIFY
CHECKSIGVERIFY will consume the top 2 items of the stack. It will check whether Lenny's signature matches Lenny's pubkey. If yes, the execution will continue.
Stack is now 0 <Alice/Bob's signature>
1 is pushed to the stack
Stack is now 0 <Alice/Bob's signature> 1
Script outside the IF/ELSE statement is pushed to the stack
Stack is now 0 <Alice/Bob's signature> 1 <Alice's pubkey> <Bob's pubkey> 2 CHECKMULTISIG
This is now a normal 1-of-2 multisig operation and will execute if Alice or Bob provides a valid signatures to their pub key.


text:
Generating Lightning invoice with an optional parameter
I have a question on the command invoice with the optional parameter.
I don't understand how to create the invoice with the optional parameter. The  c-lighitning documentations do not have an example but it says:

The fallbacks array is one or more fallback addresses to include in the invoice (in order from most-preferred to least): note that these arrays are not currently tracked to fulfill the invoice.

My question is How do I separate the address in the command invoice? As an example:
invoice msatoshi=100 label="I want to understand the invoice command" description="I am vincent" fallbacks=[2NDVm22NNuosAXFbC27Scsn1smMh1QEFZUk,2MymqReM8EaYCQKzv4rhcvafGGcddZacUtV]

The command runs good, but I want to understand if this command is correct.
Below is the command result:
{
   "payment_hash": "9610412eb4f1fa521c76d4194e93f63caae059481cdce6a11dcf369cf94a8aab",
   "expires_at": 1568312884,
   "bolt11": "lntb1n1pwhz5a5pp5jcgyzt4578a9y8rk6sv5aylk8j4wqk2grnwwdggaeumfe722324sdq5fysxzmfqwe5kucm9de6qxqyjw5qcqp2fppjmc3rmhe0wngplqnl3002udz6vj3kzwqsfppjg7ddp2vfds5hgdvsvknhdut0fxpe4z6ndth02e8reukhg4lu39xefwevkknh5r7l2ljugknfutfhhumsfprpxefpqwmjq66nkh5kfsu0esx5a0lnwk7lhhvp8tgvexl9cg8meusqty4ls3",
   "warning_capacity": "No channel with a peer that is not a dead end, has sufficient incoming capacity"
}

You are doing it the correct way. In case you have doubts, you could check if the invoice generated is according to the way you decided by running the command lightning-cli decodepay <invoice> . Below is the result for the invoice that you generated:
lightning-cli decodepay lntb1n1pwhz5a5pp5jcgyzt4578a9y8rk6sv5aylk8j4wqk2grnwwdggaeumfe722324sdq5fysxzmfqwe5kucm9de6qxqyjw5qcqp2fppjmc3rmhe0wngplqnl3002udz6vj3kzwqsfppjg7ddp2vfds5hgdvsvknhdut0fxpe4z6ndth02e8reukhg4lu39xefwevkknh5r7l2ljugknfutfhhumsfprpxefpqwmjq66nkh5kfsu0esx5a0lnwk7lhhvp8tgvexl9cg8meusqty4ls3
{
   "currency": "tb",
   "created_at": 1567708084,
   "expiry": 604800,
   "payee": "023dfbbc3011303a613645671defa34df29934ac19c507c6d42ec88df54a1532df",
   "msatoshi": 100,
   "amount_msat": "100msat",
   "description": "I am vincent",
   "min_final_cltv_expiry": 10,
   "fallbacks": [
      {
         "type": "P2SH",
         "addr": "2NDVm22NNuosAXFbC27Scsn1smMh1QEFZUk",
         "hex": "a914de223ddf2f74d01f827f8bdeae345a64a361381087"
      },
      {
         "type": "P2SH",
         "addr": "2MymqReM8EaYCQKzv4rhcvafGGcddZacUtV",
         "hex": "a914479ad0a9896c2974359065a776f16f49839a8b5387"
      }
   ],
   "payment_hash": "9610412eb4f1fa521c76d4194e93f63caae059481cdce6a11dcf369cf94a8aab",
   "signature": "304402206aeef564e3cf2d7457fc894d94bb2cb5a77a0fdf57e5c45a69e2d37bf3704846022013652103b7206b53b5e964c38fcc0d4ebff375bdfbdd813ad0cc9be5c20fbcf2"
}

However, as the docs point out, the current c-lightning implementation does not use the addresses from these arrays to fulfill the invoice in case no route to the payee is found.  So, if you show your invoice and the payer is using c-lightning, then it won't be able to make an on-chain transaction to you for that particular invoice, if its bolt11 payment fails. This might be especially pertinent in your case as the node that generated your invoice does not have any open channel that is exposed publicly which has an incoming capacity.
Also as a side-note, using fallbacks doesn't make sense for tiny or time-sensitive payments. Since your invoice amount is only 100 milli-satoshi, the payment to fallback addresses will not be routed/accepted on the Bitcoin network as it does not accept dust payments. In your invoice it is a sub-satoshi amount which is also below the smallest unit of Bitcoin.

I hadn't thought of decoding the whole hahaha, what a fool. Thank you

@vincenzopalazzo as a sidenote, since your payment is only 100msat, that payment cannot be accepted on-chain. Fallbacks does not make sense for tiny or time-sensitive payments.

I was just studying the RPC module and so I wanted to understand more about this command :)


text:
How create a personal node c-lighting for create a new channel with other node
I want to create my personal node to create a new channel with another personal node, an example my node with URL 12345678@192.164.1.7 I want create a channel with another personal node.
I could not found some reference for this problem, can you help me for creating my personal node?
I try to add and parameter to command line --addr but this I think is not enough, right?

First you will need to decide which machine will fund the channel. In the current implementation only one party can lock funds to a multi-sig address. Let's say the machine that will fund the channel is A and the other is B. Now open the c-lightning in the terminal on machine B and type lightning-cli getinfo. It will give you a bunch of information. Look for the id and address. Then open the c-lightning on your machine A's terminal and type  lightning-cli connect id@address:9735. Where id and address are the info that you got from the getinfo command on machine B.
After connection has been established, you can then open the channel using the command lightning-cli fundchannel id satoshi [feerate] [announce] on your machine A. Here id is the node id of the node with whom you want to open the channel and satoshi is the amount of bitcoins in satoshis that you want to use to fund the channel. If you want to keep your channel private you can set announce to false.
If you want to expose the node to the internet and use the internal IP addresses you can do so by adding bind-addr=<internal_IP> and announce-addr=<public_IP> in the config files. You can also pass these commands in the terminal when starting the lightningd daemon, but it is easier to do it in config file so that you don't have to do it everytime you start the daemon.

if I want to create my personal node? with a node id and an ip?

@vincenzopalazzo what do you mean a personal node? A node that is not announced?

Yes, I want create an channel on my node

@vincenzopalazzo Can you explain with more clarity as to exactly what you are trying to achieve? I'm trying to understand your issue better.

Mh yes, I want to create on the channel with node c-lightning X on my node c-lightning Y in the local network, my question is how to configure the node Y for accepting the command connect to node X

@vincenzopalazzo I have updated my answer, let me know if you need anything else.

Let us continue this discussion in chat.


text:
Can a malicious Internet Service provider launch a double spend attack?
If a malicious internet service provider completely controls a user's connection, can it launch a double-spend attack against the user? How much computational effort would this take?

Everything sent over the Bitcoin network is public so you have nothing to worry if your ISP monitors your connection (only related to bitcoin transactions; your privacy is compromised in other cases). The only thing which is private are your keys, but that is not sent over the network. Now, if your ISP controls everything that moves through its servers, it can definitely change the components of the transaction. However, when you create a transaction, you sign the entire transaction with your private key. So, if any component of the transaction is altered, that signature is rendered invalid and hence the entire transaction becomes invalid. So, no even if your ISP controls your entire network, it cannot alter your transaction without invalidating it. Just ensure that you keep your keys safe (cold storage like hardware wallets or paper wallet).

If a malicious internet service provider completely controls a user's connection, can it launch a double-spend attack against the user?

No. The transactions that users create all require a digital signature. Creating a valid digital signature requires the private key for the public key that is specified in the transaction. Because that public key is fixed (i.e. cannot be changed), the specific private key(s) that the user has are required in order to modify the transaction.
An ISP does not have that user's private keys. So they will be unable to produce a conflicting transaction as they will be unable to create a valid transaction. They cannot produce a valid signature, so a double spend cannot be made.
The only thing that an ISP can do is censor a user's transactions. All they can do is block network traffic to and from the user as everything in Bitcoin is self contained and proves itself to be valid (or invalid).


text:
Should I verify the Bitcoin Core release signing keys?
I have been running Bitcoin Core on an old Macbook for about a year, and I updated to v0.18.0 a couple of months ago which I downloaded from bitcoin.org.
I have recently been delving deeper and I realized that I never checked the keys.
Is it recommendable/indispensable?
Can I still check the keys once it is up and running?
Will I have to reinstall and download the entire blockchain again?

You should check the signature of ALL software you install so that you can trust your computer. Especially if that machine stores the secret key of a loaded Bitcoin wallet.

Is it recommendable/indispensable?

Yes, it is a MUST not just recommendable. There are a number of ways in which an attacker could modify the binary that you download from the bitcoin.org website.

Attacker can compromise the Bitcoin.org website, so any information hosted on that page could be modified by the attacker for its own benefit
Attacker can compromise the SSL/TSL connection by compromising a Certificate Authority and issuing fake website certificates
Attacker can launch a man-in-the-middle attack on people visiting the bitcoin.org website and tricking them into believing that they have landed on the original website

Once the attacker has fooled you into downloading a clone of the Bitcoin software with a malicious bug, it could steal all of your coins, use your machine to launch attacks on the Bitcoin network, crash or wipe your computer clean or might place a worm in your machine that acts as a surveillance tool.
In fact, Bitcoin Core developers issued a warning during the release of v0.13.0 binary (in 2016) as they had suspected that the release could be targeted by the state sponsored attackers.
It is also recommended that you verify the signatures and hashes of the binaries from multiple sources and developer signatures. You can use the gitian signature repository to get other signatures and verify multiple signature data through multiple channels.

Can I still check the keys once it is up and running?

You could possibly check the binary once its up and running by evaluating the SHA-256 of the binary in the /usr/bin/bitcoind. But there might be a chance that a malicious code separated from the actual binary during its first execution and now is located in some place you might never find without a thorough forensic analysis.

Will I have to reinstall and download the entire blockchain again?

If you reinstall a new binary and verify it with the signatures, you can just sync it with the network. If the blockchain that you had downloaded from the previous release matches the true version of the Bitcoin blockchain (that is it was not tampered), then you need not download it again.

Thanks for this detailed answer. I have managed to download and verify v0.18.1. In the process I also realised that I didn't have my port 8333 enabled so now have inbound  connections too, so I'm really running a full node. Next I will need help setting up Electrum Personal Server, which I also had trouble with at my first attempt.

@Saidjinn for accepting incoming connections, you should set listen=1 in the bitcoin.conf file or pass it as a parameter when starting bitcoind. And just to clarify, you are still "running a full node" even if you don't allow incoming connections. You can read more about it here


text:
Bitcoin 0.18: how to send bitcoin from one address to another using Bitcoind?
To transfer bitcoin from one account to another address, we have used sendfrom method and which got deprecated in the latest version Bitcoin 0.18.0
Bitcoin sendfrom method
On reading the document of Bitcoin 0.18.0, it is suggested to use sendtoaddress method.
sendtoaddress method has two required parameter

"address" - address to which we want to sent BTC
"amount" - amount in whole number

Unlike sendfrom method, there is no parameter where we can pass the parameter with account or address from which we want to send BTC.
Please let me know how can I define the account or address from which I need to send Bitcoins or is there any other method available.
I am using bitcoind to make RPC calls.

Please let me know how can I define the account or address from which I need to send Bitcoins or is there any other method available.

The listunspent and createrawtransaction commands together can be used to achieve what you are asking.
The listunspent command will give you all the UTXOs that your wallet controls. You can pass an address parameter to filter all the unspent outputs by address. The documentation of listunspent is available here. This command will return you the txids, output index, value and others.
Then use the createrawtransaction command and specify those txids and output indexes in the input, and the address and value that you want to send in the output.

Thank you for your response... i checked the process of transaction and we need to sign and then send the raw transaction. Is there no straight forward method like sendfrom?


text:
Transaction verification engine
In the Features of Bitcoin Core i found following Line:
"Bitcoin Core includes a transaction verification engine"
Please define Validation, is Validation = verify or not.
Does the Engine verify everything what is coming through and i just have to pay for Power, Online Time, Memory and so?
I ask because of all the time when I run a full node I NEVER had to verify a transaction coming through, no Blocks, nothing. I was wondering why my Budget didn't move a single satoshi up or down. I don't get it maybe you do.

In Bitcoin, any transaction that confines to the consensus protocol is valid. This is unlike other ledger based system like Ripple, where the validators vote on whether the transaction is valid and if it should be included in the ledger. In those ledger systems, the validators gets reward for helping maintain the ledger.
When your Bitcoin full node receives a transaction, your node will check through all the protocol requirements. This includes verifying that the inputs consumed in the transaction have not been spent in the past (i.e. no double spend), there is a valid signature that unlocks the bitcoins, inputs are greater than outputs as so forth. You can find a detailed list of the transaction verification rules here.

Does the Engine verify everything what is coming thru and i just have to pay for Power, Online Time, Memory and so?

That is correct. As mentioned above, when Bitcoin Core will receives a broadcasted transaction, it automatically verifies it against the consensus rules to determine that they are valid. Incorrect transactions are discarded and not relayed to other nodes. You only need to have your machine running with the software and it would take care of it.

Why my Budget didn't move a Single Satoshi up or down

In Bitcoin, only the miners get the block reward and the transaction fees associated with it. A full node's job is to ensure that the block which the miners have mined conforms to the protocol requirement. If say a miner turns malicious and mines a 'fraud' block, the full nodes connected to the network will reject that block and will not relay it further. Other miners also assume that the nodes on the network are honest and do not build further on the 'fraud' block mined. Thus the malicious miner not only loses the block reward and transaction fees, but also has spent his electricity and mining power to waste. Thus full nodes play a very significant role in the Bitcoin network to ensure everybody is acting in an honest manner.