text:
Litecoind answers "getnewaddress" RPC request with "Method not found"
After having compiled the Litecoin daemon found at:
https://github.com/litecoin-project/litecoin.git

and set up a username and password, calling
litecoin-cli -rpcuser=<username> -rpcpassword=<password> getnewaddress

returns the error "method not found" from the litecoin daemon litecoind.

The answer is to recompile the Litecoin daemon, making sure to call the configure script with the --enable-wallet option:
./configure --enable-wallet

From there, make and make-install will give you a binary that has the getnewaddress RPC method available.
This reason behind this is that - instead of just disabling the methods around using the daemon as a wallet, or throwing an error in the case they're used without the --enable-wallet compilation option - the methods themselves are not compiled into the binary at all. This is likely due to the use of litecoind as a wallet requiring an extra library - the Berkeley DB library - and the path of least resistance being to not try to compile that section of the code at all in order to avoid issues with referencing the library, but that's conjecture.
This should also solve similar issues in other coins forked from the bitcoin source, and for bitcoin itself.


text:
How to listen to an address or transaction_id continuously?
I am running bitcoin core on my node. I have two questions.
1) I want to get notified whenever there a new transaction in the blockchain.
2) If I give an address, I should get all the details about the address ( when there is a new payment to the address).
I want to make a callback API for the deposits. Could anyone tell how to do it?
Thanks

You can use Bitcoind's ZMQ integration. It lets you get notify instantly when a transaction is made (whatever the address), you can then do the filtering yourself if you want to pinpoint a specific address.
This setup requires a bit of work to do the transaction parsing and filtering, but lets you run without relying on third parties. You won't get the balances of each deposit address too, just the deposits (but you can get the balancet from elsewhere).
Another solution (that relies on a third party) would be to use blockchain.info's service.

Then how to check the balance of a deposit address? I want to get notified whenever there is a payment done to the deposit address.

For balances, you could use any explorer. You can also look into watch only support from bitcoind.

@merklexy Can you request to receive a ZMQ notification for every transaction that affects your wallet. You don't ever need a "balance".

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
how to get unspents of an address using bitcoind?
I am using bitcoind. I have gone through the json-rpc api's.
listunspent [minconf=1] [maxconf=999999]    version 0.7 Returns array of unspent transaction inputs in the wallet.

listunspent method gives an unspent transaction inputs of the addresses in the wallet.
I want to get an unspent transaction inputs of one particular bitcoin address which are currently not part of our wallet. Could anyone tell me how to get that?

You should send your addresses as following:
bitcoin-cli listunspent 0 9999999 '["1PGFqEzfmQch1gKD3ra4k18PNj3tTUUSqg","1LtvqCaApEdUGFkpKMM4MstjcaL4dKg8SP"]'

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What is P2PK, P2PKH, P2SH, P2WPKH - ELI5
Explain like I'm 5 years old and my other half of the brain is surgically removed :) I don't want to put a lot of stress on my brain
Also what is the difference between BTC held in P2SH and Unspent P2SH Output here?
https://p2sh.info/dashboard/db/home-dashboard?orgId=1

P2PKH: "Pay To Public Key Hash"
This is how transactions are made. You are requiring the sender to supply a valid signature (from the private key) and public key. The transaction output script will use the signature and public key and through some cryptographic functions will check if it matches with the public key hash, if it does, then the funds will be spendable. This method conceals your public key in the form of a hash for extra security.

P2PK: "Pay To Public Key"
This is similar to P2PKH; the difference is that it does not conceal your public key. Anyone using this method to send funds over the P2P network is showing people their public key in the transaction details.

P2SH: "Pay To Script Hash"
The outputs of a transaction are just scripts that, if are executed with specific parameters, will result in a boolean of true or false. If a miner runs the output script with the supplied parameters and results in true, the money will be sent to your desired output. P2SH is used for multi-signature wallets making the output scripts logic that checks for multiple signatures before accepting the transaction. P2SH can also be used to allow anyone, or no one, to spend the funds. If the output script of a P2SH transaction is just 1 for true, then attempting to spend the output without supplying parameters will just result in 1 making the money spendable by anyone who tries. This also applies to scripts that return 0, making the output spendable by no one.
This can also be used for puzzles like this one.

P2WPKH: "Pay To Witness Public Key Hash"
This was a feature of segwit which stands for Segregated Witness. Instead of using scriptSig parameters to check the transaction validity, there is a new part of the transaction called witness where the validity occurs.

Also what is the difference between BTC held in P2SH and Unspent P2SH Output here?

This means that out of all the P2SH transactions (transactions that can be spent by anyone with a set of parameters [scriptSig] that results in the execution of scriptPubkey with true), the ones with unspent outputs have not been redeemed.
I recommend this video by Andreas Antonopoulos on Bitcoin scripting.

"This method conceals your public key in the form of a hash for extra security." - just a clarification that it is not concealed after you spend with that address of course :)

any idea what is P2WSH address? these new addresses were included in BIP 173 after segwit implementation

@anacondabitch P2WSH is just like P2SH but the redeemScript is moved to the witness instead of scriptSig.

P2SH: "Pay To Script Hash" is the Old Legacy Address

QUESTION: Is there any website or script out there that can tell by typing a tx id what transaction type it is P2PK or P2PKH or P2SH or P2WPKH ????

@user380208  www.blockchain.com    type into any address

Not sure if a 5 years old kid can understand this, but great answer!

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How do I calculate the witness commitment hash for a given block?
From my understanding, every SegWit enabled block has an added output script which is used as a place to store the "Witness commitment" specified in BIP 141.
One issue I've stumbled upon was calculating the commitment hash value only given the raw block/transaction data.
For example, I've chosen Block #542213 as it only has 4 total transactions (including the coinbase generation)
Coinbase witness commitment script:
OP_RETURN OP_PUSHDATA(36) [aa21a9ed4a657fcaa2149342376247e2e283a55a6b92dcc35d4d89e4ac7b74488cb63be2]

BIP141 states that the pushed data can be broken down to the following:
0xaa21a9ed - Commitment header (seemingly random)
0x4a657fcaa2149342376247e2e283a55a6b92dcc35d4d89e4ac7b74488cb63be2 - SHA256(SHA256(witness root hash|witness reserved value))

The documentation mentions that the witness root hash can be derived similarly to the merkle root in a block header, where each wtxid acts as the txid would in a traditional header merkle root.
To test this, I'm using this PHP script kindly provided on a different thread. This generates the merkle root used in the block header perfectly (that is not what I am trying to do, but what I have done to verify the code actually works).
Our example block has 4 transactions. Only two of which have witness data (coinbase and this tx).
The commitment structure states that the wtxid of the coinbase transaction is always:
0x0000000000000000000000000000000000000000000000000000000000000000

and every other wtxid act as a leaf.
My first question:

How are transactions that do not have witness data effected in the witness hash? Are they entirely omitted?

Assuming that the non-witness transactions are just omitted entirely from the merkle tree, I simply add the coinbase and the one transaction with witness data's  wtxid into the array like so:
$txids = array(
    '0000000000000000000000000000000000000000000000000000000000000000',
    '6c6e3849acf1b570db352dc08f7776e99c344a56fbb2f019e1865d1b6e044889',
);

The result being: 3a84e2f2f1bcd42141b58f2acd9497296c9f2d710cc2164669ef6d7e3870dfe9
My second question:

Where does the (concatenated?) witness reserved value come from?

Here it states:

The witness reserved value currently has no consensus meaning, but in the future allows new commitment values for future softforks

But it is still unclear to me what this value actually is before the hash is computed.
Please feel free to correct any inconsistencies or problems with my train of thought so I can improve my understanding for this part of segregated witness.

How are transactions that do not have witness data effected in the
  witness hash? Are they entirely omitted?

It will still include transactions that are not witness, it will just compute the normal hash to use as a leaf in the tree. See BlockWitnessMerkleRoot() in src/consensus/merkle.cpp and ComputeWitnessHash() in src/primitives/transaction.cpp:
uint256 CTransaction::ComputeWitnessHash() const
{
    if (!HasWitness()) {
        return hash;
    }
    return SerializeHash(*this, SER_GETHASH, 0);
}

Also, you are forgetting to reverse endianess, and hash the witness reserved value 0000000000000000000000000000000000000000000000000000000000000000 (from the last 32 bytes of the coinbase input witness). So step by step:
$ reverse_endian 66beaceb4be99da1e9824448231ab4fd37bacaee912381e779b37cf0e1dadad7
d7dadae1f07cb379e7812391eecaba37fdb41a23484482e9a19de94bebacbe66

$ reverse_endian aecb37e25954e15489e25548eb663ffdfd8a1362cac757ad62e9614453d2a577
77a5d2534461e962ad57c7ca62138afdfd3f66eb4855e28954e15459e237cbae

$ reverse_endian 5b211bc589cbdf5ad86cab1e2fe91f01c8ab934d21536b35864d30a3ff778456
568477ffa3304d86356b53214d93abc8011fe92f1eab6cd85adfcb89c51b215b

$ hash256 0000000000000000000000000000000000000000000000000000000000000000d7dadae1f07cb379e7812391eecaba37fdb41a23484482e9a19de94bebacbe66
d822d7717f284b89ea6a97874af567e9887a26a9f7db449ce02377d8fae4643f

$ hash256 77a5d2534461e962ad57c7ca62138afdfd3f66eb4855e28954e15459e237cbae568477ffa3304d86356b53214d93abc8011fe92f1eab6cd85adfcb89c51b215b
c182a158773ea7fc7a252fa725fc726de2a171088b40528ca00de0569fe53afe

$ hash256 d822d7717f284b89ea6a97874af567e9887a26a9f7db449ce02377d8fae4643fc182a158773ea7fc7a252fa725fc726de2a171088b40528ca00de0569fe53afe
30d002641e0eb81b11c3d14f4b120e80b4c06bcb3d34d90a5a606ca85e90c841

$ hash256 30d002641e0eb81b11c3d14f4b120e80b4c06bcb3d34d90a5a606ca85e90c8410000000000000000000000000000000000000000000000000000000000000000
4a657fcaa2149342376247e2e283a55a6b92dcc35d4d89e4ac7b74488cb63be2

Which matches the op_return data in the second coinbase output: RETURN PUSHDATA(36)[<aa21a9ed> <4a657fcaa2149342376247e2e283a55a6b92dcc35d4d89e4ac7b74488cb63be2>]

Where does the (concatenated?) witness reserved value come from?

I alluded to this before, but it is in the witness field of the coinbase input:

the coinbase's input's witness must consist of a single 32-byte array
  for the witness reserved value - BIP 141

so, is the commitment header random? "aa21a9ed" doesnt decode to anything meaningful in ascii, but Bitcoin headers usually do.

@Pillagr As the BIP 141 states, the commitment header is fixed.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Can I export a xpub/zpub/ypub key in bitcoin core?
I'd like to add a master public key (any of xpub/zpub/ypub https://support.samourai.io/article/49-xpub-s-ypub-s-zpub-s) for tracking purposes to Sentinel watch-only wallet https://samouraiwallet.com/sentinel.html but I cannot find an option to do this in Bitcoin core (v0.16.1).
Can anyone suggest how this can be achieved?

You cannot. Bitcoin Core does not allow you to import master keys nor can you export extended public keys. BItcoin Core does not use extended public keys since it uses hardened derivation which does not use extended public keys.


text:
Trying to calculate the strength of a brainwallet created with a custom diceware
I want to create a brainwallet using a custom diceware since I don't like the original one. Instead of 7776 words it has 46656 because I use 6 dice for each word instead of 5.
So this is what I got so far, please tell me if I made a mistake:
2 ^ (log2(46656) * 5) / (1 * 10^15) / (60 * 60 * 24 * 365) = 7 years

2 ^ (log2(46656) * 5) is the entropy for a brainwallet of 5 words
1 * 10^15 is 1 Phash/s, which is the hashing power I'm assuming for this attacker. I don't even know if this makes sense, because an attacker doesn't need to start over for each brainwallet he wants to crack.
60 * 60 * 24 * 365 represents a year in seconds

you forgot to divide by 2.
7 years is exhaustive, but on average you'd expect it to take 3.5 years to crack.
10^-15 * 46656^5 seconds / 2


text:
A fee is added per kilobyte of data. That means 1000 bytes or 1024?
Wikipedia says kB = 1000 bytes and KB = 1024 bytes.
Which one are we using for Bitcoin?

According to the Bitcoin.it-wiki page on Transaction fees are dependent on multiples of 1,000 bytes  (rounded up).
The above page cites Gavin Andresen's post on 0.8.2 Release notes:

The default fee for low-priority transactions is lowered from 0.0005 BTC
  (for each 1,000 bytes in the transaction; an average transaction is
  about 500 bytes) to 0.0001 BTC. [emphasis mine]

Here is the latest changed(commit) made for the fee drop: https://github.com/bitcoin/bitcoin/commit/beabca2be092d0e2a1de26989d4e63a12cce1284#diff-279f95e952b2c1cff13bdfbcd7bc36ee
It seems to be a multiple of 1,000 bytes as it's seen in the source code.
So yes, it's a multiple of 1000 bytes, not 1024 bytes.
Cheers,
-Besir

authors: محمد عبدالله


text:
Do we have an estimate of how many people are using Bitcoin?
I found this map: http://bitnodes.io/
But I'm not sure what can be inferred from it.
Can the network grow much more than this?


text:
Compressed or uncompressed for brainwallet?

When creating a brain wallet, should I choose the compressed or uncompressed version? Why?
What happens if I choose the uncompressed version (which is what the official client uses, right?), and someone sends coins to the compressed address? Will I see the coins? Would I be able to spend them if I saw them sitting in the other address?
Will I have to remember which option I chose, or the pass phrase is enough? Will other options keep popping up? (eg: super compressed)

Brainwallet.org doesn't make this terribly clear: choosing a compressed versus an uncompressed address is not choosing between two different versions of the same address, but rather two different addresses based on the same secret exponent. Sending bitcoin to one address will not be accessible by the other, as they are technically independent. With that in mind, we can address your second point first:

Will I see the coins? Of course! You can see everything that happens
on the block chain.
Would I be able to spend them if I saw them sitting in the other address? No. Unless you have access to the private key for the other address, you cannot spend the coins.

As for the first point, it really depends on whether or not your client supports signing transactions with compressed private keys. Your client will always be able to receive coins at the public address, but it may not be able to process a compressed private key and thus not be able to spend with it.
Here's some info on popular clients that might be helpful:
BlockChain.info --
Currently, if you try to import a compressed private key into blockchain.info's web app, you will get a warning that they key will not be usable to create transactions on the mobile app. So, if you need to make payments with their mobile app, don't use compressed private keys. Although their web app, should work fine, even if accessed from mobile.
Bitcoin (Android - Andreas Schildbach)--
Andreas Schildbach Android bitcoin client, on the other hand uses compressed private keys. Although they are normally hidden from user, there is a (somewhat convoluted) way to import and export private keys into the app and out of the app. So, if you are making payments with this app, you can use compressed private keys. (Note: I'm actually unclear as to whether or not you can import uncompressed keys into this client, so compressed keys might actually be mandatory here.)
Armory --
I don't believe that the Armory supports importing of private keys at this time. So, if you're planning on using this, you must use uncompressed keys.
As for your third point, I would suggest remembering your chosen option just in case the clients you wish to use doesn't support it compressed keys. That way, you have a good idea of what's going on in case making a transaction fails.

I know I can see them in the blockchain, I meant see them in my balance (I guess I can't). So are you sure they are different private keys? Why call them compressed/uncompressed then? If I have the uncompressed one, can't I get the compressed one and viceversa, and then spend the coins? I think armory supports importing keys, but it doesn't support recovering them from your paper backup obviously.

I'm pretty sure that they are different keys, but I'd be willing to hear arguments to the contrary. If there is a normal private key, there should also be a corresponding compressed private key and they should each correspond to the same public key. However, the process that brainwallet.org uses to produce compressed verses normal keys results in two different key pairs -- rather than two different private keys with the same public key. (Note: the public does not remain the same when toggling.). I tried importing a compressed key into Armory, and it failed, perhaps I did something else wrong?

So, I got some down votes on this one? Anybody care to explain? I'm trying to make my answers as accurate and thorough as possible.

It makes no difference which you choose.
For more info please see the answer to this question:
What is a compressed Bitcoin key?

A compressed key is just a way of storing a public key in fewer bytes (33 instead of 65). There are no compatibility or security issues because they are precisely the same keys, just stored in a different way.

Most clients allow the import of compressed private keys, but some clients -- particularly older versions -- and the services which use those clients may not work for compressed keys.


text:
Can I use the official wallet to recover bitcoins sent to an alternative wallet?
I like alternative wallets like Armory and Electrum because they offer more features, some of which are fundamental for security and convenience. But I'm a bit worried about them not being "compatible", I mean you can't import an Armory wallet into Bitcoin-qt.
So what I would like to know is whether it is possible to recover my bitcoins (which if I didn't get it wrong, they are in the blockchain and could be spent with the private key) using the official wallet. For instance, the paper backup generated by Armory contains a "root key" and a "chain code". Is one of those the private key or what?

Most alternative wallets give you the possibility of exporting the private keys. Private keys can be imported in almost every other wallet.
In the case of Armory, the root key and chain code are used to generate a whole chain of private keys. You can ask for new addresses as you like inside your wallet, and by backing up only these two keys, you can recover all of them. This is a useful feature, but makes it harder to import in other wallets.
Luckily, Armory has an option to export the individual private keys of all the addresses you used. In the wallet menu there is a link "Backup Individual Keys" for this purpose. This exporting feature will always be available, also when Armory decides to stop, since it is independent of Bitcoin itself.
I don't know how other wallets support backing up individual keys, but I suppose most of them can.


text:
What's the mnemonic seed for in Blockchain.info, if they use non-deterministic wallets?
When you create an account, you are given a mnemonic seed that you have to write down. It's supposed to be used for recovering your coins if you lose access to your account (I think that's what it said when I created one).
But now I'm reading that the wallets are non-deterministic. So what's the mnemonic for then?

I think the FAQ answer is muddled and, as the term is normally used, their wallets are deterministic.

The FAQ specifically says there is no seed.  Is it possible that their system has changed since you created your account?

@NateEldredge It says "... you do not have to backup or remember an additional seed", which is somewhat confounding. That would be true of both deterministic and non-deterministic schemes.

Anyone got an update on this? It looks like the mnemonic is just a password recovery tool. Not a deterministic seed. Sort of confusing. That would make the mnemonic functionally identical to the password.

actually. One comment on reddit says the mnemonic encodes your password and wallet identifier.

From the source code, one can see that the mnemonic in fact only encodes your password and wallet identifier. You would still need to access you wallet data somehow to recover you coins.

Yep. The last chunk of words I know from experience represents the password (>5 words). It's 1 words in per 2 char combination so it's probably safe to say each word represents \u0000 to \uFFFF. A checksum is involved too. Changing the password changes the mnemonic's ending

@AussieCryptocurrency Each word represents two UTF-8 encoded bytes, which is two ASCII characters, or one character between \u0080 and \u07ff, or 2/3 of a character between \u0800 and \uffff, or 1/2 of any other character.

Yeah that sounds right. I haven't learned JS so I haven't checked the source code. I just plugged in different values when I noticed 42 always spat out campsites. So it doesn't start at \u0000 bc they're whitespace keys correct? (EDIT; can you explain the 2/3 1/2 part? w/o a thread hijacking taking place?

@AussieCryptocurrency I think \u0000 in a password may not work due to UI problems. Regarding 2/3: if a character is encoded as 3 bytes, and each word encodes 2 bytes, how many characters does each word encode?


text:
How is network conflict avoided between chains?
I have a question which may appear obvious but hopefully it’s not.
How do different Distributed Ledger Technologies (DLTs) and cryptocurrencies avoid network conflict?
I understand the distributed nature of the networks with nodes. But if you took a BTC node and had it try to connect to a BCH node, how is this conflict avoided? (BCH may not be a good example considering it’s a fork, then again it may be because of this.)
Is there a signature per coin on each block as a chain ID? Looking through the AcceptBlock() method doesn’t indicate it being the case.
Does it scan from block0 (genesis) for consistency to blockHeight?
I’ve customised an old version of BCH for playing with it, but want to update to the new code base (changing genesis and network IPs for discovery) but how is conflict avoided?
Hope this makes sense. Cheers.

The protocol messages on Bitcoin include a magic number to identify the network they belong to.
Bitcoin's magic values are described for example on the Protocol Documentation page in the Bitcoin wiki:

This permits the corresponding nodes to immediately recognize whether they belong to the same network and disconnect if they do not.


text:
Probability of block rewards
Assume that there are only three mining pools, all with the same hashing power, mining for bitcoins in the bitcoin network. Which mining pool is most likely to get the block reward? Do you all think that it will be equally likely? Since they have the same hashing power and working on the same candidate block.

I understood the question, but I don't get why there is a reference for 'payout methods' in the title

I have edited the title already

Yes, the same hash power means the same probability of mining a block.
But there can be other factors like connectivity: if a mining pool is facing internet connection problems between it and its members, it may fail to use that hash power effectively. P2Pool FAQ talks to its members about the importance of a good connection: http://p2pool.in/
Also, if there is an internet outage soon after it mines a block and it fails to broadcast it, the block risks been orphanated.
But it's not possible to 3 miners to work on a same candidate block, each block they are trying to mine will be necessarily different. I think you mean 'on top of the same block'.

You're welcome. I've improved the answer.


text:
Is it possible to reindex specific blocks?
I came across a block on my Fullnode that throws an error when attempting the gettrawrtransaction RPC.
No such mempool or blockchain transaction. Use gettransaction for wallet transactions.
It's only one block so far that seems like it got corrupted or improperly indexed. If I change the block/transaction, the JSON-RPC call works. I have txindex=1 set in bitcoin.conf
My question is, can I tell bitcoind to reindex only that particular block 282455, or do I need to reindex the entire blockchain? If so, what's the fastest method? Setting a higher dbcache?

can you see your transaction in that block?

none of the transactions in that block work with getrawtransaction. They aren't my txs. I'm just trying to get the raw data on them, similar to a blockchain explorer.

Can you give the txid of one of the transactions that fails for you?

@NateEldredge e73da0c9a5548e62c73b5b297f33ab7605079bcdb00ca8c5afd56bfae835a88c or any tx in the 282455 block

No, to the best of my knowledge there is no user-level command to reindex specific blocks.  It's all or nothing.
If you want such a feature you'll have to write the code.
It seems to me kind of unlikely that the index would be wrong only for one specific block, so I would suggest looking for other explanations for the behavior you're seeing.


text:
Why is the tag repeated twice in a tagged hash?
I noticed that the hash message for the tagged hash in BIP340 is prefixed with SHA256(tag) || SHA256(tag), which also describes the reason

Because this is a 64-byte long context-specific constant and the SHA256 block size is also 64 bytes, optimized implementations are possible (identical to SHA256 itself, but with a modified initial state). Using SHA256 of the tag name itself is reasonably simple and efficient for implementations that don't choose to use the optimization.

But I have some confusions

"optimized implementations are possible": What specifically is the optimization being talked about here, is it caused by the prefix being the same size as the block size of SHA-256? If so, can this conclusion be generalised to all block hashing algorithms? i.e. the same size of the prefix and the block of the algorithm can lead to optimization.
"Using SHA256 of the tag name itself is reasonably simple and efficient for implementations that don't choose to use the optimization.": Does this mean that if the implementation does not intend to adopt optimization, then it can not repeat the tag? i.e. the tagged hash would be SHA256( SHA256(tag) || msg).

"optimized implementations are possible": What specifically is the optimization being talked about here, is it caused by the prefix being the same size as the block size of SHA-256? If so, can this conclusion be generalised to all block hashing algorithms? i.e. the same size of the prefix and the block of the algorithm can lead to optimization.

Yes, it means that the SHA256 hashing state at SHA256(SHA256(tag)||SHA256(tag)|| can be precomputed. If the tag was just 32 bytes this wouldn't work, as you'd need to know 32 bytes of data before being able to process the first block.

"Using SHA256 of the tag name itself is reasonably simple and efficient for implementations that don't choose to use the optimization.": Does this mean that if the implementation does not intend to adopt optimization, then it can not repeat the tag? i.e. the tagged hash would be SHA256( SHA256(tag) || msg).

No, that would give a different result. All this is saying is that it's a simple construction to concatenate the hashed tag twice. Obviously not doubling it would be even simpler, but that would forego the optimization from the previous point.

Thank you very much for your reply.
For Q1: So this optimization needs to cache the intermediate state of SHA256(SHA256(tag) || SHA256(tag)|| locally? So in future each time the hash of the tag is computed starting from reading the cache state?
For Q2: I am very sorry for misunderstanding this statement before, now I understand that this is the reason why tagged picks SHA256 directly.

Right, the state after processing the first 64 bytes can be preconputed, and thus the computation of a tagged hash can start at that point rather than needing to redo it every time.

So, if the length of the prefix is a multiple of the block size, the optimization still works? i.e., a direct padding to the tag and using it as a prefix. For example, prefix = (tag_size, tag, padding), I just need to make sure it is a multiple of block size. But the downside is that this would limit the length of the tag. Does it correct? I will appreciate your reply!

one other thing to keep in mind in an optimized implementation of tagged hashes: SHA256 writes the number of bits into the final hashed block, this will be 512 more due to the tagging block


text:
Do hierarchical deterministic wallets require higher fees as a trade-off for privacy gains?
This is my first post here, so be kind :-|. My understanding of deterministic wallets is that they use an initial seed to create a master key from which you derive multiple receive addresses thus making it harder for others to identify from the blockchain that the funds belong to the same wallet. My question is if you spend a large amount based on multiple UTXO which used different receive addresses, presumably your wallet has to sign the transaction with multiple corresponding private keys. Will this increase the size of your transaction and lead to an increase in the fees you pay?

You are right that spending multiple UTXOs increases the transaction's size and therefore fee. However, since each UTXO is signed individually, it doesn't matter if they all belong to the same address or different addresses, there is no advantage in address reuse.
What you can do is to consolidate your smaller UTXOs into larger ones during times of low transaction fees, however you do need to be mindful of the privacy implications of that.

Understood. I appreciate you taking the time to answer.


text:
How to get all ip address of nodes in bitcoin testnet
There are many service to get all fo the ip address in bitcoin mainnet but I can't find the similar way to get all ip address in testnet.Is there a way to do it?

@Prayank You can use api.blockchair.com/bitcoin/nodes to get mainnet nodes


text:
How to buy 1,000,000 BTC
Let's imagine that I had enough fiat money to buy 1 million bitcoin ($30,000,000,000). How would I find a seller? Are there even that many bitcoins offered for sale? Or would it be easier to start a mining company with the money and mine the 1 million coins?

First, realize theres no way to get that much btc without significantly moving the price higher.
Essentially, your goal is to source as much liquidity as you can from as many sources as you can. That's so much, that you can also tap into non-spot-bitcoin liquidity with the intention of selling it at a profit after. This means buying up bitcoin cash, bitcoin sv, various correlated shitcoins and futures, acquiring options on deribit and on bitcoin etfs, and similarly with ethereum futures and on-chain, off-chain, and OTC options and futures on any and every coin for which futures exist. Make some contracts with miners to buy their future bitcoin at a premium. Buy physical mining equipment. Try to get a lot of GBTC (once we're pumping after the main buys start, the discount will narrow)
Fund a team of developers so you can directly take advantage of the DeFi liquidations that will happen as a result of the massive move. Use the liquidations to add exposure on shitcoins wherever possible. You'll sell them for bitcoin after most of the bitcoin buy is complete, to complete your buy.
Then, partner with multiple organizations over a period of time to solidify as many OTC deals as you can. Thats so much bitcoin that anyone you try to start doing these deals with will likely be able to frontrun you before you complete the sale, so you'll need to hide much of your size and appear to these giant organizations as though you're buying "only" 100m at a time.
Keep in mind, once you complete this buy and btc is at 250k-500k, you're going to have some people fight you on the OTC deals, so make those contracts specifically describe that extreme price movements will not affect the validity of the contracts. Make sure that trusted independent third parties are holding the physical bitcoin in escrow for your deals.
If you're open to market manipulation, you can get a better average price. Use your giant stack to run the typical liquidation-hunter price movements along the way. People will see what's happening and start opening leveraged longs to ride it themselves. Once you see enough leveraged longs build up around you, punish them by selling just enough spot bitcoin to liquidate all of the longs, then buy them up for yourself. Repeat on the way up.
As the price keeps going, some overconfident bears will start talking about how btc rise is getting out of hand. Find the most vocal, rich critics and make bets against them for millions of dollars.
Once you're getting near the end, with a few billion left of bitcoin buying to go, start liquidating all of your shitcoins. Start by scaling into leveraged shorts and continue as you are selling spot. Then on a Sunday or another low-volume holiday, quickly liquidate your entire altcoin portfolio. The across-the-board shitcoin collapse will affect bitcoin. Use this dip to fill the rest of your bitcoin buys.
good luck!

It is difficult to ascertain exactly how many bitcoins there are for sale at any moment. We have estimates on how much bitcoin there is on exchanges (2.26 million BTC on July 5, 2023 by Glassnode), but only some of it is currently offered on the market. Some of the rest might get put up for sale in case e.g. the price increases, but we can't know how much at which price levels. This also holds for bitcoin held by users in their own wallets, who might either put it up for sale on the open market, or be willing to sell it over-the-counter.
It is even more difficult to know how much any given amount bought will end up costing you because of the resulting market price movements. For very large amounts this could be a lot more than what you get by multiplying the amount in BTC with the current unit price. This is related to terms like market depth, market impact and slippage.
There are currently a little over 1.5 million bitcoins left to be mined, which means anyone looking to mine a million from now on would need to have a lot more mining power than everyone else combined. This obviously isn't the way to go. If we consider transaction fees rising in the future, you might be able to eventually earn a million coins with less than the majority of hashpower, but it would probably take decades at least.

Let's imagine that I had enough fiat money to buy 1 million bitcoin ($30,000,000,000).

The premise is acceptable, as any other axiom can, but the clarification is actually wrong. 30 billion USD is the current multiplied price of a million Bitcoin coins, but buying from now a million Bitcoin costs much more than 30 billion USD.
Once you buy a few thousand Bitcoin readily available on exchanges, price pumps (i.e. green candle happens) and price ceases to be as small as 30 thousand USD unless you somehow freeze the market back to such price. You would need some very worked strategy to buy that many coins while suppressing price so much.
Now the axiom implies you represent a nation state sized actor, so you could have quite such resources in order to try to suppress price globally. But those could maybe not be enough.

How would I find a seller?

You wouldn't buy from a single seller, you would need to find multiple sellers, from any imaginable source.

Are there even that many bitcoins offered for sale?

I think so, yes, currently. But that might not be the case in the future. Also, remember you absolutely wouldn't be able to buy a million coins for 30 billion USD, no way, as explained before. It would cost you a lot more USD, but yes I think you can still buy a million coins at some price.

Or would it be easier to start a mining company with the money and mine the 1 million coins?

I don't think so, because reward is so small nowadays.
Or maybe yes, but only if fees skyrocket a lot from the current small 5 sat/vbyte in the mempool since a few months from now.


text:
Why doesn't the very first bitcoin release work anymore?
I tried running the very first bitcoin client released by Satoshi in 2009, Bitcoin v0.1.0 on Windows 10. Of course I knew it wouldn't work or do anything since it's extremely outdated, but I wanted to try it anyway out of curiosity.
It runs but doesn't make any connections and can't download the blockchain. Is there a technical reason for this?

Did you give it a few nodes to connect to? The mechanism that it uses to find initial nodes won't work anymore because it's long since been replaced.

Possible duplicate of https://bitcoin.stackexchange.com/q/74323/5406, https://bitcoin.stackexchange.com/q/90811/5406


text:
What protocol do bitcoin nodes use to talk to each other?
I assume it's TCP, but is there some other software used to send and receive data?
So how would I program a very simple C++ program that sends a test message to a node?

What protocol do bitcoin nodes use to talk to each other?

Most Bitcoin nodes will primarily use:

Layer
Protocol

Application Layer
Bitcoin Protocol

Transport Layer
Transport Control Protocol (TCP)

Inter-Network Layer
Internet Protocol (IPV4 and/or IPV6), Internet Control Message Protocol (ICMP)

Network Access Layer
1000BASE-T / IEEE 802.3ab Ethernet

Obviously there are dependencies on other protocols which are not used directly by the Bitcoin node application itself (e.g. DNS via a name resolution service). The application will be using operating system libraries that manage the lower three layers. The principle APIs used will be those related to opening and closing TCP connections and writing to and reading from those connections. Perhaps using some variant of the Berkeley "sockets" API.
Here's an example network packet

Source

how would I program a very simple C++ program that sends a test message to a node?

Although the protocol isn't especially complicated, I personally wouldn't characterise writing a client as "very simple". That is, I wouldn't choose it as my first network protocol to write a client for.

Bitcoin nodes use a custom tcp-based protocol to communicate with each other. There's a work in progress to deploy a new one with some privacy enhancements.

Bitcoin uses TCP-IP protocol. The structure of data follows 2 parts:

Header
Payload

Header part

Magic Bytes (4 bytes) which indicates the selected network by nodes.
Command (12 bytes). You can see the command list here: https://github.com/bitcoin/bitcoin/blob/master/src/protocol.cpp#L14. Command field must be 12 bytes. So you can add null bytes to fullfill this field.
Payload size (4 bytes)
Checksum payload data (4 bytes). It is a double SHA256 of the payload data.

Payload part
The payload depends on the command type. You can have an overview of it from https://developer.bitcoin.org/devguide/p2p_network.html.


text:
How can wallet software implement automatic channel management for Lightning Network?
If I'm building a desktop Bitcoin wallet with Lightning support, and I don't want the user to have to worry about channel management, what's the way to achieve that?
I've seen the autopilot solutions for lnd and c-lightning.
I'm currently experimenting with lnd because it's easy to compile for Windows, macOS, and Linux. But lnd's autopilot doesn't seem to be mature enough. And it's not well documented. So I don't know if I should try to use it?
As for c-lightning, how well does its autopilot work? Well, anyway, because c-lightning is in C and its autopilot is in Python, it'll be a heavy lift for me to include them in my app.
So I'm wondering if there might be a simple temporary workaround. Like, what if I hardcode a list of popular nodes and have the app connect to those? Would that work?

I strongly advice against a fixed list of nodes. This might eventually become outdated. A better solution might be that you run the autopilot on a server and deliver the results to the user via an api. This of course would be a trusted solution. But the lib_autopilot.py which empowers the clightning autopilot plugin is a standalone library that can easily be wrapped into a small webserver (or any server sided application) instead of being used as a clightning plugin on the client side. Also it allows much more configuration and fine tuning than the clightning plugin.
Running it as a server might actually make sense as the computations that are necessary to prepare the autopilot are quite heavy and memory intensive and might not be so great with a bigger lightning network on a client's machine. Also once those computations are done the results can be used to run the autopilot several times at rather low cost making it the perfect candidate for a server side application.
That being said the autopilots are experimental features. Being the author of lib_autopilot.py I am happy you agree that the c Lightning solution might be more mature than the lnd version. But I also want to stress with my increased knowledge about pathfinding and routing that I now have in comparison to one year ago I would create a much different autopilot these days.
Edit: sorry you asked how the c Lightning autopilot works. There is a presentation comparing it with lnd https://youtu.be/N7rlHCnaBf8 and explaining some of the challanges. Again it is one year old - so a bit has changed about what should be said about autopilots but it is a good start anyway


text:
Why is the version of a bitcoin transaction is 4 bytes?
Here is a raw bitcoin transaction. I wonder what was the reasoning to make it 4 bytes rather than 1 byte(256 possible version numbers) or 2 bytes(65536 possible version numbers)?

As with many design decisions for the original Bitcoin protocol, the answer is simply "because it seemed like a good idea to Satoshi". The raw transaction format includes both variable-length numbers (called "CompactSize", used for input and output counts and script sizes) and fixed-length numbers, both 4-byte (version, sequence, prevout index, locktime) and 8-byte (output amount), so in this sense the version field doesn't really stand out. Also note blocks too have a 4-byte version field.

Only Satoshi Nakamoto knows the right answer :)

Bitcoin's transaction version is 4 bytes to accommodate flexibility and make it future-proof. Indeed, this allows for diverse types of transactions as the protocol evolves. With 4 bytes or 32 bits, the number of available version numbers is over 4.29 billion in comparison with 1 byte, which has 256 possibilities, or 2 bytes, with 65,536 possibilities. Of course, this many bytes would be more than enough for future updates, as well as for something as complex as the one introduced by SegWit. Another reason for this choice is that most programming languages process data in 4-byte blocks, so this is efficient and minimizes memory alignment issues. In general, using 4 bytes keeps consistency with the protocol and fits with other fields like timestamp and block version.


text:
If you have the required mining power, is it possible to mine your transaction in the next block without propagating it to the network?
Let's imagine that you have 51% of all mining power. Is it possible to put a single transaction in the next block without putting it into the mempool and/or propagating the tx to the other peers through the bitcoin network?

Yes, since miners (pools) construct the blocks they mine, they can insert any valid transactions they want, including transactions that were never broadcasted to the wider network. They don't even need 51% of the global hashrate, they just need to create blocks often enough that they don't have to wait too long for their transactions to confirm.
Some mining pools commonly do this. For example, here is a transaction mined by F2Pool that most likely distributes mining rewards to the pool's miners. It pays no fee which means it was never broadcasted, since zero-fee transactions aren't propagated by nodes through the network.

To add to Vojtěch's answer, take the point of view of nodes validating blocks on the network. How would they enforce that a transaction must have been propagated before it's included in a block?
Not only is this rule unenforceable on historical blocks, it also requires a synchronization mechanism across the network on what transactions are valid in the next block. But this synchronization mechanism is the block chain itself, you can see how it is circular.
No consensus rule can be based on the content of a node's mempool. Different nodes have different mempools (they usually only differ slightly). Maintaining a mempool has plenty of advantages, but it is not a requirement for using Bitcoin.


text:
Why does the block explorer show a different fee rate than what I selected in my wallet app?
I know this could have something to do with the fact that I was sending from a lightning wallet, but I would like to know why the transaction on blockchain.com shows my fee as being ~6.5 sats per byte when I selected 10 sats/byte as I sent the transaction.

Feerates on the Bitcoin network are expressed in sats/vbyte (where vbyte is bytes, but taking the segregated witness discount into account). blockchain.com shows sats/byte (where byte refers to the actual number of bytes used to serialize the transaction); this metric is meaningless for fee estimation purposes.
blockchain.com doesn't show sats/vbyte directly, but you can compute it from the sats/WU that they do show (1 vbyte = 4 WU, so N sats/WU means 4N sats/vbyte).
I recommend using a different block explorer (or none at all; your own wallet will tend to give you more correct interpretations, without needing to trust a single site).
See also: https://murchandamus.medium.com/psa-wrong-fee-rates-on-block-explorers-48390cbfcc74


text:
How to download full BTC chain with limited internet
I am running linux mint on a laptop with an external ssd, and want to set up a full BTC node. My problem is that I only have jetpack internet which won’t handle a full sync in one go.  I can take my time & download 50 gb each month until complete, if stopping & restarting is an option. Or any better solution anyone might have.

Pausing and resuming will just work; simply stop the application and start it again whenever you want.
Copying the data files from a friend might be a better option, if you can.
Or set up a Blockstream sat receiver.


text:
Can I still use old wallets after pruning?
I want to move my node to a laptop with less than 70gb free space. Would pruning make wallets older than pruned blocks unusable? If that is not an issue,  what files should I copy to my new node in order to keep using my wallets?

I wouldn't recommend it until we get a solid answer to issues like this https://bitcoin.stackexchange.com/questions/121849/loading-wallet-produces-last-wallet-synchronisation-goes-beyond-pruned-data-af

Once the wallet has already scanned a range of blocks, you can safely prune these blocks. However you won't be able to re-scan this block range in the future.
Copying your wallet file over to your fresh node should be fine as long as the wallet's tip (the latest height at which it was synced, ie opened on the old node) is superior to the last pruned height on the new node.

Copying the wallet folder to the pruned node resulted in an error and bitcoind shutting down Error: Prune: last wallet synchronisation goes beyond pruned data. You need to -reindex (download the whole blockchain again in case of pruned node) although the wallet was working just fine on the full node

Well that's exactly what i said.. The new node needs not to have pruned the blocks the wallet hasn't yet processed. Process those blocks on the old node and then copy it over.

That is what I did. The old node has already processed those blocks. -getinfo gives the correct balance on the old node. Still got that error after copying the wallet over:(

"Process those blocks on the old node and then copy it over." What does this mean exactly @AntoinePoinsot?


text:
Can we get together a group of C++ devs that work on Bitcoin apps?
I've been trying to get into Bitcoin development and I've been having a horrible time. I tried multiple Bitcoin-related IRC channels for support but most responded with hostility, except for the libsecp256k1 channel whom at least tried to lend me a hand.
I can't seem to find any C++-specific, Bitcoin-specific spaces anywhere. At first I thought the search engines were to blame but now I'm starting to think forums truly are dead and everybody else's just  coding alone in caves. The closest thing I ran into is the Bitcoin Core dev IRC channel but apparently that's strictly for Bitcoin Core development. This Stack Exchange doesn't seem too dev-oriented either although I was still encouraged to post on here by IRC users.
Are any of you here? If so, what do you say we get a group together? I don't care what platform - if you program for Bitcoin in C++ please do say what platforms you use and see what we all have in common. IRC, Matrix, Discord, Session, whatever - we can have it all, just say which you like. I really REALLY need to get in touch with you... assuming you aren't all already dead. I'm truly friggin cold and alone here. Please. I refuse to think I can't even begin to learn this stuff because of a f***ing library dependency problem. Jeez.
Edit: removed references to libbitcoin. Turns out I didn't even need that for my use case.

I’m not sure what exactly you need to achieve, but a lot of things can be done just by calling APIs on nodes. There are also numerous libraries in many languages with a plethora of language bindings, possibly some of them would work for you even if they are not written in C++. Perhaps if you ask a little more specifically about the things you want to achieve, we can suggest some avenues how to achieve them even when we are not that plugged into libbitcoin.


text:
Why mining pool has different connections ports for different difficulty
In the documentation of mining pool I found following
Connect Details:
(Difficulty 5 - GPU)    stratum+tcp://stratum.btcprivate.org:3032
(Difficulty 10 - GPU Farm)  stratum+tcp://stratum.btcprivate.org:3036
(Difficulty 15 - Nicehash)  stratum+tcp://stratum.btcprivate.org:3096
I see that only differents between difficulty. I have a farm of 6 card which connection should I use and why?
Any ideas? Thanks!

I feel sorry for you that no one had replied, so i'm gonna join in and try to think together with you, cause i was wondering the same.
Out of pure guessing, i think that when you have a mining farm, you mine a block very fast.
When you are finished with the block, the server needs to affirm it and send you a new job, which makes a small delay. This delay can decrease the profits over time.
Now the next question is, why isn't everyone using the highest difficulty? I'm not so sure about this, but i believe it has something to do with having to finish a block before you log off or your software crashes. I don't know what happens to a block that gets half mined. I think the block has to be mined in a whole for it to be confirmed.
When you mine with low difficulty, the block is split in pieces so everyone has a part they can complete in a block.
Hope this helped. It's probably not the correct answer, but it sounds obvious.
This was my first post btw. lel hah, never sell, HODL!


text:
How the Recipient's public key is used in bitcoin transaction
In the second paragraph the white paper by Nakatomo, we have this image:

It says a coin is a set of digital signature. For what I understand the whole process is the following:

Hash together the previous transaction hash plus the recipient public key;
digitally sign the hashing string with the private key of the sender;

The digitally signed result represents the transaction.
I have a couple of questions.
First of all, when the transaction is trasmitted to majority of the nodes, they have to decrypt the signed transactions by applying the public key of the sender to verify his identity.
Well, where the nodes take the public key of the sender? I thought that they can retrieve it by looking at previous transaction (where in the last transaction the actual sender will figure out as a receiver and so his public key is in the input of the hash function), but we cannot do so because the transaction is again encrypted and we need the public key of the last-last owner.
The other question is about the Receiver public key inside the data in the hash function. Does this really matter as a proof that the coin associated to that transaction are held by the new owner? Or they play a role in the user validation? I mean, the arrow with the name "verify" what stands for?

Why is this tagged "ethereum"?

Achow explained already the details, I want to give one further hint: the bitcoin system has a locking and unlocking mechanism. The locking mechanism is in the pubkey script (some refer to it also as the spending condition), and the unlocking is in the sigscript of the transaction. In the pubkey script the author of a tx defines the target address (to whom the tx shall go), and the spending condition(s). When then the receiver of the tx wants to spend the tx, (s)he needs to fulfill the condition (usually by showing the pubkey in hex format) and also signs the tx with his priv key.

All the details in Andreas' very well written book "Mastering Bitcoin" in section 6 "Transactions". It is online available...

@pebwindkraft can you link the book, if there is a free ebook available?

@pebwindkraft are referring only to bitcoin, or to smart contracts? Where do we find these spending condition in the white paper from nagatomo?
(I have not read the technical specification on github yet)

last link I had was here: http://chimera.labs.oreilly.com/books/1234000001802/index.html - a little bit of research on the net may help as well (google books), sometimes the webpages change. In my answer I am referring to transactions in bitcoin, not to smart contracts. I haven't searched the whitepaper for these spending conditions, it must not be there. But the logic can be derived in this sense.

It says a coin is a set of digital signature. For what I understand the whole process is the following:

Hash together the previous transaction hash plus the recipient public key;

digitally sign the hashing string with the private key of the sender;

The thing that is signed is not related to the previous transaction. It is actually the current transaction that is being created (or parts of it), but without the signatures.

The digitally signed result represents the transaction.

No. The result is never transmitted as it can be calculated from the transaction itself. The signature is placed inside of the transaction.

First of all, when the transaction is trasmitted to majority of the nodes, they have to decrypt the signed transactions by applying the public key of the sender to verify his identity.

No. There is no encryption or decryption happening. A digital signature does not produce any encrypted data. Nodes should be verifying that the transaction is signed with the public key.

Well, where the nodes take the public key of the sender?

The public key is either provided in the transaction output that is being spent (so nodes refer back to the previous transaction) or it is provided within the spending transaction itself. In the latter case, typically the previous transaction output will contain the hash of the public key so nodes will check that the public key provided hashes to that hash.
Note that I say typically because there are cases where none of this is true. Since Bitcoin uses a script system, anyone can create a script which does whatever they want, including signing the transaction or not. Transactions are only signed because the commonly used scripts say that the transaction must be signed in order to be spendable, but not all scripts require that.

but we cannot do so because the transaction is again encrypted and we need the public key of the last-last owner.

No. Again, nothing is encrypted. Furthermore, in an encryption scheme, the public key is absolutely useless for attempting to decrypt something. Things can only be decrypted with the private key. All blocks and transactions in Bitcoin are not encrypted and all of their details can be plainly read.

The other question is about the Receiver public key inside the data in the hash function. Does this really matter as a proof that the coin associated to that transaction are held by the new owner? Or they play a role in the user validation? I mean, the arrow with the name "verify" what stands for?

Yes, it does matter. Otherwise anyone could provide a public key which has a valid signature. The point of having the hash of the public key is to prove that the public key provided is the correct one and that the previous transaction committed to that public key (i.e. the previous transaction actually specified that public key).

Thank you.
So the flow is the following:

The sender send transaction data (plain data including receiver pub key) + digital signature (the signed hash of the data using his private key);
Other nodes use the pub key of the sender retrieved from the previous transaction (that is not encrypted), to verify the digital signature and to be sure that the sender was him and that the data have not been changed;

(if it's so) What I was missing is that this follows the normal working scheme of the digital signature.

The transaction data contains the signature itself, it isn't separate. The pubkey can also be part of that transaction data to, or it can be found in the previous transaction.


text:
Has Bitcoin Cores encryption wallet security ever been bypassable?
Every few years there seems to be an exploit found in Trezor that allows people to bypass the security. I was wondering has there every been the same case for Bitcoin Core wallet encryption?

Bitcoin core weaknesses?
Bitcoin core wallet encryption uses AES-256-CBC. So far as I know this is still regarded as a good encryption system although people have identified potential issues with cipher-block-chaining (CBC)
Other likely weaknesses might include

Poor implementation in Bitcoin core.
Poor choice of passwords/passphrases by the end-user.
Poor protection against snooping of in-memory keys etc by Bitcoin core or the OS.
etc

I don't recall any major issue being reported so far. There are currently 0 security advisories. A quick scan of previous common vulnerabilities for wallet related vulnerabilities doesn't find anything.
A careful study of the release-notes or even change-logs for Bitcoin core might reveal the fixing of an exploitable weakness. A careful web-search might throw something up. I haven't found anything but have not spent any time on this.
Trezor weaknesses
Published weaknesses include

weak proof of genuineness with respect to cloning etc.
side-channel attack by measuring power consumption during PIN entry.
extracting private key or seed from flash memory ICs

These require physical access to the device.
These don't really translate well into possible attacks against bitcoin core.

If you run bitcoin core on a PC say, and an attacker gets physical access to your PC, your security is probably gone.
If I had bitcoin core on a PC and suspected someone malicious had had physical access to it, I'd burn the PC and everything it touched, buy a new PC and recreate the wallet from offline backups, then transfer all remaining funds to yet another new/fresh/clean wallet or wallets.


text:
BIP32 recommends a 256 bit seed. Why do most Bitcoin wallets only use a 128 bit seed?
According to BIP32's "Master key generation" section, "256 bits is advised".
See https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki#Master_key_generation
Why did BIP32 consider 256 bits to be the recommended seed size, and why did the authors of so many Bitcoin wallet implementations consider 256 bits unnecessary?

The reasons for the 3 numbers:

Bitcoin uses 256-bit ECDSA signatures. These require in the order of 2128 steps to find a private key from the public key is known. This is Bitcoin's security level: we aim to always require an attacker to perform 2128 steps. If the seed has less than 128 bits of entropy, this inevitably leads to a faster algorithm, where an attacker can simply iterate through all possible seeds in less than 2128 steps. Because of that, BIP32 requires at least 128 bits of entropy.
When the seed has less than 256 bits of entropy (but more than 128), the resulting keys will also have less than 256 bits of entropy. As elliptic curve cryptography uses keys with a full 256 bits (despite there being a faster attacker than breaks it in 2128 steps), BIP32 also recommends 256 bits of entropy.
The combination of private key and chaincode (the "extended private key") in BIP32 are 512 bits in size. Because of that, using a seed with more entropy than 512 bits cannot possibly increase security more - it's just information that gets thrown away. Because of that BIP32 limits the seed to at most 512 bits of entropy.

Thanks for your answer, Pieter. I'm not quite clear though: As you say, both public keys (compressed representation EC points) and private keys (scalars) are each represented with 256 bits, but they both have an effective bit strength of just 128 bits. Therefore why is there any advantage at all in generating scalar keys from a seed that has any more than 128 bits of entropy? The only reason I can think of is that bit strength is reduced a little when the seed is hashed to derive subkeys, but that would explain a recommendation for a seed of e.g. 140 bits rather than of a full 256 bits.

Well, being able to break an EC key with just 2^128 steps is somewhat theoretical. In practice, these algorithms have significant memory/cpu tradeoffs, and in practice would still be much slower than a straight 2^128 exhaustive search on keys with just 128 bits of entropy. I guess the answer is defense in depth: less than 128 bits of entropy definitely hurts security. Less than 256 bits may hurt. More than 512 bits can't help.

Seems like a question best answered by the author(s) of BIP32.
Other than that it is the age old question of "how much entropy" do I need?!
This is a pretty interesting read re: 128 bits in general.
https://crypto.stackexchange.com/questions/48256/how-much-entropy-do-you-need-for-a-secure-certificate-generation
Also as a rule of thumb: when you insert 128 bits of entropy in a system, then it also forms the upper boundary for the entropy of anything derived from it.
If I'm allowed to speculate the intent of the author, 256 bits is probably advised because the eventual keys have that exact size, and that way the entropy matches the keysize.

Thanks for the idea, I've emailed the author of BIP32 to ask if he would consider answering this question on stackexchange.


text:
Reverse-engineering Bitcoin to a formal specification
Is there any attempt/project to reverse-engineer Bitcoin's reference implementation, not to an informal description (I have found those), rather to a formal specification?
I have tried searching but could not find anything using formal methods, but maybe I am just using the wrong keywords.  Thanks for any reference or insight.


text:
Bip32 Root Key in Metamask Signature?
I recently fell victim to an inferno contract that took a significant amount of my ERC-20 coins. I realize that this is a Bitcoin site, but a white hat hacker approached and offered to help, and told me that I am able to recover my funds, but I first needed to create a signature transaction by sending 0 ETH to myself but placing my BTC derived BIP32 root key that was converted into the hexidecimal form into the hex data portion of the Metamask transaction.
I first responded that I did not see how I could recover the funds, and secondly, if I place the converted root key in the data field, anyone could view the Tx, convert it back, and steal my private keys. He then said that this is not possible anymore due to the way elliptic curve keys are now, and directed me to this taproot link: https://cronokirby.com/posts/2021/08/taproot-and-bip32/.  This is all way above my head and if anyone is able to provide some information, I would greatly appreciate it.  Is giving out your BIP32 irrelevant now like he said and is there such thing as creating a signature transaction? I know that my tokens likely cannot be recovered, but of course, there is always a small hope that something may be able to be done.
Thank you.

Never, under any circumstances, give anyone else a private key of yours. In many cases the root private key can be computed back from the derived key. This is almost certainly a scam. Citing BIP32 itself: "One weakness that may not be immediately obvious, is that knowledge of a parent extended public key plus any non-hardened private key descending from it is equivalent to knowing the parent extended private key (and thus every private and public key descending from it)."

Thank you so much Pieter for the quick response.  You confirmed my suspicions and will definitely  keep from giving the root key out.  But this shows how much I need to learn besides just buying BTC.  Sorry, can I ask is the non-hardened private key the regular private key, and is the parent extended private key the BIP root key?

That really depends on the context. The root key is the root from which all other keys are derived, directly on indirectly (there is a hierarchy). Derivations can be hardened or non-hardened; which one is used depends on the standard/wallet/software/setting.

I see, thank you so much Pieter.  Sorry, if I could ask I more question as I was trying to learn more about the signatures.  He mentioned that I needed to create a transaction to myself.  Is this even closely how it is performed?  It just does not make sense to me.

I know nothing about ERC-20 or Ethereum

Ok, thank you Pieter, I really appreciate it!!


text:
What is an "empty vector" in Bitcoin?
EDIT: Pieter Wuille gave an answer (look at the comments) to the last situation (with OP_CHECKSIG). OP_CHECKSIG is pushing 0x00.

E.g. OP_0 pushes the empty vector, which represents the number 0 (to
numeric opcodesk).

Probably the lowest-IQ question ever, but what does empty vector represent in Bitcoin? Is it a single byte 0x00 (value 0) or is it literally nothing ("air") or maybe both depending on the situation?
In BIP342, the following is written for OP_IF and OF_NOTIF :

Consensus-enforced MINIMALIF The MINIMALIF rules, which are only a
standardness rule in P2WSH, are consensus enforced in tapscript. This
means that the input argument to the OP_IF and OP_NOTIF opcodes must
be either exactly 0 (the empty vector) or exactly 1 (the one-byte
vector with value 1).

I assume that here empty vector represents literally byte 0x00 (value 0) on the stack, thus it is not an "air" (nothing). I have that assumption because if the empty vector here represents an "air" (nothing) then it would require totally empty stack to perform these opcodes.
In the same BIP, for signature opcodes (OP_CHECKSIGADD, OP_CHECKSIGVERIFY and OP_CHECKSIG) and signature check it says that if the signature is an empty vector then different behaviors are performed than in pre-Taproot. I assume that here empty vector literally represents an "air", i.e. the witness item with length 0.
At the end, in the BIP342, it writes that if the signature is an empty vector ("air"; witness item with the length 0), then OP_CHECKSIG will push an empty vector onto the stack and execution continues with the next opcode. I assume that the empty vector in the case of opcodes (OP_CHECKSIG in this case) also represent byte 0x00 (value 0), since there is no point to push literally nothing ("air") onto the stack.

Am I right or wrong? Does empty vector actually represents the (3) option from the first paragraph, thus in case of stack it represents 0x00 and in case of witness stack it represents item with the length of 0? If not, can you tell me what the empty vector represents.
Would a witness item of length 1 and value 0x00 also be considered an empty vector so that the signature check also would not be performed in the case of Tapscript?

I think we first need to distinguish between the script stack as an abstract concept as it exists during script execution, and the initial stack serialization in the transaction input witness.
The stack, during execution, is a vector of vectors of bytes. For example, we could have a stack that consists of 3 vectors, the first being empty, the second being a single 0 byte, and the third consisting of 4 0xba bytes. We'll denote that as:

{{}, {0x00}, {0xba, 0xba, 0xba, 0xba}}

These would be serialized in the transaction input witness as {0x00, 0x01, 0x00, 0x04, 0xba, 0xba, 0xba, 0xba}. So when you're asking about 0x00, it depends on whether you're taking about the stack elements, or their serialization. An empty stack element is empty, length 0, no contents - but its serialization is a 0x00 byte.
Executing an OP_0 (encoded as 0x00) pushes onto the stack a empty byte vector, so starting from the previous example, we'd get:

{{}, {0x00}, {0xba, 0xba, 0xba, 0xba}, {}}

The script OP_3 (encoded as 0x53) pushes the 1-byte stack element {0x03} onto the stack:

{{}, {0x00}, {0xba, 0xba, 0xba, 0xba}, {}, {0x03}}

On the other hand, the script <0x00> (encoded itself as {0x01, 0x00} pushes the 1-byte value {0x00}:

{{}, {0x00}, {0xba, 0xba, 0xba, 0xba}, {}, {0x03}, {0x00}}

For the purpose of OP_CHECKSIG and OP_CHECKSIGADD, a signature equal to {} is special: this is the only "invalid" signature allowed. Any other signature stack element which doesn't actually verify correctly causes the script execution to abort immediately. An {} signature element simply causes a 0 (= {}) to be pushed onto the stack for OP_CHECKSIG, and no increment to occur for OP_CHECKSIGADD.
These rules do not apply to the stack element {0x00}, or any other value. The stack element has to be {} exactly. But again, that stack element can appear by being present in the witness stack, where it'd be serialized as {0x00}, or can be pushed using OP_0 inside script (which is encoded as 0x00 as well).

Wow, what an answer, amazing. Have 2 questions. 1) An {} signature element simply causes a 0 (= {}) to be pushed onto the stack for OP_CHECKSIG..., is it more accurate to say empty vector instead of 0 since 0 can also mean to push literally zero like {0x00}? 2) But again, that stack element can appear by being present in the witness stack, where it'd be serialized as {0x00} by this you mean that the witness item is of length 0?

Also, 3) If the witness stack have elements with the length of zero (0x00), it won't just ignore them but push empty vector ({}) onto the initial stack. For example, if the witness stack starts like 000000... it will push three empty vectors ({}) onto the initial stack?

(1) They're equivalent; an empty vector is how in the script language the number 0 is represented (as it's also what OP_0 pushed), so they can be used interchangeably. (2) Yes, the serialization of a 0-length stack element is the byte 0x00. (3) Indeed, empty stack elements are very much not ignored. The witness stack serialization {0x00, 0x00, 0x00} indeed represents an initial stack of 3 empty vectors.

For the first question. So how can I distinguish 0 in the case of {} and {0x00}? Both represent 0. For example, if someone says there must be a 0 on the stack, does that mean {} or {0x00}? Also, if it says, for example, that the opcode will push a 0 onto the stack or there must be a 0 on the stack for some opcode to pass, is it {} or {0x00} (since both are actually 0). From what I understand looking at what you wrote, it looks like 0 actually just represents the empty vector {}, but what is then the TERM for pushing 0 as {0x00} onto the stack  (if it exists)?

Also, when it says FALSE (for example, "opcode will push FALSE on the stack in order to..." or "must be FALSE on stack in order to opcode not invalidate script, otherwise...."), does in Bitcoin it represent an empty vector ({}) on the stack or {0x00} on the stack, or both? I assume just the first one ({}). For TRUE I know that's {0x01}.

False, 0, and empty vector are all the same thing. I don't think there is a common name for {0x00} as while it's indeed also treated as 0 in numeric context, it's not actually commonly used.

Okay, so {0x00} on the stack won't be threated as FALSE since FALSE is just an empty vector ({})?

Whenever a stack element is treated as a boolean (e.g for OP_BOOLAND), any byte vector that contains a nonzero byte is treated as "true", while others are treated as "false" (including {}, {0x00}, {0x00, 0x00}, ...) But when we say an opcode "pushes false", it is always {} because the shortest encoding is preferred.

That's the answer I was look for, thanks! Just one strange question. How does Bitcoin threat nothing ("air") on the stack. Also as FALSE or will fail the script execution? For example if some opcode require false on the stack, but it finds an empty stack?

That's an error. Anything that needs more stack elements than are available causes script execution to abort and treat the transaction as invalid.

Just one question related to the MINIMALIF rule from the BIP342. It requires either exactly 1 as a top stack element ({0x01} as a top stack element) or 0 as a top stack element. Since 0 in Bitcoin, as you previously mentioned, always denotes/represents {}, then the {} must be the top stack element in this case, and not for example {0x00}, {0x00, 0x00}. Correct?

Indeed, the MINIMALIF rule (consensus in tapscript, a standardness rule for other scripts) requires the input to an OP_IF or OP_NOTIF to be exactly {}, or exactly {0x01}.

Thanks for all the help!


text:
What external systems is Bitcoin commerce dependent on?
If there was a concerted effort to "shut it down", where are the vulnerabilities?
The potential for the Bitcoin client to communicate with other Bitcoin nodes has been discussed at length, but what about the other parts of the Bitcoin ecosystem that have we become dependent upon?  If they disappear (e.g., denial of service, or taken down), will Bitcoin continue to function as a currency and payment system?
What are the most important systems that we are dependent on and how are we affected as a result if services from these systems end?
For an example of one such service -- what would be the effect if BlockExplorer.com were to be forced offline.

Another example, where does an e-commerce site get the bitcoin exchange rate from.  How exposed is that vulnerability?

Any dependencies for mining?   I know if the top few pools were to find their network connections yanked all at once, there would be a scramble for miners to figure out a solution (e.g., a surviving pool, or go the P2Pool route.)

Whereas Block Explorer might be a useful tool, Bitcoin could survive without it.
Most important parts of the Bitcoin ecosystem, asides the main client that is, are the exchanges. They give the currency its value and without them, Bitcoin would become a fringe technology (no merchants would deal in it, as they couldn't pay their taxes with it.
Then we have the mining software and to a lesser degree the hardware (FPGA boards and the like). Should those be gone or less available it would be easier to launch a 51% attack on the network.
After that there are the mining pools. If they would disappear the miners would have a harder time earning a predictable wage. This dependancy has been mitigated with P2Pool though, so the impact is smaller than it was earlier.
After that the services are less essential. Losing the Bitcoin Forum, Wiki, SE, WeUseCoins or the like might make it harder for new people to get into Bitcoin, but wouldn't make it less functional.

I just saw that StrongCoin.com, a Javascript-based wallet service uses BlockExplorer.com for service.  When BlockExplorer is down so is the ability to use StrongCoin.com (though your private keys can still be accessed and used with other wallet services).

@StephenGornick One could use a couple different block explorers or other similar nodes as a form of backup. Probably in the future the Overlay Network will do exactly that - mitigate dependency on single websites in an uniform way.

If the exchanges go down, there is always person to person meetings in a coffee house.  Psst...Bud... Wanna buy some Bitcoins?.....

@shoelessjoe Well, it would be sort of like basing your economy on Magic cards - you know they will be worth something to someone, but unloading 1000 of them in a short amount of time will severely drop the local price and you might not even find a buyer. Coincidentally, MtGox was once Magic the Gathering Online Exchange...

That would only be a liquidity event, and a localized liquidity event.  People would have to not expect to sell so many at one time.  If there was a $5,000 bill, I couldn't expect to go into Starbucks and get change.   Thanks for your comment.  I like that the full circle has been made from Magic cards and back again.


text:
Does an eBay listing that accepts Bitcoin violate their payments policy?
eBay's payments policy says that the listing must accept at least one payment method that is PayPal, credit card, etc.   It does not appear to say that other payment methods cannot be offered though.

http://pages.ebay.com/help/policies/accepted-payments-policy.html

Would a listing that has in the description something like the following be a violation of that policy?

Also will accept Bitcoin for payment.

Under "Payment methods not allowed on eBay" we can see:

Send cash or money orders through instant, point-to-point cash transfer services (that are not banks) such as Western Union or MoneyGram

Pay using online or other payment methods not specifically permitted in this policy

So we have two points against using Bitcoins, technically making it a violation of that policy. This question, however, would be better asked directly to eBay.

I would like to add to this that Paypal (owned by eBay, and is the primary and virtually only payment method used on eBay) has been participating in on-going actions against the Bitcoin community.
They frequently close accounts and reverse transactions by anyone who buys or sells Bitcoin, and eBay frequently closes auctions involving Bitcoin. For some reason they have not taken action against Casascius coins though.


text:
What would happen if we provide to OP_CHECKMULTISIG more than threshold number (m) of signatures?
Asked here but still no answer.
What would happen if we provide to OP_CHECKMULTISIG opcode more than threshold number (m) of signatures?
For example, if the m = 3 and n = 6 and the output script is:
3 <pubkey_1> ... <pubkey_6> 6 CHECKMULTISIG

And we provide, for example, this input script (or witness stack for segWit 0):
0 <signature_1> <signature_2> <signature_3> <signature_4> <signature_5>

Will it pass or fail?

Since the activation of BIP147 (nulldummy softfork) in 2017, the answer is no, this will fail. Before that point it would have been generally non-standard but allowed.
An input to OP_CHECKMULISIG expects on the stack: (from back to front):

the number n
n public keys
the number k
k signatures
a dummy element (ignored pre-BIP147, must be 0 post-BIP147)

In your example, n=6 is read, followed by 6 keys, then k=3, and then 3 signatures. These 3 signatures do match the penultimate 3 keys, which on itself would work. However, after that a 0 is expected, yet instead there is another signature, which fails.
Before BIP147, the signature_2 would function as the dummy, and signature_1, plus a 1 (from the evaluation of OP_CHECKMULTISIG) and 0 would be left on the stack.

Actually I provided just 5 signatures (not 6) and the last one is signature_5 (not signature_6). Thus, (before BIP147) signature_3, signature_4 and signature_5 would be taken by the OP_CHECKMULTISIG, signature_2 is dummy (instead of zero), signature_1 plus 1 (as a result of OP_CHECKMULTISIG) would left on the stack. Also zero would left on the stack. Is that more correct or I missed something?

@MudjaAdjum That sounds right.


text:
What are the all benefits/advantages of OP_CHECKSIGADD over OP_CHECKMULTISIG?
What are all the benefits of introducing OP_CHECKSIGADD in Taproot instead of OP_CHECKMULTISIG?
I assume the first benefit is the following.
For example, in pre-taproot approach, if we have 3-of-10 multisig with the scriptPubKey:
3 <pub_key1> <pub_key2> <pub_key3> ... <pub_key9> <pub_key10> 10 OP_CHECKMULTISIG

and the scriptSig (or witness stack for segWit 1):
0 <signature 5> <signature 8> <signature 10>

the problem is that for every signature in witness data check must be done over all public keys in scriptPubKey (e.g. <signature 5> must be checked for all public keys up to 5th public key where they match; same for <signature 8> and <signature 10>). Therefore, there are a lot of unnecessary checks.
This does not exist with OP_CHECKSIGADD because there is a direct check between the signature and the public key and if they match, OP_CHECKSIGAD increments the counter on the stack by 1.
That is the first benefit.
Second benefit is that a bug with one additional element that must be passed to OP_CHECKMULTISIG in order to it work has been removed.
Third is that OP_CHECKSIG/OP_CHECKSIGVERITY and OP_CHECKSIGADD now work with Schnorr instead of ECDSA.

Am I right for these three benefits?
Are there any other benefits?

The signatures for a OP_CHECKMULTISIG must be provided in the same order as the public keys, and each key may only sign once. So in your example, the validation process would check <signature 5> against pub_key1 through pub_key5, then <signature 8> against pub_key6–pub_key8, and finally <signature 10> against pub_key9 and pub_key10, rather than needing to start over with pub_key1 for the latter two.

You've correctly described three of the benefits:

OP_CHECKMULTISIG is inefficient, since it performs up to as many signature checks as there are public keys. OP_CHECKSIGADD always performs one check per signature, since unused public keys are required to be paired with an empty witness item and are skipped.
Fixing the dummy stack item bug. This was partially addressed in BIP147 by requiring it to be an empty vector, which removed malleability, but since OP_CHECKSIGADD is a new opcode it can just not reimplement the bug.
Schnorr signatures, whose benefits are too many to describe here.

There are other benefits:

One other weakness of OP_CHECKMULTISIG is that it only allows up to 20 public keys. OP_CHECKSIGADD is limited to 999 public keys, as that runs into the stack limit. See for example this transaction whose input has a 998-of-999 multisig script.
OP_CHECKMULTISIG is incompatible with batch verification, which was the primary motivation for introducing OP_CHECKSIGADD (and disabling OP_CHECKMULTISIG[VERIFY] in Tapscript).
Tapscript doesn't contribute to the global sigop limit, instead allocating a sigop budget per script based on its size. This makes blocks easier to construct for miners, as optimizing for both weight and sigops is a hard optimization problem (specifically the multidimensional knapsack problem).

All of these benefits are correct, but the direct motivation is something else: the "retrying" behavior of needing to figure out which signature corresponds to which key is incompatible with batch validation - a design goal of BIP341/342. That's why BIP342 didn't just add OP_CHECKSIGADD but also disable OP_CHECKMULTISIG[VERIFY].

@PieterWuille Thanks, I've updated the answer.

Thanks, just have a question.  You wrote that for OP_CHECKMULTISIG it is necessary to perform as many signature checks as there are public keys. So for my example it means that 30 signature check must be performed? Is there any optimization? Like, for example: if you don't find a corresponding public key for the first signature, fail (don't try with others), or (since signatures must be in order) to continue from the public key for which you last found a match and not to always start check from the beggining?

@MudjaAdjum No, at most one check will be performed for every public key. If the check fails, the next public key is checked for the same signature, and the previous public key is never checked again.

@VojtěchStrnad And for the next signature it continues from the public key it previously stopped (from the public key it found matching signature)?

The keys and the signatures must be in the same order, so it's only n checks, not k*n.

@PieterWuille So when it finds a matching between some signature and public key, for the next signature it continues from that public key?

When a signature check succeeds, it continues with both the next key and the next signature. If a signature check fails, verification with the next key, but stays on the same signature.


text:
I forgot the password of my Bitcoin Core Wallet
In the past years, I had some Balance in Bitcoin Core wallet , but after 5 years, I forgot the transfer password and I only have the backup file (.dat).
Is there a solution that I can transfer the Balance with the help of my backup file and return its password?
Thank you very much

The purpose of the password is to prevent people who do not know the password from accessing the money.
Just pause and think about the above for a couple of minutes. Many highly skilled people put a lot of effort into ensuring you cannot do what you propose to do. Over many years, many other highly skilled people scrutinised the way it works to identify and eliminate weaknesses.
If there were a way around that, the password would be pointless. It isn't pointless.
If you remember that the password was very short, or you remember all but a few letters, you can try using a "brute force" tool that tries all possible variations. One that is often mentioned is btcrecover.
If you instead chose a sensible password,  it is almost certain no-one can ever find it in your lifetime.
Many people will contact you pretending to be recovery experts. Their intent is to gain your confidence and then trick you into giving them money.

Thank you for your help, what is the best   "brute force "tool to try this way in my Windows environment?

I don't know which is best but I would start with the one mentioned at the end of my fourth paragraph. I would make sure I was downloading it from the original source and I would try to run it on a machine that is not connected to the Internet.


text:
What is arbitrary witness data and unspent transaction output?
I'm learning about bitcoin, and I keep seeing arbitrary witness data and I'm confused what it really means. Like what do they mean about arbitrary, does that just mean the data can be anything you choose? I also saw this sentence somewhere: "It is arbitrary because it’s not a function of unspent transaction output." And I'm confused what they mean by UTXO? I'm still a beginner so any help would be appreciated!

Where did you see this mentioned? It's hard to answer properly without more details. Regarding "UTXO" a simple search on this site would give you https://bitcoin.stackexchange.com/questions/4301/what-is-an-unspent-output.

What is arbitrary witness data and unspent transaction output? [...] I'm confused what they mean by UTXO

Arbitrary
Arbitrary means what you think it does - "the data can be anything you choose". It has to be packaged in an acceptable way and there are limits on size. Part of a Bitcoin transaction input is an unlocking script, these scripts are created by the transaction creator who is spending money. There are various ways they can insert extra data into scripts that have no effect on script execution.
Witness Data
An improvement to Bitcoin called Segwit allowed some parts of transactions to be separated (segregated) into a new part called a transaction witness. This allowed the block size in Bitcoin to be increased without affecting older software and prepared for other improvements. The witness data can include scripts and those scripts can contain arbitrary data chosen by the transaction creator. This includes data that has no effect on the Bitcoin transaction itself, other than increasing its size. Some people use this to embed foreign data into the Bitcoin blockchain. This is controversial.
UTXO
Bitcoin transactions have inputs and outputs, inputs are references to outputs of earlier transactions. Inputs are thus spending older outputs.   An unspent transaction output is abbreviated UTXO.
These UTXOs represent units of spendable money, like coins. When you spend a UTXO it becomes a spent TXO and is no longer an unspent TXO. So every transaction effectively destroys the UTXOs in its inputs and creates new UTXOs in its outputs. The total value of outputs cannot exceed total value of inputs. Bitcoin nodes keep track of the set of all UTXOs so that they cant be spent twice.

Thank you that makes a lot of sense. I still don't properly understand what this sentence means: "That data is arbitrary because it is not a function of the UTXO (unspent transaction output) set. So bitcoin nodes don’t need to store it." I just don't understand where the arbitrary data is then stored.

The arbitrary data is stored inside an unlocking script for an input, as a series of data-push opcodes (pushing data onto a data-stack) that can never be executed because they are inside an if false { ... } endif statement. So these unexecuted pushes never alter the stack and never affect the end-result of the script. A script which evaluates to true means the transaction succeeds and UTXOs are consumed and created. So to normal nodes, script effect is identical to a normal/standard script which lacks the if false { ... } endif. The embedded data can't affect the result & so not UTXOs.

If it isn't obvious, think of if false {...} endif as if (2+2 == 5) { ... } endif


text:
How to change Bitcoin Core to allow MAX_STANDARD_TX_WEIGHT to be greater than 400000?
I am trying to create a transaction and I saw this issue here: https://github.com/bitcoin/bitcoin/issues/27490
I downloaded Bitcoin core source and started to compile following the documentation, then I changed on this file bitcoin/src/policy/policy.h to this new value {800000} like this:
astatic constexpr int32_t MAX_STANDARD_TX_WEIGHT{800000};

But When I am using make, I get this error
  CXX      libbitcoin_node_a-blockencodings.o
  In file included from ./txmempool.h:18,
             from blockencodings.cpp:15:
 ./policy/packages.h:25:34: error: static assertion failed
  25 | static_assert(MAX_PACKAGE_WEIGHT >= MAX_STANDARD_TX_WEIGHT);
  |               ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~

How to solve this to allow this new value? Just to clarify, I am trying to create an Inscription using Ordinals theory.
Thanks so much for any help!

MAX_PACKAGE_WEIGHT is statically checked to be at least MAX_STANDARD_TX_WEIGHT:
static constexpr uint32_t MAX_PACKAGE_WEIGHT = 404'000;
static_assert(MAX_PACKAGE_WEIGHT >= MAX_STANDARD_TX_WEIGHT);

If you want to increase MAX_STANDARD_TX_WEIGHT beyond 404000, you'll also need to increase MAX_PACKAGE_WEIGHT, in your case for example to 804000.
Note that changing your own node's standardness rules is usually not enough to relay your non-standard transaction to a miner willing to include it in a block.

Thank you friend for your response! :)

just wanted to put here what I did in case anyone later intend to try the same.
Here is what I changed:
Files: src/policy/packages.h
       src/policy/policy.h

file /src/policy/packages.h
...static constexpr uint32_t MAX_PACKAGE_WEIGHT = 808'000;

file /src/policy/policy.h
...static constexpr int32_t MAX_STANDARD_TX_WEIGHT{800000};
...static constexpr unsigned int DEFAULT_ANCESTOR_SIZE_LIMIT_KVB{201}; /** default is 101 */
...static constexpr unsigned int DEFAULT_DESCENDANT_SIZE_LIMIT_KVB{201};

See ya!!


text:
How to pick the block in which a transaction will get mined?
I am developping a bot for a customer with the following goals in mind:

to inscribe a satoshi with the block number that it has been mined in (ordinal inscription) on every block. i think this is called bitmap, and you write blocknumber.bitmap.

I have managed to build this bot, connect it to my local bitcoincore node and spam a minting transaction every block by listening to the number returned via "getblock" command to bitcoin core, but i don't know how to actually pick the block that the transaction will be minted in since this seems to be a random process decided by miners and their management of the mempool. So what happens is that the block number inscribed is rarely the correct one, since i try to write either the current block number or the current block number +1 or +2.
Is there a process that would allow me to either decide which block my transaction will get mined in, or to dynamically change the text as it is mined to get the actual block number?

There is no way to reliably pick the block in which your transaction will be mined. The best you can do is to broadcast your transaction right after the previous block is mined, and set the fee high enough to get it into the next block. However, this won't always work:

Transactions propagate through the network with random deliberate delays to protect the privacy of the originating node. This means a miner can find the next block before the transaction reaches them. You might be able to bypass these delays by submitting your transaction directly to all miners and pools that support it. Even without these delays the miner could mine the block before updating their block template, which they might do once every few seconds.
The next block could include transactions submitted privately to the miner (pool) that overpay your transactions and push it out of the next expected block. You wouldn't see these transactions in your mempool and so you might set the fee too low. Worse yet, the miner could be selling the entire block space in a private deal for a token mint or whatever, and no fee could get you in. Finally, your desired block might end up being empty.

Transaction locktimes don't help you because a transaction can only be broadcast after its locktime expires. Recently I've seen calls for creating an extension to the mempool that would accept transactions with soon-expiring locktimes, which would help in your case as all miners would have your transaction at the time it's ready to be mined.


text:
Is the USD amount the same for both sender and receiver?
I sent .004 bitcoin to someone. In USD the amount was $223. The receiving party said they received only USD $212. Is that possible?

Bitcoin's exchange rate is volatile, so 0.004 BTC can be worth $223 at one time and $212 not much later. If you're agreeing to pay someone (or be paid) in bitcoin, be sure to agree on an amount denominated in bitcoin.

The Bitcoin currency was designed as a separate currency in its own right, it was not designed as a means of sending other currencies.
The Euro currency was not designed for sending US dollars either. If you exchange $223 into Euros, send those Euros to someone else and they exchange the received Euros into US dollars, the amount of US dollars they receive is almost always going to be different from the amount originally used to purchase Euros. Typically when the recipient uses a currency exchange to exchange the Euros for US dollars, a different exchange rate applies (the buying, selling "spread"), so they might only get $212.
Every time you exchange one currency for another you are almost certainly going to pay a fee for that service.


text:
What is one word seed word exactly?
I have the following code
import hashlib
import binascii

def word_to_private_key(word):
    sha256_hash = hashlib.sha256(word.encode()).digest()
    private_key = binascii.hexlify(sha256_hash).decode()
    return private_key

word = "hello"
private_key = word_to_private_key(word)
print("Bitcoin Private Key:", private_key)

Here hello is converted into bitcoin private key. Surprisingly I see some transactions with that private key.
For any string of the word variable there is one btc private key
My question is - what is the relation between string and bitcoin private key?
Could you make me understand in simple way?
Also why people talk about 12 word seed? Here one word is converted into private key.

As you can see in the code, the word is hashed using SHA256 to get the private key. Because private keys in Bitcoin have 32 bytes, you can't just use any word directly as a private key, and one way of converting it into a private key is to run it through a hash function that outputs 32 bytes, such as SHA256. (It should be obvious that creating a private key from a single English word isn't secure.)
BIP39 is the most commonly used standard for 12-word mnemonics. It doesn't use SHA256 but HMAC-SHA512, and usually isn't used for creating just one private key but an entire chain of private keys using BIP32.