text:
What is the ecological impact of the crypto endgame
I'm interested in the long-term ecological implications of using cryptocurrencies. kraken states in The Truth About Bitcoin:

For example, according to a report from 2017, Bitcoin consumes around 8.27 terawatt-hours per year, compared to the 132 terawatt-hours per year consumed by the gold mining industry.
Further, the Bitcoin network can use renewable energy, meaning it can be as good for the environment as its source of electricity.

I don't see any reason to doubt that right now, but the comparison is a bit odd given that the legacy financial system is responsible for (let's say) most of financial transactions, and given that only Bitcoin is considered, instead of all cryptocurrencies.
But what if the crypto enthusiast's dream comes true and cryptocurrency will be used for a significant amount financial transactions, say 50%? What is the ecological impact of the crypto endgame? Will it be an ecological improvement over the legacy financial system?
Notes

I find the second point made in the kraken article (mining can be done with renewable energy) a bit sketchy, as this does not take into account (among other things) the hardware arms race that the miners seem to be involved in: As of the writing of this question, GPU prices are rising rapidly due to shortages, and it is virtually impossible to find a good buy on a mid-tier graphics card. So the answer must take into account not only energy consumption, but also the side-effects of mining. (I should add that at this point roughly 40% of mining is done using renewable energy. So a thorough answer should also consider the possibility that for the near future, a significant portion of crypto mining will probably still be done using non-renewable energy.) Edit: This article outlines some other possible side-effects which an answer should consider.

An answer should probably be based on a guess what the proof-of-work/proof-of-stake/proof-of-solvency/etc. ratio will likely be in the future. How much would a change of this ratio affect the ecological impact of cryptocurrency?

There are questions similar to this one, but so old that I would consider them outdated

References to scientific research that makes projections that take the points above into account are also welcome.

Proof of work allows the history of transactions on the network to be considered secure (ie, immutable).
When measuring the security of any thing, what we are ultimately asking is how much energy will it take to defeat the security system? The more value being stored, the more security we will desire.
The bitcoin network consumes energy in a very quantifiable way, at least compared to the traditional financial networks. As a miner, a large portion of your overall costs would be electricity, and this electricity is converted into security with increasing efficiency (each generation of ASIC chip has been more energy-efficient than the last, in terms of hashes/kilowatt hour). So it is easy to at least roughly quantify how much energy the bitcoin network is consuming.
On the other hand, it isn't really easy to quantify how much energy is used to uphold the traditional financial system. What do we even include in that calculation? How much digital infrastructure is required to support that system, and what does that infrastructure cost to run and maintain? How many humans are diverting their skills and attention to building and maintaining that system? Should we include the energy expenditure of things like the US military, when considering the ongoing security of the US dollar system? There is no clear answer here.
But this helps to reframe the question: given that a large portion of the security cost to maintain the bitcoin network is very directly and efficiently used to create security (ie, it is used to compute SHA256 hashes), we might start to wonder which system is actually more efficient than the other. The relative ease of quantifying bitcoin's security costs make it look terrifyingly inefficient at first glance.. but is it really? How can we make such a proclamation, without having a good idea of what we're comparing it to?
Consider: If we have some finite amount of energy to use, would it be more efficient to use it to hash some data for security, or use it to nurture a human being, who can then spend their life working to maintain the traditional financial system? How much of that input energy is actually going to create security in each case? With proof-of-work, it is a very quantifiable, large portion of the net energy, but with a human, there is going to be a lot of inefficiency and loss! It seems to me that using that energy directly for computation is going to be more efficient - despite the obvious issues that we'll encounter when trying to exactly quantify this relationship.
At this point it is probably also worth mentioning that the cost of this security is ultimately measured in the opportunity cost of that energy. Given the above question of 'energy spent on computation vs nurturing a human', we need to consider the opportunity cost of not having this human work on other projects that could benefit society (and likewise, the opportunity cost of using the energy for something other than proof-of-work mining).
If this idea of 'quantifying the energy cost of a human' seems strange or contentious to you, realize that only furthers the point about the how hard it is to quantify the energy costs of the traditional financial system. Where do we draw the line on what costs are included? The answer to that is certainly not clear, and the validity of the entire question really does depend on it.

A few other related remarks:

Bitcoin allows for permission-less financial innovation, so we should expect to see more and more novel financial constructions created upon it in the future. Being a digital asset, friction for economic activity using it is supremely low (it is as simple as communicating data to your counterparty over the lightning network, for example), and so we can likely expect to see entirely new economies arise, as users are capable of transacting value in ways they never were before. This would allow users to better capitalize the things which they already value-- which is to say it will 'unlock' value, growing what we might call the 'GDP' of humankind. With the potential to unlock value in this way, it is interesting to revisit OPs question, as that value would not otherwise be realized. To put it differently: what is the opportunity cost of not seeing bitcoin enable this expansion of economy?

Over time, humanity as a whole has consumed more and more energy, and there is no reason to believe this trend will reverse. There are of course better and worse ways to consume this energy, but it seems strange to just regard all increases in energy use as 'bad'. It seems very unlikely that humanity will stop this trend of increasing energy-consumption.

A lot of content you'll find from the 'pro-bitcoin' side of this question likes to highlight how much renewable energy bitcoin is using. I think this kind of misses the point, its great to not use energy that incurs negative externalities (ie global warming, etc), but more importantly we should consider the opportunity cost of using said energy in whatever way.

This thread by @nothingmuch on the topic of mining effeciency is really interesting! To quote from it:

the payoff from consuming finality is positive sum in this regime (higher velocity of money => more subjective value created) and the cost of a 51% attack increases even if the energy consumed decreases. since mining is zero sum and is in competition with all other uses of energy, that implies that as demand increases because of a need for higher efficiency, the relative fraction of energy expended on PoW can decrease without necessarily compromising finality.

tl;dr because money has positive utility even though it has no intrinsic value, if finality of transactions with that money needs PoW then the more useful that money is, the smaller the relative share of energy must be expended on PoW for a fixed level of finality assurance.

As reply to a few of your points:

An answer should probably be based on a guess what the proof-of-work/proof-of-stake/proof-of-solvency/etc. ratio will likely be in the future. How much would a change of this ratio affect the ecological impact of cryptocurrency?

I don't believe that proof-of-stake/etc offers the same security guarantees that proof-of-work does, and thus it cannot be used to secure systems in an equivalent manner. Without these security guarantees, we cannot realize the same benefits that we can with bitcoin (proof of work), and so in my opinion it becomes an uninteresting question to compare the two. There is a lot written about the deficiencies of proof-of-stake, this article is perhaps a good starting point.

This answer has become quite long, so I'll include a TL;DR:
Bitcoin mining consumes energy to create security in a very quantifiable manner, but the legacy system is hard to quantify in the same way. Considering the efficiency of computation, we may actually conclude that the bitcoin network is a more efficient system of attaining transaction finality, but I have not provided a rigorous proof of this by any means.
So what is the 'ecological endgame'? I think it is that the bitcoin network continues to use a large amount of energy to secure the history of the network, and it enables a huge amount of new economy in doing so, thus justifying the opportunity costs of the energy expenditure.

I appreciate the thorough answer. Thanks!

"Bitcoin mining consumes energy to create security in a very quantifiable manner, but the legacy system is hard to quantify in the same way." I'm not sure about this one. I understand what you're saying (I find the argument of freeing up so much manpower quite convincing), but still see the problem of hardware consumption for cryptocurrency.


text:
How to create a transaction without broadcasting
I would like the create a transaction without broadcasting it to the network.

testmempoolaccept is a RPC which lets you dry-run submitting a transaction like sendrawtransaction, without actually pushing the transaction into your mempool or to the p2p network. Both function identically other than the final operation of broadcasting, so error messages can be considered equal.

You can do so by building a raw transaction in the debug console (but be very careful, you can lose your funds if you're not experienced).
When building raw transactions you can think of it in three basic steps.

Create Raw Transaction createrawtransaction (https://bitcoincore.org/en/doc/0.18.0/rpc/rawtransactions/createrawtransaction/)
Sign Raw Transaction signrawtransactionwithkey (https://bitcoincore.org/en/doc/0.18.0/rpc/rawtransactions/signrawtransactionwithkey/)
Broadcast sendrawtransaction (https://bitcoincore.org/en/doc/0.18.0/rpc/rawtransactions/sendrawtransaction/)

The hexadecimal response you receive after signing (step 2) is your fully signed transaction.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Any study on the relative expected return of bitcoin mining and running the LBC client?
Both bitcoin mining and running the Large Bitcoin Collider client require computing hardware and electricity.  A difference is the LBC runs on CPU based hardware, mining profitably requires specialist hardware.  Both have an expected return.  Is it possible to compare the expected return of both?  Can it be predicted when the expected returns are likely to be equal?  For the purpose of this question, assume we do not care about potential legal issues.

Bitcoin would be a very terrible currency if it could have it's value stolen by individuals running a piece of software on their CPUs. The expected return from thefts using the "large bitcoin collider" is zero, no matter how many people are running it and no matter how long they run it for.

They are reporting they have found some addresses with coins, and do not have an obvious reason to lie.  The question is how does the rate of finding keys / kW relate to the mining value / kW.

Objectively, why would anybody use Bitcoin if it could have its funds stolen like that? Wouldn't the cryptography have been changed in the past 10 years to disallow it?

There are a number of possibilities.  An obvious one is that the risk of losing the funds is lower than the expected gain of holding the funds.  If the probability of finding a match is >0, then the expected return must be >0 (though may be less than the electricity cost).  No investment is risk free.  One could ask why write the software if it could never work

A lot more obviously; you're being deceived into thinking that iteratively searching 2^128 keys is somehow going to yield results, when it simply will not. The scale of the problem would have you convert all of the matter in the solar system into a computer, and still not be able to break keys iteratively. People lie on the internet, it happens.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What is a correct extranonce2 value in the stratum protocol?
I'm developing a stratum mining client. I read on stratum v1 docs on braiins.com how mining work is done on the stratum protocol and that the coinbase is done by concatenating coinb1+extranonce1+extranonce2+coinb2. At the end, mining work is submitted by sending this JSON: {"params": ["<worker>", "<job_id>", "<extranonce2>", "<ntime>", "<nonce>"], "id": 4, "method": "mining.submit"}.
It says that the extranonce2 value is generated by the miner but must be based on the extranonce2 size provided by the pool. The extranonce2_size im getting from the pool is: 4 (4 bytes). So an extranonce2 value of 00000001(in hex) can be valid.
However, when i'm trying to submit a random miner work to the pool(just testing) by throwing in random nonce and extraonce2(but of course using the current jobId and ntime provided by the response i got from the pool) i keep getting this error response: 20,incorrect size of extranonce2.
The random extranonce2 i'm sending is '00000002' and random nonce is 'b2957c02'. Shouldn't i get a rejected share and not an incorrect extranonce2 size? I'm still new to this and don't yet understand exactly how everything works so where's my mistake here? or is it a pool problem? I know there's a lot of stuff to do before sending the miner work but here i'm trying to send a random miner work here.
btw is the request id always 4 in stratum for submitting work(or id: 'mining.submit')?
I'm authorizing the miner before submitting.

Turns out that there was a bug in my code and not the invalid extranonce2 value!


text:
Repeating identical outputs in bitcoin transaction
I am building a script that is indexing all bitcoin transactions in real time with all their details etc. But I am looking here at this transaction - https://www.blockchain.com/btc/tx/8327996113c5ad326853c8af39424e12b02cadcab03a7e3b387902fe7d503081
I wonder why are there so many repeating outputs, I marked them with red. The outputs are absolutely identical with same address and same amount. Some of them might be spent, some of them might be not (actually in other example transactions I think I have seen that - identical outputs with different statuses - spent or unspent).
Also, might be silly question - why are some of them spent while others are unspent?

@RedGrittyBrick I don't understand what you really mean?

See "Change splitting" at https://blog.bitgo.com/utxo-management-for-enterprise-wallets-5357dad08dd1

Isn't it more like $40?

@Murch. ooops yes. Not sure how I arrived at that number. I blame fuzzy screenshot, bad eyesight and dumb carelessness. But anyway, looked to me like it might be change splitting to some amount the recipient deemed useful.

Change splitting is a good point, but in the context of this transaction, it doesn't seem obvious to me. The amounts are a bit small and the address reuse makes it unlikely to be done for privacy.

They're not "identical": transaction outputs are uniquely identified by their outpoint. The outpoint is the combination of the id of the transaction that created the output and the position of the output in the transaction's output list.
The protocol tracks funds based on outputs, not addresses. In fact, the notion of
addresses doesn't appear at the protocol level at all, it's just an out-of-band
mechanism to convey the receiver's scriptPubKey to the sender. Hence, while
weird, it's perfectly legal to create multiple outputs that pay to
the same address in one transaction.
Presumably the receiver has some sort of automated process that withdraws from a service whenever the minimum threshold is reached. For convenience they use the same address. The service however batches payments over a longer period of time than the receiver needs to reach the minimum threshold.
Since the UTXOs are not identical (see 1.), UTXOs are individually selected to be used as inputs in transactions. As the receiver apparently has accepted the privacy loss stemming from the address reuse, they also don't seem to mind the privacy loss from spending multiple times from funds received to the same address.
Alternatively, the receiver prefers to have multiple outputs of a smaller value over having a single output of a bigger value, as @RedGrittyBrick suggested in the comments. However, this usually appears in the context of liquidity management for enterprise wallets, and the amount seems a bit small for this explanation.

Thank you very much. Actually I was thinking something like point 3 in your answer.


text:
I sent bitcoins from one wallet to another. Next day, the receiving wallet shows balance 0
I have sent some bitcoins from bitcoin.com to the crypto app wallet, yesterday. But, the app still shows 0 bitcoin balance. I checked the receiving address on the block explorer and it showed that the transitions were successful. Today, I checked again. It strangely shows that the bitcoins send to an unknown address. So, could anyone please help to understand what's going on? Can I get my bitcoins back?

As you can see, 18deMYeiAe45x3fqbzhN15vafPNuSRYaCM was the sending address and 35qerZ6FLCshwCLwqv8JzgoLr9YDfCMquh was the receiving message. But there is another transaction which I have no idea about. Why does my bitcoin fund sent to an unknown address?
Just to clarify, "the crypto app" isn't crypto.com. It's thecrypto.app

This seems similar to Money sent to wallet was removed in unexpected second transaction 30 minutes later

Does this answer your question? Money sent to wallet was removed in unexpected second transaction 30 minutes later

If the "crypto app wallet" is the crypto.com app, that appears to be a "custodial account", not a normal non-custodial wallet. In that case it might be just crypto.com consolidating funds. You should contact their customer support about your account balance.
If the "crypto app wallet" is the crypto.com wallet, that appears to be a non-custodial wallet. This is a more normal kind of wallet. In that case it could be  your money was stolen. You should perhaps review security of computer, wallet, backups and private-keys or recovery phrases.
If the "crypto app wallet" is the thecrypto.app app, that is a custodial account  not a normal non-custodial wallet. In that case it might be just thecrypto.app consolidating funds. You should contact their customer support about your account balance.

https://help.thecrypto.app/?article=1 says

The Crypto App wallet is a hosted wallet. That means we host the private keys

Remember, "Not your keys, not your Bitcoin".
https://help.thecrypto.app/?article=12 says

Contacting wallet support
Please contact our wallet partner using this contact form. Wallet functionality in The Crypto App is provided by Crypterium. The Crypto App support team does not have access and insights into your account.

To me this seems a long way away from the original intent of Satoshi Nakamoto when he created Bitcoin as a way of removing any need to rely on trusted third parties.

They consolidated your funds with other clients inputs. If this is legit or a scam you can find out if you try to withdraw your funds again.
Anothe chance usually is tracing the bitcoin but in your case it maes no sense because the potential lst value is too low.


text:
What is chainwork?
If I run the RPC call getBlock on the bitcoin-qt client it gives me field called chainwork containing a hash. I can't find any information what this value is about.
Request:
getblock 0000000000000000073dacdd0178da5158bd78c543fbc79bd0dacf56a2021bcc

Result:
{
    "hash": "0000000000000000073dacdd0178da5158bd78c543fbc79bd0dacf56a2021bcc",
    "confirmations": 1,
    "size": 182400,
    "height": 304962,
    "version": 2,
    "merkleroot": "b144b39759d4669cb92ca8b9084d68d1f05f1e28e16887bf062f37e3b5f79fd2",
    "tx": [
        "3edc72d280edd0a7719670a5f0c97e8991a5bcfcfcfa77620969643f04d021a3",
        "841840f8a7863c8fb99e26975d45ff9151b0a5f021fb479aa9ff91a138d3cd3f",
        "a01af3f203313cdab4e8977cd2dccbd416872c9d9f76968fc7009f47ad630199"
    ],
    "time": 1402334610,
    "nonce": 2351732739,
    "bits": "185d859a",
    "difficulty": 11756551916.903952,
    "chainwork": "0000000000000000000000000000000000000000000086859f7a841475b236fd",
    "previousblockhash": "000000000000000031132699bcb917e7e6ce3cc6cce1f20b6ad36a437436c821"
}

Possibly this: https://bitcointalk.org/index.php?topic=136628.0

Not sure if this is about the chainwork or about best chain in logfiles of bitcoin-qt. On bitcointalk it says the hex data of best chain is just the first bytes of the rawblock. I checked if chainwork is the first bytes of the block but it doesnt match. Also best chain is not value of a blocks, it defines a block while chainwork changes at every block. But maybe its the sum of work put into this chain, i will check that tomorrow

The chainwork value is really just the total amount of work in the chain.
It is the total number of hashes that are expected to have been necessary to produce the current chain, in hexadecimal.
Converting 0000000000000000000000000000000000000000000086859f7a841475b236fd to decimal, you get 635262017308958427068157, or 635262 exahashes.
At june 2014 hash rates (100 petahash/s), it would require only 73 days to perform that many hashes, while in reality it took over 5 years. The hash rate has been going up so fast however that the impact of more than a few months ago is negligible.

Is this the number of single hashes or the number of double SHA-256 hashes? Or, asked in a different way, does double SHA-256 hashing the header count as a single hash or two hashes?

@StephenM347 that doesn't actually matter, as the chainwork is only used to compare different chains. But in practice, a double SHA-256 operation is counted as 1.

Pieter's answer is good, the chainwork value is the expected work amount in the chain, expressed as a 32 bytes integer, for the double SHA-256 hashes calculation work.
The chainwork is used to identify the correct chain, the biggest chainwork value means the strongest or the correct chain.
By the way, Satoshi didn't initially realize that choosing the correct chain by just counting blocks allows for some extremely easy attacks. Version 0.1 just counted blocks. That's why the paper just says "longest". The idea of "chain work" was added a little later. For detail about this, please refer to here.
I just want to give some info about how this 'chainwork' value comes, for the better or deeper understanding about what it is.
Let's take a look at Satoshi's genesis block header (part of related info):
$ bitcoin-cli getblockhash 0
000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f

$ bitcoin-cli getblockheader 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
{
  ...
  "height": 0,
  ...
  "bits": "1d00ffff",
  "difficulty": 1,
  "chainwork": "0000000000000000000000000000000000000000000000000000000100010001",
  ...
}

As you see above, the genesis block has a '1' difficulty and a [0x01,0001,0001] chainwork . If you have a question about what's a 'difficulty', you can take a look at this answer. So, that's the right definition:
difficulty '1' = chainwork amount [0x01,0001,0001]

Then, each time when a new block generated, we just accumulate the chainwork with new block's 'difficulty' field value, but remember for each '1' difficulty we add a chainwork amount [0x01,0001,0001].
As the block chain grows, the difficulty will increase (or sometimes slightly decrease) little by little.
Before the first time of difficulty changed, the chain block height had reached 32255.
$ bitcoin-cli getblockhash 32255
00000000984f962134a7291e3693075ae03e521f0ee33378ec30a334d860034b

$ bitcoin-cli getblockheader 00000000984f962134a7291e3693075ae03e521f0ee33378ec30a334d860034b
{
  ...
  "height": 32255,
  ...
  "bits": "1d00ffff",
  "difficulty": 1,
  "chainwork": "00000000000000000000000000000000000000000000000000007e007e007e00”,
  ...
}

As you can see above, the chainwork value is [0x7e00,0x7e00,0x7e00], which is exactly the height[32255]+1(for the genesis block) times [0x01,0001,0001].
[0x7e00,0x7e00,0x7e00] = 32256 * [1.0] * [0x01,0001,0001]

Then let's take a look at block 32256:
$ bitcoin-cli getblockhash 32256
000000004f2886a170adb7204cb0c7a824217dd24d11a74423d564c4e0904967

$ bitcoin-cli getblockheader 000000004f2886a170adb7204cb0c7a824217dd24d11a74423d564c4e0904967
{
  ...
  "height": 32256,
  ...
  "bits": "1d00d86a",
  "difficulty": 1.182899534312841,
  "chainwork": "00000000000000000000000000000000000000000000000000007e01acd42dd2”,
  ...
}

At block 32256, because the difficulty increase from 1.0 to 1.182899534312841, then
 [chainwork value] = [previous chainwork value] + [difficulty] * [0x01,0001,0001]
 [0x7e01,acd4,2dd2] = [0x7e00,0x7e00,0x7e00] + [1.182899534312841] * [0x01,0001,0001]

Upvoted for extra detail.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What is gettxoutsetinfo used for?
Could you please explain gettxoutsetinfo, what is it used for? What are the different fields in the output?
I assume that it's used to calculate the amount of the unspent trasaction outputs in the blockchain. What is bytes_serialized and hash_serialized?
{
  "height" : 225494,
  "bestblock" : "00000000eb0443fd7dc4a1ed5c686a8e995057805f9a161d9a5a77a95e72b7b6",
  "transactions" : 335344,
  "txouts" : 2221582,
  "bytes_serialized" : 72130407,
  "hash_serialized" : "3d14832b1b468f52ce942d4b9dc0c76a54fa6865207ff64540105403e3109f43",
  "total_amount" : 10886823.50380013
}

The command computes some statistics about the database of unspent transaction output that is maintained by a bitcoin node.

height: the height of the block up to which the UTXO set is accurate. This may be lower than the result of getinfo's height, as the node does not flush the UTXO set from caches after every block.
bestblock: the hash of the block up to which the UTXO is accurate.
transactions: the number of distinct transactions to which the unspent outputs belong (or: the number of transactions that have at least one non-spent output).
txouts: the number of unspent outputs
bytes_serialized: how large the database is in bytes, should we serialize it.
hash_serialized: the hash of the entire serialized database. You can use this value to compare with other nodes (if they are at the same block), to verify the UTXO set's integrity.
total_amount: value (in BTC) of all unspent outputs together. This is the best possible guess for the amount of currency in circulation.


text:
how to hash block header in c++
I want to hash a block header in C++ but dont know how.
Im just using a test block at the moment, so that I know what the expected outcome is. https://www.blockchain.com/btc/block/669892
I have a function which formats the block header into the hex string of 0000ff3fd90c0dfca35c8738aa92e1c5bdf706ba14eaafbb379206000000000000000000ad66d8cda0330c3af011ba6b7bc705789767c55a72a818685b3e06a0b8b9a2b054e82260b9210d17099976a6
Now, im struggling to properly hash this to the expected 000000000000000000064ec839564cc03166184f0a404d82cad9c655f714d886
static void calcHash(std::string hash){
    CryptoPP::SHA256 hash;
    std::string digest;
    CryptoPP::StringSource(data, true,
        new CryptoPP::HashFilter(hash, new
            CryptoPP::HexEncoder(new CryptoPP::StringSink(digest)
    )));
    std::cout << "digest: " << digest << std::endl;

    CryptoPP::SHA256 hash2;
    std::string digest2;
    CryptoPP::StringSource(digest, true,
        new CryptoPP::HashFilter(hash2, new
            CryptoPP::HexEncoder(new CryptoPP::StringSink(digest2)
    )));
    std::cout << "digest2: " << digest2 << std::endl;

   return digest2;
}

When I pass that header through this function, i do not get the expected.
How do I get the expected hash from my formatted header using c++?

The hash ending d886 isn't the hash of the header, it's the hash of the whole block.

Then I must be at a misunderstanding, I thought you calculate the header hash to compare with the target. Am I wrong?

Apologies, yes, it is just the hash of the header, but historically it was the hash of the whole block. Nowadays, the header is tied to the transaction set via its Merkle root.

To clarify, you are trying to compute the hash of the header of BTC block 669892, and you have computed that the binary/hex representation of that header is 0000ff3fd...?

Is the result you're getting? 86d814f755c6d9ca824d400a4f186631c04c5639c84e06000000000000000000

Yes, I am trying to has the header of the block which I have computed as 0000ff3fd.... If I use this website https://anyhash.com/sha256, and enable 'hex', I can get the hash you mentioned after doing it twice, but I cant seem to replicate that in C++.

Firstly, you have a name conflict; hash is both the std::string argument passed to calcHash(), and also the name of your CryptoPP::SHA256 object. This code won't even compile, so are you sure this is the code you're using?
The code itself is somewhat difficult to read, too, because you pass around anonymous objects to nested constructors in an attempt to be concise, but this just makes things needlessly confusing to the reader. Perhaps you have copy-pasted one-liners from other places?
Anyway, here's the approach you should take:

Use HexDecoder (not HexEncoder) to convert your input string (let's call it headerString) to a byte array of the data represented by that hexadecimal string (call the result headerData).

Hash headerData using SHA-256 twice using the CalculateDigest() method of a CryptoPP::SHA256 object, storing the result in a byte array (call it hash2, because we call the first hash hash1).

Use HexEncoder to convert hash2 into a string which is a hexadecimal representation of the hash (call it hashString).

Here's that in code:
#include <iostream>

using namespace std;
using namespace CryptoPP;

static void printHash(std::string headerString) {
    // Convert hex string to byte array
    string headerData;
    HexDecoder decoder = new HexDecoder(new StringSink(headerData));
    decoder.Put((byte*)headerString.data(), headerString.size());
    decoder.MessageEnd();

    // Hash the byte array twice
    SHA256 hasher;

    byte hash1[SHA256::DIGESTSIZE];
    hasher.calculateDigest(hash1, (byte*)headerData.data(), headerData.size());

    byte hash2[SHA256::DIGESTSIZE];
    hasher.calculateDigest(hash2, hash1, hash1.length);

    // Convert result to hex string
    string hashString;
    HexEncoder encoder = new HexEncoder(new StringSink(hashString));
    encoder.Put(hash2, sizeof(hash2));
    encoder.MessageEnd();

    // Print the hash
    cout << asLowerCase(hashString) << endl;
}

Thank you very much Jivan!! As a C++ newby, this has put to rest my couple days of trial and error! I have tried this, edited to fit, and it works as expected!!


text:
How do I hash header block with C++
So im trying to hash a header in C++ and I am having some trouble.
I have tried different libraries (cryptopp, openssl) can't seem to hash the header correctly.
This is the block which I am testing: https://www.blockchain.com/btc/block/669892
header.version = 1073676288; // 3ff0000
header.previous_block = "000000000000000000069237bbafea14ba06f7bdc5e192aa38875ca3fc0d0cd9";
header.tx_merkle_root = "b0a2b9b8a0063e5b6818a8725ac567977805c77b6bba11f03a0c33a0cdd866ad";
header.timestamp = 1612900436; // 6022e854
header.bits = "170d21b9";
header.nonce = 2792790281; // a6769909

I have a class/function which will format them into a hex string, which i get as: 0000ff3fd90c0dfca35c8738aa92e1c5bdf706ba14eaafbb379206000000000000000000ad66d8cda0330c3af011ba6b7bc705789767c55a72a818685b3e06a0b8b9a2b054e82260b9210d17099976a6
And the expected final hash of 000000000000000000064ec839564cc03166184f0a404d82cad9c655f714d886
I then use this hash function, and similar to get the final computed hash SHA256(SHA256(blockheader))
static std::string SHA256(std::string data, int size)
{
  // convert hex string to char
  unsigned char tmp;
  Crypto::hexStringToCharBinary(&tmp, &data, size);

  unsigned char hash[SHA256_DIGEST_LENGTH];
  SHA256_CTX sha256;
  SHA256_Init(&sha256);
  SHA256_Update(&sha256, &tmp, size);
  SHA256_Final(hash, &sha256);
  std::stringstream ss;
  for(int i = 0; i < SHA256_DIGEST_LENGTH; i++)
  {
      ss << std::hex << std::setw(2) << std::setfill('0') << (int)hash[i];
  }
  std::cout << "digest: " << ss.str() << std::endl;
  return ss.str();
}

I know that I am supposed to give the sha256 the bits and not the string of hex, because it works on a bit basis, but I dont know how to do that. I am not the best at byte manipulation in C++, or hashing,
does anyone know how to do this nice and easy, if that's possible?

Start by hashing twice. Bitcoin block hashes are double-SHA256 of the header.

I have done that. I think I am having trouble with variable types.. getting the sha(sha) to read the bytes of the hex string, and not the string. I dont know how to do that.

You already have them. You should just feed hash back to SHA256; not ss.str().

I have a class/function which will format them into a hex string, which i get as

You don't format as hex-string, it's 80-bytes raw bytes, serialized as:
- version (4-bytes little-endian)
- prev_block_hash (little-endian 32-bytes prev_block hash, since it's LE, you'll have multiple following 0's, not leading)
- merkle_root (little-endian 32-bytes merkle root)
- timestamp (4-bytes LE)
- bits (4-bytes LE)
- nonce (4-bytes LE)

I don't know how you serialize it, but here's a simple C code to build it from a struct like yours.
unsigned char ser_header[80];
memcpy(ser_header, header.version, 4);
memcpy(ser_header + 4, header.prev_block_version, 32);
memcpy(ser_header + 38, header.merkle_root, 32);
memcpy(ser_header + 72, header.timestamp, 4);
memcpy(ser_header + 68, header.bits, 4);
memcpy(ser_header + 76, header.nonce, 4);

Then hash it.


text:
For a block containing 'x' number of unverified transactions, how many SHA256d hashes are performed to obtain the hashmerkleroot?
I've been trying to understand this for a few days now.
I understand that the hashmerkleroot is the root node of a fully perfect binary tree. I also understand that the sha256d hashes would be the non-leaf/parent nodes of the tree and the sha256d hashes of the transactions would be the leaf nodes.
I also know that a fully perfect binary tree with 'x' leaves has 2x-1 nodes.
The question I'm trying to answer is what is the formula for the number of SHA256d hashes required to obtain the hashmerkleroot if the number of transactions to be hashed is 'x'? Is any of my understanding above incorrect?

If the number of leaves is exactly a power of two (i.e., n = 2k), then the number of hashes performed is exactly n-1. This is easy to see: every hash operation takes two inputs, and reduces them to a single one. After n-1 operations, that means n nodes are reduced to 1, and that one is the Merkle root.
When the number of leaves is not a power of two, it's a bit more complicated. Whenever a "level" of the Merkle tree has an odd number of leaves, a hash operation is still performed for the last one, but it only takes one input. So every time this happens, it means an addition hash operation on top of the n-1 that are expected otherwise.
How often does that happen? Let's look at the level sizes for n=9..16 leaves:

9 → 5 → 3 → 2 → 1: 11 hashes (n - 1 + 3 odd levels)
10 → 5 → 3 → 2 → 1: 11 hashes (n - 1 + 2 odd levels)
11 → 6 → 3 → 2 → 1: 12 hashes (n - 1 + 2 odd levels)
12 → 6 → 3 → 2 → 1: 12 hashes (n - 1 + 1 odd level)
13 → 7 → 4 → 2 → 1: 14 hashes (n - 1 + 2 odd levels)
14 → 7 → 4 → 2 → 1: 14 hashes (n - 1 + 1 odd level)
15 → 8 → 4 → 2 → 1: 15 hashes (n - 1 + 1 odd level)
16 → 8 → 4 → 2 → 1: 15 hashes (n - 1 + 0 odd levels)

The number of odd levels is exactly equal to the number of 1 bits set in the binary representation of the difference between the number of leaves and the next power of two. For example, for n=11 leaves, the next power of two is 16, the difference between those is 5, which is written in binary as 1012, which has two 1s. Thus, the number of hashing operations is 11-1 + 2 = 12 hashes.
Without knowing anything about the structure of the number of leaves, a lower bound on the number of hashing operations is n-1, and an upper bound is n-1+log2(n-1).

Thanks. Most of it makes sense. 2 quick clarifications: 1. Wouldn't the total number of hashes be 2n-1 (when txs are a power of 2) as all the transactions need to be hashed to form the leaves? 2. I'm not sure I follow the log2(n-1) part in n-1+log2(n-1)

I wasn't counting the transaction hashes. If you include those, add n to all results, but it's not entirely comparable, as transaction hashes are also more expensive (transactions are larger than 64 bytes). This is all also excluding all the witness hashes and the witness Merkle tree added by BIP141 (which roughly doubles things).

Thanks. Good point n the witness hashes and the transaction hashes being more expensive. Much appreciated!


text:
What hardware do pool operators use for calculating the hashmerkleroot?
I've reviewed the response here: How do pool operators do their hashing? and acknowledge that the ratio of hashing required by pool operators is incomparable to the miners.
I still wonder what hardware do Mining pools employ to perform the calculation of the hashmerkleroot for every miner? Would there be any performance improvement in utilising specialised hardware vs a generic yet powerful CPU? (especially when there are an average of 2000 transactions per block each being around 600 bytes in size)

They don't have any special hardware, as pools don't meaningfully produce merkle roots in any volume. Since the original answer in 2013 which pertained to GetWork, mining has moved entirely to using the Stratum protocol which has the mining client generate their own merkle roots for proof of work, based on a template that is provided by the pool and is the same for all clients. Modern miners can be exhausting more than  10,000+ nonce ranges a second so the previous system is simply infeasible to use.

Thanks, so by mining clients you mean the miner ASICs perform the hashmerkleroot computation?

Whatever local hardware is controlling the ASICs, usually a normal machine running linux, sometimes offloading some of the hashing to a FPGA.

Thanks. Makes some sense but I've seen cases where the Antminer S19 pro is practically plug and play to a mining pool without anything else controlling/overseeing it?
Surely this means that the hashmerkleroot is being computed by by the Antminer S19 pro ASIC?

Antminer machines contain an entire linux computer, the ASIC is not producing the merkle root.

Thanks. All makes sense now. appreciate it!


text:
SPV wallet connecting to personal bitcoin node
I have decided I want to run a bitcoin node both to support the network, learn more about how it works and also so I can participate while minimising trust in external parties. I have a bitcoin core full node connected with the initial block download completed but am trying to figure out how to connect a mobile wallet to it. From my understanding of SPV wallets they should be able to connect directly with a bitcoin core node to request the block headers and transactions through the SPV process, but I don't see any way of doing this in most mobile wallets. There does, however, seem to be a way to connect to an Electrum server.
My questions, then, are:

How does a mobile SPV wallet actually talk to the bitcoin network, and can I make my mobile SPV wallet talk to my bitcoin code node only?
What does Electrum server do differently than just connecting an SPV node to the bitcoin network or a personal node?

I have tried to do some reading on these subjects but am struggling to really pin down what each is trying to do and as a result how I should attempt to set up my node to achieve full self-reliance. I would rather understand what's going on rather than have something "just work".

How does a mobile SPV wallet actually talk to the bitcoin network, and can I make my mobile SPV wallet talk to my bitcoin code node only?

SPV nodes store only block headers which have the merkle root hash. When an SPV node asks a full node details of a transaction, the full node gives the sibling merkle node hashes which let the SPV node verify if the transaction was really included in the blockchain or not.

For example, your SPV wallet sends transaction hash of your interest to full node which is Hk assume here. The full node then sends back all the blue shaded merkle sibling hashes. Your SPV node then uses sibling hashes to compute parent hash and then so on till the root level.
For example,

(Hk + HL)->HKL

(HKL + HIJ)->HIJKL

(HIJKL + HMNOP)->HIJKLMNOP

(HIJKLMNOP + HABCDEFGH)->HABCDEFGHIJKLMNOP

Now because your SPV node has the root merkle hash, the end result of the above computation should exactly match. Not a match means the transaction isn't included in the blockchain or the full node is lying.
How to get around full node lying?
Here comes bloom filters. Bloom filters are like the description of a pattern that you can send to full node and ask give me all transactions matching this pattern. The beauty of bloom filters is that many patterns can be kind of etched onto the same area. And the SPV node will also include the patterns it is interested in and also some redundant patterns. This helps improve your privacy by not directly sending full node your data and also reduces the chance for full node to lie. There are wallets like BreadWallet that support custom SPV connections but I'm not sure if the feature is still available.

What does Electrum server do differently than just connecting an SPV node to the bitcoin network or a personal node?

It is just like another mobile or node less wallets. There's no node maintenance burden on you but the client/app you are using will ping a centralised server in most cases for using their hosted nodes. Electrum server is built on the full node. The ElectrumX implementation runs with txindex = 1 and downloads full blocks to your system. Check out ElectrumX which lets you run an Electrum server.
From there, your can configure your SPV wallet to connect to the electrum server you are running.


text:
What script unlocks this locking script?
The locking script I have is
OP_PUSH1(0x03) OP_ADD OP_PUSH1(0x05) OP_NUMEQUALVERIFY
I don't understand how the OP_PUSH works - where does it push from and does it push it onto the top of the stack? I thought script operations read from the top of the stack so I'd read the top item and push it back onto the stack which does nothing?
What would I have to put as an unlocking script on the left to get OP_NUMEQUALVERIFY to return valid here?

For pushing numbers between 1-16, you should be using OP_1-OP_16.

For pushing numbers larger than 16, you should be using OP_PUSH(OP_N/A).

E.g:
<5> push compiles to OP_5: 0x55
<17> push compiles to OP_PUSH1(0x11): 0x0111
<666> push compiles to OP_PUSH2(0x9a02): 0x029a02

OP_ADD pops two inputs and adds them together. So your ordering is wrong because OP_ADD must come after 3 and 5.

Stack must end with “1” so you should be using OP_NUMEQUAL instead of OP_NUMEQUALVERIFY, which leaves the stack with 1 instead of empty.

Your final script should be:
OP_3 OP_5 OP_ADD OP_EQUAL

And your unlocking script should be: OP_8
You can test it online at: https://ide.scriptwiz.app/

Why note just OP_2 then? That's the unlocking script that adds to 3 to equal 5

OP_ADD pops the two previous items which are 3 and 5. Then adds them together, resulting 8 and therefore leaving the stack with 8 8 OP_EQUAL. Finally OP_EQUAL pops the two previous items which are 8 and 8, then compares them.

The script in the question looks like "3 ADD 5 NUMEQUALVERIFY" which would be solved by a witness of "1 2" -- so "ADD" sees "1 2 3" on the stack and replaces the top two leaving "1 5", NUMEQUALVERIFY sees "1 5 5" and removes the top two and doesn't abort since they're equal, and finally just "1" is on the stack which is success.

I don't understand how the OP_PUSH works - where does it push from and does it push it onto the top of the stack? I thought script operations read from the top of the stack so I'd read the top item and push it back onto the stack which does nothing?

As I understand it, the script is separate from the stack. The script is read left to right and it's instructions operate on the stack.
So you start with your program pointer at far left of script and with an empty stack.
Then script instructions 0100 (OP_PUSH1 0x00) would push the value 0x00 from the script onto the top of the stack. The stack would then contain 0x00 and the program pointer would point to the right of the 0x00 in the script.

As Buruk pointed out, your script is incorrect and wouldn't function as you intend.
I believe you need a locking script
010301059387

which is
OP_PUSH1 0x03 OP_PUSH1 0x05 OP_ADD OP_EQUAL

with an unlocking script of
0108

which is
OP-PUSH1 0x08

or implicitly
 0108 ...  010301059387

 OP-PUSH1 0x08   ...  OP_PUSH1 0x03 OP_PUSH1 0x05 OP_ADD OP_EQUAL

which we can work through step by step thus:

script
interpretation of highlighted opcode
stack after opcode executed

0108010301059387
OP_PUSH1 0x08
08

0108010301059387
OP_PUSH1 0x03
03 08

0108010301059387
OP_PUSH1 0x05
05 03 08

0108010301059387
OP_ADD
08 08

0108010301059387
OP_EQUAL
TRUE

where

script is like machine-code
interpretation is like assembly language
top of stack is to left

It doesn't matter whether you consider opcodes as acting on the top of a stack, bottom of a stack, left of a stack, right of a stack, front of a stack or back of a stack -- so long as all opcodes are consistent in which end they act on. A stack is simply any one-dimensional ordering of items with an arbitrarily chosen direction.
Some languages/libraries might have opcodes/methods/instructions that act on the non-normal end -- you just have to know whether an opcode acts on the normal end or the other end, you could describe those ends as "near" and "far" or any other way you like.
It's just easier to imagine operating on a stack of sheets of paper, on a supporting surface in a gravitational field, where it is physically more convenient to add or remove items from the top than to extract or slide-in items at the bottom. These physical considerations have no meaning in computer operations but we should use words that are consistent with the analogy that is implicit in borrowing the word "stack" from everyday English.

Note that

OP_EQUAL is a bitwise operator that removes two one-byte operands from the stack
OP_NUMEQUAL is an arithmetic operator that removes two bigint operands from the stack. Bigints are longer than one byte. I believe each Bigint is 32 bytes (256 bits) on the stack.
OP_NUMEQUALVERIFY is OP_NUMEQUAL, OP_VERIFY
OP_ADD is listed  as an arithmetic operator (not a bitwise operator) which implies it needs two bigint operands on the stack, not two byte values. However it seems to work with byte operands.

See https://siminchen.github.io/bitcoinIDE/build/editor.html


text:
Easy to understand simple btc miner?
For study purpose , is there any simple btc miner code that are very easy to understand as long as i understand the concept of btc mining ?
I've seen similar question on this website but the answer refers to either a messy code that very hard to understand or an outdated code that does not support the current btc miner protocol (e.g : DiabloMiner) , that's why i'm asking this.
Any code which last update is behind 2020 i consider as outdated.

Welcome to bitcoin.stackexchange.com. You will find it useful to take the [tour] and have a quick look at the [help]. Please note that "Easy to understand" is subjective and invites opinion-based answers which don't fit well here.

Mining is still exactly the same problem as it always has been in Bitcoin, so it's not clear to me why code examples that predate 2020 are outdated.

@Murch because of newer miner protocol

So, you would like to understand how Stratum V2 works?

The GBT and Stratum protocols that are used for mining in practice haven't changed since 2012-2013 somewhere. Before that time, the "getwork" protocol was used which is now obsolete. But any code you find from the last 4 to 8 years will probably work fine (4 years because segwit extensions may matter).


text:
I think I was victim of a sybil attack
So I am currently working on a project using bitcoin.
I started with a full node that I tried to secure following the best practices available online.
Then, I set up an electrum server that was connected to the full node and on top of which I linked a software wallet, being most of the time a cold one.
In order to follow up this particular server' status, I added it to a notification service provided by this website.
And that's when things started to get weird.
I suddently started to see the full node being always 1 to 2 blocks behind the current block height. I then saw 3 connections on average to the electrum server from IP addresses that I didn't own. Finally, I saw that my wallet descriptor has been changed and that the receiving addresses of the wallet have been modified. So everytime I would receive a new transaction, it would generate a new address that was not derived from my private key.
But here is the most interesting part. For testing purpose, I had made an incoming transaction to that wallet, days before, using the first generated address that I will call "A". After I started to have doubts, I went back to verify the list of generated addresses and I couldn't find "A" anymore. It was visualy replaced by a random address "B", with the same transaction but no other information has changed (txid, inputs, outputs...). That list was different from the list I used to see in the wallet.
That's when I came back to the electrum server, seeing new connections every second, from different IP addresses. I thought "I'm being DDOSing". After seeing my full node being always late catching the last block, I was finally sure that I was victim of a sybil attack.
I then started to mitigate the attack by doing some tasks on the wallet, the electrum server and the bitcoin core node. I am not sure if it will be enough but like someone said to me one day, "there are lots of things to consider... and security-wise etc. it's a huge undertaking with lots of risks."
Edit : Adding the following question.
Question : How was it possible for the attacker to change the descriptor and the receiving address of my wallet, through the electrum server? Is there any other vulnerabilities I should be aware of in order to mitigate the risks ?

Hi nourou4them, I don't see a question in your post.

@VojtěchStrnad I just added it.

It sounds to me like you are either running a compromised version of Bitcoin Core (https://bitcoincore.org/) or Electrum (https://electrum.org/) or your laptop is compromised. A sybil attack is when all your peers are controlled by a malicious party and they control what transactions and blocks you see from the rest of the network. The attacker doesn't have access to your wallet, private keys, address generation, descriptors etc in a sybil attack scenario. I would recommend you install fresh verified versions of Bitcoin Core and Electrum on a different machine and if available use a hardware wallet/signer which is much harder for an attacker to compromise.

I have installed core from bitcoincore website, then verified it following the "Linux verification instructions" in the "download" page. For the electrum server, it was the fulcrum one, and I just installed from the official repo.

How would you explain that after I discovered the attack and mitigated it, and after I restarted the node and the server, everything came back like before. I found again the descriptor and the receiving addresses I used to know ?
It seems to me I was seeing a mirage, so it looked also like a MiM.

@norou4them: If you are installing the right versions of Core and Electrum then that only leaves your machine as compromised. You may be being man in the middled sure. But that's not what we refer to as a "sybil attack".

Indeed. Do you have any best practice guide to secure an ubuntu machine hosting a node ?

@nourou4them: I don't, no. You could ask on the Security StackExchange https://security.stackexchange.com/. Unless you're an expert I would install Bitcoin Core and Electrum on new hardware and not install anything non Bitcoin related on that hardware. The more you install the greater the attack surface.


text:
Multisig transactions after taproot
Before taproot when we had a multisig transaction the vout had an array of addresses(Image 1). Now, this multisig transaction in Bitcoin core v22.0 is represented only as a multisig type without the address field(Image 2). From my basic understanding, taproot improves the way multisig transactions are signed using Schnorr's signature. How are the addresses in vout represented? Are they abstracted in the same way(hiding the address field) as done for pre-taproot multisig transactions? Thanks in advance.
Image 1:

Image 2:

This is unrelated to taproot.
The "addresses" field in Bitcoin Core's RPC output was historical legacy that was at best outdated, and at worst confusing. Since 22.0 the field has been deprecated entirely, for all types of transaction outputs. It can (temporarily) be re-enabled using -deprecatedrpc=addresses, but in 23.0 it will be gone entirely. My suggestion is not to use it, because even when it works, it probably doesn't mean what you think it does.
In the modern interpretation, an "address" is a human-readable shorthand notation for a specific "locking script" (scriptPubKey) that can be placed in transaction outputs. Even when the output encodes a multisig policy, there is just one locking script, and thus just one address.
The historical interpretation (before the introduction of BIP16 in 2012, but it lingered on much longer) was that an address is an identifier for a public key. This interpretation is what led to having an "addresses" field in the RPC output; whenever there is a locking script with multiple public keys in it, the "address" identifier of each would be reported in that RPC field.
This historical interpretation is meaningless now, and has been for a long time. The reason is that multisig outputs today don't put multiple keys in the scriptPubKey. Instead they use P2SH (BIP16) or P2WSH (BIP141) to only put a hash of the script that includes all the public keys in the scriptPubKey. This is cheaper, simpler, and somewhat more private. From such outputs, despite being a multisig policy, the RPC output cannot infer the individual public keys, because they are only revealed by the spender at spending time, and for such outputs, the RPC would still only report 1 address.
It is no different for P2TR (taproot) outputs. There are no individual public keys that can be learned from the scriptPubKey, and thus the RPC output would only report a single "address" (the P2TR address).


text:
Unable to use I2P on Ubuntu 20.04 / 21.04
I configured the I2P according to this answer on Ubuntu 20.04 and 21.10.
But I have been getting the following error when I try to connect to any I2P peer:
I2P: Error connecting to a5qsnv3maw77mlmmzlcglu6twje6ttctd3fhpbfwcbpmewx6fczq.b32.i2p:0: Receive timeout (received 0 bytes without terminator before that)

This error occurs for any peer, not just the above.
How can I find the reason for this error and fix it?

UPDATE: Adding information requested by Prayank.
$ sudo ufw status verbose
Status: inactive

$ ss -nlt
State                Recv-Q               Send-Q                               Local Address:Port                                Peer Address:Port               Process
LISTEN               0                    128                                      127.0.0.1:7656                                     0.0.0.0:*
...

UPDATE 2: I was able to reach the website git.idk.i2p using the steps described in http://geti2p.net/en/about/browser-config .

UPDATE 3: Results of executing the steps: https://github.com/vasild/bitcoin/pull/4
$ telnet 127.0.0.1 7656
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
HELLO VERSION MIN=3.1 MAX=3.1
HELLO REPLY RESULT=OK VERSION=3.1
SESSION CREATE STYLE=STREAM ID=mysession1 DESTINATION=TRANSIENT SIGNATURE_TYPE=7
<no-response>

According to the text, a response (SESSION STATUS RESULT=OK) was expected. This didn't happen. No response was received.
Result of the other test.
$ telnet 127.0.0.1 7656
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
HELLO VERSION MIN=3.1 MAX=3.1
HELLO REPLY RESULT=OK VERSION=3.1
NAMING LOOKUP NAME=git.idk.i2p
NAMING REPLY RESULT=OK NAME=ME VALUE=80...AAA==
STREAM CONNECT ID=mysession1 DESTINATION=80...AAA== SILENT=false
STREAM STATUS RESULT=CANT_REACH_PEER

Received CANT_REACH_PEER instead of STREAM STATUS RESULT=OK.

@vasild also asks "Can you reach anything over I2P? For example, can you browse http://git.idk.i2p/ using the router's http proxy?"

I was able to fix this issue by going to http://127.0.0.1:7657/configclients in the browser and enabling "SAM application bridge".
This was disabled by default.

Can you reach anything over I2P? For example, can you browse http://git.idk.i2p/ using the router's http proxy?

Thank you for coming here @vasild! You are very welcome! Attempts to clarify the question should be in the comments to the question like @Prayank has done rather than posting as an answer. Feel free to edit your answer when we work out what the issue is.

Yes. I was able to reach this site (http://git.idk.i2p/) using the steps described in https://geti2p.net/en/about/browser-config .

I cannot post a comment to the question because "You must have 50 reputation to comment" (but apparently I can post a comment here to my answer).

Hello, can you follow the steps from here: https://github.com/vasild/bitcoin/pull/4#issuecomment-977719186 (I can't write long comments here and also all new lines are removed and thus formatting is screwed).

@vasild I updated the question text with the result of the tests described in the link you sent.

So it's not possible to create a session, hmm. I never experienced this. It looks like your I2P router is somehow not fully connected to the I2P network or broken in some other way. Can you take this to the I2P developers? The second test is meaningless without a session because it tries to use the session "mysession1" created in the first test in the "STREAM CONNECT" command. However I see a few strange things in its output: 1. you got "NAME=ME" in the "NAMING REPLY" I think it should be "NAME=git.idk.i2p" and 2. using a nonexistent session "mysession1" in the "STREAM CONNECT" did not

did not produce STREAM STATUS RESULT=INVALID_ID MESSAGE="STREAM SESSION ID mysession1 does not exist" but instead produced "CANT_REACH_PEER". It takes ~5 seconds for me for SESSION CREATE to return a result and create the session, it is talking to the I2P network. Maybe the session was also created in your case but it took way tooo long?

Which version of I2P router are you using?

i2pd version 2.29.0 (0.9.43) - Boost version 1.71.0 - OpenSSL 1.1.1d  10 Sep 2019
Ubuntu 20.04 on VMWare. Does it make a difference to use a virtual machine?
Does I2P require opening a port on the router?

2.29 is from Oct 2019, 2+ years old. The latest one is 2.39. The old one should work, but I have no other suggestion, other than to try a newer version :( ... or the java one from geti2p.net. Boost, OpenSSL and VM/or-no-VM should be irrelevant. "Does I2P require opening a port on the router?" -- no.


text:
What is exactly Randstorm vulnerability?
I've read the article from Unciphered about it, multiple times, and still fail to understand it
It basically says that wallets generated by BitcoinJs front end library from 2011 to 2015 are vulnerable because of the poor randomness generation. Especially those generated between May 4, 2011 to March 2012
But it's really vague on explaining what the actual exploit is. It could be just summarized as: it used Math.random() for randomness before March 2014, and it is a bad function
Let's look at the initial commit from March 4, 2011 : eckey.js is used for generating the private key, while rng.js and prng4.js in the jsbn folder are used for harvesting randomness.
rng.js
If rng_pool is not already initialized, it is filled with random values from Math.random()
while(rng_pptr < rng_psize) {  // extract some randomness from Math.random()
    t = Math.floor(65536 * Math.random());
    rng_pool[rng_pptr++] = t >>> 8;
    rng_pool[rng_pptr++] = t & 255;
  }

Math.random() according to the article has the cycle of 2^60 values before they repeat. The article also mentions that it fails modern benchmark tests, but I'm not sure about them
Is Math.random() the whole weakness of the story? What is the weakness actually about?
Later, the time in milliseconds is seeded to the pool
function rng_seed_time() {
  rng_seed_int(new Date().getTime());
}

And later for
SecureRandom.prototype.nextBytes = rng_get_bytes;

we initialize the state, and pass the pool as the key into the RC4 cipher
rng_state = prng_newstate();
rng_state.init(rng_pool);

from prng4.js
prng4.js
which creates a 256 value array
this.S = new Array();

and fills it with the loop
for(i = 0; i < 256; ++i) {
    j = (j + this.S[i] + key[i % key.length]) & 255;
    t = this.S[i];
    this.S[i] = this.S[j];
    this.S[j] = t;
  }

eckey.js
eckey.js uses SecureRandom() and creates our private key
var rng = new SecureRandom();
....
this.priv = ECDSA.getBigRandom(n);

But again, this tells us next to nothing about the actual vulnerability and what attacks might be used. Unciphered's article suggests that if we have GUID or IV (I guess that's a public key?), then we can do the work with just 2^32 to 2^64 values (2^48 most commonly)
Also, not sure about the clicks being added in the entropy pool, apart from:
<body onClick='rng_seed_time();' onKeyPress='rng_seed_time();'> comment.
In what way, other things are added into entropy pool apart from the initial timestamp seed?
Edit July 23, 2024:
Sorry, I forgot that ecdsa.js also has its own context
ecdsa.js
Basically, getBigRandom() method is realized in this file with rng = new SecureRandom();
Bitcoin.ECDSA = (function () {
var ecparams = getSECCurveByName("secp256k1");
var rng = new SecureRandom();
....
var ECDSA = {
getBigRandom: function (limit) {
return new BigInteger(limit.bitLength(), rng)
.mod(limit.subtract(BigInteger.ONE))
.add(BigInteger.ONE)
;
},

.


text:
Anyone tried assumeutxo yet?
I am attempting to use assumeutxo, but I may be overlooking something. I attempted to use my own snapshots from another node, but I consistently encountered the following error, where <BLOCKHASH> is present on the current blockheaders:
Unable to load utxo snapshot, assumeutxo block hash in snapshot not recognized <BLOCKHASH>

FYI I just updated this project and now i'm hosting the assumeutxo files as well:
https://bitcoin-snapshots.jaonoct.us

mainnet one, as Ava said, needs a patch

Currently, AssumeUTXO uses hardcoded block and snapshot hashes so you can only load specific snapshots. It does not work generically with any snapshot. If you want to use different snapshot blocks, then you will need to modify the source code and recompile.
There is no snapshot data set for mainnet yet, so you cannot use AssumeUTXO on mainnet without modifying the code.
The snapshot info for testnet can be found at https://github.com/bitcoin/bitcoin/blob/3695ecbf680a66b718f97d504308578d001eec49/src/kernel/chainparams.cpp#L269 and for signet at https://github.com/bitcoin/bitcoin/blob/3695ecbf680a66b718f97d504308578d001eec49/src/kernel/chainparams.cpp#L378

I'm working on adding support for this in bitcoin-s via RPC, would be a huge lift to allow a carve out on regtest? Otherwise this ends up being untestable. AFAICT results in this not being tested in the python test framework either. https://github.com/bitcoin-s/bitcoin-s/pull/5550/commits/585701568d616dfa1b1058ad5172399df0ade977
If it wouldn't be a huge lift, happy to file an issue on the bitcoin/bitcoin issue tracker at minimum and potentially implement myself. If its a complicated change... well maybe its an exercise best left to more experience hands :-)


text:
Don't have full block data on my full node
I am revising my blockchain parser and I'm encountering an issue where I only have block headers but not the corresponding block data on my machine for blocks past a certain height (on my machine it started at block #594606 and from there on I only had data for some blocks.)  There is a bit flag 'BLOCK_HAVE_DATA' which, if not set, indicates that we have the header in the leveldb database but not the corresponding block data on disk in the .dat files.
Unfortunately I need that data for my parser to work, so I don't know (a) why it is missing and (b) how to get the data on my hard drive.
Is there some command line option I can pass when I launch the bitcoin core node software for make sure I have data for all blocks on my hard drive?
Thanks,
John W. Ratcliff
** Note: I asked this question of ChatGPT and it provided the following response. I don't know if it is correct or not, but I'm going to try it.

To clarify, when you say "past", do you mean blocks older than 594606 or blocks newer than 594606?

Edit: I just realized that you say “past a certain height”. If you meant, that you don’t have the blocks below that height, my answer below applies. Otherwise, you might not have downloaded the remaining blocks yet?

It sounds like you are running your node in “pruning” mode. This is the default behavior when you are using bitcoin-qt, the GUI-mode. If you want to keep a complete copy of the blockchain you can either uncheck the option in the settings or add prune=0 in the configuration (bitcoin.conf). Either way, you will need to indeed reindex after changing the setting to reacquire the missing blockdata.


text:
entropy to mnemonic converter
how to make an entropy to mnemonic converter
to read a file with an array line by line
in python or javascript
there
var bip39 = require('bip39')
var mnemonic = bip39.entropyToMnemonic('00000000000000000000000000000000')
console.log(mnemonic)
but I can't figure out how to read from the file
if there is a python easier tell me

stackoverflow.com must already have excellent answers covering reading a file line by line in either Python or Javascript.

I know what it is, I don’t understand how to add

This is more a python programming question, not about BTC so much.

As stated by RedGrittyBlock this is really more of a web development question vs a bitcoin one. Also although it can be fun to create keys from file sources, typically it's a bad idea and should be more of a learning exercise than for actual use. To that end you should look at the built-in FileReader() API if you are seeking a simple javascript solution.
For example if you were doing a drag-drop to capture the file
holder.ondrop = function(event) {
  event.preventDefault();

  var file = event.dataTransfer.files[0];
  var reader = new FileReader();

  reader.onload = function(event) {
    var binary = event.target.result;

    //Do whatever you want with the data, sha256 hash it, etc

    //var mnemonic = bip39.entropyToMnemonic(binary);

  };

  reader.readAsBinaryString(file);
};

Source

I'm sorry, but can you explain it to me in more detail?
holder.ondrop = function(event) {
^
ReferenceError: holder is not defined


text:
Why is Bitcoin's closing price different from the opening price of the next day?
Why is Bitcoin's closing price different from the opening price of the next day? For example, below is bitcoin prices of the last month. Close price on day 08/21/2022 was 21380.6, but open price on 08/22/2022 was  21419.0.
Unlike stock markets, Bitcoin trading is 24 hour running. For stock markets, there is a time gap between opening and closing. For Bitcoin markets. the gap should be 0. For bitcoin charts (from Binance for example), I assume opening price should refer to the end of the day, 23h59m59s, and open price should refer to 0h00m00s, how could there be such a big price fluctuation within 1 second?
          Date  Close/Last     Open     High      Low
0   08/22/2022     21285.7  21419.0  21458.2  21271.2
1   08/21/2022     21380.6  21250.8  21489.9  21077.4
2   08/20/2022     21186.6  21146.8  21322.4  21126.1
3   08/19/2022     21135.4  22814.0  21170.7  20813.5
4   08/18/2022     22783.2  23457.8  23202.3  22718.2
5   08/17/2022     23447.6  23988.2  23512.3  23297.7
6   08/16/2022     24018.2  24045.7  24077.0  23834.6
7   08/15/2022     24101.5  24920.6  24240.8  23858.9
8   08/14/2022     24832.1  24602.8  25205.7  24193.8
9   08/13/2022     24589.3  24666.5  24623.4  24352.9
10  08/12/2022     24744.9  24008.3  24787.5  24337.5
11  08/11/2022     23997.1  24268.1  24090.0  23765.2
12  08/10/2022     24353.8  22913.8  24481.3  23923.3
13  08/09/2022     22876.6  23833.5  23150.5  22714.7
14  08/08/2022     23813.3  23305.4  23866.2  23636.4
15  08/07/2022     23307.0  22990.5  23339.8  23160.6
16  08/06/2022     22999.6  23266.8  23048.0  22852.3
17  08/05/2022     23202.8  23050.6  23344.4  23167.0
18  08/04/2022     23158.2  23073.1  23188.4  22593.5
19  08/03/2022     23069.8  22842.6  23214.5  22794.4
20  08/02/2022     22852.4  22853.6  23060.0  22698.6
21  08/01/2022     22876.7  23375.4  23412.7  22790.9
22  07/31/2022     23361.0  23741.9  23459.1  23265.9
23  07/30/2022     23706.7  23801.6  23864.6  23548.9
24  07/29/2022     23813.7  23918.9  23956.5  23698.0
25  07/28/2022     23924.0  23120.7  23938.7  23711.8
26  07/27/2022     23139.2  21074.1  23418.2  22714.1
27  07/26/2022     21068.7  21144.8  21279.8  21047.5
28  07/25/2022     21143.6  21863.6  21322.3  20916.6
29  07/24/2022     21802.8  22332.0  22653.3  21802.7
30  07/23/2022     22333.2  22838.2  22664.6  22311.0

This is normal for any kind of financial market
For example: https://www.investopedia.com/ask/answers/139.asp

The listed closing price is the last price anyone paid for a share of that stock during the business hours of the exchange where the stock trades. The opening price is the price from the first transaction of a business day. Sometimes these prices are different.

Like any other stock, share, currency or commodity - Bitcoin doesn't have a single price. A currency has a different price in each exchange where it is exchanged. Just as a share or commodity has a separate price in each market in which it is traded. The market or exchange normally doesn't set the price - it just records what people paid.
A published closing price might be the average price of the trades in the last 30 minutes of a day (e.g. between 23:30:00.0000 to 23:59:59.9999). Or it might be the price of the last transaction of the day (e.g at 23:59:57.7634)
A published opening price might be the price set in the first trade of a day (e.g. at 00:00:00.0001).
Prices are not continuous, they are not set by the exchange. The price of each trade simply reflects what that specific individual buyer was willing to pay and what that specific individual seller was willing to accept.
If you want to know for certain exactly how a specific exchange calculates the data it publishes, you'd have to ask that specific exchange directly.

Thanks. But Bitcoin trading is 24 hour running, different from stock markets. For stock markets, there is a time gap between opening and closing. For Bitcoin markets. the gap should be 0. Perhaps I misunderstand your answer?

As suggested above, the market can apply different calculations / different algorithms to calculate the opening price. Also, the closing price and opening price are each based on different trades - there's no reason for those individual trades to have the same price since they likely involve different sellers and different buyers.


text:
How do I gain access to the on-chain funds with hsm_secret?
I had a Core Lightning node running properly, but it suddenly somehow stopped working. I force-closed the only channel I owned (from my other node), and since I can't start the lightning daemon (with lightningd) of the problematic node, I can only access the funds with the
hsm_secret.
How is this going to work? I did setup another node, replaced its hsm_secret, ran the listfunds command and got nothing:
bitcoin@raspibolt:~/.lightningmobile2/bitcoin $ lightning-cli --lightning-dir=/home/bitcoin/.lightningmobile2 --conf=/home/bitcoin/.lightningmobile2/lightningd.conf listfunds
{
   "outputs": [],
   "channels": []
}

So, after I re-synced, I ran listfunds and it returned me the same, exact message (no outputs). I had some hope that I'd recover my force-closed funds that way, but bad luck.
I just want to say that I still have the problematic sqlite3 file, if that helps.

Does this answer your question? c-lightning node backup-restore

Hello. After a lot of time syncing, it doesn't work. I'm using the same hsm_secret, and once I enter listfunds I get no outputs.

Hi Angelo, please edit your question to incorporate what happened after trying what Michael proposed into the question post.

Hello @Murch, I have edited my post.

Hey @bl0ck3d, read this: https://bitcointalk.org/index.php?topic=5158920.msg60919627#msg60919627

@bl0ck3d, you can't. hsm_secret is the respective to a seed phrase. No more information is disclosed by the hsm_secret.

I guess no.....


text:
How 2 chains co-exist together?
I am not able to understand, how 2 chains co-exist together, like these were the observations which are ambiguity for me:

In one reading I have read, Only longest chain is valid and and rest chains transactions are sent back to unconfirmed transactions, but in bitcoin blockchain how more than 2 chains(Side Chains) co-exist together? There should be only 1 single valid chain right ?
If a honest miner has a double spent transaction in his block, which was done by some malicious person, honest miner after broadcasting his block, his block will be rejected because he has double spent transaction of some other malicious person, where this is not miner's fault. How miner is this problem solved?
If only longest chain is valid, after how many blocks in small chain transactions are discarded and sent back to unconfirmed pool?
If attacker tried to double spend and somehow tried to surpass the valid chain, how to deal with this problem?

Hi Arjun Reddy, as I wrote earlier, please only ask one question per topic. Also, I'm confused about your repeated questions about doublespends. I thought I had explained this twice before, but it seems that there is an underlying assumption that I keep missing. You do realize that there is only one block at each height in the best chain? Perhaps this topic will help: https://bitcoin.stackexchange.com/q/12427/5406?

Sorry Murch, I will make sure not to repeat again and ask a one question per topic

In one reading I have read, Only longest chain is valid and and rest chains transactions are sent back to unconfirmed transactions, but in bitcoin blockchain how more than 2 chains(Side Chains) co-exist together? There should be only 1 single valid chain right ?

From the point of view of the side chain, blocks in the main chain are not blocks at all, they're meaningless garbage because they don't follow the side chain's rules. From the point of view of the main chain, blocks in the side chain are not blocks at all because they don't follow all the main chain's rule.
In short, if there is more than one persistent distinct chain, it can only be because the two chains follow different rules.

If a honest miner has a double spent transaction in his block, which was done by some malicious person, honest miner after broadcasting his block, his block will be rejected because he has double spent transaction of some other malicious person, where this is not miner's fault. How miner is this problem solved?

An honest miner would never include a double spend in his block.

If only longest chain is valid, after how many blocks in small chain transactions are discarded and sent back to unconfirmed pool?

From the point of view of a longer chain that doesn't include those transactions, if they are valid in that chain, they are considered unconfirmed. From the point of view of any particular chain, any valid transaction not included in that chain is considered unconfirmed to long as it remains neither included nor conflicted due to inclusion of a conflicting transaction.

If attacker tried to double spend and somehow tried to surpass the valid chain, how to deal with this problem?

A chain with a double spend in it can never be valid because double spends are against the chain rules. Such a "chain" is treated no differently from random garbage.

Blocks in main chain are not valid in side chain and vice versa that is okay, but my doubt is "If only longest chain is taken into consideration and short chain is sent back to unconfirmed transactions(un technically deleting short chain) why there are other chains in bitcoin, there should be only 1 single chain right?"

@ArjunReddy What do you mean by "other chains" exactly? There only is one bitcoin chain. A side chain is not the bitcoin blockchain which has only one chain.

Sorry, My bad I had interpreted something wrong about side chain. Thank you for information :)

Blockchain forks, chain reorganizations, and stale blocks
You're mixing up a few separate concepts. Since there is no central coordination, occasionally two miners will randomly find a new block at the same time. As each of the competing blocks is seen first by some nodes and they both have the same amount of total chainwork, the network briefly disagrees on what the best chaintip is. Let's say the blockheight is 500 and the blocks are 500_A and 500_B.
Naturally the two competing blocks include much of the same transactions, since both miners were seeing the same unconfirmed transactions and picked whatever provided the most transaction fees. Perhaps there is even a transaction in block 500_A that conflicts with a transaction in block 500_B.
However, the best chain can only have one block at height 500! From the perspective of nodes that saw the 500_A first, the other block, 500_B, is not part of the best chain and doesn't count. From the perspective of the nodes that saw 500_B first, 500_A is not part of the best chain and doesn't count.
Eventually, another block is found. Let's say it is 501_A and builds on top of 500_A. As the chain-tip consisting of 500_A+501_A is now longer and has more work than the chain-tip 500_B, the tie is broken and every node recognizes the same best chain. The nodes that were already following that chain-tip with 500_A simply add the new block 501_A. The ones that had seen the block 500_B first reorganize to the new best chain-tip by reverting block 500_B, then applying 500_A and 501_A. We call the block 500_B which does not become part of the best chain a stale block. From the perspective of the best chain, it does not matter whether the stale block was ever found. It's not part of the best chain.
Sidechains
Sidechains are distinct other blockchain networks that are pegged to the Bitcoin blockchain. They can use different consensus mechanisms, allow other rules, and have specialized purposes such as more private ways of transacting or enabling additional smart contract functionality. Examples for sidechains include Liquid, and RSK.
"An honest miner includes a doublespend in his block"
In Bitcoin we track every spendable amount of bitcoin separately in the form of unspent transaction outputs (UTXOs). Every transaction output is uniquely identifiable. When creating a transaction, the transaction explicitly specifies which UTXOs it spends. Each transaction output can be spent only once.
A doublespend is an attempt to spend the same transaction output twice. Such an attempt could for example entail creating a transaction that spends the UTXO to send Alice money, and a second transaction that spends the same UTXO to send money to the attacker. The attacker then shows Alice the transaction sending money to Alice, but tries to get the transaction that returns the money to themselves included in a block instead. Since these two transactions conflict with each other, they can never be included in the same transaction history, let alone in one block.
An honest node validates all unconfirmed transactions they hear about before adding them to their own mempool. A miner builds their block template from their mempool. An honest miner would never have two conflicting transactions in their mempool at the same time, and therefore would never include them both in the same block, because they would notice that the UTXO is already spent when validating the second transaction of the conflicting pair before accepting the transaction to their mempool.
If a miner does not validate transactions before building blocks, and they end up producing an invalid block, it is their fault for not following the consensus rules.
No honest network participant accepts invalid blocks
Transactions that spend money that has been spent before are invalid. Blocks that include invalid transactions are invalid. Nodes do not accept invalid blocks. An attacker that tries to broadcast an invalid block is simply banned and ignored.

Only longest chain is valid

There can briefly exist multiple equally long chains. Eventually one of them wins.

where this is not miner's fault

It's always the miner's fault too, because he could've validated what he was mining.

after how many blocks in small chain transactions are discarded

In the moment the longer chain is detected, all the transactions from the "short chain" are put into the memory pool. After that the longest chain absorbs the transactions (that it confirms) from the memory pool.

tried to double spend and somehow tried to surpass the valid chain

chain containing a double spend is never valid (no matter how long it is).


text:
Can we send fund from one channel to another in Lightning?
Is it possible, without closing it, to transfer funds from one existing channel to another?

The Lightning Network is a network of routed payment channels. You can transfer funds from one channel to another by finding a route and adjusting the balances of the channel participants along the route. This only changes how the fixed capacity of channels is distributed in a channel.
You can not at the moment increase or decrease the capacity of a payment channel on the fly. This is technically feasible though, and is being worked on as part of the BOLT specifications effort. This is called splicing.


text:
How to setup Lightning Node on Windows Desktop in 2022?
I have gone through setting up lightning node, but a lot of resources stated to get hardware and to run LND through command line. But How can we setup a Lightning node in regular laptop with nice GUI and nice features without any hardware?

LND provides binaries for Windows and you can use a GUI-based web app such as Ride-The-Lightning (RTL). They provide fairly comprehensive documentation to get started: https://dev.lightning.community/tutorial/
I also wrote a tutorial, which has been updated for 2022: https://medium.com/@johnny_pham/guide-setup-a-lightning-network-node-on-windows-8475206807f
It uses command line but I mention RTL at the end. The RTL GitHub repository provides easy installation instructions.


text:
Why anonymity is important in blockchain, what are the core reasons?
I have searched a lot about this topic, but couldn't get a perfect answer. Why is anonymity is important in blockchain if the transactions are legit, What are the core reasons for importance of anonymity in blockchain?

Bitcoin is not anonymous, it is pseudonymous. Privacy is what is important here.

I think this is really a question about data protection rights.
For example see https://www.forbes.com/advisor/banking/digital-banking-consumer-data-privacy-concerns/

But the transfer of personal and financial information through digital channels raises an important question: How much of your personal data is protected and kept private?

...

Europe set a precedent for consumer privacy protections in 2018 with the passage of the General Data Protection Regulation (GDPR). This sweeping privacy law expanded consumer control over data access and use, including the right to request that personal data be destroyed by the agency collecting it.

In short, many people do not want it to be made public how much money they have saved up or the details of products and services they purchase.
To take an extreme example, a person who has been raped may not want their financial transactions to reveal to their neighbours, their employers and busybodies around the world that they received rape counselling, medical treatment from a venereal diseases clinic and other paid for services relating to the event.
Even most ordinary people leading ordinary lives, who never purchase legal but taboo or controversial products, like to keep their finances and purchasing history private. Anonymity is part of that privacy.
The creator of Bitcoin used the pseudonym Satoshi Nakamoto and took great care when registering the bitcoin.org domain to avoid that being used to identify them. Privacy and anonymity were obviously of great importance to them.

Your question boils down to why privacy is important if you aren't doing anything wrong (not-"legit"). I'll only answer in the scope of Bitcoin, but there's a more principled answer to the broader question.
First, the Bitcoin protocol allows any transaction into the blockchain so long as the inputs are properly unlocked and the outputs are validly formed and do not sum to greater than the inputs. A few other rules apply, but basically, any mathematically valid transaction is "legit" vis-a-vis the blockchain.
This definition of legit is quite different from a legal definition. Many mathematically valid transactions are considered illicit by financial and legal authorities.
As a universally open protocol, Bitcoin does not enforce these legal or financial definitions, which vary greatly by jurisdiction. Therefore, there are likely transactions being executed on the Bitcoin blockchain which are illegal in some areas.
Without anonymity, many people would be unable to use Bitcoin due to local laws.
Second, there are social/security problems with having one's full balance and financial history publicly available and tied to one's identity. Bitcoin would be unattractive to most people if it was easily discoverable how much coin you had, a list of anyone who has ever sent to you, and a list of anyone to whom you ever sent money.
Thirdly, anonymity makes Bitcoin more resilient against censorship and state-level attacks. Early on, there were few Bitcoiners. If their identities were publicly known, governments or other groups could have targeted those individuals and de facto shut the project down. Or today, they could compile a list and punish/tax/jail all current holders of bitcoin. Bitcoin is designed as a safe haven against the harms of legacy financial systems: inflation, censorship, taxation, and confiscation.


text:
ECDSA sign of transaction
What is true sign of transaction in bitcoin testnet (https://blockstream.info/testnet/tx/c6c232a36395fa338da458b86ff1327395a9afc28c5d2daa4273e410089fd433)  :
c6c232a36395fa338da458b86ff1327395a9afc28c5d2daa4273e410089fd433
Original RAW transaction : 0100000001d6ca712f68c4d3003b9558f772463d5b468ed44360184c9628c02e96276901cf000000002f21026d2204a9535443657a88a0724fbd49a0e78d305f50a82f2cc9dd9bea10a6c5cd0c093006020101020101017cacffffffff010000000000000000016a00000000
I counted sign as (use this help How to redeem a basic Tx? ) : 0xfdf34b9d1e3985d7662022a5ea4ba1def9954570752ca1d5d9b6fc648d5d6061, but ECDSA sign verification fails.
Values :
r = 0x1
s = 0x1
z = 0xfdf34b9d1e3985d7662022a5ea4ba1def9954570752ca1d5d9b6fc648d5d6061
pk = { 0x6d2204a9535443657a88a0724fbd49a0e78d305f50a82f2cc9dd9bea10a6c5cd,      0x496fb35210894a4614bd5a2d3f284c7ea6f6f1bd36510e39dcdb4be0d6e0f110 }
Transaction uses for signing:
0100000001d6ca712f68c4d3003b9558f772463d5b468ed44360184c9628c02e96276901cf0000000017a91458a994e9d5ed9baa03ecfd1137592a90ad3cdfc587ffffffff010000000000000000016a0000000001000000
What am I missing or doing wrong? And what is the true value of Z in this transaction and how can it be calculated ? Thanks for the answer.

Assuming you are talking about the transaction signatures and not attempting to sign the raw transaction data, I can see a few problems with this method.
First, it is worth understanding what a transaction signature is. A transaction contains signatures for each input, and thus, the signature cannot sign the entire transaction. When a typical (legacy) transaction is signed, the fields where the signatures would go is set to zero, allowing the signatures to be generated. Then these signatures are emplaced in those fields.
Read more from Jimmy Song's Programming Bitcoin: https://learning.oreilly.com/library/view/programming-bitcoin/9781492031482/ch05.html#idm46739596961624
Second, inputs have signatures, not the transaction as a whole. If a transaction spends multiple coins, there will be multiple signatures in the transaction. If a coin locked in a multisig address is being spent, there can even be multiple signatures per input.
Thirdly, this is an irregular transaction. It spends a P2SH input whose redeem script is a P2PK script. But here you can see a breakdown of the transaction's fields.
Read more about how P2SH inputs are validated here: https://learning.oreilly.com/library/view/programming-bitcoin/9781492031482/ch08.html#idm46739591457944
{
   inputs: [
     {
       prev_txid: "cf016927962ec028964c186043d48e465b3d4672f758953b00d3c4682f71cad6",
       prev_vout: 0,
       script_sig: "21026d2204a9535443657a88a0724fbd49a0e78d305f50a82f2cc9dd9bea10a6c5cd0c093006020101020101017cac",
       sequence_no: 4294967295
     }
   ],
   lock_time: 0,
   outputs: [{script_pub_key: "6a", value: 0}],
   version: 1,
   witnesses: nil
 }

The ScriptSig is where the signature (wrapped in some script) can be found. If we decode the ScriptSig, we see a Public Key and a Redeem Script. The Redeem Script is the last 12 bytes of data 093006020101020101017cac.
OP_PUSHBYTES_33 026d2204a9535443657a88a0724fbd49a0e78d305f50a82f2cc9dd9bea10a6c5cd OP_PUSHBYTES_12 093006020101020101017cac

If we parse this redeem Script, we get:
OP_PUSHBYTES_9 300602010102010101 OP_SWAP OP_CHECKSIG

There is the signature: 300602010102010101 Its last byte, 01 is the SigHash flag for SIGHASH_ALL. The rest, 3006020101020101 is a DER-encoded signature.
For some reason I can't explain, this signature has (r,s) = (1,1). I don't know why this is valid.

Thanks a lot for the clarification! The question is what real Z(hash) value was used to sign input 0 of the above transaction and how this Z value was obtained. I gave the transaction in the form in which the Z value is also calculated above and the calculated SHA256(SHA256()) value. But at the same time, the verification of the digital signature turns out to be incorrect. Because of the wrong Z value.

Ahh, okay. the z value is derived based on the type of SigHash flag used. The SigHash flag is the last byte of the signature. In this, and most cases, that is 1 for SIGHASH_ALL. read more: https://river.com/learn/terms/s/sighash-flag/ or https://learn.saylor.org/mod/book/view.php?id=36341&chapterid=18919

I've updated the original answer for more in depth explanation.

Thank you for your comment! The fact that the values (r=1,s=1) I indicated in the very first message (in the text of the question). There is nothing special about this, since there is a point on the curve with coordinates: (0x01, 0xbde70df51939b94c9c24979fa7dd04ebd9b3572da7802290438af2a681895441). I also used the SIGHASH_ALL flag according to the rules that you gave above, and also led a transaction to calculate the Z value in which the SIGHASH_ALL flag
is present.

0100000001d6ca712f68c4d3003b9558f772463d5b468ed44360184c9628c02e96276901cf0000000017a91458a994e9d5ed9baa03ecfd1137592a90ad3cdfc587ffffffff010000000000000000016a0000000001000000
By hashing sha256(sha256()) this transaction I get
Z=0xfdf34b9d1e3985d7662022a5ea4ba1def9954570752ca1d5d9b6fc648d5d6061.
But with this Z value, the digital signature verification does not pass, although this signature is valid (because this transaction has been added to the test blockchain and it has a lot of confirmations).

I just can't understand why the z value I'm calculating is incorrect? Where I made a mistake.

Hi Bit_user, Stack Exchange is not a forum, it's a question and answer platform. Please do not post other answer posts to reply to answer posts, each answer post must address the question independently. Each topic is based on a single question. Depending on the scope of follow-up questions, either please create a comment on the answer post to request clarification, edit your original question to clarify what your question was, or create a new topic. If your update substantially changes the question, it's better to ask a new question than to invalidate answers that fit the original question.


text:
Translating inputs (and outputs) to find addresses involved in transaction?
Is there a way for me to translate the input and output scripts in a bitcoin transaction?
I am building an application that lets you query different stats from the Bitcoin-network, down the line I want it to be able to "trace addresses" as well.
I have tried to read up on Script and how a transaction works but I am a bit confused.
Since the built-in functions in bitcoin-cli (and most all block explorers) are able to find the addresses, I'm sure I should be able to parse and translate the scripts… or?

Welcome to Bitcoin.stackexchange.com. Do you simply want to know how, in principle, to convert any Bitcoin script (in Hex say) to a standard address for each of the transaction output types for which an address is well-defined?  (If so, all the discussion of BigQuery may be a hindrance to getting an answer)

Ah thanks for the input, I agree. Just edited

An address is just a standardized way to encode an output script. They aren't part of the Bitcoin protocol, and only a few standard scripts have an associated address. As of now these are P2PKH, P2SH, P2WPKH, P2WSH and P2TR.
I won't go into exact details on how to convert each of these scripts into an address, but I'll give helpful links.
P2PKH (Pay to Public Key Hash)
A P2PKH output scripts looks like this:
OP_DUP OP_HASH160 OP_PUSHBYTES_20 <key_hash> OP_EQUALVERIFY OP_CHECKSIG

The <key_hash> part is a 160-bit hash (SHA-256 then RIPEMD-160) of a public key. The address always starts with a 1. More details here.
P2SH (Pay to Script Hash)
A P2SH output script looks like this:
OP_HASH160 OP_PUSHBYTES_20 <script_hash> OP_EQUAL

The <script_hash> part is a 160-bit hash of a redeem script. The address always starts with a 3 and is constructed similarly to P2PKH. More details in BIP13.
P2WPKH (Pay to Witness Public Key Hash)
A P2WPKH output script looks like this:
OP_0 OP_PUSHBYTES_20 <key_hash>

As in P2PKH, <key_hash> is a 160-bit hash of a public key. The address always starts with bc1q. More details in BIP173.
P2WSH (Pay to Witness Script Hash)
A P2WSH output script looks like this:
OP_0 OP_PUSHBYTES_32 <script_hash>

The <script_hash> part is a 256-bit hash (double SHA-256) of a redeem script. As with P2WPKH the address always starts with bc1q and is also specified in BIP173.
P2TR (Pay to Taproot)
A P2TR output script looks like this:
OP_1 OP_PUSHBYTES_32 <key>

The <key> part is a BIP340 public key. The address always starts with bc1p. More details in BIP350.


text:
Specification of Bitcoin RPC getbalances
I would like to track balances using address, but I have a few questions.

If I don't use the rescan option, will the balance be incorrect?
Why is the balance off when I check it against an external balance inquiry site?

The procedure using the API was to execute importaddress(no rescan) and then getbalances.
Because the balance was off, I executed importaddress with rescan, but the balance was off when I tried again.
I tried several times with other addresses, but it didn't work!
Should I change Wallet every time I look up a different address?
For example,
❯ curl --user user:password --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "importaddress", "params": [ "38XnPvu9PmonFU9WouPXUjYbW91wa5MerL", "", false ]}' -H 'content-type: text/plain;' http://--.--.--.--:----
{"result":null,"error":null,"id":"curltest"}

❯ curl --user user:password --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getbalances", "params": []}' -H 'content-type: text/plain;' http://--.--.--.--:---- | jq .
{
  "result": {
    "mine": {
      "trusted": 0,
      "untrusted_pending": 0,
      "immature": 0
    },
    "watchonly": {
      "trusted": 3239.83164697,
      "untrusted_pending": 16.05599144,
      "immature": 184.81980076
    }
  },
  "error": null,
  "id": "curltest"
}

However, when referring to an external site, it is significantly off.
https://chainflyer.bitflyer.jp/tool/balance

Any advice would be greatly appreciated!

If I don't use the rescan option, will the balance be incorrect?

Definitely. Without rescanning, the wallet won't know about any transactions that affect your balance.

Why is the balance off when I check it against an external balance inquiry site?

There is no way to answer that without knowing what you were looking at, or what you were expecting. Please update your question with details like which address you imported, what else is imported in your wallet, and what reference you used to compare with.

Thanks for getting back to me!
I've updated the question, what do you think?

Are you importing multiple addresses into the same wallet? getbalances (and other balance RPCs) report the balance of the entire wallet.

That's right! Thanks to you I understand about getbalances. So if I want to import other addresses, I have to separate Wallet. Or is there an API to remove the import of addresses?