text:
How to use getblocktemplate method with Bitcoinlib Library
I am using
bitcoinlib to communicate with bitcoin-qt
This is the code i tried:
 Dim BitcoinService As Services.RpcServices.RpcService.IRpcService = New Services.Coins.Bitcoin.BitcoinService("http://localhost:8332", "Username", "Password", "wallet password", 200)

for getblocktemplate this is the call
   Dim seg As JToken = "{""rules"": [""segwit""]}"

        RichTextBox5.Text = BitcoinService.GetBlockTemplate(seg).PreviousBlockHash

It gives me this error
BitcoinLib.ExceptionHandling.Rpc.RpcInternalServerErrorException
  HResult=0x80131500
  Message=JSON value is not an object as expected
  Source=BitcoinLib
  Arborescence des appels de procédure :
   à BitcoinLib.RPC.Connector.RpcConnector.MakeRequest[T](RpcMethods rpcMethod, Object[] parameters)
   à BitcoinLib.Services.CoinService.GetBlockTemplate(Object[] parameters)
   à WindowsApp3.Form1.Button2_Click(Object sender, EventArgs e) dans C:\Users\Hama\source\repos\WindowsApp3\WindowsApp3\Form1.vb :ligne 76
   à System.Windows.Forms.Control.OnClick(EventArgs e)
   à System.Windows.Forms.Button.OnClick(EventArgs e)
   à System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
   à System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   à System.Windows.Forms.Control.WndProc(Message& m)
   à System.Windows.Forms.ButtonBase.WndProc(Message& m)
   à System.Windows.Forms.Button.WndProc(Message& m)
   à System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   à System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   à System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
   à System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
   à System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
   à System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
   à System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
   à Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.OnRun()
   à Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.DoApplicationModel()
   à Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.Run(String[] commandLine)
   à WindowsApp3.My.MyApplication.Main(String[] Args) dans  :ligne 81

  Cette exception a été levée à l'origine dans cette pile des appels :
    System.Net.HttpWebRequest.GetResponse()
    BitcoinLib.RPC.Connector.RpcConnector.MakeRequest<T>(BitcoinLib.RPC.Specifications.RpcMethods, object[])

Exception interne 1 :
WebException : Le serveur distant a retourné une erreur : (500) Erreur interne du serveur.

I dont understand what i am doing wrong, i think it's a json problem
Help is appreciated, thank you

If anyone else need it i post the solution here:
Dim seg As JToken = JToken.Parse("{""rules"": [""segwit""]}")


text:
why halving happens each 210,000 blocks?
target time between blocks = 10m
=> 6 blocks per hour
=> 6 * 24 = 144 blocks per day
=> 6 * 24 * 7 = 1,008 blocks per week
=> 6 * 24 * 7 * 52 = 52,416 blocks per year
=> 6 * 24 * 7 * 52 * 4 = 209,664
But halving happened in block 210,000???

I don't think you'll get a definitive answer here. It's a constant that was chosen by Satoshi, and without leaving much information about it behind before disappearing.
It's possible that this was chosen in function of the total supply. If the goal was having a limit of 21 million BTC specifically, and starting with a subsidy of 50 BTC/block specifically, then halving every 210000 blocks is correct solution.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Importing private key to bitcoin node is slow. Can it be faster?
I have launched the bitcoin node and synchronized it. The issue is that importing the private key forces the node to scan the blockchain for the operations with the key. Is there a way to import the key in a fast way?

You can skip the rescan, but then you won't have transactions and can't spend existing outputs to it. Or if it was a recently created key, you can specify its birth time using importmulti, which will only rescan the chain after that point.

BIP158 blockfilterindex should help in the future.

No.
Bitcoin core is a full node and doesn't use SPV or any equivalent. Therefore, when importing a private key, it must scan the Blockchain copy it holds rather than trust third parties.
Sweeping the private key into Bitcoin core ought to be faster.

what do you mean by "sweeping"?

I don't believe it is actually possible to "sweep" into Bitcoin Core like is possible with Electrum and other 3rd party wallets.  Core requires importing the key and scanning against transaction history from 2009. I'm beginning to think that for old BIP32 and BIP39 seeds it's the only way to recover coins.  It might take hours/days to scan.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Running bitcoin node with already verified blockchain
I'm following this guide https://stadicus.github.io/RaspiBolt/ to setup a bitcoin full node on my new raspberry pi.
I already downloaded and verified the blockchain on my host machine. I also pruned it (6GB) so it can fit on my microSD card. I moved the blocks and chainstate folders to ~/.bitcoin on my raspberry. But when I run the bitcoind service with the same bitcoin.conf file proposed on the guide, the verificationprogress shows that it starts from 0.
I've tried with these options too:
prune=6000
reindex=1
reindex-chainstate=1
But none of them make the progress to be completed. I also tried waiting some minutes in order to see if the progress completes faster but that's not the case. What options do I need?

I've found it. I've removed testnet=1 and now it's fully verified. reindex* options are not necessary.
Huh, I had verified the blockchain from main net. I didn't know the testnet was using a separate blockchain?

Yes, testnet is an entirely separate system. Separate network, separate blockchain, separate currency, separate nodes.


text:
BTC deposit old address exchange
i had BTC deposit to old address in coinsbit.io exchange after 20 days they told me we dont have access to old addresses to bring back your money and i know the only one can have this access is coinsbit to transfer it. i tracked my old address in coinsbit they transfer my money after 5 days to another addreses . i sent ticket many times but they are saying we dont have access . is there anyway to bring this money by report or something? my old address is : 3ETDfiumVEBTZP1HEKDv58Uoiv2jbnomTF
my two hash link: https://blockchain.coinmarketcap.com/address/bitcoin/3BFZmH5ZYJxkqtVLNzi3YEeWQ8nNrxq7zC
https://blockchain.coinmarketcap.com/address/bitcoin/3BYeChb5aeXmWBP33318nnHyZhNYpmTqBu

Unfortunately, this is considered off-topic as your talking about an issue with a company. Giving you advice that you could have been scammed by the company. Next time when receiving the coin, consider a software wallet and never online wallet (unless it is trustworthy). TL;DR: If you do not own the private key, you don't own the coins.

You are right, the only ones who should have access to that address is CoinsBit.io.
If they claim they are unable to recover the coins, there are only three possible scenarios:

They have lost all copies of the private keys, or the root key used to derive them - this would be extremely irresponsible of any exchange, but does not appear to be the case here since the coins have been spent, which requires access to the keys.
They have been hacked and the old address' keys are compromised - this would imply that the coins were spent by an attacker, and the exchange no longer has access to them.
They are simply exit scamming and taking funds from users.

In all three cases, no one here can help you. Bitcoin transactions are irreversible, and without the private keys the funds are immovable. You should consider engaging a lawyer and going after them via the legal system.

so they are scam cause they told me 10 days ago we dont have access but they transfer my BTC 8 days ago to another addresses

We can't tell you if they are a scam or not, you should consult a lawyer

they steal our money in  many ways thats why they are scam and i saw many people that they lost money because of coinsbit.io even my friends


text:
bitcoin node startup daemon with created wallet
When I restart the Bitcoin node, I would like to load the wallet I created in the "wallets" subdirectory.

bitcoin-cli createwallet "1"
bitcoin-cli listwallets -> result : "", "1"
bitcoin-cli -testnet stop
bitcoind -testnet -daemon
bitcoin-cli listwallets -> result : ""

thanks.

You load a wallet using the loadwallet command.
To load the wallet named 1, do
bitcoin-cli loadwallet "1"

In order to have that specific wallet always loaded, add to your bitcoin.conf file:
wallet=1

Is there any way I can load it when the node starts?

You add the bitcoin.conf line I mentioned in my answer.


text:
Bitcoin Module - Hash512-EVP_ByteToToken
Is there any standalone software modules created which allow the user to complete the following:
     1. Enter in passphrase
     2. Enter a given salt
     3. Enter the number of rounds
I am looking for this in order to find the value, and locate the memory location in either Bitcoin src code or Wallet.dat file database. I have found the master key, iv, salt and rounds in wallet.dat file already. Thanks for all the help and I will pass tips to those who point me in the right direction. Thanks again.

What are you trying to do? This sounds like you have some issue for which you think that such a software is the solution. What is the problem that you are actually trying to fix?

I am tracing source code with my passphrase to understand how Bitcoin Core works exactly as far as Encrypt/Decrypt of passphrase. The function EVP_ByteToToken, I do not follow exactly, thus the request.

Bitcoin Core no longer uses OpenSSL in wallet encryption (or in the vast majority of the software). Whatever OpenSSL functions that were used have been reimplemented in Core to be backwards compatible. It is probably easier for you to read the source code for up-to-date Bitcoin Core rather than older versions that still use OpenSSL.

Thanks. I am using the source code from https://github.com/bitcoin/bitcoin.

allet encryption uses AES-256-CBC to encrypt only the private keys
that are held in a wallet.  The keys are encrypted with a master key
which is entirely random.  This master key is then encrypted with
AES-256-CBC with a key derived from the passphrase using SHA512 and
OpenSSL's EVP_BytesToKey and a dynamic number of rounds determined by
the speed of the machine which does the initial encryption (and is
updated based on the speed of a computer which does a subsequent
passphrase change).

Where do you see that information? EVP_BytesToKey appears in only one place and it is in a different comment about the function that replaced it. Wherever you are reading that is out of date and you should read the code itself.

bool CCrypter::SetKeyFromPassphrase(const SecureString& strKeyData, const std::vector& chSalt, const unsigned int nRounds, const unsigned int nDerivationMethod,

Crypter.cpp, when i=0 which is does on encryption and decryption, the if loop occurs as follows:  if (nDerivationMethod == 0)
i = BytesToKeySHA512AES(chSalt, strKeyData, nRounds, vchKey.data(), vchIV.data());


text:
LCC Electrum Wallet Set Up With Ledger Nano S Cannot Find Private Keys, Sweep or Generate Seed
I have a problem with Electrum LCC Wallet. I set up the wallet using Ledger NANO S and seems it somehow attached itself to the LTC wallet in Ledger. I get this message every time I want to send LCC from the Electrum wallet.
error: {'message': 'the transaction was rejected by network rules.\n\n16: mandatory-script-verify-flag-failed (Signature must use SIGHASH_FORKID)\n[ ... really long body of hashed content ... ]', 'code': -1}
Since it was initiated with Ledger NANO S I have no way to see private Keys, Sweep or Generate a Seed. Im stuck with a handful of LCC which I want to move. I tried restoring the wallet with no luck as the info I need is encrypted in Ledger. How can I find private key or seed to restore, thank you.


text:
Can I recover my USDT sent to a USD/PAX?
HI I sent USDT from my binance account to usd/pax blockchain wallet how i can recover that funds please ?

If "usd/pax blockchain wallet" refers to a service provided by Paxful, you should probably reach out to Paxful's customer service. Voting to close as off-topic for customer service request.


text:
Can't startup Bitgo on Windows
After installing bitgoJS, I cd into the bin directory and run the below command
bitgo-express --debug --port 3080 --env test --bind localhost I got,
'bitgo-express' is not recognized as an internal or external command,

Disclosure: I'm a Software Engineer at BitGo.
You can find a guide for running BitGo Express in the corresponding module directory:
https://github.com/BitGo/BitGoJS/tree/master/modules/express
Please direct further questions to the BitGo support team via email to support@bitgo.com.


text:
Why is the 'Estimated BTC Transacted' referring to just one of the multiple outputs?

I recently started learning about bitcoins and came by this transaction. Here there are multiple outputs so if we consider one of them to be the change received by the sender then why is the 'Estimated BTC Transacted' only summing up to one of the output values? What happens to the rest of the outputs?

It's a best effort guess made by blockchain.com, has little relevance, is not particularly accurate, and should not be used for any meaningful analysis. It has no impact on the transaction, and is not part of the Bitcoin protocol.
It is somewhat accurate for very simple transactions, but breaks down for larger ones, and ones that utilize more recent anti-tracking approaches such as randomized outputs, sending change to an address of the same type as the other address being payed, etc.


text:
MacOS: How to run Bitcoin Core GUI after building from source?
I built Bitcoin Core from source by following this guide: https://github.com/bitcoin/bitcoin/blob/master/doc/build-osx.md
And got it running on the command line via: ./src/bitcoind
Screenshots from https://bitcoin.org/en/full-node indicate that there is a GUI.

However, the guide does not mention (at the time of writing this) how to get the GUI running.
So where do I find it? How do I run it?

The GUI binary is called bitcoin-qt, not bitcoind.

@PieterWuille I don't see that binary alongside bitcoind. Any particular reason bitcoin-qt wouldn't get built along with everything else?

For example because you don't have Qt5 development files installed.

@PieterWuille I followed the official build guide (link in OP). What should I be doing differently to ensure "Qt5" files get installed?

It does tell you to install qt using brew, so I suspect it should just work. Note that the qt binary will be in the src/qt/ subdirectory. If it's not there, I can't help you as I don't have Mac hardware.

Yes, I did install the qt dependency with brew, as per the guide. I do see lots of files in the src/qt/ folder, however no binary named bitcoin-qt there.


text:
Why is Bitcoin Core full node so slow to sync?
After 12 hours, progress changed from progress=0.533751 to progress=0.541254, less than 1% progress.
$ ./bitcoind
... 0 hours:
2019-10-12T17:57:11Z UpdateTip: new best=00000000000000000126b2b7fccdfa34176fa546f2fadb91894ed4d65a9bb84e height=479855 version=0x20000002 log2_work=86.909956 tx=245200772 date='2017-08-09T21:27:46Z' progress=0.533751 cache=1.2MiB(9101txo)
2019-10-12T17:57:44Z UpdateTip: new best=0000000000000000008642926c765b5b9463613e6b3ed71044f0a4299e631975 height=479856 version=0x20000002 log2_work=86.909995 tx=245203098 date='2017-08-09T21:40:43Z' progress=0.533756 cache=2.3MiB(16894txo)
2019-10-12T17:58:25Z UpdateTip: new best=000000000000000000a5ca291f556033c0025ab3759a9faa8e090c07580faff8 height=479857 version=0x20000002 log2_work=86.910034 tx=245203927 date='2017-08-09T21:44:19Z' progress=0.533758 cache=2.7MiB(20324txo)
... 12 hours:
2019-10-13T03:41:24Z UpdateTip: new best=000000000000000000e226d20ce5690f40b0ea8c53b21b5f585bdc98d6acd4ca height=481659 version=0x20000002 log2_work=86.979252 tx=248713371 date='2017-08-23T00:57:07Z' progress=0.541245 cache=594.8MiB(4457815txo)
2019-10-13T03:41:33Z UpdateTip: new best=00000000000000000073002a56f4c75f273105b3b6efa15ed4f671c2bfbe777b height=481660 version=0x20000002 log2_work=86.97929 tx=248715388 date='2017-08-23T01:01:59Z' progress=0.541249 cache=595.2MiB(4461557txo)
2019-10-13T03:41:37Z UpdateTip: new best=00000000000000000071fa18dbf2c4a08387ea31465ba8bdc20dbe3b60fd6b1a height=481661 version=0x20000002 log2_work=86.979327 tx=248717445 date='2017-08-23T01:17:05Z' progress=0.541254 cache=595.3MiB(4462014txo)

My internet connection seems fast enough (512 KB/s down, 128 KB/s up), but that doesn't appear to be the issue, as bitcoind doesn't even seem to using much or any bandwidth anymore (it did before, but I'm guessing it downloaded enough and needs to process now).
However, I would expect the CPU to be working hard now to validate everything, but the CPU is barely busy at all.
Why is this so slow? Where is the bottleneck? The process doesn't appear to be hitting anything hard (CPU, network, or hard drive). Why not?

Incase you have't already done it, quit bitcoind and relaunch it: bitcoind -daemon -blocksonly. By default, bitcoind relays transactions while you're syncing and using -blocksonly turns that off. It probably won't make a huge difference but should save you some bandwidth.

I don't know why it's so slow, but I always kickstart the initial download by using a torrent instead of the client to download the blockchain. getbitcoinblockchain.com was a goot source for a torrent, but they stopped updating in May, so now you could use the RaspBlitz torrent, which updates roughly ones a month or so: https://github.com/rootzoll/raspiblitz/blob/master/raspiblitz-v1.3-2019-09-05.torrent

First of all you need to tail your $BITCOIN_DATADIR/debug.log and search for clues. You might solve your problem by just looking at the messages there. Here's a sample:
2019-10-13T18:17:11Z socket recv error Connection reset by peer (104)
2019-10-13T18:24:48Z socket recv error Connection reset by peer (104)
2019-10-13T18:28:17Z socket recv error Connection reset by peer (104)
2019-10-13T18:29:10Z socket recv error Connection reset by peer (104)

In my case I had several peers drop their connection but the syncing soon restarted with fresh peers.
The blocks you're stuck on are from August 2017, right after the Bitcoin Cash fork. Are you running the most recent version of Bitcoin Core? Have you tried clearing the blockchain altogether and restarting sync from zero? (Note: do not delete your wallet.dat file, just the files under blocks/ subdirectory.)
Is there other software running that could be connecting to peers' port 8333? Peers might be banning your IP for some reason. Is this PC clear of malware that could be scanning for port 8333 of network peers? This could cause your IP to be blacklisted.
Finally, have you tried forcing a IP address change? If this is network-related then rebooting your gateway to force a new IP could solve the problem.


text:
bitcoin core, deleted receiving address?
Hi I set an address on bitcoin core and after I just downloaded it for mac. It was taking a crazy amount of time to synchronise which I realised was normal. I sent some $200 to the adress and whilst mucking around with the settings i hit the reset button which at the time I thought would fix the synchronisation issue. Anyway I've searched many .dat files and the receiving address is deleted from my bitcoin core app. Will I still receive my funds? If not, can I recover them?

Did you make a backup of the wallet, or make a note of the private key, before hitting that reset button?

If your node is syncing from scratch, the payment won’t be shown in the wallet until your node gets to that point in the network history. The ‘fix’ is likely to just wait until the sync is complete, and then all will be well. Your funds are at no risk in the meantime, as long as you keep the wallet file / backup safe.

Unless you deleted anything, the Settings Reset button won't change anything except some node settings. It does not effect any transactions in your wallet. It does not effect any of your addresses.

If , at some time in the past, you had changed the directory containing your wallet.dat file, resetting options might cause the wallet to not know where your wallet.dat file was.

Did the same thing as you. In BitCoin Core, made a receive address.. then removed it. Then sent 5k worth of BTC to that removed address. Luckly.... Bitcoin Core has your back. Just don't close the wallet and wait till the transaction confirms, and sync the wallet 100%. It will eventually show incoming transaction and combine them into your main wallet. Search your wallet address you sent the transaction to with blockchain.com, see if its confirmed. Now its just a waiting game. Keeps your hair standing up

This makes no sense; you cannot delete receiving addresses in Bitcoin Core.


text:
Basic question about transaction verification
I am fairly new to crypto and had some questions about how transactions are verified by the nodes
Would it be correct to say that the full nodes check to see if the sender(Alice’s) signature is valid. And they check against double spends by seeing if the hash value that is included (of the transaction where Alice received the bitcoin) has ever been used an another transaction?
My question is, if that is the case, why can’t the verification only check against transactions signed by Alice, why does it have to check against the entire blockchain?
Why can’t it just check against Alice’s other transactions since you know she controls that coin.

How do you know that Bob doesn't also control that coin? I can make a transaction output that anyone can claim if I want to.

My question is, if that is the case, why can’t the verification only check against transactions signed by Alice, why does it have to check against the entire blockchain?

A node has no concept of who controls certain outputs. So there is no way to 'check all of Alice's outputs', how would the node be able to ascertain which outputs belong to her?
Rather, the normal operation of a node is to check all transactions, to ensure that no transaction has spent coins that were previously spent (and to check that new coins aren't being created against the network's rules, etc). In doing this check, the node creates an index of unspent transaction outputs (UTXOs), which it will use to help check validity of new transactions.

A full node knows ownership of all existing bitcoins in the sense that every sane full node keeps a database for the whole UTXO set, however obviously the UTXO set still comes from the blockchain, which contains all transactions.

@ChrisChen right, a node is aware of the lock that is put on each UTXO, but is not aware of who may own the key to that lock, or which locks may be owned by the same user. OP's question seems to imply that a node could 'only consider transactions signed by user X', but of course with that information being un-knowable, the node must actually check every historical transaction for validity, in order to determine the validity of any one specific transaction.

I thought the OP assumed that a full node would intensively read historical blocks during the validation process.

Ah, I understand your comment better now. Made a slight edit so that my answer is less ambiguous in that sense.

I see scantxoutset of Bitcoin Core displaying block height, so that it's indeedly an index, however I would rather think it as the implied "final balance" of the blockchain ledger (which contains "transaction history" only). By the way I think block height can help a wallet tied to a local pruned node to recover the final balance directly without rescanning the blockchain. (although there's issue like unknown child key index) This also seems to help in the recent hardware wallet compatibility issue caused by fixing the BIP143 vulnerability.

To be short:
Without a ledger recording everything, it won't be possible to know whether Alice had ever signed another double-spending transaction. Even if we have a ledger, or even better, a tamper-evident, append-only ledger (which is the inherent nature of blockchain), it's still enough, because the ledger itself could still be maliciously truncated & appended with dishonest new records. Therefore we need a consensus mechanism to allow every participant of the system to know which ledger is valid without a central authority. In Bitcoin, such mechanism is Proof-of-Work, aka PoW or mining.
Note that if a full node doesn't check everything, it would allow much more malicious things including inflation/counterfeit/theft.

why can’t the verification only check against transactions signed by Alice, why does it have to check against the entire blockchain?

Why can’t it just check against Alice’s other transactions since you know she controls that coin.

In reality a full node checks everything in the blockchain, including Alice's ones. Only after checking every transaction in the blockchain can a full node know all Alice's transactions. (in the beginning I didn't notice this point, thanks to @chytrik!)
Once Alice spent some of her bitcoins, any sane & in-sync full node would delete corresponding UTXO entry from its UTXO set database, so that if Alice wants to spend the same coins once again, any sane full node would spot its violation against rules & discard such transaction.

Would it be correct to say that the full nodes check to see if the sender(Alice’s) signature is valid. And they check against double spends by seeing if the hash value that is included (of the transaction where Alice received the bitcoin) has ever been used an another transaction?

Yes, it's correct. In reality it has been done in a much more efficient way without intensively reading the blockchain.
Validation of the blockchain of Bitcoin is progressive in real world, which still matches the "checking against the entire blockchain" criteria in theory.
The Bitcoin blockchain is simply a ledger containing "transaction history" only. The "final/historical balance" is not (yet) committed to the blockchain. Although Satoshi Nakamoto's whitepaper didn't mention these terms, essentially the blockchain itself is the "transaction history", and the UTXO set (which is recalculated from the blockchain by each full node on its own. In other words it's "implied" by the blockchain) is the "final balance of everyone" or "fully-detailed ownership information of all existing bitcoins".
"Ownership" means what it literally means. It's technically the scriptPubkey of each transaction output, which is a script validating digital signature against a public key hash in most cases.
"Existing" means that spending a bitcoin is essentailly deleting corresponding UTXO entry in the database. Oh, in most cases it also creates new UTXO entries, which are the actual payment to the payee and the change sent back to the payer generally.
In reality, the blockchain validation is a progressive process that, the full node just query & update the UTXO set database while reading each block, so that the historical block could be even directly deleted, which is exactly what the currently implemented "blockchain pruning" feature actually does.
Why the historical blocks can be simply discarded? Because, the historical blocks are no longer needed at all in the follow-up validation process, where only the UTXO set needs to be queried & updated. In other words, the "effects" of each transaction in each block is actually reflected on the UTXO set already.
Generally, a full node downloads & verifies the whole blockchain from the genesis block (oh, the genesis block itself is hard-coded), so that even if the blockchain pruning feature is enabled (so that all old block files are simply deleted), it still matches the "checking against the entire blockchain" criteria in theory.
(However there's still a detail that the assumevalid feature is enabled by default, so that a full node won't actually check everything of the blockchain. assumevalid is basically a block hash hard-coded by the developers, which means at least the developers themselves have fully validated all previous blocks before it, let alone thousands of other running full nodes all around the world. If you want the full node software to do real full validation that all historical blocks would be still fully validated, you need to turn off this feature)

However, preventing double-spending is not as simple as checking everything within a (individual "forked branch" of) blockchain. Even if a full node checks everything in the blockchain, Alice could still make up a "forked branck" of the blockchain ledger where she sent her bitcoins to herself, instead of the original payee, if she wants to double-spend - that's exactly why Bitcoin needs a consensus mechanism, which is Proof-of-Work, aka PoW or mining.
(PoW or consensus mechanism is actually not a simple thing like that. For example, an attacker can DoS the network by making infinite forks as well)
Anyone can be a miner, however any block needs some work (mining, which consumes electricity) accumulated on it to be accepted by other participants (full node or SPV) of the network, so that making up lies (invalid blocks) would inevitably have real costs. Making valid blocks also has costs, however this would be "compensated" by the rewards, including newly-issued bitcoins and transaction fees.
Double-spending (forked branch of) blockchain is (inevitably) valid against the rules as well (not like the case that an invalid blockchain always violates the rules), however being "valid" doesn't mean being "winning".
A valid blockchain has to accumulate the most work at same time to be the winning one, otherwise it still won't be accepted by a full node.
That's the well-known "the longest chain rule" (which was actually a tiny but serious mistake corrected by Satoshi himself, to be then changed to "the most accumulated work rule"). Together with other validation rules enforced by every full node, the ruleset incentivises miners to keep honest (only making valid blocks at the chain tip to extend the valid & winning chain even longer).
(Note that SPV client cannot validate the blockchain on its own, therefore it would simply follow the chain with the most accumulated work blindly, however it can still notice whether more than one forks are present)
A malicious miner doing the double-spending faces the risk that his forked chain could be eventually rejected by the network if he won't be able to keep his forked chain to accumulate the most work (among all forked branches, including the "honest" one) on it.
According to the calculations in Satoshi's whitepaper, an attacker (including both miners as an attackers themselves and "dishonest" miners in collusion with the attacker) with more than 50% of hashpower theoretically has 100% probability to be succeed, if he can maintain this for a long enough time. If the attacker doesn't have more than 50% of hashpower, he would inevitably face exponentially dropping success probability as the confirmation number grows.

Simply, the block chain is to prevent double spending. Otherwise Alice could present different transactions to different people, all with valid signatures, that spend the same money in alternate ways. With a ledger containing every transaction that has been executed, we know that the money can be spent exactly once.

The blockchain itself as a tamper-evident, append-only ledger cannot prevent double spending, unless the "blockchain" means not only a ledger, but a whole system including a consensus mechanism like PoW.


text:
College Economics Student in Support of Crypto Looking for Live Hashrate Data
I am a college economics student doing a Thesis project in support of the long term viability of Crypto. Does anyone know where I can get live Hashrate data for free or for sale? I appreciate it - thank you!

You can get a current estimate of this info by running the bitcoin-core full node software and calling the command:
bitcoin-cli getnetworkhashps


text:
wallet migration tool and password protected wallet
Surely the Wallet migration tool by achow will port unencrypted legacy wallets to unencrypted descriptor wallets, but what will be the output for encrypted legacy wallets? unencrypted descriptor wallet (less-is-more), or encrypted descriptor wallet (funds-are-safu)?

The migrated wallet will be encrypted with the same passphrase.


text:
restrict bitcoin core to tor
I run both bitcoind and bitcoin-qt calling with
-proxy=127.0.0.1:9050
Is this enough for Bitcoin Core to route any communication through Tor, or does this risk any name resolution or something being queried directly instead of channeled in Tor?


text:
What's the diffrence between these two bitcoin.conf format?
server=1
listen=1
daemon=1
rpcuser=X
rpcpassword=Q
rpcallowip=localhost
rpcport=Y

And
server=1
rpcuser=X
rpcpassword=Q
rpcallowip=localhost
rpcport=Y

I am a litlle confused, can these two conf file accepts solomining? And what's the difference between them?
Thank you for help

Answering the second part of your question:
server=1 tells Bitcoin-Qt and bitcoind to accept JSON-RPC commands, so you can query it
listen=1 is a bitcoin listening mode
daemon=1 starts bitcoind in the background as a daemon
rpcuser=X and rpcpassword=Q is the user/password for rpc access
rpcallowip=localhost means you can access it through rpc only from your localhost IP machine, add external IP if you want to access it from outside
rpcport=Y is the rpc port to access it (localhost:Y), for bitcoind the default is 8332
AFAIK the second config won't work for rpc, you haven't specified rpcuser, and you will see the bitcoind output in the console (not a daemon).

Thank you Tony, But about listen=1 is a bitcoin listening mode litening to what? To the rpc call?

It means, that your node accepts an incoming connection and uploads old blocks to other peers (when they are in the initial block download phase and later when propagation happens). Listening to rpc calls is defined by server=1.


text:
How to get the list of blockheader
I know that i can use getheader 'hash' from bitcoin core, but i can't use it for all blocks, it's time consuming.
Is there a site where i can find all bitcoin blockheader? Like a blockchain site but shows the blockheader?
Help is appreciated, thank you

Do you want the whole header in e.g. hex format, or do you want each field separated, e.g. version = xxx, prevHash = xxx, ...?

Hex formatted if it's possible

https://bitcoin.stackexchange.com/a/123438/34190

You can take advantage of Blockchain.com's API (formerly Blockchain.info).
You can access BTC blocks based on their block height via a URL of the form https://blockchain.info/rawblock/<height+1>. We use <height+1> because 1 is used as the index of the genesis block, which is instead usually referred to as block 0, as it is defined to have a height of 0.
By default, this endpoint provides a JSON dump of the data in the block, but you can get the raw block data as a hexadecimal string instead by appending the query string ?format=hex. For example, https://blockchain.info/rawblock/2?format=hex will give you the data in the block after the genesis block (the block with height 1) as a hexadecimal string.
The block header is 80 bytes, or 160 nibbles (hexadecimal characters), so you can e.g. use curl to fetch the data, and then head to truncate it to just the header, and then save that hex string to a file:
curl 'https://blockchain.info/rawblock/2?format=hex' | head -c 160 > block-1.txt

You can use a simple Bash script to automate this for all blocks:
#!/bin/bash

latest_block_height=670288

for ((i=0; i <= latest_block_height; i++)); do
    if ! [ -f "block-$i.txt" ]; then
        curl 'https://blockchain.info/rawblock/'"$(($i + 1))"'?format=hex' | head -c 160 > "block-$i.txt"
    fi
    echo "Downloaded block $i."
done


text:
Are miner builder/sellers profitability calculations true?
regarding to this question here How many hash calculation is needed to reach one BTC? , it takes about 1387 years with 1 TH/s power rate to reach 1 BTC.
So if someone has a machine with 53TH/s power rate and his machine works for 24/7, he can reach 1 BTC in 1387 / 53 = 26.1 years, and if he has 100 of that machine, he can gain 1 BTC in 95.5 days, or nearly 4 BTC per year.
So if the price of 1 BTC is about $9500 today, he earns 9500 * 4 = $38,000 per year(let's think the electricity consumption is free!). But in many websites like this https://www.asicminervalue.com/miners/bitmain/antminer-s17-pro-53th, it calculated the profitability of one same machine of $4,333 per year and it will be $433,000 per year for 100 machines!(It also considered the electricity power consumption).
How is it possible?

It is not 1 BTC but the entire block rewards (12.5 BTC + tx fees). Assuming just the block subsidy that is $38K*12.5 = $475K, which is roughly in the ballpark of the number quoted.

@UgamKamat: I think you can make your comment as an answer. I will accept it.


text:
Transaction in future
Is there any plan to add a feature so that you can perform transactions that will be spent in the future and stored on the network? For example, I want to send some bitcoin to my kids after each N block so they will slowly receive a long time after I may already be dead. But I don’t want to send them the whole balance at once.

About the phrasing, there is really no "plan" on Bitcoin as there would be if a centralised authority governed the protocol. Instead, there are proposals and sometimes supermajority consensus for the network to deploy a proposal.
Regarding your question you can already do this today using the locktime features of Bitcoin transaction.
You can either:

Hand your child a transaction with a nLockTime set to block X (so that the transaction cannot be confirmed before the block number X).
Create a transaction which can be confirmed but cannot be spent before the block number X (CHECKLOCKTIMEVERIFY).
Create a transaction which can be confirmed but cannot be spent before N blocks are mined after your transaction (CHECKSEQUENCEVERIFY, relative locktime).

In either way (I omitted an even more involved one purposefully), you (and your child) definitely need a specific software to create and recognize this type of transaction.
Learn more (and get more links) on the Bitcoin wiki page about locktimes.

without a process running over time that create these transactions, these methods wouldn't satisfy the condition "But I don’t want to send them the whole balance at once."
So if he wants to send X BTC every year, if he generates these transaction now planning to distribute every year he need to spend everything now, that's not what he's asking for (if I've understood correctly the request).

@NotImportant while you are right, I gave the general idea. Of course they can use 10 transactions with 10 different CSVs. I also think they were asking about if it was "doable on Bitcoin" rather than "how do I do with my grand-child now" as I don't expect them to code a wallet for this usecase.

It is possible to do that right now, but you would need a server running your Bitcoin wallet (with RPC daemon) + a custom script.
The script will see if it is time for the range of blocks you wish and then will send the Bitcoins to the addresses of your kids.
You could use a long term payment method to keep this server running with some processes to make everything running again if there is some internal issue in the server.


text:
Why in old blocks I can't see the miner address using RPC API calls?
So, I'm using Bitcoin Core RPC client version v22.0.0 and Bitcoin Core version v22.0.0.
To get the block with height 1, I call the API like this:
bitcoin-cli -datadir=/Users/dennis/Bitcoin getblock \
00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048 2

You see this block only contains the coinbase transaction. If you check the vout section you won't see any address.
"vin": [
  {
    "coinbase": "04ffff001d0104",
    "sequence": 4294967295
  }
],
"vout": [
  {
    "value": 50.00000000,
    "n": 0,
    "scriptPubKey": {
      "asm": "0496b538e853519c726a2c91e61ec11600ae1390813a627c66fb8be7947be63c52da7589379515d4e0a604f8141781e62294721166bf621e73a82cbf2342c858ee OP_CHECKSIG",
      "hex": "410496b538e853519c726a2c91e61ec11600ae1390813a627c66fb8be7947be63c52da7589379515d4e0a604f8141781e62294721166bf621e73a82cbf2342c858eeac",
      "type": "pubkey"
    }
  }
]

However, If you make the same call but for a newer block, for example with height 700,000:
bitcoin-cli -datadir=/Users/dennis/Bitcoin getblock \
0000000000000000000590fc0f3eba193a278534220b2b37e9849e1a770ca959 2

You will see the address of the coinbase transaction:
"vin": [
  {
    "coinbase": "0360ae0a2cfabe6d6d078b52bc69cf0982356d3eb439efe5489aee2c74da629dcc1a8b50a309cc0800100000000212030400000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005008a1c0000",
    "txinwitness": [
      "0000000000000000000000000000000000000000000000000000000000000000"
    ],
    "sequence": 0
  }
],
"vout": [
  {
    "value": 6.40388640,
    "n": 0,
    "scriptPubKey": {
      "asm": "0 f3b2d6544de521b5c617fc83ca6d13854f29a8a5",
      "hex": "0014f3b2d6544de521b5c617fc83ca6d13854f29a8a5",
      "address": "bc1q7wedv4zdu5smt3shljpu5mgns48jn299mukymc",
      "type": "witness_v0_keyhash"
    }
  },

Questions:

It is something related to bitcoin-cli version?
If so, It's posible to get the coinbase address for older blocks building a parser and directly parsing blk000.dat files?
If it's not possible, how blockchain.com shows the coinbase address for all blocks?

If you look at this block in a blockchain explorer such as blockchair, it shows the transaction as
{
    "txid": "0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098",
    "hash": "0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098",
    "version": 1,
    "size": 134,
    "vsize": 134,
    "weight": 536,
    "locktime": 0,
    "vin": [
        {
            "coinbase": "04ffff001d0104",
            "sequence": 4294967295
        }
    ],
    "vout": [
        {
            "value": 50,
            "n": 0,
            "scriptPubKey": {
                "asm": "0496b538e853519c726a2c91e61ec11600ae1390813a627c66fb8be7947be63c52da7589379515d4e0a604f8141781e62294721166bf621e73a82cbf2342c858ee OP_CHECKSIG",
                "hex": "410496b538e853519c726a2c91e61ec11600ae1390813a627c66fb8be7947be63c52da7589379515d4e0a604f8141781e62294721166bf621e73a82cbf2342c858eeac",
                "reqSigs": 1,
                "type": "pubkey",
                "addresses": [
                    "12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX"
                ]
            }
        }
    ]
}

Blockchair shows an address, but Bitcoin Core doesn't.
Note that "Bitcoin address" is an invention that I believe was intended to make it easier for recipients to provide senders with information needed by the sender to construct a transaction script.
Transaction data doesn't ever actually include an address.
Bitcoin core doesn't present an address, probably because they weren't in use at the time P2PK transactions were used. I guess addresses came into use with P2PKH transactions.
Note that many blockchain explorers have their own way of dreiving addresses from more exotic forms of payment such as multisignature payments. Not all software agrees about whether such transaction types can be represented in the form of an address.

This is actually a recent change. Bitcoin Core for a long time used to show P2PKH addresses for P2PK outputs, because historically, before different address types existed, P2PKH were also used as a "key id" in addition to the "shorthand for specific scriptPubKey". This is not considered a historic mistake, and fixed in recent versions.

Now I have a better idea about it. However, it's still not clear for me if I built my own parser to read the raw data from blk.dat files, could I derive the addresses of  these transactions? I mean, is this data actually in the blockchain or not?

@Dennis, if Blockchair's software can do it, so can yours. The data is in the scripts in the transaction outputs, or for input addresses, the data is in the outputs of earlier transactions referred to by the vin txid and offset. You'll need to work out how to categorise the transaction output type and work out the address algorithm for each type.

The coinbase output (the destination of the mining reward) is freely decided by the miner.
In early block, this destination was Pay to Pubkey(P2PK). Since P2PK is not assigned an address, no address is shown.
The coinbase transaction in the 700,000 height shows the address because the destination is P2WPKH.


text:
loadwallet takes too much and times out
Hardware is too old and wallet was too much unsynchronized.
Loading the wallet took so much time, it timed out (pending missing console output).

In bitcoind side, that's perfectly normal and safe operation.
Timeout happens only client side.
For a more elegant client (bitcoin-cli) operation, can use:
-rpcclienttimeout=0.

Timeout in seconds during HTTP requests, or 0 for no timeout (default: 900)


text:
Did Segwit (BIP141) eliminate all txid malleability issues listed in BIP62?

There are 9 malleability issues listed in the BIP62, but doesn't seem like BIP141 addressed all of them. If not, why not and why is it not an issue for Lightning or other DoS vectors?
What is exactly meant by the word "Nonintentional" in motivation point #1 of BIP141? Does it imply there can still be "intentional" ways to malleate txid?
What was the exact difference in PR6769 and Segwit in regards to preventing txid malleability because of ECDSA signatures? Miners could still collude and mine a different txid after this PR, which was fully eliminated by Segwit?

Most of the malleability issues listed in BIP62 stem from the fact that an input script (scriptSig) can be modified and still remain valid for the input, and because the input script is included when calculating the TXID, this modifies the TXID as well.
Example ways you can turn an input script into a different but still valid input script:

Any valid ECDSA signature can be turned into a different valid signature by inverting the s-value. BIP62 proposed that an s-value in the lower half of the range be required.
Any push operation can be expressed multiple ways. For example, pushing the byte 01 can be done with the opcode 0x51 (OP_1/OP_TRUE), but also less efficiently using the sequence 0101 (OP_PUSHBYTES_1 01). BIP62 proposed that the most efficient ("canonical") way be required.
Since the input script is a script, it can be modified any number of ways, for example by including <data> OP_DROP anywhere in the script. BIP62 proposed to restrict input scripts to data pushing opcodes.

SegWit cleverly fixed all of these malleability vectors by simply requiring the input script to be empty (or, in the case of wrapped SegWit, to push a single specific byte vector), and moving signatures and other script inputs to the witness which isn't covered by the TXID. Witness data can still be malleated by third parties, but it doesn't affect TXID malleability anymore.
"Nonintentional malleability" just refers to malleability by a third party that the original creator of the transaction didn't intend. It will of course always be possible for the creator to intentionally malleate the transaction themselves before it is included in a block, and even with SegWit you can still create transactions intentionally malleable by third parties by e.g. not requiring any signatures to spend an output.
The pull request you mention changed standardness rules, not consensus rules, so it's easily bypassed by miners. Because SegWit took a different approach to fixing malleability, high s-value signatures are still just non-standard but valid by consensus.


text:
Best script/pseudocode to convert xpriv to xpub?
I'm looking for a script/pseudocode/utility (preferably, code in javascript/c++ or a secure, verified and open-source rpc client) that can convert a given xpriv to xpub.
Only one I've found (but not tried yet) is this one  :
https://github.com/sarchar/brainwallet.github.com/tree/bip32
or I know electrum wallet client can do it. I would prefer to read through the code of a standalone script though and/or create my own script

If you are comfortable with using the command line then libbitcoin-explorer is a great utility for converting your bitcoin HD keys. It even works completely offline.
Once you have bx installed, you can run the hd-to-public command like so:
$ bx hd-to-public xprv9s21ZrQH143K3QTDL4LXw2F7HEK3wJUD2nW2nRk4stbPy6cq3jPPqjiChkVvvNKmPGJxWUtg6LnF5kejMRNNU3TGtRBeJgk33yuGBxrMPHi
=> xpub661MyMwAqRbcFtXgS5sYJABqqG9YLmC4Q1Rdap9gSE8NqtwybGhePY2gZ29ESFjqJoCu1Rupje8YtGqsefD265TMg7usUDFdp6W1EGMcet8

More info about the other HD Key related commands for libbitcoin-explorer can be found here: https://github.com/libbitcoin/libbitcoin-explorer/wiki/bx-hd-to-public


text:
What is a good approach to learning the Bitcoin codebase?
I've read Mastering Bitcoin and other high level resources and I believe I have a decent understanding of Bitcoin. I'd now like to read and understand the source code. However, I'm very new to cpp and don't know a good way to approach this task. Is there a systematic approach I can use to effectively learn the codebase?

Some good places to start are the places that the user or the network input something into Bitcoin Core. The obvious two places for these are the RPC functions and the network message processing. The RPC code is in src/rpc, src/wallet/rpcwallet.cpp and src/rpcdump.cpp. The network message processing is in src/net_processing.cpp. By tracing through the function calls and also looking up where the global variables are being added to and set, you can get into most parts of the codebase.
For example, suppose you looked at sendtoaddress. You would start at the RPC function in src/wallet/rpcwallet.cpp and trace the execution path. You would see that it eventually goes to src/wallet/wallet.cpp and how it uses the CreateTransaction function. From here, you would learn how coin selection works.
Now you would see how the wallet chooses coins for transaction spending. But how does it find out about those in the first place? Well, as you are tracing sendtoaddress, you will come across the AvailableCoins which iterates through some variable named mapWallet and pulls from stuff in mapWallet to build a list of selectable coins. Thus mapWallet must be where transactions are stored, so you can search for the variable mapWallet in the codebase. This will tell you where that variable is being used and you can trace through the uses of mapWallet to learn how it is populated and used.
Doing so will lead you to the functions that populate mapWallet. So now you want to trace backwards: what are calling those functions, how did we get to that function. For example, for populating mapWallet, you will run into AddToWallet. What calls AddToWallet? The function AddToWalletIfInvolvingMe. And you can keep going backwards to get an idea of how it all works.
The same methodology can be applied to the network message processing. For example, suppose you wanted to learn about block processing. The place to start is when a block is received, so in src/net_processing.cpp, look for NetMsgType::BLOCK within the ProcessMessage function. From here you can trace the function calls and global variables like we did with the RPCs. It will take you to the Proof of Work validation and how the target is calculated for a block. It will take you to transaction validation. You will end up looking at the script interpreter and how it evaluates scripts in a transaction. And so on.
You can repeat this for every RPC and every network message type to find out where and how things are processed. This will get you through the majority of the codebase. Network message processing will get you through all of consensus and the bulk of the codebase. Tracing RPCs will get you through basically everything else because almost everything a user can do can be done through the RPCs. These two general entry points are great places to go through the codebase and learn about what Bitcoin  Core is doing internally.

I'm not sure about "learning the codebase", but I can point you to some files which are in my opinion good starting points for explorations:

The init.cpp: "what happens when I start my bitcoind ?"
The rpc calls in src/rpc or src/wallet/rpcwallet.
For example, "what happens when I sendtoaddress ?"

In addition there are also other parts that might be less good starting point, but nonetheless very interesting. Such as the script interpreter or the network messages process.

Overall that's non exhaustive and opinionated. There are people regularly answering here who know the codebase much more than I do, so there might be other great starting points that I didn't mention.


text:
Why is it important for individuals to be able to run their own nodes?
I understand that the more people that can run their own node supports decentralization, but if miners are sufficiently decentralized what is the threat of users not being able to run their own nodes?

Every time you check your wallet, your wallet is displaying information that it fetched from a bitcoin node (usually with other software running on top of it, like ElectrumX). If you absolutely trust the server behind your wallet, then as an end-user, it's not necessary to run your node. However, you might still want to run it, because you care about bitcoin and want to help new users/companies download the blockchain, make the network more resistant to DDoS attacks or, if you live in an authoritarian nation like North Korea, you might want to make it harder for the government to censor bitcoin.
Running a node is especially important if you run a bitcoin-related company. In this situation, you shouldn't be relying on an external service as you don't know if it will be down, hacked or have bad intentions.
Also, if you receive a large payment in bitcoin, you want to make sure you received it. You can check seven different bitcoin explorers but, depending on your level of paranoia, you might want to check your own node and verify that it has at least two to six confirmations.
Check out Andreas's video about this.

There is a popular saying in Bitcoin: Don't Trust. Verify.
Even in a world where miners are perfectly benevolent and can be trusted to maintain the network's rules (highly unlikely, but lets just pretend), the user still gains benefit from running their own node:
Sovereignty:
By running a node, the user can interact with the Bitcoin network in a completely sovereign fashion. As long as they are running the correct code, the user can verify the state of the network themselves, without having to trust any third party.
Privacy:
By running a full node, the user will not have to query any other nodes directly to determine their wallet balance. Analysis of the blockchain is profitable business for several companies, so the prudent user will take steps to interact with the network in privacy-preserving ways, so that their financial data will not be compromised by these analysis firms and sold to the highest bidder. Running the node over the Tor network can help increase user privacy greatly.
It is worth noting that running a full node does not guarantee privacy, there are additional considerations here which are outside the scope of this question. But in general, running a node will be a net privacy gain in pretty much every case.

The points above are very important: by returning sovereignty to the user, we have also removed the potential for abuse from third parties. The user is in full control of their funds, and cannot be censored or restricted in access to them.

At the network level, it is full nodes that enforce the rules of the network, so by having a robust network of nodes run by individuals, we see that the network becomes resilient to change (which is a good thing, if you want the rules of your money to remain constant).
If you do not run a full node, you must assign varying levels of trust to third parties, in order to learn your balance, send transactions, or uphold the rules of the network.


text:
Can you run a full Bitcoin node on Node.js?
Are there bitcoin full node implementations running on Node.js?
Node.js is the server side language that I'm most familiar with so I was wondering if I can run a full bitcoin node on Node.js?
I just want to learn more about how a node works and possibly experiment writing my own cryptocurrency implementation in Node.js as that's what I'm most comfortable with.

Yes, bcoin:
http://bcoin.io/
https://github.com/bcoin-org/bcoin
bcoin is a modular Bitcoin library for nodejs with full node and SPV node functionality, as well as a scalable BIP44 wallet used in production by several Bitcoin businesses including Purse.io.
Bcoin has the second-fastest initial sync time out of all available full node implementations (after Bitcoin Core, of course): https://blog.lopp.net/bitcoin-node-performance-sync-tests/

You could, but this is very ill-advised to do so.
Bitcoin consensus rules being implementation-defined rather than specification-defined re-implementation are unfortunately insecure.
Thanks to libbitcoinconsensus, it has been made safer to build transactions with external tool. There might be a nodejs implementing transaction building logic out there, but you really should not validate blocks with a nodejs (or any language for that matter) re-implementation of Bitcoin.
Regarding your desire to learn more about the inner working of a node you can learn from external Bitcoin tool in your favorite language but at the end of the day if you want to learn how the protocol is really defined, you'll have to dig into the original implementation code.

Yes, i was able to find some tutorials on creating bitcoin addresses and signing transactions in Node.js, creating raw transactions and then feeding them to the network.
I thought that bitcoin peers just interact on specific ports with something like websockets but any node can choose their own implementation and how they interact with the network. And that there can be implementations in different languages.
I guess I have to delve deeper into the Bitcoin protocol.

I see that the original implementation is mostly in C++ (which I don't know any of it unfortunately). Is there any specific reason for it to be in C++? What if I want to create a new cryptocurrency different from bitcoin with nodes running in Node.js? Is that a bad idea, running on Node.js and why? Perhaps mining will be slower in Node :)

Regarding the reason behind C++ i'm afraid you'd need to ask Satoshi :-). Regarding why is re-implement Bitcoin is a bad idea, well, see the answer above. Regarding a new copy of Bitcoin, I think it would be interesting that you post a new question like "why is it a bad idea to create my new '''cryptocurrency''' ? " I and many others would be thrived to answer as that comes up really often !


text:
How can a user or wallet have multiple addresses at all if out of a private key can be generated only a single public key?
A user of a wallet, regardelss of a version of a wallet, can generate as many addresses as he wants, right?
Since an address is basically a public key, and in "private-public key" cryptograpthy there's only one public key that corresponds to a private key, how is generating lots of addresses possible at all?
This Can same private key generate multiple addresses? isn't what I'm asking about.

Well, it's very simple. Traditional wallets just generate multiple private keys.
A private key is basically a randomly generated number. You can generate several of those, and store them all in a file.
This poses a challenge with backups, because you have to keep saving new backups - and even if you do, as you continue using the wallet, your coins could be in an address for which the key is not found in your most recent backup.
The solution - which works well in most cases - is to simply generate several (by default 100 I think) keys in advance, and store them in the backup as well. As you use more addresses, keys are chosen out of the pre-generated pool, and more keys are generated in advance to replenish it. This way, a backup can still be relevant long after it was created.

but I've heard many times that "a wallet is basically your private key (or private/public pair)" which implies "one"

The word "basically" hints at the fact that this is a simplification. If "wallet" literally meant "private key", people would have just said "private key" instead of "wallet".

In general, there's not really a universal definition for "wallet", it can mean different things in different contexts. If the word is confusing, we should try to avoid using it. The Bitcoin-qt software generated multiple private keys, each with their associated address, and stored them in a file. Whether you use the word "wallet" to refer to the software, the address, the file, something else, or some combination of the above, doesn't change the essence of what's actually happening.

Today, Bitcoin Core and many other wallets use HD wallets to ensure that backups remain valid indefinitely to retrieve funds. Regular backups are still important as otherwise the meta information about transactions would be lost.

@Murch: Please read this comment by the OP - https://bitcoin.stackexchange.com/questions/98502/how-can-a-user-or-wallet-have-multiple-addresses-at-all-if-out-of-a-private-key/98507?noredirect=1#comment111576_98503. And also the specification in the question that the OP is not asking about "this", where "this" is a question and answer regarding HD wallets.

Although I see that the OP is not specifically asking about modern wallets, I think it is still misleading to declare keypools to be "the solution", since essentially nothing uses this approach anymore as far as I know.

@GlennWillen: 1. My answer starts with the words "traditional wallets". The solution is the solution to a challenge in traditional wallets. Regardless of the fact that it is now mostly of historic significance. 2. The OP is not just "not specifically asking about modern wallets", they are specifically not asking about modern wallets, as per their comment, which I took into consideration while writing this answer.

Right, and I had read your answer first, since it was on the top, and from the question and your answer it was not clear to me why you didn't mention derivation. Later, I saw Raghav's answer (being on my phone). Either way, even if the asker was asking about "non-modern" wallets, it would make sense to me to mention the modem approach for other readers as a counterpoint.

Modern wallets are hierarchical deterministic (HD) wallets.
A root key is used to derive an effectively infinite number of private keys using systems such as BIP32.
Each derivation results in a single public/private keypair, and hence a single address at that level.
Prior to the usage of HD wallets, things were a bit more straighforward - every time you requested a new address from the wallet, it would generate a new, random private key and store it in the wallet file, and give you the address created by the public key for the new private key.
The move to HD wallets was done for ergonomic and safety reasons - in older wallets, if you did not make a backup after every new address, you would lose any addresses generated after your most recent backup if the wallet file was lost. With HD wallets, only the root key (commonly a BIP39 mnemonic) must be backed up - all addresses and their keys can be recovered from it.

You haven't read it carefully -- I'm not asking about modern wallets

Not-modern wallets did the same thing, just without a single root key - For each address in a non-deterministic wallet, the wallet software would generate a new, random private key.

An important point you've left out (regarding traditional wallets) is that several keys are actually generated in advance, so that backups remain relevant for some time after they are created. The wallet would be unusable if you actually had to create a new backup everytime you send or receive money.

@MeniRosenfeld True, but I figured it was best to give the overall approach rather than specifics - not all non-HD wallets had keypools, iirc, although Bitcoin Core and other commonly used ones generally did

@RaghavSood: I think this point is a key part of what the OP was asking, even if they didn't necessarily realize it. I don't think "generating multiple keys" is a huge leap - the interesting question is is how can we actually use the wallet when it keeps generating new keys. And pre-generating keys is the missing piece of this puzzle.

Oh, I see that the discussion had already been had. -- you're both right, in that both parts are necessary to fully understand what's going on. :)


text:
tracking invoice address type migration
I'd love to see charts with stats on percent of utxos by invoice address type (specially P2PKH, P2SH, P2SH-P2WPKH, P2WPKH, P2TR), over time; and same for percent of coin supply.
No doubt I can pay a company to access these numbers. Is there any alternative free as in free beer? I could pay my own time working my way through some open source software able to do this.

While I think Pieter's answer provides exactly what you were asking for with the UTXO set repartition by output type, I think that you may also be interested in the Output Types by Count chart on transactionfee.info:

The site has also more related charts that may be interesting in the context such as

Input Types by Count
Output Types by Value

and various charts for specific output types.

You can find the first on https://txstats.com/dashboard/db/utxo-set-repartition-by-output-type?orgId=1. For a version weighed by value you can look at the graphs of the individual output types on the same site.


text:
Lightning Network state deprecation secret keeping
When using a lightning channel, two sides are sending transactions representing the current state (or balance) between them, and for each state sent, they invalidate the previous state by sending a secret for a hash puzzle in a HTLC transaction.
For example, when Alice sends Bob the new state between them, it also sends a secret for the puzzle in the previous state. Bob can use it to redeem Bitcoin if Alice publishes the previous state transaction on the blockchain.
My question is - does Bob need to save all the secrets of all the previous states until the channel is settled and closed? Sounds like Bob might need to save a lot of data when having many channels.

Unless bob uses a watchtower service where he outsources the storage of data Bob needs to know all previous revocation secrets. However he does not need to store and save all of them as the revocation secretes are derived in a deterministic way. This means whenever I know a revocation secret of a current state I will be able to derive all previous revocation secrets from it.
You can find the details of the key derivation in the key's section of BOLT 03
That being said revocation secrets are not the same as preimages of HTLCs (which I believe your question implies / confuses) The old preimages of settled HTLCs do never need to be stored or need to be reproduced. This is because in the penality transaction case from the second stage HTLCs the revocation secret is sufficient to access the funds before the timelock.

Thanks! But if revocation secrets aren't verified like a HTLC preimage - how are they verified? Are the transactions publish the public key and when the secret is published it is checked as the corresponding private key? Which Bitcoin opcode supports this kind of verfication?

When a new commitment tx is negotiated and the old revocation secret is revealed one can check if it can be used to produce a valid signature. Nodes must do this before sending a new commitment signed message


text:
Automated decentralized exchange between bitcoin and similar currency?
I want to create a cryptocurrency that's backed by bitcoin, that's running on it's own different blockchain and consensus algorithm.
Is it possible have a decentralized exchange between bitcoin and the new currency so that the only way to get the new coin is to exchange your bitcoin and if you want you can get your bitcoin back by returning your new coin?
And this to be done in a safe way without a central exchange and ANYONE having access to all of the new coin, and without risk for your bitcoin as long as both the bitcoin and new crypto blockchains are both alive and by nodes running the blockchain.
Here's an example:

You send 1 btc to a bitcoin address.
When that's confirmed let's say with 6 confirmations the "decentralized system" sends you 1 coin of the new coin to your "new coin" address of choice.

And if you want you can trade your new coins back for bitcoin.
So the total supply of the new coin = the supply of bitcoin
And no one has access to the total supply of the new coin, the only way to get the new coin is to deposit bitcoin. And you can get your bitcoin back automated and decentralised whenever you want by returning your "new coin" to the system.
Is this possible?
I know about atomic swaps but that relies on 2 parties exchanging coins. What if 1 of the parties is the new blockchain itself? Is this even possible?

No, this is impossible (so far as we know). But the reason is very, very subtle.
The crux of a decentralized system is that nobody gets to decide what the rules are. Otherwise, whoever gets to decide the rules effectively centralizes the system.
Bitcoin is decentralized because anyone who wants to can use whatever rules they want. Those who agree form a useful blockchain.
The problem with this scheme is that there has to be some mechanism to decide what controls the real bitcoins that back the system. For example, bitcoin and bitcoin cash forked off and nobody got to decide which chain was "real". Both chains now compete in the market.
If your system forks, who or what decides which side of the fork gets the actual bitcoins that back your currency? They can't both get all the bitcoins -- that would require the number of bitcoins backing your currency to double.
There is no known decentralized way to make such a choice.
Think it through -- what do you think should happen if two groups of people using your currency disagree over what the system's rules should be? They can't each get what they want because they are arguing over how a fixed amount of bitcoin should be used.

Thank you! Yeah, that makes a lot of sense. I guess (as far as we know now) the only way to this is with a level of centralisation. Let's say I create my own cryptocurrency with 21 million supply and set up an exchange where I sell 1 "new coin" for 1 BTC so this way each coin is backed by BTC but then that will be a huge level of centralisation and also people have to trust that I'm not just going to run away with the BTC lol so not sure if people will participate in my new cryptocurrency. It can still have independent nodes validating transactions but will be centralized cryptocurrency.

I'm a beginner to all this, but cryptocurrency fascinates me :) Thank you for your reply! Much appreciated. I'll mark the answer as accepted.

I wish there was a way to do it where no one individual has to control the backing BTC and people can still withdraw it back by selling their new coin with some kind of a super smart contract. :)

It doesn't have to be one individual. But whatever it is, that thing will control the backing and thus which side of the fork is real. Since a fork can change anything, one who can choose which side of a fork is real has ultimate power.

David, I got another idea :)
What if no one gets the backing bitcoin and instead this bitcoin is wasted to a random bitcoin address like "11111111111111111111..." that no one has the private key to.
Let's say that I subscribe to the idea that any bitcoin wasted to that address creates the same amount of coin in the new coin to the address that wasted this bitcoin.
The process is irreversible but now this new coin has value because you wasted real value to create it. And I believe that idea, and anyone who choses to believe that, too will participate in the new coin.

So that's 1 way to create supply from the new currency that's verifiable and all people participating in the new coin agree and believe that. People can chose not to, or believe in something else and that will form the new fork? The new coin would use the same address format as bitcoin so that people who have the private/pub key for bitcoin also have the same in the new coin.

This way, the only way to obtain from the new coin is to either waste bitcoin (which people can verify if you've actually done that) or to have someone else send it to you.

@Ivelin You can certainly do that. The problem will be that if the value of the coin is not equal to or greater than the value of the bitcoin you need to burn, nobody will create any. Realize that in no sense will this result in your currency being backed by anything.

Yeah, that makes sense. People will probably be able to buy for cheaper :) than wasting 1 bitcoin to create 1 new coin. It will still create some value for it because I will not want to sell my new coin for much lower than what I burnt to create it :) but it will not be equal value or backed. Makes sense. At least it's a way to create a new coin that's different from mining it :) or pre-mining it and distributing it in some other way. Still have problems to solve :)

Yes, it is possible with Noel Tiernan’s atomic exchange protocol

Step 1
A generates H(x)
A sends coins to <B's public key> and x for H(x)

Step 2
B knows that A has sent his coins, so
B sends coins to <A's public key> and x for H(x)

Step 3
A sends x to B.

The can now both spend the coins they received.

If A refuses to perform step 3, when A spends the coins B sent, he has to provide x to the chain.  At that point B can spend his coins.


text:
How come the bitcoin blockchain is only 285gb only?
I found some statistics that the bitcoin blockchain is 285gb.
https://www.statista.com/statistics/647523/worldwide-bitcoin-blockchain-size/
Bitcoin has been around for how long? 10 years?
with 1mb blocks every 10 mins.
6mb an hour * 24 = 144mb a day * 365 = 52,5 gb a year * 10 = 520gb
How come it's significantly smaller?
Are my calculations wrong?

Any factor limiting the size of a block is a maximum, there’s no minimum requirement.

Ah I see, so in the beginning years blocks weren't full with transactions and were less than 1mb? Because nowadays I don't think any blocks aren't full as there are so many pending transactions in the mempool all the time right?

These days almost all blocks are full, but full means 4000000 weight units (as not every byte is counted equally since segwit). Typical full blocks are between 1.2 and 1.5 MB in size on disk.

There is a minimum size though. The block should have at least 1 transaction (coinbase) which itself has a minimum size (2 byte signature script, at least 1 output).

@CodingEnthusiast Good point. I believe the minimum is 145 bytes (80 bytes header, 1 byte tx count, 1 coinbase transaction consisting of 4 bytes version, 1 byte txin count, 36 bytes prevout, 1 byte scriptsig length, 4 byte scriptsig (conforming to BIP34), 4 byte nsequence, 1 byte txout count, 8 bytes amount, 1 byte scriptpubkey length, 0 bytes scriptpubkey, 4 bytes nlocktime).


text:
How come some algorithms are better for GPU mining vs CPU mining?
How come some algorithms are better for GPU mining vs CPU mining?
From what I know (that may be wrong) most Ethereum miners use GPUs?
And bitcoin miners use CPU and ASIC mining?
What are the different algorithms out there Sha256, Scrypt, what others are being used?
And why some are better to mine with GPUs other's with CPU?
If one is to create a new PoW coin isn't there the danger that someone with a lot of ASIC mining power currently for bitcoin can just switch to your network and easily do a 51% attack and bring down your project?
How can one defend against that? Are there any algorithms that are not good to mine on ASICs?
Should a new Proof-of-Work coin use a new algorithm that most popular coins don't use?

Both are computational units. Algorithms are typically written without making assumptions about underlying chips that are going to run those algorithms, but certain type of tasks can benefit from massive parallel processing that GPUs, which increases GPUs appeal in mining, machine learning, etc.
https://blogs.nvidia.com/blog/2009/12/16/whats-the-difference-between-a-cpu-and-a-gpu/
Specific explanations about ML and mining can be found here:
https://www.quora.com/Why-do-people-use-expensive-GPUs-for-Bitcoin-mining-and-not-multiple-CPUs-like-Quad-Xeon
https://www.quora.com/Why-are-GPUs-well-suited-to-deep-learning


text:
How can I create/provide crypto currency wallets to my users?
I am interested to know, if I want to make a facility to my website users to be able to pay/buy with crypto currencies, also to gain ability to send/receive crypto currencies together( something like what crypto exchanges do) what is the solution for that?
As my understanding, crypto exchanges like Coinbase or Binance, create a wallet for each user and each currency, and users use their wallets created by the exchange to transfer currencies between themselves. If this is correct, how do they create these wallets for their customers? Are there any APIs for doing that or their create their own wallets, etc?

Satoshi Nakamoto invented Bitcoin in order to avoid any need for trusted third parties.
Cryptocurrency exchanges like Coinbase or Binance are examples of trusted third parties.
From the above, I would not expect trusted third parties to be especially facilitated by the Bitcoin network protocol initially created by Nakamoto, nor especially facilitated by what many regard as the reference implementation of a Bitcoin node (Bitcoin-core) which was originally developed by Nakamoto.
I would expect commercial businesses, such as those two, to have employed software developers to create proprietary closed-source software that suits their business objectives. They might, or might not, have used elements taken from open-source Bitcoin projects - where they can do so without violating the licence terms.
Commercial business tend to keep confidential the details of their internal systems. Especially those which relate to control of money. It should not be a surprise if they do not reveal exactly how they create, for their customers, the things they call "wallets".
Some of these businesses may have their own APIs that can be used by resellers of their proprietary services, or by others. I would expect that users of these APIs would have to seek documentation and support from the businesses concerned.

So you say there is no know/common way of doing that?


text:
QR code life span
Once the QR code is generated, how long is the code good for 24 hours, 48 hours?  I googled for the answer with no success.
Thank you

Could you be more specific in what context the QR code was issued?

I am going to send someone (that I know personally) money using a bitcoin QR code via Bitcoin ATM machine.  When the person sends me the QR code to use at the bitcoin ATM machine, what is the lifespan of that QR code 24 hours or longer?

A QR code encoding…

a payment request specific to some payment provider is subject to the specific payment provider's terms of use. A typical range could be between a few hours and a day.
an on-chain Bitcoin address remains valid indefinitely. Although address reuse is not recommended, it could be used for multiple payments.
a Lightning Invoice will be valid until the invoice times out (often 1h) or the invoice gets paid.


text:
Difference between savemempool vs getrawmempool information
Does the savemempool  have any additional information than the getrawmempool with verbose=true output?

No, in fact getrawmempool returns more information.
savemempool saves the mempool contents to disk. However it does not save several things that are precomputed and cached. These values are recomputed when the saved mempool is loaded. getrawmempool will provide those cached values.

gotcha, thanks for answering... will stick with getrawmempool then going fwd


text:
Why is restructure of mempool required with cluster mempool?
This is a follow up question from the comments in the answer of this question:

What is wrong with the current mempool data structure that it requires a complete overhaul?
Are there any conclusive numbers on how much more transaction fees will be captured with such overhaul or is it just all theoretical at this point?
How will this be finally rolled out? Opt-in or will be just a replacement in a new node client version?

What issues exist with the mempool data structure?
The current mempool design organizes each transaction based on its ancestor set. This leads to a number of issues, e.g.:

Ancestor sets overlap: one transaction can be featured in many ancestor sets. Picking a transaction into a block template requires updating all descendant transactions’ ancestor sets.
We cannot easily determine the effective feerate at which a transaction will be selected into a block. We need to evaluate all related transactions in order to determine the feerate at which a specific transaction would be picked into a block.
We build block templates by repeatedly picking the ancestor set with the highest ancestor set feerate. When our mempool data structure exceeds its limit, we repeatedly evict the descendant set with the lowest descendant set feerate. These two evaluations are not perfect opposites, and it is possible for transactions to be both in an ancestor set with a high feerate and a descendant set with a low feerate. Instead we would like to evict the transactions that will be picked into a block last. We would need to structure the entire mempool into blocks every time we evict to find out which transaction would be picked last. This is computationally expensive and causes is to stick to the imperfect descendant set feerate heuristic.
Given that it is hard to determine at what effective feerate transactions will be picked into blocks, it is difficult to determine whether a potential replacement is better, worse, or incomparable to the original. The current RBF rules allow for some edge cases in which we accept replacements that are not clearly better, and rejects some replacements that are better.
A parent transaction with two higher feerate child transactions is interpreted as two competing ancestor sets.

How does the Cluster Mempool proposal address these issues?
Where the old mempool data structure tracked transactions in the context of overlapping ancestor sets, Cluster Mempool tracks transactions in the context of clusters. A cluster is a component in the graph of unconfirmed transactions, i.e. a maximal subgraph of transactions connected via child-parent relationships. You can identify a transaction’s cluster by adding all of its ancestors and descendants, and then adding all of the newly added transactions’ relatives until there are no more to add. Each transaction is part of exactly one cluster.
For each cluster, we keep a linearization. The linearization represents a topologically valid (preferably optimal) order in which the transactions would get picked into a block. The linearization is split into chunks of transactions that would get picked into a block together. Each transaction is part of exactly one chunk, and we can calculate the effective feerate of a chunk based on its members by simply adding up their fees and weights.

Because transactions are part of exactly one chunk, no other chunks have to be recalculated when a chunk is picked into a block
We can directly read the effective feerate of every transaction per its chunk feerate.
Block building is simplified to comparing the (remaining) first chunks of each cluster, and popping the highest feerate chunk off the cluster into the template. The remaining chunks in the affected cluster do not need to be updated.
As the transactions are all grouped into clusters and the clusters are ordered, we implicitly have an order on the entire mempool. This means we can easily identify the transactions that would get mined last by comparing the last chunks in each cluster. We just need to evict the chunk with the lowest chunk feerate across all clusters.
Tracking transactions in the context of all their relatives in a linearized fashion offers "the feerate diagram" metric for evaluating replacements. This method of comparing is applicable to packages, which has led to breakthroughs in package relay and package-RBF.
A parent transaction with two higher feerate child transactions will be interpreted as a single chunk with a higher chunk feerate than either child’s ancestor set.

Are there any conclusive numbers on how much more transaction fees will be captured with such overhaul or is it just all theoretical at this point?
This is difficult to answer. If a sequence of transaction submissions were processed into blocks per the ancestor set-based (ASB) mempool design or the cluster mempool design, the total amount of fees would remain the same—just their order would change. If however some miners use ASB mempool and some miners use cluster mempool, the cluster mempool miners may be able to collect a slightly increased block reward by discovering chunks with higher feerates than the ancestor sets ASB miners are considering. Prior related research suggests that most blocks would only have a minuscule reward increase, but the new mempool design could encourage transaction patterns that increase the block reward of cluster mempool miners. Simulations suggest that cluster mempool generally performs at least as well as ASB mempool in regards to block rewards.
How will this be finally rolled out? Opt-in or will be just a replacement in a new node client version?
Cluster mempool is a drop-in replacement. The most significant changes in mempool policy would be the replacement of the ancestor and descendant limits with cluster size limits (count and total weight). This may affect some users that create long chains of transactions or other large clusters (which IMHO usually are indicators of poor operational practices, e.g. processing a large payment volume but not batching payments). The node itself would perform better at various tasks as outlined above. It may participate in package relay when that ships, and evict transactions in a slightly different order than its peers. If broadly adopted, cluster mempool may make descendants-pay-for-ancestor transaction patterns more viable, i.e. where two child transactions paying for the same parent currently compete, they would compound in a cluster mempool. This may e.g. improve outcomes of transactions spending unconfirmed outputs of batch transactions. Other that that it should not noticeable affect peer interactions, especially I would not expect any issues for nodes with ASB mempools peering with cluster mempool nodes or vice versa.
Why is restructure of mempool required with cluster mempool?
I’m not entirely sure how you mean your titular question, so let me cover a few angles:
Cluster mempool is not a required update. Any of the above improvements could be implemented with the existing mempool data structure, but overhauling the data structure allows use to get reasonable run-time performance on the new features. Beyond that,
restructuring the mempool to be designed around clusters instead of ancestor sets provides a superior abstraction that should simplify the code base around interactions with the mempool. There are many reasons why we want to always know the effective feerate of every transaction. Cluster mempool is a compelling upgrade because it provides efficient algorithms to maintain this information as well as making on-going projects like work on package relay easier to reason about.


text:
In blockchain, if no new members join the network can we prove that there will not be anymore forks before a certain point in history?
I would like to know if we can allow new members to join with the promise that they will not attempt to create a new fork however hard it may prove to be, before a certain point in history. To this effect; if no new members join the network for let's say a month, can we prove that there will not be anymore forks before a certain point in history?
I frankly feel the two statements are paradoxical. Actually I want to know if we can add a rule to the blockchain to say if it ran for a month with at most N number of nodes and all did not fork, then the network won't allow forks before that point in time, starting from let's say 2 months later.

Are you asking about a rolling checkpoint?

What happens if two people disagree on whether or not N nodes did not fork? Then won't they disagree on whether or not to invoke fork protection? You need some kind of consensus algorithm for how to agree on whether or not N nodes did not fork and then you see you have a huge chicken and egg problem.


text:
Refer to UTXO by output index or output full address in blockchain?
I'm writing my own simple crypto and wondering whether input in a transaction should refer to unspend output only by it's TXID and vout (index in outputs vector in a transaction) OR by it's TXID and full address in a blockchain - that is block height, transaction index in a block and output index in a transaction.
The first approach seems to have a flaw - it's unlikely but possible for two different outputs to have the same TXID and vout.
But in the second approach I don't even need TXID when I have the full blockchain address right? I don't want to double on bookeeping - when there's duplicate data you need to make sure it's consistent.
So my two question's are:

How to avoid the problem with two transaction's accidentally having the same hashes in an elegant way that - ideally - doesn't require storing full addresses?
How is it done in Bitcoin? (here I guess that bitcoin's Bip34 update was addressing this flaw I mentioned - but I don't understand how)

General advices on solutions are very welcome. Thanks in advance!

In Bitcoin, it was originally an assumption that miners producing blocks would use a unique address per transaction, making sure that the coinbase of each block had a unique TXID. As this was not enforced, users accidentally made duplicate transactions by setting the same destination address for multiple solved blocks. BIP34 simply adds a new field to be validated to the block, requiring that the height is also part of the hash, ensuring uniqueness for all future blocks.
For the two special cased BIP34 blocks, the outcome is rather simple. The first instance of the TXID creates the UTXO, the second invocation does nothing at all because the output already exists, and if they were ever spent the UTXO can only be removed from the database a single time. The money involved in the second instance of the TXID was destroyed, causing a loss of 100 BTC total.

thanks for answer. So coinbase outputs for the same public address naturally hashed the same. But is it the case that still - after Bip34 change - two very different transactions can hash to the same hash causing a loss of unspent bitcoin (because sha256 isn't injective after all)?

The chance is effectively zero. We don't design software with the assumption that duplicate TXID could exist, as we would have significant other issues if collisions where commonplace.


text:
Can I run a bitcoin full node on my laptop and use a external hard drive to store the blockchain?
I tried setting the datadir=/Volumes/BLOCKCHAIN (which is the path to my external drive). Then I ran bitcoind and this is the end of the output, right before it kicks my drive out and gives me a Bus error: 10.
...
2019-10-22T16:30:26Z init message: Loading block index...
2019-10-22T16:30:26Z Opening LevelDB in /Volumes/BLOCKCHAIN/blocks/index
2019-10-22T16:30:26Z Opened LevelDB successfully
2019-10-22T16:30:26Z Using obfuscation key for /Volumes/BLOCKCHAIN/blocks/index: 0000000000000000
Bus error: 10

Is there another way I can achieve this?

If you're using USB 2.0, your problem is going to be the read/write speeds.
Minimum requirements state that you need at least a read/write speed of 100 MB/s.
Typical high-speed USB 2.0 connected drives only do about 40-50 MB/s.

I believe I have a faulty SATA or SATA case because it's USB 3.0 and should have read/write speeds faster than 100 MB/s. But beyond the external drive is it okay if I do this method?

Yeah of course. It's likely something is off with your external setup, as you say.  Is the drive powered by USB as well or does it have its own power supply? Just another thought.  There's also usually a board in between SATA interfaces and the USB that might be faulty or throttling speeds somehow. Do a drive speed test and see if the numbers do indeed exceed 100 MB/s.

You definitely should use an SSD for storing the Bitcoin blockchain. A spinning hard disk gets too much stress over time and becomes quite warm, because there is a constant reading and writing going on. If you don't have any other disk, then don't store other data on it, I wouldn't trust that disk over time.
Also make sure that you have write permissions on the disk.
To see if the error comes from a faulty connector or adapter, you can run an I/O test. Depending on the operating system you're using, there's many good ones out there. For Linux, I love Bonnie++.


text:
Change IBD Location Prior to Download
I'd like to run Bitcoin Core but my laptop can't handle the load. I have an external drive with enough space, but I'm unsure if/how to change the file destination upon opening the program. Is this possible?

Hi, yes it is possible, you can easily find this information about Bitcoin Core on Google, and I'm pretty sure that Bitcoin Core will ask on first startup for data location. Besides, if you're a bit short on storage, I can also suggest you to prune the blockchain, you can trim it down to 550MB if you want (at the cost of some advances features becoming unusable though)

This and this should be helpful.
You start Bitcoin Core with the -datadir=<path> option where <path> is the path to the datadir on your external drive.
You can put the path to the datadir in your bitcoin.conf file with datadir=/your/path if you don't want to have to always start Bitcoin Core with that datadir flag.

If about 5 GB is affordable for your laptop drive, you could still keep the datadir on it, and move the block storage only to the external drive by adding blocksdir=/path/to/external/blocksdir to your bitcoin.conf. This solution allows you to keep your wallets on your laptop drive using default wallet location. If your laptop drive is SSD, while external one is HDD, this approach will give additional performance benefit by keeping the chain state and the block index on SSD.


text:
Bit coin core Settings/Receiving addresses
Under these settings in bitcoin core. does imported addresses (watch-only) appear under the settings as an official receiving address or it appears there because imported a wallet to watch.

Settings/Receiving addresses [Window/Receiving Addresses]

In Bitcoin core, under the menu "Window", "Receiving Addresses" ...

... you'll normally see a list of addresses created using the "Receive" button. For example:

does imported addresses (watch-only) appear under the settings [Window/Receiving Addresses] as an official receiving address

If you import an address, it will be non-spendable, that is watch-only. I used the console window to run importaddress  "1FeexV6bAHb8ybZjqQMjJrcCrHGW9sb6uF" "watchonly" where "watchonly" was just my choice of label for this - you could put anything there.
Then that non-spendable address will appear in the list of receiving addresses

This is potentially misleading. The tile says "these are your Bitcoin addresses for receiving payments" but you will not be able to spend any money sent to that last address. Even if you gave it some other label like "spendable" or "rainyday".


text:
Lightning Network - revocation format
Lightning Network uses Revocable Sequence Maturing Contracts.
The LN Paper specifies that:

If one wishes to permit revocable transactions [...] the output
transaction construction would remain a 2-of-2 multisig:
2 Alice1 Bob1 2 OP CHECKMULTISIG

However, the current implementation uses elliptic curve point multiplication (not multisig).
I have found this information in BOLT03 - to_local, but also here.
The benefits of the current approach are:

smaller size of the Output Script
only one signature required to unlock (vs 2 in the original spec)

Are there any other considerations besides these two for using elliptic curve point multiplication instead of multisig?

The output referred to in your citation from the paper is the funding transaction output, which is still the same: https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#funding-transaction-output. The commitment transaction (RSMC) spends this output to one or more outputs.

Thanks Mark! I'm still trying to fully understand the LN Paper.
But I think it does not refer to the funding transaction output, it does not make much sense in the given context.
Either way, do we know if originally the revocation was a multisig and then changed to eliptic curve point multiplication, and why?