text:
When exactly does Bitcoin transaction batch happen?
I need some help understanding the Bitcoin internal...
The basic idea is:
The miner assembles N transactions into a block (rather than creating a consensus for every single transaction). And hash those transactions for the consensus.
But, when exactly the miner code decides to "combine"? How long does the miner code wait until passing those transactions to a hash function?
Does it wait until the list of transactions hit the block size limit (1 MB)?
Or does it just wait X-minute?
Or does it combine immediately as soon as it receives the transaction fees?
The whitepaper says

Each node collects new transactions into a block.

Although it would be possible to handle coins individually, it would be unwieldy to make a separate transaction for every cent in a transfer. To allow value to be split and combined, transactions contain multiple inputs and outputs.

But does not explain how many it collects, or how long it waits for collection...

A miner chooses whatever transactions makes business sense to them. They don't have to include any transactions if they don't want to. They can make a block containing no transactions (other than the coinbase transaction that pays the miner the block reward).
Typically, as soon as a block has been published by someone else (or themselves), I'd expect a miner to obtain or build a block template that may or may not include a bunch of transactions waiting in its "memory pool" (mempool). Then the miner will run through trillions of different values for the nonce etc and for each value, compute the hash of the block to see if the resulting  hash is less than the current target.
But miners can start mining any time they like. The only constraint is that they just waste their time if they try to publish block number N after someone else already published a different block with block number N.

The miner assembles N transactions into a block

That's basically true. Mostly.

(rather than creating a consensus for every single transaction).

Miners don't create consensus.
Miners could create a block for one transaction at a time if they wanted to.

when exactly the miner code decides to "combine"?

Whenever they want.

Does it wait until the list of transactions hit the block size limit (1 MB)?

No.

does it just wait X-minute?

No.

does it combine immediately as soon as it receives the transaction fees?

No.
Miners don't receive transaction fees. Certainly not in advance.
Miners claim transaction fees. They do this by creating a coinbase transaction that they include as the first transaction in a block. This transaction typically moves the transaction fee amounts into an address of the miner's wallet.
They can't spend that money unless everyone involved thinks their new block meets all the rules.
One rule is they can't spend that money until after another 100 blocks has been mined.

Thank you. Then is there any benefit to include more transactions thus longer chain? What if two nodes found nonce almost at the same time?

See What happens if two miners mine the next block at the same time? and What would happen if two miners were mining two blocks with two different set of transactions?


text:
Internals of bitcoin block replication workflow from wallet, and conflict resolution

If I initiate a transaction from my wallet, does my transaction go to only one node? Does wallet client send to multiple nodes?

What happens if no miner decides to include my transaction? Would my transaction fail and waste the fee?

How does a bitcoin transaction enter the network?

every peer in the network has the relayed transaction

Usually how long does one announced transaction to be disseminated to the entire network? Is there any reference or data on this?

What do we mean by longest available chain? Is it like [1 2 3] vs [1 2 3 4] or [3 2 1] vs [5 6 7 8]? How does the protocol ensure the ordering between transactions?

Each node is in contact with a handful of peers.

If no miners include your transaction, the money never left your wallet.

Propagation may take a few tens of seconds to reach most of the  network. See
How long does it take to propagate a newly created block to the entire Bitcoin network and why?

Actually length of chain is not important, that's a shorthand way of talking about accumulated proof of work. The branch with the most work invested in it is the one that nodes will select and use. I believe this mainly affects the node's UTXO set (a list of coins in the whole Bitcoin network that are available for spending by their owners). The primary job of the proof of work mechanism and of miners is arguably the ordering of transactions.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Are there any Bitcoin payment providers with an API’s for allowing unconfirmed transactions?
I want to design my web app to instantly accept crypto currency like Bitcoin as a payment for a subscription to the app. I will design it to cancel the subscription and ban the user’s account if the transaction does not appear in the blockchain 6 blocks later.
My app is such that malicious users wouldn’t be able to do much with just 1 hour of subscription time even if they constantly make new accounts. I’m counting on them just getting bored of the first few levels and taking the loss if they don’t. If I’m not giving refunds and I make sure not to withdraw pending coins myself, what attacks am I vulnerable to like Mt. Gox?
I have a degree in Computer Science, and I’ve dabbled in crypto for a while. The idea of working on a custom full node wallet for my server excites me. However, if someone has already built exactly what I need I guess I should use that. Are there any online payment services which already accept BTC instantly with an API for reporting false transactions?

re: "I will design it to cancel the subscription and ban the user’s account if the transaction does not appear in the blockchain 6 blocks later" - banning the user doesn't seem like a good decision, there are lots of legitimate reasons that a user's transaction might not be confirmed after 6 blocks. Otherwise, questions that are asking for service reviews (ie, 'which API service should I use') are not on-topic here. See: https://bitcoin.stackexchange.com/help

Why do you need a third-party online payment service? If you want to control the BItcoin part of a transaction then using your own wallet API seems more sensible. If you want payment exchanged into a different currency then a payment service is appropriate - but then you need not be concerned with Bitcoin details at all and just cancel the corresponding subscription if the payment provider reverses a fiat transfer.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
I used a local honey badger bitcoin atm to buy 250.00 CAD worth of bitcoins, where did it go?
Ok, so it's not my first time buying/using bitcoin. However, this is the first time I've used my own wallet and purchased bitcoin solely for myself. I downloaded the app Bitcoin_wallet and used a HoneyBadger bitcoin atm to purchase 250 CAD of bitcoin. After I inserted the cash and pressed the done button I printed out a receipt which listed the transaction ID, the address of my wallet, the cash rendered (250), the exchange rate plus FEE($20198.9 + 0.00010007), and the bitcoin purchased amount was displayed. I assumed this information was confirmation that my purchase was complete. However, I still have not Recieved the funds in my wallet.... I guess what I'm getting at is "Does anyone know how long it should take for the bitcoin to show up in my wallet?" I realize that it is not an instantaneous process, but I don't know how long I should wait before contacting honey badger? If it makes any difference I am located in Calgary, Alberta, Canada.

It is advised to put your address into a block explorer such as https://blockchair.com/bitcoin (or several block explorers) to see if the network is aware of a transaction sending money to it.
You should also double-check that the address on the receipt is one of your wallet's addresses. (The main address displayed by the wallet may have changed by now, but it should be possible to open up a list of past addresses).
Most (all?) explorers show transactions even if they are unconfirmed. So if there is no such transaction on the explorer, you should take up the issue with the operators of the ATM.
If the transaction appears on explorers unconfirmed, it may mean you will see it in your wallet when it is confirmed.
If the transaction is confirmed but you still can't see it in your wallet, it may be an issue with your local wallet.

I don't know how long I should wait before contacting honey badger?

Their FAQ says

I have purchased cryptocurrency but it hasn't show up in my wallet. When will I receive it?

Under normal conditions, all transactions are sent instantaneously and are not batched
Sometimes, due to network congestion, compliance or security concerns, transactions may be delayed
Please keep your receipt and contact our support team with your transaction ID if you haven't received your cryptocurrency within 2
hours of the purchase
We reserve the right to retain all transactions until we can complete our security checks

So contact them.

Thank you very much for your help! I appreciate it. I have not yet Recieved the bitcoin, but I have contacted honey badger and I'm just waiting for a response back. As for the address I used, your right, it has changed and so far i haven't been able to find any list of past addresses.


text:
On my first shopping via bitcoin I get "The fee for this transaction seems unusually high. (20%)" What to do about it?
I have some bitcoins in a local Electrum wallet and wanted to do my first shopping with bitcoin from a shop offering payment via bitpay for an item costing (in USD) 20 dollar.
So I went through the payment process with bitpay (Bitcoin amount is around 1.8 mBTC), disabled in Electrum as they had suggested Replace-By=Fee and then pasted bitpay's payment url (bitcoin:?r=https://bitpay.com/i/....) into Electrum.
I got to the send screen. But here I received the following warning: "The fee for this transaction seems unusually high. (20%)" as shown in this screenshot:

Why is the fee so high and how can I reduce it to an "normal level" like with credit cards (say 0.5-5%)?

@Prayank The payment gateway is bitpay .

@Prayank Thanks heaps for the suggestion! Good info. However in my case it is only credit card and bitcoin via bitpay.

This is expected, given the current mempool situation.
There are over 100k transactions pending as of this answer's writing. This has resulted in the fees rising significantly, up to 10x of what you might pay normally.
As your overall transaction value is small, this results in the warning showing up. The warning itself was added to prevent people from paying absurdly large fees. In your case, as the amount is small, the fee is proportionally considered large.

Thanks @RaghavSood. So there's nothing I can do about this?

You could wait until fees drop, but that's about it

Ok, I'm not in a hurry. I will check again. Anywhere public (i.e. not as part of the payment process) where I can see fee estimations?


text:
How to send requests to a testnet faucet to collect coins at my address?
I am really new to Bitcoin and have just started reading about it.
From what I understand testnet Bitcoin can only be requested every 12 hours or so and yields a very low quantity. I would like to build a bot, so to say, which sends a request every 12 hours or so to a testnet faucet.
What I am looking for is a way to send a get/post request, so I can use shell script  or javascript to source the tBTC.
Could anybody point the right direction out to me.
What is the link? How to send the request? If there's a github repo about something related? Any documentation?
Any of these answers would be really helpful to me!

I don't know the github projects for this argument (also the question on the project here is off topic).

I would like to build a bot

Why do you want to create a bot to request the tBitcoin? you have needed a big quantity of bitcoin for your application test?
if yes I think this is not a correct street, the Bitcoin testnet is a network for the developers and in this network, the money is mined my a low difficulty.
So for the respect of other developers if you have neet big quantity of tBitcoin to do mine it. (a reference for do it).
Another possibility is using the regtest network od bitcoin, (you can build your local network, a reference of regtest)

What is the link? How to send the request? If there's a github repo about something related? Any documentation?

However, these reference of the faucet bitcoin testnet are available here but the request depend of the site that implements the faucet


text:
Relation between transaction fees and inputs
So this might appear as a duplicate of some of the other questions on this stackexchange, but I feel most of the questions have failed to ask a question that I am really interested in.
I want to calculate the fee for my transactions. Now to calculate the fee for a transaction you need to have the number of bytes for that transaction, which are determined by the number of inputs. But the number of inputs a transaction needs, is decided by fee + amount to be transfered. Although usually the fee might be very small compared to the amount one would be transferring but I want to know how to calculate the fee in this case?
Any ideas on this concept on how to solve this issue, or maybe someone who could guide me if I am wrong on this concept would be greatly appreciated.

Does this answer your question? Calculation of transaction fee for transaction if not please specify exactly which part of the answer you need help with.

No this does not answer my question. If you read my question @RedGrittyBrick it clearly says that I want to know the relation between fees and inputs, where the fee is also inclusive of inputs that will cover the fee. But the amount of the fee is not considered in the question that you have attached...

I think I see your misconception: the fee is simply the difference in monetary value between the sum of input values and sum of output values (e.g. payments + change). There is no input specifically for the fee. It isn't possible to point at any particular input and say that input is to cover the fee. A wallet is free to implement any algorithm to choose a fee and to choose inputs. Calculations of fee/vbyte for recent transactions are post-hoc. Since four other independent people felt your question was a duplicate, to reopen, edit the question to make the nub of your question clear to all.

P.S. Since the change output's amount is a fixed-size 8-byte integer, I conjecture the amount of change (and therefore the fee)  should have no effect on the transaction size in most cases. If necessary the wallet can substitute a small input for a larger one. It might be very rare for the initial excess value of inputs needed for payments to be less than the selected fee-rate * vbytes.


text:
How to use Digital Signature to Sign Unsigned Transaction?
so I was trying to sign my unsigned transaction:
010000000194d612f5ec40d483ed82bf346d8f08961d61fdf2d6b9804d7731e608d816e5330000000000ffffffff01204e0000000000001600143b36ddb4d9f75f4010450b25a8952a6725981bfb00000000
with a digital signature (ECDSA SIGNATURE in hex representation) I created based on this unsigned transaction which is:
3044022098f9780241fdd9f87a222a85cc65140f61913476c2d3c0f886e328a03f897ea4022089388ea886bb4d2440893665dc54fb7d4a0498766b4752b15415fb18dc38b72a.
How to serialize these two so that I can use this digital signature to sign the unsigned transaction. Any explanation, example or any package (in Node.js) or otherwise also would be really helpful.
I tried finding examples and test cases and even went through code for bitcoinjs-lib but I've not been able to find a single example of using a digital signature to sign and send a transaction. Most of them try


text:
Where to find the amount that is being sent in a signed transaction?
I was looking at a transaction I just signed for USDT and I decoded it to compare stuff to my original address and amount. Just to be careful.
I found the address that the data is being sent to in vout[1].addresses[0]. I now want to see where is the amount that I am sending and if it is the same as the one I had intended.
{
    "txid": "42aaf04994f7123470e8c7aff2a7fd2341b2b3d648d5839c4023432fc12cfbe",
    "hash": "42aaf04994f7123472342324f2a7fd467123445d2342d58235235345212cfbe",
    "version": 2,
    "size": 405,
    "vsize": 405,
    "weight": 1620,
    "locktime": 0,
    "vin": [
        {
            "txid": "7f589c98b41292d2180a33eac21ab7f1235456ebfd8c897305793bb793182534",
            "vout": 1,
            "scriptSig": {
                "asm": "3045022100afcd667212334568ae54e1b6e555fc756cce228923a809939382ce2d7e99d778022058a62ecf394cafb4b52ba7e1d49d543a1edef2134190f88773a30a11ef8f20b[ALL] 023b976783e66fbdc15e96954278d194829ce15735c99a11af7d6d56aae070c728",
                "hex": "483045022100afcd6671234567f8ae54e1b6e555fc756cce228923a809939382ce2d7e99d778022058a62ecf394cafb4b52ba7e1d49d543a1edef94897690f88773a30a11ef8f20b0121023b976783e66fbdc15e96954278d194829ce15735c99a11af7d6d56aae070c728"
            },
            "sequence": 4294967295
        },
        {
            "txid": "4f7dee045e338f0e68ad35dd98e1d1a07fac5d63f912e09f9c9cf88fea5fb7eb",
            "vout": 2,
            "scriptSig": {
                "asm": "3045022100f4af4fa80e2134cf84d87a42c14945717adf50b1cf0b75baafd821cb938bb02b0220061b806f0350ff9e0b20d06fce1ed35cc2fb1234569c321e4b96c479dbc5688[ALL] 023b976783e66fbdc15e96954278d194829ce15735c99a11af7d6d56aae070c728",
                "hex": "483045022100f4af4fa80e2134cf84d87a42c14945717adf50b1cf0b75baafd821cb938bb02b0220061b806f0350ff9e0b20d06fce1ed35cc2123456c549c321e4b96c479dbc56880121023b976783e66fbdc15e96954278d194829ce15735c99a11af7d6d56aae070c728"
            },
            "sequence": 4294967295
        }
    ],
    "vout": [
        {
            "value": 0,
            "n": 0,
            "scriptPubKey": {
                "asm": "OP_RETURN 6f6d6e69000000000000001f0000000006052340",
                "hex": "6a146f6d1234780000000000001f0000000006052340",
                "type": "nulldata"
            }
        },
        {
            "value": 0.00000546,
            "n": 1,
            "scriptPubKey": {
                "asm": "OP_DUP OP_HASH160 7a8cb455ccb12345617396cce512c37134d72369 OP_EQUALVERIFY OP_CHECKSIG",
                "hex": "76a9147a8cb455c1234561617396cce512c37134d7236988ac",
                "reqSigs": 1,
                "type": "pubkeyhash",
                "addresses": [
                    "1CAyzih1V3nhx98STVq7Jdya88VY4Bkna6"
                ]
            }
        },
        {
            "value": 0.00042535,
            "n": 2,
            "scriptPubKey": {
                "asm": "OP_DUP OP_HASH160 c4d18aa08bcdcabd245a7f183e09ca712c6c83b OP_EQUALVERIFY OP_CHECKSIG",
                "hex": "76a914c4d18aa08bcdca89357aa7f183e09ca712c6c3b88ac",
                "reqSigs": 1,
                "type": "pubkeyhash",
                "addresses": [
                    "1JwgTpJPZ2WtsZvoo6EsstdB9jNnhJxERB"
                ]
            }
        }
    ]
}

This is the signed transaction.
The amount I intend to send is 1.01 but I can't see that anywhere only random values in vout[1], vout[2] and weight. Can anyone please guide me where to find this value or how to recreate it? Maybe weight is the USD value of the amount I am sending in USDT Omni?
Any guidance will be much appreciated.

USDT on the Bitcoin blockchain uses the Omni layer, which encodes data in the OP_RETURN outputs. In your transaction, the amount is 0000000006052340, which is hex for 101000000. Omni has a precision of 8, so you can divide that by 10^8 to get 1.01.

BRO! I would have never guessed that! Is the value always stored in the last 16 places of the OP_RETURN hash?

Yes, the format is OP_RETURN <omni magic 4 bytes> <asset ID 8 bytes> <value 8 bytes>


text:
Why does Electrum never complete synchronization with a server?
How do I diagnose why electrum never finishes synchronizing?

I've tried selecting networks individually without success.
Can I ping a server to confirm connectivity?

Electrum 2.6.4 is over 5 years old. Have you tried with a more recent release?

huh, I was just going with snap:  https://snapcraft.io/electrum   thinking that would be up to date.  thanks @PieterWuille


text:
how do nodes agree or disagree after new block is create?
I'm new to blockchain field and I read one article.
article
In this article of Authorization chapter, it is said that "For a public blockchain, the decision to add a transaction to the chain is made by consensus. This means that the majority of “nodes” (or computers in the network) must agree that the transaction is valid."
But, I can't imagine the agreement situation by nodes.
How do all nodes agree on that new block　 should be added to blockchain ? Do nodes have to agree or disagree manually like by clicking a "Agree!!" or "Disagree!!" button?
Also, doesn't it take soooooo long time? I mean that there are tons nodes in the network and to gather all nodes agreement, it'll be sooooo long time I guess.

Read the bitcoin wiki at bitcoin.it instead.

I'm new to blockchain field and I read one article.
article
...
How do all nodes agree on that new block　 should be added to blockchain ? Do nodes have to agree or disagree manually like by clicking a "Agree!!" or "Disagree!!" button?

They run the "blockchain" software, and the software automatically accepts or rejects blocks based on what exact software they are running and what exact configuration the software they are running is using. The only manual steps are choosing the version or modification the node operator will choose to run, and configuring it.

Also, doesn't it take soooooo long time? I mean that there are tons nodes in the network and to gather all nodes agreement, it'll be sooooo long time I guess.

Accepted blocks are propagated on the network fast. Generally rejected blocks are propagated fast among the nodes who accept them, but their propagation will be slower because not every node accepts and broadcasts.
Things will be slightly different depending on if the choice creates a soft fork or a hard fork.

Consensus in Bitcoin is not really a consultative operation.
In the case of Bitcoin, consensus does not mean that all nodes have to communicate with each other to argue back and forth about whether a block should be accepted.
Each full node makes their decision in isolation without consulting other nodes.

Consensus is not absolute, and need not be
In a way, the network never reaches 100% consensus because some nodes may be offline for days or weeks, new nodes are always being added, those new nodes may spend considerable time synchronising.
The important thing is that enough nodes are applying the same set of validation rules.  That means at any given instant, the number of nodes that consequently have the same UTXOs set is sufficient to sustain a currency.


text:
Unable to sendcoins using bitgo express apis
Trying to access API URL: https://test.bitgo.com/api/v2/tbtc/wallet/walletid38u5348t9yj958jtv985j89g5/sendcoins
But every time getting same response:
{
    "error": "You have called a BitGo Express endpoint but this is the BitGo server. Please have a look at the BitGo Express documentation: https://www.bitgo.com/api/v2/#bitgo-express-rest-api",
    "name": "ApiError",
    "requestId": "irfo34r5541k06grtyy4j6n33",
    "context": {}
}

Is there any correction in API route? Or how can I successfully send coins using this api?

I don't understand why this is flagged as "customer relation with a company". Nothing in this question requires information only BitGo could have.

sendcoins is an endpoint that is only available on BitGo Express. BitGo Express is a proxy that runs on your own infrastructure, typically on port 3080.
Instead of calling test.bitgo.com, if you're running BitGo Express locally, try calling localhost:3080.

does that mean APIs will be accessible if we are running BitGo Express locally? Right now I am not running it locally.

Yes, it's a proxy that forwards requests to the API, but performs cryptographic operations client-side.


text:
How to retrieve Bitcoin transfer unconfirmed for 10 days
I’m very new to Bitcoin. I had Bitcoin sent to wallet on Bitcoin.com and then sent to wallet in cash app that I am familiar with. After 12 days it still says unconfirmed and pending. Is there a way to resend and pay higher fee or what can I do to have access and be able to cash in and use money? Please help!!

My first recommendation is for you to never use bitcoin.com.  This is a scammy site which attempts to trick new users into thinking that something called "bitcoin cash" or the ticker "bch" is bitcoin. Which it is not.
But luckily the bitcoin that you bought is the real bitcoin (often referred to as btc)
By now, your transaction has probably confirmed and already reached your wallet since the activity on the network has calmed down now over the weekend.
If not, it is usually possible to use something called "Replace By Fee" or "RBF", which increases the fee that you pay for your transaction so that it will get included in a block, and thus reach your faster.
If you are interested, consider to visit bitcoin.org to find out all the details about how to install your own bitcoin node and wallet in order to increase your security and not need to trust Cashapp for holding your funds.


text:
Change -mempoolexpiry in BitcoinCore 0.20.1
I have accidentally made a transaction with a very low fee (5 sat/byte) and I doubt that it will be confirmed any time soon.
Replace-By-Fee is not available since my transaction has no change.
Abandon the transaction option is also not available.
How can I change the -mempoolexpiry setting in my BitcoinCore (0.20.1), which I run on Windows 10, so that I don't need to wait for 2 weeks for the transaction to be automatically dropped out of the mempool?


text:
Public key collision/address reuse and HD wallet path search space
If you load the same key and the correct path into different wallets, and it properly generates a new address for every transaction, would the two wallets generate the same address in a sequential fashion? Or could it conceivably sit there for years, searching through billions of addresses?


text:
What settings required to privately run bitcoin core on tails?
I am running bitcoin-core on TailsOS which seems to work out of the box.  I think TailsOS blocks all unsafe requests but I would like to know what settings I should enable to run it really privately.  So far I start my node via
$ ./bitcoin-qt onlynet=onion

Is anything else required?  In this answer I read about a bunch of other settings but feel that isn't required on tails, correct?

Thank you @Prayank but that answer seems to be also largely not tails specific so most settings are likely not applicable.

e.g. tails disables UDP and prevents DNS leaks in general so I don't see that setting is required. https://tails.boum.org/contribute/design/Tor_enforcement/DNS/
In fact, my feeling is that bitcoin core does not need any non-default settings in tails.


text:
Blockchain.com wallet was compromised and btc stelon
I lost some money in a ico scam and googled and found a reclainfund ltd to recover fund. They ask me to  set up a blockchain.com account and then share account wallet id and password so they can transfer stolen fund. I did that. then I transferred 0.51 btc to my account and then changed my password.  now it seems to be hacked and my btc is gone and all transaction history is gone.  The weird thing is the blockchain's account left panel is totally empty now once I login and I can't see any transactions. how could that happen ? I wonder if someone inside blockchain.com is associated with hacker or scammer. that is really some high level scam I have ever see.
my question is in my blockchain.com account, where is the private key that they can steal and copy and it can bypass my changed password ?

They only needed to have access to your account a single time to have exported your private key, then that wallet is compromised for good. Your password encrypts the private key as stored on Blockchain.com's server (so that Blockchain.com can't read the key) but it doesn't change the private key itself.

but how come I can't find my private key in blockchain.com web page ? maybe I am not bitcoin expert and not familiar with blockchain.com. I only see my wallet id and addresses. Secondly, how come my account page is totally broken and nothing is showing and all transaction history is gone.

It's there, it's might also be called a recovery phrase or under a "backup/export wallet" option.

what is the best way to try to recover fund ? to find a reclaim firm or report to police or FBI ? this is not trivial money.

You don't, unfortunately, it's gone for good. There's no way to reverse a Bitcoin transaction, even if it is a scam.

in my case, I contact the guy through this website. https://www.regaincrypto.com/ they sent me a investigation CASE number and a whatsapp contact number for me to chat. if this turns out to be a fraud, I would talk to police and FBI to trace down the website owner right ? I have all the communication email and chat history.

@raymondli: regaincrypto.com was registered only a few months ago. If you check the address on their website it is a poor rural residential area. Also compare https://regaincrypto.com/Our-team/ with https://www.coinfirm.com/about-us/ - same photos, different names. scammers copy other scammers. You've probably been scammed at least three times here and need to be very careful.

If there was no withdrawal but export of the wallet then I assume you can still see in your local wallet or at blockchain.com the deposit address where you sent the 0.51 BTC to. If you post or PM this address / transaction I can have a look into with analytic tools if I can trace it already to an exchange. In case of success you could go with this information to police.

Even if you traced it to an exchange there's not very much they can do. The police probably won't take the case, and even if they did, and you did somehow find the recipient of the Bitcoin (which is unlikely) there's no way you can prove this particular transaction is a result of a scam and there's no way you can prove that particular recipient was the scammer.

It's a bit like if someone steals $5000 in cash from myself. I can report it to police and make a statement even if I cannot 100% prove that it was stolen. If 5 people report similar things then the police would ask the accused fraudster where he has all the money from.
With bitcoin this would be similar if a scammer cheats a bunch of people and the transactions can be traced to a cash out and the receiver cannot explain where he had the coins from or why the people paid him in bitcoin. I'm not saying it's easy but possible if the involved jurisdictions are cooperating.

Dunno what jurisdiction you're in but having actually tried to report cash and valuables stolen myself, unless there is evidence such as a break-in, they will usually tell you to go away because it's almost impossible to prove.


text:
Can I see transactions in bitcoin-qt without downloading blocks? if I have a server with blocks on the local network?
I have server with bitcoind (synchronized), port 18443 opened. And I tried to connect to rpc server with bitcoin-qt in my mac. With this configuration:
regtest=1
rpcuser=<login>
rpcpassword=<pass>
rpc_connect=<ip to server>

But bitcoin-qt writes that it is synchronizing, and to use the application you have to wait.
Can I use bitcoin-qt without synchronization?
P.S. Sorry for my english

It doesn’t work because the software doesn’t support operating like that. You can not connect bitcoin-qt to a remote RPC server.


text:
Possibility to retrieve BIP39 mnemonic (24 words English dict) from BIP39 (hex) seed?
I am in a situation where I have access to a BIP39 hex seed which I know has been derived from a BIP39 24 word English dictionary mnemonic.
I would like to retrieve this 24 word list - is this possible?
As far as I know there is no hierarchy per se between hex seed and mnemonic, so they should be bijective (hopefully).

This is not possible.
The seed words are converted to the hex seed through a process involving a one way hashing operation - it is not possible to go the other way.
That said, as long as your hex seed is the tree root, keeping that safely would allow you to access all your coins, although it would make importing it into a wallet somewhat difficult.


text:
Ternary Merkle Tree - concatenation and Merkle Proof?
Unlike binary Merkle trees, ternary have 3 leaf nodes. I wanted to understand how do we eventually get to the Root Merkle node -- is it by concatenating in the pair of 3 (considering 3 leaves in every set) or do we still make pairs (of 2)?
I would appreciate if someone would explain this. Thanks!

Which ternary Merkle tree? What design are you talking about? Bitcoin only uses binary Merkle trees to the best of my knowledge.

That's right! Bitcoin only uses binary merkle trees, however, in general there is a concept of ternary merkle trees as well and I wanted to understand if it works similarly by hashing then grouping individual nodes. If so, does it group and hash the hashes of all three child nodes together?


text:
"unloadwallet" and "notifywallet", work together?
I couldn't understand it very well in the places I looked and in the documentation I read, and it would take about 1 month for me to have bitcoins to test, I thought it was better to ask here.
I will need to have multiple wallets, so I could understand the correct way to work with them would be using "loadwallet" before generating addresses and doing any action on the wallet, and if I were going to use another wallet I should do an "unloadwallet" and another "loadwallet " to continue.
But in a description I saw, when leaving the wallet unloaded the "walletnotify" would not work correctly until the "loadwallet" was used again.
So what would be the correct way to use multiwallets and have actions targeted to specific wallets?

You can load multiple wallets simultaneously; no need to load and unload every time you're going to use a wallet for something.

But, for example, when I call "getnewaddress", how can I target this to a specific wallet?

If you use bitcoin-cli, you specify the subject wallet with the -rpcwallet= command line argument. If you're using JSON-RPC directly, there is a separate URI for every wallet.

NICE, thanks so much, i understand now, its work!


text:
How best to check the balance for a specific address using the Bitcoin-CLI?
I am currently calculating the balance of addresses from "listunspent"
$unspends = $btc->listunspent(6, 9999999, array($PARAMS['address']))['amount']

But I don't think this is the correct way.
Does bitcoin-cli not have a native way to get this amount?

My question would be exactly if there is a more correct way to do this than using these two functions (one of them being experimental), because it doesn't seem so correct to do this way to obtain the amount of an address.


text:
Bitcoin Core just deleted my 350+ GB of blockchain data?
I've kept the same config file and internal hard drive for my Bitcoin Core datadir for many years. I've updated to numerous new versions.
The other day, I reinstalled Windows on the system drive again. And thus reinstalled Bitcoin Core (v22.0.0) again. When it started up, I pointed it to the datadir on the other disk as usual.
Now I realize that it has deleted all of the blockchain data for the first time ever, leaving only 2 GB. It prunes by default to 2 GB now. And apparently, my config needed to have an explicit rule to tell it to not prune... which it of course didn't.
While I'm glad that I don't have to store all that stupid data anymore, it does make me question why they would enable this by default and actually delete existing data. Has pruning now reached such a level of stability that it's completely "safe" and there is no need to save the full blockchain anymore?
Or am I now running Bitcoin Core in "crippled mode"? Even though I did not actively set it to prune?

Bitcoin Core will never prune on its own; it is not default. It must be configured, either through config file, command-lirn arguments, or the GUI settings window, to enable pruning.

@PieterWuille No "prune" command in my config.

Are you using bitcoind or bitcoin-qt? Have you checked the UI settings (they are separate from bitcoin.conf)? Is something about "prune" logged in debug.log?

@PieterWuille bitcoin-qt.exe. Yes, the UI settings were and are set to prune at 2 GB, since that's now the default, but the bitcoin.conf is supposed to take precedence over the GUI settings.

I thought GUI settings override bitcoin.conf, but I'm not entirely sure about that. Still, pruning is not the default as far as I know (it's not for me, in either bitcoind or bitcoin-qt, and I don't have any pruning related settings anywhere).

@PieterWuille I don't know what to tell you. I have certainly not set up pruning in the GUI.

I also don't know what to tell you, except that no, pruning is not on by default.

@PieterWuille So I have a magical or compromised copy of Bitcoin Core, then?

It appears I'm wrong. The "first run" dialog, when selecting a filesystem with less than a certain threshold available space, will suggest have pruning enabled by default. If you then click continue without turning it off, that becomes the saved GUI setting for Bitcoin-Qt.


text:
Browser wallet BTC enabled
The Metamask extension enables the user to send and receive funds (ETH, ERC20s) to external addresses, and thus it is very easy to integrate in current dapps thanks to web3 injection.
Are there any similar solutions that also allow trading of non-ethereum tokens? In particular, I would be interested in integrating a BTC-wallet into an existing dapp, already connected with Metamask, to allow the user to send/receive BTCs to the platform.
Thanks


text:
How to generate new wallet address per token
I'm using getnewaddress bitcoin-cli function to generate wallet addresses but i need one address per token.
Is it possible to use a parameter like seed for random numbers to get the same address when i use the same token?

I finally found a solution. here is the function to get tone address per token:
try:
    addressjson=gateway.getaddressesbylabel(token)
    address=list(addressjson.keys())[0]
except:
    address=gateway.getnewaddress(token)
return address

this way if an address assiciated to the label: token exists we use. Otherwise we generate a new address with label token.


text:
Transaction amount for bitcoin payment gateway
I'm trying to add bitcoin payment for my ecommerce website. I have installed bitcoind on the server and i've set the notify wallet parameter in bitcoin.conf. the issue i have now is how to know that the client have sent the exact needed amount for products they bought.
Actually, the client can send a lower amount to my wallet and notify wallet will be generated. Please help to handle this issue.

Walletnotify needs to be defined in your bitcoin.conf file,
example:
walletnotify = /usr/bin/php some/path/newdeposit.php?tx=%s

In the above example every time there is a new transaction on your wallet, the newdeposit.php script will run and the transaction ID will be stored in a GET named "tx".
Then we can pull the transaction id from the tx GET and put it through a gettransaction command.
$tx = $_GET['tx'];
$getTrans = $bitcoin->gettransaction($tx);

From there we need to look through the API response, see if it has at least once confirmation, and how much was sent. We'll do this by running a loop, checking for the "receive" category. If there is a match you can do whatever you want in your database.
$confirmations = $getTrans["confirmations"];

if($confirmations < 1){
die();

} else {
 $countDetails = count($getTrans['details']);
 for($i=0;$i<$countDetails;$i++){
   $getAddress = $getTrans['details'][$i]['address'];
   $getReceive = $getTrans['details'][$i]['category'];

   if($getAddress == $myAddress && $getReceive == "receive"){
    $amount = $getTrans['details'][$i]['amount'];
    $amount = $amount * 100000000;
    //do something with $amount
   }

 }

}

Walletnotify only fires twice. Once when the transaction first appears on the network, and a second time after it has been included in a block(one confirmation).
Don't forget to multiply the amount by 100 million to turn it into satoshis so you're not working with floats.

If you insist on using bitcoind directly, you could always use the gettransaction rpc call to get transaction info on that transaction.
Hopefully you're using xpubs or at least a list of pre-imported addresses, and not using that bitcoind to manage your wallet/generate new addresses itself though. You shouldn't keep your private key on the live bitcoind node connected to your public ecommerce site.

thank you for your answer! actually i'm working on a project for which i have to add bitcoin payment using rpc user and password! i though i can generate a new address using bitcoind node for each payment and i didn't know that it is dangerous...

Yes, it's risky to have your private keys on a live production server because all someone needs to do is break into your ecommerce site and use RPC calls to empty out your wallet. Generally we like to use xpub keys to generate addresses and that way we can keep the private keys somewhere else.


text:
How to configure bitcoind to wait for X confirmations to make the payment submitted
I'm using bitcoind for an ecommerce website. Walletnotify setting in bitcoin.conf only fires twice (once the payment appears on the network and then when the payment is confimed the first time by a block).
The issue is that i need to set the number of confirmations before the payment is submitted. Could you please give me some help on how to proceed?

You could use the listunspent RPC command. It takes a parameter which allows you to filter out UTXOs which have less than X confirmations.
Example (from the manpage):
bitcoin-cli listunspent 6 9999999 "[\"bc1INVALIDq09vm5lfy0j5reeulh4x5752q25uqqvz34hufd\",\"bc1INVALIDq02ad21edsxd23d32dfgqqsz4vv4nmtfzuklhy3\"]"


text:
Schnorr without R in the challenge
I read the Schnorr and MuSig papers and stumbled over the problem that the use of deterministic nonces in multisignatures is not safe. The adversary would be able to extract the private key from the victim, if he initiates a second signing session with the victim on the same message, but changes his own public nonce.
This happens, because this changes also the partial signature of the victim. (s = r + c*x)
The challenge is constructed in this way : c=H(R||m)  (not key-prefixed version).
I'm wondering, would the exclusion of the public nonce from the H-sig challenge not result in a constant signature of all signers and therefore allow the use of deterministic nonces? The challenge could be c=H(m) or c=H(P||m).
Why is R (public nonce) in the first place included inside the hash function? I'm sure there is a good reason that I could not find.
ECDSA doesn't have that.
Thanks.

So the scheme you suggest is that signatures are pairs (R, s) such that sG = R + H(P,m)P, where P is the public key, and m is the message.
Pick a random s. Set R = sG - H(P,m)P. (R,s) is now a valid signature for P, without knowing the corresponding private key. So this is a trivially forgeable signature scheme.
In ECDSA, the extraction of R's x coordinate, used as a scalar, plays the role of commitment in the signature to R.

You might want to mention that essentially the same problem exists for multiparty ECDSA.


text:
Unable to sweep account using WIF, Green wallet says "No UTXO found"
I am trying to learn the process BTC transactions.
In the process, I generated

Private key : 82e8034869c77744655899d450f9fe2f01fad030bf8af4ea75b310d1a140f554
Corresponding address generated : 2MwVArnwmXQokkhn61Dgi68LnxBEuM1SrxN
WIF : 93NdjSVUcd172o55kErspBPn8jARKq4veeb3pVZyoTSa49bqnpU
Transaction : https://live.blockcypher.com/btc-testnet/tx/8c8140845faef14f860147fcd774355af665098c89cd35b59efee5c8efef3b66/

When I try to sweep using the WIF into Green Wallet by BlockStream, it says No UTXO found. Help me understand what exactly am I doing wrong here.

Are you sure you are using a compressed public key when generating 2MwVArnwmXQokkhn61Dgi68LnxBEuM1SrxN?

That WIF private key does not correspond to that address. Not even if you convert the key to its uncompressed format.
That hex private key does not correspond to either the address or the WIF private key.


text:
Bitcoin stuck in old tablet with really old wallet - Can't send out or back up
I found an old tablet I have and it had some BTC on it.  I got it in 2014 and it has a wallet called 'blockchain' on it which I think is blockchain.info at the time and now blockchain.com
I am trying to send BTC out of it, but I get the following error:

I assume this is because I am using a really old version of the wallet.  So how else can I get this BTC to my coinbase account?
When I try to backup the wallet it asks me for SD location etc but when I open that location on my computer there is nothing there.
Any help on getting the money off of this android wallet would be really helpful thank you


text:
What's going to happen if someone has access to my Bip32 root key?
Can my bitcoin get stolen if someone else has access to my Bip32 root key?

Assuming we are talking about the root private key (xprv) then yes, he will be able to steal all your current and future coins as long as you keep generating addresses from that key. It’s basically the same as him having access to your mnemonic phrase.


text:
To test if my wallet.dat encryption password is correct, MUST I spend coins?
When a Bitcoin Core wallet is encrypted, you (apparently) have to enter the decryption passphrase only when trying to spend coins. You still get to "use" the wallet as in seeing all the transactions and labels.
I'm worried that my saved passphrase (partially on the computer/paper and in my head) is not correct. But to test this, I would have to spend coins... or do I?
I suspect that you'll say: "Use the change passphrase feature!"
But that would just put me again in the same situation. Of course, I had to repeat the passphrase (two times in total) to enter it in the first place, and this would be done again. So I'd again be paranoid that my keyboard has a faulty key or something like that.
Is there some way to "test unlock" a wallet without actually spending any money?

As you are using the Bitcoin Core wallet you can look at the right bottom if your wallet is locked or unlocked in the padlock icon.
Go to Window > Console and type: walletpassphrase "your passphrase" 60
After you type this command and you get an Error "The wallet passphrase entered was incorrect" that means your passphrase is wrong so you can try it again by typing the command again.
If you can enter the correct passphrase you will get "null" as a response and you will see the padlock icon as unlocked.


text:
Why does Bitcoin Core not let met change the label for "requested payments"?
I created a receive address, entered "fdjdsgudhsgudfhgfhdgf" without thinking about it (I was going to describe it properly later) and sent coins to it from a different wallet.
Then I later changed it to "Accurate description blabla" in the "Transactions" tab, which it allowed me to do. Then I looked in the "Receive" tab and saw that it still had the stupid gibberish label there. So I went to change the label... only to realize to my horror that there is no way to change the label there, once created... ?!
Please tell me I'm hallucinating or missing something. I'll forever be annoyed by this if I'm somehow forced to keep that "fdjdsgudhsgudfhgfhdgf" label. Especially as it's now renamed in the "Transactions" tab...

I think what you are looking for may be:
In version 0.21 Open the Windows menu and choose Receiving Addresses

In older versions you had to open the File menu and choose Receiving Addresses

then right-click on any unwanted label and choose Edit.

No such option, but I assume that you meant "Window" instead of "File"? Either way, that strange list does not even list my label at all. Why is there a hidden separate GUI list like that anyway? That really confuses me. This sadly just made me have even more questions instead...


text:
Is there a reason that Bitcoin Core won't add built-in support for Lightning Network, is it dead, and what is the real website?
I've spent a lot of time recently trying to understand what Bitcoin Lightning is. It appears as if the main website for it is this: https://www.bitcoinlightning.com/
However, the site feels extremely "commercial" and sketchy, full of typos and dubious claims/language.
The only other site I can find is https://lightning.network/ , which has a way more "community-y" domain name, but almost goes to the other extreme in that it's entirely devoid of anything useful. Extremely barebones and uninviting. Empty.
I frankly did not get this feeling about the Lightning Network until today. I've heard about it for years, but not in a long time. Is it dead/failed at this point? Even after reading the entire introduction, I just have no idea how I would actually use it.
To me, this seems like vaporware. Just like I could not figure out how to actually use Ethereum to actually do anything, or even download a single sensible client.
What is the problem with "Lightning", really? Do the main Bitcoin developers/controllers have different ideas for the future of Bitcoin? I feel like I have to pull every bit of information out from people, and as if they expect me to automatically understand what to do.
Once again, I have to ask: Am I missing something? Is this promising-sounding thing a... thing?

I've spent a lot of time recently trying to understand what Bitcoin Lightning is. It appears as if the main website for it is this: https://www.bitcoinlightning.com/

I have never seen this website before. It is certainly not the official website of the Lightning Network; the Lightning Network does not have an official website.
The Lightning Network is a decentralized network like Bitcoin. In the same way that Bitcoin does not have an official website (bitcoin.org is not the official website, there's no official anything in Bitcoin), the Lightning Network does not either. Most information with the Lightning Network is found on Github and various product pages for specific software.
Some other Lightning Network websites:

LND software: https://lightning.engineering/, https://github.com/lightningnetwork/lnd
c-lightning software: https://blockstream.com/lightning/, https://github.com/ElementsProject/lightning
Eclar software: https://github.com/ACINQ/eclair

The Lightning Network is most certainly not dead and is, in fact, used by many people and services. It's just a bit hard to find information about it because the people who use Lightning currently are more power users of Bitcoin.
Additionally the Lightning Network is developed by a completely separate group of developers of those working on Bitcoin Core. It is unnecessary for Bitcoin Core to include the Lightning Network and those developers have no effect on the Lightning Network's success or failure.


text:
ELI5: is it true that Base58 doesn't prevent typo, but Base58check does, really because there is a checksum?
I am reading the book Grokking Bitcoin by Manning 2019.
It first said, if John sends the token (bitcoin) to the cafe owner but made a typo in the cafe's address (the PKH), then that token is "burned" and gone forever. Nobody can use it whatsoever.
So to safeguard it, Base58check is used.  But Base58, like Base64, is merely to convert some numbers to different representation of characters, and can be converted back and forth freely by anybody.
But it seems like, what is at work to prevent the "typo" is not Base58, but that there is a 4 byte "checksum", perhaps like some banks, when you tell them the account number, they have a one digit checksum which is the last digit of your account number, to see whether the checksum match.
So the Base58check really depends on this "checksum" to verify that you didn't make a typo, and the odds that you made a typo but the checksum still checks is by that 4 bytes, therefore is 32 bit and is 1 in 4 billion odds?  So it is not Base58 itself, but the "check" part or the "checksum"?

Yes. Every byte sequence corresponds to one Base58 string. Every byte sequence also corresponds to one Base58Check string, which contains a 4-byte redundancy, a checksum.
And this 4-byte-code's only purpose is the detection of transmission errors (typos). Newer Bech32 addresses have a different kind of checksum allowing error correction, up to a certain level in addition to error detection.

So the chance for a missed typo is 1 in 2^32 as the checksum is 4 bytes in lenght?

@RenePickhardt Approximately yes, in practice a little less than that because for example adding a character or removing a character usually changes the number of bytes decoded so the software would nevertheless reject it. As for changing a character, I suspect that changing the character after 1 sometimes causes likewise.


text:
What does it means "get a new address and then make the Bitcoin offline" and make it extremely secure?
I have been reading about Bitcoin, and one thing was about making Bitcoin extremely secure, which is to get a new address so that nobody knows, and then making it offline right away?
Is it like
0.01 Bitcoin    Address 123-456-789-876    encrypted by my public key

and then immediately change my address to something new, and then use the existing public key to encrypt it and publish it to the Ledger / Blockchain?
Should I get a new public / private key pair as well? The thing I don't understand is, don't I need to publish this new address to the Ledger?  If it is, then other people know too.  If I don't publish and other people don't know, then 1 year later, if I want to sell this 0.01 Bitcoin, how does the Ledger know it is a valid address?

Bitcoin wallets never contain money. The only important content is one secret number. This secret number is called a private-key. In most current wallets, the secret number is instead a seed from which a series of private keys are generated in a predictable (deterministic) way.
The only place you can find out how much money you control is in the public replicated journal of transactions called the blockchain. Every wallet has a copy or access to a nearby copy. Blockchain explorers let you search their copy.
You see how much money you have by making a list of the addresses* your wallet created and looking through the whole transaction journal for mentions of those addresses receiving or spending money and add up the amounts. Subtract the debits from the credits to calculate a balance.
The Bitcoin network doesn't encrypt any data.
You use a new receiving address for every transaction so that senders can't see what other money has passed through your address. For privacy.
You use offline wallets for security - a wallet on a computer that is connected to a network can potentially be hacked into. especially if it is a computer you also use for browsing web-pages and for downloading and playing games. An offline wallet in a computer that is never connected to a network is much safer.
All well-formed addresses are valid. They are just large numbers, all large numbers are valid (you may have to add a few checksum digits) but almost all of these large numbers will never be used because there are so many of them.
You don't have to publish addresses explicitly or separately, you just use them in transactions. Unlike street addresses, Bitcoin addresses do not identify people or locations - they are just a random-looking number without any special meaning. You don't have to claim ownership of an address. You can use mathematics to prove you know a secret number (private key) from which that address was derived. The proof doesn't disclose the secret number. The proof is what allows you to spend money, it doesn't matter who you are. The Bitcoin network doesn't keep accounts for people.
A Hierarchical Deterministic (HD) wallet creates new private and public key pairs as needed (usually a few in advance of need) from one master private key.
Your wallet will take care of all these details for you.

Related

Is each Bitcoin address unique?
What's stopping someone from taking over all the bitcoin addresses?
Can two private keys generate the same (public) Bitcoin address?
Can Bitcoins be duplicated if two private keys share the same public key?
Theory: brute-force key search

can my address conflict with another address... what if somebody also has the same exact address and 1 year later he used it first, before I sell my 0.01 Bitcoin? So if I keep the 0.01 Bitcoin offline, how do I check after 1 month that I still have it? I can't really check the public Ledger but have to look at this offline wallet?

If someone else has the same address, you can spend his money and he can spend yours. This will not happen because there are more addresses than there are atoms in the galaxy.

@deeper-understanding That's why you input every new address (and ONLY) the address, as a "read only account" into the wallet software of an online computer/phone (or search it in a blockchain explorer) first, just to be 100% sure there have never been any transactions on your new address before.

@RedGrittyBrick I might suggest a slight edit to "master private key" in your post, as legacy P2PKH wallets only use a "private key" and bech32 et all use a "master seed"/"master seed mnemonic".

@RobhercKV5ROB: Suggestion accepted, thanks.


text:
Can I be the transaction processor and take in transaction fees as income without mining a Bitcoin?
Can I be the transaction processor and take in transaction fees as income without mining a Bitcoin?  Would the fees be so low that it is not practical?
Or is it true that I have to be a miner in order to process any transaction? However, after year 2140, when all Bitcoins have been mined, then will people only use the computer to process transactions without doing any mining?

I think this another case where a literal interpretation of Bitcoin terminology can be misleading
Mining
The primary purpose of the process referred to as mining is not the creation (digging up?) of new Bitcoin money. The primary purpose of mining is the production of new blocks in the blockchain - the public transaction journal that is replicated throughout the Bitcoin network by all full participants, including many ordinary wallets.
The miners are currently granted, by the other nodes in the network which accept the mined blocks, a fixed amount mining-reward where the reward amount halves every few years according to a fixed halving rate that is part of the consensus rules. But this is not what causes mining to be hard work - adding to the new block a record of a mining reward of freshly created bitcoin takes almost no computational work, compared to the main task of securing the blockchain which makes that block valid.
Mining is maybe more like building an ever-lengthening structure by reusing old bricks from older sections and being allowed by the public to create two new bricks for each section completed.
The word mining is misleading if it causes you to think that the primary purpose is discovering new money. It isn't like mining gold where the primary purpose of miners is to find gold.
Processing
Lots of people process transactions, in order to create new draft unconfirmed transactions, to validate unconfirmed transactions being passed around, to calculate balances, etc. Only miners add new transactions to the public replicated transaction journal we call the blockchain.
Confirming
Miners effectively "confirm" transactions and we talk about numbers of blocks added over a block containing a transaction as "confirmations" of that transaction.
Confirming transactions is a special type of processing. It is part of mining.
Transaction fees go to whoever first confirms a transaction.
Validating
Every member of the Bitcoin network, including wallets, performs at least some validation of the new blocks and new transactions it receives. This is a significant part of the routine processing of transactions and blocks. This is the way that fake blocks and attempted fraud is detected and rejected. This is an example of an important type of processing of transactions that is not directly rewarded by transaction fees.

Confirming transactions is processing that only miners can do
Everyone is a transaction processor. Only miners are transaction-confirmers and block-makers. Only block-makers get a special reward of new money for each new block and only transaction-confirmers collect transaction fees from the first confirmation of each transaction. These two jobs are indivisible because confirmations are intrinsically part of new blocks.
For people new to Bitcoin, the word "miner" doesn't immediately bring to mind transaction-confirmation or block-making. It brings to mind digging down through soil and rock to find precious matter buried there. This isn't helpful. People realise its only an analogy but they don't know how far that analogy is valid.
So we shouldn't take terms like mining too literally. They often have subtly or significantly different meanings in the Bitcoin world than they do in everyday usage.

I have to slowly digest everything you wrote. But if mining if finding new blocks, and mining will stop in year 2140, then what about all the new transactions not having new blocks to put the data in?

Mining will not stop in 2140. Only the subsidy will stop; mining will continue, and mining income will be purely fees.

No. When miners mine blocks, that is "processing transactions".

so again, when mining stops in year 2140, then what about another 2 billion or 3 trillions transaction come in, then how can we have new blocks?

And who says that mining stops? Mining REWARDS stop, but the miners are still getting the transaction fees. The concept is that slowly the Tx fees will be the miner rewards, not the per block payout.


text:
How many computers are connected to the "Ledger" or the Blockchain of Bitcoin?
Somebody asked me about Bitcoin and I explained to him many many computers are connected to the Blockchain to keep records of all the accounts (or to keep addresses of the bitcoin and owner?). But how many computers are connected to it, as of recently or 2020?

Note: the Blockchain doesn't keep records of accounts and doesn't keep track of owners. The Bitcoin network doesn't care about owners in the normal sense - it only cares about proofs of knowledge of private-keys corresponding to addresses. Anyone who can prove they know a key is allowed to spend associated money. Ownership of money in the Bitcoin network is not the same as legal ownership.

Somebody asked me about Bitcoin and I explained to him many many computers are connected to the Blockchain to keep records of all the accounts (or to keep addresses of the bitcoin and owner?).

The blockchain is the data structure they collectively maintain. It is not an "entity" that can be connected to. The nodes are just connected to each other, and tell each other about updates they learn about.
It's hard to count how many nodes there actually are. There are sites like https://bitnodes.io that keep track of reachable nodes. However, there are many nodes on the network that only connect out to others but can't be connected to; probably in the 100000s.
A further complication is that not all of these nodes are equally relevant. It's easy to spin up a few hundred nodes on AWS, that nobody uses for anything, but do make it look like there are lots of independent users. Worse, there are also a lot of spy nodes on the network that don't do any work, except trying to monitor which nodes are the origin of particular transactions.
In the end, if you run a node yourself, it doesn't matter. It validates everything the network tells it, so if you do that, the part that matters is: 1 node, your own.

I added up the "11476 Bitcoin Core nodes" on https://coin.dance/nodes with the other nodes and that number is less than 20,000... could that means less than 20,000 nodes?

These sites only count reachable nodes. It is very hard to know the actual numbers of others. None of these sites are authoritative. It's just fundamentally hard to know about unreachable nodes.

If it is 83120 nodes... or say, 100,000 nodes, then does that mean one country can just buy 50,000 Raspberry Pi, which would be 50,000 x US$50 = US$2.5 million and be able to become the 51%? It they spend $2.5 million and can steal $20 million, that's immediately very profitable

Computers are not 'connected to the blockchain', as that implies that the blockchain is some sort of central entity in the network.
In reality, the network is just made up of a huge number of peers, known as 'nodes'. A 'full node' is one which has fully verified the state of the network, by working to verify all transactions that have occurred since the genesis block.
So each node keeps its own copy of the blockchain, and is connected to other nodes, which each have their own copy as well. The rules of the network work to allow these nodes to all remain in consensus about what that blockchain looks like, despite the fact they each keep their own record of it.
How many nodes are there? It's tough to say. Many nodes are made to be intentionally hidden, for security or privacy reasons. Many exist on the tor network, and some receive their information via satellite feed. Some websites will query nodes on the network in an attempt to figure out numbers, but ultimately the answer to this question is not really knowable for certain.
See for example:

https://luke.dashjr.org/programs/bitcoin/files/charts/historical.html
https://coin.dance/nodes
https://bitnodes.io/


text:
Why is Electrum able to generate .psbt from the address only but BitcoinCore requires a Public Key/Descriptor?
When generating a .psbt it suffices to merely import the address (hashed Public Key) in Electrum while in BitcoinCore (for the same address) I also would have to import either a corresponding Public Key or Descriptor. Is there a reason why BitcoinCore requires these additional inputs and Electrum does not?

Bitcoin Core requires the additional data so that it can estimate the size of the final transaction when it is choosing which inputs to spend. It is probable that Electrum makes assumptions about the size because it is standard, however Bitcoin Core does not do that. Instead it will make a fake scriptSig/witness to determine the size of the input, and doing that requires knowing public keys and other information provided in descriptors.

Does it need to know the size due to fee calculation? Or is there another reason I am unaware of? I would like to use BitcoinCore exclusively while avoiding to expose the public key in a hot wallet (so that its also protected by hashing). Is importing a descriptor safer than importing a public key?

It needs to know the size in order to perform the fee calculation.


text:
Bitcoin Whitepaper - Details of digital signature used in transactions
I have read the whitepaper a few times now, I just want to know precisely what Satoshi meant by 'digitally signing a hash of the previous transaction and the public key of the next owner and adding these to the end of the coin.'
Forget everything you know about Bitcoin, imagine you are reading this for the first time in late 2008 and you have only read the abstract and sections 1&2. Does he mean:

Get hash of previous transaction, Prev_Hash
Get public key of whom we are transferring coin to, Pub_Key
Concatenate these two strings, New_String = concat(Prev_Hash,Pub_Key)
Digitally sign New_String using whatever algorithm (no mention of specifics in
whitepaper)

OR

Get previous transaction, Prev_Tx
Get public key of whom we are transferring coin to, Pub_Key
Concatenate these two strings, New_String = concat(Prev_Tx, Pub_Key)
Hash New_String, Hashed_New_String
Digitally sign Hashed_New_String using whatever algorithm

Or any other ways of going about it, such as signing a hash of Prev_Tx, and then Also signing a hash of Pub_key and having 2 signatures.
I want to explore evolution of Bitcoin in a chronological way by trying to put myself in the shoes of the people working on it at the very start and so I want to know what Satoshi really meant. Can't seem to find anything online

I think the first would be best since then when you decrypt it you are left with a string and then if you know, for example, the first 32bytes refers to the Prev_Hash (because it is output of SHA_256), then you can just split this string into 2 and the first string would be a digital fingerprint of the previous transaction you are referring to and the second string would be the public key you are transferring to. This is all a miner would need in order to validate a transaction, is it not?

Also, exactly what is included in each transaction, 3 things? Pub_Key of next owner, hash of previous transaction, and the signature? [Pub_Key, Hash(Prev_Tx), Sig]? So when a miner receives this tuple, they looks at second entry to get "Tx_id" and find it in the blockchain, they then use the first entry of Prev_Tx to decrypt the Sig (3rd entry) and then they can use the first 2 entries and do: concat(Prev_Hash,Pub_Key) and check to see if both are equal?

"TX_id" being defined as the hash of the transaction...

You can (and should) edit your question rather than adding comments. In stackexchange websites, comments are temporary and disposable. Comments are mainly to assist an author to improve a question or an answer.

Ah okay, I will do in future, it's my first time on here so I wasn't aware

According to some of his other writings, Satoshi had a working prototype when he wrote the whitepaper¹. The whitepaper gives a high-level overview of how Bitcoin works, and to that end, Satoshi simplified some of the concepts.
For example, "We define an electronic coin as a chain of digital signatures." is a simplification, because transactions can have multiple inputs and multiple outputs. The resulting structure is a directed acyclic graph, not a chain.
Likewise, the "Hash" box being composed from previous transaction and public key of the recipient is a simplification. The first output type, Pay to Public Key (P2PK), locked funds to the recipient's public key (literally: <pubkey> OP_CHECKSIG). Meanwhile, inputs identify which "coin" they're spending by referencing the transaction output's outpoint, txid:vout, composed of transaction id and position in the output list. For signing, we then create a digest of the entire transaction to commit to the exact inputs and outputs as well as other details of the transaction. This hash of the transaction to be signed, the sighash, is then used as the input for the signing algorithm, along with the senders private key. As you can see, the whitepaper does not provide nearly enough information to deduce the actual implementation.
The whitepaper does a fine job explaining Bitcoin in broad strokes, but the actual implementation details only became apparent when the software itself was published. Here's for example a copy of the original release's codebase annotated by Jeremy Rubin.

¹ A whitepaper is a document meant to give a concise overview of something, usually to entice readers to learn more about a product or proposal. (And it worked! Read it some nine or ten years ago, I'm still fascinated.) However, it's a marketing document. It's not meant to be a comprehensive technical document. I don't understand why people are acting as if it were some form of ultimate truth (in this vein, see e.g. this errata).

Thanks for taking the time to answer. I was trying to put myself in the mindset of someone who would've been on the mailing list at the time and seen this whitepaper.  If you saw Satoshi's email and whitepaper back in '08 and you had only read the abstract and introduction and then you read section 2 (transactions) how would you have interpreted what he was saying? Would you reply to him and tell him to be a bit more precise? Or do you think he deliberately left it a bit vague as if to say 'the implementation could very, but this is the gist of it'?

Also, do you know where I would find a copy of the original code? I don't think his prototype was ever released publicly, not sure, where would I find earliest version?

@Kryptonite: I added a link to a copy of the originally released sourcecode.

Oh, just seeing your previous comment. I've added a comment on how I perceive the whitepaper.

Okay that's helped to clear a few things up, especially the erratum you linked to - that's exactly what I was after, thanks.

Great, glad to hear it was helpful.


text:
How a user in the Bitcoin network verifies a payment?
I'm a beginner who's trying to learn the idea behind Bitcoin.
In the Bitcoin whitepaer it's said that:

A user only needs to keep a copy of the block headers of the longest
proof-of-work chain, which he can get by querying network nodes until
he's convinced he has the longest chain, and obtain the Merkle branch
linking the transaction to the block it's timestamped in.  He can't
check the transaction for himself, but by linking it to a place in
the chain,  he can see that a network node has accepted it, and
blocks added after it further confirms the network has accepted it.

Here, A user who's gonna verify a transaction, how does it verify that particular transaction in the blockchain network?
what's the role of Transaction ID and UTXO? and how do they help the transaction find the block in which it resides to verify himself?

There's a distinction between verification and confirmation.
Verification is checking that some data, such as transaction data, meets all the Bitcoin rules. For example you could check that the sum of outputs of a transaction does not exceed the sum of its inputs. You could check that none of the inputs have already been spent. You can check signatures. You can check that the locking script is well-formed. There are many such checks. Verifying a transaction does not tell you whether the payment has been made (received).
Confirmation of a transaction is nothing more than a count of the number of blocks that have been added to the blockchain which are either the block that contains that transaction or which are linked back to that block. 0 blocks means unconfirmed, 6 or more is regarded as irrevocably confirmed. Full nodes can count this number of blocks themselves. So when you make a payment, confirmation of your transaction is what you are interested in. Confirmation is what tells you the payment has happened, that it has been irrevocably received.

A transaction-ID is simply a representation of a number that uniquely identifies some specific transaction. This Transaction-ID can be used by a node to index a list of transactions to aid retrieval. For example some software might create an index of Transaction IDs that points to the block that contain each transaction.
An unspent transaction output (UTXO) is effectively an unspent coin. A node can and should check that the inputs to a transaction of interest are all UTXOs. This is part of how Bitcoin prevents double-spending. Each full node typically keeps a list of all UTXOs and removes spent ones and adds newly created ones as it processes each new transaction.

Simplified Payment Verification is the section of Nakamoto's white paper that contains the text quoted in the question.
Full nodes do all the verification themselves. Lightweight nodes, such as those that use Simplified Payment Verification (SPV) rely to some extent on other nodes to perform verification. Unlike full nodes, they partially trust some other nodes to be honest. So the quoted text does not apply to full nodes.

Thanks for the reply. I get what those terms are now.

But I wanna know if a user (normal user not a full-node) want to verify a transaction if right or not how he does that.
I mean, How the user finds the block that the transaction is in.
I don't know if I'm thinking it the right way or not, sorry.
But I thought a normal user has access to the transactions, and to verify them It needs to find the particular block the transaction is residing in.
I don't know if I'm right or not.
If I'm thinking the right way by any chance, how the transaction finds the block it resides in.

A normal user can use a full-node Bitcoin wallet like Bitcoin core. The whitepaper section you quoted describes roughly how lightweight wallets do verification.


text:
How to serialize the CBlock Class in C++ and print the hex to terminal?
I have tried to serialize as follows without success: bitcoin tag v22.0 from github.
In the file src/miner.cpp and the function
function std::unique_ptr<CBlockTemplate> BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn)
I have placed the line std::cout << pblock << std::endl; at the very end before it returns (Here). The value that is printed to the terminal is:
0x7fc9f4009410
This is too little data, I would expect the entire block data to be serialized and printed in hex. What am I doing wrong? I know pblock is a pointer to a CBlock class, but it should still work.
What is the proper way to serialize CBlock data, and also the CBlockHeader data?
Thank you!

Since it is a std::unique_ptr, the << operator outputs the pointer value rather than the object pointed to. Try dereferencing the pointer first.

Since pblock is a pointer, you are printing an address. Dereference it to get the format you want:
std::cout << *pblock << std::endl;


text:
Customer pays in EUR and I receive bitcoin anonymously
I run a small business, and up to now I have always demanded to be paid in cash for my services.
I want to allow my customers to pay electronically through wire transfer, while remaining anonymous myself.
Is there a service which acts as a flexible always-ready EUR-to-bitcoin converter, such that my customers can pay electronically to a bank account owned by that service, whereafter an equivalent amount of bitcoin are transferred to me?

This is basically the definition of facilitating money laundering. If the funds were stolen, how would the service get the money back from you? If you're a terrorist, how would the service know?

No. I don't think there are any good services that will allow customers to pay with fiat and have you receive Bitcoin even if you were okay with completing KYC, but regardless, no legitimate fiat payment processor will allow you to stay anonymous.


text:
What is db viewer for Berkeyley DB and LevelDB?
Wallet uses Berkeyley DB. Block and transaction use LevelDB to index.
In bitcoin data we see the db data folder.
Is there some tool to view these DB data ? I mean the database management tool.

leveldb is a key value store. there’s little about it that is human readable.

You can use db-util to view BerkeleyDB contents.


text:
My Bitcoin Core Sent Bitcoins Without My Authorisation
I am running Bitcoin Core version v0.19.0.1 (64-bit) on Windows 7, and on 27 February I noticed that it sent 0.0045 bitcoins on its own.
The wallet is encrypted at all times and I enter the passphrase to send as required.
I did NOT initiate the Send.
I am greatly concerned. The PC is regularly checked for virus and malware.
The transaction details are:-
Status: 320 confirmations
Date: 27/02/2020 16:37
To: bc1qz944eascuflps4nmsc0w5cpz9p3cxdyqjmmqw7
Debit: -0.00447194 BTC
Transaction fee: -0.00002806 BTC
Net amount: -0.00450000 BTC
Transaction ID: 3c138afea7d634ce5e8b608e1a751122d6d33ba9bd8e00a02de02cb57aad039d
Transaction total size: 214 bytes
Transaction virtual size: 133 bytes
Output index: 0
The previous transaction:-
Status: 455 confirmations
Date: 26/02/2020 18:27
From: unknown
To: 3Hb61TTkbxPTUCA6EvUStb8PPMZUjPASfd (own address, label: coinfloor)
Credit: x.xxxxxxxx BTC
Net amount: +x.xxxxxxxx BTC
Transaction ID: 5697d9b298647c216a727f95dbdecfc658a9f8da3c45f98bc8c8a2439c8f1cfc
Transaction total size: 404 bytes
Transaction virtual size: 214 bytes
Output index: 1
The last time I sent was with this:-
Status: 43126 confirmations
Date: 17/05/2019 07:25
To: Coinfloor 5 10 13vk63282b4FkhyyV9mmgQTxpao9ECnnmB
Debit: -x.xxxxxxxx BTC
Transaction fee: -x.xxxxxxxx BTC
Net amount: -x.xxxxxxxx BTC
Transaction ID: d23b6c3f42339ace670d0ebdca4eabcfe45c34b0ccab44c9767a519682e415ba
Transaction total size: 932 bytes
Transaction virtual size: 530 bytes
Output index: 1
I would like to know how this could have happened, because I definitely did NOT initiate this send.
Any suggestions about how to diagnose this? I expect there's no way to recover the 0.0045 coins.

Check if it is really Bitcoin Core. Check if you really don't have Trojan horse.

When was the last time when you enter the password on your PC for your wallet?

Didn't you reveal priv key to one of your addresses? Looks like BTC from only one address was sent. BTC from other addresses were not stolen.

I last used my passphrase on 17th May 2019.

So, there was more than 0.0045 in my wallet. Why was a larger amount not taken?

My debug log also shows strange entries at the time, see below

cache=35.4MiB(261525txo) warning='56 of last 100 blocks have unexpected version.'
2020-02-27T16:37:13Z socket recv error An existing connection was forcibly closed by the remote host.  (10054)
2020-02-27T16:37:42Z [default wallet] AddToWallet 3c138afea7d634ce5e8b608e1a751122d6d33ba9bd8e00a02de02cb57aad039d  new
2020-02-27T16:38:15Z tor: Thread interrupt
2020-02-27T16:38:15Z torcontrol thread exit
2020-02-27T16:38:15Z opencon thread exit
2020-02-27T16:38:15Z addcon thread exit
2020-02-27T16:38:15Z Shutdown: In progress...
2020-02-27T16:38:15Z net thread exit
2020-02-27T16:38:15Z msgh

I always download new versions from https://bitcoin.org/en/download

The "warning='56 of last 100 blocks have unexpected version" seems odd. Is this significant?

I've done virus scan with 2 different checkers, Trend Micro and Eset NOD32. Also Malwarebytes scan. None of these show any problems

@SalP Block versions are not relevant, it's a technique miners apply to increase their efficiency, called Asicboost.

Adding to CoperNick's answer - Check the build signatures. ie sha256() some files, and compare it to the release signatures. there are guides on-line, and it's mentioned in the bitcoin-core download page.
best of luck

The SHA256 for the bitcoin-qt.exe matches

The SHA256 for bitcoin-qt.exe matches a clean install from the setup.exe, and its SHA256 is as given on the download page of bitcoin.org.  That would indicate that the bitcoin core is as created by the developers.

You do not explain why you believe your bitcoin core process sent the bitcoins. It is perfectly possible (from looking at your logs) that it was not sent from your bitcoin core, but from elsewhere on the bitcoin network. It may be possible to track down where the tx was sent from if enough nodes log the IP address of where the transaction was initiated, but there is nothing in your logs to suggest it was initiated from your node. In fact, your logs suggest that it was not initiated from your node.
Therefore, at some point the private keys for your bitcoin address were obtained. This could have been done if your computer contains malware, perhaps a key-logger, which was able to capture your wallet password, in combination with the wallet file.
The output that was spent was populated on 2018-12-28 19:12, but the private key associated with the address could have been created years earlier, as Bitcoin Core by default creates 100 addresses when it is first started, so the private key to this address could have been stored on a computer of yours for some time, perhaps even in an wallet with no password, if you initially didn't enable the password until later. So the time of intercept (of the private key) is theoretically any time between when you first ran Bitcoin Core and when the theft occurred.

The reason I am convinced that the bitcoin core process sent the bitcoin is because bitcoin core was running at the time.
I quite often just run it because I read that it's good practice. I noticed a message pop up on the screen that it had sent the bitcoins.
I saw it with my own eyes!
To say I was shocked is an understatement!
I shut down bitcoin core, and removed the wallet.dat file.
After re-running virus and malware checks, I re-booted the PC

I transferred the balance of my wallet to an exchange, to guard against any more leaks from my wallet.  So, I could start again with a new download of bitcoin core, and a thorough virus check.  Would that be enough, or should I buy a hardware wallet?,

My first use of bitcoin core was in 2013, and I  believe I never used bitcoin core unencrypted

@SalP: What you describe actually doesn't prove that Bitcoin Core sent the coins.  If I remember correctly, I believe the same popup will appear when Bitcoin Core notices a transaction broadcast on the network that spends your coins.  The precautions you took are sensible, but even so, what you describe is also consistent with someone having obtained your private keys some other way, and spending them from a completely different node.

@SalP indeed - when you run bitcoin core, it has to download the blocks to catch up with the blockchain. When it downloaded the block containing the transaction you're referring to, it would notify you at that point. The point of download, not necessarily the point in time that the transaction was broadcast to the network.

See in https://github.com/bitcoin/bitcoin/blame/971cad475f38011cf33a66671f1d5e570fd8df33/src/wallet/scriptpubkeyman.h#L48 at the time you wrote this post it was 1000 addresses already, instead of 100.


text:
Suggestion for Airgap Bitcoin Core - Comments?
Lesson learned the hard way: I lost a small value of bitcoins due to foolishly  backing up private keys to a file which was uploaded to a cloud server. My wallet used to be bitcoin core.
Right now my keys are in electrum with an air gapped laptop wallet and a watching wallet.
I'd prefer to use bitcoin core because having my own blockchain is preferable.
I know you can use bitkey, but as a Windows user, but it seems too complex.
I'm considering the following solution, and would welcome comments.
Currently, I still run bitcoin core with an empty wallet, on my desktop PC connected to the internet. The blockchain is on a separate SSD disk and I run the empty wallet to keep the blockchain up to date.
I would then keep a core wallet holding the keys to my coins, on the air gapped laptop. When I want to send or receive bitcoins I would move the SSD to the laptop. The SSD would use a SATA to USB connector, so it would be an easy plug in. As soon as I'd done the send or receive I would move the SSD back to the internet PC.
I assume that the blockchain on the SSD would resynch the send and receive transactions.
I could use Electrum Personal Server, but this means running other stuff which I don't understand. Armory is another option, but I'd prefer to stick with core.
Is my approach feasible? Secure? Overkill?

The approach does work.
From the offline setup I sent myself a small amount of BTC.
When the SSD was returned to the online computer, it caught up. The sent coins have been sent and subsequently received.
Apart from the potential inconvenience of moving the disk containing the blockchain between the 2 computers, it's very simple, and it works.

If I understand you correctly, Im not sure your system will work, because your air-gapped laptop is never connected to the internet, the transaction wont get broadcast.  You can try it, I may be wrong.
I have done a similar process using rawtransactions, where you prepare the rawtransaction on an online computer, using an explorer API (https://chainz.cryptoid.info/api.dws is one that I found works) you get the tx input and vin, create the rawtransaction and save the hex output to a small USB (smaller the better).  Then on your airgapped machine use signrawtransaction to sign it, then transfer that hex output back to the online machine and sendrawtransaction.
I automated that process in Python, you can see how it works in more detail on my github https://github.com/Scalextrix/coinofflinewallet
It worked with QR codes as well for a fully airgapped solution, but I had a lot of trouble with the Python webcam/QRcode libraries and ended up not bothering.  My code is not extensively tested, its really an example, you should validate it fully before use.

I think I might try it.

It seems to me no different to disabling the network connection and then reconnecting, which you can do from the bottom right of the GUI client. Any transactions sent to the local blockchain should be synchronised when the SSD is moved back to online computer.


text:
Is value of a wallet derived by iterating over every block in the chain, scanning each transaction?
If I open Metamask or Coinbase, for example, I see a balance. This balance is a flat number, similar to what we'd see in a bank to more intuitively check my networth or balance of a particular wallet.
I'm wondering how this is done, from a programmatic perspective. I think I kind of understand this idea of UTXO where if I have coins like (1 coin, 3 coin, 5 coin) and Alice charges 7 coin for some service, I would send 5 coin and 3 coin as input. The output would be 7 coin -> Alice wallet and 1 coin back to my wallet. (Let's ignore fees).
But how do I derive the balance of my wallet?
Intuitively, I get I could start at the genesis block and iterate linearly through the blockchain, however, each block has N transactions, so this seems like a simple derivation would entail scanning N blocks * M transactions/block.
This seems inefficient. How can I understand this better?

Your intuition is correct: in order to compute a wallet's balance bitcoind will scan through the entire blockchain, looking for a set of addresses that correspond to its own private keys. As you point out this is a rather time-consuming task. However we should note here that bitcoind needs to iterate through the blockchain and verify all transactions for validity anyway when bootstrapping, so checking if an address belongs to its wallet is rather trivial, compared to verifying scripts and signatures, so it's not adding much to that bootstrap time.
Once you are bootstrapped, you know that anything up to the point you verified is not going to change, so you can just incrementally scan new blocks as they come it, which often takes less than a second with todays code.
It is possible to build indices on which address was used in which transaction, which can help locating transactions that involve a certain transaction quickly, and there are wallets that make use of this (electrum, and all hardware wallets), however this is usefull mostly when you don't know all addresses at verification time, and you want to serve this information to users so they don't have to sync it themselves. To build the index however they themselves had to verify the entire chain. This is true for all blockchains by the way, either you are verifying the chain from its genesis, or you are trusting a third-party not to lie by omission (they can not report addresses that are yours), this is for example the case for metamask as well.
One final detail is that some seed schemas have a birthday date / height, which allows a wallet to start scanning from that height, since the seed basically says "don't bother scanning below, I had no addresses before this time". This saving is less efficient the older the seed is, since the skipped part will become smaller compared to the part that needs to be processed.

The first time you run bitcoin it will download and process every single block and will generate an index of all unspent outputs like
{ ‘outputABC’ : 1 btc
  ‘outputDEF’ : 3 btc
}

etc. When this table is constructed you use it to lookup all the values of the outputs owned by each address in your wallet.

Thanks, that makes a lot of sense!


text:
How does PoW ensure that all the transactions are recorded and that all the transactions are valid?
I just started to study blockchain and bitcoin yesterday and came up with some questions.

If miners can select transactions when creating a block, how is it ensured that all the transactions over every 10 minutes are recorded? For example, if a miner selected only a half of all the transactions over 10 minutes and found a valid hash, how can the other half of the transactions be recorded?

The answer to the first question might render this question unnecessary, but if miners can select transactions when creating a block, how is it ensured that the first one didn't include any invalid transactions over the 10 minutes?

Also, why is only the first one to find a valid hash rewarded while the first one might have left out some transactions during selection (although an answer to Q1 might render this worry unnecessary)? Is there any security reason behind choosing the first one? For example, is the first one considered to be the most trustworthy because it takes time to make invalid modifications on previous/current transactions?

The fourth question came out while I was watching 19:43~ of the following video:
https://www.youtube.com/watch?v=bBC-nXj3Ng4. It is understandable that Alice will fail to cheat because Alice can't create invalid blocks faster than the rest of the miners. However, what if Alice broadcasted the first invalid block to every other miner? In that case, wouldn't all the other miners find next blocks based on the assumption that Alice's first invalid block is valid so that Alice wouldn't have to exceed all the other miners every time?

Thank you.

Why would anyone assume an invalid block is valid?

It is not ensured. Miners can run any software they want to determine which transactions to include in a block. The game theory makes censorship hard however: it only takes one miner to include a transaction in a block for it to become valid. If a given transaction is not "selected" for a block, it remains in the mempool, a collection of unconfirmed (not in a block) transactions.

This verification is exactly what nodes do. Whenever nodes receive a new block, they verify the signatures and validity of every single transaction in that block, including whether or not the coins each transaction attempts to spend are spendable. The linear nature of the blockchain ensures that there all transactions are objectively valid or invalid at certain block heights/times.

No one chooses the "first" one to find a valid hash. The valid blockchain is the chain which includes the most work. For simplicity's sake, this is almost always the chain with the greatest number of blocks in it. When a miner finds a valid hash, they publish their block and all other miners accept it (assuming its valid) and mine on top of that block. Only one miner gets the reward for that block, but other miners hope to get rewards by finding subsequent blocks.

See #3. Blocks are not valid/invalid based on the time they were mined. The longest chain is always valid, even if a shorter chain has blocks that were mined before the blocks in the longer chain. Read more here: What Is Proof-of-Work?


text:
Best block explorer for windows with btc core synched
I have btc core synched on my windows 10 pc. What would be the best block explorer software to use to locally search the blockchain? Id prefer to stay on windows, but if there is absolutely no option I can switch to linux.

Your available options will largely depend on whether you use Bitcoin Core in pruned mode (in which case you won't have much to explore) or you have the full blockchain stored locally (then you can explore it with some limitations), and whether you're ready to dedicate another 50-100 GB of disk space for your block explorer to index the blockchain (then you can do pretty much anything).