text:
Why do hard forks need special rules to allow replay protection?
I've seen a lot of methods hard forked chains use to provide both strong and opt-in replay protection so that transactions are only valid on one of the chains. Examples of such rules include using a transaction-invalidating output, and using a specific OP_RETURN string.
My question is why are any of these mechanisms neccessary to protect a holder from replay attack? More specifically, why can't a person who holds coins pre-fork protect themselves simply by spending a pair of coin-splitting transactions on each chain like this:

Alice has sole knowledge of the private key needed to spend UTXO A which is valued at 1 "LegacyCoin" before the hard fork.
Hard fork creates a new coin called "NewCoin" based on the blockchain history  of LegacyCoin.

UTXO A now contains 1 LegacyCoin according to the LegacyCoin network and 1 NewCoin according to  the NewCoin network because that's how hard forks work.

Alice creates and signs two transactions: one that spends A to B and one that spends A to C (both of which Alice also has sole knowlege of the priavte key).
Alice then simultaneously broadcasts these signed transactions on the two networks: broadcasting the transaction A->B only on the LegacyCoin network and the transaction A->C only on the NewCoin network.
Both transactions get confirmed on their respective networks and now Alice has effectively split the original value of UTXO A into the two blockchains in complete absence of any replay protection mechanisms implemented by NewCoin. The coins can now be concidered split because even if she spends B, no one else can spend C (and vice versa) becuase they are different UTXOs and require different signatures to spend. Also, obviously no one can spend A because it is already spent on both chains.

Why is statement in step 5 not true? Where does this logic break down requiring actual replay protection mechanism to be implemented by hard-forked chains?

It's true that what you propose would effectively split the coins and prevent further replays, if it succeeded.
However, it suffers the following problems:

Every person who wants to split their coins has to create an additional transaction to do so (and pay a transaction fee).  Presumably nobody really wants to be subject to replay attacks, so everyone is going to want to do this.  That's costly for users, and it's a huge number of transactions.  For instance, Bitcoin currently has about 50 million utxos; splitting all of them would require enough transactions to fill some 5000 blocks, so the entire network would be bogged down for several weeks, and transaction fees would skyrocket.
Some people may not know about this requirement, and may unwittingly subject themselves to such attacks.  Even if it's done automatically by software, they may still be using old LegacyCoin software that doesn't do it.  That's not very user friendly.
An attacker can defeat it, at least temporarily.  Suppose Alice broadcasts her A->B and A->C transactions as you propose.  Mallory, an evil NewCoin miner, sees the A->B transaction on the LegacyCoin network, and decides to try to replay it on the NewCoin network.  He puts A->B in his candidate block (which he can do because it is replayable!) and is lucky enough to mine it before anybody else confirms the A->C transaction.  Now Alice's coins live in utxo B on both networks, which is exactly what she is trying to avoid.  Best case, she notices this happens and has to start over (and pay another transaction fee), until she succeeds in having honest miners confirm the right transactions on both networks.  Worst case, she doesn't notice and is unknowingly vulnerable to future replays.

The LegacyCoin network and NewCoin network are not distinctly separate from each other. Unless network magic and ports are changed, LegacyCoin nodes will be able to connect to NewCoin nodes and likely many will be connected to each other. So if you broadcast transaction A->B, on the LegacyCoin network, it will likely reach a NewCoin node which then causes the transaction to also be relayed on the NewCoin network.
Furthermore, even if the networks are distinctly separate, people can operate a node on both the LegacyCoin network and NewCoin network. Without replay protection, someone can take transactions that they are hearing on the LegacyCoin network and broadcast them through his NewCoin node and vice versa.
Lastly, two way replay protection should be mandatory because the process of splitting your coins with UTXO tainting is rather difficult. You need to have decent technical knowledge to do it and you will likely need to have modified software in order to do so. It is not something that ordinary users are likely able to do without some very specific instructions and there is a high probability that they will mess it up and possibly lose coins.

So to clairfy, your saying the process breaks down at step 4: Alice may not have the ability to chooses that the transactions get broadcast on separate networks because they might not actually be separate networks and even if they are there's still a chance someone can intercept one of her transactions and replay it on the "other" network anyway without Alice's knowledge or permission?

Yes. That is why it is called a "transaction replay attack"; your transaction is being replayed onto another network possibly without you knowing at all.

Right. I know it's possible to defeat this method of replay protection but wouldn't the replay event be unlikely to happen unless there was economic incentive for someone to do it (which presumably wouldn't exist bc Alice still controls private keys to both B and C)?

It's a more general reason for having replay protection. Those who don't know how to do this sort of splitting will end up having their normal transactions replayed either accidentally or possibly by the person receiving the transaction who wishes to gain coins on the other chain.


text:
Method to create P2SH address
I am trying to create a htlc smart contract manually by writing a script and finding the hex using the btcc compiler. Now node A has to create a transaction using that hex and lock money in that p2sh address for upto 150 blocks. If node B can provide the secret value before 150 blocks then he can claim the coins and if not the it will go back to node A. I am new to this concept and where ever i read about creating transactions they are explaining in a theoretical way and I could not understand what to do the redeem script hex and how to lock money and how will node B claim it before 150 blocks and how will the raw transaction created by node A will be accessible to node B. Can you please give a reference or steps to follow to understand and implement it? I want to start from the basic with step by step guide.
Which documentation should i refer for doing it and what commands should be used and how it should be used?
How to create the p2sh address. And another problem i am facing is that i am getting different hash160 everytime in different website. Not sure which one is accurate. Please guide me.
P.S: I am running bitcoin in regtest mode.
Collapse

Can you share the script you are writing and what you've tried so far?

https://bitcoin.stackexchange.com/questions/74753/htlc-hash-time-lock-contract-using-bitcoin-qt

I am trying this.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Do we have to trust at least one DNS seed?
If I understand correctly, when new bitcoin node wants to join the network, it must be introduced by guiding nodes -- DNS seeds. DNS seeds can thereafter help new node discover more existing nodes.
So during the bootstrap phase, DNS seeds are the entry points for the new nodes. By looking at the source code of BitcoinJ:
   dnsSeeds = new String[] {
            "seed.bitcoin.sipa.be",         // Pieter Wuille
            "dnsseed.bluematt.me",          // Matt Corallo
            "dnsseed.bitcoin.dashjr.org",   // Luke Dashjr
            "seed.bitcoinstats.com",        // Chris Decker
            "seed.bitnodes.io",             // Addy Yeow
            "bitseed.xf2.org",              // Jeff Garzik
            "seed.bitcoin.jonasschnelli.ch" // Jonas Schnelli
    };

They are pretty much hard coded in the software. What if all of those DNS seeds are dishonest nodes? In this case, new nodes would be surrounded by dishonest nodes and it doesn't have chances to connect even one honest node and retrieve the genuine blockchain. Therefore, new nodes would be fooled in this case.
Does that mean bitcoin network still needs some extent of trust? And would it be a concern in reality?

What you are saying is correct.
I'll quickly elaborate on why these hardcoded dns seeds are required. The attack you mention is called "peer list poisoning", a strategy that can be used to segregate someone from the network. There are two main types of peer list poisoning:

Dishonest nodes: the attacker poisons your peer list with real nodes, but malicious ones. All of your transactions would always be forced through these nodes.
Void nodes: the attacker poisons your peer list with non-working nodes, a strategy often used to dismantle malicous P2P botnets.

You may think the DNS seed require an element of trust but it doesn't. There are ways to verify that a DNS seed is honest: if you run your own 24/7 Bitcoin Node with the right setup then your address should be available in a list returned by a DNS seed. They crawl the whole network and regularly revisits known nodes to check their availability. DNS seeds do not return a full list, so it's not very practical.
There are alternatives to using DNS seeds, such as crawling the network yourself, or even bruteforcing IP addresses and attempting a connection.

Your node would only be given out if you are accepting incoming connections. The DNS seeders also don't give out their full list of nodes, just some of them (10 or 20 I think).

@Penquin Thanks for the excellent answer. But who runs the checking, new comers' node or DNS seeds? If new comers run the checking, what kind of algorithm can be used to check the honesty? If the DNS seeds run the checking, how can new comers believe in the DNS seeds is actually running the checking?

Regarding the IP brute-forcing of the who internet, it sounds a plausible method to connect to a honest node, but is it efficient enough? On average, how long does it take to connect to a node which actually can respond the bitcoin protocols?

Nobody runs the checking, but you can if you want to but I haven't seen any source code available for it yet. Bruteforcing is a really terrible inefficient method to be honest. See the introduction of this paper, given a large size it takes a few thousand attempts

@Penqin thanks for clarifying, that's very helpful!

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

authors: Лена Фирсова


text:
Was the 51% attack a real threat in the early days of bitcoin existence?
The hypothesis of 51% attack is based on a malicious entity is able to control more than 50% hashrate of all mining nodes. It's understandable that there's huge amount of mining power existing nowadays, however, they grew up gradually since 2009.
I'm wondering when there were only a few of people who knew the bitcoin back to the 2009, therefore, surpassing 50% of mining nodes was fairly easy(for instance, assuming there were only 10 participants in the early days of bitcoin). Because controlling more than 50% of mining power probably just needed to control more than 10 computers in the earliest days since bitcoin birth.
As a result, do I need to worry about the 51% attack possibly happened in the blocks of which the height number is small and are those blocks trustable?

Yes it was easier back then, that is why ASICs are a good thing and keep the network much safer. But no you don't need to worry about it, even a 51% attack can only do a limited number of things to the network, blocks and transactions must still be valid. So if a 51% attack had occurred, for example double spending a transaction, that is long in the past and would definitely not affect you now.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How to import an existing extended private key to bcoin?
I´m searching for a way to import an existing extended private key, exported from electrum, into a bcoin node. Reading https://bcoin.io/api-docs/ I can´t find a description to do this.
Am I missing a part or does someone have a solution for this?

What exactly do you have from Electrum? Is it an xprv... ?
What you need to do is create a wallet and assign the master private key:
https://bcoin.io/api-docs/#create-a-wallet
bwallet-cli mkwallet --id=WALLETNAME --master=xprv...
Note that Electrum's address derivation scheme may not match bcoin's exactly (since bip32/bip44 don't provide metadata like address type). Therefore you may also need to include the flag --witness=true which will derive bech32 addresses along the bip44 path. If Electrum is using bip49 or bip84 derivation schemes, you will have to run a modified branch of bcoin to recover the same addresses.
If you want to chat about this the developers are usually available on slack or IRC #bcoin

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
recover wallet in electrum from hex mpk
I´m trying to recover a wallet from which the Master Private Key (mpk) in hex format and the number of used addresses is known.
Till now I could manage to convert the hex mpk to the wif format and import this to electrum. But here to history seems to be totally lost.
Is there any way to recover the full wallet and history (not neccessary with electrum)?

are you sure it's the master private key and not the seed? the seed in hex is 32 characters. if it's the seed do a restore from seed and paste in the hex instead of the words.

sorry for the late respnse. yes, I`m sure that the key is a mpk and not a seed in hex. It seems that the problem is based on compressed vs. uncompressed keys.

The problem is based on the generating method I used from the the Bitwasp library.
$key = PrivateKeyFactory::fromHex($hex, true);
$wif = $key->toWif();

Will return a WIF assuming the hex key was compressed. Restoring the history while importin in electrum failed.
$key = PrivateKeyFactory::fromHex($hex, false);
$wif = $key->toWif();

Will return a WIF from an uncompressed hex key an electrum was able to restore the history of the address.

master private key doesn't lead to an address in the wallet on its own. you have to run it through hash functions and do ec math to generate child keys at various indices. that's what the type2 deterministic thing is all about. so it seems like you have a regular private key and not a master private key.


text:
How to import Multibit HD Wallet in bcoin?
I created a wallet using Multibit HD and noted the mnemonic. I sent some money to an address that showed up in Multibit HD. Worked fine. Transaktions are displayed and balance is adjusted. Can be verified on blockchain.info for example.
Using the Mnemonic Code Converter I derived the BIP32 Extended Private Key. With the REST-API of bcoin I both created a wallet using the mnemonic and using the private key. After that I did a rescan at height 0.
Now I would expect that my accounts both have the same balance as in Multibit HD, but both have zero balance and zero transaktions.
What did I do wrong?

The bcoin node is fully synced and has progress 1.
Both bcoin and Multibit are using Mainnet.
Both Wallets show the same master key (the one that was derived on the website). Only the one created by the mnemonic also displays that mnemonic.

During research I already found out that the address used for the transaktion only shows up on Mnemonic Code Converter if I choose Multibit HD as the Client under the BIP32-Tab.

MultiBit HD uses a BIP 32 derivation path that is different from basically all other wallet implementations. MultiBit HD's derivation path is m/0'/0/i whereas most wallets use derivation paths set by BIP 44 which is m/44'/0'/0'/0/i. So when getting addresses from a MultiBit HD seed, you will need to use MultiBit HD's derivation path instead of the default BIP 44 one used by most wallet software.

Is it possible to set up a wallet with the Multibit path using bcoin? Or ist is even possible to set up a wallet that "monitors" multiple paths?


text:
Finding nonce resulting in more zeros than the target
This might be a wrong question(I am new to understanding bitcoin details). But I want to ask this:
 What happens if one node finds nonce value that results in way more leading zeros than expected by the target?

Nothing special happens in this case.
If you find a nonce for which the hash has at least the necessary number of zeros (or more precisely, is numerically less than the target value), you have a valid block.  That's all.  As far as I know, the Bitcoin protocol pays no attention to whether there are "extra" zeros.
Of course, if someone submits a block header with a drastically large number of extra zeros, people might reasonably take this as evidence that they have broken the SHA256 hash function in some way, perhaps with some sort of preimage attack.  This might lead to developers wanting to phase out the use of SHA256.  But that's beyond the scope of the protocol or the software.


text:
What's the point of ETHLend?
I am trying to understand what the idea behing a site like ETHLend is.
IIUC, I can borrow coins for an interest, the same way I could borrow a traditional currency from a bank. That part is easy to understand.
But - again IIUC - they require that I block the collateral for the load - where they will only accept cryptocurrencies - upfront through the smart contract.
Translating to traditional terms, this would be as if the bank says:
Transfer 10.000 USD to us. Then we will borrow you 10.000 EUR which you can pay back in installments plus interest. After you will have paid your credit back in full, you will also get your 10.000 EUR back, otherwise we will just keep them.
What do I miss?

This question should be closed, Questions seeking product or service reviews are off-topic because they tend to attract subjective, low-quality, and spam answers. On the other hand, offerings in the Bitcoin space are still evolving rapidly which often renders answers outdated quickly.

Sorry, but I object to closing the question. Despite the fact that I am asking about a particular service here, the question is not if the service is good or bad, but about the underlying concept. I agree that maybe the questions could be reworded to say "a service like", but I think that questions about business models or trust models in this case are not necessarily out of scope.

Your assessment is correct.
But think of it this way: to get a mortgage, you put up your house as collateral, where value(house)>value(mortgage). Same idea here.
Some response to your concern in their FAQ here

Why would I borrow when I can just sell?

There are a variety of reasons you may want to borrow and unlock the liquidity of your tokens rather than selling, some include:
Borrowing Ether to participate in different ICOs,
     Purchasing other tokens from the exchange for different investment strategies without the need to sell tokens,
    Lending altcoins and tokens for short selling or hedging,
     Financing mining expansions,
     Finance for blockchain startups,
     Buying market dips.
This list is expanded even further when we include USD Pegged Etheruem loans:
Holding more ETH in a bullish market,
    Purchasing assets with $ while keeping your crypto portfolio safe,
Future use-cases could include tokenisation of property, shares, intellectual property, and other physical objects (Cars, boats, power plants, art, etc)

Thanks for the confirmation. I just wanted to be sure I did not miss anything like "on providing XX ETH I can borrow XXXX ETH" or something of that nature. But this isn't the case obviously.

Where the traditional mortgage example falls short is: If I provide my house (or maybe even a car) as collaterial for a loan, I can usually still use it (live there, drive it) for the duration of the loan. But I will not be able to spend my ETH somewehere else or use them for anything else as long as they are locked up in a borrowing contract, can I?

You can't spend your house while it's mortgaged. But you can live in it, which I think would be similar to unlocked a feature of a Token. I don't think many tokens have features yet, but I could imagine a scenario where only addresses that signed say 100 Tokens are allowed to post on a website under a certain status. Maybe.


text:
How exactly do I decode the block file info in leveldb?
Base on the links
https://github.com/bitcoin/bitcoin/blob/fcbc8bfa6d10cac4f16699d6e6e68fb6eb98acd0/src/main.h#L392 and
What are the keys used in the blockchain levelDB (ie what are the key:value pairs)?
I should simply read in the results as varints. So I wrote some Python code shown below. But I did not get the results I expected.
Below is my code to decode the file information with file number 0.
I parse through my blk00000.dat and found that the first block was the genesis block and the last block was block 177 with blockhash 00000000480c9799dd2b35009ff1833f69690f76c26a9bea7e00cd0042f67db1.
nBlocks looks to be correct with 134 blocks which was how many blocks I found in blk00000.dat.
nHeightFirst looks right with the value being 0, but I nHeightLast is not 177. nSize and nUndoSize dont look right since it doesn't match the file size with I found on the terminal doing
ls -l  ~/.bitcoin/blocks/blk00000.dat

and
ls -l  ~/.bitcoin/blocks/rev00000.dat

Also the nTimeFirst does not match the timestamp of the genesis block.
And nTimeLast does not match the timestamp of block 177, which I found to be the block with the highest timestamp in blk00000.dat.
What am doing wrong?
import plyvel
blockIndexDB = plyvel.DB('/home/chris/.bitcoin/blocks/index')
result = blockIndexDB.get(b'f\x00\x00\x00\x00')

# result value from leveldb
# result = b'\x86\xa8%\xbe\xfe\xf4E\x88\xa5\xa7}\x00\x86\xa8r\x83\xc9\xfd\xd5)\x83\xec\xd3\xa1N'
# hexString = b'86a825befef44588a5a77d0086a87283c9fdd52983ecd3a14e'

nBlocks = int.from_bytes(result[0:1], byteorder='little')
print('nBlocks', nBlocks)
print(nBlocks == 134)

s = int.from_bytes(result[1:2], byteorder='little')
# print('s', s)
nSize = int.from_bytes(result[2:6], byteorder='little')
print('nSize', nSize)
print(nSize == 134216389) # blk00000.dat file size

s = int.from_bytes(result[6:7], byteorder='little')
# print('s', s)
nUndoSize = int.from_bytes(result[7:11], byteorder='little')
print('nUndoSize', nUndoSize)
print(nUndoSize == 19502205) # rev00000.dat file size

nHeightFirst = int.from_bytes(result[11:12], byteorder='little')
print('nHeightFirst', nHeightFirst)
print(nHeightFirst == 0)
nHeightLast = int.from_bytes(result[12:13], byteorder='little')
print('nHeightLast', nHeightLast)
print(nHeightLast == 177)

s = int.from_bytes(result[15:16], byteorder='little')
# print('s', s)
nTimeFirst = int.from_bytes(result[16:20], byteorder='little')
print('nTimeFirst', nTimeFirst)
print(nTimeFirst == 1231006505) # block 0 timestamp

s = int.from_bytes(result[20:21], byteorder='little')
# print('s', s)
nTimeLast = int.from_bytes(result[21:], byteorder='little')
print('nTimeLast', nTimeLast)
# block 177 is the last block in blk00000.dat file
print(nTimeLast == 1231736557) # block 177 timestamp

Your decoding of the block file may be incorrect. Can you post the code you used to do that?

Its not incorrect because I double check all the blocks on blockchain.com and all the values matches. prevHash, merkleRoot etc.

Even if I decoded the blocks wrong, why would the file sizes not match?

You can use my blockchain-parser written on python https://github.com/ragestack/blockchain-parser. This is the parser for blk*****.dat files with simple intuitive structure for various purposes such yours.

You are deserializing every value as an integer but that is incorrect. The values are serialized as variable length integers with high bits signalling the length of the integer which are dropped for the actual integer value itself. So you need to fix your code to deserialize the bytes as varints instead of just integers with fixed lengths.
The following code was able to deserialize the entry correctly.
#! /usr/bin/env python3

import binascii

def get_max(long=False):
    if long:
        return 18446744073709551615
    else:
        return 4294967295

def read_var_int(s, pos, long=False):
    n = 0
    while True:
        chData = s[pos]
        pos += 1
        if n > (get_max(long) >> 7):
            raise IOError("ReadVarInt(): size too large");
        n = (n << 7) | (chData & 0x7F)
        if chData & 0x80:
            if n == get_max(long):
                raise IOError("ReadVarInt(): size too large");
            n += 1
        else:
            return n, pos

entry = binascii.unhexlify(b'86a825befef44588a5a77d0086a87283c9fdd52983ecd3a14e')
pos = 0

nBlocks, pos = read_var_int(entry, pos)
print('nBlocks: {}'.format(nBlocks))
nSize, pos = read_var_int(entry, pos)
print('nSize: {}'.format(nSize))
nUndoSize, pos = read_var_int(entry, pos)
print('nUndoSize: {}'.format(nUndoSize))
nHeightFirst, pos = read_var_int(entry, pos)
print('nHeightFirst: {}'.format(nHeightFirst))
nHeightLast, pos = read_var_int(entry, pos)
print('nHeightLast: {}'.format(nHeightLast))
nTimeFirst, pos = read_var_int(entry, pos)
print('nTimeFirst: {}'.format(nTimeFirst))
nTimeLast, pos = read_var_int(entry, pos)
print('nTimeLast: {}'.format(nTimeLast))

The result was:
nBlocks: 119973
nSize: 134216389
nUndoSize: 19502205
nHeightFirst: 0
nHeightLast: 120050
nTimeFirst: 1231006505
nTimeLast: 1303712078

The nBlocks and nHeightLast are expected because early blocks were extremely small yet the block file is fairly large. I think that your calculation of 134 blocks in the block file is wrong. With 134 blocks, the blocks would have to be more than 1 MB in size which is simply untrue for early blocks.


text:
From unsigned to psbt transaction
Recently, I happened to see in a codebase something like the following snippet to convert a standard transaction into a psbt format:
txHex = "01000000038caba88b36e86b253de1c22574205ad7d46598a87349b2e7f364d244246b66820000000000ffffffff8d4cefc5b67a8ae0e4c6b8fd7ee259943c29b46471c4f31a5252422177fa20770100000000ffffffff68bc5046732a7e7e57b8f0abd6f94c63f7153738cb39525d497ce1cd765e35230100000000ffffffff02404b4c00000000001976a9148eccac781789b66751f19c3a58f36526a201786488ac00093d00000000001976a9148eb446f809f526fb37059a32cf8255c4cb43d2da88ac00000000"
psbt = "70736274ff0100c9" + txHex + "000000000000"

Is this correct?
One thing I understood after reading bip174 is that the last byte of the prefix should not be hardcoded because it contains the lenght of the transaction, but I have no clue about the suffix

The "suffix" is not a fixed suffix. The number of 00 bytes needs to match the number of inputs and outputs of the transaction. Those bytes are the terminators for the key-value maps for each input and output. The automatic converter does not fill in those maps, but an empty map will still have its null terminator. So the number of empty maps (and thus the number of null terminators) will change depending on the number of inputs and outputs in your transaction.
Also, those bytes will change as the PSBT gets filled with information. It is not a fixed suffix. So as the extra data for signing for each input and output gets filled, the "suffix" of your PSBT will change.


text:
How to parse values in block index leveldb?
So I'm trying to read the block files with Python.
From this link,
What are the keys used in the blockchain levelDB (ie what are the key:value pairs)?, it tells you how to access the files with leveldb. So I wrote a class to read in the meta data.
class ReadBytes:
    def __init__(self, bytes):
        self.posistion = 0
        self.bytes = bytes

    def get_int(self, byte_count):
        self.posistion += byte_count
        return decode(self.bytes[self.posistion - byte_count:self.posistion][::-1], 256)

    def get_var_int(self):
        self.posistion += 1
        value = byte_to_int(self.bytes[self.posistion - 1])
        if value < 253:
            return value
        return self.get_int(pow(2, value - 252))

But from this link, https://github.com/bitcoin/bitcoin/blob/master/src/chain.h#L364, It turns out there is this thing call VarIntMode::NONNEGATIVE_SIGNED that is tripping me up. I can read regular var int with my Python class, but how do I read the non-negative signed ones in Python?

A non-negative signed varint is just a normal varint but should be put into a signed integer rather than an unsigned integer as with normal varints. You just need to also check that it is a non-negative integer.
For example, instead of decoding a 32 bit varint as a uint32_t, you decode it as int32_t and check that it's value is >= 0.

Your code looks like it is decoding things as signed integers already. Instead of using Python's conversions built into the various classes, I recommend that you use the struct module so that you can specify endianness and signedness of the bytes that you are decoding.


text:
Security implications of removing CHECKSIG in a test
If I conduct an htlc transaction on testnet that does not include OP_CHECKSIG will it still cost an attacker more than (say) $10 in processing power or storage to subvert my tests?
Details: I'm tinkering with BIP-0199 htlcs, the last step of which is CHECKSIG. Now, bitcoind does not (yet) expose htlc functionality through RPC, so in order to tinker I've had to implement BIP-0199 via RPC requests myself. To date I've implemented everything I need up to the final CHECKSIG. I understand of Pieter Wuille here that I would need to implement the signing logic for CHECKSIG myself, which seems logical. However, I'm not interested in the role of CHECKSIG in a htlc as I believe it indistinguishable from the same opcode in any standard transaction so I propose to skip implementing it while my tinkering remains on testnet.
Obviously an htlc without CHECKSIG is not secure. For my purposes is it secure enough: Logically,
1. While it remains on testnet, a successful attack obtains no more than testnet coins;
2. The testnet coins that could be obtained are limited to the funds committed to the p2sh address;
3. An attacker who somehow knew that a redeemscript did not contain CHECKSIG would still have to brute force a script whose sha256 hash matches that of the redeemscript in order to redeem funds;
4. An attacker who sees the seller's redeemscript has to obtain matches for the digest and public key (perhaps through hash tables) and still has to get his copy of the transaction mined first in order to cheat the seller;
5. An attacker seeing the buyer's transaction 'only' has to match on the public key then get his copy mined first to cheat the buyer;
6. An attacker that's too slow to get mined first could nevertheless succeed by getting the block containing the transaction orphaned.
And so, an attack on an testnet hashed time-locked contract that does not include CHECKSIG will still require either processing power and/or storage space well in excess of $10. Is my reasoning sound, and if not, what have I missed?

The security implication of removing a checksig operation from the encumbrance script is that the coins can be stolen while in flight by anybody, and especially by miners.  Obviously, on testnet, only someone who's bored or interested enough to grab the coins will take the time to "attack" your transactions.
There's not much sense in listing and analyzing the vulnerabilities in your method.  Testnet coins are worthless and certainly nobody will take the time to brute-force sha256, or grind through a hash table to get them.
As for checksig-less scripts in general, there are indeed bots on mainnet that listen for such transactions being relayed and try to grab any coins.  It's probably easy enough to run the same bot on testnet.
In any case, if you want to avoid this scenario altogether, you can migrate your setup to regtest.


text:
Differences to CCompactSize and CVarInt
I'm developing a parser bitcoin core and I have a problem whit the type CVarInt, I using the decodifier mode of the CCompact size but at the block 976 I Have a problem whit the type, because the value id is unusually, so I have a question why the bitcoin core decided the change implementation to
CCompact size into CVarInt?
What was introduced in the bitcoin core versions to require a change of implementation?
Sorry for my terrible English but I'm learning.

CVarInt encoding is only used in the storage of the UTXO set internally, and never in the P2P protocol. CCompactSize is used in a number of places, including the number of transactions per block, the number of inputs and outputs per transaction, and the length of scripts.
If you're parsing block data, you will never encounter a CVarInt. If you're running into problems, it's almost certainly something else, or there is a bug in your CCompactSize decoder.

Oh this is a good news, Unfortunately I don't know exactly what block gives me problems reading the script, because for every blk file I create a json file with the list of blocks, so if I get an error reading I can't generate the json, apparte if not active the logger logs the code, which I will do soon, but for now I know that the last block that reads me converting it to json is the following This

plus i use the library to read bitcoin core files the file is serialize.he exception that raises me is the following C ++ exception with description" ReadCompactSize (): size too large: iostream error "thrown in the test body.

@vincenzopalazzo That probably means you're trying to decode garbage. There is no guarantee that all bytes in the block file belong to a block; there may be gaps etc

So it is not guaranteed that all blocks have only the declared data? I mean, does this mean that in the beginning there may be other things that are not useful data for me? but in this way there must be a reference somewhere to see what is in that beginning file no?
Because if bitcoin core uses that data then behind the serialization there is logic

Blocks have well defined data only. But the files can contain garbage in addition to the blocks. There is an index database that stores which position each block is stored at. There is no guarantee that there isn't garbage between blocks (for example, for a crash while writing a block).

Is it possible that the block continues in the next file or is there some constraint that I don't know where each file must start with a block?

No, the whole block is always in the same file.

Oh thanks, this is really a problem for me now


text:
Why does the amount of sats in a lightning payment channel amounts vary if no txns are occuring?
I recently got my lightning node up and running (went the easy route via Casa node).
I noticed that I was having a difficult time tracking the amount of sats in a given channel..I wasn't doing any transactions with them, but they seemed to vary up and down 1000 sats up, 2000 down, etc.
Confused, I finally opened up a payment channel only to watch it, see how it moves, with a nice round number: 1,000,000 satoshis.
It's still in pending state (trying to open it up with ACINQ node for little miner money, still hasn't been picked up by a btc miner)...anyways..it's still in pending state, but the amount now lists "989,869 sats Pending" not 1 million sats as I tried to open the channel with. . .
Should'nt satoshi quantity stay the same for a given channel (if it's all opened up and no txns are occuring, or if as in my case it's in pending state?). Does the bitcoin price going up or down affect the amount of btc/satoshis in a channel?

Without detailed log files I can only guess the exact reason why you see this behavior. Two likely things come to my mind:
1.) Your node might have routed smaller payments. So in case you have outgoing capacity on one channel and incoming on another one someone might have used that to rout a payment. which would move some satoshi from one channel to the other
2.) If you opened the channels you are responsible for paying the fees in the case of a force close. These fees are taken away from the amount of satoshis which you own in your channel. Everytime you make a channel update (for example with a routing attampt - even if routing was not successfull) the fee estimator of your node and your peer try to estimate the onchain fees that your node would have to pay in case of a close.
Since your node seems to be fairly new and since the onchain fees have been fluctuating a lot recently I am almost certain that my second guess is the reason for the behavior that you observed.

Thx for your help. the channel finally opened after restarting the node. Casa support said the txn to open the channel likely got stuck in the mempool (ranked to the bottom as it's low value), which slowed down the opening of he channel. . . . so..it wasn't a force close, but still shows 989,869..hmmm....but on 1ML it shows that the channel has 1,000,000 sats..so not quite sure.


text:
Can my 10,000 satoshi channel with a large LN node receive multiple LN payments that are larger than the amount in the channel?
Can my LN channel for some amount "X" receive a LN payement larger than "X"..or receive multiple small payments that exceed "X"?
For example, say I have an online tea store and have one 100,000 satoshi channel opened with one of the big LN nodes out there (e.g., acinq, one of LNbig's nodes)...my thinking being, well I want to keep expenses down in opening up channels, and the big node I'm connected to will connect with a lot of people that want to buy from my online tea store.
However, say my store has one item that is 10,000 satoshi and one item that is 1,000,000 satoshi.
What happens if:
(1) I receive fifteen 10,000 satoshi order/payments routed via the 100k big LN node channel...or
(2) I receive one 1,000,000 payment routed to me via the 100k big node channel (a purchase of my one "expensive" 1,000,000 item).
I believe to run my store I would need to create multiple giant channels to accommodate any potential payments received...right? So in both cases above, the channel must have enough to receive the payments and cannot receive a payment larger than what is in the channel.

Can my LN channel for some amount "X" receive a LN payement larger than "X"

No, you can only receive payments until the entire channel capacity ("X") has been pushed to you side of the channel. At that point, you will no longer have incoming capacity.

..or receive multiple small payments that exceed "X"?

You could receive multiple payments that exceed "X", but you would also have to send payments out in between the receives (to push the channel balance back to your channel counterparty's node).

What happens if:
(1) I receive fifteen 10,000 satoshi order/payments routed via the 100k big LN node channel...or

Assuming your channel starts with 100% of the capacity on your counterparty's node, you would be able to receive 9-10 payments, and then there would be no route to complete the last 5-6 payments.
I say 9-10 instead of 10, because each node involved in a channel must keep a small amount of capacity in reserve, to initiate a channel closing in case of an uncooperative closure. So if the capacity was exact 100,000 sat, and the payments were exactly 10,000 sat, then you would most likely not be able to receive the full 100,000 sat in payments consecutively, due to this reserve requirement,

What happens if:
(2) I receive one 1,000,000 payment routed to me via the 100k big node channel (a purchase of my one "expensive" 1,000,000 item).

The same as above, you might be unable to receive that payment, due to the reserve requirements.

I believe to run my store I would need to create multiple giant channels to accommodate any potential payments received...right? So in both cases above, the channel must have enough to receive the payments and cannot receive a payment larger than what is in the channel.

Right, you cannot receive a payment larger than your current incoming capacity on that channel (the remote_balance). As a shop owner, you can actively maintain channels with incoming capacity, but you could also allow customers to open channels with you directly. There are also services forming that allow you to buy incoming capacity, etc.

That makes sense! Clarifies a lot of things. Thank you.

I believe to run my store I would need to create multiple giant channels to accommodate any potential payments received...right? So in both cases above, the channel must have enough to receive the payments and cannot receive a payment larger than what is in the channel.

@chytrik touched most of your questions, but this is one I would like to expand on. There is currently a proposal of Base AMP that solves the above issue and is likely to be merged shortly into the specs. Instead of having giant channels to accommodate larger payments, that large payment can be broken down into smaller ones and routed across multiple channels that you might have with various participants.
In order to use this feature, you will have to indicate in the invoice that your node handles multi-path payments (MPP) by setting a feature flag. The payer can then find the appropriate routes across channels and route smaller payments through them. When you have received payment onions across channels that sums to (or greater than) the invoice amount, only then will you release the pre-image that matches the payment hash.
You can read more about implementation and future releases on multi-path payments in my answer here.


text:
Blockchain Private Key
I have been paying amounts of bitcoin, with the promise from Howard Jones in Texas that I will get my private key to get the bitcoin, but when it was time to give me the key, communication went quiet, and the WhatsApp number got deactivated. Now the amount in my Blockchain wallet is going down. Have I been scammed?

Yes this sounds like a typical scam. If you didn't have the private key all along, you never had any money and weren't ever getting any.

Possible duplicate of How do I get imported Bitcoin in my blockchain wallet without having to pay more for the private key? or How to unlock private key of bitcoin or Needing a Private Key in order to receive profits in Blockchain Wallet


text:
Doesn't an Oracle go against the Blockchain principles?
From what I understand of the blockchain, the main motivation, the very foundation of this technology, is the search for peer-to-peer exchange without mutual trust, without a trusted third party, so trustless exchanges.
However, the purpose of an Oracle is to provide an external data that is invertible. It is therefore necessary to have confidence in this oracle, while it can provide false data (voluntarily knowing that it can be beneficial or not by simple measurement error (for an IoT sensor) / algorithm error / human error...).
In this context, does an oracle not go against the previous principle? Where are there ways to get trustless Oracles?
Thank you all.

The use of an Oracle does somewhat go against the founding principle of transacting without the need for a trusted third party.
As Nakamoto's whitepaper states:

What is needed is an electronic payment system based on cryptographic proof instead of trust,
  allowing any two willing parties to transact directly with each other without the need for a trusted
  third party.

Of course the majority of transactions do not need trusted third parties such as oracles or escrow agents. But for those that do, trust is a key issue.
Other people have suggested trust minimisation strategies that can be used with oracles.

multiple independent oracles
trusted hardware
Amazon AWS oracles

The important distinction here is between authenticating information native to Bitcoin (via cryptography), and information external to Bitcoin (via oracles).
The native asset of the Bitcoin network is the bitcoin currency. Users can run a node that independently validates the state of the network, without needing to trust any third party or intermediate. The cryptography that guarantees the authenticity of a bitcoin UTXO can be computed independently, allowing the user to maintain full sovereignty over their funds.
However, a problem arises with authenticating information external to the Bitcoin network, and this is where oracles are required. It is not possible to connect the external world to the Bitcoin network without the use of an oracle, and so in cases where an oracle is needed, the user must trust the oracle completely. It is important to keep this in mind: a blockchain does not illicit truth from the world, and there is no way to prove (on-chain) that an oracle misbehaved and input bad data. You may be able to prove that the data was false externally (in a courthouse, etc), but this would have no bearing on the validity of on-chain transactions related to that data. The oracle could still lie, and then initiate transactions based upon that lie.

In this context, does an oracle not go against the previous principle? Where are there ways to get trustless Oracles?

There certainly can be uses for oracles, but these relationships of trust between user and oracle are important to understand. For example, consider a simple trade of bitcoin for dollars:
Bob wants to sell 1 Bitcoin to Alice for $100, but wants Alice to send the dollars first, before he initiates the bitcoin transaction. In this case, Bob would be acting as an oracle for Alice: once the off-chain portion of the deal is complete, Bob will take action on-chain (by sending the bitcoin to Alice). So Alice must trust Bob to act honestly.
Understanding this, Alice decides to use a third party escrow agent, Joe. Alice asks Bob to send the 1 bitcoin to Joe, and then Alice will send Joe proof of the dollar transfer. Joe will act as oracle, so Alice and Bob must trust him, but Joe can be incentivized to act honestly by charging a small fee for his service.
So we can see that while trust can be minimized in certain ways, there will always be an oracle position inherent to situations where physical world happening needs to be recorded or acted upon in the blockchain. While the example was simple, it can be extended to any number of more complex situations. Oracles can be trust-minimized through clever incentive schemes, but their position can not be eliminated entirely when attempting to link the physical world to the blockchain.
Relevant note: There are no shortage of ‘blockchain projects’ which are premised entirely upon data fed from an oracle as being ‘trustless’ and ‘revolutionary’ (eg supply chain tracking projects)— but in reality they are just centralized coins with fancy marketing, that seek to take advantage of users which don’t understand the oracle-trust problem.


text:
Anatomy of mixing transactions
I've read some literature on mixing in systems like TOR and also in Bitcoin.
In Bitcoin and Cryptocurrency Technologies, the authors explain that uniform transactions are highly important in order to prevent correlation attacks by matching inputs and outputs:

Uniform transactions. If mix transactions by different users had
  different quantities of bitcoins, then mixing wouldn’t be very
  effective. Since the value going into the mix and coming out of a mix
  would have to be preserved, it will enable linking a user’s coins as
  they flow through the mix, or at least greatly diminish the size of
  the anonymity set. Instead, we want mix transactions to be uniform in
  value so that linkability is minimized. All mixes should agree on a
  standard chunk size...

However, I'm unable to find examples where this is visible in the blockchain.
I've also seen mixing in Bitcoin described differently, for example in the Bitcoin and Beyond paper:

The mixing service aggregates bitcoins received from all its users to
  re-distribute them again. Some services use randomness, i. e., they
  split the amount into smaller random chunks and transfer them after
  random intervals. The user can schedule the transactions and provide
  time constraints.

Now I'm looking for examples of these types of mixing transactions in the blockchain, especially one where I can see a fixed chunk size being used.
Is one of these schemes used more frequently nowadays?
Is the second scheme really secure? If THIS is actually mixing like I assume, then isn't it possible to match sets of inputs to sets of outputs that have the amount (or slightly less) BTC value and therefore have to be connected? I can imagine that this does also work over the course of multiple transaction in case the payout of a mixer is split into intervals.

As I see it, the techniques you are referring to are two possible but different ways of doing mixing. The first one, explained in Bitcoin and Cryptocurrency Technologies book is referring to a kind of mixing technique like CoinJoin.
CoinJoin is a mixing technique in which several users create a transaction by joining their inputs. In order to maintain their privacy, all inputs should share the same value, since in that way once the transaction is created there is no way of telling which input correspond to each output, whereas if the inputs hold different value it will be way more straight-forward. There are to ways of implementing a CoinJoin scheme, the first and more simple one relays in a third party receiving all inputs, outputs and signatures, and building the transaction from the participants. The other one, more elaborated, does not need any third party, since every user acts as a blind-signing server.
On the other hand, the technique referred in the Bitcoin and Beyond paper uses a mixing server, which acts as a trusted third party, where users will send funds and eventually receive them back in a different address (after applying a certain mixing fee). In order to avoid linkability between received and sent transactions, this kind of mixing technique uses randomized delays to redistribute transactions. blockchain.info's Send Shared, and bitmixer.io used to be examples of such a mixing technique.
Finally, regarding examples of such transactions, unfortunately I don´t have any particular CoinJoin transaction identified to let you know, maybe someone else can help you with that.


text:
Sensitive information stored in contracts
Is it a good idea to store some sensitive data on the blockchain contracts? For instance, i want to make a contract that stores vouchers (that cost real $$) and once the user pays to the contract for the voucher, the voucher is sent to the user. Let's imagine that the vouchers are just random numbers. Are the vouchers secure on the contract in this case?
I'm asking this because i've seen some controversial claims about sensitive data posted on blockchain. First one being the public image that everything is public on the blockchain, hence how the blocks are being validated. The second being that data is secure and strongly hashed which makes me wonder if this type of contract can work out. Thanks in advance, I'm new to blockchains and very interested in learning about this technology.

Data in the blockchain is public and everyone can see it.
It is "secure" in the sense that it is computationally very difficult to modify the data later, without making it obvious that you have done so.  It is also unlikely to be deleted, since a large number of computers store the entire blockchain.
The hashing provides a certificate that a large amount of computation was done using this particular data.  Someone who wants to claim that they have the authentic data would have to do even more computation.  But the hashing does not encrypt the data in any way; the data itself remains perfectly readable (in fact, this is essential to being able to verify it).
So such data is "secure" against tampering or loss, but not against being read.


text:
Am I screwed? Transferred USDT from Polinex to USD dollars in Coinbase and now it's gone
I've seen others post where they transferred their USDT to a bitcoin address and it got lost.  I specifically clicked on my USD instead of BTC and it provided me with an address to send it to.  However, the same thing happened where it says it's pending and only .0000273 getting transferred.  I transferred a decent chunk of change.
So here's where it's interesting, if I going to omnichannel and type in the address where the USDT was sent, it shows the full amount still available.  I know it's there and the omni wallet shows it but it doesn't let me get access to it.  If I try to import a private key and add an address on to my omniwallet, it let's me put in the TXID associated with the transaction but pulls up zero as the balance.
Any ideas or options out there?  I know the amount is still out there but I can't seem to get it.  I've heard something about importing private keys but I don't know how to find it.  Or am I just plain ole screwed and will have to learn my lesson.  Thanks for the help.

That happened to me too. Support took weeks to get back to me and basically said "too bad". Did you find a solution?

No luck.  I can still see the USDT available when I look it up through the blockchain but i can't get it.   At this point, i would offer have of the available funds to anyone who can get it.   Basically $1k up for grabs.

You do not have access to your Private keys on coinbase, as it is considered a custodial wallet, which means only they have access to it. I highly doubt they would give you these private keys as its not standard practice. You only option is to email support. I doubt they would care. Another option is to ask someone inside coinbase to help you, but you'll need connections to do that I imagine.


text:
Why doesn't the protocol allow unconfirmed transactions to expire at a given height?
Transactions with low fees may remain in the mempool for as long as 14 days. This indeterminism is a source of confusion for users and wallets. It's not possible to cause a transaction in the mempool to expire (become invalid) at a certain time in the future, if not yet mined.
Allowing a transaction to specify a block height after which it cannot be mined would provide certainty around how long a transaction has to confirm before it is rejected by the network and must be re-sent. This would allow a payment to be re-attempted without having to wait up to 14 days and without having to satisfy the BIP125 rules or submitting a CPFP transaction. This would also have the advantage of optimizing mempool performance by removing transactions that are no longer desired or that may not be mined.
Adding this functionality today would probably require a hard fork, but why didn't the protocol initially provide this feature?

Why would it require a hardfork?

This question was answered by Satoshi. My attempt to paraphrase: Suppose a transaction T specified an expiration height, X, and was mined in block X. Then its outputs were spent, possibly many times over through multiple generations (the original transaction had, for example, great-great-grandchildren). Now a reorg occurs and there's a new version of the block at height X that doesn't include T.
Most (now unconfirmed) transactions are simply re-mined in the new branch, possibly at different block heights than before. But T can't be re-mined, since it is now invalid (it has expired), as are all of its descendants. As Satoshi puts it,

This wouldn't be fair to later owners of the coins who weren't involved in the time limited transaction.

It's true that any reorg can cause previously-mined transactions to never be mined. But such double-spending would likely require malicious intent -- a 51% attack by miners, rather than, say, an innocuous temporary network partition.
It would be bad if this scenario (a previously-mined transaction is un-mined and then expires) could happen routinely because it would make payments less certain and would give payment receivers an incentive to verify the provenance of the coins they're being paid with, lest they be from a recent-expiring transaction. This is similar to the motivation for the 100 block coinbase maturation consensus rule.

And, importantly, this type of scenario where there's confusion about the final state of a transaction is precisely what this change would be intended to prevent. And, worse, this would be confusion about a transaction that some parties thought was confirmed. If a fix creates a worse version of the very problem it's intended to fix, it's not a very good fix.


text:
Why aren't balances appended to transactions?
As far as I know, you need to travel back through the entire blockchain to verify your balance. Surely adding the balances of the sender and receiver to each transaction would be easier? If balances are included, and after 6 blocks a transaction is considered 'final', why then can we not just go back 6 blocks and look at the balance there for verification, instead of travelling all the way back to the genesis block?

First, blockchain space is at a premium, since every node has to store the whole thing forever.  So in general it is better not to store redundant information in the block chain, and to let each node compute it instead.
Second, the "balance" in an address can be affected by multiple transactions which are not necessarily ordered with respect to one another.  Suppose that address 123abc initially has no incoming transactions, so its balance is 0.  Now suppose that Alice decides to send 1 BTC to that address.  So by your logic, her transaction should show the "final balance" of the address as 1 BTC.  Simultaneously, and with no knowledge of what Alice is doing, Bob sends 2 BTC to that address, and so naturally his transaction shows the final balance as 2 BTC.  But the "true" final balance is 3 BTC, so both of their statements as to the final balance are wrong.  So anyone who wants to know the true final balance has to calculate it themselves anyway, and can't rely on the numbers in the transaction.
To make this work at all, you would need some sort of rule that you can't send more coins to an address until the previous transaction has some number of confirmations.  This would be completely impractical for addresses which are supposed to receive lots of incoming transactions (think of a donation address on a popular website).   It would also mean that an attacker who knows your address could prevent you from receiving an important transaction by sending you lots of tiny transactions which would have to confirm one by one.

Don't forget that this would break the pseudo-anonymity that bitcoin offers.  You would have to re-use addresses for that address to have a running balance.  Alternatively, the blockchain could keep track of which addresses belong to which wallet in order to maintain a balance, but again...this breaks anonymity.


text:
With the current hashrate, how does it still take 10min to solve a bitcoin block?
No matter how high the difficuly is (or how low the target is), I don't understand how the Bitcoin network takes an average of 10 minutes to solve each block.
For any given block header, there are only 2^32 possible values for the nonce. Even if we take into account that the timestamp can be changed by up to 1 hour (for example), that's 3600 x 2^32 possible combinations, which is approximately 1.5e13
On blockchain.info it says the current hash rate of the network is 7,991,378,504 GH/s, which means that, every second, there are multiple orders of magnitude more hashes being calculated than the number of possible hashes that can result from the current block header.
So howcome blocks aren't solved in only a few seconds, and take about 10 minutes instead?
I understand that miners want to include as many transactions as they can in a block to collect the fees, so one could argue it's best to wait until there are many transaction in a block before solving it. But even that doesn't make sense, because there's absolutely nothing to lose in solving each block as fast as possible anyway.

Mining basically works like this:

Make a candidate block, try all nonces.
If none yielded a valid block, change something in the coinbase transaction.
Repeat from step 1.

Miners collect transactions and then try to create a valid block by bruteforcing nonces. The difficulty of creating a block is related to the maximum hash value a valid block may have.
For example the current blockchain head (while posting this) has a block hash of 00000000000000000077d9c18fdbd5cf45b2652fe6db9089558eb8fc9fc3b397.
As you can see it has many leading zeros, because most hashes above this value are rejected. With the 2^32 nonce values and other header fields you cannot try enough blocks to be likely to hit any block hash small enough. Miners will switch some bytes in transactions (normally in the coinbase) to create even more new block candidates.
Furthermore different miners will try different blocks, as they should have different receivers of the coinbase transactions.

So the current difficulty is so high that even if a miner computes the hashes for all possible 2^32 nonces, it is still unlikely to find one below the current target. Thus, it needs to keep trying with a different coinbase until one of the 2^32 nounces actually works. Could you please point me to some documentation explaining all about coinbase transactions and new block candidates? I think that's what's missing in my puzzle.

@EdwardEisenberg: Basically, miners are free to choose which transactions, or even if any at all, they would like to include in a mined block. Run through all the nonces? No problem, just add or remove a transaction and you've immediately got another few trillion hashes to try.

It is not necessary to remove or add transactions. There are some malicious miners which even mine empty blocks (apart from the coinbase), so have nothing to change in that regard. The coinbase transaction can contain arbitrary data and just putting some additional nonce bytes in there is sufficient.

@EdwardEisenberg https://en.bitcoin.it/wiki/Transaction#Generation

That happens because it becomes increasingly hard to mine as more hashpower is deployed to the network.
Every 2016 blocks (2 weeks) the required difficulty (leading zeroes in the block hash) for a block to be accepted as valid increases (or decreases) according to how fast the past 2016 blocks were mined. This is done by comparing the timestamp declared in the block with the timestamp that would be expected if 10 minute blocks were assumed.


text:
Would backing up my private key be enough?
Lets say I put my wallet's private key on a rocket and fire it towards mars.  In 50 years time I travel to mars for a cheap vacation and find my private key.  Would I then have full access to my wallet, including all the transactions over the past 50 years and the current balance (from when I landed on mars in person) etc?
In other words, to fully backup my wallet and be able to restore it with full control, do I only need to backup my private key?

That's exactly how private key works. All you need to have is a private key to access your wallet. And this is the reason it is not a good idea to fire your private key to mars, if someone get there before you, they will steal all your coins.


text:
General Block Chain questions / Js Block Chain
First of all. I'm very new to block chain technology so please forgive any noob questions.
So I'm going a lot on the following tutorial on youtube regarding the knowledge that I do have other than general block chain theory that I've learnt over the past week.
https://www.youtube.com/watch?v=zVqczFZr124

People say that it isn't possible to shut down the block chain but the more that I look into the subject it doesn't matter how much tech you have behind your block chain there has to be an entry point I.E to the master node with a dedicated I.P address. If this is the case then how is it not possible to shut it down? Wouldn't it be a simple fact of blocking the ip like it has been done for so long in other countries?
Proof Of Work: I get that there has to be some sort of way of proving the source of information came from a certain contributing node. However I'm not totally understanding the point of the POW. For eg, if there was 100 block nodes connected to various master nodes then the information that would be processed by each node would be then emitted by each node to the master nodes which would then process the master nodes for insertion into the ledger, correct??? If that is the case then what is the point of the POW/High powered computer specs etc?? isn't it just using up computer resources with the algorithm??? isn't there nay other more efficient way to do this. It sound like a job that any average computer could do in this day and age without having to max out the CPU etc. This is just something that I'm not understanding.
Proof of Steak and Proof of Minimum: I'm quite confused with this type of mining. I get that they say "Ok, well we want you to hold on to your tokens for a long time and then we'll reward you with coins (Mining)", But for instance if I said that I want all my customers to believe in the tooth fairy and paste it in their social profiles on my block chain based network and that is the type of mining we choose, then how does this help with the actual job at hand ??? I.E the POW method is one way to prove that they did a job etc but thats the point. There is a job to do. I don't get how POS/POM takes care of this.
Supposing that the above questions have been answered then I'm looking in to different way that I could do a different type of block chain. More like a proof of participation. Simply being a relay point of information, then I'm seeing issues.
If I did master nodes to store all the information in the ledger and insertions etc then then the information has to pass this point of entry first and then relayed to the block nodes, correct?

Presuming I'm understanding things correctly etc then can anyone point me in the right direction of how to implement the two methods above to the block chain in the tutorial?
Last question is I don't see many articles or people suggesting javascript for a block chain. I personally can't really see any issues with it. As far as I can see this is more of a theory implementation that could be pretty much implemented in any language. But if I'm understanding that wrong then I would really appreciate feed back.
Thank you.

Hello TheMan68, our website format works best if you ask one question at a time. You can find an intro to our site here: [tour]. Please create new question posts to ask about your interests separately. You may find taht we have answered some of your questions already if you take a look at the respective tags: [tag:blockchain], [tag:proof-of-stake], [tag:proof-of-work].

People say that it isn't possible to shut down the block chain but the more that I look into the subject it doesn't matter how much tech you have behind your block chain there has to be an entry point I.E to the master node with a dedicated I.P address. If this is the case then how is it not possible to shut it down? Wouldn't it be a simple fact of blocking the ip like it has been done for so long in other countries?

That is completely incorrect. There is no master node nor is there any central source that dictates what the blockchain is. The blockchain is distributed to all nodes, and all nodes update their local copy of the blockchain according to the consensus rules that the node is following. If everyone is following the same consensus rules, then everyone will converge onto the using the same blockchain. However, there are some circumstances where a node will not use the same blockchain as all other nodes, but these only happen when the node is under attack.

Proof Of Work: I get that there has to be some sort of way of proving the source of information came from a certain contributing node. However I'm not totally understanding the point of the POW. For eg, if there was 100 block nodes connected to various master nodes then the information that would be processed by each node would be then emitted by each node to the master nodes which would then process the master nodes for insertion into the ledger, correct??? If that is the case then what is the point of the POW/High powered computer specs etc?? isn't it just using up computer resources with the algorithm??? isn't there nay other more efficient way to do this. It sound like a job that any average computer could do in this day and age without having to max out the CPU etc. This is just something that I'm not understanding.

Again, there is no master node that inserts things into the blockchain.
The proof of work is something that is done by miners to show that a certain amount of work was done to produce this block. This secures the blockchain because nodes will use a blockchain which has the most proof of work and is valid under its other consensus rules. This means that if an attacker wished to defraud someone by convincing their node that a different blockchain were the real blockchain, then they would have to do the same amount of work that has been done on the real blockchain. At that point, it would be far more cost effective to simply be an actual miner instead of maliciously attacking someone as the cost of doing that much work is very high.

Wow. Thank you very much. Thats a great answer. So I'm still a little fuzzy on something then.
A website like steemit or redit that is obviously on the internet with a static ip etc then how do we connect to the blockchain from here? Where is out point of entry connecting to the system?
So I want to run a private block chain which stores wallet addresses, user logins (like steemit) that work off the block chain and other content like steemit then how do we store this? is a copy of the content copied and downloaded to everyone's machine that goes on to the site?
Thank you for your help

Nodes will perform node discovery (https://bitcoin.stackexchange.com/questions/3536/how-do-bitcoin-clients-find-each-other) to connect to many other nodes. They will then begin downloading the blockchain from all of those nodes. That is how they get the blockchain, not downloading some master copy off of a master node. The other data you want to store (user logins, addresses, etc.) do not need a blockchain, it is far more efficient and easier to just make a centralized database for all of that because that is what a website will end up being.


text:
Single-hop Payment vs Multi-hop payments
I am beginner of lightning network. When i am looking roughly lightning network whitepaper, i was confused about the types of contracts used in single-hop and multi-hop.
As far as I understand, In single-hop(Bidirectional Payment channel) case, it use revocation key concept, and there is no HTLC. And In multi-hop case, it use HTLC concept.
But When I read the example in chapter 4 of the white paper(Example about Alice and Bob), i was confused fact that 'is single-hop case use HTLC?'
Anyone can explain this?
Thank you!

First of all at this point in time the whitepaper is pretty outdated and should only be considered as a rough conceptual guide. The Lightning Network Protocol itself is better and more accurately described / specified in the lightning rfc
That being said the situation is that even in the single-hop payment case HTLCs are being used. To set up and settle htlcs revokation keys and the revokation system currently play a vital role. The reason is as far as I know to save implementation overhead.
So if Alice wants to pay Bob in a direct channel the process is like that.

Bob creates an invoice with a payment_hash that is the hash of some pre_image
Bob somehow passes this invoice to Alice. (this can happen over any communication channel)
Alice creates an onion package (which basically just contains a 1 hop payment to Bob
Alice sets up an htlc by sending the onion via an update_add_htlc message to Bob. Alice also offers a signature for the next commitment transaction via the commitment_signed message.
Bob decrypts the onion. sees the payment is for himself and he that he can fulfill the onion.
Bob revok's his hold state by sending a revocation key to alice via the revok_and_ack message.
Bob sends signatures for Alice's new commitment transaction via the commitment_signed message.
Alice revokes her old state with the revoke_and_ack message.
the htlc is settled in a similar way as steps 4 through 7. (It starts with an update fulfill_htlc message send from Bob to Alice and then the state is updated for Alice first and then for Bob. Again recokation keys will be exchanged to get rid of the htlc output and create the state in which the payment has arrived at Bob)

A visualization of this process can be found in slides 136 through 144 of this slide deck. The process itself is described in more detail in BOLT 02
I should also note that there are other suggestions to for a payment channel protocol (for example Eltoo) which get rid of the penalty based system and the revokation keys but still use htlc's for actually transfering funds (either in multi or in single-hop payments)
Wellcome to this community I hope this technical and detailed answer is not too overwhelming. Please feel free to ask more question if necessary.

So In the case of two party with direct payment channel, Are they use combination of RSMC, HTLC contract??

Yes. The RSMC is encoded to all HTLC outputs because also those promised funds will be claimed in case of a channel breach

ok. in RSMC there is a timelock concept to impose penalty to party that unilaterally shut down the channel. Also, In htlc there is timelock concept.
So.. what is purpose of the timelock concept in htlc? and
what is relationship between two timelock concept (RSMC, HTLC)?

I guess this should be a seperate question. Would you mind posting it? Otherwise I will do it and link it here

Oh i understand after read this https://upload.wikimedia.org/wikipedia/commons/b/b7/Introduction_to_the_Lightning_Network_Protocol_and_the_Basics_of_Lightning_Technology_%28BOLT_aka_Lightning-rfc%29.pdf

One reason to use HTLCs for single-hop payments in the Lightning Network would be to make them indistinguishable from multi-hop payments to the receiver. This way, the receiver does not know whether it was their direct peer who initiated the payment, or whether they are merely forwarding the payment on for somebody else. Since they look identical to the receiver, the receiver must assume that some other node, who they know nothing about, initiated it.
The way in which HTLCs are created is also useful even for the single-hop use case, because they extend the RSMC model with conditional payments. With RSMCs alone, if Alice wishes to pay Bob, she creates a new commitment transaction by reducing the channel balance paid to her, and increasing the balance paid to Bob via the RSMC output. This transaction is then signed by Alice. Assuming that nobody misbehaves, Alice has effectively already paid Bob. There is no way this payment can be undone by Alice - for example, if Bob fails to deliver a service which was expected for the payment.
In the HTLC case, the payment is conditional on Bob surrendering a secret which is the pre-image of a previous agreed upon hash. Alice signs the HTLC with the condition built in - in such a way that it is revocable by Alice after a time-out if Bob fails to deliver the keys. The rest of the balance of the channel beside the amount paid in the HTLC is unaffected by this payment and behaves as in the single-hop case.
These conditional payments have several advantages. They can be used in things such as atomic swaps and submarine swaps - where the receiving of a pre-image is sufficient to claim a balance of goods elsewhere (another cryptocurrency for example). There are probably many use cases which aren't even explored yet.

Though we should point out that Atomic swaps with htlcs as we currently know it yield the free option Problem as pointed out by ZmnSCPxj: https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-December/001752.html

That's true for different currencies, but if the two chains are the same curerency (for example, converting from Bitcoin to the Liquid Bitcoin sidechain), then there won't be an issue because they don't trade at different rates.

Yeah it is really cool how lightning can be used to integrate side chains!


text:
BIP 144 transaction format and soft-forks and old nodes
I was trying to manually parse a raw Bitcoin transaction in order to better understand how it works.
This is how it starts:
"0200000000010141035625cd030eca667c95d1729a"
I think I've parsed the Tx version just fine but then the next byte was 00. I imagine this has something to do with BIP144. Correct?
If so in BIP 144:

Parsers supporting this BIP will be able to distinguish between the old serialization format (without the witness) and this one. The marker byte is set to zero so that this structure will never parse as a valid transaction in a parser that does not support this BIP. If parsing were to succeed, such a transaction would contain no inputs and a single output.

Does this means that old nodes do not relay Segwit transactions? They will think they are invalid? But once they are put in a block it's already in a format that an old-node can understand?
Sorry for the messy question.

This is correct. As an upgrade mechanism, nodes do not relay or mine transactions which they know that they can not correctly validate. The P2P network expressly avoids situations in which it could cause invalid transactions to be included in a block. It will however accept these transactions in a block.

Thank you! I guess I'm on the right path! YEAH! :)

This response is correct in what it says but it's a little misleading.  The reason old nodes don't accept/relay segwit using txn is unrelated to the format, it's because they violate the cleanstack standardness rule.  The reason that they're accepted in a block is because they get a copy of the block recoded into the old format.

@G.Maxwell could you elaborate the answer or point me to some more detailed information? Maybe even write your own answer if you feel like it. Thank you very much.

When an old node connects to a new node the new node detects that the old node doesn't support the new transaction format and so it converts any transaction it sends it-- e.g. segwit transactions in blocks-- to the old format.  The old format does have the witness data fields in it, but old nodes don't have any requirements for witness data so they don't mind.
The format nodes send transactions to each other is not a property of the Bitcoin consensus... you and I could use a totally different format than some other nodes. No one else would even need to know if we used some other encoding... so long as it didn't prevent us from communicating every valid txn.
New nodes also don't forward segwit using transactions outside of blocks to old nodes at all. They could-- but old nodes would just drop them because they can detect that they're using "future functionality".   Nodes don't relay, mine, or show in their wallets transactions that use intentionally backwards compatible extensions because they know they don't know how to adequately validate those rules. This prevents an unupgraded miner, for example, from accidentally mining an invalid transaction that was invalid due to a newly introduced rule.  If txn using 'future functionality' show up in blocks they'll happily accept them so long as they're otherwise consistent with all the rules the node knows how to enforce.
There are several kinds of 'future functionality' some examples are NOP opcodes and unprocessed data being left on the script stack.  Old nodes see segwit transactions as leaving data on the stack.

OK, I think I'll have to give it some more thought before it gets crystal clear. Thank you!
Just as a quick question.

so it converts any transaction it sends it-- e.g. segwit transactions in blocks-- to the old format.

I was under the impression that the serialized format from getrawtransaction was the same going through the p2p network and getting in the blocks? So that is wrong?


text:
Would it be possible to create a soft-fork for reducing miner reward?
Context:
There is a bet going on r/bitcoin about the possibility of doing a user activated soft fork for reducing current miner reward to 0.25 BTC.
One of the users says the bet had two points:

the change must be a soft fork (aka it doesn't break consensus)
no miner must be able to claim more than 0.25 btc.

Would it be able to make a change to the Bitcoin consensus rules that would meet this criteria?

Maybe I'm not sure what "soft fork" means exactly in this context.  But what about a scheme where > 50% of miners agree that they will not mine on top of any block with a reward larger than 0.25 BTC?  Non-mining nodes don't have to upgrade; they might accept blocks with higher rewards from non-conforming miners, but those blocks will eventually be orphaned.  And because of the 100-block maturation time, no transaction that spends a higher reward will ever be valid.

Yes, it is possible. The rule regarding the coinbase reward is that a miner cannot take more than the block subsidy plus the transaction fees in the block. This means that a miner can opt to take less than the full reward that they are entitled to. This has happened before in Bitcoin a few times. The coins that they did not collect are gone forever.
So, you can use this rule to your advantage if you want to decrease the block subsidy via a soft fork. In your soft fork, you simply create a new rule which results in the coinbase reward being less than the current coinbase reward (e.g. the block subsidy is smaller). Because of the rule mentioned earlier, non-upgraded nodes will still accept blocks that have these coinbase transactions with a smaller coinbase reward. Thus this is a soft fork because the new rule is backwards compatible.

The way block reward is computed today is :
CAmount blockReward = nFees + GetBlockSubsidy(pindex->nHeight, chainparams.GetConsensus());

Which means that the block reward is the total fees in the block plus the current base block subsidy.
It's definitely possible to change this line to (pseudocode) :
CAmount blockReward = std::min(nFees + GetBlockSubsidy(pindex->nHeight, chainparams.GetConsensus()), 25000000);

This will be a soft fork which doesn't allow the reward to be larger than 0.25 BTC
To explain what I mean in English, the reward is set minimum taken between the current block subsidy + fees, or 0.25 BTC.  This is a constraint on the current rules (where the base subsidy is larger than 0.25 BTC), and forward compatible with the period in the future when the base subsidy is lower than 0.25 BTC.
Adding @pieter-wuille's point from the comment, the current rules don't limit a miner in how low they can set their reward to be (where the minimum is zero), only how high.  That means that a miner doesn't have to reward themselves with the maximum allowed reward.  Such occurences have happened on chain before :
Rootstock accidentally set a zero amount as their reward :
https://www.smartbit.com.au/tx/9bf8853b3a823bbfa1e54017ae11a9e1f4d08a854dcce9f24e08114f2c921182
The first satoshi taken out of money supply :
https://www.smartbit.com.au/tx/5d80a29be1609db91658b401f85921a86ab4755969729b65257651bb9fd2c10d

How is it possible to reduce/change block subsidy without breaking consensus? That seems impossible without a hardfork. Limiting included tx’s may be possible, to limit fees.

The fees, should they push the reward to be higher than 0.25 btc are simply discarded (effectively burned - taken out of circulation entirely).  This is soft fork compatible and already happened on chain.

Got it. Fees yes. Not subsidy.

@James C: the consensus rule is that the coinbase output cannot be more than the reward. It has always been allowed to be less (and there are plenty of examples of miners forgetting to claim some fees for example with badly written software).

Ah @PieterWuille I did not known it could be less :) Thanks

Changing the block subsidy amount and/or schedule is not possible without breaking consensus.
It can be possible to limit tx’s included in a block to limit the fees received by the miner with a soft fork. Empty blocks can still be valid.

It's not an arbitrary change. It's reducing the amount of coinbase reward.

Yup my mistake. I understand now.


text:
Bitcoin Wallet generation and TX signing
I am developing a Bitcoin Wallet in Javascript. I've worked on ETH but it's my first time on BTC. Here are following queries if anyone can help:

How do we generate bitcoin Wallet like we do in Ethereum. PrivateKey -> PubKey -> Address and Private Key is used to sign a rawTx, searlize and then broadcast. What I'll need to make this happen in Bitcoin.
I've gone through docs of some libraries Bitcore, Bcoin and cryptocoinjs but all of them tell to connect to a node(test/live) to generate wallet etc.
Is there any way(lib) to generate a wallet, Load wallet from privatekey, sign TX from private keys all of this offline,and then propogating this to node. How this is done in Bitcoin.
What Bitcoin node can i use for mainnet?

There is a lot here, but I would have a look at https://github.com/bitcoinjs/bitcoinjs-lib it has examples, and can do all you are looking to do. It requires a connection with a bitcoin node via RPC. Alternatively, Bitcore will run its own implementation of a full node.


text:
bitcoin full nodes vs simplified payment verification
Why do individual nodes run "full nodes" that keep track of the entire blockchain whereas mechants can get away with "lite node" that implements "simplified payment verification" needing to examine only the last few blocks?

If by individual you mean users I would argue the contrary as many wallet are SPV based or have a centralised backend running a full node (form of lightness as any client does not run a fullnode).Only few desktop bitcoin wallet such as bitcoin core or armory run as fullnode. Fullnode are necessary for miner but it's still good for anyone to run it as it does apply rules on new blocks/transactions whereas SPV's node doesn't which result with an healthier network. SPV's node needs full node to exist but worth to keep it mind that an SPV has to trust its peers as it does not verify/apply rules.
Quoting Mastering Bitcoin:

An SPV node can definitely prove that a transaction exists but cannot verify that a transaction,
  such as a double-spend of the same UTXO, doesn’t exist because it doesn’t have a record
  of all transactions.

[...]

A full blockchain node verifies a transaction by checking the chain of
  thousands of blocks below it and checks that the UTXO is not spent,
  whereas an SPV node checks how deep the block is buried by a hand‐
  ful of blocks above it.

A typical a merchant accepting bitcoin would rather be 100% that there is no attempt of double-spending which is not possible with an SPV's node.


text:
Do I need to develop a separate blockchain for my altcoin (cloned on BTC)?
Hi I am looking to develop an altcoin or more specifically to clone Bitcoin / litcoin
I have few questions in mind before starting that I’d really appreciate if someone helps.
Do I need to develop a separate blockchain or it will use the blockchain of BTC / LTC?
Do I need to construct a wallet or it will be created once I compile the code?
How would I be able to mine the coins?
Where to put the code once the coin is generated
I have basic understanding of C++ and will hire a blockchain developer as well but didn’t find one here in my country (I am from Pakistan)

Do I need to develop a separate blockchain or it will use the blockchain of BTC / LTC?

That's up to you, you could fork those, BCC is an example of a forked but separate currency. Alternatively, you can start a fresh blockchain.

Do I need to construct a wallet or it will be created once I compile the code?

"A" wallet? you need many wallets, your users will need to create wallets and not by compiling code. This question makes it sound like this is your first foray into cryptocurrency? If so this is not a good first move or a beginner project.

How would I be able to mine the coins? Where to put the code once the coin is generated

Yeah, I think you need to do more research. This project is not a small undertaking, you should at least have some experience of mining other coins, and using coins.


text:
Is 'hawala' and Bitcoin similar?
Hawala is an old indian/middle east , money transaction system , based on mutual trust, and uses an encryption system based on spoken keycodes.How is it similar to bitcoin transaction system?

The major difference is, while Hawala means trust, Bitcoin does not depend on trust. Everything is made of cryptographical signatures, making it 100%** safe, while Hawala cannot / may not be trusted.

** Two exceptions: 1) Majority of hashrate is attacking the network. See this 2) The theoretically possible, but practically impossible hash collisions. It doesn't happens, while the theoretical safety isn't 100%, rather 99.999999999...%

100% safe !what do you mean? Its a very well known fact that un hackable software/programs never exist, how do you say Bitcoin is protected then?

If there are multiple node implementations and if none of them has 50% of the network, as they're implemented by different developers, there won't be a bug that affects all of them and even if the a node software crashes, the other nodes will stay work and network will stay safe.

And the private keys can be cracked if some people manage to build a 1500 qubit quantum computer in far future. @Quantumkiller

great ,thanks @MCCCS  i heard 1000 qubit computers are on the cards!

@Quantumkiller Last paragraph: https://bitcoin.stackexchange.com/a/57977/38618 I also heard that the 1000 qubit quantum computer exist now, but do you know why they work on both the 50 qubits one and the 1000 qubits one at the same time? I heard that the 1000 qubit one is a special qcomputer that can do only a few things. Even if the qcomputers are able to break ECDSA (private key- address system) your money is safe as long as you avoid address reuse (if RIPEMD is still not cracked). More info here: https://en.bitcoin.it/wiki/Quantum_computing_and_Bitcoin

@Quantumkiller https://www.reddit.com/r/btc/comments/70v78b/cryptography_as_we_know_it_will_be_completely/dn64cfn/


text:
Bitcoin-QT Windows Safety Concerns
According to https://www.file.net/process/bitcoin-qt.exe.html
"The application uses ports to connect to or from a LAN or the Internet. Bitcoin-qt.exe is able to connect to the Internet, record keyboard and mouse inputs and monitor applications. Therefore the technical security rating is 31% dangerous."
It is my understanding that when port 8333 is enabled the node shares its blockchain data with other nodes through the network, however this is not a requirement to run Bitcoin Core and is not enabled by default. Is there any other ports used or possible vulnerabilities when a node is not configured to transmit outbound information through port 8333?
Also curious if anyone can elaborate on these safety concerns particularly relating to "record[ing] keyboard and mouse inputs and monitor[ing] applications"?
Thank you

It is my understanding that when port 8333 is enabled the node shares its blockchain data with other nodes through the network, however this is not a requirement to run Bitcoin Core and is not enabled by default.

Your understanding is incorrect. Bitcoin Core is an application that requires network access to work. Outbound connections are always possible and do not require port 8333 to be open or available. Port 8333 is only required to be available and open to the internet if you want to receive incoming connections. Listening on port 8333 is on by default, but you typically need to configure your firewall(s) to allow incoming connections to port 8333.

Also curious if anyone can elaborate on these safety concerns particularly relating to "record[ing] keyboard and mouse inputs and monitor[ing] applications"?

Bitcoin Core has keyboard and mouse input in order for people to actually be able to use it, therefore it has to be able to "record keyboard and mouse inputs".
I don't know what they mean by "monitor applications". That's a pretty vague description and could mean a lot of things.


text:
At today's price how much would$500 dollars be worth in Bitcoin if it goes to $20,000 dollars again
Ok you have $500 dollars to invest in Bitcoin at today's price$3,421 dollars and say Bitcoin surges to$20,000 dollars again how much money would you now have.

0.146156094709149 Bitcoin

I'm voting to close this question because it is about basic math.

Right this second, bitcoin is at about $3,454. So $500 would get you about 0.145 bitcoins. At $20,000 per bitcoin, that would be worth about $2,900.


text:
old wallet.dat with only incoming tx from mining, now shows 0 coins even after rescan and all the transactions are there
I found an old wallet.dat it contained hundreds of transactions while i was mining.
I've replaced the wallet.dat in a new Bitcoin Core wallet, rescanned, all the txid are showing but the amount of bitcoin stored there is not! It now shows 0 coins
I know i never sent them out + all the tx are inputs nothing is out, so far i tried rescan.
any idea how can i extract for example a master privat key for the entire wallet, and import it somewhere ?

Have you verified the tx ID exists on an explorer? Wallet files are often compatible across coins, you may have been mining litecoin or something else

yup, all tx are containing very small amounts (bitcoin) i used the dumpwallet command to get all public+privat keys turned out i got over 2800 transactions (my head hurts) so this command also give me an extended private masterkey (starting with: xprv) so my question is, how can i convert this extended key into simple privat key so i can sweep it into another wallte and get my coins out (if this is how it works, or maybe im wrong)

may be using  --reindex flag and adding txindex=1 in conf file solve your issue of showing 0  Coins during scanning process

@cryptoKTM this seams something worth giving a try, is there a difference if i add  -reindex flag to the end of the "Target" parameter or just -reindex ? also it appears i dont have a bitcoin.conf file. i looked everywhere since my folder is in not the default directory. What can i do to creat one so i can add thxindex=1 command

you have to create bitcoin.conf file in the data dir, by defult data directiry is
/home/user/.bitcoin
and
bitcoind -daemon  --reindex
is the command to reindex it

If you found an old wallet (prior to Core 0.13 which was released in August 2016), it wasn't using BIP32 HD. So there is very very likely no master private key,... instead, there are a large number of private keys.
You can use dumpwallet <filename> in the GUI console or via RPC to extract your private keys to a text file and import them into a different application.
But if you see transactions in Bitcoin Core and your balance is zero, this mostly means that there must be one or multiple spending transactions that have spent your mined bitcoins.


text:
Blockchain endpoint working
I wanna clear this thing that how blocks are retrieved and created in from the nodes using an end point. The end point is basically centralized and have nothing to do with network if i'm not wrong. Is that end point is also running some kinda setup to keep blocks in sync as well as connected nodes?
In a network we have nodes [B,C,D,E,F,G...] and we have endpoint A. Lets say we have to retrieve a block then on which node the request will be sent?
Take example of STORJ in that request lands on one server(kinda end point) only then it broadcast to all. Isn't the centralized system.

I don't know anything about STORJ, but decentralized blockchain systems do not have a centralized 'endpoint' or whatever; nodes always communicate directly with each other.

What about the retrieval and showing in web! I l know how it goes to mempool and all that broadcasting stuff but point is on GETTING the blocks in the form of JSON from which node it will be sent

Anyone can run a website. When you interact with someone's website, you're effectively using the operator's node, and trusting them.

Mean that website also running its own node?

Yes, of course. Otherwise it can't interact with the network.

SO! to add a node to the network how this process works. Mean on what node it interacts with initially to be added to node and sync the whole chain?

That sounds like a topic for a different question. You may want to search the site for questions relating to dns seeders.

Disclaimer: I hold STORJ. (Not that it would have any impact on the answer)
What you're looking at here is an ERC20 token. So your question revolves around the Ethereum blockchain.
When a new block is mined, it is transmitted to the nodes on the network. This is accomplished by the miner transmitting to his node, and his node transmits to other nodes, so forth and so on.
The idea of "end points" you are asking a node, lets say your node, or a 3rd party providers node for information. In an ideal world all nodes have the same exact information. In the real world they can be different but the node software resolves these differences using "longest chain wins". That's why we wait for confirmations. Ethereum works a bit differently as it uses a "confirmed" flag instead of number of times it's seen since the original block it was first found but this is the Bitcoin forum.
With Bitcoin you can ask any node for "a block" but i think you mean "a transaction". You simply don't consider a "transaction" or "block" to be valid until there are 6 blocks after it. By then it's statistically impossible for it not to "Exist" on the full chain.
In your final example: Storj doesn't really "land" (or any token/coin on any chain). A transaction is submitted to mempool which is then broadcast to other nodes. A miner sees the transaction, mines it, and adds it to a block. That block is then rebroadcast. There is no way for you to "spend" anything because someone else has to confirm that the transaction is valid and add it for it to show up on the actual chain.
Note: Apologies for any confusion. The question was a bit awkward but did my best to answer the various points in it. I highly suggest investigating how blockchains work and why they work. Bitcoin is a fantastic place to start since this all started with Bitcoin.

But on which node request falls while making a transaction or while retrieving the block and showing in the web or application?