text:
What are these bitcoin nodes and what purpose they serve?
Explain like I'm 5 years old and my other half of the brain is surgically removed :) I don't want to put a lot of stress on my brain
1.What are these bitcoin nodes and what purpose they serve?
2.and also why they are still running?
3.and does miners use them to mine or validate transaction anymore?
4.how bitcoind is related to any of them?
0-Bitcoin Core
1-Bitcoin Unlimited nodes
2-Bitcore nodes
3-Bitcoin UASF nodes
4-btc1 nodes
5-Bitcoin Knots nodes
6-bcoin nodes
7-Bitcoin Classic nodes
8-btcd nodes
9-Bitcoin XT nodes
10-TRB nodes
11-libbitcoin nodes
12-Bitcoin ABC

The rules of bitcoin, much like the rules of a board game, are fixed. In board games, there is an instruction booklet that tells you what the rules are, and anyone not adhering to the rules is breaking them, which is known as cheating.
Bitcoin's rules were laid down by Satoshi Nakamoto, its inventor, and its rules are enforced by each player (node). Nodes check if the blocks are constructed according to the rules of Bitcoin, and punish miners who construct cheating blocks.
Having played Catan quite a bit, I've found that some people enjoy playing with house rules. Some people feel that every player should start with at least one of each resource, for example. These rules can be enforced by the whole group, as if they were in the instruction manual.
The same is true for bitcoin. Some people feel that the Bitcoin protocol is better suited by having small additions to the rules, or notable omissions. These people are free to write their own software, which enforces their own rules on Bitcoin.
Some are just translations of the bitcoin instruction manual to another language. Some aim to change fundamental parts of the protocol, such as the btc1 nodes, or Bitcoin XT nodes.
edit: The most-used client is bitcoind, which is the command-line interface of "Bitcoin Core". Due to the fact that a majority of people are using this client, it dictates the current consensus rules (the rules of the game), and other clients either adhere to it, or risk getting "forked".
Node software has been politicized these past few years. This is a different topic altogether, and I recommend you google "the bitcoin scaling debate" for a primer.

so what nodes majority of miners use to mine and validate TX.. and how many nodes failed or dead .. due to not meeting consensus?

Quite a few nodes failed, though people can still choose to work with them as long as they adhere to the consensus rules used by the majority. Exact figures are difficult to find, but you could keep an eye on https://coin.dance/blocks for an overview of the last 1000 blocks. Miners tend to list their client in the block message.

if that is true than BitcoinABC is used for mining bitcoin cash and BitcoinCode for mining Segwit and old Bitcoin Tx .. right? also recently i keep hearing these nodes 1. Bitcoin knots 2. Bitcoin Unlimited? .... or this means that node B5(blocksize 5) is willing to accept B5 type of this tx however it will also accept and verify tx made for bitcoincore and segwit?

The bitcoin network is made up of many nodes, as well as miners. If you are a user you can run a node to verify that you are connected to the bitcoin network and your transactions are confirmed.
There are a few different implementations of the software, such as the ones you listed. All of the different implementations must adhere to the strict consensus rules defined in the bitcoin protocol, or else the rest of the network will ban them. The code that is not consensus critical can be altered, so each of the implementations may be a little different.
Most big miners probably run their own modified versions of the code, to suit their needs best.

"All of the different implementations must adhere to the strict consensus rules defined in the bitcoin protocol," - you mean they must at least support core requirements like (A segwit-compatible wallet MUST support pay-to-script-hash (BIP16) and its address format (BIP13)...) as specified whenever they release BIP

No, the consensus code pertains to things such as the amount of bitcoins included in the block reward, the mining difficulty, the type of proof-of-work required, the block size limit, etc. A node doesn't have to explicity support making a certain tx type, it just has to play by the same basic rules as the rest of the network.

A node doesn't have to explicity support making a certain tx type.. ..... however thats not the case with segwit? it aggregates witness in its merkel tree

The activation of segwit was a backwards compatible softfork: old nodes that don't explicitly support segwit still accept all segwit nodes/traffic as being in consensus with the network.


text:
Is it a 51% attack or a >50% attack?
Is it 51% attack or 50.0000...1% attack?
Bitcoin is getting to a point where 1% is a lot of hashing power...
My understanding is >50% due to the nature of the protocol.

Actually it's possible attempt an attack with less. You could be successful in punctual double-spends with less than 45% of the total hashing power, but it requires a bit (if not a lot) of luck.

50-51% is the point where it becomes essentially trivial to have continued control over block creation.

this actually makes a lot of sense. you should post this as an answer!

When performing a 51% attack, you will need to find new blocks faster than the remainder of the network combined. This way, your block chain will become the longest and clients will consider it the main chain.
Theoretically, this is achieved (on long term) when you have >50% of the hash power.
But as we know, a lot of luck is involved in mining. It's possible that someone with 55% or even 60% of the total hash power will only succeed in his 51% attack after many weeks because he has bad luck. (Note that with >50%, statistically you will always succeed on long term.) But it is also possible that someone with less than 50% can succeed on short term.

In most contexts referring to majority rule, 50% + 1 of votes is generally the right way to describe the majority. 51% is shorter and less confusing though.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
Why does the Genesis block coinbase input script push bits and the value 0x04 before the famous "Chancellor" message?
According to the Wiki, the input script of the Genesis block consists of these bytes:

04FFFF001D0104455468652054696D65732030332F4A616E2F32303039204368616E63656C6C6F72206F6E206272696E6B206F66207365636F6E64206261696C6F757420666F722062616E6B73

We can break this down as follows:

0x04: push 4 bytes
0xffff001d: the same value as the "bits" field of the Genesis block header
0x01: push 1 byte
0x04: push the value 4
remainder: ASCII encoding of the famous message "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks"

As far as I know, the first two values pushed to the stack (0xffff001d and 0x04) are unnecessary. Of course, so is the chancellor message, but at least its meaning is a bit more clear.
However, the push bits/0x04 pattern also appears in other blocks. For example, the same pattern shows up in the next block (0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098).
I found this question, with an answer that speculates:

So 010445 are in fact part of 2 operations - 01 pushes 04 to the stack, 45 pushes the message to the stack. As to why push 04, beats me, maybe it was just testing to make sure something works? Some internal counter used by Satoshi during Genesis block creation? Since the script of a coin generation transaction is really meaningless, we might never know.

My questions:

Are these initial push data operations necessary?
If not, why do these items (bits and 0x04) appear in more than one block?
Was adding this prefix to the coinbase transaction input script part of the default behavior of the original Bitcoin client?
If not, wouldn't this be a signature for blocks that Satoshi mined?

I found some more information on the Wiki:

Generations have a single input, and this input has a "coinbase" parameter instead of a scriptSig. The data in "coinbase" can be anything; it isn't used. Bitcoin puts the current compact-format target and the arbitrary-precision "extraNonce" number there, which increments every time the Nonce field in the block header overflows. Outputs can be anything, but Bitcoin creates one exactly like an IP address transaction. The extranonce contributes to enlarge the domain for the proof of work function. Miners can easily modify nonce (4byte), timestamp and extranonce (2 to 100bytes).

https://en.bitcoin.it/wiki/Transaction
In other words, the first Bitcoin client copied the "bits" field (compact-format target) from the block header and used it in a push data operation for the coinbase input script. But I'm still not sure why this was considered to be a good idea.
Next, a second pushdata operation pushed the "extranonce" field. This field incremented whenever the nonce field overflowed. The privacy implications of this system are discussed here:
https://bitslog.wordpress.com/2013/04/17/the-well-deserved-fortune-of-satoshi-nakamoto/
I believe I can now answer my own questions:

No.
Bits is there because that's just what Bitcoin did. 0x04 is the extranonce field.
Yes.
Yes, in part.

This behavior is part of the original client which you can see here: https://github.com/bitcoin/bitcoin/blob/v0.1.5/main.cpp#L2242. There's no particular reason to include the nBits field and it is not a consensus requirement to do so.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
transfer between different account
I'm living in Turkey and I bought some bitcoins from here.
If, for example, I have a neighbour in America and he bought bitcoin from a different platform, can I transfer my bitcoins to him? And can he transfer it to me?

Yes, Bitcoins are the same no matter where you bought them, and you can transfer them anywhere you want.
(One thing to watch out for, though, is that Bitcoin Cash is a different currency from Bitcoin, and you can't directly transfer one to the other.)

Why not? > you can't directly transfer one to the other

Because they are designed not to be interchangeable.  You can exchange one for the other, by finding someone willing to trade - just like you can exchange dollars for euros - but the exchange rate will vary.  You can't take 1 BCC and "transfer" it into 1 BTC.


text:
How can I pass Multiple addresses in Move command
I have to pass multiple addresses in move command but I am not able to pass as it is not accepting more than one wallet address Is there is any another way to move the balance to multiple addresses.

You can't pass multiple addresses to Move commands, also move will be removed in a later version of Bitcoin Core.
Please consider using sendtoaddress instead

No, don't use sendtoaddress. move is just moving internal bean counters in the wallet around. There is no equivalent for it. The correct advise is to first look whether you need move and accounts at all (hint: they're almost certainly not what you think they are), and then determine the correct equivalent. Perhaps that means using multiple separate wallets, or having your own application keep track of different users' balances.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What is the alternative of Bitcoin-RPC-Client?
I am implementing Bitcoin-RPC-Client for web application but I found bitcoin-RPC-client is better than bitcoinj and what are the advantages and disadvanatges of Bitcoin-RPC-client.


text:
Is it true that a bitcoin transaction can go unverified and eventually get dropped by the system?
I have attempted to transfer 0.5BTC from my personal wallet to Coinbase and it's been 5 days since I initiated the transfer. It is still pending. I started reading about this and learned that there is no guarantee in the bitcoin system to actually have any particular transaction verified. The transaction volume is high right now and transactions with higher fees get higher priority. And transactions with low fees get low priority and may not EVER get verified (or mined) before being dropped from the system. Is this true? That's a huge flaw in the system if it is. how can any kind of commerce happen with this "feature"?

There is also no official mempool, so technically each node can have their own policy on how long to retain a transaction. However nodes do get restarted from time to time, so the mempools do get cleared eventually.

You can also see here that the default mempool expiration time for Bitcoin Core got increased from 3 days(Bitcoin Core 0.13) to 2 weeks(Bitcoin Core 0.14).

My questions still stands as follows. Is there a guarantee that a transaction WILL get verified? If the transaction lingers unverified for a long time and then gets dropped, then this is not a very useful method of transacting business, is it? If I am beholden to some god-like entities (miners) for my transaction to be verified, then how is this better than central banks?

Bitcoin does not have, and cannot have a guarantee that a transaction will get verified. There is limited capacity, and possibly infinite demand. Miners have discretion to include whatever they want in blocks, and if you want to have your transaction included you'll need to incentivize them. However, you can always re-issue a transactions with a higher fee (see CPFP or BIP125), or just wait.

@Chak I disagree this is a duplicate. The fact that transactions expire from the mempool is an implementation detail that does not affect the question of whether transactions are guaranteed to confirm. They weren't before expiration was implemented.

Transactions with low fees get low priority and may not EVER get verified (or mined) before being dropped from the system. Is this true?

Yes.  However they can be resent, potentially with a higher fee.

That's a huge flaw in the system if it is.

Matter of opinion.

How can any kind of commerce happen with this "feature"?

You have to think about things a little differently.  Sending the transaction isn't the end of the story; confirmation is.  The sender has to be responsible for monitoring the progress of the transaction until it is confirmed, and to be willing to either increase the fee or eventually give up on the transaction as no longer economical.  Likewise, the recipient needs to not assume that a transaction is final as soon as it's sent; instead, they have to wait until it is confirmed before shipping goods or services.  Escrow services can help with this, playing a role similar to that of a credit card company.
I think everyone wishes the situation were simpler and that we had a system where nobody had to wait to have their transactions confirmed.  But we don't yet have such a system, and there doesn't seem to be any obvious design that would solve this problem while remaining scalable, decentralized, and reasonably easy to transition.  In the meantime, however, despite your concerns, an awful lot of commerce can and does happen with Bitcoin as it is.  People consider the limitations and adapt accordingly.

I am beholden to some god-like entities (miners) for my transaction to be verified, then how is this better than central banks?

Miners aren't centralized.  Anyone who buys appropriate hardware can be a miner.  So there are lots of miners competing to confirm your transaction, and you can expect that it will be confirmed just as long as you offer enough of a fee.  With a central bank or credit card company, they can block your transaction for non-economic reasons: you come from a country they don't like, your credit rating isn't good enough, etc, etc.  In that case you might not be able to make a transaction at all, or you might have to pay more than other people receiving similar service.
In other words, transaction confirmation isn't necessarily designed to be cheap or fast, but it is designed to be fair.

Miners aren't centralized - this is not really true. 3 companies are responsible for over 50% of all bitcoin mining in the world. I would consider that very centralized


text:
How many gigabyte(internet use) needed for bitcoin mining in a month?
if I use Antminer s9 full-time in a month, how many gigabyte uses?
I mean what is the internet usage((Download+Upload)) in a month?
comment about duplicate: first of all I want know the usage not bandwidth or speed of my internet, second, your mentioned linked is for near 7 years later. I want know how many gigabyte usage in a month with 14 Th/s. it is only a number which an expert can say, plz do not answer cursory.

@EricAllam first of all I want know the usage not bandwidth or speed of my internet, second, your mentioned linked is for near 7 years later. I want know how many gigabyte usage in a month with 14 Th/s. it is only a number which an expert can say, plz do not answer cursory.

You'll have to specify your pool to get an answer.  Your pool, as well as possibly your individual settings, will determine how often you receive work and submit shares, which is the main data usage for pool mining.  (If you're solo mining then the answer is completely different again, as you have to run a full node in that case.)  Your hash rate is irrelevant.


text:
Why aren't public-private keys sufficient to build a block chain? Why do we need proof of work/stake?
Taking the simple case of Bitcoin here:
If all nodes can validate published transactions by checking the signature (created using the private key of the sender) of the transaction against the public key of the sender, why isn't that sufficient to validate a block?
I understand that we also solve the puzzle of finding a nonce that when combined with a block's contents, gives us a hash that has N number of leading zeroes. Why do we need this at all? Isn't it enough to check the validity of each transaction in the block?
As for double spending, if a miner decides erroneously to include both transactions, nodes can see there isn't enough balance and reject such blocks. As for which transaction of the two to take, the code could have a way to order transactions and if an attacker decides to do the opposite intentionally and decides to spam the network (since there's no proof of work cost) but how does the attacker benefit from that?

the code could have a way to order transactions and if an attacker decides to do the opposite intentionally and decides to spam the network

This is the crux of the issue: how do you achieve this? It isn't a trivial problem to solve in a decentralized network. The more generalized version of this non-trivial problem is called the Byzantine General's Problem, it describes the ability of participants to come to an agreement in an untrusted network. The invention of bitcoin provides an economically-incentivized game-theoretic framework that could be considered a probabilist solution to this problem.  As @PieterWuille mentioned in the comments, there is no absolute guarantee of eventual consistency: there is only a (continually increasing, thus far) economic incentive for participants to consider.
The solution that proof of work provides is to rate-limit spamming via the requirement for expenditure of a scarce resource: energy. It is hard to create a new block (it takes some amount of energy, on average), but it is easy to verify if a block is valid. And remember that the method for choosing the next valid block must be perfectly consistent across all nodes in the network, or else you risk a network split/fork, which is very undesirable.

but how does the attacker benefit from that?

There are many ways an attacker could benefit. As a simple example: the attacker takes a 'short' financial position, betting against the price of bitcoin. They then attack the network, assumedly causing the network's value to drop, and thus they would profit from their short position.

just to add to your answer, split chain can still happen in PoW by honest miners and we fix this by taking the longer chain. So it seems like the main benefit of PoW is to solve transaction order problem (called "double spending"), is there any other benefit?

@SpaceMonkey I added a bit more to my answer. Chain splits can indeed happen, proof of work helps to guarantee an eventual consensus. The few blocks at the chain tip would not be considered as secure as much older blocks in the same chain.

Proof of work is not eventually consistent, at least if you mean the technical term from distributed systems - that would imply that after a finite amount of time a certain state is irreversibly agreed upon by all participants. Under PoW, there is always a chance of reorganization, regardless of how deep it is buried. What PoW adds is an incentive for miners to cooperate on a single chain rather than work against each other, and under certain economic assumptions, that means the chance of reorgs goes down exponentially as time passes - but it still never becomes zero.

@PieterWuille what is the incentive for miners to cooperate on a single chain? if miners receive transactions in different orders and create a split chain, what is the incentive for one group of miners to discard their version of the chain and adopt the other one?

Only the subsidy/fee income miners receiving in the winning chain matters. If miners build competing chains, only one of those will win.

but they are receiving their mining fees as long as they continue on their version of the chain. What makes them ditch their version and switch to the other chain?

They get a payout each in their own fork, but ultimately forks get resolved - and only one one chain will eventually be accepted by the network. The subsidies/fees received in the extinguished chain don't result in money that's acceptable as BTC.

@PieterWuille exactly, so how do the forks get resolved eventually? isn't it where the "honest" nodes accept only the longest chain?

In PoW, the most-work valid chains wins. But how this happens isn't relevant for your question: somehow one of the chains needs to win - or we'd end up with two separate currencies if they both survive, each with their own ecosystems around it. And yes, of course it is only honest nodes that can be assumed to follow the most-work chain, but they're also the only relevant ones - they're the ones miners hope to sell their earned BTC to! Note that nodes are not just miners.

Let us continue this discussion in chat.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
error while loading shared libraries: libboost_system.so.1.63.0: cannot open shared object file: No such file or directory
I am trying to use bitcoin core in airgapped cold storage with tails installed on a usb.
first i downloaded the 0.15.1 release bin files into tails storage.
i could run the bitcoin-qt just fine.
.......................................................
now i cloned the bitcoin core from github , added a commit from github.
again i copied the 4 files - bitcoin-cli,bitcoin-qt,bitcoind,bitcoin-tx from bin folder into tails.
But now when i try to start bitcoin-qt from tails, I get the below error :
"error while loading shared libraries: libboost_system.so.1.63.0: cannot open shared object file: No such file or directory"
shouldnt the bitcoin-qt file from bin folder work like an executable and run independently ?
please guide how to get the bitcoin-qt within the bin folder from latest github source to work on tails usb.

i had tried to build with make .
even after building - i got this error.


text:
I used multibit years ago - now what? (BTC, BTG, BCC etc...)
I have some old wallets from multibit era (~2013).  There is no way I can now download the full blockchain, so I need to import these keys into somewhere.  Worse still, I also need to import the same keys for BCC and BTG.
Try as I might, I can't seem to find clients that allow me to import old base58 keys.  I have a few files called something.key which came from multibit and are human readable with the private key in.  Everything seems to be "mobile" and "QR codes", "passphrases" and whatnot.
It's not a lot of money, and fragmented over multiple wallets, so with transaction costs stupidly high now, sweeping is infeasible.
How can I spend these old coins?  Is the only way to free up 150GB and wait?

Edit:  Ultimately I just want to create a transaction from all these wallets, to send the coins to an exchange.  So sweeping to a new wallet just doubles all my transaction costs for no reason.  Also I need a way to repeat the process for Bitcoin Cash and Gold...
Is there no simple tool to just "create" a transaction?

@fredsbend well at the very least i know a can do an import if i run a full node of each coin, so I'd rather do that for btc than pay the fee!

@fredsbend yeah i would only do it fot btc. Its only about 0.05 Btc but over a handful of different addresses so it will soon add up. And it's the principle dammit! :)

So I found a solution that worked for me:
BTC
Blockchain.info have online wallets but you can also just create your own transactions without using their wallet.  Add an address using import and then spend them directly using your private key.  You can manually set the fee and i managed to transfer with ~75sat/byte even though recommended was 225.
BCC/BTG
Transaction costs are still small for these so sweeping works fine using practically any wallet.


text:
I have bitcoins in Poloniex. How do I get them to CANADIAN dollars?

Coin base doesn't allow transfer to a CANADA account and Kraken's authentication is down.
it looks like I can get money out of Quadrigacx?? but how do I transfer my bitcoins from Poloniex to Quadrigacx
am I totally off and there is another way? I'm ready to cash out and would like my bitcoin in cash!

Thank you!!

it looks like I can get money out of Quadrigacx?? but how do I transfer my bitcoins from Poloniex to Quadrigacx

You will need to register and verify your account with Quadrigacx. Next obtain a deposit address for your Quadrigacx account, and initiate a withdrawl from Poloniex to the Quadrigacx address. After some time, your Quadrigacx account will be credited with the bitcoin.

am I totally off and there is another way? I'm ready to cash out and would like my bitcoin in cash!

You could find someone to sell them to in person. There are some websites that facilitate this, such as Local Bitcoins.


text:
How to create an offline wallet on Android only to sign transaction
I am new to bitcoins, I want to create an offline wallet on Android, to scan transaction QR code from online Desktop wallet, Sign that transaction with private key on my offline wallet.
1 Creating offline wallet.
2.Signing transaction offline with private key.
I want to sign transaction and generate QR code from that sign transaction.But after signing i get a sha256 in this form "MEQCIGBVDN/PkbESZdWkG6/KzrDRAEpDXVdsjKMzErBfFIWYAiA8JJOv97Dlp8Acg/L8JHI3RzoW                                                                       eYNxPW1Lx4wQaORNNQ==", so what will i do with this. So please help and review my code. Is my signing process ok or if not please comeup with some code.Thanks
So for i did the following.
// For creating wallet
private void InitilizeWallet()
throws IOException {
    BriefLogFormatter.init();

    params = TestNet3Params.get();

    filePrefix = "forwarding-service-testnet";

    walletAppKit = new WalletAppKit(params, getCacheDir(), filePrefix) {
        @Override
        protected void onSetupCompleted() {

           if (wallet().getKeyChainGroupSize() < 1)
                wallet().importKey(new ECKey());

            deterministicKey = wallet().getWatchingKey().dropPrivateBytes();
            deterministicKey = HDKeyDerivation.createMasterPubKeyFromBytes(deterministicKey.getPubKey(), deterministicKey.getChainCode());
            xPublicKey = deterministicKey.serializePubB58(params);    privateKey=wallet().getKeyByPath(DeterministicKeyChain.ACCOUNT_ZERO_PATH).getPrivateKeyAsWiF(params);
            Log.e("key", xPublicKey.toString());
            Log.e("privatekey", privateKey.toString());

        }
    };

    if (params == RegTestParams.get()) {
        // Regression test mode is designed for testing and development only, so there's no public network for it.
        // If you pick this mode, you're expected to be running a local "bitcoind -regtest" instance.
        walletAppKit.connectToLocalHost();
    }
    // Download the block chain and wait until it's done.

    walletAppKit.startAsync();
    walletAppKit.awaitRunning();

}

And for signing transaction i do the following.
public void Createtransictionhash(String recipientAddress, String amount) {
    try {

          // i am getting address and coins from QR code

        SendRequest request = SendRequest.to(Address.fromBase58(params,
        recipientAddress), Coin.parseCoin(amount));
            Signingtrasaction(MainActivity.privateKey,request.tx.getHashAsString());

        Log.e("txhash", request.tx.getHashAsString());

    } catch (Exception e) {
        Log.e("msgError", e.getMessage().toString());
        Toast.makeText(getApplicationContext(), " Version code of address did not match", Toast.LENGTH_SHORT).show();

    }
}

public void Signingtrasaction(String wif, String msg) {
    try {

        // message (hash) to be signed with private key
        //String msg = "15953935a135031bfec37d36a9d662aea43e1deb0ea463d6932ac6e537cb3e81";
        //my hash = 09b14f746bd0a93b71907ba0070a103adbee7b1a260e053a21aa0b660ad8de57
        // an example of WiF for private key (taken from 'Mastering Bitcoin')
       // wif ="KxFC1jmwwCoACiCAWZ3eXa96mBM6tb3TYzGmf6YwgdGWZgawvrtJ";

        // creating a key object from WiF
        DumpedPrivateKey dpk = DumpedPrivateKey.fromBase58(params, wif);
        ECKey key = dpk.getKey();

        // checking our key object
       // NetworkParameters main = MainNetParams.get();
        String check = key.getPrivateKeyAsWiF(params);
        System.out.println(wif.equals(check));  // true
        Log.e("wif check", String.valueOf(wif.equals(check)));
        // creating Sha object from string
        Sha256Hash hash = Sha256Hash.wrap(msg);

        // creating signature
        ECKey.ECDSASignature sig = key.sign(hash);

        // encoding
        byte[] res = sig.encodeToDER();

        // converting to hex
        //String hex = DatatypeConverter.printHexBinary(res);
        // String hex = new String(res);
        String hex = android.util.Base64.encodeToString(res, 16);

        Log.e("sigendTransiction", hex.toString());

        Log.e("decrypttx",""+ Hex.decode(sig.encodeToDER()));

    } catch (Exception e) {   //signingkey = ecdsa.from_string(privateKey.decode('hex'), curve=ecdsa.SECP256k1)
        Log.e("signing exception", e.getMessage().toString());
    }
}

you are doing all wrong from starting, do not use WalletAppKit use simple Wallet class, its offline wallet not Online, WalletAppKit needs internet to sync with the blockchain
public void wallet()
{
    wallet = new Wallet(params);
    try
    {
        if(isDirCreated())
        {
            System.out.print("New Wallet");
            wallet.saveToFile(walletPath);
            System.out.println(wallet.currentReceiveAddress());
        }
        else
        {
            System.out.println("Load Wallet");
            // do something here
        }
    }
    catch (IOException e)
    {
        e.printStackTrace();
    } catch (Exception e) {
        e.printStackTrace();
    }
}


text:
Format of a block key's contents in bitcoind's LevelDB?
I am trying to use bitcoind's LevelDB block index files (that are stored in blocks/index) to find the file (e.g. blk00029.dat) and byte position of specfic blocks.
Thanks to questions What is the database for? and What are the keys used in the blockchain levelDB (ie what are the key:value pairs)? I know where to look, however the contents of the database key for a block puzzles me. Even with the help of the source code I cannot figure out it's format.
For example for the block at height 100,000, which resides under database key b\x06\xe53\xfd\x1a\xda\x869\x1f?l42\x04\xb0\xd2x\xd4\xaa\xec\x1c\x0b \xaa'\xba\x03\x00\x00\x00\x00\x00 (block's hash string converted to bytes in  little endian order), I am getting the following content which I hexlified for readability:
87c628858c201d04009be99e5883d1e92c0100000050120119172a610421a6c3011dd330d9df07b63616c2cc1f1cd00200000000006657a9252aacd5c0b2940996ecff952228c3067cc38d4885efb5a4ac4247e9f337221b4d4c86041b0f2b5710
The first 3 bytes (87c628) seem to be the same for every block that I get from the database. I'm guessing it is some version identifier?
The last 80 bytes (or 160 characters in hex) are the block header: 010000005 ... f2b5710. They match what you get by executing bitcoin-cli getblock 000000000003ba27aa200b1cecaad478d2b00432346c3f1f3986da1afd33e506 false.
The rest is a mystery to me. I know that block 100,000 had 4 transactions, but the byte that holds the value 4 has 129 for block #1, which had only a single transaction.
Where can I find out about the format of, at least, the block keys in LevelDB?
Could it be that the format of each key depends on the version of bitcoind that serialized the contents? If so, how do I figure out which format to use?

LevelDB has its own database format and compression stuff that may make reading the data directly from disk as binary data to be difficult. You're probably running into some of that stuff.

You're right, that's why I am reading the LevelDB using a Python library (plyvel). That seems to be working correctly because I can retrieve records for blocks (bX*32 records) and files (fXXXX). I want to use the info stored in these records to seek through the .blk files directly.

Arjen can you please publish your code for retrieving the records using plyvel?

This is the Bitcoin Core v0.15.1 source code snippet that defines the serialization of CDiskBlockIndex. The various fields are:

A varint version number of the client who wrote the record (for future extensibility). This is the 87c628 you're seeing.
A varint block height.
A varint status with various flags.
A varint determining the number of transactions in the block.
If the status field has bits 8 or 16 set, a varint determining which file (blk*.dat and rev*.dat) the block is stored in.
If the status field has bit 8 set, a varint determining the offset into the blk*.dat file where the start of the block is.
if the status field has bit 16 set, a varint determining the offset into the rev*.dat file where the start of the block's undo data is.
The block header

I've now managed to decode the first five fields: version, height, status, num of txs and block number. I'm getting incorrect values for the data position, but I now know that I'm on the right track. Thank you for taking the time to answer.

I was a bit too fast: I had a bug in my VarInt parser. All is good now. Thanks again!

This topic had me searching quite a lot as this post is almost all there is as form of documentation (thank Pieter for that). The missing piece, at least from my perspective, is that those fields are encoded using base 128 / Varint. To make things more difficult, this is not the same Varint as in sizes representations in serialized Bitcoin block (as explained here).
So, in an attempt to bring some light on this topic, here are 2 pieces of code, one in python, and the other in golang, that will decode and read those fields using a base128 decoder function. The first one is adapted from the book "The Undocumented Internals of Bitcoin, Ethereum and blockchains".
The golang version is my own adaptation from this python code.
python version:
import leveldb
import sys

def reverse(hash):
    return "".join(reversed([hash[i:i+2] for i in range(0, len(hash), 2)]))

def base128(ans, off):
    n = 0
    while True:
        ch = ord(ans[off:off+1])
        print ch
        off = off + 1
        n = (n << 7) | (ch & 0x7f)
        if ch & 0x80 == 128:
            n = n + 1
        else:
            return(n, off)

hash = "62" + reverse(sys.argv[1])
hash = hash.decode('hex')
db = leveldb.LevelDB('./testindex')
ans = db.Get(hash)

v, o = base128(ans, 0)
print v, o
v, o = base128(ans, o)
print v, o
# etc ...

golang version:
package main

import (
    "encoding/hex"
    "fmt"
    "log"
    "os"

    "github.com/syndtr/goleveldb/leveldb"
)

func reverse(s []byte) []byte {
    for i, j := 0, len(s)-1; i < j; i, j = i+1, j-1 {
        s[i], s[j] = s[j], s[i]
    }
    return s
}

// binary.Uvarint is supposed to do this, except it doesn't.
// And it is not the same varint as in bitcoin serialized varints.
// this is ported from "The Undocumented Internals of The Bitcoin
// and Ethereum Blockchains" python code
func base128(b []byte, offset uint64) (uint64, uint64) {
    for n := 0; ; n++ {
        ch := int(b[offset])
        offset++
        n = (n << 7) | (ch & 0x7f)
        if ch&0x80 != 128 {
            return uint64(n), offset
        }
    }
    return 0, 0
}

func main() {
    db, err := leveldb.OpenFile("./testindex", nil)
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    h, err := hex.DecodeString(os.Args[1])
    if err != nil {
        log.Fatal(err)
    }

    b := append([]byte("b"), reverse(h)...)

    data, err := db.Get(b, nil)
    if err != nil {
        log.Fatal(err)
    }

    v, o := base128(data, 0)
    fmt.Printf("%d %d\n", v, o)
    v, o = base128(data, o)
    fmt.Printf("%d %d\n", v, o)
    // etc...
}

HTH

Ok, I have produced a standalone code snippet that can decode the block index records stored in the levelDB database. The challenge here, is that bitcoin core appears to use an extremely non-standard variable length integer representation. It does not match 'varint128' which is used by Google protobufs and other projects. It's very similar, but there is a real difference.
In this code snippet you pass the raw data stored in the levelDB record and it will populate a data structure with the results.
You can find the code reference here:
https://github.com/jratcliff63367/bitcoinstats/blob/main/include/CBlockIndex.h


text:
How specifically is transaction history modeled in blockchain blocks?
I understand that each block on the Bitcoin blockchain contains a list of transactions, but let's assume (for the sake of this question) that the block in question contains only a single transaction--and that this particular transaction has a history of previous transactions (i.e., it was not a coinbase transaction). How would I figure out which block holds this single transaction's previous transaction(s)? I know that each transaction contains the hash of the previous transaction--but how do you find that hash without looking into every block on the blockchain? In other words, how can I quickly discern a transaction hash chain for a given transaction ID? Any links to explanations or previous questions would be appreciated--I tried searching and haven't seen anything that answers my question to my satisfaction.
Thank you.

-but how do you find that hash without looking into every block on the blockchain?

You can't. Of course, if you are smart, when you receive a block, you extract the data that you need and put it into a database where you store this information and can quickly look it up. So, starting from the beginning of the blockchain, you read through every block and every transaction, compute their hashes, and store in a separate database the hash and a pointer to where you can find the block or transaction (or something of that sort). So when you need to look up the outputs that a transaction spends, you can just lookup the txid in your database and be able to quickly find the transaction output that you need.
This is similar to what Bitcoin Core does. After it fully validates a block, it adds the UTXOs that are created to a separate database. That database contains the details for all of the UTXOs, so it has each UTXOs outpoint (txid + output index), the output value, and the output script. When a transaction spends a UTXO, it can be quickly found using the outpoint specified in the transaction and then the UTXO is removed from the database. The outputs of that transaction are then added to the database as new UTXOs.

"Ah-hah!" is all I can say. :)

When nodes start up and sync the blockchain, they create and maintain something known as the utxo database. This contains all transaction outputs that have not been spent.
As new blocks are broadcasted, the inputs being spent in the transactions in those blocks are removed from the utxo, and the new outputs that were created are added in. This will usually happen even when a tx is broadcast, before it is mined into a block. Trying to broadcast an additional tx with the same inputs will fail with a mempool-conflict.
This database is saved to disk, so shutting down and starting up a node is not as intensive as the first sync and scan.
The raw blockchain itself is simply a series of series of serialized blocks and their transactions - The UTXO set is not a separate part of it, and must be constructed by iterating over it block by block.

The ledger i.e. Block chain is stored on every computer (node) that partakes in the network. ... Block chain is stored on all the computers running Bitcoin node. There are thousands of computers running full node. Each of these node record every new transaction in the blockchain.


text:
Confused About Hash In Satoshi Nakamoto Diagram
In Satoshi Nakamoto's diagram:

Consider the Hash block in the first transaction block above:
How is that Hash value calculated? I see arrows from an Owner's Public Key and another arrow, so it's confusing. Is the arrow coming from the Owner's Public Key saying that the Hash is encrypted with the owner's public key? If yes, then what is the other arrow? Is that involved during the encryption with the public key? In other words, does the previous transaction's hash get somehow combined with a new hash before being encrypted with the public key? What exactly is going on here?
Thank you.

Firstly, transactions are not blocks, and blocks are not transactions. They are two distinct data structures that you seem to think are the same thing.

How is that calculated?

By performing SHA256 double on the transaction.

I see an arrow coming from the left (I'll get to that in a moment)--it seems to depict the Hash of the previous transaction block. But the arrow coming from Owner 1's Public Key confuses me too. Is this diagram saying that owner 1's public key is hashed together with the previous block's hash value? Is that what actually happens in Bitcoin, or is there more detail?

Yes, the public key is part of the data that is hashed in a transaction.

'd like to see the original block in the depicted hash chain because that original block wouldn't have a previous block hash to reference. So would that mean that the first block's hash is just a hash of the owner's public key?

A transaction is more than just a public key and an output. But the first transaction in that transaction chain does not have any inputs so it does not have any previous transactions that it references. This transaction is known as a coinbase transaction or a coin generation transaction. It is created by miners and essentially produces Bitcoin out of nothingness.

Are there any good resources that go into more technical detail about the first block in the transaction hash chain and how these hashes are calculated?

Bitcoin.org's developer guide explains most of Bitcoin's details.

Is the arrow coming from the Owner's Public Key saying that the Hash is encrypted with the owner's public key?

No, it is saying that the public key is used to verify Owner 1's signature. This public key is specified in the previous transaction. In order to know what the previous transaction is, we place the hash of it in the current transaction (Owner 2's transaction). That way we can quickly lookup Owner 1's transaction to retrieve Owner 1's public key.
There is no encryption that happens in the Bitcoin protocol. Nothing is encrypted.

In other words, does the previous transaction's hash get somehow combined with a new hash before being encrypted with the public key?

Yes, the previous transaction's hash is combined with the other data in the current transaction to produce the current transaction's hash.

Instead of trying to understand the technical details and diagrams that Satoshi put in the whitepaper, you should try to understand how Bitcoin actually works today based on the code and other documentation available. The whitepaper does not describe how Bitcoin actually works; it was written months before the first release and lots of details changed between the concepts in the whitepaper and the actual implementation of those concepts.

Correct me if I am wrong but I am not talking about blocks on the blockchain--I know the difference between that block and a transaction block. I am talking about transaction blocks on the transaction hash chain the tracks a digital asset from genesis across all owners. Can't each link (data object) in this transaction-chain be called a "transaction block"?

I read the developer's guide and I think our disconnect is that Bitcoin doesn't actually create or discuss a hash chain of transactions. This is more of an abstraction of a data structure that could be created from the transaction hashes. So I don't think it's relevant to discuss the nomenclature of such "transaction blocks" here...

Question edited for clarity. I am interested in the first and subsequent Hash values along the ownership chain of transactions.

I think I really messed up in how I asked my question. Rather than post a new question, I decided to reduce this one to what I really am trying to ask and reduce all of the clutter in my original question. Please accept my apologies and new clarifications. I am really trying to understand this. The developers guide did not help me with the specific question I am asking.

No, it is not a transaction block. It is just a transaction. Calling them blocks or transaction blocks will just create more confusion. Just call it a transaction. Also, Bitcoin does create chains of transactions. A transaction input directly references the txid (aka hash) of a previous transaction.

I am not trying to understand Bitcoin from the diagram. I am trying to figure out what is generically done in such transaction chains. I don't even care whether it's Ripple, Ethereum, or Bitcoin. I'm trying to figure out what the intent was because I'd like to be able to teach the idea behind the transaction chain. I am just hung up on the Hash. I can't figure out if the transactions have two Hash values--one for the digital asset being tracked and the other for the transaction block, so that it can back-link to the previous block... If there is only one Hash, that helps to know...

Each transaction has its own hash. That hash is not part of the transaction itself because it is computed directly from the transaction. Each transaction (except for coinbase transactions) also contain the hash of a previous transaction. In the diagram, the hash that is shown is a different hash each time, it is the hash of the previous transaction.

OK--Thanks. Last clarification: I was not talking about the dashed arrow from the Owner's Public Key--I understand that it's used to verify. I was talking about the solid arrow that points to the Hash. What is that saying?

The solid arrow means that the public key AND the previous transaction are hashed together. Note that this is not how Bitcoin (or any coin that I am aware of) works and is completely unnecessary.

I think you just NAILED my confusion--thank you! So the public key just sits in the transaction--it doesn't do anything other than help validate the signature for the next transaction... Right? :)

Yes. The public key is in the transaction only for signature validation.

All that chat just to learn that one simple fact? Sheesh! :) :) :) You made my day. Thank you!

"There is no encryption that happens in the Bitcoin protocol. Nothing is encrypted." If this is a semantic issue then fine, but isn't "signing with a private key" a form of digital encryption? I want to learn what's correct--I see "encryption" used all the time when talking about PKC...

A digital signature is not the same thing as encryption. Encryption obfuscates a message so that no one except the intended recipients can read the message. With a digital signature, the message is not obfuscated. Rather the message is in plaintext and the signature serves as proof that the owner of the public key created the message and verifies the message's authenticity (i.e. it was not tampered).

Thanks Andrew, but I thought a digital signature encrypts a hash of the message--then the message and the encrypted hash are sent together to the recipient. I thought the recipient uses the public key to verify by decrypting the hash then recomputing the hash against the message to ensure a match. Now if we're saying the same thing, fine. Otherwise, I am curious where I have gone wrong in my understanding. I do understand that the message itself is not encrypted--but isn't the hash? And if it's only the hash, perhaps it's not good to talk about encryption at all bcse message is not encrypted.

It may be that way for some digital signature algorithms, but Bitcoin uses ECDSA which does not do that. The message hash is not recovered in any way, it must be recomputed from the message in order to perform signature verification.

Let us continue this discussion in chat.


text:
How is the trading engine from Binance developed and designed?
As a early adopter of BNB and fan of Binance's recent world domination, one thing that always got me curious was about how Binance deals with these massive inflows of requests in theirs trading engine/website.
Is the system of trading and cryptocurrency wallets fully developed and designed in-house or they have the assistance of some third-party provider? If they have some third-party technology, do you know which one?
I ask this because of the recent surge of a lot of companies providing white label services for digital asset exchanges.
Thanks in advance!
Obs: Binance is just an example, other exchanges with high volume and multiple coins listed also responds the doubt!


text:
Where can I find more resources that explain Bitcoin simply?
I have been learning crypto for a little bit now and have been helped a lot by pebwindkraft, Andrew Chow and Adam in particular.
I would like to ask about resources that help beginners like me to understand the crypto space, I have been following a guy on youtube and I believe that he deserves some credit for the github book he did and the youtube videos that explain bitcoin in a way that I can sort of understand.
If you have any other resources then please feel free to put them down below.
Edit:
The channel I was referring was : https://www.youtube.com/channel/UCHBWzcGSrtoLYcasLwGvsow

I would suggest you reading Mastering bitcoin book first and second editions. It's exactly what you are looking for.
https://github.com/bitcoinbook/bitcoinbook
if you like watching more than reading this youtube videos are amazing

Thanks Adam, You may not remember me, but you have helped a ton too. I am actually reading that book as well, I somewhat prefer visuals, so was hoping for a organised "series"

I have edited my answer, hope that helps.

Watched one or two and they are really good. Thanks for the link, I hope others can post more resources too

There is a TON of info out there, so I think one of the biggest issues for beginners is finding good information, since they may not yet understand the difference between good and bad info in this space.  I think it is very important for users to form at least a basic understanding of what cryptocurrencies are, and what sort of problems they solve, so that they can make informed decisions.
I often recommend people take a look at Jameson Lopp's bitcoin resource page, it is an awesome list for just about anyone involved in bitcoin to take a look at.
If you are looking for technical details, Andreas' book Mastering Bitcoin is one of the best resources out there. His youtube videos are also great, and much more accessible for a non-technical audience.
And of course this website is great for specific questions! I've learned a lot just browsing through old Qs/As, there is some great knowledge stored away here.

Thanks for the links, and you are 100% right about being able to differentiate between bad and good information. It's also hard to spot when the author is wrong because we don't know ourselves

The best place to go would be CryptoInfluence
They rank I think 90% of all YouTube channels over 1k subscribers in the CryptoSpace. Its done on their stats, and also a community poll. The top guys are usually most worth your time...but you can find some real gems in the middle or even near the bottom.
They also have a scam list that tells you which YouTube channels to avoid!
Also, I was recommended an "Into to Bitcoin" book written by a Dr. Julian Hosp. It even comes with a work booklet, so you can quiz yourself on the terms. Doctor Hosp also has a YouTube channel: https://www.youtube.com/user/julianhosp


text:
BIP32 Neutered function
Can the following be clarified, where I am wrong:
The Xpriv can derive a hardened xpriv.
It can also derive a non hardened xpriv, which we will use with the neutered function to make it an xpub.
The xpub derived, is the same xpub that we would get if we had done CKDpub on the original Xpub.
Xpirv and Xpub with the capital 'X' can be seen as parent keys.
So a diagram would look like:
Xpriv -> xprivH (hardened)
Xpriv -> xprivN -> xpubN (Normal: This is the path from the xpriv using Neuter)
Xpub -> xpubN (Normal: This is the same path from the Xpub using CKDpub)
My other question on top of this, is if a Wallet software has the mnemonic key, would it be better to always derive from the mnemonic?
Or would they keep the account level xpriv, it seems like a waste to have both the mnemonic and the account level xpriv on the device.
Also, Is the extended private key, the (private key + chain code), or is it the (magic + depth + fingerprint + index + chain + key) ?

It can also derive a non hardened xpriv, which we will use with the neutered function to make it an xpub.
The xpub derived, is the same xpub that we would get if we had done CKDpub on the original Xpub.

Yes. Neutering really means "Do CKDpub and delete the Xpriv". There is no neuter function on Xprivs; it's for the wallet software, not the key derivation.

My other question on top of this, is if a Wallet software has the mnemonic key, would it be better to always derive from the mnemonic?

Deriving from the mnemonic requires additional computation, however requires less storage space. I don't think it really matter.

Also, Is the extended private key, the (private key + chain code), or is it the (magic + depth + fingerprint + index + chain + key) ?

The second.

Thank you, I think I understand bip32 and 44 now. Thanks for all of the corrections Andrew.

It is not accurate to state that "there is no neuter function on Xprivs". The function is defined in the spec as taking an extended private key for input: N((k, c)) → (K, c)


text:
Is there a way to get all addresses from an Wallet?
If I was to search the blockchain for address X, then I see that it has two inputs in a transaction, X and Y.
I can safely assume that X and Y are owned by the same person?
Now if I look at all of the transactions for Y, and I see that it has a transaction with two outputs, Y and Z.
Then I can now assume that X, Y and Z are all owned by the same person, right?
If a wallet has 10 addresses, then could I not get all of those addresses using the above method?
So I essentially assume that, all inputs and outputs belong to the same person for a given address lookup.
Also, if X is both on the input and the output, then it is safe to assume that X being on the output was just used as a change address.

If I was to search the blockchain for address X, then I see that it has two inputs in a transaction, X and Y.
I can safely assume that X and Y are owned by the same person?

That's usually the case, but not always.  They could be owned by different people who have cooperated.  It is possible to do this in such a way that neither party reveals their private key to the other.

Now if I look at all of the transactions for Y, and I see that it has a transaction with two outputs, Y and Z.
Then I can now assume that X, Y and Z are all owned by the same person, right?

No, that's not a good assumption.  When a transaction has two outputs, there is no reason to think they both belong to the same person.  In fact, a more common case for two-output transactions is that one of the outputs is the "real" payment, and the other is "change" that goes back to another address belonging to the same person who sent the payment.  (See How does change work in a bitcoin transaction?)  However, there is no way to be sure which is which.

If a wallet has 10 addresses, then could I not get all of those addresses using the above method?

Nope.  If you continue in this way, you'll pull in a vast number of addresses belonging to different people who were paid by your initial target (and people who were paid by them, etc, etc).  You'll also miss any addresses from the wallet that have been used independently, if they've never appeared together in a transaction with any of the other ones you saw.

Thanks Nate for the clarification.
If we take out the possibility of a mixer and we can determine the change addresses. Can the assumptions not then be more feasible?

It would be plausible to guess that one of the output addresses (namely, the change address) belongs to the same person as the inputs, assuming you can tell which of the outputs it is (though I know of no reliable way to do that).  It's nonsensical to assume that all the outputs belong to the same person as the inputs.  That would mean that nobody could ever transfer money to anyone else.


text:
Is it wrong to spend Bitcoin from someone else's wallet if my Bitcoin client generates the same address?
I know this scenario is extremely unlikely but what-if by chance I get the same Bitcoin address as another user and the user has hundreds or thousands of Bitcoins in his wallet, is it morally/legally acceptable to spend the money from this wallet? Would you say it's considered stealing or do I now have equal ownership of the wallet just as the original owner?
My first impression was that it's the wrong thing to do. However, perhaps because Bitcoin inherently allows for this possibility (by design) one might argue that by using Bitcoin, one understand the risks associated with using Bitcoin and agrees to share the coins if such a thing did occur.
I'm interested in knowing the Bitcoin communities' take on this.

Can someone please teach everybody another word or set of words for this level of unlikelihood. Extremely unlikely is what I would use to describe my chance of winning the lottery once.

However, perhaps because Bitcoin inherently allows for this possibility (by design)

I would dispute that.  A fundamental tenet of cryptography is that "extremely unlikely" is effectively the same as "impossible".  Bitcoin has been carefully designed so that this should never happen by chance.  And people definitely don't use Bitcoin with the assumption that there is a meaningful chance of someone else taking their coins through this route.  There is certainly no implied consent.
In fact, I think that it's meaningless to discuss the ethics of what to do if you accidentally generate the same key as someone else.  To me, it's like asking whether it would be ethical to kill a time traveler.  Maybe the discussion helps you think about ethics, as a thought experiment, but it has no practical relevance, because it's about a situation that fundamentally can never happen.
In practice, if you ever do generate an address that already exists, what it means is that both of you are using terrible random number generators.  Maybe one or both of you is doing so on purpose, suspecting that the other has a weak RNG and trying to exploit it.  So if it's not you that's intentionally trying to crack something, then throw away the software (or hardware) you are using, because it's horribly broken.  And do the rest of the community a favor and responsibly disclose the vulnerability, so that people know not to use it, and so experts can investigate the problem.
In this case, you could of course take advantage of the other person and steal their coins.  But you'll know that it's not just random chance - you're actively exploiting a flaw in the software.  I think that would help most people realize that it's clearly unethical to do so.  (You could also transfer the coins to another address, so that they can't be stolen by someone else, and then try to get them back to the rightful owner.)
The other possible explanation for such an occurrence is that one or both of you has found a severe weakness in the cryptographic primitives used by Bitcoin.  This is similar to the previous case: responsibly disclose and don't steal anything.

Is it wrong to spend money from someone else's wallet that you found lying on the street?
Regular wallets also by design allow for the possibility that people other than the owner can spend the money. That design choice doesn't absolve pickpockets of ethical culpability. The fact we're talking about Bitcoin is irrelevant to the ethical balance here. Spending other people's money without their consent is wrong.

I think instead of thinking about Bitcoin where this is impossible, you can think of an imaginary cryptocurrency where this was possible due to a cryptographic vulnerability (perhaps they did something stupid like use a 8 bit key space). If such a cryptocurrency existed, and this vulnerability was public knowledge, the value of those coins would be effectively worthless, thus it would not be unethical to steal them.

Knives allow you to stab other people by design, still it's immoral to do that.  Same with randomly (or not) generating someone else's keys and taking funds already spendable by those keys.


text:
Would first-seen prevent a double spend attack?
It takes approx. 8 seconds for a transaction to propagate through the entire Bitcoin network. If we got rid of the concept of replace-by-fee and replaced it (no pun intended) with a first-seen approach wouldn't this significantly reduce the likelihood of a double spend?
Even if an attacker is running another wallet on the other end of the planet and broadcasts an identical transaction right near the winning mining node, such a double spend transaction would still reach the merchant within < 16 seconds.
Would a merchant be able to simply listen out for a double spend transaction and conclude with a reasonably high level of confidence that it is very unlikely for a double spend to occur in this case?
Note: This SE question does not address my particular concern as a merchant would be waiting 16 seconds to rule out a simultaneous transaction.

First of all, what is the goal? The Bitcoin consensus rules already completely eliminate the possibility of an "double spend" within any individual version of history ("chain"). The concern is about individual wallets accepting a transaction as a fullfilled payment, before it is certain that that transaction will end up in the winning chain. So when we speak about problematic "double spends", the term is confusing - no coin is actually spent twice - instead, it is the confusing name we give to "being convinced you were paid while in reality you aren't".
And there is an easy solution for this: waiting until the transaction is sufficiently confirmed before treating it as accepted. This approach is suggested in the Bitcoin whitepaper, together with an analysis that shows how risk of reorganization falls with increasing confirmations, assuming no attacker with a majority of the hashrate exists.
So the whole concern is about being able to accept transactions before they gather sufficient confirmations. And the answer is simply: if you want to be sure, you simply cannot. This is a fundamental problem: global consensus among untrusted participants is hard, and in Bitcoin's case, which relies on economic incentives to see that concensus emerge, you need proof-of-work to build up any guarantee, and that takes time - multiple blocks' worth of time.
If you're thinking of a rule where nodes reject blocks that contain transactions that they believe are fraudulent/doublespends, you're just introducing a recipe for the network to fork. We need a way to make all nodes agree on what transactions are valid, and if we had any other way of doing so without using slow and costly proof-of-work, we would.

It takes approx. 8 seconds for a transaction to propagate through the entire Bitcoin network. If we got rid of the concept of replace-by-fee and replaced it (no pun intended) with first-seen approach wouldn't this significantly reduce the likelihood of a double spend?

That's how things worked before the introduced of opt-in RBF (BIP 125), and still how it does work for transactions that don't signal RBF. However, its promises are widely understood to be an illusion: there were (and are) known ways to construct transactions that propagate slower than others through the network. All an attacker needs to do is create a slow-to-propagate transaction that pays you and broadcast it "close" to you in the network, while broadcasting a fast-to-propagate transaction that pays himself and publish it in many other places on the network at the same time. Most miners will see the second one first, and that transaction will have a higher chance of winning.
And if you look at incentives, you don't even need that. Miners have an incentive to accept the highest-fee transaction, regardless of when they see it. Even if the network doesn't relay it, nothing prevents an attacker from sending a higher-free transaction directly to a miner. They'd be stupid to reject it. Bitcoin software initially (and still, for non-BIP125 transactions) was written to reject such replacements, but as profit margins for miners shrink and the industry becomes more competitive, it seems very unlikely to me that miners will not eventually just accept whatever maximizes their income.

Even if an attacker is running another wallet on the other end of the planet and broadcasts an identical transaction right near the winning mining node, such a double spend transaction would still reach the merchant within < 16 seconds.

That's assuming that the network provides a reasonable guarantee at relaying conflicts. It doesn't. Nodes only relay the transactions they themselves accept. Relaying conflicting transactions so the rest of the network can learn about conflicts has many problems:

It doesn't necessarily work. If the attacker hands the conflicting transaction directly to a miner, they may not have any incentive to reveal it to the rest of the world before it is mined.
Done naively, it is a trivial DoS attack: an attacker simply creating a myriad of conflicting transactions would result in the network relaying all of them, at no cost to the attacker except the fee of the one variant that eventually gets mined.
Done less naively, with precautions in place to prevent said DoS attack (e.g. through rate limiting that kicks in at some point), it becomes trivial to bypass: first produce enough transactions to trigger the rate-limit, and only then perform the attack now the network has decided they've had enough of your conflicts.
It helps the attacker. If the conflicting transaction pays more, the network relaying it will make it easier for economically rational miners to pick it up.

In fact, there is barely any guarantee that even honest transactions propagate well in adverserial setting. It is even less the case for conflicts.

Would a merchant be able to simply listen out for a double spend transaction and conclude with a reasonable high confidence level that it is very likelihood for a double spend to happen in this case?

It is certainly possible to use heuristics to try to analyze the probability of conflicts, and for sufficiently low-value transactions, it may be fine to treat the occasional risk of a reversed transaction as a cost of doing business. But remember that past performance is not a promise of future success. Just because nobody attacks you doesn't mean it isn't trivial to do so; it may just be that it wasn't ever worthwhile to try.
TL;DR: if you want to be sure about a payment, wait for confirmations

@_Pieter Wuille thanks for your amazing & insightful answer. One question: "there were (and are) known ways to construct transactions that propagate slower than others through the network" can you provide examples of transactions that are slower/faster to propagate that an attacker might take advantage of?

Also regarding this point "If the attacker hands the conflicting transaction directly to a miner, they may not have any incentive to reveal it to the rest of the world before it is mine" presumably the likelihood of this type of attack succeeding is proportional to hashing power of that miner? The attacker is then taking a big chance that the first transaction will be included in the blockchain (unless he has direct access to a relatively large percentage of miners or the miners are concentrated in a sufficiently small area where he can broadcast his second transaction directly to them all)

Using long chains of unconfirmed transactions depending on each other you can slow down propagation. Yes, such an attack only has a probability of success proportional to the miner's hashrate; but even a 1% chance of having a (high value) transaction reverted could be devastating.

"Using long chains of unconfirmed transactions depending on each other you can slow down propagation" Can't a merchant detect when a prospective customer (attacker) submits this kind of transaction and reject it until its confirmed by the network? I mean, I think there's no way to hide the fact that the transaction consists of long chain of unconfirmed transactions.. and since it would be an anomaly to submit this kind of transaction most transaction would not suffer from this flaw..

Maybe, but now you're again talking about heurstics, not assurances. Chains of unconfirmed transactions happen naturally too, and there are probably many other ways of causing a slowdown (or just finding a miner willing to accept a replacement in return for a fee).

Maybe an important way to look at it: this is a question of cost, not probabilities. Situations and patterns of network activity don't occur randomly; they happen because people have incentives to make them happen. So the question isn't what is the likelihood of an attackable scenario happening by chance. The question is how much it costs for someone to make it happen. And I conjecture that any attempt at trying to get assurances about insufficiently confirmed transaction results in something that can be attacked, if the gain is valuable enough.

but I still think that YES first-seen would prevent merchant accepting 0-conf (NOT from double spend as Peter pointed out) from being tricked by attacker. As with first-seen all outputs of tx2 needs to be same as outputs of tx1


text:
Recover $220 Million Worth of Bitcoin Locked in a Self Destructing Hard Drive
A NY Times article recently ran a story of Stefan Thomas who has $220 million worth of Bitcoin locked up in an encrypted hard drive:

Stefan Thomas, a German-born programmer living in San Francisco, has
two guesses left to figure out a password that is worth, as of this
week, about $220 million.
The password will let him unlock a small hard drive, known as an
IronKey, which contains the private keys to a digital wallet that
holds 7,002 Bitcoin.

I am familiar with this discussion on SE which analyzes possible vectors to retrieve the actual PK using brute force. However, in this specific case we are trying to retrieve the password, not of the actual Bitcoin wallet but of the hard drive. Once the password to the hard drive is found, the PK would be known automatically. The main issue is that the hard drive will self destruct after 10 failed password attempts (8 attempts have been tried so far in Stefan Thomas's case).
What possible vectors are available to retrieve Bitcoin in this case? My first thought was to copy the contents of the hard drive to many different drives and try to brute force each copy separately. This way when one hard drive self destructs you can always continue brute forcing on the other. Obviously, this won't be cheap but it might be worthwhile depending on how much Bitcoin is at stake. However, there might be something specifically about IronKey that prevents copying?
Is copying hard drive an effective/feasible strategy? If not, what other possible vectors should one consider to tackle this problem?
Thanks

This isn't a hard drive; it's more something like a USB stick, but one that was specifically designed to be secure.

I'm not sure on the specifics of this particular model, but usually the way these work is the encryption key is stored on a hardware security module, which is a physical chip that performs the encryption and decryption of the data given the right password.
To my knowledge you cannot make a copy of this physical chip, and once your attempts at entering the password run out, the chip will permanently wipe its copy of the encryption key. This chip is the thing that is checking your password, so it's this chip you would have to somehow duplicate.
With a lot of effort, you could probably desolder the flash memory chips and attempt to make a copy of the encrypted data itself. However, this encrypted data is generally protected with a 256-bit AES key (with has nothing to do with your password), of which the only copy was on the encryption chip, which the chip wiped when your password attempts ran out. So, this would be pointless.


text:
DashCoin insight setup in ubuntu system
I have tried to installed dash insight ui & api in my local machine (ubuntu OS) and it is completed all the steps but I m facing issue while start services in frontground.
Let me share with you command, config file.

https://www.npmjs.com/package/bitcore-dash

But getting below error in my terminal:

    [2018-02-09T04:18:18.022Z] info: Using config: /home/dashcore/.bitcore/bitcore-node-dash.json
[2018-02-09T04:18:18.024Z] info: Using network: testnet
[2018-02-09T04:18:18.026Z] info: Starting bitcoind
[2018-02-09T04:18:18.028Z] info: Using bitcoin config file: /home/dashcore/.bitcore/data/dash.conf
[2018-02-09T04:18:18.030Z] info: Starting dash process
dashd: zmq/zmqpublishnotifier.cpp:101: virtual void CZMQAbstractPublishNotifier::Shutdown(): Assertion psocket' failed.
[2018-02-09T04:18:18.245Z] warn: Dash process unexpectedly exited with code: null
[2018-02-09T04:18:18.246Z] warn: Restarting dash child process in 5000ms
[2018-02-09T04:18:23.247Z] info: Using bitcoin config file: /home/dashcore/.bitcore/data/dash.conf
[2018-02-09T04:18:23.248Z] info: Starting dash process
dashd: zmq/zmqpublishnotifier.cpp:101: virtual void CZMQAbstractPublishNotifier::Shutdown(): Assertionpsocket' failed.
[2018-02-09T04:18:23.481Z] warn: Dash process unexpectedly exited with code: null
Warning: Failed to create backup, file already exists! This could happen if you restarted wallet in less than 60 seconds. You can continue if you are ok with this.

MY Config File  (bitcore-node-dash.json)

    {
  "network": "testnet",
  "port": 3006,
  "services": [
    "bitcoind",
    "web"
  ],
  "servicesConfig": {
    "bitcoind": {
      "spawn": {
        "datadir": "/home/dashcore/.bitcore/data",
        "exec": "/home/dashcore/.bitcore/data/dashd"
      }
    }
  }
}

Dash Config  (dash.conf)

server=1
whitelist=127.0.0.1
txindex=1
addressindex=1
timestampindex=1
spentindex=1
zmqpubrawtx=tcp://MyIPAddress:28332
zmqpubrawtxlock=tcp://MyIPAddress:28332
zmqpubhashblock=tcp://MyIPAddress:28332
rpcallowip=127.0.0.1
rpcuser=dash
rpcpassword=local321
uacomment=bitcore

Can you please help me with it? What is wrong in which step?

Try this configuration. It worked for me it might work for you too.
server=1
whitelist=0.0.0.0/0
txindex=1
addressindex=1
timestampindex=1
spentindex=1
zmqpubrawtx=tcp://127.0.0.1:28332
zmqpubrawtxlock=tcp://127.0.0.1:28332
zmqpubhashblock=tcp://127.0.0.1:28332
rpcuser=dash
rpcpassword=local321
rpcport=9998
rpcallowip=0.0.0.0/0
uacomment=bitcore
debug=0
testnet=1


text:
Zcoin insight setup in ubuntu system
I have tried to installed zcoin insight ui & api in my local machine (ubuntu OS) and it is completed all the steps but I m facing issue while start services in frontground.
Let me share with you command, config file.
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:bitcoin/bitcoin
sudo apt-get update
sudo apt-get install libdb4.8-dev libdb4.8++-dev
sudo apt-get install build-essential libtool autotools-dev automake pkg-config libssl-dev libevent-dev bsdmainutils
sudo apt-get install libboost-all-dev
sudo apt install libminiupnpc-dev
sudo apt-get install libzmq3-dev
sudo apt-get install libqrencode-dev
git clone https://github.com/zcoinofficial/zcoin.git
cd zcoin
./autogen.sh
./configure
make
cd src
./zcoind

But getting below error in my terminal:
   [2018-02-07T14:05:58.941Z] info: Starting bitcoin process
zcoind: zmq/zmqpublishnotifier.cpp:99: virtual void CZMQAbstractPublishNotifier::Shutdown(): Assertion `psocket' failed.
[2018-02-07T14:05:59.837Z] warn: Bitcoin process unexpectedly exited with code: null
[2018-02-07T14:05:59.837Z] warn: Restarting bitcoin child process in 5000ms
[2018-02-07T14:06:04.837Z] info: Using bitcoin config file: /home/zcoincore/bitcore-node-zcoin/zcoinnode/data/zcoin.conf
[2018-02-07T14:06:04.838Z] info: Starting bitcoin process
[2018-02-07T14:06:04.847Z] error: Failed to start services

MY Config File  (zcash-bitcore-node.json)
    {
  "network": "testnet",
  "port": 3004,
  "services": [
    "bitcoind",
    "insight-api-zcoin",
    "insight-ui-zcoin",
    "web"
  ],
  "servicesConfig": {
    "bitcoind": {
      "spawn": {
        "datadir": "./data",
        "exec": "/home/zcoincore/zcoin/src/zcoind"
      }
    }
  }
}

Zcash Config  (zcash.conf)
server=1
whitelist=127.0.0.1
txindex=1
addressindex=1
timestampindex=1
spentindex=1
zmqpubrawtx=tcp://127.0.0.1:28332
zmqpubhashblock=tcp://127.0.0.1:28332
rpcallowip=127.0.0.1
rpcuser=YANNISCOOLUSER
rpcpassword=YANNPASSISCOOL
uacomment=bitcore

Can you please help me with it? What is wrong in which step?

Your problem is with zmqpubrawtx
zmqpubrawtx=tcp://127.0.0.1:28332
zmqpubhashblock=tcp://127.0.0.1:28332

Remove them and see if it will work.

Hello @Adam,
Thanks for your feedback but it is getting below error after remove this 2 line from zcash.conf.
error: Failed to start services
error: Invalid state: "zmqpubrawtx" option is required to get event updates from bitcoind. Please add "zmqpubrawtx=tcp://127.0.0.1:" to your configuration and restart

Hello @Adam,
Working fine after change 127.0.0.1 to MyserverIpAddress.

Hello @Adam,
In dash currency, I did same thing but it is returning error.
Warning: Failed to create backup, file already exists! This could happen if you restarted wallet in less than 60 seconds. You can continue if you are ok with this.
dashd: zmq/zmqpublishnotifier.cpp:101: virtual void CZMQAbstractPublishNotifier::Shutdown(): Assertion `psocket' failed.

please add another question with more details

@Adam: Added another question: please https://bitcoin.stackexchange.com/questions/70483/dashcoin-insight-setup-in-ubuntu-system help me about it.

I stuck in blocks api & send api. I have setup in my server http://159.89.175.64:3004/insight/blocks but not able to get all block. This  url, I am not getting Amount: http://159.89.175.64:3004/insight/block/4b62193f2c4d93a945dce08e185e1842222230e4e233077070cde47b74683e5b
API, I installed it from here: https://github.com/Technoprenerd/insight-api-zcoin But still not working

I solved this issue,
Actually it was issue related confliction because I have setup more than one currency in single server Like (Bitcoin, litecoin & zcash)
For that, I have taken care rpcport & change port for "zmq" in config file and then it is working fine.
Config:
server=1
whitelist=127.0.0.1
txindex=1
addressindex=1
timestampindex=1
spentindex=1
zmqpubrawtx=tcp://127.0.0.1:28334
zmqpubhashblock=tcp://127.0.0.1:28334
rpcport=8334
rpcallowip=127.0.0.1
rpcuser=YANNISCOOLUSER
rpcpassword=YANNPASSISCOOL
uacomment=bitcore
listen=0


text:
Wondering about the viability of a Hardware Wallet
I want to roll my own hardware wallet using a USB stick. I have some concerns:

Do I need to download the entire blockchain in order to create a new wallet?
Do I need to download and store the entire blockchain on the USB in order to send and receive funds?
Do I need to plug in the USB wallet in order to send funds?
Do I need to plug in the USB wallet in order to receive funds?

.

I'm hoping that the answer to the first question is "no", as I really am not keen on downloading and maintaining a local copy of the entire blockchain. It's such a waste of space.
I'm hoping that the answer to the second question is "no". Because again, I really don't want to spend time and bandwidth downloading the entire blockchain, and also I don't want to have to store it. Those are gigabytes that could be spent on something else.
I'm hoping that the answer to question three is "no" for security reasons.  I don't want random cυnts on the other side of the world transfering my money without my consent.
I'm hoping that the answer to question four is "yes" for convenience reasons. I don't want to have to plug in the wallet every time someone sends me money.

My understanding is that there is the public key for receiving, and the private key for sending. Am I correct in thinking that the only thing you fundamentally have to keep secure is the private key? If that's the case, surely I don't have to download the entire blockchain?

I don't know if I'd call it a hardware wallet but yes you can store keys on a USB stick that also contains software that allows you to build and sign transactions. Technically all you need is the input details (txid, nout, scriptpubkey, and private key (and full input value if segwit)) to build a transaction. This would be for advanced users as building raw transactions can lead to catastrophic loss of funds if you don't know what you are doing.

Am I correct in thinking that the only thing you fundamentally have to keep secure is the private key?

Correct.

If that's the case, surely I don't have to download the entire blockchain?

Also correct. Downloading the blockchain and running a node lets you verify the chain and can help propagate other nodes but it's not required to do what you are describing.

I want to roll my own hardware wallet using a USB stick.

An ordinary USB stick cannot be made into a hardware wallet.
Hardware wallet incorporates cryptographic hardware designed for cryptocurrencies and has firmware to interact with software on another device (PC, phone, etc)

No and Yes.
Normal wallet software has some utility while it is synchronising, but until it has synchronised it can't show the amount you control and therefore may not be able to create transactions using those amounts.
An  SPV wallet doesn't need to download the blockchain but is less secure.
A full-node wallet cannot create transactions involving unspent transaction amounts it does not know about.
A wallet must be online to send transactions.
A wallet need not be online for money to be transferred to your control but the wallet won't know about it until it is made online or is able to communicate with other software that is online.


text:
My Bither Hot Wallet
Purchased a Bither bitcoin machine and can't figure out how to work it.  I keep getting bitcoins but don't know how to send them to my Coinbase wallet and therefore exchange them for cash.  Help

It is so easy to use Bither app.
Generating Coinbase bitcoin receiving Address
first of all you need to check your Coinbase bitcoin receiving wallet address which is shown by a QR-code (you can find out how, from here).
Sending fund
Then you should try scanning that QR-code by your Bither app. for doing it, you need to choose one of your wallet (from the middle menu which is indicated by a big B and showing total amount of your funds) then you should choose the preferred wallet. then use the send icon and scan QR code and type the amount go ahead and send funds.
caution
by the way, you should be aware of your transaction fees. Bither does not propose any offer automatically; so you should choose your own policy for transaction fee by monitoring mempool to get informed use this website to see which tx takes how long to get confirmed.

Bither is simple to use, you can either copy your coinbase address and paste in your Bither wallet or use the scan QR code function. Then imput the amount and send. 3 more things you shall know:

Bither also support segwit address.
Bither can use to create a cold wallet when you set up in a phone without any sim card or internet connection.
You can use Bither hot to monitor Bither cold, you can find the function by click the 'gear' button above.


text:
How do I set up a Lightning Node on Mainnet using windows?
I want to set up a lightning node on windows. I have found lightning-app which seems to work only on testnet. help me finding a good guide to set up a lightning node on windows.
the reason I am insisting in the frustrating windows platform is to make it familiar with other people showing it really works on mainnet and setting it up for non-programmers. but I have found some guide on medium which shows how to set it up on testnet.

Try Eclair.
It has a nice GUI, runs on Windows, and just a few days ago they have announced a new version that officially supports mainnet usage.
It requires a Bitcoin Core node to runs alongside it, and a little configuration.