text:
How to Construct a Ping Message
I am struggling greatly trying to figure out how to send a basic "PING" message, and receive back a "PONG" response.
import random
import hashlib
import socket
PEER_IP = "35.187.200.6"
PEER_PORT = 8333

nonce = (random.getrandbits(64)).to_bytes(8, byteorder="little", signed=False)

magic_number = bytes.fromhex("f9beb4d9")
command_name = bytes.fromhex("ping".encode("utf-8").hex() + ("00")*8)
payload_size = (int(len(nonce))).to_bytes(4, byteorder="little", signed=False)
checksum = hashlib.sha256(hashlib.sha256(nonce).digest()).digest()[:4]

final_msg = (
    magic_number
    + command_name
    + payload_size
    + checksum
    + nonce
)

sock = socket.socket()
sock.connect((PEER_IP, PEER_PORT))
sock.send(final_msg)
pong = sock.recv(1024)
print(pong)

I am following this YouTube tutorial, and added the socket layer myself.
The only response I get is
b''

Any insight into what I am doing wrong here, would be greatly appreciated.
Thanks

I believe you need to negotiate a version message before any other communication can occur. Also, receiving a zero length array on recv means the socket has been disconnected.

You need to send a version message (and receiving the correspondant verack) before sending anything else. Here is a quickly patched version of your script which handles the version message:
import hashlib
import random
import socket
import time
import urllib.request

PEER_IP = "127.0.0.1"
PEER_PORT = 8333

nonce = (random.getrandbits(64)).to_bytes(8, byteorder="little", signed=False)

magic_number = bytes.fromhex("f9beb4d9")
command_name = bytes.fromhex("ping".encode("utf-8").hex() + ("00")*8)
payload_size = (int(len(nonce))).to_bytes(4, byteorder="little", signed=False)
checksum = hashlib.sha256(hashlib.sha256(nonce).digest()).digest()[:4]

pong_msg = (
    magic_number
    + command_name
    + payload_size
    + checksum
    + nonce
)

# Magic bytes, as usual
magic = bytes.fromhex('f9beb4d9')
# Version number (PROTOCOL_VERSION in core)
version = int(70015).to_bytes(4, 'little')
# Random services, we don't care
services = int(1).to_bytes(8, 'little')
timestamp = int(time.time()).to_bytes(8, 'little')
myip_str = urllib.request.urlopen('https://api.ipify.org').read().decode()
myip = socket.inet_aton(myip_str)
nodeip = socket.inet_aton(PEER_IP)
# https://en.bitcoin.it/wiki/Protocol_documentation#Network_address
addr_recv = services + b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff'
addr_recv += nodeip + int(8333).to_bytes(2, 'big')
addr_from = services + b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff'
addr_from += myip + int(8333).to_bytes(2, 'big')
nonce = 0x00.to_bytes(8, 'little')
# You can replace this with something more fun
user_agent = 0x00.to_bytes(1, 'big')
# Let's say zero, once again we don't care for this example
start_height = 0x00.to_bytes(4, 'little')
# The message content, which we'll compute the checksum from
payload = version + services + timestamp + addr_recv + addr_from + nonce
payload += user_agent + start_height
checksum = hashlib.sha256(hashlib.sha256(payload).digest()).digest()[:4]
payload_length = len(payload)
# The command type, as NULL padded ascii
version_message = magic + 'version'.encode('ascii') + b'\x00\x00\x00\x00\x00'
version_message += payload_length.to_bytes(4, 'little') + checksum + payload

sock = socket.socket()
sock.connect((PEER_IP, PEER_PORT))
print('Sending version message..',)
sock.send(version_message)
print('Received verack: {}'.format(sock.recv(24)), end='\n\n')
print('Sending ping..')
sock.send(pong_msg)
print('Received pong: {}'.format(sock.recv(1024)))

And here is a sample output of my node:
Sending version message..
Received verack: b'\xf9\xbe\xb4\xd9version\x00\x00\x00\x00\x00h\x00\x00\x00n\x8a\xbe\xc7'

Sending ping..
Received pong: b'\x7f\x11\x01\x00\r\x04\x00\x00\x00\x00\x00\x00\x86\xc5\x08^\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\r\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xe7\xa0vQ\x16V\xb1\xab\x12/Satoshi:0.19.0.1/\x14P\t\x00\x01\xf9\xbe\xb4\xd9verack\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00]\xf6\xe0\xe2'

Here is the protocol documentation, on which you can rely for implementing protocol messages.

Great, thanks so much for your help. Much appreciated.


text:
How to get BTC transfer amount from raw tx using bitcoinjs
Suppose I have following object
console.log(bitcoinjs.Transaction.fromHex(rawTx));

The output
Transaction {
  version: 2,
  locktime: 1655960,
  ins: [
    {
      hash: <Buffer dc 4d aa 10 db 80 36 52 24 49 7c c5 7c bc c5 bc df be e1 2d 58 61 70 e1 0c 7b 76 a7 35 cc 7c 3a>,
      index: 21,
      script: <Buffer 16 00 14 5f f3 9b e6 42 af a9 f3 59 25 34 e7 6a dd b2 ee e3 41 82 15>,
      sequence: 4294967294,
      witness: [Array]
    }
  ],
  outs: [
    {
      value: 1000000,
      script: <Buffer a9 14 14 ea 69 2e 87 76 ec c6 2e 6a aa a1 7d cc d2 8f f3 30 b7 46 87>
    },
    {
      value: 3299775,
      script: <Buffer a9 14 73 e2 99 78 43 31 1e cc 32 b5 ac 38 ef c9 9b b3 ae 54 bc 7c 87>
    }
  ]
}

How can I decode transfer amount (in BTC) from this object?

What do you mean by "transfer amount"? You have the output values there...

You have the output value, which is the funds that get sent somewhere. That's probably enough for most applications. But you don't know how the value of any of the inputs, and therefore you also can't tell how much was paid in fees. If you can look up the inputs of this transaction, you'll have all the information you want!

The transfer amount (BTC) is, (somewhat dependant on the library used),  most commonly denominated as "value:"
Read: the outs => value: *
 {
      value: 1000000,
      script: <Buffer a9 14 14 ea 69 2e 87 76 ec c6 2e 6a aa a1 7d cc d2 8f f3 30 b7 46 87>
    },
    {
      value: 3299775,
      script: <Buffer a9 14 73 e2 99 78 43 31 1e cc 32 b5 ac 38 ef c9 9b b3 ae 54 bc 7c 87>
    }

Where the value is most often denominated in satoshis (a single unit.). But, some block explorers (blockchair) or other libraries will denominate in single units of Bitcoin, as is done below. (10^8 satoshis - 1 BTC ( 100 mil))
1000000 = 0.01BTC
3299775 = 0.03299775 BTC

I'm assuming you mean you want to know how many sats were sent from previous outputs (and how much was paid in fees).
This is not encoded in the tx itself. You need to get that information from your Bitcoin node (or from an explorer API like Blockstream's). You need to retrieve all previous input transactions and then look at the values in the outputs that your tx consumed. The tx contains the ids of the previous tx. You just have to go get that information.
This will do the trick:
import { Transaction } from 'bitcoinjs-lib';
import fetch from 'cross-fetch';

//I took a random recent tx from the blockchain:
const rawTx =
  '01000000000102f1950b9b14422ccb4214c0411a722d436f6b78aa35135ab87da42ec20a9ebf2900000000171600148e2d687d3dc2d5f2505809f212d90b3725bb78c0fdffffffc5e2cf7beaf2a14e57d530a88224e48673e6b8aa9f70d1c785ffc4618a0f89f001000000171600147606ee140cae2a5a47ad580e37a8e2557c05d12efdffffff02844402000000000017a914ec4c86acc0023962981f9d9585cab245e12b0859872eca0f0000000000160014330692d03439fe8ddb88df7b97fa834b975ea41002483045022100b6eced5373a7c37e9be7e4cbe070b880406fae7eba94519e8ddb35510889df4d02205cb95d1b9333f8713ea416f1a2f5d7d685418cad66d4b5c6a5f6b28ddecc955b012102d105008f859c9e9388c56fbe3432c077c731549c6f2807e44136b7de44abffe30248304502210085726ec64c7a614de6997705e86efdb78463d99a6be81128db2151e1869c6bbf022067e42c75755ea5ea922e29fae4a5a0b963de1f17faf8e5b9e4f767115a62e726012102555dabf7c0ecf054f1a0ae1a80980851d9b3b57060c59048c887709c62c7542900000000';

const outputValue = Transaction.fromHex(rawTx)
  .outs.map(out => out.value)
  .reduce((sum, value) => sum + value, 0);

let inputValue = 0;
for (const input of Transaction.fromHex(rawTx).ins) {
  const txid = Buffer.from(input.hash).reverse().toString('hex');
  const response = await fetch(`https://blockstream.info/api/tx/${txid}/hex`);
  const rawTx = await response.text();
  inputValue += Transaction.fromHex(rawTx).outs[input.index].value;
}

console.log({ inputValue, outputValue, fees: inputValue - outputValue });

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Using CLTV can I freeze some funds from a wallet, and make them available, in a single transaction?
For my scenarios, I want freeze some funds from my wallet for some given time/block and then make that amount spendable in a single transaction.
As I understand so far, I can do it using 2 transactions with CheckLockTimeVerify (CLTV) op_code as follows:

Create a transaction with an output that has the CLTV op_code to freeze some funds from the input(s).
Then create another transaction which uses the above output as the input. The output of this transaction will transfer the funds back to my wallet. The nLockTime of this transaction will be set to the expire time of the previous transaction.

But is there a way to do the same as above in a single transaction? As in create a transaction whose output has CLTV op_code but is a pubkeyhash back to my wallet. This way the frozen coins become spendable without having to create another transaction?

When you lock coins in CLTV you still have to provide an address in the output. In that field, you can just include the address that your wallet controls. When you spend those coins, all you need to do is include the nlocktime which is higher than that specified when locking using the CLTV opcode. You need not do a second stage transaction to send the bitcoins to an address controlled by your wallet.

Thanks @ugam-kamat. Using bitcore-lib, I'm unable to attach an address to the CLTV output but I'm going to open a separate question for that issue.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to add an address to a CheckLockTimeValue (CLTV) transaction output using bitcore-lib (or any other js lib)
I am using bitcore-lib to create transactions. I can add an output with CLTV to the transaction but am unable to add an output address to it. This is my code (typescript/node):
const lockScript = Script.empty()
                         .add(blockAsBuffer)
                         .add(177) // OP_CLTV
                         .add(117) // OP_DROP
                         .add(Script.buildPublicKeyHashOut(this.privateKeyAddress));

const transaction = new Transaction();
transaction.from(unspentOutputs);

// Following is where I'm adding the CLTV but can't add an output address
transaction.addOutput(new Transaction.Output({
  script: lockScript,
  satoshis: satoshisToLock
}));

transaction.change(this.privateKeyAddress);
transaction.fee(feeInSatoshis);
transaction.sign(this.privateKey);

With the above code, the transaction-output is something like the following. Note that there's no address attached to the CLTV output. Is there a way to add an address to that output using the bitcore-lib (or any other javascript lib)?
...
{
  "value": 0.00001,
  "n": 2,
  "scriptPubKey": {
    "asm": "1658883 OP_CHECKLOCKTIMEVERIFY OP_DROP OP_DUP OP_HASH160 89605b86b7ad185ebc10fe457b98776838972cf0 OP_EQUALVERIFY OP_CHECKSIG",
    "hex": "0403501900b17576a91489605b86b7ad185ebc10fe457b98776838972cf088ac",
    "type": "nonstandard"
   }
},
...

In case you are curious as to why do I want to do this, see the previous question that I had asked: Using CLTV can I freeze some funds from a wallet, and make them available, in a single transaction?

What do you mean by "address"? You have a public key hash in the script. The holder of the corresponding public key can redeem the funds. Maybe what you want to do is wrap the script in a P2SH address? We actually have a guide for exactly this in bcoin (another JS lib): https://bcoin.io/guides/cltv.html

You're putting a script directly in the scriptPubKey. That's both discouraged, non-standard, and hard to use, as you've noticed there is no corresponding address.
There are only three types of addresses defined (with common acceptance) in Bitcoin: base58-encoded P2PKH addresses (1...), base58-encoded P2SH addresses (3...), and bech32-encoded native segwit addresses (bc1...). Unless the output you want to construct has the structure corresponding to one of these, there simply does not exist any address for it.
This is with good reason: when you give an address to someone else to pay you, they have no business knowing how you want to protect your coins. As the policy you want to use gets more complex, sending to more and more complicated scripts would require them also to pay increasing fee. This is weird: it's you who cares about that policy, so you should pay for it instead.
With P2SH (and later P2WSH), the script is no longer put in the output itself. Instead, a hash of the script is put in the output, and the recipient, when spending the output reveals what the actual script was and satisfies it.
So you probably want to either do this manually by reading how to construct a P2SH/P2WSH output (see BIP16 and BIP141 respectively), or figure out how the library you're using lets you construct such outputs automatically.

Thanks for the details in the response Peter, it helped me understand the different types of addresses. I used the BIP65 demo to implement the freeze and spend transactions.


text:
How transaction fees differ between different wallets?
In particular, I'm wondering what is the difference (if any) of transaction fees between Bitcoine-core, MultiBit and blockchain.info?

Bitcoin-Core is a full node and can therefore build up a mempool and estimate fees. This is probably the most secure way how one can estimate fees. You can verify the source code and make your own improvements. The fee estimator allows you to estimate the fees for different confirmations targets (2-25 blocks).
MultiBit does either use a web-api and/or static fallback fees. You depend on third-parties, estimating fees for you. All SPV wallets cannot reasonable estimate fees.
Blockchain.info run its own full-nodes and use it's own fee estimation logic. You need to trust the estimations produced by blockchain.infos logic.

Right but how does it translate to numbers in a real world scenario, say I transfer 100 mBTC with each of the above?

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Is there a way to know at any point in time T how many BTC exist that were bought at time t<T?
What I would like to find out is whether for a specific time in the past, we can know the number of BTC that exist today and were bought at this time. The discrete time can be the time of block generation, i.e. roughly every ten minutes, but also for instance daily time.
What I understand is that on the Blockchain, we do not really keep track of the amount of BTC held by a specific address / account, but rather of the full history of transactions of this account. The sum of the unspent inputs of a specific address, which are basically routed output payments of other addresses, is the number of BTC that this address "holds", or is able to spend.
NEW
This part wasnt displayed in the beginning, due to some formatting error.
Now comes the actual, more refined question.
In order to construct the number of BTC that were bought at t[smaller than]T, where T is now, is there a way other than looking at the complete history of transactions on the BTC blockchain, in order to find this number? Sort of a shortcut?
NEW END
Thanks, best, JC

I think you're asking if there is a way to figure out the difference between the current UTXO set and a UTXO set from some point in the past, but it would be helpful if you finished/clarified your question and defined t and T.

By "bought at this time" do you mean purchased from an exchange or directly from a private individual. Does "bought" exclude Bitcoin obtained by selling goods or services for BTC? Or do you mean all transaction outputs other than mining rewards and transaction fees?

Your understanding of how the network keeps track of coins is slightly incorrect: there is no concept of addresses with balances, only UTXOs. Addresses with balances is just a nice human-readable abstraction.


text:
How can I know which user made which BTC transaction?
I am making an app, and I want my users to deposit BTC coin to my address which is an exchange address (I don't have the private keys).
How can I know which user made which transaction?
I have a way, which is to add the numeric user ID to the amount , for example if an user wants to deposit 100.00 BTC coin , and the ID is 158769, I can add the ID in the amount like this: 100.0000158769 and make it as a QR-code so the user should deposit the exact amount so I can know that he made this transaction.
But is there any other way to do it , any idea?
Thanks in advance

If your user changes the deposit amount post QR-code generation (which is very possible for them to do), then you will have no way to relate the actual payment to the actual customer. This will create a nightmare or accounting work that will be almost impossible to complete accurately.

Thats why I am asking for another solution..! In my solution, the user can change the amount, but his deposit will not be counted then, so users will be advised and explained why to deposit the exact amount. But as I said, my question is to get another solution

You give each customer a different address. That is the only reasonable approach.

I can generate addresses for each customer, but then I need to have all the amount in one address, cause all the amount will be used. There can be a solution if I can create multiple addresses inside one wallet, so I can make the transfer from multiple addresses in a single transfer, if you know an API to do this, it would be the solution..

@GesaVoIP That is not how Bitcoin works. You do not need all coins in one address. Doing so hurts your accounting, your privacy, and the privacy of everyone you interact with. Don't reuse addresses.

I will use the BTC amount man, I will be trading with the amount. Its a trading bot app

There are so many red flags here. You shouldn't be using a third party to hold your users' funds (use a self-managed wallet instead), you shouldn't be trading with your users' funds (keep your funds separate from your users'), and don't reuse addresses. This is all I will say; the way to figure out what account was credited is by not reusing addresses. If the approach you're using makes that hard/impossible, the approach is broken.

Its a trading bot app, users will agree to it, if you can give a solution to my problem it would be great !

I think the solution would be instead manage your own wallet that users pay into (so that you can give each a different address), and then move the coins into/out of the exchange as needed, with proper accounting.

That said, I don't understand why you can't request a separate address for each user from the exchange, and still do your own accounting.

Because if I have at minimum 100 users that deposit each one 0.01 BTC, I should then go to 100 accounts and transfer each of those 0.01 BTC one by one to my trading account.. This is the reason

You would have 1 wallet that every user pays into, but each through a different address. If 100 users paid 0.01 BTC each, that wallet will have 1 BTC, and with a single transaction you can move that to an exchange.

That would be the solution but which wallet API do you recommend to do that? I have tried the blockchain.com API can create addresses for each user but not in the same wallet. If you know another wallet API that can work, for BTC and USDT it would be great !

I expect that literally every wallet can do this. Don't use a hosted solution.

@PieterWuille My reply might be a little late but you wrote that "you would have 1 wallet that every user pays into, but each through a different address. If 100 users paid 0.01 BTC each, that wallet will have 1 BTC, and with a single transaction you can move that to an exchange". It is not really a "single transaction" as the transaction consists of 100 UTXO (100 inputs) (so 100 * fee). The only benefit of using HD wallet is that it simplifies generation of addresses via 1 seed/PK. Am I correct?

Your proposed approach is not practical:

Users might change the amount, or omit the fractional part
The fractional part might represent significant value - a user with an ID of 1234567 will have to pay a minimum of 0.01234567, well over a hundred USD at the moment
If users are withdrawing from an exchange or some other wallet without much control over the transactions, they might not be able to set the value beyond the first four decimals. Additionally, some custodial wallets batch payments, combining multiple outputs to one address into a single output with a summed value.

Pieter is right, the only reasonable way to approach this is to assign unique addresses for each user, and forwarding funds from there to your trading accounts. It is important to control the keys at the point money flows into your system. It is not unheard of exchanges to update their deposit addresses, either as part of normal key rollover or due to a hack. It is a lot easier for you to change a single forwarding address in your system, than to convince all of your users to update it.
All of your user addresses can be in a single wallet - managing the funds and forwarding them to a trading service will be no different from any other transaction you make. Any transaction fees incurred are simply the cost of doing business, and you can decide if you wish to pass those on to users or absorb them.

How to create all users addresses in single wallet, so then I can make a single transfer? With which wallet?

Every wallet supports an HD chain these days, which lets you derive the addresses from a single seed. You should be able to find a library in your language of choice for this (for example, BitcoinJS-lib for javascript) - Based on your questions, you might be a little out of your depth here, and should consider hiring someone to, at the very least, help you architect the part of the system that handles the crypto - making a mistake here can be very costly, and very permanent.

I need also wallets with HD for USDT in Omni, Tron and Ethereum chains..

Those get considerably more complex, as for USDT on all three chains you need additional BTC, tron, or ETH to pay transaction fees from each address - you will likely not find an out of the box, available solution for these. You will have to build it yourself, or hire someone to do it for you.


text:
LN Payment Channels: Can you have more than two nodes in a channel?
I can't seem to find an answer to this but apologies if this exists somewhere already.
From the last paragraph of section 2.0 in the lightning network paper version 0.5.9.2, "Micropayment channels create a relationship between two parties".
However the staked coins for the channel are held in a multisig address, and those addresses can have more than 2 signatures, max 20 signatures I believe, so can a lightning channel be set up that would be able to resemble more of a traffic circle than a 2 lane highway?

Theoretically you are correct. However maintaining and operating the channel would become much more complex as everybody has to exchange signatures with everyone. Also the penalty case in case someone breaches the protocol is not clear. Unless we switch to eltoo channels as described in https://www.blockstream.com/eltoo.pdf I doubt that the lightning network will see multiparty channels.


text:
Given a bitcoin block, what's the easiest way to determine the minimum net fee rate (including ancestors) for transactions in that block?
Given a bitcoin block, what's the easiest way to determine the minimum net fee rate (including ancestors) for transactions in that block?
A naive approach would be to just find the minimum fee rate for a single transaction, but that does not take ancestor and descendant chains into account. I want to know the minimum clearing fee rate for the block considering the full ancestor/descendant chains
I've been looking through the transaction selection algorithm to try and understand the logic, and figured I could try and reverse engineer it to decode an existing block... but I'm wondering if someone has already done the work and created a good tool/script/api that can parse an existing block and determine the minimum net fee rate of the block's transactions

Have you read [help/on-topic] and are you just looking for a software recommendation or did you want to know the method by which you can calculate the fee paid and fee-rate paid for transactions in a block?

I'd like to know a good way to calculate the minimum net fee rate of all the ancestor chains in a block, if one exists

First, even if you find the transaction with the lowest effective fee rate in a block, that will not necessarily mean that the mempool cleared to that level. In their recent paper "On Blockchain Commit Times: An analysis of how miners choose Bitcoin transactions", Messias et al observe deviations in the block composition from the expected norm (GetBlockTemplate) of up to 22%. They surmise that this could be caused by out-of-band prioritization or use of other algorithms.
Fact is, we don't know what software miners are running to compose blocks, how often they update their templates, how often they get acceleration requests or have their own interests to include transactions, and which version of Bitcoin Core they may be running if they even they simply use GetBlockTemplate (GBT).
Assuming that blocks are generally produced by Bitcoin Core's GetBlockTemplate method, and applying general rules for block composition, we know the following:

Transactions must be ordered topologically in a block
If a transaction spends UTXOs created within the
same block, the child transaction must follow the parent in the block's transaction list
GBT selects in order of effective fee rate
GBT will select complete transaction ancestor groups
This means either a single transaction if no child transaction is present, or the descendants have a lower effective feerate, or the complete group derived from the descendant that has the highest effective feerate.
Groups may be skipped when they don't fit in the block
If a transaction group requires more blockspace than is left, a smaller transaction group with a lower effective feerate will get included instead. This means that larger transaction groups may miss a block even though transactions with lower feerates made it in.
Only the best sibling counts
Sibling transactions do not form a shared ancestor group unless the
ancestor group is started from a shared descendant. I.e. if two siblings both have higher feerates than the parent, they each form a separate ancestor group in the candidate list and get prioritized on basis of the effective feerate of their individual group. When the ancestor group with the higher effective feerate gets selected, all remaining ancestor groups get recalculated and the sibling transactions would now likely have the highest effective feerate right afterwards (since it was higher than the parent alone). So, even if siblings and parents all appeared grouped in a block, they would be positioned at the highest effective feerate among the individual ancestor groups, the parent does not gain a special benefit from being CPFPed multiple times in parallel, and they were actually selected iteratively not together.

Altogether, it would be expected that transactions should appear in descending order of their feerate in the block (when using GBT), except where topology interferes, or transactions have been imbued with a "virtual high-feerate" via PrioritiseTransaction. I.e. you could look at the effective feerates of the last few transactions in the block and make a guess from there. Taking a low percentile instead may give you a more representative value.

@Prayank: Coin-age priority has been disabled since Bitcoin Core 0.12.0: https://bitcoin.stackexchange.com/q/49226/5406


text:
Is there a similar server like Bitcoin core that truly supports accounts?
Is there a similar server like Bitcoin core that truly supports accounts (and also RPC)?
I was hoping for something like the ethereum geth node. where each account has it's address and individually password protected.

Bitcoin Core since 0.15 supports multiwallet.
That is probably much closer to what people expected the accounts feature to be (and confusion about it is one of the reasons for deprecating it).
Using multiwallet, one Bitcoin Core instance can maintain multiple entirely separated wallets simultaneously. They all have their own addresses, own keys, own transactions, own coins, own labels.
This is in contrast to the older accounts feature, where the keys, coins, and labels are shared across all accounts - only a balance was maintained per account.

This is very new and suprising to me! I was lead to think every instance can handle a single wallet with the '-wallet' switch. Can you please give RPC exaple on how to query a specific wallet for getbalance for example or other commands without restarting the deamon? Im on 0.15.1

Read more here: https://bitcoin.org/en/release/v0.15.0#multi-wallet-support

Thanks, the big problem with this new feature is that I can't create new wallets via RPC. seems like I need some hack/workaround  to (somehow) create a new empty wallet-xxx.dat, write a new wallet entry in the bitcoin.conf file, and restart Bitcoin Core. Am I right?

please see my related question: RPC call to re-read config file?

That doesn't exist. For now, you'll need to restart if you want to add or remove wallets. The ability to change the wallets at runtime is being worked on for a later release.

Is there currently a limit to how many wallets I can load?

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Error when sending rawtransaction
I tried to write a transaction on my own in python to better understand how things work; when i decode the raw transaction, it seems properly written.
When I use the rpc method sendrawtransaction I get this error message:
error code: -26
error message:
64: non-mandatory-script-verify-flag (Non-canonical signature: S value is unnecessarily high)
This is the raw transaction:
0100000002b0b7c617588ccf1de8799221bf87a5d70ce47da4e5531d51748c10e1e7755c37000000006b483045022075ff5ff5587b6e6a2f34114bf3801cd2c9b076fe787f5705ca6acd2940d46189022100a65f3f8bd0e3f8c56567b28a37d3c60ff383fb10f416d25261c8453d278f8b81012102b4db032138335cd047878eac44db0dc1714c8ea2eda5079a705bbad6fdaa853dffffffffe59e23b45d3d65abd7b11ef45ca6e37c3193f4ee7c198edee20fbdc76e77f2b5010000006b4830450221008a9dbbe078feed10f50135a5521c18188e782be560ec4c31120dbf4551c6f23602201b1c9e34e1d32de425480b4afcdf5fef4957011511e6c859658c6ad70aca23f3012103fea6936eb5e3fbd4c45a8fae01395972619ef16ba307d1609addfe40668f3ff6ffffffff0580969800000000001976a914db9c91ed4690c016140e84d4d2ec72cb4225e59a88ac80969800000000001976a914d51779439ce3441647f5f4813177be8eb0febd3d88ac80969800000000001976a914fc5da273aea2b3b1346fa9648e4020e7697ceb3b88ac80969800000000001976a91428ef437e0e48b8ed4d65a10ca47a129149de6b4788ac00093d00000000001976a914476f97a1bdd330fae9bbde17de407fbfd409170488ac00000000
How can I fix it ?
Why "decoderawtransaction" works but I can't send it?
Is it invalid or just poorly written?
I am following this approach:
https://github.com/zeltsi/Mybitcoin/blob/master/tx%20with%20multiple%20inputs.py#L79
An these are my tx_tosign:
0100000002b0b7c617588ccf1de8799221bf87a5d70ce47da4e5531d51748c10e1e7755c37000000001976a9147c48aab4405b50ca5394fcf21bc08bb58bc91e0588acffffffffe59e23b45d3d65abd7b11ef45ca6e37c3193f4ee7c198edee20fbdc76e77f2b50100000000ffffffff0580969800000000001976a914db9c91ed4690c016140e84d4d2ec72cb4225e59a88ac80969800000000001976a914d51779439ce3441647f5f4813177be8eb0febd3d88ac80969800000000001976a914fc5da273aea2b3b1346fa9648e4020e7697ceb3b88ac80969800000000001976a91428ef437e0e48b8ed4d65a10ca47a129149de6b4788ac00093d00000000001976a914476f97a1bdd330fae9bbde17de407fbfd409170488ac0000000001000000
0100000002b0b7c617588ccf1de8799221bf87a5d70ce47da4e5531d51748c10e1e7755c370000000000b0b7c617588ccf1de8799221bf87a5d70ce47da4e5531d51748c10e1e7755c37000000001976a9147c48aab4405b50ca5394fcf21bc08bb58bc91e0588acffffffffffffffff0580969800000000001976a914db9c91ed4690c016140e84d4d2ec72cb4225e59a88ac80969800000000001976a914d51779439ce3441647f5f4813177be8eb0febd3d88ac80969800000000001976a914fc5da273aea2b3b1346fa9648e4020e7697ceb3b88ac80969800000000001976a91428ef437e0e48b8ed4d65a10ca47a129149de6b4788ac00093d00000000001976a914476f97a1bdd330fae9bbde17de407fbfd409170488ac0000000001000000
Thanks

the answer on here would be helpful for you https://bitcoin.stackexchange.com/questions/58176/what-does-16-mandatory-script-verify-flag-failed-non-canonical-der-signature

one more hint: you can create in a bitcoin core client the transaction like described here: https://bitcoin.stackexchange.com/questions/71990/how-can-i-specify-multiple-txids-in-createrawtransaction-function, and then compare it to your output :-)

The transaction has 2 inputs, and 5 outputs. I see you are on testnet3.
I looked at the S-Values:
 21: OP_LENGTH_0x21:      this is SIG S (33 Bytes)
     00A65F3F8BD0E3F8:C56567B28A37D3C6:0FF383FB10F416D2:5261C8453D278F8B:81
checking S-value is less than N/2, S-value is zero padded - ok

 20: OP_LENGTH_0x20:      this is SIG S (32 Bytes)
     1B1C9E34E1D32DE4:25480B4AFCDF5FEF:4957011511E6C859:658C6AD70ACA23F3
 checking S-value is less than N/2, yup... - ok

I don't see an issue here... What have you used unsigned tx?
UPDATE 2018-Mar-10:
Quickly decoding first part of the unsigned transaction:
 01000000
 02
  375C75E7E1108C74511D53E5A47DE40CD7A587BF219279E81DCF8C5817C6B7B0
  00000000
  19
  76A9147C48AAB4405B50CA5394FCF21BC08BB58BC91E0588AC
  FFFFFFFF
  B5F2776EC7BD0FE2DE8E197CEEF493317CE3A65CF41EB1D7AB653D5DB4239EE5
  01000000
  00
  FFFFFFFF

 05
  8096980000000000
  19
  76A914DB9C91ED4690C016140E84D4D2EC72CB4225E59A88AC
  8096980000000000
  19
  76A914D51779439CE3441647F5F4813177BE8EB0FEBD3D88AC
  8096980000000000
  19
  76A914FC5DA273AEA2B3B1346FA9648E4020E7697CEB3B88AC
  8096980000000000
  19
  76A91428EF437E0E48B8ED4D65A10CA47A129149DE6B4788AC
  00093D0000000000
  19
  76A914476F97A1BDD330FAE9BBDE17DE407FBFD409170488AC
00000000

I can see the scriptsig is only filled for one part of the tx. So ok for the first round. Then tx part two is somehow messed up:
 01000000
 02
  375C75E7E1108C74511D53E5A47DE40CD7A587BF219279E81DCF8C5817C6B7B0
  00000000
  00
  B0B7C617  <-- I think this should be FFFFFFFF (Sequence number)
  00000000375C75E7E1108C74511D53E5A47DE40CD7A587BF219279E81DCF8C58...

I thing the sequence number is missing, and then should somehow continue like this:
  Sequence number FFFFFFFF
  OutPoint hash   B5F2776EC7BD0FE2DE8E197CEEF493317CE3A65CF41EB1D7AB653D5DB4239EE5
  OutPoint index  hex=01000000, reversed=00000001, decimal=1
  Script Length   hex=19
  Script Sig      76A9147C48AAB4405B50CA5394FCF21BC08BB58BC91E0588AC
  Sequence        FFFFFFFF

and then the TX_OUT structure...

I tried to spend these two outputs:
https://testnet.blockexplorer.com/tx/b5f2776ec7bd0fe2de8e197ceef493317ce3a65cf41eb1d7ab653d5db4239ee5
https://testnet.blockexplorer.com/tx/375c75e7e1108c74511d53e5a47de40cd7a587bf219279e81dcf8c5817c6b7b0

yup, this is clear. Can you show the unsigned tx, that you used?

Modified the question!

I modified my answer, see section after "UPDATE 2018-Mar-10".

Thank you for the answer. I fixed the second message but it still gave me the same error.
I tried to use the option sigencode=ecdsa.util.sigencode_der_canonize in the method sign_digest instead of ecdsa.util.sigencode_der and it worked.
I understand that DER is the signature encoding used in bitcoin; but what does canonize means?

It is properly written but it won't work because its theoretical, there are no actual coins in there.

I have no clue what you're trying to say.


text:
Electrum - using a fixed fee regardless of transaction size
I am using Electrum 2.7.12, and I intend to use the python console to paytomany() while using a fixed fee. I have tried following directions from http://matt.ramuta.me/blog/1419787186410-how-to-write-scripts-for-the-electrum-wallet (with a slight workaround for the csv).
How can I create a transaction using paytomany() containing a fixed transaction fee?
Never mind I solved it

Fixed transaction fee often make little sense. Maybe your transaction require/select 10 inputs and therefore your transaction gets much bigger in size and require more space in the next block. This is why feerates (fee per KB) are mostly the better choice.

Direct quote from owner of service I am using: "and fee  is as below.
0~1kB : 0.0002 btc
1~2kb : 0.0004 btc
2~3kb : 0.0006 btc", and I doubt I will send transactions with more than 3 or 4 outputs


text:
Pls Help! My transactions on multibit HD is yet to be confirmed after 24hrs
please can anyone help here as I'm a beginner on btc.
I sent some bitcoins but after more than 24 hours is not yet confirmed but shows it has been seen by peers on multibit HD. I have tried on 5 occasions on repair wallet but yet no results.
https://blockchain.info/tx/b57acae08aa1aaff70d8c74bf823e6a5b6f536b1524cc777d9f4e89794ca64bc
3b1b1c3822ab74cad22ca34731cb298d906e04acc741bb5e7fc38b2b333c2ccc
b03eb0087f4c75d4bdfa2e1788a92b1402037f210f17caf71b6a66d73597356c
I decided to transfer my other wallet to a new wallet and still not showing too.
https://blockchain.info/tx/c7309ba1267012506fcb99de79f060935811fc74e049df6e22388b874903192a
Thanks
N.B: fess was set as default, been over 36 hours and the transaction not reflecting on the receiving wallet

Always check the current backlog when this happens. The network is experiencing exceptional circumstances. We just need to wait.

I didnt know about this. just started using multibit HD. I'm getting a bit worried that I have lost my money :(

@jenny Your money isn't lost - the nodes on the network disagree about who owns it.

Thanks for the response Nick. But please any suggestion of how I can recover it? Thanks


text:
ccminer JSON-RPC call failed: Method not found
I am having issue with running ccminer with bitcoin-qt.exe (Solo mining)
I am receiving error below upon start:
[2017-01-26 14:23:57] 1 miner thread started, using 'bitcoin' algorithm
[2017-01-26 14:23:57] Binding thread 0 to cpu 0 (mask 1)
[2017-01-26 14:23:57] JSON-RPC call failed: Method not found
[2017-01-26 14:23:57] json_rpc_call failed, retry after 5 seconds

How to resolve this issue?
What I did:
my ccminer running parameters:
ccminer -a bitcoin -o http://127.0.0.1:8332 -u USER -p 123456789 -R 5 -D

my bitcoin.conf file:
rpcuser=USER
rpcpassword=123456789
server=1
daemon=1
listen=1
#rpcallowip=127.0.0.1
rpcport=8332

I have tried to add -P into ccminer running parameters, and the result is
* Rebuilt URL to: http://127.0.0.1:8332/
* Hostname 127.0.0.1 was found in DNS cache
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8332 (#1)
* Server auth using Basic with user 'USER'
> POST / HTTP/1.1
Host: 127.0.0.1:8332
Authorization: Basic xxxxxxSOME_BASE64xxx
Accept-Encoding: identity
Content-Type: application/json
Content-Length: 45
User-Agent: ccminer/8.05-KlausT
X-Mining-Extensions: longpoll noncerange reject-reason
X-Mining-Hashrate: 0

< HTTP/1.1 404 Not Found
< Content-Type: application/json
< Date: Thu, 26 Jan 2017 11:19:41 GMT
< Content-Length: 76
* HTTP error before end of send, stop sending
<
* Curl_http_done: called premature == 0
* Closing connection 1
[2017-01-26 16:19:41] JSON protocol response:
{
   "error": {
      "code": -32601,
      "message": "Method not found"
   },
   "result": null,
   "id": 0
}

[2017-01-26 16:19:41] JSON-RPC call failed: Method not found
[2017-01-26 16:19:41] json_rpc_call failed, retry after 30 seconds

I thought, maybe bitcoin-qt.exe does not accepting any connections, but when I run bfgminer with parameters:
bfgminer -S opencl:auto -o http://127.0.0.1:8332 -u USER -p 123456789 2>logfile.txt

everything works fine...
P.S. everything is up to date:
Bitcoin v0.13.2 (64 bit)
ccminer v8.05-KlausT
pthreads static 2, 9, 1, 0
libcurl/7.50.3 WinSSL WinIDN

Thanks in advance

Above question was also asked at Github

Sorry, but Bitcoin solo mining is no longer supported, because they
  removed the getwork command. Trying to mine Bitcoin is useless anyway,
  the difficulty is too high.


text:
How to get the Bitcoin address from the bitcoin Public key?
I am working in js and I need a library that helps me get an address from a public key stored in a buffer format.

Try bitcoinjs-lib
The following will generate a P2SH(P2WPKH) address
const bitcoin = require('bitcoinjs-lib')
let pubkey = Buffer.from('027a2c8c567a5bb7777580d31d1fd8d9910d4b1bb398570e3a36e81d7522cf828a', 'hex')
let address = bitcoin.payments.p2sh({redeem: bitcoin.payments.p2wpkh({ pubkey })}).address
console.log('address: ', address)

address:  3F1JMjuxgNFuK4kBxv4JXMEdQ2yvjuz939


text:
Problems Setting a Transaction Fee on a Per Transaction Basis Bitcoind
I've spent hours and hours trying to find a solution to this problem, with no success.
Using Bitcoin Core on a Linux machine, so using bitcoin-cli to process commands back to the daemon.
Here's what I want to achieve: I want to send a transaction and be able to specify a custom transaction fee in Bitcoin/KB on a "per transaction basis".
So far, I haven't found a solution. These are the areas I have been looking at:
Change the paytxfee global variable and then just send the transaction. ---> If I do this, the resultant fee once the transaction has been sent is about 40% lower than what I specified in the paytxfee variable. Why is it smaller?
So.... does anyone please have any ideas how I can just send a transaction and specify a custom fee which is the actual fee used when the transaction is broadcast to the network?
Thanks in advance for your help.
Edit
Hi. Thank you for your reply. I'm panicking a bit as I have to get this sorted by tomorrow. I've tried setting "paytxfee" with the "settxfee" command. In the last example I sent a dummy transaction using "sendtoaddress". I entered "0.00170" in the "paytxfee" global variable (170 satoshis per byte). Having looked at the resultant transaction on Blockchain.com explorer, only 114 sat/B were used instead of the requested 170. When I used "fundrawtransaction" and specified a "feeRate" option, it was calcultaing the fee on the virtual size rather than the total size, so once again I was getting about a 40% reduction in fees once the transaction was broadcast. I now have loads of transactions stuck!! Anyway, that's not a problem as they were only tests... Any ideas how I can resolve this? Happy to send screenshots or maybe we can email/use messenger? Desperate to get this sorted .... Thank you so much!

Is the resulting feerate wrong? fundrawtransaction will try to predict the size of the signatures (even before they're actually added), and use that to determine the total fee.

My reply was too big so I've appended it on to the bottom of the question for you. Thank you!


text:
Bitcoin Core Vs Daemon (Windows 10)
I assume that if I've Bitcoin installed locally and I either run the GUI or the Daemon... It's the same wallet, data?
To that end, if I want to use the RPC interface, can i just use the GUI while developing (I like that at alerts on any action) rather than the daemon?
TL:DR: is the RPC 8332 port active when using the GUI, Vs the Daemon?

It's the same wallet, data?

This depends on your configuration like -datadir. This can be specified by command line, or something like a registry key value (when you are using Microsoft Windows).
Note that recent versions of Bitcoin Core supports loading multiple wallet at same time.

TL:DR: is the RPC 8332 port active when using the GUI, Vs the Daemon?

Yes, of course both bitcoin-qt and bitcoind supports JSON-RPC.
Edit: My apologies. For bitcoin-qt GUI, to enable JSON-RPC you must specify -server=1 in command line options, or edit bitcoin.conf to add server=1 to it. For command-line bitcoind, JSON-RPC is enabled by default. Thanks for Pieter Wuille for pointing this out.

You need to start bitcoin-qt with the -server=1 command-line option (or put server=1 in bitcoin.conf) in order to enable JSON-RPC access. For bitcoind it is enabled by default.


text:
Block version 0x20000000
When i run getblocktemplate, Bitcoind gives me in the version field a decimal number, when i convert it to Hex it's 20000000
I know it must be in little endian, but to build the blockheader must i put it like this 00000020 or 02000000  ?
I am asking this question because in version 1 the blockheaders are using 01000000 and not 00000010 so i am confused
Help is appreciated
Thanks to all of you

The bytes you would put in the block header would be, in order: 0x00 0x00 0x00 0x20. That's how the number 221 (= 0x20000000) is stored in 32-bit little endian encoding.
Note that this is actually a large number; that range is used for BIP9-compatible versions. Older blocks that use versions 1 through 4 predate BIP9.


text:
Safely transferring from a paper wallet
I am holding for the long term and have a significant amount of bitcoin (~10) in a paper wallet (the actual private key is printed on a piece of paper). To improve security measures and possibly sell forked coins, I would like to transfer the funds to a new address (paper or hardware wallet).
I understand the technical procedure - sweep the private key into a wallet software and send to the new address. I am concerned about taking the proper security measures.
Some online resources recommend myCelium, but while I am cautious I do not trust my Android devices to be free of malware. I plan on using a PC with a Live CD Linux distribution (e.g. Ubuntu) installed on a fresh USB disk and connected behind a home router.

Are there any other security measures I can take besides running an OS with no malware and a reputable wallet software?
Are there any specific disadvantages to using Electrum for this vs. running a full node wallet such as Bitcoin QT (besides privacy)?
Does the Electrum Cold Storage feature (signing transactions on an offline machine) has any benefits for a one time transaction?

I'm voting to close this question as it is "primarily opinion-based".

@Adam reworded the opinion-based question (the other two questions are not opinion-based IMHO)

Just go ahead with Bitcoin-core. Which is much more secure than the others.
Encrypt your wallet with some long strong password and you're good to go.
Electrum recently had a critical security vulnerability. Read More About it.

Thank you for the response. As mentioned in the question, I do not plan on using the wallet software other than for this one-time transaction. Does it make sense to download the entire blockchain for a one-time transaction (might actually increase the chances of getting infected with malware)? I understand that the Electrum vulnerability was related to wallet files, something I do not intend to use.

you should sign the transaction somehow, you can make your own transaction in any way you prefer. but the danger is when you sign that transaction, you should find some software to sign it.


text:
New Venezuelan crypt worth it?
Venezuela is about to launch its own Petro cryptocurrency
Could the chaotic economic scenario the country have lived be a good chance to  change the usual  currency to a criptocurrency?
I mean, What can significantly mean success or failure in this case?

From my point of view, any chaos in the monetary system may be an opportunity for an alternative system.
Taking control out of the banks would be the first step towards an attempt to improve the economy, but we must be very careful about the government control.
The current scenario in Venezuela is all fault of the Government and not totally of the banks, if the government itself wants to launch a monetary system without the interventions of the banks what would be the logic of this?

It seems that the international (led by US) hostility, sanctions, and support of opposition activities definitely didn't help. Though of course the government didn't do a good job as well. Still, it would be a bit absolute to say it is "all fault of the government"?


text:
As required in BIP 34, how miners put the Block Height to their coinbase transaction
I created a coinbase transaction with the createrawtransaction method as follow:
createrawtransaction '[{"txid":"0000000000000000000000000000000000000000000000000000000000000000","vout":0}]{"kgjdtruea5w3mz7t3tudtlplm9xht4sjwg0e3tihgtr":6.25000000}'
the response to this call is:
020000000100000000000000000000000000000000000000000000000000000000000000000000000000ffffffff0140be40250000000017a9145ea405d95b789aa887e229d8b1da44c2964054248700000000

I understand that the number of pushed bytes and the Height of the block must be included to the scriptsig of this raw data as follow:
Actually 03 + The height of the block in little endian
In this exemple must i put this just between the ffffffff and  0140be40250000?
I saw some exemples but i am confused, as there's the size of the scriptsig before the scryptsig
Help is appreciated
Thank you to all of you

In this transaction, the scriptSig is of size 0 because it isn't filled out yet. You have also used a txid of all 0's and a vout of 0, so this size of 0 gets lost in your large string of zeroes.
00000000000000000000000000000000000000000000000000000000000000000000000000ffffffff is the input data.
The first 32 bytes of this is the previous txid. The 4 bytes following that is the vout. This leaves us with 00ffffffff. That remaining 0x00 byte is the length of the scriptSig, currently length 0. The ffffffff is the sequence number.
What you have to do is replace that 0x00 byte with your scriptSig. The scriptSig is prepended by it's length, so what you replace that with would look like 040334cc09.
Note that currently your transaction would not be a valid coinbase transaction. The vout for a coinbase transaction is 0xffffffff, not 0.

Thank you for your answer, but when i try the ffffffff in the vout, it gives an error, and in the documentation it says the vout value must be numeric, how it can accept a hex value?

You can't. It doesn't accept it as hex and both 4294967295 and -2147483647 are not accepted (the former is too large, the latter is negative which isn't allowed). You will have to modify the raw transaction to contain this value. The easiest way to do this is to use 2147483647 as the vout. This will give it a vout of 0x7fffffff (in little endian so it will be ffffff7f). Then just change that 7 to f manually.

Yes, that's great, i put 2147483647 as the vout and it gives me this: 020000000100000000000...0000000000ffffff7f00ffffffff0140be4025000000001600140f3be243717c5bb903666c2de78c55d632e8e8a500000000, i changed the 7 to f, then added the scriptsig prepended by it's length so it become 0200000001000000000000000000000000000...00000000000000000000000ffffffff + 0403 + Block height + ffffffff0140be4025000000001600140f3be243717c5bb903666c2de78c55d632e8e8a500000000 and now it become a valid coinbase transaction


text:
How to build coinbase transaction from bitcoin core?
I am coding a program that connects to bitcoin-qt as local server. For building the blockheader i need the merkleroot, to build the merkle root, i need the coinbase transaction (which is not provided when connecting directly to bitcoin-qt) i understand that i must use createrawtransaction to build it manullay as below:
createrawtransaction '[{"txid":"00000...000000000000","vout":0}]' '{"BTC ADRESS":6.5}'

My question is: In BTC ADRESS i put any adress that i choose from my bitcoin-qt wallet?
And if i double hash the given hex result returned to me, without adding any other transaction, this can be the merkleroot (with only the coinbase transaction) and use it to build the blockheader?

My question is: In BTC ADRESS i put any adress that i choose from my bitcoin-qt wallet?
Yes!! You would basically use a generated receiving address belonging to your btc wallet and use that as your value within the createrawtransaction address parameter field.
Although you would need to call the getnewaddress command via any means available to you after which you can make use of it.
.
Assuming that you've already changed directory to where your bitcoin-cli or bitcoind executable file is located
CMD approach would look something like this:
bitcoin-cli getnewaddress  which would display newly generated address in the cmd terminal window.
bitcoin-cli getnewaddress > newlygenerated_BTC_address.txt  which would store the output into a text file
To make things more dynamic you would need to write a batch script to handle aspects that your programming language isn't able to handle. But hopefully you get the idea.

And if i double hash the given hex result returned to me, without adding any other transaction, this can be the merkleroot (with only the coinbase transaction) and use it to build the blockheader?
Yes. You would insert the double hashed result of the raw transaction data as the merkleroot without swapping it( That will be done later when swapping the blockheader for mining). If you swap twice, you'll end up with an incorrect coinbase hash.
Other than that, you would need to correct and insert a few things.
Assuming hex response of createrawtransaction produces -> 02000000010000000000000000000000000000000000000000000000000000000000000000ffffff7f00ffffffff0140be402500000000160014f68d712fa6f49cdfaaa1707a9d0234e2aabb3f1100000000
:::

02000000010000000000000000000000000000000000000000000000000000000000000000ffffff7f00ffffffff0140be402500000000160014f68d712fa6f49cdfaaa1707a9d0234e2aabb3f1100000000

____________________________________________________________________

02000000 - Tx version -- 8 bytes
____________________________________________________________________

01 - input count -- 2 bytes
____________________________________________________________________

0000000000000000000000000000000000000000000000000000000000000000 - txid -- 64 bytes
____________________________________________________________________

ffffff7f - vout -- 8 bytes
____________________________________________________________________

00 - size of scriptsig -- MAX OF 100 bytes
____________________________________________________________________

 - scriptsig size is missing - hexadecimal value
 - byte push is missing - hexadecimal value
 - block height (little endian) is missing - hexadecimal value
 - arbitrary data(extranonce/message) is missing - hexadecimal value
____________________________________________________________________

ffffffff - input sequence -- 8 bytes
____________________________________________________________________

01- number of output -- 2 bytes
____________________________________________________________________

40be402500000000 - reward/Output Value -- 16 bytes
____________________________________________________________________

160014f68d712fa6f49cdfaaa1707a9d0234e2aabb3f11 - scriptPubkey -- 46 bytes (dynamic in length)
____________________________________________________________________

00000000  - locktime
____________________________________________________________________

My C++ solution :

bHeight - Block height as given by **```getblocktemplate```** response
Headers needed - sstream, string, iostream.
Namespaces used - using std::stringstream, using std::string.

string SwapBlockHeight(string data)
{
    string bits; // global return variable
    if(data.size() == 5)
    {
        string byte1, byte2, byte3;
        byte3 = data.substr(5, 2);
        byte2 = data.substr(3, 2);
        byte1 = data.substr(1, 2);
        bits = byte3 + byte2 + byte1;
    }
    else if(data.size() == 6)
    {
        string byte1, byte2, byte3;
        byte3 = data.substr(4, 2);
        byte2 = data.substr(2, 2);
        byte1 = data.substr(0, 2);
        bits = byte3 + byte2 + byte1;
    }
    else if(data.size() == 8)
    {
        string byte1, byte2, byte3, byte4;
        byte4 = data.substr(6, 2);
        byte3 = data.substr(4, 2);
        byte2 = data.substr(2, 2);
        byte1 = data.substr(0, 2);
        bits = byte4 + byte3 + byte2 + byte1;
    }
    return bits;
}

string GETLittleEndian_BlockHeight(uint32_t bHeight)
{
    string reversedDATA;
    char i[25];
    sprintf(i, "%x", bHeight);
    stringstream aa;
    aa << i;
    aa >> reversedDATA;
    // reversedDATA = SwapBlockHeight(reversedDATA);
    // Optimized approach of above function call is applied below
    // Assuming hex conversion result is = "1d34f589"
    string byte;

    // Optimized approach
    for (int i = 0, k = 0; i < reversedDATA.length()/2; i++)
    {
        byte += reversedDATA.substr(k, 2);
        k+=2;
    }
    reversedDATA = byte; // or you could just return byte itself
    // reversedDATA.length()/2 - because we will be done with our goal in half the total size.

    return reversedDATA;
}

Bytepush = "03";
Blockheight_littleEndian = GETLittleEndian_BlockHeight(bHeight);
ArbitraryData = "abcdefgh"; // Anything within 100 bytes .. 1 char == 2 bytes

scriptsig_Size = Bytepush.length() + Blockheight_littleEndian.length() + ArbitraryData.length();

// ALL VALUES ARE HEXADECIMAL

LOOKING FOR JSON PARSER/SERIALIZATION??

Get nlohmann json library
Include in project folder

A little batch script used but nothing too advanced.
#include "json.hpp" // for json serialization parser
#include <fstream> // for ifstream
#include <stdlib.h> // for system calls
#include "windows.h" // if on windows
using namespace nlohmann;
using std::ifstream;

inline void RunCommand_With_Output_Without_SYMBOL_Defined(string Command_To_Run, string Output_FileName)
{
    string xx_combine = Command_To_Run + " >" + Output_FileName;
    char run_command[xx_combine.length()];
    strcpy(run_command, xx_combine.c_str());
    system(run_command); // execute the command
// std::cout << ifstream(Output_FileName).rdbuf(); // print to console -- FOR DEBUGGING ONLY!!
} // Successfully compiled on 20/01/2022 10:20PM

void GETBLOCKTEMPLATE()
{
    string getblocktemplate_syntax = "cd \"C:\\Users\\YOUR DESKTOP NAME\\Desktop\\Z Code\" && call gt.bat";
    string filename = "getblocktemplate_Response.json"; // Create file Name
    RunCommand_With_Output_Without_SYMBOL_Defined(getblocktemplate_syntax, filename); // create json file with getblockresponse output
} // Successfully compiled on 31/01/2022 11:30AM

uint32_t Get_BLOCK_HEIGHT_asInteger()
{
    uint32_t f;
    ifstream file_input("getblocktemplate_Response.json");
    json object = json::parse(file_input); // Parse json data
    file_input.close(); // close streamer
    f = object.at("height");
    return f;
}

WHATS INSIDE THE .BAT FILE??
gt.bat
@echo off
cd "C:\Users\YOUR DESKTOP NAME\Desktop\Z Code" && bitcoin-cli getblocktemplate {\"rules\":[\"segwit\"]}

rem or

cd "C:\Users\YOUR DESKTOP NAME\Desktop\Z Code" && bitcoin-cli getblocktemplate {\"rules\":[\"segwit\"]}

Rem copy file created to destination folder
copy "C:\Users\YOUR DESKTOP NAME\Desktop\Z Code\getblocktemplate_Response.json" "C:\Users\YOUR DESKTOP NAME\Desktop\Z Code\COINBASE ONLY\BITCOIN_Miner"

NOTE!! - Copying of json file from one path to another is only needed if your gt.bat is not located in your project root folder along with header files. This would cause errors later on because in as much as the json file was created and holds block template response, it won't be found UNLESS you tell the json reader to go into whichever folder where it will be generated and get it from there.
It's a lot more than requested but others might find the info useful.
I'll create a repository on github later.
Ignore what you don't need.
Cheers


text:
Did pools verify constantly the ouput of createrawtransaction when they build their coinbase transaction?
Pools creates their own coinbase transaction to pay their selves, and they begin the PoW by hashing different Headers, what prevent a hacker from changing the createrawtransaction with different output (His bitcoin adress instaed of the pool bitcoin pool adress)?
How pools can know that the header they are providing for miners contains a merkle root which contains a coinbase transaction created with a valid ouput (Their bitcoin adress and not another adress)?
Thanks

If a hasher takes the work given by a pool, and modifies it to pay themselves instead of the pool, then their block shares just won't be counted as valid work by the pool, and thus won't get any reward. Is that the scenario you're concerned about?

No, i dont mean that, i mean how the pool knows that the header they provided for hashers is valid one

They create it themselves? They obviously have to trust their own software, or they have a problem.

Yes, but the software will communicate with the bitcoin core, the software will submit an RPC to build the coinbase transaction, i mean in that moment what prevents a hacker of manipulating that ouput? if he did it, how pools verify that? constantly?

If the pool's operations have been infiltrated by a hacker that has access to all their internal systems, then they have a problem. If not, then there is no such problem.

Yes, thank you for your clarification :-)

I mean... this is literally the same as everywhere, and not specific to mining, bitcoin, cryptocurrency, or even financial services in general at all. If your infrastructure is hacked, you have a problem.

Yes, i understand, by my question i thought that pools check constantly if the headers they provides to miners contains their correct coinbase transaction

With the Stratum V1 protocol in use today (and the Stratum V2 protocol as it is currently proposed), mining pools actually send the parts of the coinbase transaction to their miners for those miners to modify. Specifically, they reserve space in their coinbase transactions for miners to use as an extra nonce. Those miners will construct the coinbase transaction, insert their extra nonce, compute the merkle root (they are given merkle branches to be able to complete the merkle root), and then create the block header.
However when the miner submits their work, they are only submitting the pieces that they added - namely the nonce, the timestamp, and the extra nonce. The pool will then reconstruct the coinbase transaction with the given extra nonce, reconstruct the merkle root, reconstruct the block header, and verify that the block header meets the PoW requirement for that miner. If the coinbase transaction had been modified by the miner, or by an attacker who sits between the pool and the miner, then it is unlikely that the block hash they calculate for that submitted share will meet the PoW requirement for that miner.
It is unlikely that miners are using createrawtransaction for creating their coinbase transactions as it is unable to do so anyways. createrawtransaction does not allow the caller to set an input's vout to be the right value for a coinbase transaction. So mining pool software are most likely creating the coinbase transaction internally rather than outsourcing it to bitcoind. In that case, they will be using the correct outputs. If they aren't, then either something is manipulating the coinbase transaction after the pool sends it but before the miner receives it, or the attacker is able to directly modify the pool software's memory.
In the first case, the PoW checking I described earlier is the protection. In any case, the pool can't really do much about an external entity (by external, I mean external to the pool software, it could still be running on the same compromised machine) modifying the coinbase when they send it out. In the second case, the pool can't protect against that as the attacker can just always make the software think that their coinbase is correct.


text:
Blockchain SHA256 hash and nonce
I have been reading the SHA256 hash creation with a random nonce. So, my understanding is every miner would pull uncommited transactions and would try to create a Hash of the transactions with a random nonce starting with 0 and increment from there on. So, if every node/miner is performing the same hash with the nonce sequence starting from 0, it would always be the miner with the maximum computing power always outrun the smaller ones. How would a miner with lesser computing power ever be able to create a hash quicker than the largest one?
My second question is every time a block is added to the chain, would every miner stop their current process and restart all over again since the next block would depend on the hash of the currently added block?

Your second question is answered here: Do miners have to start over whenever a block is found?

The block hash has to be below a certain value and the block hash depends (among others) on the nonce and on the Merkle root.
The Merkle root depends on the sequence of transactions. Note that it's a sequence, not a set, meaning the order is important.
Different miners choose many different lists of transactions. For every list of transactions with the other fields which go into the block hash, chances are there is no 32 bit nonce which will make the block hash low enough to qualify as a new block on the blockchain. So miners add a transaction, or remove one, or permute some to get a different Merkle root all the time.
I'll just calculate the time an AntMiner S9 with a hash rate of 14 TH/s takes to go through all the numbers in a 32 bit field (2^32 hashes) for you:
2^32 H / (14 TH/s) = 2^32 / 14 / 10^12 * s = 0.30678 ms
To answer your second question: Network traffic takes longer than 0.3 ms. To give you an impression on it: In fiberoptic cables, signals travel with about a third of the speed of light. With the speed of light, something travels 90 km in that time, so signals in fiberoptic cables can travel about 30 km in it. Many miners are more than 90 km away from each other and attempting to travel more than 90 km in that time turns out to piss physicists off, a lot.

Hi, could i bruteforce extra nonce with random bytes too? Should i update Merkle root after changing extra nonce. Maybe some good links where i can read about all this stuff explained from viewpoint of programming own mining software?

authors: Иван Пахомов


text:
What is the most cost-effective way to distribute a Bitcoin into 1000 different addresses?
If I have a wallet/address with 1BTC in it, how can I distribute it to 1000 different addresses (1k new addresses with 0.001BTC in each), paying the least amount of transaction fees?  Sending 1000 near-zero-fee transactions and hope that they all get through in an "acceptable" time is an option I guess, but are there any other options?

The best/cheaper way to split a single address into multiple is using batching transactions:
Batching explained in StackOverflow
This blog post from David Harding shows what are the saving possibilities when using batching to send your transfers and how it works:
Batching savings
Samourai Wallet supports batching if you don't want to use api's.


text:
Understanding blockchain 'under the hood' how a block is created
Bitcoin newbie here: I'm trying to understand the concept of the blockchain but what I don't get is what's going on behind the scenes?
So let's say that there are only 10 miners in the universe and Alice sends 15 BTC to Bob, what happens now?
I understand that a verification needs to happen (to ensure that Alice has 15 BTC to transfer)

Assume miner_1 gets this 'request' and verifies the request as valid. Does he build a block?
How a block is created? Is it created for every request?
How all nodes on the network are being updated that there
s a request/new block?
When SHA256(SHA256(transaction?) occurs? On verification or when adding a block? Is there a good example that shows how this works?
How the miner who managed to find the hash get's a reward?

possible duplicate of https://bitcoin.stackexchange.com/q/12427/5406

All miners are gathering transactions that they have heard about on the peer-to-peer gossip network.  This means that when someone broadcasts a transaction it gets propagated to all of the nodes it is connected to which then in turn share it with the nodes they are connected to etc etc...  These transactions are gathered by miners and stored in the mempool.  These are transactions that have not yet been included in a block or "confirmed".  The miner can choose which transactions it would like to include in the block, most miners gather as many TX's as they can to collect a larger reward, some do not some miners can build blocks with little or even no transactions included in it.  It is in the miners best interest to include as many tx's as possible to attain the fees generated by those tx's.  Miners take the aggregated transactions and the hash of the previous block and then generate a random nonce.  These items are all hashed together and if the hash of the block has an appropriate number of leading 0's to pass the current difficulty formula then a valid block has been found.  The miner then propagates the block the same way the transactions are distributed across the gossip network.  When the block is mined a coinbase transaction is created which the miner includes his address in, this address gets the block reward and fees for distribution (usually to a pool of miners).  The network continually adjusts the difficulty formula based on the hashing power of the network to ensure that one block is created about every 10 minutes.  There are some formulas in place to ensure the difficulty doesn't move too fast in the event of a large number of miners going offline at the same time.  Hope this answers your questions.


text:
Bitcoin core outgoing transactions
On The bitcoin core wallet you don't have to wait for it to sync to after your outgoing transaction in order for the transaction sent from the wallet to be confirmed and arrive at the addressee, does it? It's all about paying an appropriate transaction fee, right?
I only ask because on 2 attempts now after consulting bitcoinfees21 I have tried to establish a transaction fee that will be enough to be sent before the transaction bounces back and still no avail, separate question but it's a three day window until the transaction bounces back if still not confirmed, right?
Any help would really be appreciated. Thanks ...

First of all, transactions do not "bounce". Once a transaction is broadcast, it can confirm or remain unconfirmed indefinitely. There is no point where your transaction "expires" or the transaction "bounces" and the coins return to you; anyone who has a copy of the transaction can, at any point in time, rebroadcast the transaction and remind the network that the transaction still exists and can still be confirmed.
In order to send Bitcoin, you must be fully synced. If you are not fully synced, you risk spending coins that have already been spent and thus creating an invalid transaction. Furthermore, your wallet cannot know how much Bitcoin you actually have without being fully synced. While you can spend while unsynced, there is no guarantee that the transaction you are creating will actually be valid.

"In order to send Bitcoin, you must be fully synced." Meh. That isn't true; " If you are not fully synced, you risk spending coins that have already been spent and thus creating an invalid transaction" only if you've been using multiple wallets with the same keys-- which nothing supports and has that risk regardless of how synced you are.


text:
Active workers vs Active users
In a mining pool, it says that there are active workers and active users. What is the difference between them? And does the block get divided per worker or per user?

worker is analogous to the number of connections.

username is just a way of identifying specific miners in the list of connections and
to which account on the pool they belong to. They don't have any
impact on the work given.

For exemple you have an account in a mining pool ( You are ONE user), you have 5 ASIC in your farm, for every asic you give a worker name (You have FIVE workers)
All the work of your 5 ASIC will work for your username
PS: Generally miners give for every asic a worker name to identify a particular ASIC in case of a problem


text:
Why number of zeroes in bitcoin hash is increasing?
I know currently to mine a block there needs to be 19 leading zeroes in the hash, but why is this number increasing?

To find a valid block the hash of the blockheader must be below the target, the hash is Hex  number, everytime the difficulty is higher, the target is lower, so the number of zeroes will increase
For exemple this 32 bit number 0x00000100 contains five zeroes, if i want a number lower then this, it will contain 6 zeroes in this case: 0x000000ff
So the number of zeroes increase everytime the target is lower


text:
Can I verify my own transaction?
I'm waiting since 16/12/2020 and i still waiting in the Mempool. Exists a way how can i verify my own transaction without a Bitcoin miner? Yes, i paid a low fee.
Here you can see my transaction:
https://www.blockchain.com/btc/tx/cc32cd350dde5a9141e8180c63fb1aff4245d212d11c8249c58789a93acce4a3

Yes, there exist two possibilities to release your stuck transactions using Fee Bumping:

This page gives exact instructions on how to increase the fee on a transaction that is currently stuck in order to make it unstuck. This is always done by creating a new transaction that will either spend the coins sent by the stuck transaction (called child-pays-for-parent, or CPFP) or replace the stuck transaction (called replace-by-fee, or RBF).

Edit: I recommend consulting Johoe's Mempool Statistics next time before you broadcast a new transaction.


text:
Mini-seed of single utxo
Let's imagine we have a cold storage wallet that has multiple utxos inside of it and we have a master seed for this wallet.
If we want to spend from coldstorage but want to go first through a new wallet like a coinjoin hot wallet for example we have two alternatives: we could send one single utxo we are interestd in using to that hot wallet or even import the master key to the new wallet (not safe).
It would be very interesting to have a mini-seed of a single utxo from our stash that we could import in the hot wallet and use to spend that single utxo. This way we don't have to create a new tx towards the hot wallet, nor give up information like the whole stash or xpub or subsequent risks of hot wallet.
It would be great to right-click a single utxo on our coldstorage wallet and get the mini seed for that single utxo like another 24 words or 48 words.
Just an idea. Would this be possible?

With existing standards and wallets, no, that is not possible.
Existing standards such as BIP 39 do not directly use the value encoded by the words to produce a BIP 32 HD seed for use in a wallet. Instead they hash the words.
Furthermore, wallets that import that mnemonic do not just use the BIP 32 HD seed from it directly. Instead they use it with derivation paths and derive child keys from it.
If you produced a mnemonic from an individual private key, importing it into any existing wallet would not result in your private key being imported. You would end up with things derived from that private key, but that is not useful to you.

Instead of a mnemonic sentence, what you really want are the Wallet Import Format (WIF) private keys that most wallets can export. WIF is a standard way to encode individual private keys so just use that instead.

But would you see useful something like this added to bitcoin? a way to import the xprv for a single utxo from within a bigger wallet?

@zndtoshi I think you're missing the point of Andrew's answer. What you want to do is perfectly possible (though there is a huge caveat), but what you'd import is a private key, not a seed.

The huge caveat: you really shouldn't do this. In BIP32, when someone knows an xpub and one private key derived from it, they can compute the xprv and all descendant private keys.

ah ok thank you for the clarification Pieter.


text:
Bitcoin Core GUI gettransaction implementation
The cli users have the option to "getrawtransaction txid true" in order to find valuable information about a transaction that could be theirs. This is a very powerful api call that effectively eliminates the need of an electrum server if someone wants to check his tx sent to coldstorage or a mobile wallet.
Do you think there would be any consensus in adding such a feature to the Bitcoin Core GUI? Something like Help -> Verify external txid. This would greatly increase the use of Bitcoin core for the ease of validating a transaction.

That RPC is a wallet RPC, so it only works for transactions that are in your own wallet.

I used that rpc from my full node and verified a random transaction on the blockchain. The output of that command was the inputs of the tx and the outputs and number of confirmations and other info. So that is definitely from outside of the local wallet. Maybe I misunderstood your answer.

Perhaps you used getrawtransaction (which is a node RPC) instead of gettransaction (which is a wallet RPC)?

Sorry. You are right. Getrawtransaction. Should I edit the question to remove confusion?

Yes - though perhaps this is better as a feature request than an SE question. Note that it will only work when txindex is enabled. If it's about your own transactions anyway, why not import the relevant addresses/scripts into the wallet as watch-only?

I wanted to ask here first before making a feature request. Wanted to make sure its not a stupid request. Regarding importing watch only wallet those are extra steps users could just refuse to do or too difficult to import from HW.


text:
How does one mine a fraction of a bitcoin?
I'm new to the whole bitcoin stuff.
The stuff I've read about mining so far told me that when a node in blockchain finally finds the correct nonce for a block that can be accepted in blockchain, it then advertises it to others and they verify and put the block to the blockchain (at least this is what i understood from the explainations). I have also read that prize for mining a block is 12.5 BTC. My question here is how do nodes mine a fraction of a bitcoin when mining?(shouldn't it be 12.5 BTC per mined block for each node? is it even possible?)

Example: this

Answer to this question mentions 0.04 BTC,but how?

The current smallest unit of value in Bitcoin is a satoshi which is equal to 10^(-8) bitcoins. So when you say, the mining reward is 12.5 BTC, it is essentially 1,250,000,000 satoshis.

@UgamKamat so a node either mines 12.5 BTC or doesn't mine anything at all?

When a miner successfully mines a block, it gets the block reward + the fees of all the transactions in the block. The current block reward is 12.5 BTC. The first one to find the block header hash that matches is lower than the target hash at current difficulty, gets the block rewards

A node is not necessarily a miner. Not all nodes are miners, but all miners are nodes (at least they should be).

You either mine a full block...
-> Then you're free to take the full 12.5 BTC reward or less, at your choice.
Or..
-> You can mine with a pool. Everyone in that pool calculate hashes of trivial amounts, but their sum will be enough to mine blocks. Then, the pool will reward your work. (Search for PPS or PPLNS)

Is/can 'mining pool' implemented/be in the bitcoin protocol or a node with some bitcoins 'hires' a pool of node to the work and then splits the share?

A pool operator (that rich person) has a node. The node has a stratum (the mining protocol) end, using which miners (ASICs, FPGAs,... but not other nodes) can connect. The node assigns them templates. The miners try hashing them, and they share the results with the pool. The node publishes the block if the hash is valid for the network. At regular intervals the node sends payouts to miners.


text:
Finding a transaction in the blockchain
I wanna understand how can a user find a certain transaction he created on the blockchain without him knowing on which block it belongs ? does he have to go over all the blocks using bloom filters to find his transaction ? narrow the search to an interval of timestamps ? there must be a special mechanism ...
I searched on google but all i found were websites to find them for u, i want to understand the actual mechanism used in the bitcoin protocol?
Block Explorers seems to be storing those transactions in special databases and they use their own approach to find transactions ... My question is about how full nodes find those transactions ? do they go over the whole chain trying to find them ?

Are you asking about how to search for a transaction using Bitcoin Core? There is an RPC for that, assuming you enabled indexing. Or are you asking how Bitcoin Core does it? Bitcoin and Ethereum will be different, but you can visit ethereum.stackexchange.com

About how Bitcoin core does it

Bitcoin Core does not do this. It has a database of unspent outputs, and adds/removes outputs from that database. It (generally) does not have an index into the blockchain, or even the entire blockchain at all available.

Then my question is, how websites like block explorer works.

Seems like it was covered in here https://bitcoin.stackexchange.com/questions/58844/how-block-explorer-technology-works  but without great details or how exactly it works, just the general idea behind. But apparently, there isnt any native mechanism to the protocol that lets u search in blocks, u just need to go over all the blocks and search for transactions (in Eth, using bloom filters to know if a certain transaction belongs) in a way or another.

@rachidchami Bitcoin Core is a full node and hence maintains the entire database of the blockchain. If you are using a lightweight nodes that use SPV, those use bloom filters to query full nodes for transaction. These lightweight clients are then sent the merkle path by the full nodes that shows that the transaction is in the blockchain

@UgamKamat. How does the full nodes find such transactions ? do they go over the whole chain ?

@rachidchami The answer to your question is too broad for it to be posted as a comment. Here is an answer link: https://bitcoin.stackexchange.com/questions/81188/spv-clients-querying-the-full-node.

@UgamKamat thank you so much for your help.

After searching about this, i found out that each full node maintains a database with the set of unspent transaction outputs (UTXOs) in the memory. So When a new transaction is added, its UTXOs get updated accordingly (source). Also, a full node keeps track of all transactions that ever happened (source) and offer this:

Filtering transactions and blocks on behalf of lightweight nodes so that lightweight nodes do not need to download every transaction ever made on the network in order to find their own transactions.
Serving historical full blocks to nodes that have been offline for a while.
Transmitting new transactions from users to miners.
Broadcasting new blocks from miners to other nodes.

The RPC used for this call getrawtransaction that takes as an argument a TxID which is hashing twice the transaction data and gives you back the transaction you are searching for.

"NOTE: By default this function only works for mempool transactions. If the -txindex option is enabled, it also works for blockchain transactions."


text:
Will the amount of "usable" Bitcoins eventually drop to zero?
Let's assume that there is a non-zero probability that people lose their bitcoins, e.g. by forgetting their password. The amount of bitcoins these people possess will become "unusable" but still remain in the blockchain and contribute to the maximum quantity of 21.000.000 bitcoins.
Over time the amount of "usable" bitcoins would shrink more and more as people keep losing bitcoins. Will the amount of "usable" bitcoins eventually approach zero? What will happen then? Is there a mechanism to somehow recover lost bitcoins? At the moment I don't see such mechanism.

Also relevant: https://bitcoin.stackexchange.com/questions/484/why-doesnt-bitcoin-return-lost-coins-back-into-the-block-reward/487#487

Lost bitcoin are protected by the same security that protects bitcoin in circulation or storage, hence lost coins are lost forever.
There are many ways of looking at this "dilemma", and its effects are different depending on the economic context, but coin loss/destruction is generally considered to contribute to the value of the remaining "usable" coinbase. Since the lost coins are permanently removed from circulation and become unusuable, the principle of Supply and Demand implies that the remaining usable coins will become more valuable.
In economics, this process is called deflation of money supply. It has the effect of increasing the value of each token. It is the opposite of inflation of money supply ("money printing"/quantitative easing) where additional tokens are created with the effect that each token's value is decreased as the pool of tokens grows in relation to available goods and services in the economy.
You might be interested in this detailed discussion: Inflation, Deflation, Money and Credit


text:
How to read an exchange order book?
I am new to trading. On the left side is the order book and on the right, the trade history.
I can see that the order book keeps updating. The number in the market size column increases and decreases, but I don't see any update in the trade history. May I know why? Does the trade history only reflect part of all trades?

I'm voting to close this question as off-topic because learning how to read an exchange order book has nothing to do with bitcoins or cryptocurrencies.

Perhaps you can post this question on a different stack exchange site, like https://money.stackexchange.com/

I understand the point of contention - that this Stack Exchange should address technical matters pertaining to Bitcoin - and that a flood of questions such as "Why is Coinbase taking forever to verify my real-life identity?" is a potential liability. However, if we acknowledge that Bitcoin has an economic/financial dimension, and that its exchanges have idiosyncratic peculiarities of market making and representation, then the question is rudimentary, and my answer, below, all-encompassing. Read it, then argue this question post is not useful. Reject lesser, yes, but why destroy value?

Irrespective of bullish or bearish outlook, there are two attitudes a trader can have in the market: active or passive.
Limit orders and stop orders are "passive" trades because they don't move market price. These types of orders are offers to buy or sell bitcoin at a certain price. Passive orders are placed in the exchange order book and remain waiting there until cancelled (by the trader) or consumed by another trader.
Market orders are "active" trades because they consume limit and stop orders in the order book. A market order is executed immediately and moves market price ("last" price) to the level of the last order it consumes. Market orders are not shown in the passive order book, instead they are shown in the trade history column to indicate market activity
Example
Three traders: A, B and C. Market price is at $1,200.
A wants to sell 1 BTC at $1,201 and places a limit sell order in the order book and waits.
B wants to sell 0.5 BTC at $1,202 and also places a limit sell order and waits.
C wants to buy 1.4 BTC as soon as possible at any price near market price. Because C doesn't want to wait, C executes a market order for 1.4 BTC. It consumes A's order for 1 BTC and consumes 0.4 BTC of B's order at $1,202.
After execution of C's buy market order:

last price moves to $1,202
C has bought 1.4 BTC
A has sold 1 BTC
B has sold 0.4 BTC

C's market order for 1.4 BTC is entered in the trade history as the latest buy.
A's order is removed from the order book since it has been "filled".
The remaining (unconsumed) 0.1 BTC of B's "partially filled" limit sell order remains in the order book until another buyer consumes it, or until B cancels the order.
Summary
Traders who intend to buy or sell at specific price levels place their offers to buy/sell in the order book as passive orders. Active buyers and sellers consume passive orders via immediate market orders. Market orders move price and are entered in the exchange's trade history.
"Market size" is the total of all passive orders waiting in the order book. As traders place and cancel passive orders, market size changes.
"Trade volume" is the total of all active trades (market orders as reflected in the trade history) for a particular period of time.


text:
Maximum # of hashes for finding a block?
Since nonce is a 32 bit number, the maximum value of nonce can only be 4,294,967,296. So, if a miner doesn't find the target hash within this number, what happens?

Note: Although a signed 32-bit integer's max value is 2^32 - 1 = 2,147,483,647, that's not really relevant: The miners will iterate through all 32-bit values, so they have 4,294,967,296 tries really. Put another way, think of that 32-bit number as an unsigned integer.

The miner can change the block header hash several different ways:

Change the timestamp. This is the least intrusive, but works just fine. If you can do more than 4 Th/s, this starts to become a problem.
Add transactions. Adding any new transaction changes the Merkle root, so this gives you entirely new nonce spaces.
Change the coinbase transaction. This is the most common method. 32 bits are typically used to act as an additional nonce.

How can a miner change the coinbase transaction? More specifically, does this changeable field mean anything?

The scriptSig field of the coinbase transaction's txin can contrain 96 bytes of freeform data.


text:
Why do we need two transactions for CommitCoin scheme
I am studying BitCoin recently.
I noticed a method called CommitCoin for secure timestamping. It requires two transactions to send coins back with the same randomness so that others can compute the private key.
However, isn't it easier to just reveal the private key yourself? The whole point is to prove that at some time in the past, I knew something. If I send some coins to the address and send the coins back, and later reveal my private key and it matches the public key, shouldn't it be enough to prove that?
Thanks in advance.

Can you give a link to more information on CommitCoin?

@NateEldredge I assume asker is referencing this paper.

@NateEldredge I read it from Bitcoin and Cryptocurrency Technologies. It just briefly introduces it as a way to achieve secure timestamping without creating unspendable coins that give burdens to the network. I just want to know is there any problem if we just create a private key with the hash from data, then send coins to the corresponding address, and send it back. And when required, just reveal that private key. (Or just publish the private key somewhere.)

@LLS: CommitCoin was published in 2012, in 2014 OP_RETURN was added to cater to that use-case.

We wrote this paper before Bitcoin supported OP_RETURN. Now you would just use OP_RETURN.
In CommitCoin, you can reveal the private key off-blockchain. It might indeed be simpler. Our thinking is having it self-contained on the blockchain, without having to reference anything else or having to have a connection to the publisher was the simpler model.

Thank you very much for answering. That makes sense. It doesn't require a separate channel for additional information.


text:
Can there be 100% change transactions?
Is it possible to send 100% of the funds from one account to the exact same account?
Essentially, such an operation would be a NOP, but could be used as "proof of ownership" (e.g. by adding a comment with the transaction in reply to a proof-of-ownership challenge).

Yes, the output address can be chosen entirely independantly from the input address. So you can send all the money from one address to itself if you don't pay transaction fees.
"Proof of ownership" can be done easier and faster though, without performing a transaction at all. You can just use the private connected to the address to encrypt a random message from the other person. She can then decrypt it with your public key and assert that the result is the original message (and that the public key corresponds to your address). This process is called digital signing and actually happens when bitcoins are transacted.


text:
Can I run Armory on a Chromebook?
I want to buy a cheap computer to use exclusively as a secure offline wallet for cold storage. I am considering a Chromebook.
Has anyone managed to run Armory on a Chromebook?

Armory appears to be written in C++ and Python, two languages that are not supported by ChromeOS by default. You can take a look at the Armory source code here.
Because ChromeOS doesn't support these languages, there is no way to run Armory on a Chromebook without significant modifications, like rooting the device to get to the system shell or formatting the disk and installing Linux. I have heard of people doing both of these things in the past, but I'm not sure which specific device is required or how much effort they take.
If you are looking for a cheap computer to use for this sort of thing, the simplest bet in my opinion would be to purchase a low cost Linux desktop/laptop. This would provide you much more flexibility in what client you use, and would also help keep your wallets private. ChromeOS is pretty secure when it comes down to it, but Google+ is integrated very deeply with the OS, which may not be desirable for what you want to do.


text:
Bitcoin-Qt source code walkthrough
Has there been content published which provides an overview of how the Bitcoin-Qt software is architected?
I would like to dive into the source code, but going from scratch is somewhat taxing. Can someone help me?

https://bitcointalk.org/index.php?topic=41718.0
Don't miss the threads it links to at the bottom of the post.
But I wouldn't recommend diving right in if you don't know the basics of how bitcoin functions. You are going to need an okay understanding of cryptography used and C++, networking and program design.


text:
Can the Bitcoin script make a Google search?
This very interesting CCN Money article states

a script uses a data input such as a regular Google search to monitor
  real-world events that would automatically trigger disbursements or
  other actions

This part of the article suggests that the Bitcoin script could monitor external events, such as the result of a Google search, to trigger transfers. Can the Bitcoin script really do that?

https://en.bitcoin.it/wiki/Script and https://en.bitcoin.it/wiki/Contracts#Example_4:_Using_external_state are helpful

Bitcoin scripts are intentionally very limited, and can only access data inherent to the transaction (not even the blockchain it is in), in order to guarantee consistency (a script's validity cannot change over time).
However, there do exist ways to integrate external state into Bitcoin, through the use of oracles. See this wiki page for more information about those.


text:
Internals of MtGox
I'm curious as to how the MtGox exchange works. I'm looking for some comprehensive information. Below are specific questions, phrased in the terminology common for stock exchanges.

Is MtGox based on a limit order book? Is it an open order book?
Can I get order-by-order full feeds?
What type of orders can be sent? Only market and limit orders?
Does the matching engine proceed on a price-time priority basis?
Are there official or nominated market makers?
Does MtGox offer colocation services?

1) all orders are visible.
2) yes, you can get order feeds.
3) both. Market orders act like limit orders at what was market price.
4) yes, it had better do that.
5) no need for market makers.
6) Mtgox is too slow in general to worry about network latency.

Thanks. I've found the consolidated order book here, but not the order-by-order feed. Where can I get a full feed?

Regarding point #3: from my experience market (=instant?) orders eat up the top of the ask/bid orders. Every time I set a market order it ate multiple other orders with various prices.


text:
Is there a limited number of Bitcoin wallets?
Is there a theoretical upper bound on the number of possible wallets, similar to the upper bound of 21 million Bitcoins that can be generated?

Yes. ECDSA has a limited keyspace, just like we can't keep generating different RIPEMD160 hashes. But will this ever be a problem? No! :)

I think you are talking about Bitcoin addresses, those are considered disposable, consider this; a simple computer constantly generating Bitcoin addresses will generate all possible combinations in 69 tredecillion years.


text:
Block first transaction
The Bitcoin wiki states:

the first transaction of a block is, with few exceptions, the
  transaction that grants those coins to their recipient (the miner)

How are those "few exceptions"? When is the first transaction of a block not the miner reward?

Any time the miner chose not to include the reward he's entitled to, or chose to put the transaction elsewhere. As to whether that's ever actually happened, I don't know and am looking...

Based on bitcoin source code: https://github.com/bitcoin/bitcoin/blob/0.8.6/src/main.cpp#L2112 a valid block should have the Coinbase transaction (the one that spends the rewards and fees) as the first transaction and no other coinbase transactions must appear in the block.
Spending the full reward and fees is not enforced.
So your answer is only invalid blocks do not have a Coinbase transaction or have it at a different position from the first one.
But that transaction can be used to send the coins to any address not only miners one, so maybe there has been blocks using the coinbase transaction to pay for goods or pay other people different from the miner (for example Eligius pool uses it to pay the shares directly).


text:
Full animated order book
I have been using this Clark Moody webpage to track the price of Bitcoin on MtGox. A nice animated order book is provided, along with a good chart.
Are there similar pages but for other exchanges (because MtGox...)?

What about http://bitcoinwisdom.com/? It visualizes data from 14 different exchanges.

List of all sites I know of

http://bitcointicker.co/
http://bitcoinwisdom.com/
http://bitcoin.clarkmoody.com/
http://www.coinorama.net/ (only graphical order book)
http://bitjoy.org/~ (order book in "to do" list)

My favorite: https://bitcoinity.org/markets/mtgox/USD
16 exchanges

huobi
btce
bitstamp
btcchina
mtgox
bitfinex
kraken
bitcurex
cavirtex
localbitcoins
campbx
justcoin
bit2c
bitquick
rmbtb
icbit

20 currencies

AUD
CAD
CHF
CNY
DKK
EUR
GBP
HKD
ILS
JPY
KRW
NOK
NZD
PLN
RUB
RUR
SEK
SGD
THB
USD

10 time spans

10 minutes
1 hour
3 hours
12 hours
24 hours
3 days
7 days
30 days
6 month
1 year
2 years

Clark Moody is what I choose for quick glance as it offers basic candlesticks, MA and orderbook but orderbook is not visualised (as far as I can see), anyways if You want deeper in both directions:

http://www.tradingview.com is the very robust advanced technical charting + all asset classes
http://btccharts.com is very advanced bitcoin orderbook actually visualised

both have access to major Bitcoin exchanges, and are free.