text:
Where does the max_htlc_msat field in the getchaninfo results come from?
Using an arbitrarily chosen channel, the getchaninfo command returns a json object like this:
$ lncli getchaninfo 848897743567126529
{
    "channel_id": "848897743567126529",
    "chan_point": "e39544821899c05499ed1c3271b6668aea90246bfe181b72d20463a6579496f4:1",
    "last_update": 1673788833,
    "node1_pub": "025407ac7bd16419f4ae9e7827665c031323184514618c390278ec4a2708d3558f",
    "node2_pub": "03e865ea04a93245a0685e9dd5280514a9ff0d9162c3c3c004efab2636631229ef",
    "capacity": "5000000",
    "node1_policy": {
        "time_lock_delta": 34,
        "min_htlc": "1",
        "fee_base_msat": "0",
        "fee_rate_milli_msat": "842",
        "disabled": false,
        "max_htlc_msat": "4950000000",
        "last_update": 1673788833
    },
    "node2_policy": {
        "time_lock_delta": 40,
        "min_htlc": "1000",
        "fee_base_msat": "1000",
        "fee_rate_milli_msat": "300",
        "disabled": false,
        "max_htlc_msat": "5000000000",
        "last_update": 1673788419
    }
}

Excluding the disabled and last_update fields, there are 5 other fields that are specific to each node in the channel. It looks like four of these parameters are set from settings in lnd.conf as follows:

time_lock_delta = bitcoin.timelockdelta
min_htlc = bitcoin.minhtlc
fee_base_msat = bitcoin.basefee
fee_rate_milli_msat = bitcoin.feerate

But what about max_htlc_msat? I don't see anything in the lnd.conf documentation that looks like it could be that field.
I thought it might be set at the time of opening the channel, but in looking at the help for the openchannel command, I don't see anything that looks like it would be max_htlc_msat.
$ lncli help openchannel
NAME:
   lncli openchannel - Open a channel to a node or an existing peer.

USAGE:
   lncli openchannel [command options] node-key local-amt push-amt

CATEGORY:
   Channels

DESCRIPTION:

  Attempt to open a new channel to an existing peer with the key node-key
  optionally blocking until the channel is 'open'.

  One can also connect to a node before opening a new channel to it by
  setting its host:port via the --connect argument. For this to work,
  the node_key must be provided, rather than the peer_id. This is optional.

  The channel will be initialized with local-amt satoshis local and push-amt
  satoshis for the remote node. Note that specifying push-amt means you give that
  amount to the remote node as part of the channel opening. Once the channel is open,
  a channelPoint (txid:vout) of the funding output is returned.

  If the remote peer supports the option upfront shutdown feature bit (query
  listpeers to see their supported feature bits), an address to enforce
  payout of funds on cooperative close can optionally be provided. Note that
  if you set this value, you will not be able to cooperatively close out to
  another address.

  One can manually set the fee to be used for the funding transaction via either
  the --conf_target or --sat_per_vbyte arguments. This is optional.

OPTIONS:
   --node_key value                         the identity public key of the target node/peer serialized in compressed format
   --connect value                          (optional) the host:port of the target node
   --local_amt value                        the number of satoshis the wallet should commit to the channel (default: 0)
   --push_amt value                         the number of satoshis to give the remote side as part of the initial commitment state, this is equivalent to first opening a channel and sending the remote party funds, but done all in one step (default: 0)
   --block                                  block and wait until the channel is fully open
   --conf_target value                      (optional) the number of blocks that the transaction *should* confirm in, will be used for fee estimation (default: 0)
   --sat_per_vbyte value                    (optional) a manual fee expressed in sat/vbyte that should be used when crafting the transaction (default: 0)
   --private                                make the channel private, such that it won't be announced to the greater network, and nodes other than the two channel endpoints must be explicitly told about it to be able to route through it
   --min_htlc_msat value                    (optional) the minimum value we will require for incoming HTLCs on the channel (default: 0)
   --remote_csv_delay value                 (optional) the number of blocks we will require our channel counterparty to wait before accessing its funds in case of unilateral close. If this is not set, we will scale the value according to the channel size (default: 0)
   --max_local_csv value                    (optional) the maximum number of blocks that we will allow the remote peer to require we wait before accessing our funds in the case of a unilateral close. (default: 0)
   --min_confs value                        (optional) the minimum number of confirmations each one of your outputs used for the funding transaction must satisfy (default: 1)
   --close_address value                    (optional) an address to enforce payout of our funds to on cooperative close. Note that if this value is set on channel open, you will *not* be able to cooperatively close to a different address.
   --psbt                                   start an interactive mode that initiates funding through a partially signed bitcoin transaction (PSBT), allowing the channel funds to be added and signed from a hardware or other offline device.
   --base_psbt value                        when using the interactive PSBT mode to open a new channel, use this base64 encoded PSBT as a base and add the new channel output to it instead of creating a new, empty one.
   --no_publish                             when using the interactive PSBT mode to open multiple channels in a batch, this flag instructs lnd to not publish the full batch transaction just yet. For safety reasons this flag should be set for each of the batch's transactions except the very last
   --remote_max_value_in_flight_msat value  (optional) the maximum value in msat that can be pending within the channel at any given time (default: 0)
   --channel_type value                     (optional) the type of channel to propose to the remote peer ("tweakless", "anchors")
   --zero_conf                              (optional) whether a zero-conf channel open should be attempted.
   --scid_alias                             (optional) whether a scid-alias channel type should be negotiated.

The openchannel command includes a min_htlc_msat parameter, but no max_htlc_msat parameter. So where does that value come from?

Both the min_htlc_msat and max_htlc_msat in this case are part of the channel policy for forwarding HTLCs. The parameters are communicated in the channel_update message.
In LND, you set them with the updatechanpolicy command.


text:
How did bitcoin spread its first rewards
I know that in the genesis block, Satoshi Nakamoto got 50 bitcoin's and there are certain rumors that the first transaction destinations, were are also to his other wallet's, but if he never spent any of the hardcoded bitcoin's and assuming every hardcoded bitcoin awarded/premined wasn't sent how did people outside of Satoshi nakamoto ring of influence manage to mine blocks(if there wasn't no transactions)?
Also does any premine/early mine involve "fake transactions" in order to award coins to miners? Im considering doing a small video for friends so i need to get facts right.

The block reward in the genesis block is unspendable; that block is hardcoded in and so the UTXO set (which starts being populated in block one) does not contain a UTXO from the genesis block. Bitcoin had no pre-mine.
From that point on, anyone could have run the code on their home computer to start mining bitcoin. There was no need for any transactions to be happening on the network, and in fact a majority of the earliest blocks contained no transactions other than the coinbase transaction.

So is there a minimum amount of blocks per 10 minutes or is it the number of blocks per 10 minutes constant(i heard 10 minutes was the interval)

@Imeguras Blocks are mined at random intervals, but the difficulty of mining a block is periodically adjusted by the network so that it takes on average 10 minutes to mine a block.
If you have other things you'd like to have answered that aren't related to your original question, I recommend to make those into new, separate questions.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How do I use python-bitcoin-utils to add an OP_RETURN message into a bitcoin transaction?
I have the following code that successfully created a bitcoin transaction with python-bitcoin-utils:
from bitcoinutils.utils import to_satoshis
from bitcoinutils.setup import setup
from bitcoinutils.transactions import Transaction, TxInput, TxOutput
from bitcoinutils.script import Script
from bitcoinutils.keys import PrivateKey as utilPrivKey
from bitcoinutils.constants import SIGHASH_ALL, SIGHASH_ANYONECANPAY

setup('testnet')

# private key for tb1qj6zz96g8xgrwpgmdlvmkrjlwzz54sf47086yc9
priv = utilPrivKey.from_wif('PRIVATE KEY FOR SENDING ADDRESS')

pub = priv.get_public_key().to_hex()
addrs_for_script = priv.get_public_key().get_address()

# private key for tb1qxgm8j0cq7tnftef3t563psl56gtmzxanm5c9uy
recPrivKey = utilPrivKey.from_wif('PRIVATE KEY FOR RECEIVING ADDRESS')

# This UTXO has 0.00009839 btc
txin1 = TxInput("102172a062da813c3aa8cc2fb3d523cf2db300e54cd680c2129c23c97db9dd8e", 0)

# This UTXO has 0.00026859 btc
txin2 = TxInput("b3fcae0b28b387475a123c056298aec0ba3759cd019f9d0975f5af0874f395ff", 1)

addr = recPrivKey.get_public_key().get_segwit_address()
addr_non_seg = recPrivKey.get_public_key().get_address()

# the script code required for signing for p2wpkh is the same as p2pkh
script_code = Script(['OP_DUP', 'OP_HASH160', addrs_for_script.to_hash160(),
                        'OP_EQUALVERIFY', 'OP_CHECKSIG'])

# remaining 0.00005 is tx fees
txout = TxOutput(to_satoshis(0.00031698), addr.to_script_pub_key())

# create transaction from inputs/outputs -- default locktime is used
tx = Transaction([txin1, txin2], [txout], has_segwit=True)

txsign1 = priv.sign_segwit_input(tx, 0, script_code, to_satoshis(0.00009839), SIGHASH_ALL | SIGHASH_ANYONECANPAY)
tx.witnesses = [ Script([txsign1, pub]) ]

txsign2 = priv.sign_segwit_input(tx, 1, script_code, to_satoshis(0.00026859), SIGHASH_ALL)
tx.witnesses.append( Script([txsign2, pub]) )

signed_tx = tx.serialize()
print("raw tx below this line")
print(signed_tx)
print("raw tx above this line")

How would I modify this code to also add an OP_RETURN value to the transaction?

I fixed the script. The below code was used to make this transaction https://blockstream.info/testnet/tx/a86f0187be2eca4590a456e2423a2f5286b269c3b777846092ceb2fc44270aad.
from bitcoinutils.utils import to_satoshis
from bitcoinutils.setup import setup
from bitcoinutils.transactions import Transaction, TxInput, TxOutput
from bitcoinutils.script import Script
from bitcoinutils.keys import PrivateKey as utilPrivKey
from bitcoinutils.constants import SIGHASH_ALL, SIGHASH_ANYONECANPAY

setup('testnet')

# private key for tb1qj6zz96g8xgrwpgmdlvmkrjlwzz54sf47086yc9
priv = utilPrivKey.from_wif('SENDING ADDRESS PRIVATE KEY')

pub = priv.get_public_key().to_hex()
addrs_for_script = priv.get_public_key().get_address()

# private key for tb1qxgm8j0cq7tnftef3t563psl56gtmzxanm5c9uy
recPrivKey = utilPrivKey.from_wif('RECEIVING ADDRESS PRIVATE KEY')

# This UTXO has 0.00032557 btc
txin1 = TxInput("947145005bac29a9966403e986b8a36cb225cba504bffbb3f6c1da7dba7197be", 1)

# This UTXO has 0.00006859 btc
txin2 = TxInput("aea283b193f2179cb6a5c2bcea4b9ad7158960696ebfb5c15d22a0411f25dafe", 0)

# the script code required for signing for p2wpkh is the same as p2pkh
script_code = Script(['OP_DUP', 'OP_HASH160', addrs_for_script.to_hash160(),
                        'OP_EQUALVERIFY', 'OP_CHECKSIG'])

addr = recPrivKey.get_public_key().get_segwit_address()
addr_non_seg = recPrivKey.get_public_key().get_address()

# This is random text designed to be hashed 8hwi8a8x3lja1ukbu ---> 69eb032167c25784108b9783c9d59ce23608d00238eb9e3c03c73cdd79f74528
msg1 = "69eb032167c25784108b9783c9d59ce23608d00238eb9e3c03c73cdd79f74528"

op_return_script = ["OP_RETURN"]
op_return_script.append(msg1)
op_return_script = Script(op_return_script)
print(op_return_script)
print(addr.to_script_pub_key())

txoutOpR = TxOutput(0, op_return_script)

# remaining 0.00005 is tx fees
txout = TxOutput(to_satoshis(0.00034416), addr.to_script_pub_key())

# create transaction from inputs/outputs -- default locktime is used
tx = Transaction([txin1, txin2], [txoutOpR, txout], has_segwit=True)

txsign1 = priv.sign_segwit_input(tx, 0, script_code, to_satoshis(0.00032557), SIGHASH_ALL | SIGHASH_ANYONECANPAY)
tx.witnesses = [ Script([txsign1, pub]) ]

txsign2 = priv.sign_segwit_input(tx, 1, script_code, to_satoshis(0.00006859), SIGHASH_ALL)
tx.witnesses.append( Script([txsign2, pub]) )

signed_tx = tx.serialize()
print("raw tx below this line")
print(signed_tx)
print("raw tx above this line")

This is the raw transaction hex:
02000000000102be9771ba7ddac1f6b3fbbf04a5cb25b26ca3b886e9036496a929ac5b004571940100000000fffffffffeda251f41a0225dc1b5bf6e69608915d79a4beabcc2a5b69c17f293b183a2ae0000000000ffffffff020000000000000000226a2069eb032167c25784108b9783c9d59ce23608d00238eb9e3c03c73cdd79f7452870860000000000001600140e723634c35aa61f54d7dd4159483ed649449537024730440220188a85f11816410f8ea51a933dbf9e38b318203602a42019837534789cd3915a02200351fa3e40ac446959d295f4efcbd2f76f0546b4be8c2066e5ffac9a3688e519812102cc30cb24a244910be0a343ee2cd7cee2e79558dbc8873404f6122d40a70cf71f0247304402205cf55018d39e3377ab843ec53be895934f59520e8268bbbdc18328a6d1d55c590220322de21da63e32fc2160db526979f7799d44c190c9810e0a20cd89e70cc61bb6012102cc30cb24a244910be0a343ee2cd7cee2e79558dbc8873404f6122d40a70cf71f00000000

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Valid public key check in transaction output
Let's say I build a P2MS output script using uncompressed public keys, and let's say too that these public keys are not valid, i.e. they do not represent a valid EC Point and they don't start with 0x04.
Will this transaction be broadcasted and included in a block or, among all the validity and standardness checks, a peer also controls that public keys are valid and rejects the transaction if they aren't ?
Thank you very much !

The standardness check will verify a multisig output for valid EC points. If these points do not pass this verification, the output script will be considered non-standard and not propagated by the implementations which I know of.
This can be verified by checking the implementation source.

On transaction verification: maybe this and this. The funding transaction might find its way into the system, cause a script in a (P2SH) multisig is the hash of a redeemscript. A verification of the redeemscript cannot be done, unless the spending tx will be sent.
If then incorrect keys are used in a script, then the checksig script will fail, and as such the whole transaction will be rejected.
I haven't made a test with a non-redeem script.
A non-P2SH (also called "raw") multisig would have  in the funding transaction this lock script:
2 <Public Key A> <Public Key B> <Public Key C> 3 CHECKMULTISIG

I haven't seen that isStandard() checks the details of the public keys. Anyhow, the script has to have proper length parameters around the "invalid" keys, so script validation thinks, its just an array of hexadecimal data that shall be pushed onto the stack. And then again, data inside must not use OpCodes, which might be misinterpreted. So it's tricky, but not impossible. Need to have a proper understanding of bitcoin script though :-)


text:
Maximum standard input script size
What is the maximum size for an input script so to be considered standard ? Looking at the Bitcoin Core source code it appears to be 10'000 byte, however reading here and there, it appears to be 1650 byte.
If that's the case, if the input script is redeeming a P2SH, is the size of the redeem script (520 byte) to be considered in the count, or not ?
Thank you very much !

By input script, if you mean scriptSig, the max standard scriptSig can be up to 1650 bytes.
See policy.cpp :
// Biggest 'standard' txin is a 15-of-15 P2SH multisig with compressed
// keys (remember the 520 byte limit on redeemScript size). That works
// out to a (15*(33+1))+3=513 byte redeemScript, 513+1+15*(73+1)+3=1627
// bytes of scriptSig, which we round off to 1650 bytes for some minor
// future-proofing. That's also enough to spend a 20-of-20
// CHECKMULTISIG scriptPubKey, though such a scriptPubKey is not
// considered standard.
if (txin.scriptSig.size() > 1650) {
    reason = "scriptsig-size";
    return false;
}

Thank you very much. So it appears that also the redeem script is included in the 1650 byte. So I effectively have, if the redeem script has the max size of 520 byte, just 1130 byte.

Sounds right to me. Also, there is segregated witness data which doesn't look like it counts towards the scriptSig limit.

Technically, a 520 byte size redeemScript has an overhead of 3 more bytes, the OP_PUSHDATA2 opcode that is one byte and another two bytes 0x0802 for the actual size value 520


text:
How public key is calculated from a base58 encoded private key?
I followed this post and created a private key for bitcoin wallet. The post says after finding the random 256 bit(32 bytes) integer the version number should be prepended and the checksum should be appended to it. This produces a 37 bytes hex.
Now I am trying to find the public key associated with this private key.I learnt that the PUBLIC key is calculated by applying ECDSA secp256k1 curve on the private key. But only 32 bytes is accepted as input by the ECDSA algorithm.
So what am I doing wrong?
Thanks in advance.

The private key is just the 32B entropy you generated. Between wallets this private key can be encoded in a format, which provides further information such as the network(main/test/alt) and whether a compressed or uncompressed public key was used. This encoding is called the Wallet Import Format.

version (1B) (0x80 mainnet/0xEF testnet)
privatekey (32B)
0x01 compressionmarker (1B)
checksum (4B)

https://teachbitcoin.io/wallets.html#/2
Checksum is of everything which precedes it. The data is finally encoded in base58.
To take this WIF data and produce the public key, the wallet just needs to extract the private key 32B) and compute the public key point

private_key * Generator_point = public_key(x,y)

The resulting x, y public key point is your public key. If the compression market is on, take the x-coordinate and prepend a 0x02 or 0x03 depending on whether it is even or odd.

compressed public key: 0x02/0x03 + x_coordinate(32B)
uncompressed public key: 0x04 + x_coordinate(32B) + y_coordinate(32B)

Thank you so much..it was very helpful!!. Im just beginning to learn about blockchain. Thanks again :)

Awesome! Happy it was helpful!


text:
Is it possible to run two nodes with the same public key?
Is it possible to run two nodes with the same public key on the Lightning network? The reason is reliability -- two different servers run different instances of node software with the same setup (incl keys) -- if one fails -- the other one continue serving. Possible?
thank you.

With reliability, I assume you mean protecting channel balances against server failure (Not being able to close the channel correctly following an error).
You can reproduce the wallet in both nodes, but that doesn't contribute to securing a payment channel's balance in the case of a failure. It simply just means both node instances have identical wallet keys.
In other words, the two node instances may be signing with the same wallet keys, but they cannot be operating as a single peer for a given channel.
Since it is the individual payment channel balance (unbroadcasted, signed commitment transactions) which is at risk, it would be necessary to secure the commitment transaction data of a single node instance to protect your balance in case of a server failure.

No, because channels are very stateful, and each node MUST have the up-to-date state to be able to sign and revoke commitments. Any node which has out-of-date state will potentially cause loss of all funds in channels.
If you do need to migrate your node due to server downtime, you should ensure that you have replicated the most recent state onto your new server before attempting to reconnect to the network. Even if you have an automated replication strategy in place, if a server goes down, you can not be sure whether the most recent state was replicated to your new server until you can recover the state of the node which went down.
Clients will only keep a record of the most up-to-date gossip messages from a given node id, so broadcasting a new IP address will replace the old view of the IP address for other participants. Each client will probably use the node's public key as the primary key for indexing the node in its database.


text:
BitcoinCore Ubuntu buiild error: ‘relative’ is not a member of ‘fs’
I'm trying to build Bitcoin core client on an LinuxMint. I'm having the following error(which have not encountered before, while making it on MSWindows or another Linux machine) in make process, right after
CXX      wallet/libbitcoin_wallet_a-walletutil.o

& I can't figure out why:
wallet/walletutil.cpp: In function ‘std::vector<boost::filesystem::path> ListWalletDir()’:
wallet/walletutil.cpp:60:32: error: ‘relative’ is not a member of ‘fs’
         paths.emplace_back(fs::relative(it->path(), wallet_dir));
                            ^
wallet/walletutil.cpp:71:36: error: ‘relative’ is not a member of ‘fs’
             paths.emplace_back(fs::relative(it->path(), wallet_dir));
Makefile:6894: recipe for target 'wallet/libbitcoin_wallet_a-walletutil.o' failed
make[2]: *** [wallet/libbitcoin_wallet_a-walletutil.o] Error 1
make[2]: Leaving directory '/home/behrad/cprj/bitcoin/src'
Makefile:10232: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/behrad/cprj/bitcoin/src'
Makefile:774: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1

My machine version is uname -a output:
Linux anonymous 4.10.0-38-generic #42~16.04.1-Ubuntu SMP Tue Oct 10 16:32:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

One thing comes to my mind is the initial advice on build-unix.md doc, which was warning against relative paths:
Always use absolute paths to configure and compile Bitcoin Core and the dependencies, for example, when specifying the path of the dependency:

    ../dist/configure --enable-cxx --disable-shared --with-pic --prefix=$BDB_PREFIX

Here BDB_PREFIX must be an absolute path - it is defined using $(pwd) which ensures the usage of the absolute path.

But as I stated, I've passed configuration & this occurred while it was about finishing. I can installing w/o wallet, but I need it in this installation. Thank you for taking time reading so far, Any help appreciated in advance.

Looks like your version of Boost Filesystem is too old. Read doc/build-unix.md in the repository for instructions on what packages to use. Did that help?

Thank you Mr Wuille (@PieterWuille) for following up! Yeah, I've closely followed the steps (As I do always w/ bitcoincore installation), But this is weired! Unfortunately I'm not sure enough if its a public issue so I can post it as an issue w/ steps for recreation to help you find out more about my env.


text:
Bits target, Increase/Decrease in ease of mine
I am currently studying Block Chain technology and at this moment studying the contents of blocks.
To my understanding; A block has to be mined every 10 Minutes.
If the Bit Target sees that Blocks are struggling to be mined every 10 minutes will it then decrease/increase the Bit Target?
If the Bit Target sees that Blocks are easily being mined in less than 10 minutes will it then decrease/increase the Bit Target
I know that the nonce is random I believe it starts at 0 and then is appended to the previous merkle root and then hashed and if its below the bit target then success and if its above then it increases the nonce +1 and repeats the process (Basically trial and error)
However the reason I found struggle is because I dont know if decreasing/increasing is making it more difficult/easier since I don't know the number ones its been hashed.
If somebody could answer my two questions and confirm the things I am saying are true/false really would appreciate it.

Decreasing the target makes finding the correct block hash harder because the range of possible values is smaller. There are less numbers for 1 to 10 than there are from 1 to 100. So by making the target smaller, it becomes less likely for the correct hash to be found and thus the difficulty increases.
Increasing the target makes finding the correct block hash easier because the range of possible values is greater. By making the target larger, it becomes more likely for the correct hash to be found and thus the difficulty decreases.
The relationship between difficulty and the target is inverse.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
which cryptocurrency using Slasher algorithm for Proof of Stake mining process?
Ethash algorithm for PoW mining process on my own cryptocurrency. I need to use Slasher algorithm instead of Ethash algorithm for Proof of Stake process.
My question is Which cryptocurrency using Slasher algorithm for Proof of Stake mining process?.
This answer is more helpful for my further process.

etherieum, use the slasher
you can see here:
https://blog.ethereum.org/2014/01/15/slasher-a-punitive-proof-of-stake-algorithm/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Is it possible to have a failed Bitcoin transaction on the block chain?
In Ethereum, a transaction may get out of gas but still being included in a block with a receipt status of failed. In Bitcion, will similar things happen?

No. Invalid transactions do not get included in blocks in Bitcoin.
There is also no receipt concept in Bitcoin. All nodes run full verification and check transactions against their own UTXO set when importing a block. Therefore any invalid transaction will also cause a block to be invalid as well.


text:
How do exchanges use cold bitcoin wallet while still able to automate transfers?
Some exchanges mentions they use a cold wallet to keep the private key. As far as I understand cold wallet is a wallet either in an offline machine or kept in hardware.
However, it appears that transfers can still be made online on these exchanges and the underlying system will automate the signing of transaction.
How do exchanges able to maintain a cold wallet while still able to automate the process? Is it by setting up local network with limited access or by other means?

The exchanges that use cold storage keep most of their coins offline, but do have a small percentage of funds that is necessary for normal business in online wallets.

They are using a master public key for generating addresses and storing master private key offline.
When you make a transaction, you are not calling to get the master private key directly and signing the transaction.
Someone either processes the transactions manually or following some security checkpoints before signing the transaction.
We can't really know what is going on exactly, but it's just an example.


text:
Sending Bitcoin to Bitcoin Cash address. Restore without private key
I need help.
I use blocktrail.com for accept payment in my site by bitcoin cash.
Steps: 1. Generate bitcoin cash address 2. Sending bitcoin cash to generated address
One of customer sent bitcoin (BTC) to bitcoin cash address, and the address is available. Look at https://www.blocktrail.com/BTC/address/35HSWxagkcWxsEgDh3EWE6A1n3THyM1mqC/transactions
Question. How can I get these bitcoins?
I haven't got private key from bitcoin cash wallet, but I have got restore-file "Bitcoin Wallet Recovery Data Sheet". I guess that file can be use to restore only bitcoin cash wallet.
Any ideas?
Thanks

Try tell him to import the bitcoIN cash wallet private key into a regular bitcoIN wallet. If the person does not have the private key nor it not working then the fund will permanently loose
If your using the provider then contact them and they should able to recover it for you


text:
mining of sh256 coins
I'm new in mining
I just bought one s9 miner,
Want to know that how can i mine any sh256 coin with s9 miner ?
I was searching about it and different pools,
but everywhere they said like this

Create account.

Register here, or login if you already have account
Create a worker that will be used by the miner to login
2. Download a miner.
Intel/ATI/AMD CGMiner: Download here
Intel/ATI/AMD BFGMiner Linux/Windows: Download here
Fabulous Panda Miner Mac OS X: Download here
Minerd CPU Miner Mac/Linux/Windows: Download here.
3. Configure your miner.
Settings for Stratum:
STRATUM:    stratum+tcp://mine1.mpool.me
PORT (Low hash rates):  4012 (varDiff: 16 - 4096, initial diff: 128)
PORT (High hash rates): 4112 (varDiff: 2048 - 65535, initial diff: 2048)
Username:   Weblogin.Worker
Password:   Worker Password
If your using Linux, Then type the following into the console:

please help me how to configure miner
Thanks in advance

I have previously reviewed the steps for configuring the Antminer S9.
You need to register an account with a pool and add a worker.
The stratum information is what you need to put into the Antminer S9 configuration.
You need to know the IP address of your S9 on the network. If you do not know it, use Advanced IP Scanner (or similar, I do not know if you are running Windows?) and find it.
Open http://x.x.x.x/ (IP of your S9) in your web browser and login. In the Miner Configuration tab, enter your stratum information there.
URL: stratum+tcp://mine1.mpool.me:{PORT} - you have choices for PORT 4012/4112
Worker: {your worker name} - you should have configured this in the pool where it may be called Worker Username.
Password: {your worker password}

Save & Apply.
Your miner will  not start until it registers.
You may enjoy this video (the sound is quiet!):
How to setup Antminer S9, R4 GUI- Set mining pools, ECT
Enjoy mining!


text:
When is OP_RETURN cheaper than OP_FALSE OP_IF?
Since January 2023 there have been many transactions that embed arbitrary data in the leaf script of a P2TR input using the opcodes OP_FALSE OP_IF (also called "inscription envelope"). This is usually cheaper than using OP_RETURN because the data benefits from the witness discount, but also comes with the overhead of having to first create the P2TR output and then spending it in a subsequent transaction. So, when is it cheaper to use OP_RETURN?

TL;DR: OP_RETURN is cheaper for data smaller than 143 bytes.

The overhead of an OP_RETURN output is at least 11 bytes:

amount (8 bytes)
compact size of script (at least 1 byte)
script:

OP_RETURN opcode (1 byte)
push opcode (at least 1 byte)

The overhead of an inscription is at least 118.75 vB (84 base bytes and 139 witness bytes):

P2TR output (43 bytes)
P2TR input:

outpoint TXID (32 bytes)
outpoint index (4 bytes)
sequence (4 bytes)
compact size of input script (1 byte, script is empty)
witness data:

compact size of witness stack (1 WU)
compact size of signature (1 WU)
signature (64 WU)
compact size of script (at least 1 WU)
script:

public key push opcode (1 WU)
public key (32 WU)
OP_CHECKSIG opcode (1 WU)
OP_FALSE opcode (1 WU)
OP_IF opcode (1 WU)
push opcodes (at least 1 WU)
OP_ENDIF opcode (1 WU)

compact size of control block (1 WU)
control block (33 WU)

The witness discount cancels out the larger overhead at 143 bytes of data, which takes 155 vB using an OP_RETURN and 154.75 vB using an inscription. For anything smaller than that OP_RETURN is cheaper.
Notes:

The above assumes a best-case scenario in which the P2TR input is included in a transaction that serves some other purpose and would be made anyway. This often isn't the case, in which case the overhead is at least 129.25 vB and OP_RETURN is cheaper for up to at least 158 bytes of data.
It's necessary to split the data into chunks of 520 bytes in an inscription but not in an OP_RETURN output. This only affects data large enough that it doesn't change the answer.
We're ignoring the 80-byte standardness limit for OP_RETURN here.

You can drop the 1 byte overhead for the size encoding of the data that follows an OP_RETURN. Any bytes following an OP_RETURN do not need to be a valid (or even parseable) Bitcoin script. Thus, no size encoding is needed.

@user2084795 That would be a non-standard output script, but otherwise you're absolutely right. You might as well drop the OP_RETURN too and save another byte.


text:
Bitcoin Core uploads much more than it downloads
Since Bitcoin is a peer-to-peer network of mostly identical clients, I would expect each node to upload about as much as it downloads. So I was surprised to see that my Bitcoin Core node has (since last startup) downloaded 1 GB of data but sent 47 GB.
I am running a full node with pruning disabled, so my node might participate in IBDs. However, I don't think that should make that much difference unless there are a lot of new nodes joining the network compared to the current number. I also see that my node seems to have around 70 incoming connections and 10 outgoing connections.
Can someone explain this discrepancy?

This is interesting. Never looked at that. Maybe it's there really are that many people running non listening nodes?

While nodes are set to be listening by default, the vast majority does not permit inbound connections either because listening has been disabled or their network setup doesn't make the necessary port accessible. There seem to be in the range of 8-10k listening nodes, while estimates for non-listening nodes range in 60-400k depending on the source.
Next to full nodes there are numerous thin clients that only consume but not propagate network gossip. As a listening node has a lot more connections to hear about new inventory quickly, it will be more likely to be the first to announce new transactions and blocks to its peers in turn. Additionally, a listening node will serve nodes undergoing initial synchronization and thin clients.
To give one data point:
$ bitcoin-cli getnettotals
{
  "totalbytesrecv":   15176470806,
  "totalbytessent": 4708177804982,
  "timemillis": 1624363225070,
  "uploadtarget": {
    "timeframe": 86400,
    "target":              262144000000,
    "target_reached": false,
    "serve_historical_blocks": true,
    "bytes_left_in_cycle": 218592671495,
    "time_left_in_cycle": 46909
  }
}

In the last 11 hours, my node has uploaded 40.5 GiB. Altogether since it was last restarted, my node has received 14 GiB while uploading 4384 GiB, making its upload volume about 313× its download volume.

This is not surprising, and I believe it is actually normal behavior. It's an artifact of the flood method that Bitcoin uses to relay blocks and transactions.
Consider the following: one of your peers announces an unknown transaction of size 1000 bytes to your node, so your node requests and downloads. Now your node announces the same transaction to every other peer that it is connected to. For the sake of simplicity, let's assume that every other peer has not seen this transaction before, so there are 79 transaction announcements. Then each of those 79 nodes request that transaction, and your node sends the 1000 byte transaction to all 79 of them.
In this scenario, your node only downloaded at least 1000 bytes, but it had to upload at least 79000 bytes (this calculation does not include some additional network overhead or the announcements). In this way, you can have your upload be significantly larger than the download.

This is true, but for transaction relay I don't expect such a large discrepancy. I believe that the largest difference between in and out is due to the fact that OP is not in IBD, and their peers are. IBD is done almost exclusively through outbound connections.

But this obviously can't be true for every node, since every byte sent by one node is a byte received by another node. So why is his node special?

OP has incoming connections. The majority of nodes don't.


text:
How can taproot transactions manage to store so much data?
I don't quite understand how ordinals are able to store so much data on chain. I know about the ordinals envelope.
OP_FALSE
OP_IF
   OP_PUSH "ord"
   OP_PUSH 1
   OP_PUSH "text/plain;charset=utf-8"
   OP_PUSH 0
   OP_PUSH "Hello, world!"
 OP_ENDIF

According to the ordinals handbook, in this example, the text "Hello, world!" is stored on chain. But I thought that the taproot upgrade simply enables people to tweak the public key to embed a merkle root on chain by tweaking the public key to create a modified address. Even then, the merkle root isn't actually stored on chain. You just have to know it to spend coins from that address right?
I know my understanding is wrong because if all you needed to know to spend coins from the script path was the merkle root you wouldn't need to know the hash of any of the leaves.
I know that bitcoin transactions have some code for scripting in the actual hexadecimal transaction. For example a simple locking script in a bitcoin transaction output looks like this "scriptPubKey": "OP_DUP OP_HASH160 HASH_OF_PUBKEY_IN_HEX OP_EQUALVERIFY OP_CHECKSIG" so to spend that transaction whoever has the private key of the address in the UTXO provides an unlocking script consisting of the signature and public key. But this is just a little bit of data that bitcoin transactions store, how does taproot allow for such larger amounts of data to be stored?

The only thing Taproot changed with respect to the "inscription envelope" is removing the script size limit, which was previously 10,000 bytes for P2WSH and 520 bytes for P2SH. In P2TR, the scripts are only implicitly bounded by the block weight limit.
However, there's nothing stopping you from storing data smaller than 10,000 bytes in P2WSH (it's even slightly cheaper), or splitting larger data across multiple inputs in chunks of 10,000 (which would be slightly more expensive).


text:
How is tapscript different than P2PKH using opcodes from a developer perspective?
I'm going to use ordinals as an example because they illustrate my question the most effectively. My understanding is that ordinals are created by writing opcodes into the transaction just like you would with a non-taproot transaction.  If I did a typical P2PKH transaction I would put
"scriptPubKey": "OP_DUP OP_HASH160 HASH_OF_PUBKEY_IN_HEX OP_EQUALVERIFY OP_CHECKSIG"

as my locking script, but If I wanted to do an ordinal I believe I would just make my locking script
"scriptPubKey": "OP_DUP OP_HASH160 HASH_OF_PUBKEY_IN_HEX OP_EQUALVERIFY OP_CHECKSIG OP_FALSE OP_IF OP_PUSH "ord" OP_PUSH 1 OP_PUSH "text/plain;charset=utf-8" OP_PUSH 0 OP_PUSH "Hello, world!" OP_ENDIF"

It sounds like this means that I could do an ordinal in a non-taproot address bitcoin transaction. So I could mint an ordinal with a P2PKH address.
So what does tapscript actually do that P2PKH doesn't?

It sounds like this means that I could do an ordinal in a non-taproot address bitcoin transaction. So I could mint an ordinal with a P2PKH address.

Historically output types with address standards came in two flavors: pubkeyhash-based (P2PKH, P2SH-P2WPKH, P2WPKH) and scripthash-based (P2SH, P2SH-P2WSH, P2WSH). P2TR outputs combine both these flavors by providing both a keypath and a scriptpath spending option. While it seems to me that a modification of the P2PKH output script to incorporate an envelope should be consensus valid, there are two things to note here:

P2PKH addresses pertain exactly to the output script¹ template OP_DUP OP_HASH160 <pubkeyhash> OP_EQUALVERIFY OP_CHECKSIG. Modifying the P2PKH output script in the proposed manner results in a bare script output, because it no longer matches the P2PKH template.
The only bare scripts permitted in standard transactions are Pay to Pubkey (P2PK), Pay to Pubkey Hash (P2PKH), and bare multisig with up to three pubkeys.

So at first glance, I would expect your experiment to result in a consensus-valid non-standard transaction. It would also be more expensive than using the P2TR-based envelope.

¹ in Bitcoin Core’s codebase an output script is also knows as  scriptPubKey


text:
Deriving payable address from mnemonic using BIP44
I'm trying to go from seed phrase/mnemonic to payable address using BIP 44.  Here is a simplified version of the code I'm using.  However, when I compare with https://iancoleman.io/bip39/ the private, public, and address don't match up.
var mnemonic = require('bitcore-mnemonic')
var bitcore = require('bitcore-lib')

var code = new mnemonic("width bicycle axis tell burst outdoor tray episode where they forest meadow enhance twin focus");

var xpriv = code.toHDPrivateKey(bitcore.Networks.livenet);
var derived = xpriv.derive("m/44'/0'/0'/0/0")

var privateKey = derived.privateKey
var publicKey = derived.publicKey
var address = new bitcore.Address(publicKey)

console.log(`address: ${address}`)
console.log(`privateKey: ${privateKey}`)
console.log(`publicKey: ${publicKey}`)

This code outputs:
address: 14TomZRSGPtoPGy9HZA8WfX2veMbDXyAAJ
privateKey: 34fb6aee170bb41fe47f97ab601fea589620cf182ecf031688c25e5491c2c06f
publicKey: 027869eb83bbb44621c868fbd2a31daed4a0f8dbaa73d3fd79a597cdb1afcc9d1d

However when I go to https://iancoleman.io/bip39/
it give as the first address with this seed phrase to be:
path: m/44'/0'/0'/0/0
address: 1BogrbApan5Q9xEFEzwPioRu3ogYkzAQZU
public: 0236f0bf97fa766301e1137bef8377e6f6063b8a576bda57220c7ab6cf531cc992
private: L46hcht5ue2tsSKmPEV2v41CrGU8DEJAP1w7XYPbvJ72v5NBPsPF

When you say simplified code it's hard to tell where you are going wrong.
Here's an example of how you can derive a bip44 key pair from a mnemonic using bitcoinjs-lib v3.3.2
let mnemonicInput = 'width bicycle axis tell burst outdoor tray episode where they forest meadow enhance twin focus';
let seedHex = bip39.mnemonicToSeedSync(mnemonicInput).toString('hex');

//"14a3235efb14b096e8cc3082b89e0b629ec5c7b2c6621343b2657cb61853b0830623e97b8aeac416d3377b4da90a4838d9aea4d83e0117fd833049305af46f10"

let root = bitcoin.HDNode.fromSeedHex(seedHex);
let deriv = 44;
let account = 0;
let change = 0;
let index = 0;

let acct = root.derivePath("m/"+deriv+"'/0'/"+account+"'");
let xpub = acct.neutered().toBase58();
let pair = acct.derivePath(change+"/"+index).keyPair;

let wifkey = pair.toWIF();
let address = pair.getAddress();

//"address":"1BogrbApan5Q9xEFEzwPioRu3ogYkzAQZU"
//"wifkey":"L46hcht5ue2tsSKmPEV2v41CrGU8DEJAP1w7XYPbvJ72v5NBPsPF"


text:
How to calculate the checksum of an extended public key?
Very difficult to find definitive info on this.
From what I can find on google and looking at BIP32 it seems like the process to calculate a checksum for an extended key is to decode the base58 string to data, then perform a double sha256 on the data and the first 4 bytes from the double hash are the checksum.
I am trying to take in a non xpub (vpubs, upub ypub etc) and convert it to an xpub/tpub programmatically. Currently I have it working but the last 6 digits of the xpub are always incorrect, I am assuming this is the checksum?
In order to correctly calculate the checksum do I remove the prefix first, then remove the existing checksum then do the double sha256 on the stripped down extended key or do I keep the new xpub prefix and then perform the double sha256? Or am I completely wrong in my assumptions here?
Thanks in advanced!

it seems like the process to calculate a checksum for an extended key is to decode the base58 string to data,

If raw_xpub = version + depth + fingerprint + child + chain + DATA
than to make the checksum you need to do doublehash of xpriv/xpub (sha256), in python:
hashed_xpub = hashlib.sha256(raw_xpub).digest()
hashed_xpub = hashlib.sha256(hashed_xpub).digest()

and then take the checksum as first 4 characters (0 to 3) and add it at the end of the raw xpub:
raw_xpub += hashed_xpub[:4]

Finally, run it through Base58 to encode it. I think you've might performed base58 operation too early based on your question. It's the last step in the process.


text:
why is getdescriptorinfo returning a "function needed" error with a sortedmulti descriptor?
When trying:
bitcoin-cli getdescriptorinfo "wsh(sortedmulti(2,tpubDFck48zH8ZzteKKZCR9WMVBJwHZayhQmtCDyDRzHsU7VMTwEXohD23h1CBiizKtFtWck2fVegNDRM193nGx3VLs1RTSGNc1iePKpc1JvsS6/*,tpubDEfXroYtWB6rQ1vNuCUAErUJtxSQ6bjxtz63rWWRaByADcudNFtV6HZz5PM7yeXQLG4GWmrdcRkMEGgdXWWu5TyuRpMXURCiwJoGQYH743f/*,tpubDFSz52YB4yCdbgKeLB5BcJfW1f1AzmLcdzB3nGz6Yft2MVHy2rj4uXBFT8d3maYS4rCxjJdBfExJAM9JCjWmYzkJyaLP4Rk8nALDUSfWbT8/*))"

I get:
error code: -5
error message:
A function is needed within P2WSH

What am I doing wrong?

This call works fine for me. What version of the code are you using? sortedmulti is not in 0.19, only in master (and will be in 0.20).

ahh ok, will update


text:
Why can't a user import keys to the keypool for a blank wallet?
I am creating a blank wallet with the intention of adding keys to the keypool but get an error keys can only be imported to the keypool when private keys are disabled
Why? It would be beneficial for users to be able to import their own private keys and get full functionality of the wallet with those keys. Is there any PR for this?

When you create the wallet that you are importing your keys into, presumably with something like bitcoin-cli -rpcwallet='mywallet' importmulti, it needs to be created in watchonly mode, meaning with private keys disabled. To do that, create the wallet with bitcoin-cli createwallet 'mywallet' true, where true is the argument for disable_private_keys, creating a wallet for watch-only (public) keys.

right, but obviously it would be useful to import private keys with an xprv, it seems there is no way to add private keys to the keypool other then importing individual keys. why can't we import an xprv just like we can import an xpub?

Ah, I see. Looks like you can currently just import private keys in WIF. According to Pieter Wuille's comment in https://bitcoin.stackexchange.com/questions/87610/how-do-you-import-an-xprv-using-descriptors-and-importmulti it might in 0.19.0 , but I have not tested it yet.

yea this is also helpful, testing now https://bitcoin.stackexchange.com/questions/88463/when-i-use-importmulti-with-an-xprv-it-never-imports-the-private-keys-what-am-i/88466?noredirect=1#comment105060_88466


text:
Will bitcoind import private keys for multi-sig descriptor with watch-only flag set to true?
I am importing a "warm" multi-sig quorum into bitcoind using importmulti like so:
{"jsonrpc":"1.0","id":"curltest","method":"importmulti","params":[[{ "desc": "sh(multi(2,[eb2d2f9e/44'/1'/0']tprv8g5heqz5AnCKdYQHsHyYvyDvWFdp5RN8Z5uhvb1FrzgPmeuYseQNsknt1KFf1b1Y9dtNaNMbxkaNdkP7FLbhP9a4UosbV5rWLAMFPhUcMVT/0/*,[e19b3701/44'/1'/0']tpubDCkUBfVNYjzdX37ARqSFWNAA5WvQRGy3Y5PA13ve1dSVtBLr7tEf6c7C2sYtnRazFfsfAdCfuv9JG62BnUVvofvvvs9iyDreQhTRJJ6Mp8u/0/*,[65b2e5d3/44'/1'/0']tpubDCFM6STEuVkkh6qCCooJw4TXHRzyq6R6tBeigSQSgf4xY6ZwAeoSQH2ZTTfcsZTrJ7wJF2bixpn38XcKj3KihxH5LzMraCLwsbt3bnkAHrU/0/*))#2pztwc0s", "timestamp": "now", "range": [2500,5000], "watchonly": false, "label": "StandUp", "keypool": false, "internal": false }], {"rescan": false}]}

And occasionally getting the following response (sometimes it works without returning an error and sometimes not):
response = {
    error = "<null>";
    id = curltest;
    result =     (
                {
            error =             {
                code = "-4";
                message = "Error adding key to wallet";
            };
            success = 0;
            warnings =             (
                "Some private keys are missing, outputs will be considered watchonly. If this is intentional, specify the watchonly flag."
            );
        }
    );
}

I do not want to specify the watch-only flag to true as I am importing private keys into the node and want the node to be able to sign. It is very odd that this seems to happen randomly. It feels like a bug.
If I set watch-only to true will it still import the descriptors private keys?
Just to prove my point here is the exact same command attempted a second time which returns success as true:
{"jsonrpc":"1.0","id":"curltest","method":"importmulti","params":[[{ "desc": "sh(multi(2,[eb2d2f9e/44'/1'/0']tprv8g5heqz5AnCKdYQHsHyYvyDvWFdp5RN8Z5uhvb1FrzgPmeuYseQNsknt1KFf1b1Y9dtNaNMbxkaNdkP7FLbhP9a4UosbV5rWLAMFPhUcMVT/0/*,[e19b3701/44'/1'/0']tpubDCkUBfVNYjzdX37ARqSFWNAA5WvQRGy3Y5PA13ve1dSVtBLr7tEf6c7C2sYtnRazFfsfAdCfuv9JG62BnUVvofvvvs9iyDreQhTRJJ6Mp8u/0/*,[65b2e5d3/44'/1'/0']tpubDCFM6STEuVkkh6qCCooJw4TXHRzyq6R6tBeigSQSgf4xY6ZwAeoSQH2ZTTfcsZTrJ7wJF2bixpn38XcKj3KihxH5LzMraCLwsbt3bnkAHrU/0/*))#2pztwc0s", "timestamp": "now", "range": [2500,5000], "watchonly": false, "label": "StandUp", "keypool": false, "internal": false }], {"rescan": false}]}

response = {
    error = "<null>";
    id = curltest;
    result =     (
                {
            success = 1;
            warnings =             (
                "Some private keys are missing, outputs will be considered watchonly. If this is intentional, specify the watchonly flag."
            );
        }
    );
}

I have tested setting watch-only to true and it gets rid of the errors and the node can still sign.


text:
Why does estimatesmartfee and walletcreatefundedpsbt with conf_target show a wide disparity?
Seems that when my testnet node boots up estimatesmartfee works just fine generally:
{"jsonrpc":"1.0","id":"curltest","method":"estimatesmartfee","params":[2]}
result =     {
        blocks = 2;
        feerate = "0.00147381";
   }

However when I go to build a psbt the fee as specified by conf_target seems to be way off:
{"jsonrpc":"1.0","id":"curltest","method":"walletcreatefundedpsbt","params":[[{"txid":"019510d1084c15dc216e1a3e19e7910f3680c124953fb19c59e3c8892f51ca11","vout": 0,"sequence": 1}, {"txid":"019510d1084c15dc216e1a3e19e7910f3680c124953fb19c59e3c8892f51ca11","vout": 1,"sequence": 1}, {"txid":"bac790009314a66dda3f478df1401ed039e9519828d7f969c43d2c025e6ae328","vout": 1,"sequence": 1}, {"txid":"77bed34db35b4ac5ad056892deb5d11baa8f00fbb5727c9d2b5a1d9c8da2a57c","vout": 1,"sequence": 1}, {"txid":"a6a2af2adfb6782e4d8f3c8a6a5e6d84286aa6d971b474babd88806781225785","vout": 0,"sequence": 1}, {"txid":"488e99b9b2564383ed04df539a21394cafd6a2208e820b38847d864dd64fbbb2","vout": 1,"sequence": 1}, {"txid":"32665fd0077d0014b6fa4d33840dda3cdf561e94d8ca48e3c8b62e26629139b8","vout": 1,"sequence": 1}], {"tb1qltge5c988eaqy9960yvepxmlpkplnu3zly8j5z":0.92358192}, 0, {"includeWatching": true, "replaceable": true, "conf_target": 2, "subtractFeeFromOutputs": [0], "changeAddress": "tb1qltge5c988eaqy9960yvepxmlpkplnu3zly8j5z"}, true]}

I then sign and decoderawtransaction:
result =     {
        hash = 09aae88cce31c3a8145b13d2d3bc6e0661a8520faacc1c58ab82f00c841abd9e;
        locktime = 0;
        size = 1079;
        txid = 04f270bdd9dfece9a13cc3baf3c558db5395fab9eb2666847553c7dca78d681b;
        version = 2;
        vin =         (
                        {
                scriptSig =                 {
                    asm = "";
                    hex = "";
                };
                sequence = 1;
                txid = 019510d1084c15dc216e1a3e19e7910f3680c124953fb19c59e3c8892f51ca11;
                txinwitness =                 (
                    304402203104150c08f8692534190d57b61328ef83a52a0a3c529f63bb6d3028de5a5fbe02206b9abbbfedf350c53542865207a9c3def62cc386cc21717d90e7248308957abb01,
                    020e1ae766f28bef25b5d13cae1f1dd887cde3103d8a9eb6c9452e093e05001cca
                );
                vout = 0;
            },
                        {
                scriptSig =                 {
                    asm = "";
                    hex = "";
                };
                sequence = 1;
                txid = 019510d1084c15dc216e1a3e19e7910f3680c124953fb19c59e3c8892f51ca11;
                txinwitness =                 (
                    30440220234715c55ed55f4b4bb1d067210293e5cca7e5072fbdf362f3063dc7b8b8e8d0022053d22d2f65e9eeba00d705e2d4abf6d737d98f535294d8f7fb91c1beb699872201,
                    0335820a3d6416230ac2e793629d1a57b276b9bc5da0f0166bec1bdba9a5167652
                );
                vout = 1;
            },
                        {
                scriptSig =                 {
                    asm = "";
                    hex = "";
                };
                sequence = 1;
                txid = bac790009314a66dda3f478df1401ed039e9519828d7f969c43d2c025e6ae328;
                txinwitness =                 (
                    3044022036730e009f7b262ac026f36caed527bb897ade66e446d8d3c86b99f05bd7923b02203cbb170419145a00b1f93524ea05d68048fff826e2af4803f5053b4b6bd5c53d01,
                    02de2954b646d7c77cdc4444c53a564e41b28260960f1ea5282792f1a7193478cc
                );
                vout = 1;
            },
                        {
                scriptSig =                 {
                    asm = "";
                    hex = "";
                };
                sequence = 1;
                txid = 77bed34db35b4ac5ad056892deb5d11baa8f00fbb5727c9d2b5a1d9c8da2a57c;
                txinwitness =                 (
                    3044022010e1342cd515a5923768be66fdff1196d29cc81c601b1fb5aa0c27061935c7be022055f2527602d25cb64133118bab362af5666cd8f640e4929bd95c581a361154f201,
                    02fc37d19c8eaa953c00d4da134e17d9b92dacb32508afbc76c53ebf511c1aea36
                );
                vout = 1;
            },
                        {
                scriptSig =                 {
                    asm = "";
                    hex = "";
                };
                sequence = 1;
                txid = a6a2af2adfb6782e4d8f3c8a6a5e6d84286aa6d971b474babd88806781225785;
                txinwitness =                 (
                    304402207f8e055a44936178dffed0547ec58eb1b7e13308f265ec1f7fac96872f23bcb4022037d4095f7f9398fec3f9ae54e407a84d8b8b938a34df46fefac777be7891e89301,
                    02952f6dd04b2894a8aa633091f784845ebcb2749d634d161f56c3cb2d2366c45f
                );
                vout = 0;
            },
                        {
                scriptSig =                 {
                    asm = "";
                    hex = "";
                };
                sequence = 1;
                txid = 488e99b9b2564383ed04df539a21394cafd6a2208e820b38847d864dd64fbbb2;
                txinwitness =                 (
                    304402203fe34532c251e59912a83f01a7b709b43ec16cc42d926199880da0525efe06ca0220339af59da13b13a1bd420948956845500acfdc8afbc7c5ee335351acc9325ec501,
                    0254bb6f7e18fd63a5e804bc3d266edbb67af82af0e96395af680a0d2b4a60b772
                );
                vout = 1;
            },
                        {
                scriptSig =                 {
                    asm = "";
                    hex = "";
                };
                sequence = 1;
                txid = 32665fd0077d0014b6fa4d33840dda3cdf561e94d8ca48e3c8b62e26629139b8;
                txinwitness =                 (
                    304402206f5a22fb4125f423d2d6a84662122464cfd2f75c1eeb68a16a5d5d574191ee160220407fa17f60e1843b94b33ee5156db873b270704b83f8b041af03ffb177abcbee01,
                    02cdc5c26ef620915d31e7ed467eb79db58cf18544370bc2c488084d9e15415336
                );
                vout = 1;
            }
        );
        vout =         (
                        {
                n = 0;
                scriptPubKey =                 {
                    addresses =                     (
                        tb1qltge5c988eaqy9960yvepxmlpkplnu3zly8j5z
                    );
                    asm = "0 fad19a60a73e7a0214ba7919909b7f0d83f9f222";
                    hex = 0014fad19a60a73e7a0214ba7919909b7f0d83f9f222;
                    reqSigs = 1;
                    type = "witness_v0_keyhash";
                };
                value = "0.92357674";
            }
        );
        vsize = 516;
        weight = 2063;
    }

I then cross check the conf_target with a fresh estimatesmartfee command to deduce what the s/b fee rate ought to be, comparing it to what it actually is and get a huge disparity:
actual s/b: 1.0
target s/b: 147.381
Why would estimatesmartfee work but conf_target arg would not?
I would have assumed that you would only get an issue like this if estimatesmartfee did not have enough data to function as can happen


text:
What is the proper way to update C-Lightning?
I downloaded the latest github .zip for version 0.6 and unpacked it but when I check the version I am running it returns:
v0.5.2-2016-11-21-1644-g1616fcb

Any idea what I did incorrectly? What's the proper way to update C-Lighting from the terminal in Ubuntu?

When you run it, what command are you using? Did you build and install 0.6 in the same location your path points to?

You can see which binary executes by using which lightningd. Like @MeshCollider mentions, this is likely due to the old binary being executed instread of the new one

I am using this command to run it: lightningd/lightningd --network=bitcoin --log-level=debug --ipaddr=$my_ip --alias=someNode. Sorry I am very new to this. All I did was download the latest .zip file from the github and unpacked it in my existing lightning folder that has the previous binary. What should I do differently? TIA

Is there a lightning in your directory? You might have nested the installations. The simplest way to solve it is to move/remove the origin directory and unzip the release in a new directory.

I just deleted the original folder and installed from scratch.


text:
Why does walletprocesspsbt finalize the psbt but not return the transaction hex?
I am currently using walletprocesspsbt to optionally finalize a psbt and fill in bip32derivs if not signed already. The psbt is fully signed and finalized but I have to make an additional rpc command finalizepsbt to get the hex, is there a reason walletprocesspsbt doesn't return the final tx hex?

My understanding of the documentation is that it should return the hex by default if the transaction is final. Could you clarify that it indeed is complete before you call finalizepsbt but the result of walletprocesspsbt does not contain the hex?

It's walletprocesspsbt which is not returning the hex. I am still on v0.25.0 if that is relevant.

I realize my previous answer is wrong.
BIP174 defines a number of distinct rules, including:

Updater (fill in information about a transaction)
Signer (produce signatures)
Finalizer (construct scriptSig/witnesses using all information/signatures)
Extractor (convert PSBT to a raw transaction that can be submitted to the network)

The walletprocesspsbt RPC implements the Updater, Signer, and optionally Finalizer roles. The finalizepsbt RPC implements the Finalizer, and optionally Extractor roles.
Nothing really prevents walletprocesspsbt from also performing the Extractor role if possible, but that just has not been implemented.

Ah that makes sense, it finalizes but does not extract.


text:
When I use importmulti with an xprv it never imports the private keys. What am I doing wrong?
When I call:
bitcoin-cli getdescriptorinfo "pkh(tprv8iDvhA6eaMDbRuH3D2DTNuH6N6LvMTVrcv7z6sVxyyYvThSaCSa3i3qFVqUgVL4NbVkACxjx3KPWLoP5jvDU3i16hjTnzEFnZuqsnTJ5F4n/*)"

it returns:
{
  "descriptor": "pkh(tpubDEuxqa8tiiuGKNJq6ft3nJwCw7rrWngmCDimPPYGQFMKJBhLpqPdtYT7fzuDbk1AppxRdYjzeUrLaVNQRP7EYghzLDBMguY2LHHHeHMtzhm/*)#0wkkha79",
  "isrange": true,
  "issolvable": true,
  "hasprivatekeys": true
}

I then call:
bitcoin-cli importmulti '[{ "desc": "pkh(tpubDEuxqa8tiiuGKNJq6ft3nJwCw7rrWngmCDimPPYGQFMKJBhLpqPdtYT7fzuDbk1AppxRdYjzeUrLaVNQRP7EYghzLDBMguY2LHHHeHMtzhm/*)#0wkkha79", "timestamp": "now", "range": 100, "watchonly": false, "label": "Hot Storage", "keypool": false, "rescan": false }]'

It returns:
{
    "success": true,
    "warnings": [
      "Some private keys are missing, outputs will be considered watchonly. If this is intentional, specify the watchonly flag."
    ]
  }

I am importing an xprv because I want to import the private keys, but they always get imported as watch-only.
Why is this? Is it intentional?

You aren't doing anything wrong. This behavior is expected for Bitcoin Core 0.18. The Pull Request allowing private keys to be derived from descriptors and imported with importmulti was merged recently. This functionality will be included in Bitcoin Core 0.19.

I am testing this with 0.19.0 rc3 but its still behaving the same way, forcing the wallet to be cold. Any idea why?

Did you import the descriptor with the tprv of the tpub? getdescriptorinfo returns the descriptor with only pubkeys. To get the checksum for the private key descriptor, check the checksum field and add it to your descriptor.


text:
channel count differs on my site and 1ML
I wonder why:
lightning-cli getinfo;listfunds
{
  "id": "035aef5661e1a6e370db60dc0455796800afd5b51fbc12a0a8b34836b15f5d7ef6",
  "alias": "TWronald✅",
  "color": "15c315",
  "num_peers": 41,
  "num_pending_channels": 3,
  "num_active_channels": 38,
  "num_inactive_channels": 0,
  "address": [
    {
      "type": "ipv4",
      "address": "220.133.223.206",
      "port": 9735
    }
  ],
  "binding": [
    {
      "type": "ipv4",
      "address": "0.0.0.0",
      "port": 9735
    }
  ],
  "version": "v0.6.3rc1-118-g0da4054",
  "blockheight": 563691,
  "network": "bitcoin",
  "msatoshi_fees_collected": 82538
}

157278 satoshi available to fund channels
7318449 satoshi owned in channels
19485999 satoshi total channel capacity
---------------
Total funds: 7475727 satoshi
Percentage of total channel capacity owned: 37.5575
---------------
Add command line parameter [B] for BTC, [m] for mBTC, [y] for microBTC, [b] for bits and nothting or anything else for satoshi
by Rene Pickhardt.
Consider a tip: via curl -i -H "Accept: application/json" -d '{"amount":ENTER_AN_INTEGER_AMOUNT_OF_SATOSHIS_INSTEAD_OF_THIS}' http://ln.rene-pickhardt.de/invoice

On 1ML.com:
Capacity
0.19280979 BTC (0.027%)
$751.22
Channel Count
37 (0.132%)
Connected Node Count
37 (1.120%)

Rene, I tried to send you some thanks, but it should be https and even then it didn't work.

1ml says that the information about your mode was updated a month ago, no wonder it is different. The question is why does not 1ml reindexes your node more often. I would just ask them

Since 1ml is not open source I can not give the exact reason. But some things are known that add to the discussion. The information about your node that you get from getinfo or listfunds are accurate and up to date.
New channels have to be announced over the gossip protocol so it will take some time until 1ml is aware of your most recent channels. This is one certain reason for the difference.
Another reason is that 1ml does not see your private channels since they are not announced over the gossip protocol. This could be the reason if the difference exists for a longer time.
Also it seems to me from my user experience from using 1ml that 1ml does not update its info in real time but does some batch processing (this part is speculation as their software is closed source) anyway this could also slow down the process of updating your node info.


text:
clightning uses all my resources, ... till I have to shutdown!
clightning runs for a while great. However, every second day the computer locks up,
I use docker-compose for that:
version: "3"
services:
  bitcoind:
    image: nicolasdorier/docker-bitcoin:0.16.3
    container_name: bitcoind
    environment:
      BITCOIN_EXTRA_ARGS: |
        testnet=0
        whitelist=0.0.0.0/0
        server=1
        rpcuser=*****
        rpcpassword=*****
    expose:
      - "8333"
    ports:
      - "0.0.0.0:8333:8333"
    volumes:
      - "/media/ronald/data/scratch/bitcoin/mainnet/bitcoind:/data"

  clightning_bitcoin:
    image: elementsproject/lightningd
    container_name: lightning
    command:
      - --bitcoin-rpcconnect=bitcoind
      - --bitcoin-rpcuser=****
      - --bitcoin-rpcpassword=****
      - --network=bitcoin
      - --alias=TWronald✅
      - --rgb=15C315
      - --log-level=debug
      - --announce-addr=220.133.223.206:9735
      - --bind-addr=0.0.0.0:9735
    environment:
      EXPOSE_TCP: "true"
    expose:
      - "9735"
    ports:
      - "0.0.0.0:9735:9735"
    volumes:
      - "/media/ronald/data/scratch/bitcoin/mainnet/clightning:/root/.lightning"
      - "/media/ronald/data/scratch/bitcoin/mainnet/bitcoind:/etc/bitcoin"
    links:
      - bitcoind

lightning-cli getinfo
{
  "id": "035aef5661e1a6e370db60dc0455796800afd5b51fbc12a0a8b34836b15f5d7ef6",
  "alias": "TWronald✅",
  "color": "15c315",
  "num_peers": 15,
  "num_pending_channels": 0,
  "num_active_channels": 15,
  "num_inactive_channels": 0,
  "address": [
    {
      "type": "ipv4",
      "address": "220.133.223.206",
      "port": 9735
    }
  ],
  "binding": [
    {
      "type": "ipv4",
      "address": "0.0.0.0",
      "port": 9735
    }
  ],
  "version": "",
  "blockheight": 560987,
  "network": "bitcoin",
  "msatoshi_fees_collected": 0
}

Why the version of lightning is "" ??? scratch/bitcoin/mainet/clightning/config is empty.
Currently it is running fine.
ps ax |grep lightn
 7278 ?        S      0:03 /usr/bin/lightning_channeld
 8524 pts/20   S+     0:00 grep --color=auto lightn
16579 ?        S      0:06 /usr/bin/lightning_channeld
31660 ?        S      1:19 lightningd --bitcoin-rpcconnect=bitcoind --bitcoin-rpcuser=**** --bitcoin-rpcpassword=**** --network=bitcoin --alias=TWronald✅ --rgb=15C315 --log-level=debug --announce-addr=220.133.223.206:9735 --bind-addr=0.0.0.0:9735
31665 ?        Z      0:00 [lightning_chann] <defunct>
31666 ?        Z      0:00 [lightning_closi] <defunct>
31667 ?        Z      0:00 [lightning_conne] <defunct>
31668 ?        Z      0:00 [lightning_gossi] <defunct>
31669 ?        Z      0:00 [lightning_hsmd] <defunct>
31670 ?        Z      0:00 [lightning_oncha] <defunct>
31671 ?        Z      0:00 [lightning_openi] <defunct>
31675 ?        S      0:00 /usr/bin/lightning_hsmd
31677 ?        S      0:00 /usr/bin/lightning_connectd
31678 ?        S      5:45 /usr/bin/lightning_gossipd
31862 ?        S      0:00 socat TCP4-listen:9835,fork,reuseaddr UNIX-CONNECT:/root/.lightning/lightning-rpc
31868 ?        S      0:06 /usr/bin/lightning_channeld
31869 ?        S      0:09 /usr/bin/lightning_channeld
31870 ?        S      0:06 /usr/bin/lightning_channeld
31871 ?        S      0:07 /usr/bin/lightning_channeld
31872 ?        S      0:07 /usr/bin/lightning_channeld
31874 ?        S      0:10 /usr/bin/lightning_channeld
31875 ?        S      0:10 /usr/bin/lightning_channeld
31876 ?        S      0:06 /usr/bin/lightning_channeld
31879 ?        S      0:10 /usr/bin/lightning_channeld
31988 ?        S      0:06 /usr/bin/lightning_channeld
31989 ?        S      0:06 /usr/bin/lightning_channeld
31992 ?        S      0:07 /usr/bin/lightning_channeld

... but still some Zombies.
When it grinds to a hold, it uses most with gossip.
Can you spot some changes I should make to improve my system?

This is most likely the gossipd bug that was causing a request from a peer to be processed very slowly. It was fixed in PR 2268. You don't have that fix in your installation yet, which can be seen by the 5 daemons that are not being reaped correctly (the <defunct> processes in the list). This was also fixed in PR 2224.
Both fixes are not yet part of a release, but we will release them soon.

I just updated the image you are using to include the mentioned fixes, so a simple docker pull elementsproject/lightningd should get you going again. I'm investigating why the automatic image build broke on Docker Hub.

Thank you very much. I updated the image and wait for the version fix, which is not that a priority.


text:
"Cannot afford transaction"
version 0.7 clightning
Listfunds shows me:
963692 satoshi available to fund channels
6280512 satoshi owned in channels
14832105 satoshi total channel capacity

When I try to fund a channel with 400000 I get:
lightning-cli fundchannel 02b414e4e29a685b8699152be47a0420fa3c8ab59629d2da2d6c0a95c582636350 400000
{ "code" : 301, "message" : "Cannot afford transaction" }

Why?

Could it be that the outputs depicted in listfunds do not have yet enough confirmations? Maybe you got them after you closed a channel and c-lightning did not allow you to use them right away. Could you check again?

That's my expectation as well. c-lightning 0.7 will only consider outputs with at least 1 confirmation by default. To use unconfirmed outputs you can use the minconf=0 parameter, but be careful since that can lead to chaining of unconfirmed transactions if the first one never confirms.

When you use the fundchannel command, a 2-of-2 multisig transaction is made on-chain.
From The BOLT #3: Bitcoin Transaction and Script Formats:

"The amounts for each output MUST be rounded down to whole satoshis. If this amount, minus the fees for the HTLC transaction, is less than the dust_limit_satoshis set by the owner of the commitment transaction, the output MUST NOT be produced (thus the funds add to fees)."

Probably your funding ammount is under the ammount of the dust_limit_satoshis + transaction fees.

No 400k Satoshi is far more than the dust limit. I checked the Api of fundchannel command. It expects Satoshi and not msatoshi in which case we would be below dust limit

Ah ok, good to know. Thx.

I just tried again, and it gone through. I have no idea, when and why it happens.


text:
Clightning with BTCpay server failed
I have setup WooCommerce and BTCpayserver in Google cloud, and synced. Test to buy with BTC seems to work.
Adding Lightning fails:
I modified in BTCpay Lightning nodes in the Store management
"clicking here" added the connection string:
Type=clightning;server=unix://etc/lightning_bitcoin/lightning-rpc

I submit and came back to test connection, it fails:
Error: Error while connecting to the lightning node via 34.73.111.47:9735 (The operation was canceled.)operation was canceled.)

I used during setup the following Exports:
 export BTCPAY_HOST="btcpay.elmit.com"
 export NBITCOIN_NETWORK="mainnet"
 export BTCPAYGEN_CRYPTO1="btc"
 export BTCPAYGEN_REVERSEPROXY="nginx"
 export BTCPAYGEN_LIGHTNING="clightning"
 export BTCPAYGEN_ADDITIONAL_FRAGMENTS="opt-save-storage-xs;opt-save-memory"
 export BTCPAYGEN_ADDITIONAL_FRAGMENTS="$BTCPAYGEN_ADDITIONAL_FRAGMENTS;opt-add-woocommerce"
 export WOOCOMMERCE_HOST="store.elmit.com"

Did I miss something?

I found it. The port 9735 is not open to the public.
Go to google cloud console
Home -> VPC network -> Firewall rules

+ Create firewall rule:

Name      Type    Targets   Filters          Protocols / ports  Action  Priority    Network
lightning Ingress Apply to all  IP ranges: 0.0.0.0/0 tcp:9735       Allow   1000        default


text:
Track the expenditure of an output with Bitcoin-qt via the REST API
How do you find the transaction whose input, vin, is a given output, vout? For example, the following is the JSON representation of the block with height 1000; here the mining reward is locked behind the given scriptPubKey, and I am interested in finding a transaction where this output is referenced as input.
{
    "hash": "00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09",
    "confirmations": 716370,
    "height": 1000,
    "version": 1,
    "versionHex": "00000001",
    "merkleroot": "fe28050b93faea61fa88c4c630f0e1f0a1c24d0082dd0e10d369e13212128f33",
    "time": 1232346882,
    "mediantime": 1232344831,
    "nonce": 2595206198,
    "bits": "1d00ffff",
    "difficulty": 1,
    "chainwork": "000000000000000000000000000000000000000000000000000003e903e903e9",
    "nTx": 1,
    "previousblockhash": "0000000008e647742775a230787d66fdf92c46a48c896bfbc85cdc8acc67e87d",
    "nextblockhash": "00000000a2887344f8db859e372e7e4bc26b23b9de340f725afbf2edb265b4c6",
    "strippedsize": 216,
    "size": 216,
    "weight": 864,
    "tx": [
        {
            "txid": "fe28050b93faea61fa88c4c630f0e1f0a1c24d0082dd0e10d369e13212128f33",
            "hash": "fe28050b93faea61fa88c4c630f0e1f0a1c24d0082dd0e10d369e13212128f33",
            "version": 1,
            "size": 135,
            "vsize": 135,
            "weight": 540,
            "locktime": 0,
            "vin": [
                {
                    "coinbase": "04ffff001d02fd04",
                    "sequence": 4294967295
                }
            ],
            "vout": [
                {
                    "value": 50.00000000,
                    "n": 0,
                    "scriptPubKey": {
                        "asm": "04f5eeb2b10c944c6b9fbcfff94c35bdeecd93df977882babc7f3a2cf7f5c81d3b09a68db7f0e04f21de5d4230e75e6dbe7ad16eefe0d4325a62067dc6f369446a OP_CHECKSIG",
                        "hex": "4104f5eeb2b10c944c6b9fbcfff94c35bdeecd93df977882babc7f3a2cf7f5c81d3b09a68db7f0e04f21de5d4230e75e6dbe7ad16eefe0d4325a62067dc6f369446aac",
                        "type": "pubkey"
                    }
                }
            ],
            "hex": "01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff0804ffff001d02fd04ffffffff0100f2052a01000000434104f5eeb2b10c944c6b9fbcfff94c35bdeecd93df977882babc7f3a2cf7f5c81d3b09a68db7f0e04f21de5d4230e75e6dbe7ad16eefe0d4325a62067dc6f369446aac00000000"
        }
    ]
}

This information is not stored by Bitcoin Core. Outputs are stored until they are spent, and then removed from the UTXO database. There is no information inside a transaction that indicates where it was spent (in the future), so you would manually have to compile this information yourself by iterating over all transactions in the blockchain.

Can't we use the index of transactions created when using the -txindex=1 argument? Iterating through all blocks/transactions is possible but a very expensive operation, leveraging index to some extent could prune some of the processes at least.

I don't think txindex=1 would help in this case. I suggest building your own database while parsing which you can make more complicated queries to, like block explorers do.


text:
Access REST endpoint of Bitcoin-qt from another computer on the local network
I have bitcoin-qt running on a Windows machine in my network started with the following command and empty config file (hence, all default configurations).
bitcoin-qt.exe -server=1 -rest=1

I can query the API endpoint of the node on the 8332 port; for instance:
curl http://localhost:8332/rest/chaininfo.json

I want to access the same REST API endpoint from the second computer on the network. On the second computer, I can reach the 8333 endpoint, but not 8332.
$ nc -vz 196.168.1.2 8333
Connection to 192.168.1.2 port 8333 [tcp/*] succeeded!

$ nc -vz 196.168.1.2 8332
nc: connect to 192.168.1.2 port 8332 (tcp) failed: Connection refused

On the first computer (the one that is running bitcoin-qt), it seems port 8333 can be reached with the computer's IP in the LAN while port 8332 is only reachble via localhost:
$ netstat -aon | findstr 8333
  TCP    0.0.0.0:8333           0.0.0.0:0              LISTENING       117384
  TCP    192.168.1.2:1033     84.212.130.217:8333    ESTABLISHED     117384
  TCP    192.168.1.2:1034     50.39.164.136:8333     ESTABLISHED     117384
  TCP    192.168.1.2:1035     45.132.158.199:8333    ESTABLISHED     117384
  TCP    192.168.1.2:1045     95.84.137.91:8333      ESTABLISHED     117384
  TCP    192.168.1.2:1065     82.197.215.125:8333    ESTABLISHED     117384
  TCP    192.168.1.2:1076     209.237.133.54:8333    ESTABLISHED     117384
  TCP    192.168.1.2:1091     178.48.160.142:8333    ESTABLISHED     117384
  TCP    192.168.1.2:1093     95.110.133.223:8333    ESTABLISHED     117384
  TCP    192.168.1.2:1106     5.188.62.18:8333       ESTABLISHED     117384
  TCP    192.168.1.2:1112     108.248.87.92:8333     ESTABLISHED     117384
  TCP    [::]:8333              [::]:0                 LISTENING       117384

$ netstat -aon | findstr 8332
  TCP    127.0.0.1:8332         0.0.0.0:0              LISTENING       117384
  TCP    [::1]:8332             [::]:0                 LISTENING       117384

Mind [::] for port 8333 and [::1] for port 8332.
I am wondering what changes need to be made in order to reach the 8332 port of the node from another computer in the network.

The rest interface uses the RPC options for setup. You can use the -rpcbind option to bind the interface to a different address and port, and set the -rpcallowip. Note that this will also affect the RPC interface.

Could you provide a working example? I used rpcbind according to the following docs, and still cannot get the address correct. https://github.com/bitcoin/bitcoin/blob/c5f0cbefa369b0e4d99a4f871e6334955d537c1f/src/init.cpp#L580-L582

For example, if the my computer's LAN IP is 192.168.1.10, then to expose my the rpc interface on LAN, I would do -rpcbind=192.168.1.10.

Thanks! I have actually tried that, but that does not seem to be enough; e.g., above-linked docs read "This option is ignored unless -rpcallowip is also passed.". I have tried passing -rpcallowip arg as well, but that does not seem enough either.

What value did you pass for -rpcallowip? In my example, you would do -rpcallowip=192.168.1.1/24.

Could you please elaborate? e.g., what is 24?

That is CIDR Notation

I was not able to connect from the second computer when bitcoin-qt was launched on the first computer with the following arguments. --rpcbind=192.168.1.2/24 (the IP address of my computer on LAN) and --rpcallowip --192.168.1.1/24` (the IP of the default gateway and router in the LAN).

I think -rpcbind should not have the the /24. You can also check the debug.log file to see if there are any warnings or errors related to the rpc interface there.

I don't see any binding to port 8332 in the logs, though I could see Bound to 0.0.0.0:8333.

By default, the logs won't say anything about what addresses it is binding to for the RPC interface. You can enable that by starting with -debug=http. Are you checking netstat or are you checking reachability with another computer? For the latter, you may need to allow port 8332 in the windows firewall. I'm not sure why these options wouldn't work as I do observe them working for me.

I've added -debug=http but still cannot see entries for 8332. I'm checking the connection with nc -vz 192.168.1.2 8332 on the second computer which returns `Connection refused. I have also opened the port on my firewall.

Could you please paste the full command your are using to start bitcoin-qt that works for you in this scenario?

./bitcoin-qt -debug=http -rpcbind=10.21.1.10 -rpcallowip=10.21.1.1/24 -server is my command (my computer's IP is 10.21.1.10). There is a log line that says [http] Allowing HTTP connections from: 127.0.0.0/8 ::1/128 10.21.1.0/24 127.0.0.1/32 and another with [http] Binding RPC on address 10.21.1.10 port 8332

Thanks for sharing the command you use. I executed the very same command (replacing my IP and gateway with yours), though still had the same issues are before. I do not have any of those entries in the log either. I'm using bitcoin-qt v22.

when I use -rpcbind=<IP> -rpcallowip=<IP> I read in the logs  option -rpcbind was ignored because -rpcallowip was not specified, and when I swap their order to -rpcallowip=<IP> -rpcbind=<IP> logs read: option -rpcallowip was specified without -rpcbind. Does it ring any bells?!

Log lines like that sound like the shell is improperly escaping/passing the arguments to the program. Instead of using command line arguments, you could try putting the arguments in the bitcoin.conf file with one argument per line without the leading -. For the boolean options (e.g. -server and -rest), you also need to append =1 to those (e.g. server=1).

Thanks! I guess all the confusion was because of the shell not parsing the arguments correctly, since setting the values in bitcoin.conf worked as expected.


text:
How to handle incomplete blocks when reading blk*.dat files?
I'm trying to write a parser for blk*.dat files from scratch. Right now, I can read blocks, extract the header fields and calculate the correct block hash based on the header fields.
I just tried to run the parser through all blk-files and it failed after successfully reading the first ~300 blk-files. When inspecting the failing file with a hex-editor, I found two "magic byte fields", only separated by 4 other bytes.

Since the mandatory block-header is supposed to be 80 bytes in length, I'm now quite confused.
I found three probably related questions:

blk file error when reading - did something change in the format?
How can you tell if you're at the end of an incomplete blk*.dat file?
Zeros in blk00*.dat files

My guess, based on these other questions, is the following:

This is an 'incomplete block'
This happens when a block can only be partially downloaded, or written to disk.
Deleting the affected blk files and re-downloading parts of the blockchain doesn't make sense, since this can be expected to happen again.
The parser should be able to skip these blocks.
The overall blk file is not missing actual data and is not corrupt.
The 'incomplete block' will be downloaded and written again further down in the blk file (or to an other blk-file).

Which of my assumptions are correct? .. which are not? Am I missing something?

The node was probably killed while writing to disk.

If you go on 0xd0 bytes further, is there another magic 4-bytes?  Theoretically the f9beb4d9 could occur within the data of the block itself, so you can't be sure that any occurrence marks the start of a block.  You should be using the lengths.  Though the first bytes of the block should be the version number, I think, and I am not sure whether f9beb4d9 would be valid there.

@eponymous I deleted and re-downloaded all blk files. This time, I made sure that the node didn't stop a single time. After that, I had no issues reading all of the blk files. This supports your theory.

Yeah. My files are in a strange state due to many forced node restarts during IBD. The software is fine with this because there’s an external database of pointers to the blocks on disk which can just skip the corruption.

@NateEldredge No, there is not. It's also very unlikely for those 4 bytes to occur within the data. https://bitcoin.stackexchange.com/questions/2337/how-was-the-magic-network-id-value-chosen

Unlikely to occur by chance, perhaps, but there is nothing preventing anyone from intentionally inserting those 4 bytes in a transaction (via OP_RETURN, for instance).  A miner who was feeling ornery could also twiddle the coinbase transaction until those bytes appeared in the Merkle root; it would take something on the order of 100 million tries, which is not much.  In that case you'd see those bytes in the block header.

Given your additional description, I agree that this is probably not the issue here, but it shows that your code definitely needs to handle this case properly.

Thanks @NateEldredge, I didn't think of somebody generating those bytes on purpose. I wonder if this has already been tried?  Your are absolutely right, though. My parser has to handle this properly.

@forgemo: I'd be kind of surprised if it hasn't.  Maybe I'll search the blockchain sometime to see.  If nobody has done it then maybe I will :-)

@NateEldredge I couldn't resist to search for it. :) As you already guessed, somebody did it. The following block from 2012 seems to contain such a case. https://blockchain.info/block/00000000000005D7E684BEB913BD73FDC33BFD06C1FDF247E599F4D9D6061B91?format=hex This further stresses your suggestion to handle these cases properly.

I'm going to answer my own question. These are my conclusions based on the comments from Nade Eldrege and eponymous.
Assumption 1: This is an 'incomplete block'
Yes, this is an incomplete block. More specifically, it is not part of the blk file format.
Assumption 2: This happens when a block can only be partially downloaded, or written to disk.
Yes, evidence strongly suggests that this happens when the bitcoind is interrupted / killed while writing to disk. If restarted, it just continues to download and write new blocks to the blk file. It doesn't clean up incomplete block.
Assumption 3: Deleting the affected blk files and re-downloading parts of the blockchain doesn't make sense, since this can be expected to happen again.
Yes and no. Yes, because deleting and re-downloading the blk files without interruptions solves the issue. When writing a blk-parser, however, it should be able to handle these incomplete blocks. For example, by skipping to the next valid block.
Assumption 4: The parser should be able to skip these blocks.
Yes. See #3
Assumption 5: The overall blk file is not missing actual data and is not corrupt.
I'm not sure, but since the bitcoind just continues with downloading blocks and works without any issues, one can assume that the incomplete block is just  re-downloaded afterwards.
Assumption 6: The 'incomplete block' will be downloaded and written again further down in the blk file (or to an other blk-file).
It seems so. See #5


text:
Regarding Mnemonic Code words and salt key
My aim is to make a hardware wallet.
I was studying HD wallet generation (BIP 39) and have figured out how things work under the hood, except one thing.
At high level :

Mnemonic code (128bit or 256 bit) + Salt('mnemonic'+passphrase)
  -----do PBDKF2 (2048 rounds)---> 512 bit seed

Users are requested to back up their mnemonic code. If I am on a Hardware wallet like ledger nano, the passphrase above serves as my password.

Different passphrase creates different seed.

Now if i want to open the same wallet from anywhere else, theoretically i can do so just from the Mnemonic code(12 or 24 words).
My doubt is will the passphrase have no use if wallet is being opened from third party clients like Greenwallet, greenbitz, breadwallet etc?
To rephrase : How is passphrase consumed by third party wallets if I have generated an account on ledger?

Third Parties should extend your mnemonic phrase with your passphrase else they will get a different wallet with different addresses.
Electrum and some other wallets call the passphrase a "seed extension" or "mnemonic extension".
Third parties should have an option to extend mnemonic phrase with your passphrase, It's a weird idea to possibly have enough money to purchase the entire building just sitting on a sheet of paper without any protection. For this reason, many wallets make it possible to encrypt a mnemonic phrase with a password.
Take a look at this open-source project, you can import play around with any mnemonic phrase and extend it with your passphrase and see the changes

well if they do support, then its fine.  thanks Adam


text:
P2SH in Hardware wallet?
I am designing a hardware wallet and the software that will work with it.
I plan to provide P2PKH transactions support in the software for hardware wallet to sign.
Is it mandatory to support P2SH transactions ?
In my learning and use of bitcoin I have not come across P2SH transactions.
If its mandatory how does Ledger nano/ Trezor do it?
Im a little lost with how to support P2SH transactions from UI/UX point of view too.
P2PKH are simple one-one transaction to receiver Pubkey. Any help on P2SH would be really great. Thanks in advance. Sorry if this question sounds novice, Im in learning curve.

If you have a look at https://github.com/bitcoin/bips you can find all BIP's. Scrolling down to the README, you will find the list number and title of every BIP and the current status. The BIP process with definitions for the status codes and other aspects is in BIP0002. At the top of the page you can search for P2SH or Pay to Script Hash and find many matches. BIP0016 seems to be particularly relevant but is certainly not the only source.
There are several BIP's dealing with aspects of P2SH and I would suggest that you should support P2SH transactions as they are a standard type of transaction that your end users may receive, although, being able to create P2SH transactions is probably not necessary. Check the BIP's.
You may also be interested in the [bitcoin-dev] mailing list.

Thank You , I am planning to support incoming transactions (Not much to be done for receiving end as all of it will happen on the blockchain). Hardware wallet will create outgoing transactions only.


text:
Bitcoin difficulty: One block with very high difficulty vs. long chain with accumulated difficulty
First assume the hypothetical scenario, that someone has infinite hashing power, starts with the genesis block and mines one block, that has higher difficulty than the whole "normal" blockchain accumulated. Let's call this one-block blockchain (B) and the regular one (A). Would this be the new "real" blockchain with only 50 BTC in circulation? To extend the question, secondly assume our attacker splits their hash power to mine a whole blockchain with similar length to the current one but higher accumulated difficulty, how would we know if it's the real blockchain? All non-fake transactions are in Blockchain (A).
There is no concept of time in the blockchain, but everyone on the planet would know that (A) is the real blockchain because it had been used for so long. Is there some mechanism to prevent this, some kind of sanity check?

The problem with assuming infinite hashing power is that it has much larger and much more interesting consequences - like vaporizing the Earth and the rest of the universe at the speed of light. BItcoin would end along with the planet.  https://what-if.xkcd.com/109/

that has higher difficulty than the whole "normal" blockchain accumulated.

That's not possible. The difficulty of a block is solely a function of the timestamps of the blocks in its history. It does not depend on the block itself, it cannot be chosen by the miner, and isn't dependent on how low the hash is. The only thing that matters is if the hash is low enough or not to meet the difficulty target. The difficulty can go up at most 4x, and that happens only when the past 2015 blocks took less than 3.5 days.
In general, if there are two chains and both are otherwise valid, the one with the highest accumulated difficulty will win. There is nothing to protect against here, (short) reorgs are a fact of life, and long ones cannot be prevented. The economic assumptions underlying proof-of-work imply that no attacker will want to pay the cost for such a deep reorg. If that happens to be wrong, we should move to something else.

Thanks, this is what I did not have straight.


text:
Is BTC inherently much more energy-hungry than other currencies, or just been around longer?
A beginner question, but my understanding is that BTC (and many others) rely on solving ever increasingly complex algorithmic problems, which acts as a self-gating mechanism on the ever-decreasing cost of CPU power(?)
BTC's energy use has been in the news a lot recently and there's discussion about relative transaction cost in other coins. Do they work fundamentally differently, or is their blockchain just so much smaller that they are where BTC was, say, 2 years ago?
Or am I confusing mining with transaction cost?

Related: https://bitcoin.stackexchange.com/q/106298/5406, https://bitcoin.stackexchange.com/q/95356/5406

other coins. Do they work fundamentally differently,

To secure the record of transactions, Bitcoin uses a type of mechanism called Proof of Work (PoW). Some other cryptocurrencies use any of several other mechanisms such as Proof of Stake (PoS).

BTC (and many others) rely on solving ever increasingly complex algorithmic problems

The complexity is not increasing. A "difficulty" target is adjusted that affects the number of iterations required. That difficulty target can go down as well as up.

BTC's energy use ...

The energy consumption of Bitcoin's PoW depends on the amount of computing power that miners have invested in. To ensure that new technologies and miners with deep resources do not become able to forge the transaction journal, Bitcoin increases the difficulty of Mining proportionally to the total computing power applied by miners. Mining is competitive so more powerful mining equipment provides a higher proportion of the rewards. So the energy consumption of the Bitcoin network is largely a reflection of the return on investment (ROI) that miners predict in their business plans. It isn't affected directly by number of transactions.
Transactions provide a transaction fee to miners, so the transaction rate and fee-rate has some influence on ROI.  However the transaction rate does not directly drive energy costs in the network.
To put it another way, if all but ten† miners left the Bitcoin network and those ten† only used one† small† PC each for mining, the transaction rate might not be affected and energy costs would be far far smaller. However, in the absence of any other limiting mechanism, the network would be insecure, a miner could step in and use overwhelming computing power to commit fraud.

† These numbers are not real, just illustrative. there is some minimal number that would be adequate to process current transaction rates if all humans were always honest, trusted all other humans in all circumstances, never had disagreements and never made mistakes. However Bitcoin was made for the real world and this requires some energy be expended to keep money secure

There's an entirely different way of thinking about energy costs. Consider the functions that the Bitcoin network provide, the corresponding functions in banks, and how much energy they consume. How much energy is consumed by:

Citibank
Chase
Hundreds of thousands of banks and branches

Where is the Bitcoin Building? All the electricity for all of the desks in the buildings? What about all of the inefficient processes performed by banks that take man-days, overnight batch processes, etc.?
Here's a recent article that sheds some light on the topic: "Banking System Uses Significantly More Energy Than Bitcoin.

while this is interesting, it's not actually an answer to the question but BTC apologetics.

In a way the large energy expenditure is what secures the large sums of money being transacted today. These days it's not uncommon to see a single transaction of hundreds of millions of dollars. 5 years ago something that size probably would have been riskier because the total energy used to secure the network was far smaller.
The energy usage helps secure and support large transactions, try doing the same large transactions on lesser coins that runs on PCs, you run the risk of being attacked and double spent hence no one would do it.


text:
How can BTC become a useful currency when the cost-per-coin is so large?
A couple of linked questions really. Firstly, let's say BTC is worth $50k US, that means to purchase your groceries might be 0.002BTC. A sandwich might be 0.0000598. Has a system already been proposed to have better units? With 1 US cent equating to 0.000002BTC could we ever have precision issues?
Secondly, with a hard limit of 21 million BTC and a world population of 8bn, we're talking each person on average could own around .0025BTC. Which at current market cap, is about $100 US. By contrast there's approx $40tn US in regular cash, about 40-60X the market cap of BTC if I didn't make a silly mistake (and depending the value the day you read this).
Doesn't this imply that for BTC to be a real-world currency, it's already large value must end up more like $2m US per coin. At which point your $2.99 sandwich is going to cost 0.000001495 and we're needing 9 decimal places for accurate accounting to the equivalent of one US cent today.
As a newcomer to this world, how does BTC plan to work around these issues to be a practical everyday currency? Is it a trivial thing, or actually a problem?

Today 1 US dollar is 0.000027 BTC which is 2,700 Satoshi
So one US cent is 27 Satoshi
When was the last time you purchased something with a total transaction value of one cent?
If Bitcoin's value increases as you say then there will be a need for a smaller unit. I think the Lightning Network can already have transactions measured in millisatoshi - a thousand times smaller.
So, It isn't a major problem.
It isn't especially unusual for the major unit of a currency to have a high value. For example the UK currency known as the pound sterling was created in the early Anglo-Saxon period one and a half millennia ago and was essentially a pound weight of silver. That represented a huge amount of money at the time, enough to buy several thousand chickens or a hundred knives at a time when ordinary people could probably afford few of either.

Related questions:

What is a millisatoshi? I thought satoshi is the smallest unit?
Why can we have small fractions of a Bitcoin?
What is the smallest fee you can pay for an on chain transaction?

I haven't purchased for 1c for a long time, but I frequently buy things where the number of cents is not a multiple of 5/10. e.g. buying gas, or things which are $0.99c. To how many decimal places is BTC stored, I'm assuming more than 9 :) edit: oh your link mentions we have 1^8 integral subdivisions

Indeed, at the protocol level "1 BTC" doesn't exist. It is all represented as multiples of satoshis (each being equal to 0.00000001 BTC).


text:
How to see the balance change from each block
Given a new bitcoin block, I want to see:

Which address have balance change.  I.E., send / receive
The amount of that change

Is there any RPC endpoint to achieve that purpose?
Basically, I maintain the balance of ALL bitcoin addresses. And after each new block, I want to update the balance of the changed address.

Addresses are not stored on the blockchain in the typical format (Base58 or Bech32). They are stored as ScriptPubKeys.
To update each address, you would have to run getblock and parse each output's ScriptPubKey to an address and then find the Address of each input and remove those.
Bitcoin does track the existence of every currently existing coin in what's called the UTXO set.