text:
Why is a minRelayTxFee of 1000 sat/kb not equivalent to 250 sat/kw?
Naively, we would expect a minRelayTxFee of 1000 sat/kb to correspond to 250 sat/kw. However, apparently this is not so according to this issue (ACINQ), which also references this issue (c-lightning).

  /**
    * why 253 and not 250 since feerate-per-kw is feerate-per-kb / 250 and the minimum relay fee rate is 1000 satoshi/Kb ?
    *
    * because bitcoin core uses neither the actual tx size in bytes or the tx weight to check fees, but a "virtual size"
    * which is (3 * weight) / 4 ...
    * so we want :
    * fee > 1000 * virtual size
    * feerate-per-kw * weight > 1000 * (3 * weight / 4)
    * feerate_per-kw > 250 + 3000 / (4 * weight)
    * with a conservative minimum weight of 400, we get a minimum feerate_per-kw of 253
    *
    * see https://github.com/ElementsProject/lightning/pull/1251
    **/
   val MinimumFeeratePerKw = 253

I thought vbytes were equal to weight units divided by 4 and rounded up, so I don't understand where (3 * weight) / 4 comes from.

I think this is a typo. It's (3 + weight) / 4, not (3 * weight) / 4.

I thought vbytes were equal to weight units divided by 4

It is [1], but rounded up to the next integer, so the implementation in bitcoind is :
int64_t GetVirtualTransactionSize(int64_t nWeight, int64_t nSigOpCost, unsigned int bytes_per_sigop)
{
    return (std::max(nWeight, nSigOpCost * bytes_per_sigop) + WITNESS_SCALE_FACTOR - 1) / WITNESS_SCALE_FACTOR;
}

(Where WITNESS_SCALE_FACTOR = 4).
In addition, bitcoind not only uses vbytes for user interface but also for mempool logic and the DEFAULT_MIN_RELAY_TX_FEE constant is set in vbytes :
static const unsigned int DEFAULT_MIN_RELAY_TX_FEE = 1000;

In order to check it for standardness bitcoind will compute your transaction size as rounded up (if not a multiple of 4), and will record your transaction feerate as rounded down.
It will then compare this to the minimum relay feerate (1000) : a feerate of 250sat per KW will pass only if the size of your transaction (in weight units) is a multiple of 4.
It might be clearer with code, here is a Python function which illustrates this behaviour :
>>> def bitcoind_fun(tx_weight, feerate_perkw):
...     fees = tx_weight * feerate_perkw // 1000
...     print("Your transaction will pay {} sats of fees, is {} WU large (a feerate of {}sat/Kw)".format(fees, tx_weight, feerate_perkw))
...     tx_vbytes_for_bitcoind = (tx_weight + 3) // 4
...     tx_feerate_vbytes_for_bitcoind = fees * 1000 // tx_vbytes_for_bitcoind
...     print("bitcoind reads your transaction as paying {} sats of fees for a transaction of {} vbytes, so a {}sat/perKvb feerate".format(fees, tx_vbytes_for_bitcoind, tx_feerate_vbytes_for_bitcoind))

Which if you run with a tx_weight which is a multiple of 4 will pass, otherwise it won't :
>>> bitcoind_fun(1600, 250)
Your transaction will pay 400 sats of fees, is 1600 WU large (a feerate of 250sat/Kw)
bitcoind reads your transaction as paying 400 sats of fees for a transaction of 400 vbytes, so a 1000sat/perKvb feerate
>>> bitcoind_fun(1601, 250)
Your transaction will pay 400 sats of fees, is 1601 WU large (a feerate of 250sat/Kw)
bitcoind reads your transaction as paying 400 sats of fees for a transaction of 401 vbytes, so a 997sat/perKvb feerate
>>> bitcoind_fun(1602, 250)
Your transaction will pay 400 sats of fees, is 1602 WU large (a feerate of 250sat/Kw)
bitcoind reads your transaction as paying 400 sats of fees for a transaction of 401 vbytes, so a 997sat/perKvb feerate

So we use 253 as the feerate floor to always be safe :
>>> bitcoind_fun(1600, 253)
Your transaction will pay 404 sats of fees, is 1600 WU large (a feerate of 253sat/Kw)
bitcoind reads your transaction as paying 404 sats of fees for a transaction of 400 vbytes, so a 1010sat/perKvb feerate
>>> bitcoind_fun(1601, 253)
Your transaction will pay 405 sats of fees, is 1601 WU large (a feerate of 253sat/Kw)
bitcoind reads your transaction as paying 405 sats of fees for a transaction of 401 vbytes, so a 1009sat/perKvb feerate
>>> bitcoind_fun(1602, 253)
Your transaction will pay 405 sats of fees, is 1602 WU large (a feerate of 253sat/Kw)
bitcoind reads your transaction as paying 405 sats of fees for a transaction of 401 vbytes, so a 1009sat/perKvb feerate

By the way the comment in the code you put is wrong : it's not (3 * weight) / 4  but (3 + weight) / 4.

[1] Excluding sig OPs limits that artificially increases the virtual size to further prevent DOS.
PS: In the same vein, see https://github.com/bitcoin/bitcoin/issues/13283.

I see, so the core issue is rounding up when calculating the tx size, and rounding down when calculating the fee rate. That is very tricky. Thank you for such a clear explanation!

The round down is actually a consequence of the round up but yes that's the point. You are welcome :)


text:
Why are Sphinx forward messages not used in Lightning's onion packets?
It is useful to send some additional information with a payment in LN. For example, spontaneous payments in LND send a payment_preimage to the destination of a payment. LND uses a non-trivial EOB format to try and encode this information into the hops_data. The hops_data must be a fixed size to prevent intermediary nodes knowing where they are in a route, which limits the size of information which can be forwarded.
The Sphinx protocol on which LN's onion routing is based already supports forwarding encrypted messages as a separate field. The notation (M,δ) is used for an onion message, where the M corresponds to the onion_packet format in BOLT4. δ is the encrypted payload, which can be of arbitrary size, and is not used.
Would it not make more sense to utilize this message forwarding rather than trying to overcomplicate the protocol with the EOB format, which gains us a trivial number of bytes?
I can see that arbitrary size may be an issue for an attacker monitoring traffic flows. Perhaps δ can be made a fixed size and padded such that each update_add_htlc_with_payload remains a fixed size. We could potentially transfer larger and more useful information to the destination (in particular, I would like to be able to embed a refund invoice in the payment).
Are there any other issues with forwarding a separate encrypted payload which I'm unaware of?

Questions like these are always leading to an opinionated response and are a little bit tough to answer. I will give it a shot though.
First of all I think youelf already answered your question to an important degree. Forwarding messages via the SPHINX protocol yield the possibility of message length attacks.
I think Another reason was the goal to be simple in the beginning. I know there is a lot of discussion to extend onion routing to HORNET which would create a connection oriented encrypted communication layer on top of the lightning network and routing protocol. This obviously yield some advantages but also complexity and possibilities to introduce exploits.
Let us remember that sphinx mix format was created to send and receive messages / emails and we "abuse" it for lightning. So I guess we liked at the core feature we needed.
I think the main reason for using the EOB field in spontaneous payments is that it was easy to implement this in the current system without too many changes. But to be sure roasbeef might give the exact reason.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
What OP_CODEs would push 254 bytes of data?
From BIP62 I would expect to use OP_PUSHDATA1 followed by uint8_t:

Pushing 76 to 255 bytes must use OP_PUSHDATA1.

But then from this answer, the link to variable-length integer makes it seem that a value of 253 (0xfd) should be encoded by 0xFD followed by the length as uint16_t.
EDIT-1: Here is a dummy transaction I am trying to get to parse.
Here I have a tx with 252-byte scriptSig (all 1s). I have fc to encode its length and it parses:
0200000001517adf7ba2ab9fd2c514687b0774c93d87d3e959d202b66fed76782d832a94f800000000fc111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111ffffffff0280d1f008000000001976a9143bc28d6d92d9073fb5e3adf481795eaf446bceed88ac80f0fa02000000001976a914cc1b07838e387deacd0e5232e1e8b49f4c29e48488ac00000000
Then I tried adding two bytes to the start of the scriptSig (2222) to make it 254 bytes long, and replaced the length fc with 4cfe (OP_PUSHDATA1 fe), but it's not parsing anymore:
0200000001517adf7ba2ab9fd2c514687b0774c93d87d3e959d202b66fed76782d832a94f8000000004cfe2222111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111ffffffff0280d1f008000000001976a9143bc28d6d92d9073fb5e3adf481795eaf446bceed88ac80f0fa02000000001976a914cc1b07838e387deacd0e5232e1e8b49f4c29e48488ac00000000
EDIT-2: The answer is that encoding the length of the scriptSig is a varint rather than a push OP_CODE. So the correct way to get this to parse was to set the sigScript length to 0xfd 0xfe 0x00:
0200000001517adf7ba2ab9fd2c514687b0774c93d87d3e959d202b66fed76782d832a94f800000000fdfe002222111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111ffffffff0280d1f008000000001976a9143bc28d6d92d9073fb5e3adf481795eaf446bceed88ac80f0fa02000000001976a914cc1b07838e387deacd0e5232e1e8b49f4c29e48488ac00000000

Bitcoin Script opcodes, and P2P variable-length integers are distinct concepts.
To push a 254-byte element in Bitcoin Script, use OP_PUSHDATA1 0xfe (followed by the 254 bytes to be pushed).
To encode e.g. that a transaction has 254 inputs, its varint encoding 0xfd 0xfe 0x00 would be used.

Thank you very much. I tried using 4cfe but I'm not getting the transaction to parse. I added a concrete example to the question with a dummy scriptSig.

Ah okay I got it, I guess what goes at the start of the scriptSig is not a OP_PUSHDATA operation but is instead a varint encoding of the scriptSig length. I put 0xfd 0xfe 0x00 in the length field instead and it works now I'm updating the question.

Yes, the length of a script is not part of the script itself, and thus can't use script opcodes.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Output has an "Unknown" status?
Looking at transaction https://blockstream.info/tx/c362fe7f8cb94b363c0c7cd5a0f82cf0dfa7733ab8a3b90df3eb9f0bff912148 some outputs are having type: nonstandard.
There are 2 questions related to it:

Wouldn't this construct be interesting from the perspective of obscuring data against blockchain analytics?
Where can I learn more about transactions like this one? Why does it have something that looks like a multisig definition in the output?

Those are bare multisig outputs, which are in fact standard. There just is no address format for them.

If it's the standard, is there a place I can read more about it? Maybe it was introduced in some BIP?

Standardness is generally not specified in BIPs, as it is up to individual nodes to decide, and depends on the exact software you're using and what version. Bare multisig is accepted by most nodes, though.


text:
bitcoind: How to get rawblock data with ZeroMQ
I am trying to use the zeromq api for bitcoind to get rawblock but its not working with the code I've provided below. I am able to get hashtx and rawtx if I remove the filter but not hashblock and rawblock. Not sure if this is an important detail or not but my bitcoind is still syncing so I'm not sure if those topics are only triggered when bitcoind has synced up.
// Implementation of ZeroMQ in node.js.
// From the maintainers of the ZeroMQ protocol.
var zmq = require("zeromq");

// Create a subscriber socket.
const sock = new zmq.Subscriber();
var addr = "tcp://127.0.0.1:28332";

// Initiate connection to TCP socket.
sock.connect(addr);

// Subscribe to receive messages for a specific topic.
// This can be "rawblock", "hashblock", "rawtx", or "hashtx".
sock.subscribe("rawblock");

(async () => {
  for await (const [topic, msg] of sock) {
    console.log(topic.toString())
  }
})();

My bitcoin.conf file looks like:
server=1
txindex=1

# [zeromq]
zmqpubhashblock=tcp://127.0.0.1:28332
zmqpubhashtx=tcp://127.0.0.1:28332
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28332

What sort of error messages are you getting? And do you see anything relevant in your debug.log file from bitcoind?

@torkel no error messages, just not receiving the messages coming from my node when rawblock is being used. If I try to get rawtx or hashtx it works fine which is weird. I didn't find anything in the debug.log either.
When I run bitcoin-cli getzmqnotifications I get the following as expected:
[{ "type": "pubrawblock", "address": "tcp://127.0.0.1:28332", "hwm": 1000 }, ... ]

ZMQ notifications for new blocks only start when bitcoind is in sync with the network.


text:
What does PSBT stand for?
What is PSBT? What does it stand for?
bitcoin-qt has menu items "Load PSBT from file…" and "Load PSBT from clipboard…"; the dialog box says the file format is "Partially Signed Transaction".

It stands for "Partially Signed Bitcoin Transaction".
Bitcoin Core's documentation describes it as:

PSBT is an interchange format for Bitcoin transactions that are not fully signed yet, together with relevant metadata to help entities work towards signing it. It is intended to simplify workflows where multiple parties need to cooperate to produce a transaction. Examples include hardware wallets, multisig setups, and CoinJoin transactions.


text:
For SIGHASH_SINGLE do the outputs other than at the input index, have 8 bytes or 9 bytes including a VarInt?
The bitcoin wiki says:

All other txCopy outputs aside from the output that is the same as the
  current input index are set to a blank script and a value of (long)
  -1.

https://en.bitcoin.it/wiki/OP_CHECKSIG#Hashtype_SIGHASH_SINGLE
It's hard finding what this means in the C++ client. Does it mean that the outputs, other than the output with the same index as the input being signed for, only have 8 bytes for the value or does it mean they also have another byte for the VarInt set to zero; 9 bytes in total?
Thanks.

Transaction outputs have a "value" field that's an 8-byte signed integer (always 8 bytes) indicating how many satoshis (0.00000001 BTC each) are available for that output.  That is the thing that gets set to -1 on the other outputs in SIGHASH_SINGLE mode.
This looks to be the code responsible for making this happen, from script.cpp:
/** Serialize an output of txTo */
template<typename S>
void SerializeOutput(S &s, unsigned int nOutput, int nType, int nVersion) const {
    if (fHashSingle && nOutput != nIn)
        // Do not lock-in the txout payee at other indices as txin
        ::Serialize(s, CTxOut(), nType, nVersion);
    else
        ::Serialize(s, txTo.vout[nOutput], nType, nVersion);
}

If we're in SIGHASH_SINGLE mode, and the index of the output is different from the index of the input, then the thing that we sign in this spot is just a dummy CTxOut object, whose parameterless constructor (in core.h) calls SetNull(), which sets the field called "nValue" to -1 (-0.00000001 BTC) and clears the scriptPubKey, which ensures that it's a 0-byte vector.
When CTxOut gets serialized, it writes the nValue and scriptPubKey per this (core.h again):
IMPLEMENT_SERIALIZE
(
    READWRITE(nValue);
    READWRITE(scriptPubKey);
)

nValue serializes as the 8-byte twos-complement representation of -1, or 0xFFFFFFFFFFFFFFFF.
scriptPubKey serializes as a 1-byte CompactSize that indicates that the script is 0 bytes long, or 0x0.

so, the short answer is: 9 bytes.  ;>

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What are the alternatives to downloading the 6GB block chain?
I hail from a state where bandwidth is yet at a premium. Typical download speeds, in my experience as a dialup ADSL user, range from 10KBps up to 70KBps. Inclement weather may cause packet loss if not outright link failure. Over the last few months several ISP hereabouts have responded to service requests speaking of issues at the gateway.
Granted the issues I face are local but my intent here should be pretty clear - What are the alternatives to downloading the multi-GB (and growing!) block chain?

Alternatives to downloading the data yourself in reference client:

Take your computer to somewhere with Internet that you can be for several hours, perhaps a friend or relative's house, or a library or coffee shop.
You can employ bootstrap.dat by having someone else download it and give it to you somehow. Then you'd only have to get caught up and download ~ 2 GB. Alternatively, you can have that same person create for you a bootstrap.dat that is 100% up to date, as that bootstrap is behind by a few months.
Use a web wallet. You'll be sacrificing some security for the convenience, though.
Use an SPV client, such as Multibit or Bitcoin for Android, that only has to download data related to your addresses. You'll be sacrificing the ability to verify any payment. Many people find this acceptable.
Use a server-reliant client such as Electrum, where your client must download virtually no data. You'll always rely on a central server for transmission and verification of transactions, but your private keys will always be stored locally.

www.multibit.org <--This seems to be a pretty good client that doesn't need the full blockchain locally.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Is Ripple's Contract language intended to be Turing complete or sub-Turing complete ?
The wiki seems to imply Turing-completeness because no restrictions on jumps are specified here https://ripple.com/wiki/Contracts#Foundational_Ops.
Which is it and what (if any) is the actual intention ?
Cheers ...

Cool, I didn't even know Ripple had contracts!

If the language is Turing complete, they will promptly have to be disabled :-)

It should be Turing complete. However, every time a contract is invoked, the maximum number of operations the contract can perform is limited. Operations that access a contract must specify an operation count limit to get a limit higher than the default. Higher limits require higher transaction fees.

Does the Turing completeness of Ripple allow for denial of service attacks with infinite loops?

@makerofthings7 No, there's a strict limit on how much work a contract can do.


text:
What is the best way to make Bitcoin Core recognize and sign transaction for custom descriptor?
The problem is that I have a custom witness script and I want Bitcoin Core to be able to sign transactions for that script.
Initially, I wanted to create a new standard address and use its key for signing, but this way bitcoin core can't catch up on the same public key it already knows. This issue: https://github.com/bitcoin/bitcoin/issues/27336
So, now I want to use the private key of the current wallet with the hd-path.
But it requires strange manipulations like:

listdescriptors
Take random xpriv
Add this xpriv with some special hd-path to new descriptor
Import this descriptor
Then I can sign

I wonder is there a better way?
Descriptor for an example:
wsh(or_d(pk(<key bitcoin core should know>),and_v(v:pk(<another pk>),older(timelock))))


text:
Bitcoin Node starts initialblockchain download on every start
I setup a pruned bitcoin node with the following configuration
prune=550
dbcache=6192
par=4
maxconnections=20
server=1
rpcuser=testuser
rpcpassword=testpassword
datadir=/root/.bitcoin/blocks
rpcallowip=127.0.0.1
walletnotify=/usr/bin/php8.1 -f /var/www/html/btcnotifyme.php %s

But i cannot get it to be consistent on node starts - every time i start the node itll start a initialdownload again and i have to wait 3-4 days for it to synchronize again and again - does anybody have an idea what could be the reason for this ?
I just start the node bitcoind -daemon and stop it with bitcoin-cli stop. Running the bitcoin-cli getblockchaininfo also shows
"initialblockdownload": true and the blocks folder gets reset. I am using the 0.27.0 package.

Has been solved, you have to include -datadir in the startup command, somehow the datadir in bitcoin.conf does not work properly.


text:
bitcoin daemon crashing on initial block download
I'm trying to do the initial block download so I can run a full node. Didn't experience any issues until the download got to about 87% then bitcoind crashed without logging any errors. I've tried restarting it several times, however each time when I come back to my computer a few hours later to check on it I find that it has crashed and any download progress has been lost.
I am doing this on my personal laptop, using a 1 terabyte external hard drive (not SSD) and a wired Ethernet connection. I have disabled sleep on my computer and have been leaving it plugged into power.
Details:

OS: Ubuntu 22.04.5 LTS
CPU architecture: x86_64
CPU: Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
RAM: 16gb
Internet speed: 90 Mbit/s download
Bitcoin Core version: v27.0.0

This is the command I'm using to run the daemon:
bitcoind -daemon -debug -datadir=/media/me/TOSHIBA\ EXT/bitcoin/ -dbcache=10000

This is the last 25 lines from the debug.log file before it crashed:
2024-09-14T03:44:04Z [validation] Enqueuing MempoolTransactionsRemovedForBlock: block height=830276 txs removed=0
2024-09-14T03:44:04Z UpdateTip: new best=00000000000000000001914a6db5b53b3f8d673fbd9ebc1a569f37ccbbf97861 height=830276 version=0x20c98000 log2_work=94.729419 tx=965218034 date='2024-02-13T10:37:40Z' progress=0.888359 cache=2550.2MiB(20688484txo)
2024-09-14T03:44:04Z [bench]   - Connect postprocess: 0.40ms [3.08s (0.70ms/blk)]
2024-09-14T03:44:04Z [bench] - Connect block: 2522.60ms [6781.75s (1545.87ms/blk)]
2024-09-14T03:44:04Z [validation] Enqueuing BlockConnected: block hash=00000000000000000001914a6db5b53b3f8d673fbd9ebc1a569f37ccbbf97861 block height=830276
2024-09-14T03:44:04Z [validation] Enqueuing UpdatedBlockTip: new block hash=00000000000000000001914a6db5b53b3f8d673fbd9ebc1a569f37ccbbf97861 fork block hash=000000000000000000018e3f91a815472d0d994b832624751bbfbac25aa739cf (in IBD=true)
2024-09-14T03:44:04Z [validation] MempoolTransactionsRemovedForBlock: block height=830273 txs removed=0
2024-09-14T03:44:04Z [estimatefee] Blockpolicy estimates updated by 0 of 0 block txs, since last block 0 of 0 tracked, mempool map size 0, max target 0 from current
2024-09-14T03:44:04Z [validation] BlockConnected: block hash=000000000000000000031be2ac7b85a7db0493498a39f4580be01431134e4c13 block height=830273
2024-09-14T03:44:04Z [validation] UpdatedBlockTip: new block hash=000000000000000000031be2ac7b85a7db0493498a39f4580be01431134e4c13 fork block hash=00000000000000000003238c6e2e4551b0b5375ae52499bf03bf5de12a29e2bf (in IBD=true)
2024-09-14T03:44:04Z [validation] MempoolTransactionsRemovedForBlock: block height=830274 txs removed=0
2024-09-14T03:44:04Z [estimatefee] Blockpolicy estimates updated by 0 of 0 block txs, since last block 0 of 0 tracked, mempool map size 0, max target 0 from current
2024-09-14T03:44:04Z [validation] BlockConnected: block hash=00000000000000000000f3e8fcccf8ee483ebed02694c5773780a61a17a95037 block height=830274
2024-09-14T03:44:04Z [validation] UpdatedBlockTip: new block hash=00000000000000000000f3e8fcccf8ee483ebed02694c5773780a61a17a95037 fork block hash=000000000000000000031be2ac7b85a7db0493498a39f4580be01431134e4c13 (in IBD=true)
2024-09-14T03:44:04Z [validation] MempoolTransactionsRemovedForBlock: block height=830275 txs removed=0
2024-09-14T03:44:04Z [estimatefee] Blockpolicy estimates updated by 0 of 0 block txs, since last block 0 of 0 tracked, mempool map size 0, max target 0 from current
2024-09-14T03:44:04Z [validation] BlockConnected: block hash=000000000000000000018e3f91a815472d0d994b832624751bbfbac25aa739cf block height=830275
2024-09-14T03:44:04Z [validation] UpdatedBlockTip: new block hash=000000000000000000018e3f91a815472d0d994b832624751bbfbac25aa739cf fork block hash=00000000000000000000f3e8fcccf8ee483ebed02694c5773780a61a17a95037 (in IBD=true)
2024-09-14T03:44:04Z [validation] MempoolTransactionsRemovedForBlock: block height=830276 txs removed=0
2024-09-14T03:44:04Z [estimatefee] Blockpolicy estimates updated by 0 of 0 block txs, since last block 0 of 0 tracked, mempool map size 0, max target 0 from current
2024-09-14T03:44:04Z [validation] BlockConnected: block hash=00000000000000000001914a6db5b53b3f8d673fbd9ebc1a569f37ccbbf97861 block height=830276
2024-09-14T03:44:04Z [validation] UpdatedBlockTip: new block hash=00000000000000000001914a6db5b53b3f8d673fbd9ebc1a569f37ccbbf97861 fork block hash=000000000000000000018e3f91a815472d0d994b832624751bbfbac25aa739cf (in IBD=true)
2024-09-14T03:44:04Z [bench]   - Load block from disk: 194.05ms
2024-09-14T03:44:04Z [bench]     - Sanity checks: 4.44ms [12.23s (2.79ms/blk)]
2024-09-14T03:44:04Z [bench]     - Fork checks: 0.04ms [0.14s (0.03ms/blk)]

Things I've already tried:

run the daemon with debugging enabled and check the debug.log file for obvious problems
check if the external hard drive is corrupted: I've run some tests using smartctl and badblocks and they haven't reported any issues
experiment with different values for dbcache
look up the last block hash listed in the debug.log file in a public block explorer to verify it exists
look at the bitcoin source code where the last log comes from (validation.cpp line 2592). If I had to guess I'd say maybe something between that and the next log on line 2683 is causing bitcoind to crash without logging anything.

I would really appreciate any help with this. I'm finding it really hard to debug because I have to leave my computer running overnight to see if it crashes every time I try something new.

Is your datadir an external drive? Is it possible that that drive is becoming disconnected momentarily? In the past, people have experienced issues with using external drives.

Yes it is. I guess it's possible but I don't see how that could happen; I've been leaving my laptop on a shelf which doesn't get used for anything else, so the chances of it getting moved and the drive disconnecting momentarily seem pretty low.

i've had this same heisenbug for a longtime with a similar setup and think have found a workaround:
i've been accumulating datfiles on and off for years on one particular laptop and have been having the same problem across different bitcoind versions for a while using precompiled bitcoind binaries, always kept up to date.  if there are a lot of blocks to download and verify, it will most times, not always, segfault well into syncing.  i could run the daemon, let it accumulate about a months' worth of blocks, then stop it normally and restart it, and bootstrap my way to the current tip like that avoiding segfaults, but that's not ideal.
the laptop has two (internal) ssd's, the executable and conffile are on one, and the datadir on the other, hence i specify the datadir in the conf file.
i suspected hardware failure or corrupt datfile somewhere.  after running memtest86 (32gb ram) and ruling out ram, i deleted the datadir for a fresh run with v28.0, but got the same segfaults (never at the same point in the download and verification, usually ~300k, 400k or 500k blocks in, and i'm not going to bootstrap on and off from there!).
i tried gcc compiling source tagged v28.0 with --disable-wallet to see if that made a difference.  i tried compiling bleeding edge master also with and without wallet.  each executable segfaulted at some point well into the initial download (eg 9hrs).  never got anything from a strace.  on one occasion i ran it under gdb and it segfaulted at /usr/include/c++/14.2.1/bits/hashtable.h:2071
however i really doubt it was an issue in the c++ standard library.
anyway, i then compiled v28.0 using clang instead of gcc, and haven't experienced the problem again.
the precompiled binaries are compiled using gcc.

Id try running a 'pruned' node to see whether its a problem with hard drive space.
bitcoind -prune=100

This will download the latest blocks whilst keeping total blockchain size < 100Mb.
If you get this working you know its a size issue.
Should be faster for debugging too.
Id also invest in a SSD for better performance if your running it longer term.


text:
Upon opening the app, Bitcoin Core says I need to rebuild the block database. It then says error loading block database and crashes
I am relatively new to BTC but i’m all in on the principles behind it. I know nothing about coding or cryptography, but I love the concept of decentralization and I am a huge Austrian Econ nerd.
I had about 0.045 BTC in Coinbase Wallet, and transferred about 1/3 of it to my Bitcoin Core node on my macbook for the sake of having it out of a centralized exchange. I’ve been running the Bitcoin Core software for about a year and a half now, but never used it as a wallet until a week ago.
On 9/22/24, at 1500EST, the Core app stopped synchronizing with the network. I did some research as to why but didn’t make any changes. This morning, 9/24/24, I decided the best thing to do would be to increase the RAM allotment for the Bitcoin Core app in the preferences. Several people on various forums said this helped them when the Core app stalled in synching with the blockchain.
A dialogue box popped up and said I needed to restart the app for the change to initiate. I did so, and then everything went to hell.
As soon as I reopened the Bitcoin Core app, the following dialogue box popped up  [You need to rebuild the database using - reindex to go back to unpruned mode. This will redownload the entire blockchain.
Do you want to rebuild the block database now?]
When I selected {OK}, this dialogue popped up next.  [Error: Error opening block database].
When I select {OK} on the second dialogue box, the app crashes.
As I said, I am relatively new to Bitcoin, and I have gone down the principles rabbit hole for probably over a thousand hours, read Bitcoin Standard, Layered Money, Dom Frisby's Books, listened to every episode of Breedlove's What Is Money show, etc. But I don’t really have expertise on the use of the app, or of command prompts or coding whatsoever. I also, like a complete fkn idiot, did not write down my private key, as I didn’t even know how to extract it from the Core app, and I hadn’t dedicated that much time to figuring it out because I really didn’t think I would need it yet. Lesson learned.
Any wisdom regarding how to get the app to work again, how to update it, how to force it to start reindexing immediately on opening, how to find my private key from the db files, etc would be EXTREMELY welcome. Like I said, I only have about 0.016 BTC in this wallet, which isn’t a crazy amount, but it’s not nothing (~$1000 as of 9/24/24) so yeah, please and thank you to anyone and everyone here who might be able to throw me a rope.

You didn't mention backups. When did you last use the backup option in the menus to make backups ?


text:
Bitfinex API limit
There is currently an API limit of 60 requests per minute on Bitfinex.
Question is whether this limit is related to:

1) single IP address,

If so, can I have more IPs linked to one account each of which having the 60request/minute linit?

2) user account,

If so, can I have more accounts with the same IP address?

3) single API key

If so, can I have more keys to one account each of which having separate request limit?

I emailed them about this:

The limit is measured per IP address and per account. So for one
  account (regardless of the number of key/secret pairs) 60 requests per
  minute can be made via our API. When you are using multiple
  connections (IP addresses), the combined sum of all request for the
  account in question will used to determine if you are below the
  60/minute limit.
We hope this response has sufficiently answered your questions.

How do you measure the amount of the requests? Do you make it every minute or you start a minute and calculation only when the new request arrives after a delay?
Can I for ex. make one request every second during one minute and then make 60 requests at once during the first second of the second minute?

@Roces Presumably, they do it by incrementing a counter every time a request comes in. Or am I misinterpreting your question?

@NickODell sorry, updated my question. Hit enter too early.

What about WebSocket technology from Binance? Are there any rate limits?

My finding for bitfinex rate limit is this: 1 second is divided up in milliseconds. 60 requests must span the entire second.
For example, if you have a foreach loop you'll get a rate limit error in less than 15 requests, because you are querying too fast. So add usleep (microsecond sleep in PHP) to span your requests for the entire second and or have multiple IP's assigned to your VPS or dedicated box and use this curl option.
  <?php
  $ips = array('1.1.1.1','1.1.1.2','1.1.1.3');

  curl_setopt($ch, CURLOPT_INTERFACE, $ips[array_rand($ips)]);

I tried to comment on the other answer, but my rep isn't high enough.  I wanted to point out that Nick's presumption about using the counter makes sense but doesn't answer the question, which, assuming his presumption is correct, leaves this unanswered: When is the counter reset?
I would implement it this way, and (to be safe) assume that this is how they did it: On each request, if the number of requests matching either your account or your IP (or both) in the last 60 seconds is over 60, then the request is denied.  This means that one of the "60 requests at once during the first second of the second minute" would get answered and the rest would fail.  There would be no counter, but instead a record of all requests (IP, account, timestamp) that is purged of records over a minute old as often as necessary to keep the server healthy.


text:
Getting keys from multibit to export for electrum
I want to switch from Multibit to electrum. I can't figure out how to make use of the key multibit exports when it does a backup. Electrum and blockchain.info both seem to want something I can paste in but multibit exports in a binary format. I can't figure out how to use pywallet either.

You need to export the private keys from Multibit in text format using the 'Export Private Keys' option.  You can then copy and paste the private key strings into your new wallet.

Seems I missed the extension and was looking at the .wallet files. Thanks.

link directs to "Page Not Found"

You can import the private keys from your MultiBit wallet, but you will have to keep a backup of those keys separately as they won't become part of your Electrum seed.
The best way to switch to Electrum is to send all the bitcoins you have on your old wallet to one of the addresses on your Electrum wallet. This way you'll have all your bitcoins secured with your seed.


text:
create a node, issue new money
I represent an organization that belong to a network of not for profit organizations placed all around the world. We have a centralised office which register internal transactions between members. Each member of the network has an account number. Everything is done manually.
When I read about the Bitcoin I thought we could create a node of the network in order to issue new money and keep track of transactions. We could fix our exchange rate of 1 BITCOIN equal to 1 Euro.
Do you think this is possible?

I'm not sure I understand what you want, but I'm sure you have completely misunderstood how bitcoin works.

possible duplicate of What client to use for an alternative special purpose blockchain?

Can you be more precise about what you're trying to do? Are you trying to set up your own mechanism to efficiently move a private-label currency?

Not really. If you want to establish a fixed exchange of Bitcoin, it would be pretty hard. You could set the minimum, but the maximum would be up to the sellers. Similarly if you'd want to establish a fixed rate for gold.
You could create an alternative cryptocurrency owned only by yourself, but it would be way too much hassle for what you need.
If you want to use Bitcoin for tracking transactions, it can also be a bit problematic. It all depends on what do you want to do exactly. Bitcoin will definitely be a way to cut costs of international transactions and allow transparent tracking.
All in all, issuing your own money with Bitcoin is very hard, using Bitcoin as a means of exchanging money is very useful. If you want some more specific help, please describe your problem or current use in more detail.

Bitcoin sacrificed a lot of things to be completely decentralized. Among other things, that means it's not possible for a single entity to "issue" Bitcoins, nor is it really possible to use a pegged exchange rate.
You can certainly design an electronic currency that you back with Euros. You can issue the currency to people who pay you Euros and allow people to redeem the currency for Euros. There are all kinds of trust and security issues, but it's doable. However, you should totally ignore the way Bitcoin does pretty much everything. None of that is suitable for a model where one entity guarantees that the currency will be redeemable.
For one thing, the Bitcoin model secures transactions with computing power, and if you couldn't get enough of it, your currency wouldn't be secure. Since you're pegging the currency to the Euro, that would mean you would have to pay out Euros to those who secure the currency. That's not going to be much fun for you.


text:
listreceivedbyaddress missing transactions after restore
After restoring wallet from wallet.dat balance is identical,
but listreceivedbyaddress 0 do not have the transactions associated with wallet.
listunspent 0 show all the transactions, but it will not list anymore after spending.
is it a bug? or do i have to perform some special scan? rescanblockchain found all the transactions, but they are still missing in the listreceivedbyaddress.
EDIT #1
Is there a way to get all the transactions associated with specific address, what will get accurate result from blockchain? Was supposed to be bellow command what is not working sometimes.
listreceivedbyaddress 0 true true {ADDRESS}

EDIT #2
Output of getaddressinfo {ADDRESS} differences between recent, and backup wallet:
+  "solvable": true,
+  "desc": "sh(wpkh([6f8c0001/0'/0'/306']034e{SCRAMBLED_LONG_CODE}e50f))#ejlphmc0",
+  "ischange": true,
-  "hdmasterkeyid": "f9248{SCRAMBLED_LONG_CODE}53587b4",
+  "hdmasterfingerprint": "6f8c0001",
   "labels": [
-    {
-      "name": "default_wallet",
-      "purpose": "receive"
-    }
   ]

If you run getaddressinfo on the address whose incoming tramsactions are missing, what does it say about being change?

@PieterWuille, getaddressinfo shows that restored wallet consider address as "change"

Can you give an as detailed history of how you got into this state? It's possibly a bug that this address is being considered change.

I've had the same issue after restoring from wallet.dat. It's probably a bug described in https://github.com/bitcoin/bitcoin/issues/20293. Possible way to fix it is by setting an empty label to the address that became change:
bitcoin-cli setlabel "{address}" ""

FYI, this bug should be fixed in the next release :)


text:
Can't perform remote JSON RPC to my Bitcoin node (C#)
I'm trying to perform a remote JSON RPC. It works fine using localhost but not if I try to do it from another computer.
Here's what I have so far:
            int index = 0;
        HttpWebRequest webRequest = (HttpWebRequest).Create("http://fakeip:8332");

        webRequest.Credentials = new NetworkCredential("fakeuser", "fakepw");
        /// important, otherwise the service can't desirialse your request properly
        webRequest.ContentType = "application/json-rpc";
        webRequest.Method = "POST";

        JObject joe = new JObject();
        joe.Add(new JProperty("jsonrpc", "1.0"));
        joe.Add(new JProperty("id", "1"));
        joe.Add(new JProperty("method", "getinfo"));
        // params is a collection values which the method requires..
        Dictionary<string, string> Params = new Dictionary<string, string>();
        if (Params.Keys.Count == 0) {
            joe.Add(new JProperty("params", new JArray()));
        } else {
            JArray props = new JArray();
            // add the props in the reverse order!
            for (int i = Params.Keys.Count - 1; i >= 0; i--) {
            }
            joe.Add(new JProperty("params", props));
        }

        // serialize json for the request
        string s = JsonConvert.SerializeObject(joe);
        byte[] byteArray = Encoding.UTF8.GetBytes(s);
        webRequest.ContentLength = byteArray.Length;
        Stream dataStream = webRequest.GetRequestStream();
        dataStream.Write(byteArray, 0, byteArray.Length);
        dataStream.Close();

        WebResponse webResponse = webRequest.GetResponse();

which gives this exception:
    Unable to connect to the remote server ---> System.Net.Sockets.SocketException:
A connection attempt failed because the connected party did not properly respond
after a period of time, or established connection failed because connected host
 has failed to respond fakeip:8332

I've tried using curl too:
   curl --data-binary
    '{"jsonrpc":"1.0","id":"curltext","method":"getnetworkinfo","params":[]}'
-H 'content-type:text/plain;' http://fakeuser:fakepw@fakeip:8332

curl: (7) Failed to connect to fakeip port 8332: Timed out

My bitcoin.conf:
testnet=0
server=1
rpcuser=fakeuser
rpcpassword=fakepw
rpcallowip=0.0.0.0/0
txindex=1

Anyone have any ideas?
Edit: rpcbind allows me to make the RPC query using C# but the curl method is still not working.
Running the following:
curl --data-binary
'{"jsonrpc":"1.0","id":"curltext","method":"getnetworkinfo","params":[]}' -H
 'content-type:text/plain;' http://fakeuser:fakepw@fakeip:8332

results in:
{"result":null,"error":{"code":-32700,"message":"Parse error"},"id":null}

I get the same error for any command.

You'll need to specify an IP address to bind the RPC interface to. Since Bitcoin Core 0.18, it's not sufficient to have "rpcallowip=*", as it will by still default still only bind to localhost.
So add:
rpcbind=fakeip

Hi Pieter, big fan. The C# code is working now but I'm getting error code -32700 when using curl. Any idea why? If not, using C# should be enough for what I want to do. Thanks a lot.

The getinfo command was deprecated in version 0.16, and subsequently removed. Use the more specific getblockchaininfo, getpeerinfo, getnetworkinfo, getwalletinfo, ... instead.

Oh yeah, I figured that out. The error is the same for any command. The error message is "Parse error".

Can you update your question with the exact command and output?

I tried your command on my system and it works fine.


text:
Is SegWit similar to/ derived from Hyperledger Fabric?
Segregating the witness (signature) data from the transactional data is similar to the structure used in Hyperledger Fabric.
Curious about the derivation of SegWit?

I know almost nothing about Hyperledger, but maybe this helps writing an answer: Segregated Witness (on a Bitcoin-like system) was first deployed in Elements Alpha, released in May 2015, while the Hyperledger project only started in December 2015. It's also a very straightforward design (and arguably Bitcoin's legacy one was the weird one), several from scratch projects never considered anything else

Thanks for your valuable response. It clears out everthing.

OP clearly felt his question was adequately answered by Pieter Wuille's comment:

Segregated Witness (on a Bitcoin-like system) was first deployed in Elements Alpha, released in May 2015, while the Hyperledger project only started in December 2015. It's also a very straightforward design (and arguably Bitcoin's legacy one was the weird one), several from scratch projects never considered anything else

Is SegWit derived from Hyperledger Fabric?

No, the dates above make this very unlikely.


text:
how to enable wallet in a confirmed node
I installed/compiled my Bitcoin Core RPC client version v0.16.2.0 and i used --disable-wallet flag.
I think this is the reason I am seeing "Method not found" errors when trying to GetNewAddress
how can i install/enable it after having fully confirmed node with up to date blockchain?
os: Ubuntu 16.04.5
solution
accepted the answer below but in my case since i set it up sing apt-install bitcoind the wallet feature was already there, i just needed to flip the disablewallet=1 flag in my bitcoin.config

Yes, that is a wallet feature. You would have to recompile without the --disable-wallet flag, starting back at executing the autotools configure script: ./configure. See https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md#disable-wallet-mode.
You also have to make sure you don't set the -disablewallet flag when you run bitcoind or have it in your bitcoin.conf file.
If you need to recompile, you could still back up the data in the .bitcoin folder, which includes the blockchain and UTXO set and copy it back over when the installation is complete. That way you don't have to download the blockchain again.

hmm disablewallet is a setting in my bitcoin.config. after setting it to 0, how do i 'recompile' it ?  there was no strict compilation process when i set it up

-disablewallet is a runtime flag, which is different than the --disable-wallet compile flag. So did you compile it or not? Did you run ./configure --disable-wallet when you first set it up? Instructions to compile are here: https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md

also not sure what ./configure refers to.. i don't have that dir..

in installed with apt-get install bitcoind

That is autotools configure script, it scans your machine for dependencies and configures the build process.

Oh ok, so you didn't compile. Just remove thedisablewallet line in bitcoin.conf and restart bitcoin $ bitcoin-cli stop then $bitcoind -daemon and see if it has the wallet functionality.

Let us continue this discussion in chat.


text:
running live and test node simultaneously
it took me 4 days to fully synchronize my node, but its finally running and i can create wallets etc..
I wanted to experiment with some transactions on testnet, so need to either change my config or create a separate instance.
But I don't want to lose my synchronized live node, so what's the best strategy to switch to testnet while still being able to switch back?
# [core]
datadir=/mnt/volume-nyc1-03/Bitcoin
dbcache=2000
txindex=1

# [rpc]
# Accept command line and JSON-RPC commands.
server=1
# Username for JSON-RPC connections
rpcuser=bitcoin
# Password for JSON-RPC connections
rpcpassword=sayblockchainonemoretime
rpcport=8332
rpcallowip=x.x.x.x/x (ip masked)
# [wallet]
# Do not load the wallet and disable wallet RPC calls.
disablewallet=0

You can run both at the same time:
$ bitcoind -daemon
$ bitcoind -testnet -daemon

Then you can issue commands on either using:
$ bitcoin-cli <commmand>

or
$ bitcoin-cli -testnet <command>

The blockchains are stored in ./bitcoin/blocks for mainnet and /bitcoin/testnet3/blocks for testnet.
Update: With the bitcoin.conf updated in the question remove rpcport=8332, so it doesn't try to use the same port for mainnet and testnet RPC.
Update 2: With 0.17 release, it is now possible for a single configuration file to set different options for different networks in bitcoin.conf:
main.uacomment=bitcoin
test.uacomment=bitcoin-testnet
regtest.uacomment=regtest

For more information, see the 0.17.0 Release Notes

$ bitcoin-cli getblockcount: 538439

$ bitcoin-cli -testnet getblockcount: 538439

looks like its going against main

hmm i do see the testnet3 folder under bitcoin now ..

Hmm. might be an issue with your bitcoin.conf

seeing this in debug.log
2018-08-25 21:08:24 Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
2018-08-25 21:08:24 Using 2 threads for script verification
2018-08-25 21:08:24 scheduler thread start
2018-08-25 21:08:24 Binding RPC on address :: port 8332 failed.
2018-08-25 21:08:24 Binding RPC on address 0.0.0.0 port 8332 failed.
2018-08-25 21:08:24 Unable to bind any endpoint for RPC server
2018-08-25 21:08:24 Error: Unable to start HTTP server. See debug log for details.

You might want to restart the daemons using $bitcoin-cli stop and $bitcoin-cli -testnet stop and then start again.

hmm tried that  but keep seeing Binding RPC on address 0.0.0.0 port 8332 failed.

Is something using that port?

so i set testnet=1 in bitcoin.conf which seems to work but now the main node is only running test i think. not sure if i can run both at the same time

Clear that line. If you only put the flag in the command line you can run both.

thanks! good to know :) for some reason w/out the line testnet wasn't working for me though.. keep getting that error in debug.log

Is it still not working? Would be helpful to post bitcoin.conf

Let us continue this discussion in chat.

Run bitcoin.exe with conf and datadir params
And set the configuration file with testnet=1

i saw the config and am aware of these settings. but not sure how to ensure that my live blockchain stays untouched

By default the two networks will operate in different directories.


text:
How do I make a Bitcoin "memory wallet" (or paper wallet without paper)?
I'd like to see a video tutorial that explains in detail how to make a paper wallet without paper, AKA, a "memory wallet" -- thx.

this is a dangerous thing to do because if you forget, you will loose your bitcoins forever

Related question: How to save bitcoins as paper

step 1 - memorize all the private keys that would otherwise be stored in some sort of wallet file
step 2 - there is no step 2

You can use http://brainwallet.org to generate a so-called "Memory Wallet". Do not use the live website, download the sourcecode from github.
How this works is you select a phrase, like "Correct Horse Battery Staple", and that phrase will consistently compute to a keypair you can use to store and later retrieve bitcoins.
BE EXTREMELY CAREFUL, or maybe don't do this at all. There are untold numbers of robots perpetually checking the blockchain for any conceivable passphrase. If your passphrase is not EXTREMELY strong, any coins you put in there will likely be swiped in a matter of hours, if not minutes.
See: http://www.reddit.com/r/Bitcoin/comments/1ndsxi/a_test_of_brainwallet_passphrases/
If you really want to make a brain wallet, I recommend something absolutely impossible to brute force, like
my favorite superhero, childhood address, my social security number
giving you
Captain Underpants, 112 Elm Street, Oakland, CA, 555-1212-555

This is most secure way to place your Bitcoins in cold storage.
This quick Video Tutorial explains how to make
a Dr. Evil Paper Wallet or Bitcoin "Brain Wallet."
Get the ZIP file for Dr Evil Paper Wallet Generator @GitHub
https://github.com/obesityspray/Dr.-Evil-Paper-Wallet

Another option without the paper is using:
Memory Paper Wallet: https://xcubicle.github.io/memorypaperwallet
It's a fork of the original warp wallet code base: https://github.com/keybase/warpwallet with added features:

Generate multiple crypto addresses besides Bitcoin.
Printable and easily use for gifting occasions.
Ability to split a private key using shamir secret sharing.

Well, the brain wallet is essentially the mnemonic.
You can try using the BIP 39 tool. But keep in mind to save the code from Github and run it while Offline. Preferrably in a Tails OS or some disposable system/hardware (one can never be too paranoid).
After that, just keep those words in your head. When you need to interact with the wallet you download some standard wallet and while on setup, hit "recover" and insert the mnemonic that is in your brain.

% echo -n "Your Crrrappy Hi Entoppi 8rain 5ollet Praize" | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new
captain license mother demise merge must soldier good exit very wheat prefer useful congress riot airport dune ghost purpose sausage hospital myth hole federal,  and don't forget to use a BIP 39 passphrase to complement the your unique quoted echo string.


text:
bitcoin rpc api createrawtransaction with python returns -3: Expected type array, got object
experimenting with JSON RPC API using bitcoinrpc.authproxy for python.
so far it works in all instances accept when i try to create a raw transaction.
somewhere along the way im passing parameters in wrong format.
i created helper methods to create my inputs/outputs like this:
def getTransactionJsonInput(self, sourcetransaction) -> str:
    return {"txid": sourcetransaction['txid'] ,"vout":sourcetransaction['vout']}

def getTransactionJsonOutput(self, new_address, amount, returnaddress, returnamount) -> str:
    return {
        new_address: amount,
        returnaddress: returnamount
    }

and when i print my call to createrawtransaction it appears to be correct
    print('TRANSACTION inputs', [input], type([input]))
    print('TRANSACTION outputs', output, type(output))
    return svc.createrawtransaction(input, output)

TRANSACTION inputs [{'txid': 'sourcetransactionid', 'vout': 1}] <class 'list'>
TRANSACTION outputs {'destinationaddress': 0.012, 'changeaddress': 2.56559703} <class 'dict'>

error:

bitcoinrpc.authproxy.JSONRPCException: -3: Expected type array, got object

solution
thanks to @m1xolyd1an i was steered in the right direction. All that was missing in my implementation was adding [] around inputs. i added them in print statement but not in the actual call!
before
return svc.createrawtransaction(input, output)

after
return svc.createrawtransaction([input], output)

new result:

new transaction:  0200000001005134d493cb536a5000b491b0b6dbca6c5187eec67ffb8769fdedac9ec25db40100000000ffffffff02804f12000000000017a914190150e1a961a2509b1d48c4d6cf6b644454cd968757ca4a0f0000000017a91496711dceda7240a6c492b2fabc5d44ab5d4bd19e8700000000

I think you should use double quotes in input like [{"txid":"id", "vout":n}]

@AbhishekSinha i tried that initially, but then it was '"xxxx"' also the error was the same

You are providing it an Object in JSON, not an array just as the error states.
 return {"txid": sourcetransaction['txid'] ,"vout":sourcetransaction['vout']}

I would try creating it from a dictionary and then adding it to a list instead of building it from a string, something like:
>>> inputs = {}   #create the dictionary
>>> inputs["txid"]=sourcetransaction['txid']  #add the value to the dictionary key 'txid'
>>> inputs["vout"]=sourcetransaction['vout']  #add nout value to key 'vout'

>>> inputs
{'txid': 'a9d4599e15b53f3eb531608ddb31f48c695c3d0b3538a6bda871e8b34f2f430c', 'vout': 0}

>>> inputsList=[inputs] #add dictionary input to list
>>> inputsList
[{'txid': 'a9d4599e15b53f3eb531608ddb31f48c695c3d0b3538a6bda871e8b34f2f430c', 'vout': 0}]

Also noted in the example from the source:
def createrawtransaction(self, inputs, outputs):
    """
    Creates a raw transaction spending given inputs
    (a list of dictionaries, each containing a transaction id and an output number),
    sending to given address(es).
    Returns hex-encoded raw transaction.
    Example usage:
    >>> conn.createrawtransaction(
            [{"txid": "a9d4599e15b53f3eb531608ddb31f48c695c3d0b3538a6bda871e8b34f2f430c",
              "vout": 0}],
            {"mkZBYBiq6DNoQEKakpMJegyDbw2YiNQnHT":50})
    Arguments:
    - *inputs* -- A list of {"txid": txid, "vout": n} dictionaries.
    - *outputs* -- A dictionary mapping (public) addresses to the amount
                   they are to be paid.
    """
    return self.proxy.createrawtransaction(inputs, outputs)

thanks! yeah i figured im just still clueless how python typing works .. :)  trying it this way

thanks! looks like the string is still ok, i just forgot to include []'s around input when calling the actual api


text:
json-rpc api works locally but getting "Connection Refused" remotely
code:
from bitcoinrpc.authproxy import AuthServiceProxy
class ProxyService:
    def getProxy(self) -> AuthServiceProxy:
        return AuthServiceProxy("http://%s:%s@xx.xx.xx.xx:8332" % ("xx", "xxx")) # this only works locally as 127.0.0.1

bitcon.config
datadir=/xxxx/Bitcoin
dbcache=2000
txindex=1
server=1
rpcuser=xx
rpcpassword=xx
rpcport=8332
disablewallet=1

I am running this on Ubuntu droplet. connection works when i connect locally as 127.0.0.1:8332
but when i try connection to the ip of my node (locally or remotely) I get "connection refused"
also i can confirm my node is up

Make sure to set
rpcallowip=<ip>

for each of your remote IPs, to whitelist them.
You can also use a netmask instead of IP. If you have a dynamic IP, use this feature!
For example, if your laptop's IP always starts with 237.94, you can set it to
rpcallowip=237.94.0.0/255.255.0.0

Then it'll allow all IPs that start with those numbers. Make sure to set a really strong password, though.


text:
Bitcoin core setup how to prune without downloading?
I want to play around with code to learn about Bitcoin such as synchronizing the chain locally, verifying the Merkle tree, creating a new wallet programmatically.
from what i've researched I will need to prune the initial blockchain, but i don't see the option in setup.

is there some other way i can install it [with prune] given my low available storage?
update
i pressed ok, and added the config file afterwards.
I am not sure if this config is used, and the GUI is giving me an error when i try to open it.

@Cluster2k i guess i should have mentioned why i am installing this. I want to play around with code to learn some things like synchronizing the chain locally, verifying the Merkle tree, creating a new wallet programmatically.
I simply don't have anywhere near the 200GB of space on my laptop it is askingn me for :(

Yes you can run in prune mode, set prune=550 (or a number >=550, which is a size in MB). As blocks are downloaded others will be pruned to free up space as needed so you'll be fine with 17GB.
You can set the prune values in bitcoin.conf or as a startup parameter.
More details here https://coinguides.org/bitcoin-blockchain-pruning/

not having much luck. according to https://coinguides.org/bitcoin-blockchain-pruning/ the configuration file on a mach is supposed to be here:

$HOME/Library/Application Support/Bitcoin/

but /Bitcoin folder was not created.  Would I need to install the bitcoin wallet first ? I just downloaded Bitcoin Core. Seems all these help pages are for the wallet

You downloaded Bitcoin Core for mac and installed it? Maybe say what steps you've done or share screen shots, etc. Maybe check in /Users/<your user name/Library/Application Support/Bitcoin or use Spotlight search.

updated my question with latest state please take a look

You have a bitcoin.conf file in the file listing. That is the configuration file. Did you edit it manually at all? Maybe something is wrong with it. Not sure about that error you show. How are you starting bitcoin? You could try posting the bitcoin.conf file (blank out any passwords or IP addresses, etc).

not sure, it was acting weird on Mac :/ i have since created a droplet on Digital Ocean and did this on Ubuntu and it was a more straight forward process for me

If you can't locate the config file, you can try to launch the client from the terminal window.  Go to the folder where the executable is installed, and launch with ./bitcoind --prune=550.  All other settings will come from the default .conf file.

thanks, but if prune needs to be supplied via command flag, how do i know the other settings are used from config ?

It doesn't "need" to be supplied there, but it's an option.  And I believe that once you set it to prune, it has to keep pruning going forward.  I've never tested this.  If you prune and then start without prune, does it keep pruning?  I don't know.


text:
HTLC in blockchain regtest mode by creating transactions manually
I am trying to create a htlc smart contract manually by writing a script and  finding the hex using the btcc compiler. Now node A has to create a transaction using that hex and lock money in that p2sh address for upto 150 blocks. If node B can provide the secret value before 150 blocks then he can claim the coins and if not the it will go back to node A. I am new to this concept and where ever i read about creating transactions they are explaining in a theoretical way and I could not understand what to do the redeem script hex and how to lock money and how will node B claim it before 150 blocks and how will the raw transaction created by node A will be accessible to node B. Can you please give a reference or steps to follow to understand and implement it?
Which documentation should i refer for doing it and what commands should be used and how it should be used?
How to create the p2sh address. And another problem i am facing is that i am getting different hash160 everytime in different website. Not sure which one is accurate. Please guide me.
P.S:  I am running bitcoin in regtest mode.

Are you looking for a script that you can use or Bitcoin-core commands to create such a transaction?

commands to create such transaction

I have created the redeem script hex , now i don't know what to do with that.

Check @pieterwuille answer over here: https://bitcoin.stackexchange.com/questions/80902/transaction-still-unspendable-with-cltv. Bitcoin Core by default does not support signing transactions with CLTV scripts

I want to know how to create a transaction and how to sign, send and claim it step by step and what are the required fields for doing that. And which privkey should be used and which address should be used. later when the recipient has to claim, then where should he produce the preimage to claim the money.

sending a transaction to that htlc script is simple. Just use the standard Bitcoin core commands to send it to a P2SH address. However, it is while spending such script that issue arises in Bitcoin Core as it does not support dog ing transactions with CLTV scripts.

Which commands do I use. How to put the condition. I mean where? Where should i put the locking and unlocking script , how to spend it. Suppose we don't focus on the cltv scripts and if it is a basic script,  What do I do with it. How to get a p2sh from   the redeem script and how to proceed.


text:
Would proxying a lightning hot-wallet with a lightning cold-wallet make the hot-wallet safer?
(Edit: I realize how Mt.Gox-y the below setup sounds, but I'm coming at it from a "lightning routing problem perspective", not necessarily a straight "bitcoin security perspective"...though...those two things overlap quite a bit.)
The setup/problem:
I have two lightning nodes: (A) a Casa lightning node ("Casa-Node") that I trust in my apt, and (B) a BTCPayServer lightning node hot-wallet ("hot-wallet node").
BTCpayServer makes it REALLY easy to integrate a btc and lightning payment pipeline into my website, however the btcpay website has a warning the lightning wallet is a hot wallet, unsafe(ish) :(
I'd like to use my Casa node with BTCpayServer but they (casa) don't have that functionality yet. I don't want to simply put thousands of dollars on my BTCpay LN hot-wallet, as I don't trust it. The think of the casa node = trustworthy-node, and the Hot-wallet = less trustworthy-node.
A potential solution?
Came up with this idea: use my Casa Node as my primary routing node with payments coming in from strangers/customers, and have only one channel between my Casa Node and my Hot Wallet Node..so all LNpayment requests generated by the Hot Wallet Node get routed through the Casa Node, and I can easily balance the Casa/HotWallet, channel balance as needed (at the end of the day, move the funds back to the Casa Node and close that one channel out on-chain).
Example flow:

CasaNode = CN
BTCPAY Hot-wallet = HWN
Other nodes trying to pay HWN = L1, L2, L3

Step 1: Setup my website with BTCPay server, and enable HWN for the lightning payment pipeline. (assume my website is CatToys.info, and I sell four different $25 dollar items..made up but it helps the picture).
Step 2: Create a big LNchannel between my CN (more secure) and HWN (less secure storefront node). E.g., create a $1,000 payment channel between my Casa Node and the BTCPay Lightning Hot Wallet Node.
Step 3: Create a bunch of random payment channels between CN and other big nodes in the lightning network. These random payment channels increase the odds of a routing path being found between a given website customer's LN node and HWN node that generated the LNrequest (e.g., the BTCpay server that is generating the lightning payment request).
Step 4: Random customer browses CatToys.info, checks out via, which causes BTCPayserver to generate a lightning payment request for $25 via the HWN node.
Step 5: Customer uses his lightning node (L1) to pay HWN...the routing proceeds as follows:

$25 (in btc) from L1 --> CN --> HWN.

Other customer use the same flow:

$25 (in btc) From L2 --> CN --> HWN
$25 (in btc) From L3 --> CN --> HWN

Step 6: At the end of the day, HWN has 75 dollars (from the three customers paying 25 each for a cat toy). When I want to "close the store down" (which is a brick/morter thing to say, but just stay with me), I move all the funds in the CN--HWN channel back to CN, so the full $1000 is back on CN (i.e., move $75 back to CN), then close the channel out on-chain.
Step 7: if I need to "open the store" for the next day, I just need to recreate a single channel from CN---HWN, which is cheap as it's only a single channel to open up, and CN is ready to go (always connecting to lots of nodes (and CN is trust worthy)).
This approach would: let me use BTCPay server's hot wallet (or any wallet that I don't trust too much) with larger-ish funds, and give my hot-wallet channel instant liquidity by connecting to CN, and I would only need to worry about closing/opening/rebalancing a single lightning payment channel between CN----HWN, instead of trying to constantly open channels with different amounts of funding to keep my website enabled to receive lightning payments.
Thoughts? Would the above sequence work in the lightning network? And be safe/secure, and would payments actually be successfully routed to the website okay?
Any input appreciated, thx.

First of all at this stage all lightning network nodes implementations are hot wallets. There are people / teams working on hardware wallets in lightning. Also I don't get your base assumption why you would trust casa which I believe  is not open source more than trusting btc pay server which I believe is open source. The only reason I see is that you control the hardware of your casa node and it is not some machine somewhere on the web in some datacenter.
But let us assume your setup and assumptions are valid then in step 6 and 7 of your process you don't need to close the channel. You could just withdraw the funds from your HWN to your casa node. As the entire channel balance is now at casa side there is no risk with HWN being compromised. This is as good as closing a channel moving funds and create a new one but saves onchain tx and fees.
In step 3 it is important to note that if you casa ode owns those payment channels your casa node can't receive funds or error payments to the HWN. Casa needs inbound capacity. So either you have others open channels to you (e. G. Your customers) or you have to use one of the many submarine swap services that exchange onchain to offchain funds. Such a service could also be used in step 6 and 7.
Overall the node of btc pay server could be private and configured to only accept traffic on port 9375 from your casa node. In this way you make sure only you have a payment channel with it. Since without rendez vous Routing all invoices of your store point to your casa node your cars node might now be at risk.
That all being said it seems to me that for the concerns that you have you don't need this setup with a proxy node. I am not saying that such a setup might not make sense for other reasons.

This is a fantastic detailed answer, thank you for answering. I'm still parsing the many points/suggestions.   I think I misunderstood something from the get-go: that all lighting wallets are currently hot wallets (I devised the above plan under the assumption that the Casa lightning wallet was not a hot wallet, or inherently safer..which is not necessarily true as you noted). . . . Very good suggestions, thanks again!

If you are happy / satisfied with the response you can mark it as correct next to the voting points. This signals to others that the answer is answered. I am happy I could help


text:
Original Bitcoin (Core), how did it generate private key?
I'm a newbie at this, but I have a question.
How did the original Bitcoin (Core) v0.1 program generate random private keys? I was hoping for the exact source code or at least a good description of the method used.
I understand (almost) any 32 byte integer will work as a private key. I assume they used a Windows based OS random generator CryptGenRandom() since it was released for Win XP initially, I think.
Thanks!

The Bitcoin software has always used OpenSSL's random number generator in some way to generate private keys. Originally, it just used OpenSSL's EC_KEY_generate_key function to generate a new key and OpenSSL would handle the RNG and object creation. Even now, Bitcoin Core still uses OpenSSL's RNG and mixes it's output with several other sources of randomness in order to generate private keys.


text:
How many blocks per second can sustainably be created using a time warp attack?
In a time warp attack, an attacker can reduce proof of work difficulty to its minimum, which allows even a single piece of modern mining equipment to create large numbers of blocks per second.  However, these blocks must follow the median time past consensus rule of each block having a block header time greater than the median of the previous 11 blocks' header times.  The blocks also can't use a header time more than two hours in the future if they want to be accepted immediately by full nodes.
The block header time field is an integer representing seconds since the Unix epoch.
Given those constraints, what is the greatest number of blocks per second that an attacker can produce?  It's clearly possible for them average one block per second, but I've heard that they may be able to produce more than that.

An attacker would be able to sustain a cadence of 6 blocks per second without increasing the difficulty.
The attacker would start by holding the timestamps back by more than 14 days to erode the difficulty. Once the difficulty was reduced to the minimum, they would be able to increase the timestamp only every sixth block, keeping all blocks’ timestamps at least two weeks in the past except for the timestamp of each last block per difficulty period, which they would timestamp with the present time or up to 2h into the future at least 14 days ahead of the first block in the same period.
While the timestamps eventually settle on a series of the same value repeating six times before incrementing it, there is a quirk in how the timestamps sort to find the median when timestamps are first shifted to increasing minimally.
Assuming we start with 11 blocks spaced at 10 minutes, the subsequent minimum timestamps would be as follows:

Block height
Timestamp [s]
Timestamps of prior 11 blocks (median bold)

1
0
-

2
600
-

3
1200
-

4
1800
-

5
2400
-

6
3000
-

7
3600
-

8
4200
-

9
4800
-

10
5400
-

11
6000
-

12
3001
0, 600, 1200, 1800, 2400, 3000, 3600, 4200, 4800, 5400, 6000

13
3002
600, 1200, 1800, 2400, 3000, 3600, 4200, 4800, 5400, 6000, 3001

14
3003
1200, 1800, 2400, 3000, 3600, 4200, 4800, 5400, 6000, 3001, 3002

15
3004
1800, 2400, 3000, 3600, 4200, 4800, 5400, 6000, 3001, 3002, 3003

16
3005
2400, 3000, 3600, 4200, 4800, 5400, 6000, 3001, 3002, 3003, 3004

17
3006
3000, 3600, 4200, 4800, 5400, 6000, 3001, 3002, 3003, 3004, 3005

18
3007
3600, 4200, 4800, 5400, 6000, 3001, 3002, 3003, 3004, 3005, 3006

19
3007
4200, 4800, 5400, 6000, 3001, 3002, 3003, 3004, 3005, 3006, 3007

20
3007
4800, 5400, 6000, 3001, 3002, 3003, 3004, 3005, 3006, 3007, 3007

21
3007
5400, 6000, 3001, 3002, 3003, 3004, 3005, 3006, 3007, 3007, 3007

22
3007
6000, 3001, 3002, 3003, 3004, 3005, 3006, 3007, 3007, 3007, 3007

23
3007
3001, 3002, 3003, 3004, 3005, 3006, 3007, 3007, 3007, 3007, 3007

24
3008
3002, 3003, 3004, 3005, 3006, 3007, 3007, 3007, 3007, 3007, 3007

25
3008
3003, 3004, 3005, 3006, 3007, 3007, 3007, 3007, 3007, 3007, 3008

26
3008
3004, 3005, 3006, 3007, 3007, 3007,  3007, 3007, 3007, 3008, 3008

27
3008
3005, 3006, 3007, 3007, 3007, 3007, 3007, 3007, 3008, 3008, 3008

28
3008
3006, 3007, 3007, 3007,  3007, 3007, 3007, 3008, 3008, 3008, 3008

29
3008
3007, 3007,  3007, 3007, 3007, 3007, 3008, 3008, 3008, 3008, 3008

30
3009
3007, 3007, 3007, 3007, 3007, 3008, 3008, 3008, 3008, 3008, 3008

this causes the minimum timestamp to increment one second each block at first before it settles to incrementing only after repeating six times.

6 blocks per second can be sustained.
Satisfying the MTP rule
The timestamp of each block must be strictly larger than the median of the timetamps of the 11 blocks before it.
This can be satisfied using a pattern consisting of repeating the same timestamp 6 times, then incrementing by one second, and repeat again.
So the series would be:
... 0 0 0 0 0 0 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 3 ...

We can abstract over the actual values. With that, there are just 6 distinct "blocks", based on what the 11 timestamps before it were:

T+0 T+0 T+0 T+0 T+0 T+1 T+1 T+1 T+1 T+1 T+1: MTP=T+2
T+0 T+0 T+0 T+0 T+1 T+1 T+1 T+1 T+1 T+1 T+2: MTP=T+2
T+0 T+0 T+0 T+1 T+1 T+1 T+1 T+1 T+1 T+2 T+2: MTP=T+2
T+0 T+0 T+1 T+1 T+1 T+1 T+1 T+1 T+2 T+2 T+2: MTP=T+2
T+0 T+1 T+1 T+1 T+1 T+1 T+1 T+2 T+2 T+2 T+2: MTP=T+2
T+1 T+1 T+1 T+1 T+1 T+1 T+2 T+2 T+2 T+2 T+2: MTP=T+2

Which in each case continues the pattern.
Keeping the difficulty low
Using the pattern above naively would result in the difficulty adjustment period quadrupling the difficulty every period (the maximum allowed).
To prevent it from doing so, it is necessary to exploit the timewarp bug.
The winning strategy involves setting the timestamp of every block exactly 2 weeks in the past, except those blocks whose height modulo 2016 equals 2015; those are given the current time (or up to two hours in the future). This will make the adjustment algorithm see every period as having taken 2 weeks, and thus not cause any adjustment.


text:
Taproot signing with PSBT: How to determine signing method?
As far as I can tell there are three ways you can produce a valid Schnorr signature in a Taproot environment:

For a p2tr output that uses a BIP-0086 commitment (key spend, sign with internal key, tweak with h_tapTweak(internalKey)*G)
For a p2tr output that commits to a script tree root hash (key spend, sign with internal key, tweak with h_tapTweak(internalKey || scriptRoot)*G)
For a p2tr output that commits to a script merkle tree root hash (script spend, sign with any key to satisfy leaf script)

How do I find out which of the above signature "methods" is expected to be produced by a PSBT signer?
The updated BIP-0174 document only mentions all the new Taproot related fields for the inputs and outputs but does not mention how to interpret them and what combinations of values are valid.
I found some test vectors in https://github.com/bitcoin/bitcoin/pull/22558 that gave me some clues, but I'm not sure if I'm missing an important resource that describes the PSBT signing process in more detail?
This is what I was able to scrape together so far:

(p2tr BIP-0086 key spend): Set PSBT_IN_TAP_BIP32_DERIVATION with internal key and zero/empty leaf hashes.
(p2tr script root hash key spend): Set PSBT_IN_TAP_BIP32_DERIVATION with internal key zero/empty leaf hashes, set PSBT_IN_TAP_MERKLE_ROOT to merkle root hash.
(p2tr script spend): Set PSBT_IN_TAP_BIP32_DERIVATION with internal key and the leaf hash of the leaf that is being executed. Also set PSBT_IN_TAP_LEAF_SCRIPT with the control block and the script of the executed leaf.

Does that look correct?
If yes, then what is the significance of the PSBT_IN_TAP_INTERNAL_KEY field compared to using the internal key in the PSBT_IN_TAP_BIP32_DERIVATION field? It feels like I'm missing something here.

Yes, that looks correct.
I suppose you are correct that PSBT_IN_TAP_INTERNAL_KEY is not strictly necessary, but one of the main ideas with PSBTs is to include as much information as possible, and to be explicit what the information included is. So instead of inferring the internal key from PSBT_IN_TAP_BIP32_DERIVATION, we explicitly say what it is with PSBT_IN_TAP_INTERNAL_KEY.
It is also possible to have the internal key also be a key inside of a script (dunno why you'd want that, but it could happen). In that case, it would not be possible to infer which key is the internal key from just PSBT_IN_TAP_BIP32_DERIVATION.

Thank you for your answer! It makes sense to me to add as much information as possible. But when implementing the signer role it's useful to know what the minimum required fields are in order to perform some validation.

In your example, I assume that the same key would appear in the PSBT_IN_TAP_BIP32_DERIVATION field twice? Once with no leaf hashes and once with a leaf hash?

@guggero no, it would be there once with one leaf hash. A PSBT cannot have duplicate keys.

Ah, right, I mixed it up in my head with PSBT_IN_TAP_SCRIPT_SIG where the leaf hash is in the key along with the pubkey. Now it all makes sense to me, thanks a lot!


text:
Why was the RIPEMD-160 hash algorithms chosen before SHA-1?
Bitcoin uses both SHA-256 and RIPEMD-160 hashes. Most often a double-round SHA-256 is used, but for address generating, RIPEMD-160 is used because it generates a shorter hash value. RIPEMD-160 has a 160-bit or 20-byte hash value while SHA-256 has a 256-bit or 32-byte.
So RIPEMD-160 is used for it's shorter hash. However, SHA-1 also produces a 160-bit hash. RIPEMD-160 is a less popular algorithm but in fact achieves exactly the same as SHA-1 does.
The only real difference I can find on the internet is in the following fragment from RIPEMD-160's Wikipedia page:

RIPEMD-160 was designed in the open academic community, in contrast to
  the NSA designed SHA-1 and SHA-2 algorithms. On the other hand,
  RIPEMD-160 appears to be used somewhat less frequently than SHA-1,
  which may have caused it to be less scrutinized than SHA. RIPEMD-160
  is not known to be constrained by any patents.

Are patent issues the reason? Why is SHA-1 a problem but SHA-256 not? I know that SHA-1 and SHA-2 (of which SHA-256 is a part) are different iterations of the SHA initiative and so probably have very different legal implementation.
Does anyone really knows why RIPEMD-160 was chosen before the more popular SHA-1?

There are a multitude of reasons.

As @ThePiachu mentioned, there is a theoretical 2^60 bit attack that is possible on SHA-1, meaning that the algorithm is weaker than designed.

RIPEMD-160 was designed in the open academic community, in contrast to the NSA designed SHA-1 and SHA-2 algorithms.
It is worth noting that Satoshi could've used SHA2-256 twice and truncated the second digest to 160 bits as this is equally secure. The fact that he didn't is some evidence to show that his decision was a conscious decision to use RIPEMD-160 over the NSA suit of algorithms.

Lastly, I believe RIPEMD is based on a different design to the SHA1/SHA2 algorithms, and thus attacks that apply to one may not be transferable to the other (which is good).

Only the first is really an argument. The other two can be refuted by the fact that SHA-256 is used instead of another 256-bit hash.
SHA-256 is probably chosen for it's popularity, so why wouldn't SHA-1 be chosen for the same reason?

I don't quite agree. Just because one of the algorithms (SHA-256) is NSA-designed, doesn't mean that it still isn't a benefit to use an algorithm designed by another institution.

It is worth noting that Satoshi could've used SHA2-256 twice and truncated the second digest to 160 bits. The fact that he didn't I think is some evidence to show that Satoshi's decision was a conscious decision to use RIPEMD-160 over the NSA suit of algorithms.
(http://crypto.stackexchange.com/questions/3153/sha-256-vs-any-256-bits-of-sha-512-which-is-more-secure)

I don't say that he should have used SHA-1 because it's NSA-designed. Just saying that it's weird that he uses SHA-256 for a 256-bit hash but RIPEMD-160 for a 160-bit hash while the same family of hashes that SHA-256 is in has a 16-bit variant as well. (I know that SHA-1 and SHA-2 are not entirely from the same family, but they share the same name..)

It could be due to a theoretical 2^60 bit attack that is possible on SHA-1, meaning that the algorithm is weaker than designed. RIPEMD does not appear to have such weaknesses.

Hmm, I didn't know that. But still, chances are high that this theoretical flaw was found only because the algorithm is more popular than RIPEMD-160...

I'm looking into this again. The use of RIPEMD-160 in Bitcoin is not for security. Finding a collision would only allow you to find the public key of the address, which won't grant you access to the coins.
The security of the algorithm is not crucial for this case, so popularity would be the main deciding factor because it can strongly facilitate adoption.

@StevenRoose Actually, if you can generate a RIPEMD collision for an Address with a different private key that you control, you could spend all of the money of that address. We would have to revert to "pay to public key" payment algorithm.

Well, indeed. But collisions are generated by creating a specific input that will match a desired output. The inputs would be public keys. The public key from a private key is unpredictable and when you crack a hash function to generate collisions, I bet you want will do so with specific inputs, not random ones.

authors: Альфир Хазиев


text:
BIP-32 - how can parent extended public key exposed parent extended private key?
The below is the snippet from bip-32. I do not understand under what circumstance would 'parent extended public key' be exposed? In the case of e-commerce a 'child extended public key' is derived from 'parent extended public key'. The e-commerce website only needs to know 'child extended public key' to derive public key/addresses. In such case, will knowing 'child extended public key' expose 'parent extended private key'?
"One weakness that may not be immediately obvious, is that knowledge of a parent extended public key plus any non-hardened private key descending from it is equivalent to knowing the parent extended private key (and thus every private and public key descending from it)."
Example of code snippet in Javascript
function testXPub() {
 const mnemonic = 'ddddddddddddddddddddddddddddddddddddddddddddddd'
 const seed = bip39.mnemonicToSeed(mnemonic)
 const node = bip32.fromSeed(seed)
 const xpubNode = node.neutered()
 console.log("xpubNode :%o", xpubNode)
 console.log("xpubNode (string): %o", xpubNode.toBase58())

 const xpub_m_0_node = xpubNode.derive(0)
 console.log("xpub_m_0_node :%o", xpub_m_0_node)
 console.log("xpubNode (string): %o", xpub_m_0_node.toBase58())
 const xpub_m_0_0_node = xpubNode.derive(0).derive(0)
 console.log("xpub_m_0_0_node :%o", xpub_m_0_0_node)
 console.log("xpubNode (string): %o", xpub_m_0_0_node.toBase58())
}

So instead of sharing xpubNode, i would share xpub_m_0_0_node.

https://bitcoin.stackexchange.com/q/55383/7272

Not sure how is related to the suggested link? I am interested to understand as per bip32, why would anyone share the 'parent extended public key'? The use case for xpub I assume will be the 'child extended public key' derive from parent extended public key.

I am interested to understand as per bip32, why would anyone share the 'parent extended public key'?

The purpose of sharing the parent extended public key is so that child public keys (and thus addresses) can be derived from it without needing to know about a bunch of addresses individually. This is easier for separated setups where an online computer only has the public keys as the online computer does not need to be constantly refreshed with more addresses to watch for. It can generate the addresses itself.

In such case, will knowing 'child extended public key' expose 'parent extended private key'?

No. Only knowing both the parent extended public key and a child private key derived with unhardened derivation exposes the parent private key.

For the first question, wouldn't the online computer only needs 'child extended public key' to be able to generate new addresses without need of private key? I assume 'child extended public key' will be derive from another machine by another person using 'parent chain code', 'parent public key and non hardened index.

A child public key cannot derive its siblings. It can only derive its own children, so it itself becomes a parent public key.

I am only interested to understand if i have a child extended public key which is derived from extended public key, will the 'child extended public key' with any non-hardened private key descending from it susceptible for someone to know super parent extended private key? i.e. 'extended private key' --derive--> 'extended public key' --derive--> 'extended public key'

No, as I said in my answer, it is impossible to derive any parent keys from just child key, extended or not, public or not.

Sorry, Andrew, I am still not getting it. I have edited my answer with a sample code in Javascript. If you see from the above, I can derive many level of extended public key. Is that not true?

Yes, you can derive many levels. So? Each level is derived from the key at the level immediately preceding it. The only direct relation is between that key and its parent. There is no direct relation between a key and its grandparent, only an indirect one. Because all of the keys use an algorithm which is not reversible (it uses one way functions), it is impossible to derive a parent key given a child key. In order to get a grandparent key of a key, you need to get its parent key. But because that is impossible, so is getting the grandparent key.

Exactly, it is impossible to derive a parent key given a child key. However, as per https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki "One weakness that may not be immediately obvious, is that knowledge of a parent extended public key plus any non-hardened private key descending from it is equivalent to knowing the parent extended private key (and thus every private and public key descending from it).", if one share 'parent extended public key', you are susceptible for someone to hack to the 'parent extended private key'.

So I am curious why would someone ever share 'parent extended public key'? why don't just share xpub_m_0_0_node  as per my code, which is few level down the parent, hence not susceptible to being hack?

It is susceptible. m/0/0 is the parent of m/0/0/i. Given any private key at m/0/0/i and the xpub at m/0/0, you can drive the private key at m/0/0. Parent key does not just refer to the key at the root. Any key can be a parent key.

Agree, m/0/0 is hackable. Can the grandparent be hackable for the code example i.e. m/0 once m/0/0 private key is disclosed?

No. You can only recover the private hey if you have the xpub. So you need m/0's xpub in order to get it's private key once you have m/0/0's private key.


text:
Create wallet address on user registration for trading platform
I am creating a crypto-trading platform . I wanted to know how to create wallet address dynamically for all new registered user and also create new address for every transaction.
I am using bitcoind for bitcoin and will be doing RPC calls directly. Also let me know if there is any better way to do it.

Are you asking for an implementation or is it a theoretical question? Also will your application have access to the keys?

We'll need more information to specifically answer your question: what Bitcoin node software are you using? Are you connecting through a client library, straight RPC calls, or just using the command line?

@Motoma I am using bitcoind and will be doing direct RPC calls for now. Let know if there are any other better options.

@JBaczuk I am asking for implementation , yes application will have have access to keys.

since i have worked on exchange platform and payment gateways you can message me in persion about the doubts, i will be happy to help you

creating wallet address dynamically can be done with RPCs but for a exchange you wiil be needing extra services like mempool crawler, block crawler and soo on

The call you want is getnewaddress, which will generate a new Bitcoin address, add it to the current wallet, and return the address as the result of the call.
You can find documentation for the call here: https://bitcoincore.org/en/doc/0.16.0/rpc/wallet/getnewaddress/
If you were writing this in Python, it would look something like this:
from pyjsonrpc import ServiceProxy

bitcoin = ServiceProxy("http://127.0.0.1:8332")
bitcoin.username = 'testuser'
bitcoin.password = 'testpass'
print(bitcoin.getnewaddress())

You would then need to track the addresses generated and record them with your user accounts, monitoring transactions and crediting users as they come in.

Thanks for your prompt reply. I will try this approach and update my result here.


text:
Statement: for an attacker to change history, they must solve computational puzzles at a faster rate than the rest of the participants combined
I'm new to cryptocurrency and blockchain. I just started reading the textbook Bitcoin and Cryptocurrency Technologies by Narayananm, Bonneau, Felten, Miller and Goldfeder.
The statement in the title is given in the introduction to the book (page 17 in the pre-publication draft for anyone interested)
I'm assuming it is accurate and that it is connected to the architecture of the bitcoin blockchain. However, I don't quite understand why this is the case. Is it because in order to change history you need to change the entire chain up to the block you want to change? And why do you need do be faster than all others combined?
Any explanation would be greatly appreciated!
Edit: Later, on page 33:

as long as we store the hash pointer at the head of the list in a
  place where the adversary cannot change it, the adversary will be
  unable to change any block without being detected.

Does this mean that if the head pointer is unalterable (the genesis block) the statement is false?

I will answer my own question for anyone that might be wondering the same.
If a majority of CPU power is controlled by honest nodes, the honest chain will grow the fastest and outpace any competing chains. To modify a past block, an attacker would have to redo the proof-of-work of the block and all blocks after it and then catch up with and surpass the work of the honest nodes.


text:
Check txout against the circulating supply instread of MAX_MONEY
https://github.com/bitcoin/bitcoin/blob/aaaaad6ac95b402fe18d019d67897ced6b316ee0/src/consensus/tx_check.cpp#L27
If txout.nValue is more than the circulating supply and less than MAX_MONEY, CheckTransaction would return true.
Why not check txout.nValue against the circulating supply? and if it is above then just return false at that point.

There is no point. It would just be slower and more complex for no gain.
Despite what the name may imply, that test is not there to enforce the inflation schedule. It's there to prevent exploiting integer overflow (something which happened in Bitcoin's early history).
The test could be replaced with > 1000000000000000000 instead and be equally effective.

Just to elaborate a tiny bit: Five lines above the line linked in the question there is a comment saying // Check for negative or overflow output values (see CVE-2010-5139). Looking up that CVE (any search engine) yields the explanation regarding integer overflow. The only reason I looked is because when I read Pieter's answer my immediate thought was - "that's the sort of thing I'd put in a comment against that line of code" - so I looked to see if someone had done that or not.

Added explanation: There is also no direct way of calculating "circulating supply". Circulation is not something the protocol can identify. By circulating supply we simply mean
TotalSupply - ((coins estimated to be lost) + (satoshi's coins)).
Trying to enforce against circulating supply in the protocol will be unnecessarily heuristical and will provide no functional benefit.
To verify no new money is created, its enough to just check against max money.


text:
Does my segwit node only have segwit blocks?
My segwit node makes a gettxout RPC query and checks the scriptPubKey.type is "witness_v0_keyhash" and does things accordingly.
To my understanding old nodes have different blocks and if I happened to fetch a block from an old node, what will my node do? Will it simply store it as is and ultimately result in not getting the desired "witness_v0_keyhash" response when I expect?

BIP 144 equipped nodes doesn't fetch blocks from old nodes since they are only capable of providing witness stripped blocks.
If someone sent it a stripped block (e.g. by lying about their support or whatnot) and it contained any segwit using transactions then it would get dropped and the peer disconnected-- basically treated like merkle tree malleability.


text:
What files do I need to copy to a new computer to prevent having to sync with the network?
I think this used to be the blk0001.dat file, but it seems that the block chain is now (since version ?) moved to the blocks/ folder, containing a lot of blk* files, as well as an index file and multiple rev* files.
What files do I have to copy to a new machine if I want it to be synced up?
I know I will have to run with -rescan the first time after I changed the block chain files. And I know that the blocks are not verified (but they come from another machine I own, so there are no trust issues).
Please mention the current version number in your answer to not confuse people if the file configuration would change over time.

I believe this question http://bitcoin.stackexchange.com/questions/10364/transfer-blockchain-from-pc-to-mac asks something very similar...

Not very similar. I don't want to move my whole setup, only the minimum block chain files. Also, a recent update changed the whole folder organisation.

In that case, you may be interested in a very similar question I asked http://bitcoin.stackexchange.com/questions/10787/how-to-transfer-only-the-blockchain-to-a-friends-computer
Be warned, though: for what I read, most of the time spent when importing the blockchain is in validating all the entries - even after you'd copied the files, you'd still have to wait several hours...

@JoePineda, the blocks are not verified when you import them from another computer. The block chain is just scanned for transaction outputs of your addresses.

@StevenRoose Depends on what you mean by 'import'. If you copy just the block files (the contents of the blocks/ directory), they will be verified as if they were received from network. If you only copy the chainstate/ directory as well, there resulting state is already known and nothing will (need to) happen.

https://en.bitcoin.it/wiki/Data_directory
This might be of some help.I followed that and successfully moved the files from C:\Users[username]\AppData\Roaming\Bitcoin\ to E:\BitcoinData.
Just cut and paste the whole content of from the Bitcoin folder (C:\Users[username]\AppData\Roaming\Bitcoin) and set the new target as
"C:\Program Files (x86)\Bitcoin\bitcoin-qt.exe" -datadir=d:\BitCoinData and that is assuming you have bitcoin Qt installed in your C: drive.Change the path accordingly.

With the time between transactions, it currently can't be done.  The closest you can do is ( on windows ) copy the contents of (boot drive):\documents and settings(user)\application data\Bitcoin on Linux systems /home/(user)/.bitcoin
I am not fluent on Mac systems so no idea where it is stored there.
The entire contents contain the block chain, peers, your wallet and status of confirmation of transactions.  It will still try and re-sync even with these files, but at least you will not have to depend on network traffic to get it.