text:
"Lost" password to BIP38 encrypted private key, encrypter may be faulty (Bippy)
Few years ago I encrypted my Bitcoin private key for cold storage with an offline app called Bippy. Recently I tried to access my wallet by decrypting the private key, but my password was rejected.
I've checked out this Bippy program again, and I'm beginning to suspect it's faulty somehow. It seems that private keys beginning with a '5' - encrypted with Bippy - won't decrypt with the same password. Other prefixes seem to work OK.
Now, this Bippy app is open-source and written in Python, and I got it from here. I've tried, as a test, to encrypt the private key 5Ka1Bv4RpKayZygPhZbBcREejqeYwq2iSRzWPKsg49SPgVRTnpW with password testingOneTwo (it's a fresh pk, no bitcoins there ;)). In Bippy this encrypts to 6PYMN8WT4CW7xLYczLvYfXZCtQWND7XEesk32QbXujMgKhf2v5A2ZQANpL, however, using the above password fails to decrypt it to its original... I've tried several other '5'-private keys and passwords, and the same happens every time.
Luckily the program is open-source; sadly, the project seems abandoned a few years back and my knowledge of Python and encryption is not at a level good enough to pinpoint the bug at play here, and hoping for some help here :)

What did you use to try and decrypt the encrypted private key? Bippy?

It should be noted this was originally asked on reddit and someone there even posted some code.

It turns out Bippy does have a bug in the encryption algorithm where it mixes up the public key compression flag. As pointed out, I posted this problem on reddit as well, and a solution has been posted there:

Bippy is definitely doing something wrong.
Technical Explanation
BIP38 encrypted keys have 4 components (in the simplest case).
The first two letters of the key, 6P, are a marker that say this is a BIP38 encrypted key.
The next letter, Y in this case, specify some attributes of the key. In this case it says "This is a simple BIP38 key, and the public key should be compressed."
The next couple letters (4 bytes in the hex representation) specify a "checksum". We'll get back to this.
The rest of the key is the real meat of it. It's the encrypted private key. That's what we care most about.
Now let's get back to that checksum. The checksum is 4 bytes derived from the Bitcoin address associated with the original private key. A big caveat here is that there are actually two Bitcoin addresses associate with a plain Bitcoin private key. One is based off the compressed public key, and the other the uncompressed public key.
That's why BIP38 encrypted keys specify whether they are using the compressed or uncompressed address.
In this case, Bippy spit out a Y which means it used the compressed address. In your screenshots it even says as much. BUT here's where the problem is.
When we check the checksum Bippy spit out, it doesn't match with a compressed address. But if we use the uncompressed address, it matches! So Bippy says it's using the compressed address, but it's actually using the uncompressed address.
So Bippy definitely has a bug where it's using the uncompressed address, but saying in the resulting BIP38 key that it used the compressed address.
This isn't too big a deal. I'll discuss this after the non technical explaination.
Nontechnical Explanation
Bippy is encrypting correctly, but it's screwing up its encoding of the encrypted key. It's saying it used public key compression when actually it didn't. This isn't a huge deal.
Solution
So, the good news is that it looks like, other than that minor bug, Bippy is encrypted the important stuff correctly. It should be possible to take your "lost" Bippy key and recover the private key from it. Assuming your password is correct.
I should be able hack up my script to take care of that for you. I'll to to work on that and get back to you. (I need to fold in some Base58 encoding/decoding into the script and make it a bit user friendly for ya ;) ). The code will be quite simple Python code, so you can do a cursory glance to be sure it doesn't do anything evil.

A python recovery script has been supplied as well, available on github - all credits to fpgaminer

If there is a bug in Bippy and it's generating incorrect encrypted private keys, that would make your unencrypted private key unrecoverable, unless the bug in Bippy was something as simple as changing the passphrase before putting it through the encryption algorithm.
I haven't been able to get Bippy to build on my machine so I haven't been able to test. But I used bitcoinpaperwallet.com to test your above keys, and when I use the passphrase testingOneTwo to encrypt 5Ka1Bv4RpKayZygPhZbBcREejqeYwq2iSRzWPKsg49SPgVRTnpW, the result is the encrypted private key of 6PRLKpRdHRsMJcjuMYoE4wpKYUZEX11oB9LejdU8JuDLxynpDZFZWMNkDv.
I haven't been able to find anyone else running into this problem. I would suggest trying to decrypt your private key using another BIP38 tool such as the "Validate or Decrypt" section on bitcoinpaperwallet.com.
If that doesn't work, it could help if you really walked us through step by step (with screenshots) how you are both generating the encrypted private key and how you are decrypting it.


text:
Using xpub key derivation for user account balance?
I want to accept payments, and to keep the wallet secure I believe using a xpub key would be the correct solution. The site uses Google Login, which gives me a user id.
My idea is that given the xpub on the server code, I can derive a unique payment address per user (using the Google user as the derivation seed), and then simply check the blockchain for new transactions. I would store the amount of used balance used on the server database, but since there are no physical products being sold, that shouldn't be of concern if hacked.
My question is if this would be secure, in case the server got hacked? If an hacker obtains the xpub and perhaps even some of the previously generated key derivations, is there anything bad the hacker could do?
(PS: Although I'm familiar with BTC inner-workings, this is my first project/implementation)

xpub is short for Extended Public Keys. Public keys, as their name suggests, are meant to be public. If the xpub is stolen, then all that will happen is that the thief will know how much money you have and public keys associated with those coins. All you lose is your privacy, no money is at risk.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Could someone provide an example of a tx spending from an input with a relative LockTime?
I want to see an example of a raw transaction which spends from an input which has a relative LockTime applied to it. An example would be a transaction from a Lightning payment channel in which a user force closes a channel and claims their 'local' balance, after having waited for the 'to self' delay in their commitment transaction.
It would be very helpful for understanding how to spend from an input encumbered with a relative LockTime.

If you look at https://blockstream.info/address/bc1qdv0fk4c73fem89s708pzdny8q25d7k5nrckckp87c8alwaw803fq3mqrpz you will see that there are two transactions.
The first one  https://blockstream.info/tx/943f18e17aa2ba716f2d13785268a1bd6b25c39d2718f3b4785559a5eeaba883 is the funding TX to open the payment channel included in block 587093 (at least according to what was send over gossip)
The second one https://blockstream.info/tx/a175425ee2f3c01acc97ef32ab1082db14eaa94e5af4c54f02f3b01dec72558e is the force closed timelocked commitment tx included to block 587314
The output 0 of the commitment tx has a relative timelock and was self spend / sweeped with this transaction https://blockstream.info/tx/0be2b9a3a6a9dc99443d6ce7bd3beaf33c713a4938af3bf0fec1e918257bbd32 which was included in block 588801
If you decode the transaction you see nothing special
304402204a54d94068512c22847cc07fa85f2b93feaf758411920a9b20ce9b85831e8aa5022005924cb82c331c5173e21477a4d8298eb25c0d967ace065b883455f2537c3d0a01 <empty> 632103f96d73ffe138690f032e97408af933a71782ad27a28df66a750b48c56b2d7fa96702cf05b275210304f332f50363cd01b679d9fb50fb8691fb0fb638737867bd02f06850c6191a2268ac

p2wsh witness script
OP_IF OP_PUSHBYTES_33 03f96d73ffe138690f032e97408af933a71782ad27a28df66a750b48c56b2d7fa9 OP_ELSE OP_PUSHBYTES_2 cf05 OP_CSV OP_DROP OP_PUSHBYTES_33 0304f332f50363cd01b679d9fb50fb8691fb0fb638737867bd02f06850c6191a22 OP_ENDIF OP_CHECKSIG

nsequence
0x05cf

previous output script
OP_0 OP_PUSHBYTES_32 c9452fef6bd1c72c8648d6ac78ce552543a1e5bf80a470fb259ff670aa5ced26 (v0_p2wsh)

However you can observe the nSequance = 0x05cf which encoded in decimal is 5*256 + c*16 + f = 5*256 + 12*16 + 15 = 1487
As the commitment tx was included in block 587314 the timelocked spend from it would have been rejected any block earlier than 587314 + 1487 = 588801 which was exactly the block height of the third tx.

It's a case when a local node closes the channel unilaterally with broadcasting its local commitment transaction that contains to_local output below. This output is spent by a transaction with nSequence field set to to_self_delay (which can only be valid after that duration has passed.)
OP_IF
    # Penalty transaction
    <revocationpubkey>
OP_ELSE
    `to_self_delay`
    OP_CHECKSEQUENCEVERIFY
    OP_DROP
    <local_delayedpubkey>
OP_ENDIF
OP_CHECKSIG

It's a commitment transaction on Testnet as following.
txid: 4f8d776c85b1fc15c1125e7043a9aee70e33f0793b472823e3946a8de075bec4
to_local output:
6321026644cb387614f66421d14da3596c21cffa239011416c9adf3f351ee8551a9fc767029000b27521029654f80732769d7c435a184a3559f12178315526c53bbf003349390811c7590a68ac

decoded output:
{
  "asm": "OP_IF 026644cb387614f66421d14da3596c21cffa239011416c9adf3f351ee8551a9fc7 OP_ELSE 144 OP_CHECKSEQUENCEVERIFY OP_DROP 029654f80732769d7c435a184a3559f12178315526c53bbf003349390811c7590a OP_ENDIF OP_CHECKSIG",
  "type": "nonstandard",
  "p2sh": "35M7MGVGd3ajyiTMvriUiyUM3H2cT5sgb4"
}

The commitment transaction got mined in 1356328 block height, and to_self_delay is 144, so the output can be spent after it reaches 1356183 block height.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What goes in the tx hash preimage when signing with 0x83 (SINGLE|ANYONECANPAY)?
I have script to create a P2WPKH to P2WPKH transaction (1 input, 1 output) and sign it with the sighash type 0x01 SIGHASH_ALL.
Which fields would I need to change in the tx digest preimage to turn the signature into one with a sighash type of 0x83 (SINGLE|ANYONECANPAY)? I'm having trouble understanding the documentation in bip_143.
For reference, the tx digest preimage:

Double SHA256 of the serialization of:
       1. nVersion of the transaction (4-byte little endian)
       2. hashPrevouts (32-byte hash)
       3. hashSequence (32-byte hash)
       4. outpoint (32-byte hash + 4-byte little endian)
       5. scriptCode of the input (serialized as scripts inside CTxOuts)
       6. value of the output spent by this input (8-byte little endian)
       7. nSequence of the input (4-byte little endian)
       8. hashOutputs (32-byte hash)
       9. nLocktime of the transaction (4-byte little endian)
      10. sighash type of the signature (4-byte little endian)

From BIP143:

If the ANYONECANPAY flag is not set, hashPrevouts is the double SHA256
  of the serialization of all input outpoints;
Otherwise, hashPrevouts is a uint256 of 0x0000......0000.

All the details you need are in the BIP

Thanks. It feels like I've tried every combination of setting hashPrevouts, hashSequence, hashOutputs to 0x0000......0000, as well as the sighash type itself, but I keep getting the error message:
error message:
non-mandatory-script-verify-flag (Signature must be zero for failed CHECK(MULTI)SIG operation) (code 64)

Have you looked at the code? You might find it more helpful: https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L1238-L1282 or in bcoin: https://github.com/bcoin-org/bcoin/blob/master/lib/primitives/tx.js#L637-L720

Unfortunately I don't know cpp so it's a bit hard to follow

@Darius bcoin is JavaScript, if that helps. What language are you using to compute this sighash?

I'm not familiar with JavaScript either. I am using the ecsda and hashlib libraries in python. I am creating a script based off this tutorial: https://github.com/zeltsi/segwit_tutorial/tree/master/transactions
In case you were curious this is what the (non-working) script looks like:
https://github.com/DariusParvin/temp/blob/master/P2WPKH_to_P2WPKH_anyonecanpay.py

Just took a quick look - you may need check that your txid is little-endian...

thanks a lot. I know everything else is okay because I used it to create a valid sighash_all tx. From that all I did was change the sighash flag to 0x83, set hashPrevouts, hashSequence, hashOutputs to 0s in the digest, but btc core is telling me the signature is invalid.

glad to say I finally solved it. It was a stupid mistake. When converting from string to hex, I put 32 zeros which of course only corresponded to 16 bytes of zeros. So yes, all the info was clearly in the BIP and it was my bad!


text:
Cannot sign txn with change address using bitcoinjs-lib psbt
I am having a difficult time figuring out how to sign a spend from a change address. I get Error: cannot sign for this input with the key <value>. The initial transaction works fine, where I generate a transaction with multiple inputs to two outputs, one of them being the change address - it is the same as the spender addresses. Here are the steps:
1) I grab all UTXO for our spending address.
const unspentTx = (await listUnspentTxByAddresses([mainAddress])).data.result;
2) I add all inputs and outputs, sign, finalize, and submit.
  const fee = new BigNumber((await getEstimateSmartFee(2)).data.result.feerate)
    .times(100000000)
    .div(3);

  const output = {
    address: destinationAddress,
    value: parseInt(pullAmount.minus(fee).toFixed(0)),
  };

  const senderBalance = sum
    .times(100000000)
    .minus(pullAmount)
    .minus(fee);

  const change = {
    address: sender,
    value: parseInt(senderBalance.toString()),
  };

  // Adding the output
  psbt.addOutput(output);
  psbt.addOutput(change);

  for (let i = 0; i < txids.length; i++) {
    const transactionObject = await getTransaction(txids[i]);

    const input = {
      hash: txids[i],
      index: 0,
      nonWitnessUtxo: Buffer.from(transactionObject.data.result.hex, 'hex'),
    };

    try {
      psbt.addInput(input);
    } catch (e) {
      console.log('Add Input Failed e =>', e);
      return null;
    }
  }

  const key = bitcoin.ECPair.fromWIF(privateKey, testnet);

  for (let i = 0; i < txids.length; i++) {
    psbt.signInput(i, key);
  }

  psbt.finalizeAllInputs();

  const rawTx = psbt.extractTransaction().toHex();

  const withdrawPendingTx = await sendRawTransaction(rawTx);

This works the first time around, but as mentioned, the second time I call
await listUnspentTxByAddresses([mainAddress]) and follow the same exact procedure, the js library validation prevents signing with the main address key. It doesn't seem to make sense, so I hope there is an obvious error here. I'm pretty sure it's sent as a default P2PKH with SigHashAll.

Simple solution! index in input object was hardcoded to 0 instead of taking the vout nonce from UTXO list. I was going to delete, but I'll leave the post up.


text:
do miner hashrates vary during block production?
When a block is mined, the remaining mempool txs can be mined into a new block.
Is there a difference in blocktime between accepting all streaming mempool txs into the mining block and sticking with first 100txs in a block?
If so, how does the applied hashrate fluctuates between two blocks based on txcount for instance, otherwise is the hashrate steady between blocks and whats the reasoning for that?
Thank you,

When a new block is found, miners will automatically switch to mining on top of the new block headers with an empty transaction set. Occasionally you can see this in the blocks, when two blocks are found immediately after one another, the second one tends to be empty.
Within a few seconds though a non-mining PC constructs the merkle tree of the highest paying mem-pool transaction that can fit into a block, and passes the merkle root to the mining hardware, which then mines on top of the previous block and includes the transactions.
This merkle root of highest paying TX is updated every 30 seconds or so.
So here's something important: you don't pick which TX to include after you have mined it, you do it before.
Also: The hashrate of the hash-hardware stays constant. It's reliant on getting accurate previous blocker header, merkle root and a few other bits from somewhere, so that what it is mining will be a valid block.

Thank you but I still don't how hashrate remains constant or does it affect blocktime adding more transactions? The reasoning is that when addind a new tx to the merkel it invalidates the current block nouce/space search (same nounces can be recalculated).

@tlr: hashrate is a fixed characteristic of the mining hardware used. Nothing else.

The hasher hashes at approximately constant speed. It only hashes the block header, in which all transactions are through the merkle root. So whether there are 0 or 2000 transactions planned for the block doesn't matter to the hasher or the hashing speed.


text:
Confirmed transaction but unspent

Hi, I have a serious issue as this transaction is successful but the recipient insists that it's not in her wallet neither did she get any sort of confirmation about the transaction, I have been trying to understand what went wrong and why I keep seeing this transaction as 'unspent'. How can I fix this please.

The issue is with the recipient's wallet. There is nothing that you can or need to do.

why I keep seeing this transaction as 'unspent'.

It is not saying the transaction is unspent. It is saying the output of that transaction is unspent, which is perfectly fine and expected. Bitcoin transactions use a system of spending and creating transaction outputs. If a transaction output is unspent, that means it is not spent by another transaction and indicates that the output is available to be spent. This means it is part of your wallet as your "balance".
In this case, you created a transaction which creates an output that is spendable by the recipient. Since the recipient has not spent the Bitcoin yet, the transaction output is unspent. That the recipient's wallet does not detect the transaction means that there is an issue with her wallet, not yours.

Okay thanks, what can she do to fix her wallet? Any clues?

It depends on what wallet software she is using. If she has the option, she can do a rescan which will scan the blockchain for all transactions in he wallet.


text:
Can I double my money through the upcoming Segwit2x fork?
Today, I have received an e-mail from GDAX that says:

We wanted to remind you that the upcoming Bitcoin Segwit2x hard fork is projected to take place on November 16 and will result in two bitcoin blockchains.

And especially interesting:

Any customer storing Bitcoin (BTC) on GDAX at the time of the fork will be credited with an equal amount of the Bitcoin2x (B2X) asset on the Bitcoin2x blockchain. GDAX will continue referring to the current bitcoin blockchain as Bitcoin with ticker BTC and the forked blockchain as Bitcoin2x with ticker B2X

Does this mean that I have the double amount of money after this fork? If yes, why should I not buy a lot of bitcoins before the fork and sell all Bitcoin/Bitcoin2x immediately after the fork?

The "Segwit2x" fork is scheduled to activate at block height 494,784. After this block, there will possibly be 2 independent blockchains that grow independently of each other. If that happens, then any BTC you held before block 494,784, would also be available on the B2X fork.
This doesn't necessarily mean you would double your money. It's possible, but it depends on how the market prices both BTC and B2X. For example, if you held 10 BTC @ $5000 before the fork, and then the fork happens and exchanges come back online and the market prices BTC at $2000 and B2X at $2000, your 10 BTC + 10 B2X would be worth less than before the fork. It's impossible to know how the markets are going to react to the fork and how messy it's going to get.

It is possibly helpful to think of an extreme example. Forks are not magical; I can create thousands of Bitcoin forks in my basement if I want, just by making incompatible tweaks in the protocol.  It would be absurd to think I can multiply my money thousandsfold in this way.  What will happen is that nobody cares about any of my forks, so nobody will give me goods or services in exchange for the forked coins, and they effectively have zero value.  Likewise, the value of the BTC and B2X coins after the fork will depend on what people are willing to pay to acquire them.

Yes ok, makes sense. But after the Bitcoin Cash fork in August, Bitcoin prices didn't drop as well. So what do you think? What will happen now after the Segwit2x fork with the Bitcoin price?

The Bitcoin Cash hard fork and the 2x hard fork are shaping up to be quite different though. Bitcoin Cash was never trying to be the "official" Bitcoin, whereas the 2x folks are trying to make 2x the "official" Bitcoin. It changed the protocol more drastically than the 2x fork is proposing to do. And it implemented strong replay protection causing minimum disruption to the existing chain, whereas 2x is not (at the moment) implementing any replay protection. So this hard fork has the potential to be disruptive to the ecosystem.

If yes, why should I not buy a lot of bitcoins before the fork

Because of limited supply on markets :) You can not buy more than 21mln bitcoins because such quantity does not exist
This is the question of the year. I vote for it.

With "a lot" Bitcoins, I mean in the range of 1-10 or any other affordable quantity.

I think we can also make "fork of fork" in each fork to quadruple money. What about next steps?

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Is the electrum android private key secure?
I have used electrum for many years on my desktop pc, and have alway felt very secure in doing so because of my ability to use a very strong password to encrypt my seed phrase with. However I just started using the android version and have some concerns.
The android version requires me to select a PIN but it is only 6 digits long? How does the android version of electrum ensure that this tiny keyspace (1 million) stays secure?

Edit 15/3/2021: Electrum now lets you set a longer password.
Android is also better at keeping apps from stepping on each other's toes. The OS isolates Electrum's files and prevent other apps from accessing them.

+1 Good answer, but I would like more details. Why does rooting after wallet creation not work? Why can the files not be accessed using adb?

the answer is wrong now. they allow you to set a longer password and the wallet file can be saved to microsd card.

Please edit your answer to include this new information, as well as clarify the relevance of rooting the device either before or after creating the wallet.

@BrianDrake afaik rooting wipes app data so can't be used for this purpose

Rooting should not erase data, according to Android SE.

The android version requires me to select a PIN but it is only 6 digits long? How does the android version of electrum ensure that this tiny keyspace (1 million) stays secure?

The PIN is not used as a master seed in this way. It may be used as an additional source of user-supplied entropy, but the wallet's seed will be generated in a cryptographically secure way, using other sources of randomness to provide sufficient security.
I am unfamiliar with the specifics of Electrum's implementation, but there are generally two possibilities here:
1) the password / PIN is simply used to encrypt the wallet, and is not used as input for the seed
2) the password/ PIN is used as a '13th/25th word' that acts as a password on the mnemonic. Find more info here. Also, from BIP 39:

A user may decide to protect their mnemonic with a passphrase. If a passphrase is not present, an empty string "" is used instead.

Electrum uses option 1 - it has support for option 2, but you configure that during wallet set up, and don't need to reenter it unless you are restoring a wallet (I'm not sure if that applies to the mobile version, but since the desktop works that way, I imagine it does)

Option 1 is what I was afraid was happening, and what I meant when I asked if the private seed is secure. A 6 digit pin as an encryption key for the wallet, which contains the master private key, is not secure. I'm going to have to re evaluate using electrum on my android device because of this. If anyone ever gains access to the encrypted wallet file, it would be trivial to brute force the wallet file encryption.

@user258667 Oh! I understand the question better now. Your comment is correct, if anyone gains access to the encrypted wallet file, the 6-digit PIN encryption is not extremely strong. Nevertheless, the security model provided by such a mobile wallet may still be useful in some situations (ie a small amount of 'daily spend' BTC, not a large amount that you want to keep extremely secure).


text:
Is it possible to connect to BTCPay server remote c-lightning node?
I have BTCPay server and would like to use external c-lightning node which i have funded and prefer more over BTCPay server local docker container (DC).
In store configuration i see that there is option to connect to c-lightning over tcp or unix socket (this one is used when i connect internal node because c-lightning DC socket is linked to BTCPay directly).

How can i connect over TCP? As i uderstand this is not public nodes port (9735) which is used to communicate between nodes but some API port? How can i enable it on c-lightning? Is data sent plain and i should secure communication (lan, fw, vpn etc.)?

Can't give you the full answer but a hint. Unix domain sockets are local. So you could only connect to a Unix domain socket on that machine / within that docker container. One way is you could migrate your external lightning node inside your dicker container. Shut it down and copy ./lightning dir. I am however sure there is a more elegant way. Also you will most likely not talk to your c Lightning node via port 9735 on tcp as that is what is used on the lightning network. It is new to me that clightning offers a tcp api. Which is the reason I could only comment here and not answer

yeah, i see your point. i thought about "sharing" exactly lightning-rpc socket file over ssh or smth similar if its possible to share unix domain socket over lan at all. i have btcpay server and c-lightning on separate physical hosts in same lan, but would like to prefer not to move one into another if its possible.  i also could not find any mention about tcp listeners in lightningd help..

Just a word of caution. In case you decide to migrate the node make sure to really migrate the latest channel state and don't run two seperate instances with the same files in the lightning-dir as this will most certainly produce channel beaches with lots of funds

Port 9735 is the lightning port. What BTCPay wants is the API.
C-Lightning does not support TCP natively, but one can use socat as I did in the docker entrypoint to bridge unix to TCP.

I'll add additional description if anyone like me will be searching for direct steps there are two ways to connect:

use socat. install and run command socat TCP-LISTEN:22345 UNIX-CONNECT:/path/to/.lightning/lightning-rpc
as suggested before, use https://github.com/ElementsProject/lightning-charge

But what is the EXPOSE_TCP setting for then? I'm confused. I created a new question specifically about that, so that one of you can reap the benefits of answering another question. :)
https://bitcoin.stackexchange.com/questions/88862/is-it-possible-to-connect-remotely-to-a-c-lightning-node-with-an-exposed-tcp-por

EXPOSE_TCP just mean that the container is running socat to expose RPC over TCP.


text:
wallet will not sync it has been a week and I still do not have my money in the wallet. It took out my account
I have video proof that this has been going on for a week I have restarted the computer multiple times and have yet to receive any money into my account. I do not even have a wallet number.

Hello, welcome to Bitcoin.SE! In order for someone to help, you'll need to provide more info, such as: what software are you using? What sort of errors are you receiving from the software? What do you expect to happen, and what actually happened? Where did you buy bitcoin from? Do you have a transaction ID for a transaction that was sent to you? Etc

I am using a Windows laptop, using Bitcoin Core Wallet. When i went to the bitcoin website to buy bitcoin it said i needed a wallet, therefore I downloaded BCW. It has now been six whole days and I have yet to see the deposit I made to the account to receive money. I do have a transaction ID from blockchain and it shows that all was approved, now I am just not seeing it in the wallet. As of now I want to know why for six days the BCW app on my laptop has shown that it is syncing for six days and has yet to give my money?


text:
Cannot recover my wallet
I have an Android backup that I am sure I am using the right password. I have exported the file, and trying to decrypt it with openssl:
openssl enc -d -aes-256-cbc -md md5 -a -in ./backup_carteira_2017

or
openssl enc -d -aes-256-cbc -a -in ./backup_carteira_2017

I fill in the password when prompted and it seems to decrypt as I don't get a "bad decrypt", however the output is not readable at all. Any password different from the right one throws a bad decrypt error.
Am I missing anything here?
The backup has been generated in Jun, 2017, not sure if this matters.
Regards,

Possible duplicate of How to restore from backup of Android Bitcoin Wallet where the software author explains the proper process.

Note: Anyone reading https://github.com/bitcoin-wallet/.../README.recover.md should probably PAY CAREFUL ATTENTION TO THE FIRST SENTENCE there. If you proceed beyond that, note that the output of the SSL command is not expected to be readable.


text:
will segwit2x fork affect bitcoin cash
I read about the fact that segwit2x will split the BTC in BTC 2X. What will happen to BitCoin Cash BCH.

BitCoin Cash and Bitcoin are two different blockchains. Technically, segwit2x won't affect the BTC blockchain nor the BCH blockchain (the only thing that might be affected is the price of each coin).

SegWit2x is a hard fork that will occur around November 16, 2017, and as a result of the hardfork there will be two chains, one will be Bitcoin Core (legacy Bitcoin) and the other will be the SegWit2x version of Bitcoin.:

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What are the benefits of running a crypto exchange website?
Getting straight to the question. Say I want to run a cryptocurrency exchange platform where people could come and buy/sell Bitcoin or Altcoin. The way I understood I have to partner up with a Bank in Order to be able to buy coins and have them in my websites depository to sell and even be able to buy from people instantly in exchange for cash. What are the benefits of this? How much profit is in running such a website? Would be that much profit that could convince a Bank to invest in it?
And is there any way that I could run it with less deposit? like taking the money from user and the buy the Coin instantly and send it to them? like, being a middle man.
Bests.


text:
How do websites sell cryptocurrency with fiat payments?
I'm trying to create a platform to sell cryptocurrencies in my country. the way it works is, the user enters my website, select a coin he wants to buy then the amount he/she wants to buy, he pays in our common currency (say USD) and we should send him/her the cryptocurrency he has bought. the procedure is like most of the crypto shops, but the question is how should I get cryptos to sell? say a user wants to buy 5 Bitcoins, how should I have 5 Bitcoins to give him? how do other websites do this?
Any help would be appreciated.
Thanks in advance.

Hey and welcome to bitcoin SE, you would need a hotwallet to store the bitcoins that users buy.  When a user buys, most sites use an automated system to send from the hotwallet to the user. Use a hotwallet so as to avoid someone exploiting an issue in your code and stealing all your btc!

Essentially you want to start a cryptocurrency exchange? If that is true, you need liquidity for people to be able to buy these coins. There are two ways of going about it:

Use a broker/third party exchange, such as Kraken.com or any other in your region and have your orderbook go through their's. You would have to price it so you are profitable however.
Prefund your own exchange. For example, you could buy 5BTC and keep it in a wallet designated for your website. So when a customer comes there you have that "cash" flow to sustain orders up to 5BTC. Obviously you would have to scale this up if needed.

I don't know which country you are in but normally this requires a money transmitter license (at least in the USA) for fiat to crypto gateways among others. If your question is referring to the technical/programmatic aspect of the exchange, that's a different answer.

Thanks for your answer, no I have nothing do to with the programming aspect of it. yes, what I wanna do is an exchange platform. so I need to buy those Bitcoins or Altcoins in advance an have them in my websites wallet before I sell any?

Yes, that would be called providing liquidity.

If the answer is sufficient, you can mark it with the green check mark underneath the up/down arrows. Thanks


text:
Who first defined/coined The Double-Spending Problem?
The Double-Spending problem is a variant of the Byzantine Generals Problem which has a nice write up by Lamport, Shostak and Pease. But who first coined/defined the specific Double-Spending problem? Or linked the BGP to the digital currency domain? Does there exist a proper academic write up, defining the problem, that predates bitcoin?
I searched for a bit and can only find write ups after Nakamoto's publication.

Does there exist a proper academic write up, defining the problem, that predates bitcoin?

Google ngrams shows the phrase being used as early as the 1970s but that is probably not in the context of digital currencies.
The earliest reference I could easily find is in several articles in "Advances in Cryptology - CRYPTO '93"
For example in "Untraceable Off-Line Cash in Wallet with Observers" by Stefan Brands.
Since several articles mention double spending I expect the coinage of this phrase in this context is older than 1993.
However there were no mentions in CRYPTO '92.


text:
How do bitcoin miners determine the next valid state of a ledger?
I know bitcoin miners compete for a reward by seeing who can solve a difficult hashing problem first (proof-of-work).
However, how does this random hash problem help validate the transactions in the next block? It seems like this hashing problem is completely irrelevant to the ledger validation issue at hand. And when do the other miner nodes validate that the proposed block is correct after one miner finishes?

You are mixing the two things. It is not just miners, but the entire network of full nodes that ensure only valid transactions are included in a block. When a block is mined by a miner, it is propagated on the network. All the full nodes go through a detailed checklist (see here) to ensure that the block and the transactions within the block confine with the consensus protocol. This is unlike ledger based systems where an 'authority' is appointed to vote on transactions to maintain a state of the ledger.
Since Bitcoin is peer-to-peer, there is no central authority to control it. As a result, anyone can send any kind of transaction to the network, whether or not it is valid. You could simply send a transaction that sends someone else's coins to yourself. Now, if a miner tries to mine a block with those invalid transaction, the rest of the network will reject the block. So the entire effort (read electricity and time) that the miner put in will go waste as that block will not be added to the blockchain by the rest of the participants.
The 'random hash problem' also has other benefit. It ensures that Bitcoin blocks are mined every 10 minutes. Every 2016 blocks (or ~2 weeks) the difficulty is adjusted to ensure that the average block time remains at 10 minutes.

And when do the other miner nodes validate that the proposed block is correct after one miner finishes?

As soon as the miner mines a block (finding a valid block whose header hash confines with the current network difficulty), it will propagate the block through the network. Now, the miner not just sends it to other miners, but also all the full nodes that are connected to it. If the block is valid, the full nodes/mining nodes will relay the block to the nodes connected to them. If invalid, the block will be rejected outright and not relayed. When other mining nodes see this block, they will mine the next block on top of this block (provided the block they received was valid).

By mining, we mean that they are searching (through modifying the block and rehashing) for a block hash that is below the current difficulty target expected by all other network participants.  This "target" is adjusted every 2016 blocks to maintain an average block time of about 10 minutes.

how does this random hash problem help validate the transactions in the next block?

The network consensus rules require that the block hash meet agreed upon difficulty target as one of the many pieces of criteria for a "valid block". The list of transactions and block structure must already be valid in themselves before the Proof Of Work hashing starts. Various pieces (such as the nonce) are modified to modify the resulting hash.  The randomness of hashing the block is a way to ensure that the miner exerts tremendous effort (uses electricity).  This provides sybil attack resistance without the need to identify miners in that there is no difference between 1 miner (IP address) with a 1000 ASICs or 1000 pretend miners (controlled by 1 entity) with 1 ASIC each.
It also ensures that the miner has invested a significant amount of money (electricity & mining hardware) in that block.  This incentivizes them to "play by the rules" and not attempt to attack the network.  The block subsidy cannot be spent immediately, so if they damage bitcoin, they damage the value of their future payout as well.

when do the other miner nodes validate that the proposed block is correct after one miner finishes?

When a miner finds a valid block hash, they must broadcast that block to the the other bitcoin network participants.  Each miner validates it individually, and, if it is valid (has correct structure, valid hash and valid transactions), will remove those transactions from their mempool and begin building a new block.  The new block will contain a hash of the previous block which explicitly defines the order of blocks.  By this process, the block is added the the "blockchain" as miners begin building on it.


text:
Finding mixed transaction
I have a hard time understanding how mixed transactions can be recongnized on the blockchain.
Is this type of transaction a mixed one ?
https://www.blockchain.com/fr/btc/tx/cf903272cac49e28ff0e5fa109758954ed403480f68224e8ca5ccce7fcabccaa
Thank you for your attention

Mixing techniques are designed in a way so that it cannot be identified. There is no sure shot way of telling whether a transaction is mixed unless the mixing is shoddy. It might as well appear that this is a transaction where one party is using multiple inputs to pay multiple parties. Example would be an establishment accepting bitcoins for its services from its customers and pays employees/suppliers/partners at the end of the day using bitcoins. It would have to use multiple inputs and multiple outputs.
Having said that, there are some techniques for determining which outputs are created by which inputs and if it is mixing, but these techniques are not foolproof and often make assumptions that are not guaranteed.

"Mixing techniques are designed in a way so that it cannot be identified" this is not absolutely true, in fact many mixing transactions are quite obviously mixing transactions. The link between inputs and outputs is broken, but looking at the history of each output, it can be determined that it came from a mixing transaction. That said, some types of mixing (eg P2EP transactions) are in fact designed to be indistinguishable from 'normal transactions', but other other legitimate mixing techniques (eg Wasabi's mixing scheme) are not.


text:
Bitcoin core trying to sign a transaction loaded from psbt file, but getting "could not sign any more inputs" error
I am trying to sign a transaction loaded from psbt file, but I'm getting could not sign any more inputs error.
I did it successfully on the first try. However I cannot do it any more now.
I changed my passphrase of bitcoin-core before I did the second try. Does it matter?


text:
How to get number of transaction confirmation from bitcoind?
I have a question,
How can I get the number of transaction confirmation by using bitcoind?
for example:
https://live.blockcypher.com/btc-testnet/tx/9fd745521000d1aadb1f4857e44817c3342d02585af93c83d19dc3f3f4425594/
you can see the number of CONFIRMATIONS on that page
thank you in advance!

Use bitcoin-cli getrawtransaction "txID" 1
The output will include a "confirmations" field, telling you how many confirmations the transaction currently has.
Note that to get this information for transactions not related to your wallet, you will need to set txindex=1 when syncing your your node, or provide bitcoin-cli with the blockhash of the block the transaction was included in. Here is some more info about this, copied from the output of bitcoin-cli help getrawtransaction, running v0.17.0.1:
NOTE: By default this function only works for mempool transactions. If the -txindex option is
enabled, it also works for blockchain transactions. If the block which contains the transaction
is known, its hash can be provided even for nodes without -txindex. Note that if a blockhash is
provided, only that block will be searched and if the transaction is in the mempool or other
blocks, or if this node does not have the given block available, the transaction will not be found.

I tried decoderawtransaction 02000000000101fca5041df91d39bca183f0d477e44fef535f60fd8ba6b829aee8781407abda0800000000171600140976d58d4db681de489b95b549bcfa1333190ee9feffffff02661c43000000000017a9149a3aa2465229a8a9fc1614f8c83eb7107c94fa4887d8940300000000001976a9142745fd0fcf649cf333009589020bc2b40de12b3c88ac024730440220180d48c51fc0ad33d77a078d2bf3ee829bdf52e483ecac56c4c59e983cf30c8e02206f5712841bd6a97cce238b736aabee60c3f5b9365c2c38c11a0604e3c2c7de2201210366a92cde1eed31af6348ff72d6dab13b82ffec92fc7d8546c05530175e616c495e261700
but I can't see confirmations in there

@bbnn that is because decoderawtransaction doesn't give you the number of confirmations, hence why I suggested the use of getrawtransaction, which does.

the example that you gave produced error: parameter 1 must be of length 64 (not 4, for 'txID')

oh I think I got it now I should replace the txID with the txID.. sorry

@bbnn no problem, happy to help


text:
What are the chances bitcoin big mining pools to unite and do 51% attack?
Looking at this bitcoin hash rate distribution chart (https://www.blockchain.com/en/pools), it only takes 3-4 mining pools to unite and do a 51% attack.
With this possibility, how can people still think that Bitcoin can be the money of the future?
PS: I'm also a bitcoin believer

Possible duplicate of Are big miners a threat for Bitcoin?

Won't happen.
Bitcoin's value (like that of most currencies) is based entirely on the belief that it has value.  If someone were to perform a 51% attack on Bitcoin, it would greatly weaken the belief in Bitcoin, and the value would collapse.  If a pool were to perform a 51% attack, they'd get a little money from the success of the attack, but at the cost of giving up a large amount of money from future success of Bitcoin.
In fact, there have been cases where a pool has come close to having 50% of the mining power, and when that's happened, the pool has voluntarily stopped allowing new miners to join in order to avoid even the appearance of being able to perform a 51% attack.

It won't happen if the bosses of the pools are all ideal.
In the real word, they may be lunatics or terrorists, they may be cheated or threatened. As long as they have the ability, the world is in danger.
The traditional security is based on economics, or on rational persons. That is not enough. The goal of cryptocurrencies is to upgrade the security to be based on cryptography.
To consider security like "they can but they won't" is no more than the traditional system, cannot reflect the superiority of cryptocurrencies.


text:
What is the best way to generate bitcoin wallet?
I am trying to create a cold storage, and I'm becoming so paranoid about everything on software and hardware (also afraid of myself forgetting/losing the wallet) related to the setup.
I have read some information online about cold storage setup, and I have come up with these questions:

What is the safest way to generate bitcoin wallet? From the existing software such as bitcoin-core? electrum? (I thought of writing my own codes to generate bitcoin wallet, but there are so many types of bitcoin wallet BIP32, BIP39, etc which makes me think that it's not a good idea to do so)
How are we sure that bitcoin-core and electrum is generating a safe wallet? (not pre-seeded)
Which standard is the safest way to generate bitcoin wallet? BIP32, BIP39?
Does bitcoin-core uses BIP32 to generate the wallet? if not, which one does it use?
Does electrum uses BIP39 to generate the wallet? if not, which one does it use? (I read that electrum is using an extended BIP39.
I also read that bitcoin-core is not using BIP39 because it is not sure, but why BIP39 is still used in many wallets?
Is it possible that a hardware (CPU, motherboard, SSD, old computer that has been used by me before) is infected with a malware (from previous use or even from the manufacturer)? This malware might be able to mess with the entropy and create something like a pre-seeded wallet?

Thank You

I would not consider generating the wallet as the primary security related issue, what i consider more important is the proper storage and encryption of your generated wallet. If you use open source wallets you will most likely be save.
I also thought about this a while ago and stumbled upon this guide for diy cold storage:
https://steemit.com/crypto/@guidedcrypto/diy-make-your-own-crypto-hardware-wallet
In essence it tells you how to create your cold storage which is:

create your wallets (advice is to use official ones, but you can use any you choose) to the usb-thumbdrive
store your access keys and passwords to the usb-thumbdrive
test all is working
encrypt everything to make it immutable (like this, malware or viruses have no chance destroying it and you are locking unauthorized people out)
store the encryption key somewhere save (offline, please use long keys which are random to make it difficult to guess or brute force)

If you are paranoid about not catching viruses or other tempering you could do all of this from a live open source linux disto of your choice as to be sure nothing was altered in the OS. (it runs off RAM and basically gets recreated from scratch every time you start it)
I personally consider VM's not to be save as there have been security related issues with some VM infrastructure and the separation is not absolute from the base system.

As my journey in crypto assets progressed, i'am now a firm advocate of hardware wallets and i'm using a ledger nano x, the brilliance is that you generate your privat key completely offline and only connect wallets to it with the public key, any transaction needs to be confirmed on the device (your privat keys can only be regenerated by the initial seed phrase which you can keep offline)

A reputable hardware wallet is generally considered the best way for most users.
BIP39 refers to the mnemonic recovery phrase, but BIP39 wallets still use the BIP32 architecture.
The questions about why BIP39 is not implemented in bitcoin-core has already been answered (Is there a reason to why Bitcoin Core does not implement BIP39?)


text:
rust-bitcoin Non-canonical DER signature
I am trying to create a raw transaction spending a P2PKH output using rust-bitcoin, but whenever I am pushing the transaction to the testnet I receive the following error mandatory-script-verify-flag-failed (Non-canonical DER signature)
My signing code looks like this:
/// Sign a bitcoin transaction spending P2PKH outputs
///
/// # Arguments
///
/// * `tx` the transaction to be signed
/// * `script_pubkeys` list of script pubkeys of the inputs (has to be indexed in the same order as the inputs in the transaction)
/// * `skeys` list of secret keys with which to sign the inputs (has to be indexed in the same order as the inputs in the transaction)
/// * `pkeys` list of public keys for the inputs which are spent (has to be indexed in the same order as the inputs in the transaction)
/// * `curve` reference to the elliptic curve object used for signing
pub fn sign_transaction(tx : Transaction, script_pubkeys : Vec<Script>, skeys : Vec<PrivateKey>, pkeys: Vec<PublicKey>, curve : &Secp256k1<All>) -> Transaction {
    let mut signed_inp : Vec<TxIn> = Vec::new();

    for (i, unsigned_inp) in tx.input.iter().enumerate() {
        let script_pubkey = script_pubkeys.get(i).unwrap();
        let signing_key = skeys.get(i).unwrap().key;
        let pub_key = pkeys.get(i).unwrap();
        let sighash = tx.signature_hash(i, &script_pubkey, SIGHASH_ALL);
        let msg = Message::from_slice(&sighash.as_ref()).unwrap();
        let sig = curve.sign(&msg, &signing_key);
        // Standard P2PKH redeem script:
        // <sig> <pubkey>
        let redeem_script = Builder::new()
            .push_slice(&sig.serialize_der())
            .push_int(1)
            .push_key(&pub_key)
            .into_script();
        signed_inp.push(TxIn {
            previous_output : unsigned_inp.previous_output,
            script_sig: redeem_script,
            sequence : unsigned_inp.sequence,
            witness : unsigned_inp.witness.clone()
        });

    }

    Transaction {
        version : tx.version,
        lock_time : tx.lock_time,
        input : signed_inp,
        output: tx.output
    }
}

Here is an example transaction 01000000010b41a0a10f36c39b057053281d1d4d9f67410c40a093c351024f57af4e133c21010000006a46304402202d99eb85a14f483f4679b93eb1cf0f67b64d788dc4cd5ad782e75540508922ce0220752a9237e320497ba289a91edd7135dd6c53f9332dc33c71a3020bf0555607fc512103202430a99091407e6c724c5c88504e69ef6917f042a65cb990537922f823d9dfffffffff02e80300000000000017a9141831af16119be20b532668d5995bd05ac955153b872a091e00000000001976a9146b538e889cf0df7b69112b50db8faadb4e457b9288ac00000000
and in decoded form
{
   "result":{
      "txid":"b36457604e0ccd491e97c30c06a011fd906bb83d6fe81c676efb9fc60f17c5a5",
      "hash":"b36457604e0ccd491e97c30c06a011fd906bb83d6fe81c676efb9fc60f17c5a5",
      "version":1,
      "size":223,
      "vsize":223,
      "weight":892,
      "locktime":0,
      "vin":[
         {
            "txid":"213c134eaf574f0251c393a0400c41679f4d1d1d285370059bc3360fa1a0410b",
            "vout":1,
            "scriptSig":{
               "asm":"304402202d99eb85a14f483f4679b93eb1cf0f67b64d788dc4cd5ad782e75540508922ce0220752a9237e320497ba289a91edd7135dd6c53f9332dc33c71a3020bf0555607fc 1 03202430a99091407e6c724c5c88504e69ef6917f042a65cb990537922f823d9df",
               "hex":"46304402202d99eb85a14f483f4679b93eb1cf0f67b64d788dc4cd5ad782e75540508922ce0220752a9237e320497ba289a91edd7135dd6c53f9332dc33c71a3020bf0555607fc512103202430a99091407e6c724c5c88504e69ef6917f042a65cb990537922f823d9df"
            },
            "sequence":4294967295
         }
      ],
      "vout":[
         {
            "value":0.00001000,
            "n":0,
            "scriptPubKey":{
               "asm":"OP_HASH160 1831af16119be20b532668d5995bd05ac955153b OP_EQUAL",
               "hex":"a9141831af16119be20b532668d5995bd05ac955153b87",
               "reqSigs":1,
               "type":"scripthash",
               "addresses":[
                  "2MuT9izFXKft5umHqr1gviyhus6JGLCxGd6"
               ]
            }
         },
         {
            "value":0.01968426,
            "n":1,
            "scriptPubKey":{
               "asm":"OP_DUP OP_HASH160 6b538e889cf0df7b69112b50db8faadb4e457b92 OP_EQUALVERIFY OP_CHECKSIG",
               "hex":"76a9146b538e889cf0df7b69112b50db8faadb4e457b9288ac",
               "reqSigs":1,
               "type":"pubkeyhash",
               "addresses":[
                  "mqJShCuqM7FK5zPa7Z1PNJySMHTVPrK9bb"
               ]
            }
         }
      ]
   },
   "error":null,
   "id":"curltest"
}

Any help on what could be wrong is much appreciated

Can you please remove the line .push_int(1) first? This may or may not be your problem but that line needs to be removed.

I believe the intention of your push_int(1) is to add SIGHASH_ALL to the transaction.
However, one of Script's many needless inflexibilities is that the sighash flag actually needs to be part of the signature itself, rather than as a separate stack element in the script.
This means that you need to take the Vec<u8> returned from Signature::serialize_der, mutate this with sig.push(1), and then add it onto the script stack. By manually editing your transaction to change the 51 (OP_1) to 01 (the 1 byte), and increasing the length in the signature opcode to consume this, I was able to make your transaction pass validation on testnet. (I didn't submit it though, to save you the annoyance of dealing with "input already spent" errors!)
I'm not thrilled with this API -- I suppose at the very least that Signature should have a a method serialize_der_with_sighashflag or something to eliminate the need for the caller to use a temporary variable and mut, but it's not super common to be manually constructing scripts like this and I guess it hasn't come up.
You might consider looking at rust-miniscript which does a lot of this stuff for you.


text:
ScriptSig does not validate with seemingly valid signature
I am trying to build raw transactions from scratch to improve my understanding. For comparison I am using the hexoutput from https://coinb.in/ as I can choose Testnet on there.
I am trying to construct a P2PKH transaction without Segwit (for now) which should use this tx 577735da1fccf6df79eac9ff42049a7729703ec5506efeb1602bca99df3bafd2 (index 0) and send 6889990 satoshi to mv4rnyY3Su5gjcDNzbMLKBQkBicCtHUtFB, the raw public key of the input address is 034323ee9ac23504779b40c1b02578d27f70c12c22b7fc4f452dbbde681e45073d
So for the unsigned transaction my library produces 0100000001d2af3bdf99ca2b60b1fe6e50c53e7029779a0442ffc9ea79dff6cc1fda357757000000002221034323ee9ac23504779b40c1b02578d27f70c12c22b7fc4f452dbbde681e45073dffffffff0106226900000000001976a9149f9a7abd600c0caa03983a77c8c3df8e062cb2fa88ac00000000 which matches what is produced by coinb.
So to generate the signature for the sigscript I use the following code:
def sign_transaction(self, secrets):
    """
    Sign all inputs of the transaction with
    a secret key.
    :param ECPrivKey[] secrets: a list of private keys corresponding to the inputs
    :return: Transaction Signed version of the transaction ready to be published
    """
    unsigned = bytes.fromhex(self.serialize_to_hex())
    message = sha256(sha256(unsigned))
    copy = deepcopy(self)
    inputs = self.__inputs
    for i in range(0, len(secrets)):
        sk = secrets[i]
        inp = inputs[i]
        sig = SignatureFactory.gen_ecdsa_signature_bytes(message, sk)
        copy.set_signature(sig)
        sigscript = sig.get_asn1_encoded() + OP_SIGHASH_ALL
        l = len(bytes.fromhex(sigscript))
        script = compact_size_int(l) + sigscript + inp.get_sig_script()
        copy.__inputs[i].set_sig_script(script)
    return copy

Informally I use the hexoutput from the unsigned transaction, convert it to bytes, run two sha256 hash functions on it and append the SIGHASH ('01') byte
I validated the produced signature and it is valid so I feel like there is something wrong with the serialization. Here is my transaction after signing it:
0100000001d2af3bdf99ca2b60b1fe6e50c53e7029779a0442ffc9ea79dff6cc1fda357757000000006a47304402207da5c1e2d7db828e415db1ae060d65e4a44de2b13b71c50be61d6c52da82425b022010da9abb70c7b6047b559c5c572f68e078d6ae78ba03dfd6aa1890e512e376760121034323ee9ac23504779b40c1b02578d27f70c12c22b7fc4f452dbbde681e45073dffffffff0106226900000000001976a9149f9a7abd600c0caa03983a77c8c3df8e062cb2fa88ac00000000

{"version": "01000000", "flag": "", "inputLen": "01", "inputs": [{"prevOutHash": "d2af3bdf99ca2b60b1fe6e50c53e7029779a0442ffc9ea79dff6cc1fda357757", "prevOutIx": "00000000", "sigScriptLen": "6a", "sigScript": "47304402207da5c1e2d7db828e415db1ae060d65e4a44de2b13b71c50be61d6c52da82425b022010da9abb70c7b6047b559c5c572f68e078d6ae78ba03dfd6aa1890e512e376760121034323ee9ac23504779b40c1b02578d27f70c12c22b7fc4f452dbbde681e45073d", "sequence": "ffffffff"}], "outputLen": "01", "outputs": [{"value": "0622690000000000", "pubKeyScriptLength": "19", "pubKeyScript": "76a9149f9a7abd600c0caa03983a77c8c3df8e062cb2fa88ac"}], "locktime": "00000000"}

In more detail the value of sigscript:
47 Script length
30 DER signature marker
44 Signature length
02 r value marker
20 r value length
7da5c1e2d7db828e415db1ae060d65e4a44de2b13b71c50be61d6c52da82425b r value
02 s value marker
2010da9abb70c7b6047b559c5c572f68e078d6ae78ba03dfd6aa1890e512e37676 s value
01 Opcode SIGHASH
21 pubkey length
034323ee9ac23504779b40c1b02578d27f70c12c22b7fc4f452dbbde681e45073d pubkey value

If I try to broadcast the transaction on https://live.blockcypher.com/btc-testnet/pushtx/ I get Error validating transaction: Error running script for input 0 referencing 577735da1fccf6df79eac9ff42049a7729703ec5506efeb1602bca99df3bafd2 at 0: Script was NOT verified successfully..
Does anybody have an idea what could be wrong? Also the r and s value of the signature I did not encode in little endian and I am not sure if maybe this could be the problem?

Signing transaction is a bit more complicated than just signing the hash of unsigned transaction
Look at the explanations here:

Redeeming a raw transaction step by step example required
How to redeem a basic Tx?

Thanks I tried to sign with the scriptSig filled with the scriptPub from the previous output, so what I am signing now looks like:
01000000 01 d2af3bdf99ca2b60b1fe6e50c53e7029779a0442ffc9ea79dff6cc1fda357757 00000000 19 76a9143ebad4eac79369b9c8fd6c3033386729a041f65d88ac ffffffff 01 0622690000000000 19 76a9149f9a7abd600c0caa03983a77c8c3df8e062cb2fa88ac 00000000
unfortunately still fails to validate.

where is SIGHASH_ALL (4 bytes LE) at the end?

Shouldn't the SIGHASH_ALL only be included after the signature is added? According to your reference "Now comes the scriptSig. For the purpose of signing the transaction, this is temporarily filled with the scriptPubKey of the output we want to redeem."

Look to the step #13 in the instructions above


text:
OP_CHECKMULTISIG Signature must be zero for failed CHECK(MULTI)SIG operation
I am trying to spend a P2SH multi-signature output which I have created with rust-bitcoin as follows:
let builder = Builder::new()
        .push_opcode(opcodes::all::OP_PUSHNUM_2)
        .push_key(&pub_a)
        .push_key(&pub_x)
        .push_opcode(opcodes::all::OP_PUSHNUM_2)
        .push_opcode(OP_CHECKMULTISIG);
Script::new_p2sh(&builder.into_script().script_hash())

For creating the scriptSig I have the following code:
let sig_a = secp.sign(&msg, &sk.key);
let sig_x = secp.sign(&msg, &x.key);
let sig_a_der = serialize_sig_der_with_sighash(&sig_a, SIGHASH_ALL);
let sig_x_der = serialize_sig_der_with_sighash(&sig_x, SIGHASH_ALL);
let lock_script_bytes: Vec<u8> = lock_script.to_bytes();
// Now we need to combine the original P2SH script and the actual redeem script
let fin_script = Builder::new()
    .push_opcode(opcodes::all::OP_PUSHBYTES_0) /// Needed because of and issue on OP_CHECKMULTISIG
    .push_slice(&sig_x_der)
    .push_slice(&sig_a_der)
    .push_slice(&lock_script_bytes)
    .into_script();

Now I have tested this on the testnet and am receiving

I have tried to debug this using btcdeb and am seeing the following stack before the OP_CHECKMULTISIG executes:
#0015 OP_CHECKMULTISIG
btcdeb> stack
<01>    02      (top)
<02>    02952453888ebcb21720aa81ef9c43ffdb3127b30c5e2d0f61c1c1550b7a29babf
<03>    03dcc4a286cab7209043e5658bfe6ef08adde7128ee5d9bf74862f27ff15f9d519
<04>    02
<05>    3045022100ae1fe1fb0b02586b012ff01be7835269e6079b85f089123ecaf6a482e0324ca602206d2236fd0d07e018097c19e2c9bb8d0b31e18cf51624275b36e596b1969ac3b901
<06>    3045022100bbf92e527688209bacb3e75501fdf1744a7e6e8cd2c922b499d7b40b8dff6260022045845247d1a1fe04aa1ab1866473acf06abc63d5ecdebe6dae0ff7be3200795f01
<07>
btcdeb> step
                <> POP  stack
                <> POP  stack
                <> POP  stack
                <> POP  stack
error: Signature must be zero for failed CHECK(MULTI)SIG operation
btcdeb>

Afterwards, the stack looks like this:
btcdeb> stack
<01>    3045022100ae1fe1fb0b02586b012ff01be7835269e6079b85f089123ecaf6a482e0324ca602206d2236fd0d07e018097c19e2c9bb8d0b31e18cf51624275b36e596b1969ac3b901                                       (top)
<02>    3045022100bbf92e527688209bacb3e75501fdf1744a7e6e8cd2c922b499d7b40b8dff6260022045845247d1a1fe04aa1ab1866473acf06abc63d5ecdebe6dae0ff7be3200795f01
<03>

The message for the signature is created as follows:
let sighash = tx.signature_hash(ix, &script_pubkey, SIGHASH_ALL.into());
println!("msg: {}", hex::encode(&sighash.as_ref()));
let msg = Message::from_slice(&sighash.as_ref())
    .unwrap();

which outputs 0312f22fd74e55216e51c2724051d8cf9c8833edc1596871fc357a71b3826634.
It seems to me that something is still wrong structurally because after the OP_CHECKMULTISIG has been executed we see that both signatures are actually still on the stack and haven't been popped.

Usually problems like this are the result of constructing the message incorrectly. What is the data in script_pubkey? It needs to be the multisig script that you are using, not the actual scriptPubKey.

The data in script_pubkey when constructing the signature message is OP_HASH160 <hash> O_EQUAL so the scriptPubKey of the input spent. Do I understand correctly that this is incorrect and should actually be the script that is being executed (and wrapped in the P2SH)?

Yes. For P2SH, the script you should be using is the script being executed, i.e. the redeemScript.

@AndrewChow Wow, that fixed it! (well that and I also had the order of the signatures mixed up.) Thanks a lot for your help!


text:
Where does this pattern in the feerate distribution per block originate from?
While building a tool/visualization that displays the feerate of a transaction according to it's position in a given block I came across a patter in some blocks.
I expect a typical block to start with the highest fee per byte ratio (feerate) transactions and then continue with transactions where the feerate slowly decreases. Often there are some CPFP transactions where a lower feerate is followed by a higher feerate.
However, what I'm seeing is, that in some blocks (from different miners) are mixed 9 sat/B transactions with 5 sat/B transactions in between. At a later position in the block there are 5 sat/B transactions again. This happens over multiple blocks and different miners. I guess it originates from getblocktemplate or something similar.
I've forked a version of my tool an highlight 5 sat/B transactions in blue (Can be found here, feel free to explore). Since this is still under active development, I'll include screenshots of some blocks in case the links stop working. I deliberately selected a set of blocks where I think it's clear what I mean.

Sample Blocks:

Block #517361 mined by ViaBTC (?)

Block #517363 mined by BTC.TOP (?)

Block #517357 mined by BTC.com (?)
and here is one from Slush too.

Questions:
1) Is this known behavior of an algorithm that creates block templates?
2) What motivation is there behind this feature. Is it a bug?
3) Are there commonly used getblocktemplate alternatives?

1) Is this known behavior of an algorithm that creates block templates?

Yes. This is behavior that is present in Bitcoin Core.
Bitcoin Core bundles transactions into "packages" of one or more transactions. Each package consists of an unconfirmed transaction and its children (if any) to cover the case of Child-Pays-For-Parent. The transaction fee rate is calculated for the entire package (total fees paid by the transactions in the package divided by size of the package).
When transactions are selected, it's really the packages that are selected. Because the packages are put inside the block in the order of package fee rate, you will sometimes get a few low fee rate transactions followed by a really high fee rate transaction because they were all part of the same transaction and the high fee rate transaction was a child that paid for it's parent(s).
This is called "ancestor fee rate ordering".

2) What motivation is there behind this feature. Is it a bug?

It is neither a feature nor a bug, it is just a quirk of how the transaction selection works.

See also: https://bitcointalk.org/index.php?topic=2058831.0

Thanks Andrew, but this does not really answer my question. I mentioned CPFP transactions in my question with the typical lower fee tx followed by a high/higher fee tx which are easy to see in the visualisation. But that's not what I mean. E.g. in block #517363 are ~30 5 sat/B tx which are followed by a single 9 sat/B tx and then again by multiple 5 sat/B tx (around position 1200). This (I believe at least, will check later) does not seem like CPFP, or at least is not the correct position in the block for a big package of these 31 tx's. (305sat/B + 19sat/B)/31tx would be later in the block.

I think you are actually calculating the feerate incorrectly. I double checked and I am not getting the same feerates as you are for those transactions at the given block indexes. The numbers that you have match up to sat/byte, not sat/vbyte. I think you aren't calculating the feerates for transactions spending segwit inputs correctly.

yes, you are correct. I discovered it like 5 minutes ago while looking at the transactions mentioned above. Thanks!

As an aside, I wrote a quick python script to calculate what transactions are packaged together in a given block: https://gist.github.com/achow101/dfb1016f78e653656ec40cd019f8116b. I think it works.

As Andrew Chow pointed out, my feerate calculations were wrong. I used the totalSize of the transaction. Correct would have been to use the virtualSize.
With correct calculations the visualization looks far more as I expected.

Block #517361 (compare above)


text:
Where do you get the transaction data & where do you submit your work?
The concept of bitcoin mining (hash functions & the goal) are well elaborated on many sites. There are also lots of examples.
However, when going into real production I guess you need data like the latest hash, info about the block and transactions and so forth.
In addition, once you make your guess, you have to submit the work somewhere.
I can however not find any resources that explain where to do this? Is there an API to which you submit your work, to which you attach also your bitcoin address of your wallet, and once you make the right guess the btc is transferred to this account?
What are the bitcoin endpoints in my case?

when going into real production I guess you need data like the latest hash, info about the block and transactions and so forth.

If you're using a mining pool, you would use Stratum to subscribe to a mining pool's job notifications and their notifications would include this information. If you're using bitcoind to mine on your own, you'd use getblocktemplate. Additionally, you can build the template yourself.

Is there an API to which you submit your work

If you're trying to submit a block to the Bitcoin network itself, there is the submitblock bitcoind RPC call that will allow you to submit a full block. If you're trying to submit shares to a mining pool, you'd probably use the mining.submit Stratum RPC call.

to which you attach also your bitcoin address of your wallet, and once you make the right guess the btc is transferred to this account?

If you're using a mining pool, you specify this information at the mining pool's website. The block work they give you will be paying to their address, so it's irrelevant.
If you're mining yourself, you'd specify this by including a "coinbase" transaction at the beginning of the block that pays an address you specify the appropriate block reward and transaction fees.

Thank you for the explanation. I still don’t understand to which address I have to connect? What are the configuration details?

@JohnAndrews If you're mining on a pool, then the pool will provide you with their Stratum URL and allow you to configure your worker credentials. If you're mining directly on the Bitcoin network, you would be running a regular Bitcoin node just like anyone else and broadcasting your blocks that way.

What I dont like with the pools is that it seems that its not possible to configure your own methodology. Basically what U am looking for is: 1) give me the inputs for guessing, 2. I do my own guessing, 3 I submit the work. But for that I have to run a full node with all the historical transactions downloaded?

@JohnAndrews Unless you're doing it only to learn how mining works (and understanding that you will not actually make any money), it's completely pointless to mine on your own because you will literally never earn a single penny. If you want to generate a valid block template (for educational purposes) though, I believe all you need is a reliable source of the latest mined block, the current full UTXO set, and the mempool. You'd need to validate the latest block and include its hash, then you'd need to validate the unconfirmed transactions you're including in your block against the UTXO set.


text:
Trying to run bitcoind gives Segmentation Fault
Trying to run bitcoind on Ubuntu 16.04 and have all the required libraries after following the guide in the Mastering Bitcoin Book. After building from source without any parameters triggered i get the following error:
bitcoind
Segmentation fault (core dumped)
Apparently this is due to trying to access a memory index out of bounds however does anyone know how i might be able to resolve it?
Any help would be great :)

This sounds like an issue for the GitHub issue tracker, not stack exchange :)
https://github.com/bitcoin/bitcoin/issues
Please include the core dump, and which version of bitcoin you built.

Sharing a core dump file isn't a good idea, because it might contain your private keys, and then anyone would be able to steal all your coins (or coins you might receive with that wallet in the future).  If you've already done so then you should create a brand new wallet and send all your coins there immediately.


text:
Is there a guide to uninstall bitcoin core ? Installation mess
Hi i'm wanting to do a full uninstall of bitcoin core and to follow a book that i've recently bought on programming on the bitcoin blockchain.
I've tried removing all the data and running:
sudo systemctl disable bitcoind
however BC still starts on startup and the above command does not work
Failed to disable unit: No such file or directory
My installations are probably a mess i've got the normal bitcoin core install pulled down and running as well as the github project all build and installed also therefore i'm not sure exactly whats running.
Please can someone provide some help.

Have you tried to uninstall it using your package manager, e.g. sudo apt-get remove bitcoind?

Package 'bitcoind' is not installed, so not removed
The following packages were automatically installed and are no longer required:
linux-image-extra-4.10.0-19-generic linux-image-extra-4.10.0-20-generic
linux-image-extra-4.10.0-21-generic linux-image-extra-4.8.0-46-generic
Use 'sudo apt autoremove' to remove them.
0 to upgrade, 0 to newly install, 0 to remove and 267 not to upgrade.
Ommitted all the packages - There's still the problem too of locating where the actual data is


text:
Setting up the bitcoind to run at startup Ubuntu 17.04
I've created a bitcoin.service file as found here:
https://github.com/bitcoin/bitcoin/blob/master/contrib/init/bitcoind.service
However i've modified it as i build bitcoin-core from source to the following:
https://gist.github.com/anonymous/7e1bca97739147d698327d7f3a119992
It is installed to:
/lib/systemd/system
As apparently in this version of Ubuntu it must be there
It runs when i run this command:
bitcoind -daemon -pid=/home/chris/.bitcoin/bitcoind.pid -
conf=/home/chris/.bitcoin/bitcoin.conf -datadir=/home/chris/.bitcoin/
-disablewallet

However trying to run:
systemctl start bitcoind

Gives this error
bitcoind.service - Bitcoin's distributed currency daemon
   Loaded: loaded (/lib/systemd/system/bitcoind.service; enabled;
vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2017-10-15 17:27:38
BST; 2s ago
  Process: 4180 ExecStart=/home/chris/.bitcoin/bitcoind -daemon -
pid=/home/chris/.bitcoin/bitcoind.pid -
conf=/home/chris/.bitcoin/bitcoin.conf -datadir=/home/chris/.bitcoin/
-disablewallet (code=exited, status=203

Oct 15 17:27:38 chris-GE60-2PL systemd[1]: bitcoind.service: Unit
entered failed state.
Oct 15 17:27:38 chris-GE60-2PL systemd[1]: bitcoind.service: Failed
with result 'exit-code'.
Oct 15 17:27:38 chris-GE60-2PL systemd[1]: bitcoind.service: Service
hold-off time over, scheduling restart.
Oct 15 17:27:38 chris-GE60-2PL systemd[1]: Stopped Bitcoin's
distributed currency daemon.
Oct 15 17:27:38 chris-GE60-2PL systemd[1]: bitcoind.service: Start
request repeated too quickly.
Oct 15 17:27:38 chris-GE60-2PL systemd[1]: Failed to start Bitcoin's
distributed currency daemon.
Oct 15 17:27:38 chris-GE60-2PL systemd[1]: bitcoind.service: Unit
entered failed state.
Oct 15 17:27:38 chris-GE60-2PL systemd[1]: bitcoind.service: Failed
with result 'exit-code'.

Does anyone know what i'm doing wrong here ?

Usually, -conf={file} not {path}/{file}

It runs when i run this command:
bitcoind -daemon -pid=/home/chris/.bitcoin/bitcoind.pid -
conf=/home/chris/.bitcoin/bitcoin.conf -datadir=/home/chris/.bitcoin/
-disablewallet

When you run that command, where are you in your server? At the base level of your home folder? Your service file ExecStart= must point to the same location, and currently it's looking for bitcoind in /home/chris/.bitcoin/. Are you sure bitcoind is inside the directory .bitcoin in your home folder?
As a side note, that is a odd location for the binary. Consider placing bitcoind and bitcoin-cli in /usr/local/bin/. That way you can run it from anywhere in your server without specifying a path. But for the service file, you'd then should specify the full path, such as
ExecStart=/usr/local/bin/bitcoind -daemon -pid=/home/chris/.bitcoin/bitcoind.pid \
-conf=/home/chris/.bitcoin/bitcoin.conf -datadir=/home/chris/.bitcoin/ -disablewallet

You should also consider changing the location of the PID file and you can also omit -datadir=/home/chris/.bitcoin/, as bitcoind will by default look for the data dir in that location, since the service is being started by the user 'chris'. These are suggestions taken from the official Bitcoin template you linked in your post.
Don't forget to sudo systemctl daemon-reload after you make changes to systemd service files, before sudo systemctl restart <service-name.service> for example.

sudo systemdctl daemon reload should be sudo systemdctl daemon-reload note the dash.

I happen to run across this, and I don't know if you ever got this working.  Try using the correct executable in the bitcoind.service file for the ExecStart line (11 of your gist).  My bitcoind is installed in /usr/bin/bitcoind.
I also commented on your gist above with an example.

authors: Сергей Мишин


text:
Why would one buy a Bitcoin future or perpetual as opposed to actual Bitcoin via the spot market?
I often hear that the futures and perpetual markets for bitcoin help its price discovery. So this must mean that these markets for bitcoin are quite active.
But would people want exposure to bitcoin through futures or perpetuals when they can just buy the asset directly using a spot market?


text:
Is there a standard metric or approach to measure Bitcoin’s price volatility over time?
I am wondering whether there’s a standard here. By standard I mean a common or generally used method.


text:
Transaction propagation
I`ve got one really basic but still important question that is very confusing:
Does the newly created transaction needs to be propagated to ALL nodes in the network in order to be validated?
Lets suppose I have made some payment and it needs to be added to the mempool of few miners to be included in the next block. How many nodes must validate that transaction so that it will be added to the mempool?
Thanks in advance!

Does the newly created transaction needs to be propagated to ALL
  nodes in the network in order to be validated?

No. The nodes relay valid transactions to each other. In corner case you can send your transaction directly to miner node. No one will have this transaction until it included in block

Thanks for the answer! So if I understand it correctly: when node receives any transaction it will try to validate that and (if that tx is valid from the perspective of that node), that node is going to put tx in the mempool and send that tx to another neighbor node? And this proces repeats N times, basically until that tx is included in the block or all nodes have that tx in the mempool?

Yes, you are right. Note, that nodes are not obliged to relay or/and keep txs in their mempool.


text:
Blockchain and anonymous data inside the chain
A Blockchain is often said to be a log of all transactions that were ever verified on a network.
I understand that therefore a purpose could be to make sure that the records are not altered. If they would, the chain would fail by its construction.
However, what do you do when you have sensitive data that you want to store in these transactions? The reason I am asking is because the Blockchain being a Distributed Ledger, everyone has access to view all the transactions/chain.
Use case example: bank transactions
A bank could launch a Blockchain to make sure that the transactions of their clients are irreversible recorded. However, these transactions might contain sensitive information (like the details of their client). When a new block is created, the entire Blockchain is validated, meaning that the person who is creating the transaction has access to the whole chain and thus the details of the client in the previous transactions.
What is then the use case and how do people look at it this type of problem from their perspective?

Bitcoin blocks do not store any sensitive information. They store transactions where each transaction is composed of one-or-many inputs and one-or-two outputs. Inputs and outputs are simply public addresses.
If you can identity addresses, then you might be able to discern some sensitive information. But the blockchain does not store any personal information.

However, these transactions might contain sensitive information (like the details of their client).

Bitcoin transactions simply do not contain sensitive information, they need no other information other than the amount and the receiver, meaning that the publicity of it is of lesser concern. There was still implicit privacy issues in Bitcoin, but you avoid any issues with personal information being stored in the chain by just simply not doing anything that would cause this information to be included.


text:
What makes the bitcoin network decentralized? is it the nodes or the miners or both?
I wonder what makes the bitcoin network decentralized?
Do nodes participate in making the network decentralized? or is it just the miners and all about the hash power to take over the network aka 51%< attack ?

'Decentralized' is not a binary distinction. There is no point at which suddenly something is decentralized, or not. For the bitcoin network to function there is some level of decentralization required though, as otherwise the guarantees of censorship resistance will become less strong. Bitcoin is defined from the bottom-up, not the top-down!
Bitcoin full nodes and bitcoin miners are distinct in their function. In short:
A decentralized network of full nodes helps to ensure that the network's rules remain unchanged, as each node will independently verify the network state for it's own purpose. For example, if there weren't a lot of nodes, then a majority of the nodes that did exist could perhaps collude to change the rules (there is nuance to this though).
A decentralized network of miners helps to ensure that bitcoin remains secure and censorship resistant, as a more distributed network hashpower helps to ensure that not single entity (or group of colluding/coerced entities) will be able to perform a majority attack.
Without a sufficient amount of decentralization of both nodes and miners, bitcoin ceases to be very interesting or useful.

It depends on what you mean by decentralization. There is a technical meaning, and an intuitive one.
I'll talk about the first one first. A decentralized system is one which by design functions and remains secure without the presence of a trusted central party (or parties). I can't tell you what part of Bitcoin makes it have this property - it just happens to be the case that it does, at least under certain assumptions. Both miners and nodes matter. We need miners to avoid a central transaction approves with censorship rights. You avoid the need to trust that miners produce honest blocks by validating their chain yourself (= running a node).
Often the word "decentralization" is used in a much more practical sense, where far more aspects of the design and real-world implementation are included. It's hard to say what this includes, as there isn't a good definition. Does the entire ecosystem just go along with what a single, perhaps small, developer team proposes? That's arguably a form of centralization in practice, but it's one at a social/human layer, not one that is a technical design property of the system. Similarly, does all trading go through one or a few exchanges? That too can be considered a central party if its relevance to the ecosystem becomes too great. The same is true for wallet software for example - if everyone is using the same wallet software that auto-update, then perhaps the person with the keys to push such updates is a centrally trusted party.
I personally dislike this use of the term decentralization as it is far too vague, and unbounded. Are Intel and AMD and ARM central parties, because everyone uses chips produced/designed by them? For this reason I'd reserve the use of the term decentralization for the technical design, and use something like "trust minimization" for all the rest.


text:
How to secure bitcoin node (bitcoind) that is used only to broadcast transaction?
I always wanted to run a bitcoin node at home. But I always worry because it is related with bitcoin (BTC has value).
I have been broadcasting my transaction using Bluewallet by scanning qr code, or copy pasting a signed transaction string. I am wondering if there is any security risk in doing so. From here I thought it's better to set up my own node just to broadcast transaction using sendrawtransaction
So I have some questions on what I am about to setup.

Is this bitcoin.conf secure for the purpose?

# network
testnet=0
#proxy=
#addnode=
#connect
#noirc
maxconnections=20

# JSON-RPC
# accept JSON-RPC?
server=1
rpcuser=secretrpcuser
rpcpassword=secretrpcpassword
#rpctimeout=
rpcport=8332
rpcconnect=192.168.11.5
#rpcssl
daemon=1

As I disabled rpcssl on my bitcoin.conf, is it secure to sendrawtransaction without SSL?

Is it more secure to broadcast transaction using sendrawtransaction on my private node? Or is it better to do it on public nodes like bluewallet, or sites like pushtx.com or sites lsited here: https://en.bitcoin.it/wiki/Transaction_broadcasting

What are the possible security risks of running own node at home?

What are the possible risks on doing sendrawtransaction from private home bitcoin node? Also the risk for doing it from public nodes?

Thank you in advance

Thank you for your answer. I’m afraid to connect to Tor since I watched and read stuffs about the darkweb, there seem to be a lot of illegal and immoral stuffs in there. Don’t want to get involved in there. Is there any risk for running a bitcoin node without tor? And any benefit running it with tor?

Thank you for your answer. Is there any security risk if I expose my IP address with my transactions? I think I have been exposing my IP addresses with my transactions using bluewallet, electrum, etc without tor. Should I create a new wallet?


text:
Transaction to "dynamic" wallet doesn't arrive?
about one week ago I tried to make a deposit to a service called "escrowbitcommunidad" (some of you might know it). They are using some kind of "dynamic" wallet addresses, meaning each time one reloads the payment page, the BTC address they display changes (please excuse my lack of professional terms).
Now, I sent the bitcoins to the address they displayed using the etherum BTC client, the transaction ID is:
1968cc2b31b396c3ef0677c69ed68a588f6bdec1ed9e60fcb8500fddda46b068
From this, it seems like the transaction is completed already, however the BTC did not arrive to my account so far and from what I've heard from other users, this is a common problem caused by some delay in the blockchain (?) system.
As you will have noticed, my understanding of how bitcoin transactions work is very rudimentary. Can someone briefly explain me where the problem could be?

It appears the transaction with that id actually exists, you can verify it because Bitcoin transactions are public https://www.blockchain.com/btc/tx/1968cc2b31b396c3ef0677c69ed68a588f6bdec1ed9e60fcb8500fddda46b068. Indeed since it has ~900 confirmations, at an average block time of 10 minutes (Bitcoin protocol rules), it was first confirmed about a week ago.
The only explanations I can think of is related to the service and their application. If they do not choose to acknowledge the transaction or that it was sent to an address you own, that is between you and them.

from what I've heard from other users, this is a common problem caused by some delay in the blockchain (?) system

To answer this question, when you first submit a transaction to the Bitcoin network, the nodes will validate it, and if it passes, it will propagate throughout the nodes on the network. It is added to each node's mempool, which is where the transactions wait until a miner decides to add them to their block, and attempt to mine that block for a reward. When a miners decide to add your transaction to the next block is entirely up to them, but is generally dependent on the fee you decide to give them. The wallet software you used lets you select this fee, so the higher the fee the faster it will be mined (generally).
Another thing to understand is the application that is waiting for your transaction can decide how many blocks need to be mined after your transaction was submitted. This is known as confirmations. The reason for waiting more than 1 block is that if there is a conflict and 2 miners mine a block at the same time (unlikely), there is a reorganization of the blockchain, and your once confirmed transaction may become unconfirmed. So, it is a way to establish how permanent that transaction must be to be considered final. 6 is a typical minimum confirmation in Bitcoin, but can be higher or lower depending on the service or application. ~900 confirmations is much higher than usual and there should be no problem with confirmations with the service you used.

Thank you fore the explanation! Now, the only thing that I still don't understand is how the "dynamic" (changing) wallet addresses work. From what their support old me, the problem might be caused by the fact that I copy+pasted the wallet address they displayed, but then reloaded the page before I sent the BTC, so a new wallet address was already generated. And therefore, it might take longer for the system to recognize the transaction? That is the only way that I could think of...

You're welcome, unfortunately this is more of a question for them, since it really depends on their system. My guess is that they generate an address, that they then "watch" for your transaction, and when they see it, the system marks it complete. It sounds like they only watch the latest address you generated, so you'll have to work it out with them.


text:
Confused about UTXO management and consolidation
I have a few questions regarding utxo management and consolidation. I know that the more UTXO's you have, the heavier your transaction will be, the more fees you'll have to pay.
The things I don't understand:

How can you effectively avoid that? Let's say I generate a new address every time I self-custody bitcoin, it won't solve the problem, does it? When I'll want to send either the max balance or let's say a rather large amount that needs a lot of utxo's to be complete, the full tx's will be heavy no matter what, correct? So using the same address for cold storage (i.e just a receiving address) or using a new one every time doesn't change a thing?
If a wallet has coin control, does it really matter to do utxo management?
Let's say I have a single address with 100 UTXO associated to it. I want to consolidate everything. I first generate a new address on my wallet, test that address by sending a small amount (from an exchange let's say). If everything seems correct, I can then use the send max feature of my wallet to that address (everything being under the same seed)? Even though it will effectively also send the utxo I just used for testing the validity of the address back to itself?

I think that's all my questions, thanks.

Here is a brief outline on how to do UTXO management:

You should consolidate your UTXO on weekends for whenever bitcoin network is less congested. This way you consolidate UTXO with lower fee (around 1 satoshi/byte). Later when you spend money for ecommerce purpose you can afford to pay higher fee/byte as your UTXO count is low. Services like blockonomics will show a TxFee score indicating fragementation of our wallet, which you can optimize
Always use native segwit wallets so that when you spend the UTXO you save upto 38% on fee
Using a single address to receive funds will not help in decreasing UTXO count. Even in case of single address, UTXO count will increase as each receiving txid will have a new output
Note that when you spend bitcoin there will always be some loss of privacy which should be fine for normal users. For high level of privacy you have to use things like coinjoin/mixer which will then again increase the fee you pay

UTXO management is not a singular goal. instead, it's a method for balancing multiple priorities, including privacy, short-term fee savings and long-term fee savings.
Having lots of UTXOs is probably better for privacy, especially if they're each in different addresses. But like you said, that will cost more in fees if you want to send your entire balance.
On the other hand, having all your bitcoin in one UTXO is probably not best for privacy or security.
Coin Control is a feature of some wallets, that allows users to directly control their UTXO management strategy. Wallets that don't use coin control make those decisions for the user instead, so Coin Control is usually best for advanced users who know what UTXO management strategy they want.

But so it means no matter what, if you are a long term holder/has been in Bitcoin for a while and you don't transact a lot (i.e you've been stacking sats regularly), at some point in time if you wish to transfert your entire balance for whatever reason you're going to find yourself with a huge transaction fee (unless you did consolidate within one UTXO when the mempool was empty but you gave up on privacy).
Also, can you expand on how having only one UTXO is not good for security?

I'm no expert in this matter, but I'll try to answer.

Consolidating all your bitcoin into one UTXO means, if an observer knew one of your UTXOs, they now know all your UTXOs, and can trace your other UTXO histories.

Yes, it's a problem that sitting on lots of small UTXOs may, in the long term, cost you lots of fees. But I'm not sure why someone would ever consolidate all their UTXOs at once.


text:
Loading private key (missing bitcoin)
I exported the private key from a 0.7.2 because I could not how to upgrade the wallet.  I then imported it into 0.18.0 I was expecting to see a  balance and the transactions and nothing was found.  I also searched the blockchain through blockchain.com using the public address from the original wallet and again found nothing.  Would greatly appreciate advice

Can you post the address here? Perhaps it is an old address, that held some of your coins at one point, but no longer does? Are you familiar with what a 'change address' is?

If you have bitcoins in any address, they should be there unless someone else got access to your wallet and stole them, in which case you should see in a block explorer the corresponding transaction.


text:
How can I clone bitcoin 0.16.1 version (not the latest one ) with git command
this is the version that I want to clone :
https://github.com/bitcoin/bitcoin/releases/tag/v0.16.1

You cannot directly clone a tree you have to first take the full repository then checkout the commit you want.
For the 0.16.1 version execute these commands in a shell:
git clone https://github.com/bitcoin/bitcoin
git checkout dac5d68

more informations here: https://stackoverflow.com/questions/43411941/i-cannot-clone-git-tree

Easier: git checkout v0.16.1


text:
Where can I find a high performance search capabilities in a Bitcoin node implementation?
Where can I find a Bitcoin node implementation, which is capable of thousands of searches per minute, without degrading performance of the node?
I am aware of two Bitcoin node implementations.

Bitcoin Core
Bitcoin Ops btcd

I am looking for a BTC node implementation, which implements higher performance search capabilities, so I can search using syntax similar in capabiilty to SQL. I understand why LevelDB was chosen in the Bitcoin Core implementation (not sure what is used in the btcd implementation), but search performance does not stand up to my use case of executing thousands of searches (using bloom filters) per minute, as described here.
I have written a searchable database back-end using PostgreSQL, but its database must be pruned in order to archieve the high performance I require, which is not optimal. Thinking out loud, it seems like a NoSQL database like Cassandra, MongoDB, or the like might be better suited to this use case. But, I have not been able to find a Bitcoin implementation, which uses a NoSQL or comparable database.
With that that in mind, does anyone know of a Bitcoin, or altcoin, node, which meets my use case?

Perhaps you're not really looking for node software, but separate indexing software? The complexity of fully validating nodes is nontrivial, and part of the reason why Bitcoin Core doesn't have advanced indexing functionality is that it can also be provided by other software (e.g. electrs) which then doesn't need to perform validation.

That sounds like what I am searching for.  Googling a bit I found https://github.com/spesmilo, but I am not sure if there is an "official" source repo for electrs.  Do you know of one?

After a bit more googling and reading, it seems like you might be meaning ElectrumX? https://github.com/kyuupichan/electrumx  Is my understanding correct?

No, I meant https://github.com/romanz/electrs. But there are probably dozens of other pieces of software that can provide various kinds of blockchain indexing.

Thanks, that answers my question. If you want to submit an answer for it, I will mark it as the selected answer.