text:
What is the most successful method to automate circular rebalancing?
What is the most successful method to automate circular rebalancing?
We have tried the Torq and the LNDG app but so far they both do not wotk for us so we are still performing manual circular rebalancing using RTL. Please can someone suggest a tutorial on how to use Torq and or LNDG or even another method?
Thank you. :)
Orange_Bonzo

"circular rebalancing" these stuff really works?

Just In Time rebalancing makes sense if you don't hold on the HTLC for too long, it also makes it easier to reason about the benefit/risk of triggering a rebalance (since you only do so if it comes at a lower cost than what you'd get from forwarding the HTLC). It's not always possible though.

If you run lndmanagej by c-otto there are a bunch of tools and scripts for finding cheap offchain liquidity. But as antoine noted. JIT routing seems more reasonable

Hi Antoine,  Thank you for this information. Being somewhat of an adolescent newbie I am not familiar with "if you don't hold on the HTLC for too long" and also this "than what you'd get from forwarding the HTLC" We are using Umbrel and know what an HTLC is but do not know how it affeccts liquidity or how it can be used to gain more liquidity as oppsoed to a circular rebalance. Please forgive my ignorance as I have not seen this information on any of the websites I have studied or from forum posts I have received an answer too.  Thank you

In answer to "vincenzopalazzo" Yes Vincenzo Circular Rebalancing really works and we use Ride The Lightning where 100k of sats at 125 PPM (Fee rate Rebalancing) almost always works especially if you have LNBig as one of the peers you are sourcing your sats from and the peer Nicehash also works and a few others too. When that does not work we sometimes use the percent type of rebalance in RTL where 0.25 percent mostly works and sometimes up to 500k of sats will go through successfully. Best of luck to you.

Thank you Rene. We will try Carsten's lndmanagej script and see how that works. So is there any way to Automate the process of aquiring inbound liquidity that works and if not do you see any new tools being created in the future that will perform this function? Thank you for your input.


text:
For mobile wallets that don't ask for a password, how are keys encrypted on your device?
I've used wallets that give a seed phrase, but don't ask for a password to send coins (signing a transaction).
I can't imagine they encrypt with the seed phrase since you don't ever re-enter the seed phrase to send a tx (only to restore an account on another device). I also can't imagine they would store the private keys in plaintext on the device.
So how does the mobile wallet access the private keys to sign your transaction when you send coins?

Since the wallet doesn't ask you for your seed phrase every time you need to sign a transaction, the logical conclusion is that it is indeed storing your private keys. The specific method and security measures might vary between apps and operating systems, but one such method is the Android Keystore.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
I used support bitcoin wallet address by mistake! How do I get that back to put into my wallet
New to bitcoin and I use their support Bitcoin wallet address instead of using my wallet. How do I get those funds back?

I use their support Bitcoin wallet address instead of using my wallet

Tell them you sent them money by mistake, mention the transaction ID, and ask them to send you the same amount back (less transaction fees).
How you do this depends on who they are (and whether you know who they are).
Since there are no Bitcoin police and no-one in charge of Bitcoin, you will likely have to rely on the goodwill of the recipient.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Do nodes receive information from which they can discern how long a miner has withheld publishing a block from the time it found it?
In a recently Medium article, Joe Kelly described an attack that he calls "Monopoly Mining" in which a state actor or some other very well resourced party that obtains majority hashpower does more than double-spend, they deny all other miners block rewards and thus disrupt the network (https://joekelly100.medium.com/how-to-kill-bitcoin-part-3-no-can-defend-cd6affe3fc44). The attack requires mining ahead but not publishing the chain being created until an opportune time when the majority miner's secret chain is sufficiently far ahead of the canonical chain. Then the majority miner publishes enough of the blocks to establish the new canonical chain taking away all of the block rewards of the now orphaned blocks. Then when any other miner tries to add a new block, the majority miner releases two more withheld blocks to keep denying any other miner block rewards.
Disregarding for the moment the expense required to carry out this attack and the question of whether any entity could attain majority hashpower and build a sufficiently long alternate chain to execute this attack in secret, it seems to me that withholding discovered blocks serves no useful purpose and that this attack could be dealt with by a soft fork that disregards any block that the applicable miner did not publish within a certain amount of time after it was found. But do Bitcoin nodes receive enough information to determine the amount of time that has passed from the time the block was found to the time it was published?
An Ivan on Tech article describes how the Horizen protocol penalizes nodes that delay publishing blocks (https://academy.moralis.io/blog/breaking-down-proof-of-work-mining-and-51-attacks). Has anything similar been seriously considered for Bitcoin?

In principle, a node knows the date and time at which transactions in the new block were added to this node's mempool. That provides an approximate upper limit to the time the block might have been withheld.
For example if at 10:00 the node receives a block which contains a transaction the node had received at 9:00 it is unlikely the block could have been withheld more than an hour.

But do Bitcoin nodes receive enough information to determine the amount of time that has passed from the time the block was found to the time it was published?

There is no feature of a block which would reveal this information. A block can be created at any time, with any details within, and it will be accepted by the network as valid so long as it passes the consensus rules. There are no changes that could be made to Bitcoin in order to make this sort of thing possible.

This answer is not correct. The time a block was withheld can be determined (not necessarily with perfect accuracy, but that doesn't matter) by looking at how much time elapsed between when the block could have been mined (which you can determine based) and when you first saw it. You can penalize blocks that could have been mined long ago. (I don't see any point in doing so, but whether it would serve any purpose is another question.)

This answer is correct. You can not prove when a block was received, or when other transactions were received to any other participant on the network, so details like that can't be part of the consensus system. If nodes were to "penalize" blocks they received based on local observation the consensus system would fail. A node can make a guess that a block is new only if it contains a transaction they produced, as they have knowledge of when it was signed, which is completely meaningless.

This question isn't about the consensus system at all. It's about the eventual agreement part of it. You're absolutely wrong that it can make the consensus system fail for two reasons: 1. The consensus system is independent. 2. Nothing with purely local effects can make the consensus system fail. (continued)

For example, consider a rule that says you don't treat any new transactions as confirmed if you haven't seen a newly-mined block for twenty minutes until you see at least two blocks mined with 15 minutes. That will cause nodes to disagree over which blocks are confirmed, but that's perfectly normal and would in no way break the consensus scheme. A node's decision to wait longer before trusting a transaction or refuse to mine on the "longest chain" because it seems suspicious cannot break consensus.

this attack could be dealt with by a soft fork that disregards any block that the applicable miner did not publish within a certain amount of time after it was found The discussion is about consensus rules.

So your answer isn't a response to the part of the question you cited in your response? If so, why cite it? In any event, that suggestion is already implemented in nearly all bitcoin nodes. Checkpoints do exactly this. So if that breaks consensus rules, it's a break that most people consider acceptable once a block is "old enough".

It's literally part of the same paragraph, I quoted the part with the direct question of if it is possible or not, which it isn't. The checkpoint code doesn't do what you think it does.

I know precisely what the checkpoint code does. Among other things, it causes nodes to ignore other blocks with the same sequence as the checkpoint if their hash is different -- even if they are part of a longer chain than the chain the node is on. This is precisely what you said could not be done. And the rationale is the same -- because there is a conflicting block that has been accepted for a long enough time.

The question was about nodes being able to do this dynamically, within these bounds arguing that checkpoints (which don't meaningfully exist in modern Bitcoin Core, they haven't been touched in 7 years) somehow fits the description is idiotic. This bottom-shelf, belligerent, and wholly incorrect argument is explicitly why I'm not interested in contributing to this Stack Exchange further.

If you wish to continue to remain wrong, I can't stop you. But you are wrong. Nodes can absolutely make local changes in how they decide when to consider a block accepted and it has no effect at all on the stability of consensus. They're completely logically independent functions. (This is the same reason a bitcoin node can accept transactions (or blocks!) after 1 confirmation, 6 confirmations, or 80 confirmations and it has no effect on consensus.)


text:
Loosing your bitcoins due to a glitch on your mycelium app, and how do you get the back without a backup?
I went and purchased some bitcoins on my mycelium wallet. Once I went back to my car and opened the app to do a backup for it the app just stopped working and it wouldn't open back up. I couldn't do the back up to it, so I sent an email and asked if I uninstall and reinstall will I loose the coins that I just purchased? it took them an hour to respond to my email and by that time I had already uninstalled and reinstalled my wallet. The coins are gone but I looked up my transaction on Blockchain and they are there. How can I recover my coins when Mycelium had the glitch (that was fixed with an update) and there email was an hour late to respond to my question, how can I recover those coins?

If you do not have a backup of the wallet, either in the form of seed words or a exported file from Mycelium, then your coins are permanently lost.
Ideally, one should be backing up and restoring their wallet to make sure they've got all the necessary information before sending a significant amount of coins to it.

As others have pointed out, your money is irretrievably gone forever. There are some observations that might help you or might help future readers of this question.

I went and purchased some bitcoins on my mycelium wallet [later I] opened the app to do a backup.

Some people (maybe not you) have the idea that you need to make a backup every time your wallet receives money. This is not true.
It is superficially plausible, after all if the contents of the wallet have changed, surely the old backup won't know about the new contents?
The reason why this is not so is because Bitcoin wallets don't contain Bitcoins, the only important thing they contain is the secret number called a private-key. The globally replicated blockchain is what keeps track of who controls various amounts of money. You don't need to backup the blockchain yourself because every full-node in the worldwide Bitcoin network has a copy of it you can retrieve.
Immediately after installing a new wallet with no money in it, you can backup† the recovery/seed-phrase or the master private-key(s). After that, it does not matter how many times you receive money into the wallet, you can always re-create the wallet from the saved recovery/seed-phrase or private-key and it will recover control over all the unspent money received by that wallet since you made the backup.
There are some potential issues that are important to remember. If you import private-keys into a wallet, the saved recovery/seed-phrase or original private-key can't be used to recover control over money controlled by those imported keys.
I believe most wallets also allow you to make a full-backup the wallet which can be used to recover imported keys as well. So it may be prudent to make and test a full wallet-backup before and after making any significant change like updating a wallet to a new version, importing private keys, etc.

it took them an hour to respond to my email ...

Getting a response in less than a week from the developers of free software seems pretty good to me. If I was actually paying a monthly support fee I'd expect better than a week but a lot of commercial software support contracts don't guarantee a 1-hour response (and certainly not a 1-hour fix).

... and by that time I had already uninstalled and reinstalled my wallet.

That's a shame. If you feel a strong need to do something, often the best course of action is to shut-down and turn-off the computer/device and don't do anything with it until you have thoroughly researched and tested a solution using a different computer/device.

† write down on paper, imprint onto a metal sheet, etc. Put that somewhere very safe.

Yea it's a hard lesson to learn. The coins are gone forever but look at the valuable lesson you learned. Always copy those seed words down on paper and put them in a safe place. They are your private keys. I had a similar situation with electrum on tails. The wallet would not sync and I had $200 worth of bitcoin in it. I was able to regenerate the private keys to a different wallet with my mnemonic word seed and got my money back. So sorry this happened to you. It's not easy to be your own bank.


text:
Does coinjoin provide better privacy than coinswap?
In July 2020 one Twitter user shared a zcash transaction id which had shielded addresses however another user was able to get the transparent address from it. This was possible because t-z conversion was mentioned in tweet and user found one such transaction in last few blocks
Later user suggested a solution about gap in such transactions.
Zooko considers all swaps bad for privacy. Assuming this is true, how does it affect Bitcoin in different swaps and does it make coinjoin better than coinswap in terms of privacy?

What Zooko meant is that, if you have X coins in transparent, use a shielded address, and then convert them back to transparent, it's easy to link those two transactions because the amount of both is X.
However both Coinjoins and Coinswaps try to make amount correlation hard to effectively break input-output linkage.
In Coinjoins usually input and output amounts are equal, or the number of inputs and outputs would need to be sufficiently large so that there are different possible interpretations. In Coinswaps several unrelated transactions are used, which makes it hard to link them all together.
CoinJoin (3 participants)      CoinSwap (2 participants)

A(0.5) -|-> B(0.5)             Tx1 A(0.2) --> B(0.2) | B receives 0.5
        |                      Tx2 A(0.3) --> B(0.3) |
B(0.5) -|-> C(0.5)             -----------------------
        |                      Tx3 B(0.1) --> A(0.1) | A receives 0.5
C(0.5) -|-> A(0.5)             Tx4 B(0.4) --> A(0.4) |

The biggest difference between them is that Coinjoins are easy to identify, especially if they are equal amount, while Coinswap transactions may look like usual unrelated transactions. The anonymity set of a Coinjoin is all the users of it, while the anonymity set of a Coinswap is all regular transactions (at least the ones using the same address type). It's easier to target Coinjoin users because of this, and some exchanges have already rejected coinjoined coins.
In summary, Coinjoin effectively breaks the transaction graph and has been tested. Coinswap could allow to break such graph while also being hard to detect, but it's a relatively new technique.


text:
Bitcoin multisig scheme that prevents theft by middle-man operator
Background
I'm considering building a service that allows participants in my industry to strike OTC deals with each other using my centralized matchmaking service.
The deals always involve two players, both of whom deposit Bitcoin collateral at the start of the deal. Let's call our two players PlayerA and PlayerB. As the Operator, I track how their deal progresses and internally move Bitcoin back and forth between the two players as the deal unfolds. This reconciliation takes place at the close of each day and it's internal to my system. It's not posted to the Bitcoin blockchain.
If one or both parties want out of the deal, the deal is considered "settled" and one party will have a realized loss and the other a gain. It's a zero sum game.
Problem
"Not your keys, not your coin."
Players have asked to interact with one another in such a manner that settlements are made peer-to-peer without me as Operator having access to the private keys that control their collateral deposits. This would prevent me from absconding with their funds.
I suspect I'm not smart enough to come up with the correct solution, or even if said solution is possible, but I want to at least try.
What I've Sketched Out So Far
Consider this example:
Beginning Transactions: PlayerA spends 1 BTC such that the resulting UXTO can be redeemed by:

PlayerA only: sixty day timelock, or
PlayerA AND Operator at any time

The idea behind the sixty day wait is that if Operator bails on the project, never to be heard from again, PlayerA can ultimately redeem his funds.
Let's assume PlayerB does the same.
PlayerA and PlayerB find each other via the app and construct a deal. Each day the collateral is rebalanced according to how the deal is progressing. Let's say after four days of keeping the deal alive, PlayerB is ahead by .50 BTC and wants to settle up. The system then submits a transaction to PlayerA's node with the outputs:

PlayerA: .50 BTC (PlayerA's remaining collateral)
PlayerB: .50 BTC (PlayerB's winnings)

PlayerA signs the transaction. The Operator signs it too and posts it on the blockchain. PlayerB gets his .50 BTC and PlayerA gets his remaining .50 BTC collateral. The deal is now closed.
What I Can't Figure Out
I don't understand how to prevent PlayerA from walking away from the transaction. He knows he's down .50 BTC and he knows I can't force him to sign the closing transaction. His incentive is to walk away.
I could of course arrange things so that I as Operator can unilaterally move the funds to PlayerB but that would give me complete control of PlayerA's funds -- exactly what we're trying to avoid, as I'd have the ability to steal the collateral.
Is there some clever way of using multisigs and time locks to incentivize PlayerA to realize his loss and sign the closing transaction? The logic would have to be constructed in such a way that if he refuses to sign after some predetermined period, he forfeits all of his collateral. And that collateral would be forfeited to PlayerB, not to me as the Operator.
With my limited knowledge of the Bitcoin programming language, I'm unsure how to pull it off. Any clever ideas? Not looking for code, just high-level overview of the Bitcoin primitives and tools that are available to me to accomplish my goal.

I can think about some possibilities.
You can use an DLCs (Discrete Log Contracts) with your platform as oracle. Each output can be redeemable by any part at any time, if you sign an attestation saying that it's indeed true of a given statement. You not even need to know which UTXO each participant holds, and let then verify this p2p. The only trust here is that you won't create a fraudulent attestation. I can even imagine a protocol for blinding the transactions, giving more privacy to your users, but this is more evolved and hard to do correctly.
Another way is asking for a signature for each participant every time an order is matched, so you'll have an up-to-date sig from each one, and can build a tx with the current state at any time. This is similar to how Lightning works, and need to be implemented carefully.

Thanks, friend. I'll research both of these options.


text:
Is there a flag or specific message in a bitcoin pcap to show us our bitcoin transaction is confirmed or not?
Assume that I send a specific value from a source wallet to a destination wallet and simultaneously capture the input and output traffic with Wireshark.
I can extract output address and it's value from captured traffic, but how i can detect (from packets of traffic) that my transaction confirmed from miners?
In the other words, when i check out the bitcoin traffic, i see that the specific value which i sent(and it's output address), exists in some packets with different IP addresses, which i don't know which one is a miner. is there a flag or specific message sent to me that help me to understand that my transaction is confirmed?

A transaction is confirmed when it is included in a block, so you will need to watch for a message that alerts your node (or whatever software you are using to watch the P2P gossip traffic) to a new block, that includes the transaction in question. It doesn't matter if a node run by a miner relays your transaction to other nodes on the network, it only matters that the transaction is included in a block that is included in the longest (most work) valid chain.
Note that the relaying of this information has been optimized through various BIPs, so that the network is not naïvely using more bandwidth than required. This means that a message explicitly listing your transaction as confirmed may never be encountered, instead a node can gain this information implicitly, through the use of their mempool, and computational filters included with the new block message. You should take some time to learn about these optimizations if you're working to understand the network's P2P traffic, there are some questions on here that have more info. This one may be a good starting point.

Of course, just because you see your transaction in a block, that does not prove it is a valid block.  To check that, you need to verify that the block meets all Bitcoin protocol requirements and links into the longest chain - which basically entails writing your own Bitcoin node.  So it's probably better to solve this problem by actually running an off-the-shelf Bitcoin node, and having it notify you when the transaction is confirmed.


text:
Unknown command name and packet magic for bitcoin packets at the start of blockchain transactions
Recently I captured a part of blockchain when my bitcoin wallet started to syncing for the first time. I analyze about 10 Gb of traffic which was for primary transactions of bitcoin.
wireshark can detect them as bitcoin traffic but these packets don't have a valid command name and a valid packet magic. according to this link:
https://en.bitcoin.it/wiki/Protocol_documentation#Network_address
all magic values are 0xD9B4BEF9, 0xDAB5BFFA, 0x0709110B and 0xFEB4BEF9.
my first question is why a lot of packets at the start of blockchain don't have a valid magic packet and valid command name, and how wireshark can detect these packets as bitcoin while they don't have a valid header?

It may be useful to show exactly what you are seeing instead of only what you're expecting to see.

@PieterWuille there a lot of incorrect values, for example for a packet i see
"0x45022056d20ede6d23d563b5" as it's command name which isn't a valid printable value and 0x008b4830 for it's packet magic.

Presumably you're incorrectly splitting data into packets. Can you post the full data you see for a representative selections of packets? Preferably subsequent ones?

I just copy these values from wireshark. they have large size so i can't post full data here. wireshark detect them as bitcoin protocol but shows them as black packets.

If you're not showing what you're seeing, there is no way anyone can guess. I'm sure there are some small packets among them, or if not, there are paste sites where you can easily put a few MB.

as an example, start of one of these packets is shown below:
082d2026eda71ec445dbf97b346cb65692e23b712a11fc500220174bbfea9f0c591c9efb91ee004386ad22d13eb04d748df3f1957b3a3c69b18401210294c3fba7a78fcfaae34a7154e53b03ee1d78fa87499eefdb631c84e9cfd85614feffffff02e8166500000000001976a914d6c094481a567c316af56676c4df77d23ecf3a3088ac18c00900000000001976a914808b475d6a12476fb854e8aa6dd6e85aa217c1ae88ace6c106000100000001024db340a993dadcfa7b4f4d350057e38efa0339f58dc52c4cecb027a42ff7f300000
as wireshark's info, packet magic = 0x082d2026 and payload length is 92e23b71 which is equal to 1899750034.

@PieterWuille  do you have any opinion?


text:
Generating TXID from transaction's data
As I understand, we can generate a TXID by hashing transaction data through SHA256 twice, and as you know probably, A bitcoin transaction is just a bunch of data that describes the movement of bitcoins.
after that, to search a TXID in the blockchain, we should search for it in reverse byte order.
I tried this instruction to generate a TXID for some sample tx messages, but i didn't find it in blockchain explorers.

The most common mistake many people do is hashing the serialized transaction as an encoded text rather than using the bytes that the hexadecimal represents.

@Ugam I open my pcap file with wireshark and copy tx message structure as hex stream. and then hash it twice.

follow the process that is mentioned in the question I have linked.

It will be easier to help you if you share the steps you took and the data you are hashing so it can be reproduced.

@Ugam
Finally I completed my C++ program to calculate TXID with double hashing of TX data and after that reverse it.
but sometimes generated TXID is not in some bitcoin explorers like https://www.blockchain.com.
why some TXID's are found and some others not found?

@Saeed As @Jbaczuk said, it would be easier for us to help you if you post the data related to the transaction and the steps that you took to get the txid. Otherwise we are shooting in the dark as we do not know which data matched and which did not.

@Uragm. you are right.
this is my tx raw hex data:
01000000000102baec426dc550de28b4380ffa66d3f19bdce6ed6bc51a708145dbccadd6bd38400100000000ffffffff8cbc8d7ad78f1675ad3806
I do double sha256 on this raw data and the output was :
e38cc8977dba1999cf03640a997dacd1e9e6bcae255522b38d5ea43296e2fbbd
after reverse it, final TxID is :
bdfbe29632a45e8db3225525aebce6e9d1ac7d990a6403cf9919ba7d97c88ce3
but it is not found!

@Saeed First of all your raw hex data is not a valid transaction. Secondly, your dsha256 is also incorrect related to that data you posted. The dsha256 of the incorrect data you posted should be 'af40edfde85a4a53256ed85df9f9b19a3c5181167431e0442421b5e6c5032b89'

@Ugam I just coppied tx message structure hex stream data from wireshark. and I created that trasnaction myself. so the hex deta should be true, also I double hash it with sha256 online in this site:
sha256online
so it is a true way, especially that with this way I generate a lot of TXID for other transactions which all were true and found in blockchain.

@Saeed for the incorrect transaction that you have mentioned, the website hashes to the data I have mentioned. You need to check the binary option next to the box to tell the program that you are hashing hexadecimals. Now, coming back to your transaction, I don't mind how you are capturing it, but it is not a valid Bitcoin transaction.

@Ugam, Thank you. I got it. you are right. it's ok now.


text:
How i can extract all bitcoin output addresses from tx message?
I want to extract output addresses from tx messages in any bitcoin pcap.
currently, i extract tx messages which it's output script starts with "0x76" or "0xA9", but i don't know how can i extract the other types of op_codes which is introduced here:
https://en.bitcoin.it/wiki/Script

Most of the output addresses follow one of the following 'standard outputs':
Legacy Outputs

P2PK: scriptPubKey: <public_key> OP_CHECKSIG. Output pays the public key directly and hence does not have a direct address.
P2PKH: scriptPubKey: OP_DUP OP_HASH160 <hash160 of pubkey> OP_EQUALVERIFY OP_CHECKSIG. For this kind of output you just need to base58check the hash160 of pubkey with version 0x00 and you will get the addresses that starts with 1.
P2SH: scriptPubKey: OP_HASH160 <redeem_script> OP_EQUAL. You again need to just base58check the redeem_script with version 0x05 and you will get an address starting with 3.
Multisig: scriptPubKey: M <public_key1>...<public_keyN> N OP_CHECKMULTISIG. In this case, you cannot get the addresses as the output pays the public keys directly.

Segwit Outputs

P2WPKH: scriptPubKey: <0x00 (version)> <hash160 of pubkey>. You will need to create a bech32 address using the version and 20-byte redeem script and you will get an address starting with bc1. You can find the python script for bech32 encoding here.
P2SH: scriptPubKey <version: 0x00> <sha256 of redeem_script>. Again a bech32 address starting bc1 but the redeem_script is 32 bytes rather than 20 bytes for P2WPKH.
P2SH(P2WPKH): scriptPubKey: OP_HASH160 <redeem_script > OP_EQUAL. The encoding is same as legacy P2SH. Base58check with 0x05 version. For a more detailed overview on how the 'redeem_script` and addresses are generated for P2SH(P2WPKH) see my other answer here.

OP_RETURN
Many times people like to encode some data in the Bitcoin blockchain. For that, you use a OP_RETURN opcode. The scriptPubKey:<OP_RETURN><OP_PUSHDATA1><bytes to push><script>.

This is technically not true - you have outputs on the chain that don't match any of the above, the most obvious being P2PK outputs from the early blocks before hash160 addresses were commonly used

So, Tx messages which contains output scripts start with  0x00, is belongs to which category?
it seems that it is a P2SH (like output scripts which start with 0xA9), but generated output address from that way is not valid.

@Saeed It's segwit outputs. Legacy P2SH needs to have OP_HASH160  OP_Equal. However, Segwit P2SH is 0x00 <redeem_script>. For a non-segwit aware client, this looks as if this is anyone can spend output. But Segwit aware clients will look for the scriptSig in the witness section of the transaction.

@UgamKamat How can i detect between legacy outputs and segwit outputs?
as you said before, there are 6 different way to extract output address based on legacy or segwit outputs. is all these ways?

@saeed as I said earlier, those are the current STANDARD outputs. Bitcoin is a programmable money, and as a result accept a number of ways in which you can lock the output, but that will not be standard


text:
Operation not valid with the current stack size
According to bip65
https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki
I can use expressions like IF ... ELSE ... ENDIF
Here is example script from this document:
IF
    HASH160 <Hash160(encryption key)> EQUALVERIFY
    <publisher pubkey> CHECKSIG
ELSE
    <expiry time> CHECKLOCKTIMEVERIFY DROP
    <buyer pubkey> CHECKSIG
ENDIF

But when I trying to spend transaction with the following script:
OP_IF
    OP_SHA256 4bf5122f344554c53bde2ebb8cd2b7e3d1600ad631c385a5d7cce23c7785459a OP_EQUAL
OP_ELSE
    10 OP_CHECKLOCKTIMEVERIFY OP_DROP OP_SHA256 dbc1b4c900ffe48d575b5da5c638040125f65db0fe3e24494b76ea986457d986 OP_EQUAL
OP_ENDIF

https://tchain.btc.com/dbf477b700b7e159c07f15e1b6f5917e1247175e055fe533068a6fbc4ce2c374
I get error:
bitcoin-cli -testnet sendrawtransaction 020000000174c3e24cbc6f8a0633e55f055e1747127e91f5b6e1157fc059e1b700b777f4db000000000151ffffffff01c0d401000000000023a820dbc1b4c900ffe48d575b5da5c638040125f65db0fe3e24494b76ea986457d9868700000000
error code: -26
error message:
mandatory-script-verify-flag-failed (Operation not valid with the current stack size) (code 16)

Where is my error ?
P.S.

SHA256 from 1 ->
  4bf5122f344554c53bde2ebb8cd2b7e3d1600ad631c385a5d7cce23c7785459a
SHA256 from 2 ->
  dbc1b4c900ffe48d575b5da5c638040125f65db0fe3e24494b76ea986457d986

Your transaction only has 0x51 (just OP_1) in its scriptsig,  the OP_IF consumes the 1, and OP_SHA256 then operates on an empty stack, causing this error.

Like arubi said, you're scriptSig is just 0x51:
$ bitcoin-cli decoderawtransaction 020000000174c3e24cbc6f8a0633e55f055e1747127e91f5b6e1157fc059e1b700b777f4db000000000151ffffffff01c0d401000000000023a820dbc1b4c900ffe48d575b5da5c638040125f65db0fe3e24494b76ea986457d9868700000000
{
  "txid": "2e7c6efc9dc6104f132a10d0d64263269d1ece326b4d64cb1061fe5a81b4d53b",
  "hash": "2e7c6efc9dc6104f132a10d0d64263269d1ece326b4d64cb1061fe5a81b4d53b",
  "version": 2,
  "size": 96,
  "vsize": 96,
  "weight": 384,
  "locktime": 0,
  "vin": [
    {
      "txid": "dbf477b700b7e159c07f15e1b6f5917e1247175e055fe533068a6fbc4ce2c374",
      "vout": 0,
      "scriptSig": {
        "asm": "1",
        "hex": "51"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 0.00120000,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_SHA256 dbc1b4c900ffe48d575b5da5c638040125f65db0fe3e24494b76ea986457d986 OP_EQUAL",
        "hex": "a820dbc1b4c900ffe48d575b5da5c638040125f65db0fe3e24494b76ea986457d98687",
        "type": "nonstandard"
      }
    }
  ]
}

This means if the scriptPubKey of the input you're spending is:
OP_IF
    OP_SHA256 4bf5122f344554c53bde2ebb8cd2b7e3d1600ad631c385a5d7cce23c7785459a OP_EQUAL
OP_ELSE
    10 OP_CHECKLOCKTIMEVERIFY OP_DROP OP_SHA256 dbc1b4c900ffe48d575b5da5c638040125f65db0fe3e24494b76ea986457d986 OP_EQUAL
OP_ENDIF

It will execute this branch (since the top stack element is true):
OP_IF
    OP_SHA256 4bf5122f344554c53bde2ebb8cd2b7e3d1600ad631c385a5d7cce23c7785459a OP_EQUAL

You must also provide whatever hashes to the value 4bf5122f344554c53bde2ebb8cd2b7e3d1600ad631c385a5d7cce23c7785459a in order for it to return successfully.


text:
Is it safe to store bitcoins on p2sh (no multisig)?
I've just generated p2sh btc address and look in the output script:
HASH160 PUSHDATA(20)[e5c32bdc7f7fb1e6ae7aa1baf42a429803054d2c] EQUALVERIFY

while normal (legacy btc address) transaction has signature check
DUP HASH160 PUSHDATA(20)[e5c32bdc7f7fb1e6ae7aa1baf42a429803054d2c]
EQUALVERIFY CHECKSIG

Is it safe to store bitcoins in this address with this scriptsig ?
Or it use a same key for the signature check?
If yes, what a reason to check the signature ?
Thanks

Is it safe to store bitcoins in this address with this scriptsig ?

That is actually the scriptPubkey, and it is only safe if you use a safe redeemScript.
A P2SH script is a Pay-to-Script-Hash script, meaning when you spend the coins, you must provide the script (and the hash must match), but this script you provide, called the redeemScript must also succeed when it is executed, and can be any valid script.
Typically this is a script similar (if not exactly the same) as the one you provided, which is a P2PKH or Pay-to-Public-Key-Hash. Thus, it also requires a signature, so if you use a P2SH(P2PKH) it is still just as secure as using a P2PKH because of the signature requirement in the P2PKH, you just have to also provide the redeemScript.


text:
Does asic API exists?
I have an antminer asic, and it has web interface. I could see temperature and hashrate through it.
Now i want to retrieve and store that data (from multiple asics), within another program on another device. Obvious but inefficient way is to send HTTP request and parse HTML responses.
But does asic has some way (for example JSON output via some GET/POST requests, or even custom TCP protocol) to get information in machine-friendly format?
How do such aggregated statistics monitoring and control usually implemented in big farms? Is there any existing software for solving this obvious control/monitoring task?

Most Antminers have a WebSocket API that connects to the built in CGMiner software on port 4028.
You can test this in a command prompt in Windows, Mac, or Linux with (nc is a program called netcat which you may need to install):
echo -n "stats" | nc IP_OF_MINER_GOES_HERE 4028

There may also be a way to do this with CURL instead of netcat, but I have not had a chance to test it yet.
Code Examples:

api-example.php - a php script to access the API   usAge: php
  api-example.php command  by default it sends a 'summary' request to
  the miner at 127.0.0.1:4028  If you specify a command it will send
  that request instead  You must modify the line "$socket =
  getsock('127.0.0.1', 4028);" at the  beginning of "function
  request($cmd)" to change where it looks for cgminer
api-example.rb - a Ruby script to access the API.  usage: ruby
  api-example.rb command[:parameter] [HOST [PORT]] This script prints
  the parsed cgminer API response
API.java/API.class  a java program to access the API (with source
  code)   usAge is: java API command address port  Any missing or blank
  parameters are replaced as if you entered:   java API summary
  127.0.0.1 4028
api-example.c - a 'C' program to access the API (with source code)
  usAge: api-example [command [ip/host [port]]]  again, as above,
  missing or blank parameters are replaced as if you entered:
  api-example summary 127.0.0.1 4028

Java Example
C Example
PHP Example
Python Example
Ruby Example
I've built a small script for my personal miners which will not work for most environments (due to reliance on data from SlushPool), but it may be useful for some code snippets (C#): Here

You can use my software at https://github.com/anselal/antminer-monitor. More features to come


text:
UTXO are not being listed while they have conf over 100 (btcd + simnet)
I am using BTCD and while in Simnet, I generated 104 block (Best block : 104)
But when I listed utxos, transaction from block 3 and 4 (which have 102 and 101 confirmations) don't be listed. So balance shown only 50 btc that is reward of 5. block. You can see console outputs below.
PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet listunspent
[]

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getblockhash 3
5d59b05643d0c058d1b9839db3629b8a794b8617a970d5f23e45d5891efa3f35

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getblock 5d59b05643d0c058d1b9839db3629b8a794b8617a970d5f23e45d5891efa3f35
{
  "hash": "5d59b05643d0c058d1b9839db3629b8a794b8617a970d5f23e45d5891efa3f35",
  "confirmations": 100,
  "strippedsize": 188,
  "size": 188,
  "weight": 752,
  "height": 3,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "0921fead5a3b0747ced64a18c03a0b1fafc5fecabf06fff4d523176b5754b82a",
  "tx": [
    "0921fead5a3b0747ced64a18c03a0b1fafc5fecabf06fff4d523176b5754b82a"
  ],
  "time": 1539433120,
  "nonce": 2,
  "bits": "207fffff",
  "difficulty": 1,
  "previousblockhash": "1641c6a0d5916cfc2f70b7c89fcd27dfb85fe11886516dc6f1de36b8f4761212",
  "nextblockhash": "3bc06d8a82d93c3a8839288f56d949bb047dfbd597b39c6f62cf1f6da47d1692"
}

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet gettransaction 0921fead5a3b0747ced64a18c03a0b1fafc5fecabf06fff4d523176b5754b82a
-5: No information for transaction

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getbalance
0

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf generate 1
[
  "37169e2ff42e2900394ca526ce675444e0d074dc03839c70aa81bf03b845fc84"
]

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getbalance
50

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getblockhash 3
5d59b05643d0c058d1b9839db3629b8a794b8617a970d5f23e45d5891efa3f35

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getblock 5d59b05643d0c058d1b9839db3629b8a794b8617a970d5f23e45d5891efa3f35
{
  "hash": "5d59b05643d0c058d1b9839db3629b8a794b8617a970d5f23e45d5891efa3f35",
  "confirmations": 102,
  "strippedsize": 188,
  "size": 188,
  "weight": 752,
  "height": 3,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "0921fead5a3b0747ced64a18c03a0b1fafc5fecabf06fff4d523176b5754b82a",
  "tx": [
    "0921fead5a3b0747ced64a18c03a0b1fafc5fecabf06fff4d523176b5754b82a"
  ],
  "time": 1539433120,
  "nonce": 2,
  "bits": "207fffff",
  "difficulty": 1,
  "previousblockhash": "1641c6a0d5916cfc2f70b7c89fcd27dfb85fe11886516dc6f1de36b8f4761212",
  "nextblockhash": "3bc06d8a82d93c3a8839288f56d949bb047dfbd597b39c6f62cf1f6da47d1692"
}

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getblockhash 4
3bc06d8a82d93c3a8839288f56d949bb047dfbd597b39c6f62cf1f6da47d1692

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getblock 3bc06d8a82d93c3a8839288f56d949bb047dfbd597b39c6f62cf1f6da47d1692
{
  "hash": "3bc06d8a82d93c3a8839288f56d949bb047dfbd597b39c6f62cf1f6da47d1692",
  "confirmations": 101,
  "strippedsize": 188,
  "size": 188,
  "weight": 752,
  "height": 4,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "77ddf342bf332d2aa29269e75bb8464e853c2eef4936b7ab60a29192c4132874",
  "tx": [
    "77ddf342bf332d2aa29269e75bb8464e853c2eef4936b7ab60a29192c4132874"
  ],
  "time": 1539433121,
  "nonce": 2,
  "bits": "207fffff",
  "difficulty": 1,
  "previousblockhash": "5d59b05643d0c058d1b9839db3629b8a794b8617a970d5f23e45d5891efa3f35",
  "nextblockhash": "32b72487f4f9473952db6474a999b21aa31df353daf7a66199eb6f5e58adb03e"
}

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getblockhash 5
32b72487f4f9473952db6474a999b21aa31df353daf7a66199eb6f5e58adb03e

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getblock 32b72487f4f9473952db6474a999b21aa31df353daf7a66199eb6f5e58adb03e
{
  "hash": "32b72487f4f9473952db6474a999b21aa31df353daf7a66199eb6f5e58adb03e",
  "confirmations": 100,
  "strippedsize": 187,
  "size": 187,
  "weight": 748,
  "height": 5,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "b09fd1f23884ff6e64213604ae7c385b3d3a3ddc47fb5536816800df5e782c5a",
  "tx": [
    "b09fd1f23884ff6e64213604ae7c385b3d3a3ddc47fb5536816800df5e782c5a"
  ],
  "time": 1539433121,
  "nonce": 0,
  "bits": "207fffff",
  "difficulty": 1,
  "previousblockhash": "3bc06d8a82d93c3a8839288f56d949bb047dfbd597b39c6f62cf1f6da47d1692",
  "nextblockhash": "6179c9823f4614f5255727576b955a6a4f1340706329eac314f08c55da947d98"
}

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet listunspent
[
  {
    "txid": "b09fd1f23884ff6e64213604ae7c385b3d3a3ddc47fb5536816800df5e782c5a",
    "vout": 0,
    "address": "SYYFCKLX38x6aZRsxuLmdNXSBDZ3BaJczq",
    "account": "default",
    "scriptPubKey": "76a9147b5b07b286ef494549db8630aeb37301bf99663c88ac",
    "amount": 50,
    "confirmations": 100,
    "spendable": true
  }
]

PS D:\Blockchain\bitcoin\node> btcctl -C .\btcctl.conf --wallet getbestblock
{
   "hash": "37169e2ff42e2900394ca526ce675444e0d074dc03839c70aa81bf03b845fc84",
   "height": 104
}


text:
BTC to LND Node Transfer Not Showing yet on node's wallet after almost two weeks?
I realize that this may or may not be a question to ask of the Umbrel developers but I have to sadly report to you all that they rarely answer any posts on their forum.
Please help us with this situation if you can. We sent BTC from Bitfinex to our Electrum Wallet on our Windows PC which has shown up in the Electrum's history to then be sent
directly over tor (blue icon was active) to our Lightning Node (using Umbrel) which we have done many times now but this time it still has not arrived in our nodes BTC wallet
yet after almost two weeks and over 2340 confirmations, as seen on mempool.
https://mempool.space/tx/97fdb66ab006da2c634f77f8dfa66d959609517a15878266c67b9686a9866894
Any ideas as to why it has yet to reach our node's wallet? BTW we sent a 2nd amount of BTC again using Bitfinex to our Electrum Wallet last Friday and that has also not shown
up on our node's BTC wallet and yes it is recognized in the Electrum wallet on the PC but not on the node?  Thank you for any assiatance anyone can offer us to try and fix this problem :)
Oh BTW our Electrum wallet has not reported any issues.

Hi all, Am I possibly missing a step due to old age? Please let me know. Thank you.

Is your Umbrel-hosted lightning node the on-chain recipient of the transaction you listed?

Hi RedGrittyBrick, Thank you for your reply. Well no i guess not I was under the wrong impression due to a faulty memory I suppose that our Electrum wallet on our PC (Which is connected via tor to the Electrum Server app on our Umbrel LND Node) would send the funds to the Umbrel node via the Electrum server app automatucally once we sent the Electrum wallet some BTC from our exchange account after a period of time? I seemed to remember it doing this before but I guess I was wrong? Thank you.


text:
Transaction still unspendable with CLTV
Previously I generated tx with CHECKLOCKTIMEVERIFY
Here is the transaction https://tchain.btc.com/bb1415c166fd9f767c06d34d539f4b110ef4445f74a76c5ffad1ded33032e872
It lock funds before block 1442909.
I create a new spend transaction.
When I trying to sign it:
bitcoin-cli -testnet signrawtransactionwithwallet "020000000172e83230d3ded1fa5f6ca7745f44f40e114b9f534dd3067c769ffd66c11514bb0000000000ffffffff01443dd200000000002676a92103e8546cfe53ca9014f6a352bc2e5c1f7cec1f5b344c73d4db3ed3b8f075d5c9ef88ac00000000"
{
  "hex": "020000000172e83230d3ded1fa5f6ca7745f44f40e114b9f534dd3067c769ffd66c11514bb0000000000ffffffff01443dd200000000002676a92103e8546cfe53ca9014f6a352bc2e5c1f7cec1f5b344c73d4db3ed3b8f075d5c9ef88ac00000000",
  "complete": false,
  "errors": [
    {
      "txid": "bb1415c166fd9f767c06d34d539f4b110ef4445f74a76c5ffad1ded33032e872",
      "vout": 0,
      "witness": [
      ],
      "scriptSig": "",
      "sequence": 4294967295,
      "error": "Locktime requirement not satisfied"
    }
  ]
}

I got "Locktime requirement not satisfied" ? The height of blockchain now is 1,442,912
What I'm doing wrong?

Bitcoin Core does not support signing scripts with CLTV instructions.
It only supports:

P2PK
P2PKH
k-of-n multisig
P2WSH wrapped versions of the above
P2WPKH
P2SH wrapped versions of the above

If you want to sign anything else, you'll need to implement your own signing logic.
The error you're seeing is due to the spending transaction not having the nLockTime field set. In general, CLTV works indirectly:

A CLTV output puts a requirement on the spending transaction to have an nLockTime field larger than the time specified by the CLTV instruction.
A transaction with a nLockTime field can only be confirmed when the blockchain time is after the value in that field (this is even the case when no CLTV is present).

But regardless, even if you did have a correct nLockTime field, signrawtransaction would still not know how to sign for that script.

is there a reason for it? To prevent users losing funds or a protection measure for the end-user without detailed knowledge? I was wondering if there may be activities to allow more sophisticated use cases (following s.th. like a "--experimental" switch)?

No, it's just not implemented. Bitcoin Core has no idea what that script does, or how to construct a satisfying input for it.

Is there any plan to implement this?


text:
Receiving payment on Lightning Network with LNBig as initial funder?
I was able to open a channel to myself from LNBig and withdrew 100 sats from Reddit's lntipbot. This appeared to be successful!
Output of lncli channelbalance:
{
    "balance": "100",
    "pending_open_balance": "0"
}

However, when I attempt to re-deposit 10 sats back to lntipbot, I get this error:
[lncli] insufficient local balance

I was hoping I could do this without doing an on chain transaction myself, hoping LNBig provided this piece of the puzzle. Am I having a large misunderstanding, should I be able to spend from my channelbalance without doing an actual on chain funding transaction myself?

In principle what you do should work. However there is a subtle but important feature in the lightning network protocol (meaning this is with every implementation) which we call the channel reserve.
If you want to send money along a channel your node has to keep at least 1% of the channel capacity as a reserve / collateral to protect your channel partner in case you try to steal funds.
Let's say lnbig opened a 1 million Satoshi channel with you. As long as you do not have 10 thousand Satoshi on your side you will not be able to send money. If you have 11 thousand Satoshi in such a channel you can send up to 1000 Satoshi because you have to keep above 10000.

Thanks for answering my question @RenePickhardt, I was just reading your book last night! :)

Is there any way to disable this protection?
Is there an alternative to LNBig that will open up channels with me with < 2,000,000SAT capacity? Looking to make some very small transactions.

Too small channels do not make too much sense in general. But you can open channels with as little as 20k sats


text:
Block construction
I'm in the process or preparing a lecture on cryptocurrency and have studied dozens of sources.  I have a pretty good understanding of most aspects of the bitcoin blockchain scheme. However, all of them have skipped over several points that seem important to me.  Here's two:

When does a miner submit his block for verification? Is it when the when the number of transactions reached a certain number, when their total file size reached a certain value or some other criterion?
Does the miner have any flexibility in choosing the transactions to include in his block?  Why not just big ones to reach the limit quicker?
Can there be more than one block circulating in the network seeking verification at the same time?

As soon as the miner finds a solution to the proof of work.
This is as soon as they find a combination of transactions and other data for
which the hash is less than, or equal to, the current target.
Yes, the miner can choose whatever transactions maximise their return.
Blocks can contain zero transactions (other than the coinbase transaction which pays the miner their mining reward).
Yes, there can be forks in the blockchain. Mostly resolved very quickly.
Chain with most work wins, others discarded.

Most difficult chain wins, not longest.

@Anonymous: Thanks, edited.

Can a block contain zero transactions, or is it necessary to have a coinbase transaction? I'm aware of blocks with only a coinbase, but none without.

To be clear, does a miner only do the proof of work calculations on his own proposed block?  2. When he submits it to the network for others to verify, why should other miners verify someone else's block?  What's in it for them?.

@greyfrog: Every full-node in the Bitcoin network verifies every block. That includes wallets such as Bitcoin-core. If a miner builds on a fraudulent block, their own blocks will be ignored by everyone else. That means they waste their time for no reward.

@Mark H: I omitted that for simplicitly (but have edited the answer). I'm not sure what happens after 21 million Bitcoins have been mined, after that I don't see how there can really be coinbase transactions in blocks. ...

There will still be a coinbase tx, which collects the transaction fees for all transactions in the block. Only difference is that there will be no subsidy added to the amount.

@Mark thanks for clarifying. I was re-reading Nakamoto's paper but there's a lot that isn't in there.

Blocks without coinbase are unconditionally invalid, even after the subsidy runs out. Transaction outputs can have value 0, so this is not a problem even absent fees (such outputs are nonstandard due to the dust rule, but miners are not subject to those).


text:
HD Wallet convert WIF from list to Public Address
I'm trying to convert a txt of WIF to Public Address but only convert the last line, if someone knows another way to do, i really appreciate
from hdwallet import HDWallet
from hdwallet.symbols import BTC as SYMBOL
import sys
import os
import json
import requests
from requests import get

with open('/content/wif.txt') as f:
  for line in f:
    wif = line.rstrip()

# Initialize Bitcoin testnet HDWallet
hdwallet: HDWallet = HDWallet(symbol=SYMBOL)
# Get Bitcoin HDWallet from wallet important format
hdwallet.from_wif(wif=WALLET_IMPORTANT_FORMAT)

# Print all Bitcoin HDWallet information's
#print(json.dumps(hdwallet.dumps(), indent=4, ensure_ascii=False))

print(hdwallet.p2pkh_address()), (hdwallet.wif())

from hdwallet import HDWallet
from hdwallet.symbols import BTC as SYMBOL

All_WIF = []

with open('/content/wif.txt') as f:
    data = f.readlines()
    for i in data:
        wif = i.strip('\n')
        All_WIF.append(wif.rstrip())

for each_wif in All_WIF:
    # Initialize Bitcoin testnet HDWallet
    hdwallet: HDWallet = HDWallet(symbol=SYMBOL)
    # Get Bitcoin HDWallet from wallet important format
    hdwallet.from_wif(wif=each_wif)
    print("Addrss: " + hdwallet.p2pkh_address() + '    WIF: ' + hdwallet.wif())

bro thanks for saving my life


text:
timeout on transient error: Could not connect to the server 127.0.0.1:8332
I finished installing Bitcoin on a raspberry pi a couple days ago and have a problem. Everything worked fine yesterday (I was able to run bitcoin-cli commands just fine) but now whenever I run "bitcoind -daemon" (It says Starting Bitcoin Core), and then run "bitcoin-cli getblockchaininfo" (Or any other bitcoin-cli command), it gives me the error message in the title of this post.
I have a bitcoin.conf file that contains:
server=1
rpcuser=<username>
rpcpassword=<password>

Please help :)

Is Bitcoin Core (bitcoind) still running?

How would I check if it is? I'm very new to this sorry.

For example, do new log lines appear (e.g. look with tail -f ~/.bitcoin/debug.log) or is the process still around (e.g. look if any output appears with ps ax | fgrep bitcoind).

It is telling me that /home/node/.bitcoin disk space is too low, but I have a 2TB externally connected SSD.

Sounds like your home directory isn't on that external HD.

Should I simply mount the home directory where my ssd is mounted? (/mnt/bitcoin)?

That's up to you, and perhaps more a question for a sysadmin site than this one.


text:
Error: Unable to start HTTP server. See debug log for details
I have been looking around quite a bit about ways to solve this error and none of them have worked. To give some context, I am running a full node on a raspberry pi and everything is working just fine from the command line. I am able to write bitcoin-cli commands and it gives me the results just fine. When I write bitcoind in the command line to show if everything is working, it says that it is unable to an http server (the error in the title). Does anyone know how to fix this?
My bitcoin.conf file looks like this:
server=1
rpcuser=<username>
rpcpassword=<password>
txindex=1

So what was in the debug log when you looked?

Binding RPC on address ::1 port 8332 failed.
Binding RPC on address 127.0.0.1 port 8332 failed.
Unable to bind any endpoint for RPC server
Error: Unable to start HTTP server. See debug log for details.

Unable to bind (to port)

Unable to bind any endpoint for RPC server

This is a very common error for network services of all kinds. The most common causes are that there is already another service bound to the port (8332 in this case) or that the user lacks permissions.
You can check what service is bound to port 8332 using something like the netstat -anp command.

Errors with binding
The following examples are on Windows but the Linux commands have only minor differences (e.g. Use grep instead of findstr)
here's one way to check what existing process is using the port
C> netstat -anb | findstr /N "bitcoin 8332"
63: [bitcoind.exe]
65: [bitcoind.exe]
67: [bitcoind.exe]
69: [bitcoind.exe]
70:  TCP    127.0.0.1:8332         0.0.0.0:0              LISTENING
71: [bitcoind.exe]
132:  TCP    [::1]:8332             [::]:0                 LISTENING
133: [bitcoind.exe]

The above tells you that bitcoind is already running and it is listening to the network at port 8332, waiting for requests.

Testing RPC
You can test basic connectivity as follows
C> curl http://localhost:8332/
JSONRPC server handles only POST requests

Note that I deliberately don't send credentials or a valid request. However the bitcoind receives the request and tells me something about what it doesn't like. That proves the bitcoind HTTP service is working. I do not get a connection refused message from curl.

Here's an example that shows passing credentials and receiving an error message
C> curl -v -u user:pass -d '{"method":"getblockhash","params":[0],"id":1}' http://localhost:8332/
*   Trying 127.0.0.1:8332...
* Connected to localhost (127.0.0.1) port 8332 (#0)
* Server auth using Basic with user 'user'
> POST / HTTP/1.1
> Host: localhost:8332
> Authorization: Basic dG9kYXk6dGVtcG9yYXJ5
> User-Agent: curl/8.0.1
> Accept: */*
> Content-Length: 39
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Date: Sun, 16 Jul 2023 14:27:47 GMT
< Content-Length: 74
<
{"result":null,"error":{"code":-32700,"message":"Parse error"},"id":null}
* Connection #0 to host localhost left intact

In this case there is a HTTP 500 error  because bitcoind was unable to parse data.
You will need to specify -v to be sure to see the HTTP error messages, but it isn't needed once you have JSON responses working. We can drop the -v option to reduce the clutter
I changed the quoting from single to double quotes. You shouldn't have to do this on Linux, it is a Windows feature:
C> curl -u user:pass -d "{\"method\":\"getblockhash\",\"params\":[0],\"id\":1}" http://localhost:8332/
{"result":null,"error":{"code":-28,"message":"Starting network threads…"},"id":1}

Now bitcoind doesn't have problems parsing data but says it is still busy initialising.
We can ask it a question that doesn't depend on blockchain state etc
C> curl -u user:pass -d "{\"method\": \"uptime\", \"params\": []}" http://localhost:8332/
{"result":2063,"error":null,"id":null}

bitcoind reports it has been running for 2063 seconds
C> curl -u user:pass -d "{\"method\": \"uptime\", \"params\": []}" http://localhost:8332/
{"result":2427,"error":null,"id":null}

Now it is 2427 seconds since bitcoind started.
If I switch to WSL for no particular reason other than to try an RPC call using Linux, bitcoind is has finished initialising and can answer questions about blockchain data
$ curl -u today:temporary -d '{"method":"getblockhash","params":[0],"id":1}' http://localhost:8332/
{"result":"000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f","error":null,"id":1}

On my PC, with many other processes running, this query can take several minutes, so you might need to adjust client timeouts.

The PID/program name that is listed on 127.0.0.1:8332 is bitcoind. Does this mean that it is connected to localhost? Also state of the program is "LISTEN"

Its already running and you shouldn't try to start another copy running. You communicate with the current bitcoind using bitcoin-cli or another client program. As described here and elsewhere.

I have code sending a command to this http server: AuthServiceProxy(f'http://{rpc_user}:{rpc_password}@127.0.0.1:{rpc_port}') But it tells me that the target machine refused it. Would this be because of this "Unable to start http server" error?

See extra faultfinding tip added at end of answer.

I have created a JSONRPC post request: requests.post('http://username:password@127.0.0.1:8332', json={"method":"getblockhash","params":[0],"id":1})  and the same error occurs.

I suggest testing using curl as shown in answer. That eliminates other potential problems in whatever code you may be using/writing. By "same error" I don't know which one you mean unless you explicitly quote it.

I have tried all the commands you have suggested in your answer. I found that the message I get after using: curl -v -u <user>:<password> -d '{"method":"getblockhash","params":[0],"id":1}' http://localhost :8332/ it returns the same stuff that is returned in your example, but tells me that there is an authentication error: "Authentication error. Ignoring this". I am using the user and password from my bitcoin.conf file.

@NeonTears: At some point you have to put follow on questions into a new question - I think you have reached that point because your authentication error is really a different problem from your earlier "unable to bind" problem .


text:
Where to find a people who have bitcoins, to exchange USD to BTC with no KYC offline?
I have cash in USD, i want to buy BTC in my city, but i don't want to buy it with exchange magazine, because bad exchange rate and because it requires KYC and i'm not want to my personal data will leak and someone will fake my person.
What i should do to find a people who have a BTC and ready to exchange it to my USD cash?
I think i should find some miners in miners communities, but i don't know where i should looking for.
p.s. I'm not interest to any online exchange, because i don't want to be scummed, i want to meet offline.

Does this answer your question? I am new to Bitcoin, how can I get started? which has a lot of useful links, one of which leads you to How do you obtain Bitcoins

@RedGrittyBrick unfortunately no. I've explore questions before write my own, nothing useful for me. This question are closer to mine, but answers not useful for me https://bitcoin.stackexchange.com/questions/4194/whats-a-good-resource-explaining-the-best-ways-to-buy-bitcoin-noob-friendly. Answer about localbitcoins.com are good, but this service is not working anymore

Note that service recommendations are off-topic here, because the answers tend to get outdated quickly.

Service recommendations are off-topic here, but more generally, you can just look around online to find out if there are any local bitcoin meetup groups in your area. Attend a meetup, talk to some people, ask if anyone is willing to sell some bitcoin. Thats really all it takes.


text:
What does it mean that the security of Bitcoin public keys and 256-bit ECDSA is 128 bits?
I am reading the Master Bitcoin book and came across the following:

From a security perspective, the amount of entropy actually used for
the production of HD wallets is roughly 128 bits, which equals 12
words. Providing more than 12 words produces additional entropy which
is unnecessary, and this unused entropy is not used for the derivation
of the seed in the way that one might initially suspect.

It says that using more than 12 words (128 bit entropy) is unnecessary. It sounded extremely strange to me, because the seed from which wallets are made can go up to 512 bits. Thus, 256 bit entropy would provide a higher degree of security. So I did some research and in the third edition of the given book (which is still being written) I came across the following:

The security strength of a Bitcoin public key is 128 bits. An attacker
with a classical computer (the only kind which can be used for a
practical attack as of this writing) would need to perform about 2^128
operations on Bitcoin’s elliptic curve in order to find a private key
for another user’s public key. The implication of a security strength
of 128 bits is that there’s no apparent benefit to using more than 128
bits of entropy (although you need to ensure your generated private
keys are selected uniformly from within the entire 2^256 range of
private keys).

So I understand where the claims are coming from that it's not worth having more than 12 mnemonic codewords (128 bits), although the given claim is not clear to me at all. Upon further research, I found out that 256-bit ECDSA has 128 bits of security. And this is where my confusion started, so I have two questions:

What does it mean that the security of 256-bit ECDSA, and therefore Bitcoin keys, is 128 bits? I mean, if I have approximately 2^256 possible points on the elliptic curve (thus possible public keys), and therefore private keys, doesn't that require approximately 2^256 computations to find the public key (or slightly less if the key is reached early)? From this it looks to me like the security is around 256 bits?

If security is 128 bits, then why do we even have a 512 bit seed? I mean, why isn't it 128 bits, because the extra bits don't increase security?

Something is seriously not clear to me here. Any help would be appreciated.

What does it mean that the security of 256-bit ECDSA, and therefore Bitcoin keys, is 128 bits? I mean, if I have approximately 2^256 possible points on the elliptic curve (thus possible public keys), and therefore private keys, doesn't that require approximately 2^256 computations to find the public key (or slightly less if the key is reached early)? From this it looks to me like the security is around 256 bits?

You would be right if exhaustive search was the only way to determine the private key for a given public key. That is not the case however.
There exist algorithms that can do this with approximately √n operations (if the number of points on the curve is n). They are practical too (apart from needing an infeasible amount of computation): for example, they don't need much memory. Specifically, variants of Pollard's rho or kangaroo algorithm can be used to solve discrete logarithms on elliptic curves.

If security is 128 bits, then why do we even have a 512 bit seed? I mean, why isn't it 128 bits, because the extra bits don't increase security?

Conservative security. Having a seed that is less than 128 bits of entropy would certainly be detrimental to security, and more than 256 bits is unlikely to be helpful. BIP32, the standard now almost universally used for key derivation, uses a 256-bit key + a 256-bit additional "chain code", for master keys. In retrospect that may have been overkill (disclaimer: I'm the author of BIP32, and I certainly didn't know as much about cryptography back then as now), but it's also relatively cheap, in that the master keys are rarely observed by humans.
While it's true that the security of individual keys is never more than 2128, it's not exactly true for seeds. Having seeds with more entropy does mean that one cannot just compute all keys from a wallet even with a machine present that can perform 2128 computations. This is the intuition that underlies making master keys and seeds larger, but again, this is overkill.

When you say that there are algorithms that can find the private key faster, what exactly do you mean by that? To get from a certain public key to his private one, without brute force? Also, does this mean that the given algorithm will have to be "restarted" for each key? Also, does it mean that those algorithms are better when looking at discovering a single private key, but globally that's not the case because if I brute-forced the private key, I could discover other public-private key pairs along the way and create a lookup table?

I thought the seed should be 128 bits, not less than that. As far as I understand a larger seed does not affect the security of the keys, but it can affect the security of the whole wallet because even if some key is "discovered" whose security is 128 bits, this will not be the case for the whole wallet because its entropy is much higher (say 512 bit), so the other keys will remain secure. So it's potentially good to have seed and mnemonics larger than 128 bits, even though it doesn't just increase the security of the final keys?

If you're interested in the details, read the algorithms I linked to. They exploit the structure that elliptic curve points form a cyclic group; it does not consist of just trying various things - it's a long computation of roughly 2^128 steps that in the end, with high likelihood, reveals the private key. It's not comparable to exhaustive search.

And yes, indeed. If the seed is just 128 bits, the seed itself can be found with 2^128 steps, which gives the attacker all your keys. If you have a 256-bit seed, a 2^128 attack can attack individual keys generated from the seed, but not the seed itself (which would need a 2^256 attack).

Yes, the comparison I made is probably wrong, but what I meant was that using the given algorithms, no other pair would be discovered on the way to the private key. I'm asking simply from a security perspective. In the sense that even with those algorithms it is very difficult to get the private key, because each pair would require a restart and a lot of computation.

So my understanding is that a 512-bit seed would be even more secure than a 256-bit seed as far as the wallet itself is concerned, so increasing the bit count in this context makes sense?

There are multi-target algorithms too, which can search for private keys for multiple public keys at once, I believe, but I don't have a reference for that. They don't discover things along the way at random though; you deliberately start a computation given multiple public keys.

Yes, more entropy is better, but once above 128 bits, and certainly past 256 bits, the gains diminish rapidly. It's almost certainly overkill to go significantly above 128

Yes, that's what I meant. In a sense, some pair will not be discovered along the way, as would be the case with brute force. So even those algorithms cannot "crack" Bitcoin itself. It's just that I still don't understand what you mean by saying that after 256 bits the benefits decrease. I mean, the HMAC-SHA512 function is used, shouldn't the security of the wallet itself (of the wallet globally, not of the keys themselves) actually increase with each bit up to 512?

Yes, sure, the question is just how much that's worth if the attacker can break individual keys

@Filip my understanding is that to "properly backup" 12 words mnemonic is easier that to "properly backup" backup 24 words. Eg. it is easier to remember 12 words in head than 24. There are use-cases for this scenario.


text:
How are children's private keys in HD wallets obtained and how can knowing them reveal the parent's private key?
I'm reading master bitcoin book and I came to the part related to HD wallets and how to create a Child private key (link). It says the following:

The parent public key, chain code, and the index number are combined and hashed with the HMAC-SHA512 algorithm to produce a 512-bit hash. This 512-bit hash is split into two 256-bit halves. The right-half 256 bits of the hash output become the chain code for the child. The left-half 256 bits of the hash are added to the parent key to produce the child private key. In Extending a parent private key to create a child private key, we see this illustrated with the index set to 0 to produce the "zero" (first by index) child of the parent.

The bolded sentence causes me misunderstanding. What type of addition is this? If the classical addition of the left 256 bits of the hash result to the private key of the parent is implemented, then it will lead to a 512-bit private key of the child, which is not correct (the private key should be 256-bit). Is this perhaps referring to addition in the context of the "logical and" between the left hash result and the parent's private key? What type of addition is meant here when the result should be 256-bit? Some answer is given here, but I'm still not clear. The answer is related to the use of modules.
What further confuses me is that the book says that if the child's private key is known, the parent's private key can be determined? That would make perfect sense to me if it was about adding the left half of the hash to the parent's private key. You simply remove the part related to the hash from the 512-bit result and get parent's private key. However, how can it be done here?

Does this answer your question? Derivation of parent private key from non-hardened child

If the classical addition of the left 256 bits of the hash result to the private key of the parent is implemented, then it will lead to a 512-bit private key of the child, which is not correct (the private key should be 256-bit).

As Michael Folkson explains in that answer you linked:

There is concatenation where 256 bits placed next to another 256 bits makes 512 bits. However, what you are referring to is scalar addition. A 256 bit number (256 bit parent private key) is added to another 256 bit number (left 256 bits of the SHA512) and the result modulo p (p = 2^256-2^32-977) is another 256 bit number.

It in some sense will behave like classical addition (before the modulo operation).
Although I am not 100% sure where p is specifically defined, the modulo p operation where p = 2^256-2^32-977 is the relevant operation that will shorten the resulting key to 256 bit. (because p is a 256 bit number)
So you add the left 256 bits to the parent private key to get the ( large 512 bit ) child private key, then you modulo p to shorten the result to 256 bit.

Okay, let's say you and Michael are right. How will the parent's private key be obtained knowing the child's private key? If the module's operations are really used to obtain a 256-bit key, then inversion and obtaining the parent's private key cannot be performed at all... In the book, they say that it is "relatively easy" to obtain the parent's private key from knowing the child's private key.

This was discussed partially here Xpriv can be calculated from the xpub + child private key?
according to: https://medium.com/@blainemalone01/hd-wallets-why-hardened-derivation-matters-89efcdc71671#cc82
the equation for deriving the parent key from child is:
child private key = (left 32 bytes + parent private key) % n

Bob solves for parent private key:

parent private key = (child private key - left 32 bytes) % n

Note here we are using modulo n which is the Secp256k1 curve order, this is a common operation in ECC. It is possible Michael meant modulo n but I cannot confirm this.

The same formula you provided is also mentioned here. But here they use G instead of n. I know that G is a generic point (a point on an elliptic curve) used to generate a public key from a private key. So is n actually G?)

In this section you will notice p is being used and is explained to be a global constant in Bitcoin software but nothing more is said about it:
https://developer.bitcoin.org/devguide/wallets.html#hierarchical-deterministic-key-creation
This section G is being used:
https://developer.bitcoin.org/devguide/wallets.html#id5
I know for certain G is used to derive public from private keys through scalar multiplication.
I do not want to state any incorrect facts about the utilization of these values, my opinion is that in this circumstance it makes more sense that the modulo operation is done via n because of this article that seems to detail it extremely well but since I don't know for certain I can only attempt to provide researched context.

Okay, let's say you and Michael are right. How will the parent's private key be obtained knowing the child's private key? If the module's operations are really used to obtain a 256-bit key, then inversion and obtaining the parent's private key cannot be performed at all... In the book, they say that it is "relatively easy" to obtain the parent's private key from knowing the child's private key.

moving response to this to answer

The same formula you provided is also mentioned here. But here they use G instead of n. I know that G is a generic point (a point on an elliptic curve) used to generate a public key from a private key. So is n actually G?). But here they use G instead of n. I know that G is a generic point (a point on an elliptic curve) used to generate a public key from a private key. So is n actually G?

They also talks here about it.

responding in answer

What follows contains inaccurate notation and language on purpose, in order to make the conceptual explanation lighter and more straightforward.

A BIP32 extended private ("xprv") or public ("xpub") key is composed of two parts: the key itself and the chaincode. The key is either the private key k (for an xprv) or the public key K (for an xpub). The chaincode is a 256 bits integer, identical for both the private and public versions.
BIP32 defines the CKDpriv((kpar, cpar), i) → (ki, ci) function that derives a child extended private key from a parent extended private key at a given index i. (kpar, cpar) are the private key and the chaincode of the parent, and (ki, ci) the private key and the chaincode of the child at index i. Let's call Kpar the public key corresponding to kpar.
For unhardened derivation (i < 2^31), CKDpriv is defined as (loosely speaking):

let I = HMAC-SHA512(Key = cpar, Data = Kpar || i). (With all data serialized as bytes.)
ci is the rightmost 32 bytes of I.
ki is the leftmost 32 bytes of I (as an integer), added to kpar.

The insight in the above is that you can get I without knowing any secret: this makes sense, in some cases you may want an entity (such as an online server) to be able to derive public keys without having access to any secret. But this opens up the possibility to compute kpar from (Kpar, cpar) and ki, or in other words to compute the parent's xprv from the parent's xpub and any child private key (along with its index):

From the parent's (Kpar, cpar) and the index i, compute I.
Let IL be the leftmost 32 bytes of I.
Compute kpar by substracting IL from ki.

Note by using hardened derivation, it's not possible to compute IL without the parent private key. This makes it impossible to use the above trick to get the parent private key from the child private key and parent extended key, at the cost of making it impossible to derive public keys without knowledge of the parent private key.
More on the security of BIP32 here.

First of all, thanks for the reply. Yes, I generally know the process of getting the child's private key from the parent's private key and the left half of the hash. However, there is constant talk about adding those two elements and getting the child's private key, but no one is talking about what type of addition it is. In the end, as I understand it, it's simple addition, like 2+2=4, 3+6=9, etc.

Here it says that ki is parse256(IL) + kpar ( mod n) is used to obtain the child's private key. Since they don't point out anywhere that it's a special type of addition, I'd say it's just a "normal" addition. Also here(hierarchical deterministic Bitcoin wallet implementation) addPrivKes is used which just does the "normal" addition of the left half of the hash and the parent's private key.

The whole problem arose because here Michael Folkson said that it is concatenation and not "normal" addition. I think he is wrong? See also my other question related to the same topic. Link

I don't understand your comments. What is your question? The trick is to substract IL from ki.

Yes, but the problem was that I didn't know if when getting the child's private key, the scalar addition of the parent's private key and the left half of the hash was done or if it was a "special" operation denoted with +. The problem was caused by the answer to this question, because I misinterpreted that concatenation was used. Since I thought it was concatenation, I didn't know what the opposite of that was, that is, what the - stands for.

Yes, adding by the opposite is substracting.


text:
Extract input addresses from a tx message in bitcoin protocol
Recently, I wrote a c++ code to extract some information from some messages in BITCOIN protocol. for example, I extracted bitcoin value and output address corresponded to it from tx messages of this protocol.
Extraction algorithm for output addresses is based on type of scripts such as P2PK, P2PKH, P2SH, P2WPKH and etc. but it is not clear how to extract input addresses.
can anyone explain what is the extraction method for input addresses in tx messages?

Inputs don't have addresses. Outputs do. Inputs spend previous outputs. You'll need to find the output being spent to figure out the address (which is not explicit in the protocol), but generally this is not interesting information for anything but debug purposes.

@PieterWuille You mean that every address which is shown in bitcoin explorers are previous outputs?  I know that UTXO's are same previous outputs, but what about other transactions?

Yes, that's exactly what UTXO means: unspent transaction output. An input simply spends a UTXO and gets to create new ones with the redeemed value. Bitcoin transaction inputs do not contains addresses or amounts; simply the identifier of a UTXO being spent, and the signature data necessary to authorize the spend.

@PieterWuille So how I can access the transactions which  their output addresses spent as input address for current transaction?

No, you must have seen them when they were broadcast or included in a block, and remember them. Note that in order to be able to spend them, you need this information anyway.

As a personal comment judging by the questions you're asking: trying to learn Bitcoin by trying to decipher pcap dumps is not going to teach you much. The Bitcoin protocol is  designed for interaction between Bitcoin nodes, which maintain state, and interact in various ways. Looking at the operation of those nodes directly will explain a lot more than trying to infer it from the trail of TCP messages you can observe.


text:
getting address from public of ScriptPubKey vs ScriptSig
Recently I read some articles about BITCOIN protocol and it's transaction.
I understand(maybe it is incorrect) these rules:

An input is what we call an output when you're spending it in a
transaction.
A public key is the early form of an address.
It seem's that an "input" has two jobs: Select an Output and Unlock it.
Every transaction has a unique TXID, so by using that with a specific
output number(VOUT), we can refer to any output in the blockchain.
Each output is set with a locking script. So if we want to spend
one, we need to supply an unlocking script(ScriptSig).
ScriptSig in an input is a script that unlocks the input and
ScriptPubKey is a A script that locks the output.

if all rules that I wrote is correct, why we can generate an address which is obtained from public key in ScriptPubKey but we can't generate previous output address from the public key inside the ScriptSig?

You can't find the address from the scriptSig or input in general, because you don't know what the output was. Your question makes the assumption that every output contains a (common, but not only) P2PKH script.
If you would know the output being spent was a pay-to-pubkey-hash scriptPubKey, which is spent by revealing the full public key and a signature with it, you can indeed compute the address as the P2PKH address of that public key. But you don't know that without access to the output.
If you would know the output being spent was a P2SH scriptPubKey, which is spent by revealing the full script and its inputs, you can compute the address as the P2SH address of that script. But you don't know that without access to the output.
If you would know the output being spent was a segwit P2WPKH or P2WSH scriptPubKey, which are spent with an input that contains the public key resp. script as the last witness stack element, you can compute the address as the Bech32 address corresponding to that key or script. But you don't know that without access to the output.
If the scriptPubKey of the output being spent is not one of these specific templates (P2PKH, P2SH, segwit), the answer is simply that no address for that output exists. Addresses are a convenient human-readable form for specific types of transaction outputs that are common im payments. But they are by no means the only things supported by the Bitcoin protocol.

we can't generate previous output address from the public key inside the ScriptSig?

AFAIK you can. For example see How to Generate a Bitcoin Address — Step by Step by Jordan Baczuk
However, being able to generate an address from a public key is of no benefit unless you know the private-key which allows you to spend the BTC associated with that address.
Furthermore, if I understand correctly, with an HD wallet, a third party cannot predict the addresses that will be generated because they cannot predict the public keys that will be generated. Since HD wallets usually do not reuse addresses, knowledge of a single public key does not allow you to do anything useful or discover anything interesting.

thank you. I want to generate previous output address just to save a transaction history and no more. and the link you sent it, just explain how to generate destination address which is obtain from ScriptPubKey  and not ScriptSig.
I want previous address not next destination address.

As I've commented before, what you're trying to do, accounting via packet captures, is utterly idiotic. Do not build systems like this if you want to retain control of your money.

@Anonymous I don't have any bitcoin and also i don't want to extract address from packet capture. if you read my question correctly, I just want to find an algorithm to extract previous address from ScriptSig in input, like the algorithm which is used to extract destination address from ScriptPubKey  in ouput.
if you don't understand my question or don't know my answer you are not compelled to answer me. thank you.

You literally asked how to do that in the previous question.

It’s rather funny that you don’t think people would make the connection between the two questions, or are you claiming that it’s a total coincidence?

@Anonymous I'm just a programmer who intrested in Bitcoin and extract useful information from tx messages. so you should not judge me.


text:
Parse scripts to extract address
I want to parse a script with c++ to gain an output address from it.
can anybody introduce a suitable library or a sample code which extract addresses from scripts ?

You can look at the code for decodescript in the Bitcoin source. That outputs addresses (if they exist), so should do what you want

Have a look at libbtc (https://github.com/libbtc/libbtc).
There is a function btc_script_classify() which will return what script type it is (P2PK, P2PKH, P2SH, P2WPKH, P2WSH) and it will return data elements (usually the RIPEMD160 of the pubkey).

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
unknown output script in a sample bitcoin pcap file
I start working on bitcoin protocol to extract output addresses from scripts in tx structures. Now in can produce some valid bitcoin address but some scripts are not familiar in op_code lists introduced in bitcoin wiki(script).
for example, the hex stream value of output script in first tx structure of my sample pcap is like this:
410498361908359fec5adaa624428484e7d117f36f811c7c471f4f1c7dd8184c20b32f0e2590c8d70906ebd585da2ae14ea942e4088891139379b434a26173754750ac

and no op_code is present with 0x41 value.
can anyone explain this matter?

further to Rhagav's answer, description of opcodes can be found here: https://en.bitcoin.it/wiki/Script

0x01 to 0x4b are the "push" opcodes. That is to say, whichever opcode you use, that many bytes are pushed to stack. This makes your script:
PUSH(0x41) 0498361908359fec5adaa624428484e7d117f36f811c7c471f4f1c7dd8184c20b32f0e2590c8d70906ebd585da2ae14ea942e4088891139379b434a26173754750 ac

0x41 is 65 in decimal, which is the length of an uncompressed public key. The last byte, 0xAC, is the OP_CHECKSIG
In short, this is a Pay-to-pubkey (p2pk) script, which were common in the early days of Bitcoin. Indeed, it leads to 1MeeajrKNiF8WtD24S4DmVwEPQYJxeh7Ef, which contains the coinbase Bitcoin from block 93641
Bitcoin also has a decodescript method which you can use via the RPC or bitcoin-cli to decode these scripts into opcodes and the addresses (if an address exists for that script)

Thank you. exactly i want to extract raw script to gain output address. but with this script, the generated address isn't true. my new question which is related to this question is here:
https://bitcoin.stackexchange.com/questions/74501/extract-output-address-from-raw-script-in-bitcoin-protocol

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Extract output address from raw script in bitcoin protocol
I'm working on bitcoin protocol and exactly i want to extract raw script from output script to gain address from tx messages.
for common scripts, like scripts start with 0x76(means OP_DUP),  we can access to address with these steps:

add zeros to start of raw script.
hash with sha256.
hashed with sha256 again.
grab the first 4 bytes of this result and stick them on the end
of the raw script with leading zeros
then encode with base58.
at the end, add "1" to start of last output to gain output address.

these steps are suitable for common scripts, but for scripts like this
410498361908359fec5adaa624428484e7d117f36f811c7c471f4f1c7dd8184c20b32f0e2590c8d70906ebd585da2ae14ea942e4088891139379b434a26173754750ac

which is means:
PUSH(0x41) 0498361908359fec5adaa624428484e7d117f36f811c7c471f4f1c7dd8184c20b32f0e2590c8d70906ebd585da2ae14ea942e4088891139379b434a26173754750 ac

it didn't work and the generated output isn't valid.

You need to hash160 the pubkey first

To clarify against Pieter's answer, blockchain.info and other explorers will usually convert it to a p2pkh address. This script itself will not produce an address

It won't work, because there is no address for that script.
Addresses are templates to construct outputs from. Generally, you should not go the other way, as addresses may be misinterpreted as a place to send coins to, without the receiver asking for it or aware of it.
Furthermore, there are only a few common script types (P2PKH, P2SH, P2WPKH, P2WSH) that has associated addresses. Others, like your example, don't.

You say there is no address for this script, but why when i use this site to execute above script:
http://chainquery.com/bitcoin-api/decodescript
it is produced this valid address :
1MeeajrKNiF8WtD24S4DmVwEPQYJxeh7Ef

That's common practice, but incorrect. The script is a pay-to-pubkey (P2PK) script. The site is showing the address corresponding to the pay-to-pubkey-hash (P2PKH) script for the same key. You should never do this; the recipient may not expect payment to anything but the P2PK script.


text:
Is bitcoin core secure?
Amir Taaki says Bitcoin Core is not secure because of undefined behaviour in this video https://youtu.be/EPpDpULPhHc after 30:00
Is this true and how can we fix this? What are the risks of using bitcoin core?

Undefined behavior is considered a bug, and as with all bugs, software developers try to avoid writing these things. However, as with all other software, bugs do slip in. Just because there may be a bug of a particular type does not mean that the software is insecure. Bugs can have a wide variety of effects, ranging from merely annoying (such as the software crashing), to extremely dangerous (such as one that allows for remote code execution).
There are no known bugs in the wallet that would compromise your private keys. There are no known bugs in the software that would allow for an attacker to take over your computer. The vast majority of users and developers consider Bitcoin Core to be secure.


text:
Looking for data: Fee rate Per Transaction
I'm looking for websites (dashboards, charts, etc.) and/or datasets which show,

average daily transaction fees (sat/vB)
average/min/max fees of each block (sat/vB)

Also, what would be the steps to pull and compile this data myself from a Bitcoin Core Full Node?
Cheers

The first half of your question would be considered off-topic here (asking for a product/service recommendation/review), but the second half of it would be on-topic (asking how to do it yourself). You may consider editing the question to be entirely on topic, that may elicit better answers.

@chytrik: No, the asker here is trying to solve a problem "How can I get XY data?".  That's not a service recommendation. It would be a recommendation/review if the question were "What's the best way to get XY?". I agree it's subtle, but someone trying to figure out how to solve a problem is on-topic here.

@Murch hmm I don't really disagree with you there, but as written it certainly may illicit answers that read "this website/API has the data you need", which don't tend to age well. It is indeed subtle though, I don't mean to discourage Qs in any case.

My litmus test is whether a question prompts answers to select one out of many options or requests an evaluation or comparison of service offerings. It's not clear cut because it's all part of the spectrum "How can I achieve something?" ⇔ "Which service is your favorite?".

You can for example find the "average feerate distribution in blocks per day" here: https://transactionfee.info/charts/fees-package-feerates/
I'm not aware of a direct source for "average/min/max fees of each block (sat/vB)" for free (I checked transactionfee.info, statoshi.info, and txstats.com). Coinmetrics has them as "pro metrics" but not in the free community version.
Some block explorers (e.g. blockchair.com) allow you to search transactions per block. You could use that to script retrieval of the transaction with the smallest feerate and the transaction with the highest feerate from each block (don't forget to exclude the coinbase transaction, and note that CPFP could skew your perception of the lower bound). You can estimate the average feerate via dividing the total fees of a block by its total weight.
You could extract all of that information from the blockchain directly, but that is a non-trivial amount of work.


text:
How often does target hash change and how to find it?
So basically I was researching target hash and trying to figure out how I can find it. I know there are posts out there that have said that Bitcoins target hash is a 256 bit number with 32 bits of leading zeros. But, where can I find this information. Basically is there an equation I can use to find how many leading zeros are currently in the target hash or a direct source where I can find this? Also, how often and by how much does this target change?

Bitcoin’s network has several specialized nodes called “miners” who use specialized equipment to solve cryptographically hard puzzles. If they are successful, then they will get the opportunity to add blocks to the BTC blockchain successfully.
This is how it works:

The miner picks up transactions waiting in the mempool and hashes
them.
They add a random hexadecimal value to the front of the hash and hashes the entire value.
This hash needs to be less than a particular value, which is called “target”

Difficulty is a measure of how difficult it is to find a hash below a given target.
The Bitcoin network has a global block difficulty. Valid blocks must have a hash below a target. This Difficulty changes every 2016 blocks (Approximately every two weeks). The level of Bitcoin mining difficulty increases or decreases according to the ease of mining within the protocol. Remember, Bitcoin needs to have a consistent block time of 10 minutes.
The problem is that targets are hard for people to comprehend. The difficulty is inversely proportional to the target and easier to understand.
The specific formula is: difficulty = [0xFFFF x 256^(0x1D - 3)] / target, to find the target you just rearrange the formula.
The difficulty of Bitcoin at the genesis block was 1. The difficulty can be thought of as how much more difficult mining is now than it was at the start. The current difficulty level is: 28.6 x 10^12.
You can check the current difficulty rate at: Clark Moody's Dashboard

I appreciate the deep dive but I just wanted that formula you gave me there. Although, I am a bit confused if you could clear up what the values in the brackets are specifically that would be awesome.

Fixed a typo in the formula, the correct one is with 256 not 255, difficulty = [65535 x (256^26)] / target


text:
What are bitcoin Pool APIs?
I contacted a website and asked them where they get their data for crypto like Bitcoin, as I am trying to find the place for the most up to date and reliable data on Bitcoin and other crypto. They had sent me an email back saying they get their data from block explorers and pool apis. Block explorers were pretty self explanatory and easy to find and pull data from. But I wasn't 100% confident in my idea of what a Pool API was. Is it an API that is connected to a specific mining pool that gives you real time data? How do I set it up and where does the data get sent to? Or am I completely misunderstanding what it is?

Most pools offer some kind of proprietary API endpoint that offers basic stats about the pool or stats about a specific user/rig in that pool.

Is it an API that is connected to a specific mining pool that gives you real time data?

Yes.

How do I set it up and where does the data get sent to?

The current mining pool breakdown by hash rate is shown here on mempool.space. You'd have to go to the site of each mining pool to see whether they offer an API, what data that API provides and how to use it. For example the Braiins API docs are here. When you use an API your machine requests data from someone else's server and if successful the server returns this data.

Yes, it is an API (Application Programming Interface). You are on the right track here. API's are not all the same. Pool API's are different based on the site. They are up to the site (pool) providers to setup. Some more reliable, and faster (or slower) to update than others. Pool API's are designed to cover the database of the entire pool rather than one specific miner or wallet. So again, yes, the pool api pulls the data from the whole pool. You will need to set it up based on their options they've made available per pool.The data is sent to the receiver that polls the API. Each pool (with API) have the instructions for the mining pool. You will have to set it up in whichever interface you prefer. Might be good to include what you are trying to poll the info from to get more specifics.