text:
Is there a distinction between an attack and an exploit?
I'm writing up lecture notes and attempting to put the language on the most logical footing.  I'm hoping to distinguish attacks on the consensus itself from exploits of the protocol, as these seem to be separate categories.  For example an empty-block attack is clearly an attack: the purpose is to damage Bitcoin, whereas Finney's attack is not attempting to strike a blow to the consensus, it's an attempt to exploit consensus rules in a way that is more favorable to the attacker.
Am I going against a decade of convention or is there some precedent here?
More generally, is there a definition an "attack"?

An empty block attack is when someone who wants to hobble Bitcoin acquires over 51% of the hashrate and uses this to maintain a longest chain containing only empty blocks.

An exploit makes use of a software bug or security vulnerability to induce unintended or unanticipated behavior in software to the attacker's advantage.
Neither a Finney attack nor an "empty-block attack"¹ are exploits. In both cases, the software is working as designed. A Finney attack would be classified as either a user error (relying on an unconfirmed transaction) or social engineering attack (misallocated trust), and a majority attack breaks the main security assumption of the protocol that the majority of the hashrate is not colluding to attack.
"Attack" is a hypernym collecting any sort of attempt to harm the network or take advantage of other users. Exploits are a specific subcategory of attacks. It's not obvious to me what you're exactly trying to disambiguate, but perhaps you're trying to distinguish attacks on users and attacks on the whole network?

¹ I'm operating under the assumption that "empty-block attack" is supposed to refer to a majority attacker displacing any blocks that confirm transactions, or monopolizing block creation and only mining empty block, to subject the network to a denial-of-service attack.

Yes, I'm distinguishing between attacks on the network, and ways which users of the network can take advantage of others, which can be extremely broad and would include any sort of MEV.  It seems strange to me that the same word is being used.  Selfish mining, undercutting, etc are examples where the protocol is functioning as prescribed, but not as intended.  Maybe exploit is not the best word?

I guess selfish-mining could be categorized as a protocol exploit, because the intention is for miners to earn a share of the reward equivalent their relative hashrate, and if you have more than 33% of the hashrate, you can earn more than your share by employing this tactic. I don't know what "undercutting" is referring to.

Fee undercutting is an (attack?, exploit?) that becomes available if there are significantly more transaction fees mined in the  latest block than are available in the mempool.  By mining a block at the same height with some but not all of the fees, a miner can claim more fees for themselves than if they had extended the chain tip.  Further, because they can leave more fees available, rational miners will choose to extend the block mined later,  causing the first-mined block to become stale.

Ah, I know that attack as "fee sniping". Yeah, one could say that's a bug in the game theory being exploited.


text:
How many times and where exactly is ECDSA used in Bitcoin?
I'm currently a little confused about where exactly ECDSA is used in the transaction process... some resources seem to say it's used once, whereas others seem to say it's used three times.
Which is correct here?
A:
ECDSA is just used to generate the public key, which is then used in the creation of the digital signature (thus ECDSA is indirectly used to create the signature). Verification then only requires the public key of the recipient.
or, B:
ECDSA is used to generate the public key, then again to create the digital signature together with the public key and some hashing, then a third time to verify the signature.
I don't have a programming / computer science background, so am looking for a general description of the process as opposed to a code-based explanation (although if it really helps clarify, I don't mind!). I do know the basics of how ECDSA works.

A digital signature scheme, like ECDSA, is formally defined as a collection of three algorithms:

A key generation algorithm which creates a (private key, public key) pair.
A signing algorithm which given a message and a private key, computes a signature.
A verification algorithm which given a message and a public key, verifies that the signature was created by the corresponding private key.

These three algorithms are used at distinct times in the lifetime of a Bitcoin transaction:

The ECDSA key generation algorithm (*) is used when a receiver wants to create an address. The private and public key are stored in the wallet, and the public key is transformed into an address, which is then handed to the sender. The sender then constructs a transaction crediting that address.
When the receiver wants to spend the received coins, they need to construct a transaction which "redeems" them. This involves the ECDSA signing algorithm to authorize the transaction to spend those coins, showing the owner of the coins consents to it.
When that transactions is broadcast to the network, Bitcoin nodes and wallets will verify the transaction, as invalid transactions are not allowed. This involves the use of the ECDSA verification algorithm.

So to answer your question: it depends which algorithm you're talking about. The key generation algorithm is used once per address. The signing algorithm is used once per transaction input. The verification algorithm is performed by every node once for every transaction input.
(*) = In modern Bitcoin wallets, the addresses are not actually randomly generated anymore, but derived from a seed instead. This could be seen as choosing to use a distinct, but compatible, key generation algorithm. It's also worth noting that key generation for ECDSA isn't actually ECDSA-specific, as every 256-bit number is a valid private key, and the conversion algorithm from private key to public key is (mostly) shared across all EC-based algorithms (e.g. it is compatible with BIP340 Schnorr signatures, and the ECDH key agreement protocol).

Ah! I didn't realise it was three separate algorithms under the ECDSA name. Thank you! And your clarification at the end was extremely helpful :)

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Does the length of the blockchain affect computing power needed to verify transactions?
As the blockchain for something like BTC grows, does that affect the time needed to verify new transactions and/or the integrity of the blockchain? From my understanding, miners only verify the hash of the recent block, which is adjusted for difficulty to keep mining times roughly constant. But then how do they verify/check if earlier blocks were tampered with if not by checking the validity of the entire blockchain?
What I'm most interested in is whether or not the length of the blockchain will eventually make it prohibitively costly to continue to mine. I know hash verification can be dealt with through difficulty, but what about overall blockchain length? Is it just computationally trivial to verify the entire blockchain?

In short: No.

As the blockchain for something like BTC grows, does that affect the time needed to verify new transactions

Since the blockchain is immutable - or at least all but the last few blocks. It doesn't need to be verified more than once. Nodes will create, from a single pass of the blockchain, a list of spendable coins - the unspent transaction outputs (UTXOs) and use that UTXO list to verify future transactions. The UTXO list is a lot smaller than the blockchain and doesn't grow as fast.  Spent UTXOs are removed from the UTXO list.
Changes in the size of the UTXO list are presumably driven by factors such as:

Growth in monetary amount of issued Bitcoin. The total amount of BTC grows towards 21 million, but at an ever decreasing rate due to the halving process applied to block rewards. In other words, Coinbase transactions necessarily create new UTXOs.
Fragmentation of UTXOs in the UTXO set is probably driven by, among other things, the long term trend of an increase in the value of Bitcoin. If a Pizza used to cost 1000 BTC and today costs 0.0001 BTC, one might expect that there would as a result be a lot of small UTXOs rather than a few large UTXOs.
Agglomeration (or consolidation) of UTXOs. People and organisations who control money in Bitcoin have incentives to use a Bitcoin transaction to join small amounts together into a bigger amount. This can reduce fees if managed carefully. It also diminishes the creation of unspendable dust.

Note that all nodes verify data, it isn't just miners. Other sorts of nodes, such as wallets, do not depend on miners to perform verification and do not need to trust miners. Note that verification is very different from confirmation.
I believe the average computing power (energy) consumed by a miner per block mined is primarily determined by the current difficulty/target. This reflects the total hashpower being applied to the network by all active miners. I believe other factors are relatively insignificant.
The average computing power (energy) consumed by any node per incoming block verified only significantly depends on the number and complexity of transactions in that block. It doesn't depend on the size of blockchain.

how do [Bitcoin nodes] verify/check if earlier blocks were tampered with if not by checking the validity of the entire blockchain?

If you present a node with a new version of an old block, the node will validate it in the usual way. It will presumably have to walk back it's UTXO set using a change log. It knows that prior blocks have already been validated and don't need to be revalidated. It will then assess whether the new chain-head based on that new block has greater work than the existing chain-head based on the potentially replaced block. If the new block is the start of a chain-head with more work then all the blocks in the replaced chain-head are effectively invalidated, including all coins created in transactions in them.
I believe nodes keep track of at least some alternative chain-heads. Presumably both potential replacements and recently replaced. I don't know what criteria they use to discard them (e.g. absence of growth for some threshold time?)

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
bitcoin stuck with no confirmations
I'm quite new to this. I have transferred coins to a wallet 4 hours ago and I'm still stuck with no confirmations. I do have a Bitcoin wallet on blockchain.info. I transferred from Bitcoin market website straight to another wallet if mine and now its stuck. How can I reverse it or get it moving again? I have tried looking for the trusted peer option and all that, but can't get my head around it. This is my block number on blockchain.info: a8abd5243ac486822577208f87e46f6864baeb1250c1434b3a1811a017ce6e2f
Any help would be appriciated.

I think your question would be a lot more readable if you would use punctuation, proper capitalization, and generally proofread for spelling and clarity. There is an "edit" button right below the post that you can use to make changes.

This transaction is unlikely to be confirmed for a long time.

It has very low priority because it is spending an output which is unconfirmed. None of the parents of this transaction are confirmed either, going back 5 generations.
Each transaction in the chain pays a very low fee of around 0.00000259 BTC, which is below the recommended amount and below what other people are paying to get into a block.

Just as a rule of thumb always pay a fees of 0.0001 or more and you will never have slow confirmations. The amount is just 3 cents and shouldn't be decreased .

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Estimatefee value not corresponding to actual fee
When I call estimatefee 1 in testnet it returns: 0.00044444
Then when I call sendtoaddress xxxxxx 0,1 (which got the first confirmation within a couple of minutes after sending) I see that the fee included is: 0.00001662 BTC, which is about 27 times smaller that the original estimate.
What am I missing?

estimatefee 1 will return the approx. fee per kilobyte.
Because sendtoaddress could use multiple of your unspent coins, the size of your transaction is not know until it is created (until the coin selection ran).
Example: Fresh wallet. You did receive 10 times 1 BTC. Now you create a transaction with sendtoaddress <ard> 9.5. This will produce a transaction with 10 inputs (your received 10 x 1BTC). Therefor the transaction size is bigger than if you would have received 1 x 10BTC.
At the moment there is no easy way of getting the transaction fee before sending the tx (on RPC level). You could use Bitcoin-Qt (there you will see the fee before sending within the confirmation dialog).
There is a pull request for bitcoin-core that would allow fee calculation before sending the tx: https://github.com/bitcoin/bitcoin/pull/6088

The pull request will provide a great functionality that is missing today from the RPC API, nice work!

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Correlated raw transactions' input/output chaining
So I'm in the process of implementing a BTC-based payment module where for every transaction to a non-wallet account a fraction of the amount will end up to a charity account in a external wallet not controlled by this module.
A typical round for that would be:

The module receives the payment from user
It forwards the 90% (this percentage is arbitary) of the amount to its final destination (e.g. a merchant)
It now has to forward the remaining 10% to the charity of the user's choice

In a lower level, the system:

Receives a transaction TxFromUser from the user with an amount of X BTC
Waits until it gets 6 confirmations
Creates a new raw transaction, TxToMerchant to forward the payment to the merchant
Creates another raw transaction TxToCharity to forward the donation to the charity

Going a bit deeper:
TxToMerchant:

Create new raw transaction
Add new input: TxFromUser
Add new output: Address: Merchant's address, Amount: 0.9 * TxFromUser.Amount
Add new output: Address: Module's own wallet address, Amount: 0.1 * TxFromUser.Amount
Calculate the fee for this transaction
Subtract the fee from the second output's amount
Create the transaction's hex and sign it with the right key
Send the transaction and get its id: TxToMerchant.Id

TxToCharity:

Wait for the above transaction (TxToMerchant) to get confirmed
Create a new raw transaction
Add new input: TxToMerchant (this will be the change we asked for above)
Add new output: Address: Charity's address, Amount: TxToMerchant.Amount = 0.1 * TxFromUser.Amount
Calculate the fee for this transaction
Subtract the fee from transaction's output amount
Create the transaction's hex and sign it with the right key
Send the transaction and get its id: TxToCharity.Id

I am aware of the fact that I could merge these two transactions (TxToMerchant and TxToCharity) into a single transaction, however for various reasons (business requirement) let's just say that this is not possible at the moment.
The issue I have with the above implementation is this: waiting for the TxToMerchant to confirm creates a lag, as I cannot forward the payment to the charity TxToCharity until TxToMerchant is confirmed (or can I, without having to worry about transaction malleability etc?).
On top of that, when the time comes to add the TxToCharity's input I have to keep track of how the change was returned in TxToMerchant and then get the right unspent transaction (where listunspent.txid = TxToMerchant.Id), is there a more efficient and less error-prone way to do this?
Let's suppose that the user wants to split the payment between 2 merchants (or a merchant and a freight service, which makes more sense), so in the above example there would be 1 more transaction like TxToMerchant, let's call it TxToFreightService. Let's say now that TxToMerchant goes trough well and gets confirmed but TxToFreightService fails and never gets confirmed because the input used for it was a double-spend regardless the 6 confirmations we got (for its input) before processing it. TxToCharity depends on TxToFreightService because the requested change in TxToFreightService will serve as the input to TxToCharity. How do I deal with this scenario programmatically without having to perform corrections by hand every single time this happens?

I think your business process is confusing the issue. I'm going to assume that the "business" side of this also wants to take a cut of the transaction or it wants to delay paying the charity and that is why this has become more complicated.
There are a number of considerations not least the fact that you are using one input to pay the merchant, and the change to pay the freight, and the change of that to pay the charity.
What you are concerned about is the inability that you have built-in to pay the one party because of a preceding ttransaction, but that's your business process at fault not the Bitcoin protocol.
You should also consider the minimum payment amount and the eventual network fee, neither of which are %age based.
These are not mentioned and in fact by making two (or possibly four) transactions you are incurring up to 4x the bitcoin network transaction fees.
This is one of the main reasons why (from a business perspective) you should not split the transactions up ie it is more expensive and who is going to wear the costs? The charity? the merchant? the freight? You?
Can I suggest also that you have misunderstood the "confirmations" as something other than your transaction being included in a block. When a transaction is requested by a peer prior to it being included in a block - the peer validates the transaction independently, and if for any reason it is not good it is rejected. This is NOT the process of "confirmation".
All peers in effect will do the same validation to your transaction as it propagates through the network (which BTW is extremely fast).
Confirmation on the other hand is when the transaction has been included in a block, and the only time this becomes an issue is if there is a fork in the blockchain. But even then you still can rebroadcast a transaction if the "other fork" becomes the final one provided the basis on which it was made is good.
The point I want to make is that you are placing too much emphasis on confirmations for transactions that you yourself have created, and then wait until someone else confirms what you already know to be true, and this seems completely pointless.
Perhaps you can expand your question to explain the "business reasons"?

+1 For the extensive analysis.
The delayed payments to the charities and the freight services is where the revenue stream is being generated by. I am fully aware that this is not a bitcoin protocol issue, however I am positive that more entrepreneurs who will want to make use of the built-in credit capabilities will face the same issues. The fees are included in our model (although not mentioned above) and dust payments will not qualify and will be forwarded intact to the merchant.

Regarding the confirmations I also had the same opinion as you until I stumbled upon some veterans' posts that made it clear that we shouldn't assume that just because we created transactions based on change from a previous in-wallet transaction that this change is safe to spend. Please take a look here: http://www.reddit.com/r/Bitcoin/comments/1xm49o/due_to_active_malleable_transaction_relayers_it/

@DougPeters it's not that you have to wait to spend previous in-wallet transactions, it's that you are obliged to. Otherwise no subsequent transaction will validate. On a side note, you are probably on slippery ground if you think that Bitcoin can support any form of credit. You will immediately find yourself up against a competitor who offers immediate distribution, just because they can. It's an arguable point, but not one I would like to be trying to make work;)

With raw transactions you can send a payment that has as input unconfirmed change so I'm not really obliged to wait, the thing is that this is not really a good practice. Bitcoin already supports a form of credit with the use of escrow, I decided to take this into my application, but again let's stick to the main answer's questions for now.


text:
How to detect a race attack and re-spend the already-spent outputs?
My service automatically accepts, processes and sends payments to its users. It connects to bitcoind through its JSON-RPC API and makes use of the raw transactions interface.
What is going to happen in this case:

The service receives 3 payments by 3 different users.
All payments are included into the next block so they now have one confirmation.
Upon one confirmation, the service forms a new TX that spends these 3 outputs. The appropriate fee is attached. The TX is now valid but unconfirmed.
A few seconds later a double-spend attack in one of the TX's (now-spent) outputs comes in place.

As I find it hard to reproduce the scenario above, my question is:

What is the best way for the service to detect this double-spending attack through the RPC API?
Once it detects the double-spending and realizes that the TX will never get confirmed, can it re-spend the same outputs (not including the one that has been double-spent) right away in the case of a race-attack?

I feel like this is a repeat of the answered questions you asked here and here.  If this is a real risk for your service, I highly recommend that you sit down with the regtest mode and develop a solid unit test you can run against your production code.  There's a basic example in this answer.

@DavidA.Harding thanks for the suggestion regarding the test procedure and all your great replies so far. Please correct me if I'm wrong but AFAIU this question describes a fork-based double-spending attack, while the question above refers to a race-based double-spending attack, which in my understanding is a totally different case, isn't it?

I'm confused.  In bullet #1, you receive outputs a, b, c from TX1, TX2, and TX3.  In bullet #2, these TXes get one confirmation.  In bullet #3, you use those outputs as inputs a, b, c to create TX4.  What exactly happens in bullet #4?  A double spend which invalidates TX1, TX2, or TX3 would require a fork (see previous answers).  A double spend using inputs a, b, or c to invalidate TX4 would need to be created by the same private keys your service uses or some multisig thing; is that a risk here?  Or are you paying other people and worrying about whether they double spend TX4's outputs?

@DavidA.Harding my bad for not making it clear, thanks for pointing it out. What I had in mind is that the attacker would broadcast two conflicting TXs in rapid succession with the malicious one getting propagated faster than the other that pays my service. If I had just spent the outputs of this TX (which would have 1 conf) to form another one (TX4) how should the service behave? But I think I get what you're saying: if there is at least 1 conf by the network a race-based double-spending attack is not possible, while a fork-based double-spending attack is, is that correct?

that is correct.

As Doug said in the question comments (posted here as an answer by his request):

If there is at least 1 conf by the network, a race-based double-spending attack is not possible.

Therefore the attack he describes is a fork-based double spend similar to his questions here and here.


text:
Is there a way to pay my credit card balance with bitcoins?
So imagine that I have 200 USD on my credit card balance from a bunch of small purchases and bar drinks and tips that I want to pay using bitcoins.
I know that I can but gift cards from gyft or do online purchases using joinsnapcard. But If I invite a round on my local non-bitcoin-friendly bar, I want to pay with bitcoin anyways (even I I use credit card system as proxy)
Is there any service that does that for me?
Edit 2017-07-27:  There is a debit card that can be funded with Bitcoin, Ethereum and Litecoin (via Coinbase) called ShiftCard https://www.shiftpayments.com/
Thanks,
Nico

If you live in the states the only service that comes close to this would be
https://www.xmlgold.eu/  - you can even order a pre-paid credit card that you are able to top up with bitcoins.
https://www.okpay.com/en/index.html - Okpay used to offer the same service but due to regulatory reasons they have ceased offering thier services to US citizens.
The only other way I can think of to do this would be to use http://bitcoin-otc.com/  here you  can see if you can find anyone willing to pay off your credit card for bitcoins.  Its kind of like online trading pit.  Hope this helps.

Thanks for the answer, xmlgold.eu service is new to me.   about "anyone willing to pay off your credit card for bitcoins,"  I should only give them my credit card number right? how would they pay?

UPDATE now https://www.xmlgold.eu/ says "Country restriction: We do not serve customers from USA, please accept our apologies."  Any idea why? any other service?

Lately it is observed that the Crypt Currency is used for malicious purposes, so countries have started banning the services to reduce similar usage in their country.

talk has been appearing lately of PayPal considering to accept Bitcoin. That would give you a somewhat direct interchange between the "two worlds" though at rather considerable cost (PayPal commissions, their outrageous bid/ask spreads, plus fixed base-fee)...

For Canadian credit cards, you can use the service of YesToBitcoins.com, although it supports credit cards only, so no prepaid credit cards.

This looks like a valid solution, but the site doesn't do a great job of explaining themselves (their explainer videos are worthless), and neither does your answer.  Perhaps you could edit your answer to give a better explanation about what this service does and how it solves the OP's problem.


text:
What happens if I delete wallet.dat?
I have this question, say I have my coins in my local wallet and 1 of 2 things can happen, say I somehow delete the wallet.dat file or some kind of malware deletes the file. I have the password private key to the wallet, would I need the wallet.dat file to recover the bitcoins or could I just recover with the private key to the address? Would I need both?

wallet.dat is mostly a collection of private keys for multiple addresses.

In the off chance I lose access to it, what can happen?

@fatso113: I think you may be confusing "private key" with "password". The private keys are in the wallet file, the password to the wallet will not help you if you don't have the wallet itself. You will lose all your coins if you don't have a backup of the file.

Thank you, sorry for not being very clear

Since wallet.dat is mostly a collection of private keys, you will simply lose all private keys that you have not duplicated or copied. It also by default stores the next 100 Bitcoin addresses it will give you when you generate a new address as a safety measure.
Bottom line
You will lose the copy of the private keys to addresses you've created or added but you won't destroy them. A backup will save you regardless if wallet.dat exists or not. If you restore wallet.dat from a backup, you:

will not lose past transactions, those are stored by the network, not you.
will not lose ownership of any bitcoins you may have acquired under the private keys you backed up, generating a new one and not creating a backup may result in loss. Keep in mind first 100 addresses are stored in wallet.dat.
will not need to know the address of a public key. Those are derived from the private key mathematically.

If I lose wallet.dat but have the private keys derived from wallet.dat, would I be able to have access to my coins again?

@fatso113 Yes.  You can access your coins without a wallet, so long as you have the private keys.  Be careful, though, so can anyone else!

This was confusing because I got mixed up with BIP38 encryption with paper wallets. I understand now


text:
Can I prove a double spend attempt with a signed tx with same input?
I am designing an cross-chain atomic swap in a custom blockchain - Protoblock. I think it can be done as long as Protoblock knows how to parse p2pkh bitcoin transactions. The critical part is being able to "reverse" things when it sees a double spend.

Alice has 1 Bitcoin
Bob has 100 Fantasybit
Alice will swap 1 BTC for Bobs 100 FB

Step1) Bob signs a Protoblock "Swap" transaction(TX1a) that contains an unsigned bitcoin transaction(TX1b) that must have 1. a ScriptSig input with p2pkh from Alice(bitcoin-address) and 2. a p2pkh output to Bob(bitcoin-address)
Step2) 100 FB of Bobs is locked for 24 hours or until...
Step3) a Protoblock transaction(TX2a) is seen that contains the signature for TX1b and TXID(TX1a)
Step4) 100 FB is transferred to Alice and is locked for 24 hours or until...
Step5) a Protoblock transaction(TX3a) is seen that contains a signed bitcoin transaction(TX2b) that has 1. the same ScriptSig input(s) with p2pkh from Alice(bitcoin-address) as TX1b and 2. an output that is NOT a p2pkh output to bob(bitcoin-address)
Step6) 100 FB is sent back to Bob, the same state as before Step1

The reason for Step5 is that, a signed bitcoin transaction is not enough to prove that Bob received the bitcoin from Alice, because Alice could just sign the transaction, but not spend it to bitcoin, and just double spend it.
However, Bob will see that Alice spent the input and will create TX3a that gives him back his 100 Fantasybit. In fact, anyone can be watching bitcoin and create TX3a as a service for Bob.
Question: is there a way that Alice can spend the same inputs from TX1b, that will be undetectable with above algo?
Note: the bitcoin transaction with the outputs for the TX1b inputs, will be created by the wallet software doing the atomic swap. So Alice will first have to send her 1 Bitcoin to herself to make sure its a simple p2pkh. Here is the Protoblock code that creates the p2pkh outputs and bitcoin tx:
https://github.com/SatoshiFantasy/--skillSale/blob/4e59f50b7555527046804418d6483b0df1933637/share/fantasybit-core/FantasyAgent.cpp#L444
https://github.com/SatoshiFantasy/--skillSale/blob/4e59f50b7555527046804418d6483b0df1933637/share/fantasybit-core/FantasyAgent.cpp#L525

No Atomic Swaps are by definition atomic meaning either there's a swap and both parties redeem the coins or no . I suggest you take a look at the Decred repository to understand the mechanics of the swap .

the term "atomic" swap is used, even though its not atomic. i bet decred is just as "atomic", I will take a look... anyways but the question i'm asking is in relation to a double spend, and if there can be proof of a spent input.

decred uses the standard timeOut technique, which (i believe) requires one party to move the coins again - or else the swap does not go through


text:
Is it possible to mine bitcoin in a trustless way with a Pool?
Some context
As far as I understand there are two ways in which an individual (or a company) can mine Bitcoin:

Solo: if one has enough hashrate to secure a block often enough to pay for the electricity bills, or one is just a hobbyist playing Bitcoin lottery, that's usually a person who controls very low hashrates and its willing to go mine at a loss.
Or through a mining pool: somewhere in between the previous two cases, a person that needs to secure a revenue for every kWh of energy it consumes but with a hashrate too low to produce a block in a reasonable amount of time.

When is solo mining trustless?
Mining solo through your own bitcoin full node is the ideal way to contribute to the decentralization of the consensus and strength of the network.
But I've seen there are alternative solo mining configurations, in which the miner equipment can be pointed to other people's server. solo.ckpool.org is one example and
another example is public-pool.io to which nerd miners are connecting to.
In this case I see room for bad behavior from the pool operator that could be sending
block templates to the miners in which the coinbase transaction pays to some bitcoin address that's not the solo miner address.
Unless there is a way to verify on the miner side that the hardware is actually
working on the block with the correct coinbase destination address every time
a block template is received.
Pool mining is completely based on trust
On the other hand, as far as I understand, pool mining is fully based on trusting the pool operator not to run away with the Bitcoin reward, because the coinbase address is controlled by the pool operator and not the single miners.
It has happened recently that F2Pool mined a block with a 20BTC fee on one transaction, and that money didn't go to the miners but it was refunded to the company to made that transaction. It wouldn't have happened if there was some kind of smart contract in the coinbase lock script that bind the reward to the miners instead of giving it all to the pool address.
Another possible situation is that a pool gets coerced by the local government to request
KYC from participant miners (eg. Foundry USA) or the miner share doesn't get paid.
Wrapping up
I would like to ask this community:

Are there mining pools that do not require trust?

Is there any protocol that pool operators could use to make mining trustless?

Secondary questions, just for debating the implications of this issue with respect to Bitcoin mission:

Why is it that 90% of the hashrate is concentrated on trust-based pools?

Why is it that such an important aspect of the Bitcoin protocol (mining)
has been neglected to the point that a mid-size miner has to connect
to a Pool that either requires KYC or can steal the reward at will?

Not a full answer, but look into p2pool. It's effectively a separate "blockchain", with faster "blocks", which are pool shares. It can only reduce the variance by a relatively small constant factor however, and it appears that mining pool users like more variance reduction over the need to trust the pool to some extent.

@PieterWuille, thanks P2Pool seems very interesting indeed. However, it looks the pool is defunct: the last mail in the mailing list dates back to 2017, the website for statistics is down and last commit in the github repository was in 2018.

Yeah I think the variance reduction it offers compared to solo mining is just not enough for most users' taste.

some kind of smart contract in the coinbase lock script that bind the reward to the miners instead of giving it all to the pool address.

I'm no expert1 on pools but an obvious obstacle is that the coinbase transaction has to be decided ahead of time but the distribution of mining effort by pool members is not known until a block is mined.
I think there is no obvious way for a Bitcoin script, written in advance, to measure subsequent mining effort and apportion rewards accordingly.

1. i.e. fairly ignorant

This is a good point.


text:
How to open the port 8332 on EC2 instance?
I'd like to access the EC2 instance that installed bitcoin mainnet node.
I've found out that I need to open the port 8332 of instance from the question, but I have not idea how to do it.
I tried to add port 8332 on the security group, however I cannot select 8332 port, only can select 80.

Could you tell me how to open the port and access bitcoin node?
Update
I've set port as the attached picture.
However, it returned the following error.
Connection refused - connect(2) for 'publicIP address' port 8332
It means that I still cannot connect the instance? or I've already connect the instance, but returned error by another reason?

In the Type dropdown menu, change HTTP to All TCP; enter 8332 as the port number; enter the IP of the computer that will be sending RPC requests under 'Source.'

Dear Nick
I've tried add port 8332. Please check the update of question.


text:
Is the new Google Compute Engine suitable (and affordable) for running a bitcoind instance?
Google just recently (at Google I/O 2012) announced their new cloud product, the Google Compute Engine. I don't really understand what it is exactly but I heard it's basically some sort of VPS service.
So I was wondering if it would be suitable for running a Bitcoin client for a web application. I don't know what resources are required for running such, either do I know what prices you get when doing so in Amazons platform. Google says Compute Engine's pricing gives you the same computation power for half the price of competitive products, but I cannot really compare.
Did anyone made this consideration as well? Did anyone take a closer look at this?

Comparing the GCE's pricing to Amazon EC2's pricing for the cheapest tier, it appears that Google would be cheaper should the CPU usage be small (which in case of bitcoind it should be in most cases). However, as GCE is still early in development, few people could run benchmarks on it and the prices can change abruptly (much like the prices of Google App Engine changed after the early beta period).
As it stands, Google Compute Engine looks like a viable option for cheap hosting of bitcoind.

I don't know exactly how pricing works for those products, but if you now schedule your bitcoin tasks every hour, can you cut the cost in half? Or will you be billed per running hour?

@StevenRoose It is probably possible to reduce the costs, but it depends how precisely you are billed. It probably would be possible to be billed in finer amounts than per whole hour.

@ThePiachu: In AWS EC2 you're billed per whole hour, so you're out of luck if you do a small thing every hour. I don't know about GCE.

I meant by working an hour and skipping an hour. So actually every 2 hours.


text:
What is a lower bound for global bitcoin mining energy usage?
How can we estimate a lower bound for the electrical energy consumed by Bitcoin's Proof of Work (PoW) mechanism?
Obviously there is no hard bound, but there are some market laws at work and I think it's possible to give an estimation
under some assumptions, but strangely I have not been able to find much like that.
Clarification:  I'm specifically looking for something that allows predictions for the future depending on bitcoin price
and usage, where mining technology and other factors might have changed. Compare
my own answer as an example why I think this is possible,
though I feel it could be improved. (Please note that it seems the energy consumption is
pretty independent of mining technology ). Of course that kind of answer needs some assumptions - that's perfectly fine.
I think it's important to also have such a lower bound to prepare for what'd happen if the
bitcoin price really goes above $100000, as some think, considering that the mining reward is proportional to the
bitcoin price, and since we are already (as of 10/2023) around using up
0.4% of the world electricity production (= 15GW, assuming world production is about 30000TWh/year)
... What's your take on that?

Hi Hans-Peter, thanks for your interesting and well-researched questions lately. While answering your own question is a great way to establish that you have "done your homework" , please focus on asking your question in the question post. "Could you please check my work"-style questions don’t work particularly well in our format. However, self-answers are explicitly welcome here, so I would propose that you round out your question post with some clear prompts and post the last four paragraphs as your own answer post below instead.

@Murch Thank you for your feedback! I moved the answer out and updated the question accordingly, and tried to better clarify the intention of my question.

One way to estimate a lower bound for Bitcoin’s electricity usage is to assume that the current hashrate of the network is produced by the most energy efficient technology in the market. A cursory search suggests the Antminer S19 XP with (according to Bitmain's product page) 0.022 J/GH as a likely candidate. With that baseline, today’s 471 EH/s would roughly take 0.022 J/GH * 471 EH/s * 1000 EH/PH *1000 PH/TH * 1000 TH/GH = 10,362,000,000 J/s = 10.362 GW. This closely matches the "Theoretical Lower Bound" estimated by the Cambridge Bitcoin Energy Consumption Index (CBECI) today of 9.41 GW.
A reasonable lower bound would then at least add some factor for the necessary cooling. The CBECI website provides some other figures and an explanation of their methodology which may be useful in that regard

Thank you very much for your answer. I did hope for an answer that better allows answering questions about the future where technology would have changed, though - sorry for not making that clear enough. I updated the question accordingly, sorry that the answer doesn't completely fit anymore. Perhaps the methodology https://ccaf.io/cbnsi/cbeci/methodology of the CBECI allows extracting something like that, but it's quite involved. I'll have a look on that later.

That methodology didn't help me, since this is a technical argument all about specific machines and hashrates, while my question needs to involve economical arguments, too. :-( BTW: perhaps you could add that 10GW is about 0.3% of world electricity production assuming 30000TWh / year. I guess thats much more relatable than GW.

Let me give a shot at it as an example, hopefully you can do better or phrase it better.
While it'd be more difficult to estimate the amount of energy because of varying energy prices, we might get an
estimation on the total amount of money spent on electrical energy for bitcoin POW by comparing it to the
mining reward which pays for the mining (as of 2023 6.25 BTC every ten minutes, plus some transaction costs.)
That is an obvious upper bound for the energy cost, but I think we can also
derive a lower bound from that.
I've read that the
largest expense (about 70%) in mining is energy
and that mining is a thin margin business.
Obviously there is mining equipment, maintenance and taxes, so let's say the actual costs are at most 3 times
the costs of the electrical energy, estimating high to play safe.
The margin of the miners would be the difference of the mining reward and those total costs.
Assuming bitcoin prices are stable enough for a while and chip shortage levels out,
new miners would come in until the margin is too low. If that would result in, e.g., a margin of 25%, we'd have:
global electrical energy costs spent on mining > mining reward / 4

And that's
pretty independent of the mining technology.
When I calculate with that, I
currently (10/2023) get a ballpark number
of 0.1% of the world electrical energy production, which is consistent with some estimations
(1,
2.)
Perhaps it'd be better to use the lowest mining reward rate in the last few months in that formula, to cover price spikes.


text:
Is there a standard wallet format?
Is there a standard wallet format that can be read with different bitcoin clients, and that is likely to be compatible to future bitcoin clients as well and thus is usable for long term storage of bitcoins? Or how am I supposed to transfer bitcoins between different bitcoin clients/computers/client versions?

The existing wallet.dat is likely to not be used at some point in the future by Bitcoin-Qt even.

There is the Wallet Import Format, which should be a standard for Bitcoin wallets. As to how popular it is, I don't have any data on it. Alternatively, storing the raw private key should suffice, as it can be converted to any other fancy format whenever needed.

The wallet import format for private keys is by no means a format for wallets. Wallets contain transactions and keys, in its most basic form, and tons of metadata.


text:
How to check the last commited transaction in c-lightning?
Playing with c-lightning on regtest I would like to see the last valid transaction for a channel. The closest related developer command I found to do that was dev-sign-last-tx.
It seems to produce what I want, for example if I have 4 nodes connected like this:
l1 -- l2 -- l3 -- l4

At a certain state of the channel l1-l2 I can do
$ l1-cli dev-sign-last-tx `l2-cli getinfo | jq -r .id` | jq -r .tx | xargs bitcoin-cli -datadir=$PATH_TO_BITCOIN -regtest decoderawtransaction
{
  "txid": "df1e3ce314fe9b79c1a5f5152c09cb7deac8ade5e75ebf20db0da8d66276cf6d",
  "hash": "7fd1536fdb32e6353236fb9e535e6c7baf0e29c8f2f29fc08721f1248c0df929",
  "version": 2,
  "size": 345,
  "vsize": 180,
  "weight": 720,
  "locktime": 544766664,
  "vin": [
    {
      "txid": "67ebfe5b79d7d92e199e80f93b9ca01ac287490749981a6f7649309b645f3eaf",
      "vout": 0,
      "scriptSig": {
        "asm": "",
        "hex": ""
      },
      "txinwitness": [
        "",
        "3044022056f488bd90ea49da9c5c09092f2a864b9529f92aecc596625a551c327a37838c02207f50b1701d5d74de0b1bf4e777274d76d0cffbf131254b02858d6e8bd0eadc2701",
        "304402203d0a0a022c9c228e62a1b724e07e3b2caf76f3771ef967f9d86f0df2dccc1a21022018414477c5b806e322636d52b7cec771d5b0368bf843d87fd44fa6b77874a36401",
        "5221032f30452ef442c5af3c6c37a3d14b8afaad9f991e2adbfb2340b3316949f719112103d0ce37acee074edfaebaa35ff3306c0f56168185bb89e956228357a913e31c5a52ae"
      ],
      "sequence": 2152692867
    }
  ],
  "vout": [
    {
      "value": 0.00001000,
      "n": 0,
      "scriptPubKey": {
        "asm": "0 1a3be66bacb918f02df8b927fabd1569902d1360",
        "desc": "addr(bcrt1qrga7v6avhyv0qt0chynl40g4dxgz6ymq9dyffw)#85alm0m9",
        "hex": "00141a3be66bacb918f02df8b927fabd1569902d1360",
        "address": "bcrt1qrga7v6avhyv0qt0chynl40g4dxgz6ymq9dyffw",
        "type": "witness_v0_keyhash"
      }
    },
    {
      "value": 0.00998816,
      "n": 1,
      "scriptPubKey": {
        "asm": "0 f0ddefa68335b3b84dd93cebcbb298fe88a9c7472f1ce1b2a25ae1106de776a0",
        "desc": "addr(bcrt1q7rw7lf5rxkemsnwe8n4uhv5cl6y2n3689uwwrv4ztts3qm08w6sqjchzpw)#c83sshvg",
        "hex": "0020f0ddefa68335b3b84dd93cebcbb298fe88a9c7472f1ce1b2a25ae1106de776a0",
        "address": "bcrt1q7rw7lf5rxkemsnwe8n4uhv5cl6y2n3689uwwrv4ztts3qm08w6sqjchzpw",
        "type": "witness_v0_scripthash"
      }
    }
  ]
}

and I can see the contents of some state transaction, with two outputs that must correspond to the balance of the channel: some of the sats go to l1 and some to l2.
The commitment transaction on l2's side is different, which is expected. It would be interesting to verify which are the locking scripts for the outputs. I don't see how they can be extracted from the command line interface; maybe using l1-cli sql ... somehow?
However, I am not convinced the command dev-sign-last-tx is giving me the last valid state of the channel. If I try to make a payment through this channel and the payment succeeds I can see with dev-sign-last-tx a new distribution of liquidity in the outputs.
But if I try to make a payment that fails, for instance by disconnecting the recipient and using a single trial with the primitive sendpay rpc instead of using the more complex pay plugin, I know there will have to a pending HTLC in the channel but it's not reflected with the dev-sign-last-tx; if I try that experiment I still see 2 outputs in the transaction, when there should be 3 or even more depending on how many HTLCs are pending.
To me this probably means that dev-sign-last-tx shows the last commitment transaction that didn't have HTLCs. How can I see the actual last valid transaction that gives me the state of the channel?

You are correct dev-sign-last-tx is indeed the command you want, but be careful, it is a dev command because it is used in integration tests to test a cheating party. Having an old fully signer commitment TX is very dangerous.
As you noticed dev-sign-last-tx is a bit racy, i.e., if there is a commmitment inbetween, you will end up with an old on. The other way of producing a commitment TX is to unilaterally close the channel, and in that case the commitment TX is guaranteed to be the latest agreed upon state.
You can likely compare the dev-sign-last-tx with the scratch_txid in listpeers and listpeerchannels to see if there have been changes inbetween.
Now for the lack of a 3rd output: the HTLC is only temporarily added, and then removed refunding the sender on failure and crediting the recipient on succcess. The HTLC output willl only be present for a brief period, and so you'd have to be rather fast to catch it with dev-sign-last-tx. If you want to cheat (in a test scenario) with an HTLC in the commitment you'll likely need to hold on to the incoming HTLC on the recipient side with the htlc_accepted hook, ensuring that you can call dev-sign-last-tx after adding, but before removing it again.
Also notice that for HTLCs that would be considered dust, they will not be materialized, since we'd spend more on creating and cleaning up the UTXO than the value that is actually being transferred.

I can see that the txid obtained from dev-sign-last-tx does not correspond to the scratch_txid I get with listpeerchannels. Why wasn't this command implemented to get me the actual latest transaction? Would it make sense to write a plugin to do so?

So there is a refund mechanism for failed attempts? Then situations like this would no longer happen https://bitcoin.stackexchange.com/questions/86228/is-there-a-way-to-abort-a-stuck-lightning-network-payment

So you're saying that if the amount is below the dust limit there won't be an HTLC output in the commitment transaction that represents the state of the channel? How would two nodes agree on the state of the channel if not through that commitment tx?

Re: refund mechanism preventing stuck payment: refunding requires interaction by the endpoints. So if either endpoint is faulty and doesn't sign the refund it'll still get stuck.

Even for dust HTLCs we have a state transition, and regole the old state (the sender might have dropped by a sat, or the recipient might have accumulated another sat (Dunst doesn't matter individually but it does in aggregate)

As for why we can't guarantee the latest signed commitment is the current state is because we can't stop the channel from proceeding concurrently so this is inherently racy


text:
What cryptographic guarantees apply to non-disjoint paths in basic MPP?
Imagine the following graph in the Lightning Network:
  B   D
 / \ / \
A   C   E
 \ / \ /
  F   G

and node A wants to pay node E.
A chooses to pay with multi-path-payment composed of the routes: A->B->C->D->E
and A->F->C->G->E.
According to the BOLT4 in a basic MPP all parts use the same payment hash and payee releases the preimage once a set of HTLC arrive to destination ensuring that the total amount is fulfilled. The atomicity is then guaranteed by the economic incentive of the payee.
However, I don't see how this mechanism prevents the steal of funds when the different routes of the same payment have a common routing node like in the example above.
The issue lies in the fact that the same preimage releases the funds on all parts HTLCs.
My thought experiment is as follows:

A constructs a 2 route MPP to pay E, these are
A->B->C->D->E and A->F->C->G->E.

E receives two HTLCs from D and G that add up to the payment amount, and then releases the preimage to D and G to settle these.

C receives the preimage from D and settles the HTLC in the channel C--D.

at this point C can show the preimage to B and F as proof of payment forwarding, but he doesn't need to settle the HTLC with G. He will wait until the timelock expires, thus effectively stealing funds from G.

I'm sure there must be something wrong with this reasoning.
I cannot see how the cryptographic guarantees that apply to single route payments solve this situation as well. And my fear is that MPP can only be performed securely only for disjoint routes.
This 3 year old post describes another kind of multi-part-payment called AMP (atomic multi-path), in which each route uses a different payment hash and thus the problem described above doesn't apply, with the drawback that the preimage is known to the sender in advanced and thus cannot be used as a proof of payment.
Coming back to the question:
What prevents C from stealing G's funds in a basic MPP payment?

The wrong assumption is that G needs to collaborate with C to get the refund for routing the payment.
After step 4 in the situation described in the question, G tries to settle the HTLC with C, claiming that he paid to E and showing the preimage as proof of payment. If C doesn't cooperate I guess G can unilaterally close the channel and claim the funds on-chain.
The latest valid transaction G has the following outputs:
+------------+
| funding tx |
+------------+
      |
      |        +-------------+
      \--------| commit tx G |
               +-------------+
                  |  |  |
                  |  |  | C's main output
                  |  |  \------------------ to C
                  |  |
                  |  |
                  |  |                  ,-- to G (& delay)
                  |  | G's main output /
                  |  \----------------<
                  |                    \
                  |                     `-- to C (& revocation key)
                  |
                  |
                  |                                                   ,-- to G (& delay)
                  |                           +-----------------+    /
                  |                        ,--| HTLC-success tx |---<
                  | HTLC received by G    /   +-----------------+    \
                  \----------------------<     (w/ payment preimage)  `-- to C (& revocation key)
                                          \
                                           `-- to C (after timeout)
                                           \
                                            `- to C (& revocation key)

Credits to the original ASCII graph
With the preimage and before the OP_CHECK_LOCK_TIME_VERIFY becomes valid, G can spend the HTLC output and claim the funds after the delay. C cannot do anything to prevent G from doing this.
The HTLC in the latest transaction is actually a promise that C will reimburse G if he can provide the preimage, whether C cooperates or not.

It is important to stress that E must not release the preimage unless all parts for the full amount have arrived. In that case even if C had already claimed their HTLC with B and F(for example because C had learnt it from D) C will not be able to withheld funds from G as described in your answer. Luckily E is incentiviced to release the preimage only after enough incoming HTLCs were locked in


text:
PHP Blockcypher error validating generated transaction: Error running script for input 0
Attempting to use Blockcypher's /txs/send endpoint using detailed here: https://www.blockcypher.com/dev/bitcoin/#creating-transactions and I'm getting the following error:
Error validating generated transaction: Error running script for input 0 referencing 5bf99cd9191700eb9c324a00dee0ac857b804f63c391490d9c21036e4c78ac73 at 0: Script was NOT verified successfully.

Searched far and wide and can't find a reasonable solution.
I generate the transaction successufully and here's a copy of my TX:
"tx": {
        "block_height": -1,
        "block_index": -1,
        "hash": "d64c7acf354b55c69756c4646b100245fe496633b1a5be9a7cc66f4f46cf797e",
        "addresses": [
            "n3nsT38n7k83FL5p7PPvkHgdwJAzA5QBoi",
            "tb1ql7w62elx9ucw4pj5lgw4l028hmuw80sndtntxt"
        ],
        "total": 81172,
        "fees": 29300,
        "size": 223,
        "vsize": 223,
        "preference": "high",
        "relayed_by": "20.185.53.173",
        "received": "2022-04-19T11:58:00.294345726Z",
        "ver": 1,
        "double_spend": false,
        "vin_sz": 2,
        "vout_sz": 2,
        "confirmations": 0,
        "inputs": [
            {
                "prev_hash": "5bf99cd9191700eb9c324a00dee0ac857b804f63c391490d9c21036e4c78ac73",
                "output_index": 0,
                "output_value": 10000,
                "sequence": 4294967295,
                "addresses": [
                    "n3nsT38n7k83FL5p7PPvkHgdwJAzA5QBoi"
                ],
                "script_type": "pay-to-pubkey-hash",
                "age": 2196309
            },
            {
                "prev_hash": "f21f9e457d61ba4fa5858f3a4ba4c37179c12d1275a1bd799f0c0c91d3f15269",
                "output_index": 1,
                "output_value": 100472,
                "sequence": 4294967295,
                "addresses": [
                    "n3nsT38n7k83FL5p7PPvkHgdwJAzA5QBoi"
                ],
                "script_type": "pay-to-pubkey-hash",
                "age": 2196316
            }
        ],
        "outputs": [
            {
                "value": 500,
                "script": "0014ff9da567e62f30ea8654fa1d5fbd47bef8e3be13",
                "addresses": [
                    "tb1ql7w62elx9ucw4pj5lgw4l028hmuw80sndtntxt"
                ],
                "script_type": "pay-to-witness-pubkey-hash"
            },
            {
                "value": 80672,
                "script": "76a914f45560cd98d9a84d6c568bb845db0cef81ed548288ac",
                "addresses": [
                    "n3nsT38n7k83FL5p7PPvkHgdwJAzA5QBoi"
                ],
                "script_type": "pay-to-pubkey-hash"
            }
        ]
    }

tosign:
"tosign": [
        "7a407b770128fd9cd51f95010904f05fecde80202d2fecd2340020d0994ce582",
        "1d03cbac44034426c236268684bedf169beb38210f9857a46086cdade4a9c910"
    ]

signatures:
"signatures": [
        "30450221009f04eccdc41ec135083110b1dcf2ed0edb4c68aefd627820944f9314020faa4e02202337b880a133e22c03df252cf43114158774173229efe2d2d47df6cdc70a610f",
        "3045022100de59d669e38b669b0a0d4451d7a453269da849a202cd325f4923565897d562e002202db8883506cc8f5cd87b6a61cbcfd793dd6107e3a073debf1b4d2b008fcb002e"
    ]

pubkeys:
"pubkeys": [
        "303231393839646331643236333136303662653139396564333261666531636238313362343034353565326462623232303563613962393336356134626232373330",
        "303231393839646331643236333136303662653139396564333261666531636238313362343034353565326462623232303563613962393336356134626232373330"
    ]

Address information (before hex encoding using bin2hex):
"private": "a76b15c0b931b54141f73f127535ae946d45a52872306f7b5eaf3623290a1518",
    "public": "021989dc1d2631606be199ed32afe1cb813b40455e2dbb2205ca9b9365a4bb2730",
    "address": "n3nsT38n7k83FL5p7PPvkHgdwJAzA5QBoi",
    "wif": "cTC98Ng4DP92Mgi34bJADkcDbc5giMMmmcTiSUHQQkEr92MsxYd2"

For context I use the following signing tool provided by them to do the signing:
https://github.com/blockcypher/btcutils/tree/master/signer

It's a go binary which I complie and run by logging in through SSH using phpseclib in the following manner:
protected function create_new_transaction()
{
    $input = [["wallet_name" => $this->wallet_name, "wallet_token" => self::token]];
    $output = [["addresses" => ["$this->destination"], "value" => (int)$this->value]];

    $api_settings = $this->api_settings('/v1/btc/test3/txs/new?token=' . self::token . '&inputs=' . $input . '&outputs=' . $output, 'POST', TRUE, array('token' => self::token, 'inputs' => $input, 'outputs' => $output));
    $result = \GuzzleHttp\json_decode($api_settings, true);

    $to_sign = $result['tosign'];
    $inputs = $result['tx']['inputs'];

    // hex encoded arrays of values
    $public_keys_hex = $private_keys_hex = $addresses_hex = $signatures = array();

    foreach ($inputs as $value) {
        $profile = new Profile();
        $profile->address = $value['addresses'][0];

        $address_lookup = $profile->single_address_lookup();

        if ($address_lookup['status']) {
            array_push($public_keys_hex, bin2hex($address_lookup['address_data'][0]['public_key']));
            array_push($private_keys_hex, bin2hex($address_lookup['address_data'][0]['private_key']));
            array_push($addresses_hex, bin2hex($address_lookup['address_data'][0]['address']));

        } else return [
            'status' => false,
            'message' => 'failed to retrieve address data'
        ];
    }

    foreach ($to_sign as $index => $sign) {
        $signature = $this->sign_payload($sign, $private_keys_hex[$index]);
        array_push($signatures, str_replace("\n", "", $signature));
    }

    $result += ['signatures' => $signatures];
    $result += ['pubkeys' => $public_keys_hex];

    return empty($result) ? [
        'status' => false,
        'message' => "invalid response"
    ] : [
        'status' => true,
        'message' => $result
    ];
}

The actual sign_payload function:
private function sign_payload($to_sign, $private_key)
{
    $ssh = new Net_SSH2('[ip_address]');
    if (!$ssh->login('root', '[password]')) {
        exit('ssh connect failure');
    }

    return $ssh->exec("cd /var/www/vhosts/[domain_name]/httpdocs/btcutils/signer\n./signer " . $to_sign . " " . $private_key);
}

I'm completely lost and I'd appreciate some help regarding this. PS: Tried reaching Blockcypher but they never reply emails, and there's very limited information regarding this online.
On the Blockcypher docs there's this comment:
One of the most common errors we see are users who use uncompressed public keys when compressed public keys were used to generate the address; remember to be careful to use the right keys!

I don't know what that means, and I would like some clarification. Would also like to know if that is causing my issue.
Thanks a lot.

If you haven't had the answer yet, this helped resolved mine
https://github.com/bitcoinjs/bitcoinjs-lib/issues/1707


text:
Revealing chain earlier when private attack
Private attack is to violate the safety of a block B by creating another chain of length equal to or longer than the longest chain containing B, after B has been confirmed.
As in my reference

If the adversary reveals its chain too early, i.e. before block B gets k-deep. It’ll still end up displacing B, but then it’s action do not lead to a safety violation.

Why won’t it pose a threat on safety? Isn’t displacing B in the future already means block mined by the malicious nodes be adopted as the longest chain and block containing malicious transaction might get confirmed once it become k-deep?
Ref:
Private attack section in
https://courses.grainger.illinois.edu/ece598pv/sp2021/lectureslides2021/ECE_598_PV_course_notes6.pdf

The chances of a block being reorganized because of a new chaintip displacing the current one drop exponentially as the number of blocks in the current chaintip increases. Since in theory a chain of any depth could be reorganized, it is impossible to define a "safe" depth, but in practice we commonly see recommendations of up to 6 blocks before a transaction can reasonably assumed to be final. In other words, the safety violation depends on your assumption of how many confirmations are required.
It is my understanding that this is the kind of safety referred to in the document you linked. For example, say you're selling your car for Bitcoin but you agree with the buyer that you'll only give him the car keys as soon as the transaction has 6 confirmations. If the buyer tries to attack you by secretly mining a parallel chain and double spending their payment, the safety of your agreement is not violated if they decide to publish their private alternative chain too early. After all, you've agreed to wait to give them the keys before reaching the 6 confirmations.
Note of course that successfully mining a private chain in parallel would require the attacker to have a majority of hashrate (leaving aside selfish mining), which in the current environment is extremely difficult and costly.
See also Why is 6 the number of confirms that is considered secure?


text:
nunchuk.io - "To create new wallets, additional XPUBs from the key are needed."
I'm trying to start using Nunchuk (nunchuk.io), so I created 3 software keys on 3 different devices for my account, and then was able to create a wallet using those 3 keys as signers.
But when I try to create a second wallet the same way with those same 3 keys, I get the following error: "To create new wallets, additional XPUBs from the key are needed."
I don't know what this means, I tried on the PC and phone.
Is there a limitation for using the same keys on multiple wallets?
EDIT: To give a bit more context, one key was created on an Android smartphone, another key on an Android tablet, and the last one on a Windows 11 laptop. The original error is thrown on the Windows 11 laptop. A similar (?) error occurs when trying to do the same on the Android devices: "Run out of cached xpub!"


text:
Why arent wallets made to generate more than 24 word recovery phrases?
Does the math break if you go above 24? Is there a reason its not done?  Can I code my own wallet to have, say, 27 or 48 words?  Would this add to the entropy and difficulty of hacking or would it just be an inconvenience to the user? Are such wallets incompatible with the blockchain?

BIP39 mnemonics are restricted to encode 128 to 256 bits of entropy corresponding to 12 to 24 words. Since 128 bits of security already appears to be quite enough (see this answer on Information Security StackExchange), a better question might be why it even allows you to go up to 256. However I can't imagine you could need even more than that, because if the cryptography behind mnemonics is so broken in the future that 24 words aren't enough, adding more words probably won't help.
Other than setting reasonable limits, there's nothing in BIP39 preventing you from using any number of words that is a multiple of 3 (including 3 or even 0). You could code your own wallet, or better yet you could modify an existing wallet's source code to remove these limits. However I would advise not to store any actual funds with this mnemonic seed. The standard exists so that a seed can be used with any compatible wallet, and using an incompatible seed defeats that purpose.


text:
What Exactly is Blockchain?
I am trying to learn blockchain technology but I could not find clear answers. I will appreciate your help. ^^

Is blockchain a specific algorithm or software? As I know, blockchain is the transparency of digital transactions, so when you make a transfer of what you own, or store something on the blockchain network, its details will be public for everybody and it will be stored on all nodes permanently in the related network. We all heard about "blockchain is a copybook, when you make anything on it, it will be stored in all nodes". So my question is: If I develop a basic software that stores the same record on all nodes in a network (a kind of common database), would it be as called a "blockchain network", or have I to use some libraries or specific algorithms to call it "blockchain"? Is there any rule or standard to make any software a "blockchain"?
Are Bitcoin blockchain and Ethereum blockchain, not the same things? I mean, are they two separate and unique networks/software, or is the "blockchain" a common name for all of those?
When I search my wallet address on blockchain.com or blockchair.com to track my account balance and previous transactions, it never shows me the correct results. So, the amounts of Bitcoin transactions are really trackable?

I am sure everybody wonder about those questions and let us make them more clear as of now.

Does this answer your question? What is the "blockchain"?

If I develop a basic software that stores the same record on all nodes in a network (a kind of common database), would it be as called a "blockchain network"

I would call that a replicated database.
In addition to the public nature and replication, a blockchain must be a sequence of blocks with cryptographic hashes that link each block to a predecessor. It is this last characteristic, and the use of cryptographic signatures, that provides the important immutability of the record of transactions.

Are Bitcoin blockchain and Ethereum blockchain, not the same things?

They are different. I don't know anything about ethereum but I expect it is a completely independent implementation of the blockchain idea that has no common code with Bitcoin's implementation.

So, the amounts of Bitcoin transactions are really trackable?

All the essential data is completely public. However it does not explicitly tie addresses to wallets or to people. Careless re-use of addresses can allow some inferences to be made about sets of addresses whose use suggests they may be under the control of a single entity and sometimes that can be used to  link addresses to known organisations or individuals. Used with care, Bitcoin has a moderate degree of anonymity. So a business like blockchain.com doesn't know what addresses are in your wallet and can't provide the total for all your addresses that your wallet can.

Oh, I just learned that a wallet can contain multiple addresses. So, which one should we enter on blockchain.com to track our all real transactions: our wallet address or Bitcoin address?

@daybreak: Any wallet nowadays should be an HD wallet which will automatically generate a new address for every transaction. This improves privacy. I think it is a mistake to think of any address as a "wallet address" nowadays.

I would say that: "blockchain" is a concept.  Your example is not a blockchain, at least not yet, you need more than one record (datum) to make it a blockchain. I would define blockchain is data(plural of datum) linked together in such a way that its validity depends on the validity of the datum preceding it.  If you have three pieces of data connected together in a blockchain and the block in the middle is invalid then the block after it is also invalid.

Both software employ the blockchain concept with different rules.

I'd need more info here, maybe your transactions were not yet propagated and/or put in the blockchain.  It doesn't happen instantly

Awesome comment, thank you for clarifying. Are there any standards or rules to create a blockchain network? Okay I can develop a software which includes bulk of data blocks those connected each other and needs cryptographic verification for previous blocks, then would it finally as called a "blockchain network" ? Or, does it have to comprise some specific rules or standards? If so, may you share those standards with me please? +++ Umm, actually none of my wallet transactions seems on blockchain.com or blockchair.com, it just displays fake results. I made those transactions 1-2 years ago, ie.

Blockchain is a data structure employing an ordered sequence of pieces of data known as blocks. In blockchain, every block contains a reference to the previous block, using a cryptographically strong one way hash function. This, combined with proof of work, is what makes data stored inside nearly-immutable.
Bitcoin blockchain and Ethereum blockchain are two specific instances of the blockchain data structure.
blockchain.com or blockchair.com are chain explorers, they only show all the transactions performed by the actual holders of the currency.

Aha! So, normally anyone (or a contract deployer) can update any record on the blockchain, but it needs to be revalidated by 50%+1 of all nodes, so, it is called as "immutable" - but technically it is possible to edit data if we can provide the majority, right?

You can only go going back and slowly start rewriting history starting at certain point as long as you hold 51% computing power.


text:
What feerate estimation APIs are there?
Some company's, block explorers and payment processors offer a public API for feerate estimations. Is there a list of publicly available feerate estimation APIs?

You seem to have asked this question so that you could promote your own blog post/content.

I've created this question to share my knowledge about this topic with the community. This is encouraged by the StackExchange platform. While I disclose that the answer is based on other content by me, there there is no need to open / read it to find the answer to the question.

I've listed some public feerate APIs by block explorers and payment processors below. Purposefully leaving out APIs from wallets, since they don't advertise their APIs. I leave out Bitcoin Core's estimatesmartfee RPC, which I don't consider public (as in reachable over the web by everyone).
The list is sorted lexicographically. Disclosure: This Q&A is based on a blog post by me.

bitcoiner.live API
URL: https://bitcoiner.live/api/fees/estimates/latest
Note: Estimates for confirmation in 1/2, 1, 2, 3, 6, 12 and 24 hours.
Unit: sat/vByte

Bitgo API
URL: https://www.bitgo.com/api/v2/btc/tx/fee
Note: Estimates for different block targets based on current mempool.
Unit: sat/kB

Bitpay Insight API
URL: https://insight.bitpay.com/api/utils/estimatefee?nbBlocks=2,4,6
Note: Estimates for different block targets based on nbBlocks parameter.
Unit: BTC/kB

Blockchain.info API
URL: https://api.blockchain.info/mempool/fees
Note: Estimates for a regular and priority feerate are provided.
Unit: sat/Byte

Blockchair API
URL: https://api.blockchair.com/bitcoin/stats
Note: A single estimate for a feerate is provided.
Unit: sat/Byte

BlockCypher API
URL: https://api.blockcypher.com/v1/btc/main
Note: Estimates for a high, medium and low feerate are provided.
Unit: sat/kB

Blockstream.info API
URL: https://blockstream.info/api/fee-estimates
Note: Estimates for confirmation in 2, 3, 4, 6, 10, 20, 144, 504 and 1008 blocks.
Unit: sat/vByte

BTC.com API
URL: https://btc.com/service/fees/distribution
Note: A single estimate for a 'one block' feerate is provided.
Unit: sat/vByte

earn.com API
URL: https://bitcoinfees.earn.com/api/v1/fees/recommended
Note: Estimates for the fastest confirmation, a confirmation in half an hour and a hour are provided.
Unit: sat/vByte


text:
How do BIP numbers get assigned?
Each BIP (Bitcoin Improvement Proposal) has a assigned number. For example BIP-2, BIP-42 and BIP-152. How does this number get assigned and by whom?

The current BIP process is defined in BIP-2. An author drafting a new BIP leaves the BIP number in the preamble marked as XXXX. A BIP editor  assigns a number to that BIP when the draft phase is complete. The number is chosen by the BIP editor, while generally one of the next higher numbers is assigned. Some number ranges might be reserved to allow BIPs with a common theme to have similar numbers. The current BIP editor is Luke Dashjr.


text:
Are QR codes on paper wallets necessary
I don't have a wireless printer so I'm thinking about writing down the keys by hand on pieces of paper. Obviously I can't copy the QR codes that way. Are they necessary or do they just exist just for our convenience?

It's just for convenience.
Make sure to d̶o̶u̶b̶l̶e̶ triple check and use a pen that's readable in ten years, too.


text:
Transaction vaildation
I have understood that miners validate the transactions by checking for double spends and that the transaction was not already confirmed, what I don't understand is how the miners are able to do this when all the data they have is the transactions.

Do miners maintain a UTXO set that has the UTXO details of all
nodes in the network? If that is the case it might be possible that
the UTXO sets maintained ny different miners is different at a given
point of time right.
Now once the block is broadcast to the    network, we know that
other miners express their acceptance to the    block by mining the
next block on top of this. But before that they    validate the
block that was just published. How do they validate this    block?
Also why will they have to care to do it when they can save    time
by creating the next block? Is it that we are assuming honesty
from miners who are actually interested in making money?

First of, each node on the network maintains an UTXO-set. Not only miners.
to 1. The UTXO-set is created by applying all transactions in each block in the chain from the beginning. All transactions that remain unspent when reaching the chain tip are in the UTXO-set. The older a block, the higher the probability that the whole network has a equal block. Equal blocks result in an equal UTXO-set.
to 2. Each node validates blocks, not only miners. A block is deemed valid, if it follows the consensus rules. One consensus rule is, that each spend transaction (TXO) must be in the current (pre-acceptance) UTXO-set or a TXO created early in the same block. Block validation does not interfere with the actual mining (only for a short time, when the block is accepted and a new blocktemplate has to be pushed to the ASIC). Validation happens on the host CPU and the actual mining part happens in the ASIC. If a miner doesn't validate an invalid block, he wastes his time/electricity/money with mining an invalid block on top of the previous invalid block.


text:
Does Bitcoin Gold automatically appear in my ledger nano s wallet?
How do I get Bitcoin Gold into the Ledger Nano S Bitcoin Gold (Segwit) wallet?

Have you downloaded the Bitcoin Gold Chrome app?

Yes,The Bitcoin Gold wallet is now in my Ledger Nano S. I downloaded the Ledger wallet chrome app, I went to 'Settings-Tools-Reset the application, and then went to the Blockchain (Segwit) and still no Bitcoin Gold in my wallet. Dont know what else to do. I thought the BTG would arrive in the wallet automatically.


text:
how to configure bitcoin-core (bitcoin-qt) to not use change addresses?
Can bitcoin-core wallet be configured to not use change addresses, but just one address?
If yes, how? Thanks
(PS: I know most people will try to convince me that this is not a good thing, but I have perfect valid reasons. I know the privacy implications of this, ok? :)

Address reuse is not just bad for you, but for everyone. Don't do it. And if you do anyway, smart potential users of your product will simply not use it for precisely that reason.

Could you elaborate on your reasons?  I feel that your reasons probably aren't as good as you think they are, and better solutions are likely available to you.

2 reasons: a) with confidential transactions coming to bitcoin-core soon, need for privacy via change addresses is not so important, b) transactions without change addresses are smaller, thus cheaper

Address reuse privacy issues will still be important after Confidential Transactions arrives

@cowboy4life: I didn't say it's not important, I said it would be not so important

Transactions without change are smaller. But a transaction with change but no separate change address is not smaller. Also, confidential transactions are unlikely to come to Bitcoin any time soon.

It's unclear what you want.
Do you want no change addresses, but still have change that goes back to one of the addresses from which coins were taken? That does not have any size/fee benefit. This is possible by using the coin control feature (in the Bitcoin-Core GUI) or the createrawtransaction RPC interface.
Do you want no change at all? That means the amount you are sending must exactly match the sum of the sizes of some of the unspent outputs you have in your wallet. It does result is slightly smaller transactions and thus lower fees, but it is highly constraining. You'll have to construct transactions manually if you need this.

Of course I don't refer to no change at all, that would be dumb. In regards to not having fee/size benefit, you're right, I just realized that if I want to send n bitcoins and all of my previous outpoints are lower than n, then I need many inputs regardless if I used change addresses or not. Thanks for making me think through it. (And thanks for telling me about coin-control and createRawTransaction, but I was looking for a bitcoin-core setting that would make the behaviour of createTransaction be different without recurring to different API or to UI.)

Such a setting doesn't exist.

You can just send all the money leaving no change. This way there would be no money to send to a change address. Then if you need to take part of the money back, you can send the transaction with two outputs: 1. the address you are paying to; and 2. your original address.
By the way there is by design no way to just leave coins on an address. A transaction could be either spent or unspent with no partial spend. But you can send (in a new transaction) coins back to previously used address. Nevertheless it's often bad idea.
Edit: You may also want to enable Coin Control Features, which result in UI extended like below:

(source: cryptocoinsnews.com)
This way you can just specify the old address as a change address.

you're just saying how one would do it sending transactions manually, but I asked how to do with BitcoinCore

how to enable Control Coin features?


text:
What's the POST endpoint to broadcast a transaction in copay's server?
AFAIU, copay's server implementation is bitcore-wallet-service.
From its README, I can deduce that a multi-sig wallet will use the /v1/txproposals/ endpoint to create a new transaction proposal that the other co-signers will receive.
But, if the wallet is not multi-sig, what's the endpoint for broadcasting a transaction?

Multisig and singlesig wallets use the same flow. A Tx proposal need to be created in singlesig wallets also. Copay syncronize across devices, so you can have a non-signing copayers in a 1-1 wallet that submit proposals, for later approval.
The flow is:
create => publish => sign => broadcast.
You can check the official client https://github.com/bitpay/bitcore-wallet-client for details, and also the CLI client should be useful to see examples:
https://github.com/bitpay/bitcore-wallet

I think I should leave a comment with a small correction to the flow ematiu posted after 1 day trying to figure out why I couldn't sign my transaction.
the actual flow is:
create => publish => get published => sign => broadcast.
this is because, in order to sign a transaction proposal we need a proposalSignature in the txProposal object. We only get this signature by fetching from the bws
Bitcore-wallet-client "Server response could not be verified"


text:
Estimated profit of a $400 21's RaspberryPi-ASIC computer?
So I'm thinking of buying one of the newly-announced $400 USD 21's RaspberryPi-ASIC computer.
How can I know the hashrate of its ASIC? How could I calculate the BTC profits I would get from it, per day?
PS: The purpose of buying it is not just for the mining profit, although I'm still interested to know on what scale of magnitude these profits are.

We know only some of the variables so it's hard to make a blanket statement.
The chip mines at 50GH/s in the configuration given to consumers, while consuming 8W, for the whole device the consumption is likely in the order of 15W at the power point accounting for the consumption of the Raspberry Pi and conversion losses.
Power cost varies worldwide, but here's a general idea, all amounts are in USD.
╔═══════════╦═════════╦════════════════╦═══════════════╗
║  Country  ║  $/kwh  ║ Cost per Month ║ Cost per Year ║
╠═══════════╬═════════╬════════════════╬═══════════════╣
║  Iceland  ║  5.5c   ║     $0.55      ║     $6.57     ║
╠═══════════╬═════════╬════════════════╬═══════════════╣
║ Indonesia ║   11c   ║     $1.22      ║    $14.40     ║
╠═══════════╬═════════╬════════════════╬═══════════════╣
║  Norway   ║   17c   ║     $1.89      ║    $22.33     ║
╠═══════════╬═════════╬════════════════╬═══════════════╣
║  Spain    ║   25c   ║     $2.79      ║    $32.86     ║
╠═══════════╬═════════╬════════════════╬═══════════════╣
║  Denmark  ║   33c   ║     $3.68      ║    $43.36     ║
╠═══════════╬═════════╬════════════════╬═══════════════╣
║  Vanuatu  ║   60c   ║     $6.67      ║    $78.84     ║
╚═══════════╩═════════╩════════════════╩═══════════════╝

The amount of gross income from such a device is essentially unknown, the Bitcoin network difficulty is constantly changing, and so is the value in USD of the Bitcoin being mined.

Your income from a given hashrate is inversely proportional to the wider bitcoin networks hashrate, as it goes up your income goes down. The difficulty is adjusted every two weeks, and is growing single or double digit percentages per period at the moment. Planning ahead how much a miner will make even in one months time is essentially futile as a result of this unpredictability.
Without counting the power costs, for the current difficulty of 72722780642, a 50GH ASIC will produce 0.01037 BTC per month.
If the difficulty and price has not changed, they could sell this for $3.28. If they live in Spain, Norway, Indonesia or Iceland, they made a tiny profit over their power bill which can go to paying for the $400 cost of the device, in Denmark or Vanuatu they made a loss.

I didn't want to take in account power costs, sorry, so I should have explicitly for "profit", as opposed to "net profit"

Then your answer is at the bottom.

yes thanks, I'll mark it as bold text if you don't mind

done, happy with the change?


text:
Will Confidential Transactions allow us to reuse addresses safely?
It is often said that, due to the natural transparency of the blockchain, one should have a multi-address wallet and try to not reuse addresses, so that the user's transactions are not so easily revealed.
Now, it seems at some point we will have the Confidential Transactions feature implemented in Core. So, when this moment arrives, will we be able to reuse addresses? Will it be safe to use a mono-address wallet?

My understanding of confidential transactions is that it only hides the amounts, not the addresses involved (someone please let me know if this is not the case).
Since only the amount is hidden, reusing the same address would still connect your various transactions together, linking them as likely involving the same user.

you mean because they would know who are you transacting with? or because you can always know the total balance of an address regardless of CT usage?

You can't know the balance of an address if CT is being used, but you can still know if it's being used or not.

ok, that's good enough for me, thanks for the answer

No problem. Don't forget that ole green check mark if this answered your Q ;)

I wouldn't advice that. That still leaves a huge amount of meta data visible to everyone.
Also, why would you. HD wallets are very well supported these days.

what does HD stand for?

Hierarchical deterministic wallet.


text:
Where can I find code that allows me to calculate Z1 and Z2 from an ECDSA signature?
By following Recovering Bitcoin private keys using weak signatures from the blockchain, I am able to do other calculations, but I have no idea how to calculate Z1 or Z2. There was a public code available by Sean Bradley:
http://2coin.org/tx.asmx?op=GetTXExtended
But it does not seems to work now. Is there some other public code to calculate Z1 and Z2 in any language. I can convert it to the language of my preference. Any help is appreciated.

Could you maybe describe what you want this for?  I don't want to unintentionally help someone steal other people's bitcoins.

@DavidA.Harding I am a security researcher and want to replicate this attack. All the vulnerable addresses are already on internet and empty. And vulnerable applications are already fixed. I just want to replicate this nothing else.

I provided an answer.  However, the attack Nils's describes is just a basic re-used k value attack.  If you really want to replicate it, you'll find it much simpler jettisoning Bitcoin's baggage and doing the math for your own arbitrary hashed data.  Even Wikipedia has instructions for doing this.

According to Nils's post, the z values are the hashes that get signed in the ECDSA formula.  Background on what data goes into these hashes can be found on the OP_CHECKSIG wiki page and Krzysztof Okupski's excellent developer reference PDF.
Bitcoin Core's code for generating the hashes is here.  Any other Bitcoin application that signs transactions (or verifies signatures) must have a compatible function (at least for the default SIGHASH_ALL), so you should be able to find an implementation in pretty much any popular programming language.

paste your transaction into this page
https://2xoin.com/getRSZfromRawTX/
it will give you all the R,S and Z if it was able to decode the TX.
for example,
0100000002f64c603e2f9f4daf70c2f4252b2dcdb07cc0192b7238bc9c3dacbae555baf701010000008a4730440220d47ce4c025c35ec440bc81d99834a624875161a26bf56ef7fdc0f5d52f843ad1022044e1ff2dfd8102cf7a47c21d5c9fd5701610d04953c6836596b4fe9dd2f53e3e014104dbd0c61532279cf72981c3584fc32216e0127699635c2789f549e0730c059b81ae133016a69c21e23f1859a95f06d52b7bf149a8f2fe4e8535c8a829b449c5ffffffffff29f841db2ba0cafa3a2a893cd1d8c3e962e8678fc61ebe89f415a46bc8d9854a010000008a4730440220d47ce4c025c35ec440bc81d99834a624875161a26bf56ef7fdc0f5d52f843ad102209a5f1c75e461d7ceb1cf3cab9013eb2dc85b6d0da8c3c6e27e3a5a5b3faa5bab014104dbd0c61532279cf72981c3584fc32216e0127699635c2789f549e0730c059b81ae133016a69c21e23f1859a95f06d52b7bf149a8f2fe4e8535c8a829b449c5ffffffffff01a0860100000000001976a91470792fb74a5df745bac07df6fe020f871cbb293b88ac00000000
will output
[
  {
    "sigR": "d47ce4c025c35ec440bc81d99834a624875161a26bf56ef7fdc0f5d52f843ad1",
    "sigS": "44e1ff2dfd8102cf7a47c21d5c9fd5701610d04953c6836596b4fe9dd2f53e3e",
    "sigZ": "c0e2d0a89a348de88fda08211c70d1d7e52ccef2eb9459911bf977d587784c6e",
    "pubKey": "04dbd0c61532279cf72981c3584fc32216e0127699635c2789f549e0730c059b81ae133016a69c21e23f1859a95f06d52b7bf149a8f2fe4e8535c8a829b449c5ff",
    "N": 0
  },
  {
    "sigR": "d47ce4c025c35ec440bc81d99834a624875161a26bf56ef7fdc0f5d52f843ad1",
    "sigS": "9a5f1c75e461d7ceb1cf3cab9013eb2dc85b6d0da8c3c6e27e3a5a5b3faa5bab",
    "sigZ": "17b0f41c8c337ac1e18c98759e83a8cccbc368dd9d89e5f03cb633c265fd0ddc",
    "pubKey": "04dbd0c61532279cf72981c3584fc32216e0127699635c2789f549e0730c059b81ae133016a69c21e23f1859a95f06d52b7bf149a8f2fe4e8535c8a829b449c5ff",
    "N": 1
  }
]
`


text:
can i extract internal key given tweaked-pubkey and bip341 tweak tag?
Tweaking an "internal" public_key into a "tweaked" public key, using bip341 for key spend with taproot, seems to make it impossible to extract the internal key, even when knowing the tweak.
Application is trying to identify a Nostr id from a taproot address. Even when the taproot is spent, this is still not possible, as the tweaked pubkey can't be mapped to an untweaked (nostr) pubkey.
This can only be done by getting a list of NostrIds and generating the TapTweak bip341 addresses and doing a lookup.
Am I missing something?

If you know the tweak, you can compute its additive inverse (the negation) and add the resulting point to the tweaked pubkey. The result should be the untweaked key.
The additive inverse is just n - tweak.

what is n? 0?
also are there any helper functions for this in Secp256k1? ty

nevermind - i don't know the tweak - the tweak contains a hash of the pubkey - I know the TagHash -
tweak is sha256(sha256("TapTweak") | sha256("TapTweak") | P)
so, i guess i don't know the tweak

n is the order of the secp256k1 curve. If you don't know the tweak, you cannot retrieve the internal pubkey.