text:
Other database besides UTXO?
In this post it is explained that the UTXO are removed after they are spent. This makes sense because they are not "unspent" anymore.
But don't all UTXOs have a reference/connection to their previous transaction? Is there a second "list" in the blockchain where all transactions from the past are stored? Because if not, where does that reference point to if the previous transaction is not in the UTXO list anymore.
The whole UTXO Model confuses me a little bit, so thank you for the clarification.

Once spent, the wallet removes the UTXO from its cache of unspent transactions. The transaction is not removed from the general database (the blockchain or block index). Full nodes running with txindex=1 keep every transaction forever. Some wallets though optimize the database to recover disk space - but this is a wallet-specific optimization and not part of how Bitcoin works.
For instance, Bitcoin Core has many caches and optimizations. One of them is the UTXO cache. The reason to have this cache is kind of obvious: if you had to search the entire blockchain for your own unspent coins before every transaction, that could potentially take hours. So Bitcoin Core keeps track of your UTXO's. Your UTXO's output amounts add up and become the balance you see on the wallet software.
The UTXO cache never points to previous transactions - it's just an optimization cache that makes it easy for Bitcoin Core to find TX's. The algorithm which builds the UTXO cache knows which transaction the UTXO came from, but the cache never points anywhere except to the TX index where the unspent output is located.
The pointer to a previous transaction only exists when you spend a UTXO and then a new input TX is created on that transaction. That input will point to the previously unspent transaction and that former UTXO will become spent. A new UTXO is then created after this transaction is mined. This UTXO becomes part of the Bitcoin balance of the recipient, just like the UTXO was part of your balance before you spent it.
This image (courtesy of bitcoin.org) might help illustrate how it works:

See those UTXO's dangling without a new TX ahead of them? Until they're spent, they remain as UTXO.
The arrows in that illustration show the money flow direction, but technically the arrows point the other way around : the inputs point back at some previous UTXO output. Once an input points at an output that output is no longer a UTXO, it is a committed TX. The input links the previous TX to a new UTXO and the previous one is marked spent. Bitcoin Core then removes this previous UTXO from the cache.

The chainstate (Bitcoin Core's database with unspent transaction outputs) is not just a cache though. It is the authoritative set of all unspent outputs, and spending a missing entries from it imply a transaction is invalid. It's also not an index, as you can delete the actual block data for old blocks after applying their effects on the UTXO set. Lastly, there is a cache on top of the chainstate in memory, with lots of optimizations, but those are independent of the chainstate itself.

I want add a clarification inside your good post by Jose Fonseca, the UTXO is a TransactionOutput, this structure has only 2 value, the amount and the scriptPubKey. The link to the previous transaction is inside the TransactionInput so, the TransactionInput and the transaction output is contained inside the Raw transaction.
So inside the cache is present only the UTXO (TransactionOutput).
I have created this schema for my document, I hope it can help you.

This image tries to simplify how bitcoin transactions are generated.
The terms that are used are generalized, for example, you do not enter the public key in the output transaction but the address or hash160 of public key, this depend to the type of script is used.
In the image is not represented but there is a data type called Outpoint that refers to the previous raw transaction.
Still, for my project I rebuilt the data structures used in Bitcoin core (Except for the scriptWitness) if you haven't looked at the code, they can give you a general idea. you can find it here
ps: I'm developing this documentation, it can contain some error, please verify it.


text:
How can I tell who mined a certain block?
Just like title says, how can I tell who mined a certain block.
Is it just the decoded hex value of the OP_RETURN in the first tx?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
What happened to this Bitcoin transaction? Block fee lost?
I'm writing a bitcoin blockchain indexer. I've indexed every block hash and now I'm working on indexing each transaction in each block.
I've been finding some weird stuff here and there and I don't understand this one (there are a few others just like this):
What happened to the Bitcoin? It appears that the block fee was "lost"?
txid: ee40c94cc60cb3b0b59b86c2dec1c1187c4cc357c86b173fd0e8b8d4f925e5a0

What do you mean lost

That's what I am asking lol. Look at the screenshot.
It shows and unknown transaction worth nothing.
What happened to the fee?

I think you are confused by how transactions work. On the right it's tx outputs. On the left transaction input (confusingly displaying an address, which isn't present in a tx input). The fee is the left over of the sum of tx outputs' amounts minus the sum of tx inputs' amounts.

The first output is an OP_RETURN output whose content is:
Raw Hex:
5035e332014086d1 6c5a07b4d277b5f4
95c85fd7c20167cd 31029dac6f9293fc
0b7988032d44b0d3 159bb0f423de
Which Blockchair.com displays as  "Decoded:" P52@lZw_g1oy-D#
(the other output is change)

What happened to the fee?

That is the difference between the input and the change output. The OP_RETURN output doesn't affect this because it has a monetary value of zero.

Ok I see, so some code changes need to be made to filter this out.
Thanks!

Just one follow-up, if I wanted to decode this, is it UTF-8?

OP_RETURN can be followed by up to 80 bytes of data in any encoding chosen by the transaction creator. In this case it isn't valid UTF-8 - I don't know why blockchair.com decoded it the way it did. I guess Blockchair makes an assumption. See "Raw Tx" on that page for exact data values.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How does bitcoin core API work locally (network) - bitcoinlib in python using too many web sockets
I'm trying to understand at a high level, the local network aspects of calling the bitcoin core api via bitcoinlib in python
So the context around this question is that I wanted to work on some python (for which I have very minimal experience or understanding)
so I sat down to attempt to write a blockchain indexer. I run a full node and figured I would see what I could do with the api.
Bitcoinlib offers a class called AuthServiceProxy. You pass the local api url & port to this, it looks like this:
rpc_connection = AuthServiceProxy("http://user:pass@127.0.0.1:8332")
I dug a little bit into this class to find that it's using httlib.HTTPConnection to open the connection itself.
        if connection:
            # Callables re-use the connection of the original proxy
            self.__conn = connection
        elif self.__url.scheme == 'https':
            self.__conn = httplib.HTTPSConnection(self.__url.hostname, port,
                                                  timeout=timeout)
        else:
            self.__conn = httplib.HTTPConnection(self.__url.hostname, port,
                                                 timeout=timeout)

The first thing I attempted to do was to write a loop that would iterate over all the blocks stored locally to create an array out of the hashes of each block.
        while block_counter < total_blocks:
            rpc_connection = rpcConnector.rpcConnect()
            block_hash = rpc_connection.getblockhash(total_blocks)

this is essentially how it is getting called. The rpcConnector is just that AuthServiceProxy call essentially.
So I ran this and printed the output of each block hash just to see how it was working. It worked up until about the 16200th iteration. Then it fails with the error message
"[WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted". It fails at this mark each time I run it.
I thought that maybe the connections weren't being closed out correctly so I tried a bunch of different things to try and close the connection within each loop iteration
Nothing worked.
The last thing I tried was this: rpc_connection._AuthServiceProxy__conn.close(). I figured since the AuthServiceProxy was just using HTTPConnection I could just call the .close()
I stepped through the code to see that it was in fact getting into the .close() method but it wasn't ever hitting close.
    def close(self):
        """Close the connection to the HTTP server."""
        self.__state = _CS_IDLE
        try:
            sock = self.sock
            if sock:
                self.sock = None
                sock.close()   # close it manually... there may be other refs
        finally:
            response = self.__response
            if response:
                self.__response = None
                response.close()

both the Sock and Response were None.
So maybe this is a networking question but it seems like when I call the API it's not keeping a connection open, so if that is the case then my question is:
how am I using up the addresses, and why is it always stopping around 16200?
and I guess to close it out, how can I resolve this issue?

Adding a comment here, I'm closer to narrowing down the problem itself.
Navigate to HKEY_LOCAL_MACHINE\SYSTEM \CurrentControlSet\Services\ TCPIP\Parameters .
Create a new key for REG_DWORD, and specify the name TcpTimedWaitDelay.
Type the minimum decimal value 30 (which is 0x0000001e in hexadecimal).
This got me to about 20k iterations.

I think the ultimate issue is that too many ports are in TIME_WAIT status. The default value is 240 seconds after closure, I changed it to 30 seconds and it seemed to make it a bit better.
I don't know how much more I can tune this before something breaks.
Any ideas on a potential code solution?

This seems to be an issue with how Windows deals with ports. According to https://help.socketlabs.com/docs/how-to-fix-error-only-one-usage-of-each-socket-address-protocolnetwork-addressport-is-normally-permitted, when a port is released, Windows does not make it immediately available for reuse. Instead it waits for TIME_WAIT which has a default of 240 seconds.
Bitcoin Core's RPC is a one call per connection type of service, so once you connect to it and receive the response, you should be disconnecting. As each outbound connection to the server consumes one port, when the call is done and that port is released, Windows will continue to block use of that port for TIME_WAIT. As you are making many calls very quickly one after the other, you run into a situation where all available ports are in TIME_WAIT and thus you get the error.
There are a few solutions. As mentioned in the linked article, the default number of available ports is just under 4000 (the lower 1024 are reserved, this is standard across all operating systems). However ports are 16-bit ints, so you can actually have up to 65536 ports, so you could increase Windows port range to the actual maximum. You could also try decreasing TIME_WAIT, but that could have negative side effects.
However, you could also do batched RPC requests. This is where a single connection to the RPC server contains multiple RPC requests, and those are all processed sequentially and their results returned in the same response. The results will be as if you called the RPC once per request, but it will reduce the number of actual connections, and thus ports, used by the OS to send the requests. AuthServiceProxy has a batch_() function that you can use for this. You pass it array of tuples each containing the RPC name and the positional parameters for it, and it will construct a single batched request and return an array of results.

I assume batch will still run into the same issue unless it's making multiple requests on the same port?
I'll do some experimenting today, I'll try to just catch the error message and wait 30 seconds and see how that goes

Batching makes multiple requests within one connection. All of those requests are sent at the same time from the same port, and you get all of their responses at the same time.


text:
Did blockchain.com stold my Bitcoins?
Some 5 years ago I stored some Bitcoins on Blockchain.info, I remembered a few days that I still had a tiny amount left which would be worth something today, around 0.20 BTC in total which makes around $9,500+ with the currents Bitcoin value.
I decided to check my wallet which I did not log in for years. For my surprise the wallet address did not exist, and I was redirected to blockchain.com instead of .info
The login had an input to use the wallet ID, which I did. It then emailed me for authentication which I confirmed, it then asked for my second factor authentication which I entered but for my surprise it then asked me for a second password (besides my normal one). I had a second password stored, but that was only used while spending bitcoins, it was a spending password. I entered that, and it said incorrect, then again, and again and again. I tried multiple times with errors, this was already strange since they never asked this before on login, finally for some reason the page was blank and in a surprise, I was finally logged into my account.
But bigger was my surprise when by BTC balance was 0. Instead, I had around $147 something in the account and my balance is now showing as Bitcoin Cash. Bitcoin Cash did not even exist back in 2017, I never received any email from them about any conversion ever, or I would have moved by balance out. In fact, I don't remember receiving any email from them in years about anything.
I contacted their support, 2 days later no reply. No explanation.
Did they just steal my bitcoins, or I'm missing something here? I found another person here on Stack Exchange that asked the same thing. Is blockchain.com a legit company or are they fraudsters stealing people's Bitcoins?

When did your BTC balance become zero? You didn't have to do anything to get bitcoin cash, it happened automatically because bitcoin cash began with the same state the bitcoin blockchain had.

Since we have no insight into the internal processes of blockchain.com nor the make-up of your system, we cannot determine how your wallet was compromised. As we cannot objectively answer your question as it's asked, I'm voting to close.

you should check in history/activity when your btc was sold.
Btc forked to bch and bchscv so your btc became all 3.
if you had 0.2btc then you also have 0.2 bch (at current price that is about the amount you mention)
If you dont recall selling the BTC it is also possible to see what IP address was accessing your account.

The account was restricted with everything, including 2FA. The balance had 0 BTC but 0.20+ BCH, BCH had no transactions. Bitcoin had the transactions I made years back, nothing after that time.


text:
Scammed by Bitcoin Recovery Expert
We were initially scammed for 1000.00 through blockchain. So then we did some research online about fraud recovery experts. We came across a site called stallionedge.tech and their clients had rave reviews about how long it took to get their money in a matter of days, so we were able to get in touch with them. We called the phone number listed on their website and it was disconnected. They said that they can only contact us, that is why it is disconnected when somebody tries to call them. So we thought okay. They communicated with us back and forth via email. So when it came time to pay them to help, I was told to download whatsapp and that a withdrawal specialist would contact me. The withdrawal specialist, Christopher, contacted me. We initially had to pay 250.00 for services rendered. Well, then he said that we needed to deposit 460.00 to show activity on our blockchain account, so we reluctantly did. It disappeared in a matter of minutes. They said that there was a glitch and that they would help figure it out. That we were his top priority and that they deal with clients like us on a daily basis. Well, a couple of days later, we were asked to deposit 460.00 more and they would help us get the previous 460.00 back along with over 2000.00 that was taken from our blockchain account. It supposedly accrued up to over 2000.00. We received a screenshot of the transaction that they claimed was waiting to be sent to us. It looked legitimate. Well, it wound up disappearing in a matter of minutes, too. The thing is, is that he is still communicating with us via the whatsapp. I have all the texts, emails and screenshots as proof if you need those. Please help. I don't know if there is a way to recover the money that we paid, but their website needs shut down and the people need punished that have taken advantage of us. I am assuming this is a scam, but if it is a legitimate company, they are taking advantage of me. He says that you can contact him. We have his contact info if you need it.

Does this answer your question? How do I recover from a cryptocurrency scam?. Also see how to look for signs a business is not honest in https://bitcoin.stackexchange.com/q/98743/13866. The domain Stallionedge.tech was only registered a few weeks ago - it looks like a throwaway domain and website, soon they'll probably close it and open some new ones with similar information and fake reviews.

What do you suggest we do? Is there any way we can possibly get our money back?

Report the fraud to your local police. 2) No, the money is gone and you will never get it back. I suggest learning what you can from the experience, putting it all behind you and moving on with your life.

I reported it already. What about the person that contacted me on whatsapp? What can I do about them?

Report them to WhatsApp? It's probably a throwaway account, faked.


text:
Is there a point to using taproot transactions over P2WSH if I only want to use the script path?
Interested in making an HTLC, in which case there should not be a key spending path. According to this post, one can set the internal public key to an essentially arbitrary value and just use the script path. But then is there any advantage to using P2TR over P2WSH?

See also: Is there an economic incentive to use P2WSH over P2TR?

See question: Is there an economic incentive to use P2WSH over P2TR?
The answer is, if you're absolutely sure you don't need any of Taproot's features, then a P2TR input costs a bit more than a P2WSH input with the same script.
However, Taproot does have features that could be useful to you:

Script tree: Splitting a script into multiple alternative conditions and only revealing one of them when spending is perfect for HTLCs.
PTLCs: Point Time Locked Contracts are a more efficient and private alternative to HTLCs. A successful execution of a PTLC doesn't need to use the script path at all.
Privacy: Because P2TR combines key and script spending, an unspent P2TR output reveals less information than either a P2WPKH or P2WSH output. Additionally, P2TR outputs are already more common than P2WSH outputs, so you might want to choose the larger anonymity set.

I was also interested in PTLCs, but as I understand it they cannot be used on Ethereum (section 3.2.1 here) which would be quite limiting for my use case.


text:
Bitcoin to Ethereum atomic swap with PTLCs
According to this, it wouldn't be possible to use PTLCs to swap to Ethereum:

Cross-chain atomic swaps based on adaptor signatures, or at least semi-scriptless adaptor signature schemes designed for the ECDSA curve, are incompatible with Ethereum. This is because Ethereum uses the account model, not the UTXO model. Specifically, in adaptor signature-based atomic swaps, refund transactions must be pre-signed. However, in the Ethereum system, transactions cannot be pre-signed without knowing the nonce. Therefore, one party can send a transaction between the pre-signature and the transaction execution, invalidating the pre-signed transaction (because the nonce has been used and cannot be reused).

But as I understand it, someone sending you a transaction in Ethereum would only increment their nonce, not yours, so your pre-signed transaction wouldn't be invalidated. Am I misunderstanding something?
Are there any other Bitcoin-specific things that would be blockers for an Ethereum PTLC? I know that Schnorr signatures are also not well supported on Ethereum.


text:
What kind of private information would a bitcoin "middleman" ask from clients?
I don't know much about bitcoin so forgive me for the ignorant title and terminology. My mom recently got into bitcoin and told me she was investing her money in it or something like that and she's been doing these transactions through a company (whom I referred to as "middle men" in the title). These middle men recently asked her to provide pictures of her passport with her in it hold it up and also a pic of her credit card bill with her in it as well ; apparently to verify that she's a resident where we live (in Hong Kong, China).
I thought it was a bit suspicious that they asked for such private information. Is this a normal practice of bitcoin middle men/compamies?

When dealing with established companies as a middelman (usually an exchange), it is common to go through a form of know your customer (KYC). This usually involved some form of national ID, and a bank statement or other proof of residence/address, and is usually done to comply with local laws regarding movement and sale of currencies.
You should, however, be extremely wary of dealing with random companies or people. Make sure you are dealing with a reputable exchange (check review on reddir, yours.org, steem, and other such platforms), and as far as possible, keep any BTC or other crypto in a wallet where you control the private keys (such as electrum, Ledger, Trezor etc). If someone offers to do mining on your behalf, it is almost certainly a scam. If someone says you need to pay to unlock the BTC, it is a scam.
Verify you are dealing with a legitimate entity.

If the "MiddleMan" is a regulated exchange they would have asked for ID before you could begin trading with them, if on the other hand it is something like localbitcoin.com where you are dealing with individules they all have their own country specific ID rules if required by law so from reading your post it would seem to be a scam as 1. They would have asked before trading. 2. If you are already trading with them why do they need ID, all of a sudden.3. So called BitCoin Lending or Investements where you get Interest Paid Daily are also a Scam as if they actually were capable of paying such high interest, everyone involved would be either a Millionair or Billionair very quickly and every Bank, Hedge Fund etc. in the World would be doing it, even Government People, so if it were me i would not send any id & stop trading with them. "As the saying goes if it`s too good to be true it usually is" Let the Buyer Beware.


text:
Python - Querying bitcoin's block header using leveldb
I am facing a stupid problem. I am trying to read bitcoin/blocks/index leveldb using plyvel. I've set up a regtest and i know there is a block with the hash :
27ce8199dee22f96e4f392dec29ef207fcd8a3c9e43b7d58cb40a5ba01af674b

I am trying to query it using plyvel.
Note that I already read the topics :
How does Bitcoin read from/write to LevelDB
What are the keys used in the blockchain levelDB (ie what are the key:value pairs)?
https://en.bitcoin.it/wiki/Protocol_documentation
Then, I know I have to swap the endianness so it becomes :
MYVAR = "4b67af01baa540cb587d3be4c9a3d8fc07f29ec2de92f3e4962fe2de9981ce27"

Thus, I should query something like :
b + (byte) MYVAR

However I do not get how. I guess I missunderstand something in encoding. I tried :
db.get(b'b4b67af01baa540cb587d3be4c9a3d8fc07f29ec2de92f3e4962fe2de9981ce27')
db.get(b'b'+b'4b67af01baa540cb587d3be4c9a3d8fc07f29ec2de92f3e4962fe2de9981ce27')

..
Then, I opened the bitcoin/blocks/index leveldb using a gui explorer and I noticed the key for this block is :
\x62\x4b\x67\xaf\x01\xba\xa5\x40\xcb\x58\x7d\x3b\xe4\xc9\xa3\xd8\xfc\x07\xf2\x9e\xc2\xde\x92\xf3\xe4\x96\x2f\xe2\xde\x99\x81\xce\x27

which is \x62 corresponding to the "b" + the hash with \x (escaping caracter ? I know here is my misunderstanding on hex encoding)
and obviously
db.get(b'\x62\x4b\x67\xaf\x01\xba\xa5\x40\xcb\x58\x7d\x3b\xe4\xc9\xa3\xd8\xfc\x07\xf2\x9e\xc2\xde\x92\xf3\xe4\x96\x2f\xe2\xde\x99\x81\xce\x27')

perfectly works
I also tried these methods :
https://stackoverflow.com/questions/5649407/hexadecimal-string-to-byte-array-in-python
No one of them works and I really don't get why :
bytes.fromhex(hash)
bytearray.fromhex(hash)

Thank you in advance

This answer is a little stale. But here it is anyway.
You need to watch for the endianness of the bytes. How we look at the block id (block header hash) is bigendian, most significant bit first, and how bitcoin-core manages everything behind the scenes is littleendian, least significant bit first.
Here is a simple script to access the entire leveldb block index.
import re
from plyvel import DB

# Note for POSIX users, typically we have to escape spaces,
# e.g. "Application\ Support". However this is not needed here so you
# can write "Application Support" instead.
BITCOIN_DATADIR: str ="/absolute/path/to/your/bitcoin/datadir/"
db = DB(BITCOIN_DATADIR + "blocks/index")
keys = list(db.iterator(include_value=False))
regtest = re.compile(br"^b")
block_id_keys = list(filter(regtest.search, keys))

We can access the data structure and see this:
>>> bytes.hex(block_id_keys[-1][::-1][:-1])
'0000000000429f3501d60512df277b97f539becd18a7541b433e5110b2feffff'

https://blockstream.info/block/0000000000429f3501d60512df277b97f539becd18a7541b433e5110b2feffff
Going through the [-1][::-1][:-1], the [-1] is the last item in the list, the [::-1] flips the byte order from littleendian to bigendian, and the [:-1] drops the b"b".
To see the raw key, we can see the entire structure:
>>> block_id_keys[-1]
b"b\xff\xff\xfe\xb2\x10Q>C\x1bT\xa7\x18\xcd\xbe9\xf5\x97{'\xdf\x12\x05\xd6\x015\x9fB\x00\x00\x00\x00\x00"

Note the key begins with b"b".
To get the leveldb key from a bigendian hex blockid, you could do the following:
>>> hex_data = "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f"
>>> test_key = b"b" + bytes.fromhex(hex_data)[::-1]
>>> db.get(test_key)
b'\x8b\x99@\x00\x0b\x01\x00\x08\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00;\xa3\xed\xfdz{\x12\xb2z\xc7,>gv\x8fa\x7f\xc8\x1b\xc3\x88\x8aQ2:\x9f\xb8\xaaK\x1e^J)\xab_I\xff\xff\x00\x1d\x1d\xac+|'

Also see https://bitcoin.stackexchange.com/a/29418/124452

The leveldb databases are serialized for optimization and do not work well with attempting to manually parse the data. You can more easily get details on a specific blockhash using the built-in JSON-RPC API and use the getblock call.

Thank you for your reply, and I will do so. However, what I am trying to do is to reference orphans blocks. I'm not sure I would be able to see these orphans blocks through the API calls.. I mean if I know block O is an orphan block, I will be able to getblock O. But if I am looking for orphan blocks, not knowing their hash, I will be probably face some problems Iterating over all the keys of the leveldb would allow me to distinguish orphan block and "main" blocks.

I know i shouldn't answer to my own question but I found away to do so :
I'm getting all the keys using db.iterator()
And then using binascii.hexlify / unhexlify I've solved my problems. My workspace was a bit messy and that's why I didn't even realize that simple trick works (I'm sure I had tried)


text:
Bitcoin Core client on multiple computers
I have one computer with a full node bitcoin core. I would like to have bitcoin-qt installed on a different computer but I don't want to download the full blockchain again in the other computer. I would like to have bitcoind active in the first computer and the second one get transaction information from the first one instead of downloading the full blockchain and getting transaction information from a local blockchain copy. Is it possible to do it? And what about my phone getting transaction information from my computer instead of a public blockchain server?
I know I can use the second bitcoin core on pruning mode, but what about not holding any blockchain data locally in the second computer?

Possible duplicate of How to share the blockchain across local network residing on 1 PC, not copy?


text:
Does Bitcoin security rely on the assumption of nearly continuous transactions?
So for an attacker not to overcome good nodes, I believe we rely on the fact that blocks are always being produced, making the attacker essentially outpaced. But in the extreme and ridiculously unlikely case that no Bitcoin transactions occur within a certain sufficiently large time window, could an attacker gain control of the chain and do damage? If not, why?

Blocks don't have to contain transactions (other than the coinbase). Is that what you're asking?

While your question is approaching the subject from a different angle, I've closed it as a duplicate of the above, because the question's answers explain why the assumption leading to your question doesn't apply. Thanks for your interesting questions recently! :)

Ah, yep, thank you. So if I understand right, there is still a reward for securing the blockchain (adding new blocks) even though there's no peer-to-peer transaction. So blocks are being continuously added no matter what.


text:
Can we detect contentious soft forks created by the miners?
Can we analyze the blocks or the data sent per block in order to see if it includes new data that we do not recognize and assume that's a soft fork and invalidate such a block?
Can a softforked mining node send data that a user's fullnode can't tell it's foreign?

Because a soft fork by definition follows stricter rules than the normal chain, you would have to make assumptions based on the block data. For example, if a soft fork no longer allows certain scripts, you might be able to determine that by noticing that those scripts are no longer present in any blocks from that miner. However, it would be a guess since it is still a valid block, and just might happen to not contain those scripts.
There might be other changes that would be more obvious, but still a guess because the blocks will remain valid according to consensus rules.
Edit:
Regarding Segwit (BIP141), the block indeed contains extra data regarding the witness data, specifically the commitment structure which contains a witness root hash:

A witness root hash is calculated with all those wtxid as leaves, in a way similar to the hashMerkleRoot in the block header.
The commitment is recorded in a scriptPubKey of the coinbase transaction.

However, without knowing what to expect (and without the witness data, which was not sent to old nodes), this data is meaningless. One thing that does give away data that may appear to be extraneous is the use of the OP_RETURN script code, since this allows arbitrary data to be included in a transaction, and this is used in the commitment structure.
It is also interesting to see how BIP141 handled older nodes, by simply not sending witness data, see Backwards Compatibility. The segwit transactions appear to be anyone-can-spend scripts to older nodes, so the only thing they lose out on is the ability to validate segregated witness transactions, since it will assume such transactions are always valid. They don't know about or propagate witness data.

I agree but could such a soft fork be a contentions fork? I imagine there could be contentious soft fork that the miners would adopt where we can see there is foreign data besides the block that are being propagated. Right? or the data is not going to be propagated unless all nodes installed that soft fork and decide to propagate it. Do old nodes propagate the segregated witness data part of the blocks?

@zndtoshi see edited answer with more info on segwit


text:
Coinjoin API server possible?
Is there a protocol limitation (I suspect there is) that does not enable transactions across different platforms to access a 3rd party remote server (Chaumian server?) through API and create a coinjoin every let's say 2 minutes or 100 users from all platforms/wallets? If someone implemented such a server, this would highly increase coinjoin liquidity and privacy.

There is nothing in particular stopping exchanges from using CoinJoin to construct transactions with other exchanges, but rather there does not appear to be any demand for this.  For example both JoinMarket and Wasabi1 have APIs that could be used.
In the future this situation could change if something like Schnorr signature aggregation capabilities are adopted by Bitcoin. In that situation, CoinJoin transactions could be noticeably smaller (and therefore cheaper) than many separate single-output transactions.  This still seems very unlikely though as the exchanges are more likely to simply batch their own transactions rather than working with other exchanges.
Until a significant percentage of users demand privacy, exchanges are unlikely to change anything.  There may also be regulatory issues with them providing such an option.


text:
Bitcoin Core internal Electrum Server
Is there a technical reason for which there isn't an option in Bitcoin Core that would deploy an internal Electrum Server, or something similar so that a software wallet can fetch addresses and balances?

Electrum servers are simply not compatible with Bitcoin Core on a development or operational level. They require a vast amount of indexes created which the node does not have, and must be created by an external tool. With the state of the art electrs Electrum server, it needs upwards of a day of time to create these indexes, and more than 50GB of additional data stored on disk. For the server to be performant, forks of the software use over 1TB of additional storage for these indexes.

1TB or about 50GB? that's a big difference, can you elaborate? electrs or which fork? I run an electrumx server and was thinking about changing to electrs but last time I checked electrs was not indexing mempool transactions.

50GB or so for the main electrs that is fairly slow due to passing requests back to Bitcoin Core for data consistently. 500GB-1TB for the esplorer fork which doesn't need to do RPC requests for Bitcoin Core.

wow, that's why blockstream.info is so fast! mystery solved.

Yep. Typical time memory trade off.

I think the preferred technique to use 'bitcoin core as a wallet backend' is through block filters, like the new compact block filters (an improvement over bloom filters). The protocol is known as Neutrino and lowers the CPU and storage requirements for your bitcoin core node (an electrumx database takes around 50 GB right now). To create these filters in bitcoin core, you need to launch add the bitcoind option blockfilterindex=1 or blockfilterindex=basic. With these filters, you can filter for blocks that affect the addresses (scriptPubKeys) that belong to your wallet. Several lightning wallets use this technique
This block filtering technique produces some false-positives, so you will fetch blocks that end up not affecting your addresses. Because you are requesting more than just the relevant transactions to your wallet, you end up consuming much more bandwidth than using an electrum server but it has the advantage of being more private.
Neutrino is used by several lightning network wallets:

Neutrino
Basic tutorial


text:
Decoding Base64 from an old wallet.dat
Good day! I found a wallet.dat file on my old computer and no longer know the password, now I had the idea and what I read in the forum that I read the Base64 code and then convert this code.
Unfortunately my knowledge is limited and now I'm not sure if the file is corrupted.
This is what the Base64 code looks like (not complete!):
wKOCZEPaRssAQAAAAAAAAQAAAAAAwtmcm9tYWNjb3VudAABbgM4MzkJdGltZXNtYXJ0CjEyOTkxMjQ5NTYAAAAAAAK998GAAAAAjAAECdHi6UcWqMkoj92i8H4xWg/AlJx55Qa0EIouIrCN5JjQvYgAA ......
The Base64 code consists of a total of 5,013,504 characters!?
When I convert it, my resulting text looks like this (not complete!):
 =T����|`-;:6F[*w)�����������fromaccount�n1034    timesmart 1300202056�����}`��#�tx�&vQ,P$=/S~6d5ubGb~Y[���1[X1~0$v  fx)���G0D d[ꎹka|OvkӘǟXюz  ((I=QVP+o.ݑGAfQLMFjw)w]w aIO϶4}uwHX>   !``L���vbܶ"~`4gv)p@4����v.UN<I3VL
Shouldn't I see the password here or is the file corrupted or what is it?

I found a wallet.dat file on my old computer ... I read in the forum that I read the Base64 code

"wallet.dat"  is the filename used by the wallet software named "Bitcoin Core". Maybe some other types of wallet software also use this name but in this answer I will assume Bitcoin Core.
The current version of Bitcoin Core does not use Base64 to store the wallet data in the "wallet.dat" file. I don't know of any reasons why older versions would either.
The data shown in the question is not the sort of data I find in the "wallet.dat" created by a current version of Bitcoin Core.

Shouldn't I see the password here

No, the password for a file is not stored in the file itself.
The password for a file is typically used to encrypt (when saving the file) or decrypt (when reading the file) some or all of the file contents. Storing the password in the file would defeat the purpose of having a password.


text:
Can you mine in daylight only?
Would it be possible to mine bitcoin only when the sun is up?
ie does the miner have to run continuously.
I have access to off grid solar that doesn't get used for anything else,
so would not need to deal with the cost of electricity.

A miner does not have to run continuously. You can simply switch it off and on as you wish. Of course, it would make sense to link this to the current output of your solar system. That shouldn't be too complicated.


text:
Decrypting wallet
I'm seeking assistance with understanding the technical aspects of how Electrum wallets handle parsing errors and potential solutions for resolving them. My friend has encountered a recurring issue where the Electrum wallet fails to parse its wallet file during the opening process. While this issue has occurred previously and the wallet eventually worked again, it is currently persistently unresponsive. We have access to the wallet's password but not the seed phrase.
Could you provide insights or advice on the underlying mechanisms causing parsing errors in Electrum wallets and any potential cryptographic approaches to address this issue?

Related What does the electrum wallet file contains? and What encryption does Electrum use for private keys in wallets? I would decrypt then check the JSON.


text:
Create transaction and so on
Please explain to me what should be the procedure for working with a bitcoin-cli / RPC wallet.
For example:

Create a wallet
Get a new address
Sending BTC: creating a transaction, signing a transaction, sending a transaction or some else?

Creating a wallet
--------------------
Everything is more or less clear here, we use:
curl --data-binary '{"jsonrpc": "2.0", "id": "curltest", "method": "createwallet", "params": {"wallet_name":"WalletName", "avoid_reuse":true , "descriptors":true, "load_on_startup":true}}' -H 'content-type: text/plain;' http://127.0.0.1:8332/

or
bitcoin-cli -named createwallet wallet_name=descriptors avoid_reuse=true descriptors=true load_on_startup=true

Getting a new address
------------------------
Here we use getnewaddress, and the questions begin: if we use deriveaddresses, then we get several addresses. What do these addresses refer to, how many of them can I receive?
From the documentation:
Arguments:
1. descriptor (string, required) The descriptor.
2. range (numeric or array, optional) If a ranged descriptor is used, this specifies the end or the range (in [begin,end] notation) to derive.

What is end equal to? How can I find out?

Sending BTC
--------------
There are several ways:

send
sendtoaddress

I received the transaction Id, what should I do after that? Use createrawtransaction, then signrawtransactionwithwallet, and then sendrawtransaction? Or something else?
Thank you.

1. Creating a new wallet

$ bitcoin-cli createwallet "my-wallet"

2. Get new Address
Generate a new address from the wallet which you can share and use to receive coins

bitcoin-cli -rpcwallet="my-wallet" getnewaddress

3. Sending Money
(After loading money to the wallet) Send money from my-wallet to bitcoin address "tb1qw508d6qejxtdg4y5r3zarvary0c5xw7kxpjzsx"

bitcoin-cli -rpcwallet="my-wallet" sendtoaddress "tb1qw508d6qejxtdg4y5r3zarvary0c5xw7kxpjzsx" 0.01

More information on using and managing the wallet can be found on https://github.com/bitcoin/bitcoin/blob/master/doc/managing-wallets.md

Thanks, @brandon-odiwuor.
>> Generate a new address from the wallet which you can share and use to receive coins
But what about anothers addresses whic I get from deriveaddresses?
>> (After loading money to the wallet) Send money from my-wallet to bitcoin address "tb1qw508d6qejxtdg4y5r3zarvary0c5xw7kxpjzsx"
What is this address? Why I can't use another?
>> More information on using and managing the wallet can be found on https://github.com/bitcoin/bitcoin/blob/master/doc/managing-wallets.md
Thanks. But there is so

The getnewaddress RPC is generally used to get a new address while the deriveaddress RPC is used more specifically to derive addresses corresponding to an output descriptor

bitcoin-cli deriveaddresses "descriptor" ( range )

>> The getnewaddress RPC is generally used to get a new address while the deriveaddress RPC is used more specifically to derive addresses corresponding to an output descriptor
What it's mean? Should I use it? Or not?

You should use the getnewaddress RPC to generate a new address to use from the wallet

Thanks, @brandon-odiwuor.
>> bitcoin-cli -rpcwallet="my-wallet" sendtoaddress "tb1qw508d6qejxtdg4y5r3zarvary0c5xw7kxpjzsx" 0.01
And what about tx fee in this case?


text:
What exactly is the benefit of buying a hardware wallet for Bitcoin?
Let's say I invest in a physical Bitcoin wallet. They aren't exactly free, so it's going to cost me money, effort, time and energy (to set it up and put coins on it).
Then what?
It's not like I can actually use it in physical stores. And if I'm supposed to use it as a secure storage device for all my coins, doesn't it just scream "STEAL/SEIZE ME!" to burglars and the cops? And if I keep it hidden in my fireproof safe, why buy such a hardware wallet instead of just an encrypted USB stick with a wallet.dat on it? Much cheaper and the convenience of a hardware wallet is gone if it's physically tucked away in my safe anyway.
Basically every way I look at it, it seems like a bad idea.
The only situation I can think of where it would be good is if I were to meet up with a person physically to buy something from them with Bitcoin. I wouldn't want to take a computer with me, even a laptop, so then that hardware wallet would come in handy, if I "load it up" from my Bitcoin Core before leaving my home.
But that's a pretty specific and rare (to me) scenario. It's never happened, in fact.
As nice as it first sounds to have a dedicated hardware device which in theory allows me to easily pay people Bitcoin "on the go", I don't see the actual use of this in this world as it is now. Is this only useful in some theoretical future when Bitcoin has taken over? If so, is it really a good idea to buy a hardware wallet today and risk it becoming obsolete by the time Bitcoin ever takes off?
Is there something I'm missing?

Hardware wallets are generally designed to securely store your private keys in an always-offline environment ("cold storage"). This allows the user to have confidence that their coins will not be stolen by malware/hacking/etc, since any attacker would need to have physical access to the device itself.
If nothing else, this is the main advantage of a hardware wallet: the user can securely store their funds, without needing too much technical knowledge or skill.

It's not like I can actually use it in physical stores.

Well, if the store accepts BTC payments then you could, but I agree this is a perhaps less important/rare use case at the moment.

And if I'm supposed to use it as a secure storage device for all my coins, doesn't it just scream "STEAL/SEIZE ME!" to burglars and the cops?

Yes, any educated thief/cop/etc may view a hardware wallet as a target for theft. Most hardware wallets have security against this though, such as a PIN to unlock the device (usually with the threat of wiping the device after some number of failed attempts). Each device will have different features in this way, so as a user you'd have to decide what fits your needs best.
Keep in mind that you can also just use the device to generate a wallet in an offline/secure environment, then create and test backups of that wallet, and finally destroy the hardware wallet device itself. In this way, you could create a cold storage setup, without the added risk of having a device which is obviously a bitcoin hardware wallet in your possession (of course you'll need to keep the backup, but the idea is that the backup may be easier to hide- for example using steganographic techniques).

why buy such a hardware wallet instead of just an encrypted USB stick with a wallet.dat on it?

If the wallet.dat file was generated on a computer which connects to the internet at any point in time (past/present/future), then you will not have the strongest of guarantees that it is not infected with malware. The same goes for the USB stick.
A hardware wallet, on the other hand, should be designed to have extremely restricted communication channels with the outside world, such that the user can have a higher assurance that their privkeys have not been leaked. In many cases, I've seen hardware wallets include a physical confirmation that the firmware running on the device is genuine.
Worth mentioning as well: no electronic device should be used as the only form of long-term storage for a user's funds. Bit-rot and general electronic failure are guarantees, given a long enough timeline. It is very important to have a physical backup of the privkeys as well (think: written on paper, stamped into metal, etc. This backup usually comes in the form of a mnemonic seed phrase these days).

Of course, if you're a technically savvy, then you may prefer to create your own cold storage device, perhaps something as simple as a raspberry pi running Tails as the operating system. But for many users, a hardware wallet offers a great mix of security and convenience, for a reasonable price.
The extremely adversarially-minded user may also feel alarmed at the prospect of a supply-chain attack against a hardware wallet manufacturer. They would be great targets after all! To combat this, the user may choose to use a multi-sig setup, with signing keys generated on multiple devices from different manufacturers, and perhaps even keys from bitcoin-core, etc, as well.

Ultimately, there is no 'perfect solution' for security of your funds, each user will have to make a choice based on their needs, preferences, and comfort level.


text:
Bcoin - running webpack-app issue
I have cloned bcoin repo and installed webpack package. When trynig to run webpack-app I get the error below:
btc@ubuntu:~/bcoin/node_modules$ npm run webpack-app
npm ERR! missing script: webpack

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/btc/.npm/_logs/2020-11-18T08_55_31_312Z-debug.log

I have checked the node_modules folder inside the bcoin dir and couldn't find webpack-app package. Also, I excatly followed the instructions from https://bcoin.io/guides/browser
nodejs version: v10.19.0
npm version: 6.14.4

bcoin does not use webpack anymore, and in fact the browser node as a proof of concept isn't really being maintained since it is practically very silly :-)
The webpack config files were removed form the repo here and here. If you REALLY want to try the browser node, you will need to checkout a git commit of bcoin from before those changes were made.
The bcoin library can still be compiled for browser usage in simple webapps and is still very useful for small applications like that. I refer you to this guide: https://bcoin.io/guides/webapp.html ...which uses bpkg instead of webpack or browserify.
That browser-node guide should probably just be removed, or at least updated with a deprecation notice. If you feel like it you can make a PR to the website here: https://github.com/bcoin-org/bcoin-org.github.io
Feel free to join us on IRC #bcoin or on Slack to chat more about this.

I would love to join the Slack channel but I don't have the email address from the specific domain name to be able to join. Would you hep to get that?

The link I posted https://bcoin.io/slack-signup.html is a webform that will send an invite link to any email address -- is that not working?

It worked properly and I just joined,. Appreciate you ;)


text:
The best JavaScript libraries supporting Bitcoin Core?
I have downloaded and installed Bitcoin full node on my Linux server and I'd like to start working with the bitcoin core to create wallet, addresses, and transactions. What is the best JS libraries available to facilitate using all the necessary methods in this regard?

I highly recommend bcoin:
https://bcoin.io/
https://github.com/bcoin-org/bcoin
It is a nodejs library that can also run a fully validating node, SPV node, and BIP44 wallet. It's used in production by many long-standing bitcoin companies such as Purse.io
You can even use the bcoin client to make RPC calls to bitcoin core, and of course you do not need to run the bcoin full node to take advantage of the library for TX construction and wallet functions.

You can search github for popular open-source projects with the query "bitcoin core javascript".
The results will show you the most popular libraries on github at the moment!
I used bitcoind-rpc by bitpay, and it worked well for me. There may be other libraries that work better for you. Just search on github, gitlab, or other open-source repository hosts.

https://github.com/bitcoinjs/bitcoinjs-lib
Bitcoinjs-lib lets you create and sign transactions

Couldn't get their library to create valid addresses on testnet.


text:
Permission denied when attempting to backup wallet using backupwallet
When I call bitcoin@raspberrypi:~ $ bitcoin-cli backupwallet "backup.dat", I get the following error:
error code: -4
error message:
Error: Wallet backup failed!
~/.bitcoin/debug.log shows the following:

error copying wallet.dat to backup.dat - boost::filesystem::copy_file: Permission denied: "/home/bitcoin/.bitcoin/new_wallet/wallet.dat", "backup.dat"

Is it something I'm doing wrong?  Do I need to give permissions to user "bitcoin"?
The wallet is not encrypted.
Thanks

You need to check ownership and permissions for the files and folders involved. Your current user needs permissions to read /home/bitcoin/.bitcoin/new_wallet/wallet.dat and permissions to write to the current directory for backup.dat
Try ls -ld /home/bitcoin/.bitcoin/new_wallet/wallet.dat . and pwd and id. The permissions will be shown as three triplets for owner, group and others shown as rwxr-x--- 1 bitcoin pi which means the owner (user "bitcoin") has read, write and execute permissions (rwx), members of the group ("pi") have only read and execute permission (r-x) and other people have no permissions at all (---). The id command will tell you which groups your current use is a member of. The pwd will tell you the full name of the current directory where you are trying to write your backup.dat

To further diagnose the problem try
see if the target already exists and can't be overwritten
ls -l backup.dat
mv backup.dat backup.dat.old
echo test > backup.dat

see if the source file is readable - look for messages no read permission
file /home/bitcoin/.bitcoin/new_wallet/wallet.dat

try to copy it directly
cp /home/bitcoin/.bitcoin/new_wallet/wallet.dat backup.dat

Thanks,
ls -ld /home/bitcoin/.bitcoin/new_wallet/wallet.dat returns "-rw------- 1 bitcoin bitcoin 1462272".
So the owner (user bitcoin) has read & write permissions, but the group (bitcoin) has no permissions?
pwd returns /home/bitcoin, and ls -ld /home/bitcoin returns "drwxr-xr-x 6 bitcoin bitcoin".
So for the destination folder, user bitcoin has all permissions, but group has r & x only.
Id returns "uid=1001(bitcoin) gid=1001(bitcoin) groups=1001(bitcoin),27(sudo)"

@Vanderlow: see additional suggestions at end of answer.

No filename / overwrite conflicts.
I am able to copy the file using cp, so I've tested a backup that way and it works fine.  It's strange that I can read and write using cp but am unable to do backupwallet from within bitcoind.  Could it be something about bitcoind's permissions??
To backup using cp I have to stop the bitcoind server running otherwise the wallet file doesn't get backed up properly.
That's why I wanted to be able to do it using backupwallet so I could do it while bitcoind is still running.  Nevermind.
Thanks for your help @RedGrittyBrick


text:
How I can install Bitcoin Core Light Node (not Full Node) on a CentOs server?
How I can install Bitcoin Core Light Node (not Full Node) on a CentOs server? That I can use it with bitcoind and bitcoind-cli and RPC (https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list / https://developer.bitcoin.org/reference/).
Thanks.

Bitcoin Core is solely a fully-validating node, it has no light mode.

Okay, maybe I have the terminology wrong. How can I use bitcoind locally without deploying a full node for RPC?
For example:
curl --data-binary '{"jsonrpc":"2.0","method":"listaccounts","params":[]}' -H 'content-type:text/plain;' http://user:pass@127.0.0.1:8332/

You cannot. Bitcoin Core is a full node. You can use other software perhaps that does what you want, like lightweight wallet software, but it won't be called bitcoind and you won't interact with it through the same JSON-RPC interface.

How I can install Bitcoin Core Light Node (not Full Node) ...

You can't, because that product does not exist in a "light[weight] node" version.
Perhaps running it in pruning mode would meet your needs?

... on a CentOs server?

I'm pretty sure CentOS was absorbed by RedHat and has been replaced by RedHat's "CentOS Stream". At least, it may be worth checking whether CentOS and its surviving package repositories still meet your longer term needs.
See RedHat: What is CentOS

CentOS is an open source project that releases 2 distinct Linux® distributions, CentOS Stream and CentOS Linux. CentOS Stream is the upstream development platform for upcoming Red Hat® Enterprise Linux product releases.
The CentOS Project will discontinue updates and releases of CentOS Linux® between 2021 and 2024. This means current CentOS Linux users will need to choose a migration path.

Perhaps running it in pruning mode would meet your needs?

Yep, already running.
Thanks.


text:
Problem identifying the transaction hash for a new segwit uxto wallet
How do I the transaction id or transaction hash of a new segwit wallet I just created and sent tokens to? The transaction hash I see from blochian.info refers to the hash of the previous transaction.
For example, I want to send bitcoin from bc1q9pjjydvzyqd8n83znsv6zhh56lk8yztj3m7cjn to another wallet and the transaction hash I see on blockchain.ifo is f0768c1159ab868300493e093261c1e78727b389b5c640c78f50fac3eb321d2e, which appears to be wrong.
How do I get the correct transaction hash or transaction ID for bc1q9pjjydvzyqd8n83znsv6zhh56lk8yztj3m7cjn?

How do I [get] the transaction id or transaction hash of a new Segwit wallet

A wallet has many addresses, not just one.
An address can be used in many transactions, not just one.
Therefore an active wallet is usually associated with many transactions, not just one.
Any Blockchain explorer will show a list of transactions that involve a specified address. For example, Blockchair,  btscan, blockexplorer.one and blockcypher all show that address bclq9p...7cjn has only been involved in one confirmed transaction f076...1d2e
A Bitcoin wallet will usually also provide a way for someone to view a list of addresses used by that wallet and a list of transactions that have effectively sent money to or from addresses in that list.

I want to send Bitcoin [...] to another wallet

To send money from one wallet to another, ask the receiving wallet for a new address. Use the sending wallet to send money to that new receiving address created by the receiving wallet.
Your sending wallet will find and select money you have previously received into the sending wallet.
Some wallet applications have a feature called "coin control" which might allow you to send amounts of money previously received at your choice of addresses from those under the control of your sending wallet.

Thanks for your explanations.


text:
Am I being scammed again?
I recently was scammed out of $1000.00 a person recommended this Instagram site that assists in recovering stolen money. I was asked to pay a fee to obtain a program needed to move the money I foolishly paid the $85 and then was sent to another person that handles the end of it. He had me signup for Cashapp and then Trust wallet to move the scammed money back to me. I did not link any accounts of my own personal banking or cards. But he then said there was an mistake that he made and showed me a BNB Smart Chain Watch only wallet that showed a large sum of money and he wants it back. Using Cashapp. I am so naive about this and I don't want my money back or his but if it is true would assist in sending his back. Am I being scammed again? Please help I don't know what to do.

Yes, you’re being scammed again.

Yes, this is a scam. If somebody had a magical program that could recover your money, they wouldn't sell it to you - they'd use it to steal the money for themselves. Further, if such a program were possible at all, cryptocurrency would be fundamentally broken. I'm sorry for your loss, but the only option is to report the theft to authorities, or move on.


text:
Max amount of peers connections on LND
I'm trying to connect to a long list of peers. I'm running a LND(0.17.0) node.
My list has almost 1k peers and I can only connect to 6 of them. Is my node limiting the max connections? If so, is it possible to increase that max amount?

What operating system are you running? Many operating systems allow only 1024 open file discriptors concurrently by default.

@RenePickhardt Linux (Ubuntu)

[Answer] In principle there is no limit on the number of nodes to connect to. It seems that the firewall was blocking the traffic.


text:
Lightning Network Nodes do not report Bolt9 Features
Looking at the output of the describeGraph command of my node (LND), I have seen that my node detects approximately 19,000 nodes. Of those 19 thousand, 6 thousand do not report any bolt9 feature.
lnd@node:~$ lncli describegraph | grep -c "features"
18969
lnd@node:~$ lncli describegraph | grep -A 1 "features" | grep -c }
6274

Is there any logical reason why this could happen? I thought it was mandatory to report the features that are implemented in order to use the protocol correctly.

Could it be that those 6 thousand nodes are running older versions and might not support certain features or report them correctly? or some nodes might be using non-compliant implementations that don't report BOLT #9 features at all?

@deyw It could be, at first I thought of nodes that work with smartphones. It remains to be seen even if these nodes are really useful for the network. Because if they are not, the LN numbers are far worse than they might first appear.


text:
New copay on Windows 7
How can I install new Copay wallet to my Windows 7?
I cannot use the microsoft app store.
Thank you.

Have you tried installing from source? https://github.com/bitpay/copay#desktop-linux-macos-and-windows

Thank you for your suggestion, I'll give it a try.

Copay is written in Electron which means that it is entirely cross platform by design, Windows Store is just a cute wrapping in this case. You can git clone it and use it with node and npm but this is boring work and nodejs tend to throw errors at you because the compilation process of Copay is designed more for a Linux kind of system with things like rms etc in package.json.

Use Fiddler, Wireshark (on VirtualBox) or https://store.rg-adguard.net/ (Directly) to generate links to *.appx file.
Open the downloaded file as an archive (with WinRaR for instance).
Create a new directory in the location you find most entertaining.
Extract .appx file contents into that directory.
Create shortcut to <your path>\Copay\app\Copay.exe and run the app.

Tested for Copay v7.0.3.

Download the pre-compiled windows versions from the releases tab on the github page.
https://github.com/bitpay/copay/releases

Thank you for your help. If I understand your comment in the right way you mean the downloadable exe file. It's an old version and it is deprecated. The new version is available only in the microsoft app store.


text:
Does pruning node keep old wallet's transactions?
Does pruning node keep old wallet's transactions? For example I need to use the gettransaction API to retrieve a wallet's transaction in a deleted block, should I use the pruning mode? I cannot find any document about it,
Thanks.

Yes, it does.
Pruning only affects the blockchain/validation logic. The wallet always maintains a copy of all transactions relevant to it (including e.g. unconfirmed transactions that the validation logic does not know about).


text:
Can I receive detailed tx data via command line using only a lightclient?
I recently tried to sync the Bitcoin blockchain, but my laptop does not have enough space on it. I want to be able to analyze UTXOs and raw transaction data.
Will a lightclient allow me to do this?

To be clear, you want to be able to analyze any transaction without storing all of the blocks?

Yes. I am hoping to use all of the same command line commands to analyze the transactions.

No. Lightweight wallets are designed specifically only for the wallet user. It only stores transactions and UTXOs relevant to the wallet. It will not store any historical data or data that does not pertain to the wallet, so it is useless for trying to retrieve any arbitrary transaction.

Is there anyway to call tx info (from any transaction) from a full node using command line?

If you are running your own full node using the Bitcoin Core software and you have enabled the transaction index, then you can use the getrawtransaction command to retrieve any transaction that is in the blockchain or in the mempool. Not that this requires over 200 GB of disk space and can take several hours to sync.


text:
Is there a way to verify how many Liquid Bitcoins are in existence?
Is there a way to recognize peginaddresses where bitcoins are locked up for use on the Liquid sidechain, and therefore a way to sum up the total bitcoins currently circulating on Liquid?

You can easily verify the liquid circulation, in fact any liquid node can do this in realtime as part of its operation if you run a Bitcoin node along side it.
Pegins can't, however, be reconized until after they're claimed on the sidechain.  Before then the addresses are cryptographically indistinguishable from other similar multisig transactions on Bitcoin. But when they are released into the sidechain the party that generated the address publishes the information needed for everyone to find them.
https://liquid.horse/ is a website that runs a liquid node and reports this sort of data.


text:
How many BTC addresses have balance in them?
Of all the possible BTC addresses, how many have balance in them?
And what percentage of the total possible addresses is that?

There are approximately 800,000 unique addresses on the blockchain with some Bitcoin held at those addresses at the present time according to blockchain.com.
Let's assume these addresses are all bech32 (to simplify the calculation, in reality these are the minority).
The witness program in a bech32 SegWit v0 P2WSH address is 32 bytes. In theory this means there could be approximately 8^32 = 7.9 x 10^28 different witness programs.
The percentage of unique addresses on the blockchain then works out to be:
800,000 / 7.9 x 10^28 = 1.0 x 10 ^(-23)
This is 0.00000000000000000000001 percent of the total bech32 SegWit v0 addresses.

Do "addresses on the blockchain" include addresses that had some BTC in and then out and now have 0 balance?

No if you look at that blockchain.com chart the number of unique addresses goes down in 2018 so they are removing addresses that used to have Bitcoin but no longer do. If they were including those addresses the number would never decrease, only increase.


text:
Bitcoin scriptSig with two transactions to unlock the scriptPubKey
For example, the bitcoin script of a scriptPubKey that consists of OP SUB < 100 > OP EQUAL, and is used to lock an output of a transaction. OP SUB pops the top two elements in the stacks, subtracts them, then pushes back the result. < X > pushes value X on the stack.
Also, the following, a scriptSig script that consists of < 200 >< 100 >, which is used as an unlocking script to spend that output.
In that case, will the scriptSig succeed to unlock the scriptPubKey thus allow the transaction to spend the money? Because of how Bitcoin combines the scripts and how it executes them, like what are the contents of the stack at each step of the execution?

What do you mean by "two transactions"? There aren't 2 transactions in your example.

I mean for the inputs of <200> and <100>

You cannot put your plain script in scriptPubKey hence it's considered non-standard and will not be relayed/mined. You need to hash your custom script, put script hash in P2SH output, and reveal the script later when you spend it (in scriptSig).
Here's a step-by-step demonstration:
OP_SUB < 100 > OP_EQUAL

Convert your script to hexadecimal representation:

a. OP_SUB is encoded as 0x94 (in hex)

0x94

b. 100(dec) is encoded as 0x6400 (in little endian) plus one-byte length prefix

0x94026400

c. OP_EQUAL is encoded as 0x87 (in hex)

0x9402640087

Hash your script in order to get script hash:

RIPEMD160(SHA256(0x9402640087)) =
0xa038dd58be8e9c3bd9f26fe585dcdfe85d70ac4d

Your P2SH scriptPubKey is:

OP_HASH160 a038dd58be8e9c3bd9f26fe585dcdfe85d70ac4d OP_EQUAL

In order to spend/unlock this output your scriptSig should be:

< 200 > < 100 > < OP SUB < 100 > OP EQUAL >

In Hex:

0x02C800026400059402640087

Please note that arithmetic operations like OP_ADD and OP_SUB can only accept 32-bit integers, so if any input encountered longer than 4 bytes, the script must abort and fail.
Feel free to learn more about P2SH here

Thank you for your answer, as in your answer stated that the scriptSig is < 200 > < 100 > < OP_SUB < 100 > OP_EQUAL > then the scriptSig would not be able to spend the transaction due to the insufficient fund?
Because previous transaction of scriptPubKey contains only OP_SUB < 100 > OP_EQUAL ?

I mean because of the OP_SUB that subtracts 100 - 200 then it resulted as -100. Therefore, the OP_EQUAL operation will conclude invalid because -100 is not equal with 100 ?

scriptSig elements are placed in stack from left to right. Top stack element will be "< OP_SUB < 100 > OP_EQUAL >", bottom stack element will be "< 200 >", and "< 100 >" in the middle. Top element is popped out from the stack, hashed and compared against the hash in scriptPukey. If matches, the actual script execution begins:

In the actual script execution, two left stack elements are 200 and 100. "< 200 >" at the bottom, "< 100 >" at the top. OP_SUB pops the top two items 100 (a) and 200 (b), then executes "b - a". Learn more about opcodes here: en.bitcoin.it/wiki/Script

You can have better understanding of stack and bitcoin script here: https://youtu.be/6Fa04MnURhw


text:
Sending multiple output transaction with total output exceed the total input value
I am still confused about whether it is still a valid transaction for multiple output transactions if the total output of the transaction exceeds the amount of total input.
For example, there are two inputs in Bitcoin transactions, one having a value of 1.5BTC and one
of 0.75BTC, and two outputs, one with a value of 1.3BTC and one of 1.25BT.
Is this a valid transaction? And how about the transaction fee? Because one output is still less than the size of some of the input or the total input.

No, it is not. The individual values of the inputs and outputs don't matter, only the sum matters. So the sum of the inputs is compared against the sum of the outputs. And since the sum of the outputs is greater than the sum of the inputs, the transaction is invalid.
Bitcoin does not take Bitcoin from particular inputs and map it to a particular output. Instead it's like all inputs in the transaction get mixed together, and then that total amount is split out into outputs and fees. So only the sum matters.


text:
Lost bitcoin - advise and help

Hi there,
I bought some bitcoin many years ago in 2015 I believe, and I can't for the life of me recall where I've kept it in my computer. At that time, there weren't many facility to keep these private keys and they make no sense to me either.  I've received found this crypto code in my computer - see attached - and I wonder if anyone is able to advise if this looks like a bitcoin private key or not.

That is a Machine Security ID not a Bitcoin key
prefix S-1-5-2 is used by Microsoft Windows for Machine Security-IDs
As Wikipedia explains

The machine SID is stored in a raw-bytes form in the registry. To convert it into the more common numeric form, one interprets it as three little endian 32-bit integers, converts them to decimal, and add hyphens between them.

Example
2E,43,AC,40,C0,85,38,5D,07,E5,3B,2B

1) Divide the bytes into 3 sections:
2E,43,AC,40 - C0,85,38,5D - 07,E5,3B,2B

2) Reverse the order of bytes in each section:
40,AC,43,2E - 5D,38,85,C0 - 2B,3B,E5,07

3) Convert each section into decimal:
1085031214 - 1563985344 - 725345543

4) Add the machine SID prefix:
S-1-5-21-1085031214-1563985344-725345543

The Machine SID can be used in folder names such as c:\documents and settings\%username%\application data\microsoft\crypto\rsa\s-1-5-21-1078081533-1606980848-854245398-1003
Note the similarities:
Your Image          = S-1-5-21-0902708941-4082285366-676173334-153042
Example Machine SID = S-1-5-21-1085031214-1563985344-725345543
Example folder name = s-1-5-21-1078081533-1606980848-854245398-1003

Now compare what you have with an example WIF
Your Image          = S-1-5-21-0902708941-4082285366-676173334-153042
Example WIF         = L1aW4aubDFB7yfras2S1mN3bqg9nwySY8nkoLmJebSLD5BWv3ENZ

These are really not at all alike.
You definitely don't have a private key in wallet import format (WIF).

A private key is a 256-bit number that allows you to access and spend your bitcoins. It usually starts with a 5, K, or L and has 51 or 52 characters. The crypto code you attached does not look like a valid private key, as it has only 32 characters and does not follow the format. It might be a password, a seed phrase, or something else. You should try to find the file or program where you stored your private key, or use a recovery tool if you have one. Good luck!

The crypto code that you attached looks like a Wallet Import Format (WIF), which is a way of encoding a private key in base. A WIF can be used to import a private key into a wallet program.
However, just because it looks like a WIF, it doesn’t mean that it is a valid one. There are some checks that you can do to verify if it is a genuine private key that corresponds to a bitcoin address.
One way is to use a tool like bitaddress.org or cryptex to decode the WIF and see the public key and the bitcoin address that are derived from it. Then you can compare the bitcoin address with the one that you used to receive the bitcoin in 2015. If they match, then you have found your private key.
Another way is to use a programming language or a command line tool to perform the same steps as above, but with more control and security. You can find some examples of how to do this in Python or grep.
I hope this helps you to recover your bitcoin. Please be careful when handling your private key, as anyone who has access to it can spend your bitcoin. It is recommended to move your bitcoin to a new address that is generated by a secure and reputable wallet program.
Good luck!

I am confused why you think this looks like WIF, especially since the screenshot seems to show a directory. Could you perhaps elaborate that point?

A bitcoin private key is a 256-bit number, which means it’s a string of 256 binary digits (0s and 1s). However, it is usually encoded in different formats, such as hexadecimal, base58, or Wallet Import Format (WIF). A WIF is a common way to represent a private key, and it starts with a ‘5’, ‘K’, or ‘L’ and has 51 characters. A WIF also has a checksum to detect errors.
Based on the image you attached, the crypto code you found looks like a WIF, but it is not a valid one. I checked the checksum and it does not match the expected value. This means that the code has been corrupted or modified, and it cannot be used to access your bitcoin address.

I am confused why you think this looks like WIF, especially since the screenshot seems to show a directory. Could you perhaps elaborate that point?


text:
Is it always possible to find a number whose hash starts with a certain number of zeros?
As I understand it, proof of work requires nodes to "scanning for a value that when hashed, such as with SHA-256, the hash begins with a number of zero bits" ([bitcoin paper])1.
Specifically, based on the 3Blue1Brown video, nodes are scanning for number such that, when it is appended to the block of transactions, the hash of the block has a specified number of zeros.
I understand that SHA-256 and other cryptographic hash functions are supposed to be uniformly distributed, but is there any guarantee that it's always possible to find the desired number such that the hash begins with a certain number of zeros? What would happen if there was no such number?

Does https://bitcoin.stackexchange.com/q/22978/5406 answer your question?

It is unfortunate that Nakamoto's whitepaper and code disagree on this point.

@Murch yes that gets into it. Especially, because it says that each miner is potentially working on a different set of transactions, that assuages my doubt a lot. I'm curious for more about SHA-256 though

Before we get to the real question we need to clear up a couple of apparent misconceptions
Bitcoin never considers number of leading zeroes

As I understand it, proof of work requires nodes to "scanning for a value that when hashed, such as with SHA-256, the hash begins with a number of zero bits" ([bitcoin paper]).

It is understandable why, after reading the famous white paper, you would think that. It is not true though. No bitcoin node actually cares about the number of leading zero bits. The test in  Satoshi Nakamoto's source code is if (hash <= hashTarget) -- Nakamoto wrote one thing in the whitepaper but wrote something different in the actual Bitcoin software.
Therefore the situation is this:

Item
Binary
Comments / Verdict

Target
000000001100
8 leading zeroes in target

Block A hash
000000001101
8 leading zeroes but FAILURE because hash > target

Block B hash
000000001011
8 leading zeroes but SUCCESS because hash < target

Even though the block hashes have the same number of leading zeroes, one is a failure and the other a success.
The test is not "has a specified number of zeroes"

Specifically, based on the 3Blue1Brown video, nodes are scanning for number such that, when it is appended to the block of transactions, the hash of the block has a specified number of zeros.

That is doubly wrong. Even in systems based on numbers of leading zeroes,  like hashcash, I believe it is acceptable to have more than the specified number of zeroes.

However the core of your question is certainly an important one.
Is SHA256(SHA256(n)) surjective?
No it is not.
according to an answer in cryptography.stackexchange.com to Is there a guarantee that for each possible hash y there exists a number x such that with hash function H, H(x) = y?

A notable exception is double-SHA-256 (SHA256d) used in Bitcoin mining, where overwhelmingly likely there are some unreachable outputs.

Therfore it might be possible that, for exceedingly small targets, there might be no reachable values smaller than the target

The core principle behind the hash function, like SHA-256, is that it generates a seemingly random output for any given input and the output is uniformly distributed across the possible range. This means that for any given input, the chance of a particular bit being 0 or 1 is approximately 50%. Therefore, the chance of the first bit being zero is 1/2, the chance of the first two bits being zero is 1/4, the first three bits being zero is 1/8, and so on.
Due to these characteristics, it is statistically almost certain that a valid input exists which will generate a hash with a specific number of leading zeros. It is just a matter of trying enough inputs (or "nonces" in the case of ₿ mining) until you find one. This is what ₿ miners do in the proof-of-work system - they try billions of different inputs every second until they find one that generates a hash with the required number of leading zeros.

What would happen if there was no such number?

The ₿ network would be unable to produce a new block for the current difficulty level.

Thanks for the explanation. Can you show me (or link to) the math? (I mean, expanding on the math you already showed me.) You show that the probability of getting say 30 0 bits is 1/(2^32). That is a very low probability, weighing against what you are trying to show.

What you're asking about is essentially the concept of probability. It's like flipping a coin: the probability of getting heads is 1/2, but if you flip the coin enough times, you are almost certain to get heads eventually. ₿ miners are doing something similar, except instead of flipping a coin they are generating hashes, and instead of getting heads they are looking for a hash with a certain number of leading zeros.

Right but the analogy isn't perfect because the SHA-256 algorithm isn't a perfect random coinflipper. It's actually completely deterministic.

well, theres a good argument to be made that any coinflippers not based on quantum measurments are themselves completely deterministic too, so the analogy laregly holds: the point is that its not predictable without expending (a uniform amount of) energy, not that it is literally random


text:
Overflow bug bad chain length
Can someone recall how many blocks have been mined on the overflow bug chain until the new chain with the patch became the longer one?

You can estimate this by reading the corresponding forum posts.

Uh, you don't mean the 2010 arithmetic overflow inflation bug, do you?


text:
Sent Bitcoin to my electrum wallet address without hitting request first now it says its confirmed and i have no money after requesting it
Sent Bitcoin to my electrum wallet address without hitting request first now it says its confirmed and i have no money after requesting it after Isent it. Can I still receive my coin

Can I still receive my coin

For privacy, many wallets use a new receiving address for each transaction but sending money to an old address should still work.
You can use one or more of the public blockchain explorers to find out the status of transactions involving the old address.

Addresses
To view a full list of addresses known to your wallet see I can't find my bitcoin address
If the address to which you sent the money is not on that list, it is possible you sent the money to someone else's address and have lost the money with no chance of recovery. If you have multiple wallets or deleted a wallet and reinstalled the same software or another, it is possible you sent the money to an address that no-one has control over (unless you were diligent in making and retaining backups of the wallet and records of the private-keys and/or recovery phrases)
If the address is on that list and shows a balance of zero, where blockchain explorers show a non-zero balance, it is likely there is a problem with synchronisation.

Synchronisation
See I didn't receive a Bitcoin Payment. What does it mean when the Client is synchronizing?
If your wallet is having problems synchronising, you won't see your money until it succeeds
An unofficial Electrum Beginner's Guide says:

You can monitor progress from the Network window. Access it by clicking on the circle to the lower right of the main window. When the block count under “Blockchain” equals the height indicated for the servers you’re connecting to, Electrum will be fully synchronized and all confirmed transactions should be displayed properly under the History tab.

If the servers that Electrum connects to are themselves not synchronising, your wallet might not know that it is not  synchronised with the main body  of the bitcoin blockchain network. -- check the blockchain height in your wallet in the network tab (blockchain: nnnnn blocks) - make sure it agrees with blockchain explorers --

Checking "balance" using another wallet
Download and install a new wallet (e.g. on another computer you own)
Watch-only
Import the address into the new wallet
Note that this doesn't mean the address is yours and gives you no ability to spend money at that address.
Full-Access
Export the Private key from Electrum

Wallet > Private Keys > Export.

Import the private key into the new wallet

Footnote, probably unrelated:
Electrum phishing & theft
According to https://electrum.org/#home currently

Warning: Electrum versions older than 3.3.3 are vulnerable to a phishing attack. Do not download Electrum from any another source than electrum.org.

There are fake versions of Electrum which steal your money.

Blockchain says its complete and that it's at the the address but it doesnt show up in my wallet

See I can't find my bitcoin address - is your address shown in the full list of addresses in Electrum?

Yes  it is shown on the list and the balance says zero,  but when I right click it and  click view on block explorer it says the address has the funds

My electrum wallet says it never received or started to receive anything but it says its complete on blockchain

Is there a way I can access an address from electrum wallet on blockchain

You can download & install a different wallet and either: import your private keys from Electrum or use a recovery/seed-phrase of a type that both wallets support. I think Electrum doesn't use the standard BIP39 seed-phrase system so choices may be few.

If you specify the address, people may be able to assist you to work out what is happening (or they may not be able to add much)

Ok I'm going to try that thank you for your help. It's very appreciated

See also blockchain height in revised answer. Don't forget to up-vote any answers you find helpful.


text:
I made mistake sending Bitcoin to old generated address in nairaex ...the transaction still writing unspent...how can I get my Bitcoin back
I got 22 confirmation for Bitcoin not up 50$  and still unspent ,ow can I get it reverse back ,it more than 4hours

Given Nairaex is a centralized exchange, it is highly unlikely that you would have the private key associated with the address. Your best hope is to contact the customer service of the exchange and ask them nicely if they would be able to help you. That is your only way out.

I got 22 confirmation for Bitcoin

22 confirmations of a Bitcoin transaction mean that transaction is certainly beyond any economic hope of alteration or invalidation.

not up 50$

I don't know what you mean by this but will assume you mean the transaction was for an amount that is currently equivalent to USD 50. That makes it too small to be worth much effort to recover, you wouldn't spend 1000 USD to recover USD 50.

still unspent

Someone knows the private keys associated with the receiving address. That person can create a new transaction sending the money back you - less Bitcoin transaction fees etc.

[h]ow can I get it reverse back

You can't reverse a Bitcoin transaction.
Under some circumstances you can invalidate an unconfirmed transaction if you act quickly enough and are sufficiently knowledgeable. In your case it is too late.

old generated address in nairaex

Since NairaEx is a Bitcoin exchange in Nigeria, you should ask them for help. It seems likely that they control the private keys associated with that address and are therefore the only people who can assist.

I'm not sure what you mean. Do you mean the transaction is still unconfirmed? In that case there are two options.
1) If the original transaction was replaceable (depending on your wallet this is something you can set with a checkbox) you can use RBF (Replace By Fee).
2) If you cannot use RBF, you can try to double spend the original output(s) with a significant higher fee to convince miners to mine the 2nd transaction before the 1st, making the 1st transaction invalid.
If your transaction is already confirmed, than you will need the secret key of the old generated address to create a new transaction from the old address to the new address.

The questioner already mentioned that his transaction has 22 confirmations.

@UgamKamat I wasn't sure whether he meant his transaction got 22 confirmations or whether there were 22 blocks mined since he broadcasted his transaction, but I guess you're right.


text:
Can payment from a P2SH address provide change to itself?
In the gavinandresen Multisig Test test, could change have been redirected to the originating 3QJmV3qfvL9SuYo34YihAf3sRCW3qSinyC P2SH address for the multisig payments?
In Mastering Bitcoin by Andreas M. Antonopoulos, Chapter 5. Transactions the following statement is made, and I would like to know if it invalidate my question above: "Note that you are not able to put a P2SH inside a P2SH redeem script, because the P2SH specification is not recursive."

These two things are actually not related. There are no restrictions to address reuse in the protocol. You could get away with never using more than a single address ever (although that is not recommended especially for privacy reasons) and p2sh addresses are no exception there.
The redeem script encodes the conditions that have to be fulfilled in order to spend funds from an unspent transaction output (UTXO). What the sentence says, is that you cannot have a lock to another redeem script inside of a redeem script.


text:
Force closes reasons
What are the different reasons for a Lightning Network node to force a channel closure automatically, without user interaction?


text:
Lightning Network channel reserve
How is the calculation of the amount of sats reserved to pay fees in L1 in case of having to launch a penalty transaction or make a forced closure of a channel? In bolt2 I have read that 1% of the channel capacity is recommended but it does not make much sense as this 1% could be below the current fee rate and therefore the transaction will not be confirmed.
Added to this, is there a difference between how LND and CLN treat this?

These are two separate things:

The funder is in charge of setting aside sufficient funds to get an eventual close to confirm. That either means they have to set the fee of commitment transactions high enough for it to confirm some time in the future, or a collaborative close at the time of creation. This also shows why unilateral closes are more expensive, as they need to account for the possibility of rising fees and they often need to confirm more rapidly (in case of a stuck HTLC).
Both sides of the channel are forced by the other side to keep 1% of funds in the channel on their side. This is in order to ensure cheating always costs something (at least 1% of the channel balance). The reserves cannot be used as fees generally speaking since that'd be a way to reduce the impact of an eventual punishment.

So, while both earmark some satoshis for a specific purpose, these purposes are very different.

As for differences between CLN and lnd, I'm sure there are when it comes to fee estimation, however the protocol will ensure both implementations agree on a feerate for closes and a reserve that they'll impose on the counterparty.

Thanks for the response! So there's no dynamic "fee reserve" for the commitment transactions? At the very beginning, both agree on what will be the feerate of the commitment transactions at all time?

No, the Reserve value, i.e., the amount either side cannot dip into, is determined at funding of the channel, and will not change. It is however unrelated to the feerate for commitment txs, which gets negotiated over and over again to ensure a timely settling. They are two distinct things and do not interact.

How does CLN calculate the commitment tx feerate? Does it use estimatesmartfee or estimaterawfee from core cli? If so, which confirmation target do they ask for? It uses the return value or they even increment it a bit?

It ises the estimatesmartfee method of bitcoin-cli, with 100 blocks as target in case you are using anchor channels (https://github.com/ElementsProject/lightning/blob/9a4be1a4ac0b2269e355a18619e9a776d13a9c2c/lightningd/chaintopology.c#L614-L633)

I was told that LND uses a timer that is trigger at random intervals in a range in order to re-calculate and negotiate the feerate for the commitment tx. Does CLN work in a similar way? Do you have this documented so I can dip into it?

CLN also polls the bitcoin backend at regular intervals to get up to date estimates, and then we smooth over a 5 minute interval, slowly moving from the previous to the current estimate. The smoothing is there so the counterparty does not get upset if there is a big jump in estimates, and the peer has not checked recently.


text:
bitcoinj sendCoins method and Coin value
I'm trying to use bitcoinj to transfer the balance of the address derived from a random private key if that balance is non-zero.
I generate the private key and derive the address fine, then I use blockchain.info's API to check the address balance fine, but I'm not sure what I'm doing wrong with bitcoinj.
This line:
wallet.sendCoins(peerGroup, destination, wallet.getBalance());

I don't understand where it's getting the balance from. I haven't downloaded the blockchain and I've just created a new wallet from a random private key. Does this mean bitcoinj will always see this address as empty, even when I've confirmed that it is not empty?
I want to pass the value returned from the service into sendCoins (replacing "wallet.getBalance()" with the number of satoshis in the address), but that field requires a Coin and I can't figure out how to make a Coin with a specific value that I set.
How can I get this working the way that I want to? And is there any way to test it?
public class MainActivity extends AppCompatActivity {
    private final NetworkParameters params = MainNetParams.get();

    @Override protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        BigInteger max = new BigInteger("FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141", 16);
        Random rnd = new Random();
        for (int i = 0; i < 1; i++) {
            try {
                BigInteger privkey = new BigInteger(new BigInteger(256, rnd).toString(), 10);
                while (privkey.compareTo(max) == 1) {
                    privkey = new BigInteger(new BigInteger(256, rnd).toString(), 10);
                }
                ECKey key = ECKey.fromPrivate(privkey);

                // Calculate Addresses
                String address = LegacyAddress.fromKey(params, key).toString();
                Log.d("address",address);
                key = key.decompress();
                String address2 = LegacyAddress.fromKey(params, key).toString();
                Log.d("address2",address2);

                // Use API service to check balances of the 2 address derived from the 1 randomly-generated private key
                checkBalance(privkey, address);
                checkBalance(privkey, address2);
            } catch (Exception e) {
                Log.d("Error in Main:", e.toString() + "<");
            }
        }
    }

    private void transfer(BigInteger privkey, Long sats) throws BlockStoreException, InsufficientMoneyException, InterruptedException {
        Log.d("transferring coins", "from privkey:" + privkey.toString() + "<");

        // Import the private key to a fresh wallet.
        Wallet wallet = Wallet.createDeterministic(params, Script.ScriptType.P2PKH);
        wallet.importKey(ECKey.fromPrivate(privkey));
        Address destination = LegacyAddress.fromBase58(params, "bc1quecavdwnygmfvewvavwnzv3m5gyv3svjj8dvg5");

        // send coins
        final MemoryBlockStore blockStore = new MemoryBlockStore(params);
        BlockChain chain = new BlockChain(params, wallet, blockStore);
        final PeerGroup peerGroup = new PeerGroup(params, chain);
        wallet.sendCoins(peerGroup, destination, wallet.getBalance()); // how does it get the balance???

        // Wait a few seconds to let the packets flush out to the network (ugly).
        Thread.sleep(5000);
        peerGroup.stopAsync();

    }

    private void checkBalance(BigInteger privkey, String address) {
        String url = "https://blockchain.info/balance?active=" + address;
        String balance = "";
        JsonObjectRequest request = new JsonObjectRequest(Request.Method.GET, url, null,
                response -> {
                    try {
                        Long sats = response.getJSONObject(address).getLong("final_balance");
                        //Long btc = sats / 100000000;
                        if(sats > 0){
                            Log.d("Transferring", privkey + ":" + address + ":" + sats + "BTC");
                            transfer(privkey, sats);
                        }
                    } catch (JSONException | InterruptedException | BlockStoreException | InsufficientMoneyException e) {
                        e.printStackTrace();
                    }
                }, Throwable::printStackTrace);
        mQueue.add(request);
    }
}