text:
Why was noise XK fundamental pattern chosen in the transport layer (BOLT 08) of the lightning network
I am currently reviewing the transport layer of the Lightning network protocol. It builds on top of the noise protocol framework handshake patterns.
What I don't get: Why was the fundamental pattern XK chosen?
XK
  <- s
  ...
  -> e, es                  0                2
  <- e, ee                  2                1
  -> s, se                  2                5
  <-                        2                5

First of all why not KK? Nodes are announced via gossip and on my tcp socket I should see who connected to me being able to look up the static key of my peer. Was the reason so that lightning nodes could be private and in particular on tor?
Second of all why not using protocols without static keys? Is the reason that we wanted to have property 2 and 5 for initiator and recipient respectively?
Property 2:

Sender authentication resistant to key-compromise impersonation (KCI) . The sender authentication is based on an ephemeral-static DH ("es" or "se") between the sender's static key pair and the recipient's ephemeral key pair. Assuming the corresponding private keys are secure, this authentication cannot be forged.

Property 5:

Encryption to a known recipient, strong forward secrecy. This payload is encrypted based on an ephemeral-ephemeral DH as well as an ephemeral-static DH with the recipient's static key pair. Assuming the ephemeral private keys are secure, and the recipient is not being actively impersonated by an attacker that has stolen its static private key, this payload cannot be decrypted.

It feels like I have given the answer by quoting from the noise protocol framework page. But maybe I am mistaking so it would be great to get your insights.

First of all why not KK? Nodes are announced via gossip and on my tcp socket I should see who connected to me being able to look up the static key of my peer.

It isn't really possible to tell who is connecting to you based solely on the IP information you receive. The public IP information received over the gossip protocol only lists the listening endpoints which you can connect to, but this does not imply that those endpoints will be the ones connecting to you. In fact, they can't be, because the listening port requires a dedicated socket, so any outgoing connections from that IP would be on a different port. The port for outgoing connections is usually assigned automatically, although you can bind it to a specific local port explicitly, this is not common. You can't bind to the same port as the listening socket anyway, and if you're behind a NAT you have no control over the port the recipient might see at their end.
The IP address alone is not sufficient because you could have many nodes behind the same IP on a network, or due to NAT. It would be guesswork which node you think is connecting to you to try and pick the right public key.
Since the initiator's key can't be known for sure, the other options from the Noise protocol are N, I or X. We can rule out N because it does not provide the security guarantees you listed above, and I leaks information in plaintext which could be intercepted. X is therefore the most reasonable option.
For the responder's public key, we can again rule out N, leaving either K or X. We already know the key though, from the gossip network (or DNS), so there is little point on transmitting it also. Additionally, the recipient has a little extra information hiding because their public key is never transmitted.
The Lit project from MIT uses the XX pattern. I believe this is because nodes don't have information about public keys in advance because they use hashes of the public keys to identify nodes instead. Once the public keys have been transmitted, their hashes can be compared to the previously known public key hashes.

Second of all why not using protocols without static keys? Is the reason that we wanted to have property 2 and 5 for initiator and recipient respectively?

The reasons are as you've listed from the Noise protocol document. The public keys alone are sufficient to have both encryption and authentication without relying on any third party for the security guarantees. Other protocols such as SSL have inherent points of failure - such as the PKI, which has been abused for many real world attacks on security. (In addition to it being a rent-seeking industry.)

In that case I don't understand why key rotation takes place with HKDF and not a new handshake generating new random ephemeral keys.

The purpose of the key rotation is to prevent the decryption of older messages if the keys are compromised in the future. This could also be done with a new Noise exchange each time, but from a practical point of view it is more computationally expensive to do a handshake (multiple ECDH calls) than to produce new keys with the HKDF. There may be some other risks to it that I'm not aware of though.

For the first part of the answer I believe I can choose port 9735 as an outgoing port even though I am listening on 9735 for incoming connections. The tcp header has two port fields. The src port and the dest port. As soon as the tcp handshake took place the pair 9735/9735 Rieger with the ip addresses an seq and ACK numbers are my end to end connection.

The outgoing address you connect to is not the same as the local port you're using for the socket. You connect to port 9735 because this is the listening port used by the person you're connecting to. The port used on your end is automatically chosen, and if the recipient looks at who is connecting, they'll see your public facing IP and this seemingly random port. They can't uniquely identify you based on this information.

I try an answer myself. It seems desireable to achieve forward secrecy for longterm security and protection of static key leakage and authentication to avoid spoofing. The only patterns which achieve both are the ones that use static keys and either knowing or transferring them.

In that case I don't understand why key rotation takes place with HKDF and not a new handshake generating new random ephemeral keys.


text:
Where can I see the current size of the bitcoin blockchain?
I am tying to understand the concept of the bitcoin blockchain, from what I understand everybody owns a full copy of the bitcoin blockchain, but is it possible to see the current size somewhere without having "checked it out/cloned it"?
And are there any backups being made? I know it's distributed and everyone owns a full copy - kinda like with a git repository - but is it also backed up on centralized servers?

Yes, you can use this website:
https://blockchain.info/charts/blocks-size

Ah yes, so currently around 135 GB and a pretty step rate of growth, in a couple of years this will not be feasible to work with I guess?

@u123 Welcome to Bitcoin! You've stumbled directly into the scaling debate, quite probably the most contentious part of the Bitcoin protocol. Growth of the blockchain is one of the arguments towards keeping the block size limited.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

authors: Сергей Брегеда

sort: votes


text:
Why doesn't address contain a full scriptPubKey?
This is a general question, but let's suppose a P2PKH transaction. P2PKH address starts with "1" and the sender incorporates it into the scriptPubKey as:
OP_DUP OP_HASH160 <PublicKeyHash> OP_EQUALVERIFY OP_CHECKSIG
Why does an address contain only the <PublicKeyHash> instead of the full scriptPubKey including the Script commands? At the current scenario the wallet software needs to recognize the address and assemble the scriptPubKey itself.
If the full Script code would be incorporated in the address, the wallet software would not have to understand it and would simply set the scriptPubKey equal to whatever the address contains.
This seems to me as very useful in terms of forward-compatibility for future types, where the wallet software doesn't have to be upgraded to understand some new forms of transaction.

I believe it  partly done due to security reasons? It is much easier to bruteforce something XY if you already have X, (which would be the case by already adding the public key to your adress by default). Instead of a hashed variable where you will only get the pubkey if a spent is made, rendering it useless ( bitcoins will be gone by the time you are able to bruteforce it). ( Note that it is currently ( even with the pubkey) practically impossible to bruteforce any adress, but this might not be the case in the future.) - Someone else might be able to give a more detailed answer.

That is not what the question is asking (I misread it first time too). It does not ask why is hash there; it asks why is not the full scriptpubkey there, only the hash.

I've modified the question a little bit to make it clearer.

There are several reasons for this, some historical and some more recently:

At first, addresses were probably not intended to cover everything Bitcoin Script could do. They were a fallback for the case where the recipient was not online (we're calling addresses still addresses because they were originally IP addresses you'd connect to to request a scriptPubKey). So at the time they were introduced, a focus on size was probably far more relevant than considerations about flexibility. Note that addresses have a 'version byte' (which was intended for upgrading the format, not for identifying altcoins).
Later, when P2SH was introduced the concern about flexibility went away entirely: P2SH permitted conveying any spendability concern in a constant-size address. Furthermore, it removed the concern of senders needing to create larger outputs. The fact that I am using a multisig wallet shouldn't mean you have to pay a higher fee when sending to me.
When SegWit was introduced, P2SH was no longer sufficient to encode any output, for reasons of efficiency and security. Despite the need for a new format anyway, we still chose not to propose something that could encode arbitrary scriptPubKeys. The reasoning given in BIP173:

Why not make an address format that is generic for all scriptPubKeys? That would lead to confusion about addresses for existing scriptPubKey types. Furthermore, if addresses that do not have a one-to-one mapping with scriptPubKeys (such as ECDH-based addresses) are ever introduced, having a fully generic old address type available would permit reinterpreting the resulting scriptPubKeys using the old address format, with lost funds as a result if bitcoins are sent to them.

Thank you. So if we would hypothetically invent Bitcoin again, we would still probably not opt for a generic addresses that contain a full scriptPubKey because in the case of ECDH-based addresses (such as CryptoNote uses?) got introduced it would represent a security risk, because the older wallet might simply copy and paste instead of deriving a secret value. Is that about right?

That last point is my own opinion, and I don't expect everyone agrees with its importance. However, given the flexibility of P2SH and other scripthash based outputs (second point) it doesn't matter all that much.

Most bitcoin nodes only allow a few specific types of scriptPubKey, including P2PK, P2PKH, OP_RETURN, P2SH, native witness and multisig. Those are the 'standard types', others are not related by default (initially for security reasons due to bugs).
So addresses with non-standard pubKeyScripts would be basically unusable on the network as it is, and there is no need to include the full pubKeyScript in that case - it's space saving and safer to just use the template.
Addresses have version byte at the start to differentiate between types.
It is possible to create scripts which do whatever you want, in a similar way to what you are asking, using P2SH. Still P2SH addresses have a specific scriptPubKey format which is always used, but then when spending you can provide the redeem script

Yeah, I'm aware of that. I understand it is shorter and faster. It just seems that a full scriptPubKey including Script code would allow really easy implementation on the wallet side, which wouldn't have to know what P2PK, P2PKH, P2SH etc. are. I'm not sure the benefits you mention are larger over this one

@tsusanka it's not about benefits, read the first two paragraphs :)

I'm trying to, sorry if I'm being stubborn :). Let's suppose the scriptPubKey is a full Script code. You can still easily check if it is one of the standard transactions and the restrictions you mention would remain.

Then what is the point of allowing it? You can always give someone a scriptPubKey anyway, there is no reason you have to give them an address

The use of standard address types also helps to move the cost of sending to that address from the sender to the receiver. If you tell the sender to send to some stupidly large scriptPubKey, the sender is going to be the person who has to pay the fee for that to work. However if you use P2SH and use the script as the redeemScript, the burden of paying the fee moves from the sender to the receiver; now if the receiver wants to spend the coins, he has to pay the fee.

@MeshCollider the address and the scriptPubKey would be the very same thing

@tsusanka you are just playing pointlessly with terminology now... please start a chatroom if you want to continue this conversation, comments are not for extended discussion :)

Is there maybe a mixture of two things in your assumption?

Why does an address contain only the PublicKeyHash ...

Addresses do not contain hashes. Addresses are derived from the (hexadecimal) pubkeys, which are sha256&ripemd160'ed, added with a network byte, and some checksumming, and then base58encoded.
The script you show us here for P2PKH is the spending condition. When a funding transaction is created, this condition is in the output, so that only the person, that can create the pubkeyhash is allowed to spend the funds. To create the pubkey hash, you need the hexadecimal pubkey, and sha256/ripemd it...
Now why is a combination of OpCodes and Hash not used as address?
I give it a speculative try. I am asking myself, what is the gain? The wallet to surround the keyhash with 5 OpCodes (incl. length) is not much of an effort. The keyhash is anyhow calculated (fairly easy add/multiply with integers and no floating point arithmetics), to be used later on in base58encoding. All calculation costs are at the user's machine(s), not the bitcoin network machines. So not a big cost for the wallet machinery.
Limiting an address to one specific use case would require to have many address "types" in a wallet (a bunch for P2PK, P2PKH, P2SH, OP_Return... and what else). I think at this time you had just a bunch of keys in the wallet. And the wallet pre-generated addresses (no address re-usage), that needed to be properly back'ed up after every use. Adding different address types for limited use case would only complicate wallet handling?
Length of tx was already mentioned by MeshCollider.
Astoundingly a P2PK is shorter with compressed keys, than P2PKH (but not as anonymous as a P2PKH). The lock script is only  OP_CHECKSIG. Here the address would be the pubkey + 1 OpCode.
I come to this conclusion: hen creatig a software to transfer values, I need targets (aka addresses) and logic. The targets would be "stable", and the logic gives me all the flexibility. It could be done linking the target to the logic, but can't see the immedeate benefit from it.

"Addresses do not contain hashes." ... "which are sha256&ripemd160'ed". You're very much contradicting yourself.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Is it possible to talk to bitcoin RPC API with Javascript and WITHOUT using a node server?
I am planning on building a Wallet and want to know if I can hook something like an Electron native JavaScript app to Bitcoin RPC without running node.js for the server calls, so using something like AJAX or XMLhttprequest.

It is possible? Yes definitely!
Is it secure and smart to do? NO!
Anyone can see the password of your bitcoin node and thus steal your funds.
You will be hacked in no time.


text:
Why do we need public key as part of Witness in P2WPKH transactions?
As we want to minimize the data payload in a blockchain, which also led to concepts like SegWit, and is matter of discussion in Signature aggregation etc, I wonder why do we actually need to supply public key in order to spend? It's needless 33B of data. ScriptPubKey could look like this <HASH160> OP_CHECKSIG2 and ScriptSig <Signature>. I know this requires Softfork, but as we introduced SegWit, why wasn't this the design? Is it because mapping signature -> public key is not unambiguous?
Edit:
As in the comments suggested: the issue with public key recovery from EC signature is simply trade-off between fast verifiability and space requirements. The public key recovery from signature (r,s) involves conversion of scalar r = xr into a curve point R = (xr, yr), i. e. modular square root. This raises another question:
What's the idea of segwit transaction verification? Do we have multiple levels of consensus rules validation? E.g. basic validation evaluates all transaction scripts according to pre-segwit rules (which is just push of 2 stack items, which is always True), and deep validation that triggers the witness validation logic. I'm not sure whether I should look at segwit as "We moved the data away from transaction that represent just some ownership proof which is relevant at the time of confirmation but not years thereafter" or maybe rather like "Witness data is inherent part of transaction, just as it has always been, and was actually meant as tx-malleability fix and Script language generalization allowing future protocol extension."?

The questioner seems to be alluding to the ability to recover public keys from ECDSA signatures, such as described in https://bitcointalk.org/index.php?topic=6430.0

Pub key recovery generally wasn't known about when bitcoin was designed, and has some small caveats like being more CPU intensive. Most of the resource usage of Bitcoin is CPU time already.

@Anonymous not really. It could be faster if so many steps weren’t single thread.

This was discussed today on the #bitcoin-wizards IRC channel.  The reasons offered by Andrew Poelstra (andytoshi) were:

"Segwit had enough scope already."  That is, it was already a large change that some people considered controversial, so minimizing further changes was considered desirable.
"Pubkey recovery is incompatible with batch validation."  Poelstra goes on to explain that Bitcoin doesn't currently have batch validation for ECDSA and the reasons why, but it doesn't sound to me like batch validation would be impossible to add at a later point (though perhaps it would require a soft fork); presumably devs would prefer to work on batch validation and signature aggregation using Schnorr signatures instead.

Later, Gregory Maxwell (gmaxwell) added that "Certicom has a patent on key recovery that could potentially apply" to a possible construction, which could've made using that construction controversial.
FYI: your second question asked in edits should probably be asked separately as it's a completely different issue.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
A rogue wallet was created on my Full Node. How did this happen? Can external users create a wallet on a Full Node?
I have a synchronous (1G/1G) network connection and I was willing to put up a full node that allows inbound synchs, but not without figuring out how someone opened a rogue wallet on my system.  This attack undermines the faith in supporting the network, which is alarming to me.
So a rogue wallet was created on my Full Node, the question is how did that happen.
Luckily, I didn't have my own wallet open on the system, I was just trying to contribute to maintaining the blockchain.  On 11/20 I noticed an empty rogue wallet created with the ridiculous name:
"th keyboad is clicky, WOWO what. uge supride i met you last sunday, and , you looked like you wantd to die!"
This is not my wallet and didn't have anything in it.  It looks like it was just created, and I didn't see any evident transactions, but there certainly could have been.
HOW DID THIS HAPPEN?  This PC is not used every day.  There should have been ZERO chance of this happening, which is why I'm looking at the TCP/8333 bitcoin protocol in question.  I did a forensics review of the machine in question and it has no RATs (Remote Access Terminals) or back doors.  Unless the adversary deleted them, which is always a possibility.  I've dug and dug and can't find anything, so I'm turning to you all for help.
System and network details:

SHA256 verified copy of bitcoin-22.0-win64-setup.exe
(9169989d649937c0f9ebccd3ab088501328aa319fe9e91fc7ea8e8cf0fcccede)
Fully patched Windows 10 Pro
Network commercial grade firewall with NO inbound SNATS or ports exposed
Local (on PC) commercial grade software firewall
Local (on PC) commercial grade software anomaly detection
Network traffic was TCP/8333 outbound

Updates since original post:

The system had/has a blank bitcoin.conf file.
There were no GUI Configs or CLI options.  I made no changes to the initial install.
I've cleaned the system up, removed the rogue wallets, uninstalled and re-installed bitcoincore 22.0,  and put it back on it's isolated network.  It is synching up with the blockchain after an application wipe and should be back up in about 7 hours it says.  I don't see why it wouldn't happen again.  If it happens again, next step would be a complete wipe the OS and see if it still happens.

I assume that you are running Bitcoin Core. Where does the name of the wallet appear? And do you have your node configured to make RPC accessible?

Thank you for the quick reply. The rogue wallet showed up in the user roaming directory.
c:/Users/%user/AppData/Roaming/Bitcoin/wallets/(crazy name above) I was allowing incoming connections, I'm not sure how I could make the node RPC accessible? I didn't add anything else. Again, no ports were open to the outside world, so only established connections could make it back to this node.

What is in your bitcoin.conf, your GUI config and command-line options?

bitcoin.conf is 0 bytes and blank,
There was nothing added to the GUI config or CLI.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to Properly Use a Hardware Wallet and Recover My Coins if I Forget it Somwhere?
A noob in the bitcoin world, and trying to get in by diversifying some of my assets into bitcoin. My apologies if the question sounds too ignorant or basic. I was just watching this video for beginners which made some sense about how to own and transfer coins.
https://youtu.be/tDXE99uyg4s
It recommends the ledger nano S hardware wallet. As I understand, once I get it delivered as a package, I will have to set up my private/public keys and also my receiver address. To receive payment, shall I just give my address to anyone and ask them to transfer?
In this regard, I have two questions.

How do I know they have transferred?
Unless my hardware wallet connects to the internet via a computer, will the transfer take place at all? What if I ask someone to transfer, and I do not connect my wallet to internet for a week, month, year etc.? Will my money get lost somewhere?

Also, is the hardware wallet really bullet-proof in the sense it cannot stop functioning? Is my entire saving dependent on the functioning of the USB drive? What if it drops in water somewhere, gets spoiled, like any USB drive? Assuming I have all the keys safely noted down, how do I recover my saving?
And, what if the wallet falls in the wrong hand, without the keys? Can they steal my coins?

Some answers here are recommending that you check online blockchain explorers to see if you've received funds. This is generally fine, but is detrimental to your privacy. Browsing these sites via tor can help you maintain some privacy, fwiw.

To receive payment, shall I just give my address to anyone and ask them to transfer?

Yes, but for privacy reasons, it is recommended to give people a new address for each payment, and avoid re-using addresses. The wallet software (such as Ledger Live, or Electrum configured with hardware wallet) will give you an option to generate new addresses, and all of those addresses would go to the same wallet.

How do I know they have transferred?

Either you check for an incoming payment in your wallet software, or you use a blockchain explorer like blockchain.com to look up the Bitcoin address you gave out and check if it has the expected incoming payment.

Unless my hardware wallet connects to the internet via a computer

Your hardware wallet never connects to the internet. Your computer runs wallet software that uses an internet connection. The hardware wallet is only for signing transactions.

What if I ask someone to transfer, and I do not connect my wallet to internet for a week, month, year etc.? Will my money get lost somewhere?

The payment will still go through! You don't need to be online to receive a payment. The record of your bitcoins are stored on the blockchain. Your hardware wallet is only strictly necessary when you wish to spend the Bitcoins.

Ledger Hardware Wallet
As soon as you have set up your Ledge (which involves creating a backup seed, so you can restore your private keys in case you lose it). You can then create accounts (you can do so directly with Ledger-live or other third party apps which support the Ledger hardware wallet). The Ledger device basically is a keep safe for your private keys and allows only spending coins (any coins really this is not limited to bitcoin) if you confirm transactions on the device.
To your questions
If a transfer was success can be seen on any blockchain explorer (https://live.blockcypher.com/btc/) or directly inside apps like Ledger-live. You only have to know the address and you should be able to find it.
Coins will be transferred to your account anyway. No matter if you have the Ledger plugged in or not. You can not spend the money if you haven't attached the Ledger to your computer, because you can not confirm a transaction on the Ledger. (Ledger Nano X also has Bluetooth so you can wireless attach it)
If your hardware wallet stops working you can use your seed and buy a new wallet, like this you restore your private keys and you can happily spend your coins again.
The Ledger itself is protected with a PIN (min 4 - max 8 Numbers) which you need to open it up (like your mobile phone). So if somebody just randomly stumbles across it he can have a go at it 3 times then the device locks up and you need to recover your private keys by the initial seed). So what is most important to safeguard is your seed, if it gets stolen or public anybody who has a Ledger (or buys one) can get your private keys and spend your money.
Further Infos
Hardware wallets are a lot safer than keeping your private keys on any random device, as they do protect you very well against an online attack. Make sure you give your wallet address to people correctly. There is no bank to check your mistakes you make in misdirecting coins. If you send them to the wrong address they are gone. Your seed is an important factor and to be kept offline and save too (i keep mine on multiple usb-sticks encrypted inside a password manger, in my case key-pass which is password/key protected and stored offsite and protected against the elements, you can hammer it in steel, there is kits for that, or keep it on paper, to each his own)
Strong Advice
First get yourself familiar with the coin concept and how it works otherwise it can be easy for scammers or malicious people to trick you into sending them money and you may loose it all. Ledger has a Ledger academy and they also feature YouTube videos, which are not a bad starting point to hardware wallets.
Full disclosure i own a Ledger Nano X and exclusively manage my crypto with it.


text:
How can "Lightning Loop Out" help me to receive more payments?
In his blogpost, Alex Bosworth describes that “emptying of the channel allows them to receive more payment” (https://blog.lightning.engineering/posts/2019/03/20/loop.html). But how can I allow more inbound payments by withdrawing funds from an active channel? From my understanding, the only way to get inbound capacity is a connected node that provides funds on its side of the channel?

As mentioned in other questions here there is a difference between capacity and balance.
Currently you cannot change the capacity of a channel. So let us assume you have opened a channel with a capacity of 0.1 BTC
This means you can now send up to 0.1BTC but you cannot receive any BTC on that channel.
One you used this channel to pay an invoice for example with 0.04 BTC the capacity of the channel stays the same but now you can receive 0.04 BTC on that channel which we usually call inbound capacity and you can only send 0.06 BTC (not in one payment though since the max payment limit is currently a little above 0.04 BTC) the 0.06 BTC is caked your outbound capacity. Inbound and outbound capacity together reflect the channel balance.
What loop does is that it gives you an invoice (something to pay for) and reimburses you onchain. In this way you have moved some of your outbound capacity to become inbound capacity without the necessity to spend money by actually buying something. (though you technically pay a small fee for the service)
Hope that makes it clear (:

I think Mike was asking about "loop in", right?


text:
Do I have to pay Fees on the Lightning Network if there is a Payment Failure?
Lightning nodes are talking fees for routing a payment. Will fees be charged anyway if a payment fails?

Short answer: No.
Explanation: The fees are part of the onion routing package within the HTLCs. So a routing node accepts an incoming HTLCs and forwards it if there is a difference in value that can be collected as fees. The HTLCs are conditional payments on the condition that the preimage is delivered in return. This will only happen after a path of HTLCs is successfully established. If a payment fails some node can't forward the HTLC which means that noone along the path will be able to settle the HTLCs. This in turn means no fees are being paid.


text:
How is node capacity determined on 1ML.com?
I have set up a Casa node, using lnd, and opened 15 payment channels. The capacity of my channels, after almost 3 weeks, is exactly what it was when I opened the channels, leading me to believe that I have earned no fees.
However, when I look at 1ML.com, my capacity is 22% higher. Are these fees generated (I hope!) or is my node including the capacity of some private channels that my node is linked to? In short, how does 1ML.com calculate the capacity of my node?

Since 1ml is closed source I obviously cannot be 100% sure how it calculates the capacity of your node. But from my experience I can say that it does it in the same way as any other lightning node do it.
It takes the capacity of all payment channels which have been broadcast by the gossip protocol and have been received by the node in question and adds them up. This should be handled in the same way on your casa node. In particular 1ml has no chance of adding private channels (since they are private and not announced on the gossip protocol)
I believe the difference of the amount that you see comes from a different understanding of what capacity means. Let me elaborate:

The capacity of a payment channel is the amount of bitcoin that has been provided by the person who opened the channel (as long as dual funded channels are not implemented).
Your balance (or your funds) are the amount of bitcoin that you own in that particular channel.

So I believe what happened is that after your node was online for some time other nodes opened channels to your providing your node with more capacity (which is depicted on 1ml) but your funds / balance has stayed the same since those nodes only opened a channel but did not send money to you.
Therfor when you looked on the user interface of your casa node you saw a different amount because you most likely confused capacity with funds / balance.
In particular 1ml can only see the capacity of the channels and not the actual funds that you own as that data is private to you (and for each channel to your channel partner). Also I would highly doubt that you have earned more routing fees than a couple satoshis. The node of Alex Bosworth has some high routing fees configured and despite being online for a long time and very well connected the routing fees have not been high.

Thanks Rene, that is very helpful, and clearly explained.


text:
How can a single funded channel become bidirectional?
Is it possible that funding transaction of Lightning Network channel has got just one input? And if yes will it be one-directional during whole period of channel existence? Or after some off-chain transactions into Lightning Network, channel will become bidirectional due to second participant will have some funds after this transactions?

The original title "lightning network funding transaction" was not a question. I changed the title suggesting a question which seemed to me to be the essence of your post. Please feel free to edit the title again if my suggestion does not fit.

The amount of inputs of the funding TX has nothing to do with the question if the channel is unidirectional or bi directional.
What is relevant to understand is how the commitment tx encodes the balance of the channel. (the difference between capacity and balance was discussed in this question
In the standard case the balance of the channel gives all the capacity to the party funding the channel. (currently only one party can fund a payment channel but this will change with dual funded channels which are supposed to come with the next protocol update aka BOLT 1.1). So in the beginning of the lifetime of the channel it seems unidirectional as only the funding party can send money by making a payment or routing a third party payment on this channel.
However once that has been done the commitment transaction is invalidated and a new one is negotiated which has a different distribution of the balance. Now the party funding the channel is able to receive Bitcoin on this Channel.
There is one tiny exception: every channel partner must keep 1% of the channel capacity as a reserve for the penalty if a channel brach happens. So if the party that funded the channel makes initially a payment of less than 1% of the channel capacity it still cannot receive on that channel.

One small question. Should be all inputs of funding transaction be from one address or they can be from different addresses which belong to one wallet?

Can come from multiple outputs / addresses

In the current specifications of lightning network channels, only one party contributes to the funding transaction. However, the channel itself is always bidirectional. In fact the channel can possibly become bidirectional since the very inception of channel. This is because the party funding the channel can optionally send some specific amount unconditionally to the other party when the channel is established. But yes, as the channel balance of the other party increases, it can support higher payments.


text:
How does 1ML measure the availability ranking of nodes?
If there is an 1ML Node Rank ranking explanation anywhere ?
My node is online 99.9% but "Availability" dropped down from 700 to 4000 in last 3 months. :(
I guess this is somehow associated with the number of channels ?

Could you post a link to that node on 1ml? though questions like those are hard to answer as it is not public how 1ml measures this data

Hello Rene, thank you for attention.
here is my node
https://1ml.com/node/02bfb576be997eac8afbf6fb22e8e69b58eaea2640f6aab6b7ba3f54ca5adf49f4
Yep, 1ml have not shed light on this question, and we haven`t sources to check by self, but probably there is any 1ML dev we can ask to know for sure?
I sent same question to https://twitter.com/LNstats recently, but looks like this account is driven by bot (some script), and maybe even not related to 1ml.

Sorry that does not make sense. My node has an availability of ~400 and was offline several times. Sometimes even for days. Are you positive your uptime  is 99.9%?

I sure uptime is 99.9. I connect to this node from my Zeus LN mobile wallet everyday, to check if node is online.
Also I use this node for generating receipts on OC store, to show demo of LN payment gateway for opencart.
now I made uptimerobot.com page, to collect 100% reliable uptime.
https://stats.uptimerobot.com/QOMOOIyjq

This node have only 3 channels, and only few (up to 5) connections with other nodes, probably that's why availability is low.
I mean node could be unavailable for others often, but still online .
I had an issue, when user from California was not able to connect to my node, but node was online. And issue was solved, after my connection to his node.
And maybe it's becaue of low time-out limit and high latency between Ukraine and USA :), so I need to make more local channels.
What do you think ?

So, as we can see, node has 100% uptime during last 7 days.
but availability drop from 4070 to 4311

Availability  4752 now

Looks like the problem is found.
I didn`t forwarded 9735 port on router, as LND and my router support UPnP.
But looks like UPnP works not as intended, so sometimes host was accessible on 9735 port, and most of the time not :(.
Now I forwarded 9735 port to my Raspberry Node local address , so  Availability on 1ML not dropping down.


text:
sum(VOUT) > sum(VIN) -- Is this a possible scenario?
I have a transaction. This transaction was taking a value from AddrX and sending it to itself in a "split" of sorts. In the transaction there is one VIN of 98000000. This transaction split the stack and has two VOUT. They are both 49000000.5. There is a transaction fee of 1.00.
In this case, the total VOUT is 98000001 which exceeds the total VIN of 98000000. However, that difference is found in the transaction fee.
From what I have understood about the blockchain and transactions, the transaction fee is: Fee: sum(vin) - sum(vout) so it results in a -1 transaction fee.
My issue here is that if I were building a database to track this address I would end up subtracting 98000000 from their balance, and then would add the two incoming values of 49000000.5 which would end up placing their total balance after this transaction to 98000001 which would be incorrect. Now, I could add logic to remove the transaction fee incurred from this manually; however I would have assumed that this transaction would be invalid because the total VOUT was greater than the total VIN.
Could this situation be possible? So far from transactions I've seen, the transaction fee has already been applied to one of the VOUT and thus would not cause this error. If it is possible, how can I detect this and remedy it?

Such transactions would be invalid. While you could create them, all nodes would reject them as invalid and such transactions would never confirm. If this were possible, then it would be possible to create money out of nothing (except in Coinbase transactions) and that should be impossible.
You have either constructed your transaction incorrectly, or are interpreting it incorrectly.

So if this were to have happened on a blockchain, is there an issue in the codebase?

It means that there is possibly a problem in a given client's validation code, not necessarily a problem with the consensus rules themselves or with the coin. That a client would accept such an invalid transaction means that the client has a problem.

Coming back to this later, it seems this is because this is related to a Proof of Stake blockchain!

The bitcoin protocol rules https://en.bitcoin.it/wiki/Protocol_rules#.22tx.22_messages (rule 14) prevent this. It is a reject criteria for nodes.

authors: آرادخاکی آرادخاکی


text:
How to access funds from a fork?
So I want to access funds from various bitcoin forks, and I have my keys.
Are there any sites, or services that allows me to enter my keys to get access to the funds from the forks?
Point me in the right direction

You will need to download a wallet that supports the said fork.
So for example for the "Bitcoin Cash" fork, you will need to download something like this, https://electroncash.org/
Take extreme care when downloading these wallets, as a lot of them are infected/and or malware.
Always check the authenticity of the wallet files you download.
For electronCash they can be found here https://github.com/fyookball/keys-n-hashes

So then I will be able to input my key into the wallets and have access to the funds?

Yes, import the private keys which held bitcoin at the time of the (in this case bitcoin cash) snapshot, and your bitcoin cash should be there. (Check if your adress has balance in a bitcoin cash blockexplorer such as https://blockchair.com/bitcoin-cash/blocks)

Thanks! I guess it's time to check the balances and fire up the VM and see what works


text:
Why are transactions taking so much longer in the last few days (2017-10-29)
I noticed that in recent blocks there has been lots of transactions that has been waiting for as many as 3 days.
For example, just randomly picking a block 492200, which was produced on 2017-10-29 12:04:50, there are transactions that date back to 2017-10-26 17:32:41
However, when I looked at blocks just two weeks ago, there weren't any transaction that were waiting for so long. Again randomly picking an example block 490200, produced on 2017-10-16 20:54:59, the earliest transaction in that block I could find was sent out on 2017-10-16 13:02:04.
So what has happened to the block chain in the past two weeks (except for the Bitcoin Gold fork, but I don't see how that relates to this)?
Note: I am not asking about how to solve unconfirmed transactions. I am just simply curious what is causing this to happen as block times and network hash power doesn't seem to have changed much.
Another example: Transaction is stuck and unspent 2days+

So what has happened to the block chain

Some miners moved their hashpower to Bictoin Cash blockchain because it was more profitable for a while. https://cash.coin.dance/blocks
This resulted less hash power on bitcoin main chain and larger intervals between blocks.

Good answer. Didn't know why I didn't think of that... Surprised by how many miners are actively monitoring the profitability and as a result caused an impact on the network

The result will be attack-51. While others work on BCC network the dishonest mining pool (alliance) can work on reorganizing bitcoin main chain


text:
I need to recover BCH
So I had some BTC in a Multibit HD wallet (from 2014) and in October 2017 I transferred it to an Electrum wallet as I had learned that MultibitHD was no longer being supported. I just learned about BCH, and was wondering if there was some way I could gain access to the BCH. Unfortunately, I do not know the seed phrase for my now-empty multibit HD wallet, but I do have everything for the Electrum wallet.
UPDATE: I think I found the seed phrase for the MultibitHD wallet!

To recover your bitcoin cash, you need the access to the private keys of the wallet where your bitcoins were before the fork, which happened at the beginning of August 2017.
If you have withdrawn your bitcoins from your Multibit wallet in October 2017, then you can recover bitcoin cash from your Multibit wallet only. If you have no access to your Multibit wallet anymore, then your bitcoin cash coins are lost. I am sorry.
Update:
In case you still have access to your MB seed, I would recommend you the following simple steps:

Follow this video tutorial to import your MB HD seed into Electrum. Check if there are still any bitcoins on that wallet and withdraw them to another bitcoin wallet. Just to be sure that you won't lose them during the further process. This will not affect your BCH
Follow the official Electrum Recommendation on how to redeem Bitcoin Cash. Short version below:

2.1 Install Electron Cash.
2.2 Import your MB seed into Electron Cash.
2.3 Enjoy your coins and send your promised bitcoin cash reward to :) 3AxGUDoXKtz4w6fba6HPrp8j8XFmSwFCoE

Note1: For a more safer approach, read the recommendation.
Note2: In case you will not be able to import your MB HD seed into Electron Cash, first export another seed or private key from your Electrum wallet, which you created in step 1 and use it instead of the MB HD seed.

So, you can see non-zero balances on your old addresses with BCH blockexplorer like http://blockdozer.com/insight/
But you do not have seed of your old MultibitHD wallet.
Sorry, man. These coins are lost for you and everyone.

See my comment to henk below. I have the seed phrase now.


text:
What’s the Most Frictionless Way to Get Bitcoins into My Wallet?
Total noob in crypto (bought my first few mBTCs a month back) and a proud owner of a new trezor one hardware wallet.
I want to ask what’s the most direct/frictionless way to get the bitcoins into my Trezor?
I have already set up my wallet and did my first transfer into it, it worked. But I had to first sign up and buy the coins via independent reserve, and then transfer it to my wallet address. The independent reserve charged me when I paid my dollars into their account (paid $500, they credited something like $497), then they charged me a few satoshis when I transferred from that exchange to my trezor. Seems very convoluted.
Is there a more direct way, without going through these extra transfers?
Related, the trezor one suite (I am using the Linux Deb version) has an address in it, along with a QR code. Is it safe to keep a copy of this QR code on my phone, and flash it at a bitcoin ATM (or share it via whatsapp to any sender) to receive Bitcoin? Will the ATM send it directly to my trezor one?
Sorry for the noob questions, just trying to figure out the safest and most cost effective ways to navigate the landscape.


text:
Is it possible to have multiple receiving addresses (bc1q) related to the same public key hash
I am trying to get the big picture of the bitcoin entities starting from seed to btc receiving addresses in Hierarchical deterministic wallets without being too technical about hash functions or too specific about each entity. If there are other entities (like PKH) that should be included in this "Entities relationship diagram", please let me know.
So trying to summarize the main concepts as I understood them:

Seed is used to generate Master Private Key (and the mnemonic 12/24 words).
From Master Private Key, many child private keys can be derived. Together with the derived public keys, they represent the "accounts" in the wallet-family-bank (ex. My Savings account, My Spending account, My son's account etc)
In order to receive payments to a specific account, the PHK (double hashed public key) or the short version of it (bc1q...) needs to be provided to the payer.

Now my question is: Is it possible to have multiple receiving addresses (bc1q) related to the same PHK, thus to the same account? As per my understanding from practical experience with Ledger, it is so, but after reading Grokking Bitcoin I got a bit confused.

Read BIP39 then BIP32 then BIP44

Seed is used to generate Master Private Key (and the mnemonic 12/24 words).

Better way to think, convert the entropy to a BIP39 mnemonic. Use that mnemonic plus a (technically optional but no) passphrase as the inputs to produce the BIP39 numeric seed. End of BIP39.
The BIP39 numeric seed is used to derive the extended master private key (aka master xprv or BIP32 root key, corresponds to the root node "m")
That xprv includes the master private key plus the chain code
and from that master xprv is derived every child xprv for a given derivation path in the tree below.
The public key and address are derived from the private key. Only extended keys (with a chain code) can derive children.
Accounts are at level 4, so #7 below
m / purpose' / coin_type' / account' / change / address_index
e.g.  m/44'/0'/7'/change/index

Now my question is: Is it possible to have multiple receiving addresses (bc1q) related to the same PHK, thus to the same account?

Thus, no, 1:1 with the keypair

thanks for the clear answer. I get it.
Now suppose I have a Ledger HD Wallet with 1 account m/44'/0'/0'
I received two payments on two different addresses. What did the ledger do behind the screen to generate the 2 addresses associated to same account and how?
My understanding is that from the extended private key of the account m/44'/0'/0' a new child private key is generated and from it is derived a new child public key and its PKH with the human readable address. So I get smth like this:
m/44'/0'/7'/0/0 --> bc1q ** first address**
m/44'/0'/7'/0/1 --> bc1q second address

the addresses are children of change (internal vs external)

they get derived with incrementing index values

ok, after reading again the BIP 32 39 and 44, I finally got it. Reading the same concept with different words, helped me a lot. BITCOIN is quite a long and beautiful puzzle to solve :) it seems it's neverending.

No, there is a one-to-one relationship between the public key hash and the address. The address is not a short form of the hash, it is actually a human readable form of the hash. The address is an encoding of the hash (and a bit of extra data indicating what to do with it) itself, so there is only one address per public key hash.


text:
Can the difference be determined between two nonces k1, k2, having a linear relationship if the nonce and private key are unknown, i.e., k2 - k1 = n?
My question is related to weak signature vulnerability testing where a hypothetical attacker does not know the values of the nonce or private key itself, but can determine that the nonce 'k1' is increased by private key 'd' to create k2, such that:  k2 = k1+d.
I have searched the stack exchanges and various articles and research papers and have not found a workable solution to this problem.  My own linear algebra is not as strong as I would like it (it's been a number of years), and my attempts have not been successful.  I have used the k2 = k1+1 equations but they do not specifically produce the necessary solution of k2-k1 = d.  Other solutions for solving for k did not work, either.
Is there a way to find the difference of k2-k1 = d, where k1, k2, and d are unknown, where k2 = k1+d, but all other values (r, s, z) are known?
(I have provided the values of d, k1, and k2 for verification of possible solutions, but for the purposes of the question, assume an attacker does not know the values but can spot a relationship).
Let:
N:       Finite field of the secp256k1
d:       Private key
B(x,y):  Public key coordinates
k:       Private key of ECDSA nonce
R(x,y):  Public key coordinates of nonce
r:       R.x integer
s:       signature integer
z:       Hash value of message
N= 0xfffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141
B.x= 0x7697a1f8d952c373285bd441b1716344a61748b48279be15b03538d80e60f901
B.y=
0x431aed8857c99064b316a0c9fb6e7cba3eeade88f44d2c31dee4fa8965740f54
d=
0x8b5069be43c35cbc6530d473a9f102dc57f8f5771982eba3988a7588c6e284a7
k1= 0x248548dcbf7ff4b25134ee12f78fadaa8283025734b905640b8c400580b869e
s1= 0xec585a4b0317e38b75008930607a32224b3d11a8a373ed15da32723b1418f852
z1= 0x12dd3658e48a15016dc1f5af7725ad3a5e7b2d6a52112dbd4a0d6df8772a141a
R1x= 0xb2dcf2acc6fdd003e65e8d9d77ee4d84a9c0cccbac8eb43bdeeecb69a35bab4b
R1y= 0x4a8774fd7ac69caf2d92154324c2280b5f29652b7f458baa28545f1b38d969b8
############################
k2= 0x8d98be4c0fbb5c078a442354d969fdb70021259c8cce7bf9d94339891eee0b
s2= 0x2df1c9e8b3984088ec521c92e9f001b836841e8a14cda6977072d5b23bed171e
z2= 0xdae540b9f50337b2dab2f226a687084e8a1758dbc73505b7ff288b01cc4dae3f
R2x= 0x37df00ccef04bfcf0f2b0da82a685125ddd67b9f66320a9619e6e4df7bd57439
R2y= 0x5ff78c4ef0caf4eff68f938ee475e2e99a7b9be31642eb0e5cb68b30a843b46b
Basic signature equations:
d= (k * s - z) * r^-1 mod N
k= (d * r + z) * s^-1 mod N - or- (d * a + b) mod N
s= (d * r + z) * k^-1 mod N
r= (k * s - z) * d^-1 mod N
z= (k * s) - (d * r) mod N
a= r * s^-1 mod N
b= z * s^-1 mod N
Is there a way to solve this to obtain the value k2-k1=d (k1+d-k1=d) given the equations above, or to solve either k1 or k2,  or would the k generation scheme be relatively secure from attack?

This is trivially broken.
Given:

(1) s1 = (z1 + r1d) / k1
(2) s2 = (z2 + r2d) / k2
(3) k2 = k1 + d

Substituting (3) into (2), eliminating k2:

(4) s2 = (z2 + r2d) / (k1 + d)

Solving (1) for k1:

(5) k1 = (z1 + r1d) / s1

Substituting (5) into (4), eliminating k1:

(6) s2 = (z2 + r2d) / ((z1 + r1d) / s1 + d)

Solving (6) for d:

(7) d = (z2s1 - z1s2) / (s2(r1 + s1) - r2s1)

The right hand side of which only contains public information.
Note that all these variables are elements of F, so the division symbol above refers to multiplication with the modular inverse of the denominator modulo N.

k2= 0x8d98be4c0fbb5c078a442354d969fdb70021259c8cce7bf9d94339891eee0b

I believe you're missing a "45" after those digits.

You are correct, there was a 45 on the end of the hex string, but the secp256k1 calculator would produce the incorrect result when running through the Generator Point while that was appended.  I don't know the particular reason for this, but, deleting the 45 in the calculator produced the correct R(x,y) values.  Thank you for your answer, I can see where my error was in my equations.

Would solution (7) still hold true for k2=k1+d+1, if (7) were modified to:  d=(z2s1-z1s2)/(s2*(r1+S1) - 1-r2*s1-1) mod N?  I am manipulating the parameters on the small curve [N=79, P=67,G=(2,22), d=7, k1=19, r1=38, s1=36, z1=23 | k2=27, r2=17, s2=4, z2= 68].

You cannot fix this. If your nonce values satisfy a simple polynomial expression, it will either be trivial to break, or at least be impossible to prove it is secure. If you want a provably secure technique, your nonces need to be actually unpredictable (e.g. computed using hashes with secret input, or randomly).

I can generate secure random nonces without any issues.   That is not the problem.  I am trying to generate a decent honeypot for my employer so that potential employees can demonstrate their abilities.  Thus, I am playing around with signature equations and I have honestly noticed and developed a lot of different patterns (granted, using the small curve) and equations to match.  Back to your answer, I generate nonces using randomness on secp256k1 calculator, and then sign per protocols.  This question is simply for my further education. Thank you so very much, I really appreciate you.

It appears stackexchange butchered my earlier URL. Retry: https://tinyurl.com/429ye2ye gives the solution for k2 = a*k1 + b*d + c, with known constants a, b, c.

Thank you, that cleared up a lot of confusion on my end.  :-).   I sent you an email on your github email.  My apologies for not getting permission from your first.

Mr. Wuille, is there a way to ask a question that is private between two parties, so that a malicious actor cannot take advantage of the Q&A here on stackexchange? If so, will you direct me to the help section as to how? I cannot seem to find it. If not, is there any way I can ask you a question outside of the public forum? Thank you.

You can email me.

Sent the email to the address posted on github just now.  Thank you.


text:
Is it possible to transfer blockchain data from one machine to another?
I want to run a bitcoin full node at home but the initial sync-up is taking days because of my slow internet connection. Can I download the blockchain data from a machine with faster connection and transfer over to my home machine? Is this possible and if yes, how?

You may want to ask yourself whether the bottleneck is really your connection, or you CPU.  While the blockchain is large, for most people, the bottleneck is the verification of the blocks/transactions rather than network speed.

My CPU usage is about 15-20% while running the application so I was thinking more about the download speeds.

@Regmi It's like I/O bound, rather than network bound. Block validation needs very large amounts of database operations, which often translates on the CPU waiting for disk. You can often improve this by increasing the -dbcache setting.

Increasing the cache size helped quite a lot, thanks!

@PieterWuille The processing of the last blocks, Oct-Nov, took the longest and yes, those depended very much on my hd throughputs. I should have used an SSD but the node is up and running now, thanks.

You can simply copy the entire data directory from an existing bitcoin-core installation on another machine over to your machine, then start the bitcoind daemon.
Be aware you'll want to create a backup of the wallet.dat file from the destination machine so you don't inadvertently overwrite it.  You may also want to confirm the bitcoin.conf settings work for the destination machine.  Specifically, you may want to modify the rpcpassword, memory configuration, etc.
On Windows, the data folder is installed by default into %AppData%\Bitcoin

If you trust that machine, no problem to do it. All blocks in chains (which means "mined ones" (which means confirmed ones) ) are same in all nodes. But if somebody changed some parts of the chain you fetched from "somewhere", they can spoof received/sent transactions which can be observed as "correct" only in your node. To make sure, you can just compare hashes of the blocks (which contains your in / out transactions) in your node with ones in another node (maybe in blockchain.info).


text:
How to display {block number} instead {date value} on Wordpress posts?
Ideally, the goal is to show the block number (ie: 782667) instead of the date value (ie: 26 March 2023 4:44PM CT) on all wordpress posts and pages that currently display it.
There's maybe a script we can add to the functions.php in the wordpress child-theme currently in user?

Do you know which plugin you are using for WordPress?  There are a lot of different plugins.

Ideally we're not using (and we'd prefer to do not use) any plugin, that's why I was asking if a simple function in the functions.php file on the active theme could do the job.
But if there's any plugin that allow the date to be replaced with the blocknumber... could you please share it here?
We did post the same question in the WP stack too and see if a merge of both knowledges can help. WP users do not know about blockchain and vice versa.

Response to this question has been provided here


text:
How to display Woocommerce Products prices in Bitcoin or Satoshis?
In a WooCommerce website, we aim to display the product prices in satoshi/bitcoin based on the current market value. Ideally we don't want to use any existing plugin and we are looking to get the updated market value from Coingecko or any other free API.
There's an easy way to do it? Maybe with a script to be added in the child-theme/functions.php?
Thanks

We ended up building Bitcoin Price Converter, a plugin for WordPress and compatible with WooCommerce
Converts WooCommerce product prices to Bitcoin using exchange rates.

Custom settings available from the admin sidebar menu  Woocommerce > Bitcoin Converter .
Select your preferred denomination (BTC, mBTC, sats)
Select to display fiat price, Bitcoin price or both
Select API price source between CoinGecko, Coindesk or your custom source

For more info:
Repo on GitHub
Demo web site
Donate on Geyser
.


text:
How should I invest $10,000 in crypto-currency?
I have $10,000 which I have decided to invest in crypto-currency.
How should I go about investing this money for maximum profit? E.g. should I invest in a mining setup? Trade the currency? A combination of both?
A few investment parameters and some background on myself is provided below:

Risk sensitivity would be very high. Although I wouldn't want to be stupid with how I go about investing the money either.
I am a technically minded person (I am a mechanical and software engineer and I develop progressive web applications for a job and build computer hardware systems as a hobby).
I would be okay with constructing and maintaining rigs. Although I don't know what coins to mine but I have a basic idea on how to mine them using either ASIC hardware or graphics cards with off the shelf software.
I wouldn't say I have an investment style. I have never really invested in anything before beside putting my money in high interest bank accounts.

@AR71 Risk sensitivity would be very high. Although I wouldn't want to be stupid with how I go about investing the money either. As for being technically minded I am a mechanical and software engineer and I develop progressive web applications for a job and build computer hardware systems as a hobby.

@AR71 I would be okay with constructing and maintaining rigs. I don't know what coins to mine. I have a basic idea on how to mine them using either ASIC hardware or graphics cards with off the shelf software. I wouldn't say I have an investment style. I have never really invested in anything before beside putting my money in high interest bank accounts.

Fact you are asking question means you should not. There is learning curve to have it replied and in different ways. Read and find your personal answer.

Requests for investment advice are off-topic.

@Murch I actually initially posted this question on https://money.stackexchange.com/ however the moderator told me to move it here. Where should I post a question like this if not on the finance or crypto currency sub domains?

I am afraid that I don't know why they told you to move it here. Questions about investing are subjective and outdate quickly, which doesn't match well what we're trying to do with the rest of the site. We've been considering requests for investment advice to be off-topic for a number of years.

I'm going to assume Bitcoin since that is by far the most common cryptocurrency by mindshare for people who are new to the space.
Since you have $10k to invest, you could buy around 5 to 7 bitcoin mining rigs with a capacity of 13 to 16 tera-hashes per second each.  Investing in these mining rigs will give you about 1 year of mining time before the rigs cost more money to run than they can make.  You could make approximately $10,000 to $30,000 profit during that time assuming you have cheap electricity, and buy 5 x 15 TH/sec bitminers.  This path requires far more technical knowledge than a simple purchase of  bitcoin.  To get 5 mining rigs for $10k, you'll need to buy direct from a manufacturer.  Bitcoin mining rig manufacturers typically require a wait of 3 or 4 months until delivery.
If you buy bitcoin, you might lose the entire investment if bitcoin goes to zero; on the other hand you might make 10 times your investment if bitcoin repeats the past 12 months of performance.  If you stay invested for a longer timeframe that risk/return is extended.
There is no simple answer to your question since it depends on your technical knowledge, risk appetite, and timeline.


text:
What are the implications of min_htlc_size in "Pickhardt Payments" on the Lightning Network?
I've read the paper "Optimally Reliable & Cheap Payment Flows on the Lightning Network" by Rene Pickhardt & Stefan Richter and I'm tempted to assume the answer to my question, but I still have some nagging doubts.  So, I'll ask here.
It's clear that setting base fees to zero helps the pathfinding system because base fees don't scale linearly - they're zero when the value transferred is zero, and a fixed non-zero amount when the value transferred is non-zero.
What's not clear is whether setting a minimum HTLC will affect the pathfinding.   Some people have pointed out concerns that with a base fee of zero, the routing fee can be rounded down to zero on small payments.  That'd allow someone to spam a larger payment through as a series of tiny payments at no cost.  Setting a sufficiently sized min HTLC (sufficient for your fee rate so that all routes are at least 1 msat) can avoid this, but obviously at the expense of no longer being able to route arbitrarily small amounts.
My main question is therefore simply: does setting a minimum HTLC value influence the pathfinding in Pickhardt payments at all?
Related question: If yes, in what ways?  Is it scale related?  To get an idea of this: How harmful to the overall system would it be if everyone had small (e.g. 10sat) min HTLC settings?  How about larger (e.g. an unrealistic value like 10000sat)?

Hi Ben, thanks for this interesting question. I was wondering in this context whether Pickhardt Payments would be compatible with rounding the fees up. Perhaps that would satisfy both the needs for the routing algorithm but give a semblance of a base fee.

I don't see a problem there, and I think rounding up fees could be an interesting approach anyway.

In theory, it isn't clear to me yet if finding a min-cost flow is any harder when there are minimum flows. In practice, it's not at all a problem as long as the min HTLC size is small. That's because we calculate the optimum flows with a global minimum flow size (resolution) anyway, and typical values for that are 10k-100ksat. At the moment, the maximum observed min-HTLC-size in the network is 50ksat, and even that isn't a problem. For smaller sizes, the optimal flow will always be a single path anyway.
With regards to effects on the network, that's more speculative, but obviously, if everybody used min-HTLC-size, very small payments wouldn't work anymore.

Note: The resolution that we use finds a flow that is slightly away from the optimal solution. This was sufficient to deliver 0.3679 BTC on mainnet. In edge cases the actual liquidity on the network might be distributed in a way such that the used resolution (stepsize) might prevent us from finding all of the existing liquidity. Imagine 2 channels: The first one has a liquidity of 517k sats the other one has a liquidity of 415k sats. With a step size of 10k sats we could find a flow that delivers 920k sats but we wouldn't be able to find the flow for 932k sats even though it clearly exists.


text:
20% difference between settxfee and actual transaction
I set transaction fee with :
$ bitcoin-cli settxfee 0.00003000

Here is what i observe from block explorer for my transaction :
https://testnet.blockexplorer.com/tx/789d578d22ee07e11ceca9e94aacbf51d36a41223f792035cac26e91830f39c3
Fee Rate    0.000036 BTC per kB

I've tested with different transaction fees for several times; difference always 20% more.
I am on testnet3; using bitcoind version 0.16.0
Edit :
As Adam's answer; now i am more skeptical about byte calculation of blockchain monitoring sites (or whatever they are called).
I've inspected my transaction with getransaction call.
But i still dont see fee per kB is matching with calculation. According to hex parameter; it should be 250 bytes. So total fee should be 0.00000750. Shouldn't i take hex  parameter as size of transaction?
If adam's calculation correct; (seem like it is) then 180 * 0.00003000 / 1024 = 0.00000527. Closest answer so far with %4.5 difference.
Is my calculation correct?
bitcoin-cli gettransaction 789d578d22ee07e11ceca9e94aacbf51d36a41223f792035cac26e91830f39c3
{
  "amount": -0.00005000,
  "fee": -0.00000504,
  "confirmations": 190,
  "blockhash": "0000000000000287661d724d67bed9c1a8861179413e056ee0d16a8b132ad380",
  "blockindex": 12,
  "blocktime": 1527979564,
  "txid": "789d578d22ee07e11ceca9e94aacbf51d36a41223f792035cac26e91830f39c3",
  "walletconflicts": [
  ],
  "time": 1527979516,
  "timereceived": 1527979516,
  "bip125-replaceable": "no",
  "details": [
    {
      "account": "",
      "address": "momFk8PtAZoN4j7Y6jnbucEfc6teUKECEB",
      "category": "send",
      "amount": -0.00005000,
      "vout": 0,
      "fee": -0.00000504,
      "abandoned": false
    }
  ],
  "hex": "020000000001014bb52fb3493598763b53d5ba3cb41f2486cd1ec1b61bae2727d2b9fc74a7454800000000171600148febee6ac4a40d4474c2665c77bbf02350d1bb63feffffff0288130000000000001976a9145a756559278c3795092c07f49c6cf47d360bb1be88acc7c70f000000000017a914849901fe3e003ebf5be1afaf737305158d8865a58702483045022100e23a7e22a21da25f3363bbccbe0bfc94f8044e48236f20b595a456f6bc6d708b02206fc6acdc3636904458cd81770a346a75cc7d5e8ad3868168e387c9cff521b9680121038dc51cc27a4ae2f7ae52503ae4a23862cde107001b59a65639522925527a3e4788291400"
}

It's likely a difference between fee calculation using segwit and non-segwit sizes.

The problem is not with bitcoin core, it's definitely of the block explorer.
Transaction size is 104 + 32 + 34 + 10 = 180 byte, they are saying transaction size 140 byte. I don't understand how did they calculate it.
Blockchain.info saying transactions size is 250
You should not trust them for that matters, try
bitcoin-cli gettransaction 789d578d22ee07e11ceca9e94aacbf51d36a41223f792035cac26e91830f39c3

and see transaction size.

You are right. But still i coundnt reach the exact calculation. I have shared output of gettransaction call.

You can't reach exact calculation because transaction header size isn't always 10 byte and input size as well, take a look at this answer here

Got it! How about bitcoin-cli command output. I cant see 180 but 250 as blockchain.info

where do you see 250 byte?

hex parameter in gettransaction returns 250 byte value

The size that matters is the virtual size (or 1/4th of the weight), which takes the SegWit discount into account), not the byte size.

Thanks for explanations. @PieterWuille can you suggest comprehensive documentation where i can calculate my transaction in detail for different address types. Seriously; i am sick of wandering around Github repos and Stackexchange topics.

Weight is defined in BIP141 as 3*bytes_when_serialized_without_witnesses + bytes_when_serialized_with_witnesses.


text:
Can Bitcoin Core Blockchain initial sync time in BitcoinQT be sped up?
Given the following platform hardware and internet data speed (provided below), what settings are recommended for fastest possible blockchain download and sync?
This is the hardware I'm running:

Note: The storage is completely Bitcoin and Monero blockchain data.

The Bitcoin Core/BitcoinQT client is dreadfully slow.  I've been syncing for a week now.

My settings are:

Is there anything I need to do programming-wise or hardware upgrade?
I have tried all the other answers found on the stackexchanges and none have worked.  I restart the client each time I make a change.

What's your Bitcoin Core version? And are you using an HDD or an SSD? I think you could improve performance a bit more by increasing your db cache to e.g. 8 GB but it probably won't make a huge difference.

I am using an HDD drive and the version of Bitcoin Core that I am using is v22.0.0.  I have already increased the dbcache to 8000 Mb (8 Gb),  to no avail.  If anything, it's performance has slowed down even more.

I don't have reputable benchmark data on hand, but there are quite a few sources (including anecdotal evidence from myself) that indicate that using an SSD can significantly speed up initial block download, but most of those mentions I've seen were about external drives. I assume you're using an internal drive?

Correct, I am using an internal drive.  I complete rebuilt my machine from the motherboard out.  My hardware consists of fast AMD 64-bit processors, Radeon GPU, two 1TB SSD internal SSD drives, and 16GB of RAM.  I switched my OS from Windows to the latest stable Linux Ubuntu (will probably switch to Qubes later).  This resolved the problem.   Bitcoin blockchain downloaded in under 24 hours and SYNCED.  Monero blockchain fully downloaded and synced in a few hours.  Thank you for your guidance!


text:
Why doesn't changing MAX_MONEY change the maximum number of coins?
I've changed MAX_MONEY to decrease the number of coins issued in my altcoin, but it's not working.
Why?

Check where MAX_MONEY is used, and you'll see it does not affect subsidy. It only limits the values being allowed in transactions.

Despite the name, the MAX_MONEY constant is not used in determining the total maximum number of coins.  Rather, it is used only for a sanity check: any transaction output larger than MAX_MONEY is automatically rejected as invalid.  They simply set this new constant equal to the eventual maximum number of Bitcoins, since that is a value that certainly should never be exceeded by any output.  This was part of the fix to an early bug where only the sum of all outputs for a transaction was checked, leading to the possibility that the addition might overflow.
The block subsidy (aka reward) is computed in the
GetBlockSubsidy function in validation.cpp.
CAmount GetBlockSubsidy(int nHeight, const Consensus::Params& consensusParams)
{
    int halvings = nHeight / consensusParams.nSubsidyHalvingInterval;
    // Force block reward to zero when right shift is undefined.
    if (halvings >= 64)
        return 0;

    CAmount nSubsidy = 50 * COIN;
    // Subsidy is cut in half every 210,000 blocks which will occur approximately every 4 years.
    nSubsidy >>= halvings;
    return nSubsidy;
}

Here you can see the initial block reward of 50 coins is hard coded.  The halving interval consensusParams.nSubsidyHalvingInterval is set in chainparams.cpp in the constructors for CMainParams, CTestNetParams and CRegTestParams respectively (the respective default values are 210000, 210000, 150).  Note that MAX_MONEY does not appear.
So in order to change your coin's total money supply, you'll need to do a little math to decide what the initial block reward and halving interval should be, and modify them accordingly.  If you want the reward adjusted by any other mechanism than "continue halving until the reward reaches zero", then you'll have to rewrite the
GetBlockSubsidy function.  (And you should also change MAX_MONEY to an appropriate value that should never be exceeded by any transaction output.)


text:
lncli pendingchannels output: difference between `pending_closing_channels` and `waiting_close_channels`
When plotting lncli pendingchannels, two of the outputs are oddly similar:
pending_closing_channels and waiting_close_channels. I have only seen my closed channels either as the output closedchannels or as waiting_close_channels. I suppose pending_closing_channels refer to the step previous of negotiation for cooperative close? or is it after, during the amount of confirmations needed to wait?

Found an answer for this by asking the slack community:

pending_closing_channels: waiting for confirmation blockheight after cooperative closing and publishing on the blockchain (standard case)
waiting_close_channels: closing request being processed
'pending_force_closing_channels': waiting for confirmation blockheight of unilaterally closing the channel (Commitment Transaction)


text:
Why do self-loops exist in the bitcoin transaction graph?
I have obtained bitcoin data from block explorer and created the transaction graph. The graph consists of millions of nodes and edges.
i wrote a query that creates a relationship from source address to destination address and visualized a portion of the network; approx. 4500 nodes along with 50000 edges. What this means is that I omit the nodes depicting the blocks txs belong to, nodes depicting txs etc.
The nodes in the graph illustrate addresses connected by a relationship " sent_money_to "
There are many self-loops. In several instances a node u has n out-links to node w, while the same node u has n self-loops.
By no means I am an expert on the graph properties or protocol of the bitcoin network.
Any possible explanations? Trivial or non-trivial.
Thanks.

What do you mean by self loops?

You must be confused. Loops are not possible.

Do you mean transactions sending funds that were previously received to an address back to the same address?

I edited the post, I hope it s more insightful now. Possible answer: A transaction with change, source: https://hal.inria.fr/hal-01449078/document . Search for key word: " self ", result number 2.

Possible duplicate of How does change work in a bitcoin transaction?

Those transactions are simply sending change back to the same address as one (or more) of the inputs, it is normal behaviour for certain types of wallet systems.

You are looking at transactions the wrong way. Bitcoin does not actually use addresses and their balances. The protocol has no concept of what an address is, nor does it need to. It also has no idea of an address or an account balance.
Rather how Bitcoin works is with transaction outputs. Transaction outputs are spent as the inputs to other transactions, and created as the outputs of transactions. Addresses are merely a human abstraction to quickly specify the locking conditions (the conditions that need to be met in order to spend) of a transaction output. They are not an identifier, just shorthand for an output script.
Since addresses are just shorthand for an output script rather than identifiers, nodes do not care about the contents of the scripts except when verifying that an input has met the conditions to spend the output it specifies. Other than that, a transaction output can contain whatever the user wants in the script, including invalid scripts. Outputs with such scripts just cannot be spent.
So these self loops exist because people can create a transaction which spends an output with a particular script, and then they create a new output which has the same script. Because nodes don't check the scripts, this is perfectly fine and valid.

So, correct if I am wrong, but the protocol can be used to send anything in a anonymized (or pseudo-anonymized) way. That may include pictures, videos, etc.?
However, I don't understand the last paragraph. Could you elaborate more? If not, can you recommend sources to study the bitcoin protocol/blockchain?

Arbitrary data can be sent only by abusing the fact that output scripts aren't checked for any form of validity in the transaction that creates the output. There isn't anything that explicitly allows or disallows arbitrary data, just that people have realized they can abuse the laxness in order to embed aribitrary data in the blockchain. The point of the last paragraph is just that because you can have anything as a script, having the same script as the output that is being spent is allowed and that causes your "self-loops".


text:
How do I use an issued API key in my python 3 script to bypass the request limiter?
I got an API key from blockchain.info to bypass the request limiter, but I don't know how to use it and I can't find any relevant info online.
 url = "https://blockchain.info/rawblock/" + str(s)
        usock = urlopen(url)
        data = usock.read()

where variable s is the block height.
As the python blockchain.info API states:
https://blockchain.info/rawaddr/$bitcoin_address
Address can be base58 or hash160
Optional limit parameter to show n transactions e.g. &limit=50 (Default: 50, Max: 50)
BUT it also states:
Request limits and API keys
In order to prevent abuse some API methods require an API key approved with some basic contact information and a description of its intended use. Please request an API key here.
The same API key can be used to bypass the request limiter.
However, I don't know how to incorporate it in the function.
Any advice is appreciated!

You need to get an "api_code", then use that when making calls.
You can use their official python client.

Hi Jonathan...I have the api key (i assume it s the same as api code you re referring to).
I have been using the python client and getting the data I want but the issue is that for a certain wallet address I can get only 50 transactions of let s say 800.
My question is how can I incorporate the API key in the python script and download all of the transactions instead of just 50.

@AndreasManousakis It looks like you can just add the api_code as a parameter: https://github.com/blockchain/api-v1-client-python/blob/52ea562f824f04303e75239364e06722bec8620f/blockchain/blockexplorer.py#L11


text:
Is it possible to retrieve input addresses and input values of bitcoin transactions from raw blk.dat files?
I'm developing a script that processes blk.dat files from bitcoin core. So far, I am able to retrieve the following attributes for transaction inputs:
"vin",
"index",
"sigScript",
"sequence",
"witness"

Is it possible to get the address and the value of the input from the blk.dat files?
In case you need more info or the question is not clear, please let me know.

Is it possible to get the address and the value of the input from the blk.dat files?

Yes. It is possible. After all, that's effectively what blockchain explorers do.
I would try to do something like the following

Use the previous transaction id to look up and obtain the contents of the previous transaction.
Use the index to retrieve the specific output of the previous transaction being used as an input in the current transaction
Obtain the value from that output.
Derive an address from the locking script (AKA ScriptPubKey) of that output.

Potentially useful:

What do the different .dat files contain?
How to parse transaction script to address the correct way?
Can you break down what data is encoded into a bech32 address?
How to get addresses from scriptPubKey in segwit transactions?
In what format does a block store the transaction data?
What are the parts of a Bitcoin transaction in segwit format?

I do not really have a full answer, but maybe to add to RGB's: Yes, but a transaction itself only identifies the outpoint of the UTXO it spends in an input. So, you need to find that previous transaction to read the output data there. It would probably help to run the full node with txindex to that end.

It's a bit of a mystery to me how I can look up a specific transaction from raw data. I can use the txid of the input with a web api and get the transaction contents but this has proved to be a bit slow due to exceptions raised from too many requests to the server.

The txindex configuration option will cause Bitcoin Core to maintain an index of txids. You can use its RPC API - maybe gettxout (which I guess would work for UTXOs without txindex)

E.g. in Python with https://github.com/alecalve/python-bitcoin-blockchain-parser:
#!/usr/bin/env python3

# stdlib
import os
import sys

# Third party
from bitcoin.core.script import *
import blockchain_parser.blockchain

datadir = '/home/ciro/snap/bitcoin-core/common/.bitcoin/blocks'
blockchain = blockchain_parser.blockchain.Blockchain(datadir)
for block in blockchain.get_ordered_blocks(
    os.path.join(datadir, 'index'),
    cache='cache.pkl',
):
    for txno, tx in enumerate(block.transactions):
        print(tx.txid)
        print('inputs')
        for inp in tx.inputs:
            print(inp.sequence_number)
            print(inp.witnesses)
            print(inp.script)
            print(inp.transaction_index)

requirements.txt
#git+https://github.com/alecalve/python-bitcoin-blockchain-parser.git@c06f420995b345c9a193c8be6e0916eb70335863
git+https://github.com/cirosantilli/python-bitcoin-blockchain-parser.git@plyvel-1.5.1
requests==2.31.0

I needed a small patch on my fork: https://github.com/alecalve/python-bitcoin-blockchain-parser/pull/112
Sample output:
tx 63f9df910b54c60a391e0842cd485c455f9c7fb7c7c79eb07706b2c0ef7ebe93
4294967295
[]
Script(ffff001d 8101)
4294967295
tx 58d089afb985c772232e17dec60c9eed5c281d985cace881fb1f6b34c43e7500
4294967295
[]
Script(ffff001d 5e)
4294967295

At https://github.com/cirosantilli/bitcoin-strings-with-txids/blob/b560fc0bcfb6f0f64a256bdcd15b48810262dc67/main.py I used some more of their functionality for my purposes which may serve as further reference.
I haven't researched a lot before choosing this one, but it kind of works. Sometimes I wonder if there's a faster one though in some compiled language, but this one gets there eventually.


text:
LTC transaction fees and p2pool
I'm experimenting with LTC p2pool mining. However, it generates lots of micropayments (0.02 LTC). Such small payments generate big transaction fees (in latest litecoin-qt from github). Example: 0.6 LTC transaction fee for sending 0.3 LTC.
How can I avoid such big transaction fees?

Transaction fees are calculated according to priority and transaction size.
Priority is a function of age and value of a single input in the wallet. The larger the value and the older the input age the higher the priority will be. Input value and input bytesize are static values. Input age is a variable that increases with time. So, in short, you can decrease your fees by allowing the coins to "age" (aka wait).
priority = (input_value * input_age)/input_bytesize
priority >57600000 is free (not accounting for transaction size)
Transaction size is going to incur fees anytime 5 or more inputs are included in an output. The only way to avoid incurring transaction size fees is to create outputs containing 4 or less inputs. I am not aware of any coin control extensions for the litecoin client so achieving this will likely prove very difficult for very small inputs.
The litecoin client is expected to receive a significant update sometime in May 2013 that may address some of the transaction fee concerns in the current client.

Hi! What are the units in that equation? Value is... whole coins? And age is in... seconds? Cheers!

Age is counted in blocks

I have used this method to reduce my fess to 1.5% . It basically joins a bunch of small p2pool payments into one large payment with 0 transaction fees. But the catch is that there is a 1.5% developer donation, still much cheaper than any other alternative.
https://forum.litecoin.net/index.php/topic,3870.0.html

It looks like there's a bug in your code: http://ltc.block-explorer.com/tx/7ab6fc85cd656df7f43167ae3c79585ceaf970faf9c7084b19bccbe2a375c672 In fact, it kinda looks like it lost a decimal place.


text:
electrum windows client, standalone executable versus windows installer
From the download page, http://electrum.org/download.html

What is the difference between the two windows versions? standalone executable and the windows installer version?
What are the advantages and disadvantages of each, and how should I choose which one I should use?

The Windows Installer will install the executable in "Program Files", create shortcuts and other registry entries. The Standalone Executable is exactly that, just an executable to run the Electrum program.
Benefits of Windows Installer would be ease-of-use for those most familiar with installed programs, plus it will be available to multiple users (whether that is desired or not, is another question). Benefit of Standalone is don't create registry entires (cleaner) and don't require raised privileges (as not installing globally).

In either case - user-specific data will be saved in C:\Users\USERNAME\AppData\Roaming\Electrum
There are some more details in this blog post

They also have portable now. It basically stores stuff in it's own folder instead of the local folders.

Try this link - simple explanations and instructions:
http://www.bit-buddy.com/electrum-setup.html

while your link answers the question it is good to give a short overview of the link's contents in case the page expires and to save people time to assess whether it is useful to them.


text:
The nmap script bitcoin-getaddr often does not return a response
For a project I need to write a scraper that sends a GetAddr request to all Bitcoin nodes. I determine the reachable nodes using the bitnodes.io API. Then I go through all the IPs and send a request using the python-nmap library and the bitcoin-getaddr script from nmap.
The code to do this looks like the following:
def send_get_addr_requests(ip, port):
    ip_addr = ipaddress.ip_address(ip)
    # check if it is an ipv6 address and fix arguments in this case
    arguments_get_addr = '--script bitcoin-getaddr -d -Pn'
    if isinstance(ip_addr, ipaddress.IPv6Address):
        arguments_get_addr += ' -6'
    # send get_addr request to the node
    nm = nmap.PortScanner()
    response_get_addr = nm.scan(ip, port, arguments=arguments_get_addr)
    return response_get_addr

However, as I have found, the nmap script (especially the bitcoin.lua file) is relatively unreliable and in most cases does not return a result on the request. I have already modified the GetNodes function as follows:
 getNodes = function(self)
    local req = Request.GetAddr:new(
      self.host, self.port, self.lhost, self.lport
    )
    local status, err = self.socket:send(tostring(req))
    if ( not(status) ) then
      return false, "Failed to send \"GetAddr\" request to server"
    end

    local status, response = Response.recvPacket(self.socket, self.version)
    local all_addrs = {}
    -- limit is not usable because it is unclear how many packets are returned
    -- thats why we take a timer
    -- local limit = 20
    local startTime = os.time()
    local endTime = startTime + 60  -- wait 60 seconds max
    -- Usually sends an addr response with 1 address,
    -- then some other stuff like getheaders or ping,
    -- then one with hundreds of addrs.
    while #all_addrs <= 1 and os.time() < endTime and status do
      -- limit = limit - 1
      status, response = Response.recvPacket(self.socket, self.version)
      if status and response.cmd == "addr" then
        for _, addr in ipairs(response.addresses) do
          all_addrs[#all_addrs+1] = addr
        end
      end
    end
    return #all_addrs > 0, all_addrs
  end,

I added a timer and removed the limit. Since the requested nodes send many packets before the GetAddr response, this approach works much better. I also set the replaced version to 70015 and adjusted the ping/pong messages of the script to the newest implementation.
However, in many cases (about 2000 out of 7000) I still do not receive a GetAddr response. The function Response.recvPacket(self.socket, self.version) from the bitcoin.lua file gets at some point only a status = false and a response = EOF ...
So my question is if this is normal or how I can further customize the script to get a more reliable response? When I manually type in the terminal "nmap -Pn -d -script bitcoin-getaddr -p 8333 " I get a response from time to time, but not reliable and rather rare. So the requested nodes don't block the GetAddr requests, which makes the whole thing even more complicated to understand.
Thanks in advance for any help!
Here is another Wireshark screenshot. I executed the command several times in a row (14x) in the terminal. In the last attempt I finally got the GetAddr response. The [unknown command] packets are "sendheaders" and "sendcmpct" packets (and in the last attempt "feefilter").

Have you used a network sniffer, such as wireshark, to examine to detailed packet by packet conversation for a successful vs unsuccessful query?

Yes, I have actually done that. I have added a Wireshark screenshot to the question. There I simply executed the command several times in a row (14x) in the terminal. In the last attempt I finally got the GetAddr response.

Well, that didn't shed any light on the matter, I'm out of ideas, sorry.

Okay, thanks for thinking about it anyway!


text:
Running Bitcoin SPV Node
How can I run a Bitcoin SPV node? Am I able to run SPV nodes with Bitcoin core? If so, can I use JSON-RPC by sending requests to full nodes? I couldn't find a good way to retrieve transaction data other than installing a full node.

Hi @xor, these are some good questions, but I'd suggest breaking these questions down into multiple posts. The more detailed and specific you can be in each question, the better

@josie Thanks for the tip! I will do definitely do that.

How can I run a Bitcoin SPV node?

You run Bitcoin SPV software. They're often called "lightweight clients" or "lightweight wallets". Several mobile wallets implement SPV validation, as does Electrum (though note that Electrum is not technically a node; it connects to Electrum servers, not to P2P nodes).

Am I able to run SPV nodes with Bitcoin core?

No, Bitcoin Core only implements a fully-validation node.

If so, can I use JSON-RPC by sending requests to full nodes?

No, Bitcoin Core's JSON-RPC interface is designed for the owner/operator of the node to connect to their own trusted node. It's not designed to be exposed to third parties.
Furthermore, the JSON-RPC interface is specific to Bitcoin Core. Other software will have different ways of communicating or interacting with it.

I couldn't find a good way to retrieve transaction data other than installing a full node.

That's pretty much by design. Bitcoin is defined to minimize trust; if you want validated data there is really no way but validating it yourself using a full node.
The alternative is trusting other parties to do it for you. If that security model is acceptable to you, with the security and privacy risks that entails, you can e.g. use block explorer websites, or hosted wallets.

How can I retrieve transactions data I want with hosted wallets?

You don't. The wallet will tell you what your balance is.

So in order to retrieve transaction data, I have to either run full node or use block explorers.

Any node can request blocks from their peers; it's just that lightweight clients generally don't fetch (all) blocks, because they're specifically designed to be lightweight. So yes, I'd say the best way of getting transaction data, if you want all transactions, is to just run a node yourself.

To be more clear with my question, with the lightweight client, can I send a request to full node "I want all the transaction data of certain blocks"?

@xor My comment was abstract: any node software can indeed ask other nodes for whatever it wants. If you write a simple P2P node yourself, it can do that. But existing lightweight node software you find typically won't do that, as its purpose is exactly not needing to download everything, and just act as a wallet for the users, downloading only the relevant parts. If you want to just have all the block data, you're much better off running an existing full node.

I see thank you so much for the help. Last question, if I want to write a simple P2P node myself, what are some resources I can reference to? Do you have any recommended resources?


text:
BTC not confirmed after 6+hrs
Can anyone help with why a simple transfer of BTC from BLOCKCHAIN to COINBASE wont confirm?  Fee paid and debited from my BLOCKCHAIN ac
2c4a037812e32a02ca12f4eb38577c11d65900872b5a795516c59f300afdbb9b
....and how I can rectify it!

Would be helpful if you provided the transaction ID.

Yupp, 40sat/B transaction fee is way too low. Check out the link mentioned by Nate Eldredge above this comment

Your fee is 40 sat/b so it may take at least 48 blocks ( ~480 min) To confirm.


text:
What is the python equivalent of this php code $values = unpack("C*", pack("H*", $scriptpubkey));
I'm trying to convert this php code:
$scriptpubkey = '4104678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5fac'
$values = unpack("C*", pack("H*", $scriptpubkey));

to python. So far, i have done the following:
PHP: pack("H*", $scriptpubkey) -> Python: binascii.unhexlify(scriptpubkey)

I am not sure if it's correct though. If someone could explain the pack, unpack functions and what the characters "C*", "H*" do, it would be much appreciated.

If someone could explain the pack, unpack functions

The PHP manual covers pack and unpack

Pack given arguments into a binary string according to format.

Unpacks from a binary string into an array according to the given format.

So I would expect the code to produce a character array. This might be useful in PHP but not useful in Python.
The format characters are explained under pack

Code
Description

a
NUL-padded string

A
SPACE-padded string

h
Hex string, low nibble first

H
Hex string, high nibble first

c
signed char

C
unsigned char

s
signed short (always 16 bit, machine byte order)

S
unsigned short (always 16 bit, machine byte order)

etc
However I would avoid performing a statement by statement or function by function translation and try to understand what the whole section of code is trying to achieve and then work out how best to accomplish that using the standard libraries available in Python. I find libraries and idioms vary greatly from language to language and an overly literal translation can end up being fragile, inefficient and incomprehensible. Your experience may differ though.
Note that you are asking about PHP and Python rather than about Bitcoin specifics. This question might do better on stackoverflow.com.

These pack/unpack php functions seem to be exactly as the identically named perl functions.
So pack("H*", ...)  converts an ascii string to an byte-string.
And unpack("C*", ...)  converts a byte-string to an array of byte values.
In python, pack("H*", $scriptpubkey)  would be:  binascii.a2b_hex(scriptpubkey)
The second step is not needed in python, since byte-arrays are already lists of byte values in python. If you access a single element: a[0] you would get the integer value  0x41  in this case. While accessing a subrange: a[1:] would get you a byte-array containing the pubkey.
Then another remark, if your variable contains a hexadecimal representation of your script, it might be clearer to indicate that in the variablename, for instance by naming it hex_scriptpubkey.


text:
Question Regarding Send_to_many function of blockchain wallet api
I am using send to many function of blockchain wallet api and i am facing issue.
when upto 20 transaction done then after i get error in terminal that
1538652825520 - error: ERR_PUSHTX
1538652826930 - error: ERR_PUSHTX
and until that upto 20 transactions get 1 confirmation on block i am unable to send any other transaction from that wallet and it's specific address.
i want to use sent to many function for sending unlimited transaction from one wallet to my site users bitcoin addresses when they submit payment request from their account.
kindly help me regarding this.
Thanks

Is your fee too low? See https://github.com/blockchain/service-my-wallet-v3/issues/105

thank you very much it was really help full for me.

but i still want to ask one question, what if fee is set to normal and still after 20 transactions it's show
1538652826930 - error: ERR_PUSHTX

This isn't really answering the question but it is a suggestion. If you're sending out that many transactions, wouldn't it be easier (and cheaper) to combine your inputs and outputs into a single transaction? Instead of spamming a bunch of smaller TX's, you could create 1 or more large TX's with many more inputs and outputs.

What you need to manage is the fee rate. Bitcoin core nodes will not relay transactions (or include them in blocks) below the minimum fee rate of 1000 satoshi/kB (-minrelaytxfee), so if you keep increasing the transaction size, but not the fee, you risk not getting relayed or even mined (-blockmintxfee), since miners typically maximize the fee rate in the block being mined.

But when i am adding fee in satoshi still it's taking too much time to get 1 confirmation. which is required to process further transaction from 1 address after 24 transactions are pending.

https://bitcoin.stackexchange.com/questions/9046/why-is-my-transaction-not-getting-confirmed-and-what-can-i-do-about-it