text:
Are there btc in this multibit wallet
I have several multibit wallets I put in storage years ago, and I'd like to check if there's btc in them . I see wallet IDs for 'receive address'  in the .info files - if those show 0btc does this mean the wallet is empty? Or is the receive addr not where btc would be stored?
I'd like to verify there's actually something in there before banging my head against how to retrieve from what looks like an unsupported wallet type.
The dirs look like:
wallet1.info
wallet1.wallet
wallet1-data
    key-backup
    wallet-backup
    rolling-backup
    wallet-unenc-backup

and the .info files look like (for wallet1 in this case):
multiBit.info,1
walletVersion,3
receive,1GFM...RGdp,
property,walletDescription,wallet1
property,walletBackupFile,C%3A%5CUsers%5Cjeremy%5CDocuments%5C%5Cwallet1-data%5Crolling-backup%5Cwallet1-20131201123906.wallet
property,receiveAddress,1GFM...RGdp
property,receiveLabel,

Is your wallet encrypted? If not the wallet file is plain text, if yes you'll have decrypt it first.

There is a receive address in the (unencrypted) wallet1.info  - my question was whether this revc address  ( whose balance can be checked on a blockchain explorer) necessarily reflects the wallet contents.

First dump your keys using the instructions here
Then open a new Electrum wallet, ask it to "Import private keys", and paste your private keys there.
Then you will be able to spend from there if you have coins.

thanks - tho I was actually after whether I can determine wallet contents/address before actually extracting, by use of the plaintext receive addr. in the .info file

I'm not very familiar with Multibit's wallet format, so I don't know a shortcut.

I added the .info file contents, my question is basically whether that receive wallet addr necessarily reflects the wallet contents


text:
What is a millisatoshi? I thought satoshi is the smallest unit?
I encountered 'millisatoshi' (msat) while browsing Lightning Network..., see screenshoot below:

And subsequently I found it mentioned here:
https://github.com/ElementsProject/lightning/blob/master/README.md#sending-and-receiving-payments
Obviously, I am curious... I thought satoshi is the smallest unit?

It's not a mistake, it's literally a thousandth of a satoshi. A satoshi is the smallest unit for bitcoin, but lightning can transact with even smaller units while channels are open. The amount is rounded down to the nearest satoshi when the channel is closed and broadcast to the blockchain to adhere to bitcoin's limit.

Since the satoshis commitment transactions are always necesserily atomic, are fractional satosh just some kind of unenforcable gentlemens agreement?

Now, what happens if the fee is very low? Let's say the base fee is zero and the fee rate is 25 PPM, and you route 10,000 satoshis. The fee would be 0.25 satoshis. Since the smallest amount in Bitcoin is 1 satoshi, you can't earn and be paid 0.25 satoshis (or 250 milli satoshis). However, the money is not lost. If you were to route four such transactions through the same channel, the partial satoshi earnings will add up, and you will earn a single 1 satoshi (1,000 milli satoshis) eventually.

https://lightningnetwork.plus/posts/568

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Bitcoin transaction script debugging strategy - how to debug 'invalid script' errors
I've been playing around with DLC transactions but I can't seem to create a valid CET transaction to conclude simple all or nothing bets. Further details below on my specific case, though this question also relates to methods for debugging invalid bitcoin transaction scripts in general.
In this example, when attempting to broadcast an invalid transaction, the most verbose error I am able to obtain is as follows:
Error validating transaction: Error running script for input 0 referencing f8736021c8144f6991cd05a6702197f307565b11e9b7abaaa19b0348d3d6da16 at 0: Script was NOT verified successfully..

...which isn't very helpful, since it only states that the transaction script is invalid. It does not divulge specifically what part of the script/transaction is invalid.
Having spent a lot of time debugging my own DLC logic, I am now interested - how can I approach the debugging of an invalid Bitcoin transaction itself in search for clues on what has gone wrong during transaction creation?
Specific (invalid) transaction in question here:
{
    "addresses": [
        "bc1qewcpdlhklv427mtxefvzym800mqrdvkapym2kmqleetxwa08w3gq7y6n6c",
        "bc1ql7np4a4elfkv6ae43wrfaup6c0rmvyxkqx88aw"
    ],
    "block_height": -1,
    "block_index": -1,
    "confirmations": 0,
    "double_spend": false,
    "fees": 170,
    "hash": "a740198694a1a2e8e1850970cf25e3ef795c8b5fb8da0732375644f8d5ad58c8",
    "inputs": [
        {
            "addresses": [
                "bc1qewcpdlhklv427mtxefvzym800mqrdvkapym2kmqleetxwa08w3gq7y6n6c"
            ],
            "age": 711675,
            "output_index": 0,
            "output_value": 1370,
            "prev_hash": "f8736021c8144f6991cd05a6702197f307565b11e9b7abaaa19b0348d3d6da16",
            "script_type": "pay-to-witness-script-hash",
            "sequence": 4294967295,
            "witness": [
                "",
                "304502210097aa3e394986bcc74178c16d7fee160c78e2f6b3389995d2f930162c3da1448b0220585d29c92dd4f5878ddd2da0f6cc7a50825eab20187c5e6de41a8f54c054cf9101",
                "304402206744c1ff93471ed94b4b9fe61c3e84c4f49554d15245c590245c6eba200f26c8022026d3154d08db41db10765aad4ba76dfdb8f91c0f37a5ca324c292ec5fb91b34b01",
                "5221033e7464943ac55e9f2a8fdc30325df313270fd15ef92744327241ba18a72c506d21035b00cdb5056b7c644f17dde11067e91384c7b9ed97c16d7feb28df569df1371e52ae"
            ]
        }
    ],
    "outputs": [
        {
            "addresses": [
                "bc1ql7np4a4elfkv6ae43wrfaup6c0rmvyxkqx88aw"
            ],
            "script": "0014ffa61af6b9fa6ccd77358b869ef03ac3c7b610d6",
            "script_type": "pay-to-witness-pubkey-hash",
            "value": 1200
        }
    ],
    "preference": "low",
    "received": "2021-12-20T23:08:32.284953769Z",
    "relayed_by": "44.193.204.249",
    "size": 303,
    "total": 1200,
    "ver": 2,
    "vin_sz": 1,
    "vout_sz": 1,
    "vsize": 138
}

and associated txhex:
0200000000010116dad6d348039ba1aaabb7e9115b5607f3972170a605cd91694f14c8216073f80000000000ffffffff01b004000000000000160014ffa61af6b9fa6ccd77358b869ef03ac3c7b610d6040048304502210097aa3e394986bcc74178c16d7fee160c78e2f6b3389995d2f930162c3da1448b0220585d29c92dd4f5878ddd2da0f6cc7a50825eab20187c5e6de41a8f54c054cf910147304402206744c1ff93471ed94b4b9fe61c3e84c4f49554d15245c590245c6eba200f26c8022026d3154d08db41db10765aad4ba76dfdb8f91c0f37a5ca324c292ec5fb91b34b01475221033e7464943ac55e9f2a8fdc30325df313270fd15ef92744327241ba18a72c506d21035b00cdb5056b7c644f17dde11067e91384c7b9ed97c16d7feb28df569df1371e52ae00000000

Thanks to @meshcollider's pointer, I used kallewoof's script debugging tool with the above txhex setting txin as the txhex for the associated funding transaction that created the input utxo of the CET transaction:
btcdeb --tx=0200000000010116dad6d348039ba1aaabb7e9115b5607f3972170a605cd91694f14c8216073f80000000000ffffffff01b004000000000000160014b007f44f21a153706746e09e508f43193315996b0400473044022053beeedca8004419528365422d7f41bf1c458696a7a5ca045685e1fa23114f0202200dd522014dae15d3edb28d25a7190d262ec3a4b6a4aa5aaac56de8c9cdb3b0b50147304402202de278ef40eaa143704950842cde5ac0119056ec1302413e871c3d202bb0182b022057bafc3eb34f8d8bfd884d812436a78c6aea7931589c7131b5dd80e686040ac701475221033e7464943ac55e9f2a8fdc30325df313270fd15ef92744327241ba18a72c506d21035b00cdb5056b7c644f17dde11067e91384c7b9ed97c16d7feb28df569df1371e52ae00000000 --txin=0200000000010433d8a2db6a4fc1ac1ae69b2c0c315c002195d9e6ba6a485f52601322170469400000000000ffffffff3c0f4ab770449808b9fd3fe430221bde77ed979002ff5b8f86664f12abeb7c560000000000ffffffff33d8a2db6a4fc1ac1ae69b2c0c315c002195d9e6ba6a485f52601322170469400100000000ffffffff3c0f4ab770449808b9fd3fe430221bde77ed979002ff5b8f86664f12abeb7c560100000000ffffffff035a05000000000000220020cbb016fef6fb2aaf6d66ca58226cef7ec036b2dd0936ab6c1fce566775e77450d10200000000000016001411b853839dd15f691f390611c2ebd7b7b2f51fadd10200000000000016001409973c8035089d42c2ab1328aa00dc1998a1f9bc0247304402205db089047b17d5769e34f541cba8b35cff9b8fddceac37ec4b0296b50176b3980220435932954b97688973cdc058590e77e4b9124e79ebbbceb5b78e4677dd198b82012102e01b2ba3f457708bb6317fb9d5f5a1d8a7e8c6331c0fb6d28b3c00d8930ad1fc02473044022014182c1f10edf4c44601ce4d859490544f9a3fd8abe99dd7d21f413e2a22d01e022003118089e598f7a1a989634ba61356e64a94df70b9b28ff8a953cb11cf0737b1012103a1e0a48f62065f043d71ac1565861ca49ecf0a4b9858a227a2453bfa7a3198550247304402206499708afd271848f24eef30b589834197a709123208152c93cc853fd600c12302207e3a7e0b1957f1083fb1b61a8548ffebdb71908906e73c5b842e2156594c393c012103d5d4b6f0a52331ea5f7a713104610ce946038a4aa8fcad6a5905ca0cb6c0792e02473044022063fb8c225317258bf9b6715a1117d4faeb43330cec79ed0fee97f286ddf01eff02200f810cd6080331dd82b44827f29d3ef8499af3c4c8f27ef58c66f0b1bf37d11b0121035482f5bf4ab770f9f88c6db13406a9c624aae1adff4b031f7fa1addb966f83ee00000000

VerifyECDSASignature() failed (see below). What does this tell me specifically? How can I use the output below to find the problem?
btcdeb> step
stack has 7 entries [require 1]
stack has 7 entries [require 4]
stack has 7 entries [require 7]
scriptCode = 5221033e7464943ac55e9f2a8fdc30325df313270fd15ef92744327241ba18a72c506d21035b00cdb5056b7c644f17dde11067e91384c7b9ed97c16d7feb28df569df1371e52ae
looping for multisig
loop: sigs = 2, keys = 2
- got sig 304402202de278ef40eaa143704950842cde5ac0119056ec1302413e871c3d202bb0182b022057bafc3eb34f8d8bfd884d812436a78c6aea7931589c7131b5dd80e686040ac701
- got key 035b00cdb5056b7c644f17dde11067e91384c7b9ed97c16d7feb28df569df1371e
GenericTransactionSignatureChecker::CheckECDSASignature(71 len sig, 33 len pubkey, sigversion=1)
  sig         = 304402202de278ef40eaa143704950842cde5ac0119056ec1302413e871c3d202bb0182b022057bafc3eb34f8d8bfd884d812436a78c6aea7931589c7131b5dd80e686040ac701
  pub key     = 035b00cdb5056b7c644f17dde11067e91384c7b9ed97c16d7feb28df569df1371e
  script code = 5221033e7464943ac55e9f2a8fdc30325df313270fd15ef92744327241ba18a72c506d21035b00cdb5056b7c644f17dde11067e91384c7b9ed97c16d7feb28df569df1371e52ae
  hash type   = 01 (SIGHASH_ALL)
SignatureHash(nIn=0, nHashType=01, amount=1370)
- sigversion == SIGVERSION_WITNESS_V0
  sighash     = 9b315f4e8bf72ff16827a12d832820592af81e62d3764a50ce09715a2e398bff
  pubkey.VerifyECDSASignature(sig=304402202de278ef40eaa143704950842cde5ac0119056ec1302413e871c3d202bb0182b022057bafc3eb34f8d8bfd884d812436a78c6aea7931589c7131b5dd80e686040ac7, sighash=9b315f4e8bf72ff16827a12d832820592af81e62d3764a50ce09715a2e398bff):
  result: success
- sig check succeeded
loop: sigs = 1, keys = 1
- got sig 3044022053beeedca8004419528365422d7f41bf1c458696a7a5ca045685e1fa23114f0202200dd522014dae15d3edb28d25a7190d262ec3a4b6a4aa5aaac56de8c9cdb3b0b501
- got key 033e7464943ac55e9f2a8fdc30325df313270fd15ef92744327241ba18a72c506d
GenericTransactionSignatureChecker::CheckECDSASignature(71 len sig, 33 len pubkey, sigversion=1)
  sig         = 3044022053beeedca8004419528365422d7f41bf1c458696a7a5ca045685e1fa23114f0202200dd522014dae15d3edb28d25a7190d262ec3a4b6a4aa5aaac56de8c9cdb3b0b501
  pub key     = 033e7464943ac55e9f2a8fdc30325df313270fd15ef92744327241ba18a72c506d
  script code = 5221033e7464943ac55e9f2a8fdc30325df313270fd15ef92744327241ba18a72c506d21035b00cdb5056b7c644f17dde11067e91384c7b9ed97c16d7feb28df569df1371e52ae
  hash type   = 01 (SIGHASH_ALL)
SignatureHash(nIn=0, nHashType=01, amount=1370)
- sigversion == SIGVERSION_WITNESS_V0
  sighash     = 9b315f4e8bf72ff16827a12d832820592af81e62d3764a50ce09715a2e398bff
  pubkey.VerifyECDSASignature(sig=3044022053beeedca8004419528365422d7f41bf1c458696a7a5ca045685e1fa23114f0202200dd522014dae15d3edb28d25a7190d262ec3a4b6a4aa5aaac56de8c9cdb3b0b5, sighash=9b315f4e8bf72ff16827a12d832820592af81e62d3764a50ce09715a2e398bff):
  result: FAILURE
- failed: VerifyECDSASignature() failed
- sig check failed
loop ended in failure state
        <> POP  stack
        <> POP  stack
        <> POP  stack
        <> POP  stack
error: Signature must be zero for failed CHECK(MULTI)SIG operation

This may be helpful: https://github.com/bitcoin-core/btcdeb

Very cool tool, thanks for the pointer

@meshcollider I tried the btcdeb tool with the contract ending transaction hex as tx, and the funding transaction hex as txin (please correct me if this is wrong). I included the command and associated output as an edit to my original post above, which shows a validation error as expected. How would I go about understanding why VerifyECDSASignature() failed in this instance?

As mentioned in my comment, for answer completeness, a great way of debugging Bitcoin script is using the script debugger.
VerifyECDSASignature passed on the first of the two signatures in the multisig, but failed on the second signature. Unfortunately it is difficult to know exactly why - it is likely the transaction being signed has changed slightly from when it was created.
How exactly did you generate the signatures?

I generated the CET signatures (as well as the rest of the DLC transactions) using the cfd-js and cfd-dlc-js libraries - https://github.com/p2pderivatives/cfd-dlc-js. I'd assume that in this case the correct signature was that of the local counterparty, who constructed the CET, while the incorrect signature is associated with the adaptor signatures that should have allowed the local counterparty to unilaterally settle the contract (by making use of the signed oracle attestation).

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Can non mining full nodes prevent 51% attack? Does it actually strengthen the network? I am confused of its real purpose
From what I understand, miners secure the bitcoin network by making it very-very expensive to do a 51% attack or double spend bitcoins through the sheer amount of hash power needed to overwhelm the network.
That implies, if the hash power happens to be very low (for whatever reason), an entity could theoretically mount this attack and thus reverse transactions or double spend the coins.
So what is the role of non-mining full nodes then? Does not seems to serve any real purpose to me?

Possible duplicate of What is the difference between a miner and a full node?

@JBaczuk None of the answers there answers my question. Title edited to avoid being labelled 'duplicate'

Non-mining full nodes cannot prevent a 51% attack, but they are essential in preventing other attacks.
In particular, full nodes verify that the chain produced by miners is valid. This means that  no coins are transferred without proper authorization from their owner, that no new coins are being created out of thin air (except for those permitted by the inflation schedule), and a few other things.
Full nodes are what keeps miners honest. To anyone who verifies incoming transactions themselves using a full node ("economically relevant full node"), it guarantees that the chain is valid. Miners have no way to cheat those (apart from a 51% attack, see further), and as a result, they have no incentive to produce blocks that violate the rules, as such blocks will be dropped on the floor and ignored by full nodes. To anyone not running a node themselves, the knowledge that a sufficient number of other independent parties are running full nodes helps, because hopefully those parties are important enough that miners wouldn't want to waste their money creating an invalid block they would detect.
Bitcoin's security is based on auditability, not trust. You know the chain, in its entirety, is valid, because you're able to independently verify that it is.
Unfortunately, there is no way to verify which of two conflicting (but otherwise valid) transactions is the "real" one without a central clearinghouse that blesses one of the two. This is called the double spending problem, and it is the reason why we need miners: a decentralized clearinghouse that anyone with the right hardware, even anonymously, can join. Producing blocks costs money for them, and they're only paid if the network of full nodes accept their blocks. This is why they're incentivized (but not forced) to produce blocks that satisfy the rules, and build on top of each other's blocks.
But it is important to see that the 51% attack is the exception here. For almost every rule in Bitcoin, full nodes verify everything, and there is no way they can be fooled. The only thing that cannot be verified independently is double spends, which leads to a 51% attack if exploited (but the theory is that this would be expensive for miners so not economical; furthermore, if miners aren't doing their job well, others are incentivized to become a miner themselves).

Thanks for this detailed answer. Most answers I found simply says "miners keep the network secure but full nodes enforce the rules..." that statement alone confuses the hell out of me...

May I add the following:
An additional full-node does not automatically equal more Security
Full-nodes may validate, but if they are not validating transactions received by the node operator, they do not offer any security, in fact, they decrease it by adding latency to the network.
Only a full-node validating transactions received by a person who is offering economic value (Goods/Services etc.) in return for the received Bitcoin is securing the system, as he/she is part of the "economic majority" who is actively deciding which definition of Bitcoin is "worth" trading value for. This person has the power to reject trade (Bitcoin for Value) with his/her full-node, and therefore is protecting/enforcing the validation rules agreed upon by the economic majority (the group of people who are in agreement (consensus) what "Bitcoin definition" to trade for economic value.
So it is important to note that simply adding nodes to the network, without actively using them to validate incoming transactions, (for which you paid for with goods, services) is not a system-securing act and therefore meaningless.

So what is the role of non-mining full nodes then?

Relaying valid transactions. Keeping blockchain for others. Serving light clients. Nothing more.

... relaying valid blocks, allowing users to validate the network state independently, allowing users to interact with higher level layers (eg LN) trustlessly...

When Bitcoin was released, nodes and miners were the same thing, a computer which:

validates and relays transactions
validates, mines, and relays blocks.
See Bitcoin Whitepaper.

When mining started being done on GPUs and eventually ASICs, it was run on separate hardware than the nodes. This is because the mining hardware was designed to be good at only 1 of the things listed above, mining. They perform trillions of hashes per second until a solution is found.
Nodes perform all of the transaction and block validation as well as relaying that information to other nodes on the network. Generally speaking, miners receive the information necessary to mine from a node, and then submits its work back to the node to be validated.

Can non mining full nodes prevent 51% attack? Does it actually
  strengthen the network? I am confused of its real purpose

Not without a miner, because the 51% attack refers to the hashpower or the amount of work the miners are able to do. Nodes just validate, store, and relay the blocks, but they cannot create new blocks without a miner to generate the proof of work.


text:
Does balance broadcasting hinder scalability of LN?
Rene Pickhardt states in his research that broadcasting the balance values would hinder the scalability of the Lightning Network meaning that broadcasting the local and remote balance of each channel would hinder the scalability of the network.
How is the broadcasting of this info related to the scalability?

Assume the lightning network wants to support 100k payments per second globally.
Now every time a payment is attempted, aborted or settled the balance of almost every channel in the payment flow changes. Assuming 5 channels being involved in a payment (and assuming no failed payments as the balance values would magically instantly known to all peers) this would still result in 1M messages that would have to be broadcasted to every participant in the network per second.
Assuming 100 Bytes per message (as we have not only the balance value but timestamps, shortchannel ids, signatures to prevent spoofing and hmacs) this would be at least (!) a datastream of 100 MByte /second for every participant wich is almost 1 Gigabit/sec
Even if the internet would not break if every participant was constantly requiring such a throughput the nodes would have to be able to processes all that information. A regular webserver on commodity hardware can handle up to 10k http requests per second. So also the nodes would have some serious challenges.


text:
What means "fallbacks" in LN invoices?
Just saw a "fallback" property in LN invoice. Never seen before. Can you please describe what that means and how I can make an invoice with same property?
I am using c-lightning. This is what i had decoding invoice:
{
   "currency" : "bc",
   "created_at" : 1564513459,
   "expiry" : 3600,
   "payee" : "03efccf2c383d7bf340da9a3f02e2c23104a0e4fe8ac1a880c8e2dc92fbdacd9df",
   "msatoshi" : 1000000000,
   "amount_msat" : "1000000000msat",
   "description" : "satoshis-tennis-1564513459.3110697",
   "min_final_cltv_expiry" : 10,
   "fallbacks" : [
      {
         "type" : "P2SH",
         "addr" : "3MXqbMwf457U4Jaw35WnJdnL99mq7Q8oQQ",
         "hex" : "a914d9a625fb64a9faa4600548ae97e8efda9294dd0a87"
      }
   ],
   "routes" : [
      [
         {
            "pubkey" : "0276e09a267592e7451a939c932cf685f0754de382a3ca85d2fb3a864d4c365ad5",
            "short_channel_id" : "557233x2338x0",
            "fee_base_msat" : 1000,
            "fee_proportional_millionths" : 9,
            "cltv_expiry_delta" : 14
         }
      ]
   ],
   "payment_hash" : "de210472f7e9513358a22db99ee5b987e9fe96d523d0d833fd900837aa71199a",
   "signature" : "3045022100c79fc20087480d302a68c3b92b34de935956202c0499cf8acc182837c871923202200879fd3979aa688a46703c31f7e56ea5e7917f4820da42177d31fc2a9290c91a"
}

I am in particular interested in the following part:

    "fallbacks" : [ { "type" : "P2SH", "addr" : "3MXqbMwf457U4Jaw35WnJdnL99mq7Q8oQQ", "hex" : "a914d9a625fb64a9faa4600548ae97e8efda9294dd0a87" } ],

As can be read in BOLT 11 of the lightning protocol spec

the f field allows on-chain fallback; however, this may not make sense for tiny or time-sensitive payments

it is one of the fields encoded into a BOLT 11 invoice

f (9): data_length variable, depending on version. Fallback on-chain address: for Bitcoin, this starts with a 5-bit version and contains a witness program or P2PKH or P2SH address.

In case a payment can't be routed it is possible to pay this invoice with an onchain bitcoin transaction instead of opening a new channel. This obviously can't be done for sub dust payments and even for tiny payments above the dust limit this might not be economically viable.

Is there any option to add such fallback using c-lightning when creating an invoice?

Yes check lightning-cli help invoice to be able to read the manage (:


text:
Why does my freshly installed lightningd not show funds that I transferred to it?
I have a box with Ubuntu, bitcoin: fully indexed, and c-lightning.  I made several new addresses, and put money into one of them.  The money was transferred to it in block 655761.  I see the bech32 address when I run dev-listaddrs, it was sent to the right address.  Why would list funds show no outputs?

Clightning actually needs to scan all blocks to check for output's and Adresses it controls.
I assume most likely your c-lightning node is so fresh that it has not seen such a late block yet. Once your c Lightning node is up to date funds appear directly in listfunds once they are confirmed. So I guess you can do two things.

Check your log file and see how far c Lightning is with parsing through blocks
Wait until clightning is finished


text:
How to recover funds from a force-closed Lnd lightning channel?
I lost my Lnd channel backup in a boating accident, but I still have the mnemonic. I had one freshly opened channel to my own c-lightning node which has never been used. I force closed the channel from the c-lightning side.
The funds are now in a P2WPKH address which I've imported into Bitcoin Core. Now I need to figure out the private key in order to sweep it. This requires knowing how to derive it, and then how to dump it.
There is wallet utility pull request that I could tweak in order to dump the key I need once I know how to derive it.
I also know the funding transaction id, the channel id and all the other channel details c-lightning spits out with listfunds.
Key derivation is explained in derivation.go:
m/1017'/coinType'/keyFamily'/0/index

The coinType for Bitcoin is 0, and I believe the key family is 3: "used in scripts that pay directly to us without any delay"
My guess is that the index is 0 since it was the first channel.
BOLT 3 then explains how to do tweak this key for a commitment transaction. Because the other side closed this channel, I need the to_remote output, which requires the remotepubkey.
pubkey = basepoint + SHA256(per_commitment_point || basepoint) * G
privkey = basepoint_secret + SHA256(per_commitment_point || basepoint)

The spec says "the remotepubkey uses the remote node's payment_basepoint". Is that the key we just derived?
So that leaves the per_commitment_point to be figured out, derived from per_commitment_secret. I'm confused where that comes from.
What Go incantation should I use to dump the private key I'm looking for?

I cannot give you the answer out of the box but I wanted to chase down that logic since a long time. if noone finds the answer I would volunteer to have a hangout (or what ever service you prefer) to chase down the derrivation of the the per_commit secretes from the mnemonic seed

In c-lightning the per commitment point is derived from the hsm_secret using per_commit_point from:

The node id / public key (can be found with listfunds, until the channel is purged some time after closing)
The channel database id (can be found with cat ~/.lightning/debug.log | grep $node_id), shown as chan #$node_id
The commitment number (0 when opening the channel, increases during fee negotiations and other channel updates, |  grep commit)

We care about the commitment point, not the secret, because the private key we need is derived as follows:
privkey = basepoint_secret + SHA256(per_commitment_point || basepoint)

I created a tool for c-lightning to dump this point (with help from @cdecker): https://github.com/ElementsProject/lightning/pull/3115
Back to Lnd... One easy place to dump a private key is in the lnwallet walletinfo command. Add something like this to commands.go:
basePointSecret, err := keyRing.DerivePrivKey(keychain.KeyDescriptor{
  KeyLocator: keychain.KeyLocator{
    Family: keychain.KeyFamilyPaymentBase,
    Index:  0, // Same commitment index as used in c-lightning
  },
})

// Parse per_commitment_point dumped by c-lightning:
bytes, err := hex.DecodeString(".........")
commitPoint, err := btcec.ParsePubKey(bytes, btcec.S256())

// Obtain SHA256(per_commitment_point || basepoint)
h := sha256.New()
h.Write(commitPoint.SerializeCompressed())
h.Write(basePointSecret.PubKey().SerializeCompressed())
commitTweak := h.Sum(nil)

// Obtain and dump private key
tweakedPrivkey := TweakPrivKey(basePointSecret, commitTweak)
commitmentWif, err := btcutil.NewWIF(tweakedPrivkey, &chaincfg.MainNetParams, true)

fmt.Printf("Dump commitment priv key: %s\n", commitmentWif)

Where TweakPrivKey() is lifted from script_utils.
Import the resulting private key into Bitcoin Core et voila!

So that leaves the per_commitment_point to be figured out, derived from per_commitment_secret. I'm confused where that comes from.

This comes from the open_channel/accept_channel messages in BOLT2. It is called first_per_commitment_point.
Unfortunately I can't find where you can access this on lnd documentation.
An idea: Look at the lnd backup documentation and try to figure out how that works. per_commitment_secret must be backed up to trustlessly redeem funds. Since it looks like they do not offer easy access to that, you may be able to find it in the backup information.


text:
Multisig with oracle
I am trying to develop an use case in which a user has its funds on a multisig 3-2 (one key staying with him, another with my app and another one with a type of oracle that can attest if some conditions are met). These conditions would be retrieved with some type of script that can consult API's with BTC price, etc. Would that be possible to do in the base layer of bitcoin? Can anybody point to me some articles that explore this question?

DLCs, or 'Discreet Log Contracts' could be interesting for you to explore given the use-case that you appear to have in mind.
DLCs allow for the creation of onchain Bitcoin contracts using an external oracle to determine the contract outcome. The DLC spec has built in refund logic to prevent locked funds for if for some reason the oracle is unable to attest, following a timelock. Given your stated goals, I would suggest that you explore the DLC specs and specific implementations linked above as DLCs already achieve your stated goal using 2/2 multisig and adaptor signatures.

2-of-3 multisignatures are indeed possible with Bitcoin's base layer scripting language. In fact, there is an opcode decidated to it - OP_CHECKMULTISIG.
Note that all the Bitcoin base layer sees is three keys, and two signatures from those keys. It doesn't understand price, APIs, or any other oracle logic. The oracle would have to be independently developed and operated to check if whatever conditions you want are satisfied, and if so, to sign a message of your choice (a transaction). That would be the real challenge, and would have to be trusted and secure.


text:
Block 9 Transaction 1 Output 0 ScriptPubKey openssl validation fails
I'm trying to check the public key which lies in the ScriptPubKey of output #0 of transaction #1 bloc #9 with openssl. I'm using Python to create a DER file, and openssl to check it.
ScriptPubKey value is:
0411db93e1dcdb8a016b49840f8c53bc1eb68a382e97b1482ecad7b148a6909a5cb2e0eaddfb84ccf9744464f82e160bfa9b8b64f9d4c03f999b8643f656b412a3

My Python code to create the DER file is:
scriptPubKey = block9.transactions[0].outputs[0].scriptPubKey[1:-1]
x = scriptPubKey[1:1+32]
y = scriptPubKey[1+32:]
publicKey = bytearray ([0x03, len (scriptPubKey) + 1, 0x00]) + scriptPubKey
algorithmIdentifierId = b'\x06\x07\x2a\x86\x48\xce\x3d\x02\x01'
algorithmIdentifierParameters = b'\x06\x08\x2a\x86\x48\xce\x3d\x03\x01\x07'
algorithmIdentifier = algorithmIdentifierId + algorithmIdentifierParameters
publicKey = bytearray ([0x30, len (algorithmIdentifier)]) + algorithmIdentifier + publicKey
publicKey = bytearray ([0x30, len (publicKey)]) + publicKey
file = open (f'public.der', 'wb')
file.write (publicKey)
file.close ()

After running this code, I get a public.der which I check using openssl. First; I check the ASN.1 syntax, which is correct:
$ openssl asn1parse -inform der -in public.der
    0:d=0  hl=2 l=  89 cons: SEQUENCE
    2:d=1  hl=2 l=  19 cons: SEQUENCE
    4:d=2  hl=2 l=   7 prim: OBJECT            :id-ecPublicKey
   13:d=2  hl=2 l=   8 prim: OBJECT            :prime256v1
   23:d=1  hl=2 l=  66 prim: BIT STRING

Then I check the key itself, and this doesn't work:
$ openssl ec -pubin -inform der -in public.der -text -noout
read EC key
unable to load Key
140542543959360:error:1012606B:elliptic curve routines:EC_POINT_set_affine_coordinates:point is not on curve:../crypto/ec/ec_lib.c:812:
140542543959360:error:10098010:elliptic curve routines:o2i_ECPublicKey:EC lib:../crypto/ec/ec_asn1.c:1158:
140542543959360:error:100D708E:elliptic curve routines:eckey_pub_decode:decode error:../crypto/ec/ec_ameth.c:157:
140542543959360:error:0B09407D:x509 certificate routines:x509_pubkey_decode:public key decode error:../crypto/x509/x_pubkey.c:125:

This code relies on this post. It works well if I use the x and y coordinates which the author of the post gives as an example:
x = b'\x81\x01\xEC\xE4\x74\x64\xA6\xEA\xD7\x0C\xF6\x9A\x6E\x2B\xD3\xD8\x86\x91\xA3\x26\x2D\x22\xCB\xA4\xF7\x63\x5E\xAF\xF2\x66\x80\xA8'
y = b'\xD8\xA1\x2B\xA6\x1D\x59\x92\x35\xF6\x7D\x9C\xB4\xD5\x8F\x17\x83\xD3\xCA\x43\xE7\x8F\x0A\x5A\xBA\xA6\x24\x07\x99\x36\xC0\xC3\xA9'
scriptPubKey = b'\x04' + x + y
publicKey = bytearray ([0x03, len (scriptPubKey) + 1, 0x00]) + scriptPubKey
algorithmIdentifierId = b'\x06\x07\x2a\x86\x48\xce\x3d\x02\x01'
algorithmIdentifierParameters = b'\x06\x08\x2a\x86\x48\xce\x3d\x03\x01\x07'
algorithmIdentifier = algorithmIdentifierId + algorithmIdentifierParameters
publicKey = bytearray ([0x30, len (algorithmIdentifier)]) + algorithmIdentifier + publicKey
publicKey = bytearray ([0x30, len (publicKey)]) + publicKey
file = open (f'public.der', 'wb')
file.write (publicKey)
file.close ()

No problemo for openssl here:
$ openssl ec -pubin -inform der -in public.der -text -noout
read EC key
Public-Key: (256 bit)
pub:
    04:81:01:ec:e4:74:64:a6:ea:d7:0c:f6:9a:6e:2b:
    d3:d8:86:91:a3:26:2d:22:cb:a4:f7:63:5e:af:f2:
    66:80:a8:d8:a1:2b:a6:1d:59:92:35:f6:7d:9c:b4:
    d5:8f:17:83:d3:ca:43:e7:8f:0a:5a:ba:a6:24:07:
    99:36:c0:c3:a9
ASN1 OID: prime256v1
NIST CURVE: P-256

May somebody tell me what I missed? I tried to inverse the byte order of x and y, and/or to exchange x and y, but I could not make it work with the x and y values from ScriptPubKey...
According to pages 73 to 74 of this book, x and y are supposed to be given in this order, and in big endian...

You have the wrong elliptic curve parameters in your PEM file. Bitcoin uses secp256k1, but you have (copied) the curve parameters for NIST P-256 (aka secp256r1).
algorithmIdentifierParameters should be b'\x06\x05\x2b\x81\x04\x00\x0a' for secp256k1.

Exactly! Thanks! I just found it while reading this useful table giving the byte notation for OID : https://thalesdocs.com/gphsm/luna/7/docs/network/Content/sdk/using/ecc_curve_cross-reference.htm

If you know a better resource for converting OID from strings to bytes, it may be useful.

See MSDN for this conversion...


text:
Block 170 transaction validation in the old days
I'm trying to understand how to validate the first non-coinbase transaction, that is transaction 1 of block 169, as it was validated in the old days.
This transaction has one input, which decoded ScriptSig is exactly:

PUSH (71 bytes: 70 bytes ECDSA signature ANS.1 in DER format and trailing SIGHASH_ALL byte)

This input refers to output 0 of transaction 0 of block 9, which ScriptPubKey is exactly:

PUSH (65 bytes: not-compressed flag byte and 64 public key x and y)
OP_CHECKSIG

My question is: in those old days, how did we know that we had to run a P2PKH to validate the transaction? Simply put, did we just concatenate ScriptSig and ScriptPubKey...:

PUSH (71 bytes) PUSH (65 bytes) OP_CHECKSIG

..and run the resulting script (that is: running OP_CHECKSIG as it is coded in EvalScript() in src/interpreter.cpp of the Bitcoin Core), which was supposed to be a P2PKH script, or did we create a more elaborate script from those scripts to validate?
Because nowadays, after some BIPs (witnesses, and so on), the P2PKH script looks more elaborated:

OP_DUP OP_HASH160  OP_EQUALVERIFY OP_CHECKSIG

Addendum
For those like me who are interested in learning about the bitcoin by coding it from scratch and using the first blocks as a data reference, I later found this very good book. You may also find interesting to check the first Bitcoin code here.

The input on transaction f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16 is P2PK not P2PKH.

This really has absolutely nothing to do with any differences in modern bitcoin, it’s just how it has always been. That’s a pay2ip transaction and they weren’t P2PKH “offline addresses” in the nomenclature of the time.

Thanks, it's more clear now. A follow-up: nowadays, what would be the criteria for a node to identify this transaction as a P2PK transaction ? The length of the scripts ? The contents of the scripts ?

You could tell because there would be no need for the "OP_DUP OP_HASH160 Public-Key-Hash OP_EQUAL" sequence.

In the original Bitcoin, you sent money by putting an IP address in the "to" field, your node connected to that IP address directly and requested a public key, and then you paid to that public key. Addresses were originally "offline addresses" for when the person wasn't running a node at the time, or for some other reason it wasn't possible to use this method.

@StashOfCode Nodes don't classify transactions or inputs as P2PK or P2PKH or whatever. That's something we do as humans, because some have well-defined address representations. But nodes don't care - they just execute the scripts as they are told.

My question is: in those old days, how did we know that we had to run a P2PKH to validate the transaction?

We didn't "know that we had to run..." Rather, in the old days, one would pay to public key instead of pay to public key hash. This made the validation scripts look simple.
The combined validation script for P2PK literally just looks like:
(sig) (pubkey) (checksig)

Contrast this with the P2PKH combined validation script, which looks like:
(sig) (pubkey) (dup) (hash160) (pubkeyhash) (equal?) (checksig),

where all the stuff in the middle of the P2PKH script, between (pubkey) and (checksig), is basically just to go from the public key hash (which is the only thing the person creating the transaction knows) to the actual public key (stored on the sigScript side for P2PKH).
So, you can tell them apart because the validation scripts look different. For example, you could look for the "OP_DUP OP_HASH160 public-key-hash OP_EQUAL" sequence.

Thanks for the explanation!

P2PK was used with P2IP, meaning nobody had to interact with raw public keys. An exception is block rewards, which remained P2PK long after that went out of fashion.

Support for P2PKH addresses was present in the first released Bitcoin code. And I think this answer misses the most important point: the script interpreter doesn't have to tell scripts apart (and doesn't). It just executes them, indiscriminately.

Yeah, that's true. I guess I didn't parse the question fully. (But apparently well enough to answer the question OP must have had in mind.) I didn't mean to imply that the script interpreter has to tell the difference--just that OP can tell the scripts apart using the suggested method. I edited my answer.

My question is: in those old days, how did we know that we had to run a P2PKH to validate the transaction?

We didn't, and we don't.
The validation code simply executes the scripts, instruction by instruction. The fact that we (now) call these particular scripts P2PK or P2PKH scripts is just a human designation, that determines how they get shown by wallets. But the validation code doesn't care.
Perhaps this helps: script is a programming language. Any time a transaction output was created, the wallet would construct in the transaction outputs a little program that determined how that output can be spent. Most commonly, the transaction output would be one of a very small number of specific types (e.g. P2PKH), which we've given special names. But transactions weren't, and aren't, restricted to just those two types. Scripts could use whatever opcodes they wanted, to encode whatever conditions they wanted.

Now, some things have changed since "the old days", which do factor in here:

Very early on, the scriptSig (unlocking script in the input) and the scriptPubKey (locking script in the output) were concatenated, and then executed as one, together. This is no longer the case since this was discovered to be trivially insecure, in 2010. Now the scriptSig is executed first, and then the scriptPubKey is executed separately, copying just the final state of the first as initial state for the second.
The introduction of P2SH in 2012 (BIP16) did actually introduce a special pattern that validation nodes do specifically recognize. BIP16's rule in short is "IF the scriptPubKey exactly matches this template, some additional validation rules are triggered". That's because P2SH moved the actually executed script logic from the scriptPubKey to the scriptSig, and the scriptPubKey only contains a magic hash of the real script, redirecting execution to the script in the scriptSig. So since P2SH, senders don't actually "know" what script they are sending to anymore, only its hash.
Something similar to P2SH was done again in 2017 with the introduction of segwit (BIP141).
There used to be support for pay-to-IP addresses, where your node would literally connect to that IP, ask it for a scriptPubKey to send to, and then construct a transaction that paid that did. This was very insecure (no encryption or authentication, just whatever the IP responds), and has long since been removed. Such pay-to-IP scripts were generally P2PK ones.


text:
What prevents a transaction being added to the block chain twice?
As I understand it, a transaction basically consists of a from address, to address, amount and some component that verifies that the owner of the from_address has requested the transaction. A transaction is broadcast to the network in the hope that it will get included into a freshly mined block.
If the miner can choose which transactions to add to a block. How does the Bitcoin protocol stop a miner from re-adding an existing transaction to the chain? and thus recreating the transaction and doubling the effect of the transaction.
Surely it is not feasible to search the entire transaction history to make sure the transaction hasn't already been included.
EDIT - from a bit of further research, it looks like including a confirmed transaction will render a block invalid and it will be rejected by other Bitcoin nodes. But how can you determine if the block contained an already confirmed transaction in an efficient way?

In Bitcoin, each full node follows all transaction activity on the network by digesting the blockchain. To that end, every full node enforces their ruleset on the data that they receive. As (usually) the full nodes all follow the same ruleset, every node indivdually converges on the same state.
Hereby, the blockchain's transaction data represents a journal of the activity, which if digested according to Bitcoin rules yields a ledger of all balances at the current blockchain tip, the Unspent Transaction Output Set (UTXO Set).
Every full node maintains this complete set of UTXOs. Each UTXO is uniquely identifiablee by the transaction it was created in, and the position in that transaction's output list.
When a transaction is confirmed, the UTXOs in the transaction inputs get spent and removed from the UTXO set, while the transaction's outputs in turn get added to the UTXO set. If someone tries to replay the same transaction again, the referenced inputs are no longer in the UTXO Set as they were previously removed and the transaction is just as easily identified to be invalid as a transaction spending made-up UTXOs.

Thanks. So the UTXO set must be rather large? A look on http://statoshi.info/dashboard/db/unspent-transaction-output-set suggests that there are 22,500,000 unspent transaction outputs? and this all has to be searched in order to validate a block?

Would distributing coins to a large number of addresses serve to increase the UTXO set, and gathering coins into a single address reduce the UTXO set?

Yes, sounds about right. Searching 22.5M UTXO is fast if you put them into an adequate data structure. If you meant sending into a single UTXO in one address, yes on both counts.

So UTXO is another data set, different from the blockchain, right? How is consensus over UTXO reached?

@MarkNeuhaus: I've improved my answer to better explain your follow-up question.

authors: Роман Кадышев

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Why has BCH not received a 51% attack yet?
BCH right now has around 10% of BTC total hashrate.
There are multiple pools that control 10%+ of the total BTC hashrate why haven't any of those just allocated a few hours of hashrate to do a 51% attack similar to the one we saw recent on BTG. If they have 10% of BTC hashrate then it would only cost them around $60,000 per hour to control the network (to pay the users using the pool)

Because of the cost, specifically the opportunity cost.
If a miner has 10% of the BTC hashrate, then if they pointed all of that mining power to BCH, they would still be losing money. With 50% of the hashrate on BCH, they would mine roughly 50% of the blocks in a day, which is 72 blocks. At 12.5 BCH per block,72 * 12.5 = 900 BCH. Convert that to BTC at the current exchange rate (0.135 BTC/BCH) 900 * 0.135 = 121.5 BTC.
However if the miner keeps mining BTC, at 10% of the network hash rate, they would mine ~14 blocks per day. With at least 12.5 BTC per block, a miner would be making 14 * 12.5 = 175 BTC per day, much more than if they were mining BCH.
Thus a miner will earn more money from mining BTC rather than mining BCH. Performing a 51% attack on BCH isn't really worth it; all you can get is possibly some double spends, and less value in BTC.

I understand from that point but they could make way more than 175 BTC in value from attacking multiple exchanges? BTG attackers managed to take several million and that coins daily volume/order books is vastly smaller than BCH.

@masterq you must not ignore the risks and potential costs of being caught attacking multiple exchanges (or simply having your attack fail for some reason). If it is profitable to just mine BTC, why risk your profits?

Yea that's probably a risk most wont be willing to take but BCH being safe on the notion of no company risking being caught doesn't really make it safe. Plus with the rise of decentralised exchanges wont this just become much easier?

One also has to take into account the possibility that performing such an attack will cause the price of the currency targeted by the attack to drop. That would make the attack even less profitable.

@kasperd Yes, maybe worth it may not. If BTG is a example of any then it would defiantly be worth it.

Mining profitability doesn't have much to do with 51% attack.  If they defraud an exchange for $10 MM, then it's well worth missing out on BTC's blockrewards.

when you control 51% of the hash rate, you mine 100% of the blocks, not 51% of the blocks.


text:
Run a full node with multiple IP addresses
I run a full Bitcoin node on a powerful server however rather than setting up multiple I was wondering if it's possible to use multiple IP addresses on one installation and one running process? Without people being able to detect it's the same node (not easily anyway).

If you can allocate multiple IP addresses to one machine (hint: you can), then yes, you can have one node bound to multiple IP addresses and serving peers on all of them. It's impossible to tell whether two nodes you are connected to are actually the same node so that doesn't matter. Note that doing this does not help the network any more than having one IP address and one node.

Why doesn't it help? Whats the difference between one node with 5 IPs vs 5 nodes with 1 IP each. In terms of helping spread the blockchain and signalling no 2x

If your node goes down, all of the nodes connected to your 5 IPs lose a connection, and that could be many nodes. If there are 5 nodes and one goes down, then only the nodes connected to that one IP address lose a connection. More nodes adds more robustness to the network and is generally better.

Well I only have one server so the other option would be to host multiple VMs which could cause the same problem and I guess running one node with 100+ connections spread across 5 VMs would use more resources than having 1 node with 100 connections active. How do I bind multiple IPs to the node?


text:
How to upgrade Bitcoin Core Version on Raspberry PI?
I want to upgrade my Bitcoin core version to latest release 0.16.1. The release docs say just copy over bitcoind/bitcoin-qt (on Linux). What does that mean? what to copy? Copying from where to where?
https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.16.1.md#how-to-upgrade

Would you edit your question to include a link to the document where you find this instruction ?

added docs link

They are referring to executable binary files, either the bitcoind daemon or the bitcoin-qt GUI app.
The page with download links is https://bitcoincore.org/bin/bitcoin-core-0.16.1/
The only link for ARM (Raspberry Pi processor) is https://bitcoincore.org/bin/bitcoin-core-0.16.1/bitcoin-0.16.1-arm-linux-gnueabihf.tar.gz and expanding it
$ tar -xf bitcoin-0.16.1-arm-linux-gnueabihf.tar.gz

I find these directories:
~/bitcoin-0.16.1$ ll
bin/
include/
lib/
share/

~/bitcoin-0.16.1$ cd bin
~/bitcoin-0.16.1/bin$ ll

bitcoin-cli*
bitcoind*
bitcoin-tx*
test_bitcoin*

I don't find a bitcoin-qt GUI version; perhaps they have not compiled one yet. But you can certainly use the command-line versions, bitcoind, bitcoin-cli and bitcoin-tx.
The place to put them is where they will be found from anywhere, for example, /usr/local/bin   or /usr/bin.


text:
Ho to identify a multisig transaction?
01000000013dcd7d87904c9cb7f4b79f36b5a03f96e2e729284c09856238d5353e1182b00200000000fd5d01004730440220762ce7bca626942975bfd5b130ed3470b9f538eb2ac120c2043b445709369628022051d73c80328b543f744aa64b7e9ebefa7ade3e5c716eab4a09b408d2c307ccd701483045022100abf740b58d79cab000f8b0d328c2fff7eb88933971d1b63f8b99e89ca3f2dae602203354770db3cc2623349c87dea7a50cee1f78753141a5052b2d58aeb592bcf50f014cc9524104a882d414e478039cd5b52a92ffb13dd5e6bd4515497439dffd691a0f12af9575fa349b5694ed3155b136f09e63975a1700c9f4d4df849323dac06cf3bd6458cd41046ce31db9bdd543e72fe3039a1f1c047dab87037c36a669ff90e28da1848f640de68c2fe913d363a51154a0c62d7adea1b822d05035077418267b1a1379790187410411ffd36c70776538d079fbae117dc38effafb33304af83ce4894589747aee1ef992f63280567f52f5ba870678b4ab4ff6c8ea600bd217870a8b4f1f09f3a8e8353aeffffffff0130d90000000000001976a914569076ba39fc4ff6a2291d9ea9196d8c08f9c7ab88ac00000000
you can decode above tx at http://chainquery.com/bitcoin-api/decoderawtransaction
From bitcoin explorer, I found that above transaction is 2-3 multisig tx. I understood that bolded part fd5d01 is script-length. its value is 349 Bytes. In a normal tx, Within the ScriptSig, the first byte which tells us how long the signature is. followed by the size of the second element and then the second element which is pubkey. Now, how to identify the number of signatures involved in a multisig tx like above example. I mean how did the explorer find out that it is 2-3 multisig. In the first place how to identify a multisig? And what is the significance os OP_0 at the beginnning of scriptsig?

As you have found out correctly, the length field describes the following script length. Then the SigScript follows:
004730440220762CE7BCA626942975BFD5B130ED3470B9F538EB2AC120C2043B445709369628022051D73C80328B543F744AA64B7E9EBEFA7ADE3E5C716EAB4A09B408D2C307CCD701483045022100ABF740B58D79CAB000F8B0D328C2FFF7EB88933971D1B63F8B99E89CA3F2DAE602203354770DB3CC2623349C87DEA7A50CEE1F78753141A5052B2D58AEB592BCF50F014CC9524104A882D414E478039CD5B52A92FFB13DD5E6BD4515497439DFFD691A0F12AF9575FA349B5694ED3155B136F09E63975A1700C9F4D4DF849323DAC06CF3BD6458CD41046CE31DB9BDD543E72FE3039A1F1C047DAB87037C36A669FF90E28DA1848F640DE68C2FE913D363A51154A0C62D7ADEA1B822D05035077418267B1A1379790187410411FFD36C70776538D079FBAE117DC38EFFAFB33304AF83CE4894589747AEE1EF992F63280567F52F5BA870678B4AB4FF6C8EA600BD217870A8B4F1F09F3A8E8353AE

The script starts with a hex "0", which compensates for a very early "off by one error" in the execution routines of the checksig routines. An element too much was popped off the stack. This "bug" is so long in the system, that everyone got accustomed to it, and removing it requires an enormous effort. So it became a feature :-) Even more, if there are tx, which have an execution time in the future, they wouldn't work anymore...

how did the explorer find out that it is 2-3 multisig. In the
  first place how to identify a multisig?

The explorer finds this in the script sig. There is the two signatures, and the part for the multisig:
<sig1> <sig2> <length> OP2 <pubkey1> <pubkey2> <pubkey3> OP3 AE

to be better able to view, I have decoded the script a bit. The sigs start with hex 0x47 or 0x48, and end with 0x01. They are ASN1-DER encoded, providing a view into the R and S fields, and have surrounding length information. After the second 0x01, follows the length field, and then the multisig OPcode (0x02) and the 3 pubkeys.
00: OP_0, OP_FALSE:      an empty array is pushed onto the stack.
<sig1>
47: OP_DATA_0x47:        push hex 47 (decimal 71) bytes on stack
30: OP_SEQUENCE_0x30:    type tag indicating SEQUENCE, begin sigscript
44: OP_LENGTH_0x44:      length of R + S
02: OP_INT_0x02:         type tag INTEGER indicating length
20: OP_LENGTH_0x20:      this is SIG R (32 Bytes)
02: OP_INT_0x02:         type tag INTEGER indicating length
20: OP_LENGTH_0x20:      this is SIG S (32 Bytes)
01: OP_SIGHASHALL:       this terminates the ECDSA signature (ASN1-DER structure)
<sig2>
48: OP_DATA_0x48:        push hex 48 (decimal 72) bytes on stack
30: OP_SEQUENCE_0x30:    type tag indicating SEQUENCE, begin sigscript
45: OP_LENGTH_0x45:      length of R + S
02: OP_INT_0x02:         type tag INTEGER indicating length
21: OP_LENGTH_0x21:      this is SIG R (33 Bytes)
02: OP_INT_0x02:         type tag INTEGER indicating length
20: OP_LENGTH_0x20:      this is SIG S (32 Bytes)
01: OP_SIGHASHALL:       this terminates the ECDSA signature (ASN1-DER structure)

4C: OP_PUSHDATA1:        next byte is # of bytes that go onto stack
C9: OP_Int(0x01-0xff):   201 bytes onto the stack
52: OP_2:                the number 2 is pushed onto stack
    ################### we go multisig ####################################
41: OP_DATA_0x41:        uncompressed pub key (65 Bytes)
41: OP_DATA_0x41:        uncompressed pub key (65 Bytes)
41: OP_DATA_0x41:        uncompressed pub key (65 Bytes)
53: OP_3:                the number 3 is pushed onto stack
    ################### 2-of-3 Multisig ###################################
AE: OP_CHECKMULTISIG:    terminating multisig

You find a very good explanation in chapter 6 "Transactions" and 7 "Advanced Transactions and Scripting" of Andreas' wonderful book "Mastering Bitcoin", which is also online available.

I didn't completely understand how you are able to find that tx has 2 signatures. When parsing the script, first we encountered 0x47 (71 Bytes). As script starts with signature, next 71 bytes must be SigA (understood till here). Now we got 0x48 (72 Bytes). Did you consider this 72 Bytes as SigB just because it is bigger than standard length of compressed/uncompressed pubkey? or do you some other logic to consider this 72 Bytes as SigB

Correct. Sigs are encoded by the ASN1 standard. Sig1 is the first 0x47, ending with "01". The second hex 0x48 defines a length, at this point in time it is unclear, what will follow - could be anything (e.g. redeemscript). Then follows 0x30 and 0x45, defining length of R+S, and ending with 0x01 - so it must be a signature again, cause it follows ASN1 DER encoding scheme.


text:
How does Bitcoin Core know it's on the longest chain?
In trying to implement a client, I am wondering what is the most efficient approach to figure out if you are on a fork.
Let's say you pick a peer at random and go through IBD using the headers-first approach. After reaching the tip of the chain, what is the most efficient way to check whether this is the longest chain?
I'm guessing you have to ask your peers what their best header is, and based on the response and some heuristic, decide whether you are on the longest chain or not. Is this simply done via getheaders? What is the heuristic? And what do you do if you find out you're on a fork?
Pointers to the relevant code in bitcoin or btcd would also be appreciated.
Thanks
EDIT: In case this isn't obvious, I'm not asking whether it can know with certaintiy that it is on the longest chain (it can't), but rather what is the best effort attempt to remain on the longest chain at all times.

First of all, finding the longest chain is not the goal, but the chain with the most accumulated proof-of-work [1]
Now, finding the most-work chain is the entire goal of the full node. The whole meaning of its existence is finding the most-work chain, and so you can pick almost any line out of the networking and validation modules of the code and explain it in those terms.
To be a bit more specific:

The full node connects to at least 8 peers. More recent versions connect to 2 extra peers in 'blocks only' mode. Peers are chosen to be as diverse as possible (by IP subnet or AS map). If a new block has not been added within a certain timeout (I think it's 40 minutes) Bitcoin Core adds one more outgoing peer.
The full node sends a getheaders request to each peer and keeps track of the best block each peer knows about.
If a peer has a valid block header with greater accumulated work than our current tip, we ask it for the corresponding blocks. If those blocks are valid, it becomes our new chain tip. A reorganization may or may not be necessary to get to that state.

If you understand conceptually how headers-first sync works, the function FindNextBlocksToDownload() is the next step. This is where the node decides if a peer has headers with greater work than its current chain tip, and requests those blocks from that peer.
So all we can do is connect to a lot of peers and assume that at least one of them knows about the most-work chain. This makes peer discovery and choosing which peers to connect to of critical importance.

Thanks, this is really helpful. Looking at FindNextBlocksToDownload, the part I'm still trying to understand is: if after finishing your header sync, a peer gives you headers you've never seen, and which don't connect to your chain, how are you going to judge whether they have more accumulated work?

Since we can not verify "orphan" headers, we do not request those blocks. The getheaders message includes something called a locator that the receiving peer uses to determine where their chain connects to your chain, and sends you headers from that point: https://en.bitcoin.it/wiki/Protocol_documentation#getheaders

I see, so peers will end up sending chains that connect, by using the locators.. makes sense!


text:
Is the `mempool` message reliable?
According to https://github.com/bitcoin/bitcoin/blob/master/src/net_processing.cpp#L3775-L3801, it seems like the mempool message is only responded to in certain cases. Can someone help understand how this works and why a full node would not always respond to this message? Is it being deprecated?
I'm looking for ways to estimate fees through the p2p layer, and getting the mempool content seems like the best option so far, if nodes reliably reply to the message.

Full nodes have absolutely no use for the mempool message, it is vestigial from bip35 and has had a history of causing privacy leaks due to its poor implementation. It has previously been used to bypass transaction trickling, or just cause an enormous amount of waste data transfer. Modern versions of Bitcoin Core do not ship with bip37 enabled, which means they will not respond to the message without configuration changes.
It would not be a good solution for attempting to estimate fees regardless, as you would cause potentially gigabytes of transfer of transactions you have absolutely no way of verifying. Estimating fees, by and large, needs to be delegated and there are many APIs for doing so.

The mempool message was actually introduced in BIP35, which predates the BIP37 Bloom filtering. However, later it was changed to only be permitted when combined with BIP37 filtering, to prevent the worst problems with it. Even later, BIP37 was made optional and defaulting to off, so in general, no, most modern Bitcoin Core nodes will not respond to it anymore, unless specifically configured to allow it.

@PieterWuille Corrected, my memory of where it came from was wrong. Thank you.

I believe there are several ways to configure bitcoind to support mempool requests: (1) the "-peerbloomfilters" config option - this allows BIP37 bloom filters for all peers, and by extension also adds support for mempool requests. (2) the "-whitebind=bloomfilter" setting to allow BIP37 bloom filters and mempool requests on all inbound connections to a specified local IP address. (3) the "-whitelist=bloomfilter" setting to allow BIP37 bloom filters and mempool requests for a specified remote IP address (4) the "-whitebind=mempool" or "-whitelist=mempool" for just mempool permissions


text:
Howto convert mnemonic->seed->root key->derived addresses with c library (libWally)
I am trying to use the cryptowallet library LibWally to convert a 12-word mnemonic into a set of derived addresses.  I believe my mistake is not with the library itself, but a process flow/algorithmic one, so even if you don't aren't familiar with the library, you may be able to help.
First I validate and convert the mnemonic to a seed with the following code:
    char *mnemonic = "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon about";
    bip39_mnemonic_to_seed((const char *)mnemonic, "",
                          (unsigned char *)bytes_output, HMAC_SHA512_LEN,
                           &size_written)
     wally_hex_from_bytes((unsigned char*)bytes_output, size_written, &hex_output)

This gives me the proper seed value; at least it matches what I see on Ian Coleman's Site.  Next I take the seed value in bytes_input (not the hex coded value) and call:
bip32_key_from_seed( (unsigned char*) bytes_output,
    BIP32_ENTROPY_LEN_128,
    BIP32_VER_MAIN_PRIVATE,
    0,
    &key_structure)

And then retrieve the private key with:
bip32_key_to_base58(&key_structure, BIP32_FLAG_KEY_PRIVATE, &output_privkey);

The value in privKey does not match the Bip32 root key shown in Ian's site, though interestingly enough, it matches on the first 16 characters, before diverging (and winding up 4 chars longer).
Am I missing a step here?  Or doing something else wrong?
Edit: Adding some output data.  Using the phrase given above: my (and IanColeman's) seed is:
5eb00bbddcf069084889a8ab9155568165f5c453ccb85e70811aaed6f6da5fc19a5ac40b389cd370d086206dec8aa6c43daea6690f20ad3d8d48b2d2ce9e38e4
Root Key results:
Mine: xprv9s21ZrQH143K2fH5FXCJ4B1iTN2wJSq1h9z2wTZkytm3YgVRxP621bDZ7yqo4KdNWVNcMEyWBHrjThJ9jXMabog7R1RTWFNdSAz2mhX2VjL
IanColeman: xprv9s21ZrQH143K3GJpoapnV8SFfukcVBSfeCficPSGfubmSFDxo1kuHnLisriDvSnRRuL2Qrg5ggqHKNVpxR86QEC8w35uxmGoggxtQTPvfUu
"

I added some output data above. Do you mean "to" base58, the function I'm already calling?   Also, the docs for bip32_key_from_seed says it creates a new master root key.  But the output is an "extkey", an extended key structure w/ public/private pair on it.  It feels like I should first have to create the root key, then use that to create the extended key, no?

The extended private key on Coleman's site doesn't match the return value from my to_base58() call either.  After the same "xprv9" prefix, it diverges.

The Trezor API in that link is so different from libWally that its difficult to see what I might be doing wrong.  But if I can't get this to work, I may have to switch to that.   I could probably just write the code from scratch myself, but all the online BIP32 references I've seen are somewhat opaque for those not well-versed in elliptic-curve mathematics.


text:
Calculating P2WKH nested in P2SH addr/redeem script hash from pubkey
My calculation of the hash160 doesn't seem to be correct - what are my sins?

Tx spending the input: https://www.blockchain.com/btc/tx/d490cc0a5b9799e09b02dd4b20a6d7ca297c09c425983efc13b382f6023d0376

Spent output's scriptPubKey: OP_HASH160 032d2c15611c7b4b6941e62011e58cb5e204baa5 OP_EQUAL

Spending signature:

'txinwitness': ['3045022100b36d2efceb87bd89466d10ee31e5c76c775d9df71095a39c38bd8c5d64fc78d202201dddf5579d993d013eb73b58110e3d280364ec47b1d7617c2f6d842e7d243d6f01',
'02d0a9b9f420d22e3c1de27e87c6ff2ec33da21302cf07688b53d72c07e88d0422']

So the pubkey is '02d0a9b9f420d22e3c1de27e87c6ff2ec33da21302cf07688b53d72c07e88d0422'

My understanding is that the redeem script should be:

2102d0a9b9f420d22e3c1de27e87c6ff2ec33da21302cf07688b53d72c07e88d0422ac (classic P2SH with OP_CHECKSIG)
or

0002d0a9b9f420d22e3c1de27e87c6ff2ec33da21302cf07688b53d72c07e88d0422 for P2WPKH-nested-in-P2SH.  [this is wrong, see below]

The expected hash output of the redeem script is:
032d2c15611c7b4b6941e62011e58cb5e204baa5 (see scriptPubKey above)
My hash outputs:

ripe160(sha256('2102d0a9b9f420d22e3c1de27e87c6ff2ec33da21302cf07688b53d72c07e88d0422ac')) = 9e75bbd28314946ea5bcea5e217982fce698b041

ripe160('0002d0a9b9f420d22e3c1de27e87c6ff2ec33da21302cf07688b53d72c07e88d0422') = 1c868bec66729c87e4cb60ec6d1c8846ad86d0ee

Saw this answer (P2sh redeem script hash), my calculations work fine with those inputs, but not with the above.
Related question: what is the role of the sigScript
0014cc38458a6e5c8c09e1d8fac905fb1024c8678d88 in the input if we have the txinwitness?
EDIT: mystery solved
Since scriptSig was not empty, it means that the transaction was a nested P2SH transaction. In this case it is a P2WPKH nested in BIP16 P2SH.
So in order to get the expected redeem script hash 032d2c15611c7b4b6941e62011e58cb5e204baa5 (which is base58 transformed, with a 0x05 byte appended at the beginning, to obtain the bitcoin address 31yp4jocJSRG4Wa8aV7ocKSXKcdWzkHy61):
using the scriptSig asm = 0014cc38458a6e5c8c09e1d8fac905fb1024c8678d88
so
ripe160(sha256('0014cc38458a6e5c8c09e1d8fac905fb1024c8678d88')) = 032d2c15611c7b4b6941e62011e58cb5e204baa5 = desired result (which can be base58 transformed into the bitcoin address).
The scriptSig asm is in this case a P2WKH redeem script, which is obtained as from the pubkey:
0x00 + ripe160(sha256('02d0a9b9f420d22e3c1de27e87c6ff2ec33da21302cf07688b53d72c07e88d0422')) = 00 cc38458a6e5c8c09e1d8fac905fb1024c8678d88


text:
Are there any POS (point of sales) machines that accept BTC?
We would like to know if there are any POS in the market that accept BTC or any other Cryptocurrencies? How do you think if these kinds of POS will be the trend in future?

Yes. General Bytes, the manufacturer of ATMs and POS in the Czhech Republic has several in action in Prague.
https://www.generalbytes.com/pos/
I hope that Lightning POS will be the new standard though. There are a few companies working on that too.

Yes, depending on what country you are in. A quick check of Big Internet Search Engine for the term "Bitcoin POS" finds me two top of the list answers, once country specific with invoicing, payment and POS and, the other a global.


text:
Is blockchain protected from history spaming?
As I understand, full nodes accept block even if it is not mined on top of most recent block and store it as alternative chain as long as it fullfills protocol rules.
What would happen If one start mining millions (or more) of new versions of block height 10 (based on block 9) and keep sending them to the network? Could this cause full nodes to fill their hard drives and make them inoperational?
In other words - is there a limit for side chains enforced somehow by Bitcoin Core or protocol rules?

Full nodes use checkpointing: they decide a past block (and all the blocks before it) to be definitive. Any block relayed to them that'd have happened before that checkpoint would be ignored. See this article for more details.

Actually, how does that work?  The first checkpoint is at height 11111, so if I'm initially downloading the blockchain, an attacker can feed me an unlimited number of candidates for block 10, and until I get blocks 11-11111 I won't know which of them leads to the checkpointed block, so in the meantime I have to store them all (and I don't think that Bitcoin Core will go back and delete them afterward).  Are we just hoping that I will get blocks 11-11111 quickly enough that the effect won't be too severe?

In recent versions of Bitcoin Core, headers first synchronisation is used. Your client would download first the headers, checking some consensus rules along the way (PoW, etc..) and also checking checkpoints. This way you know that you are downloading the valid main chain.

Your information is outdated: For many years now nodes do not fetch, much less store, blocks which aren't memebers of a candidate for the longest chain according to their headers.


text:
Miner modifies unconfirmed transactions and put it into the block
Is is possible that a miner modified a unconfirmed transaction (like changing the output of transaction to the miner himself) and put it into the local block, and after 10 mins the miner luckily solved the PoW, putting the block on the chain and broadcast it to others?
If a transaction was modified and finally linked onto the chain, how others check whether the tx was valid or not? (others only check if PoW answer was right or wrong, not checking tx, am I right?)
And If modified tx was on chain, how do affected people recover their assets from the loss in modified tx (in which output was modified to the miner himself)?
Could they ask to revoke the block or how could they do?
Thanks

If a miner could modify your unconfirmed transaction, I could also modify your unconfirmed transaction even without mining.

Is is possible that a miner modified a unconfirmed transaction (like changing the output of transaction to the miner himself) and put it into the local block, and after 10 mins the miner luckily solved the PoW, putting the block on the chain and broadcast it to others?

No, because the transaction would be invalid. The vast majority of transactions contain digital signatures which signs the entire transaction. If the transaction is changed, the signatures will become invalid and thus the transaction will be invalid. And if a transaction is invalid, the block containing it is also invalid.

If a transaction was modified and finally linked onto the chain, how others check whether the tx was valid or not? (others only check if PoW answer was right or wrong, not checking tx, am I right?)

No, this is completely incorrect. ALL full nodes verify all transactions in all blocks that they receive (as well as transactions received outside of blocks). Just because a block has a valid proof of work does not mean that the block is valid. It must still build upon a valid block and must only contain valid transactions. Full nodes still verify that transactions contained within a block are valid.
Contrary to popular belief, miners do not say what transactions are valid. Their job is to determine the order of transactions, within certain constraints. It is the job of full nodes to verify transactions, and all miners (or the mining pools) should be running full nodes.

Thank a lot, it's really helpful. So, it is that full node would verify every transactions in the block which was newly created by miner? For my scenario as example, if someone modified a tx and successfully mined the block containing it, and as the block was sent to a full node, the full node would verify it and find out the invalid tx, and then what would this full node do? broadcasts to other nodes and informs them that this block is invalid? is it so? and then every node just discards the block that is just mined, which causing no new blocks found in last 10 mins?  Thanks

Every full node that receives that block would reject it as invalid. They just discard the block; there is no notifying other nodes that the block is invalid. All full nodes will figure out it is invalid. It will be just as if no block had been found. Most nodes will also ban the node that relayed them the invalid block. Such bans are local to the node, not network-wide. Note that the block being discarded does not mean that work was wasted. Mining is stateless and every block tried has the same probability of having a valid PoW as every other attempt for a block at a given height.

"If a transaction was modified and finally linked onto the chain, how others check whether the tx was valid or not? (others only check if PoW answer was right or wrong, not checking tx, am I right?)" You can't check anything about a block until you understand the block. If the transaction doesn't comply with the rules, then you don't understand the transaction and thus don't understand the block. That makes it indistinguishable from random garbage. A node doesn't try to figure out what particular invalidities mean since there is no such specification that says what they mean.

Every valid bitcoin transaction can only be created if you have the sender's private key, public key, and the receiver's public key. You can check that the transaction is valid without having the private key, but to modify it you would need to recreate the transaction from scratch, which would only be possible if you had the sender's private key.
So no, it is not possible.
Edit: For a good explanation, see https://youtu.be/Lx9zgZCMqXE?t=193

Normally everything in a transaction except scriptSig is cryptographically signed. So nobody can alter your transaction outputs. This is the default case.
But it's indeed possible to craft valid transactions where the outputs can be altered.
Have a look at the configurable signature hash types. Using DER signatures with the SIGHASH_NONE flag (0x00000002) appended in OP_CHECKSIG will produce a valid transaction of which the unlocked funds can be redistributed to any output.

It is possible, but not for regular transactions

A bit of a short answer to say that it's possible to craft transactions which can be altered (without signatures).


text:
Can I figure out the size of a transaction before its sent?
I want to know the size (in bytes) of a transaction before it's sent.
Is it possible? Maybe with raw transactions? or..
EDIT:
Not a duplicate. The 'duplicate' doesn't address how to find out how big it is. Just addresses how to calculate the bytes (inputs and outputs). I even commented on that thread. How do you find how many inputs and outputs there are. I understand how to calculate it. But how do I figure out if there are 10 outputs or 5 outputs?

Not a duplicate. The 'duplicate' doesn't address how to find out how big it is. Just addresses how to calculate the bytes (inputs and outputs). I even commented on that thread. How do you find how many inputs and outputs there are. I understand how to calculate it. But how do I figure out if there are 10 outputs or 5 outputs?

I've voted to reopen.

@MarcAlexander: Thank you for clarifying. I've also voted to reopen.

Yes, this is actually what the raw transaction API is for.
The ordinary send RPCs (sendtoaddress and sendmany) do everything at once:

Construct outputs based on the addresses/amounts you want to send.
Do coin selection to find inputs to satisfy the amounts, determine the fee, and add change if necessary.
Sign the inputs with the keys in your wallet.
Broadcast the transaction to the network.

Since the fee depends on the size of the transaction, which depends on the coin selection, you cannot "intervene" in the middle of these RPCs.
The raw transaction API lets you do all these steps independently:

Constructing outputs based on addresses/amount using createrawtransaction.
Do coin selection to find inputs, determine the fee, and add change using fundrawtransaction.
Sign the inputs using signrawtransaction.
Broadcast the transaction using sendrawtransaction.

If you need to know the actual fee before broadcasting, you can use createrawtransaction + fundrawtransaction + signrawtransaction to construct a fully valid transaction without broadcasting. You can then inspect it, and decide to broadcast it using sendrawtransaction or not.

can you post a link for this?

A link to what?

the raw transaction api documentation if you know where it is?

Seems to me that it would be useful to  allow for user to input the current fees that result in acceptable delay (in satoshis per byte, as published, for example at https://bitcoinfees.earn.co), , and then do these steps automatically up to "signrawtransaction", caculate a transaction fee accordingly , and then as for an OK before setting the tx fee and actually broadcasting the transaction.


text:
Creating a multi-sig wallet with bitcoind
Our server has bitcoind on it and is running on the test network.
I want to make the wallet multi-signature so no one in our company can withdraw any bitcoin from our server without at least 3 signatures.
Can you do this with bitcoind? I tried to follow: How can I create a multi signature 2-of-3 transaction? and How users collectively sign a transaction in a multisig scenario?
.. but could still send the bitcoin from our server with:
bitcoin-cli sendtoaddress "address_here" 0.01

That just means you had 0.01 tnBTC in the wallet which didn't come in through the multisig address.

Is there a way to make the whole wallet multi sig? Instead of just a single address within the whole bitcoind node wallet.

Just never send any money to it, except through the multisig addresses. There is no way to automatically derive multisig addresses.

a bit late, but maybe it helps future readers. Whereas you cannot "make the wallet" multisig, you can create a multisig address.  And while you are on testnet, here is a step by step example: https://gist.github.com/gavinandresen/3966071


text:
Just curious can you make $ off the spread?
I noticed there was a 2.5% spread on a certain exchange. Is there a way to make money from large spreads?


text:
Bitcoind rpc AddLocal(ip:port) and Disover: IPv4 eth0: ip missing Amazon AWS CentOS
I have a server at DigitalOcean with Ubuntu installed and bitcoind with rpc which IS working.
With my Amazon AWS CentOs server bitcoin-cli is working but not EasyBitcoin-PHP, it's returning 0:
$bitcoin = new Bitcoin('username','password','myipaddress','8332');

echo "Status: ".$bitcoin->status;

Bitcoin.conf
server=1
daemon=1
rpcuser=username
rpcpassword=password
rpcport=8332
rpcallowip=myipaddress

Checkout this image. On the working DigitalOcean Ubuntu server it includes AddLocal(192.81.219.224:18333,1) AND Discover: IPv4 eth0: 192.81.219.224
I also tried to add -discover=1 to the AWS CentOS but it didn't do anything new and still isn't working.
I'm guessing it may have to do with Amazon's Elastic IP's or something?

The fix was:
setsebool -P httpd_can_network_connect 1

Amazon AWS apparently limits httpd network connections initially.


text:
Bitcoin Script - Beautify
My question is if something exists to do the job in the opposite direction of miniscript.
Miniscript's intention is to help:

    ...writing (a subset of) Bitcoin Scripts in a structured way, enabling analysis, composition, generic signing and more

 So, for example, given the below Bitcoin Script:
OP_SIZE
32
OP_EQUAL
OP_IF
  OP_HASH160
  'dea3...'
  OP_EQUALVERIFY
  '301...'
OP_ELSE
  OP_CHECKLOCKTIMEVERIFY
  '19323434...'
  OP_DROP
  '303...'
OP_CHECKSIGVERIFY

  It would create something of this form (in JavaScript):
function canUnlock(preImageOrSig) {
    let pubKey1 = null;
    if (preImageOrSig.size() === 32) {
        if (hash(preImageOrSig) !== 'dea3...') {
            return false;
        }
        pubKey1 = '301...';
    } else {
        if (!checkLockTime('19323434...')) {
            return false;
        }
        pubKey1 = '303...';
    }
    if (!checkSignature(pubKey1, preImageOrSig)) {
        return false;
    }
    return true;
}

  The resulting JS code is NOT to be executed or compiled back to Bitcoin Script, but only to help a developer reason about the logic of the original Bitcoin Script.

Would anyone else find this feature useful?

I would look at Bitcoin script debugging tools like those described at https://bitcoin.stackexchange.com/q/42576/13866, I'm not sure I would be comfortable with a tool that translates a stack-based reverse-polish language to javascript.

Your code is more complex than a miniscript expression. A miniscript expression is intended to succeed completely without failure - because that's how bitcoin scripts operate. It's basically your code without all the IF + return true/return false lines.


text:
I need help on how to change the disk to run my bitcoin core wallet
Long time ago I transferred my wallet on My Passport an external hard disk. I have not opened bitcoin core for over 8 months. I opened it 2 days ago, the dialog box opened, which normally doesn’t and I was asked to choose where to open bitcoin core. I chose the external disk.Something went wrong. Finally I understood that there was a problem with the disk and it was not mounting on my macbook pro. I fixed that problem. However bitcoin core continues to open at the default file on my mac which is empty.  My wallet shows no transactions and it is updating since 11 years back.
I need to be able to open the dialog box at start of bitcoin core to choose again the correct disk but I don’t know how to do that. My knowledge is very limited

Close core, find your bitcoin.conf file, and change the datadir setting. e.g.

# [core]
# Specify a non-default location to store blockchain and other data.
datadir=/opt/yourdisk

Then re-open core. It should pick up where your wallet last synced to on the external drive.


text:
Pseudo code to find Bip39 checksum?
I'm trying to find the checksum for a 12-word BIP39.
I can find the first eleven words just fine, but my 12th word is always off by a few positions.
For example, take the 128-bit entropy e84eaa26330ccbb2f866a1dc155e94ba.
In binary that is 11101000010011101010101000100110001100110000110011001011101100101111100001100110101000011101110000010101010111101001010010111010.
The first eleven 11-bit words are:
11101000010 (bin) => 1858 (decimal) => tribe
01110101010 (bin) => 938  (decimal) => inspire
10001001100 (bin) => 1100 (decimal) => maze
01100110000 (bin) => 816  (decimal) => gravity
11001100101 (bin) => 1637 (decimal) => smile
11011001011 (bin) => 1739 (decimal) => sunny
11100001100 (bin) => 1804 (decimal) => thunder
11010100001 (bin) => 1697 (decimal) => stage
11011100000 (bin) => 1760 (decimal) => swing
10101010111 (bin) => 1367 (decimal) => priority
10100101001 (bin) => 1321 (decimal) => pioneer

There are still 7 unused bits, the last ones: 0111010.
The SHA-256 hash of the hex string is 5355d54a3d673c4b1ac20b839ead09af3c6fea6dc24199b477f6ff64e7a68262.
The first 4 bits of that are represented by the first hex character, 5, which in binary is 0101.
I append those 4 bytes to my binary string:
111010000100111010101010001001100011001100001100110010111011001011111000011001101010000111011100000101010101111010010100101110100101
Now I have the last 11-bit word:
01110100101 (bin) => 933 (decimal) => input

But when I try to validate that against Ian Coleman's Bip39 I see the checksum should be 0011 (not 0101) and that would make the last word be
01110100011 (bin) => 931 (decimal) => inner

What am I doing wrong?

The SHA-256 hash of the hex string is 5355d54a3d673c4b1ac20b839ead09af3c6fea6dc24199b477f6ff64e7a68262. The first 4 bits of that are represented by the first hex character, 5, which in binary is 0101.

You need to take the shasum of the raw binary, not the hex string:
$ echo "11101000010011101010101000100110001100110000110011001011101100101111100001100110101000011101110000010101010111101001010010111010" | shasum -a 256 -b

Gives output
3043b02ceb37bb00ab48c91f76bb23d487e1ec822c90e1575967e8838a655f7c *-
When converting this hexstring to binary we get:
0011000001000011101100000010110011101011001101111011101100000000101010110100100011001001000111110111011010111011001000111101010010000111111000011110110010000010001011001001000011100001010101110101100101100111111010001000001110001010011001010101111101111100
The first four bits of this string are 0011, which is consistent with the output given by the Ian Coleman tool.

when I want "$ echo "11101000010011101010101000100110001100110000110011001011101100101111100001100110101000011101110000010101010111101001010010111010" | shasum -a 256 -b" convert python language to "sha256_hash = hashlib.sha256(data.encode()).hexdigest()", it return "7529ea5a2f86b91b4b72e41aa36c190e8173e2fef6bbda828f74e55022eefec8", wrong result, BUT I don't know which error came from?


text:
Starting bitcoind for the express purpose of creating a wallet?
Referencing this solution:
My favorite method:

grab out a old/unused pc/mac
install a linux over usb (maybe Talis) (make sure your USB stick is clean!)
download a recent version of bitcoin-core (check hash, verify gitian signatures)
copy bitcoin-core over usb to your offline machine
start ./bitcoind
./bitcoin-cli getnewaddress (gives you a new public key)
./bitcoin-cli dumpprivatekey (gives you the corresponding private key)
print or write down both strings (address can be recreated from private key, but print it anyhow)
verify the printed or written down strings
try to delete printers cache (power off, reset)

Now you could delete/format the pc/mac.
But maybe it's worth keeping your new offline computer to generate new addresses or sign transaction in a "cold space/wallet".
I've installed bitcoin-core using snap on Ubuntu and ready to start a dry-run of this for reference.  However, disk space is limited, and, even then, I have no interest in downloading the blockchain itself.
How do I start bitcoind for generating a wallet?
I have:
nicholas@mordor:~$
nicholas@mordor:~$ bitcoin-core.cli status
error: Could not connect to the server 127.0.0.1:8332

Make sure the bitcoind server is running and that you are connecting to the correct RPC port.
nicholas@mordor:~$
nicholas@mordor:~$ bitcoin-core.cli getblockchaininfo
error: Could not connect to the server 127.0.0.1:8332

Make sure the bitcoind server is running and that you are connecting to the correct RPC port.
nicholas@mordor:~$
nicholas@mordor:~$ bitcoin-core.cli start
error: Could not connect to the server 127.0.0.1:8332

Make sure the bitcoind server is running and that you are connecting to the correct RPC port.
nicholas@mordor:~$

and am just looking to run bitcoin-core in pruning mode to play with.
I want to run a command like:
bitcoin-cli -testnet getblockchaininfo
Because I installed with snap I can run snap info bitcoin-core but man bitcoin-core doesn't give any results.
Starting the daemon:
nicholas@mordor:~$
nicholas@mordor:~$ bitcoin-core.daemon
2020-11-11T20:55:32Z Bitcoin Core version v0.20.1 (release build)
2020-11-11T20:55:32Z Assuming ancestors of block 0000000000000000000f2adce67e49b0b6bdeb9de8b7c3d7e93b21e7fc1e819d have valid signatures.
2020-11-11T20:55:32Z Setting nMinimumChainWork=00000000000000000000000000000000000000000e1ab5ec9348e9f4b8eb8154
2020-11-11T20:55:32Z Using the 'sse4(1way),sse41(4way)' SHA256 implementation
2020-11-11T20:55:32Z Default data directory /home/nicholas/snap/bitcoin-core/common/.bitcoin
2020-11-11T20:55:32Z Using data directory /home/nicholas/snap/bitcoin-core/common/.bitcoin
2020-11-11T20:55:32Z Config file: /home/nicholas/snap/bitcoin-core/common/.bitcoin/bitcoin.conf
2020-11-11T20:55:32Z Using at most 125 automatic connections (1024 file descriptors available)
2020-11-11T20:55:32Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
2020-11-11T20:55:32Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
2020-11-11T20:55:32Z Script verification uses 1 additional threads
2020-11-11T20:55:32Z scheduler thread start
2020-11-11T20:55:32Z HTTP: creating work queue of depth 16
2020-11-11T20:55:32Z No rpcpassword set - using random cookie authentication.
2020-11-11T20:55:32Z Generated RPC authentication cookie /home/nicholas/snap/bitcoin-core/common/.bitcoin/.cookie
2020-11-11T20:55:32Z HTTP: starting 4 worker threads
2020-11-11T20:55:32Z Using wallet directory /home/nicholas/snap/bitcoin-core/common/.bitcoin/wallets
2020-11-11T20:55:32Z init message: Verifying wallet(s)...
2020-11-11T20:55:33Z Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
2020-11-11T20:55:33Z Using wallet /home/nicholas/snap/bitcoin-core/common/.bitcoin/wallets
2020-11-11T20:55:33Z BerkeleyEnvironment::Open: LogDir=/home/nicholas/snap/bitcoin-core/common/.bitcoin/wallets/database ErrorFile=/home/nicholas/snap/bitcoin-core/common/.bitcoin/wallets/db.log
2020-11-11T20:55:33Z init message: Loading banlist...
2020-11-11T20:55:33Z Using /16 prefix for IP bucketing
2020-11-11T20:55:33Z Cache configuration:
2020-11-11T20:55:33Z * Using 2.0 MiB for block index database
2020-11-11T20:55:33Z * Using 8.0 MiB for chain state database
2020-11-11T20:55:33Z * Using 440.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
2020-11-11T20:55:33Z init message: Loading block index...
2020-11-11T20:55:33Z Opening LevelDB in /home/nicholas/snap/bitcoin-core/common/.bitcoin/blocks/index
2020-11-11T20:55:33Z Opened LevelDB successfully
2020-11-11T20:55:33Z Using obfuscation key for /home/nicholas/snap/bitcoin-core/common/.bitcoin/blocks/index: 0000000000000000
2020-11-11T20:55:43Z LoadBlockIndexDB: last block file = 186
2020-11-11T20:55:43Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=338, size=108899992, heights=325785...326349, time=2014-10-17...2014-10-21)
2020-11-11T20:55:43Z Checking all blk files are present...
2020-11-11T20:55:43Z LoadBlockIndexDB(): Block files have previously been pruned
2020-11-11T20:55:43Z : You need to rebuild the database using -reindex to go back to unpruned mode.  This will redownload the entire blockchain.
Please restart with -reindex or -reindex-chainstate to recover.
: You need to rebuild the database using -reindex to go back to unpruned mode.  This will redownload the entire blockchain.
Please restart with -reindex or -reindex-chainstate to recover.
2020-11-11T20:55:43Z Aborted block database rebuild. Exiting.
2020-11-11T20:55:43Z Shutdown: In progress...
2020-11-11T20:55:43Z scheduler thread exit
2020-11-11T20:55:43Z Shutdown: done
nicholas@mordor:~$

what throws me for a loop is that if I sudo into root there's a bitcoin.conf file in snap -- but there's also a bitcoin.conf file for my user account!  I have the GUI open now so will play with it.

Have you started bitcoind at all? Anything in the debug.log file?

I installed with snap, and, frankly, this hdd isn't suitable for blockchain usage.  I might have to just use different hardware.  But, to answer your question, @PieterWuille I'm not sure how to start bitcoind when installed from snap.  What you see above was my attempt to ascertain if it was even running.

You start the snap binary using bitcoin-core.daemon (or the GUI using bitcoin-core.qt). The bitcoin-core.cli tool is to communicate with a running daemon.

ah, that's the answer to my poorly asked question @PieterWuille


text:
How to generate a very secure wallet with Electrum?
For a poor-mans glacier protocol, looking for:
One private key – an alphanumeric string used to secure the funds
The cold storage address – an alphanumeric string designating the virtual “location” of the funds
The “redemption script” – an additional code needed to access funds, shared by all private keys.

but can I accomplish something similar with the Electrum wallet?  Based on IRC discussions, I've been pointed to libsecp256k1 for generating the keys.
see also:
https://github.com/spesmilo/electrum-docs/blob/master/libsecp256k1-linux.rst
Where I get stuck is with the what, precisely, to write down and what medium to use.  Because writing down a private key in its entirety seems a questionable endeavor, at best.  so, it's a matter of noting the "words" used to deterministically re-generate the private key?

In that context, how do I use electrum to generate a wallet approaching the security provided by something like the glacier protocol?  Assuming all else is equal, particularly with regards to quarantined hardware et. al.

additional context:

There are two types of digital wallets: hot wallets and cold wallets.
Hot wallets are connected to the internet while cold wallets are not.
Most digital wallet holders hold both a hot wallet and a cold wallet.
Hot wallets are most often used to make quick payments, while a cold
wallet is generally used for storing and holding your money, and has
no connection to the internet.[citation needed] Another difference
that is apparent when comparing the types of digital wallets, or
e-Wallets, is the price. While most hot wallets are free, cold wallets
can be expensive.[citation needed]

https://en.wikipedia.org/wiki/Digital_wallet#Types

the seed can be written down as a paper wallet.  the seed can then be used to generate the private key, and the private key to generate the public key.  or so is my understanding.


text:
What needs to be written down for a paper wallet?
I'm installing bitcoin-core with snap on Ubuntu for a dry-run at generating a paper wallet.
Perhaps naively, I thought to "write down" something like:
nicholas@mordor:~/bitcoin$
nicholas@mordor:~/bitcoin$ ls
nicholas@mordor:~/bitcoin$
nicholas@mordor:~/bitcoin$ ssh-keygen -t ed25519 -C saunders.nicholas@gmail.com
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/nicholas/.ssh/id_ed25519): ./bitcoin_wallet
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in ./bitcoin_wallet
Your public key has been saved in ./bitcoin_wallet.pub
The key fingerprint is:
SHA256:MtZQUFLqDWU0fJbkFhner+ZRF1WZ1MacM0WOBbGPvCc saunders.nicholas@gmail.com
The key's randomart image is:
+--[ED25519 256]--+
|      o*O.++  +B&|
|       *.+=o   @*|
|      +  o+ . oo+|
|     . = .   o o.|
|      = S     = o|
|     . o     o o |
|            + E .|
|           o . o |
|            .    |
+----[SHA256]-----+
nicholas@mordor:~/bitcoin$
nicholas@mordor:~/bitcoin$ ls
bitcoin_wallet  bitcoin_wallet.pub
nicholas@mordor:~/bitcoin$
nicholas@mordor:~/bitcoin$ cat bitcoin_wallet
-----BEGIN OPENSSH PRIVATE KEY-----
b3...AAtzc2gtZW
QyNTUxOQAAACD...AAAKBLLY/ISy2P
yAAAAAt...jpB6ZXAZLiLoXx+D9kw
AA..JYB62mUlxCESA
KyOkHp...LmNvbQEC
-----END OPENSSH PRIVATE KEY-----
nicholas@mordor:~/bitcoin$
nicholas@mordor:~/bitcoin$ cat bitcoin_wallet.pub
ssh-ed25519 AAA..IuhfH4P2T saunders.nicholas@gmail.com
nicholas@mordor:~/bitcoin$

where I need the public and private key.  Obviously, very easy to make a mistake, but isn't that what's required??
I'm not understanding all this mention of writing down seed words or passphrases.  Isn't the key pair required?

leaving aside QR codes or other automated tools for storing keys.

Bitcoin Core doesn't really support paper wallets. It's possible to kind of do it if you really know what you're doing, but it's not advisable. The supported mode of operation is backing up your wallet.dat file.

The design/structure of a paper wallet is not something is set in stone, but the main idea is writing public and private key pairs on a piece of paper.
The universal flow for generating an address is:

A public key is derived from private key
A bitcoin address is derived from public key

The common flow (Mnemonic) for generating an address is:

Master private key is derived from mnemonic phrases
A private key is derived from master private key
A public key is derived from private key
A bitcoin address is derived from public key

As you can see 3rd and 4th steps of Mnemonic key-generation flow are the same as universal key-generation flow. You can unlock your Bitcoin with plain private key without mnemonic phrases. Mnemonic phrases are just used to derive private key, so you can simply write down public key(65-byte or 33-byte) and private key(32-byte) onto a piece of paper regardless how you generated your key-pairs.