text:
bitcoin block time analysis with conjuction to propagation
For both scenarios that I will describe below, assume the 2 things.

that network propagation for blocks always 6 seconds.

that we start looking at my analysis from time = t0 and at that t0, every node has the same exact chain.

Scenario 1: block time = 10 min
If block time is 10 min and minerA solves blockD at time t0 + T, and shares it immediatelly, during the time t0+T and t0+T+6, others miners might solve their own block that they were working on. Let's call the variable how many miners solve it during this time to be "X"
Scenario 2: block time = 4 min
If block time is 4 min and minerA solves blockD at time t0 + T, and shares it immediatelly, during the time t0+T and t0+T+6, others miners might solve their own block that they were working on. Let's call the variable how many miners solve it during this time to be "Y"
As it turns out, Y > X.
Question 1: Am I right that Y would be greater than X ? ofc, not in 100% cases, but in terms of probability.
Question 2: How do I make myself sure that it's mathematically true that Y > X ? I know it must be about how difficulties and target are set. It seems like the less difficulty, The higher the chance to solve a block during ANY 6 second period. (NOTE the word: "ANY"). This is important because we don't know exactly when minerA solves the block, but as I've read, probability that Y > X is true for ANY 6 second period, and this 6 second period doesn't have to be closer to 4 minutes or 10 minutes or whatever it is. What would be a mathematical approach to this so I believe in this ?

Am I right that Y would be greater than X [...] in terms of probability.

block time := actual interval between recent blocks
Not if by "block time" you refer to a recent interval between the datestamps on two successive blocks. Or between the arrival times at one node of two successive blocks.
It is entirely possible that three successive blocks are produced with successive intervals of 10 and 4 minutes.
The probability of any miner succeeding in producing a hash less than the network target in any small interval, like 6s, is proportional to their hashpower. It is also inversely proportional to the current network difficulty. It is not affected by recent intervals between block publication.
I think the maths supporting this can be found by using this sites search facility and looking for "poisson"

What is the formula for inferring hash rate from difficulty and block frequency?

block time := target for average interval between blocks
To make the average of intervals shorter, the network would have to set a lower difficulty. At the least, this would be the simplest and most obvious way to cause that result so far as I know.
As noted above, probability of a hash being less than a target is inversely proportional to the difficulty. So the probability would be higher.
This sort of change alters the ROI of miners and would affect the number of miners and the amounts they were willing to periodically invest in new or additional equipment. I think the dynamics of this might mean that a miner who doesn't invest in extra equipment might not see the expected change in success rates endure for long.

by block time, I mean, the bitcoin's 10 minutes block time that in 2008, satoshi decided. I think that block time is averagely 10 min because target gets adjusted such as it takes averagely 10 min to solve the block.  and i bring 2 scenarios, what if target were adjusted always to cause 4 min block time averagely instead of 10. then read the question again and In this definition of block time, does anything in your answer change ?

@Giorgi: Yes it does change my answer. See above.

ah, so I was right claiming that Y > X(I meant block time as in your 2nd approach) . correct ?


text:
Is every copy of the blockchain the same? If I have an error within file should I be able to copy this file from a backup and reindex chainstate?
I recently moved the first 2000 files of my blockchain from the SSD to a HDD. The HDD gets mounted in /mnt/ext.
A symlink per file is then put into blocks that points to the remote files. So my blocks folder is still on SSD, but the files are on HDD.
I tried a bitcoind -reindex-chainstate to be sure everything still works but now it is giving an error:
2020-11-26T12:52:24Z ERROR: ReadBlockFromDisk(CBlock&, CBlockIndex*): GetHash() doesn't match index for CBlockIndex(pprev=0x44288c0, nHeight=255, merkle=4309bfeed77a70f309da08bcf8948906b9cc26120c0b0ef86e0ac67284bbd79e, hashBlock=00000000d0a75c861fabf9ff7b92022f60e4afeed9331fe5aa073d8e4706fe3c) at FlatFilePos(nFile=0, nPos=58808)
2020-11-26T12:52:24Z *** Failed to read block

Now I have several questions. I think nFile=0 means that the block is in blk00000.dat and rev00000.dat. Am I right?
So I copied this file again from a backup of the blockchain. This is from a second node. But I would suspect the files are the same, right?
But I get the exact same error at the same position - even with a new -reindex-chainstate.
I can see in the log that all blocks up until number 254 get indexed but each time it fails on block with height 255.
I think those old files contain always the same blocks. How can it be 254 blocks are correct but it fails on the next one?
Could this simply be a problem of the mount?

EDIT: I am currently -reindexing. In this case the files I already have linked to from my blocks folder get used like this:
Reindexing block file blk00000.dat...
Loaded 119964 blocks from external file in 133291ms

Apparently, the first file contains 119,964 blocks. So when I have an error with height=255 it should definitely be in the first file.

So I copied this file again from a backup of the blockchain. This is from a second node. But I would suspect the files are the same, right?

No. Blocks are downloaded out of order, but are written to disk in the order that they are downloaded. So the block index (which states where each block is located) will be different for every node because every node will likely have the blocks stored in a slightly different order.
Furthermore, if your node has received stale blocks, those stale blocks are also stored on disk and have an entry in the block index. Not all nodes will have the same stale blocks, and they may not be stored in the same order. Newly synced nodes will not have stale blocks from before they were synced, and often times stale blocks become stale due to propagation delays, so a node may not even receive a stale block to store.
You can't just copy individual block files from one node to another. You need to copy the entire blockchain and the block index. Otherwise you will probably corrupt the whole thing and need to redownload the blockchain.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
What is the difference between PPS+ and PPS?
Some pools do offer such a payment method as PPS+. This method is claimed as better than the PPS method, it should insure against losses on orphans and blocks with bad luck. But this explanation is too fuzzy and obscure for me so I do not understand how to calculate exact payments.
Is it true that the PPS+ is not only insure against the losses on the bad-luck blocks but also prevent to earn a surplus on the block with high luck? And I need a formula to calculate rewards on per-block basis.

PPS+ meaning that you receive payment per each share + block fee reward distributed using PPLNS.

So payment for a block will be block_reward*my_shares/total_block_shares + PPLNS, is it correct?

Block reward = Block reward + all transactions fee. PPS+ = block_reward*my_shares/total_block_shares + PPLNS for  all transactions fee.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Is there an easy way to check out the market cap of the alternative cryptocurrencies?
Ideally, I'd like to see a pie chart with the whole pie being "Bitcoin-derived cryptocurrencies", and each major alternative represented as a market cap in Bitcoins and USD.
Currently, from what I gather we have:

Bitcoin - 7,415,750 BTC = $34,854,025 = 99.7%
Namecoin - 959,150 NMC = 25,897 BTC = $121,716 = 0.3%

I don't know where to get data on other cryptocurrencies right now, and if they even have any significant market cap.

Now that I think of it, building a site that shows up to date stats on this is super easy with a bit of javascript (at least for these two). I might make one this weekend if I find a few hours.

And ... done! http://bitcoin.stackexchange.com/questions/1385/is-there-an-easy-way-to-check-out-the-market-cap-of-the-alternative-currencies/1399#1399

http://dustcoin.com/mining is the only site I know right now that shows this information.

Introducing ... Bitcoin Pie.
Supported Coins are Bitcoin, Tenebrix, NameCoin and SolidCoin. If your favorite coin is not supported, point me towards an active exchange (with or without an API), and a block explorer, and I'll integrate it.

Good stuff! One of these days I really must learn regex. I get 99% of your code but the regex might as well be written in Sanskrit :P

@David - Really? The regex is the simplest part of the code to me. There are a lot of inner closures there, I assumed that would be most complex.

It's not necessarily that the regex is complex for what it is, it's just that I never ever bothered to learn it at all. Not sure how I got this far in my career without learning it but perhaps it's time... I'm off to find some good resources.

Difficulty/hashrate would be a great thing to have in there too, since it approximates network security.  Great site, though!

@eMansipater - I'll put those up when I get a chance (code is open source, anyone is invited to send a pull request)

Site is dead right now, I haven't maintained it in a while (pull requests welcome).

At the moment I don't believe there is any such site. It wouldn't be too hard to create, I might even do it myself, I just need to find a good data source for pricing.
The coin counts can be found easily here:

Bitcoin
Fairbrix
GeistGeld
I0coin
Ixcoin
Namecoin
SolidCoin
Tenebrix

Given an API to pull price data from, calculating the numbers would be trivial, after which it's just a matter of how pretty a chart you'd like. I have a question here if anyone knows of such a data source at which point I'd be happy to whip up just such a chart.

I beat you to it (only Namecoin and Bitcoin are supported so far)

Eh, Namecoin is easy. Wake me up when you've got APIs for all the others ;)

Added Tenebrix :) I'm surprised at its relatively high cap. When API fails, I'll go regex (already implemented it for Namecoin)

Added SolidCoin.

It's 2021 and none of the existing answers work.
Behold coinmarketcap.

Why is this downvoted? This is literally the only answer that actually works in 2021. Downvoter, don't cower behind your computer: speak up.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Why is the full Merkle path needed to verify a transaction?
If I have transaction A and wish to verify that it is in the block, and I also have the hash of transaction B and the hash of the hashes of A and B, then can't I just hash A and then see if what I get by hashing A's hash and B's hash is equal to what is in the Merkle tree? Why do I have go all the way to the root?

If you have A, HB and HAB you can obviously check whether A fits. As Jestin noted this is essentially a full Merkle tree with two leaves.
However, as a thin client, you only have readily available the Merkle root (which is in the block header) and get told about A. The intermediate levels of the Merkle tree are not provided, therefore, to calculate them, you'd need the block's complete set of transactions.

Image via Mastering Bitcoin
So, for a thin client, we calculate the Merkle branch instead. For the Merkle branch, we just need the transaction's position in the block's transaction list and the hashing partners at each level, instead of the complete set of transactions. A Merkle branch is impractical to fake because it would require finding of a hash collision (which is not doable, or mining wouldn't work). So by going up the tree and combining our result with the respective hashing partner at each level, we finally get the Merkle root. Thus we can prove membership of the transaction in the block.
In the example image, you only need to provide the blue information to link HK to the Merkle root, whereas checking the whole Merkle tree would require all transactions from  A to P and the Merkle root.

But don't you need the path which in fact consists of intermediate levels?

@Jeff: I've edited to clarify. :)

I assume the client who wants to verify the existence of the transaction tells the full node (?) the transaction id (?) and the full node then provides the path? What makes the client believe a given transaction is in a particular block? And when would we first detect that transaction K is not in the block? When you say "blue" I think you mean solid blue?

Thin clients only get the block headers usually. They give a bloom filter to the full nodes which resolves to transactions they are interested in. In response full nodes tell them about transactions that are found with the bloom filter. The thin client then requests the Merkle branch for any transaction they are interested in. It is hard to prove that a transaction is not in a block, but it is easy to prove that it is. If nobody can provide proof that K is in the block, it probably isn't. –– Yes, I meant solid blue.

I am still not understanding why we need the full Merkle branch: why would not a single hashing partner be sufficient?

@Jeff: If you were trying to verify that K is in the block, and you only get HL, you can merely calculate HKL, but you don't know whether HKL is actually in the tree. To check that, you have to hash it with HIJ, but again, you don't know if the resulting HIJKL is in the tree. So, you go up until you can calculate the Root, because if your calculation matches the information that you have from the block header, you know that the transaction fits the block header.

@Jeff: Meanwhile, you're fairly sure that the block header is correct, because it adheres to the difficulty requirement and it would be quite the investment to fake a block header that adheres the difficulty requirement. Even better when you learn about new block headers that reference the block header of your transaction's block and also adhere the difficulty requirement.

Thanks for your explanation. Is there an example with a small tree that demonstrates this?

@Jeff: The example I gave corresponds to the image in the answer. Or what do you mean?

Well, one question is, one you say: You don't know whether HKL is actually in the tree, does this mean that a "malicious" full node might deliberately give you a false hashing partner?

@Jeff: If they give us a wrong hashing partner, a few levels further up the result will not match the Merkle root, unless they managed a successful preimage attack i.e. found a second input that creates the same given output. The latter is thought to be infeasible for SHA-256d.

When starting with Hk and given Hl, how does one know which of  (Hk + Hl) or (Hl + Hk) to compose when checking back up to the root?  Does the Merkle Path include a branch-left/right indicator?

@GoZoner: Good question! The full information set includes the transaction you're getting the branch for, the transaction's position in the block's transaction list and the list of its hashing partners up to the Merkle root. You can calculate from the position whether it's the left or right hashing partner for each hashing partner. — I've updated my answer.

Merkle trees are used extensively by SPV nodes. SPV nodes don’t have all transactions and do not download full blocks, just block headers. In order to verify that a transaction is included in a block, without having to download all the transactions in the block, they use an authentication path, or a merkle path. To understand why we need authentication path, you need to understand how Merkle trees work.
A merkle path is used to prove inclusion of a data element. A node can prove that a transaction K is included in the block by producing a merkle path that is only four 32-byte hashes long (128 bytes total). The path consists of the four hashes (shown with a blue background in A merkle path used to prove inclusion of a data element) HL, HIJ, HMNOP, and HABCDEFGH. With those four hashes provided as an authentication path, any node can prove that HK (with a green background at the bottom of the diagram) is included in the merkle root by computing four additional pair-wise hashes HKL, HIJKL, HIJKLMNOP, and the merkle tree root (outlined in a dashed line in the pic below)

Are those intermediate hashes (e.g. HMNOP) cached somewhere? Or do you also need to compute them from the other transactions in the block?

What you are proposing is essentially a Merkle tree with 2 transactions.  The hash of the hashes is the Merkle root, and the hash of B is the rest of the Merkle path.  You are going all the way to the root.
When you scale this up, you can provide the next highest node in the tree...but what verifies that it belongs in the tree?  You'll need to provide the hashes all the way up in order to verify, because it's the Merkle root that is hashed into the proof of work algorithm.

 But why would this fail if transaction A was one of 100s of transactions? If A were not in the block, then A's hash would not be in the hash of the hashes of A and B, right?

@Jeff, are you saying that 100 hashes would be concatenated and then hashed?  If so, that's a o(n) solution, compared to the o(log n) solution that a Merkle tree provides.

I may not be /probably am not understanding the procedure which involves a "Merkle path" but it seems to me that the absence of A in the block would become apparent without having to use all of the nodes in the path.

I think the part that you are missing is that intermediate nodes in the tree are not always available, only the root.  Therefore, the path back to the root must also be provided.  Check the "transaction-data" section of the developer guide (https://bitcoin.org/en/developer-guide#transaction-data), specifically the part about Simplified Payment Verification.


text:
IBM Hyperledger Fabric Running Multiple Nodes on Multiple Datacenters & PCs
At the moment I'm playing around with some of the Hyperledger Fabric demos that IBM has on their cloud platform Bluemix.
When you initially spool up an instance you're offered more than one option of location, such as US South, and... at least one more, I forget, California or something.
Running a blockchain application on someone's private cloud somehow feels a bit hollow.
I'd like to know if it's possible at least to have multiple nodes running in multiple IBM datacenters simultaneously maintaining a ledger between eachother and exposing an IP address to demonstrate where they're located.
Is it possible that I could add more nodes to such a network that I can control, i.e. my own machine?

In the recent technical steering committee meeting of Hyperledger they discussed the planning for an initiative to connect across multiple cloud providers, to demonstrates that Fabric can be deployed to multiple cloud platforms and integrated into a highly distributed and heterogeneous consortia network.
This effort (as of the time of writing) is under way with a number of community members interested in contributing.
You can read about it more in the meeting minutes here.


text:
Benchmarking on Hyperledger Fabric
Is there any public benchmarking code and results for Hyperledger Fabric v1.0 blockchain?
IBM claims a transactions rate of 3500 tps, but I am not able to find any public code of the benchmarking experiments.
Paper link: https://arxiv.org/abs/1801.10228

You can run Hyperledger Caliper https://www.hyperledger.org/projects/caliper/ , a benchmarking tool that supports Hyperledger Fabric (IBM), Hyperledger Sawtooth, Ethereum, and other blockchains.
Click on "Get the Code" link in the link above, or go to https://github.com/hyperledger/caliper for source code. It's thousands of lines of code, including benchmark blockchain apps, so that's too much to post. The original link above has documentation and is the main Caliper project page.

Just FYI, if the code is open-source and has a GitHub repo, you may want to link to it, and perhaps provide a short piece of code relevant to the question, if possible.  Short, link-only answers tend to suffer from link-rot, and become unhelpful.

That's the kind off detail it would be useful to have in the answer... use the [edit] link.  Cheers!


text:
HyperLedger Fabric from IBM is fork of Tangle or Etherium?
I read as if HL Fabric was the most interesting thing ever created on the use of blockchain, but when I read the documentation (very short) about Hyperledger Fabric I came across some coincidences.
the install is very common:
https://github.com/hyperledger/hyperledger
but the documentation mentions "tangle":
https://github.com/hyperledger
but the blog documentation:
https://hyperledger-fabric.readthedocs.io/en/latest/
is equal Ether documentation:
https://ethdocs.org/en/latest/ether.html
Does anyone know of any tool that I can compare the code of 3?

No, they are not related at all. The Hyperledger documentation is not short. The Hyperledger Github project does not mention Tangle as far as I can see. The only similarity with the Hyperledger documentation and Etherium documentation is that they are hosted on Read the Docs and look similar, the content is completely different. Hyperledger, Tangle, and Etherium are developed completely independently.

There is confusion here with the term Hyperledger.
Hyperledger is a Linux Foundation consortium of multiple open-source enterprise blockchain technologies (unrelated to Tangle). It includes:

Hyperledger Fabric, contributed by IBM (and not a fork of anything)
Hyperledger Sawtooth, contributed by Intel. Also not a fork. Both are Apache 2 licensed
Hyperledger Burrow an Ethereum VM (EVM) compatible blockchain (also included in Sawtooth)
other block chain technologies and tools

So you can see there is no forking--these are independent platforms with independent code bases (Fabric, Sawtooth, Burrow, Tangle, Ethereum).
The only relations are
- Ethereum and Burrow both implement EVM
- Hyperledger platforms are all under the Linux Foundation Hyperledger consortium, but are independent platforms and code bases.

Tangle is alternative distrubuted database design to Blockchain, it's got nothing to do with Hyperledger as far as I know.


text:
REST API documentation
I'm trying to use REST api of bitcoin core, i can do basic things such as getting latest block, or getting a transaction detail. But I couldn't find detailed explanation of the REST API. I checked the official release notes from here, but it's not complete as well.
I want to give an example, I couldn't find the information that I could get inputs and outpus of transaction when i request to tx/hash/populate instead of tx/hash, or i can get last block via block/tip, I want to find informations like this
I can see complete JSON RPC reference from here, is there a REST equivalent of this?
Last but not least, shall I use the rpc, or is it okey to use the rest api?
thanks, dante


text:
Getting error "non-mandatory-script-verify-flag (Witness program hash mismatch)" when trying to send raw signed transaction
I'm trying to send a fairly simple segwit transaction with multiple input and outputs using bitcoinlib, here's how the simplified code looks like:
from bitcoinlib.transactions import Output, Key, Transaction

# (prev_txid, output_n, address, value in satoshis)
inputs = [
    (u'4ab923ee400a3899dc0790c7658af9931e54fc40dc1a8a79281a989e6a56ddd2', 125, 'bc1p0val59xpljwfr52hdg667v0nnnjzusfmvgys6warv7hklrfjum7sjgj75j', 1077938),
]

outputs = []
outputs.append(Output(15260, address='bc1pylrxedd66287hl0e000qy9c86u9tagg206yvyymp4j2c6c5npfjseyjpdu'))

 # 1000 = 10.00 sat/vB
fee_per_kb = 1000

t = Transaction(outputs=outputs, witness_type='segwit', version=2)

for ti in inputs:
    ki = Key("PRIVATE_KEY")
    t.add_input(prev_txid=ti[0], output_n=ti[1], address=ti[2], value=ti[3],  keys=ki.public(), witness_type='segwit')

fee = round((t.estimate_size(1) * fee_per_kb) / 100)

total_inputs = 0
for ti in inputs:
    total_inputs += ti[3]

total_output = 15260
change = total_inputs - total_output - fee

# Add change output
t.add_output(change, address='bc1pyr9x8f7z5t9cxykfzzx66y2xc3davshzmd3xmf9dcxdpkg4tjuusschhpq')

icount = 0
for ti in inputs:
    ki = Key("PRIVATE_KEY")
    t.sign(ki.private_byte, icount)
    icount += 1

t.update_totals()
t.verify()
t.info()
print()
print("Raw Signed Transaction: %s" % t.raw_hex())

Transaction b2b765f26c3d8f766e205f1ae4bd82950f6283013ed23c0dd6d337c15b30d97a
Date: None
Network: bitcoin
Version: 2
Witness type: segwit
Status: new
Verified: True
Inputs
- bc1p0val59xpljwfr52hdg667v0nnnjzusfmvgys6warv7hklrfjum7sjgj75j 0.01077938 BTC 4ab923ee400a3899dc0790c7658af9931e54fc40dc1a8a79281a989e6a56ddd2 125
  segwit sig_pubkey; sigs: 1 (1-of-1) valid
Outputs
- bc1pylrxedd66287hl0e000qy9c86u9tagg206yvyymp4j2c6c5npfjseyjpdu 0.00015260 BTC p2tr U
- bc1pyr9x8f7z5t9cxykfzzx66y2xc3davshzmd3xmf9dcxdpkg4tjuusschhpq 0.01061148 BTC p2tr U
Size: 236
Vsize: 153
Fee: 1530
Confirmations: None
Block: None

Raw Signed Transaction: 02000000000101d2dd566a9e981a28798a1adc40fc541e93f98a65c79007dc99380a40ee23b94a7d00000000ffffffff029c3b00000000000022512027c66cb5bad28febfdf97bde021707d70abea10a7e88c21361ac958d62930a651c3110000000000022512020ca63a7c2a2cb8312c9108dad1146c45bd642e2db626da4adc19a1b22ab973902473044022019d43d372c8d6967f377f76d20a24d31fb3cb046a5640e9a02197afa526738dd02201afdc3e421665b35397ef2adf40908aeb3031569e49aab367285a9015effbd1f0121023f5bdab1fb426681f1a020998ffc69c844f380fe6e0d67b73e7fc9bc3291018b00000000

Everything seems ok when running the script and it even shows Verified as True but when I try to send it through bitcoin-cli I get a "non-mandatory-script-verify-flag (Witness program hash mismatch)" error.
I think I'm making a mistake signing my inputs but I'm not sure how, I checked the private key multiple times and it does match the address that currently holds the UTXO.
Can anyone advise what to do?
EDIT:
I just tried creating the same transaction manually with Sparrow using the same wallets and they look very different, I'm able to send the one from sparrow just fine:
Sparrow (raw unsigned)
0200000001d2dd566a9e981a28798a1adc40fc541e93f98a65c79007dc99380a40ee23b94a7d00000000f400000002083110000000000022512020ca63a7c2a2cb8312c9108dad1146c45bd642e2db626da4adc19a1b22ab97399c3b00000000000022512027c66cb5bad28febfdf97bde021707d70abea10a7e88c21361ac958d62930a6500000000
Sparrow (raw signed)
02000000000101d2dd566a9e981a28798a1adc40fc541e93f98a65c79007dc99380a40ee23b94a7d00000000f400000002083110000000000022512020ca63a7c2a2cb8312c9108dad1146c45bd642e2db626da4adc19a1b22ab97399c3b00000000000022512027c66cb5bad28febfdf97bde021707d70abea10a7e88c21361ac958d62930a65014076dca5b3231f6bca4b2142f2f388158f57ec61845edc34358f4b8d9da9532810512cbac2bb9ff49905c15f1a7bafab8cb9e7aa6a9761f9e8f611207a048b7c1600000000
bitcoinlib (raw unsigned)
02000000000101d2dd566a9e981a28798a1adc40fc541e93f98a65c79007dc99380a40ee23b94a7d00000000f4000000029c3b00000000000022512027c66cb5bad28febfdf97bde021707d70abea10a7e88c21361ac958d62930a65083110000000000022512020ca63a7c2a2cb8312c9108dad1146c45bd642e2db626da4adc19a1b22ab97390000000000
bitcoinlib (raw signed)
02000000000101d2dd566a9e981a28798a1adc40fc541e93f98a65c79007dc99380a40ee23b94a7d0000006a47304402203a0962afe594a0f1d15ff1b8a55e72c566157c1ac02af659509c925b9122d83e02207533c881f368c89be823fbcaed39a431670b9a6c141829bf8f57c73b00eb0c3e0121023f5bdab1fb426681f1a020998ffc69c844f380fe6e0d67b73e7fc9bc3291018bf4000000029c3b00000000000022512027c66cb5bad28febfdf97bde021707d70abea10a7e88c21361ac958d62930a65083110000000000022512020ca63a7c2a2cb8312c9108dad1146c45bd642e2db626da4adc19a1b22ab97390000000000
EDIT 2:
If I add script_type="p2wpkh" to my inputs then the transaction looks more similar to Sparrow's but the witness data is completely different:
Sparrow
"inputs": [
        {
            "addresses": [
                "bc1p0val59xpljwfr52hdg667v0nnnjzusfmvgys6warv7hklrfjum7sjgj75j"
            ],
            "age": 835519,
            "output_index": 125,
            "output_value": 1077938,
            "prev_hash": "4ab923ee400a3899dc0790c7658af9931e54fc40dc1a8a79281a989e6a56ddd2",
            "script_type": "pay-to-taproot",
            "sequence": 244,
            "witness": [
                "76dca5b3231f6bca4b2142f2f388158f57ec61845edc34358f4b8d9da9532810512cbac2bb9ff49905c15f1a7bafab8cb9e7aa6a9761f9e8f611207a048b7c16"
            ]
        }
    ],

bitcoinlib
"inputs": [
        {
            "addresses": [
                "bc1p0val59xpljwfr52hdg667v0nnnjzusfmvgys6warv7hklrfjum7sjgj75j"
            ],
            "age": 835519,
            "output_index": 125,
            "output_value": 1077938,
            "prev_hash": "4ab923ee400a3899dc0790c7658af9931e54fc40dc1a8a79281a989e6a56ddd2",
            "script_type": "pay-to-taproot",
            "sequence": 244,
            "witness": [
                "30440220387c744f51688777465cb0621d94a419c7b92fe6f9b75c66186ffba46eed0f680220383da6b94e24e3ef80b0c6287b28b5f5befece93a14e3541d2b539d1cd5b0cd201",
                "023f5bdab1fb426681f1a020998ffc69c844f380fe6e0d67b73e7fc9bc3291018b"
            ]
        }
    ],

I couldn't figure this out but I switched to using bitcoinutils instead and it worked perfectly.


text:
Manual Bitcoin transaction assembly: which output type should be used and how to compose a segwit field?
I try to sign the following transaction:
020000000001012ca7795bc5ca18f0fd5a1ffb27f31d693b813f6b73b0fd31f4868f88f2d727980100000000ffffffff01e803000000000000160014c221f41ace22b7272a0ab3289bb2df3c36a9e0250000000000

Parsed:
{
"addresses": [
    "tb1q4rmg2ah0f5skv9yg43kwzgp457430navtmvwv6",
    "tb1qcgslgxkwy2mjw2s2kv5fhvkl8sm2ncp9xanaux"
],
"block_height": -1,
"block_index": -1,
"confirmations": 0,
"double_spend": false,
"fees": 0,
"hash": "01071bbceb3ab921873325bfadef0382fba0e6a74709f96cc4f8799af635cab3",
"inputs": [
    {
        "addresses": [
            "tb1q4rmg2ah0f5skv9yg43kwzgp457430navtmvwv6"
        ],
        "age": 2868502,
        "output_index": 1,
        "output_value": 1000,
        "prev_hash": "9827d7f2888f86f431fdb0736b3f813b691df327fb1f5afdf018cac55b79a72c",
        "script_type": "pay-to-witness-pubkey-hash",
        "sequence": 4294967295
    }
],
"outputs": [
    {
        "addresses": [
            "tb1qcgslgxkwy2mjw2s2kv5fhvkl8sm2ncp9xanaux"
        ],
        "script": "0014c221f41ace22b7272a0ab3289bb2df3c36a9e025",
        "script_type": "pay-to-witness-pubkey-hash",
        "value": 1000
    }
],
"preference": "low",
"received": "2024-07-22T11:23:44.915543755Z",
"relayed_by": "44.192.67.93",
"size": 85,
"total": 1000,
"ver": 2,
"vin_sz": 1,
"vout_sz": 1,
"vsize": 83

}
I use the following input (index 1): https://blockstream.info/testnet/tx/9827d7f2888f86f431fdb0736b3f813b691df327fb1f5afdf018cac55b79a72c?expand
According to this doc (https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki#user-content-Native_P2WPKH), the pre-image hash is:
Double SHA256 of the serialization of:
 1. nVersion of the transaction (4-byte little endian)
 2. hashPrevouts (32-byte hash)
 3. hashSequence (32-byte hash)
 4. outpoint (32-byte hash + 4-byte little endian)
 5. scriptCode of the input (serialized as scripts inside CTxOuts)
 6. value of the output spent by this input (8-byte little endian)
 7. nSequence of the input (4-byte little endian)
 8. hashOutputs (32-byte hash)
 9. nLocktime of the transaction (4-byte little endian)
10. sighash type of the signature (4-byte little endian)

My pre-image before double sha256: 0200000032a17c92a47c5d4be8d6238132c848017f59ddacdac33b6c8d3c3ac4508a64ef3bb13029ce7b1f559ef5e747fcac439f1455a2ec7c5f09b72290795e706650442ca7795bc5ca18f0fd5a1ffb27f31d693b813f6b73b0fd31f4868f88f2d72798010000001976a914a8f68576ef4d21661488ac6ce12035a7ab17cfac88ace803000000000000ffffffff39faa805af04ff44720fa4e3de5476ee3cb833676531f48ad8ebf1e6d3001fc20000000001000000
After the signature: 020000000001012ca7795bc5ca18f0fd5a1ffb27f31d693b813f6b73b0fd31f4868f88f2d727980100000000ffffffff01e803000000000000160014c221f41ace22b7272a0ab3289bb2df3c36a9e025024830450221008dab2d55de0b7862b5f2f4bca0d8517915fad356ac38ae1f802034e6e9ec15aa02200f84f0f9e1a239a4bbd1bde5c3d8b0900d91ed0af9fd666922a794c182b658fb01210318e6317e8cb99d5be4fe863a63c9a365f58361f9196448c569a8ee3802ddd70300000000
Public Key: 0418e6317e8cb99d5be4fe863a63c9a365f58361f9196448c569a8ee3802ddd703cd7cd3767ad5330d27ef4db83adb02554decc2ed45fd7a16aa0cd663384693a7
Compressed Public Key: 0318e6317e8cb99d5be4fe863a63c9a365f58361f9196448c569a8ee3802ddd703
when I try to broadcast it, I get: error code: -25 error message: bad-txns-inputs-missingorspent
my questions are:

What should be the output script 76a914 output_address 88ac, or 0014 output_address? Why does this doc describe the legacy format of the output https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki#user-content-Native_P2WPKH ?

Is my pre-image correct?

is my segwit script for the input correct?
{
"stackitems": "02",
"0": {
"size": "48",
"item": "30450221008dab2d55de0b7862b5f2f4bca0d8517915fad356ac38ae1f802034e6e9ec15aa02200f84f0f9e1a239a4bbd1bde5c3d8b0900d91ed0af9fd666922a794c182b658fb01"
},
"1": {
"size": "21",
"item": "0318e6317e8cb99d5be4fe863a63c9a365f58361f9196448c569a8ee3802ddd703"
}

What type of public key must be used for address generation and segwit? Full or compressed?

What else can be wrong here? :)

Update:
I reduced the output amount:
Now pre-image is: 0200000032a17c92a47c5d4be8d6238132c848017f59ddacdac33b6c8d3c3ac4508a64ef3bb13029ce7b1f559ef5e747fcac439f1455a2ec7c5f09b72290795e706650442ca7795bc5ca18f0fd5a1ffb27f31d693b813f6b73b0fd31f4868f88f2d72798010000001976a914a8f68576ef4d21661488ac6ce12035a7ab17cfac88ace803000000000000ffffffff9ad50ca3ecd096afa146edd608c66350fd1546c36876d728004cc40428b474250000000001000000
The final transaction is: 020000000001012ca7795bc5ca18f0fd5a1ffb27f31d693b813f6b73b0fd31f4868f88f2d727980100000000ffffffff019001000000000000160014c221f41ace22b7272a0ab3289bb2df3c36a9e02502483045022100d13d1f8ec58cf0df2963986f36977bd7d12304bc69d375c8f02d959b23261f3702206d2e7f08eb3f67cb517a0ae9515ff22d676b944cf564d40ab54df6410b92a5c101210318e6317e8cb99d5be4fe863a63c9a365f58361f9196448c569a8ee3802ddd70300000000
And the error is: {"code":-26,"message":"non-mandatory-script-verify-flag (Signature must be zero for failed CHECK(MULTI)SIG operation)"}

How can I verify the signature?

How are you broadcasting it? Are you sure the node broadcasting it is on the right network (testnet), and is fully synchronized? The error indicates that the UTXO being spent does not exist, not a problem with the signature or witness.

I use bitcoin-cli -testnet sendrawtransaction.

Is the node you're sending the transaction to fully synchronized? If it hasn't synchronized to the point where the UTXO you're spending (9827d7f2888f86f431fdb0736b3f813b691df327fb1f5afdf018cac55b79a72c:1) is created, you would see this error.

Now I faced another error: {"code":-26,"message":"non-mandatory-script-verify-flag (Signature must be zero for failed CHECK(MULTI)SIG operation)"}

Ok that last error means the signature is actually invalid (or computed over the wrong sighash).


text:
"non-mandatory-script-verify-flag (Invalid Schnorr signature)" error despite successful Schnorr signature verification?
I am working on verifying a Schnorr signature for a Bitcoin transaction, and despite the signature verification succeeding, I'm encountering the following error when broadcasting the transaction:
Status Err("RPC error: Object {\"code\": Number(-26), \"message\": String(\"non-mandatory-script-verify-flag (Invalid Schnorr signature)\")}")

Here are the details of my transaction:
Unsigned Transaction Hex:
0200000000010274e1be2d0692b65bd22f61ab77af5840446529bb43c21f99d86461250126563e0000000000ffffffffc68b4cfa8bda7eb1f473b0c42c51bde3d62df313416b090bebc9b8f36b1c48250000000000ffffffff0417270000000000002251200ea2746ae09dfcd1bfbf0f1bc819f7d1192c8fefd65d4ce3a0f510737a18d61a10270000000000002251207c019aba4fe8376b2ca2e4afa89c6a995d973ae55039a505badf55cb3b8fd21b6ad0029500000000225120b8df80071cd758e45bb1345559936864c6ba38bb78663eb13e96b56a459ef3520000000000000000096a5d06008b01016301000247304402204317d0cbaa1f264f536ba1bca4fa66d8c5a54228b8935065f7bee4dc3f7ce32f02205ef2ae85228797824f3ad1e94ec6662992a3a83dcb485a638e7ba5843a8f24b4012102787804fc6954ad0777e2e0717ed775e5344bb9da6307c1d3f3b4df642782622b00000000

Signed Transaction Hex:
0200000000010274e1be2d0692b65bd22f61ab77af5840446529bb43c21f99d86461250126563e0000000000ffffffffc68b4cfa8bda7eb1f473b0c42c51bde3d62df313416b090bebc9b8f36b1c48250000000000ffffffff0417270000000000002251200ea2746ae09dfcd1bfbf0f1bc819f7d1192c8fefd65d4ce3a0f510737a18d61a10270000000000002251207c019aba4fe8376b2ca2e4afa89c6a995d973ae55039a505badf55cb3b8fd21b6ad0029500000000225120b8df80071cd758e45bb1345559936864c6ba38bb78663eb13e96b56a459ef3520000000000000000096a5d06008b0101630101407032ca6ad2a3e473f03dd594d9b071ff22bb12e2da4648d33e89a898e4fbd0c423f14a4ddf13f083413877487bc868ae8affb9f9a672bf54197c5b17391409a40247304402204317d0cbaa1f264f536ba1bca4fa66d8c5a54228b8935065f7bee4dc3f7ce32f02205ef2ae85228797824f3ad1e94ec6662992a3a83dcb485a638e7ba5843a8f24b4012102787804fc6954ad0777e2e0717ed775e5344bb9da6307c1d3f3b4df642782622b00000000

I've verified the Schnorr signature using the appropriate sighash, and everything checks out. Yet, when I try to broadcast the transaction, I receive the above RPC error.
What could be causing this error, and how can I resolve it?
Additional Context:

I'm working with SegWit v1 Taproot outputs.
The signature verification process uses the correct sighash.
The nLockTime and nSequence fields are properly set.

Any insights or suggestions would be greatly appreciated!
Code to generate  signature  and verify

pub fn sign_transaction(
    tx: &mut Transaction,
    priv_key: &PrivateKey,
    mpc_group_pub_key: String,
    funding_script: String,
) -> Result<Transaction, String> {

    let secp = Secp256k1::new();

    // Create sighasher using a mutable reference to tx instead of cloning it.
    let mut sighasher = SighashCache::new(tx.clone());
    //Vault Pub Key
    println!("MPC key :{:?}",mpc_group_pub_key);
    let prev_out_1 = ScriptBuf::from_hex(&format!("5120{}", mpc_group_pub_key)).unwrap();

    let prev_out_2 = ScriptBuf::from_hex(&funding_script).unwrap();

    // Define previous UTXOs
    let prev_utxo_0 = TxOut {
        value: 10007,
        script_pubkey: prev_out_1,
    };

    let prev_utxo_1 = TxOut {
        //   value: tx.output.get(1).unwrap().value,
        value: 2500000000,
        script_pubkey: prev_out_2,
    };

    let prev_utxo = vec![prev_utxo_0, prev_utxo_1];
    let prevouts: bdk::bitcoin::psbt::Prevouts<_> = Prevouts::All(&prev_utxo);

    let sighash: bdk::bitcoin::sighash::TapSighash = sighasher
        .taproot_key_spend_signature_hash(
            0,
            &prevouts,
            bdk::bitcoin::sighash::TapSighashType::Default,
        )
        .map_err(|e| e.to_string())?;

    let msg = Message::from(sighash);

    // Sign the sighash using the secp256k1 library (exported by rust-bitcoin).
    let key_pair = KeyPair::from_secret_key(&secp, &priv_key.inner);

    let signature = secp.sign_schnorr_no_aux_rand(&msg, &key_pair);

    // Update the witness stack.
    let signature = Signature {
        sig: signature,
        hash_ty: bdk::bitcoin::sighash::TapSighashType::Default,
    };
    println!("Signature :{:?}", hex::encode(signature.to_vec()));

    let witness = sighasher.witness_mut(0).ok_or("Failed to get witness")?;
    witness.push(&signature.to_vec());

    Ok(sighasher.into_transaction().clone())
}
let sk = SecretKey::from_str("8756415bbbbf1ce134ca7b97e2ca584cb52150783cb37f78cc41ccc7aacc10f1").unwrap();

let secp = Secp256k1::new();
let hex_msg = "0b5d2c9103c794eec24dd3dbbbfc8c8efc377d3fd613b958bdfc9d649166dc5f";
let msg = bdk::bitcoin::secp256k1::Message::from_slice(&hex::decode(hex_msg).unwrap()).unwrap();
let sig = bdk::bitcoin::secp256k1::schnorr::Signature::from_slice(&hex::decode("b9ca544040e9dee3b547b5c781ae13d15a5b20369e522b684c240a9e353b9b4bde1bca69642963801b33dd8ecc50c8714cfa4bb386390a867baabafde59dfd30").unwrap()).unwrap();

let pubkey = bdk::bitcoin::key::PublicKey::from_str("020ea2746ae09dfcd1bfbf0f1bc819f7d1192c8fefd65d4ce3a0f510737a18d61a").unwrap();

println!("Status {}", secp.verify_schnorr(&sig, &msg, &pubkey.to_x_only_pubkey()).is_ok());

Are you sure that the sighash is correct? That's the only reason why an otherwise correct signature would not validate.

@PieterWuille I added the sighash code now , kindly check, Also added verification code, Kindly check it

Does your signing implementation pass the BIP 340 test vectors?

@Murch I have attached the code to verify signature,Same vectors

@Murch I have added code to sign_transaction,


text:
Witness Program Hash Mismatch Error When Sending a Signed Transaction on Regtest
I am trying to send a transaction on my Bitcoin regtest node, but I keep encountering the following error:
Status Err("RPC error: Object {\"code\": Number(-26), \"message\": String(\"non-mandatory-script-verify-flag (Witness program hash mismatch)\")}")

Here is the transaction in hex format:
01000000000102a2ffd68c47fb375a78e8e5a8c62a8a2bb978cebee138c01d7af081d0841d0140000000000001000000e1f85331fdecdcb29d3a43d6e63c447e37747e761b66030d69b9325fc48ff68c00000000000100000004a68601000000000022512036d5748a7817424eca90e0d5a9f41cc2a0c843528872ba9d05a20eabd09c4065ae5b01000000000024512251207062783e946190c8d1043a2491633cfb20ac91f06869514471bf98fe0245f34310270000000000002251204238ecdae980e5ce65c70d50043d58ef8014bb2ef7adf9bfa43ddadc33d493ad0000000000000000096a5d06008b0101630100024730440220448546af85c40d4be09872013ef1a8cfeb9516cd39cf0bc22be28e61eb666350022018f5be97709b5b980aa5ce81f8a27e45621444d6385c6548209912bc48b48d9701210301cc8e443c2c8bfc286f01d203c325b4088ecc1106cd7421321a720253fc91ad0c000000

My RPC URL is:
http://localhost:18443

Below is the decoded transaction:
{
    "txid": "e7b3866f253f2464b39893f3f74f8a7aabf810dd33444685fac0e64be6b7e89f",
    "hash": "099fb30f2ad1ae01ec974047b9a7c387c3b86bd8efc4d227e93f28ddd57bdb74",
    "version": 1,
    "size": 351,
    "vsize": 269,
    "weight": 1074,
    "locktime": 12,
    "vin": [
        {
            "txid": "40011d84d081f07a1dc038e1bece78b92b8a2ac6a8e5e8785a37fb478cd6ffa2",
            "vout": 0,
            "scriptSig": {
                "asm": "",
                "hex": ""
            },
            "sequence": 1
        },
        {
            "txid": "8cf68fc45f32b9690d03661b767e74377e443ce6d6433a9db2dcecfd3153f8e1",
            "vout": 0,
            "scriptSig": {
                "asm": "",
                "hex": ""
            },
            "txinwitness": [
                "30440220448546af85c40d4be09872013ef1a8cfeb9516cd39cf0bc22be28e61eb666350022018f5be97709b5b980aa5ce81f8a27e45621444d6385c6548209912bc48b48d9701",
                "0301cc8e443c2c8bfc286f01d203c325b4088ecc1106cd7421321a720253fc91ad"
            ],
            "sequence": 1
        }
    ],
    "vout": [
        {
            "value": 0.00100006,
            "n": 0,
            "scriptPubKey": {
                "asm": "1 36d5748a7817424eca90e0d5a9f41cc2a0c843528872ba9d05a20eabd09c4065",
                "desc": "rawtr(36d5748a7817424eca90e0d5a9f41cc2a0c843528872ba9d05a20eabd09c4065)#gp0fk5kf",
                "hex": "512036d5748a7817424eca90e0d5a9f41cc2a0c843528872ba9d05a20eabd09c4065",
                "address": "bc1pxm2hfznczapyaj5sur26naquc2svss6j3pet48g95g82h5yugpjs25muxj",
                "type": "witness_v1_taproot"
            }
        },
        {
            "value": 0.00089006,
            "n": 1,
            "scriptPubKey": {
                "asm": "1 51207062783e946190c8d1043a2491633cfb20ac91f06869514471bf98fe0245f343",
                "desc": "addr(bc1p2ys8qcnc862xryxg6yzr5fy3vv70kg9vj8cxs623g3cmlx87qfzlxsc3cylxd)#sp4wxhtc",
                "hex": "512251207062783e946190c8d1043a2491633cfb20ac91f06869514471bf98fe0245f343",
                "address": "bc1p2ys8qcnc862xryxg6yzr5fy3vv70kg9vj8cxs623g3cmlx87qfzlxsc3cylxd",
                "type": "witness_unknown"
            }
        },
        {
            "value": 0.0001,
            "n": 2,
            "scriptPubKey": {
                "asm": "1 4238ecdae980e5ce65c70d50043d58ef8014bb2ef7adf9bfa43ddadc33d493ad",
                "desc": "rawtr(4238ecdae980e5ce65c70d50043d58ef8014bb2ef7adf9bfa43ddadc33d493ad)#2x7uc7k6",
                "hex": "51204238ecdae980e5ce65c70d50043d58ef8014bb2ef7adf9bfa43ddadc33d493ad",
                "address": "bc1pgguwekhfsrjuuew8p4gqg02ca7qpfwew77kln0ay8hddcv75jwksufvfww",
                "type": "witness_v1_taproot"
            }
        },
        {
            "value": 0,
            "n": 3,
            "scriptPubKey": {
                "asm": "OP_RETURN 13 008b01016301",
                "desc": "raw(6a5d06008b01016301)#53e7sytq",
                "hex": "6a5d06008b01016301",
                "type": "nulldata"
            }
        }
    ]
}

Also, here is the detailed hex for the signed transaction:
01000000000102a2ffd68c47fb375a78e8e5a8c62a8a2bb978cebee138c01d7af081d0841d0140000000000001000000e1f85331fdecdcb29d3a43d6e63c447e37747e761b66030d69b9325fc48ff68c00000000000100000004a68601000000000022512036d5748a7817424eca90e0d5a9f41cc2a0c843528872ba9d05a20eabd09c4065ae5b01000000000024512251207062783e946190c8d1043a2491633cfb20ac91f06869514471bf98fe0245f34310270000000000002251204238ecdae980e5ce65c70d50043d58ef8014bb2ef7adf9bfa43ddadc33d493ad0000000000000000096a5d06008b01016301014020ef582e42d51c0879f5ef445272107ea3146be874a9370b1dbba3e3f3ded31eddc05af7b1afe06d3d597dd2c3ee3ebf04803a764835804e6a724f6fb1c63056024730440220448546af85c40d4be09872013ef1a8cfeb9516cd39cf0bc22be28e61eb666350022018f5be97709b5b980aa5ce81f8a27e45621444d6385c6548209912bc48b48d9701210301cc8e443c2c8bfc286f01d203c325b4088ecc1106cd7421321a720253fc91ad0c000000

Signed Txn

{
    "txid": "e7b3866f253f2464b39893f3f74f8a7aabf810dd33444685fac0e64be6b7e89f",
    "hash": "af75b76a6b5de75acdd4f27121939f574135c78e46807127ea3df118f09e7189",
    "version": 1,
    "size": 416,
    "vsize": 285,
    "weight": 1139,
    "locktime": 12,
    "vin": [
        {
            "txid": "40011d84d081f07a1dc038e1bece78b92b8a2ac6a8e5e8785a37fb478cd6ffa2",
            "vout": 0,
            "scriptSig": {
                "asm": "",
                "hex": ""
            },
            "txinwitness": [
                "20ef582e42d51c0879f5ef445272107ea3146be874a9370b1dbba3e3f3ded31eddc05af7b1afe06d3d597dd2c3ee3ebf04803a764835804e6a724f6fb1c63056"
            ],
            "sequence": 1
        },
        {
            "txid": "8cf68fc45f32b9690d03661b767e74377e443ce6d6433a9db2dcecfd3153f8e1",
            "vout": 0,
            "scriptSig": {
                "asm": "",
                "hex": ""
            },
            "txinwitness": [
                "30440220448546af85c40d4be09872013ef1a8cfeb9516cd39cf0bc22be28e61eb666350022018f5be97709b5b980aa5ce81f8a27e45621444d6385c6548209912bc48b48d9701",
                "0301cc8e443c2c8bfc286f01d203c325b4088ecc1106cd7421321a720253fc91ad"
            ],
            "sequence": 1
        }
    ],
    "vout": [
        {
            "value": 0.00100006,
            "n": 0,
            "scriptPubKey": {
                "asm": "1 36d5748a7817424eca90e0d5a9f41cc2a0c843528872ba9d05a20eabd09c4065",
                "desc": "rawtr(36d5748a7817424eca90e0d5a9f41cc2a0c843528872ba9d05a20eabd09c4065)#gp0fk5kf",
                "hex": "512036d5748a7817424eca90e0d5a9f41cc2a0c843528872ba9d05a20eabd09c4065",
                "address": "bc1pxm2hfznczapyaj5sur26naquc2svss6j3pet48g95g82h5yugpjs25muxj",
                "type": "witness_v1_taproot"
            }
        },
        {
            "value": 0.00089006,
            "n": 1,
            "scriptPubKey": {
                "asm": "1 51207062783e946190c8d1043a2491633cfb20ac91f06869514471bf98fe0245f343",
                "desc": "addr(bc1p2ys8qcnc862xryxg6yzr5fy3vv70kg9vj8cxs623g3cmlx87qfzlxsc3cylxd)#sp4wxhtc",
                "hex": "512251207062783e946190c8d1043a2491633cfb20ac91f06869514471bf98fe0245f343",
                "address": "bc1p2ys8qcnc862xryxg6yzr5fy3vv70kg9vj8cxs623g3cmlx87qfzlxsc3cylxd",
                "type": "witness_unknown"
            }
        },
        {
            "value": 0.0001,
            "n": 2,
            "scriptPubKey": {
                "asm": "1 4238ecdae980e5ce65c70d50043d58ef8014bb2ef7adf9bfa43ddadc33d493ad",
                "desc": "rawtr(4238ecdae980e5ce65c70d50043d58ef8014bb2ef7adf9bfa43ddadc33d493ad)#2x7uc7k6",
                "hex": "51204238ecdae980e5ce65c70d50043d58ef8014bb2ef7adf9bfa43ddadc33d493ad",
                "address": "bc1pgguwekhfsrjuuew8p4gqg02ca7qpfwew77kln0ay8hddcv75jwksufvfww",
                "type": "witness_v1_taproot"
            }
        },
        {
            "value": 0,
            "n": 3,
            "scriptPubKey": {
                "asm": "OP_RETURN 13 008b01016301",
                "desc": "raw(6a5d06008b01016301)#53e7sytq",
                "hex": "6a5d06008b01016301",
                "type": "nulldata"
            }
        }
    ]
}

I have verified the signatures and the public keys. Can someone help me identify the issue and fix this transaction error?
For the input of the account tx:
# bcmd -rpcwallet="mywallet" gettransaction 40011d84d081f07a1dc038e1bece78b92b8a2ac6a8e5e8785a37fb478cd6ffa2
{
  "amount": 0.00000000,
  "fee": -0.00009901,
  "confirmations": 162,
  "blockhash": "67b7007b27e35047f7c7dff6412c49458546bfc196b6208e72bef24380fcc44f",
  "blockheight": 143,
  "blockindex": 1,
  "blocktime": 1722672138,
  "txid": "40011d84d081f07a1dc038e1bece78b92b8a2ac6a8e5e8785a37fb478cd6ffa2",
  "wtxid": "287650b835e50a27f272834d1d13863745eaffce8c8913f46a2af4ab68e0425d",
  "walletconflicts": [],
  "time": 1722672138,
  "timereceived": 1722672138,
  "bip125-replaceable": "no",
  "details": [
    {
      "address": "bcrt1q7hz4y2eq3r6pj4y8fwvc53cyetv3k6ucfv4nhc",
      "category": "send",
      "amount": -0.00010006,
      "label": "",
      "vout": 0,
      "fee": -0.00009901,
      "abandoned": false
    },
    {
      "address": "bcrt1q7hz4y2eq3r6pj4y8fwvc53cyetv3k6ucfv4nhc",
      "parent_descs": [
        "wpkh(tpubD6NzVbkrYhZ4YA6tv2BJdjxPVGx3qgb8QVsTqxbB9YwEQY8dKej5FyX5wQXxVHtKxqQgSw7D9XucsnKZ2iodSFXM4EpjdENvruaGxyMp5Y4/84'/1'/0'/0/*)#a5rks7y5"
      ],
      "category": "receive",
      "amount": 0.00010006,
      "label": "",
      "vout": 0
    }
  ],
  "hex": "02000000000102dd9c61c543ef92e03a39f584b517a7fd10937bcc033e9704f833561a6eae2f9e0000000000fdffffffbaec46fce03948d7f19ad6d856867b366b0bc6b7e5a4b05d0c7d212aa5fa28ac0200000000fdffffff011627000000000000160014f5c5522b2088f41954874b998a4704cad91b6b9802473044022039f78eda2a2926e4e722ddcd09c866dd6183fb623001bbf63208d4206620b03302204af1a6f7c55b33ae2826995100bfc6b93dd0322b43745f23158c2f760d4e09d6012103e6b1b50b7f1ec077385a978a0be708e11767245251075cb7f88ba11645783031024730440220192d300622f517aef52505b5c4b2a4566e843b74688de43e96d21a7634c03bf702203781b12538ff5dcc3ed96f96029a4bb136814a79c07d27d73ee72fa09a0355e8012103e6b1b50b7f1ec077385a978a0be708e11767245251075cb7f88ba1164578303100000000"
}

For the input of the funding:
# bcmd -rpcwallet="faucet" gettransaction 8cf68fc45f32b9690d03661b767e74377e443ce6d6433a9db2dcecfd3153f8e1
{
  "amount": 25.00000000,
  "confirmations": 143,
  "generated": true,
  "blockhash": "30cb027232dfe5905b4c996908bf39f436e4cfd2855e5447bd46d210f20bb30b",
  "blockheight": 165,
  "blockindex": 0,
  "blocktime": 1722684106,
  "txid": "8cf68fc45f32b9690d03661b767e74377e443ce6d6433a9db2dcecfd3153f8e1",
  "wtxid": "4b0989ba070a394bf837b949def1d87dfd004ddf8212897d207b3977021c55cd",
  "walletconflicts": [],
  "time": 1722684104,
  "timereceived": 1722684104,
  "bip125-replaceable": "no",
  "details": [
    {
      "address": "bcrt1q23mtqw3hnsyvux5wr2kpwes6lkc3zsy5zaxzzm",
      "parent_descs": [
        "wpkh(tpubD6NzVbkrYhZ4YcdAPzo9agcpA4gyTtW1gZCKtEpcVnGfJ5Qg4txwbUFMfYs4RrMJXXf51MUsdj5mroJtZMuTWCrHhpdfdBUqVSrQp6XnHgs/84'/1'/0'/0/*)#peesvaay"
      ],
      "category": "generate",
      "amount": 25.00000000,
      "label": "faucet",
      "vout": 0
    }
  ],
  "hex": "020000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff0402a50000ffffffff0200f90295000000001600145476b03a379c08ce1a8e1aac17661afdb11140940000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000"
}

Hi Vinay, can you also include in your post the two transactions whose outputs are being spent?

@VojtěchStrnad Hi I have updated the post and posted the two UTXOs I am trying to spend. Your help and response is greatly appreciated.

I just noticed your first input has no witness data, that's probably not intentional?

@VojtěchStrnad Thats for unsigned Transaction it should be empty, Check one more hex I added its for Signed Txn

The witness stack of the first input only has one item, even though it's spending a P2WPKH output which requires two items. Since the item is 64 bytes long, I'd presume it's a Schnorr signature and you meant to spend a P2TR output instead.

I fixed it. but right now its giving me invalid schnorr signatue


text:
How to ./configure a built for Raspberry Pi?
I want to use this guide to set up a headless bitcoin core full node and came across the line:
./configure CPPFLAGS="-I/usr/local/BerkeleyDB.4.8/include -O2" LDFLAGS="-L/usr/local/BerkeleyDB.4.8/lib" --disable-wallet

Unfortunately, the blog does not explain why he uses this.
First of all, I want to use my node with wallet. Second, I want to configure it to be usefull with a Raspberry Pi which has only 1GB of RAM and is also running other stuff, for example my Logitech Media Server for audio streaming. So I would like to make the node as lean as possible.
I know that I can use a config file for running  bitcoind with entries for upload limit and other setups. Do I need to run the skript also with options or are both setups interchangeably?
Also, I get the error Found Berkeley DB other than 4.8 upon configure. Can Ihave a wallet with the BDB other than 4.8? The error message suggests, that 4.8 is needed for "portable wallets".
Here I found out about dependencies for the wallet and also about the option to built without GUI. Unfortunately, I cannot successfully ./configure with those neither, because BDB 4.8 is also not found this way.
However, to do it without GUI the option --without-guican be used, which is very usefull.

EDIT: I encountered another problem while make:
/usr/bin/ld: cannot find -ldb_cxx-4.8
collect2: error: ld returned 1 exit status
Makefile:4779: die Regel für Ziel „bitcoind“ scheiterte
make[2]: *** [bitcoind] Fehler 1
make[2]: Verzeichnis „/home/pi/Downloads/bin/bitcoin/src“ wird verlassen
Makefile:12975: die Regel für Ziel „all-recursive“ scheiterte
make[1]: *** [all-recursive] Fehler 1
make[1]: Verzeichnis „/home/pi/Downloads/bin/bitcoin/src“ wird verlassen
Makefile:776: die Regel für Ziel „all-recursive“ scheiterte
make: *** [all-recursive] Fehler 1
19:07:10 [pi@raspberrypi bitcoin]$ /usr/bin/ld: cannot find -ldb_cxx-4.8
-bash: /usr/bin/ld:: Datei oder Verzeichnis nicht gefunden

Here I found an answer to a related error and I used ld -ldb_cxx-4.8 --verbose and got this:
attempt to open //usr/local/lib/arm-linux-gnueabihf/libdb_cxx-4.8.so failed
attempt to open //usr/local/lib/arm-linux-gnueabihf/libdb_cxx-4.8.a failed
attempt to open //lib/arm-linux-gnueabihf/libdb_cxx-4.8.so failed
attempt to open //lib/arm-linux-gnueabihf/libdb_cxx-4.8.a failed
attempt to open //usr/lib/arm-linux-gnueabihf/libdb_cxx-4.8.so failed
attempt to open //usr/lib/arm-linux-gnueabihf/libdb_cxx-4.8.a failed
attempt to open //usr/local/lib/libdb_cxx-4.8.so failed
attempt to open //usr/local/lib/libdb_cxx-4.8.a failed
attempt to open //lib/libdb_cxx-4.8.so failed
attempt to open //lib/libdb_cxx-4.8.a failed
attempt to open //usr/lib/libdb_cxx-4.8.so failed
attempt to open //usr/lib/libdb_cxx-4.8.a failed
attempt to open //usr/arm-linux-gnueabihf/lib/libdb_cxx-4.8.so failed
attempt to open //usr/arm-linux-gnueabihf/lib/libdb_cxx-4.8.a failed
ld: cannot find -ldb_cxx-4.8

Now trying with sudo apt-get install gcc-4.8 g++-4.8. That did not help.
Maybe I have to try this guide: https://medium.com/@meeDamian/bitcoin-full-node-on-rbp3-revised-88bb7c8ef1d1

I want to add that with wallet it is necessary to sudo apt-get install libdb++-dev.

You might find this helpful https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md#ubuntu--debian

Yes, I found it, but unfortunately, it sais nothing about how to compile a debian version that includes BDB 4.8. It just sais: "Ubuntu and Debian have their own libdb-dev and libdb++-dev packages, but these will install BerkeleyDB 5.1 or later."

Have you tried https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md#berkeley-db

As configure worked, make now takes quite a while at  CXX      libbitcoin_server_a-init.o. Not sure if the process crashed or if it is just hard work for a small CPU.

Yes if I remember correctly I ran into that issue as well when I didn't have enough memory.

If it doesn't crash it's probably still compiling... Good luck!

You can follow the instructions for Debian at the build docs:
https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md
For Berkely DB, specifically:
Berkeley DB

It is recommended to use Berkeley DB 4.8. If you have to build it yourself,
you can use the installation script included in contrib/
like so:

./contrib/install_db4.sh `pwd`

from the root of the repository.
Note: You only need Berkeley DB if the wallet is enabled (see Disable-wallet mode).

To be able to make with just 1GB I needed to use a 4GB swap: fallocate -l 4G /swapfile && chmod 600 /swapfile && mkswap /swapfile && swapon /swapfile && swapon --show (https://github.com/bitcoin/bitcoin/issues/11820)

Following parts of this guide I was able to install a full node with wallet (without GUI) on my already running Raspberry Pi. I did not use all of the steps.
I did use my external hdd as swap:
sudo swapoff --all
sudo nano /etc/dphys-swapfile

Putting in the following
CONF_SWAPFILE=/mnt/hdd/swap
#CONF_SWAPSIZE=100
CONF_SWAPFACTOR=2

But I had to add this (as root) to make configuring work:
fallocate -l 4G /swapfile && chmod 600 /swapfile && mkswap /swapfile && swapon /swapfile && swapon --show

(It is also possible that the guide just missed the swapon --all after configuring /etc/dphys-swapfile.)
I installed dependencies according to the guide:
sudo apt install git build-essential libtool autotools-dev automake pkg-config libssl-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-program-options-dev libboost-test-dev libboost-thread-dev libminiupnpc-dev libzmq3-dev jq

But I have to say that I already installed a lot and probably more dependencies from the bitcoin github site.
I did ./contrib/install_db4.sh $(pwd) to install Berkeley DB 4.8.
I used the git I already cloned (in /home/Downloads/bin/bitcoin).
This was something I missed during the first attemps: ./autogen.sh.
Then I export BDB_PREFIX=$(pwd)/db4 (I am actually not sure if I did this first part which is included in the guide and is setting a variable which is used in the ./configure if I understand correctly.) But I definitely did
./configure BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" BDB_CFLAGS="-I${BDB_PREFIX}/include" CXXFLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768" --enable-cxx --without-gui --disable-shared --with-pic --enable-upnp-default

And then of course (which might take a few hours)
make
sudo make install

This time I had no errors and no hangs while configuring.
I also added the bitcoin user and a firewall as decribed in the guide. In my case I had to open up more ports for the firewall because my Raspberry is running other stuff too (SMB, LMS, minidlna).


text:
How to remove private keys from wallet.dat and create "receive only" wallet?
I want to create a wallet for monitoring incoming transactions. So it should be receive only wallet that will be place on hosting, that is potentially not secure. If I understand it well, this won't allow attacker to stole funds if server is compromised. How I can create such a wallet with only public keys? And can I inject private keys later to spend from that wallet from a secure machine?

possible duplicate of how to monitor +20,000 addresses for new received transactions?

7 years old and still no answer. I get "Keys can only be imported to the keypool when private keys are disabled" when trying to add a public key and google took me to this page with no answers

You're right, storing your private keys on a server is a bad idea - it means you're liable to lose all of your funds in the case of a hack.
But as you aren't storing them, do you need to install the bitcoin daemon on your server? You could use an API from one of the online blockchains to monitor transactions from your server, and maintain your encrypted wallet.dat on your own computer.

Thanks for your input unfortunately there are severe security issues with blockchain.info see https://bitcointalk.org/index.php?topic=199965.0

I'm not suggesting you use blockchain.info or one of their wallets - I'm suggesting you use the API of a blockchain website (there are a bunch of them, blockexporer.com being one) and store the addresses on your server. That way, when you are expecting a payment, you can run an API call, check whether a payment has been made to an address and voila. The private keys never touch the network so completely safe. Regardless of perceived blockchain.info wallet insecurities, their API is solid and if you cross-check with another API then you should be good.

Ah, got your idea. Thanks for the detailed answer!

Commands can be entered after opening Window -> Console
in Bitcoin Core
Create a new empty wallet with
createwallet 'wallet_name' true
https://bitcoincore.org/en/doc/0.17.0/rpc/wallet/createwallet/
Then use
importmulti command with "watchonly": true
https://bitcoincore.org/en/doc/0.17.0/rpc/wallet/importmulti/

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How to install bitcoind (Bitcoin Core) on Ubuntu?
I want to install a command line-only full node bitcoind on a mini PC with 4GB RAM and Ubuntu. What are the steps to do it and how can I check if it works?

useradd bitcoin and passwd bitcoin
link /home/bitcoin/.bitcoin to external HDD with chown -R bitcoin:bitcoin
git clone bitcoin core in /home/USER/GitStuff
add bitcoin ppa repository
sudo apt install libdb4.8-dev && sudo apt install libdb4.8++-dev
/autogen.sh in /home/USER/GitStuff/bitcoin
/configure --without-gui (to build without GUI pass --without-gui)
make
make install
su bitcoin
start bitcoind (as user bitcoin)
curl -sL https://bitnodes.earn.com/api/v1/nodes/me-8333 | jq

Dependencies
To get add-apt-repository, which we will use later to add the bitcoin PPA:
sudo add-apt-repository ppa:bitcoin/bitcoin
sudo apt-get update
sudo apt-get install bitcoind
Configuration
Before starting bitcointd you'll want to create directory and the bitcoin.conf configuration file:
cd ~/
mkdir .bitcoin
cd .bitcoin
vim bitcoin.conf or nano bitcoin.conf
In bitcoin.conf these content are important:
server=1
testnet=1
rpcuser=UNIQUE_RPC_USERNAME
rpcpassword=UNIQUE_RPC_PASSWORD

If you wish to use this installation on the regular bitcoin network, remove testnet=1

I think is an unnecessary command mkdir .bitcoin

When adding ppa:bitcoin/bitcoin it says: NOT MAINTAINED. The OS-library linking packages here had a series of issues.
PLEASE DOWNLOAD DIRECTLY FROM bitcoincore.org (and verify the signatures of said files).

The custom apt repository now states it is unmaintained, and to use the snap found here:
https://github.com/bitcoin-core/packaging/tree/main/snap
sudo snap install bitcoin-core

Config file wants to live at: /home/yourusername/snap/bitcoin-core/common/.bitcoin/bitcoin.conf

There is a guide in the docs of the bitcoin source code that goes over how to build bitcoin as well as the dependencies needed. Within that same directory of the source (doc/) there are various other guides and explanations that you may find useful as well.
Linux build doc

Thanks, I actually would have added this and others once I refine my answer. I just wanted to put it out there for a start yesterday.


text:
How to make sure Electrum connects to my private Electrs server and to no other server?
I want to connect my Electrum wallet app to my private Electrs server in my local LAN which is also reachable via onion. I want to just use this one server, no other server. My ~/.electrum/config is
{
    "alias": "",
    "auto_connect": false,
    "blockchain_preferred_block": {
        "hash": "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
        "height": 0
    },
    "check_updates": true,
    "config_version": 3,
    "gui_last_wallet": "/Users/$USER/.electrum/wallets/default_wallet",
    "is_maximized": false,
    "oneserver": false,
    "proxy": "socks5:127.0.0.1:9050::",
    "recently_open": [
        "/Users/$USER/.electrum/wallets/default_wallet"
    ],
    "rpcpassword": "",
    "rpcuser": "",
    "server": "something.onion:50002:s",
    "show_channels_tab": true
}

This seems to work fine in the sense that I am connected. But I have a lot of nodes in my list under the "Network" settings in Electrum. And this is because the oneserver property is set to false. If I try to set it to true in the config manually, Electrum starts and works fine but the config file is reverted to its old state with false in it.
How can I make sure this works?

I don't know why this happened several times. But in the end the config worked with true and now my nodes show only one connection and this is the private server.

I don't know id this is relevant in this case, but I was facing a similar issue.
I read this post, and when I invoked with -1 --server <host:port> the connection worked.
Turned out the issue was that my server value in the config was postfixed with :s which specified an ssl connection, whereas the server I set up was unencrypted tcp. (:t).
Even if this is not what happened to you, perhaps it can be useful to someone else if they read this with the same issue I was facing.


text:
How to get the block hash in the chainstate leveldb?
From this link What are the keys used in the blockchain levelDB (ie what are the key:value pairs)?,
to get the block hash I use the obfuscation key and xor it with the result I get from the db. So I look up the block hash I got on blockchain.com,
but the block hash doesn't exist. So I check if I'm xoring right by using the website http://xor.pw/#. My xored value matched the website.
Then I look back to the example on How does Bitcoin read from/write to LevelDB and xored their example
26c326d7353661dc7005d274976f458691f24f0f05d141335f4ad5927e41 and 27c78118b731610527c78118b731610527c78118b731610527c78118b731 on http://xor.pw/#
and I got 104a7cf820700d957c2536c205e2483b635ce17b2e02036788d548ac970 not 01028820010b2a00367244680f6da18acd861a08f0a89cb3b49ab50e.
Me and and the website can't get the same result and be wrong right? What am I missing?
import plyvel

def get_xor_key(o_key, result):
    xor_key = ''
    while len(xor_key) < len(result):
        if len(xor_key) + len(o_key) <= len(result): xor_key += o_key
        else: xor_key += o_key[:len(result)-len(xor_key)]
    return xor_key
def xor_two_str(s, t): return hex(int(s, 16) ^ int(t, 16))

db = plyvel.DB('/home/chris/.bitcoin/chainstate')
o_key = db.get(b'\x0e\x00obfuscate_key')
print('o_key', o_key.encode('hex')) # my o_key is 0899b9c2314a85c9b6

result = db.get(b'B')
result_hex = result.encode('hex')
print('result_hex', result_hex) # b6c9854a31c2b999b6da5ee266a8047f3c8c5fc82479ab03af9272a3a57372a2
xor_key = get_xor_key(o_key, result)
xored = xor_two_str(xor_key.encode('hex'), result_hex)
block_hash = xored[2:-1].decode('hex')[::-1].encode('hex')
print(block_hash) # 870b3cd33974701cfceb9c5bffa0b2b6b9c66e0a9de0a3b56617389267caebaa

Can you explain to me why in the answer for https://bitcoin.stackexchange.com/questions/51387/how-does-bitcoin-read-from-write-to-leveldb/52167#52167 it says 26c326d7353661dc7005d274976f458691f24f0f05d141335f4ad5927e41 xor 27c78118b731610527c78118b731610527c78118b731610527c78118b731 is equal to 104a7cf820700d957c2536c205e2483b635ce17b2e02036788d548ac970 and not 01028820010b2a00367244680f6da18acd861a08f0a89cb3b49ab50e?

Oh, I see what's going on.  Your answer is mathematically right, but you have dropped a leading zero so it is harder to see.  It's more clear if written as 0104a7cf820700d957c2536c205e2483b635ce17b2e02036788d548ac970.  However, I don't understand either where the 010288... answer comes from.  It's not even the right number of bytes.

The author of https://bitcoin.stackexchange.com/questions/51387/how-does-bitcoin-read-from-write-to-leveldb/52167#52167 has acknowledged that their result seems to be in error.

Hmm. Then I wonder what I'm doing wrong.

What information you trying to get from blockchain that you need to work with 'leveldb'?

Just trying to understand how it works.

The decoded block hash ought to start with zeros, and I'm noticing that the beginning of the database value looks like the end of the obfuscation key, reversed.  So this is probably a big/little-endian issue.  Try reversing the bytes in one of the values before xor'ing.

Tried it. That is not the case.

This is a bit late, but I've been struggling with this myself so hopefully I can help out anyone who comes across this thread with a similar question:
Looks like the chainstate obfuscation key begins with an 08, which probably refers to the length of the key. If you remove the 08, you get an obfuscation key of (for the above example) 99b9c2314a85c9b6, which you'll note is the reverse endian-ness of the beginning of result_hex (as it should, since the beginning of block hashes are zeros).
So to get these pieces of data to match endian-ness, you can reverse the key above and you'll get b6c9854a31c2b999. Per this answer, to de-obfuscate data you need to append the obfuscation key to itself until it matches the length of the data, leaving us with:
result_hex = b6c9854a31c2b999b6da5ee266a8047f3c8c5fc82479ab03af9272a3a57372a2
xor_key    = b6c9854a31c2b999b6c9854a31c2b999b6c9854a31c2b999b6c9854a31c2b999

If you enter those two values on https://xor.pw/# (as the OP did) you get 13dba8576abde68a45da8215bb129a195bf7e994b1cb3b, which you'll notice is quite a bit shorter than the inputs. This is because that site omits leading zeros. Add the missing zeros in and you get 00000000000000000013dba8576abde68a45da8215bb129a195bf7e994b1cb3b, which is a block hash from around the time this question was asked.

Your logic of xoring obfuscation key with the key seems to be causing an issue. Key and level DB value should be first made the same length. The result block hash should start with zeros. Feel free to copy code from this repo which is in working state


text:
Frequency of MtGox market orders
Im currently trying to estimate the frequency of incoming market orders to MtGox order book.
Assume the following buy limit orders:
BID 2@150
BID 3@149
against which ONE market order to sell 5 btc is submitted. The transactions would be recorded for instance as:
Trade_ID   Unix_date   Price   Amount   Type   Properties
1          123456789   150     2        sell   market
2          123456789   149     3        sell   market
The key problem is that despite there was just one incoming order, it is recorded in two rows because it was matched against two outstanding limir orders.
Possible solutions:
1) If both trades have same TRADE_ID, it is possible to group the transactions. Does it really work that way? Do trades resulting from one order but matched against more orders same Trade_ID?
2) Grouping according to unix time stamp. I assume that these two transactions would have same time stamp? If yes, is is possible that any other market order arrives just at the same microsecond as the one above? Or is the microsecond so small time interval that it is not possible for two orders to have same unix time stamp?

just look at the websockets depth event. All new orders will be positive and executed orders will have negative volume. Orders will not appear redundantly

I'm afraid I don't understand. Can you elaborate a bit, please?

How does websocket depth event look like? Can you give an exaample? thanks.

here is the api that details how to use websockets and you can also get the data from bitcoin charts

Well, I went through the API specs, namely the depth channel. However, it contains only following information: currency, item, price, total volume, price_int, type, type_str, volume, volume_int.  But it does not contain the information whether the order was limit order or market order. How can you know then what type of order it was?

Well I guess it's impossible to know for certain (as far as I can tell) but I think it would be a safe assumption to make that if the the price of the depth event is equal to the inside price (or within depth - inside_price * order_volume) and it was immediately executed on (< 1 sec) then that is probably a market order.

OK, sounds like a reasonable assumption. So i guess the depth update of the market order above would be something like -5BTC, price 149??

yeah, it was being executed, but you should also look for non negative ones going up in a close time frame

OK, sounds like a reasonable assumption. So i guess the depth update of the market order above would be something like -5BTC, price 149?? I understand that if the price is lower/higher (depends whether buying order or selling order) than the best bid/ask*order volume then it shoul be an market order. But how do you know how lond did it take to execute that order? DOes it have two times? Time of entry and time of execution? And what about the canceled orders? I guess they have negative volume as well. Maybe it will be youseful, if you can say how the log of order above would look like.Thanks!


text:
Bitfinex API - clarification of "timestamp" in orderbook API request
Does anybody know what value "timestamp" in orderbook POST actually means?
The response is structured as follows:

bids (array)
price (price)
amount (decimal)
timestamp (time)
asks (array)
price (price)
amount (decimal)
timestamp (time)

documentation: Bitfinex API
Does it say when the corresponding limit order entered the orderbook? Assuming that this is true, what if there are two limit orders at the same price level each of which entered the orderbook at different time? Will there be two orders at that price level or will it be groupped to one order with the latest (more recent) timestamp?


text:
Is there any chance of the Ripple gateway market NOT consolidating into a small transnational oligopoly?
Bigger and better reputations might beget more business and so on. What can possibly prevent emergence of a small oligopoly of transnational Ripple gateways ?

That wouldn't really be a bad thing if they were reliable and kept costs down. But I think it's inevitable that, if Ripple catches on, small gateways will always be around at least to serve more remote and underbanked areas and populations that transnationals can't be bothered to target.
However, I don't think the barrier to entry will be that high. You might think that a small company couldn't compete with a larger company because trust is required. But actually, a gateway can operate without people trusting it. The gateway can basically resell another gateway's IOUs for cash. And for getting cash out, the gateway can accept other gateway's IOUs. I think there will always be "gateways" of this type selling and buying IOUs in person.
Over time, if the gateway continues to operate reliably, people will hold its IOUs. This is especially likely if the gateway offers some advantages such as a lower transfer fee or the ability to costlessly exchange their IOUs for IOUs from a number of other gateways.
Of course, transnationals may be able to eliminate this opportunity by providing their own costless inter-gateway exchanges and keeping their own transfer fees low. But if they serve everyone at low cost, nobody should much care that there isn't much competition.
A model that I frequently use to think about the relationship between gateways is Internet service providers. They compete with each other, but if they don't also inter-operate with each other, none of their customers will be happy and they will move to other providers that inter-operate with their competitors better. Small Internet service providers don't exactly compete on a level playing field, let's be honest, but they still have little problem finding niche markets.


text:
Can Ripple facilitate business-to-business barter transactions?

Beverly Hills Hotel participates in the Ripple network and issues its USD-denominated IOUs which are redeemable for hotel stays.
Beverly Hills Performance Car Rentals participates in Ripple network and issues its USD-denominated IOUs redeemable for car rentals.
Both the hotel and the car rental shop have unrented inventory.
Daily room rental rate in USD is roughly the same as the daily rate to rent a Ferrari.

Question:
Can the hotel owner and the owner of the car rental business exchange their respective IOUs through either a mutually trusted gateway/nexus or directly (assuming they trust each other) or through a web of trust to allow a room to be rented out for a day of stay and a Ferrari rented out for a day of driving ?
Bonus question:
Is this the real purpose of the "issuer" field in the current Ripple user interface, i.e. to allow any participant to issue IOUs denominated in any currency of their choice ?

I can answer your last question in the affirmative. The goal is definitely for it to be possible for anyone to issue an IOU in any currency (standard or even customized like "owe you a beer") if someone else is willing to accept it. The underlying system is designed to fully support it. For example, currencies are just 160-bit numbers to the network. Issuers are just regular accounts.

Currently currencies are limited to three-character codenames, so although this can be done and will work it will probably be useful to establish some convention(s) that will help people to not get too confused.
For example your examples seem to amount, in general, to "store credits", like airmiles or giftcards in a way. Thus it might be better to use a code for them that is different from the national cash they might happen to be useable on par with "at the store". So maybe instead of having lots of "different" IOUs labelled as USD most of which are not really someone owing an actual USD but, rather, someone owing in store credit, in other words goods or services, whose retail price happens to be in USD, it might be good to use USC or USX or whatever, the three letter code conventions only use two for the nation, deliberately leaving one for distinguishing between variants (such as day old dollars or tomorrow dollars or some such strangeness I think).
In general though, ripple docs already say that it is okay for a gateway to refuse to cash out people who are not actual members of that gateway, so that gateways can insist on knowing the customer and having anti money laundering data about that customer and so on. So we are told up front already that gateways are not under any obligation to actually make good on their IOUs to just any Tom Dick and Harry. Since that is already said, why not take it a step farther and say "oh yes I am a gateway, but only to people who buy my merchandise, and only up to the actual amount, per such person, that person spends at that moment on my merchandise".
I just think that would be a heading toward being kind of sleazy as compared to right up front setting your IOUs as not being USD but store credits, so people don't end up confusing them with actual fiat debt.
-MarkM-

In the example above presumable both businesses would want to denominate their IOUs in USD but make them redeemable only at their place of business, hence their are effectively private currencies but measured with a common USD metric.  Still, presumably the whole existing b2b barter $70 Billion USD per year industry as represented by https://www.imsbarter.com/ and 10's of other "barter exchanges" most of whom are  just poorly designed websites ... My hope would be that Ripple could thrive and enable such "excess inventory without hard cash" exchange which would otherwise not happen at all.


text:
cgminer fails to start with error "error opening terminal: dumb"
I'm using Windows 8 x64. I've downloaded the latest cgminer (3.4 at this time), but when ever I try and start it I get the error message:
Error opening terminal: dumb

I've seen various post that suggest this relates to the TERM env var and setting this does change the message to end with what ever is in TERM, but nothing I've tried fixes it. Anyone know what I need to do to get this working?

I had the same problem and for me it worked to set the TERM variable to nothing directly before you execute cgminer. Like this in the cmd shell:
set TERM=
cgminer-nogpu.exe ........
If you set it to "" (meaning nothing) in the system settings it will use the default value "c:\windows\system32\cmd.exe". You can check the the variable in the shell with:
echo %TERM%
I hope this will help you :)

Wow, thanks! I'd tried setting TERM to just about everything but the empty string!

Sure, forget about that sorry.

This seems kinda coincidental and doesn't get at the root of WHY that error message is being produced. It also doesn't work for me: $ TERM='' ./cgminer --scrypt -o stratum+tcp://dogehouse.org:3333 -u zbuc.zbuc -p p4ssw0rd --worksize 256 --lookup-gap 2 --thread-concurrency 2047 -g 1 --intensity 11 Error opening terminal: unknown.

Start the cgminer script with the -T flag. That worked for me. It disables ncurses and uses text-only output.


text:
Can I long USD/BTC instead of shorting BTC/USD?
Let's say that I think that bitcoin has hit a peak. I want to short bitcoin, but in a far less risky way. In order to do this, I want to go long on USD/BTC. Let's say (as an example) that I longed this pair when Bitcoin was $13,250 (on the dropping side, I set that as strike price), and sold it when it was $8,000 and going up. I would have made a 65% return as opposed to a USD/BTC short of just under 40%. I want to do this to be able to make a higher profit during a bear market. I feel that this is a good idea.
The question is, to clarify: can I long the USD/BTC pair? This pair is equal to one divided by the USD price for one bitcoin. I think it is good because it offers an alternative to shorting, one with a theoretically unlimited upside and a limited downside. If one bought $250 worth of bitcoin at $100 and sold the position at $6,000 in the beginning of November of 2017 or 2018, it would have brought $4.50 as opposed to a theoretical owed amount of over $14,000.
As a commenting individual in the comments said, this move could have theoretically brought (and maybe has brought) very high returns. Even going back 1-2 years ago (roughly speaking, as of writing), such a move timed at the right time could have brought 500% or more returns depending on how good timing the trader had.

Hypothetically, if you did this in 2011-2012, you could have turned $100 into $1000 or more using realistic timing.

It is unclear to me what your question is.

I'm in favor of keeping this question open as I believe it is a question related to fundamentals of trading practices and can be useful for number of people who have questions related to that. This question will not get outdated until trading practices changes (so for a number of years).

Long USD/BTC is the inverse of short USD/BTC, which is identical to short BTC/USD.

Going long USD/BTC is identical to  selling the amount of BTC that you have for USD. To put it into perspective, assume you had 1 BTC when the price of BTC was $13,250. When you went long USD/BTC, what you essentially did was you sold 1 BTC and received $13,250 in your account. When the price of BTC dropped to $8,000, you still had $13,250 in your bank account. The value that you own in terms of USD remains the same. The only difference is that it now represents 1.656 BTC. So if the currency is BTC you made a profit of 65.6%. If currency is USD, you made a 0% return.
Short-selling is a different beast. In short-selling, you borrow some BTC from a third party, sell them in the market and then buy BTC at a later price. So in the above example, you borrowed 1 BTC from say John at $13,250, sold them in the market and later bought 1 BTC at $8,000 and gave 1 BTC back to John. So you have netted $5,250 in profit. Your starting capital was very small (which includes interest payments to John for lending you 1 BTC and some margin requirements that needs to be kept for safety.) So with a very little capital investment you have netted $5,250 in profit. This can be also used to buy Bitcoins at the current market price. So with short-selling you can gain profit in terms of USD and BTC.
However, I like to note that short-selling is a high-risk, high reward game. If the price of BTC had moved higher as compared to USD, you would have lost your margin capital, and still made interest payments to John. So in essence, you can lose 100% of your initial investment in the short selling game if the price moves in the opposite direction in which you have bet.


text:
Verify Receive and Change Addresses
I run a coldcard with Electrum and Electrum-Personal-Server. As I understand it, we use hardware wallets because we don't trust our laptops.
However, I've only been able to verify the sending addresses on the coldcard's screen when I sign transactions, as well as verify my own personal receiving addresses using the addresses explorer function: https://blog.coinkite.com/address-explorer/
What remains is to verify my change addresses.
I'd like to do this manually on my laptop, using a virtual machine (I imagine it'd have to be pretty specialized malware to interfere with that).
Does anyone have some resources for manually deriving native segwit addresses from xpubs that they can point me to?
Thank you,

Upon further research, the coldcard has built in change-address fraud detection: https://blog.coinkite.com/troublesome-change/
Also when signing a PSBT, after looking at the send address on the coldcard, you can also scroll down further to see the change address. (Once again, if anyone knows of good resources, please point to me to instructions for manually deriving change addresses manually from xpubs).


text:
Create Raw Transaction in BitcoinJ offline starting from UTXO info
I would to create offline Raw Transaction (without signing it) in BitcoinJ. The information I know about the inout is UTXO's hash, index, amount, address, and script. I also know the receiver address of the final transaction, the change address, the amount and the fee.
I'm fighting with the TransactionOutput constructors to build the input for my transaction. I don't know where to set UTXO hash and index - i think in the "parent" parameter, but i'm failing. This is my snippet:
Transaction parent = ???
Coin value = Coin.valueOf(satoshi(utxo.getAmount()));
Address to = Address.fromBase58(params, utxo.getOwnerAddress());
//where do i put utxo hash and index?
TransactionOutput input = new TransactionOutput(params, parent, value, to);
tx.addInput(input)

I'm fighting with the TransactionOutput constructors to build the input for my transaction. I don't know where to set UTXO hash and index

I think you can build the input based on a previous transaction output (outpoint) as follows:
byte[] scriptBytes = {};
Sha256Hash vinTxId = Sha256Hash.wrap(txId);
TransactionOutPoint outpoint = new TransactionOutPoint(params, index, vinTxId);
Coin value = null;
TransactionInput input = new TransactionInput(params, transaction, scriptBytes, outpoint, value);
transaction.addInput(input);

Not sure if this is the best ways to do it.
EDIT: This does the same as the code above.
transaction.addInput(Sha256Hash.wrap(txId), index, ScriptBuilder.createEmpty());


text:
What is the current size of txindex folder when txindex=1 set?
Due to reputation limitations, I cannot ask for an update on this question:
How much extra disk space does txindex=1 require?
What is the current size of the txindex folder when txindex=1 is set in the bitcoin.conf file on a fully synced full node?
As pointed out in the referenced post, the size should now be around 28GB?
Thanks!

27GB as of 2020-06-14

You yourself can check this via going to .bitcoin/blocks/index folder. Where .bitcoin folder is usually hidden folder on home when using linux.
It depends if you have pruned than the index folder might appear less in size. As of today my index folder is 105.4 MB since I've prune=1000 and it would remain the same in future as well.
Our question is answered here as well. Please check comment by @anonymous

-txindex and -prune are exclusive, only one of the two may be active at the same time. -txindex is inactive by default, and your folder size is smaller because you don't have a complete transaction index.

@Murch so if I have pruned=1000 doesn't it effect the size of transaction index? I think it will reduce the size.

What I meant was that txindex builds a complete transaction index which is 27 GB. This is incompatible with running a pruned node. Your node is running in pruned mode, so it cannot be running with a txindex at the same time. I'm not sure whether the index folder of a pruned node would be smaller than that of an archival node that doesn't build a transaction index.

@Murch yes it is smaller I have tested


text:
Unspendable bitcoins on blockchain
Gooday, please i want to know how to transform my non spendable bitcoins to spendable bitcoins on blockchain account for the sake of being able to spend them please someone help me out I'm lost. I dont know how to retrieve the private key?
I got the bitcoins from an account manager that traded for me and supposedly made a blockchain account for me, where all the money was being sent to but a non-spendable account. The broker used to sending the money was IQ options, i do not know what to believe but please, please and please i want those bitcoins to be spendable and usable...Please someone or anyone, give me a SOLUTION!!!... I have my backup phrases if that helps and to be honest I have an average knowledge of bitcoins. SOMEONE PLEASE HELP ME!!!

If you're referring to watch-only addresses, and coins you have in them, the answer is almost certainly that they were never yours - this is a common scam. Though you'll need to expand your question (how did you end up in this situation, what do you see, what site you use, where you got it, ...) to be sure.

Someone helped me trade and supposedly sent the bitcoins to my blockchain wallet, I use blockchain.com and like I said the money was gotten from a trading account, IQ options to be exact.

That sounds like you were scammed.

Damn, I can't get them back?

The money is still in my non-spendable account!!!!!!!!

I can't tell without more details, but if it's the common watchonly wallet scam, it's not that you can't get them back - it's that you never had any. A watch-only wallet is for watching someone else's coins.

What the hell?, seriously?...I got duped?...so non-spendable bitcoin wallets are just facades damn

I wish I could send u a pic or a screenshot

Does this answer your question? How can I get a private key for this address

non spendable bitcoins

That usually means either

a watch-only address in a cryptocurrency wallet
an account with a custodial service with some withdrawal restrictions.

In either case, you are not the owner of those Bitcoins so far as the Bitcoin network is concerned. The watch-only address is a favourite trick of scammers/conmen. They like to set up accounts and wallets that look like they contain money but which, in a non-obvious way, don't give the victim any control over money - that helps them to trick people. The conmen then ask the victim for a series of invented special "release" fees or extra deposits.

Someone helped me trade

Usually a sign of a scam. Unless its someone you met in person in real life and have known well for years.

a trading account, IQ options ... an account manager that traded for me

Very often the sign of a scam
I'm pretty certain your money was taken by a scammer and not used in the way they claimed.

blockchain account

The only recourse in this situation is to report this theft to Blockchain.com, to the police and if the "helper" is identifiable and located in the same country, explore pursuing it through the courts. It is extremely unlikely the money can be recovered and attempts to recover the money through the legal system are likely to cost more money.

I have my backup phrases

I would use that to recreate a clean wallet on a clean freshly installed PC (not one your helper has ever known about) and see if it actually contains anything. Most likely it is an empty wallet. Probably the watch-only addresses in your wallet were added separately and any money shown there is not under the control of your recovery phrase.


text:
The bitcoin miner lags my cpu 99% and how i will earn bitcoins
Bitcoin Miner program uses my CPU 99% can you just little make it no laggy and how do i earn bitcoins? in the internet sites and forums they dont sya anything about earning bitcoins

Are you sure they say nothing about mining or earning bitcoins by other means?

Mining with a CPU is just not a sensible way to get bitcoins.
Basically, you earn bitcoins the same way you earn any other currency. You earn dollars by finding people who have dollars and offering them products, services, or items in exchange for those dollars. It's the same thing with bitcoins. Find people who have bitcoins and offer them something in exchange for them.
It's extremely difficult to mine at a profit these days. That's probably the hardest way to get bitcoins.


text:
The general principle of smart contracts for bitcoin
As far as I know, there is the possibility of realizing smart contracts using the Bitcoin blockchain. Explain how a smart contract works on the Bitcoin blockchain. I am interested in the general architecture: in what language is the contract written, what is and where is the machine that processes it, etc.
This is the most general question in order to understand and clarify the mechanism, for better immersion in development.точнения механизма, для лучшего погружения в разработку.

Bitcoin doesn't have native smart contracts. Ethereum is famous for smart contracts. https://ethereum.stackexchange.com/ can answer in that case.

@MCCCS that isn’t really true, Eth just has more expressive smart contract capabilities than BTC.

@chytrik This is what I had in mind. What does more expressive smart contract capabilities mean? Can you tell me where to dig?

@Gybernaty the Bitcoin scripting language is designed to be verifiable, secure, dependable. It is slightly limited in some functions because of this. On the other hand, Eth's smart contracting language is designed to be easy-to-use, more full featured, ~Turing complete, etc. But the trade-off is verifiability and security, which are in fact very important considerations (see: eth DAO hack, eth multi-sig bugs, etc). The question I linked above does a better job explaining this


text:
mining: losers get nothing?
if the winning miner takes the 12BTC+fees, unless you own the biggest mining operation out there, or are not a member of the biggest pool, why bother mining, if there is no prize for second place?  does the randomness still ensure the smaller pools get lucky  just enough to keep them interested in mining BTC?
the link below claims the massive Bitmain solves only 3 blocks a day ?  what could a smaller operation/pool hope to achieve? thanks
https://coincentral.com/the-top-5-largest-mining-operations-in-the-world/

In the very early days of Bitcoin blocks were typically mined by one individual person, often referred to as "solo mining"). This made sense because with only CPU mining each person pretty much had the same probability of solving a block as the next person (unless they had several CPUs). Eventually as mining expanded, became more distributed, and hardware such as FPGAs, GPUs, and ASICs came along, the idea of pooled mining was developed. This allowed a group of people to put their computational power together and share the rewards across each "pooled" miner in accordance with their contributions once the pool had solved the block. Each pool is different, and will use different reward algorithms such as PPLNS, PPS, etc.

why bother mining, if there is no prize for second place?

Mining has very profitable rewards, especially if you are lucky enough to solve the block while solo mining. If you do not have to share the reward among others, 12.5 BTC will likely be more than enough to pay for the electricity used to mine the block depending on location, etc. For smaller miners, pooled mining will give them a portion of the rewards depending on how much hash rate is directed to the pool, regardless of whether they have personally solved the block themselves.

does the randomness still ensure the smaller pools get lucky just enough to keep them interested in mining BTC?

Exactly. Imagine you and a friend were the only people mining in a pool you have created. Lets say you both have one 14Th miner. If the pool solves the block, you will split the reward because you have both agreed to work together and share the rewards.

ok, thanks for the explanation. but do you still hear of  people who are not in a big pool who actually find the golden nounce before the other  big setups?  i found some posts of solo miners celebrating after mining a block,but these date back to 2014/5 .   if i  were to take fishing as an analogy,  its not impossible for a little kid on his first day fishing with some string tied to a piece of cane to outperform  a seasoned  fisherman with a $1000 rig, if the fisherman is having an unlucky day.  is this analogy valid for mining BTC today? thx

@Fred Yep it is defiantly possible. Back then the difficulty was much lower so the chances were higher too

Mining is not a race where the fastest one wins every round. It's more like rolling 20 dice in quick succession where the winner is the one who throws all sixes. The quicker you roll, the more chance you have to win, but even someone who rolls once a second still wins every now and then.