text:
Consider Bitcoin Price goes down to $3000, what will be the effect on miners (will they stop mining) and the affect on difficulty to mine a Bitcoin?
Over a set period of time (around 10 mins for BTC), a number of transactions are compiled into a block by a miner, and then the block is verified on the general ledger and added to the list of blocks which make up the blockchain.
Consider the price of Bitcoin goes down to $3000, will this effect Mining operation go on?
What actually going to happen to the difficulty of solving problems in mining?
Thank you in advance.

Regards,
Raghu Ariga

More than a question it seems like you are asking for a prediction of behavior. What miners will do eventually, and how getdifficulty will be affected, has been central to many conversations around Bitcoin.
In my honest opinion, even at 3000 USD per BTC it is still highly profitable for miners. The key factor is if miners need to move their generated Bitcoins to fiat money to pay bills.
If a miner or mining pool manages to avoid these payments based on future price -something they should aim at-, they will be doing quite good with even 1000 USD per Bitcoin.
Remember that we are creating the Bitcoin/Frame/Economy. Trading for fiat currency is just not the main idea behind cryptocurrencies.

There was a survey recently about cost of mining bitcoins that's relevant to this discussion: http://www.ibtimes.com/bitcoin-mining-cost-lowest-venezuela-highest-south-korea-report-finds-2646191


text:
If Bitcoin addresses are truely random, then why are some more difficult to generate than others?
In this post TierNolan explains why 'Bitpoin' is much easier to generate than 'bitpoin', and that mainly has to do with the position (first character). But I remember that certain characters (I believe 'i' and 'o') are much more difficult to generate, independant of their position in the address. How is this possible?

related: Why are Bitcoin addresses starting with 1s or 1S so rare?

The 160-bit hash that is encoded in addresses is uniformly distributed ("truely random" as you call it), but the base58 encoded form is not. Some characters are more likely to occur at the start, for example.
To illustrate, consided the set of all integers between 0 and 1999. Even though each of those numbers is equally likely to be chosen, this is not true for its decimal representation. There, half of the numbers start with a 1!

Nice example, and I understand why the first digit is not always random. But between 0 and 1999 the last digit always has an equal distribution, and if 'o' or 'i' automaticly increase the difficulty, even the last character is not equally distributed. I saw the author of vanity-gen once explain this on bitcointalk, but couldnt find his post.

Specific answer why 1Bitpoin is easier to generate than 1bitpoin:
The address is a base-58 representation of a 192-bit integer (160-bit hash plus a 32-bit checksum). The largest value you can represent is (2^160-1) with the corresponding checksum: 0xfffffffffffffffffffffffffffffffffffffffffa06820b, or the 34-character address "1QLbz7JHiBTspS962RLKV8GndWFwi5j6Qr".
In the base-58 alphabet, 'B' < 'Q' < 'b'. So any 34-char string that starts with '1Bitpoin' is valid as long as checksum is correct. But 34-char strings starting '1bitpoin' are all out of range. You can still generate 33-char strings starting with '1bitpoin' but obviously there are much fewer of these.

The characters 'i' and 'o' are only hard to generate in the case-insensitive sense, because capital 'I' and 'O' are not valid in base-58.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
How to create a SPV client?
What are the steps a SPV client takes in order to sync?

Connect to a peer
Set bloom filter for wallet addresses
Download all block-headers since latest checkpoint
Request every block again with filters set
Update wallet balance based on returned blocks

Obviously the above steps are wrong since it makes no sense to first download all blockheaders to determine the correct chain, and then re-request all blocks to calculate the wallet balance.
But what is the correct procedure?

This is what I do:

Connect to peer
Set bloom filter
Send 'getblocks' message
Send 'getdata' message with MSG_FILTERED_BLOCK set for any new blocks

Note that 'getblocks' returns a list of block chain hashes from the specified starting point, not the blocks themselves.  Then 'getdata' returns 'merkleblock' messages instead of full blocks.  The peer follows the 'merkleblock' message with any transactions that match the bloom filter as separate 'tx' messages.
More details are in BIP0037
If you want an example, my Java wallet can be found on GitHub

A couple examples may help you:
https://code.google.com/p/bitcoinj/
My project https://github.com/richardkiss/pycoinnet is not complete (in particular, as of this writing, there is no bloom filter implementation), but it may eventually do what you need.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How does lighting network ensure all parties cooperate
I somewhat understand the two way channels used in lightning. But I fail to see how trust is avoided.
If I'm the middle node I'm perfectly fine with forwarding a payment, if I received the payment first. The initial payer can reason that he will only make the payment if all intermediate nodes have shifted the payment to the receiver. It seems like until at least one party takes the risk the payment is in some kind of Mexican standoff.
How is this avoided / cryptograpically enforced?

Thanks, I think I get it: all the receiving parties can pull money out of their channel once they know a secret from the seller. To claim the funds on the final channel, the seller has to publish this secret.

Exactly. If the linked question resolves your question, you can vote to close it as a duplicate yourself. (Otherwise please specify what points you would like to see answered here still.) Thank you.

That will do for now. Edit: unfortunately I don't find how to mark this as duplicate


text:
How does a node find a transaction in the blockchain?
If a (light) SPV client asks a full node for the Merkle branch containing a specific transaction (or the value of an address), how does the full node find this transaction (or the UTXOs) in the blockchain?
Does the full node scan linearly through the entire blockchain or is there a faster way to link transactions and blocks?

Love your question, I never really thought about that in detail. As I'm not sure at all, and rather just speculating I'm posting this as a comment (perhaps someone can use it as a starting point for an answer):
SPV clients mostly ask nodes about patterns such as a bloomfilter that resolves to payments made to their addresses.
Since SPV clients are rather exclusively interested in coins that still exist (or are about to exist), all of this information will be found either by searching the UTXO set or the memory pool. -End of speculation-.

@Murch The UTXO doesn't contain merkle roots to prove an output exists, in that model the node could just make up nonsense. You're right in that the UTXO can be searched in milliseconds though, but this wouldn't get any history for the client connecting.

You can read the bip37 specification for all of the gritty details.

how does the full node find this transaction (or the UTXOs) in the blockchain?

The client builds a filter which contains what they're interested in, be it output script (addresses), public keys, or TXID. They then send this to the node they are connected to. When they request data for a particular block the node filters it to only contain things which match the clients bloom filter, and sends it to them. To find all of their unspent UTXO, they request every single block in the chain in turn and look for things that could be theirs. As you mostly worked out, this does involve for every client the node loading 80GB of blocks from disk and performing extremely expensive filtering operations on them.
Modern nodes recognise how terrible of an idea this is, and allow users to disable the functionality completely due to the massive load and denial of service risk involved. With specially crafted transactions, a remote peer with a single command of less than 50 bytes can cause a node to load a 1MB block from disk, hash it over a million times, and then return nothing.

Don't they just ask for the data on blocks for which they don't have information yet? I.e. if a wallet was created at block 423,000, it would only ask for blocks later than that. And then, since it'll know that it caught up to 430,000 last time only newer blocks after that?

Yes. They can also skip blocks they know happened before the wallets "birth date", as there can't possibly be transactions from that time which they can spend. This doesn't help much in practice because the bulk of the size of the chain is within the last few months.

Usually, the light client asks "is this transaction in this block" not "which block is this transaction in". The latter can be done without too much trouble if there's a txindex (transaction index), which is something bitcoind has an option for.

Thanks for your answer! Still, a couple of things are not clear to me: (1) How does the SPV client determine X in the question "Is this transaction in block X"? By proping all blocks (let's say that I just installed a new SPV client and it should determine how much bitcoin I own based on my public addresses)? (2) How does it work in case of a Bloom filter? To my understanding, the full node has to calculate the hashes of all transactions and compare them to the filter? This seems too time-consuming for more than 80 GB of data.

Nodes do not maintain a txindex by default because it would be forever growing and prohibitively expensive (enabling this option significantly slows down block insertion speed and increases disk writes).


text:
Unconfirmed in the blockchain
I was sent a bitcoin transaction 18 hours ago  and it has still yet to be confirmed.  Any idea on what I can do or when it will go through?
https://blockchain.info/tx/da635a68d9db0448b1c10feee07a91cd6901f904d609b2f0a63b202e38d641f2?show_adv=true

Your transaction's fee was pretty low compared to the transactions that are currently happening on the network.
Your transaction was big with 2580 bytes, and that makes your fee work out to be only 22.3 satoshi per byte. Currently, transactions are confirming in the next block that are paying about 60 satoshi per byte.
There are about 35k transactions ahead of yours in line. (But new ones join the line, so it's going to be a while.) You might want to resend the transaction with a higher fee, if you know how to do that.
Otherwise check out Why is my transaction not getting confirmed and what can I do about it?


text:
How can I transfer my Bitcoin Android wallet to another platform?
I have the wallet backup and I was able to decrypt it using these instructions: https://github.com/bitcoin-wallet/bitcoin-wallet/blob/master/wallet/README.recover
When I went to do the withdrawal the system hung and I now recieve an error telling me the wallet is empty (CouldNotAdjustDownwards did not occur first try). I don't see any record of the transaction on any block explorer.
Is there some other way I use my decrypted wallet? Can anyone suggest a desktop wallet that would likely support this?
I tried replacing wallet.dat in Bitcoin core with no luck.

Specifically, which was the last line of README.recover that you ran successfully? And which is the first line where you ran into problems? Also, if you reached this point in README.recover, did you discover that you have a PROTOBUF WALLET FORMAT or a BASE58 KEY FORMAT?

./wallet-tool dump... says the wallet does not exist, so wallet-tool sync would be the last successful. wallet-tool send was where it hung. It is protobuf.

If your wallet was created with Bitcoin Wallet for Android 4.0 or later, you can extract the mnemonic (seed) with this, and then restore that seed into a wallet which supports a BIP-39 mnemonic with a BIP-32 path to gain access to the same funds, such as MultiBit HD or breadwallet.


text:
Concept of Validating Transactions
Trying to get my head around Bitcoin. Been watching tutorials, unsure if I still understand
Alice wants to send Bob 5 Bitcoins
Alice hashes the transaction with 5 Bitcoins and send's it to Bob's mailbox, i.e his Public Key
What is the job of the Miners?
Before the transaction get added to a public ledger (i.e blockchain) do the miners validates that Alice has 5 Bitcoins to give (based on info in public ledger) by recreating the hash.
And if validation is successful, the miners earn Bitcoins, the transaction is added to Blockchain, and Bob gets his 5 Bitcoins.
Is it true that validation takes about 10 minutes or so?
Really, I've been struggling with this concept.
Thanks!

First off, Alice doesn't hash the transaction as part of validation.  The hash Alice produces is simply a hash of the transaction, which is how transaction IDs are created.  In truth, anyone participating in the network could hash the transaction body and arrive at the same hash.  It is when the miners hash an entire block of transactions that hasing is part of the validation process.
Secondly, Bob doesn't have a "mailbox".  That's not what his public key is.  The blockchain is a public ledger, so Alice isn't sending the transaction to just Bob, she's announcing it to everyone.  Bob can see this transaction, just like everyone else, and now has to wait for a miner to include it in a block.
Miners create blocks of transactions, and they have to create them in such a way that the rest of the network will accept them.  One of the requirements is that the transactions in the block are all valid transactions.  So yes, miners will validate that Alice has 5 bitcoins to give before they add the transaction to a block.  If the miner cheats, and puts an invalid transaction into a block, then the rest of the network will reject that block, and the miner would have wasted their time doing the proof of work on that block.
The 10 minutes is a product of the proof of work algorithm.  It is retargeted every 2,016 blocks such that on average, it takes the entire network 10 minutes to construct a block whose hash is lower than the target value (starts with a certain number of zeros).  It is not a single transaction that takes 10 minutes to validate, but the construction of the entire block of transactions with the required proof of work.
Once a block with the proper proof of work has been found, it is broadcast to the network, which will accept it if the block meets all criteria.  After that, Bob can see that a transaction to the public key he supplied to Alice has been included in a block.  This is called a "confirmation".  Once another block is mined on top of that block, the transaction is said to have 2 confirmations, and a block later, 3 confirmations...and so on.
Here's some other answers you may want to read:
What exactly is Mining?
What is proof-of-work?

The hash Alice produces is simply a hash of the transaction, which is how transaction IDs are created You mean Alice's signature creates the transaction ID?

@Rhonda, no.  The hash cannot contain the signature, because the signature signs the hash.  This gets complicated in bitcoin.  What you should focus on is understanding the proof of work system, and not get so caught up in the details.  Make sure you know what a hash is, a signature is, a key pair is, a nine is, and a Merkel tree is.  Then rewatch those tutorials, and you'll probably understand better.

Wow, haven't heard of a nine ... will re-watch tutorials. Thank you!

Whoops...that should say "nonce".  Sorry, autocorrect.

Ah, ok then. Will mark question answered in this case.

The mailbox would be more equivalent to the Public Key Hash, which is derived from the Public Key.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Why does Bitfinex allow shorting with BTC as collateral?
I made a mistake today where I shorted BTC on Bitfinex using BTC as collateral. Why is this even possible, I cant figure out why someone would want to do that?

Shorting is normal trading transaction on real financial markets. Without shorting you would not be able to speculate and profit on price decline. Assume that btc price is 400 USD and you think that the price will go down. By shorting one bitcoin you simply borrow one bitcoin from some lender and sell the bitcoin for 400 USD. If your assumption is correct and price goes down, say to 100 USD, you buy that bitcoin back and return it to the lender. Your profit then is 400 - 100 = 300 USD.

Yes, that's how shorting works.  The question is why it makes sense to pledge your own bitcoin (instead of dollars or something else) as collateral for the bitcoin you are borrowing.  On the face of it, this would seem like a no-op.

Shorting always includes borrowing some of the bitcoins on the top of those you sold, meaning that you sell your bitcoins and broker always sells some more bitcoins borrowed from the lender, typical ratio is 1:4, one btc ours, 4 btc from the lender. Selling bitcoins without lending bitcoins is not shorting, it only protects your portfolio from loosing its value, but does not generate profit.

In well regulated markets (Bitfinex's aim) traders should have the ability to profit from movements in both directions.
Shorting also assists in the formation of price bottoms. The only way a short seller can capitalize after a down move is by buying back the shorted BTC. This has the net effect of creating buyers after a down move which helps bouy the BTC price.
Bitfinex can also adjust its overall risk and generate some additional revenue by lending out BTC's to short sellers.

I know what shorting is, the question is about the collateral.

By using BTC as collateral instead of USD,EUR etc you can mitigate some of the exchange rate risk.


text:
Is there a chart of all combined orderbooks?
I would like to view a chart where all bid/asks are combined from the various exchanges, instead of keeping multiple tabs open for each exchange. Is there a website that allows this kind of meta-overview?

All of the exchanges have APIs http://pubapi.cryptsy.com/api.php?method=singlemarketdata&marketid=155
you can create a script to keep the numbers updated with wget and a cron job
Cryptsy has a pusher API for live updates.
https://www.cryptsy.com/pages/pushapi

data.bitcoinity.org has come up with such a website:

There is also a combined depth chart at http://bitcointicker.co/, but it is not actually clear if it is actually combined from more exchanges (and which ones) or if it is from single exchange.
There are some code projects on this topic, like: https://github.com/coinorama/coinorama/wiki/Coinorama or https://github.com/Pechalka/combined-order-book/
You find more (though old) answers here: Full animated order book


text:
Does anyone have a formula for dynamic block durations?
I want to implement this, so does anyone have a formula for dynamic block durations instead of a fixed target of 10 minutes?

Essentially, you would have to change the difficulty every block, or define the difficulty in terms of passed time. AFAIR The Kimoto Gravity Well does both: How does the Kimoto Gravity Well regulate difficulty?

@Murch I already use Kimoto Gravity Well, but it changes the difficulty but not the blockduration?

Sorry, it's been a year since I read about KGW. If I recall correctly, there was some altcoin that accepted blocks at a receding difficulty the more time had passed, I thought it was KGW that did that, but alas I seem to misremember. Also, now that I think about it, that might come with some unique attack vectors to replace the chain leaf by mining a higher difficulty block redated to an earlier time.

How do you want the block duration to change? That's the hard part. After that, the formula is essentially the same, one of the input parameters (the target block time), just changes.

@StephenM347 I want a faster interval when needed, and slower if the network is underutilized.  A good indicator could be the number of transactions in the past X blocks I think? But maybe there are better measures? But for example: if the past 100 blocks are filled by more than 50% of the allowed max, raise the speed. if their are filled less than 50%, decrease the speed. what do u think? What will be a challenge is that I also like to have a floating maximum blocksize, so the previous example would be very complex to implement, since the max blocksize could also grow in the mean time :)

@Muis, be careful with solutions like that, it sounds easily manipulated, and the problems introduced could be worse than the ones it is trying to solve.

@StephenM347 I know, its just for a personal experiment, no noobs will be harmed :)

It seems like what you really want is a formula to keep block duration static, and block difficulty dynamic.

@NickODell No? Suppose you have a chain that almost receives no transactions for 10 years, but it must be ready for 1 month of high-traffic. If you make the block-duration shorter to be prepared, then you have 10 years of empty blocks to store. If the duration was dynamic, you could have blocks which span 1 month (only 120 for 10 years), and then switch to 1 minute when needed.

Block duration is only controlled by difficulty (although indirectly) and network hashrate, so you'd want to increase difficulty when there are very few transactions in the last X blocks, and vice versa, decrease difficulty when blocks become too full, but also decrease difficulty when there is a low hashrate, and increase difficulty when there is a high hashrate.
At the same time, you'd need to incentivize mining, so you might want to have a reward relative to target time, instead of constant per block, as otherwise miners could spam the network in order to increase revenue.
A draft
Let's say, we are aiming to adjust from looking at the last 100 blocks for blocks that are 50% full.
findNewBlockTime():
  newTimeTarget = (avBlockInterval100)*(0.5/averageFullness); //shorter interval with many transactions, longer with less

setReward():
  newReward = newTimeTarget*totalIntervalReward/rewardInterval; //rewardInterval could be four years like Bitcoin, or some other interval, with four years totalIntervalReward would be 11.5M coins for first interval

calculateDifficulty():
  newDifficulty = sumDiffLast100/last100BlocksTime*newTimeTarget;

You might be interested in what I wrote about the subject here. You can make a protocol change where miners are given a choice of their block's "weight", and their decision can then be used to set the global frequency parameters.


text:
What makes the initial download of the blockchain so slow?
If you download a bootstrap copy of the blockchain through torrents, it will be faster than getting that same data through the Bitcoin network.
Ofcourse the BitTorrent protocol is more optimized for bulk data transfer, but is there any other reason for the huge difference in bandwidth? Is it artificially limited in the client right now, or by design?

I found this post by Jeff Garzik (one of the developers):

Initial blockchain download occurs from one peer at a time.  If that
  peer is slow, then your entire blockchain download is slow.
  Yes, it's stupid and yes it needs fixing (patches welcome).

Bitcoin blockchain download is slow because on default bitcoind will connect to random nodes (home connections for example).
See What is the fastest possible way to download the blockchain?
.... too accelerate the download.

Bitcoin and torrent are two different things, there are no trusted sources for the blockchain in the torrent network as im aware - INSTEAD  you can predefine trusted node servers in the bitcoin.conf which can be found on various "trusted sites"


text:
Why did Satoshi not choose for Segwit?
I know that the very early version of the Bitcoin code had a lot of advanced features, and was already very mature.
But Segwit (storing signatures separately from the transactions in the block) is a very simple change, and so obvious that I'm sure it must have passed Satoshi's mind many times.
So I'm wondering: what could be a reason that Satoshi choose to store the signatures inside the transaction? There is always the possibility that he just made a stupid mistake in the design, but I find that hard to believe.

I know the reason, but I won’t post is as answer because it will be heavily downvoted.

Satoshi made many design decisions that are now criticized (like signatures in transactions, 2-week difficulty period, halvings, etc..). He also made some maths error in the whitepaper. Satoshi wasn't a perfect designer and his C++ code was not up to the highest standards.
We will maybe never know for sure why he chose what he chose, but I'd personally argue it was for simplicity sake.
On a more tangent note: while SegWit looks simple on paper (just put the signatures on the side), in practice, it's quite a complex change to implement. It took months for the most experienced developers to review it.

It is only complex to implement it as a soft-fork, but to make the change long before the launch (or shortly afterwards using a hardfork) is not complex at all, you can simply adjust the format / structure of the blocks. Maybe he talked about this subject on Bitcointalk, but otherwise I agree that we can only guess for his motives..

Satoshi had a definite reason.


text:
How to estimate mining fee for raw transaction?
In our application we are generating a trasaction hex using the createrawtransaction call to "handcraft" a transaction. The problem we are facing is that we would like to estimate the mining fee for this transaction. I realize that we could call estimatefee x to estimate the mining fee per byte. But the problem is we are generating the transaction, so we don't know the size of the transaction yet! Chicken or egg?! I realize that we can generate a transaction and then account for the mining fee based on the hex, but this sounds like a hack to me.
So the question is: what's the best way to estimate the optimal mining fee while using createrawtransaction ?

Have a look at fundrawtransaction (if you are okay using the internal wallet).
fundrawtransaction will give you back the used fee (next to the hex byte-array of the funded transaction) before signing or broadcasting the transaction.
The ideal workflow is:

createrawtransaction '[]' '{"<addr>": <amount>}' //set no inputs
fundrawtransaction <hex>
you'll get back {"hex":<tx>, "fee": <amount>, "changepos": <pos of the change output>}
Check the fee, maybe increase it (requires Bitcoin Core 0.14, use the feeRate of fundrawtransaction.
signrawtransaction <hex>
sendrawtransaction <hex>

This is a great suggestion! But we want to spend from a particular address in the wallet which we retrieve from listunspent .. so it won't work for us as there is no way to do this filtering in fundrawtransaction

In that case you need to calculate the transaction size by yourself by assume 72byte per EC sig. But it depends what types of inputs you support (P2PKH, etc.).

Well they are all multisig addresses .. So P2SH


text:
Extra bytes in coinbase
I am working on blk file parser and there is problem with one of coinbase tx.
hex:
00000000: 0b11 0907 cf44 0e00 0000 0020 7f00 6e33  .....D..... ..n3
00000010: 4287 2eb8 c047 5d7c 16de 070e 5d88 1958  B....G]|....]..X
00000020: 91c2 6460 7104 0000 0000 0000 bef7 a5e4  ..d`q...........
00000030: 017b 7455 9f92 e883 d556 e21f ce14 aeb7  .{tU.....V......
00000040: 88c9 b2e2 ffec 2c6f 4b14 a901 0cb5 1358  ......,oK......X
00000050: cc2c 051a e9b8 2286 fdaf 0201 0000 0000  .,....".........
00000060: 0101 0000 0000 0000 0000 0000 0000 0000  ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 ffff ffff 3603 7e76 0f00 040c b513  ......6.~v......
00000090: 5804 b761 0630 0ce8 3a09 58c5 f700 0000  X..a.0..:.X.....
000000a0: 0000 000a 636b 706f 6f6c 122f 4e69 6e6a  ....ckpool./Ninj
000000b0: 6150 6f6f 6c2f 5345 4757 4954 2fff ffff  aPool/SEGWIT/...
000000c0: ff02 0e2a d213 0000 0000 1976 a914 876f  ...*.......v...o

Let's decode it according to bitcoin developer reference specification.
0b11 0907 - testnet3 magic number
cf44 0e00 - block size
Block header begins:
0000 0020 - block version
7f00 6e33 4287 2eb8 c047 5d7c 16de 070e 5d88 1958 91c2 6460 7104 0000 0000 0000 - previous block hash
bef7 a5e4 017b 7455 9f92 e883 d556 e21f ce14 aeb7 88c9 b2e2 ffec 2c6f 4b14 a901 - merkle root hash
0cb5 1358 - time
cc2c 051a - bits
e9b8 2286 - nonce
Block header ends;
fdaf 02 - transactions in block
And now the problem:
01 0000 00 - transaction version
00 - inputs counter
0101 - 2 unknown bytes.
I didn't found any information about this 2 unknown bytes. Any help appreciated.

this block is valid. blk file from bitcoin-core client

I found the answer. This 'unknown bytes' described in bip144


text:
Is there an API for price estimation?
I need to display at my website what a certain amount of Bitcoins will likely cost. I can fetch the latest price using the MtGox API and multiply it by the amount, but for large amounts this is inaccurate because the buy itself will move the price. The only other option seems to fetch the whole orderbook, and calculate it myself each time, but isn't there an API for this?

You might want to post this as a feature request for the XChange project in GitHub. See https://github.com/timmolter/XChange

Unless you want to move really big amount of Bitcoins, the order itself shoudldn't move the Bitcoin price too much. You can always go for the current price +- some spread.

    while amount > 0:
        amount_i = int(mtgox_depth['bids'][index]['amount_int'])
        price_i = int(mtgox_depth['bids'][index]['price_int'])
        if amount_i < amount:
            total_value += amount_i * price_i
            amount -= amount_i
        elif amount_i >= amount:
            total_value += amount * price_i
            amount = 0
        index -= 1
    spot = int(total_value / float(total_amount))
    return spot

Something like this, where mtgox_depth is a json object of https://mtgox.com/api/1/BTCUSD/depth?raw, should work for you.

Thanks. I already use code like that right now, but it requires to the download the whole orderbook each time, so I hope there was some API to do this serverside.

Although I don't have the answer you want, be sure you're not actually downloading the whole order book. I believe that link I provided in the answer is only a partial order book, even as it is a lot of orders.


text:
Why are balances not in the blockchain?
Currently one needs to traverse the full blockchain to determine the balance of a certain address. Wouldn't it be much more efficient to include the new balance along with each transaction? That way clients wouldn't need the whole blockchain to verify a transaction, since miners already verified the balance.

That would not work because the way the system is designed, accounts do not have a pile of coins to spend. Spendable coins exist in the form of transaction outputs that have not yet been used as the input of a subsequent transaction. To form a valid transaction, you must pull in coins from specific transaction outputs, not from specific accounts.
One could imagine a Bitcoin-like system where accounts have balances. In such a system, you could simply report the new balance of an account in any block that modified that account's balance. Then a transaction could pull in from accounts rather than specific transaction outputs. But that system would not be Bitcoin.
Essentially, Bitcoin was not designed around accounts. Accounts are just a convenient way for human beings using Bitcoin to think about the set of transaction outputs that can be claimed with the same key. (Or to form transactions that we know can only be claimed by a particular person.)

Technically, you should use a new address for each new incoming transaction to protect your anonymity. The standard client likewise sends the remaining balance after a transfer to a new keypair. This would make the transaction amount synonymous with the balance.
As for addresses that are used more than once - it would be first of all a bit wasteful (having to add an extra field for each transaction), and moreover - you don't need to claim all the inputs for a given address each time you send coins. Sometimes you don't want to claim each input - for example with low-confirmation transactions. Say you have a transaction that has 10 confirmations and one that has 1 - if you want to make sure your transfer will go through even if some blocks get overwritten, you would only use the 10 confirmation input and leave the 1 confirmation one to mature. Such operations would make the balances go wrong very quickly.

I think you would have to add more than one field per transaction.  Every input and every output would change the balance of an address, so every input and every output would have to state the new balance for the address it touches.

@ChrisMoore A very low price to pay in exchange for assuring that there is agreement on what those balances are.

When you receive a payment, you could presumably find from the transaction which block an input came from, and then request a copy of just that block to verify the balance.  But without having all the blocks from that block to the latest one you wouldn't know whether the inputs had already been spent in another transaction.
Proposed schemes which do appear to be workable maintain a list of the balance of all addresses in a separate blockchain.


text:
How does Segwit prevent ASICBOOST?
I know that ASIC BOOST exploits the fact that the blockheaders are not a multiple of 64 bytes, but what is the change in Segwit that prevents this? Are they always a multiple of 64 bytes now for example?

There's 2 versions of ASICBOOST:

Overt where miners use bits in the version number as extra nonce space
Covert where miners "mine" merkle trees with 4 bytes collisions

The overt version is very easily detectable, whereas the covert one isn't.
To mine these merkle trees for the overt version, miners need to shuffle the transactions in the block.
Without Segwit transactions in a block, it is possible to do it in a very low number of SHA-256 operations ; with Segwit transactions, a change of order forces miners to recompute the witness commitment which is stored in the coinbase. That makes it way more expensive to mine merkle trees and destroys the advantage covert ASICBOOST provides.
You can find a detailed explanation here.

SegWit doesn’t eliminate the ability to achieve the full performance of covert ASICBOOST. Study more carefully the source you cited. If the block has only 1 or 2 transactions, then the difference in covert ASICBOOST performance is not very significant. Given Lightning Networks, we can expect the possibility of settlement transactions that can occupy an entire block. So then even blocks with only 1 or 2 transactions will not be conspicuous evidence of covert ASICBOOST. Let’s presume LN Mt.Box hubs can possibly collude with covert ASICBOOST miners.

It is true that the less transactions, the overhead is lessened. However, Lightning settlement transactions don't take more space than other transactions. The coooperative case is a simple 2-of-2 multisig spend, and thousands of those happen per day already.

Bitcoin will become a settlement network between whales with huge transactions with a plurality of inputs and/or outputs. Think of centralized exchanges which is what LN Mt. Box hubs will become. The on-chain transaction fees with be $50,000. Only the uber wealthy will transact on-chain and open LN channels on-chain. Everyone else will be on fractional reserve banking. That is the master plan and the only way the technology works.


text:
Calculating address balance
Is there some method how you can calculate the balance of an address without resorting to web-based API's? When I import a private key the rescanning of bitcoind takes hours. I know to get a balance you would need to iterate over all transactions in the whole chain, but maybe there is still some optimization possible.

possible duplicate of How to get an address's balance with the bitcoin client?

@StevenRoose It's not a duplicate, I know how to do it via bitcoind, I'm looking for alternatives.

I see, sadly I cannot revert my vote, but maybe no additional close votes are cast. (http://meta.stackexchange.com/questions/915/can-we-have-the-ability-to-rescind-a-close-vote-before-it-closes)

I would like to know how to this in bitcoind. The question Steven linked is also not properly answered. They all say "its not possible". What I'd like to know is how to traverse all unspent outputs, and find those corresponding to a given address, possibly not in the wallet.

see also How to know there is a new transaction?

You would need to use a client node that has a different architecture, from the satoshi's client.
Specifically, the client must be able to quickly browse through all the unspent outputs.
You may want to check this table for clients that support Multi-wallet.
Among them, there is a new client Gocoin. It is currently in a testing phase, and you need to build it yourself from sources, but if you have a PC powerful enough to run it, it needs less than a second to give you an unspent balance of any standard bitcoin address.

This is probably not a full solution, but maybe a possible optimization.  Since you send all BTC out of the address in every send transaction, you should only have to look at the transactions that happen after the last sending transaction to calculate the value at an address.


text:
How do I restore my Multibit wallet without seed words?
So I created my BTC wallet back in 2013 and had always kept a backup of the files handy. The computer I used has since died and now that I've installed the latest version of Multibit on my new computer, I can't figure out how to restore my wallet. It asks me for seed words, but I do not recall ever having seed words when the wallet was created.
I have the files multibit.info, multibit.wallet, and wallet.aes.json among other backups (along with my password). How do I get to a point where I can get my wallet up and running again?

In 2013, MultiBit was not using Hierachal Deterministic (HD) wallets. The latest version is MultiBit HD. If you want to use the older version you can use MultiBit Classic and that should let you import your multibit.wallet file.

Take your private keys you had with Multibit, go make a new wallet on the blockchain.com site and restore your wallet there

Please elaborate on the steps involved in doing so. This could in general help others who would have faced similar problems.

You should not store more in an online wallet than you can afford to lose. If the online wallet provider goes offline, goes broke, is hacked, is shut down or, if they just steal your coins then they are lost.


text:
Can we experiment on Signet with multiple proposed soft forks whilst maintaining the ability of full nodes to validate the Signet chain?
One of the goals of Signet is to test proposed soft forks before they are activated on mainnet. Not only does this mean proposed soft forks such as Taproot that at the time of writing (August 2020) are at an advanced stage (BIPs reviewed, PR under review in Bitcoin Core codebase, activation discussion started) but potentially other proposed soft forks that are at a less advanced stage. It is also worth noting that despite Taproot being at an advanced stage there has still been discussion on changing the BIP (340). With less advanced proposed soft forks they could change and be updated at multiple points before either being activated on mainnet or rejected entirely.
How will Signet allow for this experimentation and testing on proposed soft forks whilst maintaining the ability for full nodes to validate the Signet chain from genesis?

The way Signet will likely address this is to have two classes of Signet full nodes. The first class of Signet full nodes will stay on a stable Signet version and only update once the proposed soft fork is activated on mainnet. The second class of Signet full nodes will run experimental Signet versions and as a result will need to update every time a new proposed soft fork is added or a change to an existing proposed soft fork is added. If they don't they risk being forked off the Signet chain. This is similar to full nodes needing to update immediately assuming there were regular hard forks.
As an example (credit to AJ Towns for this explanation), imagine there were three classes of full node.

Doesn't enforce Taproot at all
Enforces the current Taproot rules from block 800
Enforces a future new set of Taproot rules from block 2400 (assuming R tie breaker changes)

Class 1 is able to validate the entire chain by ignoring the new soft fork restrictions in the same way as pre-SegWit versions of Bitcoin Core on mainnet can treat SegWit spends as anyone-can-spend. Once the proposed soft fork is activated on mainnet these Signet full nodes can safely upgrade to what would likely be a new stable and non-experimental Signet version.
Class 2 though does start enforcing Taproot rules from block 800. When those Taproot rules change at block 2400 unless the Class 2 nodes update they will start rejecting transactions following the new Taproot rules as they won't recognize them. They will evaluate those transactions according to the old Taproot rules and they won't be valid Taproot spends according to those old rules.
Therefore Class 2 Signet full nodes will need to upgrade every time there is a new experimental Signet version. Obviously this isn't ideal but it seems a reasonable trade-off given that the whole purpose of Signet is experimentation and testing with no real value on the line. If you can't test things for fear of hard forking Signet it isn't providing the testbed and staging ground that is required before changes are considered for mainnet with hundreds of billions of dollars on the line.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Is it recommended not to take hardware wallets through international airport security scanners?
I suspect this won't get an answer as it is highly speculative (on the scanning technology in airports) but does anyone have any information or informed views on whether you can take hardware wallets/signers through international airport security scanners and not leak any (private) key material? Or is it recommended one does not take hardware wallets/signers through international airports unless they are part of a multisig and the other signing devices aren't on you at the same time?

Yeah this question does seem potentially speculative. It is the whole point of a SE though to avoid leaking a secret by all means, so it sounds unrealistic just observing through a scanner could breach its protections. That said, it seems to me the bigger risk at TSA is "unlock this thing or we [confiscate / lock you up / enter threat]". So yeah, speculative question. :) Maybe try a more objective one at a hardware stackexchange?

@AntoinePoinsot: Thanks, I'll close.


text:
How is the progress field of Bitcoin Core's synchronisation/Initial Block Download debug logs calculated?
The debug.log file during Inital Block Download (IBD) typically includes something like the following1:
2023-09-09T10:04:32Z UpdateTip: new best=00000000000000001004395fb5a59a4d44e51e7f9ff3504c2a5d6e7f06885680 height=371337 version=0x00000003 log2_work=83.254204 tx=81017610 date='2015-08-24T17:00:31Z' progress=0.093041 cache=226.8MiB(31135360txo)

It includes
progress=0.093041

How is the progress field calculated?

What are its possible values/units?

What is the function in bitcoin core that does the calculation?

Possibly related:
https://bitcoin.stackexchange.com/a/123281/34190

1. Copied from this Github Issue for illustration (the question is unrelated to the issue, it was just the first example I found to copy).

The progress value is the ratio between the total number of transactions in the currently-validated part of the active chain, and an estimate for the total number of transactions in the chain. That estimate is computed based on statistics that are hardcoded in the software, and updated for major releases (including a known point in the chain at a known timestamp, with a known number of transactions, and a predicted rate of transactions per second afterwards).

It is a dimensionless unit, taking values between 0.000 and 1.000.

For the latest Bitcoin Core release (27.1), the code can be found here.


text:
What are the mean, median, range, limits and trends in the size of coinbase transactions?
I'm attempting to estimate how much "spare capacity" there is for merged mining protocols.
As I understand it the scriptSig in the coinbase transaction must be less than 100B. However, the miner can include multiple OP_RETURN statements. For instance this recent block has a coinbase tx with 3 OP_RETURN statements and is 353 bytes.
A. Is there a limit in the overall size of the coinbase tx?
B. Is there a limit to the number of OP_RETURN statements in the coinbase transaction?
C. What is the mean, median, range and general trend in the size of coinbase transactions?

A. Is there a limit in the overall size of the coinbase tx?

I thought the limit for coinbase was the same as the general limit for transactions.
What is the maximum size of a transaction?
But as Murch noted:

coinbase transactions are not subject to mempool policies and standardness rules, because they are not relayed on the network outside of the context of being included in blocks. Therefore, they are only limited by the blockweight limit instead of the weight limit of 400,000 wu for standard transactions.

n.b. How many weight units does an input use?

B. Is there a limit to the number of OP_RETURN statements in the coinbase transaction?

So far as I know, you can only have one OP_RETURN in any one Bitcoin script, any following data is not parsed.
Like any transaction, a Coinbase transaction can have many outputs, although only one input. So it can have numerous OP_RETURNs.
As Pieter Wuille noted:

There can be multiple OP_RETURN opcodes within a transaction output too. The typical standardness rules about OP_RETURN outputs require the opcode to be the first script byte, but again because standardness rules don't apply to coinbase transactions, that doesn't matter.

It is probably worth saying that scripts in coinbase inputs should never get executed, although the first opcode should push the block height. Other than that, their contents can be considered as arbitrary byte data which might contain the value 0x6A (OP_RETURN) multiple times.

C. What is the mean, median, range and general trend in the size of coinbase transactions

You can work this out if you collect data using a full node such as Bitcoin core. See

Open-source forensic analysis tools to analyze full-node's local copy of Bitcoin blockchain?
How can I take a "snapshot" of the bitcoin blockchain and analyze it?

You can write your own analytic tool using a blockchain parser. I wrote my own parser library (unpublished) which could be used as follows
package main

import (
    bc "blockchain/blockchain"
    "fmt"
    "os"
    "time"
)

func main() {
    b, err := os.ReadFile(`blk02000.dat`)
    check("Reading data", err)

    parser := bc.New(BlockHandler)
    err = parser.ParseBlocks(&b)
    check("Parsing data", err)
}

// Extract length of coinbase transactions
var i int = 1
func BlockHandler(b *bc.Block) {
    blockDate := time.Unix(int64(b.Header.TimeStamp), 0).Format("2006-01-02 15:04")
    coinbase :=  b.TransactionList[0]
    txBytes := coinbase.Size()
    fmt.Printf("Block %5d dated %s has coinbase of %5d Bytes\n",
        i, blockDate, txBytes)
    i++
}

// Rudimentary convenience function for error handling
func check(where string, err error) {
    if err != nil {
        fmt.Println(where, err)
        os.Exit(1)
    }
}

results
Block     1 dated 2020-03-14 06:24 has coinbase of   300 Bytes
Block     2 dated 2020-03-14 14:20 has coinbase of   300 Bytes
Block     3 dated 2020-03-14 08:42 has coinbase of   246 Bytes
Block     4 dated 2020-03-13 23:08 has coinbase of   300 Bytes
Block     5 dated 2020-03-14 04:19 has coinbase of   306 Bytes
Block     6 dated 2020-03-14 15:47 has coinbase of   295 Bytes
Block     7 dated 2020-03-13 14:39 has coinbase of   216 Bytes
Block     8 dated 2020-03-14 13:04 has coinbase of   362 Bytes
Block     9 dated 2020-03-14 09:31 has coinbase of   308 Bytes
Block    10 dated 2020-03-14 10:16 has coinbase of   306 Bytes
Block    11 dated 2020-03-14 15:39 has coinbase of   346 Bytes
Block    12 dated 2020-03-14 08:05 has coinbase of   377 Bytes
Block    13 dated 2020-03-14 13:48 has coinbase of   362 Bytes
Block    14 dated 2020-03-14 17:15 has coinbase of   299 Bytes
Block    15 dated 2020-03-14 06:18 has coinbase of   342 Bytes
Block    16 dated 2020-03-14 11:37 has coinbase of   377 Bytes
Block    17 dated 2020-03-14 12:35 has coinbase of   377 Bytes
Block    18 dated 2020-03-14 17:32 has coinbase of   377 Bytes
Block    19 dated 2020-03-14 13:22 has coinbase of   316 Bytes
Block    20 dated 2020-03-14 18:37 has coinbase of   291 Bytes
Block    21 dated 2020-03-14 15:10 has coinbase of   377 Bytes
Block    22 dated 2020-03-14 09:35 has coinbase of   362 Bytes
Block    23 dated 2020-03-14 12:58 has coinbase of   295 Bytes
Block    24 dated 2020-03-14 06:37 has coinbase of   317 Bytes
Block    25 dated 2020-03-14 18:46 has coinbase of   299 Bytes
Block    26 dated 2020-03-13 23:12 has coinbase of   298 Bytes
Block    27 dated 2020-03-14 16:27 has coinbase of   362 Bytes
Block    28 dated 2020-03-14 16:53 has coinbase of   377 Bytes
Block    29 dated 2020-03-14 19:04 has coinbase of   217 Bytes
Block    30 dated 2020-03-14 07:26 has coinbase of   342 Bytes
Block    31 dated 2020-03-14 13:42 has coinbase of   289 Bytes
Block    32 dated 2020-03-14 20:10 has coinbase of   295 Bytes
Block    33 dated 2020-03-14 12:32 has coinbase of   295 Bytes
Block    34 dated 2020-03-13 08:50 has coinbase of   342 Bytes
Block    35 dated 2020-03-14 18:07 has coinbase of   308 Bytes
Block    36 dated 2020-03-14 16:59 has coinbase of   263 Bytes
Block    37 dated 2020-03-14 21:43 has coinbase of   296 Bytes
Block    38 dated 2020-03-14 20:27 has coinbase of   377 Bytes
Block    39 dated 2020-03-14 18:33 has coinbase of   289 Bytes
Block    40 dated 2020-03-14 18:15 has coinbase of   290 Bytes
Block    41 dated 2020-03-14 23:05 has coinbase of   362 Bytes
Block    42 dated 2020-03-13 16:19 has coinbase of   377 Bytes
Block    43 dated 2020-03-14 13:48 has coinbase of   263 Bytes
Block    44 dated 2020-03-14 09:30 has coinbase of   308 Bytes
Block    45 dated 2020-03-14 20:07 has coinbase of   316 Bytes
Block    46 dated 2020-03-15 01:10 has coinbase of   289 Bytes
Block    47 dated 2020-03-14 19:19 has coinbase of   362 Bytes
Block    48 dated 2020-03-14 19:51 has coinbase of   300 Bytes
Block    49 dated 2020-03-14 22:40 has coinbase of   377 Bytes
Block    50 dated 2020-03-15 02:16 has coinbase of   296 Bytes
Block    51 dated 2020-03-14 21:43 has coinbase of   217 Bytes
Block    52 dated 2020-03-14 15:43 has coinbase of   308 Bytes
Block    53 dated 2020-03-14 22:42 has coinbase of   306 Bytes
Block    54 dated 2020-03-14 20:45 has coinbase of   306 Bytes
Block    55 dated 2020-03-14 12:13 has coinbase of   296 Bytes
Block    56 dated 2020-03-15 04:06 has coinbase of   342 Bytes
Block    57 dated 2020-03-15 01:09 has coinbase of   285 Bytes
Block    58 dated 2020-03-15 05:13 has coinbase of   299 Bytes
Block    59 dated 2020-03-15 00:31 has coinbase of   289 Bytes
Block    60 dated 2020-03-14 17:39 has coinbase of   308 Bytes
Block    61 dated 2020-03-15 01:12 has coinbase of   290 Bytes
Block    62 dated 2020-03-14 22:31 has coinbase of   299 Bytes
Block    63 dated 2020-03-15 08:39 has coinbase of   362 Bytes
Block    64 dated 2020-03-14 13:30 has coinbase of   295 Bytes
Block    65 dated 2020-03-15 01:45 has coinbase of   377 Bytes
Block    66 dated 2020-03-15 02:08 has coinbase of   311 Bytes
Block    67 dated 2020-03-15 03:10 has coinbase of   377 Bytes
Block    68 dated 2020-03-14 19:02 has coinbase of   290 Bytes
Block    69 dated 2020-03-15 01:31 has coinbase of   377 Bytes
Block    70 dated 2020-03-15 03:41 has coinbase of   377 Bytes
Block    71 dated 2020-03-15 09:34 has coinbase of   342 Bytes
Block    72 dated 2020-03-15 03:25 has coinbase of   285 Bytes
Block    73 dated 2020-03-14 16:20 has coinbase of   261 Bytes
Block    74 dated 2020-03-14 21:04 has coinbase of   298 Bytes
Block    75 dated 2020-03-15 04:01 has coinbase of   362 Bytes
Block    76 dated 2020-03-15 05:09 has coinbase of   377 Bytes
Block    77 dated 2020-03-15 04:32 has coinbase of   296 Bytes
Block    78 dated 2020-03-15 05:59 has coinbase of   289 Bytes
Block    79 dated 2020-03-15 03:24 has coinbase of   362 Bytes
Block    80 dated 2020-03-15 05:22 has coinbase of   377 Bytes
Block    81 dated 2020-03-14 18:40 has coinbase of   313 Bytes
Block    82 dated 2020-03-13 17:48 has coinbase of   298 Bytes
Block    83 dated 2020-03-15 10:38 has coinbase of   316 Bytes
Block    84 dated 2020-03-15 06:46 has coinbase of   362 Bytes
Block    85 dated 2020-03-15 06:28 has coinbase of   342 Bytes
Block    86 dated 2020-03-13 10:28 has coinbase of   306 Bytes
Block    87 dated 2020-03-15 06:41 has coinbase of   362 Bytes
Block    88 dated 2020-03-14 23:00 has coinbase of   306 Bytes
Block    89 dated 2020-03-15 04:09 has coinbase of   339 Bytes
Block    90 dated 2020-03-14 21:11 has coinbase of   295 Bytes
Block    91 dated 2020-03-15 11:18 has coinbase of   266 Bytes
Block    92 dated 2020-03-15 09:37 has coinbase of   306 Bytes
Block    93 dated 2020-03-15 09:07 has coinbase of   296 Bytes
Block    94 dated 2020-03-15 11:51 has coinbase of   299 Bytes
Block    95 dated 2020-03-15 09:26 has coinbase of   306 Bytes
Block    96 dated 2020-03-15 02:03 has coinbase of   289 Bytes
Block    97 dated 2020-03-15 12:26 has coinbase of   263 Bytes
Block    98 dated 2020-03-14 23:34 has coinbase of   377 Bytes
Block    99 dated 2020-03-15 10:32 has coinbase of   342 Bytes
Block   100 dated 2020-03-15 11:35 has coinbase of   342 Bytes
Block   101 dated 2020-03-15 10:51 has coinbase of   291 Bytes
Block   102 dated 2020-03-15 10:18 has coinbase of   377 Bytes
Block   103 dated 2020-03-15 05:58 has coinbase of   299 Bytes
Block   104 dated 2020-03-15 11:57 has coinbase of   306 Bytes
Block   105 dated 2020-03-15 13:38 has coinbase of   377 Bytes
Block   106 dated 2020-03-15 03:58 has coinbase of   298 Bytes
Block   107 dated 2020-03-15 02:17 has coinbase of   306 Bytes
Block   108 dated 2020-03-15 05:25 has coinbase of   296 Bytes
Block   109 dated 2020-03-15 06:43 has coinbase of   299 Bytes
Block   110 dated 2020-03-15 10:41 has coinbase of   342 Bytes
Block   111 dated 2020-03-15 12:15 has coinbase of   300 Bytes
Block   112 dated 2020-03-15 11:54 has coinbase of   299 Bytes

Obviously its a small step from that to accumulate min, max, median values by year. Or just feed that data into some other tool.

Thanks, the linked coinbase tx has three OP_RETURNS, no? Perhaps I'm misunderstanding something...

.. interesting; a miner could choose to make the coinbase tx take up the entire block

Yes sorry, I was thinking of one OP_Return per script so one per input or output. A coinbase transaction can have many outputs. (Answer updated to clarify)

Great answer. Small correction: coinbase transactions are not subject to mempool policies and standardness rules, because they are not relayed on the network outside of the context of being included in blocks. Therefore, they are only limited by the blockweight limit instead of the weight limit of 400,000 wu for standard transactions.

@Murch: thanks for the correction. Answer updated.

There can be multiple OP_RETURN opcodes within a transaction output too. The typical standardness rules about OP_RETURN outputs require the opcode to be the first script byte, but again because standardness rules don't apply to coinbase transactions, that doesn't matter.

@Pieter thanks. Answer updated. I'm a bit puzzled though. Don't coinbase outputs get validated by nodes so that valid outputs without OP-RETURN can be added to the UTXO list? I'll see what I can learn about standardness vs other validation/checking but more hints welcomed. I guess OP's question B is already answered adequately now though.

There are almost no validity rules that apply to transaction output scripts. Most rules you'd imagine are only run when the output is attempted to be spent, but OP_RETURNs obviously can't ever be spent. There is some "shortcut" implementation rule in Bitcoin Core that detects certain subsets of outputs as unspendable, so they can avoid being inserted into the UTXO set (as they can't ever be spent anyway), but that's a local optimization and doesn't affect what transactions are valid.

A. Is there a limit in the overall size of the coinbase tx?
Yes, but its only limited by the size of the block. A miner could choose to have the coinbase transaction fill the entire block!
B. Is there a limit to the number of OP_RETURN statements in the coinbase transaction?
It is only limited by the transaction size limit mentioned above.
C. What is the mean, median, range and general trend in the size of coinbase transactions?
Based on analysis of my local blk.*dat files (up to blk3063.dat):
Mean: 259 bytes (3 sig fig)
Range-Max: 168 bytes
Range-Min: 90 bytes
Range was the same for all parsed blk.dat file aside from blk0000.dat (129 - 3434 bytes)
Trends in mean and median:


text:
How do pre-Segwit nodes interpret transactions containing Taproot OP_CHECKSIGADD?
When pre-Segwit nodes encounter blocks with Segwit transactions, they'll see them as "spent anyone can spend" transactions and so can validate them and move on to the next block.
However, when they encounter a transaction with the opcode 0x6a (Taproot only OP_CHECKSIGADD) they'd see it as an undefined opcode and thus see the transaction and its block as invalid. They would then fork or become stranded.
After reading this, though, it seems there is some way by which the old nodes handle/avoid OP_CHECKSIGADD.
What is the mechanism by which the old nodes handle the 0x6a/OP_CHECKSIGADD opcode or are able to see these transactions as a "spent anyone can spend"?

Tapscript can only appear in leaf scripts. Leaf scripts can only appear in the witness stack of a P2TR script path spend. Pre-segwit nodes don’t see witness data. Ergo, a pre-segwit node will never encounter an OP_CHECKSIGADD.

Oh, in my comment on the other answer I assumed OP was asking about pre-taproot nodes. But indeed, pre-segwit nodes won't see the script at all.

@PieterWuille And if one did try to include a tx with non-witness script with an 0x6a opcode in it; it would fail on pre-Segwit/pre-Taproot as undefined, and on post Taproot as it detects that it's not a P2TR.

OP_CHECKSIGADD is only defined for SegWit v1 (Taproot). A pre SegWit v1 node that doesn't recognize Taproot rules will treat all SegWit v1 outputs as "anyone can spend" regardless of whether it has a OP_CHECKSIGADD opcode in the script or not. (Of course it doesn't know about the OP_CHECKSIGADD rules). It is only doing partial verification on the transactions in a block and accepting any SegWit v1 spends as valid. Other nodes that do recognize SegWit v1 rules will do full verification of all transactions in the block and hence ultimately the pre SegWit v1 node will still follow the same chain.
In short the SegWit version is given precedence over the presence of an undefined opcode. An undefined SegWit version is "anyone can spend".

Thanks, my understanding is that the 0x6a opcode must only occur in the witness field of a transaction. If it was in the scriptSig or scriptPubKey field then all nodes would reject it.

@Lee When a pre-taproot node sees a taproot spend, they do not even get to seeing/interpreting any opcodes. They just see it as an output of an unknown type, and say "ok, no rules apply, all good". From their perspective, the spending of such an output does not involve any script at all.


text:
Should the BIP 39 mnemonic sentence checksum be eliminated from the standard? Does it do more harm than good?
Should the BIP 39 checksum be eliminated from the standard? Should the BIP39 checksum be made optional to improve security?

No, I don't think the checksum should be removed or made optional. It doesn't do more harm that good.
Bip39 is a simple way to copy computer-generated randomness from the computer onto, eg a piece of paper. This Bip39 mnemonic sentence can then be used to restore your wallet, should you lose your computer. The checksum provides some degree of certainty on the correctness of the mnemonic you enter.
However, lately, people have begun to explore how to create their own randomness when creating a new wallet. This can be done with dice, coin-flips, etc. You might want to do this for example if you don't trust that the computer's random-number generator is doing its job well.
Since most wallets offer a way to restore a wallet using bip39 mnemonic sentences, people are trying to use that feature as a way to create new wallets from manually generated randomness. This is where the problem lies.
Actually, it's stated in the Motivation section of bip39:

This guide is meant to be a way to transport computer-generated randomness with a human-readable transcription. It's not a way to process user-created sentences (also known as brainwallets) into a wallet seed.

So we're actually using bip39 in a non-intended way, and that's why it's so awkward. The solution to this is not to modify bip39, but to acknowledge that we're doing it wrong.
Instead of entering manual randomness into a computer through bip39, we should invent other means of transferring the manual randomness into the computer. Some wallets already offer this. For example, ColdCard lets you input dice rolls one by one until you think you have enough randomness and then the device will provide you with a bip39 mnemonic sentence for you to backup on paper. That's one way to do it.
As you may have noticed, since writing about this on Twitter (cited in Michael Folksons's answer), Mastodon, and Reddit, I've changed my mind about this. I no longer think that making bip39 checksum optional is the solution. The solution is to not use bip39 to input randomness into a computer. The purpose of BIP39 is to transfer computer-generated randomness to a human, not the other way around.

Not having a checksum means that users would be able to easily pick their own mnemonic seed phrase in a non-random way, and that presents a really huge security risk to those users!
While it would also enable easier provable physical entropy gathering methods, this enabling of extremely poor entropy seed phrases could likely be of huge detriment to naive users. See for example, the brain flayer project, which can crack brain wallets with astonishing efficiency. To be clear: users that put funds in a wallet generated from a low-entropy seed are at a greatly increased risk of having all funds stolen.
Not having to trust software to produce the checksum is a rather small gain, considering that the user is still going to have to trust software to produce keypairs and bitcoin addresses from the mnemonic seed phrase already. Calculating keypairs or payment addresses by hand is pretty obviously untenable, and without these keys / addresses, the wallet is of no real use.
So considering the extremely large risk to naive users, and the extremely small gain in having less software-dependance, it is likely a bad idea to consider removing checksums from the BIP39 standard.

Kalle Rosenbaum argued for this on Twitter.

The checksum is between 4 (for 12 words) and 8 (24 words) bits. This is a very weak checksum. For 12 words, this would mean 1/16 probability of failure to detect a mistyped sequence of VALID words!

The words themselves are a much stronger checksum. Any misspelled word has a much higher probability to be detected due to word missing in wordlist; It's unlikely to misspell a word so that it becomes another valid word.

When does the checksum help you? It's not when you write down the mnemonic on paper. The paper/pencil doesn't check the checksum.

The checksum could help when you want to recover the wallet. If the checksum is wrong AND all words exist in the wordlist, it will help you realize there's something wrong with it.

But you'll also realize there's something wrong with it if there are no transactions or bitcoin in your "recovered" wallet, or that you don't recognize the addresses generated in it. The checksum, at best, only helps you realize it earlier.

The checksum also has a major turnoff: It's hard to generate a mnemonic without a computer. You can roll dice to generate 128 bits of entropy, but then you need to put those bits into a computer to calculate the checksum, "the final word".

This "final word" step is a security risk. If you're a novice, it's very hard to do the final word generation in a secure way. This risk is far worse than the dubious benefit explained above.

If wallets would accept mnemonics without checksum you could create a mnemonic with ONLY dice and a wordlist and give it to a hardware wallet or airgapped computer and be done with it. You don't have to setup a secure environment to generate a nearly useless checksum.

Exactly how BIP39 could be modified to make checksum optional is another question, but it's doable.


text:
What are the various denial of service vectors on the Lightning Network?
What are the various denial of service (DoS) vectors on the Lightning Network?

This is a draft answer. Alternative answers (as always) are welcome.
Alex Bosworth listed a number of DoS vectors on Twitter. This isn't necessarily an exhaustive list.

TCP level flooding
P2P protocol flood
Too-many graph updates
Huge unsettled HTLCs
Many slow HTLCs
Pathfinding distraction
RBF pinning
0-Base PPM floor HTLCs
Many channel open proposals
Fake Neutrino data
KeySend spam


text:
How would one test the impact on the Lightning Network of a network split on the base layer?
How would one test the impact on the Lightning Network of a network split on the base layer (the blockchain progressing on two separate branches with each branch unaware of the other competing branch)?
Do any Lightning Network spec or implementation test frameworks currently allow you to simulate a network split on the base layer for testing?
This was asked by DrNo_21M on Twitter and has been paraphrased.

Christian Decker answered this on Twitter.
pyln-testing which is used for testing in c-lightning uses a proxy in front of bitcoind allowing you to mock responses. This allows you to get any node stuck at a height while the others get new blocks. A cooperative close on a smaller base layer subnetwork followed by network unification and chain tip rollback can also be tested on c-lightning but without long chain splits given how unlikely they are to happen (in normal times).
Jamal James added that Polar can simulate this scenario (use 3 bitcoind nodes then take the middle one offline, mine blocks on the other two, this results in a chain split).


text:
How do I move my c-lightning node to a different machine without closing my channels?
If I wanted to move my c-lightning node to an other machine, is there a way to do that (and how)? Specifically, without changing my node key or closing my channels?
This question was asked by plant-boy on IRC.

This question was answered by Christian Decker on IRC.

Yes, it's a bit tricky, because you never want to start the old copy
once you started the new one, but absolutely doable. 1) stop the old
node, 2) rename the .lightning directory so you can't accidentally
restart the old node (maybe verify that starting will create a new
node), 3) copy the renamed .lightning to new host in the place where
you'll be running from, and 4) point lightningd to the copied
directory and start it.
I'd also delete any copy remaining on the old host for safety
(starting that one may lead to loss of funds). While you're at it,
make a backup of hsm_secret files in the directory (those are static
and safe to copy)

No, more a footgun than a bug. Like accidentally broadcasting a revoked state could lead to you being punished and losing all your funds in the channel. Of course in an ideal world these footguns wouldn't exist but we don't know how to magic them away in the Lightning protocol (and it may well be impossible without designing a completely different protocol).

I guess we are talking definitions/semantics. I would describe a footgun as something that is addressed with user guidance, user docs etc but it is well known and understood in the protocol. I would describe a bug as something that is discovered out of the blue and needs squashing asap (ie there is no reason for it to exist). Of course in consensus systems some bugs (like the CHECKMULTISIG OP_0 bug) can't be squashed without a hard fork so the definitions do get confusing and create grey areas.

Correct, losing any state transition is deadly in the LN protocol, which is why we came up with a more forgiving protocol with eltoo. The protocol itself makes backups and restores pretty dangerous (and this migration is pretty much just a backup + restore), because even missing a single update may lead to loss of funds (and that's true for all implementations).

@Prayank: With onchain funds you can't lose your private key and you can't leak your private key to an attacker. Is that a bug? No it is a well understood footgun and limitation of cryptographic protocols. SImilarly Lightning requires you to monitor the chain (stay online) and not to accidentally broadcast a revoked state. Another well understood footgun and limitation of Lightning. Pre-eltoo if a malicious channel counterparty can trick you into broadcasting a revoked state you can lose your funds in the channel. No experimentation needed, it is a known footgun.


text:
What would maxdelay practically be used for in the Core Lightning pay plugin?
What would maxdelay practically be used for in the Core Lightning pay plugin?
This was asked on Telegram.

This was answered by Christian Decker on Telegram.

At each hop the node forwarding the payment will require to increase the CLTV delay. This is used so it can settle the downstream HTLC, turn around and still be in time to settle the upstream HTLC. The sum of these delays is the maximum time that funds may be locked if the HTLC is stuck, and so it is important to set up a maximum delay (so you get your funds back in a reasonable amount of time). On the flip side, setting that delay too low will result in some possible routes not being selected, as they'd exceed that delay. If not set, pay will use the --max-locktime-blocks option's value, which defaults to 2016 blocks (14 days).


text:
Can you help me get started running a signet Lightning routing node with intended 100 percent uptime?
I'd like to run a signet Lightning routing node with ~100 percent uptime. Until now I've just spun one up and shut it down but if a reliable signet Lightning Network is going to be a thing we need persistent, reliable signet Lightning nodes. So a bunch of questions (and any additional guidance would be welcome).

How do I obtain some signet coins?
How do I identify signet Lightning nodes that are also aiming for ~100 percent uptime?
Are there any publicly advertised signet Lightning nodes with large numbers of channels, large channel capacities that I should connect to first?
Are there any Lightning explorers that support signet?
How much would it cost (rough estimates per month) to run a signet Lightning node on a cloud provider (Digital Ocean etc) and allow anyone to connect and open channels with my node?

I know some of the answers to these questions will over time become outdated but at least it will be a starting point for someone who wants to do similar.

This is a draft answer. Alternative answers (as always) are welcome. Answers may become outdated over time. The time of writing is May 2023.
For more guidance on getting set up with signet generally (not Lightning specific) see these previous questions (1, 2).

How do I obtain some signet coins?

There are currently two signet faucets I am aware of. There is signetfaucet.com maintained by DG Lab and signet.bublina.eu.org maintained by jsarenik. You can get very small amounts of signet (up to 0.01 sBTC) from these faucets and they have in the past been subject to DoS attacks so if you want more significant amounts you should try to contact individuals privately. As long as I don't get spammed I'd be happy to provide larger amounts of signet Bitcoin if a signet address is supplied.

How do I identify signet Lightning nodes that are also aiming for ~100 percent uptime?

This is also somewhat difficult to do on the mainnet network although as there are routing fees with actual monetary value available there is more of a motivation to stay online. On the mainnet network you can look up particular Lightning nodes on Lightning explorers such as 1ml.com and amboss.space.....
Temporary dump of related links:
https://plebnet.wiki/wiki/Plebnet_Playground
https://ex.signet.bublina.eu.org/
https://mempool.space/signet
https://ln.anyone.eu.org/


text:
What are the differences between the design of BOLT11 and BOLT12?
What are the differences between BOLT11 and BOLT12?
Should I be using BOLT11 as of today (November 2022) even I like some of the features of BOLT12?
Relevant resources are bolt12.org, this tweet thread from LN_Capital and this presentation from Elle Mouton at Advancing Bitcoin 2022.


text:
What is the argument for not using proofs of possession in Bitcoin multisig and threshold key aggregation schemes?
A number of multisig and threshold sig key aggregation schemes developed outside of the Bitcoin ecosystem (e.g. SpeedyMuSig, FROST) use proofs of possession yet those developed in the Bitcoin ecosystem (MuSig1, MuSig-DN, MuSig2, a future Bitcoin specification of FROST too perhaps) generally avoid using proofs of possession. Why are proofs of possession generally avoided in Bitcoin schemes?
This was discussed during this online Socratic on MuSig2.


text:
How would you build a bill splitting app using Lightning? Does one already exist?
How would you build a bill splitting app (20+ people) on Lightning? Does one already exist? Setting up a BTCPay Server shop seems like over engineering.
This was asked by fulmolightning on Twitter.


text:
How should Bitcoin exchanges complete their UTXO consolidations and what factors should they consider?
How should Bitcoin exchanges complete their UTXO consolidations and what factors should they consider i.e. regularity, fixed or variable intervals, consolidate into how many UTXOs etc?

Will post Murch's tweet thread as an answer if he doesn't beat me to it https://twitter.com/murchandamus/status/1595809884058132480?s=20&t=OxlATNytppe2y3xYR7FM2w

Overview
Exchanges tend to have a much higher transaction volume than other entities, both inbound and outbound. Since payment batching greatly simplifies liquidity management and reduces the overhead per withdrawal by requiring only a single change output and a small number of inputs per transaction, many exchanges can operate their withdrawal wallets on a small count of UTXOs, but tend to grow their UTXO pool over time. Since every UTXO needs to be spent eventually to make use of the funds, the size of the UTXO pool as well as the output types it’s composed of translate to a corresponding future blockspace allocation.
An exchange has several goals in their UTXO management. They want to:

Minimize funds in their hot wallet to curb exposure to hacks
Maintain sufficient liquidity to send transactions throughout a sequence of slow blocks
Maintain a variety of UTXO amounts in case the wallet tries to make changeless transactions
Be able to service occasional larger withdrawals
Process withdrawals in a timely fashion for good customer experience and to minimizing support tickets
Minimize the on-going cost of their wallet operation
Minimize future blockspace obligations by delimiting UTXO pool growth

A wallet operator should identify the minimum necessary funds, the minimum useful amount for a UTXO and the minimum count of useful UTXOs to be able to timely serve withdrawal requests (including occasional larger withdrawals) and to bridge slow blocks. These metrics depend on the volume of the exchange’s operation and will shift over time. Beyond these operational needs, all other goals align with minimizing the UTXO count and funds in the hot wallet.
Flow of Funds in Wallet Setup
Two common wallet structures involve either receiving deposits to a warm wallet and restocking a hot wallet for withdrawals from the warm wallet, and receiving deposits directly to the hot wallet. I have more comprehensively described these two schemas in my article UTXO Management for Enterprise Wallets.
In either of these setups, the wallet operator should automate absorption of excessive deposits: when funds in the deposit wallets exceed the acceptable risk limit, the system should automatically create a transaction to move the excess of the funds to cold storage. (Obviously, the wallet should be locked down in a way that only the preset destination wallets can be paid with the automated transactions, and neither change outputs nor excessive fees can be used to extract funds.) Since this operation should conclude in a timely fashion, it is usually implemented as a regular send operation using the same coin selection and feerate estimation as withdrawal transactions.
Consolidation
Some of the exchange’s wallets (especially the deposit wallet) may accumulate a large count of UTXOs. Having a variety of amounts in the wallet increases the chances of changeless transactions, but with the common imbalance of inbound deposit transactions compared to the less frequent batched withdrawal transactions, especially UTXOs with lower amounts will end up not getting used for a long time. While some exchanges notoriously make all of their consolidation transactions once every three months, overpaying frighteningly and monopolizing blockspace, it is easy to automate an on-going consolidation process.
The goal in a consolidation transaction is opportunistically spend a lot of inputs when blockspace is abundant in order to reduce the amount of blockspace that must be bid on in the future at an uncertain feerate. Since consolidation transactions are inherently sent at a low feerate, they may take a lot of time to get confirmed. The wallet operator should take care that outputs from consolidation transactions are not eligible to be used as inputs for withdrawal or absorption transactions. Those high-priority child transactions would either get stuck, or if they CPFP the consolidation transaction to a competitive feerate, waste a lot of fees unnecessarily. Further, the wallet operator should not have too many consolidation transactions in flight at the same time, since this reduces the liquidity and flexibility of the wallet.
A simple approach for an automatic consolidation process would be to check once per hour:

is there already a consolidation tx in flight?
if not, are there 200+ UTXOs that have more than 2,000 confirmations and less than the “useful amount”?
if so, submit a consolidation tx with the 200 oldest UTXOs  from the above set as inputs at a feerate that might confirm within 24h

This approach makes sure that the wallet does not flood the mempool with consolidation transactions, may perform multiple consolidation transactions in a lull (e.g. weekend or over night), would consistently delimit the wallet’s future blockspace obligation for most wallets, not tie up undue liquidity, gives each UTXO a chance to be picked organically before considering them for consolidation, and be fairly cost-efficient. Only consolidating UTXOs older than two weeks also usually prevents overconsolidation and helps maintain a healthy minimum UTXO count. Naturally, this process would pause whenever the feerates exceed the chosen consolidation feerate, but since the wallet consistently compact’s its UTXO pool, the wallet should be well-prepared for a feerate peak.
If the wallet becomes too fragmented such that withdrawal transactions take many inputs, it may be useful to additionally have a process that automatically produces a consolidation transaction with a slightly higher feerate that picks the largest amounts from the eligible UTXOs instead of the oldest.
If the system registers a persistent growth of the UTXO count, e.g. by the total UTXO count in the wallet exceeding a multiple of the minimum count of useful UTXOs, the wallet should automatically generate a consolidation transaction.
Assuming that the wallet is below its upper bound for funding, the wallet could just pay to itself otherwise it could sweep funds into the cold storage wallet. The consolidation should not go to the withdrawal wallet, since it would make it harder to keep the available liquidity within bounds, and stocking up the withdrawal wallet after some big withdrawals can be an urgent matter.
Corollary Remarks

To minimize the future blockspace of consolidating these UTXOs, deposits should be received to an output type that minimizes the input weight. At the time of writing this is P2TR.

This is a bit raw and unedited, open for improvement suggestions


text:
How to properly monitor incoming transactions for a specific address with bitcoin-cli?
I would like to monitor incoming transactions for a single address using
bitcoin-cli listreceivedbyaddress 1 false false <myaddress>

My problem is the returned array will become longer and longer and I cannot output the last 10 incoming transactions using the arguments. I don't want to rely on any online service, rather I want to use the CLI.
Can this be done?

You could use the -walletnotify startup option for bitcoind/bitcoin-qt. This will execute a program whenever the wallet receives a transaction. So you can write a program which is executed for every transaction and looks for ones which have outputs for the addresses you want. Then this program can do whatever to notify you.

I'm glad for your answer. I was hoping I could give an additional argument to -walletnotify for inline commands, but it seems I have to modify the conf to contain a call to a script. I.e. walletnotify=/home/scripts/transaction.sh %s. Apparently, the flag only activates the feature. Your answer helped me to get on the right track, though! :)


text:
sendrawtransaction RPC format across core versions
Has there been any changes across core versions for the sendrawtransaction interface. I am trying to understand if upgrading the core version from say 12 to 15 needs a change in serialization. I'd also like to know if there are any versions across which such upgrade would fail.

sendrawtransaction takes transactions in the network serialization format. There is a new network serialization format for segwit transactions, so such transactions would be broadcast in one format for 0.12 and a different one in 0.15. If you are not making segwit transactions, then sendrawtransaction has not changed and you can use it in 0.15 as you did with 0.12.

@achow101 :  Is it possible to know across which versions was the serialization changed in the past as well ?  I assume this applies to all clients across the board ( Unlimtied , ABC etc). Thanks!

You can read the release notes of each release. The serialization format will only change when a fork happens (both hard or soft), but not every time there is a fork. The only time the serialization format has changed is between 0.12 and 0.13 with the introduction of Segwit. The format has only changed to allow a new format; the original format is still supported by all versions.


text:
Merkle tree structure inside blocks - what are they for?
I'd like to check if my understanding is correct. This is regarding the purpose of having the Merkle tree structure for all transactions inside a block. Is the only reason it exists so that thin clients can verify that a transaction exists in a block without being told about all the other transactions of that block?
Seems quite complicated to implement that (instead of just concatenation of transactions) for a rather special case. Particularly since the maximum size of a block is not very large - so there isn't really a problem with verifying all the transactions and ensuring that the hash of the block matches up to those.

There is no merkle tree structure inside of blocks. Transactions are just concatenated with each other. The order of concatenation does matter though. It is necessary to verify the merkle root, which is the root node of a merkle tree. The merkle tree is not encoded in the block, rather it is generated on-the-fly using the data in a block in order to verify that the provided merkle root is correct. SPV wallets will often request for a transaction and proof that that transaction is in the block, so a node will provide part of the merkle tree to prove that the transaction is in the block.

That particular use case is significantly less narrow than the use case suggested by the question and an excellent justification for using a Merkle tree.


text:
How is the hash target set and verified by all nodes?
As in the title - I understand that nodes look at the last two weeks of calculated hashes and adjust the difficulty to bring the average time to 10 minutes. But how exactly is this done?
In particular, all the nodes would have to agree on the same target, else a newly mined block would be valid for some nodes and not others. This seems like a problem of consensus (in the traditional sense, not the Bitcoin way) which is very difficult, isn't it?

The difficulty adjustment algorithm is as follows (from the source code):
unsigned int CalculateNextWorkRequired(const CBlockIndex* pindexLast, int64_t nFirstBlockTime, const Consensus::Params& params)
{
    if (params.fPowNoRetargeting)
        return pindexLast->nBits;

    // Limit adjustment step
    int64_t nActualTimespan = pindexLast->GetBlockTime() - nFirstBlockTime;
    if (nActualTimespan < params.nPowTargetTimespan/4)
        nActualTimespan = params.nPowTargetTimespan/4;
    if (nActualTimespan > params.nPowTargetTimespan*4)
        nActualTimespan = params.nPowTargetTimespan*4;

    // Retarget
    const arith_uint256 bnPowLimit = UintToArith256(params.powLimit);
    arith_uint256 bnNew;
    bnNew.SetCompact(pindexLast->nBits);
    bnNew *= nActualTimespan;
    bnNew /= params.nPowTargetTimespan;

    if (bnNew > bnPowLimit)
        bnNew = bnPowLimit;

    return bnNew.GetCompact();
}

The adjustment is only run every 2016 block. What it does is it takes the timestamp of the block at the beginning of the last period (2016 blocks prior) and the timestamp of the block that came before the one with the adjusted difficulty. Those timestamps are then used to calculate the amount of time that passed between the first block in the interval and the last block in the interval. Then that timespan is clamped to be at least 1 quarter of the target timespan and at most 4 times the target timespan. To calculate the new difficulty for this interval, that target is multiplied by the actual amount of time that passed, and divided by the targeted amount of time.
Since all nodes are calculating the target at the same block and with the same previous data, the target calculation is deterministic and everyone will land on the same target.


text:
Why is the block reward slightly higher than what it should be?
https://blockchain.info/block-index/143685
The block reward here is 50.04 BTC instead of 50. Why is this so? I notice many blocks have this slight excess.

The coinbase transaction(the transaction that pays miner bitcoins) gets bitcoins from two sources.

Block reward: This part goes on declining like 50/25/12.5 and so-on in a pre-defined schedule
transaction fees: This part is total sum of all transaction fees of all transactions in the that particular block. This is determined by market conditions.

Miner Income =  Block-reward + Transaction fees

Ah I see - so the transaction fees are considered newly generated coins? In other words, they are not considered as outputs from the transaction itself?

Yes, that is correct. There are no special outputs for fees. Fees is the inplicit difference between sum of inputs and outputs.

Transaction fees are not counted as newly generated coin. Here new is 50 but 04 is your total transaction fees. As a total reward it is 50.04 but newly generated only 50.


text:
Can two different nonces and different blocks produce the same hash?
Say, I want to erase the record of a transaction by changing a block. Can I go back to that block and find a new block and nonce combination that makes the same hash?
Is it a case that finding such a new combination is computationally hard or is it actually impossible?

One of the main properties of a hash algorithm is collision resistance. If you would find an identical hash with two different datasets, then we would speak of a hash collision.
This is extremely unlikely to happen. The mathematical chance that a sha256 collision would happen is smaller then the actual chance that the universe will implode in this moment.
https://crypto.stackexchange.com/questions/24732/probability-of-sha256-collisions-for-certain-amount-of-hashed-values

Thank you for your reply. I understand that one of the basic principle of a hash is that it should be infeasible to find two messages that give the same hash.
Am I then correct in saying that it is computationally feasible to find a nonce that fits the requirements of the hash in bitcoin but it is computationally not feasible to find a message nonce combination that fits a given hash?

Also, is there a chance that a given message has no possible nonce exists that produces the right hash value or is it only a matter of time before such a nonce is found?

The collision resistance result in the behavior that you won't find a nonce that would result in the same hash when changing other data in the block (example: remove a tx). The hash of a block does always go over the complete header (not only the nonce). The assumption is hash(X) != hash(Y) regardless if X=none or X=nonce||moredata.


text:
Bitcoind block chain structure
I am trying to understand the structure of the underlying block chain stored in the core implementation (or rather just the BLK files used by a fullnode). My question is ; does the block chain store the complete tree structure , i.e all the forks which were encountered? Or does it purge the entries once a fork is resolved.
If i were to read the blk files sequentially , can i assume that every single block read is correctly ordered and no forks are present?
Thanks!

the BLK files are not an API. You shouldn't directly read those files.
What you should do is requesting the headers/block via the JSON RPC interface (or the REST interface).
You can get the genesis block hash via:
bitcoin-cli getblockhash 0

Then you can get a block (without transactions):
bitcoin-cli getblock <hash>

Then get the next block by looking at the nextblockhash item in the block JSON response, etc.
Also, consider using the REST interface. There you can get binary responses which are much faster (no JSON encoding/decoding overhead).

To answer the question more directly: if you read the blk files directly, you would find the chain's blocks out of order, potentially with gaps in between, including forked off blocks, and the format may change in future versions.

@PieterWuille many thanks for the clarification.

@PieterWuille you'r wrong. While reading blk files with some math you can find order of blockchain exclude of any gaps or forks. Perhaps.

I wrote that code in Core. Yes, you can scan for all blocks and afterwards figure out the longest chain, and filter out garbage. But that's not what the question was about. On disk, the blocks are out of order.


text:
Buying a Gridseed Mini ASIC Miner -Beginner
I'm new to the field of cryptocurrencies.It seems that without good hardware i cannot earn anything.As a starter - is  Gridseed Mini ASIC Miner a good choice to buy http://www.makeuseof.com/tag/gridseed-mini-asic-miner-review-and-giveaway/
Will it cost too much electricity? Can i make something out of it?
Please advice

Do you really think that the "device for printing money for everybody" exists?

No. It's old, slow and obsolete. Buy modern mining hardware instead.

I got one of these pretty cheap last year just to play around with mining, and it's neat looking and wasn't difficult to get working once I found the right software, a custom fork of cgminer I think it was. It's too bad I burned it up when I took it apart and put it back together wrong. Anyway, you're never going to make any money on it mining bitcoin because it is truly obsolete, but it can also mine litecoin and it's more efficient at that, so depending on the exchange rate you might make a few pennies that way.
So if your object is to have fun, it's a neat toy and you'll learn a bit about mining and be able to say you've done it. On the other hand, if you're looking to make money or contribute to the network, you'll need something more modern.


text:
Easy to Mine Cryptocurrency
I'm new to this field.It seems that mining Bitcoins using a normal computer has become tough.I have got a normal laptop with Intel i5 processor and GeForce 930MX graphics card.Which is the easiest coin to mine and get some real money(how less does not matter)
Just wanna try and see how it works.
Out of curiosity.
Thanks

I managed to get a couple of mBTC from mining on a GTX 745 using things like NiceHash and ZPool, which can both pay you in BTC for mining altcoins. If you have Windows, the profit switcher on NiceHash works quite well. It's not like you're going to get rich from this though, but if you're just doing it to see how it works, try that.

@bcsb1001 Thanks for the reply.It seems I need to use cloud mining and purchase mining power... Right?

No... what I did was mined on my own PC. I probably lost more in electricity than I earned, but I did mine a few mBTC. See this for a profit-switching miner on Windows.

@bcsb1001 Thanks.. Will check it out.Did you get your BTC exchanged for currency ?

Create your own coin, do a Ninja launch, know how to point appropriate pre-booked hash at low rates, do exceptional marketing, make lot's of false promises about a road map, ensure all the pools that mine altcoins pick up for the launch (pre-negotiated). Have a roadmap for development filled with false promises and tons of nonsense, a gimick can help.
Launch your nodes; launch, mine the majority stake of the supply, then manipulate the price using multiple accounts on the exchanges using automated trading software, creating tons of volume thus drawing attention.
Cycle this endlessly, up and down, and basically milk every newbie who enters the game on how it works... You only learn the hard way.

Which is the easiest coin to mine

Bitcoin testnet-in-a-box. Just connect two instances of client to each other.

and get some real money(how less does not matter)

is "zero money" acceptable answer?


text:
RTX 2070 and GTX 1660 super in same rig with 1000W Corsair power supply - rig shuts down
I just got my computer back from the shop. I have an MSI RTX 2070 and a GTX 1660 Super in my machine. I swapped out the power supply for a 1000W PSU. Whenever I run ethminer for more than 10 minutes my machine shuts down. I had a technician install the parts because my physical handicap makes me susceptible to damaging the hardware.
I set both of the cards fans to 100% in Afterburner, as well as giving both cards maximum power (electricity is free for me so I don't care how much power I use). Still, regardless of my afterburner settings, my machine shuts down.
I'm pretty sure I have the correct drivers. I didn't give the shop my Windows 10 password so I had to download nVidia's drivers myself. What I don't get is I am maxing out the fan and power settings, so I honestly have no idea why my machine shuts down.
When I just browse the web and do basic computer stuff, everything's fine. However, whenever I run ethminer that's where my problem arises. I'm not a gamer, but I have a few installed whenever I do feel like playing computer games, However, I'm too scared to run ethminer or games because I don't want to damage my computer.
I'm still fairly new to crypto mining so forgive me if this sounds stupid. Thanks a bunch!

I think I figured it out guys.
I removed the Plexiglas off my case because I suspected my rig was overheating. No shutdowns so far, looks like I have to investigate water cooling