text:
How does solving math problems create bitcoins?
If I understand correctly, Bitcoin mining software "creates" bitcoins by solving complex mathematical equations. Is there a need for these problems to be solved, is there a value to solving these equations? If not, where does the solving of the math problems fit into the whole Bitcoin scenario?

The problem that mining solves is the problem of providing secure transactions without a central authority. There is value to solving these problems because otherwise, there would be no way to securely exchange Bitcoins.
Bitcoin uses proof of work as its means of solving the double spend problem without a central authority. If I try to send the same Bitcoin to two different accounts, the miners choose one transaction or the other and use it as the basis of the computations they do. Because nobody else can outcompute all the Bitcoin miners, this makes it possible for others to determine which transaction "wins". Without this mechanism, there would be no way to resolve conflicting transactions.
Because this proof of work costs money to do, it's not likely that it could be accomplished without some incentive. Mining rewards provide this incentive.

Miners are essentially putting a notary stamp on a batch of transactions.  That's all they are needed for.
But how do you prevent from having a corrupt notary?  Bitcoin does this by having tens of thousands of potential notaries and one of them will happen to be the lucky one that gets to do the stamp.   The lucky one is the one who happens to solve the problem.   All the potential notaries try to solve the puzzle over and over, but it will take about ten minutes for one to become successful.
Others will see the solution, and if it is valid, will continue on to the next puzzle.   This builds a chain of batches.
So to go back in history to change the batches of transactions requires having to redo all the problem solving attempts that occurred previously.  The further back you want to go, the more problem solving work you need to perform.
So the problem being solved is a system that makes it enormously expensive to change history.  That's the job that miners are paid to provide.

No, no-one is paying for the solutions to the math problems. In fact, they are only math problems in the same sense that watching a Youtube video is doing a computation.
Miners are checking transactions of other users. The Bitcoin protocol is built in such a way that this process sometimes creates new bitcoins. The creation of new bitcoins is also regulated by the protocol - only 21 million can be created, and the last Bitcoin will be created around 2140. The creation of coins is distinct from the value of the coins, which come from their usefulness, supply and demand.
There is another cryptocurrency, Primecoin (XPM), which uses calculation of prime numbers in the "proof of work" process of the currency. These prime numbers do have potential scientific value, but the value of Primecoin is not linked to the value of the prime numbers. The prime numbers are publicly available, and value again comes from supply, demand and usefulness. The goal is that the computation involved in "proof of work", which must be done anyway, should have a side-benefit.

There are two mistakes in your answer:

There will be only 21M Bitcoin
The last Bitcoin will be created whenever block #6,929,999 is found. This would be 2140 if the network actually found one block every ten minutes, but our average time is much quicker, we will be way ahead schedule.

Mining essentially tries to brute force decrypt an encryption. This is not so much a complex mathematical problem as rather systematically trying a multitude of potential solutions until one fits the current prerequisites (context and difficulty).
The purpose of mining is to validate the transactions of the network. The more computational effort is applied to mining, the more resilient is the network against attacks. Therefore the Bitcoin protocol has been designed to dispense rewards among the miners to incentivize behavior that is beneficial to the network – whoever finds the winning solution is allowed to transmit a number of newly created coins to an address of his choice.


text:
What if my Internet Service Provider decides not to let me send bitcoins?
Since bitcoins are sent through the internet, what would happen if my Internet Service Provider prevented me from sending bitcoins?  If they can do that, does that mean that I don't really control my bitcoins?

Since bitcoins are sent through the internet, what would happen if my Internet Service Provider prevented me from sending bitcoins?

There really is no practical technical way to do this, but they could just have a rule.

If they can do that, does that mean that I don't really control my bitcoins?

No, since you choose your ISP.

You can write Bitcoin transaction data on USB stick using your private key.
Then you go to a Internet provider or country where they do not block connecting to Bitcoin nodes.

Indeed. You can also set up a server elsewhere or ask a friend to help you, such that you tunnel your Bitcoin transactions over another protocol like HTTPS.

Easy! Use Tor, or any VPN service. Problem solved! The ISP can't monitor encrypted traffic.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

authors: สุภชัย บัวฉาย

sort: votes


text:
Is Bitcoin a corporation?
Does Bitcoin have legal status as a corporation? If so, what kind, and in what jurisdiction(s) is it registered?

No. The whole point of Bitcoin is that it is decentralized and there is no single corporation behind it.

So there's no liability protection or "conventional" official benefits from corporate structures?

@corporeal: Correct.

I'd say it's not so much that there is no liability protection, but rather that the concept doesn't apply. "Bitcoin" can't be liable for anything, so there's no liability to protect from. Bitcoin can't have revenues or expenses, either. People using Bitcoin might have liabilities, assets, P&L, but it's up to them to decide which legal entity to use for these.

So, if Bitcoin value suddenly tanks, each user is liable for their own losses independently?

@corporeal: Yes, certainly.  Your question is kind of like asking if copper is a corporation.  If the market price of copper drops, everyone who owns supplies of copper will lose value.  No corporation is responsible for this.

Thanks @NateEldredge that clears it up for me pretty much perfectly!

You may have a claim against whoever sold you the bitcoin, depending on the regulations and jurisdiction.

The Internet might be a better comparison. If the Internet disappears tomorrow, you might have actions against particular people or companies for particular promises they made to you. But you can't have an agreement with "the Internet". You can't use the Internet.

@DavidSchwartz: Should that have been "you can't sue the internet"? Because I use the internet just fine.

@MeniRosenfeld Yes. Too bad you can't edit comments after five minutes. :(

The debate of bitcoin in the legal system is just in the beginning. It's not even that clear (worldwide) whether bitcoin is legally consider money or not. Up to now, most of bitcoin space is unregulated (including the issue of corporation status).
But this doesn't mean that it will be unregulated forever. Nobody knows what the future holds, maybe one day the government might decide to debate whether miners should be responsible for bitcoin issues (as one day happened with Napster, Torrent and several other internet sectors).

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Can i combine 2 devices to get more power for mining
As An example lets say i have 2 USB Miners each one has 16 GH/s
can i combine these 2 to mine with 32 GH/s ?
And if yes pls tell me how.

Yes. To do this simply mine with both at the same time. A mining pool will never give out the same work twice. So mining with two 16 GH/s devices will be the same as one 32 GH/s device.
If you mine at the same mining pool with both then the pool will show you mining at 32 GH/s.

The answer is yes for USB based mining H/W hosted on the same computer. The bfgminer and cgminer mining applications can easily support multiple USB mining devices on the same computer.
However, the answer is no if you are mining from home behind a NAT firewall and are trying to point two or more different IPv4 based miners to the same mining pool server supporting Stratum (STM) or GetBlockTemplate (GBT) protocols. This is a NAT firewall issue.

Mining with multiple devices behind NAT works fine


text:
Confidential Transactions on main chain
I have seen discussion of CT being used on the a side chain. What are the chances it will be applied to Bitcoin Core itself?

Pretty good actually as it has become a lot easier after SegWit allowed for new script opcodes through soft forks. It might still be a few years out, as there are other priorities, but Gregory Maxwell is pretty active on CT.

This is not a question anyone can answer. If you'd asked me 2 years ago what the chances of raising the block size in the future were, I'd probably have said that they were pretty good.
I think we can all definitely agree that CT is unlikely to be merged into Bitcoin Core itself any time soon, if at all, as it would require major wallet changes across the board, and would be a massive hard fork...with all the push-back that goes along with that.

SegWit allows CT to be done through soft fork. Wallet support would be opt in, I think. Also block size increase is going to happen within weeks now, so you were right 2 years ago.

If it's not enforced on a protocol level it's not even worth discussing. Then it's about as useless as centralised mixers, marred by exceedingly low entropy and full of central failure points. As to the block size, where do you get that there's going to be a hard-fork increase within weeks? Segwit doesn't count.

Saying it's useless doesn't make any sense. First of all it would be immensely easier and faster than centralised mixers and trustless. 2nd even if a small but steadily growing number of people start using CT then that's an improvement for the people involved and even others. That's the only way to transition to a mostly CT situation in a backwards compatible manner anyway. Next: i never mentioned hard fork in 2 weeks, i meant segwit which perfectly counts as it literally increases the block size. But this is not a discussion site, so don't bother please.

Don't bother with the blocksize debate, is what i meant there. Valid points on the topic of this question+answer are welcome of course.


text:
I spilled a little bit of candle wax on my Bitcoin private key and now it won't scan electronically what can I do?
I need help. What can I do to still use my paper key?

You can make a manual copy of the QR code. Get out a sheet of graph paper, a #2 pencil, and start copying the grid from the QR code. Count the number of rows, and make sure your graph paper is large enough. Each square in the QR code should be either entirely filled in or entirely blank. As long as each square in the QR code is legible to you, you can make a clean version.
You don't need to get every square correct - QR codes have an error correction code.

If the string of numbers and letters that should also be printed on the paper wallet (usually below the QR code) is legible, use that to import your wallet. It is likely prepended with "Private key: " and starts with a "5".

Else:

Try googling "Remove wax from paper"
Read instructions carefully for plausibility
Follow instructions.

You might be able to soak up the wax by covering it with baking paper or newspaper and carefully ironing it. I have not tried this with printed documents before, use this idea at your own risk.

Make some copies before trying to do anything to the paper. Is there not also a written code in letters and digits on the paper?

@Jannes Perhaps OP spilled wax on that part too.

@NickODell Still important to know as part of the key might still be readable and be used in some brute forcing (depending on how many digits are missing). Same with QR code. And with some trickery combine the two to fill missing digits using the other.


text:
Why was BIP34 (Block v2, Height in Coinbase) not implemented via the coinbase tx's locktime or nSequence?
At some point in 2011/12 miners started using custom mining algorithms ignoring the best practise to mine to a different new public key (hash) for each subsequent block. This lead to a lack of uniqueness of the coinbase transaction id and destroyed some bitcoins forever.
So it became mandatory with BIP34/v2 blocks to put the height of a block in a special encoded format into the coinbase transaction's input "script".
But why not use existing datastructures like the coinbase transaction's very own locktime field?

That would have made perfect sense, and I see no reason why that wouldn't have been the superior way of doing it.
However, as far as I remember, nobody suggested this at the time.

The nLockTime value can't be set to the height of the block it's included in. This is because the locktime check (IsFinalTx()) is performed for all transactions in a block, including the coinbase transaction. And the nLockTime value represents the last block at which a transaction is invalid, not the first block at which it is valid.
Of course this can be dealt with, for instance by using height - 1 as the nLockTime value or by requiring that the nSequence field of the coinbase transaction's input be set to the maximum value. However this added complexity might have played a role in the decision to not use this method at the time.

Interesting, I don't think I considered this when writing my answer here.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Does mining with multiple computers add up?
Let's say I have four computers, each mining bitcoins at 5 Mhash/s. Is this the same as mining bitcoins on one computer with a speed of 20 Mhash/s?

Yes, the hashrate adds up.
But GPU mining ended in 2013. The tiny bitdust you will earn will never reach a mining pool's minimum cash out limit. So you will never see any bitcoin. This will be a pure loss consisting of spent electricity and worn-out hardware.
You could buy a USB stick for mining, for 10 USD. It would mine at a loss due to electricity costs. Even if you have free electricity it would never earn back the 10 USD. So this would also be a loss.
The only way to mine now is with modern stand-alone miners. They have a much higher hashrate and efficiency and they use a tiny built-in computer instead of connecting to your PC. You'll want to get one that runs 1 TH/s (1000 GH/s) or faster. You will not want to pay more than 0.30 USD per GH/s for a miner with power supply. You will need cheap electricity.

Yes, and that'll net you around 0.01 USD worth of BTC per year (as of January 2016), assuming your power is free.

Alternately, you can sell your keyboards, and that'll net you more money. You can still use a computer with just a mouse.

.01 seems generous for 20 Mhash/sec

Yes. More generally though:
x mining machines with an average hashrate of y hashes per second will have the same chance of finding an arbitrarily low blockhash in a period of time as one mining machine with a hashrate of xy hashes per second in the same length of time.


text:
Determine which transaction output is a change address and which is not
Is there a way to determine which transaction output is a likely a change address and which is not?
Looking at this transaction, there are two outputs - a change address and the output address. Blockchain.info says "Estimated BTC transacted: 0.5 BTC", but I'm not sure how it's estimating that.

This is usually (intentionally) not possible.
Well designed wallets attempt to obfuscate the change vs. non-change addresses to make it harder to find out which part is the payment for privacy reasons.
Traditionally, some clues existed, which are now fixed in any decent wallet:

The change would go to an address of the same type as the input - This was an issue when P2SH was new, and when segwit just started out. These days, wallets such as Bitcoin Core will try and match the change output to a similar address type as your regular output. That is to say, if you use a native-segwit address to send BTC to a P2SH address, Bitcoin Core will send the change output to a P2SH address as well, even though your wallet is predominantly native segwit addresses
Change outputs often appeared last in the list of outputs - This allows people to guess at the change address by simply looking at the output ordering. Today, many wallets implement BIP 69, which orders outputs lexicographically, thus removing any guarantee that the change address will be the last output.

It should be noted that the above are not adopted by every wallet. In fact, many systems, such as several exchanges' withdrawal systems, will still send the change to the last output, and also to an address of the same type as the input. That said, there are currently enough wallets that do follow some/all of the above to make change detection unreliable without additional data (such as knowing if some input was spent by a specific wallet program, and how that wallet would behave)

You can guess at which output is a change address, but the only way to be certain is if you are the person initiating the transaction.
In fact, some wallet software could be designed to intentionally obfuscate this data, to improve user privacy. At best, services like blockchain.info can only guess.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Does Bytecoin use SHA-256?
Also, doesn't this mean that I can use hardware such as a bitcoin usb block erupter to mine it?

Bytecoin (BCN) bytecoin.org uses the CryptoNight algorithm and cannot be mined with bitcoin mining hardware.
Bytecoin (BTE) bytecoin.biz uses SHA-256 and it can be mined with a block erupter (and any other bitcoin mining hardware).


text:
Is there dedicated mining hardware available to mine cryptonote coins?
I'm talking about hardware such as USB miners, etc. I'm asking, because I see people mine with 86kH/s on my pool…

I've removed the bit where you ask "what the best mining hardware is", because we don't do product recommendations here as they would outdate quickly anyway.

From here

CryptoNight is a proof-of-work algorithm that mixes graphics processing unit (GPU) and central processing unit (CPU) mining to create a system resistant to both application-specific integrated circuits (ASICs) and fast memory-on-chip devices.

So, there is no dedicated ASIC hardware (yet). You can currently only use CPU plus GPU (which of course counts as hardware but I assumed you meant dedicated to mining hardware like ASICs).


text:
How profitable was GPU mining in 2011-12?
I am not interested in mining myself, I just want to know how true my friend's claims are since Bitcoin is a subject that I try to follow and understand.
He says he mined 31 bitcoins in 7 months using 2 gtx 460 in a regular gaming PC. This supposedly happened around 2011-2012. I told him this sounded like too much to me and he says it was possible because he was in a "hard to access" mining pool.
So, I just want to know if these numbers are remotely possible. My limited knowledge and the browsing I have done says it's too much but, of course, I could be wrong.
What do you guys think? Thanks.

Question in title: similar to how profitable it is now.
The numbers he is giving you don't seem unlikely. Using the Bitcoin price of those days he probably could have gotten more Bitcoins than 31 by simply straight up buying them instead of the graphics cards. And if he accurately accounts for the electricity costs it was probably not profitable at all.
His luck (and the reason you are asking this question now) came when the price of Bitcoin went up.

because he was in a "hard to access" mining pool.

That sounds like a bit of a misunderstanding tbh.

This answer could be improved by explicitly addressing the plausibility of mining a number of bitcoins with GPU mining then. The value of 31 bitcoins then doesn't seem to be the central issue of the askers question. (Although I may have made it look that way, by editing the title.)

@NickODell I said the opposite of implausible: not unlikely.

@Murch sounded to me the OP didn't believe his friend statement, as 31 BTC sounds like a lot of money today. Back then however 31 BTC wasn't that much and probably worth less than the 2 GPUs+electricity used. And therefore it sounds very plausible his friend is talking the truth.


text:
How can I find out where to buy Ripple?
Hi I just heard about a new alt coin called "Ripple". How can I find out where to buy Ripple?

Ripple has been around for years.

Ripple is actually not really new but still an interesting coin.
Their website lists several possible exchanges (https://ripple.com/xrp-portal/how-to-buy-xrp/). I have no strong opinion about which of these is most reliable. There are others (https://poloniex.com for example) where you can trade XRP/BTC, too, so if you already have an account at an exchange that you trust that might be the easiest way.


text:
Re-generate a past private key
I know that every public key has a private key... I am wondering how to regenerate a private key that was generated by my Blockchain app (using a plain text passkey, which I am assuming was converted into the proper format by the app)
I have the wallet address where the bitcoins were sent, but cannot use the plain text private key I used in the application to spend the bitcoins...
The bitcoins have been sitting at this address since 2015, And I guess I was hoping if there is a way to convert the passphrase I used in the app into the proper private key to access the bitcoins at the address they were sent to...
I hope that this makes sense.... I can see them there, and not being able to access them is so frustrating! I lost the data in the application, I think due to a glitch. I was hoping since it is a mathematics thing to generate the private key that there must be a way to go backwards, using the public address and the plain text passkey I had used to find the proper format private key.

I've updated the tag to blockchain.info, as I assume that's the service provider that you're referring to.

There's no way to go backwards from the public key to the private key, that would mean that anyone could steal your Bitcoin.
You might be able to do something with that passkey you have saved though. Is it a list of words? Do you remember what app you were using?
Edit: Looks like you actually have a passkey that was used to generate a private key and not a Mnemonic backup. Have you tried putting the private key into the 'wallet details' section of this website: https://www.bitaddress.org/ ?
Note that you can use the site offline and I recommend it.

Thank you I will try it, I was using the Blockchain application on Android.
When I bought some bitcoins I created a new address for them to be sent to and was prompted with a "Please type a passkey for this address" type thing, in which I typed just a short 11-16 character pass phrase which I was assuming would be used to generate a private key for that address... Now I only have the wallet address where they were sent and the passkey I set doesn't work to spend the coins, I assume it is the wrong format... I will try this and let you know, thank you!

Take a look at what's the steps to generate a private key here.
If your passkey is the raw 256 bit sequences or any intermediate results, you may be able to generate the end private key by following those steps, assuming the app uses the same methods. That's the only way to you can generate a private key again. There is no way to go back from public key to private key. Hope this helps and you can get back your coins!

I'm not sure what raw 256 bit is... sorry! When I created the address for the bitcoins to be sent to I was prompted to enter a "passkey" which I assume was used to generate the proper private key, in which ever format that may be... If I knew I may be able to look up if there is a way to convert a simple 11-16 character passphrase into the actual import format needed to access the coins at this address... Sorry my understanding of Bitcoin is a bit limited so I am not sure what to look for to try and convert the passkey into the right format. Thank you for your help!

Hmm, you should ask someone who you can trust and have knowledge of page I posted to take a look at your passkey. Maybe you can contact the app owner to see what that passkey is. Maybe it's a form of retrieving actual private key for you.


text:
Could Bitcoins (or any other blockchain based cryptocurrency) feasibly ever be used for purchases that need to go through fast?
Today, one of the problems with cryptocurrencies like Bitcoin is that it takes quite some time for a transfer to go through and enough confirmations to take place. This really isn't a problem when it comes to inherently slow buying procedures, such as for example ordering something online and receiving it a couple of days later by mail. However, other types of purchases requires a much faster process (for example if you're buying a digital good and want it immediately or if you're buying a physical good and want to leave the store with it right away), and as it is now, cryptocurrencies don't seem suitable for these as it simply takes to long time to confirm the payment and defend against double-spending.
Is this a problem that is inherent with the Blockchain technology, or could purchases go through much, much faster (at least so that they can compete with credit card transfers) given enough resources or faster computers in the future?

Blockchain technology, at least without accepting central party that controls access to the chain, inherently has slow blocks, or strong centralization incentives.
The reason is that blocks need to propagate much faster than the block interval time. If they don't, miners who are further away from the majority of the hash power are put at a disadvantage.
To explain this intuitively, notice that when miner A creates a block, he can start working immediately on a successor, while another miner B first needs to wait until A's block reaches him. During that time, B works on an outdated chain, and is unlikely to produce a block that ends up being accepted. The closer A and B are located together, and the more trust they have in eachother about not lying about new blocks they produce, the more they start to behave like a single party.
Thus, the ratio of block interval and propagation speed across the network is a means for keeping the advantages of larger miner groups over smaller ones in check.
This does not mean that the bitcoin currency is inappropriate for fast payments. They just require another mechanism than on-blockchain transactions. For example, look at the proposed Lightning network, which allows many small transaction between parties to occur, while only settling the resulting balance on the blockchain. It requires more trust, but has higher performance and convenience, and lower costs.

If faster computers had started producing blocks at higher rate the algorithm would just increase block difficulty. It is intentionally designed to give next block (and therefore next confirmation) at average once per ten minutes.
But it does not mean the money needs an hour to be transferred. The transaction is emitted immediately and the fact of sending money may be detected by receiver in seconds. Confirmations are required just to make sure the transaction is irreversible for the sender. And there are payment processors (like Coinbase) which accept payment just after the transaction is detected (i.e. without waiting for confirmation). Of course it is not suitable method for high value transactions (or if service is highly abused for some reason), but it does work (as making double spend requires some knowledge).
On the other hand if you need payments confirmed in shorter time you can use other cryptocurrencies. For instance DogeCoin block time is one minute (and after 10 minutes you have 10 confirmations making the transaction quite irreversible). There are also some altcoins with shorter times (e.g. 40 seconds for FlorinCoin) but they are not as popular as BTC or DOGE.

The last paragraph is a common misconception - "more confirmations" doesn't inherently mean "more irreversibility".  One Dogecoin confirmation represents only one-tenth the work (and hence one-tenth the irreversibility) of a Bitcoin confirmation, relative to the total hashing power of their respective networks.  Getting more security in less time is not as simple as choosing a shorter target block time.  See also this discussion, and this one.

@NateEldredge But there is a pretty big difference between the irreversibility of a transaction with 0 confirmations, and a transaction with 1 confirmation.

@NickODell: True, sort of.  But in the limit, as the target block time becomes small, tending to zero, this difference approaches zero as well.

@NateEldredge Theoretically, yes, but miners have different rules for whether to include transactions into their blocks, and it ends up being difficult to predict what percentage of miners will try to include it.

There's no inherent reason why it couldn't be done that way, but then there's no inherent reason why you couldn't charge an electric car wirelessly.
The present time between blocks represents a compromise: set the block time too high, and you'll inconvenience users. Set it too low, and blocks take too long to propagate around the network, which means that nodes won't mine efficiently. The existing block time is probably too long.
It's not so much that it couldn't be done, it's just that there are a lot of stakeholders with a lot of different ideas about what the block interval should be, and this requires agreement from a lot of people.
@Jakub Jagiełło discusses the mechanism that adjusts difficulty so that the actual block interval matches the target block interval, but the target block interval could be adjusted deliberately.

"The existing block time is probably too long." Too long for immediate purchases, or too long in general?

Given the fact that miners currently find it profitable to not fully validate blocks they learn about from others, I would say that the current time is even too fast.

@Murch Way too long for immediate, in-store purchases, but I don't think that's fixable solely by tweaking nTargetTimespan. Too long in general, I think.

I was just trying to point out that the statement is ambiguous in this context. ;)


text:
Reduce Wallet Sync Time
So I am new to Bitcoin and I run the Bitcoin Core on my computer. I know that you need to wait for your wallet to sync before you can make transactions, but it says it won't finish syncing for 4 years.
I'm not sure if this is a problem and if there is a way to fix it, or I just need to suck it up and wait  for it to finish syncing. I am trying to learn how to mine and I know that I can't use a lite wallet (sorry if that's not what it is called) because large transactions will clog it up. Is there a way that I can fix this or should I just use an online wallet? Thanks for your help

The "four years" is a description of how far back in time the current synchronization progress is, it won't literally take four years to complete. For most people it will take under a day to completely sync from cold, and there's not really any way of speeding it up.


text:
Is it necessary for all miner to reset mining when the tip of blockchain has changed?
Each block in the blockchain contains a pointer to the previous block (block header). If multiple blocks reference certain block as their parent, only the one that is part of longest chain is accepted.
Suppose that the hash of the current blockchain tip (last block) is 'A'. Two miner concurrently works on new blocks (containing different transactions) referring to A as a parent. Is it necessary for the slower miner to reset his work when faster miner finishes his block (with hash 'B') and broadcasts it, because the next block has to reference 'B' instead of 'A' as a parent?
If this principle holds, does it mean that slower miners have to restart their work more frequently because they are outrun by faster miners who publish their blocks and so change the blockchain tip?

Slower miners and faster miners have to restart their work just as often -- every time a block is found.

Correct: when block B is broadcast and validated, all miners need to construct new blocks containing a hash of block B's header.
However, it takes only a minuscule amount of computer processing time to construct a new block, so there is effectively no such thing as "restarting work".  Mining is specifically designed to be progress-free: you have the same chance of finding a valid block after a millisecond of work as you do after an hour of work.
Progress-free mining means that a miner with x hash rate should produce n blocks over time, and a miner with 2x hash rate should produce 2n blocks over time---there should be no special advantage to the more powerful miner.


text:
How are unconfirmed transactions divided into different blocks?
According to https://bitcointalk.org/index.php?topic=216938.0 one transaction can only be contained in one block. If I understand correctly, miners can freely decide what transaction they include in their block and what they don't.
Why is it that multiple miners concurrently deciding what block to build on, and what transactions to include, produce blocks without any duplicate transactions? Is there a reservation list of transactions to be confirmed and miners that will include them?

possible duplicate of If a block chain is considered invalid, how about transactions in this block chain?

When a miner (call him Pete) learns that a new block (call it X) has been mined by someone else, he will normally update the header of the block he is trying to mine, so that it lists X as the previous block.  (That way, Pete is now trying to build on top of X, rather than trying to replace it.)  At the same time, Pete will note which transactions were included in block X, and remove them from his "memory pool" of unconfirmed transactions that he was planning to include in the block he was trying to mine.  He will rebuild a Merkle tree with the remaining transactions he wishes to include, and update his block headers to match.
In this way, when Pete does succeed in mining a block, it won't contain any transactions that have been contained in previously mined blocks.

Does this mean that mining pools work most of the time in vain - their work is lost because the lack of planning? I mean suppose there are 4 mining pools. All of them are working on different blocks but all of these blocks contain one same transaction (lets say A). When fastest pool publishes the newly mined block others has to change previous block reference and replace the transaction A. Because data of their blocks has changed they need to search the nonce space from the beginning. So all the work of remaining 3 pool done before block publishing by fastest pool is lost. Is that correct?

@czerny: That is a very common misconception about how mining works. Every nonce has an equal chance of success; a winning nonce could be anywhere. So there is really no progress to be lost. Whether you've already tried a trillion nonces for a particular block or are just starting, your chances of success on the next one are the same. It's like a fresh roll of the dice each time.

@czerny, Just like Nate said, there's not really a progress bar when mining. It's sort of like flipping coins on video. If you get lucky enough, you can show your video to others, and they let you get to pick the next transaction set and get paid. Every other time, you throw out the old video, start filming and try flipping the coins again.

@StephenM347: That's a nice analogy, I've never heard that one before.  I may have to start using it.

Thanks for answers. I asked in separate question about the progress of mining and as far as I understand there is a progress of negligible size. So it is practically like coin flipping but strictly speaking each nonce has a (tiny) bit higher probability to be the proper one.

When a newly discovered block is announced to the network, every node must verify that all the data contained in it is valid. If such a block lists a transaction that was already listed in any ancestor block(*), the node must reject the new block entirely. The block will be ignored and the network waits until someone announces a block that is valid.
(*) Every block contains a reference to its parent block, ensuring that bitcoin's entire transaction history is verifiable by anyone; hence the term block chain.


text:
How is it that concurrent miners do not subvert each other's work?
Every time a new block is added on top of block chain, the miners have to restart their work because the next block has to have a proper reference to previous block.
Let's suppose that there is some nonce for each block such that there is also sufficiently small hash (smaller than target). In general, there are 2256 possible hashes. Let the target be t. The target can be also understand as a number of acceptable hashes. So there is t / 2256 probability to find a proper hash in each try i.e. to find a block.
The number of unsuccessful tries before a block follows a geometrical distribution with parameter p=t/2256. The expected value of a variable following such a distribution is EX = 1/p = 2256/t. So, each mining pool has to spend 2256/t tries in average to find a block.
How is it that concurrent mining can be efficient, providing that each time some pool publishes a new block all other pools have to restart their work and thus throw out their tries on blocks that now can't be used any more?
Note: Please be a bit detailed. I've already read explanations like: Every try has equal chance to success. But I can't get it from such short hints.

Do you not agree that every try has an equal chance to success? Or do you not understand the consequences of that? (Because that is the canonical short answer.)

I did not understand that every try has (exactly) an equal change to success. The discussion below and question How can we be sure that a new block will be found? helped me to clarify that.

The number of hashes a miner has tried in the past does not affect the probability that a miner will get the correct hash in the next immediate calculation. Thus, it does not matter for the miner from an efficiency viewpoint if he starts work on a new block since the probability of getting the correct hash is exactly the same as if he kept working on the old block.
Think of this coin flip example: The first goal is to try to get heads. You failed the first several flips and than the goal changes to getting tails. The probability of you meeting your goal is still exactly the same.

I understand the coin flipping example. However a I see a difference between coin flipping and finding proper nonce. I think that there is finite number of all possible hashes (2^256) and finite number of acceptable hashes t. So after trying (2^256 - t) nonces I can be sure (up to hash collisions) that I met at least one proper hash. Thus although it is not possible to establish a concept of reaching to a block finding (distribution of proper hashes is random), it is possible to say that one is reaching the point when it is sure to find a block. Why is this reasoning incorrect?

Your reasoning is correct, but 2^256 is such a large number (1.15x10^77) that the difference in probability is going to be extremely small considering that the current total network hash rate is around 300 GigaHash / sec (3*10^11).

@czerny, I'm not convinced what you are saying is true, because you're not guaranteed to get a different value every time, and neither are you guaranteed to ever find a nonce that hashes low enough. It's like flipping 8 coins and hoping to get the first 4 be all heads. There are only 2^8 possible outcomes, 2^4 positive outcomes, and having a fail on the previous attempt makes you no more likely to succeed on the next.

@czerny: Actually, while there is a finite number of 2^256 results of the hash, the number of possible inputs for the hash function is infinite. Therefore you cannot be sure that you have found a winner after 2^256 inputs, because you would have hit some hash results multiple times, and some never.

@Murch, right, hence you're no more likely on one try than the next. Although, technically not infinite because the block header is 80 bytes and that's the only thing you're hashing in mining. Version, hashPrevBlock, nBits, and nTime can't really change either. That leaves merkle root (32 bytes) and nonce (4 bytes). So there are really only ~2^288 possible inputs.

In other words: To establish a term reaching (to point where at least one proper nonce is find) it is necessary to have a collision free hash function (which used sha256 is not) and an assurance that these exists a nonce such that block hash is smaller than target (there can't be such insurance). Hence finding a proper hash is exactly like coin flipping with one side of coin lighter. Correct?

@StephenM347 I'm not much convinced about the flipping 8 coins analogy. The probability of success in one toss is p=2^4/2^8. Probability of success if one tries long enough is q=lim n->inf (1-(1-p)^n)=1. So there is always chance to success. However in the block hashing there might be no suitable nonce so there might be cases when there is no change for success.

@czerny: Even if there is no nonce that works, you can switch around the order of the transactions, change the extra nonce, wait one second to get a new time, and so forth, so the space searched is much bigger. Also see: How can we be sure that a new block will be found?

I kind of see what you are saying @czerny, because there are an unlimited number of tries in the coin flipping game, but a finite number of tries (2^(36*8) ish) in mining. So, yes, I see that the analogy is not perfect, but I don't see any reason why that would change the probability of success from trial to trial.

An analogy for mining would be the following:

You are at a lottery booth. The lottery booth has 1,000 lottery tickets in a bowl (which are always mixed perfectly). There is only one ticket with a prize in the bowl. Every time somebody buys one lottery ticket, the booth owner prints a new ticket and adds it to the bowl, replacing a loser with a new losing ticket, and a winner with a winning ticket, so that there is always 1 winning ticket and 999 losing tickets in the bowl.
Whenever somebody buys a ticket, chances are 1 in 1,000 that he wins.
You might win with your first ticket, but you might win with your 10,000th.

As you already mentioned, every try at finding a new block has an equal chance of being the winning ticket. That means that there is no such thing as "progress towards the block". It either works or it doesn't.
So now, somebody found a block. If people keep mining on the old block, they would have to find two blocks in a row in order to profit, because they would have to find one to catch up, and another to overtake the current best chain. Rather they will just want to compete for the next block again!
Mining is not supposed to be efficient. Mining is supposed to be inefficient, in that it aims to prove that a sufficient amount of computational work has been expended to protect the network from being subverted.

Does this mean that computing private parallel branch is computationally easier then computing public blokchain because there would be no competing? (I understand that computing private branch in hope that it will become longer than main chain requires improbably large amount of computational power. Just theoretical question.)

Based on @kaykurokawa's comment I would a bit reword the analogy. There are so many bowls that the probability of picking the winning one changes just negligibly by removing some bowls.

@czerny: Well, if you'd want the right numbers, the bowl should have 2^256 tickets, and there'd be target winning tickets among that. You'd then start a new bowl every time somebody wins a bowl.


text:
Is there a way to find how many nodes are there?
As a single bitcoind node I can only detect how many connected peers I have. Is there a way to have a vision of the whole bitcoin network?

There are websites such as Bitnodes.io that track the number of nodes currently reachable in the network. The site also gives an account of the versions the nodes deploy and what countries they are located in.
According to Bitnodes.io, there are currently 6426 nodes reachable.
To collect your own list of nodes, you'd ask your peers for more peers, i.e. getaddr, then recursively request more peers from the newly discovered peers, until you have polled everyone you have discovered.
Note that this doesn't guarantee you to find all nodes, as some don't accept connections from the outside, and getaddr doesn't necessarily give you all peers. Also see: Does getaddr.bitnodes.io find all Bitcoin nodes or only one node per mining pool?

Thank you! How do they get this data by running multiple nodes around the world and converge?

@moshaholo: See my updated answer.

Also note that this is considered spamming and a DOS attack. It goes against the whole idea of P2P and the network implements methods to defend against such activities. The bitnodes people are in close contact with the bitcoin core developers and try to be as non aggressive as possible. Still plenty of nodes ban them anyway.

If I understand the answer correctly, it is about the active nodes. Can we know how many nodes there are in total? Regardless whether they are active or not. Also, does this exist for other cryptocurrencies? And are there historical data for it?

There are other estimates, like for example the data Luke-Jr collects with his seed node: http://luke.dashjr.org/programs/bitcoin/files/charts/software.html Some historical data can be found here: https://coin.dance/nodes Other cryptocurrencies have fewer of these private sites as these spring up with a bigger community.

authors: Сергей Курков


text:
Mining Litecoins as User in Debian
I followed this tutorial https://litecoin.info/Debian_Linux_cryptocoin_mining_rig_setup_guide and set up a Mining rig (Debian 6.0.10). Sometimes I modified it (downloaded later graphics drivers than in the Tutorial sugestet etc) but it works and I can mine on a pool.
But I can mine only if I log in as root. If I log in as a normal user and type cgminer --verbose -T, I get the following error:
 [2015-02-15 22:36:50] Started cgminer 3.7.2
 [2015-02-15 22:36:50] Loaded configuration file /home/martin/.cgminer/cgminer.conf
No protocol specified
No protocol specified
No protocol specified
 [2015-02-15 22:36:50] CL Platform 0 vendor: Advanced Micro Devices, Inc.
 [2015-02-15 22:36:50] CL Platform 0 name: AMD Accelerated Parallel Processing
 [2015-02-15 22:36:50] CL Platform 0 version: OpenCL 2.0 AMD-APP (1642.5)
 [2015-02-15 22:36:50] Error -1: Getting Device IDs (num)
 [2015-02-15 22:36:50] clDevicesNum returned error, no GPUs usable
 [2015-02-15 22:36:50] All devices disabled, cannot mine!

I tried setting all privileges in the folders in root - which are used for litecoin mining - to 777 and to change the owner to the usual user, but it didn't work. I copied the file /root/.cgminer/cgminer.conf to /home/user/.cgminer/cgminer.conf, set the user and the right to this file, but this didn't solve the problem too.
But why can only root access the OpenCL functions which are used, why can not the normal user do so?
Thank you.

You should be able to add your current user to the plugdev group in Linux to be able to access the GPU as a standard user.  This can be accomplished by the following command:

sudo usermod -G plugdev -a whoami

There may be some additional bits of information that are relevant in the README on the cgminer website.

Again, I am not familiar with cgminer but when I attempted to use it, I ran it with sudo cgminer --verbose -T This should work.

I'm not familiar with cgminer -- i hope to learn more about it this year -- but a wild guess would be that it tries to configure some device driver, and that needs root priviledges.
If you are worried about using the root user for mining, maybe you can add this device driver priviledge to a common user using sudo.


text:
Why do we refer to previous transactions in each transaction?
Isn't it enough to just have the Input Address, Output Address, Amount and Signature for each transaction?

We don't refer to previous transactions, but rather to specific outputs of previous transactions. When transactions sign over money to a recipient, they create a uniquely identifiable "Unspent Transaction Output" or UTXO. Transaction inputs in turn spend these "UTXO" to fuel transactions. Since addresses can be reused, addresses alone are insufficient to precisely identify the bitcoins that are being spent.
If you only had "Input Address, Output Address, Amount and Signature for each transaction". This would be trivially broken:
Let's say Alice keeps all her funds in address A1, and pays Bob 0.1BTC in rent to address B1.
Bob pays Mallet to fix Alice's heater from B1, using the funds that he received from Alice.
When Alice pays Bob next month again from address A1 to address B1, Mallet can replay Bob's transaction to himself in order to get paid again.

I suppose the question then becomes, why do you need to precisely identify the bitcoins being spent? Why isn't it sufficient to ensure only that an address doesn't spend more than it has received?

@BT: Because precisely identifying what money you spend makes it much simpler to prevent doublespending.

Could you explain that further? I don't quite see why that's the case.

@Murch So you are saying that it isn't possible to prevent double spending without having previous transaction outputs as a reference?

It is possible, e.g. Ethereum with its account-based design doesn't have UTXOs. However, then the order of transactions needs to be tracked and you need sequence ids. While the UTXO-based design is a bit harder to grok, it's much simpler to implement and more versatile for transaction creation.

@Murch Re versatility: what's an example of something you can do with a UTXO design that you can't do with an account-based design?

@BT: Send a high priority transaction with a high fee while a low priority transaction is still unconfirmed.

Bitcoin doesn't track account balances like traditional accounting, it only keeps a record of where BTC came from and which address it is sent to. So, your receiving address doesn't technically have a recorded balance, only a calculated balance for all the transactions received on that address. Once you send Bitcoin, the unspent transaction output you reference when sending becomes spent and the calculated balance for that address you originally received on becomes zero.
You may find this video of assistance:
How Bitcoin Works Under the Hood


text:
How do SPV (simple payment verification) wallets learn about incoming transactions?
I try to understand a SPV implementation.
I read the wiki pages on Protocol Specification and Thin Client Security as well as the white-paper's chapter 8 .
What I don't understand:
Assume user A has Bitcoin address X.
User A can do getheaders and load all block headers and then have all block hashes and timestamps.
So: how does a SPV client then find possible transactions to the user's Bitcoin address X without loading all blocks (and all transactions)?

You are right that SPV as described in the Bitcoin paper does not explain how to discover payments to yourself, without downloading full blocks. My guess is that Satoshi either planned to develop this later, or that he assumed you'd just be told about payments to yourself (pay-to-IP, as existed next to pay-to-pubkeyhash as is used now).
Practical implementations of SPV nodes today however use a protocol extension called bloom filters, described in BIP 37. They use getheaders to fetch block information prior to their wallet's birth timestamp, and request filtered blocks afterwards. To do so, they submit a bloom filter that describes the addresses and transactions they are interested in to the peer. The peer then only includes the relevant transactions in blocks it submits, together with their Merkle paths to prove that these transactions were indeed part of said block.

If it helps https://gist.github.com/TOMOAKI12345/7e0aa1c6b8ace4a70ca6
Breadwallet source code is really good source to learn about the network protocol of SPV wallet.

Breadwallet source is not available any more.

Just found something in bitcoinj sourcecode:
/**
 * The "getheaders" command is structurally identical to "getblocks", but has different meaning. On receiving this
 * message a Bitcoin node returns matching blocks up to the limit, but without the bodies. It is useful as an
 * optimization: when your wallet does not contain any keys created before a particular time, you don't have to download
 * the bodies for those blocks because you know there are no relevant transactions.
 */

In addition to above answers, You could refer to this diagram if it helps from this link.

I think SPV is only meant to verify transactions, not calculate a balance or list of all transactions to a certain address.

But how does Multibit (a SPV thin-client) check for incomming payments so fast?

@Jonas, Multibit uses bloom filters, that why it can check for incomming payments that fast.

I downvoted. Verifying transaction is exactly the thing SPV does not do: it only verifies the block chain, not the transactions in it.

SPV clients can calculate balances and can list transactions to certain addresses by requesting filtered blocks. SPV clients also verify transaction validity by verifying the signatures; what SPV does not verify is inclusion of outputs in the utxo, so double spending is possible.

@dionyziz SPV clients cannot verify signatures, as they don't have the UTXO entries to know what to verify against.


text:
How do I determine my transaction fee in an automated way?
When I send Bitcoins through Electrum, I just pay whatever transaction fee the software recommends. Now I'm writing a program that will have to determine the size of the transaction fee on the fly.
How is this computed? It looks like Electrum is consistently recommending .0002 BTC for each transaction but I read elsewhere that resolving the fee amount depends upon the size of the transaction. In an extreme example, if I wanted to redeem 15 tiny unspent outputs and send them to 2 destinations (the receiver + my change address), would the unusually large number of inputs for the new transaction justify a larger transaction fee? If so, how much larger?

How is this computed?

Bitcoin Core offers something called fee estimation, where you can gauge the amount of fee that is probably necessary to get a transaction into the next block. It returns the fee per kilobyte to get into one of the next n blocks, assuming the miners making them have similar transaction acceptance rules as prior ones.
1 confirmation 0.00026816 BTC
2 confirmation 0.00011483 BTC
3 confirmation 0.00002048 BTC
4 confirmation 0.00001255 BTC
5 confirmation 0.00001255 BTC
6 confirmation 0.00001255 BTC

Emulating this without running a fully validating node that can evaluate its memory pool over a long period of time is probably not particularly doable at present. Centralized wallets like Mycelium make HTTP calls to their own endpoint which returns this information on their behalf, there will probably be more endpoints for this in the future given the difficulty of estimating fees for non fully validating wallets.

would the unusually large number of inputs for the new transaction justify a larger transaction fee? If so, how much larger?

The transaction fee is per kilobyte. Adding additional inputs requires both the public key for the address and a signature, and some miscellaneous overhead to be included. It will often be about one hundred bytes per vin, but this varies if the input is P2SH or not and if the keys used were compressed or not. Generally if the input is below 0.00005 BTC it is probably going to add more in cost in fee than the value of it to include.
This transaction on mainnet is similar in size to the one you describe, it has 16 vin, 2 vout, for a total size of 2291 bytes. The key for 1FkiXxeKektpHVjvfknVvC6eQmjM9DEu9D is compressed (begins with 02 and is only 33 bytes long) which contributed to the small size of the transaction.


text:
What is NTime rolling/nonce range? And what miners support it?
From Slush's pool's homepage:

Pool now supports Long polling and NTime rolling. Older miners have many problems with those new features, which can negatively affect your rewards. Please update your miners to latest version!

What does NTime rolling and nonce range mean for me as a miner?
And what miners support it? (+versions)

NTime is the integer timestamp in seconds.  It is one element on the block header. Each second NTime increases by one.    "NTime rolling" is a getwork parameter indicating how many seconds forward a miner can still consider this getwork valid (by incrementing NTime locally).
To understand how this is used lets first take a look at Bitcoin Block Header.
For a given set of transactions only two elements in the block header change between hash attempts.

NTime - current timestamp (changes every second)
Nonce - 2^32 number (changes every hash)

So to construct a valid header a miner must have the current value of these two rapidly changing elements.  The other elements of block header are also needed but changes occur much less frequently and trivial for the pool to handle.
The basic (inefficient) method to handle these two variables in a getwork request is for the pool to explicitly provide both values in every single header. The pool will provide one getwork every second.  A miner with less than 4.2GH/s however will on average find < 1 nonce per getwork. As an example, a miner which can execute 100MH/s will cover the nonce range (2^32) in 40 seconds.  The pool will need to issue 40 getworks per share.  The avg # of getworks per share will depend on the speed of individual miners however a significant amount of excess communication is occuring.
Now both nonce range and NTime are highly predictable so a miner could simply be given a starting point and locally change both values.  Doing this significantly reduces the number of getwork requests required to cover the same number of hashes.
By allowing the miner to roll the NTime forward for 10 seconds the pool can cut the number of getworks from 40 to 4 seconds.
There is also a benefit for fast miners which are already efficient.
A 2 GH/s miner will find on average one nonce every 2 seconds.  With an NTime rolling value of 10 seconds.  The same getwork will enable the pool to acheive 4 nonces per getwork.
The NTime rolling value instructs the miner how far it can increment NTime (in seconds) before needing to request new work.  So a NTime rolling value of 4 means the miner can use this header for 4 seconds before considering it stale and likewise a value of 40 indicates to the miner it can use same header for next 40 seconds.  Pools can use increasingly large NTime values however there are diminishing returns.  A block is found on average every 600 seconds, and periodically transactions need to be included.
Still an optimal Ntime rolling value ensures the pool is only updating getworks when work has actually updated in a meaningful way (elements changed other than nonce or time).
An example:
A pool provides a 100MH miner a block header containing the current ntime, a nonce range of 0 to 2^32 (full nonce) and an NTime Rolling value of 40.  The miner would hash until the local time increments 1 second (roughly 100 million hashes) then it would increment NTime value and continue to hash for another 1 second (roughly 100 million more hashes).  Once it has incremented NTime 40 times it must request new getwork.
Proper use of NTime rolling improves pool efficiency by reducing number of getworks per share.  In your example I speculate that Slush looked at the pool logs and determined that some miners were requesting work "too often" indicating they were ignoring or incorrectly using NTime Rolling value.   Obviously a pool operator would want all miners to use latest protocols such as NTime rolling, and Long Polling as it allows a pool server to handle a higher throughput with same amount of hardware, bandwidth and other resources.
I don't have an exhaustive list but as far as I know any "relatively recent" miner supports NTime rolling (as in released in last 6 months).

From your description it appears that normally a miner must discard its getwork after a second and get new assignment, but NTime rolling allows it to use the same getwork for more than 1 second. Shouldn't it instead work in a way that would allow a miner to not only increment nonce, but also increment timestamp? That is, after going through the whole nonce range it would set timestamp to a different time within NTime Rolling and work on the previously checked nonces, as the timestamp change would make the hashes different than the ones calculated for previous timestamp?


text:
Can you run P2Pool without having to keep the blockchain updated?
I'm guessing the answer is no, but I thought I'd ask anyway.
I'm running my miner off a USB pen at the moment and I'd like to run P2Pool but the main issue for me is there wouldn't be enough space left on the USB pen to store the blockchain.
Is there a way I can run P2Pool without having the blockchain stored locally?

The blockchain will probably be under 2GB for at least the next year. Isn't a bigger usb drive a trivial cost compared to your hardware and electricity?

I'd prefer a longer term solution

then you can buy a 16gb stick, for about 15-20usd, and that would last you for quite a while. :) but yes, as the answers suggest, you can run a bitcoind anywhere and connect to it remotely.

You could run the Bitcoin client on a separate machine, and connect to it over the network.

When you run p2pool, you can provide extra flags to tell it where to find the bitcoind to connect to:
bitcoind interface:
  --bitcoind-address BITCOIND_ADDRESS   : connect to this address (default: 127.0.0.1)
  --bitcoind-rpc-port BITCOIND_RPC_PORT : connect to JSON-RPC interface at this port (default: 8332)
  --bitcoind-p2p-port BITCOIND_P2P_PORT : connect to P2P interface at this port (default: 8333)
  BITCOIND_RPCUSERPASS                  : bitcoind RPC interface username, then password, space-separated


text:
Why does mining profitability tend towards zero?
Why does mining profitability tend towards zero?
Is it different for short term miners than it is for long term miners?

Example: if I mine now and sell them next year, they may be worth more as next year they'll be harder to mine

Thanks

Do you mean "the mining reward"? The current reward is 50 BTC per block, and will be halved in 2013. Profitability has to include other things like difficulty = cost of mining, price of electricity, etc...

Can you try elaborating on your question, Sean?

Sorry for the confusion, I meant profits in terms of how much a bitcoin is worth if exchanged for goods/services, as opposed to the "mining reward" of bitcoins per block. Hope this helps clear things up!

As with all things related to supply and demand, the market will eventually find its equilibrium, where the cost of generating the Bitoins will be close to their sale price. If the costs were bigger, less cost efficient miners would not mine. If the price was higher, there would be more room for investment that would increase the difficulty, thus increasing the cost to mine every Bitcoin.
If you want to calculate how profitable mining is, just put the data into any Profitability Calculator.

This is a bit of a generalization, since mining profitability is different for different people depending on their comparative advantages.
But the profitability of mining is affected by the difficulty, which in turn is determined by the total hashrate of all miners. If at any time the profitability is too high, this will attract more miners causing the profitability to decrease. If it is negative, it will cause miners to quit, increasing the profitability. The result is that there will be very few untapped opportunities to profit from mining.
Note that the profitability is only close to 0 if all costs are considered - hardware, electricity, space, maintenance, risk and so on. Every potential miner will have his own costs/benefits analysis.
The causality in your quote is reversed - if bitcoins are worth more, they will be harder to mine. (At least, if we ignore the effect of generation reward halving.)

This is a loaded question, as it assumes that profitability will tend towards 0.
However, for any given piece of mining hardware it's almost certainly a fair assumption for the foreseeable future that it's profitability will tend towards zero. This is because, from an overall perspective, as new processing hardware is developed (improving both efficiency and processing power), the cost per hash of mining decreases.
While this holds true, there will always be a point in time when the cost of operating a given graphics card (or other processor), with all ownership costs included, exceeds the cost of purchasing new hardware.
The most important issue for miners purchasing new hardware is whether the break even point can be reached before the hardware becomes essentially obsolete (because the costs exceed the revenue). Predicting this currently involves taking a gamble on what the future price of bitcoins will be.
All this means that the profitability of mining doesn't necessarily approach zero, as long as you continue to optimise your mining equipment to maximise efficiency.

You're right about the question. It was only worded this way as I've heard time and time again how profitability tends towards zero, even though you're right, as it is just an assumption. Interesting take on mining hardware becoming obsolete! Gave me some good insight that I hadn't thought about previously

The answer to your question can be found in the interplay between two independently varying factors: price and difficulty.
There are a number of factors which play into the variance of price, and it's too complex an issue to discuss here. Perhaps you could find your answers at the new Economics StackExchange (in public beta).
Difficulty is much simpler to track - the more people mine (more accurately, the higher the combined hashrate of the network) the higher the difficulty climbs. When price climbs, it tends to lead difficulty, making mining profitable so long as the price continues to increase. The increased potential for profit draws more miners until eventually the difficulty rises to meet or exceed the profit point. At this point, less efficient miners begin to switch off their operations and difficulty slows or reduces once again.
The equilibrium of such commodities is a well-understood economic principle based in supply and demand. Another way to think of it is that there's only so much "demand" for Bitcoin miners and so the "supply" of miners rises to meet that demand, typically overshoots a bit, then adjusts back to equilibrium like any market.


text:
How do I know when my mining GPU has been worn out?
How do I know for sure when my mining hardware (GPU in particular) has failed?
How can I be certain that my card is at fault and not my software or overclock settings?

Example: On my rig one of my card's hashrates updates the hash/sec display very slowly and after about 5 minutes of mining phoenix miner will crash and refuse to start up a new instance.

Thanks

You can try running different software for mining after restoring the overclock settings back to factory settings. This way if the graphic card fails, you know the problem exists regardless of the mining software and the overclocking.

I have tried this. I moved from linuxcoin to BAMT and changed the overclocks back to stock factory settings. The problem still occurred, where phoenix would crash or freeze and the box would become quite unresponsive, but still ran basic commands. However when I took the box apart to investigate, I restarted my miners and BAMT has now been running all night without any problems (overclocked again now). So I honestly have no idea what is the problem.


text:
Which mining software/script provides the greatest MHash/sec and power efficiency balance?
Hopefully this is not too specific..
Which mining software/script provides the greatest MHash/sec and power efficiency balance? (I'm currently using an old version of phoenix miner)
Preferably for the 5xxx series of cards, but if some mining programs are generally more efficient than others in terms of MHash/sec to power usage ratio, those are also welcome.
Thanks

My understanding is that it differs for each piece of hardware, and the OS may affect things too, which complicates things. Because of this, the usefulness of any answers to this question will likely reduce over time (as older hardware becomes obsolete). But others seem to have voted it up...

@HighlyIrregular good point(s). It was a bad question but I was interested in asking to find out if there was a general trend of some being more efficient than others.

@SeanChapman I think you'll find very little difference between the popular miners, as long as you use a recent version. That point is important though, older versions tend to have less performance.

It seems that way. However, I have had reduced performance with some of the newer versions of phoenix that's why I stuck to an older version. I think the problems with that have been fixed by now in 1.7 though. I am using a custom kernel so that helps also.

poclmb always had superior speed for me on 6970's

My experience with a 6990 rig is that m0mchil’s Python OpenCL GPU miner is the most efficient one for me.
But I would try them out on a 12 hour basis, and do a comparison. Maximum output might be very different between software/hardware combos. Would love to see the results.


text:
Is there any point attempting to vote when my hash power is so marginal?
With regards to /P2SH/, is there even much point in voting with 500MHash or so of hashing power given that the chances of finding a block within the voting timeframe is incredibly slim? Not to mention that my vote will be shadowed out by miners with larger operations.
Is the voting system unfair in this sense? Or if I want my vote to count for more should I just be piling on more MHash to show a greater support of the network? (as in, people with more MHash are entitled to more votes because they help support the network more than I do?)

There isn't a real answer to this question. Is there a point to mining where you reward is so marginal? The answer is subjective. The voting is not "unfair" like Bitcoin is not "unfair". I suggest you post this at the forums.

@ripper234 I see your point but I thought it was a valid question. Perhaps a better phrased version would've been "Do pool operators have too much voting power if people don't choose"

I'm not sure it helps a lot. It's a known fact that today Bitcoin is rather centralized due to pools. How much is "too much"? I'm not sure what kind of an answer would be valid here.

I was implying that "too much" is simply more than the average miner

If you change your question to be about facts, then it has a place. You can change it to "do pool owners actually vote for their entire pool", for which the answer will be "yes".

You can still vote by supporting a pool voting in the same way as you.
Voting is rather fair, as bigger pool operators have a bigger responsibility to their miners to keep the software up to date and operating according to their best interest. Thus mining in a pool is like voting for your representatives - you give your trust to them so they can work on benefiting you.


text:
Is the "double geometric" payout method for pooled mining fair to part time miners?
The "double geometric" payout method as used on the Ozco.in pool is supposed to be fair and prevent pool hopping, but is this method fair to miners that only mine for a few hours per day, as opposed to miners that mine 24/7?

Some linking would be appreciated. Also, you might want to check this resource out - https://bitcoil.co.il/pool_analysis.pdf

Related: http://bitcoin.stackexchange.com/q/370/516

Yes. If you mine at a given point in time, you do not know when a block will be found, and thus your reward for shares you submit is a random variable. DGM is constructed so that the average reward for a submitted share is always equal to what you would get from it in PPS (assuming equal fees). The total reward for all shares you submit is additive - so it follows that the average reward you get in DGM is equal to what you would get in PPS, no matter what is your mining pattern.
However, the reward you will get is random and has some variance, and the variance is greater if your mining is intermittent. The exact degree of this depends on the parameters used by the pool.
More background information about mining pool reward methods can be found in Analysis of Bitcoin Pooled Mining Reward Systems.

Not quite always equal I think. I believe with plain PPS the pool operator  wears the cost of block withholding attacks, but with DGM the other miners wear most of the cost.

@HighlyIrregular: This is true, the analysis assumes finding shares and submitting them is an atomic action. However, block withholding is also more expensive to carry out against DGM; and, this is not relevant to the specific question about intermittent mining - in the broadest sense, the payout in PPS will be lower because of the higher fee (which exist to compensate for the risks, one of which is block withholding).

As far as I know, yes. The expected value (in BTC) of a share is constant in time, for a given difficulty/pool-hashrate/etc.

If you are only mining a few hours a day, this will not likely work for you.
The way DGM works is that at the start of a new block all miners start to gain credit for the block.  All miners will gain a cut.  For instants when I mine with 1.6 GH in a pool with about 1.8 TH I'll receive 0.043xxx BTC when the block is solved.  If the block takes 5 minutes I get that for 5 minutes work.  If it takes 5 hours then I have to work for 5 hours.  Over time this averages out so long runs and short runs are inconsequential.  However if you mine on a 5 hour block for only 2 hours then your reward is going to be substantially lower.
Another point worth mentioning.  On many pools your award is not paid until the block is confirmed.  This usually takes about a day.
My recommendation is to use a PPS Pool if you are going to only be mining from time to time.

I'm afraid this is incorrect. You will be rewarded on average in exact proportion to your work on DGM. You don't know in advance when a block will be found, so you're as likely to get more than average as less.

The process of averaging works if you mine constantly.  It you mine for a few hours a day, you are not going to receive the same proportional share as if you mined constantly.  Since the end goal for DGM is to provide an average value that is equal to PPS, my suggestion is to mine PPS if you are going to mine only for a few hours a day.

I have mined both ways and I mine constantly with 1.6 GH and I still perfer PPS.  With DGM if a block is invalidate then you don't get paid.  What i saw was I made less the two days I mined DGM and I had wait 24 hours for the 120 confirmations of the blocks before I was paid.  I think that DGM favors the pool operator, where as PPS favors the miner.

Again, this is wrong. However, SE comments are not suited for debates, and I will be happy to take this elsewhere.

There's no free lunch, PPS only pays for invalid block by taking an extra fee to compensate for them. Mining rewards are random and depend on the pool's luck in finding blocks so you can't learn anything by experimenting for 2 days. PPS has more variance for the operator and less for miners, but correspondingly the operator will take more fees.

Moving away from the debate (I agree this is not the place for it), but back to the fact this user only wants to mine for a few hours a day. Which is better?

It depends on personal preferences. From a purely economical point, DGM will be better because the variance is not high enough to be meaningful. But psychologically PPS will be much better for his peace of mind, and if the fee is reasonable like 5% I think it's worth it.

I find it interesting that the creator of DGM is here, claiming user statements are incorrect. As with any "hop-proof" payout schema...
It has the net effect of punishing miners who don't mine for an entire round. DGM greatly reduced the risk to the pool op by transferring those losses to the miners. PPS is exactly the opposite, pool ops take all the risk & variance instead of miners.
DGM is for pool ops who can't manage their business well enough to keep correct reserves. Miners will quickly find their earnings decreased and start looking for other more favorable payout schemas.

Like it or not, PPS increases risk for the pool operator. Risk costs money. That money must come from somewhere.

Risk and loss are not the same thing. In DGM miners have more variance, but this means they can get either more or less than normal. On average they will get the same amount in both PPS and DGM, assuming the same fee - which means, on the long run they will get more on DGM because of the lower fee.


text:
Does the size of a pool affect the number of invalid shares?
Does the size of a pool affect the number of invalid shares?  If so why does it?

Do you actually see fewer invalid shares with a large pool? Or are you just wondering if the size of the pool might explain the number of invalid shares?

There's no reason pool size should affect the number of invalid shares. If you're seeing different amounts of invalid shares on different GPUs, the most likely explanation is that the GPU is too hot, improperly overclocked, or failing.

Many things can affect your stale/reject rate. An overclocked or faulty GPU could be making miscalculations. The pool's long polling might not be working properly. Even network instability could cause rejects due to duplicate shares (the share was received by the server, but your miner got an error so it tries again).
The way pool size would affect rejects is that a large number of miners slow down long polling on the server side. Everything else being equal it takes longer for the server to notify 10 000 workers of a block change than it would 500 workers.
So in general you can expect lower stale rate on a small pool, unless the small pool effect is negated by really slow server hardware or some other factor.

An invalid share might be a share you submitted with an error in calculations, or possibly a stale share, depending on the pool implementation.

A possible reason for a rejection of a share might be that the pool doesn't use difficulty 1 as the target, while some miners might try submitting those shares
There is also a possibility that your miner makes some mistakes while calculating hashes, but that is less likely
The pool might be poorly implemented and make some mistakes

A reason for a stale share might be that:

The pool does not use long polling, thus it doesn't inform the miners when to update their getwork

All in all, you should ask other people whether they have the same problem for the given pool, and try mining using a different miner and a different computer and see if the problem persists.


text:
Why do miners seem to be using consecutive letters for the Coinbase transaction text?
If you look at coin.dance, it looks like the coinbase transactions text start with consecutive letters. For example, block 478529 starts with "AM" and 478530 starts with "BM" and 478531 starts with "CM" and so on. This continues with DM, EM, ... all the way to ZM despite being mined by lots of different miners.
This looks coordinated in some way. Why is this? Can't the coinbase text be whatever the miner wants?

According to BIP 34 the coinbase transaction is required to start with the block height.  The format is a single byte giving the count of bytes to follow (currently 3) and then the block height in little-endian. See Why block height is required in coinbase? for more on the rationale.
478529 decimal is 0x074d41 in hex, so this block's coinbase must begin with the bytes 0x03 0x41 0x4d 0x07.  0x41 0x4d happen to be the ASCII codes for the letters A M.
Likewise, 478530 decimal is 0x074d42, so we start with 0x03 0x42 0x4d 0x07, and 0x42 0x4d is B M.


text:
Where is the Mycelium wallet file stored?
I have a fairly straightforward question that's still unanswered after googling for days for a definite answer, I hope you guys can help me out with this one here.
Background: I have Mycelium Bitcoin Wallet installed on my Android phone [ROOT], I have backed up my HD wallet which also contains Local Trader account, and have been successful in restoring said backup[s] (by means of un-installing app, wiping it's old data and re-installing / loading wallet from backup (Master seed)). I also have redundant backups of the app and it's data (I assume that includes the wallet file?) made with Titanium Backup.
Question: I however still do not know where Mycelium's wallet resides, is it on my SD extCard? SD Card? Data  partition? Protected Storage? Is it just determined by the Master Seed and hence resides nowhere?
Final Note: I'm eagerly awaiting this piece of information to be at ease with my policies, knowing that  I can manually backup/encrypt my wallet file (if there is one) is really important to me.
Thanks in advance!

For Mycelium, the actual file that contains the encrypted keys are in the sqlite3 file. You can see in the source code that though there is in-memory storage, it's used mostly for testing and that the main way in which Mycelium data is stored is in a sqlite3 file. Sqlite, if you aren't familiar, is a very simple relational database that contains the whole database in a single file.
You can see the actual relational database table layout in the source code.
To answer the question about exactly where the wallet is, mycelium does not have the installLocation attribute and therefore is only going to be stored in internal storage, or on the phone itself.

Thank you very much! Time to review that wallet handling code.

Thanks again, your answer proved most useful. Yes I'm used to work with SQL frameworks. Will wait a bit before I mark your answer as best, see if anybody else has anything to add.

I was just looking for this very same information. My response is based on the current version of Mycelium (Sep 2016). This is what I found:
The backup is embedded in the 12 words Mycelium generates. So let's say you drop your phone in the ocean and get a new one. When you install the Mycelium app and restore the wallet, Mycelium will regenerate the wallet from those 12 words. Nothing is stored outside of the phone or on a server.

Does not answer the question -- this is about the actual wallet file.


text:
How many addresses does my wallet.dat contain? Do I need to make a new backup every time I make a transaction?
I'm having little problem here, trying to figure out the safest way to backup my wallet.dat to make sure that no BTC may be lost in case of recovery.
I'm not sure how the process of adding new addresses may affect a, say, a recently created backup.
So, basically, do I need to backup continuously to make sure that no BTC are lost? do I need to backup after each new address is created?
Can you provide me with some general golden rules when backing up wallet.dat?

https://en.bitcoin.it/wiki/Key_pool

By default, Bitcoin Core creates 100 addresses the first time it starts, and tries to keep 100 not-yet-used keys in wallet.dat 'keypool' (when you unlock the wallet to make a payment it will generate more).
So you do not need to backup continuously. The 'getinfo' RPC command will tell you the time when the oldest not-yet-used key was generated ('keypoololdest').
Any wallet backup created after the keypoololdest time will be good enough to recover all of the bitcoins sent to your wallet, but I'd suggest you figure out about how long it will take you to send and receive 100 transactions (e.g. two per day would be 50 days) and then backup about twice as often as that (maybe once a month if you're making 2 transactions per day).
Be sure your backups are encrypted with a strong passphrase, and make sure you store them someplace off-site that is safe.
And for long-term storage of more than one or two bitcoins, use a hardware or paper wallet.

thanks a lot for your answer. Not only it is very detailed but it also provided me with general guidance about backups, and addresses generation. Further reading on the wiki is on my ToDo List.

when I tried to watch the "keypoololdest" after your answer I noticed the following : it was 1392447127 which is Sat, 15 Feb 2014 06:52:07 GMT , I created new address then it goes 1392449568 which is Sat, 15 Feb 2014 07:32:48 GMT , I'm wondering should the time change to be today as it is too old (like one year ago) . and if I took a backup today is it quite efficient until the "keypoololdest" changed to be more than 1422866692 (Mon, 02 Feb 2015 08:44:52 GMT) which is the date of the backup ?


text:
Can you make a profit mining with CPU/GPU?
A Radeon HD 7950 has the following hash rates:

Around 600 MH/s for Bitcoin
Around 600 KH/s for Litecoin

I used the profitability calculator at bitcoinwisdom to estimate revenue:

Bitcoin: 0.06148 USD / Month
Litecoin: 0.6661 USD / Month

Am I missing something? Less than 1 dollar per month? Is it because of the specialized mining hardware?

No you are not missing anything. CPU mining has been unprofitable since 2011, GPU mining just slightly later.
Today, ASICs rule mining. The Bitcoin network has more than 300 Phash/s by now, that is 300,000,000,000 Mhash/s.
The upcoming ASIC miner generation is announced to provide more than 9,000 Mhash/s per Dollar. Times for GPU/CPU mining are long gone.
The situation is largely the same for Litecoin with a slight delay.
Also see: Why does mining profitability tend towards zero?
There might be some altcoins that can still profitably CPU/GPU mined and traded for Bitcoin thereafter, but I am not sure even they are worth when you factor in time investment.


text:
What options are there to combine mining over multiple computers?
Assuming you have multiple (30+) computers, what options are there available to do a distributed solo-mine?  Can I have them all checking into one bitcoin application over the network, or do I need to create/join a pool?

Don't use the same wallet on different computers. See also http://bitcoin.stackexchange.com/questions/490/can-i-use-my-wallet-on-different-computers

I'm not planning on running the bitcoin application on all of the computers, just the miner, if that's possible.

shouldn't be a problem.

You can run as few or as many Bitcoin clients as you want. You can have them all pull from the same client, or you can have each miner pull from its own client. Just remember, the mined coins will go to the wallet of the client the miner pulled the work unit from.

Are there any guides to setting this up?

There is no advantage to "pooling" solo miners. Simply install the client and a miner on each machine.  Point the miner at the local client and start mining.
While you can have all the machines pointed to a single client (just give the miner the machine-name or LAN IP address of machine w/ bitcoind/wallet) doing so introduces a single point of failure.  If the machine with the wallet goes down you lose the entire network until you fix it.  If each machine is using a local bitcoind/wallet and it goes down you only lose 1/30th of the network.
If you feel you must have centralized control I would strongly recommend setting up a private pool and pointing all your rigs towards that.  While you still have a single point of failure the server can be simplified to only run the client and pool daemon with no other applications.  That combined with no OpenCL/GPU driver complexity and no massive heat/power draw should result in a more stable single point of failure.  If your rigs are no climate controlled (warehouse).  THe pool server could be put in a clean climate controlled enviroment which should improve uptime.  While I strongly recommend fully decentralizing your rigs I would consider this option rather than pointing multiple rigs to same wallet on one rig.  Of course it makes more sense the more rigs you have.

Is there an automated way of transferring any earned coins to a central wallet?  Or would I have to check every miner by hand?  I'm looking for automation.

@kiyote Not as far as I know.  One reason if there is that blocks occur very infrequently.  A 2 GH/s rig will only find a block on average once every 36 days.  It would be possible to write an automated bitcoind that say once per hour checks value of addresses and if it >0 creates a transaction to send balance to a "central" wallet address.  When I solo mined I just logged into each rig once every few days and did a manual transfer.  Like I said it occurs rather infrequently.

@kiyote.  I though of another option today.  BAMT is a linux distro w/ built in miner.  It can be setup so that you can monitor and control multiple rigs running BAMT remotely.  While it doesn't do wallet transfer automatically it will notify you when a rig finds a block and which rig found it.  This could save some time in doing transfers if you do them manually as you would only need to login to the rig which found a block and do a ontime transfer.  This should occur roughly once per 72 days for each GH of hashing power the rig has. days on a 2GH rig has.


text:
My raw transaction destroyed 0.0284377 BTC. What did I do wrong?
Years ago I designed a .NET module which facilitates the transmission of BTC to my customers. It creates a binary representation of the desired transaction based on the material presented here and here:
The binary representation is then converted to hex and pushed out through various free services like BlockExplorer, BlockCypher, etc.
The system has been running flawlessly for years. Until yesterday. A customer requested that 0.0284377 BTC to be sent to 38MRMGjMBMp4k7vZhKLHhcM9Pm8AMLy18v. He never received it. Sure enough, he was right. It was never sent.
I looked through all of my logs and saw that my software did in fact submit a transaction with the following inputs and outputs:

input: 13P38hMYJXFdxDJJn8TtPUJZFXmcpf2J99
output #1: 38MRMGjMBMp4k7vZhKLHhcM9Pm8AMLy18v (my customer)
output #2: 1Ny3CV3rAsNMWpLfpxhXW3Fh71YmMEXXU7 (my change address)

Everything looked fine on my side but sure enough, when I looked on block explorer, what I saw shocked me. Consider transaction 9a138b14dcc8ae740073c06933ae04e3b08fe6be6ada0dc175e6484250dfe269 and look at the inputs and outputs:
As you can see, my input is correct. My change address XU7 is also correct. But now look at my customer's address. It says 17fQRjEudTVgexE8aDfhGyzDFEqSnnJJCA (which I'll refer to as JCA) instead of the expected 38MRMGjMBMp4k7vZhKLHhcM9Pm8AMLy18v (which I'll refer to as 18v). What the heck? The JCA address is known neither to me nor to my customer. It's a completely unknown address.
Clearly the problem is in my code somewhere so I dug deeper to find out what made this particular transaction unique. I identified that his desired address (18v) begins with a '3' whereas every other outbound transaction I've completed throughout the history of my app has targeted addresses beginning with '1'.
My stopgap measure was to force users to only specify addresses that begin with a 1. But that's a temporary solution. I need to find out what I'm doing wrong.
I turned to Google. Research indicated that addresses beginning with '3' are typically SegWit addresses where as those beginning with '1' are traditional, old-school addresses. I remember all the talk about SegWit a few months back but I figured it didn't affect me and my legacy transactions would still clear properly. Obviously that was a bad judgement call on my part so now I have to figure out what I did incorrectly and where the bogus JCA address came from.
That's where I'm stuck. I think my problem might have to do with uncompressed vs. compressed public keys based upon discussions here and here:
Here's what I do know: When it comes time to create the outputs for my transaction, my code does the following (explanation follows):
Func<String,byte[]> makeOutScript=(btcAddrHex)=>{
    byte[] addrBytes=BTCUtils.Base58Decode(btcAddrHex);
    byte[] pubKeyBytes=addrBytes.Take(addrBytes.Length-4).Skip(1).ToArray();

    using(MemoryStream ms=new MemoryStream()){
        using(BinaryWriter bw=new BinaryWriter(ms)){
            bw.Write((byte)0x76);   //op_dup
            bw.Write((byte)0xa9);   //op_hash160
            bw.Write((byte)20);     //size of public key
            bw.Write(pubKeyBytes);  //public key
            bw.Write((byte)0x88);   //op_equalverify
            bw.Write((byte)0xac);   //op_checksig

            bw.Flush();
            return ms.ToArray();
        }
    }
};

What I'm doing here is: Converting the output address to bytes by Base58 decoding it. I strip off the last four bytes, which are the checksum, and the first byte, which is always 0x00 (apparently a network indicator of some sort). That leaves me with the raw public key. Actually, looking at my internal notes, it's not actually the public key but rather 0x04 prepended to the public key and then passed through SHA256 and then RIPEMD160. But I'll refer to this blob as the public key. From there the byte sequence is 0x76, 0xa9, 20 (the size of the public key blob), the public key blob itself, 0x88, and 0xac.
I don't pretend to understand what all those single-byte entries mean but the blog posts above said to include them so that's what I did, and until now it's worked just fine.
The question is: How on earth did the 18v address get transformed into the JCA address when it was submitted to the network? Is my "size of public key" value (20) incorrect, perhaps? At a gut level, it does seem odd that a compressed (18v) and uncompressed (JCA) public key would both have a constant size of 20. But maybe I'm on the wrong track entirely and compression has nothing to do with it.
I don't have to tell you how awful it would be if this transaction were 100 BTC instead of 0.0284377. I lucked out that my bug was triggered on a low value transaction. But I really want to solve this. Can you point me down the right path?

Addresses that start with 3... are P2SH addresses, and they've been around for almost 6 years now. Historically they saw limited use - used by multisig wallets like Copay and GreenAddress. The compatibility mode of SegWit however also uses P2SH, making their use more common recently.
The main issue is that your code for converting an address to a scriptPubKey does not look at the version byte. After Base58 decoding the address, you should get <1-byte version> <20-byte hash> <4-byte checksum> structure. If that version is 0, which it is for 1... addressses, that hash is a public key hash, and you correctly construct the corresponding scriptPubKey.
However, if that version number is 5, it is a P2SH address, which was specified in BIP13, which uses a corresponding scriptPubKey specified in BIP16.
While SegWit initially just used the old P2SH format, A new address format is being introduced for SegWit as well (which gives slightly better performance, flexibility, and error detection capabilities), described in BIP173. (disclaimer: I'm an author of BIP173). These addresses will start with bc1....
I advise you to stop your service, and at the very least implement verification that the version number is expected, together with a test set you can find in many implementations. If you feel like it, you can also implement BIP13 and BIP173 addresses, both of which will likely become more prevalent soon.

Whoa, this is crazy interesting. I don't know how I've spent all these years never dealing with a Bitcoin address starting with something other than '1'. Even outside of my app, all of my Electrum addresses start with '1' too. Ok, so you're right, when I unpack the 18v address it does in fact have a leading byte set to 5.  So you're saying: (1) check that version byte! and (2) if it's a 5, I need to change the structure of that script? What exactly is "BIP"? Is that the reference doc? Why so many people claim that leading '3' is a "segwit" address? So much bad info out there :(

No, the first thing you need to do is check the checksum and the version number. Thinking that the version is always zero is the cause of this problem. Even if you believe it will always be zero, your software should fail if it isn't.

BIPs, or Bitcoin Improvement Proposals is a process for publishing ideas around changes to Bitcoin's rules or their use: https://github.com/bitcoin/bips

SegWit needed a way to be compatible with old software (so that old wallets could send to SegWit wallets), so it can be used either in compatibility mode (which uses P2SH 3... addresses) or native mode (which uses BIP173 bc1... addresses). Before SegWit, the only software that would hand out P2SH addresses is multisig wallets like GreenAddress or copay.

Gotcha. When the user submits an address to the app, that's when I'm running the checksum validation. I've got that part covered. My mistake was thinking that what I now know is the "version byte" was just a constant zero. Now I can check for it and take action. Thanks! I'm still a bit confused on how the format of my scriptPubKey will change if the version number is 5. Presumably I'll be able to hunt that down somewhere.

You'll find that in BIP16.

hi guys @PieterWuille thanks for this superb QA, awesome.  it has led me to ask a question .. https://bitcoin.stackexchange.com/questions/64166

Wow, such a great read guys.