text:
What are some of the advantages & disadvantages of the following: Colored Coins and Counterparty
Seeing the release of coinprism wallet for colored coins and I'm also playing with Counterparty.
What are the main differences among these two competing decentralized asset managers?
I see that counterparty.co has their own currency XCP, but colored coins does not. Now the question is how many of these XCP coins does the developers control or the first few people that created them?
And couldn't colored coins adopt what Counterparty is currently doing and make them obsolete? If not, why?

Having done my research on both services (colorcoins & counterparty). I was sold on coloredcoins initially, but I ran acrosss the NXT system, which is a separate coin but also ties into BTC via their www.multigateway.com system. I can see NXT being the major underdog that can outtake these services in the longterm, imho.

There is this new site comparing Colored Coins with Mastercoin and Counterparty: www.bitcoin2comparator.org

Colored coins just "mark" coins with a "color". This is done by using unused data space in the Bitcoin transactions.
Counterparty on the other hand has:

the same functionality as colored coins
decentralized asset trading with automatic order matching
assets that can pay distributions (dividends)
assets can be customized (divisibility, callback, locked or issueable, total amount, description)
peer-to-peer betting
contracts for difference
BTC and XCP trading
Is fully open-source, and built to support higher level applications. (Such as SWARM, Vennd, Xbet aka countersports, etc.)
XCP, which is a scarce (2.6 mil) and deflationary currency inside Counterparty that makes many of its features possible. It is not mineable and supply doesn't increase.
Counterwallet, which is a client-side web wallet that provides access to most Counterparty features.

This looks like a quite one-sided comparison. I'm sure Colored Coins can do more than "just mark a coin with a color"...

@StevenRoose no. lol. this isn't a problem, it does what it was made to do.

Actually there are issues with counterparty as the exchange doesn't really work on its own. You have to be online for the trade to happen which is kind of silly imo. I went with the NXT platform instead, they seem to have alot of decentralized services on top of it already. Definitely a hidden gem among all the noise.

The main difference between these protocols is how they associate and store data about asset ownership. Colored coins at least as implemented by Coinprism, put data in the form of a transaction output that must be retained in later transactions involving the associated standard transaction outputs (spending money). If a colored coin is handled by a wallet which is not colored-coin aware, the output containing data would not be reproduced and the colored coins involved would be lost.
Counterparty takes a different approach in that all of the ownership, issuance, order, betting and other data is stored in the blockchain with no relationship between money spending bitcoin outputs and the existence of such data. Instead, each counterparty transaction adds data to the global counterparty history. If your address has 500 FOO tokens, it will continue to have those tokens until a counterparty transaction is signed by your address that sends them somewhere else.
The model Counterparty uses, makes it possible to do things like locking the issuance of a token so no more can be created in the future.
Another fundamental difference is that colored coins use the 40-bytes available after an OP return whereas Counterparty encodes its data in the form of additional multisig addresses. Each allows spent bitcoins to be reclaimed but Counterparty can store more data in this way as long as Bitcoin's fee structure and mining community allow it.
In regards to how many xcp the developers control, that is unknown but is related directly to how many bitcoins they burned during the 30-day proof-of-burn period. At that time ~2100 bitcoins were destroyed to create 2.6 million xdp.
Colored coins would basically need to restart itself under a new model to copy what Counterparty is doing including its own solution to the initial distribution of a valuable, native XDP-like token.

in your opinion, which one would you prefer and why?

They basically let you do the same thing.
The differences are that:

Counterparty relies on the XCP currency whereas colored coins work directly on Bitcoin, so you don't have to get price exposure or volatility risk with a new currency when you use colored coins.
Counterparty uses a monolithic protocol where breaking changes are introduced about every 2 weeks. This makes it semi-centralized as the developers have the power to change the protocol as they see fit. For example, they can change the fees, disable features at will, change how orders are matched, etc... They even decided to add support for a "rock-paper-scissors" feature. On the other hand, colored coins (in particular open assets) are following a minimalistic approach, where it only does one thing, but does it really well. The protocol is final, so it is more robust and future-proof than counterparty.
Colored coins is cheaper to use than counterparty as counterparty has specific fees you need to pay, in addition to the miners fees. Colored coins only has the miners fees.
Colored coins being a thin layer on bitcoin, and working with outputs rather than addresses, can use features bitcoin offers out of the box, whereas counterparty can't: unconfirmed transactions, micropayment channels, SPV, etc...
The customization of assets with colored coins goes well beyond what Counterparty can do (pictures, long description, full range of divisibility - not just 8 places or nothing).
Colored coins use Proof of Authenticity to verify issuers, whereas it's very easy to create scammy assets with Counterparty.
Counterparty is tied to an economic model because of the XCP currency. Sooner or later, a group of people unhappy with the current distribution of XCP will fork the code of Counterparty, and create a clone with a different distribution, but the same features. That will create two separate networks, and split the liquidity in two (this is basically what already happened with Mastercoin and Counterparty). This makes Counterparty a risky bet for the long term. This can't be the case with colored coins since they are independent from any economic model.

hello, as someone that used to like colored coin more than counterparty, let me correct you on some things. 1) counterparty does not rely on XCP, first look at all the things that are similar with counterparty and colored coin, counterparty does all those things directly on the blockchain - just like colored coin. counterparty has additional scripting parameters that require XCP, so if you want to make a contract for difference, then thats when you would use XCP, if you want to simply issue an asset only bitcoin is involved. your first 4 points are invalidated here, as well as your last point.

Colored Coin's customization of assets is not a standout feature, this metadata is hosted by the issuers on the issuer's own servers. If their server goes down you have to rely on a mirror or not have that information at all. There is nothing stopping a future counterparty GUI from reading and parsing externally stored metadata.

and this appears to take apart your final point.... proof of authenticity is going to be an ongoing concern with all assets. The ability to store even MORE metadata - externally - to verify an issuer is again not unique to colored coin. This is simply coinprism's GUI being used again. An asset co-signing mechanism or the exact same SSL mechanism could easily be done (ie. read) by counterparty protocol or their GUI

finally: the ability to accidentally uncolor colored coin assets are a unique problem to colored coin. So anyway, I gravitated toward colored coin because I hated mastercoin's concept, and I mistakenly treated counterparty like a mastercoin for a very long time. I urge you to take a different look at it

I don't think your comments invalidate any of my points, they still hold true. In response to your comments: "if you want to simply issue an asset only bitcoin is involved": this is wrong, there is a 0.5 XCP fee for issuing an asset.
"Colored Coin's customization of assets is not a standout feature": yes, and it proves it can and should be external to the protocol itself.
"If their server goes down you have to rely on a mirror or not have that information at all.": The issuer has a strong incentive to keep it online. Using the blockchain as highly available data storage is misguided at best.

"The ability to store even MORE metadata - externally - to verify an issuer is again not unique to colored coin. This is simply coinprism's GUI being used again. An asset co-signing mechanism or the exact same SSL mechanism could easily be done (ie. read) by counterparty protocol or their GUI": what could be done is not relevant to the discussion. What matters is what exists today.

what is the strong incentive to keep it online? secondly  I don't agree with "what matters is what exists today", because this whole analysis is based on inherent limitations of the protocol and none of these aforementioned things are limitations. ie. Mastercoin has inherent liquidity limitations that colored coin and counterparty do not have


text:
Is there a block explorer for Ardor Testnet?
I'm looking for a block explorer for Ardor Testnet, but can't find one. I know of http://ardor.tools but that is only for main net.

The ardor portal block explorer also monitors the testnet, see https://test.ardorportal.org/monitor

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Can you use any Bitcoin address as a Counterparty token receiving address?
I have a bunch of bitcoin addresses and I would like to just use them for receiving counterparty tokens. I've read that counterparty addresses are the same thing as a Bitcoin address. Is this correct? And if so, doesnt this mean every Bitcoin address can effectively receive counterparty tokens?
I also can't seem to find an option to import an existing Bitcoin address into the counterwallet website. I was only able to find an import sweep option. screenshot would be nice.

Yes, you can use your Bitcoin addresses to receive Counterparty tokens. Make sure you have the private keys to your addresses backed up.
Counterparty is a layer built on top of the Bitcoin protocol which basically allows you to embed data into the Bitcoin blockchain. This data can be used to represent tokens which will be recognised by the counterparty protocol as being owned by a particular Bitcoin address. You will need a small BTC balance at the holding address in order to send those tokens to a new address.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Difference in hash rates between Bitcoins and Litecoins calculation
I recently discovered the Hash rate of my graphic card is different with Bitcoins than Litecoins. I understand this is due to the difference in the algorithm. Bitcoin has a double sha256 instead of Litecoins.
If I have a Miner of 1THash/s at Bitcoins what will be the average hash rate I get on calculating the Litecoins?

If you are using general purpose mining hardware (i.e. CPU, GPU) then it really depends on lots of factors, not only hardware, but also mining software and even its configuration.
For details on mining hardware you can check the tables for Bitcoin and Litecoin.
But with the inevitable introduction of ASIC miners (already for Bitcoin, and soon for Litecoin) the question is actually irrelevant.
ASIC miners can only work with the hashing function for which they were designed.
So a Bitcoin ASIC miner (SHA256) cannot mine Litecoin (scrypt) at all, and vice-versa.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Are my bitcoins safe?
As I understand, the safest way to keep your bitcoins is to have them in cold storage.
After reading a few articles I just wanted to make sure the technique I will use will allow me to pull my bitcoins out of storage whenever I want:
1) Boot up a computer using a linux live cd that is not connected to the internet
2) Transfer a copy of armory over using a USB stick
3) Generate a new wallet and 1 address. Encrypt it, and keep a paper backup of it.
I can now send all bitcoin 'savings' to the 1 address that the wallet has, and even if the computer that made the wallet is destroyed I can still recover the wallet from anywhere (even if I didn't copy the wallet file anywhere) because I have a paper backup?
As I understand it this should work?

if you're going through all the trouble to get a fresh-boot cold address you might as well generate dozens of addresses for use later

That seems right except there are some gotchas if you try to spend from your paper wallet.  The "change" will go to you another address and the paper backup address will be empty.  This should not be an issue with Armory though because it has deterministic addresses.  Be sure to follow the instructions carefully.


text:
Can free bitcoins from faucets be actually spent?
Suppose that I have a wallet with 0.001 bitcoins, from dozens of micro transactions (6000 satoshi in average). That means if I want to spend them, it must be concatenated and the transaction fee would be higher than the actual value?

There's a No forced TX Fee version of bitcoin client that (as name says) do not enforce any fee on transactions.
Note, however, that miners might be quite unwilling to include such transaction is their blocks, so there's the chance the transaction will never be processes.

it will never be processed, or it will just have a slow processing time? (days?)

If it isn't processed, it can't also be canceled, right?

It will be processed slow, and might even be never processed (and you can't cancel transaction). So that's quite a risky way.

You could consider creating a new address with 0.01 BTC on it, then sending the concatenated amount at 0 trx fee to each of the addresses containing the smaller amounts.
So:

0.01 -> first address
First address (now including the 0.01) -> second address
second address (now including the 0.01+first address -> third address

... ad nauseum.

authors: Александр Сорвачев


text:
Is there anywhere to get free testnet bitcoins?
I'm building a project with bitcoins and want to test it with testnet. However, I don't have any testnet bitcoins, and despite the difficulty being less than 100 usually, I still haven't been able to mine a block on my PC (CPU only) after a day. Is there anywhere I could get just some small amount of testnet bitcoins for testing?

Also see: How can I get more than 1 BTC on testnet?

(Link to faucet URL removed... please see the community wiki answer further down the page for links of faucets.)
The term "faucet" has been used for quite some time for a site giving away small quantities of coins for free. If all links in this page stop working, a web search for "bitcoin testnet faucet" may find a different site.

The important part of this answer is the term "faucet"; even if a specific source ceases working, a web search as suggested should be able to find another source of testnet coins.

One more site to get free test bitcoins...
http://kuttler.eu/bitcoin/btc/faucet/

manual sending there, so it might take some days - manu's is instant, see answer of Pavel

http://bitcoinfaucet.uo1.net/ - about 4000 testnet coins now

keeps saying I'm bot, but I'm not. My wife had me tested!

it's trying to detect adblockers and fails badly by overblocking non-blockers :D

Worked for me, Instantly! (as of 15 Nov 2018; i know mentioning the date is redundant :)

Website Shutdown
This worked for me today, they give 1BTC per request, theres captcha
https://testnet.manu.backend.hamburg/faucet

It does not work now

This is a useful frequent topic, but answers outdate quickly. I’m starting this community wiki to collect all current answers in one place and to make it easier to update the list of Testnet 3 and Testnet 4 faucets. Please record the date when you update a line.—Please upvote this community wiki for more visibility (community wiki posts do not confer reputation to any users).

Testnet 4 Faucet
Status
Last checked

mempool.space
Works (needs sign-up)
2024-08-02

anyone.eu
Works
2024-08-02

coinfaucet.eu
Works
2024-08-02

Testnet 3 Faucet
Status
Last checked

coinfaucet.eu
Works
2024-08-02

uo1.net
Works
2023-10-06

bitaps.com
Works
2023-10-06

kuttler.eu
Empty
2024-05-05

onchain.io
No response
2023-10-06

manu.backend
No response
2023-10-06

luis.im
Abandoned domain
2023-10-06

blocktrail.com
Business shut down
2023-10-06

xeno-genesis.com
Abandoned domain
2023-10-06

Edit: Website Shutdown
http://faucet.xeno-genesis.com/
working as of 2014-04-06 with about 2000+ testnet coins.

I got the error: DNS_PROBE_FINISHED_NXDOMAIN

Just launched Bitcoin testnet faucet for developers!
https://tbtc.bitaps.com

https://blocktrail.com has a testnet faucet but requires login

http://www.bitcointestnetfaucet.com/ Free unlmited withdrawals at 0.05 BTC max.

Testnet3 faucet as louis is down: https://testnet.coinfaucet.eu/en/
Edit: Defunct as of 2023-10-06

BTC Testnet faucet:
https://coinfaucet.eu/en/btc-testnet
Working as of Oct 5, 2018
Defunct as of 2023-10-06

Testnet3 bitcoins in http://faucet.luis.im
Active as of December 2015.

authors: Артем Веремеенко

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Trying to understand how bitcore-node stores the data in the DB
I'm looking into the data and source of bitcore-node ( http://bitcore.io/ ), I'm a bit confused how all the information is stored in the database, probably it's in a similar way to others block explorers, so maybe someone could help me.
In this case for example https://blockchain.info/tx/acb2e895a9b3664cfcfb13dd0321ef0c1205cf2aa082681dc6ba424af6456a42
In the database I found two txo- txo-acb2e895a9b3664cfcfb13dd0321ef0c1205cf2aa082681dc6ba424af6456a42-0
{"prevTxId":"8697331c3124c8a4cf2f43afb5732374ea13769e42f10aa3a98148a08989af5e","outputIndex":1,"sequenceNumber":4294967295,"script":"4730440220106c316c22d4bf8c970f8e4b1af1edd95121397cfe29ab987cb000b478ec8e4302202b77986b89d77ab716da3cf53424efacf35e9097b1c89194df220e99503b8cae01","scriptString":"71 0x30440220106c316c22d4bf8c970f8e4b1af1edd95121397cfe29ab987cb000b478ec8e4302202b77986b89d77ab716da3cf53424efacf35e9097b1c89194df220e99503b8cae01","heightConfirmed":37514}

txo-acb2e895a9b3664cfcfb13dd0321ef0c1205cf2aa082681dc6ba424af6456a42-1
{"satoshis":10100000000,"script":"76a91427d25a1ff9a6da31eeb991c48bb6cd95191a6b2c88ac"}

Clearly different data, not sure why in one case we store the previous txid and in the other just the satoshis sent. It has to be obvious, but not sure what I'm missing here.
Another example https://blockchain.info/tx/00eff263871e6d44abed66d90abb10d5a211441668511dba3146e380360efe4b
txo-00eff263871e6d44abed66d90abb10d5a211441668511dba3146e380360efe4b-0 (from 0 to 5 all have similar data)
{"prevTxId":"6aebb4a764a027436f703ca9d42d0735423798f56973f9f236832a91fc2e9026","outputIndex":0,"sequenceNumber":4294967295,"script":"473044022057801e812f964bcb9e8ff17b37acef55afbb7119a66f24a148ba65e86a7aa1e3022064af657d280068b979ea17a29969976aedee8a664b3f1b575ebb88ef9275302601","scriptString":"71 0x3044022057801e812f964bcb9e8ff17b37acef55afbb7119a66f24a148ba65e86a7aa1e3022064af657d280068b979ea17a29969976aedee8a664b3f1b575ebb88ef9275302601","heightConfirmed":10712}

Also,... they are inputs, not outputs of the transaction.
I'm trying to read the code, but I can't get to understand all the pieces.
As someone points me in reddit, it looks like it's using txo-- to store both, outputs and inputs. So I'm not sure how bitcore manages to get the inputs/outputs separately later (the API allows you to request only the inputs for example).

Are you using one of the bitcore sub-projects? I don't see any references to a database in bitcore, or in any of the sub-projects that I looked in, actually.

No, I was using the bitcore-node only, but because there were no subproject/module at the moment. They changed quite a bit the code since them, I need to review it again. I'm a bit confused by bitcore (only the library?) and bitcore-node (a full node with DB?), as you have a module for block explorer for bitcore (not bitcore-node) https://github.com/bitpay/bitcore-explorers . The node use a database : https://github.com/bitpay/bitcore-node/blob/master/lib/db.js
Anyway, what I want is the DB key structure, to replicate it on my own

Bitpay's bitcore-node project uses the same datadirectory as Bitcoin Core. You can see this in their sample configuration in their README:
var configuration = {
    datadir: '~/.bitcoin',
    network: 'testnet'
};

As such, the you can see: What are the keys used in the blockchain levelDB (ie what are the key:value pairs)? for information on what is in this database. However, bitcore-node does provide another interface for storing extra data about addresses and their balances. This is defined in this file. You have to enable this in your configuration and re-index the blockchain to enable this module, as their README details.
As these lines show, for each output, key-value pairs are added to the leveldb database in this format:
key:   [AddressModule.PREFIXES.OUTPUTS, address, timestamp, txid, outputIndex].join('-'),
value: [output.satoshis, script, height].join(':')

And, similarly for inputs, this is the format of the key-value pairs:
key: [AddressModule.PREFIXES.SPENTS, input.prevTxId, input.outputIndex].join('-'),
value: [txid, j].join(':')


text:
How does the Bitcoin client encrypt the wallet.dat file?
What is the algorithm used for encrypting the wallet.dat file?

A detailed explanation can be found in the application's README: https://github.com/bitcoin/bitcoin/blob/6b8a5ab622e5c9386c872036646bf94da983b190/doc/README
In short:

The passphrase is converted to a key/iv pair using EVP, with a dynamic number of rounds
This key/iv pair is used to encrypt a randomly-generated master key, using AES-256-CBC
The secret part of wallet keys are then encrypted using that master key, again with AES-256-CBC

Hmm, the README seems to be updated every now and then and the wallet encryption description is no longer there. Is this the version you were linking to? https://github.com/bitcoin/bitcoin/blob/6b8a5ab622e5c9386c872036646bf94da983b190/doc/README

@ThePiachu Thanks for the updated link it saved me a few minutes.  I updated the answer to reflect the permanent link.

Pieter's answer covers the technical details so I won't repeat any of that.  I would add that users should remember that the weakest point in Bitcoin-core wallet or any cryptographically strong encryption system is the passphrase.  The system converts a passphrase to an AES-256 key and that password derived key is used to encrypt a randomly generated master wallet key.  The master wallet key can not be brute forced barring a compromised or flawed PRNG however attackers can attempt to brute force the passphrase itself.
The client adds attempts to harden against this attack by using both a salt (random number) and multiple rounds of hashing.   The salt prevents a precomputation attack as the same passphrase with different salts will produce different keys.  The use of multiple rounds of hashing slow down brute force attacks because the attacker will also need to perform multiple rounds on each passphrase attempted.
The client uses dynamic key hardening.  This means the number of rounds varies based on the computational power of the client (to keep time to encrypt or decrypt below 1 seconds).  The more computing power your hardware has the more rounds it performs, and the more difficult it will be to brute force the password.  When you change your passphrase the client will adjust the number of dynamic rounds based on your current hardware's computing power. By updating your password periodically you can ensure the password derived key keeps up with Moore's law.
Key hardening through the use of salt and key stretching can only go so far.  If your password is "p@ssw0rd!" or some other known commonly used password it will still be trivial to brute force the wallet.  To illustrate why lets assume the attacker has hardware which can attempt 100M hashes per second and your wallet is using 100,000 rounds of hashing.  This means the attackers hardware (100M hashes) can attempt 1,000 passwords per second.  So the wallet does slow the attacker down but if the password is weak then it can't slow it down enough.  There are lists of previously compromised/stolen/hacked passwords publicly available with tens of millions of passwords.  Without key stretching an attacker could attempt all possible known passwords in less than a second.  Key stretching slows that down to a few hours.  On the other hand lets say that your password was unknown and strong enough that with a single hash it would on average take one day to brute force.  With 100,000 rounds that increases the time to centuries.

I would add that the passphrase is called passphrase for a reason. Using "This is a very long - yet very simple - passphrase to remember" is tons of orders of magnitude more secure than "fl@a%%zG", even if using a dictionary attack.

@AndreasBonini: Thanks to your comment, that particular passphrase will now be part of the dictionary though ;-)


text:
How does Bitcoin Wallet know someone has sent you some money? Does it keep scanning blockchain continuously for transactions having your Public Key?
How does my wallet know that someone sent me some money as soon as the block containing the transaction gets mined? Does my wallet check for new blocks on the blockchain continuously 24x7x365? And every time it sees a new block does it scan each transaction in that block & check if any one of them have my public key in the ScriptPubKey part of the transaction?

From the comments on my answer, I got the impression that you may be asking about a specific wallet. Could you please clarify whether you are asking about a specific wallet software, one type of wallet, or wallets in general?

What you describe is what happens
If the wallet is not running 24/7 it will catch up by requesting missing blocks from peers.
See also SPV


text:
Get raw transaction from tx id without txindex=1
I'm trying to get the raw transaction hex string of a transaction that I know the id from.
I would like to do that without using txindex=1.
As I assume the transaction is unspent (and confirmed) I'm doing the following steps:
bitcoin-cli gettxout 'txid'
bitcoin-cli getblockcount
bitcoin-cli getblockhash 'blockcount - txout.confirmations + 1'
bitcoin-cli getrawtransaction 'txid' 0 'blockhash'
I have a race condition when getting the blockcount.
Is there a better way to do that?

To avoid a race condition with getblockcount, you could use getblockheader with the bestblock result from gettxout and get its height. Then do the height calculation with the number of confirmations to get the block hash to pass to getrawtransaction.


text:
CVE-2012-2459 , possible code and performance improvement, is my logic correct?
ok so, in my effort to create a fullnode from scratch both for learning purpose and need (not a topic for discussion, thanks), I was looking at the code that check Merkle Root to not be vulnerable to CVE-2012-2459.
That vulnerability basically allows you to create a block that has a valid merkle root but contains duplicate transactions, causing a node that receive such block before the correct one, to be stuck on a fork (because that block will be flagged as incorrect and having the same hash of a correct block will prevent the node to ask again for that block, causing a node to be stuck.
Anyway, this is an old CVE that was fixed back in 2012 and code changed during time.
Now in bitcoin core the code is not that pretty and the method to compute merkle root are polluted from a boolean parameter that when read back reflects if the block has been mutated (malleated) or not.
The code is this
uint256 ComputeMerkleRoot(std::vector<uint256> hashes, bool* mutated) {
    bool mutation = false;
    while (hashes.size() > 1) {
        if (mutated) {
            for (size_t pos = 0; pos + 1 < hashes.size(); pos += 2) {
                if (hashes[pos] == hashes[pos + 1]) mutation = true;
            }
        }
        if (hashes.size() & 1) {
            hashes.push_back(hashes.back());
        }
        SHA256D64(hashes[0].begin(), hashes[0].begin(), hashes.size() / 2);
        hashes.resize(hashes.size() / 2);
    }
    if (mutated) *mutated = mutation;
    if (hashes.size() == 0) return uint256();
    return hashes[0];
}

now thinking about the problem, I think I found a better approach that is O(log n) and way better both in readability and speed... if it works and the logic isn't flawled... so I'd like to know your thought about this.
my paint skill can show you some of the logic:

A, B, C... etc... are transactions
the vertical line is what I call "safe point", basically all transactions before that safe point are guaranteed to be not duplicable (this is one of the assumption I do and one thing to check if it's correct)
then, based on how merkle root is computed and how the CVE uses that to do nasty things, you can see some example of malleated blocks
I created then a gist, containing a LINQPad code that can be run as it is and produce outputs to see if the logc is correct and can spot malleated blocks.
https://gist.github.com/MithrilMan/27985e4f5bcc3853e792aa39631b9647
this is an output example of that gist

The core logic relies into checking the last transaction against the previous, moving exponentially to the left at each iteration (see the picture with the vertical blue arrows showing which element it checks, or see the linqpad result where it's explicitly detailed)
I handle both the cases where the tx count is even or odd.
To me it seems to work but would like to have some more eyes on that to see if the logic sounds correct or not.
Note how for 9000 tx I have just to compare 14 transactions instead of thousands like current bitcoin code is doing.
Apart from that in my case it has another pros that allow me to split the markle computation and markle cve check in two different places, without having to use a solution like that mutated boolean parameter
UPDATE
for those of you not having LINQPad I adapted the POC script to be run on dotnetfiddle here
for those who doesn't have linqpad I adapted the script to be run on dotnetfiddle here
https://dotnetfiddle.net/wT87D2

any comment on this?

@Prayank I started doing it, then I stopped because I focused on other projects but the project is quite interesting and have already lot of stuff into
here is the repo: https://github.com/mithrilman/mithrilshards/
I tried to focus both on architecture and performance, I will resume it for sure, not sure when :)
That repo has a broader goal that's to be a foundation for p2p software, you can read something about it in the documentation site (yeah, I worked on that too :) https://mithrilshards.mithrilman.com/

Trigger warning: I'm not about to be nice. Because this code is responsible for my elevated blood pressure.
uint256 ComputeMerkleRoot(std::vector<uint256> hashes, bool* mutated) {
    bool mutation = false;
    while (hashes.size() > 1) {
        if (mutated) {
            for (size_t pos = 0; pos + 1 < hashes.size(); pos += 2) {
                if (hashes[pos] == hashes[pos + 1]) mutation = true;
            }
        }
        if (hashes.size() & 1) {
            hashes.push_back(hashes.back());
        }
        SHA256D64(hashes[0].begin(), hashes[0].begin(), hashes.size() / 2);
        hashes.resize(hashes.size() / 2);
    }
    if (mutated) *mutated = mutation;
    if (hashes.size() == 0) return uint256();
    return hashes[0];
}

'First thing I notice is the function signature.
uint256 That's 32 Bytes (octets) that's the size of your whole data cache on x86-64. And it's obtained by (dereferencing) using the first element of the vector of cache misses (or uint256, you pick the name you prefer).
The one time a pointer would have sufficed… (It's ok…)
bool* Indicates the return type is wrong.
In fact, I know what's wrong with all of this… It's not a member function to the TYPE that is a vector<char[32]>…
It's the tree, isn't it? Or rather "the latest branch of the[…]"
if (mutated) That doesn't check if anything mutated. That if statement should be taken out and shot.
That checks if the bool* mutated is nullptr which it will rarely be.
(And it SHALL NOT become nullptr inside this function. Otherwise, you have a data race and you can stop reading here because the rest is UB.)
if (hashes.size() & 1)
If the size is odd, sure.  How many times are we going to call that function to check if the size that so far hasn't changed? (Still the first iteration of this…
hashes.push_back(hashes.back());  Think about this…
We just did:
while (hashes.size() > 1) { // First check of size.
  if (mutated) { // It's not `nullptr`.
     // You SHOULDN'T pass a bool as a raw pointer.
     // FOR GOD SAKE.

    for (size_t // size_t is unsigned.
         pos = 0; // using auto you wouldn't have the problem of:
// EVERY SINGLE loop iteration you will be checking for overflow.
// I can tell you right away overflow is impossible in this case.
// But since the code that is coming up is so wrong…
         pos + 1 < hashes.size(); // THIS HAS TO BE A JOKE.
         pos += 2)
    {
             if (hashes[pos] == hashes[pos + 1]) mutation = true; // WHY?
// And now we're looping over the whole vector.
// Continually setting mutation to true.
// EVERY ITERATION will cause a cache miss
//  [if the optimizer wasn't able to optimize this…
//   (And it wasn't. The code is just wrong)].
// Every time we re-enter the while loop,
// because cache misses are slow,
// AND since you CANNOT reorder a LOAD with a STORE.
// It will waste time. `mutation = true` this is unbelievably costly on arm.

// This is a polynomial-time implementation of an algorithm which
// SHOULD be IN THE WORST CASE linear in time.

     } // for (slow; slow; fine) cache miss; cache miss; store; recurse…
  } // if (some pointer to a bool is not `nullptr`.
    // Which is constant over the whole [dis]function)
  if (hashes.size() & 1) { // currently false for vector of size 2.
// You'll not that we never bothered to check if the size was 2. NB: Θ(1)
    hashes.push_back(hashes.back()); // more cache misses.
// And potentially a new allocation of an object of obscene length.
// At this point it's just intentional.
// Nobody is that good at randomly deoptimizing code.
// Fun fact: hashes can be hashed.
// I'm pretty sure you could hash your uint256 with 2 hash functions,
// then simply xor with the other hashes of hashes (),
// then if, and only if ONE of the sets of 2 hashes(of a hash) mismatches
// you would bother checking the whole. 'should be Θ(n^2) worst case.
// A cache miss from L1 to L2 is a lil' bad.
// L2 to L3 is already very bad.
// L3 to ram?
// Better write it in Java it will have a higher throughput with a well-tuned GC.
   }

I don't care what the rest does. It's just too slow as it is.
return hashes[0]; may not even be NRVOed…
I looked at your code. It's nice, you're doing a clz, we only just got that in C++ in C++20 with bitcast, so we could integrate something similar. And by "we" I mean… Does nobody think it could be interesting to, now that concept has proven to generate public interest, start again with a "blankish" slate…
Hint: the bitcoin network is essentially the biggest decentralized neural network, that I know of. I'm not sure you people realize that.
The complexity doesn't have to be artificial. It was just like that to prove the concept. We've done that.
Nitpick: uint safePoint = ((uint)Math.Pow(2, higherBitPosition));
Isn't that just a binary left shift on 1 by higherBitPosition?  (it is. For higherBitPosition any Natural up to (excluding) the largest register width) Though floating-point pow is also 1 instruction, in this case it's slower (for integer powers of 2), otherwise, it's a perfect job for a gpu.


text:
ECDSA r, s encoding as a signature
An ECDSA algorithm when signing a given messages produces a pair of outputs, r and s. How, given a sigStr from a Tx can one extract r and s? Are they just concatenated byte arrays of a specific length, or is there more to it?

If you want, you can pay $100 for the standard, ANSI X9.62. Or, you can cheat and look at RFC3278, section 8.2. It is in DER format consisting of a SEQUENCE of two INTEGERs. The first INTEGER is r, the second s.
If you look at this transaction you can see that one of the signatures is:

3045 0220
316eb3cad8b66fcf1494a6e6f9542c3555addbf337f04b62bf4758483fdc881d
022100
bf46d26cef45d998a2cb5d2d0b8342d70973fa7c3c37ae72234696524b2bc812
01

If we parse that as DER, we get:
 0:d=0  hl=2 l= 69 cons: SEQUENCE
 2:d=1  hl=2 l= 32 prim: INTEGER :316EB3CAD8B66FCF1494A6E6F9542C3555ADDBF337F04B62BF4758483FDC881D
36:d=1  hl=2 l= 33 prim: INTEGER :BF46D26CEF45D998A2CB5D2D0B8342D70973FA7C3C37AE72234696524B2BC812

You can also peek at the OpenSSL source code, file ecdsa/ecs_asn1.c:
ASN1_SEQUENCE(ECDSA_SIG) = {
        ASN1_SIMPLE(ECDSA_SIG, r, CBIGNUM),
        ASN1_SIMPLE(ECDSA_SIG, s, CBIGNUM)
} ASN1_SEQUENCE_END(ECDSA_SIG)

Hmm, I managed to encode and decode the two integers, but my encoding seems to be missing the 01 byte at the end. Any ideas why that might be so? I used the code form here - http://stackoverflow.com/questions/8693513/go-der-and-handling-big-integers/8708864#8708864

I actually have no idea what that 01 is doing there. I'll try to find out and update my answer.

01 is SIGHASH_ALL, used by OP_CHECKSIG to decide how to hash the transaction that is being signed.

R and S are visible in each of the tx inputs.
You can see them already extracted on this site. I also provide the Z value.
https://2xoin.com/tx/711b6457b4b2b51e56b94ab541a75d02908648a9de26a3c0ce5b2c3b10573d4e
See the bitcoin protocol specification for more info about the byte order.
https://en.bitcoin.it/wiki/Protocol_specification#tx
Below is an example of a TX input hex,

4830450220657912a72d3ac8169fe8eaecd5ab401c94fc9981717e3e6dd4971889f785790c022100ed3bf3456eb76677fd899c8ccd1cc6d1ebc631b94c42f7c4578f28590d651c6e0141049b5506df53ff5eff7dc553131043bb993f55d2b0fddd866984f593777023c8226920ff05747ccb963f0fe459cb217d502e57dcf8afec786c3dcee4d1558f85fa

Now separated to make it easier to read,

48

3045

0220
(The Hex 20 says the next 32 bytes are the R value)

657912a72d3ac8169fe8eaecd5ab401c94fc9981717e3e6dd4971889f785790c

0221 (The hex 21 says the next 33 bytes are the S value)

00ed3bf3456eb76677fd899c8ccd1cc6d1ebc631b94c42f7c4578f28590d651c6e

0141 (The hex 41 says the next 65 bytes are the public key)

049b5506df53ff5eff7dc553131043bb993f55d2b0fddd866984f593777023c8226920ff05747ccb963f0fe459cb217d502e57dcf8afec786c3dcee4d1558f85fa


text:
Are there any services offering blockchain.info-like API for multiple currencies?
I am wondering whether there are any services that offer some API like blockchain.info does but for multiple currencies at the same time. I know there are a lot of block explorers out there, but they appear to be disjointed and focused on one currency at a time.

BlockCypher is adding support for Litecoin, we're currently downloading the LTC block chain and will put it out next week. All the  Bitcoin APIs will work in the same way for Litecoin.


text:
Can*t call chainActive.Tip()->nHeight in core.cpp
I want to call chainActive.Tip()->nHeight in core.cpp. It is not possible, I am interested in every idea.
https://github.com/LIMXTEC/BitSend/blob/DEV-joshafest/src/core.cpp
Here is an example :
bool GetH()
{
    int nBlockHeight = chainActive.Tip()->nHeight;
    if(nBlockHeight < FORKX17_Main_Net )
        return true;
    else
        return false;
    }
}

uint256 CBlockHeader::GetHash() const
{
    if (GetH())
    {
        return HashX11(BEGIN(nVersion), END(nNonce));
    }
    else
    {
       return HashX17(BEGIN(nVersion), END(nNonce));
    }
}

Also that is not possible:
CBlockIndex* pindexPrev = chainActive.Tip();
if (pindexPrev->nHeight <= FORKX17_Main_Net)

Looks like you need to #include "main.h"?

Yes I use main.h.
#include "core.h"
#include "util.h"
#include "main.h"
#include "uint256.h"

This is due to missing the object of the class which has 'chainActive()' method. So including which ever the file that has the method. In case if it gives further different undefined reference errors; find the file that contains the method; include it; until it compiles ;)

@SajithP chainActive is a variable not a method :) More likely, it's something to do with the makefile, perhaps main.h cannot be included when building -cli instead of the Daemon? Unsure

Got it! Then modify the make file to include the object of the class which has the chainActive as a member. e.g 'x.o' where chainActive is a member of class 'x'

As SajithP commented:

modify the makefile to include the object of the class which has the chainActive as a member. e.g 'x.o' where chainActive is a member of class 'x'


text:
Does testmempoolaccept check all possible rejection rules?
Is there any rejection rule that testmempoolaccept will not check, or is it safe to assume that if it passes testmempoolaccept validation it will be at least broadcast?
Mostly concerned about complex reasons, such as too-long-mempool-chain.

No it's not, or at least not at any point in time.
There are both invariant and time-dependent rules for the mempool to accept a specific transaction.
The testmempoolaccept command will check your transaction against the former (policy and consensus rules), but there is nothing you could realistically do to check the latter.
As an example of a time-dependent rule, you could have a software checking pre-signed transactions paying 253sat/kW of feerate against testmempoolaccept.
Valid transactions would pass the check, nonetheless they would not broadcast if the mempool minimum feerate is > 253sat/kW at the time of broadcast.

Yeah, fair enough. I was thinking more on non-transitory checks about the tx correctness, such as not double-spent, standard (ruling out fess),... I'm less concerned about transitory rejections.


text:
Preventing excessive rescans on startup
I upgraded my node at coinos.io from bitcoin 25.0 to 28.0 the other day and got hit by a bunch of somewhat lengthy rescans that caused me about 30 minutes of downtime.
I have close to 200 watch-only descriptor wallets loaded at startup and each one of them had to go through about a 30-40s rescan as the node rebooted.
As this was occuring the node was unresponsive to any other RPC calls.
I attempted to abort some of the rescans with bitcoin-cli -rpcwallet=<wallet id> abortrescan to no avail. I can't recall if it gave me an error message or just did nothing.
I'm wondering if there is anything I can do to avoid this in the future. Were the rescans really necessary in the first place and is there any way to opt out of them, abort them, or improve the efficiency of them without missing incoming transactions or losing track of the balance of my wallets?
I anticipate adding significantly more watch-only wallets to the node in the future and was hoping to be able to manage thousands of them on a single machine. Is that unrealistic?


text:
Transferring bitcoins from my blockchain addresses to an exchange?
What are the specific steps, in order, required to transfer my bitcoins from their blockchain wallets to a bitcoin exchange?  Please be specific.  Thanks

Are you asking about a blockchain as it relates to bitcoin, or the service named blockchain.info? 2) Are you asking for a technical description, or instructions? 3) Have you tried to research your question elsewhere?

Assuming you mean "blockchain.info".

Go to your exchange, deposit/bitcoins section. It should give you a bitcoin address, copy it.
go to your wallet, "send money"
enter the copied address and the desired amount
hit "send"

Your wallet contains one or more bitcoin addresses that are yours. Pick one and copy it to your clipboard.
Go to the deposit section of your exchange, select "Deposit bitcoins".
Paste in your bitcoin address, plus type in the number of bitcoins to transfer in.

That's it.

This post is a mess, who upvoted and why?


text:
Bitcoin Transaction Unconfirmed
I made a transaction on March 13th, transferring .88 BTC from my desktop wallet to another account. I have a Mac and am using Bitcoin-Qt version v0.8.1-beta. The transaction did not go through, my wallet is "out of sync", and I do not have the .88 BTC in my wallet anymore. I have scoured the internet trying to solve this problem and have not found anything yet. Any help is MUCH appreciated!

Let your wallet sync up. Then let us know what you see on the transaction page of the client. Is it there, but showing up as unconfirmed? Not there at all?

You can see when is your transaction with : https://blockchain.info
If your wallet is out of sync don't worry, it's normal that you don't see the transaction.
If you sent BTC from A to B, B will see the transaction when he's synced.

My transaction does not show up on the blockchain, and it did not go through to btcsportsbet, the recepient. How do I recover my BTC?

You got something to search for the transaction in BlockChain ? ATM let A and B sync completely :) If you don't have enter the wrong address it's impossible that you lost your BTC.

I solved my own problem. If others have this problem, the easiest solution is to find your wallet.dat file on your Mac, which is in in ~/Library/Application Support/Bitcoin/
Import this folder into a new wallet on Blockchain.info. Your balance will show up in there and you will have access to your BTC.
That was fairly simple and I spent way to much time on that!

Can you clarify this process please? I transferred my bitcoins from one machine to another and it is unconfirmed. Does this mean I need to find my old machine to copy the wallet.dat file? I am using a restored version of my OS and data on a new machine from a Time Machine backup


text:
How does a client know it is connected to the right P2P network?
Global botnets of "pwned" machines exist. Hypothetically, such a botnet could be configured to look a lot like the Bitcoin peer-to-peer network.
It could even have a long (fake) block chain, full of fake timestamps showing years' worth of history where the difficulty just never got very high.
My question is simple: How does the Bitcoin client know it is connected to the "real" Bitcoin P2P network and not to an impostor? That is, what mechanisms does it use to make this identification with confidence?
The more such mechanisms you identify, the better your answer is, in my view. Thanks.

There's a list of hardcoded blocks in the bitcoin client: https://github.com/bitcoin/bitcoin/blob/master/src/checkpoints.cpp
However, you present an entirely valid attack. If I can control one DNS seed and DDOS the others, I could cause new clients to only connect to my network.

You need to start at the Genesis block so you can't go back in time. And like Nick wrote, then the bitcoin-qt client has a list of "checkpoint" blocks which is hardcoded.
Also, the "correct blockchain" is defined by the one with the most work behind it (starting at the Genesis block ofc), so if you can see just one IP with the 'not-fake' blockchain, then he can prove that the millions of IP's you can see (the botnet) has a 'fake' blockchain.


text:
How does bitcoin prevent someone from sending more money than he/she has?
I am new to bitcoin, I was wondering how bitcoin implements this particular functionality efficiently. Does it check through all blockchain records for the address of the sender. Or does each transaction store a list of previous transactions by the sender?
I am curious as to how this can be implemented efficiently?

Essentially for each transaction that you want to initiate, you (or more specifically your client) indicate where you got the money from that you want to use for that transaction. These are called the inputs to your transaction. Your recipient (and actually every node on the network) then looks to make sure that you haven't spent that money from that transaction anywhere else. (This is the double spend issue that is referenced in another answer).
It's easy to implement efficiently because every synced bitcoin client has a complete list of all transactions. And they can index those transactions based on the transaction id so that they can very quickly lookup whether a transaction has already been spent or not. Or for that matter whether that transaction exists at all or not.
In other words, if you tried to spend 10 BTC that you didn't really have, you would have to "make up" a fake transaction of where you got that 10BTC from. And any client on the network would instantly detect that your made up transaction didn't exist and refuse to propagate your new transaction.
@Zim Brott
Actually this is a little different than a double spend problem the way I read it.

This is called the "double-spending" problem. There's an excelent article on the Bitcoin Wiki that covers the entire mechanism.
https://en.bitcoin.it/wiki/Double-spending


text:
Bitcoin network: data-intensive or compute-intensive system?
Is it more accurate to consider the Bitcoin network a data-intensive system due to the massive amount of data that must be stored by miners, or instead, a compute-intensive system due to the PoW and mining process?

Why can it not be both?

@PieterWuille , If it has both properties. But I'm not sure. I just want to know which adjective / property is more accurate for the Bitcoin network.

I don't believe this is an interesting distinction to make. It's a complex system with many aspects. Would you say airplanes are more accurately described as "big" or as "fast"?

This is probably one of those situations where any simple answer is inaccurate and misleading.
For any mining node, I expect computing costs much more than storage. By a long way.
For an individual non-mining node that doesn't use pruning, neither are really huge but maybe the data storage element dominates for at least some. Even so, the amount of data easily fits onto a single hard disk of the sort that can be bought at low cost. These are mass-market commodity items. It isn't at all like a large corporate data storage problem. It isn't Petabytes.
If you take the network as a whole, the data may become more significant since it is replicated on thousands of nodes but even so, it is the energy consumption of the computing part that is often compared to the energy usage of whole countries, not the data storage part.


text:
Testing pruning logic with bitcoind
I'd like to test pruning logic using bitcoind in regtest mode, however, given the nature of regtest, blocks are rather empty (if not fully). Is there any way of easily generating full blocks so the pruning logic can kick in?
I'm interested in solutions involving only bitcoin-cli.

There is the test-only -fastprune bitcoind option, which results in a much smaller maximum blockfile size (16 kb) and a lower minimum prune height. If that option is used, you could just bitcoin-cli -generate a couple hundred empty blocks and then use the pruneblockchain RPC to prune some blockfiles manually.
It's being used in some of the functional tests, for example.

Is that documented somewhere? I'm interested in what the range values for -fastprune may be, given it looks like it's not simply a flag.

it is simply a boolean flag (and the option is hidden and not documented well, because it's basically just a hack for devs to be able to tests pruning better, nothing any user should ever do). Probably the best way is to grep for fastprune in the codebase.

So just for completeness, the easiest way of doing so is using both -fastprune and manually pruning with bitcoin-cli pruneblockchain


text:
Will Bitcoin Core still respond quickly to a rpc-call when new blocks come in fast?
I had my Core Lightning node crash a few times because Bitcoin Core didn't respond in time to a rpc-call. I found that this happens when new blocks are generated quickly, like below:
849683  2024-06-27 14:46:02
849682  2024-06-27 14:44:11
849681  2024-06-27 14:43:30
3 blocks are found within 3 minutes, so 10 times as fast as on average (which can happen and is normal).
I increased my cpu-limit to mitigate the problem but that's beside my question below:
Does Bitcoin Core process rpc-calls during or in between incoming new blocks validation? What has precedence? How does it work in (more) detail?

Possibly related: Increase timeout in client - see my answer to seemingly unrelated question using timeout=120 in Python RPC client. See bitcoin-rpcclienttimeout in https://docs.corelightning.org/reference/lightningd-config

Thanks RedGrittyBrick, this is very useful on the CLN side


text:
Why doesn't each node store only a portion of the blockchain?
Why doesn't each node only store a portion of the blokchain instead of the entire blockchain? Is it possible and why is it possible or impossible?

They store a part of the blockchain. When you edit your quesiton to change it to something along the lines of "only part of the block chain" or – even better – to something explicitly stating that the block chain should be distributed, please also state which advantages you expect from your idea.

@UTF-8: I mean,why does not node store blockchain as the distributed database? Why doesn't each node store a portion of a blokchain?

Your question is vague, but I assume your real question is why every node must store the full blockchain.
The answer to that is that they don't. In fact, nodes can choose to not store any part of the blockchain at all. The blocks are only needed when validating history, to build up the state of the global ledger (the "UTXO set").
Nodes that do store the full chain advertize so on the network, so that new nodes that join can fetch history from them. However, they themselves do not need the chain at all after validating it, and since Bitcoin Core 0.11, it is in fact possible to configure your node to throw away older parts of the chain to reduce disk space.
To summarize: the only data that every fully validating network node needs is its own validated copy of the UTXO set.
Given that information, maybe your question becomes: why isn't the UTXO set stored in a distributed fashion?
The answer to that is more complicated:

Trust model we cannot trust other nodes to store the UTXO set for us. The reason we went through the entire blockchain to build up the UTXO set is because the blockchain is auditable while the UTXO set is not. It allows us to check whether the entirety of history is valid. If we go outsource the UTXO set, and rely on other nodes to keep part of it, they could modify it to their advantage (for example, to make us believe they have more money than they do).
Performance Access to the UTXO set must be very fast. Faster nodes often are configured to keep large portions of it in RAM rather than on disk. The reason for that is that it directly influences the time we need to verify new blocks that are being broadcast. Delays in verification cause blocks to propagate slower across the network, which results in more stale blocks, and indirectly an advantage for a 51% attacker.

I guess what he meant was probably like: why can't bitcoin work more like BitTorrent, in which a brand-new node can bootstrap itself by fetching different parts of blockchain from different nodes which only keep a small portion of the blockchain individually.

The blockchain determines whether a transaction can be made. When a miner creates a new block, they are only allowed to include valid transactions. If they include an invalid one, their block is invalid which costs them a lot of money (currently the equivalent of currently over 8'000 USD). So each miner has a big interest to only include valid transactions. If they don't know which unspent outputs (those basically are the "coins") exist, they can't be sure which transactions to include.
Asking others about which transactions are valid, is cumbersome:

How do you know whom to ask?
Why would you trust them to not tell you a transaction is valid if it fact it's invalid? You shouldn't trust others if you don't have to.
It takes time. If you mine, you need to know which transactions to include. If that's slow, you can start mining with a few transactions and only hash with regard to the ones you already managed to get verified. If you find a block during that time, you lost the transaction fees you could've included if you included more transactions. Furthermore, it's bad for the users as fewer transactions are processed than would've been possible. If you include a transaction that's already in a previous block, you just lost the equivalent of over 8'000 USD.
It requires a good internet connection which currently isn't required.

Furthermore, nodes need to verify the blockchain. If you're given blocks, how do you know they are valid? Full nodes verify every block including all of their transactions all the way from the genesis block. Without knowing about the entire blockchain, someone could sent you trash so you mine on top of trash and everything you mine is thus trash. Ergo, you don't make any money.
Storing about 100 GB on a hard disk is so much easier than the hassle of any of the above.


text:
Why zero amount transaction outputs are kept in Bitcoin Core chainstate database?
They can be added as an input to a transaction, but what is the benefit of that? Isn't it the same as adding any random data to the transaction and pay the fee? You must know the key(s) to sign, and that may be useful to someone in some way, but does that justify the chainstate bloat? Are there any ways such an input can really be useful?

Are there any ways such an input can really useful?

It's not a question of usefulness, but necessity.
It is legal to spend such outputs in the Bitcoin consensus rules. If these outputs were not added to the chainstate, Bitcoin Core would reject such spends, violating the rules.


text:
Installing insight blockchain explorer - is bitcore mandatory?
What I have: bitcoind on Linux running.
What I want: to have insight api+ui local website running.
Earlier it was possible to add insight querying your bitcoind. Now it seems they made some bitcore base for insight.
If I install bitcore is it gonna replace bitcoind? How do they interact?
What's the best way to have local insight explorer running now?

Bitcore is a javascript library for interacting with Bitcoin. Bitcore is built on top of bitcoind and doesn't replace it.
Insight is probably the simplest way to run explorer-style queries if you've already got bitcoind set up.
Alternatively: I believe btcd and Toshi offer similar functionality, but with different implementations than bitcoind.


text:
What fields of a coinbase transaction can be empty?
Is it correct to say:

As the coinbase transaction doesn't need to identify an output, there
is no need for the input scriptSig.

Would a block be accepted in which the coinbase transaction has an empty input scriptSig field? Is there an example of such?
Can any other fields be left out, e.g. could the input TXID and VOUT be omitted (as have implied values)?

Specifically, given this example coinbase transaction (from here):
01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff4503ec59062f48616f4254432f53756e204368756e2059753a205a6875616e67205975616e2c2077696c6c20796f75206d61727279206d653f2f06fcc9cacc19c5f278560300ffffffff01529c6d98000000001976a914bfd3ebb5485b49a6cf1657824623ead693b5a45888ac00000000
Would it still be accepted as (input scriptSig removed):
01000000010000000000000000000000000000000000000000000000000000000000000000ffffffffffffffff01529c6d98000000001976a914bfd3ebb5485b49a6cf1657824623ead693b5a45888ac00000000
Or even (input TXID and VOUT removed):
0100000001ffffffff01529c6d98000000001976a914bfd3ebb5485b49a6cf1657824623ead693b5a45888ac00000000

Possible duplicate of https://bitcoin.stackexchange.com/q/20721/5406

Would a block be accepted in which the coinbase transaction has an empty input scriptSig field? Is there an example of such?

No, for unknown reasons there is a consensus rule that the coinbase input scriptSig must be between 2 and 100 bytes in length. Its contents however does not matter; it does not need to consist of valid opcodes even.
Since BIP34 activated in 2012, there is an additional rule that the coinbase scriptSig starts with a push of the block height (in a particular format). This forces all coinbase transactions to be unique.

Can any other fields be left out, e.g. could the input TXID and VOUT be omitted (as have implied values)?

No, because there is not even a way to signal missing fields.
Overall, a coinbase transaction is subject to these special rules:

It has a version number and locktime like other transactions, but their values are ignored.
It must have exactly one input:

Its prevout hash must be the all-zero hash (indicating it is "spending" new coins, if you will)
Its prevout index must be 0xFFFFFFFF, a value that cannot occur for normal UTXOs.
Its nSequence value must be present, but is ignored.
Its scriptSig must be between 2 and 100 bytes inclusive, and since BIP34 start with a push of the block height.

It must have 1 or more outputs. These outputs are treated like all other outputs; without them, there would be no way to signal where the created/mined coins go.

re. "all-zero hash" my interpretation is that the input prevout field should be zeros 0x00000... , not SHA256(0x0000)?

Also, the coinbase version and locktime can be used for arbitrary bytes? It would be interesting to find an example in the wild...

@Lee Correct. The prevout hash must be all zero bytes, not the hash of all zeroes.

Also, the coinbase version and locktime can be used for arbitrary bytes? It would be interesting to find an example in the wild...

@Lee check this question

It seems nSequence can be arbitrary too.

Yes, I said so above.

Apologies, intention was to correct/complete my previous comment.


text:
Is there a simpler form of BIP32-like key derivation?
I've read about BIP32 key derivation functions, and also:

Any Elliptic Curve could work in the BIP32 scheme. The only property
of a Curve that BIP32 relies on is that a * G + b *G = (a + b mod N) *
G, which is true for any Elliptic Curve.

(from Is BIP 32 Technology Cryptographic Curve Agnostic?.)
Are there simpler forms of weak-child-only key derivations?  (I'm guessing that the HMAC, splitting and concatenation is unneeded for a simpler/weaker scheme.  Such a scheme would just have a large number of weak children and no layers/tree.)
Is a simpler scheme possible?  (I'm asking about cryptography generally, not whether it's a good idea for Bitcoin.)

SHA256(entropy | index). 1st key = SHA256(entropy | 0x00000000), 2nd key = SHA256(entropy | 0x00000001),...

@CodingEnthusiast Could you expand that into an answer, please?  I can't understand how these keys relate to the parent keys, as they do not depend on them at all.  In particular I want to derive child public keys from just a parent public key, knowing that the owner of the parent private key can derive the associated child private keys.

It is a very very weak way of deriving child private keys from an initial entropy. If deriving pubkey in a weak way is desired then a similar scheme to BIP-32 for non-hardened children could be used to compute SHA256(parentPubKey | index) where child private key is the parent+child % N

Is parentPrivKey+index % N the private key for parentPubKey | index?  (I cannot see how the unreversable SHA256 can help here at all, but that may well be my lack of knowledge.)


text:
In compressed public keys, is the 2 or 3 the parity or the sign?
In different places I've read that the 2 or 3 is the sign or is the parity of the elided Y coordinate?
Which of these is it, or are the sign and parity equivalent in some way?

According to https://www.secg.org/sec1-v2.pdf, points are encoded in compressed format as:

0x02 + X coordinate: implicit Y coordinate is even
0x03 + X coordinate: implicit Y coordinate is odd

You can call this parity or sign - it doesn't matter; they're the same thing in a finite field (and arguably, both are inaccurate). As the coordinates are numbers modulo an odd prime p (the field size; p = 2256 - 232 - 977 for the secp256k1 curve used in Bitcoin), -x and p-x are the same coordinate. One of these will always be odd, and the other will be even. One of them will be negative and the other will be positive.
The convention is that "even" refers to the coordinate which manifests as an even number when seen as an integer in the range [0..p-1], and "odd" the opposite. This is an arbitrary choice, as another range could be chosen (such as [-(p-1)/2..(p-1)/2]) in which different coordinates would be seen as even or odd.
Yet, it helps distinguish. If (x,y) satisfies y2 = x3 + 7 mod p, so does (x,y) = (x,p-y). To identify the solution, the criterion of "even/odd when restricted to [0..x-1]" is used in practice.


text:
How do I compute a private key, if I have two messages signed using the same nonce?
If I sign two messages with the same private key and nonce, it compromises security, making it possible to calculate the private key.
I'm implementing a system (outside of bitcoin) where a message, with a unique property, must be signed only once.
In order to enforce this, I'm making it mandatory to use a hash of the unique property as the nonce.
This means that signing a two messages, having the same unique property, will give away the signer's private key.  (Which is not something they want to do, in this system.)
How do I use the secp256k1 functions to obtain a private key from two signed messages which use the same nonce?  (This is alluded to in this question.)

Presumably, if the unique property for two messages is the same, the content of those messages is also the same (wouldn't be much of a unique property, otherwise) - if your systems is designed correctly, you would end up with the same signature in both instances then, which would not leak the private key. For leakage to occur, two different messages must be signed with a shared R value.

@RaghavSood No, the messages might be {"election":2018,"vote":"green"} and {"election":2018,"vote":"blue"} with the unique property being just the election and NOT the vote.

This does not work. The reason the private key leaks is because attackers now know a relation between those two signatures. If they know the nonce up front (as required by nonce=H(msg)), attackers will learn the private key from one signature directly.

@PieterWuille I'm confused, I thought nonces were not secret information (at least in encryption).  Can you verify a signature without knowing the nonce for secp256k1?

The term nonce as used in signatures is confusing. It security requirement is not the same as those in encryption. It is not enough that they are just used once. They have to be completely unpredictable to an attacker, which implies they are secret (but even knowing one bit of information about each nonce in many signatures may be enough to recover the private key).


text:
OP_HASH256 of a negative number
transaction 9 in block 251684 spends transaction 25 of block 249957. the script to be evaluated here is:
OP_PUSHDATA0(20) 16cfb9bc7654ef1d7723e5c2722fc0c3d505045e OP_SIZE OP_DUP OP_TRUE
OP_GREATERTHAN OP_VERIFY OP_NEGATE OP_HASH256 OP_HASH160 OP_SHA256 OP_SHA1
OP_RIPEMD160 OP_EQUAL

the problem i have is with how to sha256 a negative number. so far i have the following:
op: OP_PUSHDATA0(20) 16cfb9bc7654ef1d7723e5c2722fc0c3d505045e,
                    new stack: [0x16cfb9bc7654ef1d7723e5c2722fc0c3d505045e]
op: OP_SIZE,        new stack: [0x16cfb9bc7654ef1d7723e5c2722fc0c3d505045e, 20]
op: OP_DUP,         new stack: [0x16cfb9bc7654ef1d7723e5c2722fc0c3d505045e, 20, 20]
op: OP_TRUE,        new stack: [0x16cfb9bc7654ef1d7723e5c2722fc0c3d505045e, 20, 20, 1]
op: OP_GREATERTHAN, new stack: [0x16cfb9bc7654ef1d7723e5c2722fc0c3d505045e, 20, 1] (since 20 > 1)
op: OP_VERIFY,      new stack: [0x16cfb9bc7654ef1d7723e5c2722fc0c3d505045e, 20]
op: OP_NEGATE,      new stack: [0x16cfb9bc7654ef1d7723e5c2722fc0c3d505045e, -20]
op: OP_HASH256,     new stack: [0x16cfb9bc7654ef1d7723e5c2722fc0c3d505045e, ???]

This is an extremely useful site for debugging bitcoin scripts: http://webbtc.com/script.
Bitcoin uses a custom way to serialize integers, shown here. This is confusing, but keeps the number of bytes on the stack small as possible. Let's walk through the basic execution:
exec : 20 OP_NEGATE OP_HASH256
stack:

exec : OP_NEGATE OP_HASH256
stack: 0x14

exec : OP_HASH256
stack: 0x94

exec :
stack: 0xaf462e245a0f463a93e23a54eb0b4d0d8da7a951e4d0b41015782a27e80a2e2c

The confusing step is really the OP_NEGATE. To negate an integer that fits into 1 byte without having the MSB set, you just XOR with 0x80. Hence, 0x14 ^ 0x80 -> 0x94.
The script debugger I linked to above verifies this result. This python code verifies the result as well:
import hashlib
import binascii

def SHA256d(string):
    bin = binascii.unhexlify(string)
    hash = hashlib.sha256(hashlib.sha256(bin).digest()).digest()
    raw = str(binascii.hexlify(hash))[2:-1]
    return raw

print (SHA256d("94"))

Prints:
af462e245a0f463a93e23a54eb0b4d0d8da7a951e4d0b41015782a27e80a2e2c

Oops, it looks like you beat me to it :-)

webbtc isn't heaps useful here as it writes -20 instead of 0x94. but your answer was very useful, thanks.

The value that is hashed is the single byte 0x94.
The stack contains vectors of unsigned chars. OP_NEGATE converts the int -20 into a vector of unsigned chars by calling CScriptNum::getvch() here:
stack.push_back(bn.getvch());

CScriptNum::getvch() serializes ints into bytes by taking its absolute value, encoding the result in little-endian form using the fewest number of bytes possible, and then guarantying that the most significant bit of the most significant byte is set if and only if the value is negative.
Therefore -20 is serialized into the single byte 0x80 + 20 == 0x94.
This can be further verified by completing the rest of the script:
$ python
>>> from hashlib import *
>>> h = chr(0x80 + 20)
>>> h = sha256(sha256(h).digest()).digest()
>>> h = new('ripemd160', sha256(h).digest()).digest()
>>> h = sha256(h).digest()
>>> h = sha1(h).digest()
>>> h = new('ripemd160', h).digest()
>>> print h.encode('hex')
16cfb9bc7654ef1d7723e5c2722fc0c3d505045e

As an aside: that's a rather, uhm, interesting script you've found....

heh yeah its interesting allright. thanks for pointing out where negative integers are handled in the original source code


text:
Mycelium Gear testnet confirmation
When I send payment to a testnet address generated by Mycelium Gear, it doesn't see the transaction even when it's included in a block.
Blockcypher.com, blockexplorer.com, blockchain.info and blocktrail.com see the transaction, though blocktrail seems to have indexing lag: it can't find the transaction by payment address, only by transaction id.
If Gear uses blocktrail API, this could be the reason.
Does anyone know a workaround, without using block chain exploration apis by myself?


text:
My transactions aren't being recognized in Blockchain
I have a Bitcoin core 0.11.0.0 on a server and it always worked fine. However, when I try sendtoaddress now, it generates a TXID but the TXID is not recognized in Blockchain (it says "Sorry we could not find any blocks or transactions matching this hash") and the transaction is stuck forever on the transactions list, with a status of 0 confirmations. No error is showed. Also, I can receive bitcoin with no problem in that install.
Have them went to limbo? Is there a way to recover these btc? And how to make my btc core work again?
Here are my latest 2 transactions (that aren't recognized by Blockchain):
{
    "amount" : -0.01000000,
    "fee" : -0.00034039,
    "confirmations" : 0,
    "txid" : "8307bac81b4eb857c16b1599436acc9d3f49e7d2253fc688cb9292fb47fe820d",
    "walletconflicts" : [
    ],
    "time" : 1485898700,
    "timereceived" : 1485898700,
    "details" : [
        {
            "account" : "",
            "address" : "1Gz8MxXiTgUo5Ycn7DgJziggkdGMZww191",
            "category" : "send",
            "amount" : -0.01000000,
            "vout" : 0,
            "fee" : -0.00034039
        }
    ],
    "hex" : "0100000002e1e327fea800282aa00921c43a26517acc64cb8c9c3261cb6dae7d0f668bf723000000006a47304402203d7af77e66f004f1a4cc83b8ac21ff6eabecbfe5e96905c0350d38a28cb8d5a802204101771b4a0350ccfe77b96f6b1b0e2f3ae1f620a521f38cb0fe1621cdbcce03012103a2e9b25ce7560da57259e076ad6101f4c7de67176cf29bbd7eed6a8117074e6bfeffffffbe1fe9973216a6e43cf1bdd4f7424d7e3e68620ad0889e693e1b92517616b14d010000006a4730440220627623dc6a66854b3ed71583d7a928c166bf3e9422bc7c25f15d4e43c156a7e9022024fca12c62c444d57004d8bbf1bf0960f3b64a90c9f62fa08ffb54c01920ee15012102cda9f734d610bd2c03b6c7ead5e493f8dd42bf310a13c34fd4dd57b4c6d5f042feffffff0240420f00000000001976a914af57bdd3af7711005e4b25ec58535c8693f6c63188acbfa20200000000001976a9146c15c3513c71e01d0b0d2a9b0c62fa038405fc3b88ac61e10600"
}

{
    "amount" : -0.05445000,
    "fee" : -0.00112313,
    "confirmations" : 0,
    "txid" : "4db1167651921b3e699e88d00a62683e7e4d42f7d4bdf13ce4a6163297e91fbe",
    "walletconflicts" : [
    ],
    "time" : 1485812669,
    "timereceived" : 1485812669,
    "details" : [
        {
            "account" : "",
            "address" : "1Jtw9hcx76C3CTwy5CoYETLu8eLSRbtFkT",
            "category" : "send",
            "amount" : -0.05445000,
            "vout" : 0,
            "fee" : -0.00112313
        }
    ],
    "hex" : "0100000007c5284391640ea0733c599a2c8c99389c7e00bef8bbd956b8cbbf907f19042f13000000006a473044022052d6e5682c47dc0872db24088e2829677cb60e9f5ac22d4db559e585b5a36060022037ae271d99b5667f9ba6e69db0f7632ecc5c6dc6581e161db258d89c96984301012103a7cca30a179d47b1056a378a82e0c9efb125139f0a1d3f8556a393d98bed27eafeffffff3c236945b01925e883cbcd0b212cc1c884cde4023a470a1a32e94bdd94ce8e9a000000006a47304402204a548e7d894aa48d2830c28ff47dbf25f63d868a7b9fd4e53d731ff1c9264d1202200da544ffbd0ba48631ce2c55abdcf76ff49ba28a44ffd59858bed6bfa911567d012103b1a903334b67e6c294ba03aeb415dc60fc6febfcc6770eb179c6e853d5937013feffffff1e5167deda344ecaf241049c2a4ba297c7105b4b4291b126c9564d7af2d71f5f000000006b483045022100d54fffb1cc35cbf198fd0847b7a30cd71d58815ad20bbb1436e4f82aea1d9897022076dc25e7cea43f985bae2c7c06c3616ed9a4d7d5f52030e1625964a8568cbeb5012103807e48bb9ff408e6ea92c3d2d0403d67d766696af258b551183bba2728eb995cfeffffff7a41d3b69ea4ed2dffe3ceccad9492832a80455c9004367dbd642c369e0aafc2000000006b483045022100e33e5b87640554350ca383aad327f38e5a3892e90165f60e96960216a2cfd7b1022041c8fe1e5f3c7cc4a099b2167d2e8854a6bb07a29d53baac613ac333b3ffe82d012103822cdda2be9a8b3a2bce5a7b3cf560ebb43989b32c0a8fa4b6d592aa9727bd9ffeffffff045b30cf4f348a9e758a481ff765c6175ab9406fb8c7b25810ed9979c6d0c660000000006b483045022100fdb2dc74573c58d6ff5bb4a9095e46524fc846797e37553b194df052f1588714022073ad03034f6ac806f765062e4a6942d9af1362322d9d24c201dfd2349348f15a012103df687d701e4ed1d5fe6e713407747f101df4bd921d98320013fad0be00623b17feffffff81349df39942b9cd41f72eafd89bfdc37fb76544c6b134e7880c124287a94ce7010000006a473044022006dad36e316bc1a08bc8ac3cbe93abd6cb2b804ab864adf351f0516713e11d0d02207658184768ee209f40a92c9f1bb7b59bfa2b63ddf268a2c5afd6ed6f50d51d60012102a5faeca03736ea8bf48dae4572f2b28cc2bb973c362605d9b285e7ac3e693e26feffffff7051a8e038a4a4f08da327b1dce671e55a67f99f5a6be8fd914f7a094c6163aa000000006a47304402201b057397f8ef1fcf7bd4f213672f855b1e7cabaa2f7e916c829fca3c92bffac0022039caa1a8b30693078c342a33747562e791646de09a45e9ebe143a17ebed9968f0121026ec8e0c398708d277007a3df35f7675d05f024ddfb224b44fee147b1566d7a71feffffff0288155300000000001976a914c44c8df549684f25467b9da3717c0aa5ac32abdf88accc420f00000000001976a914098e19275394d464a479cb8c22e92af8ff450e8c88ac98e00600"
}

They were all sent through bitcoin-cli sendtoaddress, which never gave me problems before. Also, this is not happening just to some transactions, but to all transactions that I do now using that installation.
Here is the transaction history from some transactions before this error appeared:
{
        "account" : "",
        "address" : "1DzHVQEmwhJcPFZHa4mqmEtpYDeMivV9HQ",
        "category" : "send",
        "amount" : -0.00425424,
        "vout" : 0,
        "fee" : -0.00009600,
        "confirmations" : 96,
        "blockhash" : "0000000000000000009af402250b020f9bc0b52a8499b35794e3fdefc6427ce2",
        "blockindex" : 1816,
        "blocktime" : 1485846115,
        "txid" : "5f1fd7f27a4d56c926b191424b5b10c797a24b2a9c0441f2ca4e34dade67511e",
        "walletconflicts" : [
            "4248afebb80fb18d7a5624be00b015298b4ad0be629d11bb535c6f0904a83d52"
        ],
        "time" : 1485804855,
        "timereceived" : 1485804915
    },
    {
        "account" : "",
        "address" : "1Jtw9hcx76C3CTwy5CoYETLu8eLSRbtFkT",
        "category" : "send",
        "amount" : -0.00350608,
        "vout" : 1,
        "fee" : -0.00034416,
        "confirmations" : 143,
        "blockhash" : "000000000000000001ad830c8eff6baa49c44c8e4485ac5b180dda19e81f7885",
        "blockindex" : 1449,
        "blocktime" : 1485818240,
        "txid" : "9a8ece94dd4be9321a0a473a02e4cd84c8c12c210bcdcb83e82519b04569233c",
        "walletconflicts" : [
        ],
        "time" : 1485804916,
        "timereceived" : 1485804916
    },
    {
        "account" : "",
        "address" : "1Jtw9hcx76C3CTwy5CoYETLu8eLSRbtFkT",
        "category" : "send",
        "amount" : -0.05445000,
        "vout" : 0,
        "fee" : -0.00112313,
        "confirmations" : 0,
        "txid" : "4db1167651921b3e699e88d00a62683e7e4d42f7d4bdf13ce4a6163297e91fbe",
        "walletconflicts" : [
        ],
        "time" : 1485812669,
        "timereceived" : 1485812669
    },
    {
        "account" : "",
        "address" : "1FLGvKgjkS69h1KVUpwKXB3Mmii1S3jrtc",
        "category" : "receive",
        "amount" : 0.00206634,
        "vout" : 0,
        "confirmations" : 99,
        "blockhash" : "0000000000000000012b03745b24b041a3fce605153cdd7d049923416d844d46",
        "blockindex" : 1525,
        "blocktime" : 1485845061,
        "txid" : "23f78b660f7dae6dcb61329c8ccb64cc7a51263ac42109a02a2800a8fe27e3e1",
        "walletconflicts" : [
        ],
        "time" : 1485816901,
        "timereceived" : 1485816901
    },
    {
        "account" : "",
        "address" : "1Gz8MxXiTgUo5Ycn7DgJziggkdGMZww191",
        "category" : "send",
        "amount" : -0.01000000,
        "vout" : 0,
        "fee" : -0.00034039,
        "confirmations" : 0,
        "txid" : "8307bac81b4eb857c16b1599436acc9d3f49e7d2253fc688cb9292fb47fe820d",
        "walletconflicts" : [
        ],
        "time" : 1485898700,
        "timereceived" : 1485898700
    }
]

Edit: I ran bitcoind -zapwallettxes and I have my funds back and I am able to send bitcoin again. But why did it happen and what can I do to prevent it from happening again? I mean, I'm running bitcoin core in a server for a site that I'm building, so it really can't happen when it's up.
Edit 2: The problem happened again. I disabled the router's UPnP (afraid it had some problem with BTC Core running as a node and having 12 connections). I took a transaction I made that hasn't been recognized by Blockchain (.05 BTC, .001 as fee) and got its raw through getrawwtransaction, copied it to Blockchain's pushtx tool and this is what I got:
An outpoint is already spent in [DBBitcoinTx{txIndex=217064367, ip=5.9.28.10, time=1486082953, size=223, distinctIn=null, distinctOut=null, note='null', blockIndexes=[], nTxInput=1, nTxOutput=2}, DBBitcoinTx{txIndex=217073562, ip=82.134.66.146, time=1486085288, size=223, distinctIn=null, distinctOut=null, note='null', blockIndexes=[], nTxInput=1, nTxOutput=2}, DBBitcoinTx{txIndex=217229864, ip=46.183.223.186, time=1486124142, size=519, distinctIn=null, distinctOut=null, note='null', blockIndexes=[], nTxInput=3, nTxOutput=2}, DBBitcoinTx{txIndex=217230789, ip=5.189.176.14, time=1486123915, size=370, distinctIn=null, distinctOut=null, note='null', blockIndexes=[], nTxInput=2, nTxOutput=2}] [OutpointImpl{txIndex=217057962, txOutputN=1}, OutpointImpl{txIndex=217227286, txOutputN=0}, OutpointImpl{txIndex=217227286, txOutputN=1}, OutpointImpl{txIndex=217066625, txOutputN=0}, OutpointImpl{txIndex=217066625, txOutputN=1}, OutpointImpl{txIndex=217420371, txOutputN=1}]

So are my transactions not being propagated properly? If so, how do I find out the reason and how to prevent it?
Edit 3: I think I found the root of the problem.
I used decode-tx tool to decode the hex I got on  getrawtransaction and it showed:
{
   "lock_time":451510,
   "size":1257,
   "inputs":[
      {
         "prev_out":{
            "index":1,
            "hash":"d134ef1458216fb826d3f1b474127946e1ca11f01a87fc422068917128d2141d"
         },
         "script":"47304402201d32169037ed4d027aa158b95a6168c9b24fb4652ab976a3d385a57bba34c53102206b80c850c8d67de47a058cb0c6d95ca00b72a0e18e4cc7177203d0be176661120121035d6dd7e514d0b9ca5dc90363982910f0602e72cd458ee3fff3b88d0742708909"
      },
      {
         "prev_out":{
            "index":1,
            "hash":"3b76aaf5d8e8dd5cf93d81009c670fc2e98f2a8b60726027fa792e240f732534"
         },
         "script":"47304402202f0824ccbe197bcf3cdd454ae109aa7acd964e476fba9c09eff505dcd6087a3102202d541ee6650448343e094f545f22a328b52ff945a48131a3712025ad984e029a01210318bc5f03c55dc90016d355f0740042b0c5bd5c6d381ea03be0de25a619c26247"
      },
      {
         "prev_out":{
            "index":0,
            "hash":"bded0c615a5868535ba587a06769f680d7500dda842ff2f843d11627a6b90a52"
         },
         "script":"473044022039513d1e0b37505fecef6de2593ef87766bdab08178e5fcd227a6eae0e444fc102202adacee6ef7aa8c6c26a40e253e36296ff758c458a18355b5a0fe91be1d01d3c0121039ed4708293468074f437a8cb7574bb87efb4787c9cca68d44b257529b9ee5f7b"
      },
      {
         "prev_out":{
            "index":1,
            "hash":"bded0c615a5868535ba587a06769f680d7500dda842ff2f843d11627a6b90a52"
         },
         "script":"483045022100ae06f71c73511992128a7ba2bc5f0f6b93091458910df5ae9c09e510485f5a34022042830e4f3f54d4163f3570d7422d60bb397eb2a74fb76423df5efbafd449b72b012102e0bec5b4f66054a707bf7af1d7f244be82d62ccd6574f736ff0c6e13bb5139b2"
      },
      {
         "prev_out":{
            "index":0,
            "hash":"1fa7e497f2cd31d4e9ff8ace167a8f3e458efe8a740736a3029c9a2772595258"
         },
         "script":"47304402200915d67fe78538661edaad9703d5e515a97ee4792377ea95657a95699d05296502207dc64db869c686081e6babec063907a7965f4d980abce7c11914d4486fce781b01210256248c0aabdf513a7a11729b65401bae90884a6830a266198b1bf6db7f2aea41"
      },
      {
         "prev_out":{
            "index":1,
            "hash":"847901326c7300463266c25ceb3cb13d82a4d11027487f106be09dd72421727f"
         },
         "script":"483045022100d50660416846fe3ed5f327d90b7daa97397b509230d3ec6b264f9367e7c09f3c02206aed59b88f51c7deec5ec2d183d071a7c253894701aa03e4fd02ea2ec23a71bb01210208cabe6be609c05278ea26393841ae517bd8e4c575fb637a571ea94fffda15d9"
      },
      {
         "prev_out":{
            "index":1,
            "hash":"a34ee9e1f370c128cbf4bf454b534e0f3dce05ccb39b069567ff5230ab9c1153"
         },
         "script":"483045022100cb906854c02938d78d930ec79403aa7cc1c81f8399091afb30f791207eddbc1602200f91e7d1a1942ff25cdad4e844c92bf574b02d4093960aa431adb63084a036a80121024830e992b2f652309d23d9a821755f69f3f27c670a53e8b35a32feb8cbda1a50"
      },
      {
         "prev_out":{
            "index":0,
            "hash":"8d2c13c9f17312d71c25119520ba74e5a3a03b9e6d7a651612be834135e2f224"
         },
         "script":"473044022065c557aa65825d6fe3feab9be00f3653f895eafcff2882f4a014715c46a5116d02202b1066e9c47eba2a6d951549ea09a4ed2b6d1fe03840e5cdbfcaee04b67d1c6f012103062530388cf0bf7d0a61ccd0bcb114d63970adf30b1c5b4be9272c89f503d292"
      }
   ],
   "version":1,
   "vin_sz":8,
   "hash":"25cb7cec2db1b4660bd8020614b88224e13fc805200e7eea624a3a91aa5869f6",
   "vout_sz":2,
   "out":[
      {
         "script_string":"OP_DUP OP_HASH160 af57bdd3af7711005e4b25ec58535c8693f6c631 OP_EQUALVERIFY OP_CHECKSIG",
         "address":"1Gz8MxXiTgUo5Ycn7DgJziggkdGMZww191",
         "value":5000000,
         "script":"76a914af57bdd3af7711005e4b25ec58535c8693f6c63188ac"
      },
      {
         "script_string":"OP_DUP OP_HASH160 7ff04e9b54b7028182c9ec930c2abe814850535c OP_EQUALVERIFY OP_CHECKSIG",
         "address":"1CfUhBzdSWMB1tfghYyAKECfbCHTwAa5Xy",
         "value":1074498,
         "script":"76a9147ff04e9b54b7028182c9ec930c2abe814850535c88ac"
      }
   ]
}

Some of the inputs it shows are already spent:

https://blockchain.info/tx/a34ee9e1f370c128cbf4bf454b534e0f3dce05ccb39b069567ff5230ab9c1153
https://blockchain.info/tx/d134ef1458216fb826d3f1b474127946e1ca11f01a87fc422068917128d2141d
https://blockchain.info/tx/8d2c13c9f17312d71c25119520ba74e5a3a03b9e6d7a651612be834135e2f224
(was this last one really spent?)

So it looks like some outpoints were already spent and my wallet does not know it. How can I "tell" it that they were already spent (preferably without downloading the entire blockchain again, as it delays about 1 week)? getwallettx, getwallettxes and rescan didn't help. Also, when I run -rescan, the balance is about 0.06 BTC. When I run -zapwallettx(es), the balance is about 0.1 BTC. Which one is true?

0.11 is 1.5 years old, and things change quickly in this space. Try upgrading to the latest version (0.13.2) first.

zapwallettxes simply made your client forget those transactions - they're not confirmed, your client just thinks you got your money back.

I recompiled BTC Core's latest version on the same machine (keeping wallet.dat) and tested again. Results in OP. Unfortunately no luck :(

The 'An output is already spent' warning means that bc.i thinks you're creating a double spend. That's likely because it still known about the first transaction you sent, despite your wallet having forgotten about it because of the zap.

I think I found the root of the problem. OP edited. @PieterWuille I really had to doublespend a transaction some days ago (the btc core was set with low fees)...

Blockchain.info has been not up to date a few times in the past months. Have you tried looking at your transactions on another blockexplorer? If your result for -rescan and -zapwallettxes differ, I'd say there are still unconfirmed transactions created by your wallet floating around.

Hi, @Murch, thanks for your response! I've tried in multiple blockexplorers and all gave the same results (transaction not found). As an emergency act, if I just delete wallet.dat (and recover the funds later in another wallet), will bitcoin core work well again? Is this a safe way to completely reset ("factory reset") bitcoin core?

The wallet.dat stores your private keys, which is what allows you to spend your money. The wallet.dat is the one thing that you should never delete or lose. – If your Bitcoin Core is synchronized, you should be able to see which transactions remain unconfirmed. Then, you either wait for them to confirm, or you replace them by a new transaction with a higher fee. This would of course lead to some transactions never confirming as only one from two conflicting versions of a transaction can confirm.


text:
Help! Unconfirmed transaction for the past 12 hours!

I am a noob in the bitcoin world, I made a transfer of 0.0577201 BTC , with a transaction fee of 0.000221 BTC,  but the transaction has not yet been confirmed. Please help me! Here is the link for the transaction: https://blockchain.info/tx/72b2b1191693f5d916daf899ef6e43171599b065779e394fc7096c8fc5262091

It confirmed now.

There is currently a backlog of about 69,000 unconfirmed Bitcoin transactions and only the transactions with the highest fees will be confirmed until the backlog clears.

Please help me!

For similar situations going forward, you have a few options:

Submit your transaction to the ViaBTC Transaction Accelerator
If you control either of the receiving addresses, send a new transaction from one of them with a much larger fee, encouraging a miner to pick up the whole chain of transactions (Child-Pays-For-Parent).
Wait. The transaction should be either confirmed or removed from the memory pool within about 72 hours.

21.co has a good fee estimator for future transactions.


text:
MtGox Daily Withdrawal Limit
Simple question about Mt Gox's terms for "unverified" "level 0" accounts:
See this URL: https://support.mtgox.com/entries/20919111-AML-Account-Statuses
It says that the daily limit for USD "or equivalent" is 1000USD.
And daily limit for bitcoin is 100BTC.
So my question: if current exchange rate is 100USD/BTC, I can withdraw 100BTC from MtGox to my personal wallet in one day, and then exchange the 100BTC in my wallet for 10,000USD at another exchange, potentially, assuming that another exchaneg will offer me 100USD/BTC?
On other words, is the daily limit of 1000USD only for withdrawing directly to USD?  Or does it mean that the equivalent of 1000USD (10BTC) is the limit no matter whether I'm withdrawing USD or coins?

Yes, but you'll have to deal with whatever AML rules the other exchange has.

OK, but I can always withdraw up to 100BTC per day from mtGOX to any bitcoin address, and then the receiving party deals with AML?

The USD limit is only for funds being removed as USD.  They do not value BTC in USD to see if it crosses the limit.


text:
What are the risks of keeping USD in an exchange account?
Bitfloor, mtgox and others allow me to keep a balance of both btc and usd that is not being traded. So the usd is apparently in the hands of the exchange company rather than at my bank. This is an advantage because it allows me to quickly buy in and out of btc without waiting for a wire transfer or paying for something like bitinstant.
Is my usd held by the exchange protected in anyway from hacking or other losses, the way it is at my bank?

Hacking of what? Your account, or the entire exchange?

Well this question assumes I'm smart about protectin my login credentials.

Each exchange can be different.  Essentially with every bit of funds held at an exchange, either fiat or bitcoins, the account holder becomes an unsecured creditor of the exchange.
Regulated exchanges of other securities, commodities and currencies and brokers of those exchanges have clear instruction to keep customer funds segregated from the operating funds so that those funds are not at risk if the exchange itself runs into financial issues.
Whether any of these regulations affect a Bitcoin exchange or brokerage may vary based on jurisdiction.  It is possible these exchanges are operating without any regulatory oversight and thus no guarantee that they are holding all customers funds without lending those funds out or using those to fund operations.
An exchange that accepts deposits from an account-to-account (A2A) transfer could help lessen the risk.  For instance, Camp BX and Mt. Gox accept deposits from Dwolla.  So holding dollars at Dwolla until they will be used for trading is one approach to lessening the risk of keeping the funds at the exchange the whole time.   BitFloor accepts CapitalOne 360/ING transfers, as another A2A example.  And multiple exchanges accept transfers from OKPay.
Additionally there are site security concerns.  If the service gets hacked, or an individual account gets hacked, those funds could be used to buy bitcoins and withdrawn -- with the losses assessed to the account holder.  Use of two-factor authentication is essential a core requirement.

Bitfloor's shutdown offers a great example of the risk of holding fiat in an exchange:
Withdrawal can be cumbersome. Bitfloor requires you to submit a photo ID and a checking and routing number.
Withdrawal can be risky. If you give Bitfloor what they require, they have almost all the information someone needs to conduct ID theft.
Withdrawal means you lose your anonymity. All transactions that originated from Bitcoins bought via Bitfloor can be traced back to your real identity.
Withdrawal can be SLOW. If you read the thread above, you can see that users still haven't received their USD after months of waiting.
I would try at all times to have the minimum amount of fiat in any exchange.


text:
Bitfloor's "average price" for buys
When you buy bitcoins with USD or other currency on Bitfloor, it quotes you an "average price" per bitcoin and says the "actual price may vary" once the order is completed.
The "average price" stated changes periodically, but it always lags the present actual market price on Bitfloor.
Question 1: when I hit the BUY button, what price do I get?  The "average price" or the current market price on Bitfloor?
Question 2: how does bitfloor calculate this "average price"?

The average price is calculated by taking into account the current market conditions (i.e. other buyers/sellers) against the amount you wish to buy/sell. Given that the market is always changing, your actual average price may vary. When you finally place a market order, the system will execute against the best available prices at the time.
If you would like to have more control over the pricing and order execution, use limit orders. These may not execute immediately (depending on how you price them and their size) but you will be able to control those factors.


text:
Bitcoin HEISTS: Has USD, EUR, etc ever been stolen from an exchange?
Many bitcoin exchanges allow users to store two balances: one in bitcoin and one in a fiat currency like USD.  This is convenient because it allows one to trade in and out of the bitcoin market without incurring the fees and delays of moving money in and out of the exchange.
There have been numerous cases of theft from bitcoin exchanges, but have any of them lost users' fiat funds stored in this way?  Or have only bitcoin balances been depleted by thieves?
Have any of the major exchanges made any statements (please link!) about how secure a user's fiat currency stored by them is?

Someone that gains access to an exchange's system has several options:

transfer out BTC
transfer out fiat
manipulate/corrupt the exchange

Option 1 is fairly trivial to do nearly anonymously. Governments and lawyers are good at tracking down people that take option 2. Option 3 isn't the most straightforward way to profit.
Why would anyone take the risks with #2 instead of just doing #1?

Can anyone skilled in lawyering indicate whether lawyers/govts would be compelled to track down "#2 thieves", and whether the victims (users) could expect USD to be returned to them if the thieves are prosecuted?

@themirror, perhaps I misspoke when I said governments and lawyers... I don't know who would be responsible or if the exchange would contract out an investigator or whatever. I only meant to say that there are people/organizations/whatever that are pretty good at tracing/tracking financial transactions to find the end account holder. It gets way harder with BTC.


text:
Why should I keep my coins in an exchange?
If I have 1BTC in a wallet anywhere, which I bought at mtGOX for 20USD, and now that 1BTC is worth 130USD on mtGOX, I can simply send my 1BTC from the wallet to mtGOX and withdraw to a bank as USD and have netted over 500% profit after fees right?
So my question is: why would I keep my BTC in the "live" trading of an exchange rather than just keeping it in a Wallet?  There's no difference to me right?

There is a difference, from a security perspective.
When you deposit your bitcoins to MtGox they credit your website account with what are essentially MtGox-bitcoin-credits. These are not bitcoins. If they chose to operate with a fractional reserve system (find out more here), they may have even spent some of the bitcoins, and you wouldn't know unless there was a run on the exchange (like a run on a bank).
Further, MtGox may or may not be able to reimburse you if they somehow lost all your bitcoins. They probably wouldn't reimburse you if someone managed to obtain your login details and withdraw your balance.
However, storing bitcoins yourself has risks too; they can be stolen if your computer is hacked (through malware or a virus). You can however reduce the risk of that to essentially zero by generating an address from an offline computer, printing the details, destroying all electronic traces of it, and transferring your balance there. Of course, then you need to protect your piece of paper...
There is also a difference when it comes to liquidity. If you have the balance stored with MtGox, you have the ability to trade it immediately (provided the website is online).

And if I have my coins stored on a piece of paper and want to sell them for use immediately on mtgox then it might take a few hours for the coins to transfer in?

@themirror, yes that's correct. I'm not sure what their policy is for deposits, but it's probably to wait for at least 6 confirmations before you can trade them.

@themirror If you have coins stored on a piece of paper you can use "redeem private key" function of MtGox, I believe it's near instant.

authors: Сергей Чуркин