text:
Walletnotify not working
Here's my bitcoin.conf:
server=1
rpcallowip=127.0.0.1
rpcuser=userrunningbitcoind
rpcpassword=passwordofuserrunningbitcoind
rpcport=7788
walletnotify=/home/userrunningbitcoind/transaction.sh %s

And here's my transaction.sh, which I've tried running manually already. This works without issue on its own:
#!/bin/sh
curl -d "txid=$1" http://127.0.0.1/api/process-payment

What else can I try to get walletnotify going?
EDIT:
I've additionally found this in my debug.log:
2018-01-19 05:49:22 Binding RPC on address 0.0.0.0 port 7788 failed.
2018-01-19 05:49:22 No rpcpassword set - using random cookie authentication
2018-01-19 05:49:22 Generated RPC authentication cookie /home/userrunningbitcoind/.bitcoin/.cookie

It should be working, note that it will only trigger for transaction to addresses in your wallet. Have you made a transaction ?  To debug what is happening, you can see .bitcoin/debug.log

@darkknight I've made transactions to attempt to test it. The debug log is enormous, and seems to be filled with unrelated messages. How would I filter it appropriately?

Search for transaction.sh in the log ...it should show up

grep -i "transaction.sh" debug.log returned no lines. Is it possible that my bitcoin.conf is being ignored?

@Fibericon have you tried connecting to the RPC to see if the other settings have taken effect?

How would I test this?

@MeshCollider It's apparently not binding RPC. Please see my edit.

Yeah if it is failing to bind the port its more likely a server issue, try a different port maybe

@MeshCollider I tried a few different ports, but no dice.

What about the default port? Maybe it's a permission issue

@MeshCollider Looks like it defaults to 8332 (when I remove the port line in bitcoin.conf), but it still gives the same error in debug.log

Have you tried curling it? walletnotify=curl /home/userrunningbitcoind/transaction.sh %s

Problem is here:
2018-01-19 05:49:22 Binding RPC on address 0.0.0.0 port 7788 failed.

if you are using rpcallowip, you need to also use rpcbind. By default, the recent BTC versions are limited to 127.0.0.1 so there's no need to use rpcallowip or rpcbind if your BitcoinD and your app are running on same server.
So either you remove rpcallowip from config, or add rpcbind=0.0.0.0 and then restart bitcoind. Also, everytime you restart bitcoind, you need to load the wallet, as default loading of wallet is removed.  So if wallet is not loaded, walletnotify will not work.

Remember to restart bitcoind again after you make any change in bitcoin.conf file. It seems as if you set an rpcpassword in the conf but didnt restart the bitcoind service. You have everything set up properly i guess.


text:
What's the difference between anonymous and pseudonymous electronic money?
You often hear people say that Bitcoin is a pseudonymous system, and that it's not anonymous. What is the actual difference in the context of electronic money?
Pseudonymity means that you can see the payment history of a pseudonym (eg an address), but there's no way (in the system itself) to link the pseudonym to a person.
What would an anonymous system look like? I am aware of Chaumian e-cash (https://bitcoin.stackexchange.com/a/10666/69518), but what properties must anonymous electronic cash have? Is it what Andrew Poelstra alludes to:

Total anonymity, in the sense that when you spend money there is no trace of where it came from or where it's going, is theoretically possible by using the cryptographic technique of zero-knowledge proofs.

So you can see a payment, but you can't see where the money is coming from or where it's going (just like chaumian e-cash). Would anything less than this be considered not anonymous? I'm not looking for solutions, only what properties are needed for an electronic money system to be anonymous.

Would anything less than this be considered not anonymous?
That depends who is doing the considering.
I suggest thinking about degrees of anonymity rather than absolutes such as anonymous vs psedonymous vs identifiable. There are systems which claim to offer more anonymity. There may be trade-offs in other aspects of security or in other factors such as convenience.
what properties are needed for an electronic money system to be anonymous.
What reduces anonymity in Bitcoin is the property you already identified. The public nature of the record of addresses used in transactions. So removing that property would increase the degree of anonymity. An example might be the gradual transition in usage of transaction types from ones that explicitly include a public key in the locking script towards types with locking scripts that only disclose something like a hash of a script? I don't know if the issue is fundamentally tied to the UTXO model though.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How could a 2 byte witness program make sense?
BIP141 requires the witness program to be 2-40 bytes for segwit versions 1-16 (for v0 it must be either 20 or 32 bytes).
When would a 2 byte witness program ever make sense?
The 16 bits entropy is nearly non-existent, so it can't make sense for cryptographic purposes.
I can come up with some possibly silly suggestions, wild handwaving:

The usefulness of 2 byte witness program is unknown at present time, but BIP141 doesn't want to unnecessarily close doors. It might prove useful in the future.
Use it as some kind of 0 value OP_RETURN but the data is stored in the witness of the spending transaction.
A certain witness version and a two byte witness program, for example 0x52020000, on a 0 value output activates some new rules for encoding outputs, possibly to enable sub-sat amounts, or some fancy privacy feature. The actual output is attached elsewhere in a similar fashion as in segwit.
Use it as an "exodus" address to recreate all spent UTXOs to another system external to Bitcoin.

I don't have any concrete cases which could benefit from such short witness programs. It seems very unlikely to me that there can be one with say less than 128 bits programs (as anything less would appear to be unable to achieve the typical 128-bit security level aimed for in Bitcoin), but your example of using it to trigger additional rules is an interesting one.
I believe the original rationale for the lower end of the range of valid witness program lengths is that 2 bytes is enough to guarantee no ambiguity of how the program would be pushed (some 1 byte values can - and according to standardness, must - be pushed with OP_n, and dealing with those would have complicated the matter).

Gregory Sanders proposed Segwit Ephemeral anchors, in which a 2-bytes witness program in a transaction's output to make it opt in to different relay rules (BIP, Optech).

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How does the Bitcoin client check for double spend?
When the Bitcoin client looks at the input of a transaction, it know the hash of the previous transaction from which the funds are coming, however then, how does it know that this output has not been spent already? Does it need to scan the whole blocklist for another transaction spending that same output? Isn't that very inefficient?

Bitcoin-Qt marks outputs as spent in the database once they are spent. To check whether an input's prev-out is spent, Bitcoin just needs to look up the transaction and check whether the output is marked spent.
Other ways of doing it are to have a set of all unspent outputs and check whether the prev-out is in this set (future versions of Bitcoin will do something like this), or to index inputs by their prev-outs.

Thanks for clarifying. So the sum of all unspent outputs is the balance for an address?

@bananer Yes, that's right.

Where does this appear in the code?

@KinnardHockenhull For 0.7.x and below, search for vSpent.

So I imagine there is some sort of hashtable indexed on the transaction hash, with some flag indicating if it was spent.

It is similar to the question here
Suppose A, a peer, receives 'B' BTCs from various transactions: B1,B2,B3, etc. Also, A had spent 'S' BTCs in various transactions S1,S2,S3, etc. Now, whenever a new transaction is initiated from A to C (eg: A wants to transfer X bitcoins to C), the transaction has to be verified as follows.

The full nodes have the UTXO (Unspent Transaction) details. The full nodes go through this UTXO database and verify if S1,S2,S3 are unspent or not. UTXO's size is around 512 MB. So, not difficult to go through all of the UTXO.
Verification can also be done via SPV nodes, a lightweight client. These SPV nodes do not have the UTXO details. However, they have the header information of the entire blockchain. So, a new transaction is initiated, SPV client requests the full node to give information of the S1,S2,S3 transactions. Full node gives this information in the form of Merkel Trees. SPV node verifies the validity of these Merkel trees using the blockchain header info. The blockchain header would not change for all the transactions which are 'old enough'. As a conservative rule, people usually wait for at least 6 more blocks to be built on top of this present block, before delivering their goods or service.

Please refer to this link as well-  Good article on how individual transactions are processed.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Order of transactions within a block
Does the Bitcoin protocol guarantee that within a valid block, if transaction B spends the outputs of transaction A, A will always precede B?
Or is it possible to have B before A?

The Bitcoin protocol requires transactions to appear in topological order in the blockchain. Given two transactions A and B where B spends an output of A:

Both A and B may be included in the same block.
A must precede B in the transaction list.

Colloquially, a miner will never include a transaction in a block template until the transaction's requirements are met. Since B spends an output of A, B is not valid until A has created said output. To satisfy this dependency, B can only be included after A was included.
Also see: Can multiple transactions transferring the same bitcoin be done in one block?

I cannot understand this answer it seems to contradict itself.  It says 1) all transactions in a block happen at the same time, 2) A & B can be in same block but they will stand in chronological order.  2) means that 1) is not true, 2) means that transactions in a block are considered to happen in the order they are listed, not at the same time.  Fix!?!?

They don't have to be in different blocks, i.e. they will all be timestamped at the same time. It doesn't even matter if a miner receives knowledge of B first. However, if a miner were trying to first determine the validity of B, he would find it invalid, as he doesn't know about the transaction output created by A yet. Thus, even though the transactions can happen at the in the same block, the protocol enforces that the transactions will be written out in the block in the correct order.

I am curious about unrelated transactions - how are they ordered? Surely a timestamp is insufficient? If it is a UUID, how is it ordered?

They don't need to be in a specific order, but some miners seem to keep them in the same order they selected them into the block, i.e. ordered by the transactions' fee rates paid.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Broadcast raw transaction without input validation
What tool/software can I use to broadcast a raw transaction, which would not reject a transaction because of invalid or double spend?
Bitcoind with RPC has a validation step which rejects any raw transaction where the inputs are not known, or if the inputs are double spending.
Also, I don't want to rely on a third party service like blockchain.info.

AFAIK It does not exist. Looked for this myself as well. You have to develop your own tool.

Seems like the easiest way might be to just comment out the validation code in bitcoind and recompile.

What tool would do this? No tool, because if it was possible, which it is not, you would be able to break Bitcoin.

Thanks god double spend attempts don't break Bitcoin, that would make it rather useless.

And yes, it is certainly possible, it happens a few hundred times per day: https://blockchain.info/double-spends.

Turns out there are many ways to do this:

Node.js: Bitcore lets you broadcast a raw transaction to the network without any kind of validation, see the "broadcast a transaction through P2P" example.
Python: Protocoin is a library that can do that.
Java: BitcoinJ can do it.
C#: Bitcoincsharp, which is a port of BitcoinJ can also do it.

There is absolutely no reason trying to broadcast invalid transaction.
It is wasting your time and your traffic.
Your peer(s) will check it and throw it away. They will not relay it to network.
They also can ban your IP-address for some time (1 day?)
May be you want to send valid but non-standard transaction?
Or may be you want to have a custom tool, which sends transactions to a network?
    // [...]
    void NetSocket::pushTx ( const QByteArray& data )
    {
      const MyKey32 key ( data.constData ( ), data.size ( ) );  // calculate txid
      outTx.insert ( key, data );                               // store data
      write ( "inv", invPacket ( MSG_TX, key ) );               // send inv packet
    }
    //--------------------------------------------------------------
    void NetSocket::write ( const char* type, const QByteArray& data )
    {
      socket -> write ( packet ( type, data ) );
    }
    //--------------------------------------------------------------
    void NetSocket::procGetdataPacket ( const QByteArray& data )
    {
      Stream d ( data ); d.skip24 ( );               // i do not check header now
      const int count ( d.readVar ( ) );
      xassert ( count == 1 );
      const quint8* ss = d.readAdvance ( 36 );
      const quint32 tag ( *(quint32*)ss );
      const MyKey32 key ( ss + 4 );
      xassert ( tag == MSG_TX );
      xassert ( outTx.contains ( key ) );
      write ( "tx", txPacket ( outTx.value ( key ) ) );
    }
    // [...]

I think it could very well be that the various bitcoin implementations have slightly different notions of when  to accept a transaction or not. the only way to find out is to be able to send a transaction unfiltered to lots of different nodes.

Of course, there is a very good reason. You may be spending unconfirmed outputs, which your client doesn't know about, but still exists on the network.


text:
Transaction with OP_RETURN rejected by bitcoind 0.9
The raw transaction below is being rejected by bitcoind 0.9.1 (error: {"code":-22,"message":"TX rejected"})
01000000015594a8c1f84b926e84d70c3a3d5e517e0c12dc07cb1a774b587121fef08f91b8000000006a47304402203ea43861e1e22603873f49e73c6ace08d32ef024e1feaca5a6242f74bf152810022044aeee3231e81d927accf20b0b0106ec327728b6a61aedd75596c5c12e5d9999012102add59262e234c0045d1f6a3d40a144b47ea0b4214916f55fb6029a079cc0b3cbffffffff0358020000000000001976a9140f763005e063382f8f4138f75cdc64d14f8ec16f88ac0000000000000000196a054f4101000111753d687474703a2f2f7265642e70696c6cd8720000000000001976a9140f763005e063382f8f4138f75cdc64d14f8ec16f88ac00000000

And the following error appears in the logs:
ERROR: AcceptToMemoryPool : nonstandard transaction: scriptpubkey

The decoded transaction uses OP_RETURN and looks like this:
{
   "lock_time":0,
   "inputs":[
      {
         "prev_out":{
            "index":0,
            "hash":"b8918ff0fe2171584b771acb07dc120c7e515e3d3a0cd7846e924bf8c1a89455"
         },
         "script":"47304402203ea43861e1e22603873f49e73c6ace08d32ef024e1feaca5a6242f74bf152810022044aeee3231e81d927accf20b0b0106ec327728b6a61aedd75596c5c12e5d9999012102add59262e234c0045d1f6a3d40a144b47ea0b4214916f55fb6029a079cc0b3cb"
      }
   ],
   "vout_sz":3,
   "hash":"4dd1b7130045f84e7dd75a03dc802063f76687812cf877dacff2a645a1b0991b",
   "vin_sz":1,
   "out":[
      {
         "address":"12QkihKUyE1hAkv7wmaMj6V3QiN8FfMvpv",
         "script_string":"OP_DUP OP_HASH160 0f763005e063382f8f4138f75cdc64d14f8ec16f OP_EQUALVERIFY OP_CHECKSIG",
         "value":600,
         "script":"76a9140f763005e063382f8f4138f75cdc64d14f8ec16f88ac"
      },
      {
         "script_string":"OP_RETURN 4f41010001 753d687474703a2f2f7265642e70696c6c",
         "value":0,
         "script":"6a054f4101000111753d687474703a2f2f7265642e70696c6c"
      },
      {
         "address":"12QkihKUyE1hAkv7wmaMj6V3QiN8FfMvpv",
         "script_string":"OP_DUP OP_HASH160 0f763005e063382f8f4138f75cdc64d14f8ec16f OP_EQUALVERIFY OP_CHECKSIG",
         "value":29400,
         "script":"76a9140f763005e063382f8f4138f75cdc64d14f8ec16f88ac"
      }
   ],
   "size":259,
   "version":1
}

However OP_RETURN is standard in Bitcoin 0.9, so why is this being rejected?
Note: This works in testnet.

Given that bitcoin-qt is the reference  implementation, if it thinks your transaction is nonstandard, then it's nonstandard. (Note that nonstandard transactions are allowed on testnet.)

@NateEldredge Please post this as an answer, not a comment :) .

Please elaborate on why it is not standard...

@Flavien: I would have to single-step through the source code and see exactly why the check for standardness fails.  I don't have the time or inclination to do that at present, but I encourage you to do so.

I suspect that the second output is being rejected because it contains an OP_RETURN with two data pushes.  From browsing the code, "Solver" in script.cpp only categorizes output scripts as "standard" if they meet one of a limited set of templates.
All such templates containing OP_RETURN:

OP_RETURN, by itself
OP_RETURN, followed by a data push operation that pushes not more than 40 bytes onto the stack.

Since your script is "OP_RETURN, followed by a data push operation that pushes not more than 40 bytes onto the stack, followed by another data push operation", it looks like it wouldn't meet either of those templates.


text:
To use Dwolla with Mt. Gox I need to become Verified. Do I need Apostille?
Mt. Gox now requires at least Level 1 Verified in order to deposit or withdraw Dwolla funds.
Level 1 says Government Photo ID + utility bill

http://en.bitcoin.it/wiki/MtGox#AML
https://support.mtgox.com/entries/20919111-aml-account-statuses

But then I see an update saying an apostille is needed.  Is that just for a Level 2 (Trusted) account?

http://support.mtgox.com/entries/20861901-changes-to-aml-policies

An apostille is only required for level 2. Their original announcement of the change was confusing, but they have since clarified it on the forum - https://bitcointalk.org/index.php?topic=60074.msg716245#msg716245.
Edit: You also need a bank account linked to your Dwolla account that has been verified over 30 days.

That is now true for transferring funds to Mt. Gox, but that 30-day requirement is not necessary when withdrawing funds from Mt. Gox.


text:
Does 'listunspent 0' return my non-mempool unconfirmed outputs?
If I have an unconfirmed transaction with a UTXO that belongs to my Bitcoin Core wallet, and that transaction is evicted from my mempool, for example due to high fee-pressure, will bitcoin-cli listunspent 0 still return the transaction?
I've tried to experiment in regtest mode. I restarted my node with -minrelaytxfee=<higher than my tx>. In this case I didn't get my tx in the response from listunspent 0, but I'm not sure this correctly mimics the scenario described above.

No. listunspent, even if passed 0 as the minimum number of confirmations, will not list coins created in wallet transactions that are not part of the node's mempool at this time.
Link to the relevant code in the Bitcoin Core wallet.


text:
How would a PSBT signer identify a non-bip32 key?
In a PSBT, version 0 or 2, how would a signer know what keypair to use for an input or an output if the keypair isn't derived through bip32?
For inputs, assuming p2wpkh for simplicity, I suppose one can use PSBT_IN_HASH160 with the pubkey hash from the output script as keydata and the pubkey as valuedata.
But how would you identify the pubkey for an output? There is no way to convey that information via the PSBT, it seems.
(The signer thus needs to maintain a map from pubkey hashes to pubkeys for all its pubkeys to verify that it owns an output. Alternatively, it can calculate the pubkey hash for each pubkey it knows about until it finds a hash that matches the pubkey hash in the output. It also has to do this for every output of the transaction, since there are no hints in the psbt on which outputs are signable by the signer. The pubkeys needs to be hashed at most once, though, because you can check each pubkey hash against all outputs.)

I don't know what answer you're after. Yes, if the key isn't derived from something else, there is no way to convey how it was derived...

I'm asking how to convey what key to verify ownership of the output with, not how it was derived.

I see; you're right, indeed you need a map from pubkey hashes to pubkeys. I don't think I ever imagined someone would write signing code that doesn't have such a map. Before PSBT that'd be unthinkable - you need that map anyway, as you need it to identify incoming UTXOs, but from a post-PSBT perspective, I guess that's not necessarily a given. The Signing entity could be distinct from the entity that recognizes incoming UTXOs.

Yes. I'm thinking of a signer as just a signer, ie no other responsibilities. The map isn't needed as long as the party feeding the unsigned tx (psbt or not) to the device also supplies necessary info to identify all keys needed. It seems PSBT output map isn't designed for non-bip32 keys.

Then again, you need a map from public keys to private keys anyway, so I'm not sure what would be gained by removing the need for it to be pubkeyhash-based.

I take your responses as "The answer to the original question is that the signer must keep a pkh->pk map in order to verify that an output (of the transaction to sign) belongs to the signer". Thanks. I just wanted to verify that I had understood psbt correctly.

For outputs, the signer doesn't have to do anything, but they may want to, in order to verify what is change, and e.g. report the amount transacted to a user for confirmation. And a signer could also reasonably not support anything but BIP32 derived keys from one or a small test of xprvs.

PSBT_*_BIP32_DERIVATION can and should be used even for keys that are not BIP32 derived. It would simply be the fingerprint of the key itself, followed by a derivation path of 0 length. Every key can be represented as being derived from itself with the derivation path of m.

For inputs, assuming p2wpkh for simplicity, I suppose one can use PSBT_IN_HASH160 with the pubkey hash from the output script as keydata and the pubkey as valuedata.

I would not recommend doing that. Signers will not be expecting keys to be provided in PSBT_IN_HASH160. That field is really for finalizers so that they can build the scriptSig/witness.


text:
Unconfirmed balance in MultiBit HD plus unconfirmed transactions (failed bitpay orders)
I've had a little bit of BTC in a MultibitHD 0.5.1 wallet for a while now. With the latest surge in price, I tried spend it at a place that uses bitpay. My first transaction probably had too little of a fee and is unconfirmed. I attempted to redo with a larger fee but it too told me was too small a fee and my attempt to buy from the store failed.
After some searching I found that Multibit is not good to use anymore / discontinued. I also came across something telling me to repair the wallet which I did but then the remaining balance became unconfirmed and not sure what to do next.
How could I get back the unconfirmed balance or confirm it? Is there anything I can do about my two unconfirmed transactions? Would it make sense to transfer everything to a new wallet software (can't when unconfirmed balance)?

Following this suggestion in the GitHub issues page solved the unconfirmed balance in MultiBit HD.
I performed the following in Windows 10:

Closed the MultiBit HD Application
Navigated to the wallet data in %AppData%\MultiBitHD\
Rename the folder mbhd-12345678-... to old-mbhd-12345678-...
Reopen the MultiBit HD Application.
Restored from wallet words (Requires words and datestamp).
Note: I did not have to close the application during synchronization like the GitHub comment suggested.
After synchronization, the unconfirmed balance was then confirmed. I was then able to transfer my balance to another address and software.

At the time it just appears to have been an over flooding of transactions making confirmation slow especially with my low fees (0.1 mbtc, 0.5 mbtc, 0.75 mbtc). Wallet balance transfer and the two unconfirmed transactions have been processed and confirmed.


text:
Why do people apparently "look up Bitcoin addresses on websites"?
I often hear things such as:

Remember, if you look up an address on Blockchain.info or something, they will have a record of your IP address being interested in that address and thus associate it with you.

Sure. I follow that logic. But what I don't understand is why they would do that in the first place. Why use some "website" when the whole point is that Bitcoin is decentralized and you have your own wallet/client/node that you can look at?
And what is their "objective" with inspecting some address that they have just used? Why not look in Bitcoin Core (or whatever they use) to simply see if it's in memory pool or has confirmations or whatever it is they are trying to look up?

@Prince M's answer is great, I have just a couple things to add:
Wallet software will generally only show you transactions relevant to the wallet, so if you ever want to explore transactions beyond your own, you might want to use a block explorer to do so.
If you are running a full node, then you can look up transactions using bitcoin-cli, but a full node does not index the blockchain according to addresses, and so the intuitive way to explore transactions that most block explorer services provide does not really exist when using software like bitcoin-core. Likewise 'address balances' are not explicitly retrievable from a full node, though there do exist some open-source self-hosted blockchain explorers that you can run on top of a node, that will compute and display such things for you. But in any case, simply running a full node does not lay open the blockchain for you in a human-digestible format, getting to that point requires additional work and software.
On another note, some wallets don't have very informative user interfaces. For example, I've seen wallets which do not display the fee-rate your transaction includes, and so if you're waiting for confirmation, just looking at your wallet will not provide any information about where in the mempool your transaction resides. Looking up the transaction using a block explorer may be helpful to the user in this case.
Of course, this can be tragic, as using such an online block explorer service can be detrimental to your privacy. For this reason, I would recommend using a block explorer that offers a .onion domain, so that you can view it over the tor network, to help preserve your privacy.

Let me preface by saying I am not an expert. There are a lot of experts active in this community, so you will hopefully get a more holistic answer from a more knowledgeable user. I can answer your question with my personal experience however, because I do look at those websites!

Why use some "website" when the whole point is that Bitcoin is
decentralized and you have your own wallet/client/node that you can
look at?

I use blockchain blockexplorer and blockstream block explorer several times a day as an easy and quick way to cross analyze and verify things I am learning about related to bitcoin. As I learn new concepts regarding addresses, transaction weight, segwit, PKskript, sigscript, etc, I can go look at transactions or blocks to help confirm my understanding, and using those sites is easy because someone has already organized the information contained in the transactions and blocks into easily human readable format. It is also fun to look through historical blocks to verify where "halving" took place, or special transactions like the transaction puzzles I posted about the other day. Sometimes I just watch block explorer for fun, look for patterns or observations.

And what is their "objective" with inspecting some address that they
have just used? Why not look in Bitcoin Core (or whatever they use)

I don't typically look for my own transactions (I've made very few). I already described above why I look on these websites, and for those reasons I wouldn't care at all if my IP address was linked to those transactions (because the ones I click on are more-or-less random). It is probably true that I could do all of the functions I described above by running a full node and running the queries myself, but there are some obvious reasons why myself and other probably don't. The minimum requirements for running a full node are not terrible, but are more than my computer can handle. Also, the average person is not always savvy enough with programming to navigate bitcoin core to verify their own transactions. Most people are probably not even savvy enough to cooperate in bitcoin transactions let alone verify their own.
In general I would say you are right, the general mood among bitcoin enthusiasts seems to be "don't trust, verify". But there are also probably a lot of people scanning explorer websites like me just for fun - and a lot of people checking transactions that don't have the skills to verify themselves, or simply just don't care if their IP address is connected to that transaction.
plot twist
Also, if lots of people like me click on random transactions on the block explorers, then their will be lots of IP addresses associated to a given transaction!


text:
How can I see the on-chain evidence of a lighting channel?
I am learning about lighting now and I'm trying to check my understanding on-chain w/ a block explorer. I have opened several channels already and closed some also.
I would like to confirm my understanding of the script logic (e.g. the 2-of-2 multisig, the HTLCs, and the conditional refunds) as described in the lightning white paper and other sources.
Unfortunately the block explorers I have tried only show "OP_0 " under the Pkscript fields for these transactions, Blockchain.com for example.
Is there any on-chain evidence of a lighting channel (before or after channel closure) or is that something that is discarded before settling on-chain?
Thanks in advance
Update: I found the 2-of-2 multisig tag on the input of a commitment transaction but there is no similar tag on the funding transaction (the one that creates the multi-sig output).
The commitment transaction (the one that spends the mult-sig output) shows the following (op_code inputs omitted)
OP_PUSHNUM_2
OP_PUSHBYTES_33
OP_PUSHBYTES_33
OP_PUSHNUM_2
OP_CHECKMULTISIG

There is also an nSequence=0xffffffff and a prev output script of
OP_0
OP_PUSHBYTES_32

But I don't see things mentioned in the whitepaper like:
OP_IF/ELSE/ENDIF, SIGHASH_NOINPUT, or HTLC
again i am just learning and I only want to know where the interesting bits are as I continue reading.

The lightning-rfc is a better source than the whitepaper. It would also help if you could provide transactions in hex or pointers to them in some blockexplorer otherwise it is hard to help. You could also check the issue tracker for some implementations where discussions of channel points (transactions) took place for debugging e.g:  https://github.com/ElementsProject/lightning/issues/1738

Transactions that create funding outputs have no special characteristics. The funding output is merely some type of segwit-output that can facilitate 2-of-2, i.e. P2SH-P2WSH or P2WSH, with P2TR to become available soon as well.
A transaction in which the channel owners collaborate to close a channel amicably does not have special characteristics except that it has a 2-of-2 input. If a channel gets closed by one channel owner unilaterally, it will use a commitment transaction with the characteristic timelocks on the closing party’s outputs, as well as optionally some anchor outputs or HTLC outputs.


text:
Why should a miner include transactions when creating a new block?
As I understand it, if I'm a miner, and I have 5 transactions in my queue, I then would use the previous blocks hash and my list of transactions to attempt to create a new block. To do this, I must verify that the 5 transactions are valid, this costs computational power.
If all a miner is trying to do is create the next block, then why bother including transactions at all? It costs my computational power and time to verify those transactions. Why can't I simply look for the next block and have no transactions in my list?
It seems like a miner wouldn't even bother including transactions.

Transactions include a fee, which miners are allowed to claim if they include that transaction in their block. While it is legal to not include any transactions, miners who do this will miss out on the fees.
Furthermore, the computational cost needed to verify a transaction is absolutely negligible compared to the work of producing PoW. What is more important is the fact that verifying transactions and building a block on top takes time, time which the miner isn't mining. For this reason, some miners will briefly work on a temporary empty block just after the predecessor block is found, and will then switch to a full one once the full processing pipelining is through with it.
Lastly, if a miner were to skip transactions for computational reasons, those transactions might (and likely will) still be included by other miners. And when this happens, the miner still has to verify them, in order to build on top of it.

While working on the empty block, what if they find the solution? Will the empty block go into circulation? or do the nodes reject the block as there're no txns in it?

Blocks without transactions are perfectly legal, so those are accepted by the network. It wouldn't make sense to outlaw them as it's trivially bypassed (miners can just stuff them with their own transactions).

Got it. What generally prevents miners from stuffing the blocks with their own txns though? Or do they already do that? Sorry for noobish questions.

The fact that it doesn't gain them any money.

Transactions pay transaction fees to the miner which includes that transaction in a block. So miners are incentivized to include transactions because it means they get paid more. This becomes more important as the block subsidy goes down as then a miner's revenue will primarily come from transaction fees rather than the generation of new Bitcoin.


text:
Timelocked bribing
Did anyone observe any instance of Timelocked Bribing in Bitcoin transaction? Are there any specific articles that points out that the attack has been mounted in reality?


text:
unlocking lightning script in windows
I want to automate the process of restarting lnd lightning after windows automatic reboot. For win update, power failure or whatever.
I cannot find the way to feed the password in the lncli unlock command is there a way to do this with a batch file using powershell?
Thanks


text:
Is it possible to calculate the correct y coordinate from x coordinate given only x coordinate without any prefix
If I have a x coordinate without any prefix such as 03 or 02
fff97bd5755eeea420453a14355235d382f6472f8568a18b2f057a1460297556
will it be possible to compute the correct y coordinate that would be usually generated normally given that private key is smaller that (n - 1) / 2, because public keys after that have same x coordinates but with different y coordinates.
If not, can someone explain me how bitcoin public key generation selects the correct y coordinate normally?
sorry incase answer was too easy, I am not too knowledgeable about all this stuff.

There is no known relation between the parity of the Y coordinate and any property of the private key (like whether it's in the low or high half).
In BIP340, which defines the Schnorr-based digital signature scheme used in Bitcoin, x-only public keys are used, which just consist of an X coordinate without information about the Y coordinate.
Internally, this scheme treats the Y coordinate as implicitly even (as if there was a 02 in front). This is accomplished by, if the provided signing private key has a corresponding odd Y coordinate, negating the private key (which switches the Y coordinate to even) at signing time.

thanks for answering that
I tried taking the even keys as correct, and when I try adding the generator point to it, it sometimes goes in the backward direction, is there any way I can generate the next x coordinate from the previous x coordinate without using y coordinate?
is it okay to ask questions in comments?

If you're going to do point additions, you really need to know both X and Y coordinates.


text:
Transaction in local mempool not broadcasted further
I created a bitcoin transaction from lnd (lncli sendcoins), and it seems to be created successfully and I can see it in my local bitcoind mempool:
bitcoin-cli getmempoolentry abdctxid {
 "vsize": 130,
 "weight": 520,
 "time": 1692559301,
 "height": 804102,
 "descendantcount": 1,
 "descendantsize": 130,
 "ancestorcount": 1,
 "ancestorsize": 130,
 "wtxid": "bcdetxid",
 "fees": {
   "base": 0.00000786,
   "modified": 0.00000786,
   "ancestor": 0.00000786,
   "descendant": 0.00000786
 },
 "depends": [
 ],
 "spentby": [
 ],
 "bip125-replaceable": false,
 "unbroadcast": false
}

It is however not being broadcasted further, the receiver did not see the transaction, and it is not visible in various online explorers.
Using Bitcoin Core v23.0.0. The only non-default setting in my config is minrelaytxfee=0.00000011 blockmintxfee=0.00000011 maxmempool=300.
As far as I know this is the first time in years it happened to my node. What could be the reason for the transaction being stuck in my local mempool and is there a way to force-rebroadcast it further?

It's hard to tell with so little information. Anything else uncommon? Do you have peers at all (getpeerinfo)? Any high CPU usage from the bitcoind process? Anything peculiar in the debug.log? Have you tried upgrading you bitcoind to the latest (25.0) version or at least the patched (23.2) version?

Thanks for directions! getpeerinfo shows ~50 peers, no suspiciously high CPU usage, upgrading to 23.2 does not help. Grepping debug.log for the txid or "mempool" or "err" does not show anything interesting. Really not sure what else to check...

Out of all the things, restarting lnd helped! Maybe there is some logic on the wallet side that forced transaction to be rebroadcasted, but now it is visible to other nodes as well. Still super weird that this happened though.


text:
Could Neutrino technically as side-effect, incentivize centralization of the bitcoin network?
TWO questions on Neutrino::
1)  Could Neutrino technically as side-effect, incentivize centralization of the bitcoin network?  Or other formulation: ...hinder higher degree of decentralization?
Why I am asking:  This article by Dorier pointed me to this thoughts. https://medium.com/@nicolasdorier/neutrino-is-dangerous-for-my-self-sovereignty-18fac5bcdc25
He basically says neutrino users are still not FULL NODES. Therefore doesnt make sense to switch from SPVs or webwallets to neutrino enabled wallets....
Could the technical knowledgable members here maybe do an explanation, why Dorier might come up with this thesis,  what technical arguments are speaking for it?  Or do you think its a baseless thesis?
2)  He basically says, why do we need neutrino, its essentially the same as SPV... Why is there technical advantages with Neutrino, what are its differentiators to SPV?
Thank you

Dorier is correct that Neutrino/SPV nodes are not full nodes, because they do not verify the entirety of the blockchain, leaving potential room for attack. However, the portrayal that Neutrino is no better than other SPV modes is not correct.
Neutrino is a form of SPV which improves greatly over other implementations in regards to privacy. All SPV modes inherently leak some hints about transactions you wish to receive because the information must be queried. Neutrino improves over previous SPV modes by limiting precision of information leaked to the blocks which contain the transactions they are concerned with. A full node does not leak any information because all blocks are archived and no querying needs to occur.

The trouble with SPV nodes is that they only verify that transactions have occurred using some SPV proofs, but they do not guarantee that you are on the correct, most proof-of-work chain. Any single deviation of the full validation rules could fork off onto a separate chain, where the full nodes would reject the chain which SPV users might follow.
To give a concrete example, the maximum block size is part of the validation rules. A full node will never accept a block which is over the allowed size. An SPV node may accept any sized block as truthful, because it doesn't know the block size and cannot measure it - it only receives the block header and it must trust the servers delivering the block header information to have properly validated the information.
The only way to counter this is to perform all of validation rules expected by a full node, which includes the block size, and therefore, requires you to receive all of the transactions in a block to measure its size.
Another validation not performed by current SPV modes is the rules regarding the coinbase transaction, which mints new coins. Deviation from this rule could lead to inflation of the supply of coins on the chain followed by an SPV node, who would be none the wiser.

The above do not mean that SPV nodes are inherently bad or must necessarily be avoided - but highlights that there is an attack surface which SPV nodes are vulnerable to, but which full nodes are immune to by design.
The concern that Dorier and others have, is that if it becomes commonplace to ditch running full nodes because of the expectation that a Neutrino node is good enough, then full nodes might begin dropping off the network and leaving fewer and fewer entities responsible for the validation of the full set of rules. If an overwhelming majority of nodes simply follow the rules of a few validators, then protection of the economic majority against inflation and block size expansion would be weakened compared to a network where a majority of people validate all of the rules.

"A full node does not leak any information because all blocks are archived and no querying needs to occur." Seems like technically, all querying occurs, and thus no information is leaked because there is no division into queried/not queried.


text:
how to find the block height from merkle root?
I have the merkle root, using this value, how to get the block hash or its height?
There is this merkle root available in getblock RPC output:
./bitcoin-cli getblock 0320d6c1bd3c4cd2a08d6f76acb50b06a3ed766b058d247fbda3147aecfef388
{
  "tx": [
    "bccf4c873984245694f64263a3392c4d67c6a6f60efe4ed53aa4965f6d8b7dc0"
  ],
  "hash": "0320d6c1bd3c4cd2a08d6f76acb50b06a3ed766b058d247fbda3147aecfef388",
  "confirmations": 2,
  "size": 180,
  "height": 102,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "bccf4c873984245694f64263a3392c4d67c6a6f60efe4ed53aa4965f6d8b7dc0",
  "num_tx": 1,
  "time": 1593523883,
  "mediantime": 1593467398,
  "nonce": 1,
  "bits": "207fffff",
  "difficulty": 4.656542373906925e-10,
  "chainwork": "00000000000000000000000000000000000000000000000000000000000000ce",
  "previousblockhash": "1664a604a6c8e603e311c6759f7693343ad77896c53de1bcef37de04dbcbbbc5",
  "nextblockhash": "1924c8db1b986f398a12d5481fcc293913a2dab2ee7fbcb0e131d7941e5201c2"
}

using different transaction hashes, I got the merkle root value, now I need to know which block has this merkle root as in getblock.

Do you have the entire coinbase transaction? The height is included in its scriptSig

The block hash is the output obtained when hashing the header of a bitcoin block. The header of a bitcoin block is comprised of several pieces of data, one of which is the merkle root.
So having the merkle root alone is not sufficient to calculate the block hash, you would need additional information (the rest of the block header).
You could write some code to query the block database for a specific merkle root, but otherwise I am not aware of any software that currently includes this functionality.
(this seems odd to me though: if you have calculated the merkle root, you assumedly have all the transaction data from the block in question. I'm not sure in what situation you would have all of this data, but not the rest of the block header data)

I am in the verge of validating a new functionality of verifying the merkle proof, I used different positions like changing LEFT to RIGHT and got some value of merkle root, hence using that merkle root value, I need to know the block hash / height to verify whether its correct or not.


text:
Can stolen or otherwise tainted coins be laundered through mining fees?
Do chain analysis techniques consider the fee portion of block rewards as tainted funds?
e.g. if known stolen funds are moved in a certain block, the fees the sender pays end up "polluting" the block reward. I want to know how much weight this taint actually gets when it comes to chain analysis. Would the largest exchanges accept these coins as deposits?
If so, then a thief could launder the funds by colluding with a miner. e.g. do a self transfer that pays 10% in fees; don't broadcast the tx, just pass it to the colluding miner who mines it and later returns a portion of the fees to the thief out of the block reward.
If not, then a thief could taint every block's reward by simply self transferring a minimal amount with sufficient fees for next block. Then none of the blocks rewards can be deposited in any heavily regulated exchange. I guess miners could censor those txs if the thief's UTXOs are well known, but it's still an attack vector I guess.

It was proposed by some regulators, but the whole idea isn't applicable to bitcoin (highly decentralized).
First of all, there are multiple mining pools and whoever tries to launder bitcoins in this way would have to collude with all of them. Second, pool is just a collection of smaller entities mining together and rewards are being distributed based on the hashrate. There may happen to be some out-of-bound mining because it's fairly hard to look for events like that, but if some percentage of the fees wouldn't be distributed properly, members of the pool would know about it.
Also it's impossible to "mine block first, add transaction later", this kind of attack would require mining pools to adopt custom bitcoin node software, which is not rebroadcasting said transaction further.
Overall it's way easier to set-up darknet market shop and sell "tainted" bitcoins for 10% discount, or just coinjoin, and most of the "high fees" incidents in past were just overlooking and misunderstanding of how bitcoin change works.

Why would the thief have to collude with all of the pools? Just one would suffice, since the tx isn't broadcast, just secretly given to it?

Also, nothing suggested "mine block first"... the pool defines the txs that get mined and just hands over the hash to the grinders. Nobody would know this particular tx is included. Grinders can't censor, to boot.

Third, coinjoins carry major taint. The question is how chain analysis treats block reward, not how to obfuscate utxo paths.

I cannot look into the chainalysis code but if someone cooperates with a miner by paying unusual fees with tainted coins then the high fees are suspicious and I'm pretty sure analytics will monitor this.
Usually it's not a question whether or not the funds on a bitcoin address are from a criminal source, it's a question of what % is polluted. If you go inbound in the transaction trees of an address back to the miner rewards then you might have 1000 addresses and a little percentage will be polluted. So e.g 1.5% of the starting addresses funds are from a criminal source. It's the same with the bills in your wallet or with your ancestors. Going 10 generations back we have upto 1024 great...grandparents and if 10 of them were criminal then we could say that 1% of our ancestors were criminal.
So the criteria aren't the same at all exchanges but I assume that only addresses with a relatively high percentage of pulluted 'ancestors' will be rejected otherwise the exchange would have to stop it's business.

So any block reward that includes a major "generous" fee would be considered taint? That's an attack vector.

I do bitcoin tracing and would  consider an unusual high fee as a likely taint. Of course it could be as well a mistake by the sender or by a buggy wallet. Like all attacks the outcime isn't 100% safe.

Can stolen or otherwise tainted coins be laundered through mining fees?

They can, and for sure they will.

Do chain analysis techniques consider the fee portion of block rewards as tainted funds?

In general, they should not, because the miner pool is, unless proven otherwise, protected by the good faith principle.
Demonstrating break of good faith requires demonstrating only the miner pool manager knew the transaction.
Once a miner pool manager bad faith is demonstrated, the first time they'll have a huge win, then regulation will be enacted, then if they do once again, government crackdown will come hard on them.

e.g. if known stolen funds are moved in a certain block, the fees the sender pays end up "polluting" the block reward. I want to know how much weight this taint actually gets when it comes to chain analysis.

None unless bad faith proven.

Would the largest exchanges accept these coins as deposits?

Yes.

If so, then a thief could launder the funds by colluding with a miner. e.g. do a self transfer that pays 10% in fees; don't broadcast the tx, just pass it to the colluding miner who mines it and later returns a portion of the fees to the thief out of the block reward.

No, the higher the fee, the most solid suspicion for bad faith. Eventually blocks chain police would build enough evidence to demonstrate bad faith.

If not, then a thief could taint every block's reward by simply self transferring a minimal amount with sufficient fees for next block. Then none of the blocks rewards can be deposited in any heavily regulated exchange. I guess miners could censor those txs if the thief's UTXOs are well known, but it's still an attack vector I guess.

No, they don't even need to censor it, if tainted fees are low, reward gets a very low taint, because most fees are legit.

What I can now unsee is a future where, long after inside countries anti mining laundry regulation has been enacted, nations hack each other's reserves and use their own mining rigs to launder coins stolen to the enemy, in blocks the enemy is forced to accept because multipolar global nation state consensus. Until a straw breaks the camel's back. I think you have discovered a casus beli for WWIII.


text:
I cannot broadcast my raw tx (P2PKH) on testnet: Signature must be zero for failed CHECK(MULTI)SIG operation
I am trying to broadcast my transaction on the test network with no success so far. I get always this error:
sendrawtransaction RPC error: {"code":-26,"message":"mandatory-script-verify-flag-failed (Signature must be zero for failed CHECK(MULTI)SIG operation)"}

I have reviewed my code at least 10 times the last two hours and can´t find what is wrong with it. It looks like everything is okay but obviously not.
Could someone spot the cause of the issue?
This is the raw transaction in hex:
010000000119385e43e5d69ce6dd8aa465cdfc897c80782d1b44fcc07239bf17e000fdf7db000000006a47304402207fd3680d14fe8de7ef71d35d9af0c60b2f878532cdd912fd3b293ae30c8bd3a902203de1b6330657487d4fe115a90916d55fbea0445433eb206b04073594ef5bcdd1012102ce96cbb4508b6555c0968a90cc2e24e511e7431e78e69a04144f5017390c9612ffffffff0150dc0000000000001976a914655b68715774105f48b475dc2a15026ade9e013e88ac00000000
And here the same hex deconstructed:
version: 01000000

tx in count: 01

prev_tx_id (flipped): 19385e43e5d69ce6dd8aa465cdfc897c80782d1b44fcc07239bf17e000fdf7db

v_out: 00000000

sigscript size: 6a

sigscript: 47304402207fd3680d14fe8de7ef71d35d9af0c60b2f878532cdd912fd3b293ae30c8bd3a902203de1b6330657487d4fe115a90916d55fbea0445433eb206b04073594ef5bcdd1012102ce96cbb4508b6555c0968a90cc2e24e511e7431e78e69a04144f5017390c9612

sequence: ffffffff

tx_out_count: 01

amount: 50dc000000000000

scriptPubKey_out_size: 19

scriptPubKey_out: 76a914655b68715774105f48b475dc2a15026ade9e013e88ac

lock time: 00000000

This is my code on Python. I just removed the private_key_hex field, anything else remains untouched. Also, notice that I got no change, the difference between the full UTXO amount and the value sent is the fee for the miners.
import base58
import hashlib
import ecdsa

def hash160(x): # Both accepts & returns bytes
    return hashlib.new('ripemd160', hashlib.sha256(x).digest()).digest()

def doublesha256(x):
    return hashlib.sha256(hashlib.sha256(x).digest()).digest()

def flip(string):
    flipped = "".join(reversed([string[i:i+2] for i in range(0, len(string), 2)]))
    return flipped

def hashed_pubkey(pubkey):
    return base58.b58decode_check(pubkey)[1:].hex()

private_key_hex = ''
pub_address_in = 'mpfmNgZAkSdhDoP5UJKhh6wahfGboXhFmj'
hashed_pubkey_in = hashed_pubkey(pub_address_in)

version = 1
tx_input_count = 1
tx_output_count = 1

#TX_IN_VARIABLES
prev_tx_id = 'dbf7fd00e017bf3972c0fc441b2d78807c89fccd65a48adde69cd6e5435e3819'
v_out = 0
scriptPubKey_in = bytes.fromhex("76a914{}88ac".format(hashed_pubkey_in))
scriptPubKey_in_size = len(scriptPubKey_in)

#TX_OUT_VARIABLES
amount = 56400
pub_address_out = 'mpkt3ZpTfZJDeN9p5LgkAHj3NmVMaND7vR'
hashed_pubkey_out = hashed_pubkey(pub_address_out)
scriptPubKey_out = bytes.fromhex("76a914{}88ac".format(hashed_pubkey_out))
scriptPubKey_out_size = len(scriptPubKey_out)

class raw_tx:
    version                             = (version).to_bytes(4, byteorder='little')
    tx_in_count                         = (tx_input_count).to_bytes(1, byteorder = 'little')
    tx_input                            = {}
    tx_out_count                        = (tx_output_count).to_bytes(1, byteorder = 'little')
    tx_output                           = {}
    lock_time                           = (0).to_bytes(4, byteorder = 'little')

rtx = raw_tx()

#TX_IN
rtx.tx_input['prev_tx_id']              = bytes.fromhex(flip(prev_tx_id))
rtx.tx_input['v_out']                   = (v_out).to_bytes(4, byteorder = 'little')
rtx.tx_input['scriptPubKey_in_size']    = scriptPubKey_in_size.to_bytes(1, byteorder = 'little')
rtx.tx_input['scriptPubKey_in']         = scriptPubKey_in
rtx.tx_input['sequence']                = bytes.fromhex('ffffffff')

#TX_OUT
rtx.tx_output['amount']                  = amount.to_bytes(8, byteorder = 'little')
rtx.tx_output['scriptPubKey_out_size']   = scriptPubKey_out_size.to_bytes(1, byteorder = 'little')
rtx.tx_output['scriptPubKey_out']        = scriptPubKey_out

raw_tx_string = (

    rtx.version
    + rtx.tx_in_count
    + rtx.tx_input["prev_tx_id"]
    + rtx.tx_input["v_out"]
    + rtx.tx_input["scriptPubKey_in_size"]
    + rtx.tx_input["scriptPubKey_in"]
    + rtx.tx_input["sequence"]
    + rtx.tx_out_count
    + rtx.tx_output["amount"]
    + rtx.tx_output["scriptPubKey_out_size"]
    + rtx.lock_time
    + (1).to_bytes(4, byteorder = 'little')#SIGHASH_ALL type of sighash

    )

#DOUBLE SHA256 OF RAW_TX
hashed_tx_to_sign = doublesha256(raw_tx_string)

#GETTING PUBLIC KEY IN BYTES FORMAT
sk = ecdsa.SigningKey.from_string(bytes.fromhex(private_key_hex), curve = ecdsa.SECP256k1)
vk = sk.verifying_key
public_key = vk.to_string()
public_key_hex = public_key.hex()

#COMPRESSING THE PUBLIC KEY
# Checking if the last byte is odd or even
if (ord(bytearray.fromhex(public_key_hex[-2:])) % 2 == 0):
    public_key_compressed = '02'
else:
    public_key_compressed = '03'

# Add bytes 0x02 to the X of the key if even or 0x03 if odd
public_key_compressed += public_key_hex[:64]
public_key_compressed = bytes.fromhex(public_key_compressed)

#SIGNING RAW_TX HASHED
signature = sk.sign_digest(hashed_tx_to_sign, sigencode = ecdsa.util.sigencode_der_canonize)

#CREATING SCRIPTSIG
signature_SIGHASH_ALL_byte = signature + (1).to_bytes(1, byteorder = 'little') #SIGHASH_ALL type of sighash
signature_SIGHASH_ALL_byte_size = len(signature_SIGHASH_ALL_byte)

public_key_compressed_size = len(public_key_compressed)

scriptSig = (

    signature_SIGHASH_ALL_byte_size.to_bytes(1, byteorder = 'little')
    + signature_SIGHASH_ALL_byte
    + public_key_compressed_size.to_bytes(1, byteorder = 'little')
    + public_key_compressed

    )

scriptSig_size = len(scriptSig)

real_tx = (
    rtx.version
    + rtx.tx_in_count
    + rtx.tx_input["prev_tx_id"]
    + rtx.tx_input["v_out"]
    + scriptSig_size.to_bytes(1, byteorder = 'little')
    + scriptSig
    + rtx.tx_input["sequence"]
    + rtx.tx_out_count
    + rtx.tx_output["amount"]
    + rtx.tx_output["scriptPubKey_out_size"]
    + rtx.tx_output["scriptPubKey_out"]
    + rtx.lock_time

    )

print(real_tx.hex())

The modified transaction you're hashing (raw_tx_string) is missing the output script (scriptPubKey_out), which is why the signature then fails to verify.


text:
Obtaining other portion of my private key
I have only a partial private key due to hard drive damage. I have the full public key I plan to extrapolate back on the secp256k1 elliptic curve to locate the full private key. What are my chances of getting it correct?

It really depends on how much of the private key you're missing. Every missing byte increases the search space by a factor of 256. Assuming you can search one million keys every second (a reasonable approximation for a modern CPU utilizing all its cores running a decently optimized search algorithm):

4 missing bytes, 232 = 4,294,967,296 keys, about 1 hour to search through
5 missing bytes, 240 = 1,099,511,627,776 keys, about 12 days to search through
6 missing bytes, 248 = 281,474,976,710,656 keys, about 8 years to search through

As you can see, the chances of recovering the rest of the key within a reasonable timeframe fall pretty quickly. If there's a lot of funds to recover you can of course spend funds on acquiring more computing power, but even increasing it 256 times still offsets just one missing byte.

Thankyou. I will ponder my next step


text:
Sent to Wrong address
Someone gave me the wrong address to send the bitcoins to on my cash app.  Now I’ve lost everything. Can I get it back? I’ve contacted Cash app they said that once it is cashed out they have no way to track it. I’ve tried searching for it but can not find it.


text:
How do I cash in Physical Bitcoins?
I was gifted 2 physical bitcoins by a family friend who has passed. How do I go about cashing them in? I have spoken to my bank, retirement company, paypal, coinbase, etc., and no one has a clue. Anyone know?

This question is similar to: How can one redeem a Casascius Physical Bitcoin?. If you believe it’s different, please edit the question, make it clear how it’s different and/or how the answers on that question are not helpful for your problem.

See What is a physical Bitcoin token

Although Murch has already given an excellent high-level answer, I'll add on some more details as an avid collector.

The first step is identifying the kind of item you have. This is going to fall into, broadly, three categories.

Bitcoin-themed tokens - These are a dime a dozen, and sold on common marketplaces like Amazon, Etsy, etc. They're simply metal coins or bars with Bitcoin stamped on them, typically accompanied by a design of a face or circuit board on the back of the coin. They have no identifying marks (such as a hologram, serial number, QR code, or bitcoin address), and are simply commemorative tokens. They contain no Bitcoin inside them, and are functionally no different from an arcade token.
Commonly, they look something like this

Loaded Bitcoin Token - Loaded tokens have been made by a handful of creators over the years, most famously Casascius. Other makers with sizable quantities produced (and thus more likely to randomly crop up like this) are BTCC, Satori, Lealana, AlpenCoin, and Denarium. Many other smaller series with tens or hundreds of pieces produced exist as well.
Identifying a specific series can be hard without photos, but you can look for indicators that it is a collectible item - it will typically be made of metal (Brass, Silver, Titanium, Copper, and in some cases a mixture of them), Clay or Plastic (usually, these take the form of poker chips from Satori, BTCC, or MoonBits). It will usually also mention the name of the creator (such as "BTCC" or "Casascius"), and come with a holographic sticker on some part of it (for coins, usually at the back - for bars/blocks, usually in one corner of the face). The sticker may or may not have a small window in the middle showing the first few characters of a Bitcoin address). The holographic portion itself may contain some or all of the address, and may be marked with a serial number. You may have to try viewing the sticker from multiple angles to reveal text or finer details, as it may not always be visible head on.
In the event that the token has a unbroken holographic sticker, it is possible that it contains a BTC value - the sticker typically contains a private key behind it, which has an associated bitcoin address. In many cases, this address is loaded. However, several series have been made over the years that were sold as "Buyer Funded", where it is up to the eventual owners to decided whether or not to send BTC to that key.

Redeemed Loaded Token - It is also possible that you have an item which has a holographic sticker that has been peeled (look for honeycomb patterns or the term "VOID" appearing in the sticker), or have had the sticker removed entirely (look for a suspiciously blank spot which would be good for pasting a round or square sticker and stands out from the otherwise patterned/design heavy body of the rest of the item). In this case, any BTC it may have contained is likely gone, but the redeemed item itself may be worth a small amount of money as it is still typically collectible.

If you find your item falling into categories 2 or 3, it is best to try to identify exactly what you have - some of them can contain sizable amounts of BTC (0.0001 through 1 BTC are common, with amounts above 1 BTC being fairly rare).
I have built a site at collectible.money that indexes a significant portion of the most common/well known creators and items - if you find any identifying marks such as the name of a creator, series, partial/full BTC address (look for a string of random looking characters starting with 1, 3, or bc1), the search function on the website may help to narrow down which item you have (and potentially report the balance, if any).
It is also advisable to post photos to https://bitcointalk.org/index.php?board=217.0 - the Collectibles forum on Bitcoin Talk is fairly active, with many extremely knowledgeable folks who are acquainted with nearly the entire spectrum of Bitcoin collectibles. Make sure to avoid responding to any 1:1 messages, and if you are uncertain about the status of your item's funding, cover up any QR codes/paper/stickers - simply posting the face of the item will usually be sufficient for folks to identify it.

I guess your first step would be to establish whether what you hold is actually a Casascius physical bitcoin or another physical carrier¹ of bitcoin. After that I would recommend that you take your time to continue gathering information in public (such as here or in forum threads) and be wary of anyone approaching you in private messages. Be careful what information you reveal by posting pictures, under no circumstances should you reveal the information under the hologram imprint, and very likely, you do not want to show the backside of the coins with the hologram to anyone either. You may want to consider using a pseudonym while collecting information, otherwise you may be contacted by any number of people that offer you help with a variety of intentions.
I say that it should be worth taking your time to figure out what you got and how to proceed, because assuming that you are talking about Casascius coin which were only sold up to 2013, you may have collector items on your hand. If you do identify your coins as Casascius coins, they were created in denominations of 1–1000 BTC, so you might be looking at a life changing amount of money.
My understanding is that Casascius coins have a hologram imprint which when peeled off reveals a private key that can be imported into a Bitcoin wallet.  I assume that the coins are likely to be more valuable intact, but it may be easier to import the stored bitcoins instead of auctioning off the coins. The Bitcoin Talk Collectibles forum users are giving mixed recommendations.
If you do end up redeeming the coins, you could afterwards spend the acquired bitcoins directly for goods and services, or trade them for your local currency e.g. on a Bitcoin exchange.

¹There have also been other physical bitcoin coins made which may be more or less similar to Casascius coins, but generally, be sure not to reveal any numbers or hexadecimal strings on the coins, especially anything that is revealed by peeling or scratching off a cover.


text:
Is mainline Bitcoin planning on switching to deterministic wallets?
Deterministic wallets enable reliable backups but the mainline client doesn't use them.  I was told that bitcoind and Bitcoin-QT are planning on switching to deterministic wallets by default, is this correct?  Any idea of when?

It seems very likely, however it also seems to be a fairly long-term goal with no ETA as of yet: https://github.com/bitcoin/bitcoin/issues/5761


text:
How to send Bitcoin in the Testnet?
I'm using this library: https://github.com/lian/bitcoin-ruby
And here's my code:
require 'open-uri'
require 'net/http'
require 'net/https'
require 'json'

require 'bitcoin'

Bitcoin.network = :testnet3

def get_response(prev_tx)
  url = "https://testnet.blockexplorer.com/api/rawtx/#{prev_tx}"
  uri = URI(url)
  response = Net::HTTP.get(uri)
  response = JSON.parse(response)
  response = response['rawtx'].to_s
  return response
end

def build_transaction(prev_tx, prev_out_index, key, satoshi_to_spend, addr, message)
  include Bitcoin::Builder

  new_tx = build_tx do |t|
    t.input do |i|
      i.prev_out prev_tx
      i.prev_out_index prev_out_index
      i.signature_key key
    end
    t.output do |o|
      o.value satoshi_to_spend
      o.script {|s| s.type :address; s.recipient addr }
    end
  end
  puts ""
  puts "--- new tx ---"
  puts new_tx.to_json
  return new_tx
end

def bin_to_hex(s)
  s.unpack('H*').first
end

bitcoin_key_1_pub = "0367e6251a89c5afc1212c3e2169866ccfe9d6059a4cfc15dc139b534b974fd7ca"
bitcoin_key_1_priv = "*****24c56528becd02d23076123a2edc2f3cf52c3c0202178c1b7fa9ac*****"
bitcoin_key_1_addr = "mxz4hu6JrTHevn22W4GDVjS3ko9MwXuHND"

# the previous transaction that has an output to your address #1
prev_tx_1 = "427138ed5f2443fbfbf8065e990f1bf9ce0236e14376e8abca0b2876eabca653"

# the number of the output you want to use
prev_out_index = 1

# 0.5 BTC in satoshis
btc_to_spend = 0.5
satoshi_to_spend = btc_to_spend * 100000000

bitcoin_key_2_pub = "02989d0578f715a1980d4d6e968bcba487077e74aaf8a219d11ee4384ea1fc1eda"
bitcoin_key_2_priv = "*****cb521a27f9782cc503b461bf5c6b470469ca00ba1d7dc26f4ab61b*****"
bitcoin_key_2_addr = "n4SYP6TJyK7pUouW65fnC1hDBrbb7Za8RR"

# fetch the tx from whereever you like and parse it
response = get_response(prev_tx_1)
$prev_tx = Bitcoin::P::Tx.new(response.htb)
prev_tx = $prev_tx

tx_value = prev_tx.outputs[prev_out_index].value
puts "tx_value: #{tx_value}"

key = Bitcoin::Key.new(bitcoin_key_1_priv, bitcoin_key_1_pub)
tx = build_transaction(prev_tx, prev_out_index, key, satoshi_to_spend, bitcoin_key_2_addr, "hello")

puts ""
puts "--- bin_to_hex ---"
puts bin_to_hex(tx.to_payload)

Here's the output:
➜  examples git:(master) ✗ ruby sandbox.rb

tx_value: 75972791
/Users/glaksmono/.rvm/gems/ruby-2.4.2/gems/bitcoin-ruby-0.0.18/lib/bitcoin/script.rb:281: warning: constant ::Fixnum is deprecated
/Users/glaksmono/.rvm/gems/ruby-2.4.2/gems/bitcoin-ruby-0.0.18/lib/bitcoin/script.rb:455: warning: constant ::Fixnum is deprecated

--- new tx ---
/Users/glaksmono/.rvm/gems/ruby-2.4.2/gems/bitcoin-ruby-0.0.18/lib/bitcoin/script.rb:261: warning: constant ::Fixnum is deprecated
{
  "hash":"88217866a40829f144e2d2c19b2e506812ae013d08d9335ce8fa839582f1b18e",
  "ver":1,
  "vin_sz":1,
  "vout_sz":1,
  "lock_time":0,
  "size":191,
  "in":[
    {
      "prev_out":{
        "hash":"427138ed5f2443fbfbf8065e990f1bf9ce0236e14376e8abca0b2876eabca653",
        "n":1
      },
      "scriptSig":"304402201cf2d99b7f10c71a1c77ae629ec134a3f255c0027f47815aff3e43d500e6da46022008409ea148d6cce6876910c7a7218b8197c3a827a6a7bf3d5f064a3ed3a0616701 0367e6251a89c5afc1212c3e2169866ccfe9d6059a4cfc15dc139b534b974fd7ca"
    }
  ],
  "out":[
    {
      "value":"0.50000000",
      "scriptPubKey":"OP_DUP OP_HASH160 fb75447d6038b05bb736e5b05f18a3c9da2b34db OP_EQUALVERIFY OP_CHECKSIG"
    }
  ]
}

--- bin_to_hex ---
010000000153a6bcea76280bcaabe87643e13602cef91b0f995e06f8fbfb43245fed387142010000006a47304402201cf2d99b7f10c71a1c77ae629ec134a3f255c0027f47815aff3e43d500e6da46022008409ea148d6cce6876910c7a7218b8197c3a827a6a7bf3d5f064a3ed3a0616701210367e6251a89c5afc1212c3e2169866ccfe9d6059a4cfc15dc139b534b974fd7caffffffff0180f0fa02000000001976a914fb75447d6038b05bb736e5b05f18a3c9da2b34db88ac00000000

It seems to be successful, but I don't see the BTC actually transferred from bitcoin_key_1_addr to bitcoin_key_2_addr on the Blockcypher Explorer:

bitcoin_key_1_addr URL: https://live.blockcypher.com/btc-testnet/address/mxz4hu6JrTHevn22W4GDVjS3ko9MwXuHND/
bitcoin_key_2_addr URL: https://live.blockcypher.com/btc-testnet/address/n4SYP6TJyK7pUouW65fnC1hDBrbb7Za8RR/

As you notice in the Blockcypher Explorer, the BTC amount is not yet transferred from bitcoin_key_1_addr to bitcoin_key_2_addr. Ideas?

Did you broadcast the transaction? I don't see any code that looks like the transaction is actually being sent to the network. A transaction needs to be broadcast before it can be seen by block explorers and included in the blockchain by miners.

@AndrewChow very good point - any ideas how to do that in bitcoin-ruby?

I figured, you need to use this: https://live.blockcypher.com/btc-testnet/pushtx/ but then I'm getting an error saying that the transaction has "too high fees", ideas? @AndrewChow

You can use that to broadcast. If you get an error, then you probably built your transaction wrong. Too-high-fees usually means that you did not create a change output so the change ends up all going to fees.

Did you see anything wrong with my code above? Not really sure which part of it was wrong @AndrewChow

The problem is that you are only creating one output whereas you need to have two outputs, the second being for the change output. The change output is exactly like any other output except that the output is spendable by you, not by your recipient.

@AndrewChow I don't understand. Do you have a sample code I can take a look?

It seems you are having a hard time grasping the idea of change. Since this is testnet, just make your output value almost match the input value. I say almost match because you will still need a transaction fee.

Decided to use BlockCypher API, and it just works. Just follow this CURL example: https://gist.githubusercontent.com/matthieu/b07c5ba27bc99188a15f/raw/870a9d0e1bdd05a6230acde359e6953add15621a/gistfile1.sh
And you can always translate that into whatever language you want to use. In my case I'm using Ruby.


text:
Increase transaction fee with Bitcoin Core client fails because of no change output. Why?
Context
I created a transaction (using the full balance of an output) with the Bitcoin Core Wallet (running as full node). The transaction got into the mempool of major nodes, as expected. I set the fee to what the client recommended for a slow transaction (little more than 9 sat/B) , but these days this turns out to be very low. Given the current state of the mempool at hoenicke this may get confirmation in weeks, if at all.
Increasing the fee fails
I initially, deliberately, set the transaction to "Request Replace-By-Fee", and I want do increase the fee now. I expected to take some part of the "target amount" and use it as fee, leaving the total amount of the transaction the same (still spending the complete output in total)
However, this does not seem to be possible by design. I get a "Fee bump error" saying "Increasing the transaction fee failed (Transaction does not have a change output)".
What to do about it?
Why am I not able to change the target/fee balance of a transaction, and how am I expected to increase the fee at this point?
Transaction Details (as per Request)
Summary
Size    1xx (bytes)
Weight  7xx
Lock Time   Block: 49xxxx

Inputs and Outputs
Total Input 0.01xxxxxx BTC
Total Output    0.01xxxxxx BTC
Fees    0.00001xxx BTC
Fee per byte    9.xxx sat/B
Fee per weight unit 2.xxx sat/WU
Estimated BTC Transacted    0.019xxxxx BTC

Both input and output addresses start with "1"

BIP125 RBF (Replace-by-fee) transactions in bitcoin-core 0.15.0.1 appear to require a change address with a balance in order to facilitate a future RBF-based fee increase.
By way of an example, consider the following transaction:
Input  - 1.00000 BTC
Output - 0.99998 BTC
Fee    - 0.00001 BTC
Change - 0.00001 BTC

Assume the fee is too low.  Now, we want to increase the fee for the transaction, so the RBF replaces the prior transaction with one like:
Input  - 1.00000 BTC
Output - 0.99998 BTC
Fee    - 0.00002 BTC
Change - 0.00000 BTC

Now the fee is enough that the transaction is confirmed.
Without "change" present in the first transaction, there is no source for the increased fee.  RBF could be implemented such that the output is reduced to allow for a greater fee; I'm not sure why that is not supported.  Line 2928 of the wallet.cpp source code in the bitcoin-core github repository states:
break; // Done, able to increase fee from change

The way around this problem for a transaction that has RBF enabled, but has no change address is to follow the directions contained in one of the answers in the canonical "transaction not getting confirmed" question.  I experienced the exact issue last week and was able to successfully recover using the -zapwallettxes option in the bitcoin-core wallet CLI.


text:
Using Bitcoin ATM's, how does selling work?
How do Bitcoin ATM's work? I have a wallet on my computer and wanted to know how would the bitcoins there be transferred to the ATM and exchanged for cash.
Could somebody explain this process to me?

Atm will generate a bitcoin deposit address
Client scan it using smartphone and send bitcoin to that address
Atm will wait for confirmation and emit fiat money from dispenser

Things to consider:
KYC level for bitcoin amounts
Generation of addresses
Confirmation number
ATM connected full node
Rate exchange
And more , but I believe this is the basic.

There are various types of Bitcoin ATM's, only some of them offer to sell Bitcoin. Here is a list of some of those types and a good explanation of the selling process:
https://coinatmradar.com/blog/how-to-sell-bitcoins-using-bitcoin-atm/
Typically you receive a voucher and an address to send the Bitcoins to, then redeem the cash using the voucher, after the transaction has confirmed.

Following are the steps are followed for the usage of the bitcoin ATM machine :

Open the QR code for your Bitcoin wallet on a smartphone. You can get a bitcoin wallet from Coinbase if you don’t already have
    one! Don’t forget to download Coinbase’s app for iPhone or Android from the AppStore or GooglePlay.
Then run the QR code under the bitcoin ATM machine’s scanner.
Enters the amount of cash to be exchanged for Bitcoins.
Waits for the machine to process the transaction and give out a confirmation.
Check your bitcoin wallet to see if the Bitcoins transaction is processed

This is a good answer, however, the OP is asking about selling his Bitcoin's, not buying them.


text:
UXTO DB structure
After update Bitcoin Core to version 0.15, chainstate database changed format of data.
Old prefix of key has been 0x63, now 0x43, and key/value are different
Where is the new format described, or can someone have work on this?
UPDATED
Sample
Key:
43000000dcf3713a29f3da53dbde07ea93ace52a4393ba9a009d8ac17c853c18e500

Value:
d42ade0dfd6a93b6ecbc6098336feedaa21b79128e29d5b7ee2d42

Prefix BYTE:
0x43

TransactionId:
e5183c857cc18a9d009aba93432ae5ac93ea07dedb53daf3293a71f3dc000000

Index VARINT:
0x00 -> 0x00(0)

Code VARINT:
0xD42A -> 0x2AAA(10922)

Block Height:
Code >> 1 = 0x1555(5461)

Is Coinbase:
Code & 1 = 0x00(0)

Value VARINT:
0xDE0D -> 0x2F8D(12173) -> decompressMSB128 -> 0x21084(135300)

Type VARINT:
0xFD6A -> 0x3F6A(16234)

If you want to find UTXOs, I find scanning the blockchain to be a better option: https://bitcoin.stackexchange.com/a/65083/2075

Each entry in the new 0.15 format is defined as outpoint:coin, and has the following structure:
Outpoint is formed by: key | tx_hash | index.
Where the key corresponds to b'C', or 43 in hex. The transaction hash in encoded in Little endian, and the index
    is a base128 varint. The corresponding Bitcoin Core source code can be found here.
On the other hand, a coin is formed by: code | value | out_type | script.
Where code encodes the block height and whether the transaction is coinbase or not, as 2*height + coinbase, the value is a txout compressed base128 varint, the out_type is also a base128 varint, and the script is the remaining data. The corresponding Bitcoin Core source code can be found here.
Version 0.15 still used an obfuscation key to avoid triggering anti-virus software, you can check this answer to know how it is stored in the database and how to use it to de-obfuscate you data.
Finally, if you need a decoder for both versions 0.14 (0.8-0.14) and the new 0.15 format, you can check this GitHub repo (particularly functions decode_utxo and decode_utxo_v08_v014 from utils.py).
Disclaimer: I'm the author of the repo.

I parsed Key, But with Value i have a problem. Question updated

What's the problem?

There can be still something? Data does not converge. In question i try parse example from db. I get txhash and index from outpoint. https://blockchain.info/tx/e5183c857cc18a9d009aba93432ae5ac93ea07dedb53daf3293a71f3dc000000?format=json. But any other fields in coin invalid. The simplest field is the script, since it remains unchanged. I cant find any part of script in transaction

For what I see, the coin and the outpoint you are giving as example do not match. The corresponding coin for '43000000dcf3713a29f3da53dbde07ea93ace52a4393ba9a009d8ac17c853c18e500' is 'bafe78849575007b8268c6115b707d17cccfdf9be636467a80f9e4'. Either way, the data you have decoded in your updated question match with the coin you provided, but since the coin seems not to be correct, the parsing goes wrong at the end.

Very strange. I wait full synced of my wallet and double checked value for 43000000dcf3713a29f3da53dbde07ea93ace52a4393ba9a009d8ac17c8 in two different leveldb driver https://github.com/syndtr/goleveldb and https://github.com/golang/leveldb and my value still is D42ADE0DFD6A93B6ECBC6098336FEEDAA21B79128E29D5B7EE2D42. Data is somehow compressed, encrypted? Is the database unique for each node?

I've update the answer with information about the obfuscation key and links to a repo with tools for utxo set decoding.

Thx! Obfuscation key - i confused. What's the point in this? This is simply XOR who stored in this db.

The chainstate use to trigger some anti-virus software in windows systems, by XORing the information the signature of the data changes and there is no more false positive with AV.

I followed your instructions and successfully decoded all but the script part:
TxID: e5183c857cc18a9d009aba93432ae5ac93ea07dedb53daf3293a71f3dc000000 DecodedValue: bafe78849575007bba135794915ad83d8f25cbd406b9c92b52963a Code: 983032 Height: 491516 Amount: 942800 (84853) OutType: 0 (P2PKH) Script: 7bba135794915ad83d8f25cbd406b9c92b52963a
The actual hash160 must be 7b8268c6115b707d17cccfdf9be636467a80f9e4
Is the script encoded somehow?

How are you getting 7bba135794915ad83d8f25cbd406b9c92b52963a? That's the response I'm getting: {'coinbase': 0, 'tx_id': '000000dcf3713a29f3da53dbde07ea93ace52a4393ba9a009d8ac17c853c18e5', 'height': 491516, 'outs': [{'amount': 942800, 'out_type': 0, 'data': '7b8268c6115b707d17cccfdf9be636467a80f9e4'}], 'index': 0}


text:
How it is possible to validate existing Pay to Public Key Hash transaction without public key?
These are my understanding or assumptions.
Sign a document requires secret key and message only as below
SignDocument(secret key, message) => signature

And verification of signature requires public key, signature and message as below
VerifySignature(signature, message , public key)

Purpose of scriptPubKey is a script for validating signature for P2PKH transactions.
It is clear that we need public key in order to validate transaction but existing transaction data doesn't have public key data but hash of public key only.
I can easily understand Pay to Public Key transaction because public key is part of transaction data.
If public key is not stored in blockchain so how it is possible to validate P2PKH transaction without public key?

http://www.righto.com/2014/02/bitcoins-hard-way-using-raw-bitcoin.html explains manually signing a transaction, and helped me alot (next to the online readable book "Mastering Bitcoin" from Andreas). Looking at the example tx from before: https://blockchain.info/rawtx/e46a88ed211c1ee7f34f0f4828611da52404c3282416ae1e3b7096f9dddc6c4e?format=hex
one can see the sigscript in this section:

483045022100B8E264B50017806D4095740E5523188C78884DCFCBA3AA7C1E608F28A1A763AF0220159680F7870BE5B46B6CD3110629AEAB221E6D658DD396AE319B071E0D2B339601210214A307355361CC5571154EEBA68E7F4799F5520656E1AA6BA5E80BA65C6AA190

when decomposing, it looks like this:
48: OP_DATA_0x48:        push hex 48 (decimal 72) bytes on stack
30: OP_SEQUENCE_0x30:    type tag indicating SEQUENCE, begin sigscript
45: OP_LENGTH_0x44:      length of R + S
02: OP_INT_0x02:         type tag INTEGER indicating length
21: OP_LENGTH_0x20:      this is SIG R (33 Bytes)
    00B8E264B5001780:6D4095740E552318
    8C78884DCFCBA3AA:7C1E608F28A1A763
    AF
02: OP_INT_0x02:         type tag INTEGER indicating length
20: OP_LENGTH_0x20:      this is SIG S (32 Bytes)
    159680F7870BE5B4:6B6CD3110629AEAB
    221E6D658DD396AE:319B071E0D2B3396
01: OP_SIGHASHALL:       this terminates the ECDSA signature (ASN1-DER structure)
21: OP_DATA_0x21:        length compressed Public Key (X9.63 form, 33 Bytes)
        0214A307355361CC:5571154EEBA68E7F
        4799F5520656E1AA:6BA5E80BA65C6AA1
        90

This last line(s) after OP_DATA_0x21 is the pubkey, which allows to verify the signature. The corresponding bitcoin address is: 1FNtjbmxmGyt3MvHpWjhoR4ztTabUox3vp.
This whole structure is part of the "input" of a transaction. It shows that the owner had the right priv key to sign the tx, so he is allowed to spend the funds. Now the specification "Pay 2 Public Key Hash" is output related, aka "where shall the funds go to". And this is part of the output structure in this transaction. This string:

76A914E1315C0FA59687EF4E035C184151CFCB096BE4EE88AC

can be decomposed to this:
    76: OP_DUP
    A9: OP_HASH160
    14: OP_Data14 (= decimal 20)
        E1315C0FA59687EF:4E035C184151CFCB
        096BE4EE
    88: OP_EQUALVERIFY
    AC: OP_CHECKSIG
  This is a P2PKH script

which contains after the OP_DATA_14 the public key hash. You can base58check encode this value, which returns this bitcoin address:       1MXiAjDmJzySYm6Re6vV3WgSP7rVc57LMh
So coming back to your question ("How it is possible to validate existing Pay to Public Key Hash transaction without public key?"):
I think it is phrased in a way, that doesn't fit into the logic of bitcoin. You have to prove with the signature, that you are able to spend the funds, and this is done in the input part of the tx. And yes, therefor you need the pubkey, and this pubkey is usually appended after the signature.
The wording "p2pkh tx" specifies an output related way of spending the funds. You could also spend to a multisig, or even to a smart contract tx. This doesn't change the rules of the input/signature/pubkey part.

Thank you for very detailed answer. I will read this and the articles again and again. A question, Can you please tell me how do you decode scriptSig which is this one 4830........ in your example

effectively I have a set of (OpenBSD/MacOS/Linux) shell scripts, which do this for me. I created them based on this site here: https://en.bitcoin.it/wiki/Script.

A transaction which spends from a P2PKH output provides both the signature and public key in the input script. So the public key is part of the transaction data, just the spending transaction, not the output creating transaction.

It is Pay 2 Public Key Hash so it is expected that it will only have Public Key Hash. I can only see scriptSig and Public Key hash but not public key.
Can you please give me an example from blockexplorer ?

The second output of https://blockchain.info/tx/e46a88ed211c1ee7f34f0f4828611da52404c3282416ae1e3b7096f9dddc6c4e?show_adv=true is a P2PKH output. The only input of https://blockchain.info/tx/6bb09e10bb685a032f88689cb4c1b881ce3834f8b918a175032cdc20ba98dc0f?show_adv=true spends that output. In that input script, there are two pushdatas, the first is the signature, the second is the public key.

I see script data there but not scriptPubKey. I am expected that there should to be scriptPubKey there if it is P2PKH transaction. Am i wrong ?

The scriptPubKey is the output script

And verification of signature requires public key, signature and
  message as below VerifySignature(signature, message , public key)

... and the public key itself can be derived from signature and message :)

Can you prove this ? At least  can you point an article for me ? I spent my days understand these details : (

https://bitcointalk.org/index.php?topic=6430.0 note: this is not used in bitcoin


text:
Looking for revenue data of bitcoin related websites
I was wondering if we can share pointers or anectodes about revenues of web projects related to bitcoin or general cryptocurrencies.

The blockchain itself is literally a database of every transaction ever, so if you download it and parse through all of the transactional data, I guess that probably qualifies as 'revenue related stats'.
Blockchain.info has a lot of stats that you may find interesting.

Technically that is a valid answer haha but I meant something like projectx.com made $y in z duration.

Look up the stats on recent ICOs/token sales. They are literally a public sale for a stake in a project/tokens/whatever-the-idea-is, so you should be able to find numbers on how much money each of these projects / companies was able to raise. Not really revenue, but probably relevant.

true but majority of them have no proper business model let alone revenue haha


text:
Metcalfe's law for bitcoin price
Here's an inline link to article
article

the square function number of users times the average transaction value. 94% of the bitcoin moved over the past four years is explained by that equation.

It does't make sense for me about a mathematical equation of 94% explained bitcoin price in this article.
If I'd like to predict a bitcoin price of following day using this equation, is this as following?
　　square function number of users × the average transaction value
　     　　　　     ↓
　　square number of unique addresses per day ×
(trade volume(USD) per day / number of transactions per day)

　　　　　↓
604,054^2 × (597,502,689 / 271,995) = 801,644,073,110,452
date: 2017/11/14 00:00
Here's an inline link to [reference unique addresses
(https://blockchain.info/charts/n-unique-addresses?timespan=30days)
Here's an inline link to [reference trade volume]
(https://blockchain.info/ja/charts/trade-volume?timespan=30days)
Here's an inline link to [reference transactions]
(https://blockchain.info/ja/charts/n-transactions?timespan=30days)
Answer is very huge number.
Could someone explain that concretely where the wrong is?
Thank you


text:
BTC number of transactions
I'd like to get number of transaction's data with API.
I already use API of blockchain.info.
Dose someone know other API?
I want to get number of transaction every a minutes(or less than ten minutes).
It's not one day total average.
I mean BTC.
Thank you

Yeah, you could use Bitcoin.com They don't provide API for total transactions but i've got the link they use to get bitcoin total transactions
Here is it:
https://charts.btctest.net/api/chart/daily-transactions
Response Format is JSON.
I guess you could use it.

Thank you very much.
This is number that is transaction-rate(transaction/second) in a day.
I want to know transaction-rate in a minutes or ten minutes,
like a 3.5 transaction / second in a minutes.
thanks

I guess its another question. BTW You have to use some programming language to do this math.


text:
What did Gavin mean when said minimum fees would be automatically suggested?
From the release notes of the Bitcoin version that introduced minimum transaction fees:

... note that we intend to replace the hard-coded fees with code that automatically calculates and suggests appropriate fees in the 0.9 release ...

Has Gavin expanded on the implementation of this automatic calculation?

Yes, this has actually been going on for a longer time, and it is expected to arrive in the next (0.10) release.
Citing Gavin:

Instead of using hard-coded rules for what fees to pay, the code observes how long transactions are taking to confirm and then uses that data to estimate the right fee to pay so the transaction confirms quickly– or decides that the transaction has a high enough priority to be sent for free but still confirm quickly.

In practice, you'll be able to select the priority of your outgoing transaction and the fees will be calculated accordingly, taking the current "market situation" into consideration.
Once implemented, it will no longer be necessary to dictate or suggest the optimal fee centrally, as it is now.
You can read more in the original article by Gavin Andresen.


text:
Why is anonymity important in bitcoins?
Before I get started I would like to say I am an advocate of a virtual, peer to peer currency and bitcoins to an extent. I think some of the worst aspects of capitalism as well as huge amounts of power have manifested themselves in the finance sector and it would take something as radical as bitcoin to take the power back.
I am interested in why anonymity is important in bitcoins. Some of the other "worst aspects of capitalism" are the tax dodging multinationals that lie and cheat and as a result prevent the fair distribution of wealth amongst the population. Surely if bitcoin was adopted widely as a currency this would only worsen matters as there could be no means of taxation and thus projects like public health, education and law would break down.
What is the problem with making transactions traceable. Would this not make a virtual peer to peer currency more likely to succeed?

Bitcoin is private by default, but one of the Bitcoin devs maintains a noprivacy version.

"as a result prevent the fair distribution of wealth amongst the population".   That's something other than free market capitalism, incidentally.

@StephenGornick who said anything about free market capitalism?

@zode64 fyi, this question is getting votes to close not because people disagree with your position (though I'm sure some do) but because it's hard to give a clear factual answer to this rather than an opinion, and stack exchanges are designed to serve as reference material rather than as a forum.  Perhaps you'd have more luck on bitcointalk.org or in the chat?

@eMansipater thanks for clarifying, I am familiar with how the SE sites work and totally understand. I do however think this question creates a bit of a paradox in this case because if there were a clear answer then it could be provided but the fact that there isn't answers the question quite definitively but also results in it getting closed.

@zode64 I don't think the lack of a precise answer means what you think it does here.  Rather, I think your question would inherently require a personal opinion.  Consider asking the same thing of other technologies, i.e. "why is anonymity important in TOR" or "why is anonymity important in email" or even  "why is text important in email".  All of these questions end up quite philosophical rather than factual or specific.  Then when you add to this the fact that Bitcoin is not especially anonymous anyways and only becomes roughly so when users go to great lengths to hide their actions...

@eMansipater on the contrary, the answer to your example questions is that, on a technical level they are not and that might have been informative in the early years of those technologies. In addition to that, your last sentence is also informative and doesn't address a philosophical point. I think my mistake is that the way I have phrased the questions might encourage a philosophical answer.

Anonymity is part of the Bitcoin culture. It was created to circumvent the powers at be, and anonymity plays into that idea quite well.
However the nature of the block chain is that every transaction is totally traceable. You can look at the block chain and determine where every bitcoin has ever been. This seems to contradice most peoples conception of bitcoin as being an anonymous currency.
In some ways it is. If you mine your own bitcoins and generate your own wallet then, even if I can see every transaction, that particular bitcoin or wallet made, I can not know who is responsible for making it.
But in the new age of bitcoin, where the average user does not know anything about encryption, and contracts third party services such as mt. gox to manage their bitcoins for them, anonymity is lost.

I guess the argument would be that this culture might be it's down fall. Especially if it serves no purpose other than just to be anonymous.

This answer leads me to a question I been meaning to ask: if I deposit fiat currency to an exchange, such as Mt. Gox, then buy BTC with it, and then send those to an offline wallet I created, will there be any mention of my fiat bank account, or other personal information, in the blockchain? Eventhough Mt. Gox may have a record of me depositing fiat currency to buy BTC, surely this is not broadcasted for all bitcoin-users to see, is it?

@fireeyedboy I am almost certain that this information isn't available in the block chain.

Not everyone in the Bitcoin community cares much about anonymity.  However the general rule for cryptographic tools is that they shouldn't force you to reveal information you don't want to.  We can all think of cases where there would be a legitimate use of pseudonymity or increased privacy(oppressive regimes, paying confidential informants, and so on), so since the problem exists technology might as well have a crack at solving it.  As Loourr notes Bitcoin itself doesn't really provide anonymity.  However Zerocoin, a recently proposed inclusion in the protocol, might.

I guess one of the problems with traceability is the creation of wallets and addresses. To trace transactions by entities you would need to verify the wallet owners and this would require some central authority to decide whether the entity was indeed real or just someone who has created a second wallet for the purpose of  funneling money into a second non-taxed account.
It could be possible to put inplace some lame non-central verification system but buy declaring anonymity it doesn't give the illusion of something that could be regulated.


text:
Logical flow to check for receipt of a bitcoin payment
I'm experimenting with writing some Bitcoin software in Node.JS using the node-bitcoin lib to talk directly to my own bitcoind via JSON-RPC, and I am trying to work out the best-practice way to recognise when a bitcoin payment has been received by an address.
The only way I can really think of doing is is as follows:

a recipient provides the Bitcoin address they wish to receive payment to (presumes they have their own wallet already - else I could generate one for them)
My webpage generates a QR code in the proscribed format for a user's smartphone wallet app to scan.
The user scans the QR code and sends payment via their wallet app
I then assumed I could poll getreceivedbyaddress with the recipient's address until the transactions showed up, but the docs explicitly state that this:

Works only for addresses in the local wallet, external addresses will always show 0.

So I assume the solution is to generate a new receiving address for the recipient rather than let them provide their own receiving address. Ideally in this situation I'd generate a new receiving address for each sale the recipient makes.
Then I can easily poll getreceivedbyaddress, analyse the raw transaction when I see it come in, and, assuming it all makes sense, ping a web-socket event to the webpage telling it to close the QR code and spit out a formal receipt.
Does that make sense or is there a preferred way to do this?

related: How to know when a payment has been received by bitcoind?

Cool thanks!  I'll go read up on -WalletNotify

You're welcome. If the linked question answers your question completely, please kindly flag your own question as a duplicate.

Good idea.  I've done that now.