text:
Information to recipient of the transaction
I have some questions of the transactions bitcoin,
In a transaction bitcoin, there is not information on the recipient, right?
How do services such as blockchain.com track information about a transaction, from which wallets have they started and to which wallets have they arrived?

In a transaction bitcoin, there is not information on the recipient, right?

Correct, the address/script that the payment is sent to does not have any user or wallet-identifying information included in it.

How do services such as blockchain.com track information about a transaction, from which wallets have they started and to which wallets have they arrived?

Simply: by making a guess based on certain heuristics. As @torkel mentioned, the ownership of some addresses are publicly known (eg donation addresses, known exchange cold wallets, etc), but for many addresses, the service will just algorithmically group addresses, based on certain spending patterns/conditions. Note that in many cases, these guesses may be incorrect, it is extremely difficult to be absolutely certain about the nature of each transaction output, without being party to the transaction.
As a user, having your wallet addresses identified and grouped is very bad for privacy, and for the fungibility of Bitcoin in general. So there are many techniques being developed and deployed to break the heuristics used by analysis services, for example P2EP coinjoin transactions are effective in breaking some of the most simple heuristics used (all of the inputs for a transaction belong to the same wallet).
This paper from 2013 lists some of the analysis heuristics that chain analysis services may be using. To quote the two most basic heuristics discussed in the paper:

HEURISTIC1.If two (or more) addresses are inputs to the sametransaction, they are controlled by the same user; i.e., for any trans-actiont, allpk∈inputs(t)are controlled by the same user.
HEURISTIC2.The  one-time  change  address  is  controlled  bythe same user as the input addresses; i.e., for any transactiont, thecontroller ofinputs(t)also controls the one-time change addresspk∈outputs(t)(if such an address exists).

Block explorers know some information about wallets because they belong to public entities with known addresses.
Other wallets are known because Bitcoin wallets typically follow a spending pattern where it's very easy to link together outputs and inputs belonging to the same wallet.
For example:

in most transactions all inputs belong to the same wallet
if a transaction has one output of 0.5 (or any other round number) BTC and one output of 0.158293 (or any other non-round number) it's likely that the second output is machine generated and belongs to the wallet of the sender

Thanks for your answar,  then the sites using a matching addresses into transaction input for the Match the transaction input/output, right? Sorry for my terrible english but i'm learning


text:
bitcoin reorg question
if a block takes 10 minutes to be mined
does it need 21 minutes to be reorged?
if the reorg started immediately after the block was mined
because 100% was mining and 51.5% will reorg
how long does it take to reorg 5000 block ? 70 days?
is 70 days of reorging a huge thing that would make the people stop believing in BTC ?
all of my questions and availability here is because i am selling my business for 5300 BTC and i dont need to be F&&&ed with reorging later if i will not use them
this was also my point in the last questions for creating custom wallet offline

what is mean "reorged"? in this context?

refunding the sent transactions with huge reversal mining power

mh with on bitcoin exist the reorged action?

yes but you need much money to reorg

Possible duplicate of Why is 6 the number of confirms that is considered secure?

First you need to define "reorg". Under the definition I'll use, the minimum reorg is two blocks. The definition would be:
"A reorginization occurs when a block is broadcast with a history that is incompatible with the previously work-heaviest** known blockchain, under the same consensus rules"
Say a block is mined onto the height 593000 block, with hash 000000000000000000a11... so that apparently, block 593001 has been found, and many miners are now mining based off of this hash.
Now imagine that someone broadcasts block 593002 with hash 000000000000000000c84...but upon examining block 593001, you see that the hash is 000000000000000000b5c...
A reorg has just taken place. Someone mined block 593001, and without broadcasting it (or perhaps after broadcasting it too late), mined 593002 on top of it. Now any transactions in the block with hash 000000000000000000a11... have not been confirmed (one of which was the original miner sending the block reward transaction to themselves), since the (valid) block that contains them has been orphaned.
As far as "how long", a reorg is relative to the observer, so one could technically take place before you observe the original chain at all.
** essentially, the longest chain, but there are exceptions where a "reorg" could be a shorter chain in terms of number of blocks, when a shorter chain contains "more" proof of work. This is outside the scope of this answer.

so lets assume they want to reorg a 5,300 BTC transaction after 30 days of its creation what is the situation ? if around 100 blocks daily is trying to reorg after 3000 blocks, and here am talking about huge reorging power. i just need to know when there are no chance for reorging to i can gave up the company to them after this time frame from their transaction.

The size of the transaction is completely irrelevant. Realistically, waiting for just 1 confirmation will give you an extremely high chance of your transaction being permanent, especially in the absence of double spend attempts, which are trivial to detect. (I am ONLY referring to the Bitcoin blockchain here. All other blockchains have significantly weaker security. This is the advantage of using Bitcoin vs any altcoins)
For sending bitcoin, 2 confirmations is overkill for any realistic scenario.

its not irrelevant because if its low the miners wont accept to help them reorging but with a huge reword they would accept, so if they risk 2650 btc as fees to the miners to get net of 2650 btc back? that's my point.

No. First, you'd need over 51% of the hashing power to collude to do this, which is essentially impossible to coordinate at all, even if you could convince them. Second, you'd need to convince them to basically break Bitcoin, after they have spent millions of dollars buying specialized equipment that is useless without Bitcoin. And all of this without your counterparty suspecting anything, all for what is essentially still a relatively small tx vs miner rewards (1800 btc are mined daily). It's just not something that would be in anyone's financial interest that had mining machines.

and then on top of that, you've only saved yourself bitcoin, which is now nearly worthless in traditional currency terms due to the 51% attack. So you don't even have upside in this ridiculous scenario.

do you mean that i should giveup taking it as btc ?

could you elaborate? Taking what? Against what alternative?

55,000,000 USD or 5,300 BTC. in both cases i will not use the money for years.

While there are certainly some risks to manage when holding that much bitcoin, a reorg is not one of them.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Connection Refused when running bitcoind in a non-root user and trying to make RPC calls
I'm new to Bitcoin development and I'm currently having difficulties trying to make RPC calls to a Bitcoin-Core daemon running in a SSH server. I suppose that the problem is with Firewall or closed ports, but I also do not know much about Network settings.
I'm using nbobtc/bitcoind-php package to make the RPC calls with HTTP requests, and it is running in a Docker container. I'm sure this is working and is not the problem.
So here's what happening: when I run bitcoind in a normal user in my SSH server, and using nbobtc package to make the RPC calls, the connection always get refused in port 18332 (that is conffirmed when I try to telnet that port in SSH's ip address):

Failed to connect to 192.168.xx.xxx port 18332: Connection refused

However, when I run bitcoind in super user (root), the RPC calls works just perfectly. But as I did not want to run bitcoind in root user, I'd like to find out what may be blocking me from making RPC calls to a normal user - or if is it even possible.
I've tried to open ports with 'sudo ufw allow 18332' and also with IPTABLES (but actually I'm not sure which one is being used by the SSH - it wasn't me who configured it). I'm not sure if they have worked as with nc and telnet commands connection is still being refused.
Here's what my bitcoin.conf looks like (I want to use testnet in here):
server=1
debug=net
txindex=1
testnet=1
rpcuser=userb
rpcpassword=test
test.rpcport=18332

# I've already tried allowing the IP these 3 ways:
# rpcallowip=192.168.xx.xx  # My machine's IP
# rpcallowip=172.19.x.x/xx  # Docker's NBOBTC container IP
# rpcallowip=0.0.0.0/0      # Allowing all IP

datadir=/home/bitcoin-dev/.bitcoin
debuglogfile=/home/bitcoin-dev/.bitcoin/debug.log

PS: I'm using Bitcoin-Core "subversion": "Satoshi:0.17.1"
Here's what appears in debug.log right after I run Bitcoind:
2019-05-06T14:43:10Z Bitcoin Core version v0.17.1 (release build)
2019-05-06T14:43:10Z InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1
2019-05-06T14:43:10Z Assuming ancestors of block 0000000000000037a8cd3e06cd5edbfe9dd1dbcc5dacab279376ef7cfc2b4c75 have valid signatures.
2019-05-06T14:43:10Z Setting nMinimumChainWork=00000000000000000000000000000000000000000000007dbe94253893cbd463
2019-05-06T14:43:10Z Using the 'sse4(1way),sse41(4way)' SHA256 implementation
2019-05-06T14:43:10Z Default data directory /root/.bitcoin
2019-05-06T14:43:10Z Using data directory /home/bitcoin-dev/.bitcoin/testnet3
2019-05-06T14:43:10Z Using config file /home/bitcoin-dev/.bitcoin/bitcoin.conf
2019-05-06T14:43:10Z Using at most 125 automatic connections (1024 file descriptors available)
2019-05-06T14:43:10Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
2019-05-06T14:43:10Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
2019-05-06T14:43:10Z Using 4 threads for script verification
2019-05-06T14:43:10Z scheduler thread start
2019-05-06T14:43:10Z Binding RPC on address 0.0.0.0 port 18332 failed.
2019-05-06T14:43:10Z HTTP: creating work queue of depth 16
2019-05-06T14:43:10Z Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcauth for rpcauth auth generation.
2019-05-06T14:43:10Z HTTP: starting 4 worker threads
2019-05-06T14:43:10Z Using wallet directory /home/bitcoin-dev/.bitcoin/testnet3/wallets
2019-05-06T14:43:10Z init message: Verifying wallet(s)...
2019-05-06T14:43:10Z Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
2019-05-06T14:43:10Z Using wallet wallet.dat
2019-05-06T14:43:10Z BerkeleyEnvironment::Open: LogDir=/home/bitcoin-dev/.bitcoin/testnet3/wallets/database ErrorFile=/home/bitcoin-dev/.bitcoin/testnet3/wallets/db.log
2019-05-06T14:43:10Z net: setting try another outbound peer=false
2019-05-06T14:43:10Z Cache configuration:
2019-05-06T14:43:10Z * Using 2.0MiB for block index database
2019-05-06T14:43:10Z * Using 56.0MiB for transaction index database
2019-05-06T14:43:10Z * Using 8.0MiB for chain state database
2019-05-06T14:43:10Z * Using 384.0MiB for in-memory UTXO set (plus up to 286.1MiB of unused mempool space)
2019-05-06T14:43:10Z init message: Loading block index...
2019-05-06T14:43:10Z Opening LevelDB in /home/bitcoin-dev/.bitcoin/testnet3/blocks/index
2019-05-06T14:43:10Z Opened LevelDB successfully
2019-05-06T14:43:10Z Using obfuscation key for /home/bitcoin-dev/.bitcoin/testnet3/blocks/index: 0000000000000000
2019-05-06T14:43:19Z LoadBlockIndexDB: last block file = 161
2019-05-06T14:43:19Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=755, size=30875345, heights=1513309...1514061, time=2019-04-29...2019-05-03)
2019-05-06T14:43:19Z Checking all blk files are present...
2019-05-06T14:43:20Z Opening LevelDB in /home/bitcoin-dev/.bitcoin/testnet3/chainstate
2019-05-06T14:43:20Z Opened LevelDB successfully
2019-05-06T14:43:20Z Using obfuscation key for /home/bitcoin-dev/.bitcoin/testnet3/chainstate: 2686d59caeb1917c
2019-05-06T14:43:20Z Loaded best chain: hashBestChain=00000000b3b6a5db140b6058b7abe5cb00d8af61afd2a237ae3468cd36e387fa height=927391 date=2016-09-08T15:04:00Z progress=0.311180
2019-05-06T14:43:20Z init message: Rewinding blocks...
2019-05-06T14:43:29Z init message: Verifying blocks...
2019-05-06T14:43:29Z Verifying last 6 blocks at level 3
2019-05-06T14:43:29Z [0%]...[16%]...[33%]...[50%]...[66%]...[83%]...[99%]...[DONE].
2019-05-06T14:43:29Z No coin database inconsistencies in last 6 blocks (500 transactions)
2019-05-06T14:43:29Z  block index           19450ms
2019-05-06T14:43:29Z Opening LevelDB in /home/bitcoin-dev/.bitcoin/testnet3/indexes/txindex
2019-05-06T14:43:30Z Opened LevelDB successfully
2019-05-06T14:43:30Z Using obfuscation key for /home/bitcoin-dev/.bitcoin/testnet3/indexes/txindex: 0000000000000000
2019-05-06T14:43:30Z init message: Loading wallet...
2019-05-06T14:43:30Z txindex thread start
2019-05-06T14:43:30Z [default wallet] nFileVersion = 170100
2019-05-06T14:43:30Z [default wallet] Keys: 2005 plaintext, 0 encrypted, 2005 w/ metadata, 2005 total. Unknown wallet records: 1
2019-05-06T14:43:30Z Syncing txindex with block chain from height 694205
2019-05-06T14:43:30Z [default wallet] Wallet completed loading in             123ms
2019-05-06T14:43:30Z [default wallet] setKeyPool.size() = 2000
2019-05-06T14:43:30Z [default wallet] mapWallet.size() = 7
2019-05-06T14:43:30Z [default wallet] mapAddressBook.size() = 4
2019-05-06T14:43:30Z mapBlockIndex.size() = 1515581
2019-05-06T14:43:30Z nBestHeight = 927391
2019-05-06T14:43:30Z torcontrol thread start
2019-05-06T14:43:30Z Bound to [::]:18333
2019-05-06T14:43:30Z Bound to 0.0.0.0:18333
2019-05-06T14:43:30Z init message: Loading P2P addresses...
2019-05-06T14:43:30Z Loaded 10420 addresses from peers.dat  36ms
2019-05-06T14:43:30Z init message: Loading banlist...
2019-05-06T14:43:30Z Loaded 0 banned node ips/subnets from banlist.dat  29ms
2019-05-06T14:43:30Z init message: Starting network threads...
2019-05-06T14:43:30Z net thread start
2019-05-06T14:43:30Z dnsseed thread start
2019-05-06T14:43:30Z addcon thread start
2019-05-06T14:43:30Z msghand thread start
2019-05-06T14:43:30Z init message: Done loading
2019-05-06T14:43:30Z opencon thread start

Can you provide the contents of the debug.log file?

Sorry @AndrewChow I forgot to put that. I'll edit the question with the content that appears after I run bitcoind, because after that all that appears in there are "UpdateTip", "Requesting block", "received block" and "getdata" messages.

It looks like bitcoind is unable to bind to port 18332. Given that running as sudo works, it sounds like there is a permissions problem with your system which is preventing non-root users to bind to port 18332.

Thanks, it indeed makes sense. Is there any way I could possibly verify why or how my 18332 port is being prevented from being binded?

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to extract raw 64 byte EC signature from an ASN1/DER encoded signature (which ranges from 70 - 72 bytes)
I see that my ASN1/DER signature contains the individual r and s values, is there a way to derive the generic 64byte signature?
edit:  Ive looked through the DER/ASN1 spec, seems that what Im looking for is simply s appended to r.  BUT both r and s can possibly have lengths of 32 or 33.  so its possible for my raw EC signature to be 66 bytes long, which completely breaks the application I use now.
But Ive noticed that whenever the DER specifies eother r or s to be 33 bytes long, the very first byte is 0x00.  I suppose I can simply throw that out and then have 64 bytes?

simply s appended to r.

It is not that simple. DER encoding uses a Tag-Length-Value format. In this format every "value" (like a number, string,...) that you want to encode starts with its own special "tag", a single byte that indicates the type (eg. Boolean is 0x01, integer is 0x02, sequence is 0x03) followed by a number that indicates the length of the "value" called "length" (eg. 32 is 0x20, 130 is 0x8182) and finally the "value" itself.
Since Signatures we are using in bitcoin are 2 integers we use a sequence of integers so the format turns into SequenceTag+TotalLength+SequenceValue where the sequence is the 2 integers each formatted the same: IntegerTag+IntegerLength+IntegerValue.
Now that we covered the format let's look at the integers. You have to remember that DER encoding is designed to encode a ton of things including both negative and positive numbers. One way of including sign of a number in its byte array representation is to use its most significant bit as the "sign bit". So when we have a byte array starting with 0b1000_0000 (128 decimal or 0x80 hex) and any number above that, that is a negative number. As a result when we have a positive number that has its most significant bit set, 1 zero byte is appended to it to make sure it is positive.

r and s can possibly have lengths of 32 or 33. so its possible for my raw EC signature to be 66 bytes long, which completely breaks the application I use now.

The application you use must not hard code values like 32, 33 or 66,... at all because these are maximum lengths not the only possible ones. for instance both r and s can be smaller like 20 bytes so your signature can end up being like this:
30380214<20bytes>0220<32bytes> a total of 58 bytes.

You are on the right track. R and S data bytes typically encode 256-bit values (32 bytes). However, integers in DER are signed, which means that if their highest bit is set, they are interpreted as negative. For that reason, if the R or S value being encoded has their first byte >= 0x80, it would mean a negative number, and hence a null byte is appended taking the length to 33 bytes. So to encode the signature to 64 bytes, just take the last 32 bytes each of the R and S values.


text:
How would these parameters affect mining and the size and content of the mempool?
Q1 and Q2 are theoretical questions to confirm my understanding.  I know doing the following serve little practical purpose.
Q1: If a mining pool were to set blocksonly=1 in their bitcoin.conf, would all blocks created be "empty blocks" (only the coinbase transaction)?
Q2: If a mining pool were to set mempoolminfee very high, would all blocks created be "empty blocks" (only the coinbase transaction) or possibly only a few transactions?
Q3: I can't seem to find the syntax to put mempoolminfee in bitcoin.conf.  Any help appreciated.

Q1: If a mining pool were to set blocksonly=1 in their bitcoin.conf, would all blocks created be "empty blocks" (only the coinbase transaction)?

Yes. The mempool would be empty so the miner would not have any transactions available to add to a block except for their own coinbase transaction.

Q2: If a mining pool were to set mempoolminfee very high, would all blocks created be "empty blocks" (only the coinbase transaction) or possibly only a few transactions?

Yes.

Q3: I can't seem to find the syntax to put mempoolminfee in bitcoin.conf. Any help appreciated.

The option to set is actually minrelaytxfee. This is because a node will only add to its mempool something it will relay, so the minimum relay fee is what controls the minimum feerate to be added to the mempool. Note that this is a feerate, not an absolute transactionfee.
The documentation for this option is:
  -minrelaytxfee=<amt>
   Fees (in BTC/kB) smaller than this are considered zero fee for relaying,
   mining and transaction creation (default: 0.00001)


text:
Bitcoin address generation and UTXO scanning
I'm developing bitcoin payment processing application.
I've two question to ask:

In each bip32 address, How much UTXO each address should have to consider generating new address to avoid address re-using? for example If address A used twice should I generate a new address or does it have specified number for address re-use before generating new one?
When new address generated, from my understanding , I've to scan following address for each address derivation path to see if each index have fund or not:

Legacy
Segwit
Change address for Legacy
Change address for Segwit

What's best way to scan addresses without submitting so much request (for each index) to blockchain explorer to scan funds?
Thanks in advance.

How much UTXO each address should have to consider generating new address to avoid address re-using?

Preventing address re-use by itself means that an address should not be used more than once. So unless, someone sends you two outputs to the same address in the same transaction (highly unlikely) you are going to end up with one UTXO per address. Preventing address re-use helps your privacy. For example, say you used an address to receive funds from your customer. If you use the same address again, these customers can actually scan the blockchain to see how much amount you have received from others. Not re-using addresses more than once helps protect your privacy.

When new address generated, from my understanding , I've to scan following address for each address derivation path to see if each index have fund or not?

Depends on what address format are you using for your payment gateway. If you have always generated bech32 addresses, you need not scan the blockchain for P2PKH addresses to see the amount received from your customers. You only need to scan the blockchain for the addresses that you have generated (receiving or change).

I still don't get it how to handle address scanning,let's say I don't use P2PKH addresses anymore, over time, When transactions grow and that means generating more new address, Is there good way to handle address scanning considering speed and performance? for example If single user does have 100 transaction(100 addresses?), Should I scan 100 blockchain addresses to find out how much funds user have? In over time, when I have more users and they also generate address does it not going out of control?

@nobit You cannot do away without scanning the blockchain. If you want to generate more addresses, you would need to scan them to see you have not magically received some funds (after their use) or if there is a leak in the private keys and your bitcoins got spent.

What's best way to scan addresses without submitting so much request (for each index) to blockchain explorer to scan funds?

You should not build your service to trust some third party blockchain explorer service. If you are handling payments from customers, you should be running your own node, and looking to it for confirmation of payments.


text:
Bitcoin Core v.0.14. doesn't recognise recipient address starting bc1
I have some problem with sending Bitcoin from Bitcoin Core v.0.14. When entering the address in recipient field it becomes red. And I noticed that the address to which the translation needs to be made does not correspond to my. My address contains digits, small and bit letters and address that i need to send contains digits and small letters only and it it has a more length. First 4 characters are "bc1q".  Can you help me in resolving this problem. Thanks!

What are the first 4 characters of the address you're trying to send to?

First 4 characters are "bc1q"

In that case you're just using a way too old version of Bitcoin Core. This is a BIP173 address (using Bech32 encoding). Support was added in Bitcoin Core 0.16 (released in 2018). I would encourage you to update to the latest version (0.21.1 currently), and see if the problem persists.

Thank you for such fast response. Can I immediately set the latest version or is better alternately, starting from 0.16, and then 0.21.1

You can just update to the last version immediately. Make a backup of your wallet.dat file in case something goes wrong, but that's unlikely. There are several questions about details on this site.

As Pieter Wuille explained:
An address starting bc1 is a BIP173 address (using Bech32 encoding). Support was added in Bitcoin Core 0.16 (released in 2018).
You can just update to the latest version immediately (currently 0.21.1). Make a backup of your wallet.dat file in case something goes wrong, but that's unlikely.
(community wiki answer to remove this question from list of unanswered questions)


text:
What actual data format does a Bitcoin Core wallet.dat have?
I'm trying to extract the transaction log from a bunch of wallet.dat files. Not private keys.
All I want is to get access to the transaction log stored inside the wallet.dat.
I can't find any mention of what format this ".dat" file actually has.

Does this answer your question? What do the different .dat files contain?

Your previous question has an answer which covers CSV export and prevention of rescan.

Try here, it mentions "bitkeys" format.
https://bitcointalk.org/index.php?topic=4448.0
Also, on the following wiki it mentions Berkely DB and that pywallet can manipulate the file (so you should be able to reuse the raw reading from pywallet):
https://en.bitcoin.it/wiki/Wallet

That's the format of the dumps created with the dumpwallet RPC, not the wallet.dat format itself.


text:
coinbase script arbitrary data in coinbase input
01000000 .............................. Version

01 .................................... Number of inputs
| 00000000000000000000000000000000
| 00000000000000000000000000000000 ...  Previous outpoint TXID
| ffffffff ............................ Previous outpoint index
|
| 29 .................................. Bytes in coinbase
| |
| | 03 ................................ Bytes in height
| | | 4e0105 .......................... Height: 328014
| |
| | 062f503253482f0472d35454085fffed
| | f2400000f90f54696d65202620486561
| | 6c74682021 ........................ Arbitrary data
| 00000000 ............................ Sequence

01 .................................... Output count
| 2c37449500000000 .................... Satoshis (25.04275756 BTC)
| 1976a914a09be8040cbf399926aeb1f4
| 70c37d1341f3b46588ac ................ P2PKH script
| 00000000 ............................ Locktime

when building the coinbase transaction Arbitrary data is the coinbase script? can you make it whatever you want? :$

BIP 34 requires the script to start with a push of block height.
The script length cannot exceed a limit ( ~100 bytes IIRC)

In the coinbase transaction, the arbitrary data in the coinbase script can indeed be customized, but it’s not just a free-for-all. Miners typically use this space for things like including the mining pool name, extra nonce, or even creative messages. However, it’s essential to ensure that the data inserted doesn't interfere with the transaction’s proper function, as it’s still part of the validated script. Keep it fun, but keep it functional!


text:
How do nodes fetch unconfirmed transactions?
I'm taking up a project to build a software in python to mine cryptocurrency (preferably bitcoin).
I have an overview of the computation that needs to be done.
I would like to know how nodes fetch unconfirmed transactions.

I think that's too big a question to be answerable here. You could break your question down into much smaller parts and research those separately and ask questions here about specific individual points. See what is mining etc.

@RedGrittyBrick Thanks!

So, there are three sub problems to focus on here: 1.) Fetching unconfirmed transactions 2.) Finding the nonce for hashing based on the difficulty and creating a new block 3.) Submitting/broadcasting the new block to the network for verification. I'm stuck at subproblem 1 and 3 from a programming point of view.

You seem to be struggling a little. I will edit your question for you -- though you can do this yourself. If you don't like my edits you can use the "rollback" option under "edit".

Thanks @RedGrittyBrick for making the question more appropriate. I'm gathering a lot of info from the materials provided.

In brief you would use your Bitcoin libraries functions to perform peer discovery and then periodically communicate with a few peers to collect data.
Your question isn't an exact duplicate of any question I can find but existing answers to the following questions will help:

How transaction happen in blockchain?
How do Bitcoin clients find each other?
How are transactions broadcast to the rest of the network?
How does a node get information from other nodes?

Other useful resources

https://en.bitcoin.it/wiki/Protocol_documentation
https://developer.bitcoin.org/devguide/p2p_network.html
This covers the following topics

Peer discovery
Connecting to peers
Initial Block download
Block broadcasting
Transaction broadcasting
This is the part where your node listens for inv messages
and issues corresponding messages such as getdata.


text:
Transaction haven`t got
5 days ago some Pokerroom sent to my Bitcoin Core wallet a slight amount of bitcoin. I checked it on Blockchain, it's OK. But, unfortunately, still there is no bitcoins in my wallet. I've also sent email here - contact@bitcoincore.org with explanation of the problem.
Who's already gone through situation like this please give me advice on what to do.
Many thanks,
Kirill

It would be helpful if you could provide more information, such as: what is the receiving address you gave the Poker site? Do you have a tx id for the withdrawal? Is your bitcoin-core wallet fully synced to the network?

Address         16QQtvsmKExMFojtq5hM6Xpks9Uc5XkmxF
Hash 160  3b46249208fc3d73da62001f238499b52c96d138

Until now my wallet isn`t fully synced with the network.

Your Bitcoin-core wallet will need to finish syncing with the network before it will display your balance.

How could I synce it more faster? Number of the not synced blocks - 91 451 (progress 57,%). I don`t have enough free space on my computer as well - 10 GB / 227 GB.

If you don't have enough free space for the full blockchain, you can instead run a 'pruned node', so that old and un-needed transactions are deleted from your hard drive after they are validated. Look up 'pruning' for more info. There isn't really any way to speed it up, except for using a faster computer, a high-speed internet connection, or an SSD instead of an HDD.

You could also try using a light-wallet that doesn't sync the entire chain instead, but now that you have sent funds to your bitcoin-core wallet, you would need to dump the private keys from it, and import them into the light-wallet, which is a little bit more of an advanced operation, but it is possible.


text:
Does someone know how to recovery a private key
I have more than 60 Bitcoins in an account since 2014.
At the time I was using Bitcoin Armory to manage wallet. After some health problem I have lost the private key but I still have the password. There is any way to recovery my Bitcoin?
As I understand after extensive search it seems to be hard to find the private key of my own BTC address but i fund an interesting site called https://privatekeys.directory/ where seems to be possible to find (theoretically) any private key.
It's really so simple (if someone is lucky) to find the private key of an account ??
If so.. there is no way to protect the account with a password?

After some health problem I have lost the private key but I still have the password.

If you don't have the wallet file, the password is useless, as the password only encrypts the file.
If you don't have the wallet file + password, or the private keys, there is no way whatsoever to recover your coins. Sorry for your loss.

There is any way to recovery my Bitcoin?

No. If you want to have any hope of recovery, you need a backup of your wallet file.

As I understand after extensive search it seems to be hard to find the private key of my own BTC address but i fund an interesting site called https://privatekeys.directory/ where seems to be possible to find (theoretically) any private key.

That site is a (possibly cruel) joke. Yes, it lists all private keys, but in order of increasing private key. It cannot search by public key. Since there are 115,792,089,237,316,195,423,570,985,008,687,907,852,837,564,279,074,904,382,605,163,141,518,161,494,336 valid private keys, and 25 keys per page, you'd need to scroll on average through 2,315,841,784,746,323,908,471,419,700,173,758,157,056,751,285,581,498,087,652,103,262,830,363,229,887 pages before you'd encounter your key. That number is enormous. Even if you had every human on earth go though it, at 100 pages per second, it would take billions upon billions upon billions of times the age of the universe before you had any chance of hitting your key. To be clear, the site just computes the keys on the fly as they are requested. They do not have a full database of all keys - such a database can simply not be constructed.


text:
who controls the price of BTC?
latest edit [ how does the whole world know the price this two people agreed in the same second they decided to do the trade? while most of transactions run anonymously. ]
i also wanna ask how does the BTC prices are posted and moveable every second automatically allover the world around the same price range.
is there's an AI for the BTC who're placing and changing its prices based on math for example with an open source to understand how it works ?
not convincing answers

the point is not why all are the same price for sure they will all do similar prices, the point is deciding the price range

the question is why the btc price is $10,371.68 now, and 2 seconds ago $10,370.63. based on my work for years in the fx market we know there is an AI software run to make the daily and instant small movements to move the market for the traders until whales trade. this softwares is owned by banks. as the source of prices comes from them. now i am trying to understand and discover the same for the btc. where are all prices getting the source of market movement ? did satoshi for example create beside the BTC an AI beside it that decides its prices based on the blockchain transactions? if not, how

This is not a place for conspiracy theories. The price you see is simply the last price any two people anywhere choose to trade at. There are bots involved written by people who have various strategies, but ultimately the choice to trade and at what price is made by those whose money it is. There is nothing more to it.

okay, how does the whole world know the price this two people agreed in the same second they decided to do the trade? while most of transactions run anonymously.

They get published by exchanges. Of course people are free to trade outside of exchanges, and their price is not known to the world. The vast majority of trading happens in open markets, however.

thank you very much. i need to mark your reply as best answer and situation as solved, but your comment seems not direct answer to be able to mark it, i cant find the button to press it

who controls the price of BTC?

No single person.
Like any market, the price is determined by the individual buyers and sellers.
Since most buyers and sellers are human, the price they are willing to buy or sell at depends in part on irrational or emotional factors in part driven by news about various aspects of the global economy as well as news about cryptocurrencies in general and news related to Bitcoin specifically.
Some buyers or sellers may attempt to manipulate the market but, so far as I know, there is nothing in the Bitcoin peer-to-peer protocols that would facilitate this in any unique or special way.

did satoshi for example create beside the BTC an AI beside it that decides its prices based on the blockchain transactions?

No, AFAIK.
Evidence suggests that the person or people behind the name Satoshi Nakamoto are no longer active in the Bitcoin world, certainly not in Bitcoin development. It is of course possible that they are active under another name but there is no way to tell. This part is unanswerable.
So far as I know, there is no significant evidence that price movements involving BTC in currency exchanges act in any way that would contradict what one might expect from reading Satoshi Nakamoto's whitepaper.
I've seen no evidence that there is a AI active in the BTC marketplace that acts any differently (ie. with any special privileges) to any AI active in other currecy markets (e.g. the USD-EUR market)
So far as I know, there are no whitepapers or emails published by  Satoshi Nakamoto that refer to an AI intended to manipulate or determine exchange rates between BTC and another currency.

Footnotes:
This Q&A website is not intended nor suited for discussion in general, it is also not suited to examination of conspiracy theories. There may or may not be other places where those kinds of discussion can occur.

The price of any asset on a market is determined by the last trade that came to pass: whenever a buy order and a sell offer overlap, the current price has been found. Exchanges publish their order books at least to their customers if not publicly altogether.

i also wanna ask how does the BTC prices are posted and moveable every second automatically allover the world around the same price range.

When prices differ across different markets, there is an opportunity for market makers to do arbitrage: They buy the asset where it is cheap, and sell it where it is valuable. Moving the asset from one market to another may incur significant friction, e.g. maintaining fiat balances denominated in various currencies, withdrawal fees, or transaction fees to move cryptocurrencies. As long as there is a significant price gap, the market makers are profiting from arbitrage. When the prices get close to matching the cost of the friction exceeds the gains from arbitrage. Given that all of this data is digitally available and the arbitrage is highly automizable, market makers use trade bots to close arbitrage opportunities in moments. In other words, markets converge in price because it is profitable for market makers to facilitate price convergence.

thank you very much and yes we arbitrage in the forex but we know the source of the prices there.


text:
What happens when bitcoin drops and i add more money to my wallet
If i have $1000 worth of bitcoin with the current price of bitcoin at $7000, then later on bitcoin dropa to $6500, normally i know ths money in my wallet will drop. But my question is what will happen if i add more money in my wallet then later bitcoin goes back to $7000, will i have more than $1000?
I have been wondering for a while and yet haven't come up with a clear explanation.

This really is just a math problem more than a bitcoin question, but it’s simple enough so here is your answer:

If i have $1000 worth of bitcoin with the current price of bitcoin at $7000

($1000)/($7000/BTC) = 0.14285714 BTC
This is the amount of bitcoin you would own. Changes in the dollar value of a bitcoin will not affect how much Bitcoin you own, it will only affect the dollar value of what you own.

then later on bitcoin drops to $6500, (...)  what will happen if i add more money in my wallet then later bitcoin goes back to $7000, will i have more than $1000?

Yes.
To illustrate: let’s say you buy just $1 worth of bitcoin when the price is $6500/BTC:
($1)/($6500/BTC) = 0.00015384 BTC
Now add that to your original BTC balance:
0.14285714 BTC + 0.00015384 BTC = 0.14301098 BTC
Now, at a price of $7000/BTC:
(0.14301099 BTC)*($7000/BTC) = $1001.08

Now i get it. Btc value is unchanged. Thanks


text:
undefined is not an object error when adding BIP39 to React Native
I have a React Native Project and it was worked correctly till I added bip39 package to the project. when i try to require bip39 in my component i get the below error.
undefined is not an object (evaluating 'superCtor.prototype')

inherits
inherits_browser.js:5:45
<unknown>
index.js:23:9
loadModuleImplementation
require.js:320:6
<unknown>
index.js:3:22
loadModuleImplementation
require.js:320:6
<unknown>
browser.js:3:17
loadModuleImplementation
require.js:320:6
<unknown>
index.js:2:24
loadModuleImplementation
require.js:320:6
<unknown>
BIP39.js:1:19
loadModuleImplementation
require.js:320:6
<unknown>
Mnemonic.js:5
loadModuleImplementation
require.js:320:6
<unknown>
App.js:5
loadModuleImplementation
require.js:320:6
<unknown>
index.js:7
loadModuleImplementation
require.js:320:6
guardedLoadModule
require.js:207:45
global code
<unknown file>:0

React Native Version 0.57.7
React Version 16.6.1
bip39 version 2.5.0

Can you provide more information, like when is this error thrown?

@JBaczuk when i add const bip = require('bip39') in component


text:
How to calculate the risk-free rate of cryptocurrencies?
For a conference, I need to show the Sharpe ratio's of Bitcoin portfolios using different trading strategies. To calculate the Sharpe ratio, I need Bitcoin's risk-free rate. The risk-free rate for normal stock markets is usually taken from Treasury documents. But cryptocurrencies have no central government and are not affiliated to any country. So how to determine the risk-free rate for, for example, Bitcoin? Is it simply 0?

Bitcoin itself has no returns: 1 Bitcoin is worth 1 Bitcoin, always.  Presumably you actually want to discuss the returns of a Bitcoin plus ordinary currency pair, such as BTC/USD.  In that case you'd would treat it the same as you would treat any other asset trading against USD and use some presumed near zero risk bond rate, like US treasuries, in your analysis.

I think you should use Treasury bonds from your country to calculate the risk-free ratio.
Risk free rate is just a theoretical rate of return of an investment with "zero risk", which would be your local Treasury bonds.
If you are in US, you should use about 2% for 10 years, which is the approximately rate of return of US bonds.
Anything below this rate of return would not be acceptable for investors, theoretically, as you could invest in a "zero risk" treasury bonds and get 2%. I use quotes because there is no such thing as zero risk, even US bonds.
Bitcoin expected return is much more than 2% a year, so I believe you shouldn't be using this kind of metrics when thinking about investing in bitcoin.


text:
how to calculate Entropy from series of outcome?
I want to generate my own private key with dice and/or other very entropic phenomenon. But how can I calculate if my data have a good entropy ? I mean if I throw dice in a certain way too much time maybe my outcome will not be trully random, maybe my dice is not a very good dice and have imperfection etc.
So can I just throw it 300+ and if I don't have 0.166666% each result (1,2,3,4,5,6) it's not good ?
Also I want to write my own series of dice result just to compare how deficient is my brain when I try to generate true randomness.
Thanks

This seems more a questions for statistics SE or mathemticals SE site.

I'm voting to close this question as off-topic because it's a statistics question unrelated to Bitcoin

Try using the an open source program called ent. Here are the results for rolling a die six times with the result vector of <666666>.
% echo -n "666666" | ./ent
Entropy = 0.000000 bits per byte.
Optimum compression would reduce the size
of this 6 byte file by 100 percent.
Chi square distribution for 6 samples is 1530.00, and randomly
would exceed this value less than 0.01 percent of the times.
Arithmetic mean value of data bytes is 54.0000 (127.5 = random).
Monte Carlo value for Pi is 4.000000000 (error 27.32 percent).
Serial correlation coefficient is undefined (all values equal!).

You may find this old thread on bitcointalk helpful -> https://bitcointalk.org/index.php?topic=24268.0
There are 2^160 possible addresses in Bitcoin. To put that in perspective there's an estimated 2^63 grains of sand on all of Earth's beaches, citation.
If you have a die and roll it 99 times, you have 6^99 different possible combinations. The number of different possible outcomes is so large that even with a dice that has imperfections causing less entropy the likelyhood of someone else rolling the same 99 numbers is astronomically unfeasible.


text:
Server cannot connect to Peers With Some Internet Connections
I use the internet at my work, at my house, and at various other places. The server can connect with other peers and get block data at all of these locations except for my home. What could be the reason for this? I also can't push to GitHub using SSH whenever I am at home, so I'm guessing these problems are related.
I have tried turning firewalls off, and resetting the router. I still always get the "No block source available error".
I know this is similar to the question asked here, but I think it is different because this question asks why it would be happening only on a particular internet connection.

What does your debug.log say? Is it trying to connect to peers?

A Gist can be found here, but I think the important parts might be the last three lines.

What have you got in your Bitcoin.conf file?

It happens with all of my QT wallets, regardless of .conf file. Some of them don't even have conf files.

@GeorgeKimionis, thank you, it seems I am getting closer to an answer. I don't have TOR enabled on my clients, though, and isn't that an opt-in part of the client? Could Comcast be blocking my bitcoin related activity even though I'm not using TOR? Thanks! (Also, you might want to post that stuff as an answer, in case it does lead me to solving my problem)

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

authors: เกเร นนทบุรี


text:
In the BIP39 specification, are seeds generated from encoded or decoded passphrase?
From the BIP39 spec:

From mnemonic to seed
A user may decide to protect their mnemonic with a passphrase. If a passphrase is not present, an empty string "" is used instead.
To create a binary seed from the mnemonic, we use the PBKDF2 function with a mnemonic sentence (in UTF-8 NFKD) used as the password and the string "mnemonic" + passphrase (again in UTF-8 NFKD) used as the salt. The iteration count is set to 2048 and HMAC-SHA512 is used as the pseudo-random function. The length of the derived key is 512 bits (= 64 bytes).

I assume that (in UTF-8 NFKD) means that encoded characters are hashed, rather than the original binary data. Is this correct? If so, is there a security reason why it was done this way?
One consequence of this is that, if the seed is generated from the actual encoding rather than the original bit sequence, then one cannot encode the same bit sequence into multiple languages to give access to the same private keys. This is a pretty big down-side for wallets that provide a language setting. If the user changes their language setting, for example, then they cannot retrieve and enter their wallet words in their new language.

@Bitcoin I don't mean literally translate the same words to produce the same wallet. I mean if you have 12 English words as your wallet words, there should be 12 Spanish words that would produce that same wallet as well, but those 12 words are not language translations of each other at all, they are simply derived from encoding the same binary with dictionaries in different languages.

Right, sorry I wasn't thinking with that comment and got completely the wrong end of the stick. I don't know why that design decision would have been made, a number of things BIP39 design seems to be a little off base. There's a lot of discussion about the shortcomings on github, though ultimately without much in the way of resolution.

@Bitcoin great link, thx. Are they imploring Slush to implement an Electrum 2.x algorithm which uses a checksum? I'm checking my understanding is correct\

I'm not sure of the timeline. I got the impression that Electrum has just deviated and BIP39 isn't going to be changed to fit.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What is the format of the databases that are used in the core bitcoin client?
This question tells what each of the files in the bitcoin directory are, but it doesn't explain the structure of the databases used. What is the table structure (columns, data type, etc) for each of the tables that the bitcoin client maintains?
Thanks!

Is this what you're looking for? http://bitcoin.stackexchange.com/questions/28168/what-are-the-keys-used-in-the-blockchain-leveldb-ie-what-are-the-keyvalue-pair


text:
How are validity flags in block index set in concurrent blocks when one of them is chosen?
I'm trying to understand block validity flags, and what happens when a branch occurs.
I found a block status flags in the bitcoin core source code:
enum BlockStatus : uint32_t {

    BLOCK_VALID_UNKNOWN      =    0,
    BLOCK_VALID_RESERVED     =    1,
    BLOCK_VALID_TREE         =    2,
    BLOCK_VALID_TRANSACTIONS =    3,
    BLOCK_VALID_CHAIN        =    4,
    BLOCK_VALID_SCRIPTS      =    5,

    BLOCK_VALID_MASK         =   BLOCK_VALID_RESERVED | BLOCK_VALID_TREE | BLOCK_VALID_TRANSACTIONS |
                                 BLOCK_VALID_CHAIN | BLOCK_VALID_SCRIPTS,

    BLOCK_HAVE_DATA          =    8, //!< full block available in blk*.dat
    BLOCK_HAVE_UNDO          =   16, //!< undo data available in rev*.dat
    BLOCK_HAVE_MASK          =   BLOCK_HAVE_DATA | BLOCK_HAVE_UNDO,

    BLOCK_FAILED_VALID       =   32, //!< stage after last reached validness failed
    BLOCK_FAILED_CHILD       =   64, //!< descends from failed block
    BLOCK_FAILED_MASK        =   BLOCK_FAILED_VALID | BLOCK_FAILED_CHILD,

    BLOCK_OPT_WITNESS        =   128, //!< block data in blk*.dat was received with a witness-enforcing client
    BLOCK_ASSUMED_VALID      =   256,
};

Supposing that block arrives, the node validates it successfully. I suppose that block status is set to 5 (BLOCK_VALID_SCRIPTS) in this case. Then it is stored in the block index database.
Now another sequence of two valid blocks arrives starting at the same height as the block above. I assume that the working node will consider this sequence as the active branch and the previous block will be abandoned.
What will happen to the index entry for the abandoned block? Will it remain in the database? How these flags or other fields will be set then to mark this block as abandoned?
Is there any easy answer to the question of which branch to follow when traversing the index database or do I need to visit every branch to the very last leaf and choose the longest sequence each time?

What will happen to the index entry for the abandoned block?

Nothing. The block index contains information about the available blocks and branches, and their validity. It does not contain any information about which chain is actually currently considered active by the node. That information is in the chainstate database (which also holds the UTXO set).

Will it remain in the database?

Yes. It is a valid block.

How these flags or other fields will be set then to mark this block as abandoned?

Nothing changes.

Is there any easy answer to the question of which branch to follow when traversing the index database or do I need to visit every branch to the very last leaf and choose the longest sequence each time?

That's a possibility. An alternative is also looking at the chainstate database.


text:
My wallet is protected by AES256-CBC, but I forgot part of the password
My AES256 encryption password is 15 characters. The password uses upper and lower case letters, symbols and numbers. I can remember the first 7 characters of the password, but I have forgotten the last 8 characters which are all digits.
To retrieve the password, I'm planning to use brute force using Python to get the last 8 digits, is this possible?

8 digits might be a date (YYYYMMDD or MMDDYYYY etc) so I would try some birthdays, anniversaries etc.

8 digits, assuming you mean decimal digits, only have 100000000 possible values.
That's trivial to brute-force.

Thanks a lot, I've done it, I use python to brute force to combine the last 8 decimal digits


text:
Does an empty bitcoin.conf file cause my Bitcoin-core wallet to have a balance of 0?
I need help with THE bitcoin -qt logiciel (V23.0) ! two questions : what is normally in bitcoin-conf please!!
and why, this file has no script and does not see the transactions, why? please ?! why ??
is a link that : bitcoin.conf = 0 bytes AND not seeing transactions on my wallet .dat type file opened and synced to the blockchain!?
Thank you, if we check the coins are in the wallet (and auxiliary on various keys privates)  who help me I render thanks !! very very !!

what is normally in bitcoin-conf

It is normal for it to be empty (0 bytes).
It only has contents if you deliberately change the configuration.

[I am] not seeing transactions on my wallet.dat type file [which was] opened and synced to the blockchain

If the blockchain is not fully synced you will see the "catching-up" icon at bottom right of the bitcoin-qt window:
If you have upgraded Bitcoin-core, you should check that a copy of your old wallet.dat is in the place expected by bitcoin-core. For example, in Windows 10 a default location would be %AppData%\Roaming\Bitcoin\wallets

is [there] a link [between] bitcoin.conf = 0 bytes AND not seeing transactions on my wallet

No.
In ordinary use of Bitcoin core, there is no link between these two observations.
If you had previously changed the bitcoin.conf to relocate wallet.dat when using an older version of Bitcoin core but did not follow a normal upgrade process that preserves bitcoin.conf then there could be a causative link. I would check one of my backups to see the previous contents of bitcoin.conf.

Also see

Wallet balance zero after restoring wallet.dat
Exported wallet.dat from bitcoin-qt. Then uninstalled bitcoin-qt. Now the wallet is empty!
How do I import a private key into Bitcoin-Core?


text:
How to retrieve a public key from all types of transaction?
Is there a way to get sender's public key based only on the raw transaction data no matter the P2... type?

Is there a way to get sender's public key based only on the raw transaction data no matter the P2... type

Probably, if you exclude all current transaction types whose most popular names don't begin with the words "pay to ...":

Script type
Public Key in locking script
Public Key in unlocking script

Pay-to-Public-Key-Hash (P2PKH)
No
Yes

Pay-to-Public-Key
Yes
Yes

Multi-Signature
Yes
Yes

OP_RETURN
No
N/A

Pay-to-Script-Hash (P2SH)
No
Yes (in redeem script)

Transaction Puzzle
N/A
N/A

...
?
?

References:

https://www.oreilly.com/library/view/mastering-bitcoin/9781491902639/ch05.html
https://en.bitcoin.it/wiki/Script
https://www.quantabytes.com/articles/a-survey-of-bitcoin-transaction-types

The last of these is the most comprehensive of those listed, however it lacks full details.

The method that can be used to retrieve the public keys in each case (i.e. answer to How?) can be found by inspecting the detailed descriptions in the linked references above. Exact details depend on available tools and skillsets.

@Prayank: Good point, I'll update the answer


text:
Coinbase tells me to send from electrum to my private key?
A few years ago I sent my BTC from Coinbase to Electrum on windows PC.  That apparently went fine because Electrum showed the correct balance.  Then I received 12 keywords and a hash (starting with 17) which I always thought was my bitcoin key.  I saved these on paper and deleted Electrum, thinking OK I have all I need on paper.
Now  want to transfer my BTC back to Coinbase.  I entered my 12-seeds in Electrum and the balance was right. I hit {Receive] in Coinbase and I get a window giving me the Coinbase(?) address to send my BTC to from Electrum.  But this address is the same as what I thought was my private key (starting with 17).  This is weird especially because 1) How would Coinbase still know this number and 2) Electrum says [Pay it] to (what I thought was) my private key.  What am I missing here.  I don't have many bitcoin but the price is too high for me to make a mistake and get an education.

Tl;dr: what you have is

a seed-phrase for a Bitcoin wallet (where you look after your own cash)
a receiving address associated with a Coinbase account (where you let Coinbase have ownership of your cash in return for an IOU)

You don't have a private key (and you don't need one so long as you keep that seed-phrase written down somewhere safe - not on a computer)

I received 12 keywords

That is almost certainly a seed phrase, also called a recovery-phrase or backup-phrase.
Your master private key is generated from that seed phrase.
In a hierarchical deterministic (HD) wallet, that master private key is used to generate many pairs of private keys and public keys.
For the most common types of Bitcoin transaction, a public key is used to generate a bitcoin address to which money can be sent.

and a hash (starting with 17) which I always thought was my bitcoin key.

A string of around 32, 33 letters and digits starting with a "1" is an old-style of Bitcoin address that is still valid and popular. It is a Bitcoin address, not a private key.
To see some examples look at some reference pages such as

https://en.bitcoin.it/wiki/Invoice_address
https://en.bitcoin.it/wiki/Private_key
https://en.bitcoin.it/wiki/Seed_phrase

Note that the same private key can be shown to you in several different ways (encodings, display-formats).
From the above you should be able to reassure yourself that what you have is a 12-word seed-phrase for your wallet and an  unrelated bitcoin address for your Coinbase account.

Thanks RedGrittyBrick.  I believe I understand it all now--and it explains what I have observed.  I had been thinking that what was really the Coinbase destination address for my account was co-equal with the seed phrase, ither woud work m

Thanks @RedGrittyBrick.  I believe I understand it all now--and it explains what I have observed.  I had been thinking that what was really the Coinbase destination address for my account was co-equal with the seed phrase, either would work to access my coin.  Thanks for taking your time to answer my beginner question.


text:
Bitcoinj Spend from a address directly without wallet
I am stuck trying to send coins from arbitary privateKey in bitcoinj. How can I make the following function work without using the Kit or Wallet classes?
@Override
public String transferCoin( ECKey sendingAdress, String destinationAddress, BigDecimal coins) throws Exception {
    return null;
}

Additionally I need to receive the notifications of coins being received on any address in the database, how can I achieve it without the wallet Class?


text:
Why is my "spendable_msat" much lower than "msatoshi_to_us"?
Today I was attempting to send a Lightning Network payment via c-lightning, version v0.7.1-64-gbb30104, using a channel I established in 2018. Strangely, my payment kept failing even if the destination was routable. Finally, I noticed the reason was a relatively large disparity between  spendable_msat and msatoshi_to_us.
"msatoshi_to_us" : 49947931,
"to_us_msat" : "49947930msat",
"msatoshi_to_us_min" : 49947930,
"min_to_us_msat" : "49947930msat",
"msatoshi_to_us_max" : 50000000,
"max_to_us_msat" : "50000000msat",
"msatoshi_total" : 50000000,
"total_msat" : "50000000msat",
"dust_limit_satoshis" : 546,
"dust_limit_msat" : "546000msat",
"max_htlc_value_in_flight_msat" : 18446744073709551615,
"max_total_htlc_in_msat" : "18446744073709551615msat",
"their_channel_reserve_satoshis" : 546,
"their_reserve_msat" : "546000msat",
"our_channel_reserve_satoshis" : 546,
"our_reserve_msat" : "546000msat",
"spendable_msatoshi" : 35617930,
"spendable_msat" : "35617930msat",
"htlc_minimum_msat" : 0,
"minimum_htlc_in_msat" : "0msat",
"their_to_self_delay" : 144,
"our_to_self_delay" : 144,
"max_accepted_htlcs" : 483,
"status" : [
  "CHANNELD_NORMAL:Reconnected, and reestablished.",
  "CHANNELD_NORMAL:Funding transaction locked. Channel announced."
],

As you see, my "spendable_msatoshi" is much lower than "msatoshi_to_us", a difference of 14373070 msatoshis. Even considering "their_reserve_msat" and "our_reserve_msat", it can only explain an amount of 1092000 msatoshis, or ~7% of the unspendable fund, where does the rest go?
Why does this even occur? And what should I do to fix it?
I tried reading the source code of c-lightning, and clearly HTLC and transaction fee is also deduced from "spendable_msat", however, those values are invisible in lightning-cli listpeers and I still cannot determine the reason.
/* Compute how much we can send via this channel. */
if (!amount_msat_sub_sat(&spendable,
             channel->our_msat,
             channel->channel_info.their_config.channel_reserve))
    spendable = AMOUNT_MSAT(0);

/* Take away any currently-offered HTLCs. */
subtract_offered_htlcs(channel, &spendable);

/* If we're funder, subtract txfees we'll need to spend this */
if (channel->funder == LOCAL) {
    if (!amount_msat_sub_sat(&spendable, spendable,
                 commit_txfee(channel, spendable)))
        spendable = AMOUNT_MSAT(0);
}

/* We can't offer an HTLC less than the other side will accept. */
if (amount_msat_less(spendable,
             channel->channel_info.their_config.htlc_minimum))
    spendable = AMOUNT_MSAT(0);

/* We can't offer an HTLC over the max payment threshold either. */
if (amount_msat_greater(spendable, get_chainparams(ld)->max_payment))
    spendable = get_chainparams(ld)->max_payment;

json_add_amount_msat_compat(response, spendable,
                "spendable_msatoshi", "spendable_msat");
json_add_amount_msat_compat(response,
                channel->our_config.htlc_minimum,
                "htlc_minimum_msat",
                "minimum_htlc_in_msat");

You basically answered the question already yourself. Besides the 1% channel reserve you as the funder are responsible for paying onchain fees in case you need to force close the channel by spending the commitment transaction.
C Lightning (and lightning nodes in general) are rather overestimating onchain fees as they cannot rely on the fact that the fees that where taken when the commitment transaction was signed will work in the future if force closing of the channel takes place.
Look for example at this transaction in which my c Lightning node had a channel to test the lightning app (running lnd) and a cross implementation bug produced a forced channel close. You can see that I paid about 100 satoshis per byte in fees resulting in 18312 satoshis. This is the same ballpark number as the value which you describe here. (looking at the mempool this number sounds realistically to me)
Things you can do

Don't force close channels (fees will be much smaller with mutual closes, since the fee is more accurate and only one tx is needed)
Have bigger channels then the relative fee is not so high
Look at the code where fees are set. Propose better strategies in the c Lightning issue tracker or provide an api to set / read them. (as far as I remember there is currently no api for that)
You cannot even do too much as the fees being used are negotiated between you and your channel partner. If the channel partner starts out with a really high fee there is hardly anything you can do to defend against this.

Btw welcome to the rabbit hole of onchain fees with lightning. Doing this for a while now I realized it is one of the most subtle and present challenges that we are facing.

Thanks for the informative and comprehensive answer! From reading those Lightning 101 tutorials, I falsely assumed on-chain fee was something negligible as long as the channel is open. The force-close txfee is really a trap for our young players ;-) I will try to always remember that each outgoing channel has a nonspendable txfee on my side and avoid creating too-small channels in the future...


text:
How to calculate txid for segwit transactions?
I'm having difficulty calculating txid for recent transactions. I followed bip-141 at https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki
For the first transaction in the first block found in blk00165.dat (testnet), I get the following when I aggregate the required data to be hashed:

010000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff2303cdb21400fe56ca615bfe8b8804000963676d696e6572343208020000000000000000ffffffff0220f68e11000000001976a91446868f669409e6e744c6a270767cc27e81220ead88ac0000000000000000266a24aa21a9ed5d06bb0f22a0ffa6d7c63f2769b13f917787554e1235f7eed140fe675ee29c050120000000000000000000000000000000000000000000000000000000000000000000000000

I also printed out the byte stream of the file and it matches. But when I double hash it, I get an invalid txid. Am I missing something for segwit transactions?
My code works for non-segwit transactions, tested with the first few transactions in the very first block.

The txid of a segwit transaction is computed by first dropping all the witnesses, and encoding it in legacy notation, and then computing the double-SHA256 hash of it.
This is the only possible way, as anything else wouldn't be compatible with old clients (which is required by it being a softfork).

Now that makes a lot of sense... I was wondering why txid and wtxid were two separate things.

Sorry for the intrusion, so the hash of transaction before segwit is calculated at the same mode of the transaction after segwit?

Yes, the definition of txid has not changed. Segwit simply added extra data to transactions that does not contribute to the txid.

Another question on Segwit, if the Segwit was been introducing because the txid is malleability, how the softfork fix this problem if the hash is calculated to the same mode?

Because the malleable part (the signature data) is now segregated, and not contributing to the txid anymore. The place where signatures go in pre-segwit transactions is empty for segwit; it's still there, but only containing a dummy. The real signature is elsewhere.

Thanks so much, the my question is stupid because the reason was write in the BIP 0141, sorry


text:
Create a transaction from addresses A and B sending to address C
Related: Using bitcore to create a transaction with two addresses and change address
I would like to create a transaction from addresses A and B sending to address C
In the above question I get a little confused (it looks like hes doing one transaction that sends from addresses A and B to addresses C and D receiving change at address E) because it looks like he is signing it with the two receiving addresses!
Usually when I create and sign a tx with Bitcore I sign it with the sending address's private key not the receiving address (in my case the receiving address would be a user I do not personally know)
The bitcore docs say:

To send a transaction to a multisig address, the API is the same as in the above example. To spend outputs that require multiple signatures, the process needs extra information: the public keys of the signers that can unlock that output.

var multiSigTx = new Transaction()
.from(utxo, publicKeys, threshold)
.change(address)
.sign(myKeys);

var serialized = multiSigTx.toObject();

The two parts to this that confuse me are:

Bitcore quote - To send a transaction to a multisig address...

Is this different from a regular address? I notice that the example is lacking .to(address, amount)

Why is Richard from the related question signing with the receiving address's private keys?

Logically shouldn't my desired action look something like this?:
var transaction=new Bitcore.Transaction()
.fee(fee)
.from([output_A,output_B])
.to(pubkey_C,satoshis-fee)
.change(address_A)                    //don't loose the change
.sign([privatekey_A,privatekey_B]);

output_A is from address A and output_B is from address B (both my addresses)

You are right in how your transaction should look like. However, you are getting confused since you are looking for the wrong concept.
You are not sending funds to a multisig address, that is an address shared by multiple users in which some of them need to sign a transaction or order to spend funds from the address. You are (if I've got it right) willing to spend funds from two different addresses (A and B) in order to send funds to a third address: C.
Therefore, what you should build is a transaction with inputs from both addresses A and B, in which each input may be sign by the corresponding address from the latter two.

In my example the .from function takes two inputs. I wrote output_A,output_B as the inputs because I was thinking of them being the outputs of the previous transaction... we are both correct; thank you!


text:
In Bitcoin core 0.21 which command replaces getaccountaddress
In Bitcoin core 0.21, which command replaces getaccountaddress?
Is there any way to get the same data as getaccountaddress in 0.21?

That very much depends on what you were using getaccountaddress (and accounts in general for). Depending on the answer to that, there may be solutions that replace it with getnewaddress, possibly combined with setaddresslabel, separate wallets, or an entirely different custom solution.

Can give an example ?

which command replace getaccountaddress ?

None, see the notification two and a half years ago in https://bitcoincore.org/en/releases/0.17.0/

Deprecated Method
New Method
Notes

getaccount
getaddressinfo
getaddressinfo returns a json object with address information instead of just the name of the account as a string.

getaccountaddress
n/a
There is no replacement for getaccountaddress since labels do not have an associated receive address.

getaddressesbyaccount
getaddressesbylabel
getaddressesbylabel returns a json object with the addresses as keys, instead of a list of strings.

etc.

is there any way to integrate getaccountaddress to new release BTC ?


text:
Will "Taproot" replace Lightning Network?
Can somebody explain to me if Taproot will fix the issues that Lightning Network solves, without the Lightning Network?
That is, near-zero fees, privacy and fast settlements?
Will Taproot allow me to skip Lightning Network and simply use pure Bitcoin Core?

Does this answer your question? Who would benefit from Taproot deployment?

I have edited the question to focus on the main issue. If you disagree with the edits, please use the "edited..." link and select "rollback".

Will Taproot allow me to skip Lightning Network and simply use pure Bitcoin Core?

Well, Bitcoin Core doesn't support Lightning for now, and probably won't for the foreseeable future, but there is no reason why that can't change over time. I'm going to assume your question is:

Will Taproot allow getting all the benefits the Lightning Network provides with just on-chain Bitcoin transactions?

Absolutely not.
Taproot is an improvement to the script infrastructure that exists on chain. It makes some transactions slightly cheaper, in particular more complex ones, and most importantly makes many types of transactions look indistinguishable, improving privacy.
However, on-chain transactions still suffer from all the same problems that make it much more appropriate as a settlement system for higher layers than as payment system directly:

Slow to finalize transactions
Unpredictable confirmations/fees
No feedback between payer and payee: they can't instantly tell you "your payment was accepted"; you need to just broadcast your transaction out and hope something happens

These are side effects from the fact that on-chain transactions are broadcasted and verified to the whole world, and the whole world needs to get consensus on their occurrence. Lightning solves these by building on the security properties of the on-chain system, and building a fast, reliable, cheap, payment system on top at the cost of additional setup and additional assumptions. You simply can't build a usable payment system where every transaction needs to be broadcast and verified by the whole world - Lightning solves this by making the payments simple participant-to-participant interactions, and only broadcasting settlements to the whole world, or to counter misbehavior.
Lightning however can take advantage of the new features Taproot offers, and be upgraded to use it. In particular, it's possible to make non-adverserial Lightning settlements looks like ordinary payments on-chain.

With respect to the original wording of the question I believe Taproot will actually make the Lightning Network pointfull instead of pointless.
While Taproot allows us to save some space on the bitcoin base layer, it will not make the Lightning Network obsolete. Remember, the main issue with respect to on-chain scaling is that a protocol in which every transaction is broadcasted to every participant is just too inefficient if those transactions should represent a substantial fraction of monetary payments in the world.
With respect to the Lightning Network there are actually many improvements possible for the Lightning Network if Taproot is activated. Most notably, we can have Point Time Lock Contracts (PTLCs) instead of HTLCs which give us way more flexibility in the payment process as the addition of points and secrets is linear whereas the addition of payment hashes and secrets is not linear. E.g.:
H(r1+r2) != H(r1) + H(r2)

The property of linearity allows us to have applications like

stuckless payments
payment decorrelation
escrow services


text:
Money sent to wallet was removed in unexpected second transaction 30 minutes later
I got sent bitcoins to my cake address then the coins went into the account as unconfirmed for about half an hour then they were sent to another address, and the confirmations have finished and i dont have any coins in my account they are sitting in this other account which is similiar address to the one i sent to , so its a cake address, im wondering if i hit refresh bitcoin addy it will go to this new address that was created ? am i on the right track ?
cf5471592253c04cf12a6aba559a7aec531df42cee4a87f85968ce4187dddc16
anyhelp would be great

What is a 'cake address'? It isn't clear to me what your question is, or what you're trying to do.

cake the android app, populated a bitcoin address which i used to send to, but it went there then got sent on to another address simliar to the first one but i dont know how to access it, im wondering if i press refresh new bitcoin addy in the cake app it will be the new address it was sent to

Does the wallet currently show a balance? It is strange behaviour to have the coins forwarded to another address. Are you sure it is a legitimate wallet? I haven’t heard of cake wallet before.

How did you determine "the coins were sent to another address"? Do you mean that the initial transaction got replaced by another transaction? Which address were you trying to get paid to? Is cf547159… the transaction that paid you or the transaction that paid the other address that it went to eventually?

It looks like your money was stolen. Either the app is fake or broken or someone had access to your wallet at sometime (maybe they helped you set it up).
A blockchain explorer shows (most recent at top)
Transactions involving address

Hash: c423c3d9d4e72b16cbfb3a3c1a47c578a56f51bf00be4ecb07c588d957f7cf46

2021-05-21 01:01  55 confirmations  Privacy score: 55

Senders
  0.05796927 BTC    bc1qs69asfc0c0we5ynauzsfjtm7hwp6du6vs40srf

Recipients
   bc1q8tdu705hjqxlzar2mretdrcesuezkzphad2amc  0.05784046 BTC Unspent

Hash: cf5471592253c04cf12a6aba559a7aec531df42cee4a87f85968ce4187dddc16

2021-05-21 01:01 55 confirmations  Privacy score: 49

Senders
  0.01025588 BTC        35KKBAGr5ch6dwMYzrxW1oVdCvMHHoCuKn
  0.04729191 BTC        3QEUGgo3LsLFFqAV38q5bCxP3nH1phbyPi
  0.00069646 BTC        3JervgE4EoMhjbAYH11pQnsxTDQBeK7FKu

Recipients
  bc1qs69asfc0c0we5ynauzsfjtm7hwp6du6vs40srf

So money was sent in and a second transaction immediately removed it.
Just because both addresses start bc1 doesn't mean they are both Cakewallet addresses. Thats not how addresses work. Apart from the bc1 or 3 prefix, they are visually random.


text:
Upgrade bitcoin-core on Ubuntu
I have an installed and running full node (version 0.21.1) on a ubuntu machine. I saw the next version (22.0) is out so I looked at the official release notes to see how I had to upgrade. The only instructions I found were to cleanly and fully shut the node down, then "copy over bitcoind/bitcoin-qt". This raised a number of questions that I hope someone out there can help me with ...

The 0.21.1 unzipped/untared install media created a directory structure which had the directories bin, lib, include, share.

The install instructions that I followed after extracting the media was to "install" the binaries using the command:
sudo install -m 0755 -o root -g root -t /usr/local/bin bitcoin-0.21.1/bin/*
This seemed to copy over the contents of the bin directory to /usr/local/bin and gave appropriate ownership/permissions.

There were no instructions for the lib, share or include directories, so those contents were not copied over to /usr/local/*
The installed bitcoind seems to work regardless, at least I have seen no obvious errors running it for the past month.
So my questions are ..

Should I have "installed" the contents of the lib, share and include directories? If not how are they being seen/referenced while in their extracted location? If they are not being used, why are they there?

When the new version (22.0) is extracted, The official upgrade instructions are to "copy over" one or two binaries. I'm guessing reissuing the install command I put earlier would be appropriate? If not, what about the rest of the contents of the bin directory? What about the contents of the new version lib, share and include directories?

I am used to installing software on linux, this just seems somewhat lacking.
Thank you.

Does this answer your question? How to upgrade Bitcoin Core from 0.18 to 0.20 on Ubuntu 18.04? or (maybe better) Install guide for Ubuntu 18 -- I'm pretty sure both are still applicable to current and recent Ubuntu versions

It doesn't answer my specific questions, but thank you for taking the time. Pieter did provide an answer after yours in case you are interested.

Should I have "installed" the contents of the lib, share and include directories? If not how are they being seen/referenced while in their extracted location?

Bitcoin Core doesn't need "installation" - it's distributed as a set of binaries and other files, but the binaries can be run from any location, and do not access any of the other files.
The other files are:

README.md and share/man/*: documentation
include/* and lib/*: the libbitcoinconsensus library, which can be used to build other applications that rely on Bitcoin Core's validation rules. The logic in these files in built into the bitcoind and bitcoin-qt binaries too, so those don't need the library.

If they are not being used, why are they there?

They're there because you may use them, but if all you do is run bitcoind and bitcoin-qt, that's not the case.

Pieter, thank you for answering my questions about the lib and include contents. That's what I needed.

Here's what I did, with current directory where I wanted the bitcoin-22.0/ directory to be:
curl -O https://bitcoincore.org/bin/bitcoin-core-22.0/SHA256SUMS
curl -O https://bitcoincore.org/bin/bitcoin-core-22.0/bitcoin-22.0-x86_64-linux-gnu.tar.gz
sha256sum --ignore-missing --check SHA256SUMS.asc
tar -xf bitcoin-22.0-x86_64-linux-gnu.tar.gz
sudo install -b -m 0755 -o root -g root -t /usr/local/bin bitcoin-22.0/bin/*
sudo install -b -m 0644 -o root -g root -t /usr/share/man/man1 bitcoin-22.0/share/man/man1/*

HashHouse, thanks for suggesting the man pages can also be "installed".


text:
What is the difference between wallet versions 159900 and 169900
The wallet from 2018 version,159900 to the one i have now which is version 169900,what is the difference in the two diff versions?

169900 is a version that implements a certain feature required for upgrading older wallets to support split HD chains (introduced in version 139900). It exists because the upgrade mechanism required a new version for this necessary feature to be implemented.
The feature added is a pre-HD split keypool. This allows users who upgrade from prior to HD chain split to still use their pre-split keypool. The version number had to be incremented in order to allow the upgrade to work - it is not possible to upgrade a pre-HD split wallet to 139900, the upgrade must be to 169900.
New wallets use 169900 as their version number because it happens to be highest version number.

Thanks andrew,but it still saying i have one version 15990 on chainquery and version 169900 main on the core, is the chainquery one just a mock and untrue then,

159900 just means the wallet as made on an older version. Wallets are not upgraded automatically so the version number will not change until you explicitly upgrade it.

@AndrewChow I think OP is trying to say they're comparing with the output of running the same command on chainquery.com. Marc: it is almost certainly simulated, but we don't have any insight in how that site works.

ThanSk Andrew i thought chainquery was simulated, so the only true info is what comes from the core..only got 8000 blocks to go then i can get to work.,

The difference between version 159900 and 169900 is a minor feature, allowing the upgrade from pre-HD wallets or not. It is unlikely to matter for an end user.
Both versions imply the wallet was last created or upgraded using a Bitcoin Core version from 2018 or later.


text:
Verifying a bitcoin Signature and what is the transaction data that is signed?
Given a bitcoin transaction where it is able to redeem it's UTXO's, how can I verify in Java if the signature is actually a valid signature or not. I was trying to use the Bouncy Castle library where I was trying to recreate the message that the private key signs over.
From what I know I need the following things -
1.The version number(4 bytes)
2.The number of Inputs(1 byte)
3.Transaction hash of the input being redeemed
4.Index of the output being redeemed (4 bytes)
5.ScriptPubKey of the output being redeemed prepended with the length of the script
6.Sequence Number(4 bytes)
7.No of outputs we are signing over(1 byte)
8.Total value of the output(8 bytes)
9.ScriptSig for the output prepended with the length
10.locktime field(4 bytes)
11.hashcode type(4 bytes)
I have tried to capture this information from the transaction data but when I try to verify the signature, it fails. Am I missing something?I want to be able to verify if a signature is correct. Thanks.
From what I have read, Step number 5 is a temporary step and the ScriptPubkey is replaced by some other value but I am unable to understand what that value should be.

What you described only works for transactions with one input and one output.
In general, what you do is you take the transaction that you want to verify, remove the scriptSigs from all of the inputs, and for the particular input you want, you place in the scriptSig the scriptPubKey (prepended by its length) of the output that input spends. If the output that was spent is P2SH, then you put the redeemScript instead of the scriptPubKey. Then you put the sighash type at the end of the modified transaction.
Once you have that message, you need to hash it with SHA256. Most cryptographic libraries take a message and hash it once. However in Bitcoin, the message is actually hashed twice. So to verify a signature in Bitcoin with a general purpose cryptography library, you need to hash the message first, and then allow the library to hash it again when it verifies.
Lastly, the message is different for inputs that spend Segwit outputs. This message is defined in BIP 143.

So in the case where I have one single input spending to two outputs how would this work? Say I hold 500 BTC in one private key and I am sending 200 to Address A, 200 to Address B and 100 to Address C(for sake of simplicity we omit the transaction fee). Then which ScriptPubkey should be put in the scriptSig for input of 500 BTC. @AndrewChow

Awesome! I found the bug. I realised that except for the Scripts everything else is stored in little endian format! It works now.


text:
How does a wallet get the public key from an address?
I was under the impression that addresses are just an encoded, hashed public keys with a prefix and that only the public key is used on the blockchain (not the address) for transactions.
But if the public key is hashed, how does the wallet get the public key from the address to create a transaction?
I did some research which confirmed my doubts that this is not possible. So I was wondering, how does the wallet create a transaction if it doesn't have the public key?
EDIT:
I mean how does someone trying to send 10 BTC get the public key of the receiver for pay to public key from the receivers to address. I fully understand the cryptographic element of private & public keypairs and how bitcoin is spent just not how the public key of the receiver is derived from the address. What I have seen is that (most commonly) transactions are pay to public key hash (P2PKH) rather than what I originally thought which was all txns are pay to public key (P2PK).

Simply put, you don't. The address is a hash of the public key with a checksum and a prefix (1). The reason the addresses are not just public keys is unknown, you would have to ask satoshi that, however it is likely make them shorter. BTC does have P2PK but it is not commonly used as the public key is much longer than an address. Hope this helps!

Perhaps you're confused by the phrasing "the wallet", as it's only the receiver's wallet who needs to know the public key, and he has it, because he created the address in the first place.
In what follows, I'm going to assume a standard 1... address (pay to pubkey hash). That's not the only type of address, but similar ressonings apply to every type of addresses.
In the normal workflow, there are two wallets involved. Let's call them Suzy (the sender) and Rudy (the receiver).

Rudy generates a private key x.
Rudy computes the public key corresonding to x, called P.
Rudy computes the address A corresponding to P by hashing it.
Rudy sends A to Suzy (by sending it in person, or putting it on a website, or showing it as a QR code, or ...).
Suzy constructs a transaction that contains an output addressed to A. Suzy has no clue what public key A corresponds to, but also doesn't care. This address really encodes "require a public key and a signature valid for that key, and the hash of the public key needs to be A".
Suzy broadcasts the transaction on the network, where it will be picked up by miners who put it in a block, finalizing the transfer.
Now when Rudy wants to spend the coins he received from Suzy, he needs to satisfy the condition that Suzy put in the transaction. This requires the public key P (as that's the only known public key whose hash is A), and a signature created using private key x (which is the only private key that can construct signatures that are valid for P). Thankfully, Rudy has both x and P, because he was the one who created them.

Thank you, I always thought transactions where P2PK.

Just to comment that the pubkey hash is not the address. The address is the double hash of the pubkey [ i.e. pubkeyhash=sha256(pubkey), address=sha256(pubkeyhash)=sha256(sha256(pubkey))=sha256d(pubkey) ]

@Jose In common terminology, "pubkey hash" refers to SHA256+RIPEMD160

I had the same question about how wallets get the pub key hash from an address. Here's a nice diagram that illustrates the ability to go from a bitcoin address to and from a public key hash - https://learnmeabitcoin.com/guide/p2pkh

This answer could perhaps explicitly mention that the receiver defines what the output should look like and the sender should never make guesses at what else might be acceptable but either follow the receivers instructions or follow-up.

There are different kinds of outputs a transaction can pay to.
For simplicity I'll explain it with a Pay-to-PubkeyHash (P2PKH) output as an example. I think this is what your question is based on.
With a P2PKH output you are right. An address is just an encoded and hashed public key with a prefix. You can remove the prefix and reverse the encoding, but you can't find the preimage to the hash (in our case the public key).
But you don't need to. As the name Pay-to-PubkeyHash suggests, you only pay to the hash of the public key in your transaction. The receiver can derive the public key, as long as he has corresponding the private key, if he needs to.

thanks, so on the blockchain transaction records dont use the public key of the reciver but the hash of the public key? and then i presume when getting your balance you scan the blockchsin with your address rather than yoir public key?

There are different kinds of transactions. For P2PKH only the public key hash is included when paying to an address (someone spending a P2PKH output reveals the public key though). A wallet knows the private keys, calculates the public keys and their hashes and 'scans' for them.

@oxb10c ok so for P2PKH the wallet doesnt need the public key, but what about P2PK transactions? how does the wallet send those?

For P2PK you don't hash the public key in the address, just encode and add a prefix.

I see, so the receiver would generate his address like this Publickey --> encode (eg AES?) --> add prefix = address and then the sender reverses that proccess?

There is no encryption anywhere in the Bitcoin protocol. For P2PK the sender puts the public key directly in the crediting transaction, and to spend it, the receiver provides a signature. But there are no addresses for P2PK outputs, so the sender needs some other way of figuring out the receiver's full public key. Such transactions are very uncommon now.

I was under the impression that addresses are just a encoded

Yes, addresses are encoded (base58 or base32) to help human readability. They include a checksum to ensure protection against errors in address transcription and entry. A bitcoin address can represent the owner of a private/ public key pair, or it can represent any other script that can be represented through P2SH or P2WSH.

But if the public key is hashed, how does the wallet get the public key from the address to create a transaction?

To create a transaction that spends your bitcoins, you need access to the private key. That is because almost all of the transactions need to include the signature of the private key you hold in order to spend the funds that were locked.
However, you can have a watch-only wallet in case you are apprehensive of using private keys on a network connected device. A watch-only wallet allows you to watch what happens to those cold storage bitcoins without exposing the private keys to the online world. This includes importing your master public key at the account level that helps you create different addresses for that account. With the master public key at the account level, you can create receiving/change addresses as you have access to the public keys that will be used to create those addresses. The software just hashes the public keys and encode them in the way you like (P2WPKH or P2PKH).

@ArielHurdle You cannot derive public key from the address. Addresses are mostly hash160 of the public key and hashes are one way mathematical functions. If you want to create a P2PK transaction, you would need to ask the payee directly for the public key.


text:
Sent bitcoins to my unencrypted wallet and encrypted the wallet before file synced, is there any way i can get them back?
Thanks in advance for any light you can shed on my problem..

I think I sent bitcoins from an online wallet to my offline wallet BitcoinQT(0.81) in august.
The transaction has been confirmed, but not redeemed, (due to my offline wallet not being synced i believe)
I encrypted today, before bitcoinqt had completed initial sync.
Now i've just realised that I've probably lost the associated change address in the unencrypted wallet. as the address it was sent to was not the same as the receive address that showed in bitcoinQTfrom the unencrypted wallet.
i have no backup of the unencrypted wallet.dat

Questions:
Q1. Am I correct that i've lost my bitcoins?
Q2. Does bitcoinqt overwrite the wallet.dat file when it encrypts the wallet? or does it create a new copy, such that I may be able to recover the old wallet.dat using recovery sw like testdisk?
Q3. If answer to Q2 is not overwritten, what is the likelihood of being able to retrieve the unencrypted wallet.dat or change addresses considering the bitcoinqt client was syncing/processing blocks for about 20mins before i shutdown?
Q4. Would it be possible to recover the wallet.dat from the app data folder in a system restore (winxp)?
Thanks again,
22/11/2013
Update Q1: Looks likely that i've either sent them to a different wallet - or my blockchain.info account (with 2FA) was hacked. :(
Update Q2: Bitcoinqt does not immediately overwrite the unencrypted wallet.dat after encrypting the file.
Update: Q4: It may have been possible to recover the walllet.app from the app data folder in system restore (but i didn't have this enabled)
Lesson learnt? too early to say, but always keep a backup...

Why wouldn't the key be in your encrypted wallet?

hey david, thanks for the comment, because it was a private key for a change address?

The (public?) address that the bitcoins currently reside at shows in blockchain explorer is different to the receive address I saw in bitcoinqt with the unencrypted wallet, so i guessed it was a change address (one of the 100 pregenereated that is only kept with my unencrypted wallet, and reset/removed when i encrypted?)

I sent coins from an online wallet to an offline unencrypted wallet, and then encrypted the offline (w/out backup, hence discarded) without either of them having completed initial sync on bitcoinqt

because the address it was sent to wasn't the same as the receive address on bitcoinqt?

Didn't you pick the address to send it to when you requested the send from the online wallet? Where did you get that address from?

I can't remember.. I can remember wanting to put it in cold storage, but i don't remember the actual transaction. it's not been spent though?

The public address where the bitcoins currently are is not in the encrypted wallet, but I think it would be in the unencrypted wallet, can i use the transaction log to find it?

*(10mb transaction log) /rebuild it using the dbprintlog (works) or dbrecover (failed)?