text:
Get a correct SHA256 of a PublicKey as bytes
I’m new in code develop and I’m studying it by myself (books, tutorials, official help of sites…) since a year VB.NET. A lot of information I read is in C# that I rewrite in VB.NET.
I’ve started four months ago to read and write some code about SHA256, Aes, ECDsa… and there are some things those I understand with time and practice, but there are other things that I can’t understand (or find) until read a lot of information. Well in this case, I think I can understand but I don’t know how write the code or understand how SHA256 works.
I’m following these instructions:
http://gobittest.appspot.com/Address
ripemd160(sha256(publickey)) - Where am I going wrong?
<< I think I have the same problem.
I’ll try to be specific and write all necessary code for other people that it could be for their interest. Thank you for advance for all help, links, information that you can give me to follow the right path. Well, this is my situation:
I’ve created a Public key (x coord and y coord) and a private key by ECDsa with SecP256k1 curve.
This is the code:
Private Sub GetKeys()
Using myECDsa As New System.Security.Cryptography.ECDsaCng(vCurve)
myECDsa.HashAlgorithm = CngAlgorithm.Sha256
Dim PrivateKey() As Byte = myECDsa.Key.Export(CngKeyBlobFormat.EccPrivateBlob)
Dim PublicKey() As Byte = myECDsa.Key.Export(CngKeyBlobFormat.EccPublicBlob)

Console.WriteLine(Convert.ToBase64String(PrivateKey))
Console.WriteLine(Convert.ToBase64String(PublicKey))
End Using
End Sub

*vCurve is a var that contains the string “SecP256k1”
Output maybe this:
Private Key: RUNEViAAAADvJ2dk9gUnYF2XDIr5dNITLUXnXJRgXD8A3WHeDe6YkBBTJAlNizJCp5rMG0163U2JfNWEnBKEE7AjU2guVar63+jTifUg8/UrjTfoLkoIgdk8VLnTz1HYDxEvR2GtE70=
Public Key: RUNEUCAAAADvJ2dk9gUnYF2XDIr5dNITLUXnXJRgXD8A3WHeDe6YkBBTJAlNizJCp5rMG0163U2JfNWEnBKEE7AjU2guVar6

I can convert to string and clean data if I write this code into Using
'Private Key
Dim StrPrivateKey As String = String.Empty
For Each bt As Byte In PrivateKey
StrPrivateKey &= bt.ToString("X2")
Next
'Public Key
Dim StrPublicKey As String = String.Empty
For Each bt As Byte In PublicKey
StrPublicKey &= bt.ToString("X2")
Next
Dim StrDigits As String = StrPrivateKey.Substring(0, 16)
'Clean data
StrPrivateKey = StrPrivateKey.Substring(StrPublicKey.Length)
StrPublicKey = StrPublicKey.Substring(16)
Dim StrXCoord As String = StrPublicKey.Substring(0, 64)
Dim StrYCoord As String = StrPublicKey.Substring(64)

Output is:
Private Key: 6624B7D42BA98F09B4782B755DEE4A050A3E5F15D1DFB152E5A0571ABD045797
Public Key: C400BED535B984BADCB6DA96D4167709314085B6E5405C7CE1A3D2CF76FA991DC34F2FAA9533FEFD80E6EAA20ED0BD2D7A30F2696A08570B053159FA7068CE90
Digits: 4543445620000000
Xcoord: C400BED535B984BADCB6DA96D4167709314085B6E5405C7CE1A3D2CF76FA991D
YCoord: C34F2FAA9533FEFD80E6EAA20ED0BD2D7A30F2696A08570B053159FA7068CE90

First Question:
I saw that every Private/Public key has a similar number or 8 bytes. In this example is: 4543445620000000
What is this number? Is it a version of something?
Second Question:
I’ve found a little information about how to get a SHA256 of Public Key. If I get the SHA256 of this example, the result is:
Public Key: 04C400BED535B984BADCB6DA96D4167709314085B6E5405C7CE1A3D2CF76FA991DC34F2FAA9533FEFD80E6EAA20ED0BD2D7A30F2696A08570B053159FA7068CE90
SHA256: f7beda57bd9e0c9838dd4e1336aaa363c2bfa88eca25407b472a620fcaa1626f

Sha256 is correct if I take Public Key as a string (yes, I write “04” at the start and I know that is not a byte). But I’ve read that Public Key must be in bytes, and this is the part where I lost the way. I have PublicKey() as byte, but I don’t know how work with it to get a SHA256. All sha256 tutorials and help about it, hash a single string or files, and as you can see, I need some information that I don’t know how to search.
Thanks a lot.
'------ Edit to write how I'm using SHA256 code. A lot of info I'm reading to learn is form https://docs.microsoft.com
Public Function FxSHA256(ByVal textHashed As String) As String
        If textHashed.Length <= 0 Then
            MsgBox("No text to hash.")
            Return 0
            Exit Function
        End If
        Dim hash As String = String.Empty
        Using mySHA256 As SHA256 = SHA256.Create()
            hash = ModuleConvert.GetHASH(mySHA256, textHashed)
        End Using
        myHash256 = hash
        Return hash
    End Function

In other module I have this:
'HashAlgorythm
Public Function GetHASH(ByVal hashAlgorithm As HashAlgorithm, ByVal textHashed As String) As String
    Dim data As Byte() = hashAlgorithm.ComputeHash(Encoding.UTF8.GetBytes(textHashed))
    Dim sBuilder As New StringBuilder()
    For i As Integer = 0 To data.Length - 1
        sBuilder.Append(data(i).ToString("x2"))
    Next
    Return sBuilder.ToString()
End Function

To second question, can you use sha256.ComputeHash as in Dim hash As Byte() = sha256.ComputeHash(bytes) in https://codeshare.co.uk/blog/sha-256-and-sha-512-hash-examples/?. (Yes, 0x04 is a byte and you should add it to beginning before hashing it, as you did in the last code snippet)

Thanks. Yes, I read this link a few days ago. But works like my code. Now I will edit my Post to write what I'm using for get SHA256

@mcccs, sorry, I want to say that I wrote "04" as a string, and not add as a byte. I know that I need to add a byte in front of Public Key, but in this example I write as string.

Well, I was lost in hashing bytes. Finally this web helped me to understand what happened: https://jackdye.co.uk/generating-a-bitcoin-address-in-c/
In my post I was hashing STRINGS, and this is why SHA256 never will works well. We can display on screen these values as strings, but when we have to do SHA256 or RIPEMD160 we need bytes (a lot of people have or had this problem to understand how to add bytes and hashing them as bytes, not as strings. It's a problem of understanding).
To convert Strings to bytes I use this function:
Public Function HexToByteArray(ByVal HexString As String) As Byte()
    Dim ByteArray((HexString.Length / 2) - 1) As Byte
    If HexString.Length Mod 2 <> 0 Then
        MsgBox("Invalid Hex string.")
    Else
        For i As Integer = 0 To ByteArray.Length - 1
            ReDim Preserve ByteArray(i)
            ByteArray(i) = Byte.Parse(HexString.Substring(i * 2, 2), NumberStyles.HexNumber, CultureInfo.InvariantCulture)
        Next
    End If
    Return ByteArray
End Function

Ok let's start:
Work with strings and convert to byte():
When get the public key as string add a byte:
Dim myString as String = "04" & "C400BED535B984BADCB6DA96D4167709314085B6E5405C7CE1A3D2CF76FA991DC34F2FAA9533FEFD80E6EAA20ED0BD2D7A30F2696A08570B053159FA7068CE90"

Yes, this still a String and we are adding a string "04", but we will convert in a byte. Now Hashing it as a byte:
This is the function for SHA256 for bytes
Public Function FxSHA256(ByVal ByteArray() As Byte) As Byte()
If ByteArray.Length <= 0 Then
MsgBox("No Byte Array to Hash.")
Return ByteArray
Exit Function
End If
Using NewSHA256 As SHA256 = SHA256.Create()
Return NewSHA256.ComputeHash(ByteArray)
End Using
End Function

And this is the SHA256 function for strings (wrong function to get the correct SHA256 in this example)
Public Function FxSHA256(ByVal textHashed As String) As String
If textHashed.Length <= 0 Then
MsgBox("No text to hash.")
Return 0
Exit Function
End If
Dim hash As String = String.Empty
Using mySHA256 As SHA256 = SHA256.Create()
hash = ModuleConvert.GetHASH(mySHA256, textHashed)
End Using
myHash256 = hash
Return hash
End Function

And now, you can understand why I get the correct SHA256 and why I was lost, wrong and confused.
To Hash correctly the string in bytes:
FxSHA256(HexToByteArray(myString))

And if you want to display on screen, you can do that:
For Each bt As Byte In FxSHA256(HexToByteArray(myString))
Console.Write(bt.ToString("x2"))
Next

The same for RIPEMD160.


text:
Security concerns with walletpassphrase
I have some security concerns with walletpassphrase:

If it stores the passphrase for some unit of time. ie. so you can send funds, surely if the server is compromised, within that timeframe, the attacker can obtain the passphrase in its raw format?
If I have somehow misunderstood, ie. perhaps the raw passphrase isn't easily accessible (how?), an attacker could still empty the wallet within that timeframe seeing as the wallet is in an decrypted state.

All roads are leading me to use bitcoin libraries that allow me to create transactions and sign them using encrypted private keys that are only decryptable at the time of signing. I feel way more comfortable using my node for transaction broadcasting / indexing capabilities. Rather than leveraging its in-built wallet.

Wallet encryption protects funds at rest, it does effectively nothing about protecting funds on a live machine that has loaded the wallet to make a transaction.
There's no expectation that the passphrase will do anything in particular to protect online wallets on machines that contain malicious software, as you've surmised.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Syncing blockchain with prune enabled
I'm syncing a node with prune enabled. Will it download the whole blockchain and then prune to my given settings after, or does it prune as it syncs?

I'm currently 34.85% synced
Blockchain directory is 1.3G
Prune depth is 500

It will prune as it synchronizes. Disk usage shouldn't ever reach more than a few GB plus what you configured the pruning limit to.

Thanks and should I enable reindexchainstate with pruning?

No, that'll just make it start over.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Automatically restarting bitcoind five minutes after reboot on ubuntu
How can I run bitcoind automatically five minutes after restart?
So far I have crontab set up to reboot my computer every morning at 4AM (it runs into issues when it stays on for more than a couple days). I also have tried to use crontab to restart bitcoind on @reboot, but haven't had any success with that.
I am currently using Ubuntu 18.10, and bitcoind v0.18.0.0.
Here is my current crontab:
0 4 * * * /sbin/shutdown -r +5
@reboot /usr/bin/bitcoind -daemon -conf=/home/<username>/.bitcoin/bitcoin.conf

Don't you mean @reboot /path/to/bitcoind -daemon ...?  You left out the command name.

If you want to wait five minutes, you can do @reboot sleep 5m ; /path/to/bitcoind -daemon ...

Fixed to reflect actual crontab. I think I've diagnosed the problem: looking at debug.log, bitcoind can't start an http server on reboot because when my computer shuts down, it doesn't kill the original bitcoind process. As a result, when I start up, bitcoind doesn't have access to port 8332. As a result, I added the following to the first crontab entry: 0 4 * * * bitcoin-cli stop; /sbin/shutdown -r +5
I added the sleep 5m. Appears to do the job.

If you just need it to shut down and start as your computer reboots, why not use the systemd service?

@Marty When the computer shuts down the original bitcoind process does get killed. There's something else causing the issue. You may want to look at the at command and trigger it at startup. https://unix.stackexchange.com/questions/146615/run-at-command-in-two-minutes-time


text:
BTC Blockchain WALLET ACTIVITY
Although I can see all transactions in my "wallet activity" I pasted my wallet address into Blockchain Explorer expecting to see the history of transactions in that wallet. It directs to a page that shows no transactions....what am I missing and how can I see this info on the Blockchain?

a wallet usually holds several addresses (HD wallets). maybe you should try putting the ones created before? Or try a different explorer (blockchair, blockstream.info, smartbit, blockcypher, etc). Give us the txIDs of the transactions you are talking about if you have them

Many wallets that are designed to facilitate an easy user-experience will simply display one receiving address at a time. Once that address receives funds, a new address will be displayed. This helps adhere to the best-practice of only using an address once, which is very important for privacy.
With that in mind, if you're just copying the receive address as a means of looking up your funds on a block explorer, then it is expected that you'll be shown an address with zero balance. To view your addresses with a balance, you'll need to look up the receiving addresses of your past transactions (and, the change addresses of those transactions, if you've sent any BTC out).
Note: when you look up addresses on a block explorer, you are giving information to that service: for whatever reason, you are interested in that address. This can be a privacy concern, as the block explorer service can keep logs of devices/ips that query the service for specific addresses.


text:
How to tax trading in colored coins representing fiat?
Say Kraken and Bitstamp both issue colored coins for USD, called KrakenDollars and BitStampDollars. Then I find an colored coins exchange where I can trade between the two, and deposit some KrakenDollars. In the eyes of the IRS, these are still dollars I think. Then I start trading between KrakenDollars and BitStampDollars, and make a respectable profit. Then I put my KrakenDollars back into Kraken, and withdraw to my bank account. How on earth should that be taxed? I was just trading between dollars and dollars...

You trade a bunch of stuff, you wind up with more dollars than you started with, you have a taxable profit.

Don't you have to tax each trade based upon the price at the time and whether it's a short term or long term gain/loss?

@lurfjurv That's going to depend on your jurisdiction, but often no. That would be a like-kind exchange.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
brainwallet.org Low Entropy Generation
I was reading that brainwallet.org uses low-entropy RNGs, so any address generated using their website will be unsafe. I used their website to make a cold storage address last December, and so far no one's stolen anything from it. Is it worth it to make a new address on a more secure website? I've put a large amount of time into the current one; I memorized the first 2/3 of the private key and have the last third written down (most secure method in my opinion).

You should move them. Read this: http://www.reddit.com/r/Bitcoin/comments/295las/35_of_my_btc_gone_pc_not_compromised/

I used WarpWallet. I booted my computer into Linux, copied WarpWallet off my flash drive, ran it, put in my passcode (39 random hex digits that I've memorized), and I had my address. (I triple checked that the same passphrase always gave me the same private key, and all this time the computer was, of course, disconnected from the internet) I wrote down the public key on a piece of paper, then transferred all my old cold-storage bitcoin to it. https://blockchain.info/tx/13fd0f4ef557317871965f12d02e17069dc782d3dce9f2f44c4b751c3c856595

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Where is my account?
I received 0.2 bitcoins on this site and deposited it in my wallet, why it was not deposited.
https://cryptofree.casa/free#

why it was not deposited.

This happens sometimes when people give money to a website without first checking whether that website is genuine and whether the offered benefits are realistic.
Investors who don't check carefully are especially vulnerable to fraud.

Your account is with a website that is often connected with fraud or scam.

Is cryptofree.casa genuine plateform to earn free BTC
What happens to the bitcoin mining if the device changes?
Is "cryptofree.best" website a scam for mining? How does it works?

See also

How to tell if a cryptocurrency exchange or trading platform is trustworthy?
How do I recover from a cryptocurrency scam?


text:
Can a bitcoin address be duplicated?
I'm new to this site. A few years ago I got a bitcoin wallet address but honestly never did anything after that but think there is a few dollars

Unless you have the private key or seed words leading to that address saved somewhere, there is no feasible way to recreate it.

Thank you. Excuse my ignorance, what is seed words?

Seed words are a set of words that can recreate all addresses in your wallet; depending on how old your wallet was, it may predate this process, in which case you would need 1 private key for each address


text:
"Sweep failed" message when sweeping a paper wallet into a Bitcoin.com wallet
I have a bunch of coin in BTC cold storage (paper wallets).  And for the life of me, I cannot figure out how to sweep them out of there.
The best attempt I've had so far was to download the Bitcoin.com app to my phone.  There in the settings it allows you to sweep a wallet.
In both cases, I put in the private key, and it shows me how much is in the wallet.  So it was able to unlock it.  But when I "slide to sweep", I get a simple toast message "Sweep Failed".  And nothing else.  No help.  No other information.  No indication as to what the problem is.  And doing a Google search on this has yielded exactly zero help.
So if anyone has any ideas, like something better, I'm all ears (eyes).

I edited the question to remove the digression because of https://bitcoin.meta.stackexchange.com/q/1005/13866. If you disagree with the edits, click the "edited..." link and choose "rollback"

Well, okay.  But it would have been way better if you were to offer some help with the problem.  :)

I agree it would have been, but I don't use the Bitcoin.com wallet/account so don't know that message from that wallet. I figured I should make the change in order to prevent your question being closed as off topic - that way it is more likely to get an actual answer.


text:
Forgotten password for Bitcoin core wallet
My old computer was trash my new computer btc core wallet with my private keys and password I made my backup wallet locked I forgot my password how can I find my password please help

If you forgot your password, nobody here will be able to help you (that is the point of a password!). If it was a weak password, perhaps you could use some password cracking software to attempt to guess it. If it was a proper strong password and you've forgotten it... you're probably out of luck :(

I wouldn’t delete it, perhaps one day you’ll remember the password, etc

See https://bitcoin.stackexchange.com/search?q=forgotten+password


text:
Can you recover a wallet with only BIP39 seed or BIP32 root key?
Can we access and withdraw BTC if we have bip32 seed and bip32 root key? But we don't have the bip39 passphrase

Was the wallet created using a bip39 passphrase? If it was, you'll need it. If it wasn't, then you won't.

No I mean if someone get my bip39 seed and bip32 root key, he can withdraw my fund or not?

Yes
It's unusual, but yes, you can recover the wallet with EITHER the BIP39 Seed (hex) or the BIP32 Root Key (xprv).  But if your root key starts with xpub then it has no private keys so likely won't help.
To do this on Ian's site (in JPG), simply enter the hex BIP39 Seed, or the xprv BIP32 Root Key.  Of course you will need to pick the correct derivation path tab, either BIP44, BIP49 or BIP84.  Assuming your wallet used standard derivations.


text:
The effect of block arrival time and average block interval on blockchain forking and resolution
Solana docs say:

In Proof of Work consensus, these block times need to be very large
(~10 minutes) to minimize the odds of multiple validators producing a
new valid block at the same time. There's no such constraint in Proof
of Stake consensus, but without reliable timestamps, a validator
cannot determine the order of incoming blocks. The popular workaround
is to tag each block with a wallclock timestamp. Because of clock
drift and variance in network latencies, the timestamp is only
accurate within an hour or two.  To workaround the workaround, these systems lengthen block times to provide reasonable certainty that the median timestamp on each block is always increasing.

My assessment:
Let's quickly discuss BTC's proof of work. Block time is 10m, and it's mostly this much so that two blocks can not be solved at the same time. Even that, there's still a chance this could happen, if so, one of the blocks will be discarded and another one will be added. Now, chance is nodes don't know which block they should add to their chain, but I guess, whichever gets propagated first to them, they add that. Le's say in Bitcoin, 3 blocks were mined at the same time.
Some call this a huge problem that when node receives 2 blocks at the same time, it doesn't know which one was solved first and one block gets discarded while there's a chance that second one should have gotten discarded.  So it turns out timestamp is used only for managing/calculating difficulty.
Am I right what I assessed and what Solana describes as a problem above ?

Solana describes proof of work as a problem because it allows them to assert their extremely fragile (has gone down countless times) "consensus algorithm" called proof of history which is censorship enforcing. They need to sell you on insecure consensus mechanisms in order to pump their own pre-minted coins.

Block time is 10m, and it's mostly this much so that two blocks can not be solved at the same time.

I don't believe this is true.
I believe the primary purpose of the fixed average block time in conjunction with the upper limit on block size is to keep the processing and storage burden on ordinary nodes to an acceptable level (see quotations and references at end).
It is also impossible for a node to receive two blocks at the same time. An ethernet interface receives packets serially not concurrently.  Even on a Gigabit interface, there will be differences of the order of nanoseconds in packet completion times. Even if a node uses multiple independent network connections, it is trivial to arbitrarily serialise concurrently received packets from eth0 before eth1.
When someone such as Solana say "at the same time" I imagine they really mean "sometime during the same block interval". What they probably should have said is something more like "with the same parent block".
Bear in mind:

Real block intervals can vary from seconds to hours.

Receiving nodes are not timing the intervals in order allocate blocks to intervals.

Block arrival time has no influence on block selection in a timescale of several actual block intervals.

When miners see another's block, they immediately stop work on their block with the same parent and start work on the next. Even if that block arrived within microseconds of the prior block leaving almost 10 minutes remaining of the notional current block interval.

It seems to me the last of these points has the most effect in reducing publication of competing blocks with same parent.

Some call this a huge problem that when node receives 2 blocks at the same time

Reordering the last one or two blocks is not uncommon and is well catered for. So I suspect block ordering is not a primary factor in choosing block intervals. The Bitcoin design allows for each node's initial ordering to be somewhat arbitrary. Consider the effect of physical (or effective) distance etc on message propagation times from two distant miners to two other nodes of varying distance.

What did Nakamoto say in 2008?
The Bitcoin white paper has only one mention of "10 minutes":

If we suppose blocks are
generated every 10 minutes, 80 bytes * 6 * 24 * 365 = 4.2MB per year. With computer systems
typically selling with 2GB of RAM as of 2008, and Moore's Law predicting current growth of
1.2GB per year, storage should not be a problem even if the block headers must be kept in
memory.

So the block interval is associated with the storage burden on ordinary nodes.
Earlier in the whitepaper, Nakamoto writes

Nodes always consider the longest chain to be the correct one and will keep working on
extending it. If two nodes broadcast different versions of the next block simultaneously, some
nodes may receive one or the other first. In that case, they work on the first one they received,
but save the other branch in case it becomes longer. The tie will be broken when the next proof-
of-work is found and one branch becomes longer; the nodes that were working on the other
branch will then switch to the longer one.

So this kind of activity is allowed for in the design and is not the alleged "huge problem".


text:
Versions of protocols for BTC and TEST?
I'm dorking around with creating a TEST transaction using libbitcoin's bx command.  Besides setting the inputs and outputs, the bx tx-encode sub-command allows one to specify lock_time and version.  Any idea where one can look to find details about what the current version is for BTC and TEST transactions?

You should use TX_VERSION = 1 and TX_LOCK = 0xFFFFFFFF for mainnet and testnet

Noticed reading through https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki#deployment that nVersion is now 3 after the July 4th soft fork.  Should this mean the bx --version argument should now be 3?

https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki#backward-compatibility defines condition for when nVersion was set to 2.

You're confusing transaction versions with block versions. The answer is correct.

% bx fetch-header -t 365307 -c bx-mainnet.cfg
header
{
    bits 404063944
    hash 00000000000000000eecb8c5dabfede194627d53fddfdb685103e5317a7e175e
    merkle_tree_hash 8b787ac50ec4e7d3890a917ff196938972f940304badd4f3ebb3c1a480d6eeeb
    nonce 3395429232
    previous_block_hash 00000000000000000641fa532675b3e4fcf879cb5e4358895fa7b87e883b13d9
    time_stamp 1436897984
version 3
}
% bx fetch-header -t 503223 -c bx-testnet.cfg
header
{
    bits 486604799
    hash 0000000001eee9c08d12c48f11414c45379e2e7b1b5b5318b012ba0e677609f1
    merkle_tree_hash bb6dc83519a3246adca03a4eeb4d2a7c326693f8c1ed28b6e13824e90ecbde8a
    nonce 561008572
    previous_block_hash 00000000000266ea01bc19141dffa2230e9897259d9ddc058cf41d97328477cb
    time_stamp 1436904967
version 3 }

This is about block versions (which are at 3, since BIP66 took effect). The question was about transaction versions (which have been at 1 since Bitcoin's creation).

Thanks for clearing up the confusion between Block Versions and Transaction Versions.

authors: Татьяна Кукушкина


text:
Do the keys that are derived from a BIP-32 HD wallet depend on an initial seed?
I did not understand the concept of a seed in BIP0032 that well. Would you mind telling me do the keys that are derived from a key tree of Hierarchical Deterministic Wallet (HDW) depend on a root seed? In another words, do the derived keys differ for different root seed values or not?
Thanks in advance
Best Regards

The short answer is yes.
A real example is provided below that applies bitcoin-explorer (bx) client on a UNIX command line interface to synthesize a very simple "low entropy" Bitcoin brain wallet seed that is 256 bits in length:
% echo “Hello its me” | bx base16-encode | bx sha256
cc64b0cb02990d2a17ba0f4274df3e38448454da4fb22a9ca8d23944be144e1d
The $seed above feeds the synthesis process of BIP 32 m/0/12d/4 private extended key below:
% echo $seed | bx hd-new | bx hd-private -i 0 | bx hd-private -d -i 12 | bx hd-private -i 4
xprv9yimFehUHrTzsGso7fg1yXqzGgAAbYJxubKbgZWzVYvf2tSNcQvbReku1xEiqmoHhhHzRsedx2a4YVQ5qf4TtkdP5DMV7dqhnK1ihKZA5Cj
Different seeds feeding the "bx hd-new" command will yield different xprv* extended private keys.
The $pvt_ext_key above can be converted into a traditional elliptic curve WIF key  below:
% echo $pvt_ext_key | bx hd-to-wif
KzWMQhXEz1Z8s5s16ArUvUEgUmX6rFahacnaMi8HMEUKFDSQ6Pks
The $pvt_ec_WIF_key above can also be directly converted into a Bitcoin address with below:
% echo $pvt_ext_key | bx hd-to-address
1ATcSQCeuQB8xNvCH7XSksNUs8SHU1dfEe


text:
RPC call gettransaction does not work on bitcoin 0.10.2
Hello I have upgraded my bitcoind to 0.10.2, some of my rpc request now doesn't work.
This is an example:
bitcoin-cli gettransaction 6f5dfa31bef79d0c8cdd58530fc9f0ed2427e7085d421755f3fe78c
a6ac326ef
error: {"code":-5,"message":"Invalid or non-wallet transaction id"}
i have in bitcoin.conf
txindex=1
reindex=1
any suggestions?

If the transaction is not in your wallet, the RPC call you want to be using is getrawtransaction not gettransaction.
$ bitcoin-cli getrawtransaction 6f5dfa31bef79d0c8cdd58530fc9f0ed2427e7085d421755f3fe78ca6ac326ef 1
{
  "hex": "01000000022b53ea7294eb61c19f45a3923baa0b78ed838131c797fdea02e424fe9c368fc8010000006b483045022100acc3143388da78db06726c89f5c05a2e25560e6df27ff0df1918f6e1f12629b202205ab9ae4b639742450f64e2014eadd6433d269f0d37a3c3fdad2b19076e546ab70121034a506701c7698e39a20f3de3f81b76c623ee7530ccac43465ea69f73c6fbe444ffffffffc555e7d7b46ad83f3d3a43b8152a6b1bf3389e5ca382be9beecc5652be34077b010000006b483045022100bf63e952641d061c4f0d65bc3755d7fec7b503f85f05b38661833ee2089d2587022001e6664c10abd94462ecaa6696ed4d9ab7b48d6e5bd13c06dd3348ac00992202012103fc4d38d770cacce092809eee44433dcca0d75286ff7cfddb267bd777485bf0d0ffffffff0200f90295000000001976a914b5f0f59ed466f998aae81497a2b895e89525d98888acbfc89600000000001976a914d3a9ea24ce1a4448a03abcc8ea50eb8f8d571f1788ac00000000",
  "txid": "6f5dfa31bef79d0c8cdd58530fc9f0ed2427e7085d421755f3fe78ca6ac326ef",
  "version": 1,
  "locktime": 0,
  "vin": [
    {
      "txid": "c88f369cfe24e402eafd97c7318183ed780baa3b92a3459fc161eb9472ea532b",
      "vout": 1,
      "scriptSig": {
        "asm": "3045022100acc3143388da78db06726c89f5c05a2e25560e6df27ff0df1918f6e1f12629b202205ab9ae4b639742450f64e2014eadd6433d269f0d37a3c3fdad2b19076e546ab701 034a506701c7698e39a20f3de3f81b76c623ee7530ccac43465ea69f73c6fbe444",
        "hex": "483045022100acc3143388da78db06726c89f5c05a2e25560e6df27ff0df1918f6e1f12629b202205ab9ae4b639742450f64e2014eadd6433d269f0d37a3c3fdad2b19076e546ab70121034a506701c7698e39a20f3de3f81b76c623ee7530ccac43465ea69f73c6fbe444"
      },
[....]


text:
How to create Ethereum brain address?
I have used bitaddress.org to create a brain wallet for my Bitcoin.
Now I would like to create a brain wallet for my Ethereum using Bitcoin Improvement Proposals (BIP) 32/39/44/49 technologies. Is there any reputable tool (preferably similar to bitaddress.org) or open source command line tools I can use to generate a brain address for Ethereum?
Thank you all

I'm voting to close this question as off-topic because it would be better asked on the Ethereum stackexchange.

Don't use a sha512 brainwallet generator like bitaddress.org to generate your brainwallet. Many people have lost their money by doing so.
Instead use a more secure salted scrypt like warpwallet.
For Ethereum, I made a warpwallet implementation here, that is available as a web version and as a go executable. For extra security you should generate the addresses offline.

Thanks @David I read some problems with brain wallets. As far as I understood, it is because people used not secure passphrase. I chose something complicated and "not human". I was not too sure from the article...Am I safe? In case I chose some weird passphrase to generate the keys, am I safe? Thanks!

@Yura If you choose the passphrase in a random way (e.g. by using a dice), have a big alphabet (upper and lower case letters, numbers, symbols..) and it's long enough (20+ characters for the sha512 brainwallet, 8+ characters for the salted warpwallet) the it's safe.
If the passphrase you choose just looks random but it really isn't (e.g. you used the first letters of each word of a obscure song) then it's not safe.
Using a harder algorithm and salt makes an attach orders of magnitude harder, is easy and there is no reason not to do it.

The approach below is very flexible because it applies not only to Ethereum but other cryptos as well that follow the BIP 32,39 and 44 Hierarchical Deterministic (HD) wallet standards. Because the BIP 39 Mnemonic conversions inherently applies Password Based Key Derived Function 2 (PBKDF2) that uses 2048 rounds of HMAC-SHA512 hashing that will partially compensate for poorly chosen brain wallets with little entropy.
Obviously, your brain wallet phrase, your BIP 39 seed words, and your BIP 39 Passphrase should be kept private, and all operation should be performed on an offline trusted computer.
Step 2 calculates the BIP 44 private key path for Ethereum, m/44'/60'/0'/0/0.
Results for Step 3 can be easily validated by configuring a Trezor with the BIP 39 seed words provided below and a BIP 39 Passphrase. (Ledger wallets use a different non-standard HD path for Ethereum.)
1. Computing BIP 39 seed words from from what should be a complex brain wallet:
% echo -n "This is how you can create a BIP 39 brain wallet that is applied to Ethereum" | bx base16-encode | bx sha256 | bx mnemonic-new
game bless rather reward fun person negative manage walnut blanket normal swamp glue garbage deer flip immense assault sponsor matter ethics scare cushion loop
See mnemonic-new for details for using bitcoin-explorer commands.
2. Computing the first of billions of Ethereum HD Wallet Private Keys, m/44'/60'/0'/0/0:
% echo "game bless rather reward fun person negative manage walnut blanket normal swamp glue garbage deer flip immense assault sponsor matter ethics scare cushion loop" | bx mnemonic-to-seed -p "Your private HD Passphrase" | bx hd-new | bx hd-private -d -i 44 | bx hd-private -d -i 60 | bx hd-private -d -i 0 | bx hd-private -i 0 | bx hd-private -i 0 | bx hd-to-ec
469225ff5a9fe8eea420568657792a2a1beb313db2dba607bdc59661f058c376
3. Computing the associated address for the private key:
% echo -n 469225ff5a9fe8eea420568657792a2a1beb313db2dba607bdc59661f058c376 | bx ec-to-public -u | sed 's/^..//' | ./kec | cut -c 25-64
d3de5bec3f1947563af2f0512bdf9d793225089a

This is a fork of warp wallet with the ability to generate ethereum and other coins using your passphrase. It's meant to be memory and time intensive to prevent brute force guessing.
fyi, i'm the creator of it. If that gives you a piece of mind. ;)
https://xcubicle.github.io/memorypaperwallet/


text:
Fee too high because of a bug in Bitkey application for Apple?
I am new here, hello, yesterday evening I sold 0,99930000 Bitcoins with my Bitkey app. But I was very suprised when it went wrong, and it was seen as a minersfee which Haobtc mining as intermediar transferred it. Please help me Haobtc would not help me.
Thanks Edwin
The transaction details are here.
(304f8e03294ba3dbf0cb66ca9dc5f0a2ae99fff4f880ef252c08ed8e75f877de   0.99927577 BTC 454,889 2017-02-26 21:51:03
1HXfAYqpNnawryLgdare836GFJS9KMLa3f 0.99930000
1DouiKSPh4jqaK5YKeHmMEuKkGkvERcvjx 0.00002423)

Bitcoin: Be Your Own Bank ®

Welcome.  Please keep asking questions.  That is how we build the community here.  Be sure to take the tour to get a point and so you understand how this system works.

I am also new and did my fair work of research. However looking at your scenario if the miner doesnt want to return the fee because of your accident its your problem. Generally good miners will return the fee but that isnt always the case, Its a risk of transactions with cryptocurrency. txid

https://bitcointalk.org/index.php?topic=360099

Welcome to you, too.  The tour is well worth taking.


text:
How to get balance of a bitcoin p2sh address?
I send some BTCs to a p2sh address. How can I get the balance of the address?
Local regtest mode

I've provided an answer... let me know if you specifically wanted to know how to do it with Bitcoin core though

Do a search for it in blockchain.info or similar service.
Eg for an example address 3P14159f73E4gFr7JterCCQh9QjiTjiZrG you can see the result of such a query here.


text:
Is it possible (in some way or other) to expire a txout?
I read Contracts: Will this be possible? where part of the question related to expiring a transaction and one of the replies said that it would never be possible since it causes problems. Unfortunately the reply didn't indicate what problems it would cause.
I can't quite understand from the question/reply whether it was a question of expiring an (unconfirmed?) transaction (which might well be problematic) or expiring a txout from a previously confirmed transaction.
Is the latter entirely impossible by any current method, e.g. scripting and/or a combination of transactions?
One way that I perceive that it might be possible (and I'm only a bitcoin newbie so please take pity on me and help me understand if this suggestion is patently ridiculous to bitcoin experts) would be to have a new bitcoin script operation that could be used in the scriptPubKey of the txout to push the 'current' time on to the stack so that it can be compared (using OP_LESSTHAN, OP_GREATERTHAN, OP_LESSTHANOREQUAL or OP_GREATERTHANOREQUAL) to a fixed expiry time (which is in scriptPubKey and pushed to the stack by scriptPubKey) and only if the current time is less than the expiry time will the rest of the scriptPubKey allow the transaction (i.e. bitcoin payment) to be confirmed.
A note on what I mean by 'current' time...there could be a lengthy debate over the source of current time and the trustworthiness of that source but I'd propose that 'current' time doesn't truly need to be the actual current time so long as it's a reasonably close approximation in the bitcoin realm (about 10 minutes on average but maybe longer, maybe shorter). Thus it could be the timestamp of the previous block in the blockchain; that way every node (assuming no fork) will be judging the txout expiration against the same 'current' time.
An undesirable but arguably acceptable side effect is if a transaction spending the txout is submitted and confirmed in a short blockchain fork then it will get requeued on the longer fork and there is a chance that txout might have expired before the transaction is reprocessed and thus the transaction will fail the second time around.
Another subtlety to aid usability might be to offer two variants of the new 'current' time operation to mirror the two 'variants' of lock_time, one variant to push the previous block's timestamp on to the stack and the other variant to push the previous block's height on to the stack. This way a user can write transactions which are self consistent in time reference or self consistent in height reference.

Forks work a bit differently than you assume: As soon as they diverge from a common source, they are completely independent from each other. They are diverging child branches from a common root, which will never find back together. Each fork considers its competitors to be invalid as they are incompatible to each other. Thus, there would never be any requeueing due to interaction with other forks, instead each fork will consider each transaction for itself as soon as it becomes available.

Expiring a txout doesn't really make sense, since it would destroy the coins, and there is no way to destroy coins. (Coins not spoken for by any txout are contributed as fees.)
You could create a transaction with a lock_time in the future that sends the coins to another bitcoin address. If you give this transaction to a counterparty, they would then have the option to broadcast the transaction to the network after it becomes due, and it would take effect.
Before that time, you could create another transaction without the lock_time restriction, move the coins, and make the counterparty's transaction useless.

My comment is too long so is coming in several parts...
PART 1
This is a somewhat contrived example but illustrates the kind of thing I was contemplating. NB: The example would require the use of (currently) non-standard transactions featuring conditional scripting operations.

PART 2
Conditions:

Alice wants to make give some bitcoins to Bob right now (time t0) but wants to stipulate that they have to be spent by a given future time (t2). Alice doesn't care what Bob spends the bitcoins on.

If Bob doesn't spend the bitcoins by time t2 then Alice wants the coins to be donated to a charitable foundation which is run anonymously by Eve.

Alice expects to die roughly at time t1 where t1<t2.

PART 3
Corollaries:

Alice needs to submit a transaction (Tx0) at t0 with a txout that transfers the bitcoins to Bob so that Bob can spend them.

Alice and Bob don't want Alice to be a signatory on Bob's (as yet unwritten) spend transaction (Tx.Bobspend) principally because if Alice dies at t1<t2 before Bob has generated Tx.Bobspend, Alice is no longer around to sign Tx.Bobspend even though we haven't reached t2 yet.

PART 4

Although Alice could generate a transaction (Tx.Alicecharity) now with a txout that transfers the bitcoins to the foundation she doesn't know Eve's identity and thus has nowhere to send Tx.Alicecharity to.

PART 5
So the Alice's Tx0 would have a txout that works something like this when somebody tries to spend it with a transaction (Tx.future) at some time in the future (t.future):
If t.future < t2 and signatures on Tx.future match Bob's bitcoin address then let Bob spend the bitcoins,
Else if t.future > t2 and signatures on Tx.future match the foundation's bitcoin address then let the foundation spend the bitcoins,
Else t.future is invalid.

There is no way to put a reference to the current time (or other monotonically increasing time-like function) in the script, so I don't see a way to do this with just one transaction. However, you could write a 1 of 2 multi-sig that transfers the coins to A1 and B1 (where A1 is controlled by Alice and B1 is controlled by Bob), and send Eve a transaction moving the multi-sig coins to E1 (an address controlled by Eve) with lock_time set to t2. Then, Eve is free to broadcast the transaction, snagging the coins from the multisig transaction after time t2.

did you mean 'currently no way' (agreed) or 'never be a way'? I was floating the idea of one or two new scripting function(s) for scriptPubKey use that would push the previous block's height or time stamp to the stack preceded or followed by a push of a constant (expressing the 'expiry' time) and then an op_lessthan or op_greaterthan to determine if the 'current' time is before or after the constant. Given that nodes already have some concept of the 'current' time to check Tx NLock_time it didn't seem to much of a step onwards...but my understanding is probably too simplistic. :-/

by the way, your (thoughtful) answer appeared to suggest that Alice would send a signed, lock_timed Tx to Eve but my (very contrived) example had already explicitly stated that Alice didn't know who Eve was so had no way to send Eve anything. (Notwithstanding the possibility of Alice's use of a trusted 3rd party to broadcast the lock_timed Tx on behalf of Eve once the lock_time had passed.)


text:
Bitcoin mining on headless Ubuntu 12.04 server
I've googled a lot and also read threads here but I'm still struggling.
Background:
I'm a eager but novice Linux user
I've an Ubuntu 12.04 webserver sitting pretty and is not doing much.
I would like to use the free processing power to mine Bitcoins - makes cents (sense).
Thus, I have Ubuntu 12.04. I have installed bitcoind and made a bitcoind.conf file as per Ubuntu Linux -- How do I start bitcoind as a service to run automatically?
(Basically, copy and paste)
I've installed bfgminer and I've register an account with Eclipse Mining Consortium (eclipsemc).
Now my questions:

I've read the bitcoin needs to download a large data file (+- 6Gb).
I can't find any log files to tail.  How do I know bitcoind is
working properly?
$ ps aux | grep bitcoind
1000     30080  0.0  0.0   9364   368 pts/1    D+   21:21   0:00 grep --color=auto bitcoind

When I run bfgminer (bfgminer -o http://eu.eclipsemc.com:8337 -u [MyUserName_worker] -p [password] -L bfgminer.log), I have to press "M", then "+" and enter "auto" before it will pick CPU 7 as the device.  How can I ensure that it will pick CPU 7 as the default device? How can I run this as a background process?
I still need a bitcoin wallet?  I'm using Armory on my Windows box, what is a compatible one for Ubuntu webserver?
How do I ensure that bitcoind and bfgminer doesn't kill my server?

Thank you very much!
André

A bit off-topic: it doesn't really make sense anymore mining Bitcoin with CPU, you do realize that?

@Jori Not even in a pool? Any suggestion on how to earn using free CPU power?

No, not even in a pool. I'm sorry. You could take a look at LiteCoin or some other Scrypt based cryptocurrency, but I doubt that you will make any significant profit on that too (unless you have some serious CPU power).

I don't think there is anything that will make you a profit purely from that small amount of CPU power, but you could share it for scientific purposes (which is unpaid of course) like calculating new prime numbers / molecule modeling, etc.

It seems that CPU based mining is not viable any more.  Even going to other cryptocurrency, which difficulty level should rise quicker as people mining bitcoin using hardware will mine these too.
Ah well, missed the bus :)


text:
How to check that I control an address
I recently created a bunch of bitcoin addresses (public starting with 1, PK starting with 5) from a paper wallet, that I downloaded from their github (https://github.com/walletgeneratornet/WalletGenerator.net) and ran offline.
I imported the first PK in my full node, it translated to the pub address I expected and I control this address.
I don't feel comfortable sending coins to addresses I don't know for sure I control. How can I check securely and "easily" that I control them ? Adding all of those PK in Bitcoin core is long and decreases security.

Even though you control the private key, private keys generated by distrusted code shouldn't be used. You might not be the only person to have generated that private key. They might have a long but "short enough that they can scan it" list of private keys and they might be querying the blockchain explorer until they see someone make a payment to it.

It's impossible to "check" if the address is controlled only by one entity (you).

Test the windows python code below with the private/public key generated from
website. Just to verify the correctness. If correct proceed.
Throw coin 256x get your private key.
Throw dice whatever, until you get 256 bits.
enter the private key in the code below.
you should get your bitcoin address.

Code:
# COMMENTS
import bitcoin
import binascii
PrivKey1 = 'e8ee7398abbbdc8920f53c8fd454e86eab74beae2a9c42d28bf86402f3d87fcb'
PrivKey1Type = bitcoin.get_privkey_format(PrivKey1)
print("Private Key types")
G = bitcoin.encode_privkey(PrivKey1, 'hex',0)
print("Type: hex", G)
G = bitcoin.encode_privkey(PrivKey1, 'hex_compressed',0)
print("Type: hex_compressed",G)
G = bitcoin.encode_privkey(PrivKey1, 'wif',0)
print("Type: wif",G)
G = bitcoin.encode_privkey(PrivKey1, 'wif_compressed',0)
print("Type: wif_compressed",G)
PubKey1  = bitcoin.privkey_to_pubkey(PrivKey1)
PubKey1type = bitcoin.get_pubkey_format(PubKey1)
H1 = bitcoin.encode_pubkey(PubKey1, 'bin')
print("Public Key")
print("Bin format", H1)
H2 = bitcoin.encode_pubkey(PubKey1, 'bin_compressed')
print("Bin Compressed format", H2)
H3 = bitcoin.encode_pubkey(PubKey1, 'hex')
print("hex format", H3)
H4 = bitcoin.encode_pubkey(PubKey1, 'hex_compressed')
print("Hex Compressed format", H4)
K1 = bitcoin.decode_pubkey(H4)
print("decoded format", K1)
Hash1 = bitcoin.bin_hash160((H1))
print("PubKey Hash from Uncompressed PubKey", binascii.b2a_hex(Hash1))
Hash2 = bitcoin.bin_hash160((H2))
print("PubKey Hash from compressed PubKey", binascii.b2a_hex(Hash2))
BTCaddress = bitcoin.pubkey_to_address(PubKey1, 0)
print("Bitcoin Address", BTCaddress)

This doesn't ensure that only you control the address.  If you have malware on your computer, your private key could have been sent elsewhere the instant you typed it in.

in my answer, i asked the user to generate their private key by throwing coin 256x. He can run the code offline. Malware issue is beyond the scope of discussion. Even hardware wallet got bug with malware.[link] (https://twitter.com/ericsavics1/status/1271589769336598528)

Since 2016, i go through some length advise explanation here and there but in the end i still got nowhere. So, I decided randomness is best using throwing coins. and the algorithm to generate addresses is fixed. So I wrote the above code. U can use it. rather than lengthy comment, u get a code for secure generation of BTC address.  If you are paranoid, then you look at the library call as well.

this is close to what I want. However it doesn't check on the bitcoin network, just because the pub key generated from the PK is the same that I had originally, it should mean I can sign with this key ?

well, at least it is correct! if in doubt U can always try it with a tiny amount first! but then most wallet already have BIP 32 /44 where address is used only once. that one i need more time to code as i not a good programmer.

don;t sign with a doubtful key ! U can always generate a new one. Just spend your time throwing coins.

I do not feel safe with wallets generated keys that I can't see nor store.I want my PK on a piece of paper

calculate the public key on paper by hand, so no computer every gets access to it. a bit hard but maximum security if you dont trust computers (or do it on a computer without a network connection and wipe it before reconnecting)

The correctness of calculation of an address (and a public key) given a secret-private key can be double checked using another software (if you don't want to use Bitcoin Core), e.g.:

Seed Tool https://bitcoiner.guide/seed/ -> single address.
pubkeybitcoin https://github.com/GregTonoski/pubkeybitcoin (public key only).

If a public key (or address) is the same as the one calculated by WalletGenerator.net for your secret-private key then I would conclude that your control of the address  has been checked and confirmed.


text:
ECDSA: (v, r, s), what is v?
Deterministically signing a Tx with RFC6979 returns v, r, s, where r and s are the 2 values used in standard ECDSA signatures. v = 27 + (y % 2), so 27 + the parity of r, as pybitcointools indicates.
So for even r, we get v = 27, odd r we get v = 28.
How is the value of v (27 or 28) important? Why is it necessary to have a v value? Also, why is it 27?

Seems to be some sort of a version byte, similar to the version byte used in encoding addresses. Not sure of the significance of 27/28.

Your link to github is broken; I would fix it but apparently I need to change at least 6 characters for it to be a valid edit.

@DavidGrayson Link fixed. Perhaps Peter Wuille can comment, as I believe I've read that 27 was chosen arbitrarily because it was "halfway" between used version byte choices, though I can't find that source for the life of me

That's correct. It was an arbitrary choice for a number at the time, because we needed a byte to convey public key recovery information. I'll try to write a full answer later.

This has nothing to do with RFC6979, but with ECDSA signing and public key recovery.
The (r, s) is the normal output of an ECDSA signature, where r is computed as the X coordinate of a point R, modulo the curve order n.
In Bitcoin, for message signatures, we use a trick called public key recovery. The fact is that if you have the full R point (not just its X coordinate) and s, and a message, you can compute for which public key this would be a valid signature. What this allows is to 'verify' a message with an address, without needing to know the full key (we just do public key recovery on the signature, and then hash the recovered key and compare it with the address).
However, this means we need the full R coordinates. There can be up to 4 different points with a given "X coordinate modulo n". (2 because each X coordinate has two possible Y coordinates, and 2 because r+n may still be a valid X coordinate). That number between 0 and 3 we call the recovery id, or recid. Therefore, we return an extra byte, which also functions as a header byte, by using 27+recid (for uncompressed recovered pubkeys) or 31+recid (for compressed recovered pubkeys).
Strictly speaking the recid is not necessary, as we can just cycle through all the possible coordinate pairs and check if any of them match the signature. The recid just speeds up this verification.
In general, if h is the cofactor, the maximum number of different points with given "X coordinate modulo n" will be 2(h+1). In the case of secp256k1, which has cofactor 1, we get 2(1+1) = 4.

Can you link to the code which implements this? I'm going to start learning C++ using functions like this which I understand

The signing code is in libsecp256k1, which also has verification code, but that is not currently used. For verification, there is C code in Bitcoin Core (see pubkey.cpp) which does a series of OpenSSL calls to do the bulk of the work. I am not sure that reading cryptographic C code is the best way to learn C++ :)

It's more a way of learning things I understand pretty well already, and am interested in. I've learned Python that way and the ultimate goal is to know the core software too

@PeterWuille There can be up to 4 different points with a given "X coordinate modulo n". (2 because each X coordinate has two possible Y coordinates, and 2 because r+n may still be a valid X coordinate). I understand the former (2 y values for each x, because of the symmetry)... But how does the latter work? ie r+n may still be a valid X coordinate??

X and Y coordinates are numbers modulo p, the field size, which is around 2^256 - 2^32 for secp256k1. The value r and s in the signature however are modulo n, the group order, which is around 2^256 - 2^128. When R.x is between n and p, r is reduced to R.x-n. Thus, if you have an r value below 2^128-2^32, there may be 2 valid R.x values corresponding to it.

@PieterWuille - Is this specific for secp2561k, or can this be generalized for all elliptic curve signatures? Also, how does it relate to this answer on crypto SE - https://crypto.stackexchange.com/questions/18105/how-does-recovering-the-public-key-from-an-ecdsa-signature-work  ?

ECDSA always supports public key, regardless of the curve. For curves with cofactor > 1 it is slightly more complicated. That linked answer explains the procedure.

Which, please, of the 4 possible output (27..30) corresponds to which of the possible results. Eg: 0 = positive Y and X1? where is this documented?

@Sentinel You can get full information on the recovery method here: http://www.secg.org/sec1-v2.pdf

27 = lower X even Y. 28 = lower X odd Y. 29 = higher X even Y. 30 = higher X odd Y. Note that 29 and 30 are exceedingly rarely, and will in practice only ever be seen in specifically generated examples. There are only two possible X values if r is between 1 and (p mod n), which has a chance of about 0.000000000000000000000000000000000000373 % to happen randomly.

@Souza / Pieter - Thanks. I had already managed to discover the answer to that - it took me forever! I did actually find that ASN.1 in sec1-v2.pdf after an enormous amount of digging. When I found it, I thought I'd struck gold. The problem is that the Ethereum documentation (I am implementing an Ethereum light client) does not include detailed protocol info, and official implementations tend to rely on libraries that go where Bitcoin already trod, so details like the 'v' parameter are lost/buried in very, very obscure code.

@Sentinel my strugle's the same. I am working on top of Ethereum's protocol and they say little to nothing about this. And you're right, they both use libsecp256k1.

@Souza - agreed. It is a struggle. I am trying to contact Ethereum foundation to put together accessible protocol documentation and get funding to do it. I am putting together a light client using reactive extensions and .NET, where I am treating all protocol messages as streams flowing through stack-layer adapters, and trying to insulate the top level protocol from lower level differences (discv4 vs v5 etc for example). By the time I am done I should be able to write a book on it.

As all the other answers already outline: v is required to recover the correct public key for a signature because there are sometimes (even with low probability) more than one valid public key to be retrieved by a signature.
Here's a cheat sheet:

27 uncompressed public key, y-parity 0, magnitude of x lower than the curve order
28 uncompressed public key, y-parity 1, magnitude of x lower than the curve order
29 uncompressed public key, y-parity 0, magnitude of x greater than the curve order
30 uncompressed public key, y-parity 1, magnitude of x greater than the curve order
31 compressed public key, y-parity 0, magnitude of x lower than the curve order
32 compressed public key, y-parity 1, magnitude of x lower than the curve order
33 compressed public key, y-parity 0, magnitude of x greater than the curve order
34 compressed public key, y-parity 1, magnitude of x greater than the curve order

For any v >= 35 you might be dealing with Ethereum signatures as per EIP-155:
v = recovery_id + CHAIN_ID * 2 + 35

I don't think that the v you describe is part of RFC6979 because I cannot find it in that document.  This v happens to be part of the ecdsa_raw_sign raw sign function in pybitcointools, which calls deterministic_generate_k, which is an implementation of RFC6979.
RFC6979 just helps you generate a deterministic k value for signing.  It does not help you generate r and s.  To generate r and s, you just use the normal ECDSA algorithm; you don't have to refer to RFC6979 after you have used it to generate k.  RFC6979 generates k, which is an input to the signing algorithm that can generate r and s.

Right, but does v appear across implementations? Pybitcointools isn't the only implementation to use it, I believe.

The only other implementation I have seen is libsecp256k1, which does not return a v number.  That library provides a function named secp256k1_nonce_function_rfc6979 whose only output is a 32-byte buffer of data.

secp256k1_ecdsa_sign_compact (old API) or secp256k1_pubkey_serialize_compact (new API) do return the recovery id (the value v, but without the 27 constant term). See my answer.

Yeah, so v has to do with signing, not RFC6979.

@DavidGrayson yeah, have amended title :)

v is needed to recover the public key. As a result of recovering the public key from ECDSA signature, 0, 1, or 2 points can be returned. In order to strictly indicate which point corresponds to the "original" public key, an additional byte is used


text:
How does bitcoin prevent DDoS amplification via the `addr` p2p message type?
If I am not mistaken the mainline bitcoind will accept unsolicited  addr messages, and upon receiving this message - the struct is deserialized, and then verify each address from the unsolicited sender.  The addr struct allows for up to 1,000 arbitrary addresses:port tuples and there isn't a port restriction... so what exactly is preventing an attacker from flooding the 100k or so Bitciond nodes with the address to some target web servers on 80 and 443?
...To make things more abusable, the client knowingly accepts information drafted in the future, and will retry three times regardless of the error - even on a protocol mismatch. This means that flooding a control-plane service like sshd or a vpn or even an entire IP range of services could cause a lot of problems for a target entity with a large network footprint.
A similar issue to this was exploited on the DC++ network.

For a long time, there was a restriction to strongly prefer connecting to addresses with port 8333. This restriction was recently removed in PR 23542 and PR 23306, although a list of "bad ports" was introduced that bitcoin core nodes would still avoid to connect to.
If you send an address to a node, it will often save it in the new table of the AddrMan database (if there is no collision with an existing one), but it won't automatically connect to it. That might happen at some later point when they have less than the default 10 outbound connections and therefore need more, or when they make a periodic feeler connection. But since addresses are gossiped over the bitcoin network all the time, they will know about thousands of potential peers, so they would connect to a specific address of the victim only with a low probability.
So even if you send a victim's address to all bitcoin nodes, that doesn't mean that they would receive so many connections that it would qualify as a DoS attack.
There has been a rough calculation in this post by vasild, which comes to the conclusion that even in the unrealistic scenario where nodes made new connections all the time, the number of connection attempts the victim would receive would be ~2 per second.
In reality, it should be much less because nodes don't make connections all the time - the victim might get the occasional connection attempt from a bitcoin node, but nothing that would qualify as a DDoS attack.
Also note that your verification link from the OP is part  of the AddrMan::Unserialize() function, only used to load saved addresses from the peers.dat file at startup, the function in AddrMan that is executed when receiving addresses from peers is AddrMan::Add().

ah thanks for the details, I am new to the codebase.  Wait... So does the client forward unvalidated addrs?  Also, seeing as i don't see a test case for this I don't see how anyone could know it is "10 connections."  Every nasty bug in prod has an excuse.

Yes, it does forward them (otherwise it would be very hard for nodes new the network to find incoming peers), although with some anti-spam restrictions. There actually was an addr spam attack last year, see this paper by Grundmann et al.: https://arxiv.org/abs/2108.00815v2
The fact that a bitcoin node makes 10 outgoing connections per default (8 full outbound + 2 block-relay-only) is documented in many places, see e.g. https://github.com/bitcoin/bitcoin/blob/master/doc/reduce-traffic.md

Oh cool, my concern was that it was never tested because the design team thought it wasn't exploitable.   Forwarding dead nodes and connecting to 80/443 and not rejecting cross-protocol is a bug, this can and should be fixed.  If it is known to be an HTTPD - then don't forward it.

This specific DDoS vulnerability was exploited in 2021, which lead to an emergency patch.  PR #22387 is how Bitcoin prevents DDoS, clients running a version older than v22 may still be used to amplify attacks.

The causal relationship was more complicated than that: It wasn't an emergency fix against an ongoing attack. The spam only started several days after #22387 was opened, so the patch might have provoked the attack, or maybe it was just coincidence. Also note that AddrMan is designed to be able to cope with spam, even without rate limiting, so it didn't really cause much chaos - most node operators didn't notice the spam at all.

@Lightlike The operator was never the victim, and of course, the operator should not be concerned with such attacks.  Suggesting that the node is the victim means that you are missing the point entirely.

@Lightlike also, really?? Bitcoin knowingly propagates known HTTPD's that make their way into an ADDR message? This is underpants on head level crazy.

I was refererring to the spam from 2021 where unknown people spammed the network with randomly distributed IP addresses, presumably with the goal of atttacking the bitcoin network. I was not talking about the scenario you describe in your OP. And of course node operators need to be concerned with such attacks, either because it might affect their ability to find peers, or because they run out of memory, which happened to an alternative implementation. (https://bitcointalk.org/index.php?topic=5348856.msg57469495).


text:
Bitcoin-qt data on external drive
I have done the instructions listed here:
How to move Bitcoin data from Application Support on an external drive [Mac]
It works fine for a flash drive connected the mac, but I wanted to have the data read from my network storage device (ReadyNAS). Bitcoin-qt comes up with the following error when I link it to the NAS:
EXCEPTION: N5boost12interprocess22interprocess_exceptionE
Operation not supported
bitcoin in Runaway exception

Any ideas on what this could be?

It is trying to acquire a lock on the lockfile in the data directory, which likely fails due to not being supported on the network file system you use to access your NAS.
Even if that could be safely ignored (the lockfile is just there to prevent you from running multiple Bitcoin Core instances on the same directory), it is a warning that more problems exist. In general, running a database on a network filesystem is a bad idea because of lack of support for operations or guarantees that databases require. It would either just not work, be very slow, or get corrupted very easily.
However, this is all because of the databases used (blocks/ index/, chainstate/ and wallet.dat). The block data itself is not a database, and could be moved elsewhere. Bitcoin Core does currently not support moving just the block data elsewhere, though.

I quite literally just had the same problem.
Downloaded 0.90 on Mavericks, connected to my FreeNAS box in Finder, made a directory to hold the block chain, told Bitcoin-QT to save the block chain there in the dialog and got the same message:
A fatal error occurred. Bitcoin can no longer continue safely and will quit.
EXCEPTION: N5boost12interprocess22interprocess_exceptionE
Operation not supported
bitcoin in Runaway exception

This could be better as it's own question or a comment adding more info the problem.


text:
Meaning of SigScript in Satoshi Nakamoto's "Genesis" coinbase transaction
The Satoshi's coinbase transaction includes this following scriptSig:
04ffff001d0104455468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73

There's an unexpected thing that I don't fully understand. What is this bytes sequence:
04ffff001d0104. Like that, I could eventually determine as 04 (size) + ffff001d + 01 (size) + 04
But I don't think it's the decode structure, so are you able to tell me what is this sequence number and what is its role ?

What is this bytes sequence: 04ffff001d0104

Non-functional data that probably only has meaning to the miner?
See What is the coinbase? where Murch answers

The coinbase transaction can have an arbitrary input of 100 byte size. E.g. this was where the genesis block famously contained "The Times 03/Jan/2009 Chancellor on brink of second bailout for banks". This is also the field that may be used for the extranonce.

And David Schwarz notes

BIP 34 changes this a bit.

Though obviously the genesis transaction predates the ideas of extranonce and BIP34

As you worked out, this part of the unlocking script (sigscript) is a push of a 4-byte value onto the stack and a push of a one-byte value onto the stack.
Since this is in the input of a coinbase transaction, so far as I know it has no functional purpose for other nodes in the rest of the Bitcoin network. There's no previous transaction with a locking script to be satisfied.
I'm not sure if that is known what this data meant to Nakamoto, so far as I know there was little restriction on what a miner could choose to put in the unlocking script of the inputs of the coinbase transaction.
Nakamoto seems to have pushed 4 bytes FFFF001D for the first 30000 or so blocks, the second push was sometimes 1 byte sometimes 2 and looks a bit like a sequence number but isn't continuous and doesn't always increase.
I wonder if Nakamoto ever publicly explained this. As I said, I believe it was arbitrary data that has no function for other nodes.
I looked at the coinbase transactions in blocks 1-9, 10,20,...90, 100,200...900, 1000, 2000...9000 etc
    Unlocking Script:.....04FFFF001D0104455468652054696D65732030332F4A616E2F32303039204368
    Unlocking Script:.....04FFFF001D0104
    Unlocking Script:.....04FFFF001D010B
    Unlocking Script:.....04FFFF001D010E
    Unlocking Script:.....04FFFF001D011A
    Unlocking Script:.....04FFFF001D0120
    Unlocking Script:.....04FFFF001D0123
    Unlocking Script:.....04FFFF001D012B
    Unlocking Script:.....04FFFF001D012C
    Unlocking Script:.....04FFFF001D0134
    Unlocking Script:.....04FFFF001D0116
    Unlocking Script:.....04FFFF001D010D
    Unlocking Script:.....04FFFF001D0109
    Unlocking Script:.....04FFFF001D013A
    Unlocking Script:.....04FFFF001D028300
    Unlocking Script:.....04FFFF001D02B500
    Unlocking Script:.....04FFFF001D0103
    Unlocking Script:.....04FFFF001D0122
    Unlocking Script:.....04FFFF001D0145
    Unlocking Script:.....04FFFF001D029600
    Unlocking Script:.....04FFFF001D023801
    Unlocking Script:.....04FFFF001D024F02
    Unlocking Script:.....04FFFF001D020204
    Unlocking Script:.....04FFFF001D029F06
    Unlocking Script:.....04FFFF001D02B400
    Unlocking Script:.....04FFFF001D02A500
    Unlocking Script:.....04FFFF001D02D202
    Unlocking Script:.....04FFFF001D02A003
    Unlocking Script:.....04FFFF001D027802
    Unlocking Script:.....04FFFF001D02E700
    Unlocking Script:.....04FFFF001D020204
    Unlocking Script:.....04FFFF001D02E606
    Unlocking Script:.....04FFFF001D02EA04
    Unlocking Script:.....04FFFF001D02E502
    Unlocking Script:.....04FFFF001D023909
    Unlocking Script:.....04FFFF001D026606
    Unlocking Script:.....04FFFF001D02DA03
    Unlocking Script:.....04FFFF001D02C30C
    Unlocking Script:.....04FFFF001D0173
    Unlocking Script:.....04C38C001D021801
    Unlocking Script:.....0415112A1C02CC00
    Unlocking Script:.....045C670F1C02ED0D
    Unlocking Script:.....04FD68011C024705
    Unlocking Script:.....04E6ED5B1B028701
    Unlocking Script:.....046332151B026402
    Unlocking Script:.....044C86041B0145

The script causes 3 items to be pushed onto the stack.
The first byte 04 is the length of the first item. That means the following 4 bytes get pushed onto the stack as one value.
After that is the byte 01, so the byte following that is pushed onto the stack.
Then comes the byte 45. This is 69 decimal, and is the length of the remaining part of the script. It gets pushed onto the stack as a 69-byte value.
So, when the script executes, it produces the following stack:
ffff001d
04
5468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73

The 3rd stack item, is hex for the text string:

The Times 03/Jan/2009 Chancellor on brink of second bailout for banks

You’re right. However, can you tell me why 04ffff001d0104 in the script is needed ? Can you provide more information please.

@Loopite: I believe it isn't needed (at least not by anyone other than the miner)

It is likely that he was using those fields as a test, since it was the first block ever. There are quite a few transactions that contain data that makes no sense to anyone other than the person who created it, especially in coinbase transactions.


text:
An unexpected transaction removed money from my wallet
I sent a transaction from Kraken to my wallet Exodus. All went smooth and the transaction was confirmed in my wallet yesterday around 12 AM.
This is the ID: fc933cc0456e8d8927ef1feb6392bf697279466e85d6f886a1637b09c7add45a
Suddenly one hour later I have 0 Bitcoin/EUR and a random transaction showing up.
I went in this website and saw that a transaction came up with this ID:
ba9aa12d2d28f4b786803e24a27722ceceaa1ec13349368cb93310f6deeeb632
I really have no idea what the hell happened.
Thank you for your time.
Kind regards,
Joanna

If you didn't create the second transaction it means that one of the following probably applies:

Someone else has, at some time in the past, known a previous or current password for your Exodus wallet.

Someone has had access to your written down backup of your recovery phrase or private key.

Your Exodus wallet is a fake wallet or has been compromised by some malicious add-on feature.

Your computer or device has been infected with malware.

In any case, the money is gone and you won't be able to retrieve it.
It is certain you will be contacted by people claiming that they, or someone they know, can get your money back. Their intent is to trick you into giving them money using invented fake fees, fake charges or anything else they can dream up that they think you will fall for.


text:
Where to download "Bitcoin core" wallet application from?
Why there are 2-3 bitcoin core applications online?

bitcoin-qt at bitcoin.org,
bitcoin knots
bitcoin core on bitcoincore.org

Which should I use?

Please split up your questions into topics (one question per topic). The variety of Bitcoin client downloads seems unrelated to the rest. About the rest: you'll need to give a lot more information. Which transactions? Why do you expect them to show up? Are they yours? How did you import or create them? Where are they coming from?

that was a very long time ago so, so given the weather, I'll be as specific as I can!
Comes from mining, bought more than 10 years ago.
it's all bitcoin-qt coins, in wallet.dat s!
I can open them on bitcoin-qt, but nothing, no trace, for me, who knows how to read, little or not, the outputs of the DEBUG file

buying, coins gathered, on a bitcoin-qt client, no passphrase (bitcoin-qt)

But, two months ago, I have out of the client -qt, priv key on electrum, But the file WAS in .txt file ... That It's sure

and haveing more file wallet.dat, more than this alone

Of course you will be paid

This site is not a job ad.

CAN help me ??please ?

Does this answer your question? How do I retrieve my backed up wallet to Bitcoin Core?

loadwallet and -rescan = replace new wallet.dat out of wallet folder and replace the old save wallet.dat in wallet folder and -rescan ?? it's = they 2 commands ?

do not can, other option please !

one wallet.dat of 2012-1014, I have an completely downloading blockchain, What I do ? no prejudiciable, option know also exprimed, juste help me ! what I talk !

I have wallet.dat from 2011 to 2014 (save bitcoin core) with private keys in it, a blockchain uploaded to bitcoin-qt client.
how to recover keys WITH COINS and TRANSACTIONS? please

I have edited this question to focus on the first of the many questions in the original text. I believe the other questions will be answered by answers to one of your several other questions. If you dislike what I have done you can use "rollback" in the "edited2 link above.

Websites
As of June 2022 the website I would advise you to use to obtain Bitcoin Core is bitcoincore.org.
Bitcoin knots is a completely different program.
The bitcoincore.org website is operated by the maintainers of the Bitcoin Core repository and is the project's own site, while bitcoin.org is operated by independent people. Historically, the Bitcoin Core software was published on bitcoin.org, but as ownership grew apart, this transitioned into it just being mirrored there. At at June 2022 the maintainers of the bitcoin.org website have not mirrored on their website the latest software from bitcoincore.org.
Therefore I would not download Bitcoin core from bitcoin.org.
The bitcoin.org website is a good source of information about Bitcoin and about a variety of different Bitcoin wallet applications. I believe that is its main purpose.

Application names
There can be confusion over similar names for Bitcoin software
The original Bitcoin application created by Satoshi Nakamoto was named simply "Bitcoin".
As multiple Bitcoin wallets were developed by many different independent developers, It was decided to rename "Bitcoin" to "Bitcoin Core". This gave it a separate name that would be less likely to be confused with other independent applications. "Bitcoin Core" is not the same thing as "Bitcoin Knots"
If you download and install the Bitcoin Core application you will probably find you have several distinct executable programs.

bitcoin-qt.exe - a wallet application with a simple graphical user interface. This is the main application and is all that would normally be used by anyone who just wants a uncomplicated Bitcoin wallet for ordinary simple Bitcoin transactions. This is why some people refer to "bitcoin-qt" instead of saying "Bitcoin Core". The history is a little more complicated but this is all we really need to know.

In a "daemon" subdirectory might be

bitcoin-cli.exe
bitcoind.exe
bitcoin-tx.exe
bitcoin-wallet.exe

These are a group of programs that communicate with each other. They are intended for more advanced users and for more advanced purposes. They should not be used at the same time as bitcoin-qt.exe.
You can find out more using the help built into these programs. Open a command prompt, use cd <directory path> to navigate to the installation folder and type bitcoin-wallet -?

I have made this answer community wiki to acknowledge and make clear that it is the result of community effort. Any errors are mine alone as at 2022-06-09

This is mostly right, but perhaps missing this: bitcoincore.org is operated by the maintainers of the Bitcoin Core repository and is the project's own site, while bitcoin.org is operated by independent people. Historically, the Bitcoin Core software was published on bitcoin.org, but as ownership grew apart, this transitioned into it just being mirrorred there. Now it appears that it's not even up to date anymore.

@Pieter: Thanks, I have edited my answer and replaced my paragraph with yours.

bitcoin core, created and coded by satoshi nakamoto? that's what it says! why is it not true?

Hi signbertrand, we are only a Q&A site. We do not allow job postings, and everyone here is a volunteer. Please tune back your expectations. Perhaps you could try the French Bitcoin Talk forum: https://bitcointalk.org/index.php?board=13.0


text:
Sent to wrong btc address help!
I sent my btc to another address which i found it unspent on block chain how do i return it to my account? I contacted coinjar my wallet and still have no response 1 week!

I downloaded bitcoin core and used the address[recieve] to finalise the transaction so im not sure why its not in my wallet as the address i got was from my bitcoin core wallet?

There is no way to reverse a transaction once it has been confirmed. Do you control the address that you sent the bitcoin to? If not, then you will need to contact the person (or business) that controls that address, and ask them to return your funds.

I downloaded bitcoin core and used the address[recieve] to finalise the transaction so im not sure why its not in my wallet as the address i got was from my bitcoin core wallet?

if you mistyped the address and it was somehow still valid (very low probability but still possible) you are definitely screwed. but just finding it "unspent" on a blockchain explorer doesn't mean anything went wrong; it just means the recipient is holding on to it. how about giving more details as to what happened?

The probability of a mistyped address being valid is about 1 in 4 billion.

You may have entered a invalid address by accident, there is a VERY low probability it was valid, so it is probably unspent at this moment.
However, once it is sent, its sent. There is no way to reverse it.

If the transaction still in mempool then you should be able reverse it but if it on blockchain then u cannot reverse it


text:
How to check where I recently sent some Bitcoin Cash using Copay wallet
Hey sorry but i'm totally new to this. i bought serval different currencies. i have sent bitcoin and bitcoin cash to my copay wallet. i can see the bitcoin but not the bitcoin cash. the balance has left my coinbase account. i don't understand enough to decipher what im looking at. i see input consumed and outputs created. and i see a load of confirmations. i copied and pasted the address from my wallet so i don't think i could have pasted them wrong. what are my next steps to take. ho do i check where i have send them.

Can you edit your post to include the addresses your transactions went to?

how do I check where I have sent them?

View Recent Transactions.

If they are not there, you may be able to refresh the history from the Copay servers

"Settings" > "Wallet" > "More Options" > "Transaction History" > "Clear cache"
"Settings" > "Wallet" > "More Options" > "Wallet Addresses" > "Scan addresses for funds"

You can paste the address or transaction ID into any BCH clockchain explorer (example) to check another view of the transaction in a different copy of the blockchain ledger.


text:
How to recover bitcoins from personal wallet app in android which is uninstalled?
I am using a bitcoin Non-deterministic wallet created by a private firm.
In which there are few bitcoins worth 100$
Somehow I clear data of the android App, and Now all the memories related to my wallet is removed from android device.
How can I recover my bitcoins from android wallet app ?
UPDATE
I am not sure but If I have backup the wallet file in either form wallet.dat or wallet.bin
Even then my internal memory data would have get removed as well by Clear app data.
Isn't it?

Without a backup, there is no way to recover the coins.
If this happened very recently, you might be able to enlist a data recovery service to pull the keys off the device. This is likely to cost more than $100 though.
Wallet generation is based on random data, and if you do not backup individual keys, or the mnemonic for HD wallets, you cannot recreate the keys.

I am not sure but If I have backup the wallet file in either form wallet.dat or wallet.bin
Even then my internal memory data would have get removed as well by Clear app data.
Isn't it?


text:
Setting up a miner with Eligius
In the last few hours I am getting 100% connection rejections from Eligius. I am using a Mac and the Asteroid miner GUI. I also have MacMiner setup. both are not failing after about a month of successfully working.  But that is not the point of the question.
My question is trying to understand the pool URL, here is what Eligius provides:
Stratum Host: stratum.mining.eligius.st Port: 3334 | Getwork Host: getwork.mining.eligius.st Port: 8337 GBT Host: gbt.mining.eligius.st Port: 9337
I have been using "http://gbt.mining.eligius.st:9337"
Question 1: Why is the scheme missing, that is "http://", etc. It needs to be entered as part of the pool URL, what if any scheme should be entered?
Also is there some reason the scheme is not provided? Or it just a lazy web developer?
Question 2:
Please direct me to some information about the host types: Stratum, Getwork and GBT.

In the last few hours I am getting 100% connection rejections from
  Eligius. I am using a Mac and the Asteroid miner GUI. I also have
  MacMiner setup. both are not failing after about a month of
  successfully working. But that is not the point of the question.

Check your ISP, check luke's threads on btctalk and on eligius or their IRC chan #Eligius on freenode. They can give you the best help with support. A stackexchange unfortunately is not a support page for a website.

My question is trying to understand the pool URL, here is what Eligius
  provides: Stratum Host: stratum.mining.eligius.st Port: 3334 | Getwork
  Host: getwork.mining.eligius.st Port: 8337 GBT Host:
  gbt.mining.eligius.st Port: 9337
I have been using "http://gbt.mining.eligius.st:9337"

Thanks for some detailed info.

Question 1: Why is the scheme missing, that is "http://", etc. It
  needs to be entered as part of the pool URL, what if any scheme should
  be entered?

stratum+tcp://stratum.mining.eligius.st:3334
I believe i explained the rest of the question below.

Also is there some reason the scheme is not provided? Or it just a
  lazy web developer?

It's assumed that in bitcoin mining you have an idea of what a stratum's protocol is and the requirements/settings for it. Plus some newer miner programs auto detect stratum/gbt then fall back to getwork if there is not a template available. I wouldn't call it a lazy web developer. Just one who is more technical and not the best with making n00b friendly how to's.

Question 2: Please direct me to some information about the host types:
  Stratum, Getwork and GBT.
  Let me give a light explanation of each along with some links

Getwork was the original bitcoin mining protocol. It is still around, but not used very much except by some pools for legacy support (mostly asicminer blades/cubes). Getwork uses the standard rpc connect commands to get new work. Since a third party miner does not have the block chain, it has to ask for a new piece of work to hash.
https://en.bitcoin.it/wiki/Getwork
Stratum mining is an overlay protocol that allows thin clients as well as a method of sending templates via the stratum protocol.
https://en.bitcoin.it/wiki/Stratum
GBT (getblocktemplate)
This is simply allowing you to get the entire template of a block. A template is the entire list of transactions needed to hash out the block. This is usually updated every minute. This allows ASIC miners not to have issues with as many rejected shares and stale shares (like they would with GetWork)
https://en.bitcoin.it/wiki/Getblocktemplate
Source: Me. (I'm an alt-chain pool op and a longtime bitcoiner)
As always
Any errors/questions/comments feel free to let me know in a positive manner.

Great into on the protocols.  Sorry, I have t go with lazy. Giving full information is part of a good user interface. If the information is not there there is no way to get past noob-hood. That some clients will automatically fill-in the correct scheme is a partial answer if they supply that information, both in the positive and negative. But they seem to fall back on the pool supplying that information. I spent substantial time looking for the definitive answer before I asked here, thanks for you answer. As for bitcointalk I wish I could get an account setup there.


text:
Get transaction data from full node
Background Information
I am building an application that requires the ability to query any bitcoin full node (from a python script using raw tcp sockets) in order to read the OP_Return value listed in the following transaction (https://live.blockcypher.com/btc-testnet/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1/)
What I have done to date
I have successfully managed to establish the version + verack, however I was faced with two issues.

When I send a getdata for the necessary transaction Idit does not return the result. I have identified this is because my initial assumptions that getdata returns the necessary data was incorrect and instead it will only return transaction data for transactions still in mempool.
This lead me to my main issue, I was hoping to replicate the initial block download on my python script, starting from the block containing the transaction I'm interested in. However when I do getblocks/getheaders no matter what I seem to put in the hash filter I get a dump of 500 (or 2000) blocks/headers. Below is a hex dump of the requests I am making (just the bitcoin protocol frame):

getblocks
0b110907676574626c6f636b00000000450000002a0af9950100000001000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7000000000000017a09017d52db538d7a9ddcc48311866d7e5fdbbbec7d0faad5
getheaders
0b110907676574686561646572730000450000002a0af9950100000001000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7000000000000017a09017d52db538d7a9ddcc48311866d7e5fdbbbec7d0faad5

I am looking for the following help:

How can I query for transaction data without using bitcoin RPC if I can't use the getdata message?
How do I get my block or headers response to only contain 1 inv item instead of 500/5000

EDIT (4/5/20)
The implied requirement which wasn't made clear here was that we needed to do so in a decentralised way using the blockchain RPC api or equivalent.

You want to download 250GB of blocks from a remote peer in order to find a single output?

No I don't want to do that, but there doesn't seem like there are too many alternatives. I'm trying to pretend like I am a bitcoin full node trying to sync (but attempting to sync from the necessary block that I need, this way I would only be syncing 1 block worth of data) I just don't think I'm using the API properly

If your best choice is to do something absurd, maybe reconsider your strategy. The p2p network isn’t designed for arbitrary data access like you’re attempting.

I've gone through a few alternatives but the main goal here is to be able to look up the data in a decentralised censorship resistant manner. So I can't rely on looking at blockchain explorer apis. Also if I use bitcoin RPC I would need to be connecting to a specific bitcoin full node which could get taken down.

If you want something censorship resistant, your only choice is running your own node, and indexing the blocks in its data yourself, and then querying that.

is there no way for me to mock what a full node does (but only for part of the sync process)? Like surely I can say, hey I'm a full node and I'd like to start initial block download from X block and then validate the transactions in there. Is this not handled through bitcoinP2P ? Sorry in advanced if I have made any errors here

@PieterWuille I might not fully understand your comment from December, but in actual fact I managed to do what I was hoping, in theory you can 'mock' a full nodes functionality perfectly fine if you simulate the initial ver-ack messages. You can view the full source code here: https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py

It turns out the main issue with the above code was that the the endianness in python was defaulting to my hex payload being mixed up. I simply needed to correct the ordering of the payload. The correct code can be viewed here: https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py#L284-L299

The bitcoin p2p protocol does not directly permit transaction lookups this way. Maybe some bloom filtering would minimize the footprint, but it's still relatively complex to implement.
Electrum servers are maybe the next best thing - they do support the lookups you want, but they're less decentralized, and if your application is doing naughty things they could collude and manipulate only your results.

How do I get my block or headers response to only contain 1 inv item instead of 500/5000

I don't think you can limit the headers results (you have to know all the headers to do so) But once you have the header chain, you can send a block locator with start:HASH(n) end:HASH(n+1) and you can download a single block.
Re bloom filtering, if you find nodes with the BLOOM_SERVICE bit you can tell them your filter, and the block data only contains your txs and the false positives allowed by your filter.

I don't see any endpoint to get the whole transaction in Blockcypher API documentation, so I used blockstream.info API:
>https://blockstream.info/testnet/api/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1
{
   "txid":"2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1",
   "version":2,
   "locktime":0,
   "vin":[
      {
         "txid":"382bda6ee2bac22afd051104af16146436895ecca382a76c2d66535a837254bc",
         "vout":1,
         "prevout":{
            "scriptpubkey":"00144ecdb2867150ff16c7d6e3258adca9ed8bac3ac1",
            "scriptpubkey_asm":"OP_0 OP_PUSHBYTES_20 4ecdb2867150ff16c7d6e3258adca9ed8bac3ac1",
            "scriptpubkey_type":"v0_p2wpkh",
            "scriptpubkey_address":"tb1qfmxm9pn32rl3d37kuvjc4h9fak96cwkpjyyjmk",
            "value":2948920
         },
         "scriptsig":"",
         "scriptsig_asm":"",
         "witness":[
            "304402205e9b17758d1daf2a501a1fabe06bdbf15adaadfcf6ff80c4cfb34176bd988c0c02207450185bde5c20e06270b3b7e681e0042072c6af25fc10c62430df0f04ec5bf201",
            "023532fa6b866f37729719a3615ef3a776407048989736ac2b7fafa5678a519df3"
         ],
         "is_coinbase":false,
         "sequence":4294967295
      }
   ],
   "vout":[
      {
         "scriptpubkey":"6a0e3132372e302e302e313a38313831",
         "scriptpubkey_asm":"OP_RETURN OP_PUSHBYTES_14 3132372e302e302e313a38313831",
         "scriptpubkey_type":"op_return",
         "value":0
      },
      {
         "scriptpubkey":"a914171f697fe358f7d7238b6128930bb1fa7363b44d87",
         "scriptpubkey_asm":"OP_HASH160 OP_PUSHBYTES_20 171f697fe358f7d7238b6128930bb1fa7363b44d OP_EQUAL",
         "scriptpubkey_type":"p2sh",
         "scriptpubkey_address":"2MuMVAFK2gbTDNVBhLtpZfA4Xi62mWh6bU5",
         "value":948920
      },
      {
         "scriptpubkey":"0014af8abd63a00c40ad75e47870b1b0a1ddb2ba7e66",
         "scriptpubkey_asm":"OP_0 OP_PUSHBYTES_20 af8abd63a00c40ad75e47870b1b0a1ddb2ba7e66",
         "scriptpubkey_type":"v0_p2wpkh",
         "scriptpubkey_address":"tb1q479t6caqp3q26a0y0pctrv9pmket5lnx24twcx",
         "value":1500000
      }
   ],
   "size":248,
   "weight":665,
   "fee":500000,
   "status":{
      "confirmed":true,
      "block_height":1612110,
      "block_hash":"000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7",
      "block_time":1576024567
   }
}

You can see here that the first output has the data is 3132372e302e302e313a38313831, which decoding to ASCII characters seems like a local URL:
hex = '3132372e302e302e313a38313831'
decoded = binascii.unhexlify(hex).decode('ascii')
print(decoded)

Prints: 127.0.0.1:8181
EDIT:
To get the transaction that interests you without using any external API you will need to run a full bitcoin node with the txindex=1 option. If this transaction is tracked by your wallet, a pruned node would suffice. Then you can get the information about the transaction with this command:
>bitcoin-cli getrawtransaction 2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1 true

You should get something very similar to the answer from blockstream's API I pasted above

sorry if it wasn't clear, but I needed to do this in a decentralised way without relying on stored data from an API server (for example if they just respond with the incorrect data). I'm trying to read this data straight off the blockchain and can't find any way to do this

Check the edit in the final part of my answer, it explains how to do that without an external API


text:
Can't run bitcoind on OS X
I get the following error:
dyld: Library not loaded: /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib
  Referenced from: /usr/local/bin/bitcoind
  Reason: image not found
Abort trap: 6
Recompiling fails too with similar "abort trap" messages.
I am 99% sure I have messed up my openssl stuff in some way, though am unsure how to rectify that.

Did you compile yourself?

yes. though make fails now in a similar way.

brew reinstall openssl?

make still fails with: 5 errors generated.
make[2]: *** [bench/bench_bitcoin-data.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1


text:
update to 0.17.0 broke several rpc api calls that worked under 0.16.3, how to migrate correcty my code?
After updating from Bitcoin core 0.16.3 to 0.17 I've found several RPC calls are now broken, and I'm not sure on how to migrate/update my code (PHP).
Things broken so far:

getbalance : does not detect unconfirmed balance, this command shows only confirmed balance where it used to (and should) show confirmed+unconfirmed balance. Expected to work like in 0.16.3.
$btc->getbalance("*", 0);

listunspent: does not detect unconfirmed inputs, showing nothing and returning error stating "Invalid amount". Expected to work like in 0.16.3.
$btc->listunspent(0, 9999999, array(), true, array("minimumAmount" => 0.00005));

I will add more to the list as they get discovered.
There is no mention of these issues online, in the 0.17 changelog or anywhere else I could find.
Already updated some other php code that used validateaddress deprecated call and a couple more, probably still missing some editing here.
How to restore functionality as expected?
Thanks.

The first issue, with getbalance, is a known issue which will be fixed in the next release. You can see the pull request for it here: https://github.com/bitcoin/bitcoin/pull/14602
The latter, though, I cannot seem to replicate. To my knowledge, listunspent has not been modified in version 0.17, and it seems to work as expected when I test on the command line. Did it work before? Perhaps PHP is causing the issue, could you try enclosing the number as a string instead just to test it? "minimumAmount" => "0.00005"

Perhaps mention that it can be worked around using getunconfirmedbalance.


text:
Bitcoin core wallet transactions and synchronisation
I have Bitcoin Core wallet.
I wanted to know if each transaction I need sync the wallet?
(I have a new business that accept bitcoin and I must to save each transaction)

With Bitcoin Core you'll run a full node. So, for every transaction someone does to your business, your Bitcoin Core wallet need to be synchronized with the complete blockchain.

So each transaction i must sync manually ?

No, you just have to make sure you are synchronized with the head of the blockchain. When deciding if a payment is finalized you should look at the number of block confirmations. Three usually is enough.

One more question please to close my case . ill explain the steps i want figure out. For example i have an empty bitcoin core wallet . Step 1. l backup the wallet and go offline. Step 2. someone send me a bitcoin to this wallet and i can see it(in balance).  Step 3 .ill restore my bitcoin core wallet to the initial point.. What will happen then ?

I assume that with restore the wallet you start over with the blockchain from scratch. What will happen is that initially the balance will be 0 since your node is not yet aware of the transaction. So you will need to synchronize up to the block where the transaction was included (this will take some time). At that time the balance will be updated. After that the synchronization will continue until it's up-to-date with the head.

i assumed the same , but  tried to avoid making test this week because lake of time.  Thanks anyway , will update after testing if its working as you mentioned above. THANKS!

When you create a transaction, the transaction is broadcast to other Bitcoin nodes and reaches miners.
When a miner includes your transaction in a block that they successfully mine, they broadcast the block that contains your transaction - this is the first confirmation. When other blocks get mined on top of that block, those are additional confirmations of your transaction.
When you run Bitcoin core on your PC it will try to synchronise your local copy of the blockchain. It needs to get the latest blocks to add to your local copy of the blockchain so that it can find out whether your transaction got confirmed.

got you friend. thanks for the detailed answer. So actually i need to sync my wallet each transaction ?

Can you please check out my response to previous commenter ?

@Sadikov: Hello, you are new and may not know that comments are for clarifying questions or answers and are disposable. It is better to ask a new question using the "Ask Question" link above and include a link to this question if needed. That way people with the same question can find it's answers using Google or this site's search bar without having to scan through all the comments (which may get deleted in a clean up - they are temporary/disposable, Qs and A's are not)