text:
How is the whitepaper decoded from the blockchain (Tx with ~1000x m of n multisig outputs)
The whitepaper is apparently encoded at 54e48e5f5c656b26c3bca14a8c95aa583d07ebe84dde3b7dd4a78f4e4186e713, which is an m of n multisig Tx with 947 outputs (just under the scriptsig limit of 20kB!).
Using the Blocktrail Python SDK, I get a list of the outputs as hex using the following Python (2.7) code (NB, the APIKEY, APISECRET parameters are available if required from www.blocktrail.com):
from blocktrail import APIClient
bt_client = APIClient(APIKEY, APISECRET, network='BTC')
txnObj = bt_client.transaction('54e48e5f5c656b26c3bca14a8c95aa583d07ebe84dde3b7dd4a78f4e4186e713')
hashes = [(t['script_hex']) for t in (txnObj)['outputs']]

The resulting list is available here in full and is essentially all pay-to-pubkey-script Txns. An excerpt:
[u'5141e4cf0200067daf13255044462d312e340a25c3a4c3bcc3b6c39f0a322030206f626a0a3c3c2f4c656e6774682033203020522f46696c7465722f466c617465446541636f64653e3e0a73747265616d0a789cad5c4b8b24b911becfafa8b3a1da292925654253d0d55373f06d61c007e39bbd061f0cde8bffbe25c55b5266f61ab3905d419ba54728e28bb76a963777fbcfb77fdf96db7d291f93f3e599f7fafcedefb73fffe1f6aff665fdefb77f7c7bfefce6c2fa166e695bdfd6dbcfbfddfef8c3dd5cf953ae',
.....
u'514130206e200a30303030313832353430203030303030206e200a747261696c65720a3c3c2f53697a652036382f526f6f74203636203020520a2f496e666f20363720413020520a2f4944205b203c43413142304134344244353432343533424546393138464643443436444330343e0a3c4341314230413434424435343234353342454641393138464643443436444330343e205d0a2f446f63436865636b73756d202f36463732454137353134444641443233464142434337413535303032314146370a3e53ae',
 u'51213e0a7374617274787265660a3138323732370a2525454f460a000000000000000051ae',
 u'76a91462e907b15cbf27d5425399ebf6f0fb50ebb88f1888ac',
 u'76a914031c79236ff3017496cf8d9a883f494458f245f288ac']

QUESTION: How is this array of hex data parsed into the bitcoin.pdf? Specific Python framed answers would be appreciated!

I should add its 945x ? of 3(?) txns and 2x paytopublickey outputs

scriptsig limit of 20kB! I was aware of a 10 kB limit, enforced in EvalScript, but what 20 kB limit are you referring to?

@NickODell isn't the limit for scripts 20kb?

This is a fun little puzzle on the blockchain, basically. First, you need to know a little about pdf's and how they're structured, which you can find here.
Second, you'll note from section 3.4.1 that all pdf's start with this string:
%PDF-

In hex, that is 255044462d. And indeed that is in the very first output in the very first bare multisig pubkey:
<e4cf0200><067daf13>**255044462d**312e340a25c3a4c3bcc3b6c39f0a322030206f626a0a3c3c2f4c656e6774682033203020522f46696c7465722f466c6174654465

I haven't figured out what the first 8 bytes are for (♦edit: e4cf0200067daf13 = 2x 4byte little Endian "checksums", see @WizardOfOzzie comment below), but the rest of the bare multisig keys (everything in between 1 and 3 OP_CHECKMULTISIG in each output -- note the last one is a 1 of 1, so it's 1 OP_CHECKMULTISIG) are pieces of data for the pdf and they are in order. If you can put all the hex digits of the bare multisig keys into a single file (no whitespaces) called "fromblockchain.hex", you can run this very simple program to extract the pdf:
contents = open('fromblockchain.hex').read()
bytes = contents[16:].decode('hex')
f = open("bitcoin.pdf")
f.write(bytes)
f.close()

This should create a bitcoin.pdf which is the actual satoshi whitepaper. I've tested this and indeed it is the whitepaper. Good to know it's literally in the blockchain.
Alternatively, if you have bitcoind running on your machine, you can run this python script to grab the bitcoin whitepaper:
import subprocess

# raw = full hex of raw Tx using Bitcoin-cli
raw = subprocess.check_output(["bitcoin-cli", "getrawtransaction", "54e48e5f5c656b26c3bca14a8c95aa583d07ebe84dde3b7dd4a78f4e4186e713"])

outputs = raw.split("0100000000000000")

pdf = ""
for output in outputs[1:-2]:
    # there are 3 65-byte parts in this that we need
    cur = 6
    pdf += output[cur:cur+130].decode('hex')
    cur += 132
    pdf += output[cur:cur+130].decode('hex')
    cur += 132
    pdf += output[cur:cur+130].decode('hex')

pdf += outputs[-2][6:-4].decode("hex")
f = open("bitcoin.pdf", "wb")
f.write(pdf[8:-8])
f.close()

Finally checking checksum
sha256sum bitcoin.pdf
b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553  bitcoin.pdf

Ahh ha! Another fantastic answer! I was trying to get a script like the above working and couldn't parse the hex after 0x5141 and between the 0x41s. What is up with the last two outputs being standard 76a914____88ac?

Looked to me like the person spent 1 satoshi on each output and wanted the change back for the last one. Not sure about the second to last.

Perhaps the it's a dual purpose: allows output to be collected and the data length can't be padded, since 20 bytes x2 % 8 whereas doing that with 65 bytes is much harder (actually 65x3)

Did you want to edit the bottom script to add the output at outputs[-2]? The script needs this after the for loop: pdf += outputs[-2][6:-4].decode('hex'). Also open needs to be using open("bitcoin.pdf", 'wb') (binary mode). I'll have to look into multisig more b/c I still fail to see why the rest of the multisigs are pushing 0x41 bytes Vs the last only pushing 0x21 bytes

Can you edit your fantastic answer to also show reverse functionality (ie bitcoin.pdf => rawhextx) using Python? Python-specific forums/subreddits are having a hard time understanding the Bitcoin aspect. EDIT: yes I am aware this is a non_standard Tx; 'tis for intellectual curiosity only

Re: 0xe4cf0200067daf13 ("I haven't figured out what the first 8 bytes are for"). It's actually 2x 4 byte "checksums". More specifically, little Endian hex representations of: <total pdf size> <crc32 for pdf> ie 184292 330267910 ;)

Nice catch @WizardOfOzzie. I'll try to write something up to do the opposite direction (any pdf, not just bitcoin whitepaper) soon.

this code - https://gist.github.com/shirriff/bfc4df70a02732493a28#file-bitcoin-insertion-tool-py - is what was used, I believe. The problem is it depends on actual UTXOs and counts values, checking both, whereas I'm just trying to create a raw unsigned Tx, and sign it (ie no need for Bitcoincore RPC, I'll provide the input data like txid, VOUT etc)

I did a binary diff against the version previous on bitcoincore.org and it seems to differ. My guess is that it's some irrelevant metadata thing.

Also, getrawtransaction now takes an optional blockhash argument. So that removes the need for -txindex here.

@WizardOfOzzie I have done some further analysis of that uploader script at: https://bitcoin.stackexchange.com/a/105574/21282 and I agree that it is almost certainly what was used.

This is a bash command that can also give you the file:
bitcoin-cli getrawtransaction 54e48e5f5c656b26c3bca14a8c95aa583d07ebe84dde3b7dd4a78f4e4186e713 0 00000000000000ecbbff6bafb7efa2f7df05b227d5c73dca8f2635af32a2e949  | sed 's/0100000000000000/\n/g' | tail -n +2 | cut -c7-136,139-268,271-400 | tr -d '\n' | cut -c17-368600 | xxd -p -r > bitcoin.pdf

It doesn't need -txindex, but it does need a full node, not pruned. I suppose it's possible to make a variant that works with gettxout that works with pruned nodes, but that's left as an exercise for the reader.

Exercise done by @jb55 :-) seq 0 947 | (while read -r n; do bitcoin-cli gettxout 54e48e5f5c656b26c3bca14a8c95aa583d07ebe84dde3b7dd4a78f4e4186e713 $n | jq -r '.scriptPubKey.asm' | awk '{ print $2 $3 $4 }'; done) | tr -d '\n' | cut -c 17-368600 | xxd -r -p > bitcoin.pdf Source: https://bitcoinhackers.org/@jb55/105595146491662406

The PDF uses Python uploader/downloader scripts present in the Bitcoin blockchain itself
Those scripts are present at:

uploader: tx 4b72a223007eab8a951d43edc171befeabc7b5dca4213770c88e09ba5b936e17 block 229991 reproduced at: https://gist.github.com/cirosantilli/ade4dde7c2f2f5020d792872681763e8
downloader: tx 6c53cd987119ef797d5adccd76241247988a0a5ef783572a9972e7371c5fb0cc block 229991 reproduced at https://gist.github.com/cirosantilli/e90bd2e6c3fab25a20898e61e3ab3e90

These are my forks of Ken Shirriff's Gists who presumably found them while researching for: http://www.righto.com/2014/02/ascii-bernanke-wikileaks-photographs.html
Both scripts have the copyright header:
#
# File downloader
# Requires git://github.com/jgarzik/python-bitcoinrpc.git
#
# (c) 2013 Satoshi Nakamoto All Rights Reserved
#
# UNAUTHORIZED DUPLICATION AND/OR USAGE OF THIS PROGRAM IS PROHIBITED BY US AND INTERNATIONAL COPYRIGHT LAW

which is likely a joke and not really written by Satoshi in my opninon.
Both uploader and downloader are uploaded with the uploader encoding, which creates a slight chicken-and-egg bootstrap issue to download the downloader. But the encoding is so simple that you could just download the raw downloader data and fix it up manually.
After downloading the downloader manually, I managed to run it to extract a byte-by-byte match of the PDF present at https://web.archive.org/web/20210418161957/https://bitcoin.org/bitcoin.pdf (sha256 == b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553).
First you have to be running a Bitcoin core server locally. Suppose that your .bitcon/bitoin.conf contains:
rpcuser=asdf
rpcpassword=qwer
server=1
txindex=1

Then I run:
git clone git://github.com/jgarzik/python-bitcoinrpc.git
git -C python-bitcoinrpc checkout cdf43b41f982b4f811cd4ebfbc787ab2abf5c94a
wget https://gist.githubusercontent.com/shirriff/64f48fa09a61b56ffcf9/raw/ad1d2e041edc0fb7ef23402e64eeb92c045b5ef7/bitcoin-file-downloader.py
pip install python-bitcoinrpc==1.0
BTCRPCURL=http://asdf:qwer@127.0.0.1:8332 \
  PYTHONPATH="$(pwd)/python-bitcoinrpc:$PYTHONPATH" \
  python3 bitcoin-file-downloader.py \
  54e48e5f5c656b26c3bca14a8c95aa583d07ebe84dde3b7dd4a78f4e4186e713

where tx 54e48e5f5c656b26c3bca14a8c95aa583d07ebe84dde3b7dd4a78f4e4186e713 (block 230009) is the transaction in which the whitepaper is encoded.
Encoding used by the downloader script
The downloader is not super strict about the subtleties of the uploader. We can immediately understand its format from the source:
data = b''
for txout in tx['vout'][0:-2]:
    for op in txout['scriptPubKey']['asm'].split(' '):
        if not op.startswith('OP_') and len(op) >= 40:
            data += unhexlify(op.encode('utf8'))

length = struct.unpack('<L', data[0:4])[0]
checksum = struct.unpack('<L', data[4:8])[0]
data = data[8:8+length]

if checksum != crc32(data):
    print('Checksum mismatch; expected %d but calculated %d' % (checksum, crc32(data)),
          file=sys.stderr)
    sys.exit()

so we see that it:

ignores the two last outputs
ignores operands
ignores any integer constant with less than 20 bytes (40 hex bytes)

Then, from the data above:

the first 4 bytes are the payload length
the next 4 bytes are a crc32 of the rest of payload data
the payload starts at the 8th byte

Encoding used by the uploader script
The uploader is a bit more specific. Perhaps it is easiest to understand it by looking at the disassembly of a small upload, e.g. the downloader itself: https://www.blockchain.com/btc/tx/6c53cd987119ef797d5adccd76241247988a0a5ef783572a9972e7371c5fb0cc
First we have a bunch of outputs of type:
OP_1 data OP_3 OP_CHECKMULTISIG

and worth a tiny 0.00000001 BTC which contain the bulk of the data. These cannot presumably be spent, due to the OP_CHECKMULTISIG, but this cannot be proven either.
Then at then end we have two different transactions, which as we saw above the downloader ignores:

a standard P2PKH OP_DUP OP_HASH160 <hash> OP_EQUALVERIFY OP_CHECKSIG with minimal value 0.00000001 BTC and unspent. TODO why? This corresponds to the part of the code:
# dest output
out_value = Decimal(sys.argv[3])
change -= out_value
txouts.append((out_value, OP_DUP + OP_HASH160 + pushdata(addr2bytes(sys.argv[2])) + OP_EQUALVERIFY + OP_CHECKSIG))

which shows that you just pass an arbitrary target value and address from your script.

a standard spent P2PKH change transaction with a considerable value, to recover leftovers from the input. It is sent to a new address owned by you that is created on the fly via the RPC call:
change_addr = proxy.getnewaddress()
txouts.append([change, OP_DUP + OP_HASH160 + pushdata(addr2bytes(change_addr)) + OP_EQUALVERIFY + OP_CHECKSIG])

Reading the source basically confirms what we see on the disassembly. Some other pointers follow.
At:
data = open(sys.argv[1],'rb').read()
data = struct.pack('<L', len(data)) + struct.pack('<L', crc32(data)) + data
fd = io.BytesIO(data)

we see the:

payload length struct.pack('<L', len(data))
crc32

being added to the start of the file.
At:
def checkmultisig_scriptPubKey_dump(fd):
    data = fd.read(65*3)
    if not data:
        return None

    r = pushint(1)

    n = 0
    while data:
        chunk = data[0:65]
        data = data[65:]

        if len(chunk) < 33:
            chunk += b'\x00'*(33-len(chunk))
        elif len(chunk) < 65:
            chunk += b'\x00'*(65-len(chunk))

        r += pushdata(chunk)
        n += 1

    r += pushint(n) + OP_CHECKMULTISIG
    return r

we see the main data encoding loop. This clarifies from the disassembly:
OP_1 data OP_3 OP_CHECKMULTISIG

the OP_1 is fixed at pushint(1)

the OP_3 can be less than 3, it encodes how many 65 byte chunks were encoded, with a maximum of 3 chunks present on each output due to fd.read(65*3)
The last payload transaction can therefore have OP_1, OP_2 or OP_3 depending on granularity.

What else was the uploader script used for and was it popular?
At:

https://github.com/cirosantilli/bitcoin-strings-with-txids/blob/master/data/satoshi_uploader
https://github.com/cirosantilli/bitcoin-strings-with-txids#satoshi_uploader

I have indexed every single blockchain transaction that follows the above format, notably having a matching CRC32 in the first 8 bytes.
I have also managed to download and interpret every single one of the transactions as described briefly at: https://cirosantilli.com/cool-data-embedded-in-the-bitcoin-blockchain/illegal-content-of-block-229k
Basically, very soon after the uploader/downloader were uploaded on 229991, besides the whitepaper, several uploads were made intentionally containing what some countries might consider illegal data, although none of them eye-popping for my crazy standards, and as we've seen so far, the Bitcoin blockchain has survived them for the time being as of 2021.
Since the illegal content was uploaded soon after the uploader, I find it exceedingly likely that it was uploaded by the same person who created the uploader.
And after a short flurry of activity, the uploads stopped, and only one single upload was made much later at tx 89248ecadd51ada613cf8bdf46c174c57842e51de4f99f4bbd8b8b34d3cb7792 which caught my attention because it contains an ASCII art of a Buddha between two Yin Yang symbols as described at: https://cirosantilli.com/cool-data-embedded-in-the-bitcoin-blockchain/ascii-art
          69696969                         69696969
       6969    696969                   696969    6969
     969    69  6969696               6969  6969     696
    969        696969696             696969696969     696
   969        69696969696           6969696969696      696
   696      9696969696969           969696969696       969
    696     696969696969             969696969        969
     696     696  96969      _=_      9696969  69    696
       9696    969696      q(-_-)p      696969    6969
          96969696         '_) (_`         69696969
             96            /__/  \            69
             69          _(<_   / )_          96
            6969        (__\_\_|_/__)        9696

Therefore my conclusion is that the uploader and downloader scripts simply never reached considerable popularity.
They are just too over-engineered and clunkier than needed. Why would you need a CRC32 when the blockchain itself is already hashed byte-by-byte to the brim? Perhaps it was intended simply as a marker of "interesting information". But much more natural and dominant later on was to use ASCII strings, or simply magic bytes of certain file formats as markers, see e.g. my comments on more directly encoded images at:

https://github.com/cirosantilli/bitcoin-strings-with-txids#image-indexing
https://cirosantilli.com/cool-data-embedded-in-the-bitcoin-blockchain/images


text:
Are there public LN nodes that have some gRPC LND API endpoints accessible for 3rd parties?
My lnd node died but I'm doing some research on lightning channels. So I'm rebuilding a new btc/lnd node, but in the meantime: are there open nodes that I can access, especially for SubscribeChannelGraph?
Update:
I found this repo that pretty much had everything I needed. It's a modular implementation of the BOLT specification in NodeJS, meant exactly for uses cases like mine.
https://github.com/altangent/lntools

As far as I know there are no open nodes anywhere. But I have not been searching for them. Generally you don't need a full node / a full implementation to get the data from the lnd command subscribechannelgraph.
Basically you will only need to reimplement BOLT 07 ( https://github.com/lightningnetwork/lightning-rfc/blob/master/07-routing-gossip.md ) to listen to the gossip protocol.
However since the transport layer in BOLT 08 is encrypted via the noise protocol framework you will also have to reimplement this in order to be able to connect to a peer. Only your peers can be queried for the network graph from BOLT 07. The only other BOLT that you need is BOLT 01 which describes the message format of peer messages.
Implementing these allows you to interact with the peer to peer network and get information about the channel graph without the necessity to implement a full lightning node or even run a Bitcoin node. In fact you don't even need any of the Bitcoin techstack (just ecdsa for your static node key and strong authentication within BOLT 08)
I might make the network graph available via my lightning node next weekend if I remember to do so. Then there would be an open api command to query for this data. Hope that answer helped.

Should make the distinction between a peer you're simply connected to, and a peer which you have channels with. There is no requirement to have channels open to gossip, so implementation of BOLT2 is not strictly necessary. When you first run a lightning node it will perform the initial gossip sync, I believe before any channels are opened. You need channels if you wish to advertise your own node on the gossip network, in which case you would need to then implement BOLT2.

Yes true! I was not precise. No need for entire BOLT 02 but necessity to connect to the peer wich I thought I remembered was part of BOLT 2. Since I was on mobile I didn't double check. Will edit the answer now. Thanks for paying attention (:

As far a solutions go, it's a tall order... That being said, I really like the idea of it. It really fits my research to be able to "mess" with the gossip, without having to run a full node. I'm think I'll dive into this a bit. I'm undecided on the programming language to use. C seems logical, because of c-lightning, Python seems logical mainly because of academia, Go seems logical because of LND (which I like a lot) but I have no experience in the latter. Any thoughts? (I will mark this answer as accepted in a few days, but I keep it open for now to encourage others to answer as well)

You know in the electrum repository there there is a lightning branch with an almost complete implementation of the lightning rfc in python. So it might make sense to extract the gossip code from there

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
getting bch out of my paper wallet
Hi anybody who can honestly and simply tell me how to get my bch out of my paper wallet. I went to blockchain.info and transferred my bitcoin which I had on my paper wallet before Aug 1rst--into their wallet, then safely exported it to an exchange I use--all went well. But I haven't the foggiest how to retrieve my bcc from the same paper wallet now, which I've been told still has the bcc on it. blockchains website is not for a newby like me. I sent their support requesting step by step directions and haven't heard back from them--can anybody give me simple directions on how to get my bcc out of my paper wallet and into my exchange? Please don't assume I know anything and I especially only want to deal with my pc to accomplish this---I have no clue what an app is and don't want to know----Is there someone who's right for this question please?

when you pose your question like this, you might fall victim to scammers. There are already people here in the forum, that told their story of being scammed. So be very careful once you get contacted, and never (ever!) give out your private key or seed(s) to anyone via Internet - you will be robbed!

You could download a full client of bitcoin cash like Bitcoin ABC (https://www.bitcoinabc.org/).
After that you can import your private keys from the paper wallet with the console (help > debug window > console) and enter
importprivkey <yourprivatekey> "label of address"

The optional "label of address" is just a description for you and not visible for anyone else.
After entering the importprivkey command, the wallet has to rescan the wallet for transactions, which may take a few hours depending on your system. After that, your address is fully spendable from within your wallet.
If you want to import multiple private keys, you can disable the rescan option until you entered all private keys with an additional argument. For example, to import 3 private keys you can use 2 times "false" to skip rescan and on the last import "true":
importprivkey <yourprivatekey-1> "label of address 1" false
importprivkey <yourprivatekey-2> "label of address 2" false
importprivkey <yourprivatekey-3> "label of address 3" true

Beware: Ensure that you claim your Bitcoin (BTC) first, and only after successfully sent them to another address use key for claiming your Bitcoin Cash (BCH), because BTC is way more valuable than BCH and I don't know how safe Bitcoin ABC actually is.
This procedure is a bit cumbersome, because you need to download the whole blockchain. But you are not dependent of any third-party applications or websites and don't reveal your private keys to anyone online.
This method can also be applied to any other Bitcoin Fork, assuming that there is a QtWallet for that Fork available.
After claiming your coins I would suggest to never use the address again. Get a new Paper wallet address if you want to use one again.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Current blockchain hot topics in research
I'm looking for a research project for my master thesis in software engineering and computer science and i'm willing to research the blockchain technology. What are the current open issues and/or hot topics regard blockchains?
EDIT: I can see my question is way too generic, i just wanted to specify i read about 20 papers on the topic and i'm more concerned about the dencentralization aspect of it.

I think it's scalability. The cost to generate a block is getting more and more expensive. Couple months ago, CryptoKitties caused Ethereum network congestion due to the surge of transactions. For real world application, I think blockchain must be able to handle high volume of transaction. This can be achieved by reducing the cost to generate a block. AFAIK, some blockchains are developing proof-of-stake algorithms replace the old proof-of-work one.
You may interested in Ouroboros proof-of-stake algorithm, which is implemented by Cardano. Ethereum is also working on their own implementation - Casper

You could have a look at this long list of Bitcoin's weaknesses. Since your question is aimed at blockchains in general, why not also take a look at Ethereum's list of problems.

Cartel theory may be interesting to research since you say you are concerned about decentralization. Bitcoin is carefully considered to avoid cartels and I know that it is a real consideration given when discussing any potential change. You may enjoy listening in on the [bitcoin-dev] mailing list.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What is the "extended private masterkey" line of a dumpwallet file?
What is the "extended private masterkey" in a dumpwallet file? It begins with xprv.

I found this helpful: https://learnmeabitcoin.com/technical/keys/hd-wallets/extended-keys/

It is the BIP32 extended private key which all wallet keys are indirectly derived from. Anyone with access to it can compute all the wallet's private keys.

"Anyone with access to it" (plus a private key, for hardened derivation, right?) "can compute all the wallet's private keys."

Extended private keys let you compute all private keys; there is no other information needed. The extended private key is literally where the keys come from.

What is hardened vs. non-hardened derivation?

With normal (non-hardened) derivation you can get the public keys from the extended public key, as well as the corresponding private keys from the corresponding extended private key. With hardened derivation, there is no usable extended public key, and one needed the extended private key to derive both private and public keys. Hardened is more secure in certain scenarios, but is not as feature-rich.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Is Ethereum essentially Bitcoin with Turing-complete Script?
Is Ethereum essentially Bitcoin with a Turing-complete scripting language and, if I understand correctly, where the script to be executed (not its hash) is placed in the Ethereum blockchain?

If so, why doesn't Ethereum just put hashes of its scripts in the Bitcoin blockchain, send the scripts to be executed through another secure means such as via a P2P network (which would greatly reduce blockchain bloat), and avoid being an altcoin altogether?
If not, is the gas limit the real distinguishing feature of Ethereum compared to Bitcoin? Or is gas really Ethereum's equivalent of Bitcoin's transaction fees? Also, would Ethereum's adoption of proof-of-stake further differentiate Ethereum from Bitcon?

cf. Ethereum's white paper
(cf. the related BitcoinTalk.org thread)

Related: What is the gas limit for?

Is Ethereum essentially Bitcoin with a Turing-complete scripting language and, if I understand correctly, where the script to be executed (not its hash) is placed in the Ethereum blockchain?

That's a bit of a simplification, (Ethereum has other improvements like GHOST) but for our purposes (discussing how a Turing complete scripting language could be implemented) it's essentially accurate.

If so, why doesn't Ethereum just put hashes of its scripts in the Bitcoin blockchain, send the scripts to be executed through another secure means such as via a P2P network (which would greatly reduce blockchain bloat), and avoid being an altcoin altogether?

One of the important features of Ethereum is that you can condition spending funds on the output of a script. For example, you could write a lottery script that accepts money from a bunch of different people, and randomly sends all of the money to the winner. Imagine that we change this system so that only the hash of the script is included in the blockchain:

If it's not a consensus rule, then other nodes will accept a transaction that doesn't satisfy the attached script. In other words, if nodes can't see the entire script, how can a node know whether someone claiming the lottery winnings is actually authorized to do that?
If it is a consensus rule, that introduces a new set of problems. First, in order to tell if a block is valid, you need to download the scripts from the P2P network, and check that all of the scripts are valid. That increases the amount of time it takes to verify blocks, and therefore increases the amount of orphan blocks.
The second problem that this introduces is that full nodes synchronizing for the first time need these scripts, or they won't be able to check that the blocks are valid. Therefore, the scripts need to be stored indefinitely, in case new nodes join the network.
This doesn't really solve the blockchain bloat - it just moves the problem around.

There's one more solution that does use the Bitcoin blockchain, though. You could follow Counterparty's example, and create a system where you send a Bitcoin transaction with additional metadata to identify it as a colored coin. You don't change transaction validity rules on the main network; rather, you say that if the transaction doesn't follow certain rules, then it is not a valid TuringCoin transaction for the purpose of establishing ownership of TuringCoins. (Other TuringCoin transactions within the same block would still be valid, though.)
That's not an altcoin in the traditional sense (you're using Bitcoin's blockchain, and you must pay Bitcoin transaction fees) but it is an altcoin in the sense that you're creating your own token that isn't fungible with Bitcoin.

If not, is the gas limit the real distinguishing feature of Ethereum compared to Bitcoin?

The gas limit in Ethereum is present for essentially the same reason that people have proposed making transactions in Bitcoin larger than 100KB invalid: otherwise, it becomes possible to create a block that takes a really long time to validate.

What is GHOST, exactly?

@Geremia Essentially, even if one of your blocks is orphaned, it can be included as an 'uncle block' in a later block. It reduces the cost of orphaned blocks.


text:
Satoshi's IP address(es)?
Does anyone know Satoshi Nakamoto's IP address(es)? The forums he posted on should be able to figure that out. He posted on p2pfoundation.org, bitcointalk.org, and the cryptography mail archive.

Quote from: theymos [BitcoinTalk.org admin] on April 17, 2013, 11:20:31 PM:

I'll probably release Satoshi's PMs and logged IPs addresses in ~8 years. This'd probably be of great historical interest. (Though he always used Tor, as far as I can tell.)


text:
What does `OP_CHECKSEPARATESIG` refer to?
I was reading about potential OP_CAT applications and read about OP_CHECKSEPARATESIG. This seems to refer to some shorthand, as I am not aware of an opcode by that designation. What is OP_CHECKSEPARATESIG?

OP_CHECKSEPARATESIG has not been formally proposed for bitcoin, but it would amount to an opcode that consumes the parts of a bitcoin signature (r and s and sighash mode) separately, rather than from a single argument.
This separation of r and s allows the r value to be fixed by a bitcoin locking script, which ensures that only one signature for a specific script branch can be produced by the signer without revealing their secret key.
Having a branch that can only be signed once enables the construction of spending bonds which could enable specialized protocols, including zero-conf-safe wallets and a subset of multiparty channel factories.
If OP_CAT is enabled for bitcoin, there is no need for OP_CHECKSEPARATESIG as OP_CAT can be used to build a combined signature from the separate parts without a specialized opcode.


text:
How does Ocean’s TIDES payout scheme work?
This week a new mining pool "Ocean" was announced. The mining pool uses a payout scheme called Transparent Index of Distinct Extended Shares (TIDES). The Ocean website describes the scheme as follows.

Our reward scheme is called Transparent Index of Distinct Extended
Shares (TIDES). As blocks are being mined, they generate the reward by
a weighted percentage of effort to the most recently found proofs. The
proof period funds are distributed across has been chosen such that
each proof should be paid on average 8 times. Instead of a set amount
of bitcoins per proof, the block reward is divided by percent, so
transaction fees are included. Because of this design, each payout to
you is fully auditable. Previous implementations of similar systems
such as PPLNS would distort the payouts owed to miners by using
“shifts” and much smaller proof windows, resulting in miners getting
significantly less accurate payments for their contributed hashrate.
TIDES is the most accurate payout scheme available in Bitcoin today
and unlike other schemes like FPPS it does not require the pool to
serve as a custodial intermediary for payment processing.

Can someone explain in more details how much a miner gets paid in each blocks? Let’s say Alice is contributing 20% of the shares continuously to the mining pool. Bob hops mining pools and contributes 30% to the first block but then doesn’t return to mine with Ocean. How much would they get paid and when?

I see that yesterday this related blog post was published: https://ocean.xyz/docs/tides

I am also wondering how this actually works since there hasn't been
any official documentation about it. However, given the already available information
on the website, 'cause all miners shares and hashrate are public and some questions
answered on Twitter by @wk057 and @GrassFedBitcoin;
I think I can already have a full picture of it.
The reward system consist of three main concepts:

shares earn by each user,
total shares on the pool,
and earnings based on shares.

Earnings
Earnings are the simplest thing to explain. If a user has s shares and the pool
has a total of total_shares (the sum of all user shares) the moment a block is found, the user earns
s * (block_reward) / total_shares

where block_reward means subsidy plus fees.
Share Window
The share window Z is the maximum number of shares the pool will consider at any time. That is only up to the newest Z shares are considered for the reward distribution.
This number is 8 times the current block difficulty Z = 8 * network_difficulty. I will show below that this factor 8
translates into the number of blocks for which each share
is expected to generate an earning.
Once the pool is running for quite some time the number of accumulated shares is enough to have total_shares = Z hence in the examples below
we will consider that to be the case. Though in general this is not
the case at the beginning of the pool operation.
User's shares
One share is equivalent to the work needed to mine a block with difficulty 1.
Ocean, like any other pool, challenges the individual miners with some difficulty d
and each time a hash is accepted d shares are added to this miner's account.
Because there is a maximum number Z of shares considered for the
distribution of rewards,
when a miner starts contributing with a constant hashrate H,
his share number starts growing linearly until a maximum is reached,
this is when the sum of shares in the pool up to Z starting from the
most recent leaves out the oldest shares generated by the user.

In the plot, I call T the relaxation time which is in fact the temporal dimension
of the share window, or simply the time difference between the newest and the oldest share
in the share window.
For as long as the pool has a constant hashrate H_pool,
T can be
understood as the expected time for the pool to produce Z shares:
T = W1 * Z / H_pool

where W1=0x100010001 is (by the definition of difficulty 1), the expected work needed
to solve a difficulty 1 block.
Also in the plot, we see the miner with a constant hashrate H
reaches a share number <S>_T which is the expected
number of shares this miner will receive after T seconds of work:
<S>_T = H * T / W1
      = Z * H / H_pool

If a miner with hashrate H reaches the expected maximum number of shares and then leaves the pool
forever, his shares will linearly decrease as the oldest shares are thrown away from the share window.

We can understand the distribution of rewards from the point of view
of individual shares.
For instance, assuming a constant hashrate of the pool,
a share earned at any time will stay valid for a
time frame of about T. Now consider the expected number of blocks found by the pool
in this time frame <N>_T:
<N>_T = H_pool * T / W1 / network_difficulty
      = Z / network_difficulty
      = 8

hence each share won by a miner is expected to reward him 8 earnings
before it is thrown away from the share window.
Fairness
A pool has a fair earning distribution if the expected earning(W)
for a certain amount of work W
diluted over time is:
earning(W) = block_reward * W / (W1 * network_difficulty)

assuming a constant block_reward and
network_difficulty.
We stress that the denominator W1 * network_difficulty
is the expected work needed to mine a block, approximately the
number of hashes tried by the entire network in 10 minutes.
We had seen that each individual share earned by a miner in the Ocean pool is obtained after W1 of work on average
and it earns the miner a 1/Z fraction of the reward
in an expected number of 8 blocks found by the pool
from the moment the share is added until it is dropped from the
share window. Therefore we have earnings per share computed as:
earning_per_share = 1/Z * 8 * block_reward
                  = block_reward / network_difficulty
                  = earning(W1)

thus the earning distribution is fair according to our definition,
independently of the miner's online time, hashrate,
the pool's hashrate or the network global parameters.
Eventually, if we know a miner's share function in time we can compute the total expected
reward for his shares.
For instance in a time interval [0, t] the share function is s(t),
we split the interval into M small pieces of length dt = t / M.
Then the probability of finding a block in each of these time intervals is
p = H_pool * dt/ (W1 * network_difficulty).
The reward for the miner in the entire time period is a random variable R which is the sum
of the reward for the miner in each of the tiny time intervals:
R = sum_i r_i

where r_i = b_i * block_reward * s(t_i) / Z, in which b_i denotes a Bernoulli variable that takes values
1 with probability p and 0 with probability 1-p.
So the expected value of the reward for the single miner becomes:
<R> = sum_i <r_i>
    = block_reward * H_pool / (Z * W1 * network_difficulty) sum_i s(t_i) dt

that converges to a Riemann integral in the limit
of M -> infinity, the integral is the area under
the curve of shares vs time.
We can use this relation to compute the total earnings of a miner in different example
situations.
Example 1. Alice mining always
Let the hashrate of the pool be H_pool = 1.25 EH/s, consider Alice a miner with a hashrate
of H_alice = 250 PH/s, which corresponds to a 20% of the pool's hashrate.
And let network_difficulty = 80T.
The share window of the pool is Z = 8*network_difficulty = 640T and the time window
would correspond to T = Z * W1/H_pool = 330 Ms or about 25 days.
Alice has been mining non-stop since the beginning, so she has already reached the maximum
number of shares in the window: S_alice = H_alice * T / W1 = 128T.
If a block is found, Alice will get S_alice / Z = 20% of the block reward.
That means on average in a time frame t
Alice does a total work W_total = H_alice * t
and the pool finds <N>_t = H_pool * t/ (W1 * network_difficulty)
blocks so Alice earns
<R>_t = <N>_t * block_reward * S_alice / Z
      = block_reward * W_total / (network_difficulty * W1)
      = earning(W_total)

which is fair for Alice.
To put some numbers into it, it means that for blocks with
block_reward = 7 BTC, Alice is earning on average 0.003 BTC
every 10 minutes or 0.44 BTC/day, and this number only depends on Alice hashrate
and the network parameters (block_reward and network_difficulty).
Example 2. Charlie the small miner
Now consider a small miner that arrives to the pool,
it is small so the total hashrate of the pool doesn't change.
Let's say Charlie has H_charlie = 1 TH/s.
At a constant hashrate and difficulty, Charlie would have to wait
for at least T seconds or 25 days until it can reach the maximum number of shares,
see fig 3. If after some time dt Charlie leaves the pool, his shares will
start to fall linearly to zero in a time period of 25 days (relax time T).

During all the time Charlie was hashing for the pool, he contributed to
a total of W_total = H_charlie * (T + dt).
The expected earnings of Charlie in the three regions in fig. 3 are
(using the integral under the curve):
<R1> = H_pool * block_reward / (Z * W1 * network_difficulty) * (S*T/2)

<R2> = H_pool * block_reward / (Z * W1 * network_difficulty) * S* dt

<R3> = H_pool * block_reward / (Z * W1 * network_difficulty) * (S*T/2)

where S = H_charlie * T / W1 is the maximum number of shares of Charlie.
When these three are summed up we find
<R1 + R2 + R3> = block_reward * S / network_difficulty * (1 + dt/T)
               = block_reward * W_total  / (W1 * network_difficulty)
               = earning(W_total)

Which means that a fair earning is expected for Charlie if
he comes and mines for a long time (until it reaches constant shares)
and then leaves the pool.
For example, if Charlie mines for T = 25 days reaching constant shares
and then after mining for 10 more days he turns off his
miner. Then <R1> = <R3> = 2240 sat and <R2> = 1760 sat,
and a total expected earning of <R1 + R2 + R3> = 6240 sat
before his share count in the window is 0, or 176 sat/day for
the ratio of total earnings divided by the work time.
Example 3. Charlie leaves early
Consider again the small miner Charlie, but this time he mines for some time t1 < T, so his shares start to accumulate linearly
until, at time t1, he turns off his miner;
then for some time his shares remain constant,
until at time T his oldest share
falls out of the share window and his share count starts to decrease linearly until it
reaches zero at time T + t1, see fig. 4.

The total work is W_total = H_charlie * t1 and the expected earnings are:
<R1> = H_pool * block_reward / (Z * W1 * network_difficulty) * (S*t1/2)

<R2> = H_pool * block_reward / (Z * W1 * network_difficulty) * S*(T-t1)

<R3> = H_pool * block_reward / (Z * W1 * network_difficulty) * (S*t1/2)

where S = H_charlie * t1 / W1. When we add them up we find:
<R1 + R2 + R3> = block_reward * S / network_difficulty
               = block_reward * W_total / (W1 * network_difficulty)
               = earning(W_total)

Which means a fair earning is expected for Charlie even if he participates for
a short period of time.
For example, let t1 = 1 day, it follows that <R1> = <R3> = 3.5 sat
and <R2> = 169 sat, in total <R1 + R2 + R3> = 176 sat
or 176 sat/day (earnings per time of work).
Example 4. Bob the big miner
Now assume, a big miner Bob enters the pool with H_bob = 1.25 EH/s, doubling the pool's
hashrate.
Bob quickly starts adding shares and the pool would need to discard old shares
as twice as fast than previously to keep the window size at constant Z. In other words
the relaxation time changes from T = 25.5 days to T' = Z * W1 / (H_pool + H_bob) = 12.7 days.
It will take Bob 12 days, until he reaches the maximum number of shares: S = Z/2 = 320T.
After this moment, every time a block is found Bob will earn half of the block reward.
If Bob decides to leave after some time dt of having a constant share count,
then his shares will start decreasing linearly with time, see fig. 5.

But since the pool has now less hashrate than when Bob was mining, it will take 25 days before
all of Bob's shares are gone. But still the expected number of blocks is 8.
Due to the fairness argument described above, each one of Bob
shares is as good as any other miner shares.
The total work done by Bob is W_total = H_bob * (T' + dt) while the expected earnings on each
region are
<R1> = (H_pool+H_bob) * block_reward / (Z * W1 * network_difficulty) * (S*T'/2)

<R2> = (H_pool+H_bob) * block_reward / (Z * W1 * network_difficulty) * S*dt

<R3> = H_pool * block_reward / (Z * W1 * network_difficulty) * (S*T)/2

again adding them up we find that
<R1 + R2 + R3> = block_reward / network_difficulty * (S + S*dt/T')
               = block_reward * W_total / (W1*network_difficulty)
               = earning(W_total)

Also for the big miner we verify that earnings are fair.
For instance if dt = 7 days we find that <R1> = <R3> = 14 BTC
and <R2> = 15.4 BTC, for a total expected earning of
<R1+R2+R3> = 43.3 BTC or 2.2 BTC/day (total earnings per day of work),
notice this is always 176 sat/day for each TH/s of hashrate.
Example 5. Bob the big miner leaves early
Now assume, Bob enters the pool but mines only for a short
period of time t1 < T'.
While Bob is mining his shares grow linearly up to
S = t1 * H_bob, then after he stops
the number of shares remain constant for some time dt until
the oldest shares by Bob start to leave the share window:
t1 * (H_pool + H_bob) + dt * H_pool = Z * W1

Afterwards, his shares starts decreasing linearly
until zero at time t1 + T; see fig. 6.

The total work done by Bob is W_total = H_bob * t1 while the expected earnings on each
region are:
<R1> = (H_pool+H_bob) * block_reward / (Z * W1 * network_difficulty) * (S*t1/2)

<R2> = H_pool * block_reward / (Z * W1 * network_difficulty) * S*dt

<R3> = H_pool * block_reward / (Z * W1 * network_difficulty) * S * (T-dt)/2

again adding them up we find that:
<R1 + R2 + R3> = block_reward / network_difficulty * S * (t1/T' + dt/T + 1 )/2
               = block_reward * W_total / (W1 * network_difficulty)
               = earning(W_total)

Earnings are always fair.
Now let's say t1 = 1 day, during that period of time
the miner will earn <R1> = 0.086 BTC. Then, the miner turns off
and during the following dt = (1-t1/T') * T = 23.5 days
in which the shares remain constant the miner will earn
<R2> = 2.027 BTC.
Then finally the shares will start to decrease linearly for the next
T-dt = 2 days and the miner will earn <R3> = 0.086 BTC,
for a total expected earnings of <R1 + R2 + R3> = 2.2 BTC
or 2.2 BTC/day
(total earnings for time of work) or 176 sat/day for each TH/s.


text:
Unable to decode input address
It seems that some transactions, although having "prev_out" field in their "inputs", does not have "addr" in the "prev_out", i.e. they do not have input address. What is this kind of transactions for?
e.g. transaction hash: 38d4154f800cbbce0ac39772dd88d338424be21ba36ccb6a11110a029e6b5a1e

Yes, this tx has only the signatures in the V_IN part (unlocking script), no public keys directly behind it, or a redeemscript. The V_OUT part (locking script) has a redeemscript for a 2-of-3 multisig. Usually one would expect a redeemscript hash in the pubkeyscript, but here it is the plain hex codes.
So it is an old type multisig (not using P2SH). The unlocking script is the two signatures, and locking script provides the required pubkeys.
So what would happen on the stack:
<sig1>
<sig2>
<redeem script data> ("2" pk1 pk2 pk3 "3" CHECK_MULTISIG)
In a first step the unlocking script is executed, and if valid (results in a TRUE value, or no remaining operators remain on the stack), the tx would be marked as valid. It only pushes data (the signatures), so ok. Then the second step would clear the stack, and sigscript and pubkeyscript would be pushed and executed. As one can see, the three elements on the stack can be resolved to TRUE, and as such it is a simple multisig.
Decoding the pubkeys in the redeem script shows:
5221037BF122821AE522F67B7BC09387C2907607CDD66E6BD67BCB53F39D5346ADC74821033E81519ECF373EA3A5C7E1C051B71A898FB3438C9550E274D980F147EB4D069D21027AA4A8DAFFF9B6307899BB7762BF92C50ABDE14C3636719FFC4D4FABF3DC6B0F53AE

    52: OP_2
    21: OP_Data21 (= decimal 33)
        037BF122821AE522:F67B7BC09387C290
        7607CDD66E6BD67B:CB53F39D5346ADC7
        48
        bitcoin address:   18pvHhDLrGJpJ7SznnoxxFdwPnKCqjqVCj
    21: OP_Data21 (= decimal 33)
        033E81519ECF373E:A3A5C7E1C051B71A
        898FB3438C9550E2:74D980F147EB4D06
        9D
        bitcoin address:   1JXc8zsSeAPwqfAzLbBnZxNTfetZexH2bW
    21: OP_Data21 (= decimal 33)
        027AA4A8DAFFF9B6:307899BB7762BF92
        C50ABDE14C363671:9FFC4D4FABF3DC6B
        0F
        bitcoin address:   1BDp6sR3bHH8ACzbwZccY4y9GohG3KX828
    53: OP_Data53 (= decimal 83)
    AE: OP_CHECKMULTISIG

So just an "old type" multisig transaction.


text:
Unable to execute getwork logic in java
I was going through this post in stack exchange How can I code a Bitcoin JSON-RPC “getwork” request in Java?
I tried to write a simple snippet for just the getwork json rpc.
public static void main(String[] args) throws Exception {

    String request = "{\"method\": \"getwork\", \"params\": [], \"id\":0}";
    URL url = new URL("http://de01.supportxmr.com:7777");

    HttpURLConnection conn = (HttpURLConnection) url.openConnection();
    if (conn .getConnectTimeout() == 0)
        conn.setConnectTimeout(1000);
    if (conn.getReadTimeout() == 0)
        conn.setReadTimeout(1000);
    conn.setRequestMethod("POST");
    String encoded = Base64.getEncoder().encodeToString(("<my_wallet_addr>:x").getBytes(StandardCharsets.UTF_8));  //Java 8
    conn.setRequestProperty("Authorization", "Basic "+encoded);
    conn.setRequestProperty("Accept", "application/json");
    conn.setRequestProperty("Content-Type", "application/json");
    conn.setRequestProperty("Content-Length", Integer.toString(request.getBytes().length));
    conn.setRequestProperty("X-Mining-Extensions", "midstate");
    conn.setAllowUserInteraction(false);
    conn.setUseCaches(false);
    conn.setDoOutput(true);

    DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
    wr.writeBytes(request);
    wr.close();

    InputStream is = conn.getInputStream();
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    int len;
    byte[] buffer = new byte[4096];
    while ((len = is.read(buffer)) != -1) {
        bos.write(buffer, 0, len);
    }
    String content = bos.toString();
    is.close();

    System.out.println(content);

}

when I run this code, I get an error
Exception in thread "main" java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:789)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at org.litecoinpool.miner.Test.main(Test.java:42)

What am I missing here? Is stratum proxy necessary to be running on the machine? If so how do I specify the parameters to run in the java code?
I tried a direct TCP connection also to the server.
public static void main(String[] args) throws Exception {
    String message1 = "{\"id\":1,\"method\":\"mining.subscribe\",\"params\":[]}";
    String authorizemessage = "{\"params\": [\"<wallet_address>\", \"x\"], \"id\": 2, \"method\": \"mining.authorize\"}";

    Socket soc = new Socket("de01.supportxmr.com", 7777);
    System.out.println("connected");
    OutputStream outputStream = soc.getOutputStream();
    outputStream.write(authorizemessage.getBytes());
    outputStream.flush();

    BufferedReader in = new BufferedReader(new InputStreamReader(soc.getInputStream()));
    JSONObject json = new JSONObject(in.readLine());

    System.out.println("json response: " + json.toString());

    outputStream.write(message1.getBytes());
    outputStream.flush();

    in = new BufferedReader(new InputStreamReader(soc.getInputStream()));
    json = new JSONObject(in.readLine());

    System.out.println("json response: " + json.toString());

}

but no luck again :(

The getwork RPC call was removed from Bitcoin Core. It was deprecated and then superceded by the getblocktemplate RPC call.

It looks like the pool that you are using uses an unspecified protocol. It's just a tcp connection where you are sending JSON formatted strings and receiving JSON formatted strings from the pool server. I figured this out by digging through this mining software's source code.

I tried that. no luck either. :( by the way I am connecting to a monero pool URL if you see.

You have to use whatever protocol the pool is using. They probably use stratum, not getwork or getblocktemplate.

looks like they use getblocktemplate from this link : https://getmonero.org/resources/developer-guides/daemon-rpc.html

That's not necessarily true. That link points to the official monero node software documentation. But that is not the same as the pool that you are using. They don't necessarily have that exposed to miners to connect to, they may (probably) use stratum which is run by a software external to the node.

ok. let me try to use stratum and update the post. Thanks for your suggestions:)

Thanks again for your analysis. Even i suspected this and I tried direct connection but could not read anything from the input stream. Please check the edited question section for the tcp code at the end.

Finally I found the right string to be passed to the pool.
String message1 = "{\"id\":1,\"method\":\"getblocktemplate\",\"params\":[\"wallet_address\":\"<wallet_address>\", \"reserve_size\" : 60]}\n";

String authorizemessage = "{\"params\":{\"login\":\"<wallet_address>\", \"pass\":\"x\"}, \"id\": 2, \"method\": \"login\"}\n";

After that we can first authorize using the authorizemessage and later pass message1 through the OutputStream of the socket.


text:
Is there api for ripple.com to authentificate with username & password?
I'm googling, and looking for such option, to authenticate to ripple.com with out secret key, but username and password.
I've found only with secret key:
# curl -X POST -d '{ "method" : "sign", "params" : [ { "secret" : "snoPBrXtMeMyMHUVTgbuqAfg1SUTb", "tx_json" : {"TransactionType":"Payment",  "Account":"rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh", "Amount":"200000000", "Destination":"r3kmLJN5D28dHuH8vZNUZpMC43pEHpaocV" }} ] }' http://s1.ripple.com:51234

How properly to authenticate with username and password only?

Usernames and passwords are client wallet functions. They have no meaning to the Ripple network and servers would have no idea what to do with them.
In fact, servers do not have permission to obtain encrypted wallets because if they did, they could attempt to brute force decrypt them, negating the security advantages of the blob vault. By design, is is as difficult as possible to ask someone else to obtain your wallet, purportedly on your behalf. This is necessary to prevent malicious web sites from sending javascript to your browser that causes it to brute force other people's wallets as you browse the Internet.

Ripple.com does not store any information about the users of the ripple network. Instead if you want to make transactions using only a username and password you must store your secret key in an encrypted blob vault, using your username and password credentials to create the encrypted cypher-text.
The web wallet app available at ripple.com/client uses such a blob vault to store and retrieve your ripple secret key. Then behind the scenes it signs transactions to the ripple network using your key. The provider of your vault is configurable and if you desire you may implement your own blob vault by following the vault protocol and by consulting the official open source blob vault code on github.
Of course in your own ripple applications you can use whichever protocol you like to store and retrieve your secret key using a username and password for encryption. Remember it is not possible to make transactions in the ripple ledger without a signature from your account secret key.


text:
Setting up: sending and email
I've been using Bitcoin Wallet on windows 7 to receive payments successfully but after making my 1st BTC payment the transaction doesn't appear to be clearing (957 confirmations and counting).
The recipient of the payment has asked for my transaction email confirmation but I don't recall setting an email address or where to check if I ever did this.
Please can someone help?

If the transactions is shown in your Bitcoin-Qt client as having confirmations, then the transaction went through and is confirmed. Most systems consider a transaction as confirmed when it has one confirmation, some require up to 6 confirmations, but when a transaction has 957 confirmations, it is practically irreversible.
If your transaction has that amount of confirmations and the receiver did not get it yet, then he has a problem with its client, not you.
There exists no such thing as e-mail confirmations in Bitcoin. It's possible that some clients offer it as a feature, but it is certainly nothing standard to Bitcoin.
(I don't know who the recipient is and I don't want to accuse anyone, but it seems like he is trying to have you send the transaction again.)


text:
Why does Electrum's createnewaddress() only return legacy addresses?
Why does Electrum's createnewaddress() (or "Create Request") only return legacy addresses? How do I make it create bech32 addresses?

Electrum stores whether to create legacy addresses or segwit addresses at the wallet level (it is part of the Electrum seed itself), hence each wallet is configured to only produce legacy addresses or only segwit addresses. It sounds like the wallet you are using is a legacy address wallet.

I imported a mixture of address types to the wallet.

createnewaddress() isn't going to give you addresses that have been imported. It should be generating new addresses, and it will do this based on the seed stored in the wallet, which indicates the address type to create.

I created a new wallet and imported a mixture of legacy and bech32 addresses. Is "the seed stored in the wallet" a legacy seed or something? It would seem a newly created wallet would have a modern seed that can generate bench32 addresses.

Presumably, yes, the stored seed is one that makes legacy addresses. You can check by going to Wallet > Information and looking at the "Script type" field.

I see. It says p2pkh. It shows the xpub.


text:
How can I check the balance of a 10 year old wallet that can't sync?
I have a very old PC that blue screens every time I try to sync bitcoin core and I was wondering if there was any way I could check the balance of the wallet before I destroy the PC?

It has 10~ receiving addresses and they all seem to have 0 balance on cointracker.io and blockchain.com/explorer/
I've think I've also extracted the public and private keys but I'm not sure what to do next
Presumably if all of the receiving addresses are showing 0 balance then there is no point pursuing this?
I am a bit tentative especially as when I check my Ledger wallet and receiving addresses, they also appear as 0 balance on these sites even though I have more than 0 balance

Alternatively, you can export all private keys, copy them to a new computer, install Electrum (a lightweight wallet which doesn't need to download the full blockchain), create a new wallet using the private keys and check the balance.

How can I check the balance of a 10 year old wallet that can't sync?

I would

Use the backup option in the wallet software.

Copy the backup file to a new computer.

Install the latest Bitcoin core on the new computer.

Restore the wallet from the backup file on the new computer.


text:
I can't get Inbound Connections on my node
So, I've been trying to get Inbound Connections on my node, to no success.
I've got an UmbrelOS 1.1 installed on a RPi5. It's Bitcoin Node app is already set to allow Incoming Connections.
I also went into my router, added exceptions to the internal network's DHCP to the RPi5's MAC, and forwarded both IPv4 AND IPv6 TCP 8333 ports to my fixed RPi5 IP.
I also made sure that there were no Firewalls on my RPi5's Umbrel, not even UFW or something like that.
I restarted both my Node and router.
The problem is that, even doing all this, when I check bitnodes.io, it says that my IP still has it's port unreachable.
One thing I noticed is that the external IPv4 address my router indicates to be mine, is different from the one announced in whatismyip.com. This got me wondering if this is some kind of CGNAT problem. If it is: what can I do to solve it?
Thanks a lot.

Just wanted to update that my ISP does use CGNAT, so my external IPv4 is not personal but public. I'm trying right now to change that through contacting my IPS. If it does work after getting a personal external IPv4 address, I'll update this post with an answer!

Alright! I just solved it! My full node is now up and running! :D
So, there were two problems:

I indeed had CGNAT open by my ISP, so I called it and asked them to take it off;
Port forwarding on my router, for some reason, doesn't work with source IP 0.0.0.0 (?); so the source IP field has to be entirely blank in order to refer to "any IP" - which is something different from all the other routers I've ever had, by the way.

So that's it! Hope this somehow helps someone in the future! :D


text:
Is there a good intend behind the scripting system bitcoin uses for transactions?
From https://en.bitcoin.it/wiki/Script :
"Scripting provides the flexibility to change the parameters of what's needed to spend transferred Bitcoins. For example, the scripting system could be used to require two private keys, or a combination of several keys, or even no keys at all."
I'm wondering if there is a "good" reason to provide that "flexibility". Foremost why is there a possibility to ask for "no keys at all"?
I want to apologize if I'm misunderstanding something here or if I'm unaware or naive or "sounding" cryptic and for my bad english.

why is there a possibility to ask for "no keys at all"?

The statement you quoted is about "requiring no keys" in the sense of not explicitly requiring the use of keys. The statement you quoted isn't about asking for keys. requiring and asking are different.
I suspect it isn't simple to specify a scripting language that allows for key based security but which prohibits scripts omitting any kind of key based security. The designer of the system used a scripting language to create flexibility for future innovation, not to suppress it.
Here's an extract from an early conversation between Gavin Andresen and Satoshi Nakamoto, the creator of Bitcoin and its initial scripting system.
Gavin Andresen June 17, 2010, 11:38:31 AM

... First: it make me a little nervous that bitcoin has a scripting
language in it, even though it is a really simple scripting language
(no loops, no pointers, nothing but math and crypto). It makes me
nervous because it is more complicated, and complication is the enemy
of security. ...

Satoshi Nakamoto replied

The nature of Bitcoin is such that once version 0.1 was released, the
core design was set in stone for the rest of its lifetime. Because of
that, I wanted to design it to support every possible transaction type
I could think of. The problem was, each thing required special support
code and data fields whether it was used or not, and only covered one
special case at a time. It would have been an explosion of special
cases. The solution was script, which generalizes the problem so
transacting parties can describe their transaction as a predicate that
the node network evaluates. The nodes only need to understand the
transaction to the extent of evaluating whether the sender’s
conditions are met.
The script is actually a predicate. It’s just an equation that
evaluates to true or false. Predicate is a long and unfamiliar word so
I called it script.
The receiver of a payment does a template match on the script.
Currently, receivers only accept two templates: direct payment and
bitcoin address. Future versions can add templates for more
transaction types and nodes running that version or higher will be
able to receive them. All versions of nodes in the network can verify
and process any new transactions into blocks, even though they may not
know how to read them. The design supports a tremendous variety of
possible transaction types that I designed years ago. Escrow
transactions, bonded contracts, third party arbitration, multi-party
signature, etc. If Bitcoin catches on in a big way, these are things
we’ll want to explore in the future, but they all had to be designed
at the beginning to make sure they would be possible later.


text:
Why is 550 (MiB) chosen for minimum storage size for prune mode?
As you know Bitcoind can run in pruned mode, thus not advertising blocks to other nodes. Even though it has some drawbacks, saving you storage space isn't a benefit you can ignore.
My question is about the magic 550(MiB) default/minimum number. Why they've selected that limit? (& Why not less or more?)
I know it implicitly indicates blocks will be kept for the last (about) 3.8 days (91 hours) (one MiB because of one block every 10 minutes). But where that number come from? I know there is also a dust limit equal to 546 Satoshis.[182*3 ~ 550 here! Even though it is unlikely to be related to this case]
Would you please explaining what the logic behind 550 is? Is there any kind of difficulty level relating to 51% attack or any other stuff like that?
Edit:
From 0.12.0, bitcoind supports relaying NEW blocks to other nodes, thus the first assumption up there, isn't an accurate sentence!

Just got help from someone at BitcoinTalk forum, who directed me to bitcoind 0.11.0 changelog (Which pruning was introduced/implemented for the first time in bitcoind)
As indicated there, Block pruning deletes raw block & undo data:

...
  there are four types of data related to the blockchain in the bitcoin system: the raw blocks as received over the network (blk???.dat), the undo data (rev???.dat), the block index and the UTXO set (both LevelDB databases). The databases are built from the raw data. Block pruning allows Bitcoin Core to delete the raw block and undo data once it's been validated and used to build the databases
  ....

which then explains where the number 550 (Not magic anymore!) has come from:

The minimum allowed is 550MB*. Note that this is in addition to whatever is required for the block index and UTXO databases. The minimum was chosen so that Bitcoin Core will be able to maintain at least 288 blocks on disk (two days worth of blocks at 10 minutes per block). In rare instances it is possible that the amount of space used will exceed the pruning target in order to keep the required last 288 blocks on disk.

*Typo! MiB(1024) is right & not MB(1000). So:
550*1024*1024 = 576716800
576716800 / 288 ~ 2MB

Let me append it that with release of bitcoind 0.12.0, Wallet functionality & block relay that was disabled at pruned mode(& I talked about in my question), are supported again.


text:
Bitcoin rpc testnet balance and transaction not visible
I'm using bitcoind-0.15.0.1 version & My Wallet Address is: muweVvHKnK1bE88pUsrpSAPuWry1V1AmHZ
But whenever checking balance with following command:
bitcoin-cli getreceivedbyaddress 'muweVvHKnK1bE88pUsrpSAPuWry1V1AmHZ'
It appears to be 0.00000000, while the wallet has 0.0007 BTC stored!
Bitcoin.conf indicates I'm connected to the testnet: testnet = 1
What is the problem?

Is muweVvHKnK1bE88pUsrpSAPuWry1V1AmHZ an address of the wallet you're querying? What does getbalance say?

My wallet address = muweVvHKnK1bE88pUsrpSAPuWry1V1AmHZ
bitcoin-cli getbalance = 0.00000000
There is a balance in the wallet but it does not appear.

Then I think you're querying the wrong wallet. You can't ask for information about addresses that are not in that wallet itself.

wallet true. registered in node. but no balance appears.

How did you inport the address? What does "getaddressinfo muweVvHKnK1bE88pUsrpSAPuWry1V1AmHZ" say?

getaddressinfo Method not found. because i use node version 0.15.0.1

Then you should update urgently! You can use validateaddress instead of getaddressinfo in 0.15.

./bitcoin-cli validateaddress "muweVvHKnK1bE88pUsrpSAPuWry1V1AmHZ"
{
"isvalid": true,
"address": "muweVvHKnK1bE88pUsrpSAPuWry1V1AmHZ",
"scriptPubKey": "76a9149e3d4e1e102c388f26f2e4d6cabe38d7be0187ea88ac",
"ismine": true,
"iswatchonly": false,
"isscript": false,
"pubkey": "031365b3a9d073ac31fc2a92ddf3c894972a5e0323b764e963fc2d8e229dc44569",
"iscompressed": true,
"account": "testnet-icin",
"timestamp": 1567514729,
"hdkeypath": "m/0'/0'/8'",
"hdmasterkeyid": "ab73360cdaf8a69a63ed1c816129c4d9a33d4c1c"
}

Are you fully synchronized with the chain? What does getblockchaininfo say?

{
"chain": "test",
"blocks": 1012566,
"headers": 1576881,
"bestblockhash": "000000000000019bb5aa7db72fd5c4c01e42b80815a0e80d9ee2dce46aaf548d",
"difficulty": 3241932.284902167,
"mediantime": 1477459028,
"verificationprogress": 0.4805068858447337,
"chainwork": "000000000000000000000000000000000000000000000018c885775e04566600",
"pruned": false,
"softforks": [
{
"id": "bip34",
"version": 2,
"reject": {
"status": true
}
},
{
"id": "bip66",
"version": 3,
"reject": {

"status": true
}
},
{
"id": "bip65",
"version": 4,
"reject": {
"status": true
}
}
],
"bip9_softforks": {
"csv": {
"status": "active",
"startTime": 1456790400,
"timeout": 1493596800,
"since": 770112
},
"segwit": {
"status": "active",
"startTime": 1462060800,
"timeout": 1493596800,
"since": 834624
}
}
}

Ok, you're half a million blocks behind. Just wait until it fully synchronizes.

how long does it take? I encode a payment system with bitcoin. what should I do to not wait so long?

Depending on your hardware and settings a few hours to a few days. There are many questions about speeding things up.

I am using ubuntu virtualbox. I think this is the reason for the delay.i will buy vps server. What features do you recommend?

RAM, fast disk, and set the dbcache option in your config file appropriately.


text:
BIP39's checksum generation
From what i've read, the checksum is just a basic SHA-256 hashed string, when only taking (Bitlen/32) bits of the hashed string.
Yet when I do it, I get incorrect values (resulting in incorrect last words)
This is my implementation of the checksum calculator.
I've implemented the hashing algorithm by myself, but there shouldnt be any issues with it.
std::string BIP39SeedMaker::checksum(cpp_int v)
{
    int binSize = binary(v).size();
    std::string hash = sha.digest(v.str());
    std::string hashBin = binary(cpp_int("0x"+hash));
    return hashBin.substr(0, binSize/32);
}

Please let me know if you see anything wrong, im really confused

std::string BIP39SeedMaker::checksum(cpp_int v)
{
int binSize = binary(v).size();
std::string hash = sha.digest(v.str());
std::string hashBin = binary(cpp_int("0x"+hash));
return hashBin.substr(0, binSize/32);
}


text:
Why does BIP-340 use secp256k1?
If the entire Taproot/Schnorr soft-fork was going to implement an entirely new signing scheme anyway that was completely independent from the previous ECDSA scheme, why did it use secp256k1?
Couldn't it have used other curves, like Curve25519 and its signature schemes that are being used by many other projects? Wouldn't that come with a lot of advantages -- like key aggregation, signature aggregation and threshold signature schemes already well-developed and deployed in other projects? Aren't these schemes generally faster on Curve25519?
What was the advantage of sticking to secp256k1?

Is there evidence of a signature aggregation scheme that is well developed and deployed in other projects (and uses a pairing-unfriendly curve like Curve25519)?

Sorry, I had this impression from looking at a stupid comparison table in some website some time ago, but now I can't find that table nor any other evidence, I think the table probably mentioned just key aggregation, not signature aggregation.

@fiatjaf Are you talking about https://safecurves.cr.yp.to/ by any chance?

No, not that one.

Couldn't it have used other curves, like Curve25519 and its signature schemes that are being used by many other projects?

Curve25519 is a key agreement scheme, but if you're referring to the (related) Ed25519 system (which uses a variant of the same curve), yes, it could have.

Wouldn't that come with a lot of advantages -- like key aggregation, signature aggregation and threshold signature schemes already well-developed and deployed in other projects?

To the best of my knowledge, at the time, none of these things existed. Even the research aspect of practically usable key aggregation and threshold schemes like MuSig and FROST is very new. Standardization of those is underway now for multiple curves, but that was not the case back then.
Schemes predating BIP340:

The MuSig1 paper is from Jan 2018 (but it was broken, and fixed in May 2018).
The FROST paper is from Jul 2020
The MuSig2 paper is from Oct 2020

Standardization efforts postdating BIP340:

A standardization effort for FROST is underway since Feb 2021, which includes Ed25519 and secp256k1 support (but not BIP340 compatible).
BIP327 and bip-frost-dkg are BIP340-compatible, but are very recent.

There exist (much) older multisignature schemes too, but none that were ever deployed in production as far as I know, due to various issues that made them hard to use in the real world.

Aren't these schemes generally faster on Curve25519?

Yes, implementations of Ed25519 exist that are somewhat faster than secp256k1.

What was the advantage of sticking to secp256k1?

Quoting BIP340:

By reusing the same curve and hash function as Bitcoin uses for ECDSA, we are able to retain existing mechanisms for choosing secret and public keys, and we avoid introducing new assumptions about the security of elliptic curves and hash functions.

So the reasons are:

Reusability of existing key derivation infrastructure, including BIP32.
Not changing security assumptions. Bitcoin's security already relies on ECDLP not being broken for secp256k1. BIP340 doesn't change anything about that. Bringing in a new curve would involve an additional assumption.


text:
What happens if your bitcoin client generates an address identical to another person's?
Here's a what-if scenario:
Person A has a Bitcoin address with 25BTC.
Person B opens up their Bitcoin client:

which may or may not have the complete blockchain (the latter would
mean no copies of Person A's transactions)

Person B presses "New Address", and Person A's address happens to somehow be generated. Now, the blockchain finishes synchronizing.
What happens? Is this a possibilty, sans the astonishing improbability? After all, random number generation can be influenced.

They'd be able to spend each other's coins.

related: How many Bitcoin addresses are there?, Is it possible to brute force bitcoin address creation in order to steal money?

@rponder No, the labels you attach to some addresses are just on your own computer.

@SaintHill oh wow, that's bad

Generate the same keypair hash (AKA collision) equals the chances of Chloë Grace Moretz be my gf, 1,461,501,637,330,902,918,203,684,832,716,283,019,655,932,542,976 of possible addresses, how many chances do you think you have?

If this happens, then Person B will be able to spend person A's bitcoins. However, there are only two ways this can happen:
a) Person B generates the same keypair (private key) as person A
or b) Person B generates a different keypair, which (public key) hashes to person A's address (a hash collision)
Take a look at the specification for at bitcoin address: https://en.bitcoin.it/wiki/Technical_background_of_Bitcoin_addresses

A Bitcoin address is a 160-bit hash of the public portion of a
  public/private ECDSA keypair

For "a" to happen, person B would need to break "secp256k1".
I am no expert on this field, but from what I can find on bitcointalk.org, then the "strength" of this is 2^128 bits. And if you're wondering how strong that is, then look for the video "Exhaustive search attacks" from Dan Boneh. 11 minutes in the video he says "anything that's bigger than 2^90 is considered sufficiently secure" (its about attacking 3DES, which is a symmetric block cipher so I'm not sure it actually apply to secp256k1, but I really have no better argument for why 2^128 is "so secure that you won't never, ever, ever generate the same key as someone else"
And for "b" to happen, you would need to find a collision on RIPEMD-160(SHA-256(pk)). As far as I know, then neither RIPEMD-160 nor SHA-256 is known to have any vulnerabilities to hash collision attacks. So somehow breaking both or randomly generation a key which hashes is exactly the same as another bitcoin address, is highly unlikely.
There is also a chance for you computer to catch on fire, and some of the materials to melt together into a lotto coupon with winning numbers on (and a valid barcode), but it just won't happen because of the chance is so unbelievably small (it's the same with the "click and generate another persons bitcoin address").
EDIT Woops, one important thing: This all assumes that all bitcoin addresses are properly generated using "true random". All brain-addresses and addresses generated using a bad PRG, might be easy to find by either using a flaw in the PRG or exploit the (flaws in the) human brain.

Your case "b" is a (multi target) second pre-image, not a collision. You want to match the hash of an existing message you can't influence. A collision would be Person B creating two distinct inputs with the same hash.

But are all addresses verified to check for duplicates when the blockchain is synchronised by your address generating client? If there is an address collision, wouldn't it cause serious confusion and possibly substantial financial loss (if funds are transferred to person b's account since they were under the illusion they controlled it?) Can we avoid person b losing funds by alerting them an address they generated, has actually since been detected while they caught up with the blockchain? At least then person b knows to wait for sync before asking for payments to this address.

I believe case A will be far more common once Bitcoin becomes mainstream. Lots of people will use weak private keys, and plenty of them will use the same as another random guy, resulting in them both spending each other's coins :(

How can you calculate odds without taking into account processing power and its exponential increase? Yes, bitcoin has some protection against that... but it's a significant factor. An alien civilization with black-hole-like-dense quantum computers have probably already stolen all your coins in some infinity of paralellel universes... http://bitcoin.stackexchange.com/questions/6062/what-effects-would-a-scalable-quantum-computer-have-on-bitcoin

Here are some cases of "a)" on mainnet :)) https://lbc.cryptoguru.org/trophies

This is the way I see it. The total number of bitcoins that will ever be mined is 21 million. The smallest bitcoin unit is a satoshi (0.00000001 BTC). If we place all possible satoshis into a wallet of their own, we would get the maximum number of wallets that could have any balance to them (so the actual number of wallets with bitcoins is obviously less). This is 21x10^6(BTC) x 10^8 (satoshi/BTC) = 21 x10^14 wallets. It's a huge number, but it's eclipsed by 2^256 possible wallets. So in this worst case scenario, the probability of guessing a wallet with one satoshi is 21x10^14/(2^256)=1.813595x10^-62. An incredibly small number.

By the Birthday Problem, the odds of two people have the same address in that situation would be about 1-e**(-((21*10*14)2)/(2(2256))) .  That's like 1 atom compared to the number of atoms in the Earth.  Quite small.

This answer doesn't address the main question "What would happen if two users generated the same address?", rather just evaluates the likelihood of the question being relevant, which the asker already seems to have been aware of. This answer would be a better fit on e.g. How many Bitcoin addresses are there? or Is it possible to brute force bitcoin address creation in order to steal money?.

+1 for details on how unlikely (unprofitable) brute forcing an existing address is

Actually, there are only 2^160 possible addresses. More than one private key (of which there are 2^256 possible ones) would hash to the same address.

A slight addition to the existing answers:
If B has not downloaded enough of the block chain to see A's transaction, then the situation will be as described above.  When blocks are downloaded by a wallet client, the transactions therein are checked to see whether any of them send coins to addresses held in this wallet.  If so, those coins are added to the wallet's balance.  So B will see an extra 25 BTC appear in his wallet.  As mentioned in the other answers, he'll be able to spend them as if they were his own.  So it's a race between A and B to see who spends them first - either can do it.
If B has already downloaded the relevant transactions before generating the colliding address, the situation is a little different.  As far as I know, most Bitcoin clients, when generating a fresh random address, will not rescan the block chain to see whether it contains any transactions sending coins to that address.  (As described above, the probability of this happening is infinitesimally small, so for all practical purposes, such a scan would just be a waste of time and resources.)  So although B now has a private key which can spend A's bitcoins, he will not be aware of it, unless/until he manually forces a block chain rescan (e.g. with Bitcoin Core's -rescan option), or checks his address balance using an online block explorer, or something of the kind.
Of course, barring RNG faults, this question is sort of like asking "If all 50 US state governors were simultaneously struck by lightning, how would the stock market be affected?"  It's based on such an improbable assumption that it's kind of absurd to draw any conclusion.

This answer actually answers the question the OP asked, rather than a completely different question that answerer wants to answer. But even this answer could not resist throwing in a disclaimer about why it is dumb to even ask such a question, despite the fact that the OP explicitly acknowledged the "the astonishing improbability" in the question. Such is the level of discourse in the bitcoin universe (which, everyone must chime will not exist long enough for any undesirable bitcoin thing to ever happen).

I suspect the simple answer is that Person B would be able to spend Person A's bitcoins, as he would show up in the ledger as owning them. Not dissimilar from simply giving someone your wallet.

I'm not sure that this is fundamentally different than Nicolai's answer. Nicolai just goes into more details. i.e. whether it was just a public key hash that collided or a true duplicate key pair. As well as details on the improbability of that happening.

Funny how people claiming to know about crypto can't calculate probabilities.
The address space of 2128 is not the probability or "strength" of anything (other than the probability of picking a value in the address space).
The probability of 2 people having the same bitcoin address is actually a lot higher than people may suspect by (faulty) intuition.
In fact, the likelihood of collision is related to the Birthday Problem (read about it).
As the number of people and addresses generated increases, likelihood of a collision increases close to exponentially.
Given a few million users each generating a new address per month, the likelihood of a collision is such that it could occur several times in a lifetime, especially as we're dealing with hashes (or hashes of hashes).
Make it a billion users and you have regular stories of collisions appearing.

Actually, the probability of a collision goes up as the square (i.e. quadratically) of the number of bitcoin addresses used.  Not exponentially.  Now, with that correction the probability of a collision is approximately zero in the lifetime of the solar system.

I believe that I am reading wikipedia there needs to be ~2.2×10^19 bitcoin address until there is a 50% probability that there has been at least one collision. So basically don't worry.

Birthday problem on 160 bit bitcoin addresses doesn't apply, since collisions are irrelevant. It's a second pre-image attack which has cost (2^160)/n where n is the number of targets. So if all 21 million bitcoins are already mined, you need to calculate (2^160)/(2110^6) = 710^40 = 2^135 hashes per stolen bitcoin. Which is not profitable (by far), even if you assume bitcoin is used for all of our economy.


text:
How to increase the speed of synchronisation?
What ways are there to increase the speed of synchronisation of the network, assuming the use of the main Bitcoin client?

Disk seek speed is the single most important factor. It takes you 0.5 ms to verify a transaction, but 5 ms to go to disk and save it. When buying harddrives, look at that. SSD's have short enough seek times that the exact seek time doesn't really matter.

Pass -dbcache=N as a command line option. N is a number in megabytes, and setting it higher significantly improves verification speed. The default is 450, but up to 6000 or so you can notice improvements.
This is superior to using a RAM drive.

Next to installing an SSD, and using -dbcache=N to allow Bitcoin Core to take more memory, each release of Bitcoin Core has sped up the synchronization.
The upcoming Bitcoin Core 0.14.0 release will introduce the option -assumevalid=<block> which will cause Bitcoin Core to not check signatures up to the given block. This is by default enabled and set to a block shortly before the release. It has been reported that synchronization only took about 2/3 of the time with 0.14.0 than with 0.13.2.
So, if you're still on an older version because you're reindexing or hadn't turned on Bitcoin in ages, you'll probably see huge benefits by upgrading to a newer version.

One way is to pre-calculate a subset of the data - this would be up to the author of the bitcoin client.
For example, the author of a particular client could say (to the program): "The first block is 286,427, it's hash is 0000000000000000f85c21f45015dcd2a89df40b03071234e1b3211aecaea1a4, and at that time, the COMPLETE list of balances is as follows: (include the full list here).
So long as the information is correct (and verified), and it is not on the wrong side of a fork, processing could continue from there. You would lose transactions before then.

authors: Татьяна Горобцова


text:
What's the reason for the scriptCode format for a P2WPKH? (BIP 143)
When signing a P2WPKH input, the P2WPKH locking script is converted to the pattern of a legacy P2PKH when creating the scriptCode.
From BIP 143:

For P2WPKH witness program, the scriptCode is 0x1976a914{20-byte-pubkey-hash}88ac.

What's the reason for this?
Why not just use the original P2WPKH as the scriptCode?

Maybe one of the authors of BIP143 will correct me, but I think this was done to simplify the implementation in Bitcoin Core.
If you look at the code for P2WPKH validation, you can see it actually builds and executes a P2PKH script. This is just one way you could implement P2WPKH validation, and indeed BIP141 doesn't say anything about executing a script, however this simple 5-line implementation is only possible if the scriptCode for P2WPKH is the corresponding P2PKH script.

I believe the design decision was just to treat P2WPKH as a "special short notation" for a specific single-key P2WSH. That resulted in both the current code, and the effect that the scriptCode equals this specific form. In retrospect, this was a bad decision, because it means P2WPKH spends can be malleated into equivalent P2WSH spends by third parties, which slightly reduces the feerate of the transaction even.

@PieterWuille How can it be malleated when the signature commits to the outpoint(s)?

I misremembered; it's a somewhat different setting. The issue is with hardware signing devices: a malicious software wallet could create a P2WSH signature, present it as a P2WPKH spend to the signer, which would present a higher fee to the user.


text:
What exactly are the rev00000.dat files?
I've been looking at the on disk structured generated by the standard bitcoin-qt client. It looks to me that the rev?????.dat files are reverse application for blocks? So I imagine every single spent output is added to the reverse dat, so that they can be applied directly to the unspent pool during a rewind?
What exactly is stored, and what is the format?

Indeed, you can consider transactions to be 'patches' against the set of unspent outputs (the UTXO set): they consume some outputs, and they create some new outputs. Blocks are aggregated patches.
And indeed, to be able to undo the effects of a block on the UTXO set, a record with the reverse patch is written to the rev*.dat files (corresponding to the blocks in the same blk*.dat file).
The format is custom and very compact. For every transaction input (excluding the coinbase's, which does not spend anything) it contains the script and amount of the output being spent and whether it was created by a coinbase. If the output was the last of a particular transaction being spent, the transaction version and height in the chain it was created at are also stored.

I'm going to accept this answer, but I was hoping for more detail on the format itself. It seems spectacularly compact, at roughly 18 megs~ per file that seems like far less than would be necessary to revert a lot of one meg blocks.  Unless I misunderstood and the rev patch doesn't rewind a block as a single block at a time but instead rewinds in a single patch N blocks back then forward applies the correct chain?

No, it contains one record per block, which is individually applied when rolling back.


text:
What bad things could happens if Bitcoind RPC is public?
It is always suggested to enforce authentication on Bitcoind RPC. However, in case the RPC authentication get compromised (or if it is public), what kind of bad things could happens?
First I could think of money lost
First wallets are exposed and all the private key will get exposed, and it essentially mean all your Bitcoins can get stolen
Second with setTxFee function, any transaction created using the node will either has insufficient fee or just too much fee
Apart from money lost, is there any other things that may potential happens?
For example, if I just run the node as a block tracker and monitor if any money sent into my address. Does RPC has the ability to affect them as well?

Recently ELectrum had that bug, First of all, if your wallet were not encrypted you will lose your funds and that is the most dangerous thing might happen.
If safe mode is enabled disablesafemode=1 You will avoid any danger RPC Calls.

Safe mode is a feature that disables a subset of RPC calls - mostly
  related to the wallet and sending.

If you want to make your server public, Enable safe mode and disable wallet access.


text:
Does BIP44 key derivation path protects private keys derived within same account?
I understand that with

Un-hardened parent extended public key
child private key

This will leads to compromise of parent private key and so all the private keys of siblings
BIP44 suggests the path

m / purpose' / coin_type' / account' / change / address_index

Let's say I use the path m/44'/0'/0'/0 to derive  public keys for account 0
Since The change and address_index is not hardened. And in my application I will have the extended public key at level m/44'/0'/0'/0 revealed (to derive addresses), does it means that if any of the private keys derived at the address_index level is compromised, all the private keys within the same account may also be compromised?
So with this path only avoid further compromise up to

Other accounts and
Further up to the master key?

Am I correct?

Bip44 doesn't protect anything, if the user has master root key, they could get access to all the other accounts private keys.
Your path is wrong, it should be like that:
m / purpose' / coin_type' / account' / change / address_index
What you should do is generating child master private key for the following path:
m/44'/0'/1'/
Now the user has access only to account number "1" and should generate addresses using that child master private key and the following path:
m/changeAddress/addressIndex

Thanks for point out the mistake. I will try to clarify my misconception and make my question clear. I understand hardening is helps to protect the parent private key being compromised if extended public key + child private key is compromised. However, I realize in BIP44, the change and address_index is non-hardened. That's why I am concerned that if any private key I derived from this path got compromised, does it means all the other private keys with the same account derived from the path is compromised as well.

Never, they'll need the master private key to get access to other accounts private keys.

How about within the same account? e.g. the private key for m/44'/0'/1'/0/0 get compromised, with the extended public key, does it means private keys of m/44'/0'/1'/0/n are all said to be compromised?

Not using WIF private key, they need the account master private key.

with the extended public key they may just see the addresses, but never the private keys.

Let us continue this discussion in chat.

Oh I just notice in your answer you mention generate addresses using that child master private key. In my case I want to use an extended public key to generate address (because I need to generate address on the fly). So what I will putting in my application would be the extended public key at path m/44'/0'/1'/0. Does it means the vulnerability I mentions is valid?

I have rethink the whole thing and try to summarize. Does it means that: 1) If I use a master private key to derive a public key, it is a hardened public key and I do not need to worry about compromise of parent or siblings private key if child private key is exposed. 2) If I decide to use an extended public key to derive addresses, then I am subject to the vulnerability, no matter I am using the BIP44 derivation path m/44'/0'/1'/0 or other path that ends without prime.


text:
Why use Merkle Root (and not just concatonation of all hashes?)
I am studying the bitcoin protocol and do not exactly understand why a Merkle root is used to register all the transactions. What is the point of using a tree-wise structure when you can also just take a hash over the concatenation of all transactions hashes? As far as I see, that would be less operations and therefore only more efficient?

Note: the question above is not exactly the same, but the top answer on it works perfectly as an answer to this question.

I take it you have researched Merkle Trees and understand how they work?

The Merkle tree structure is essential for section 7 (Reclaiming Disk Space) and section 8 (Simplified Payment Verification) in the whitepaper.
With simple concatenation, the only way to be convinced that a transaction exists in a block is to have access to the hashes of all transactions, which requires a lot of storage space.
With a Merkle tree you can:

Prune large parts of the history, and still know that the data left intact is a legitimate part of the block.
Convince a specific node that a transaction exists in a block, by showing the Merkle branch.

Merkle trees are also used in pooled mining, allowing the miner to construct the generation transaction independently while the pool or another node builds the bulk of the block.
Some more details are in this answer.


text:
Schnorr batch verify random number generation method
Is there a specific way that you must generate the random numbers for batch verification with Schnorr signatures?
In BIP 340 it describes this method for generating each random number:
seed = SHA256(pk1..pku || m1..mu || sig1..sigu)
a = []
for (2..u) do |i|
    a[i] = ChaCha20(seed) <- not sure if this is right or how this works, but ChaCha20 is recommended
end

But if you haven't got easy access to ChaCha20, would it be perfectly fine to use SHA-256 with an index instead:
seed = SHA256(pk1..pku || m1..mu || sig1..sigu)
for (2..u) do |i|
  a[i] = SHA256(seed || i)
end

Similarly, does the order of the data matter when creating the seed for the hash function? All of the same input data is getting hashed, just in a different order:
seed = SHA256(m1..mu || sig1..sigu || pk1..pku)
for (2..u) do |i|
  a[i] = SHA256(seed || i)
end

Basically I'm wondering if you can generate these numbers in any way you like as long as they are random and based on a hash of all the input data.