text:
Reindexing after initial download of bitcoin core
I recently downloaded bitcoin core. Took about 1+ week to download 80%. One day I checked, it got an error message. When I closed and reopened the bitcoin core, it started reindexing. Any idea how long this will take? Turned it off after a day of "calculating indexing time". Was trying to mine low to no fee transactions as I start getting into bitcoin.

2017-10-18 05:36:19 receive version message: /Satoshi:0.15.0.1/: version 70015, blocks=490420, us=71.202.75.20:62505, peer=1908
2017-10-18 05:37:32 Pre-allocating up to position 0x6000000 in blk00622.dat
2017-10-18 05:37:45 receive version message: /Satoshi:0.13.1/: version 70014, blocks=490420, us=71.202.75.20:62519, peer=1909
2017-10-18 05:39:29 Pre-allocating up to position 0x7000000 in blk00622.dat
2017-10-18 05:40:07 socket send error An existing connection was forcibly closed by the remote host.  (10054)
2017-10-18 05:40:37 receive version message: /Satoshi:0.14.2/: version 70015, blocks=490420, us=71.202.75.20:62551, peer=1910
2017-10-18 05:41:03 Pre-allocating up to position 0x8000000 in blk00622.dat
2017-10-18 05:41:44 ProcessMessages(version, 113 bytes) FAILED peer=1911

"Was trying to mine low to no fee transactions as I start getting into bitcoin.": I am not really clear what you mean by that, but I suspect that it isn't possible.

What is in debug.log?

Where is the log located? Looking at my directory, I don't see this file. Trying to add a pic of the folder but not sure how on this reply.

Actually, edited the original question to add the picture here.

The debug.log file is located in the data directory which is not the same location as the install directory. In Window,s the default is %APPDATA%/Bitcoin

Ah, it's on my external drive. Using wireless external drive due to size of the blockchain now. There appears to be an error very early in the log. Can you guys take a look? There are few other failed messages after but I think this is the right one.

Also, does the error log contain any sensitive data?

Well, I figured this one out. Running a full node on my other computer with internal hard drive is much quicker.
Believe the original problem that I had was that I downloaded the application on my internal hard drive but saved the blockchain itself on an wireless external drive. Having the external drive disconnected even once I think would cause some issue.
Given the size of the blockchain, not sure what the plan is once it reaches 1TB?


text:
What is the algorithm to create a Bitcoin mining calculator?
I want to know how I can calculate the profit of bitcoin mining in bitcoins. I am interested in the formula that is used by calculators. For example lets say I have 700mh/s hashrate, current difficulty is 908350862,437. The calculator says I will mine 0.00038755 BTC per day. What algorithm is used to calculate these figures?

possible duplicate of How much Bitcoin will I mine right now with hardware X?

Check out this answer: http://bitcoin.stackexchange.com/a/135/5406

I want to give an extended answer as I found the pre-existing here or elsewhere in need of a bit more clarification, specifically for newcomers.
Formula
In order to calculate the approximated total bitcoin earnings value per month from a mining operation (not taking into consideration mining costs [electricity, hardware maintenance etc...]) the following formula can be used:
H = Hashrate (hashes / second)
D = Difficulty (Reference for values below)
B = Reward per Block (Reference for value below)
N = Number of days per month (default = 30)
S = Number of seconds per day (S = 60 * 60 * 24 = 86400)

Example
H = 21,990,232,555,520 h/s (~= 20TH/s)
D = 47,427,554,950.648
B = 25
N = 30
S = 86400

The total number of bitcoins earned per month as per the defined variables above will be ~6.995. Dismiss N from the numerator and you'll get the daily value.
Python
A quick python statement as an example implementation (you can type this directly in the terminal)
$ python -c "print (30*21990232555520*25*86400)/(47427554950.648*2**32)"
$ 6.99542703277

Some References:
Difficulty (Current Value | Wiki)
Reward per Block (Current and Future values)

The average time to find a block can be approximated by calculating:
time = difficulty * 2**32 / hashrate

Where difficulty is the current difficulty, hashrate is the number of hashes your miner calculates per second, and time is the average in seconds between the blocks you find.
Mining is a game of variance and luck. There are times that you could get lucky and generate more blocks than predicted and there will surely be dry spells when blocks do not occur. This can be seen by watching large mining pools as they solve blocks, blocks are not created on a set schedule by any means the network adjusts such that one block should be generated every 10 minutes, but we know that this will need to be adjusted regularly based on new miners coming online all the time. The network hashrate and thus the difficulty will be regularly changing.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Bitcoin - BitGO - Save all wallet objects into Array of Objects
My Setup:
With the help of Mongoose module we handle all Mongo Database operations.
For each new user a wallet is generated and stored inside a Mongo collection.
METHOD:
If we use User.find({}, function(err, docs) {. We can get each user object.
User.find({}, function(err, docs) {
 if (err) {console.log('Could not find Wallets in DB');} else {
   docs.forEach(function(address) {
     console.log(address);
   });
 }
});

// console.log(address); <== Result:

{ tokens: [],
  profile: { gender: '', location: '', website: '', picture: '' },
  __v: 0,
  balance: 0,
  playerWallet: '2N3qPHp7bNaL4hiztrzZCzCywryKUzb5aLE',
  password: '$2a$10$JRs/hyiStyhcZzHBwFnhmulKX0vTqlNMTGdxJjDdCSD0Bufe7K6wC',
  email: '111111111111111111111@11111111111111111.com',
  username: '111111111111111111111',
  createdAt: Mon May 02 2016 01:38:11 GMT+0300 (EEST),
  updatedAt: Mon May 02 2016 01:38:17 GMT+0300 (EEST),
  _id: 57268553ea2d6bb50c9d4069
}
2N3qPHp7bNaL4hiztrzZCzCywryKUzb5aLE }
{ tokens: [],
  profile: { gender: '', location: '', website: '', picture: '' },
  __v: 0,
  balance: 0,
  playerWallet: '2N2oG4nBJZU19Aks3MLvHqDKMXePmHhbEjC',
  password: '$2a$10$cCo5gQgqcRpQW9Dl/Q.48uJEQCaOpJOLY8LVpQxoQFZ6nUqQD8lEC',
  email: '2222222222222222222222222@222222222222222222.com',
  username: '2222222222222222222222222',
  createdAt: Mon May 02 2016 01:38:38 GMT+0300 (EEST),
  updatedAt: Mon May 02 2016 01:38:44 GMT+0300 (EEST),
  _id: 5726856eea2d6bb50c9d406a
}
2N2oG4nBJZU19Aks3MLvHqDKMXePmHhbEjC
{ tokens: [],
  profile: { gender: '', location: '', website: '', picture: '' },
  __v: 0,
  balance: 0,
  playerWallet: '2My1FGBG6erNXfzeGXg5DXnLk2PWxhpk9SL',
  password: '$2a$10$tL96HqPIAi6q8XnI3z4Bc.dX88fTYQXsTt.UFWJQn4k6ZdDFOZLe2',
  email: '33333333333333333333333@33333333333333333333333.com',
  username: '33333333333333333333333',
  createdAt: Mon May 02 2016 01:39:03 GMT+0300 (EEST),
  updatedAt: Mon May 02 2016 01:39:09 GMT+0300 (EEST),
  _id: 57268587ea2d6bb50c9d406b
}

HERE LIES THE PROBLEM:
Because i use a for loop to iterate all wallet addresses, I get 3 different calls and always get one object at a time. I never get more than 1 User object stored into my .JSON file.
var _24HourGame = ".24HourGame.json"; // Save Path
var user = 'Super@User.com';
var loginPassword = 'SuperSecretPassword';
var otp = '0000000';

var bitgo = new BitGoJS.BitGo();

var listWallets = function() {

User.find({}, function(err, docs) {
if (err) {console.log('Could not find Wallets in DB');} else {
  docs.forEach(function(address) {
  console.log(address);
  var allWallets = [];
  allWallets.push(address.playerWallet);

           console.log(allWallets);
  //   ALL ADDRESSEs EACH IN ITS OWN ARRAY
  //[ '2N3qPHp7bNaL4hiztrzZCzCywryKUzb5aLE' ]
  //[ '2N2oG4nBJZU19Aks3MLvHqDKMXePmHhbEjC' ]
  //[ '2My1FGBG6erNXfzeGXg5DXnLk2PWxhpk9SL' ]

  // Don't know how to use a better method that calls back recursively and sums all 3 obj received into an array....
  for( var iteration in allWallets){

        console.log(allWallets[iteration]);
  //  '2N3qPHp7bNaL4hiztrzZCzCywryKUzb5aLE'
  //  '2N2oG4nBJZU19Aks3MLvHqDKMXePmHhbEjC'
  //  '2My1FGBG6erNXfzeGXg5DXnLk2PWxhpk9SL'

  // How to call a Recursive Callbacks or a Promise for bitgo.wallets().gest({}) while iterating multiple address?
      bitgo.wallets().get({ "id": allWallets[iteration] }, function callback(err, wallet) {
        if (err) { throw err; }
        console.log( address._id + ' ' + address.playerWallet + ' ' + 'BlockChain_Balance: ' + wallet.balance() / 1e8.toFixed(4) + ' DB_Balance: ' );

        jsonfile.writeFileSync(_24HourGame, wallet, {spaces: 3});

            });
          }

      });
    }
  });

 };

 // Authenticate first
 bitgo.authenticate({ username: user, password: loginPassword, otp: otp }, function(err, result) {
   if (err) { console.dir(err); throw new Error("Could not authenticate!"); }
   console.log("Unlocking account.." );
   bitgo.unlock({ otp: otp }, function(err) {
     if (err) { console.dir(err); throw new Error("Could not unlock!"); }
     listWallets();
     });
 });

When i save result to JSON I only get one object at a time:
     {
        "_id": "5726858f42929c5c066825b6cc7f9d96",
        "id": "2My1FGBG6erNXfzeGXg5DXnLk2PWxhpk9SL",
        "label": "123labeling",
        "isActive": true,
        "type": "safehd",
        "freeze": {},
        "adminCount": 1,
        "disableTransactionNotifications": false,
        "private": {
           "keychains": [
              {
                 "xpub": "xpub661MyMwAqRbcG8zA12U9YnJU3fSxoCq857uMcFon6ViZdYMSeXbeVUGtGkBCasmE3SGdJVRr4ui1V8TzsmM7K6eKikSi9gZQiJXoVumGTnU",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "031ec65814f7e46e274a78671f6ed2b945c386e378160b19613ce8ba555e793635",
                    "chainCode": "a046f6f6ea9fd06974b333e87a483c9c3cbdeb41c0e56c8f21a6954f9f95e387",
                    "depth": 0,
                    "index": 0,
                    "parentFingerprint": 0
                 }
              },
              {
                 "xpub": "xpub6GiRC55CRvMEnS2CtwqrMNKDnpsogbX1tukjM2xWyW1PttVLSEVgDiJq7bWh6esAQWeQ9oGNR56RHqwzKLCzzneM6DGq91v8sA9DAJbUbtm",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "027f30812a8f33ffeb44515d0fe9cc99041c2369f407ecc537d492d34da48c29d2",
                    "chainCode": "4f493067187019f78a773e3e559c699b98c0e82a53cbf76cf9ba5db8103062a0",
                    "depth": 5,
                    "index": 58402,
                    "parentFingerprint": 2966462100
                 }
              },
              {
                 "xpub": "xpub661MyMwAqRbcF6cBQfmTER34dwShTVu6x2h24nxhHxMX3jAHCP9DJYuLcHABvaMeaBcSMdcte8mhAMnCcBCypK5iGe6H33aUH8JWbstSKP9",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "02ec33597217f00f1ad5689b0abbfd01d850c921d4560d1333f52f9f70eea2aafd",
                    "chainCode": "37b746ccdf38587d6a943f8007ef9e3012f52c73079f53c094f446cd43f9bb4f",
                    "depth": 0,
                    "index": 0,
                    "parentFingerprint": 0
                 }
              }
           ]
        },
        "canSendInstant": true,
        "permissions": "admin,spend,view",
        "admin": {
           "users": [
              {
                 "user": "56f9e3b2cd8f2e5906530948fe1af2d2",
                 "permissions": "admin,spend,view"
              }
           ]
        },
        "tags": [],
        "approvalsRequired": 1,
        "spendingAccount": true,
        "pendingApprovals": [],
        "balance": 30000000,
        "instantBalance": 0,
        "spendableConfirmedBalance": 0,
        "confirmedBalance": 0,
        "spendableBalance": 0,
        "sent": 0,
        "received": 30000000,
        "unconfirmedSends": 0,
        "unconfirmedReceives": 30000000
     }

I want to sum all objects into one array of objects...:
     [
     {
        "_id": "5726857642929c5c066825819a872925",
        "id": "2N2oG4nBJZU19Aks3MLvHqDKMXePmHhbEjC",
        "label": "123labeling",
        "isActive": true,
        "type": "safehd",
        "freeze": {},
        "adminCount": 1,
        "disableTransactionNotifications": false,
        "private": {
           "keychains": [
              {
                 "xpub": "xpub661MyMwAqRbcH2AHKRsNyEv4euymBzY8P52NzeodZBBvG8zPtpLeEJmjD8xevZRrDLCyCr4TjmavB9TuAw69bDH3TkKNPyEQ2UBVrGg8uVq",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "038c20add130bf5d79fc1debc658e8866f5c69962efc15350941d18cc3b88c025f",
                    "chainCode": "f8e61f519f8e5363b24fe19912a25c90395a9189f59517d5c888c2b2eaefe4ba",
                    "depth": 0,
                    "index": 0,
                    "parentFingerprint": 0
                 }
              },
              {
                 "xpub": "xpub6GiRC55CRvMEkcMvj2xtGSMhQzAys7EMV4gvqporNz5kSkzuqt3x25cnpvYWyR24y38WYij7eLr3xSQsBo7TBHWR27b5QZTrj5sVq5MMBVa",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "037e26f5594705a05a49e63dbd7eb98e0fb56d0f26b584c8e4fcdb15efd0239d62",
                    "chainCode": "984f19d3ae866a97a1b3a0bad1abf2a9aa77dbf50465180dee0cf8a8b73089c8",
                    "depth": 5,
                    "index": 58401,
                    "parentFingerprint": 2966462100
                 }
              },
              {
                 "xpub": "xpub661MyMwAqRbcGdVpPbFpvi37Ziteqj4JY3uEoV6XijMAk62Pgs2jK3bRPPVjKg4AtwbZ6xiMMHD56oHCdGQwqtWNZgwSWefw9edex9aJ9KC",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "03b323e0b6ebb0cbb845be13c5a090c4648de18c090382e899e66c389c0f6451fc",
                    "chainCode": "d1a6e364a4b44764190148e725a8c584583743d31df2d06439b70016e19799b5",
                    "depth": 0,
                    "index": 0,
                    "parentFingerprint": 0
                 }
              }
           ]
        },
        "canSendInstant": true,
        "permissions": "admin,spend,view",
        "admin": {
           "users": [
              {
                 "user": "56f9e3b2cd8f2e5906530948fe1af2d2",
                 "permissions": "admin,spend,view"
              }
           ]
        },
        "tags": [],
        "approvalsRequired": 1,
        "spendingAccount": true,
        "pendingApprovals": [],
        "balance": 533000000,
        "instantBalance": 0,
        "spendableConfirmedBalance": 0,
        "confirmedBalance": 0,
        "spendableBalance": 0,
        "sent": 0,
        "received": 533000000,
        "unconfirmedSends": 0,
        "unconfirmedReceives": 533000000
     },
     {
        "_id": "5726858f42929c5c066825b6cc7f9d96",
        "id": "2My1FGBG6erNXfzeGXg5DXnLk2PWxhpk9SL",
        "label": "123labeling",
        "isActive": true,
        "type": "safehd",
        "freeze": {},
        "adminCount": 1,
        "disableTransactionNotifications": false,
        "private": {
           "keychains": [
              {
                 "xpub": "xpub661MyMwAqRbcG8zA12U9YnJU3fSxoCq857uMcFon6ViZdYMSeXbeVUGtGkBCasmE3SGdJVRr4ui1V8TzsmM7K6eKikSi9gZQiJXoVumGTnU",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "031ec65814f7e46e274a78671f6ed2b945c386e378160b19613ce8ba555e793635",
                    "chainCode": "a046f6f6ea9fd06974b333e87a483c9c3cbdeb41c0e56c8f21a6954f9f95e387",
                    "depth": 0,
                    "index": 0,
                    "parentFingerprint": 0
                 }
              },
              {
                 "xpub": "xpub6GiRC55CRvMEnS2CtwqrMNKDnpsogbX1tukjM2xWyW1PttVLSEVgDiJq7bWh6esAQWeQ9oGNR56RHqwzKLCzzneM6DGq91v8sA9DAJbUbtm",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "027f30812a8f33ffeb44515d0fe9cc99041c2369f407ecc537d492d34da48c29d2",
                    "chainCode": "4f493067187019f78a773e3e559c699b98c0e82a53cbf76cf9ba5db8103062a0",
                    "depth": 5,
                    "index": 58402,
                    "parentFingerprint": 2966462100
                 }
              },
              {
                 "xpub": "xpub661MyMwAqRbcF6cBQfmTER34dwShTVu6x2h24nxhHxMX3jAHCP9DJYuLcHABvaMeaBcSMdcte8mhAMnCcBCypK5iGe6H33aUH8JWbstSKP9",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "02ec33597217f00f1ad5689b0abbfd01d850c921d4560d1333f52f9f70eea2aafd",
                    "chainCode": "37b746ccdf38587d6a943f8007ef9e3012f52c73079f53c094f446cd43f9bb4f",
                    "depth": 0,
                    "index": 0,
                    "parentFingerprint": 0
                 }
              }
           ]
        },
        "canSendInstant": true,
        "permissions": "admin,spend,view",
        "admin": {
           "users": [
              {
                 "user": "56f9e3b2cd8f2e5906530948fe1af2d2",
                 "permissions": "admin,spend,view"
              }
           ]
        },
        "tags": [],
        "approvalsRequired": 1,
        "spendingAccount": true,
        "pendingApprovals": [],
        "balance": 30000000,
        "instantBalance": 0,
        "spendableConfirmedBalance": 0,
        "confirmedBalance": 0,
        "spendableBalance": 0,
        "sent": 0,
        "received": 30000000,
        "unconfirmedSends": 0,
        "unconfirmedReceives": 30000000
     },
     {
        "_id": "5726855c0cb29c5b060918484524d6d9",
        "id": "2N3qPHp7bNaL4hiztrzZCzCywryKUzb5aLE",
        "label": "123labeling",
        "isActive": true,
        "type": "safehd",
        "freeze": {},
        "adminCount": 1,
        "disableTransactionNotifications": false,
        "private": {
           "keychains": [
              {
                 "xpub": "xpub661MyMwAqRbcFcyTCTTNVmYcAmPJ2yRJanFPv9To7n1wk5cZmgUpzY16pAz9m6XriziHnk6c4Y2Hov9HJSmWPSHdmSLn9KNBLMr5Qi29HGN",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "032536e93aa4cd5feb0706d118b6ecf695d26544c00f92a2fb36ef9208ee216821",
                    "chainCode": "6c4dbaed05b8a09d2d64c3688e3fb53f12e62ef64b70ad808a700830e2063bb2",
                    "depth": 0,
                    "index": 0,
                    "parentFingerprint": 0
                 }
              },
              {
                 "xpub": "xpub6GiRC55CRvMEhQVtxhHwwpmG7NVyjT1SUuyYXfafsRDW4XoYRwzPp6Af5vY83hLN8vaFCzk9FdtUqdeXoQbS7fXNJU3eFYPnU1bfkjimsT4",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "035d12105eab700c637ca9fbbea066beca88c6603f90f98c07a9e5b84713d6ff20",
                    "chainCode": "5670e6ede3379f33cc635981c7fbe9255c3f69a5e4868e5491e42bf3d5fb380a",
                    "depth": 5,
                    "index": 58400,
                    "parentFingerprint": 2966462100
                 }
              },
              {
                 "xpub": "xpub661MyMwAqRbcEZBgr8PUbKfGBaQ98GFp2wGdR6vDwTAzruGenMnn3PqDo9MMmzSsF8dFWgUfUUV1Yor2ErVrohBNHj9Ug81ih7VAohfnEQu",
                 "path": "/0/0",
                 "params": {
                    "pubKey": "024f7b2f85a3e0b859c4bdb975f9953973de4806300b2148afec66fb610af380a7",
                    "chainCode": "014ce166076a6045fb5a88fa4a51052d13c5449a4367d030e0f6bc058a9d0191",
                    "depth": 0,
                    "index": 0,
                    "parentFingerprint": 0
                 }
              }
           ]
        },
        "canSendInstant": true,
        "permissions": "admin,spend,view",
        "admin": {
           "users": [
              {
                 "user": "56f9e3b2cd8f2e5906530948fe1af2d2",
                 "permissions": "admin,spend,view"
              }
           ]
        },
        "tags": [],
        "approvalsRequired": 1,
        "spendingAccount": true,
        "pendingApprovals": [],
        "balance": 330000000,
        "instantBalance": 0,
        "spendableConfirmedBalance": 0,
        "confirmedBalance": 0,
        "spendableBalance": 0,
        "sent": 0,
        "received": 330000000,
        "unconfirmedSends": 0,
        "unconfirmedReceives": 330000000
     }
     ]

Hoping that someone has done something like this and can easily spot the error.
Cheers !

Silly me, didn't push wallet results...
Here is the working code:
  var bitgo = new BitGoJS.BitGo();

  var listWallets = function() {

  User.find({}, function(err, docs) {
  if (err) {console.log('Could not find Wallets in DB');} else {
    var allWallets = [];
    var allTXS = [];

  docs.forEach(function(address) {
    // console.log(address);
    allWallets.push(address.playerWallet);
  });

             console.log(allWallets);
    //   ALL ADDRESSEs EACH IN ITS OWN ARRAY
    //[ '2N3qPHp7bNaL4hiztrzZCzCywryKUzb5aLE' ]
    //[ '2N2oG4nBJZU19Aks3MLvHqDKMXePmHhbEjC' ]
    //[ '2My1FGBG6erNXfzeGXg5DXnLk2PWxhpk9SL' ]

    // Don't know how to use a better method that calls back recursively and sums all 3 obj received into an array....
    for( var iteration in allWallets){

          // console.log(allWallets[iteration]);
    //  '2N3qPHp7bNaL4hiztrzZCzCywryKUzb5aLE'
    //  '2N2oG4nBJZU19Aks3MLvHqDKMXePmHhbEjC'
    //  '2My1FGBG6erNXfzeGXg5DXnLk2PWxhpk9SL'

    // How to call a Recursive Callbacks or a Promise for bitgo.wallets().gest({}) while iterating multiple address?
        bitgo.wallets().get({ "id": allWallets[iteration] }, function callback(err, wallet) {
          if (err) { throw err; }
          // console.log( address._id + ' ' + address.playerWallet + ' ' + 'BlockChain_Balance: ' + wallet.balance() / 1e8.toFixed(4) + ' DB_Balance: ' );
          allTXS.push(wallet);

          jsonfile.writeFileSync(_24HourGame, allTXS, {spaces: 3});
              });
            }

      }
    });

   };

   // Authenticate first
   bitgo.authenticate({ username: user, password: loginPassword, otp: otp }, function(err, result) {
     if (err) { console.dir(err); throw new Error("Could not authenticate!"); }
     console.log("Unlocking account.." );
     bitgo.unlock({ otp: otp }, function(err) {
       if (err) { console.dir(err); throw new Error("Could not unlock!"); }
       listWallets();
       });
   });

Thank you for your help !

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How does each miner know when to create a block?
I have been trying to understand the protocol and reading related documents, there is a point where i get stuck.
As far as i understand (correct me if not), when a transaction occurs, it is broadcast to the network and some miners receive it. When a certain number of transactions is reached, they are packed in a block and hash race begins.
What i fail to understand is; in this scheme, don't all the miners have to be in perfect memory coherence and time sync, so that they know when a block is to be sealed and start to iterate for hashes? Or do they not need to mine and hash the same global block, but have separate blocks which are later validated by other miners and interblock transaction collisions don't matter?
I know i misunderstand a very basic point in the protocol, but i can't figure what.

Related: When does a miner decide to stop collecting transactions and start calculating hashes to try to win?

A miner begins creating a new block as soon as it receives a valid block from another miner.  It then begins to construct and hash the next block on top of the block it just received.
You were correct when you thought that each miner has a separate block from the rest.  Don't look at it as if miners are all cooperating to find a hash for the same block, but rather are competing against each other to make their own block the next in the chain.  If they find a valid block, they broadcast it, and other miners give up their efforts, and instead focus their hashing power on the next block.

Alright... So for example transactions a, b, c and d have been submitted to miner M1; transactions c, d, e, f and g have been submitted to miner M2. Both M1 and M2 are on head BC1. M1 finds a legit hash and announces the new blockchain BC2. Now what about M2? Current BC2 does not contain e, f and g. M2 might start a new block candidate B3, but this will cause transactions c and d to be duplicates. M2's ex-block B2' cannot be completely dropped either, that would cause e, f and g to vanish.

@corsel: M2 generates a new block header B4 that contains transactions e,f,g only, and starts looking for a matching nonce for it.

Ah, so the blockchain is always in sync and are to be validated by majority of network, whereas block candidates are local and at the miners' will completely... If a malicious blockchain appears, it will be tested and rejected by other miners, or trying to manipulate history by hashing in deep levels fails, as in less time, a new chain will be announced.

Part of your misconception is this: "when a certain number of transactions are reached".  Mining doesn't need any particular number of transactions; it's legal to have a block with no transactions at all (except for the "coinbase" block reward transaction).  The "hash race" is always going on; it doesn't start and stop.  When a miner gets a new transaction, they just include it with the  transactions they already have, generate a new block header, and continue hashing on the new header.  There is no requirement that all miners be working with the same set of transactions.


text:
How to configure a testnet node to be pruned?
I have a testnet VM node running. As I have only limited resources, I did set pruned=1000 in the bitcoin.conf file, so the blockchain is pruned down to 1GB. But when I run bitcoind and it downloads the blockchain, I get the error, that no more space is available after downloading 80% or so.
So my question: How do I make sure that the pruned mode is actually enforced?
Thanks!

The bitcoin.conf option must be prune=1000, not pruned=1000. If you specified the latter, it was probably ignored.
You could reduce the number up to 550 (MiB), bitcoind won't start if you choose a lower number.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What is the difference between secp256k1_ecmult_gen and secp256k1_ecmult?
libsecp256k1 has two implementations, secp256k1_ecmult in ecmult.h and secp256k1_ecmult_gen in ecmult_gen.h, to multiply the points of an elliptic curve.
secp256k1_ecmult_gen supports simple multiplication such as a*G and secp256k1_ecmult supports multiplication involving addition such as a*P + bG. However, secp256k1_ecmult can be used for simple multiplication by setting b=0, actually, it is used as such in eckey_impl.h#secp256k1_eckey_pubkey_tweak_mul.
How should these two functions be used properly? Is there any difference in the performance?

The files ecmult.h and ecmult_gen.h can be found here: https://github.com/bitcoin-core/secp256k1/tree/master/src . The implementations are found here: https://github.com/bitcoin-core/secp256k1/blob/master/src/ecmult_impl.h ; https://github.com/bitcoin-core/secp256k1/blob/master/src/ecmult_gen_impl.h

How should these two functions be used properly?

A simple answer is "not at all". Those functions are not exposed in the public API of libsecp256k1, and that's the reason why they don't have user-targeted documentation. Instead, they're used as internal subroutines, mostly for the implementation of ECDSA and Schnorr signatures.
Be advised that cryptography should be implemented by experts. It's very easy to shoot yourself in the foot if you don't know what you're doing (and in fact, even if you know what you're doing).
Having said this, here's a high-level overview:

secp256k1_ecmult_gen gets as input a scalar a and computes the multiplication aG with the standard generator G. This function is constant-time in the scalar a, i.e., it ensures that the timing of the computation does not leak information about a. That's why it is used whenever the scalar is a secret, e.g., for key generation when computing the public key xG from the secret key x, or for signing when computing the nonce rG from the secret scalar r. In general, the requirement for a constant-time implementation makes makes code slower (for example, you can't have early returns).
secp256k1_ecmult gets as input two scalars a, b and a point P, and it computes aP + bG. This function does not ensure constant-time computations. This computation is exactly what is necessary for signature verification, and the function is suitable because no secret values are involved in verification.
There's also secp256k1_ecmult_const. It gets as input a scalar a and point P, and it computes aP in constant-time. (So in terms of functionality and security, it is a more general variant of secp256k1_ecmult_gen that works with arbitrary points P. As such it is less efficient because it cannot rely on some precomputation techniques that require to know P upfront.) This is used as a subroutine in the ECDH (Diffie-Hellman) module for computing the shared secret in the key exchange.


text:
Creating a raw Segwit transaction from scratch and getting Signature must be zero for failed CHECK(MULTI)SIG operation
I am trying to create a Segwit transaction on TESTNET from scratch while referencing an online guide:
https://medium.com/coinmonks/creating-and-signing-a-segwit-transaction-from-scratch-ec98577b526a
My BIP-39 mnemonics code:
differ toast alert candy orbit raccoon wagon hour thunder kiwi home cigar

My TESTNET BIP-32 master public key derived Segwit Bech32 address with derivation path 'm':
tb1qzjgctgz9t98dmkwp4z26whg6tl4w9s6rl6hwwj

The address profile can be viewed from here.
Just for simplicity of debugging, the derivation 'm' will be used only.
For more context the Master Public Key compressed format:
03D7DE86033E1C815BA0E3589938B7AC906B728F1F2C771542E203E32B80A3385A

For more context the Master Public Key uncompressed format:
04D7DE86033E1C815BA0E3589938B7AC906B728F1F2C771542E203E32B80A3385AE1DEED79187355D35FDABAF74DFD186A8B8C9FE1D6357AA46750A47B86ED16CF

The raw BIP-32 Seed:
1CD4561EF82D35F9EE2E95ED63B68BB91ADA8967E1D98FA4080637A4CC310BE99DD285F82F45EE004ADC408F8B93C2A829A5B811104496A6FB0301E062CBC813

The raw Master Private Key used for the digital signing later:
13D84CAC526896CEF2594D71621D2AEDC90F9DAEA88A1751AD99E35868E0021B

I would like to send myself (tb1qzjgctgz9t98dmkwp4z26whg6tl4w9s6rl6hwwj) some funds (0x5503000000000000 / 853 satoshis) using my own address (tb1qzjgctgz9t98dmkwp4z26whg6tl4w9s6rl6hwwj).
So, I begin to create the Segwit inputs that would then be double-SHA256 hashed then signed.
It follows the BIP143 defined steps:
Double SHA256 of the serialization of:
 1.  nVersion of the transaction (4-byte little endian)
 2.  hashPrevouts (32-byte hash)
 3.  hashSequence (32-byte hash)
 4.  outpoint (32-byte hash + 4-byte little endian)
 5.  scriptCode of the input (serialized as scripts inside CTxOuts)
 6.  value of the output spent by this input (8-byte little endian)
 7.  nSequence of the input (4-byte little endian)
 8.  hashOutputs (32-byte hash)
 9.  nLocktime of the transaction (4-byte little endian)
 10. sighash type of the signature (4-byte little endian)

// 1.  nVersion of the transaction (4-byte little endian)
<<< 01000000

// 2.  hashPrevouts (32-byte hash)
>>> printf "4E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A0 00000000" | xxd -p -r | sha256sum -b | xxd -p -r | sha256sum -b

<<< 04dc164c0a705f5d2d57fd42e6896a94394e6966ed0b116a64adfaa9310b848f

//  3.  hashSequence (32-byte hash)

>>> printf "ffffffff" | xxd -p -r | sha256sum | xxd -p -r | sha256sum

<<< 3bb13029ce7b1f559ef5e747fcac439f1455a2ec7c5f09b72290795e70665044

//  4.  outpoint (32-byte hash + 4-byte little endian)

<<< 4E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A0 00000000

//  5.  scriptCode of the input (serialized as scripts inside CTxOuts)

<<< 1976a914149185A045594EDDD9C1A895A75D1A5FEAE2C34388ac

//  6.  value of the output spent by this input (8-byte little endian)

<<< e803000000000000

//  7.  nSequence of the input (4-byte little endian)

<<< ffffffff

// 8.  hashOutputs (32-byte hash)

>>> printf "e803000000000000 16 0014149185A045594EDDD9C1A895A75D1A5FEAE2C343" | xxd -p -r | sha256sum -b | xxd -p -r | sha256sum -b

<<< 7c007a193409e64199438db0ba4349809fbcc42c17e7167cbc06deec4cc0998f

// 9.  nLocktime of the transaction (4-byte little endian)
<<< 00000000

// 10. sighash type of the signature (4-byte little endian)
<<< 01000000

// Hashing with double Sha256

>>> printf 0100000004dc164c0a705f5d2d57fd42e6896a94394e6966ed0b116a64adfaa9310b848f3bb13029ce7b1f559ef5e747fcac439f1455a2ec7c5f09b72290795e706650444E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A0000000001976a914149185A045594EDDD9C1A895A75D1A5FEAE2C34388ace803000000000000ffffffff7c007a193409e64199438db0ba4349809fbcc42c17e7167cbc06deec4cc0998f0000000001000000 | xxd -p -r | sha256sum -b | xxd -p -r | sha256sum -b

<<< c2636fae9487354f47e4e92458a7e856f717533c6d01e1c5fda5b06c4a1976a8

The encoded digital signature result:
30450221008B61503427B2292F781405712DF878B207D8B89FC86D60B1A68D479CA6C6FF2302203BB1F7F5F7E2B9050BA481310C997F6F078F5ED25564AEC85277EB56BA98FD76

Finally, formatted into a raw transaction below:
010000000001014E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A00000000000FFFFFFFF015503000000000000160014149185A045594EDDD9C1A895A75D1A5FEAE2C343024830450221008B61503427B2292F781405712DF878B207D8B89FC86D60B1A68D479CA6C6FF2302203BB1F7F5F7E2B9050BA481310C997F6F078F5ED25564AEC85277EB56BA98FD76012103D7DE86033E1C815BA0E3589938B7AC906B728F1F2C771542E203E32B80A3385A00000000

The broadcast results throws the following error:
sendrawtransaction RPC error: {"code":-26,"message":"non-mandatory-script-verify-flag (Signature must be zero for failed CHECK(MULTI)SIG operation)"}

I have checked the digital signatures many times and ran validation on the signatures and public key with assumption that all the above addresses and scripts are valid.
How do I fix the failed broadcast ? What are the causes of the failed broadcast ?

OK, finally I got some things to work and there are quite some changes I had to make but it seems to have work.
I need to retrace my steps all over again later on tomorrow as its late now.
Here's the TESTNET transaction txid: e559b79981b5bcde2df389f37e2b72e5d6a132748c6dab499d10f5186568533f
Also, while trying every means to mint this transaction, it raises more questions than answers at the same time. There's quite a few "waht-ifs corner cases" I have in mind.
Firstly, I need to create a PEM file containing the private key so that I can use OpenSSL to sign the transaction later on. This is an optional step.
printf "302e0201010420 13D84CAC526896CEF2594D71621D2AEDC90F9DAEA88A1751AD99E35868E0021B a00706052b8104000a" | xxd -p -r > priv1.hex

openssl ec -inform d < priv1.hex > priv1.pem

The private key in the original post is added into a PEM format file. If you are not using OpenSSL to sign, you can omit this step.
Next is the usual layout of 10 steps.
// 1.  nVersion of the transaction (4-byte little endian)
<<< 01000000

// 2.  hashPrevouts (32-byte hash)
>>> printf "4E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A0 00000000" | xxd -p -r | sha256sum -b | xxd -p -r | sha256sum -b

<<< 04dc164c0a705f5d2d57fd42e6896a94394e6966ed0b116a64adfaa9310b848f

//  3.  hashSequence (32-byte hash)

>>> printf "FDFFFFFF" | xxd -p -r | sha256sum | xxd -p -r | sha256sum

<<< caf35e5224de16efa3ccaf41070f6e7b9432b6f79551e629fca9d1c03b43bc52

//  4.  outpoint (32-byte hash + 4-byte little endian)

<<< 4E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A0 00000000

//  5.  scriptCode of the input (serialized as scripts inside CTxOuts)

<<< 1976A914149185A045594EDDD9C1A895A75D1A5FEAE2C34388AC

//  6.  value of the output spent by this input (8-byte little endian)

<<< e803000000000000

//  7.  nSequence of the input (4-byte little endian)

<<< FDFFFFFF

// 8.  hashOutputs (32-byte hash)

>>> printf "01000000000000001976A914149185A045594EDDD9C1A895A75D1A5FEAE2C34388AC54030000000000001976A914149185A045594EDDD9C1A895A75D1A5FEAE2C34388AC" | xxd -p -r | sha256sum -b | xxd -p -r | sha256sum -b

<<< 91f7bd32e352f7cc0cb49cbdf9f7d0f51f27a4da527c094b2ab85c2e2e0ae47f

// 9.  nLocktime of the transaction (4-byte little endian)
<<< 00000000

// 10. sighash type of the signature (4-byte little endian)
<<< 01000000

Next is the serialization of the hash pre-image by concatenating all the steps results above:
0100000004dc164c0a705f5d2d57fd42e6896a94394e6966ed0b116a64adfaa9310b848fcaf35e5224de16efa3ccaf41070f6e7b9432b6f79551e629fca9d1c03b43bc524E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A0000000001976A914149185A045594EDDD9C1A895A75D1A5FEAE2C34388ACe803000000000000FDFFFFFF91f7bd32e352f7cc0cb49cbdf9f7d0f51f27a4da527c094b2ab85c2e2e0ae47f0000000001000000

Next is the double hashing of the hash pre-image:
// Double hashing and storing double hashed result

>>> printf 0100000004dc164c0a705f5d2d57fd42e6896a94394e6966ed0b116a64adfaa9310b848fcaf35e5224de16efa3ccaf41070f6e7b9432b6f79551e629fca9d1c03b43bc524E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A0000000001976A914149185A045594EDDD9C1A895A75D1A5FEAE2C34388ACe803000000000000FDFFFFFF91f7bd32e352f7cc0cb49cbdf9f7d0f51f27a4da527c094b2ab85c2e2e0ae47f0000000001000000 | xxd -p -r | sha256sum -b | xxd -p -r | sha256sum -b | xxd -p -r > msg-segwit

Next is using the PEM private key to sign in OpenSSL with a SHA-256 digest:
// Signing

>>> openssl pkeyutl -inkey priv1.pem -sign -in msg-segwit -pkeyopt digest:sha256 | xxd -p -c 256

<<< 30450221008f56dd07c3637ba6bc00e4e8e05bd2e9c3d6a5c073201df8aaf52e9847d0e01c0220752381278c83287c975a147b20daef443f68d85c84650ab2d061375b6f6ef739

Append your sighash (SIGHASH_ALL - 0x01) to the back of the signature:
<<< 30450221008f56dd07c3637ba6bc00e4e8e05bd2e9c3d6a5c073201df8aaf52e9847d0e01c0220752381278c83287c975a147b20daef443f68d85c84650ab2d061375b6f6ef739 01

Finally, serialize and merge all the transaction data with the digital signature into a single transaction:
// Serialization

01000000 // version
0001 // witness

01 // 1 input

4E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A0 00000000 // outpoint with output index
00 // 0 bytes script
FDFFFFFF // sequence

02 // 2 outputs

0100000000000000 // output amount
1976A914149185A045594EDDD9C1A895A75D1A5FEAE2C34388AC // output script
5403000000000000 // output amount
1976A914149185A045594EDDD9C1A895A75D1A5FEAE2C34388AC // output script

02 // First input has witness script of 2 elements

48 // signature length
30450221008f56dd07c3637ba6bc00e4e8e05bd2e9c3d6a5c073201df8aaf52e9847d0e01c0220752381278c83287c975a147b20daef443f68d85c84650ab2d061375b6f6ef739 01 // signature with sighash (01)
21 // compressed pubkey length
03D7DE86033E1C815BA0E3589938B7AC906B728F1F2C771542E203E32B80A3385A // compressed pubkey

00000000 // Locktime

<<< 010000000001014E2F5CCED0DC6D7B229AB43381A7DDBAB759C457C3366D8435782120493133A00000000000FDFFFFFF0201000000000000001976A914149185A045594EDDD9C1A895A75D1A5FEAE2C34388AC54030000000000001976A914149185A045594EDDD9C1A895A75D1A5FEAE2C34388AC024830450221008f56dd07c3637ba6bc00e4e8e05bd2e9c3d6a5c073201df8aaf52e9847d0e01c0220752381278c83287c975a147b20daef443f68d85c84650ab2d061375b6f6ef739012103D7DE86033E1C815BA0E3589938B7AC906B728F1F2C771542E203E32B80A3385A00000000

End Result:

A transaction (txid is on top of this post in the link) manages to fire off successfully.
The targeted receipt addresses are INCORRECT in the transaction. I will need to figure out again on the txouts side and update here again.

What changed:

Step 3 & 7. Hash Sequence & nSequence. The input transaction (a033314920217835846d36c357c459b7badda78133b49a227b6ddcd0ce5c2f4e) utilizes an nSequence of (0xfffffffd) which after formatting will become (0xFDFFFFFF) so the step 3 for hashSequence has changed. Subsequently its double hash for this step also changed.
Step 8. Hash Outputs & Outputs. For the original outputs, I used e803000000000000 16 0014149185A045594EDDD9C1A895A75D1A5FEAE2C343 which is the amount to spent in little endian with the Bitcoin opcodes of OP_0 <pubkey hash> but I changed it to the OP_DUP OP_HASH160 OP_PUSHBYTES_20 <pubkey hash> OP_EQUALVERIFY OP_CHECKSIG. I have added two transactions but the recipient address is not what I want so this has to be updated later on.

Future Questions:

What happens if there is an input with vins containing multiple inputs and each vins input uses a different nSequence, then which nSequence should be used to generate the transaction signature ?


text:
Unable to verify Bitcoin 23.0 release due to developer keys being untrusted/uncertified
I have precisely followed the instructions for attempting to verify the latest 23.0 release of bitcoind on Linux, available here: https://bitcoincore.org/en/download/
I have gpg imported all the keys defined here:
https://raw.githubusercontent.com/bitcoin/bitcoin/master/contrib/builder-keys/keys.txt
I have then ran gpg --verify SHA256SUMS.asc SHA256SUMS and when I do this, gpg is reporting multiple keys are not certified with a trusted signature - there is no indication that the signature belongs to the owner - providing an exit status code of 2.
My script therefor does not proceed to download and unpack the latest bitcoind binary.
I don't believe I am doing anything wrong and I suspect this is a key hygiene issue that the devs need to get on top of. Is anyone else experiencing this issue, and if not, can provide some insight into a fix?
Thank you.
UPDATE
There are also several can't check signature: no public key errors on the gpg verification output so this is probably the main cause of the problem.

I have then ran gpg --verify SHA256SUMS.asc SHA256SUMS and when I do this, gpg is reporting multiple keys are not certified with a trusted signature - there is no indication that the signature belongs to the owner - providing an exit status code of 2.

This error is a result of how GPG determines whether it trusts keys. When a key is signed by another key, the signer indicates that they have verified that the key belongs to the person it claims is its owner. This property is transitive, to a degree (that is configured by you, locally). So if GPG sees a key signed by a key that you trust, then it will also indicate that you trust that key, even if you yourself have not signed it. You can also indicate (locally) to GPG that you trust a key, and with what level you trust it.
This particular error message means that the GPG does not see anything that indicates that it trusts those keys. It may mean that you have not signed someone's key who has signed those keys. It may be that your GPG simply does not have anything in its trust database and so it defaults to not trusting any key (this is the likely scenario for most users).
This security.SE question goes into further detail.

I imported all the keys and signed them as trusted, using keyserver.unbuntu.com. Re-ran gpg --verify and everything checks out except a single key that has expired. So yet again, I'm at a point where I cannot use this gpg verification method to verify the integrity of the download, even though it's recommended that this process be followed before running the bitcoind binary. It's a shame because I really wanted to go with this procedure for upgrading my bitcoin node but if the key hygiene of the developers is this flakey, it is never going to work.

The purpose of the multiple signatures is to allow people to be certain of the release even if they only trust one of the developers. There is no need to have all of the keys and verify all of the signatures. One signature being valid is sufficient to verify the integrity of the download. The multiple signatures just allows verifiers to choose which developer's key they want to do that verification with. If all of the signatures are valid and it's just one key that is expired, you have still verified the integrity of the download.

Note how the instructions say "It is recommended that you choose a few individuals from this list who you find trustworthy and import their keys as above". This means that you don't have to verify all of the signatures in order to be sure that the binary you downloaded is correct.

Ok great. I've implemented a mechanism to randomly select a single signature from SHA256SUMS.asc - it then uses that for verification. Not all the keys appear to be valid/are available so it will be hit and miss as to whether this step passes but, given I run this upgrade check daily, eventually it will pick a signature which offers a pass (thus exit code of 0) and then the upgrade of bitcoind will continue as normal (SHA256SUM download verification etc). Many thanks for your help @Andrew.


text:
What's the difference between 'rpcallowip' and 'rpcbind'?
What's the difference between these two settings in the bitcoin conf file?
'rpcallowip' and 'rpcbind'

You could be running Bitcoin Core on your desktop and issuing RPC commands from your laptop. In this example you might want to put rpcbind=0.0.0.0 and rpcallowip=192.168.1.10 or your laptop’s lan IP.
Rpcbind binds to a socket, this happens in the networking layer. Without it packets won’t even reach bitcoin core. Rpcallowip is used for authorization which happens in the application layer. Packets need to be able to reach bitcoin core before the command can be authorized so in a sense rpcallowip needs to be a subnet of rpcbind


text:
Is it possible to use bitcoin-cli deriveaddress rpc call to generate ethereum address?
https://github.com/satoshilabs/slips/blob/master/slip-0044.md
As per this slip, if we just change the cointype in the descriptor, the deriveaddress rpc call should work to generate ethereum addresses from an xpub too. But it isn't? I can not find any good documentation on this rpc call whether it supports all the bip standards in the derivation path or not?

No. Bitcoin addresses and Ethereum addresses are constructed and defined differently and bitcoin-cli certainly does not support Ethereum address generation. For more details on why see this question.

Thank you for your reply. Cleared some of my doubts.


text:
I sent $50 worth of bitcoins from coinbase wallet to bittrex wallet but received only a third of it in the bittrex
I sent $50 worth of bitcoins from coinbase wallet to bittrex wallet but received only a third of it in the bittrex. Could someone please explain if the i will get more bitcoins later or did i just lose it?
This has never happened before as i have done this transfer previously as well.
It shows confirmed on coinbase but it still doesn't show all my bitcoins on bittrex wallet.
EDIT - i just found out that 66% was the fee they charged me. Why is that? They generally charge 0.25% but this is ridiculously high.

The transaction fee is a fixed amount, not a percentage

don't confuse: 0,25% are the fees of the exchange-website for trades, withdrawals, etc.. the exchange-website gets this 0,25%. the 66% of your 50$ are network-fees. when coinbase executes your withdrawal-transaction, they are sending a transaction to the bitcoin-network and set a fee (for example 0.0025btc (~35usd)). the miner which creates a new block in the blockchain where this transaction is included got the network-transaction-fee. i think it is very sad when exchanges don't allow their users to set the network-fee for their transactions. but this is unfortunately normal.

This is normal, Bitcoin's transaction fees are getting excessively expensive. That could be why altcoins like BCH are gaining popularity

Please keep politics out of answers here as much as possible :)

But 66%? I never knew that it would be this high.

you must usually pay or every transaction. the bitcoin-network-fee does not depend on the amount of btc you are tranfering. if you are transfering small funds it is unfortunately virtually normal that the percentage of fees is higher. the fee is not higher if you transfering very high amounts of btc


text:
As transactions are getting increasingly slow and expensive, how can Bitcoin compete against newer better currencies?
If I understand correctly, every transaction needs new blocks to be created (mined), which is becoming increasingly hard, slow and expensive. Not mentioning the massive energy consumption, already raising environmental issues.
On the other hand, any new better currency would offer exactly the same service, except faster and cheaper.
Then how can Bitcoin compete against it, in long term?

I have voted to close this question, cause it is merely opinion based. What is "better" in your view, might not be better in others view. What does it mean, when you say "massive" energy consumption? Compared to what? Who sets the baseline here? Also: there are already more than 1000 Altcoins, which can do things faster and cheaper. And one can see already, how bitcoin competes "against it".

@pebwindkraft "Better" is meant to refer to "faster and cheaper" in the same sentence. That its energy consumption is "massive", is discussed all over the internet, plenty of baselines as large as consumptions of small countries, just google for details. It is a serious concern and I am asking for objective facts how Bitcoin is addressing it, if there are any. Also I respectfully disagree with "one can see it already", one can only see the price based on hype and popularity and speculation, that can always change very fast. You are welcome to edit the question, if you see a better way ;)

In this sense: there is no better currency. Bitcoin is offering a path to Segwit and lightning, which makes it faster and cheaper. The bullshit on massive energy consumption is a repeating pattern, mostly by people who haven't the faintest idea, what the consumption really is. Current consumption based on Antminer S9 and Terahash: roughly 1000MWatts. Compare this to other currencies, and then look at global scope. So Bitcoin will stay for the longterm, cause other currencies are too weak, and don't provide the benefits you are asking for.

@pebwindkraft Many people had justified their claims about energy consumption and I had never seen counterarguments. The same goes for transactions getting slow and expensive, people are complaining waiting for weeks. I am asking about real solution and simply dismissing the problems is not one.

I think I do not dismiss the problem, I think the opposite is true: people dismiss, that there is segwit, but still complain. And people also complain about energy, without doing the math (on both sides). And lightning is on the way to go. For those who don't get it, there is lots of Altcoins (BCash, IOTA, ... and more). But I do understand: complaining is easier than acting :-)

@pebwindkraft How is segwit solving the energy or slowness problems?

to avoid extensive comments, you may want to create a new question, or even two, stating exactly your assumption or opinion, what you would expect to be a reference, and how "the problem" affects reality...

When transactions are created they are essentially a payment promise. They are submitted as a message to the network, where every node checks that the promised money exists and then forwards the transaction. Some of these nodes collect transactions in order to create a new block. These miners are essentially participating in a giant lottery where the winner gets to author a new block.
To that end, miners chose which transactions they are trying to confirm and add their own Coinbase transaction. The Coinbase transactions is the first in a block and gets to collect the transaction fees, creates the block subsidy, and contains the segwit commitment (in a segwit block). All the other transactions can be chosen at the leisure of the miner, although they pick the transactions with the highest fee rate, i.e. the transactions that generate the most revenue to them, by default.
The mining process is self-adjusting, in that the difficulty of blocks gets reset every 2016 blocks. Hereby, the difficulty can increase or decreaseee by up to a factor 4. Hence, the difficulty does not always have to get larger, in fact we've seen it decrease a few times this year. There is an argument to be made for it becoming slower, in that lately the demand for transaction confirmation has been much larger than the available blocksize which caused a lot of transactions to remain unconfirmed for weeks at times.

On the other hand, any new better currency would offer exactly the same service, except faster and cheaper.

The thing is: A new network actually cannot offer the same service. There is a huge demand for Bitcoin transactions. However, a new network can only transmit its own tokens. If you want USD, would you settle for Venezuelan Bolivars instead? Although there have been numerous clones, copies and forks, in the past eight years, just cheaper transactions don't seem to be a sufficient incentive for adoption.

By the same service, I mean paying for goods. If I run the company servicing international payments, I can use any crypto currency accepted by the other end. If Bolivars can do it faster and cheaper, I'd have no problem using them. It doesn't mean I would invest and hold it, which is completely different matter.

"the difficulty does not always have to get larger"  - does it not get harder to mine every new block?

@DmitriZaitsev: No, difficulty only changes every 2016 blocks, and it only gets higher if the last 2016 took less time than 14 days.

But it does get harder to mine every new Bitcoin, doesn't it? Now I am confused.

Mining for new bitcoins has nothing to do with the transaction speed.  At some point you will be unable to mine new coins, but that has no relation to the transaction speed.

@DmitriZaitsev nope, it does not get harder to mine with every bitcoin, or every block. As Murch mentioned, the difficulty target is re-calibrated every 2016 blocks. I think you misunderstood somewhere. The difficulty target is set with the aim to have a new block found every ten minutes, on average. So the rate of blocks being generated (transactions being confirmed) remains stable over time (again, on average).

@chytrik Thanks, maybe I did misunderstand, because both transaction confirmations and coin creations are called "mining". So generating a block to confirm a transaction is unrelated to mining the new coins?

@DmitriZaitsev It works like this: a miner performs proof-of-work hashing to find a new block. When the miner finds a valid block hash, they can submit that block to the network. The block they submit may be full of transactions that they chose to include (thus confirming transactions). The first transaction in every block is a 'coin base transaction', which rewards the miner with newly minted bitcoins.

@chytrik Then isn't it becoming harder and harder to mine new blocks?

@DmitriZaitsev: Both transaction confirmation and coin creation are product of the same mining process. The difficulty is retargeted every 2016 blocks (14 days at ten minutes). If more hashrate is entering Bitcoin mining, the blocks are too fast, and the difficulty increases. If hashrate leaves mining, the blocks are slow and the difficulty decreases.
May I suggest the following topics for some added background information:
https://bitcoin.stackexchange.com/q/148/5406, https://bitcoin.stackexchange.com/q/855/5406, https://bitcoin.stackexchange.com/q/12427/5406

Miners are rewarded for creating new blocks, so the more valuable the users consider the network to be (ie higher coin value), the more electricity miners will dedicate towards securing it (to gain the block reward /tx fees). This is the nature of POW: less energy-intensive networks will be less secure, and as a user the security of the network that stores your value is very important.
Being 'fast and cheap' is nice, but many altcoins already exist that are faster and cheaper, and still bitcoin remains the most valuable. Consider that there are reasons for this, beyond 'fastness and cheapness' of transactions.

I suppose Bitcoin's biggest market cap currently is only due to it being older. But how is this a competitive advantage? If I can get the same service with X-coin, cheaper and faster, why would I use Bitcoin?

Security is nice, but: Too many other ways to compromise security, as the most recent history shows. Why would I trust the guys running Bitcoin more than some new X-coin?

Well, why would you trust any chain to hold your value? I think most people would look at the history of the project, so part of the picture is indeed the fact that the bitcoin network has an outstanding record of immutability and uptime. At the protocol level, security is very important (this is different from the "don't have your private keys stolen" level). The security of the code has not been compromised recently, I'm not exactly sure what you mean by this.

I mean the security of the exchanges and their own software, not necessarily the coin software. Every exchange I check shows huge number of bad reviews, it feels frustratingly risky to trust any of them, even if the coin software is flawless. And if it is so risky to deal with any unknown, untrusted service, how can we even speak of any serious security? Like the one provided by banks with government guarantee.

There are reputable exchanges in operation, at least a few have some govt oversight of their operations in the US (those that have a 'bit-license' to operate in NY state, for example). There are certainly some sketchy exchanges and outright scams that need to be avoided too. It is prudent to not hold coins on an exchange longer than necessary, to minimize the risk that you lose coins if they are hacked. This industry is not super easy for non-tech-savvy people to navigate, but it becoming better all the time. Years ago it was even more difficult than it is now. Progress is being made.

Sound like still long way to go ;(

It is a fundamental problem: People need to feel safe buying these coins. Calling it "not super easy for non-tech-savvy" is not helping it. "Not to hold coins for long" is not solving the problem, because it creates additional whoops and yet there is still real risk to lose a lot of money. If there is no exchange to guarantee the safety of their customers' funds, how can one even talk about safety?

I think you misunderstood me: I mean that a user should not store coins on an exchange for longer than necessary. Rather, a user should transfer their coins to a wallet they control, and hold them there (for longer term storage, or for day-to-day usage). That said, I believe Coinbase offers insurance against some forms of theft from their platform.

Re: calling it "not super easy for non-tech-savvy" is not helping it. -> I'm not here to pretend that bitcoin is ready for every single non-technical person to engage it fully. The reality is that key management requires more serious considerations than perhaps any other form of cyber-security, and it is important that this is made clear to users, in order to help them learn how and why it is important to protect themselves. It is not incredibly hard to store btc securely, but pretending it is easy would be a disservice to new users.


text:
Is it possible to set up an Antminer S3 using a a Raspberry Pi?
Hey everyone I am new to mining and just getting into it. I have done loads of research and haven't found the answer to my question. I have seen an Avalon 6 being run through a raspberry pi and I really like the idea of doing this. This being said I haven't seen the ability to do this with an Antminer S# series, and I am unsure why. I understand that it has its own controller and is more of a plug and play. I would like to run this through the raspberry pi though as it is more mobile and in general takes less space. Below is the video of the Avalon 6 being integrated: Avalon 6 w/ raspberry pi
Thanks for the help!

An Antminer S3 operates as a stand alone computer, so you need no controller. The device is configured and managed through a web browser interface, so you can configure it from any web browser.
The only downloaded tool from Bitmain for the Antimer that runs on a PC is their "IP address finder" app, and you can use any of hundreds of apps to discover the IP address of devices on your LAN, including Fing.

I think the OP is talking about replacing the control board on the antminer s3 or s5 with a raspberry pi. You can use the raspberry pi with an antminer U3.


text:
Blockchain is losing me money with every transfer
I'm losing up to 45% on a BTC transaction. When it reaches the merchant they say that I'm losing it in the transfer and that the chain is the cause. How true is this and how can I check with the chain on what's going on?

45% of what? the btc sent? this does not sound correct, watch out for scamming merchants, as soon as u have sent your btc there is no getting it sent back without the merchant sending back to you etc

Have you tried looking up your transaction on a blockchain explorer like https://www.blockchain.com/explorer ? How much did you pay in mining fees as a proportion of the entire transaction amount?


text:
Is it possible to send 0.00000001 BTC?
I tried doing it using bitcoin-qt but it wouldn't let me. Is this to avoid blockchain spam?
What clients would allow me to send that amount?
Will I have to pay fee if I send such a small amount?

You can not send transactions that small. The minimum is 0.00005460 BTC, anything smaller is called dust and will not be accepted into the network. You will most likely need to pay a fee with something with this little priority, which will cost you more than the amount you're sending. Bitcoin is not intended for micro transactions. Also bear in mind that collecting a lot of transactions this small will cost you more in fees to spend them again than the value of the coins.

Will I HAVE to pay a fee for a transaction of 0.00005460 BTC? Is there any progress with regards to micro transactions via Bitcoin?

You can try without, it might go through but it will take a very long time. Hours or days, maybe never (the funds will be returned to you). It's not something the developers are concerned about at the present, as it's not the intended use of Bitcoin.


text:
Bitcoin Daemon Syncing Crashing Server. CPU through the roof
we run a Bitcoin Daemon on a Linode server.
We're seeing huge CPU usage to the point its crashing our server.
We're not mining, simply running a node.
Can you help?

Can you please add details such as client version and whether you are using a bootstrap file or downloading and verifying the blockchain from scratch?  Or is the blockchain already up to date?

The bitcoin daemon takes around 24 hours on a good server to sync to the network. During this time CPU usage is very high as it is verifying the blockchain from scratch.
Once synced, bitcoind is a far lighter program.


text:
Is mining with my computer worth it?
I have a computer with a Phenom II X6 1090T 3.2 GHz, and a GT430 gfx card. Obviously I know I won't be able to mine lots of bitcoins, but I want to start getting some money for a car and such. Is the only solution to mining by getting a ASIC Machine, or people on other forums run a pool using a huge botnet.
If you guys can give me any tips to start, it would be appreciated.

So according to this wiki[1] your GT430 can run at 20 MHash/s. And if you plug 20 MHash/s into the mining calculator[2] you're looking at around $0.08 a day in revenue. Your daily power consumption will be larger than that.
Or to think of that another way, the current rates mean $0.41 USD/24h@100MHash/s, and you're one-fifth of that.
8 cents per day. That's a long way to save for a car.
[1] https://en.bitcoin.it/wiki/Mining_hardware_comparison
[2] http://www.bitcoinx.com/profit/


text:
Securing a wallet and multiple addresses
I am trying to understand Bitcoin QT. Sorry for the questions.

If I create two new addresses, then encrypt my wallet and then create a third new address after the encryption process, are all three new addresses encrypted and safe to use?
If I then copy the wallet.dat file to usb sticks and delete the wallet.dat file on my laptop, are the funds on the usb stick still accessible? On all three addresses?
Can I then continue sending money to these addresses? Even thought the wallet isn't connected to the internet?
Imagine I have a copy of the wallet.dat file on a usb stick. If I generate a 4th address on my laptop, and then destroy the laptops wallet.dat file, will funds I send to the 4th address still go into the wallet.dat copy  on the usb stick which was made before I created the 4th address?
If I can see coins transfered to my wallet and confirmed, should I bother setting up a second wallet on my other computer and doing a test transaction with my new funds? Or can I safely assume the coins are ok?
Is there a gauranteed fail safe way to check the balance in an address without hooking up the wallet.dat file to my client? I want to keep my wallet.dat file offline.
Are the answers to the above the same for Litecoin QT?

Thanks so much for any guidance

Yes, your "wallet.dat" contains all your keys, and when you make new keys then they will be added to your wallet.dat. If you encrypt your wallet.dat, then all keys in it will be encryptet and all keys added to it will also be encryptet.
Your founds is not stored in your wallet.dat. The founds are stored in the blockchain (which everyone can download via a huge p2p network).
Yes. Your wallet.dat only contain the private keys to spend coins. Everyone can at all time send you coins (whether your connected to the internet or not).
No. The USB stick will only be able to spend coins send to the first 3 addresses (as the USB stick's wallet.dat will only contain private keys to the first 3 addresses).
As long as your client has the longest blockchain, then coins send to you can NEVER be removed again (unless someone spends them, with the private key only you know). However note that coins can first be "trusted", when there is enough confirms ("0 confirms" just means that someone has sent you a transaction, but it will first be "trustable" when it's included in the blockchain and enough blocks confirms that the blockchain will never be forked under your transaction. But usually 6 confirms should be enough).
Type your address on a trusted website, e.g. https://blockchain.info/ and look at the balance. Some clients support "watch-only" addresses (they only store your bitcoin address and not your private key), but I don't think Bitcoin-QT supports this (yet).
I don't know much about litecoin, but most of the "public/private key" stuff, apply to all cryptocoins (e.g. you can watch an address without storing the private key, etc).

Thank you :D That all makes sense to me apart from answer 5 where you say 'As long as your client has the longest blockchain,'. What does this mean and how can I check/ensure that my client has the longest blockchain?

Supposed an attacker controls all your internet. What he can do, is to block all bitcoin traffic between you and the rest of the bitcoin network (so you can't see the longest blockchain). Then the attacker creates a transaction where he sends some bitcoins to you and begin mining a block with this transaction in it. At some point he will generate a valid block and send it to you. From your point of view, the longest blockchain will contain a "1 confirmed", so you believe that the attacker has sent you some bitcoins.

So now you send the attacker some dollars/goods (believing that the attacker is a good guy that have paid you). After this, the attacker give you full access to the internet and you can see that the "attacker mined" block is not valid (not in the longest chain), but you have already send him dollars/goods. To prevent against this, your client can calculate how many blocks it should see (e.g one block pr 10 minutes). If at sometime the block generation slows too much down, your client will warn you that something is wrong. And the attacker can't alone generate blocks fast enough to avoid this

One thing I forgot about #4: By default Bitcoin-QT creates a "key pool" ( https://en.bitcoin.it/wiki/Key_pool ) with 100 addresses in it, so when you click "generate address" you actually wont generate an new address, you will just select an address from the key pool. So old backups can recover "new" keys, but you should ofc still be very careful not to delete your wallet without making sure you have an updated backup :-)


text:
How to utilise more cores when I am cpu mining?
This is purely an experiment for fun. This is for litecoins but can bitcoin people help me too :)
I am using litecoinqt to solo cpu mine, on my windows machine. Threads set between 4 and 10, scantime of 99.
My main computer has 4 cpu cores. But I am noticing the same speed as when I use my laptop which has only one cpu core. This makes me think I am not using all the cores on my main computer?
Is there a way to force the miner to use more cores?
Thanks

I don't really know about Litecoin-Qt but for a better result and to configure correctly the numer of threads I recommend you to use a software dedicated to LTC mining.
You can use https://github.com/pooler/cpuminer for only CPU and https://bitcointalk.org/index.php?topic=150331.0 for GPU.
You must have a pool but you can solo mine with the arg '-server' when launchin litecoin-qt.
I know it's more difficult than just use the LiteCoin-Qt miner but it is also more configurable and more efficient.


text:
Proof of address ownership
This question appears many times on SO, but there's something in the answers I'm not getting.
I'm designing a system predicated upon API calls made to blockchain.info, but I'm having real trouble getting my head around the relationship between Pubkey, Address and digital signatures for my use-case.
Say user Jane has previously transacted using Bitcoin and we assume the transaction is confirmed on the Bitcoin blockchain. Jane now makes a request from her app-based wallet software for some metadata concerning that specific transaction via the system's web-service.
The web-service requires Jane to prove that her wallet really instigated that transaction in order to permit her wallet access to the data, but here I get a bit hazy: From all my reading over the last 6 or 7 days, I gather the wallet software would need to sign some pre-determined message containing e.g. the TXID using the wallet's private key, sign it and send it to the system along with the public key, but once the system receives the data, and can verify the message using the public key, so what? How does that prove the txid (or address or whatever) in that signed message, was signed by the same private key from which the transaction in question is indirectly derived? There's some part missing for me that "reconciles" remote blockchain data with the data sent from the wallet app.
I understand basic PKI and that addresses are hashed derivations of a public key, but the digital signature part is messing with my head.
Thanks for reading. Any help would be greatly appreciated.

Is there some specific reason you're doing it such a complicated, kludgey way? Is there some reason you can't do it a much more straighforward way? For example, why not have Jane pay to a specific address just for her that binds whatever she bought to her account or to some other credential?

The system I wish to build pertains to metadata about a btc transaction, almost none of which is available via the public blockchain, it will eventually be generated by the system. All I'm asking is how the system as some 3rd party, might best determine that a software wallet app  really took part in a given BTC transaction, to which the metadata relates. Such that only that wallet can have access to thw data.

I opened random transaction for this example: https://blockchain.info/tx/c929454d6c83c15ecd9931c005a5a7fbacb1faba69f0a49538ab334d2848c5a1
Input scripts:
ScriptSig:
PUSHDATA(72)[3045022100eb1d36c6fe6c201e5594d90a7bdc73dd9a33d291e7e58ba74fec46ab2139c9c602207a7124819f9a004a36a0a2ac494603cfeecf38b8c12b3c6e3ff6dea74053706101]
PUSHDATA(33)[033f2223c8f6b74e75e4afd05811b8c9f55c8584abef768336e6ad61d931bef548]

First hex-value is signature of transaction. Second value is public key (elliptic curve point). And from this value we can derive address.
If you have another signature that signs something, and this signature has the same public key, that proves, that this is the same guy, who did transaction. Only one who have private key can sign something.
Elliptic cryptography works in the next way: we choose random number, and this will be private key. Then we multiply this number and EC point called generator, and we get public key.

Thank you, that does help. So am I correct then, that for the "application" outlined above ("reconciling" or "associating" 1 or N tx input addresses sent to the application from some wallet app), that it should 1). Request that Jane's wallet signs some data 2). Request that the resulting signature and Pubkey are sent 3). Verify the signed document contains the expected data by decrypting it with the provided Pubkey and 4). If verified,  compare the pubkey with the one used in the desired blockchain transaction in order to assert "ownership"?

For step 3. You cannot decrypt anything with signatures. Public key is used for verification. Verification procedure can return 2 values: signature is valid, or not. It accepts 2 parameters: public key, and data. Data is supposed to be hashed, and hash then be verified.

Right, so does the process therefore become: 1). Sign raw data 2). Send sig + hashed-data + pubkey 3). Verify?
But what I'm still missing (sorry!) is how off-chain "verification" of arbitrary data proves that the sender's prikey was used to hash the message, and is implicated in a given blockchain TX. I suppose once verification of the message-hash itself was complete, the system could check the blockchain for a TX with that/those pubkey input(s), but the pubkey differs for each TX input, so wouldn't it also be different for this off-chain procedure too?

"verification" of arbitrary data proves that the sender's prikey was used to hash the message - privkey is used to sign the hash, not to hash the msg. You are right, you cannot check anything off-chain. You have to be sure, that certain transaction is inside blockchain. Public key in TX input means bitcoins came from some address, and your verification procedure confirms, that the man who owns this address signed some message.

I understand that addresses derive from pubkeys. What I do not understand is how a signature as part of this off-chain process, relates to a Pubkey, address or anything else on-chain.

Signature is created by using private key, public key and address is created by using private key as well.


text:
Transactions with invalid or non-parseable Script
During parsing of Bitcoin data from blockchain, I found many transactions which has invalid script (example 37b828de0da38f0b3b9ec09f6f8b7883c3202a99a285b0b0087f19f9cbf526d9,f6502e80d8a8dc9e59bb518a62a6443479dd72cd153929272ac21bab26317226)
Is there a way to parse these scripts to pull out the output address? If not, what happened to the BTC send to these invalid script?

By the way, where (which block/transaction?) did you find these scripts?

Does this answer your question? Non-standard transaction scripts broadcasting

During parsing of Bitcoin data from blockchain, I found many transactions which has invalid script (example 37b828de0da38f0b3b9ec09f6f8b7883c3202a99a285b0b0087f19f9cbf526d9,f6502e80d8a8dc9e59bb518a62a6443479dd72cd153929272ac21bab26317226)

Is there a way to parse these scripts to pull out the output address?

Addresses are human-readable shorthand ways of writing scriptPubKeys that follow a certain template. Not every script has a corresponding address. Only these are defined:

P2PKH addresses (1...): OP_DUP OP_HASH160 <keyhash> OP_EQUALVERIFY OP_CHECKSIG.
P2SH addresses (3..., defined in BIP13): OP_HASH160 <scripthash> OP_EQUAL (defined in BIP16).
P2WPKH addresses (42-character bc1q..., defined in BIP173): OP_0 <pubkeyhash> (defined in BIP141).
P2WSH addresses (62-character bc1q..., defined in BIP173): OP_0 <scripthash> (defined in BIP141).
P2TR addresses (62-character bc1p..., defined in BIP350): OP_1 <tweaked key> (defined in BIP341).
Future native witness addresses (bc1..., defined in BIP350): OP_n <program>.

Any scriptPubKey not matching any of the above templates doesn't have a corresponding script. Unparseable scripts are among those.

If not, what happened to the BTC send to these invalid script?

Those coins are lost. There is no way to spend them, because the script can't be executed if it can't be parsed, and without it executing successfully it can't be spent.


text:
How can I make my node a Lightning routing node?
I've spent some time setting up Umbrel on my rPi4, I've synced my Bitcoin Core wallet and funded two channels... Which I've now come to realize that's for me to make payments.
I was really hoping to use my BTC to help the growth of the Lightning network and facilitate payments by routing payments, can this be done with my current setup? Still so little info out there...

It can be done, but it's not trivial. Technically, in order to be able to route payments, you need liquidity in both directions, incoming as well as outgoing. If you just funded two channels, you only have outgoing liquidy now (all the money is on your side).
There are many ways to gain incoming liquity. You could spend or swap the funds in your channels (or a part of them). You could ask someone to open a channel to you (https://lnbig.com seem to give these away for free). You could buy liquidy from https://lightning.engineering/posts/2020-11-02-lightning-pool/.
But that's just the beginning. If you really want to contribute, you need to always manage your node so that it provides routes where people need them. This is an art and you can surely find more hints by googling.
There are some developments underway to automate this process. If you used Core Lightning instead of LND, you could try the clboss plugin, which has the goal of making all these choices without user input. Your mileage may vary.

another option is to send funds to an exchange that supports lightning network. you could then withdraw the funds as an onchain withdraw or to another lightning node (I am sure the exchange would open a channel with your 2nd node in case you withdraw funds

This is my first time using lightning and I'm struggling to see the point really... Unless it goes like this, I stake my BTC and the network uses my BTC to facilitate payments on the network and I receive routing payments for providing liquidity... Then why would I use it? It's been around a few years and it's still way too hard for people to use, seems like and utter waste of time to me


text:
Can't verify Taproot signatures that use SIGHASH_ANYONECANPAY
I wrote a program that verifies signatures for Taproot transactions using the secp256k1 library. My program works perfectly for SIGHASH_ALL, SIGHASH_NONE and SIGHASH_SINGLE. But in February, we saw the beginning of a huge wave of Taproot transactions using 0x83 as the SIGHASH byte. I also know of one Taproot transaction that uses SIGHASH byte 0x81. My program fails on transactions that include the SIGHASH_ANYONECANPAY bit and I can't figure out why. These are all keypath spent transactions, which should be relatively simple to verify.
The problem must be that I am not producing the same message that was signed for these SIGHASH types, but I can't find where the error is. I am using the Signature validation rules section of BIP341 to determine which parts of the transaction to include.
The transaction that uses SIGHASH byte 0x81 is c0b767665e68916510126d704c1e96903275ed9e8c97e6782568cd23a4e8ef09, input 0. My program produces a message of 126 bytes as follows:
    Zero byte: 00
      Sighash: 81
   Tx Version: 02000000
  Tx Locktime: 00000000
 Outputs Hash: d32241d6cf9637a786922b700fc5c34385dc2a738d84734cae914b39ef595cfe
   Spend Type: 00
     Outpoint: 7eac3298e596858e44bdf23a06409133bf5eb5b886dda2891de452b991a7489400000000
       Amount: 3954000000000000
Output Script: 225120fd784aba5e91d18306ba722f3af50ecdaf056caf19a3632a39b1b64a2109ecf6
     Sequence: ffffffff

The outputs hash is a single SHA256 of the following bytes:
Output 0 Amount: e803000000000000
Output 0 Script: 225120fd784aba5e91d18306ba722f3af50ecdaf056caf19a3632a39b1b64a2109ecf6
Output 1 Amount: 3d46000000000000
Output 1 Script: 225120fd784aba5e91d18306ba722f3af50ecdaf056caf19a3632a39b1b64a2109ecf6

A transaction that uses SIGHASH byte 0x83 is 2e46f5b63deb421dee85cfea240a74b99d14b3286c195189f223f6260ea13943, input 1. My program produces a message of 126 bytes as follows:
    Zero byte: 00
      Sighash: 83
   Tx Version: 02000000
  Tx Locktime: 00000000
   Spend Type: 00
     Outpoint: 19ad63e54efbd351f94496789035483cd4c6962c9d63fe83dc9babcead677f4b00000000
       Amount: 1027000000000000
Output Script: 2251208102001190c6aad9a015dff1540dc9a7bda31613b8ab05a58268c4bff53fae82
     Sequence: ffffffff
  Output Hash: a29ce7981f103591964c176f85cb171e098c4d86d31ec711b2a91c0d59134adf

The output hash is a single SHA256 of the following bytes:
Output 1 Amount: 1027000000000000
Output 1 Script: 2251208102001190c6aad9a015dff1540dc9a7bda31613b8ab05a58268c4bff53fae82

Everything else is working fine. I'm using the TapSighash tag and everything else correctly, otherwise none of the other transactions would verify either. The problem is only happening for SIGHASH_ANYONECANPAY types. What am I missing?

It looks like your outpoints are serialized incorrectly. Txids are always displayed in reverse byte order, so when serializing them, they need to be in the reverse order of what everything displays them as.


text:
Unknown Address Format Parsing Bitcoin Addresses from Blockchair Data
I've obtained a list of all current Bitcoin addresses with balances from Blockchair, which is downloadable from their data repository located here.  I have downloaded the Bitcoin / Addresses TSV database dump.
The list consists of:

~23m P2PKH (1xxx) addresses
~5.8m P2SH (3xxx) addresses
~1.3m Bech32 (bc1xxx) addresses

However, there are also ~400k addresses that have prefixes that I don't recognise, including:

d-xxx (example: d-b697921d996831d1a26ca8d8e92571c1)
m-xxx (example: m-942943e227462513f33c4f32ec6f1e56)
s-xxx (example: s-2ad548f2459c0ed6123400e5d61a479e)

Are these valid Bitcoin addresses?
If they are, how do I view information about them?
If they are not, any idea why they appear in the dump of current Bitcoin addresses from Blockchair?

Having investigated this following @MCCCS's answer, the following is information about the addresses provided by Blockchair support in response to the issue I raised in their GitHub:

In fact, there are no addresses on the Bitcoin blockchain. There are
only scripts, some of them are standard and can be converted to a
Bitcoin address, some of them can’t — like nulldata (OP_RETURN)
outputs, nonstandard, and bare multisig. We use the d- prefix for
nulldata, m- for bare multisig, and s- for everything else.
This is briefly mentioned in this API documentation section:
https://blockchair.com/api/docs#link_300

Therefore, these addresses (for want of a better word) are valid transactions on the Bitcoin blockchain.
The prefixes used are:

d- nulldata
m- bare multisig
s- everything else

In conclusion, the data is valid but the outputs are not addresses in the normal sense of the word.

Those are invalid addresses that don't belong to any cryptocurrency. You should file a bug report at https://github.com/Blockchair/Blockchair.Support/issues since those might even be their database secrets.
Those are of 16 bytes. My initial guess about their format would be SHA-1, but Bitcoin does not use SHA-1. Probably it's related to their database.
For example on this page https://blockchair.com/dogecoin/address/942943e227462513f33c4f32ec6f1e56 it seems Blockchair thinks 942943e227462513f33c4f32ec6f1e56 is a valid Doge address. It's not.
To prevent this from happening again in the future, they could implement input sanitization before INSERT statements

Thank you. I've raised an issue on their support forum: https://github.com/Blockchair/Blockchair.Support/issues/273

I've received a response from Blockchair support and provided the outcome in a separate answer. Thank you again for your input, which ultimately led to me finding the answer!


text:
Miner for Non-Standard Transactions
I understand that bitcoin-core introduced the IsStandard() check in v0.9(?) to prevent non-standard transactions from being broadcast (unless in a mined block).
I'm interested to know whether there are any current mining pools or miners that will process non-standard transactions - i.e. they're running a non-default configuration that will permit the non-standard transactions to be submitted using sendrawtransaction (or another method) and mine them.
When I say non-standard, I mean that the transaction will be rejected because it contains non-standard inputs (i.e., inputs that return false when evaluated by IsStandard()).
I already understand that testnet is more permissive in this sense but I'm specifically interested in miners on mainnet.
Note: This question is for academic purposes in support a paper I am writing.

The checks in isStandard() are largely ignored for P2SH scripts and for Segwit scripts, since they sidestep a lot of the concerns that caused the standardness checks to come into existence.

Thanks for the comment. I'm talking specifically about transactions that involve non-standard inputs, so P2SH\Segwit doesn't apply here. I'll edit the question to clarify what I'm asking


text:
i find r s z from tow transaction but the r value is not same
i need your help plase
i find r s z from tow transaction but the r value is not same
from this wallet 1XCPdWb6kk7PGfvbdRbRuNh51aPc4vqC7
this is information
tx 1 b152df9fc1087066cc12ec69c441f35559d794267209fd03f6d3f1d257b69549
raw tx : 010000000134930446c8a27fb457e65fa6f88a193843bf49a73043b997cfd005b9910173a7290000008a4730440220700d06f0d24f18e89acaf250b137b25066b4abea1f3cab58ba48b3dac7e3ede7022032236390aa15f60370236a9b88a5fbcc95aca95bbc7c32be2e781daf5ed1cd1601410433e05b29670f19cbc499f207f11abe1c69f77f00d5cbb9dbec5b5fe7527e2f41fa1e90f10a05e9c0a34d255988082e190c9ee7ea05f62297d4f76d9b61d7561bffffffff020000000000000000306a2ef24afce462d82863a3686fa4071c2d451a0e8c1056250110f8c08e8966f731b5d630d7b6a89c8ba88d3d388c3d4236150000000000001976a91405b5fca463964fc8af5ce743e1cfc1102601ecfc88ac00000000
THE R1  700d06f0d24f18e89acaf250b137b25066b4abea1f3cab58ba48b3dac7e3ede7
THE S1  32236390aa15f60370236a9b88a5fbcc95aca95bbc7c32be2e781daf5ed1cd16
THE Z1  b28f435e053ba045130a39562570bb7fcb7100d6501cb1571ee7bea979ea6377
THE PUBKEY : 0433e05b29670f19cbc499f207f11abe1c69f77f00d5cbb9dbec5b5fe7527e2f41fa1e90f10a05e9c0a34d255988082e190c9ee7ea05f62297d4f76d9b61d7561b')
tx 2 4fa22a2149dd320655425ad7a88b6f6aad87afe10bcf7117c2499b1d5af9529c
raw tx 2 : 010000000134930446c8a27fb457e65fa6f88a193843bf49a73043b997cfd005b9910173a7280000008b483045022100e13d5d80bab1b5734684334c652cdaa53bd63003b3fb5044e7cfabb78e3e6621022007f8c435382190f1b7d3f0318924e886360ff2b4a7a565fb36d2ed6b936b3e3e01410433e05b29670f19cbc499f207f11abe1c69f77f00d5cbb9dbec5b5fe7527e2f41fa1e90f10a05e9c0a34d255988082e190c9ee7ea05f62297d4f76d9b61d7561bffffffff020000000000000000306a2ef24afce462d82863a3686fa4071c2d451a0e8c105531133861c0ed81e886d159436a4e351103320143bbdae92a2836150000000000001976a91405b5fca463964fc8af5ce743e1cfc1102601ecfc88ac00000000
('THE R2 e13d5d80bab1b5734684334c652cdaa53bd63003b3fb5044e7cfabb78e3e6621')
('THE S2 07f8c435382190f1b7d3f0318924e886360ff2b4a7a565fb36d2ed6b936b3e3e')
('THE Z2 1e8faed746b6bcfbc42d8c2c75b58e1ba2f9059089be669bbb9365b84937e89c')
('THE PUBKey 0433e05b29670f19cbc499f207f11abe1c69f77f00d5cbb9dbec5b5fe7527e2f41fa1e90f10a05e9c0a34d255988082e190c9ee7ea05f62297d4f76d9b61d7561b')
this is the wallet
https://www.blockchain.com/btc/address/1XCPdWb6kk7PGfvbdRbRuNh51aPc4vqC7
And I'm going to send some money to someone who will help me. Please, all I want is to get the private key for this wallet.
I searched again and again and did not find any solution or I did not understand the methods well Please my brothers I want help urgently
I found these variables R S Z, but the variable R is different in financial transactions. Is there a way to extract the private key via R different
I want a full explanation and if you can attach me the last result which is the private key in the messages
Help me even by a tool to find the private key via two variables of the value R

You can't. Otherwise Bitcoin, and ECDSA in general, would be broken.

Related: https://bitcoin.stackexchange.com/q/25814/5406

A private key is retrievable if two different messages are signed using the same r value and the same private key. Two signatures using different values for r are insufficient to recover a private key.
You are asking for help to break one of the core cryptographic assumptions that make Bitcoin work. If what you are trying to do were possible, ECDSA would be considered broken and any form of address-reuse would leak private keys. To the best of my knowledge, neither of these two are currently the case.


text:
Is it possible to earn BTC by processing transactions?
Where does the money go? Is it possible for any person to setup his wallet application to process transactions and make some more BTC?

See What exactly is Mining?

There are ways you can provide services and earn bitcoins as a result: http://bitcoin.stackexchange.com/questions/4858

If you run an Escrow service, yes, you could get paid to process transactions. But that's probably not what you're asking about.
In normal cases, transaction fees go into the reward for a new block being mined and are earned by the miners that create the new block. This takes specialised hardware and software. If you want to get your feet wet in this area, you ought to look into mining at a pool.


text:
How to turn off "change" address in bitcoind?
I would like to keep backup of my wallet on paper, but that's impossible or at least impractical when the bitcoind keeps creating new addresses for "change". I know it is possible in theory to do it by sending the change back to the sending address. And I know there are privacy implications when doing this. But I care more about security than anonymity in my case.
Is there an option to turn off "change" addresses in bitcoind (the original Satoshi client)?

Bitcoind isn't designed for that. There isn't option for that and probably won't be. If you are interested in keeping paper backup, you should look at clients such as Electrum.

I guess "electrum or electrum" isn't what you meant to write : P


text:
Which bitcoin script forms should be detected when tracking wallet balance?
I am implementing a library for tracking wallet balance of given bitcoin addresses.
How should the transaction output script be parsed to find if the output belongs to the given address?
Is it dirty if I just match pay-to-pubkey-hash and pay-to-script-hash transaction by simply matching
OP_DUP OP_HASH160 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx OP_EQUALVERIFY OP_CHECKSIG

and
OP_HASH160 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx OP_EQUAL

or should I implement more sophisticated script parser which can detect "non-standard" scripts as well?

These are the scriptPubKey's that are standard, arranged in order of popularity:

P2PKH (Pay to public key hash)
OP_DUP OP_HASH160 <20 bytes of public key hash> OP_EQUALVERIFY OP_CHECKSIG

P2SH (Pay to script hash)
OP_HASH160 <20 bytes of script hash> OP_EQUAL

P2PK (Pay to public key)
This is now mostly unused, except by miners, but there are still unspent outputs in the form:
<33 or 65 bytes of public key> OP_CHECKSIG

Segregated witness outputs
There are two ways of signaling that a scriptPubKey should be interpreted as a segregated witness program:

Native witness program
<version byte> <2 to 40 bytes of witness data>

The version byte must be equal to or less than 16.
P2SH wrapped witness program
OP_HASH160 <20 bytes of script hash> OP_EQUAL

The scriptSig must have exactly one item. That item is interpreted as a witness program, as above.

Once you extract the witness program, you must match it to one of the witness program types:

P2WPKH
0 <20 bytes of public key hash>

This is hashed identically to P2PKH.
P2WSH
0 <32 bytes of witness script hash>

This is pretty similar to P2SH, except that the script hash is SHA256, not HASH160.
Other version 0 scripts, not 20 or 32 bytes in length, are nobody-can-spend, so ignore them.

Note that any witness program format can be combined with any witness program. For example, you can have P2SH wrapping P2WSH, like this:
scriptPubKey:
OP_HASH160 <20 byte hash of scriptSig element> OP_EQUAL

scriptSig:
<34 byte serialized witness program>

unserialized witness program:
0 <32 byte hash of first witness item>

Then, the first witness item is unpacked and interpreted as the actual program.
More information here.
m-of-n bare multisig
Also pretty rare.
<m> [n <public key>s] <n> OP_CHECKMULTISIG

Data-carrying output
Only one of these is allowed per transaction. These outputs can never be spent, so you probably don't need to detect them.
OP_RETURN <less than 80 bytes of data>

Should I implement more sophisticated script parser which can detect "non-standard" scripts as well?

No. It's a waste of development time, when nobody uses transactions like that. (If they did, those transaction types would be added to the list of standard transactions. :))
It's also going to be very difficult to implement correctly. If you don't do it correctly, then you'll think you have coins when you really don't. For example, imagine someone sends you a transaction with a scriptPubKey like this:
OP_DUP OP_HASH160 <20 bytes of public key hash> OP_2DROP OP_CHECKSIG

Note that this is like a P2PKH output, except that OP_EQUALVERIFY has been replaced by OP_2DROP. That means that the script checks that the signature is correct, but doesn't check that it's signed by the right key.
That means that anybody can spend it, not just you. Correctly detecting edge cases like these make writing a program that can understand nonstandard outputs more trouble than it's worth.

You have forgot about bare multisig outputs

Isn't multisig also wrapped inside P2SH?

@Jus12 Conventionally, but there's no protocol rule requiring that.

or should I implement more sophisticated script parser which can
  detect "non-standard" scripts as well?

You can create two "virtual" accounts while parsing the blockchain.
First account receives all non-standard transactions which are provable unspendable.
There will be no spendings from this "account". The amount of this account will be the answer to a question "How many bitcoins are lost"
Second account receives all other non-standard outputs.


text:
Is it possible to create a feature for custom wallets that also have a bank account number?
The main problem I'm attempting to tackle is to create a feature that allows for wallets that are linked to a bank account which needs monitoring and transparency to prevent embezzling. Think something like a housing society's funds in a bank account no. 123XXX. If all of 123XXX's transactions are shown on the blockchain(by linking this account to a wallet), all spendings will be more transparent.
As pointed out by Murch, my attempted solution was this -
I am trying to modify the bitcoin protocol(possibly fork it and create an altcoin) that allows for the creation of non-anonymous wallets that have a publicly visible bank account(identity theft not an issue) as one of the fields on the Blockchain every time these wallets are part of a transaction.
Please guide me toward this.

Are you talking about forking Bitcoin into an alt-coin and adding this feature, or trying to do this on the existing Bitcoin blockchain?

"will also be displayed on the blockchain" doesn't make sense. This question seems like an XY-problem to me. What do you actually want to do?

Wallets are not announced to the network. Perhaps you should elaborate what you're trying to do, right now it sounds more like you should be aiming for a website than a modification of the Bitcoin protocol.

To be transparent in Bitcoin, there is little you need to do:

All transactions are visible on the blockchain already.
There is no need to connect addresses to your bank-account or any such thing.
Instead, just use a single address for all your transactions and require documentation and vouchers for every payment on your website.

You seem to have a poor understanding of how Bitcoin addresses work. When you create a Bitcoin address it doesn't appear anywhere. Not on the blockchain, not on someone else's computer. It exists only as a private-public key pair on your computer. When you first receive funds that is when it appears on the blockchain.
In other words, you don't give your address to someone by saying 'please look at transaction 7 in block 317592', you just give them your address. It is the same with your bank account, you just give them your bank account!

First, publishing anyone's bank account number is not a good idea as that can be used for identity theft. Second, there are a lot of identity solutions like onename, which may be closer to what you want to do. There's also BIP47, which allow anyone to publish a single public key whose transactions are not known to anyone but the sender and receiver.

authors: Александр Борисов