text:
Blockchain / Bitcoin / Previous address / Next address
New to learning about the Blockchain ...
The Blockchain requires a Token for use. It is my understanding that the Token (or rather the code implementing the Token) is responsible for filling up the Data portion of a block (that is to be added as part of the Blockchain).  Is this correct?
If so, this means that what exactly goes into the Data area of a block is dependent upon the Token that fills it.Does that mean that Bitcoin is a type of Token?
In the Block (that sits on the Blockchain), there is a Previous address and a Current address. The Previous field and the Current field is used to support the Linked List structure of the Blockchain (with the Previous address pointing to the Previous block in the block chain and the Current address being the actual block id of the Current block).
Does the Current address of the block take into account the Data area (or contents) of the Current block? My understanding is that the Current Data area of the block would hold Merkel Trees - and - that the root of the Merkel tree is somehow incorporated as part of the Address of the Current block.
TIA

You're confusing a few of the terms here. The "data area" of a block contains transactions. Transactions generally redefine Bitcoin ownership by signing over funds from one owner to another. "Token" doesn't seem well-defined to me, so I'm not sure whether a bitcoin would constitute a "token" in the way that you think of "token".
Blocks are identified with the hash of the blockheader, not an address. A new block points at its parent block by including the parent's hash in their blockheader. The blockheader's hash covers the content of the block by including the Merkle root of the transaction tree which commits the block to specific set of transactions.
I've written a bit more about how the blockchain works here:
Can someone explain how the Bitcoin Blockchain works?

thanks for the response. Above you said ""Token" doesn't seem well-defined to me, so I'm not sure whether a bitcoin would constitute a "token" in the way that you think of "token""
I was following the explanation here: https://youtu.be/_160oMzblY8?t=11m49s
From what you are saying, the "Blocks are identified with the hash of the blockheader" is the ID of the block.

Yeah, the hash of a block's header seconds as its ID. I've watched a short bit of the video, it seems to me that the author is using token as some sort of abstraction.


text:
Difference between Byzantine Fault Tolerant Protocols and Byzantine Agreement protocol (i.e., BA*)?
What is the difference between Byzantine Fault Tolerant Protocols and Byzantine Agreement protocol (i.e., BA*)?
Is it just a naming convention or both concept differs fundamentally?
Nathan Aw

There are many problems in the family of "Byzantine" questions with slightly different names and all related to each other in various ways. Generally, the problem concerns a network of nodes all connected with all, some of which are honest and some of which are adversarial, i.e., can behave arbitrarily, even maliciously and send different data to different peers or refuse to send data altogether. Each of these nodes have some "input", potentially different for each, and they want to agree on a common view. The "input" can be the set of new transactions to confirm in the case of bitcoin, for example.
The canonical problems in the space, which are equivalent (if you can solve one, you can solve the other), are formulated as follows:
The Byzantine Agreement Problem. A group of nodes needs to agree on some value. The protocol needs to (a) terminate within finite time, (b) have all honest nodes reach the same result, and (c) the result reached must match the input of one of the nodes.
The Byzantine Generals Problem. A group of nodes has a designated "leader" called a "general". The leader has an input value and the rest of the nodes need to agree on a value. The protocol needs to (a) terminate within finite time, (b) have all honest nodes reach the same result, and (c) if the leader is honest, then the value agreed by everyone should be the leader's value.
These problems were put forth by Lamport in his seminal paper.
These problems can be modified to allow for less connectivity or to allow for failures that are not byzantine (i.e., do not allow for malicious intelligence, just failure that isn't trying to subvert the system actively). You can also add digital signatures to the mix to make the problems easier.
In bitcoin's case, the problem solved is a harder variant; anonymous Byzantine agreement. In this case, the number of parties can change arbitrary and the parties do not know who is sending each message. The solution to this is to use Proof-of-Work.
To answer your original question about Byzantine Agreement versus Byzantine Fault Tolerance. Broadly, these titles refer to the same problem of agreement. However, more specifically, BA is used to refer to the specific problem of reaching consensus about a decision under byzantine settings; while BFT refers to designing any generic protocol (not necessarily requiring agreement), for example the Byzantine Generals protocol, for solving a problem under byzantine faults, i.e., faults that are malicious. The generic problem of BFT and the problem of BA are equivalent - if you have agreement, you can be fault-tolerant; and if you are fault-tolerant you can reach agreement.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Mapping the Bitcoin Lightning Network: Why not use the Blockchain?
Researching about the available routing algorithms for the Bitcoin Lightning Network, I have been reading the Flare whitepaper, and this idea/question came to mind:
Since all payment channels are created with a blockchain transaction, and Flare itself checks the Blockchain to see if a channel is live or has been closed, wouldn't it be possible to parse the blockchain for all live payment channels to create a complete, up-to-date map of the Network so that you do not have to use flooding or beacons on the Network itself?
Do/can on-chain transactions that create the payment channel store nodes' ids? If yes, then this seems a trivial and maybe effective tactic.

You can add data to the blockchain, like IP address or node name, yes. This would lead to a bloat of the blockchain, where everyone is already complaining about it's size ... Then the client would also have to parse the blockchain, to find it's counterparty? Maybe this is a good point for a discussion in bitcointalk.org? I have seen several lightning threads there ...

wouldn't it be possible to parse the blockchain for all live payment channels to create a complete, up-to-date map of the Network so that you do not have to use flooding or beacons on the Network itself?

No because it is not possible to know which transactions are related to a payment channel opening, and channel closing transactions can be ambiguous. The funding transaction sends money to a 2-of-2 multisig, but these are in outputs as a P2WSH or P2SH-P2WSH script which means that the redeemScript that specifies the 2-of-2 are not known. You cannot assume that any P2WSH or P2SH (because a P2SH-P2WSH looks just like a P2SH) is for funding a payment channel.
For closing transactions, if there is a cooperative close, then the spend of the funding transaction's outputs will look just like a normal 2-of-2 multisig, which is not indicative of a payment channel. In a unliateral close, you would be able to identify the unilateral close only after the person closing the channel spends their output.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Atomicity on HTLC payment chain
I have a doubt about HTLC on a lightning network payment. I have this scenario:
Alice wants to pay 100 sats to David using a route she found with Bob and Chad, this would be:
Alice -> Bob -> Chad -> David

Alices sends the HTLC to Bob, Bob to Chad and Chad to David, everything goes as expected, David wants to reveal the secret to Chad and they update their channel state, now David have +100 sats on his channel balance and Chad have -100 sats.
Chad do the same with Bob and now Chad have +100 sats on his side of the channel and Bob have -100 sats on his side.
At the moment Bob wants to do the same with Alice his node crash and go offline (for days), Alice can wait for the timeout and get her funds back, Bob would lose the funds.
I read before that an HTLC chain was atomic, all payments are successful or none of them, is this true? in this case David and Chad got their payments updating the channel state but not Alice so this case is not atomic.
What I am missing here?

It is atomic
If a node goes offline the htlc will not be settled off chain but onchain. of course that is more expensive with respect to fees but that is true for channel closes in any case

What happens with the channel update between Chad and David?
How they know if Bob or other hop goes offline?
Why they would want to go onchain if they can update they channel state offline?

Chad and david have settled off chain. Alice can't settle onchain as she did not get the payment secret as bob is offline. Bob looses money but he is offline he has to actively claim the payment. Slice cancled the htlc onchain as the timeout starts to kick in and not resolving the htlc might make her pay Bob later

"Chad and david have settled off chain. Alice can't settle onchain" <== not atomic, correct?

Yes and no. It is atomic in the sense that a routine node cannot mess up. And make Alice or David loose funds. If the routing node disappeared it might have lost money though as David got Paid but Alice had not to reimburse Bob and cancles the htlc. If Alice goes onchain but Bob appears in time Bob can get reimbursed on chain too.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to find bitcoin address from bitcoin privatekey
How to find bitcoin address from bitcoin privatekey using bitcoind json rpc or any other way?

while not an exact duplicate, the logic of deriving keys is mentioned in this thread: https://bitcoin.stackexchange.com/questions/69315/how-are-compressed-pubkeys-generated/69322?noredirect=1#comment80498_69322. When having a private key, you can load it into bitcoind with "importprivkey", explanations in the API calls list (https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list). I hadn't found a direct way to see, which address belongs to a private key, but a "getaddressesbyaccount" before and afterwards did the trick for me.

and a second link: https://bitcoin.stackexchange.com/questions/1389/how-are-public-private-keys-in-an-address-created/1715#1715 - if this is not sufficient, maybe you can outline your question a bit more, so we can understand, what tool(s) you would like to use? There is python, Java, shell scripts, ...


text:
Bitcoind estimatesmartfee
I am trying to get transaction fee for sending all btc to another account.
I use estimatesmartfee to get tx fee, I have total btc amount of 0.0195btc, so I pass 0.0195, but I get JSON integer out of range (code -1)

estimatesmartfee expects the number of blocks you want the tx confirmed in as the parameter, not the amount of BTC you are sending.
Try estimatesmartfee 5 to get an estimate for confirmation in the next 5 blocks, for instance.

thanks, but I followed your command but it still throws some error {   "errors": [     "Insufficient data or no feerate found"   ],   "blocks": 1 }

Your node has not seen enough blocks to make an estimate yet, or does not have the feerate configured correctly


text:
What is the motivation for miners to include all recent transactions in a block?
I think I understand the basic operation of the Bitcoin protocol.
Blocks are the "long-term memory" of the network used to prevent double spending (and the way the network agrees in first place). Proof of work is sort of brute-forcing the nonce so that the hash has at least the desired number of zeros. When a dishonest node generates a new block other nodes will refuse it should it contain transactions that were not seen by them, so the network will "converge".
I guess the transactions need to be included in that PoW (or else it would be simple to alter them afterwards). So every time a new transaction is seen it will require the miner to start brute-forcing "from scratch"? I read some sort of tree is formed, so it actually doesn't mean the input to the hash function gets longer, but still every transaction will change the "root value", won't it?
Why would a miner even bother to include all transactions that occurred on the network (given that transaction fees are minimal)? There is no lower limit on the tx number per block (perhaps in that time there were really no transactions). It would be simpler for them to just include perhaps one and then use their (CPU) time to solve the puzzle and earn the reward.
There must be some sort of thing in the Bitcoin design which I don't get yet...

I've probably fallen for the "gambler's fallacy". The probability that the next calculation will win is the same no matter whether I change just the nonce or something else changes in the block. On average is takes that many tries to get a correct solution.

Your answer is based on many misconceptions. For one thing, 2^31 is half of 2^32, not 2^16. For another thing, a typical miner will try all 2^32 possible nonces in a second or so. Lastly, your chances of finding a working nonce are completely independent of how many prior successes or failures you've had. It may be that no nonce in the 2^32 possible nonces work. It may be that ten of them work. Each nonce is an independent trial. Typical miners won't interrupt a set of 2^32 nonces just because of a new transaction, but a few seconds later, their next run will use the new transaction.

Miners have a long-term investment in the health of the Bitcoin network. If Bitcoin collapses, then their expensive ASICs are worthless. Miners particularly need Bitcoins to remain valuable over the long term because their hardware produces Bitcoins over time.
If nobody includes transactions in blocks, then Bitcoins would be useless and therefore worthless. That would hurt the miners' long-term investment.
If this ever became a problem, transactions would just wind up with higher fees to encourage miners to include them. Right now, enough miners will include a transaction with a miniscule fee that there's no point in paying more.

I see. So the few additional hash computations for the Merkle tree update are probably for the "greater good" (but might get rewarded by tx fees). However mining is still exactly at same speed as without including transactions (because there is no "from scratch") - I did not realize that yesterday. Well ASICs probably would be worthless but a general purpose CPU won't. We are probably off-topic but I guess that's the motivation behind Proof of Stake (more BTC, more to lose if the network collapses).

"However mining is still exactly at same speed as without including transactions (because there is no "from scratch")" - there is a from scratch every time a transaction is added (due to a new merkle root), but because 2^32 nonces can be hashed on the header so fast, adding another transaction(s) to attempt the next round of hashes with all nonces isn't a big deal. Additionally, the timestamp in the header adds variation enough for a new group of hashing with the 2^32 nonces. Can someone confirm if this is correct. Whatever the case I've voted up the question & accepted answer, thanks!


text:
How does one manage the shifts of Channel Balances in the Lightning Network
I saw the video below that shows the possibility of a shift in Channel Balances. It shows that the amount that one uses to fund a channel may "disappear" if the channel is used as part of a route to fulfill the payment for someone else.
Someone Else --> Me --> Store
https://youtu.be/pOZaLbUUZUs?t=2m18s
It this scenario:

-> I get ready to go to the store
-> I establish a payment channel between myself and the store and add $100 USD to it
-> Someone else uses my channel (i.e. me) as part of their payment routing - a go-between - to pay the same store (since they don't have
a direct channel to the store).
-> My direct channel to the store is now empty of funds.

How does one come about this problem?
TIA

Beware that that video was specifically produced by Bcashers for maximum FUD effect. It starts with a kernel of truth and then pulls you towards insane and nonsense conclusions.

It's actually a non problem deliberately set up in this video as a strawman. Professionally produced FUD.
In practice you're very unlikely to have a direct channel with the specific store in the first place, since for privacy reasons you actually want to be separated from that store at least 3 hops. But in either case you'd simply route your payment in the other direction through a different route towards that store. That's the whole point of this thing being a network. Payments can flow in both directions and the network is full of cycles.

thanks for the information. How can one determine that this video was done by Bcashers? Is there a way that one can spot FUD in the future? Also, you said: "you'd simply route your payment in the other direction through a different route towards that store" - to do so would mean the transaction cost would increase - right - since it is going thru more hops to get to the store.

Hard to say what gives it away. The sort of people/forums that link to it. The comments below it. The way the video is structured by staying with a small kernel of truth but then quickly derails. But all of those require some level of knowledge of (recent) topic history as well as the technologies involved. So it's hard and time consuming.

Re tx cost: After the sunk cost of opening and closing on chain transactions, all further LN transactions are really just a few bytes flowing over the internet between a few peers. Bittorrent does a trillion times more, for free. The biggest reason to charge a fee is to offset my cost of moving my abacus to the side i don't want it to move (bringing me closer to having to close on chain). But that's offset by transactions moving in the other direction. Competing software managing that balancing act and finding optimal routes among those competing alternatives will drive fees very low.

thanks again for the response :) You said "Competing software managing that balancing act and finding optimal routes among those competing alternatives will drive fees very low" - can you suggest such a software?


text:
Using bitcoin-core RFCs to create REST APIs
I am new to using Bitcoin and the Lightning Network.
I was looking at this: https://bitcoin.org/en/developer-reference#bitcoin-core-apis  I saw REST APIs as well but did not seen any functions for POST, DELTE, etc. Would be realistic to try to make a REST APIs using these functions. One should not have to worry about authorization since the Blockchain is public - right? I wanted to create functionality that would be called when openning/closing a Channel on the Blockchain from the Lightning Network.
Also, I was hoping that in creating a BTCD daemon (and synching it with the Blockchain), one could perhaps have their transactions commit faster. But, it looks as though this can only be done by miners(?) Is my understanding correct?
TIA

I can't speak too much about the lightning network, but you should be able to build a REST endpoint yourself by utilizing the Bitcoin-Core wallet and the JSON-RPC calls as you referenced. In terms of committing transactions, I assume you want fast confirmation time. A natural way would be paying higher fees, which incites miners to include your transaction sooner. You could potentially pay miners directly to do that, but I found that a little bit unreliable, plus you just can't get away with no extra cost

Thanks a lot for your response. Should I be a miner? The thought would be  add "open/close" transactions to my block, "solve the puzzle" for it and then place it out there on the Blockchain - does this make sense? This way, one would not have to depend upon another miner to incorporate transactions.

You don't have to be a miner. All you need is just to broadcast your transaction. As I said, faster confirmation time is achieved by either paying higher fees or pay a miner to include your transaction asap. On a regular computer, you'll probably never be able to miner a block, so becoming a miner is pretty much pointless.

Thanks again. Out of curiosity, would the confirmation time be much faster on the "testnet" (the testing Blockchain)?


text:
Difference between Hashcash proof of work algorithm and Ethash proof of work algorithm?
Understand that each proof of work algorithm has its pros and cons.
Difference between Hashcash proof of work algorithm and Ethash proof of work algorithm?
Nathan Aw

Hashcash and Ethash are two different things and not directly comparable. Hashcash is the idea of using hashes for a proof of work system. it can be used with any hash function.
Ethash is a specific hash function. It is used in a hashcash proof of work system, specifically the one used in Ethereum.
Hashcash is not comparable to Ethash because it is not a hash function at all.

One major difference between them is that Ethash(Modified Dagger-Hashimoto Algorithm) is ASIC resistant due to its memory hardness (Algorithm works on precompiled dataset of size > 1GB) , whereas Hashcash algorithm is not ASIC resistant as it uses SHA-256 hash function which do not require any memory to work with.


text:
Proof of Elapsed Time Susceptible to Random number generator attack?
Since Proof of Elapsed Time (POET) uses a Random Wait time to elect a leader, would it be susceptible to Random number generator attack?
Nathan Aw

Bitcoin Stack Exchange on topic seems to have changed; sorry if migrated incorrectly.

True random number generation is difficult (unless random number generators have improved?) and many can be seeded, some bad ones producing predictable results with careful seeding. So, as I understand it the question becomes, what random number generation system is uesd, who controls it (how is it accessed?) and, how is it seeded? An attack that is theoretically possible is not necessarily feasible.


text:
Bitcoin Pluggable Consensus with Algorand?
Algorand is an extremely promising consensus protocol -- Could Bitcoin be configured to use Algorand?

No.
You can't change Bitcoin's consensus algorithm. If you do, and you don't convince literally everyone to do the same thing, you'll end up with a fork in the chain. At best, that means you won't see any new blocks anymore. At worst, you'll end up with two active chains, and a split currency.


text:
What are the tangible benefits of decentralisation?
What are the tangible benefits of decentralisation?
This question was posed to me by my parents. Under that this question might incur a lot of wrath from this community.
When my parents lose their ATM cards, they can go to the bank to queue up to get their ATM card if they provide their identification card.
Unlike the above scenario, when one loses his/her private key, one loses access all his BTC.
Hence this question -- Other than removing intermediaries, what are the tangible/immediate benefits of decentralisation? Frictionless commerce, etc.
Nathan Aw

Some of the tangible benefits of decentralization, blockchain technology and triple-ledger accounting include:

Higher trust between parties, with a lower need to trust other parties due to transparency and immediate reconciliation of transactions when a blockchain/triple ledger accounting system is used.
Resiliency, Robustness: by eliminating single points of failure (a single company, a single database, set of computers/network), the network itself can be much more resilient and able to deploy resources most efficiently/effectively.

I'm argentinian and in 2001, banks converted all USD money in the argentinian accounts to argentinian pesos. A lot of people lost money, because the peso plummeted in value after that. With a central authority things like this can happen at any time.
Descentralization is good because you remove a central authority that can modify the economic policy in an arbitrary way, it can seize or reduce your funds for arbitrary reasons, etc.
If Bitcoin had a centralized authority that would mean the policies that were set in the beginning (e.g. the quantity of coins that will be created, the fact that is a deflationary currency) could change. The fact that Bitcoin is descentralized means that the policies can't change, actually if they change it would not be Bitcoin anymore, it would be a new currency.
So from the beginning you know the "rules" in Bitcoin and you know they are not going to change because an arbitrary authority decided to change it. You are protected against arbitrary change in economic policies.
Also, being descentralized, you can do whathever with your money and hipotetically nobody can take it from you. A central authority could take your funds because arbitrary reasons (e.g. a new tax, etc.).
Also, being descentralized means it's really hard for a government to "shut down" Bitcoin.

I think that we take advantage of the situations that we are in.
There are some countries that will not greatly benefit from decentralisation because we trust our government. This is not the case in most circumstances, in lesser developing countries the government cannot be trusted and because of the power that we have given them, they can at any moment create money out of nowhere, take money from you and a plethora of other things.
The question for me, boils down to "who governs the government?" when we have a centralised structure, we normally need people in place to watch them. For example, there are people in charge to watch the bank and make sure that the banks don't run off with your money. In my opinion, I think the banking crisis showed that they were not doing a good job. Even if we did not have the banking crisis, who is governing those that are watching the bank? Who governs them then? There comes a point where they can't be governed. This is where we as human beings start to take advantage of the situation. This is why we need decentralisation. Who governs in decentralisation? From what I have seen, the majority, the way it should be.
TLDR: It is hard to think of the reasons, we need decentralisation until your government starts spying on you and taking your money/property because they can. It is a great benefit for those in developing countries, and by us using these technologies, we are steering the ship towards the global decentralisation.

Thanks! Is trust the only reason for decentralisation?

@NathanAw The umbrella reason, why you would want decentralisation is that, you do not want a central authority. Trust is one reason why you would not want a central authority, I do not want to be hasty and say it is the only reason.


text:
Why is mining described as validation of a block through proof of work when it is the validation that is critical not the proof of work?
That is reward for proof of work provides incentives for miners to validate transactions; it does not validate them through proof of work.

You are correct in that the proof of work itself does not validate transactions, that is done separately, so that only valid transactions will be included in the block.
However, the block itself must contain a valid proof of work (the block hash). Without this proof of work, the block would not be valid (even if the transactions are valid). So it can be said that a block is made valid by its proof of work.
Edit: it is worth noting the distinction between the act of performing the proof of work hashes, and the object that is the proof of work (block hash). My answer references the object, not the act.

"Validation through proof of work" is indeed a wrong description of what mining is. Mining is what operates a universal clock of sorts which solves the problem of verifiably ordering the blocks of the blockchain. The Bitcoin paper refers to it as the "timestamp server".
Note that miners do not verify blocks. They get a very rare and expensive chance (once every 10 minutes on average, enforced by Proof-of-Work) to propose a block, which better be valid or they've wasted their electricity. But it is all other full nodes (including the ones that are not mining) that validate the block proposed by the miner who solved it and broadcast it to the network. The nodes do this by simply ignoring it if it is not valid in their "opinion".
For a much more detailed description of what I mean by "timestamp server" or "clock", see this write up: https://grisha.org/blog/2018/01/23/explaining-proof-of-work/

While it is true that what a miner does for the PoW does not involve block verification, miners do still verify blocks so that they are not wasting their time and electricity on an invalid chain.


text:
Why do I need a private key to send some my bitcoins?
Why do I need a private key to send someone my bitcoins?

Paradoxical, because without the key you cannot even call it "my bitcoins" :)

A private key is the only way to assure everybody else you and only signed a specific transaction saying 'give X of my coins to Y'.
If you publish your private key, anybody can sign transactions like that and your money can be stolen. You should keep your private key private.

Standard Bitcoin addresses encode a user's public key. When someone sends bitcoins to such a "pay-to-public-key-hash" address, they sign them over to the owner of the specific public key. The public key hereby is part of a unique key pair. The public  key (as in public knowledge) can be shared freely, while the private key should be kept secret.
When the new owner of the funds wants to spend them, they have to prove their ownership. To that end, they provide a signature that corresponds to the public key that the funds were assigned to. This signature can only be produced with the knowledge of the corresponding private key, but can then be checked by everyone that knows the public key.
You therefore need the private key to prove your ownership in order to spend funds.

technically speaking, the private key is used to sign a hash of a transaction. This signature process proves, that only the person with a private key belonging to its public key (from which the address could be derived) could sign this hash. As this is somewhat of a very generic question, I think it is a good idea to look up the principles of transaction processing in bitcoin in Andreas' book "Mastering Bitcoin". It is an amazing book that'll definitely get you started. It's freely available online here ...

At the time when the new owner of the assets needs to spend them, they need to demonstrate their possession. That time they give a signature that belongs to the public key that the assets were assigned to. This signature can only be invented with the instruction of the corresponding private key, but can then be checked by everyone that knows the public key. You always need the private key for showing ownership of this assets in order to spend funds.


text:
Eltoo channel - can opening be off-chain
In Eltoo the set-up phase is similar to Lightning. You fund a n-of-n multisig (usually 2-of-2). Of course you don't want to publish this until you have a bail-out possibility - called "settlement transaction" here.
But what I found out is this "setup transaction" is exactly the same as later "updates" - you could also call it Tu0. So is there any need to even broadcast this first transaction to the blockchain (before doing further ones)? Or is it enough if both parties just keep the properly signed Tu0 (with both signatures)? What else do you need in case of an uncooperative close? You broadcast Tu0, latest Tui and (time-locked) Tsi ?
I guess one problem would be the other party trying to "double-spend" his input into the multisig?

I think you have answered your question yourself in the last question that you raised. The problem with not publishing Tu0 is that the funder of the channel could double spend the input of Tu0 and in that case "steal" all the money from the channel. so with eltoo we still need the bitcoin network / blockchain give proof of the existence of funds in the 2-2 multisig

Yeah, I realized when writing the question that double spend could be a problem. What about any "clever hacks" like requiring inputs to have nSequence 0xffffffff to prevent replace-by-fee? And broadcasting the 2-of-2 with 0 fees to the mempool then. Or what exactly is the minimum transaction fee (since else you could have some sort of DoS)?

No DoS since UTXO set is limited. But still silly, because the 2-of-2 will eventually need to be mined for the Tui to be "attached" to it and I basically prevent this with the low fee. Guess you can't have the cake and also eat it :)

Or could you use the same SIGHASH_SINGLE hack that is also described in the paper for Tu-s? Like have a 0 fee 2-of-2 lurking around in the mempool, so if eveybody is cooperating there is no cost associated (and people cannot double spend), but when other party is uncooperative you can "pump-up" Tu0 and do the rest normally after that one gets mined. I am probably again missing something?


text:
IOTA quantum resistance
I've read a great deal about the impact of quantum computing on cryptocurrencies in:

https://github.com/theQRL/QRL/blob/master/QRL_whitepaper.pdf
What effects would a scalable Quantum Computer have on Bitcoin?
https://bitcoinmagazine.com/articles/bitcoin-is-not-quantum-safe-and-how-we-can-fix-1375242150/

IOTA also claims to be quantum resistant, unfortunately in their whitepaper that is just briefly mentioned in the last paragraph.
I believe they are talking about the impact from Grover's algorithm where doubling the bit-size help. I can't find any information on how they deal with signatures (which would be broken due to Shor's algorithm) however.
Their wallet code mentions:

You can use an address for receiving as long as you have not used it for any outgoing transaction. What this means is that once you have sent a transaction with a specific address as input, you should never use it again. This is because IOTA uses Winternitz one-time signatures which degrade security exponentially after each reuse.

So I suppose they are using a Winternitz variant of Lamport's OTS scheme. But how exactly?
I also found an interesting pull request

In order to prevent the reUse of adresses a message could be display to say that if you re-use the same adresses for transaction the security of your wallet will decrease quadratically .

What does it mean that the "security of your wallet decreases quadratically"? Do they have some Merkele tree with keys to be used? Or how is the relation between address and public/private key in IOTA? If there can only be one transaction with a given input that somehow implies that you always have to transfer all funds?

You seem to be referencing the QRL whitepaper too. In that case, XMSS is being used.

Yes, it uses a Winternitz OTS scheme. It is believed that Lamport signatures would still be secure against a quantum adversary.
No merkle signature scheme used with IOTA for transactions, as the result of a number of design and security decisions, for instance, signature size. It was a conscious decision to not use a stateless merkle scheme like sphincs. So yes, you should always transfer change to a new address. The implications of reusing a private key in a lamport scheme is that each reuse halves the security level of the signature (assuming a random message).

So basically it boils down to: if i have two bitstrings of identical length, how many bits with same index (on average) will be different? Because for
all those positions it is possible to spoof the signature bit in whatever way you want (for others just the exact value)
Probability of zero or one is 0.5 like with a coin toss. This means that for half of the indices we win. With 3 bitstrings the question is basically what is the probability I tossed 3 same results.  (0.5*0.5) and then for n (0.5)^(n-1). That is the unfavourable case and that the outcome played nice is then 1 - (0.5)^(n-1).

Instead of solving it through protocol IOTA then relies on a "smart" client. I'd expect that the change is automatically returned to a newly generated address of mine? Not completely sure, what does it mean that an "address is attached to the tangle" (which happens when you generate a new address through the light client)? Isn't that something that can be done locally?

Found https://www.reddit.com/r/Iota/comments/6jknwe/purpose_of_attaching_an_address_to_tangle/ already regarding "attachment".


text:
How to build a p2sh who's scriptsig contains a Signature
I have created a P2SH address, funded it, and am attempting to move the funds with a TX.
The redeemScript for the TX requires the TX signature (OP_CHECKSIGVERIFY), but in order to create this signature, the TX preImage must be created and hashed.  However, the preImage for a P2SH contains the TX redeemScript itself...this seems to create a loop wherein this functionality is not possible.
Am I doing something incorrectly, or am I overthinking things?

You might want to review BIP16 (pay to script hash)
https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki
The signature is not included in the redeem script, typically just the public key. (Your script could be anything, all I know is it has a CHECKSIGVERIFY in it, so I'm answering broadly...)
The INPUT SCRIPT aka scriptSig of the transaction SPENDING from the P2SH output would then need to contain the signature and the entire serialized redeem script (the preimage to the script hash in the output).

After more reading, it looks like the redeemScript is the serialized script of the spending TX...and after dumping some variables, looks like my underlying issue is an incomplete preImage.


text:
Fast bitcoin load service
I'm freelancer and I'm willing to get paid with bitcoins. Hence I need to find some service which allows to top up bitcoin wallet with minimum efforts (even without registration). Something similar to: https://paynow.payoneer.com/ (but for bitcoins). Simplicity is really important.
May anyone suggest some?

Do I understand correctly, that you're trying to find something to recommend your customers in order for them to buy bitcoins to pay you?

@Murch yes, correct. But ideally they don't even need to know the word 'bitcoin'. Any of those 'computer stuff things' freak them out really well. It would be great if my client could fill some simple form like: their email address, my bitcoin address, amount to pay, credit card info.

There was a great service back in the day called https://www.moneypacket.org/. You can send bitcoin with email. I think it still works so it seems like a great solution to your problem.

I made the wallet Bolt, and it is extremely simple.
It does not need registration.
All you have to do is generate your address and save a file.
That file could be encrypted or not.
Every time you open the link you should import that file.

Does not answer the question, OP was looking for a service to buy bitcoin on.


text:
Payment has not arrived at destination!
I do not know yet whether is this transaction is confirmed or not because not yet reached the destination !!
https://blockchain.info/tx/962a428acb10ed071b1cf954cbaf763016751b986429df5268424880ee95a7d6.
Can someone help me with advice on what to do

The transaction was confirmed according to the transaction ID you provided (727 Confirmations as of this moment). Have whoever you sent to Bitcoin to check again.

How can i check again if my wallet is empty? tanks

You need to make sure your wallet sees all the recent blocks. Check it's got connectivity to peers to get the blockchain from. Or if you are using a SPV wallet, maybe find another node. If you are using a third party wallet (ie, one you don't have the private keys to), you're at their mercy.

And how i do this?thanx


text:
Bitcoin Widgets & API
can anyone recommend the best api/widget for getting Bitcoin, Crypto Currency Charts, Currency Convert and tickers?

Which platform do you want widgets for? Android, iOS  or something else?

authors: Александр Вязанкин


text:
Does bitcoin stop periodically for finalising the next block?
I have read that every once in a while, the new block is determined by a winning miner. The miner takes about 10 min to form a block out of the transactions he has received so far. Does that mean the network tells each miner all at the same time that it is the start of the race and that they are supposed to compute the next block? During this race does Bitcoin stop accepting new transactions?

No, block discovery is a memory-less process. Every try of discovering a valid block is completely independent from any other. Miners continuously search for a new block. The only thing that changes over time is which transactions they're trying to include and what preceding block they are building on-top of.
When some miner finds a block that fulfills the required criteria, they just publish it to the network. The propagation of the new block also works as the signal for all miners to switch work to a successor of this block instead of the predecessor.
Transactions are relayed in the network independently and continuously. Miners may update their block template to include new juicy transactions every once in a while, but it's up to them how often and when they do this.

How does the miner know that it has to start looking for a block now? I understand that a bunch of transactions in some order is a block for the miner. The miner now has to try different things so that the block's hash condition is matched. Who tells the miner that it should do this activity and when is it told?

@Yash: They never stop. Miners continuously look for a block, as in always.

No.
Transactions are always accepted and included whilst the mining process takes place. The mining process itself is memory-less, therefore there's no "progress towards the next block".
Each time a new transaction is included in a block, transaction inclusion priority are recalculated based on fee rate (as miners are always incentivised to mint the highest paying transaction to reap the reward), then a block template gets produced with the highest-paying transactions included in a block and it is then distributed to workers to mine on.
Transaction selection and mining all take place simultaneously.


text:
Is it possible to / should one use a Lightning Network client?
When looking at all of the examples with the Lightning Network, they involve "Alice" and "Bob". Alice is usually sending some kind of payment to Bob (or vice-versa). The thing is, "Alice" and "Bob" are Lightning Network daemons. From my understanding, these LN daemons require "BTCD" in order to work properly (or work at all).
Payment Channels also seem to require the use of the "Public ID" of a Lightning Network node in order to be created.
FROM: https://themerkle.com/what-is-the-lightning-network-daemon/

LND requires BTCD as its direct connection to the Bitcoin blockchain.
  The daemon has to receive on-chain event information in some way, and
  using BTCD is the most viable solution right now.

I have also seen that BTCD needs to be synched(?) properly (which could take > 12 hours to do).
So, with that, how excactly is one to run a LN Node on their Smartphone (especially since a BTCD node - which is not pruned - is needed for the LN Node to function in the first place)?
Is some other kind of client software supposed to be used? If so, what is it?
TIA

The Eclair mobile wallet by Acinq allows you to run lightning without a full node.

Lightning Network wallets can run in a SPV node just like how a normal Bitcoin wallet can be a SPV wallet. However just because it is possible does not necessarily mean that all software will support this.
LND has the option to use Neutrino, a light client that uses a different SPV protocol. this protocol requires a supporting node which currently exists as a special version of btcd. It should still be possible to run a Lightning wallet that uses BIP 37 SPV although I do not know of any software that does.

thanks for the information. Above, you mentioned "However just because it is possible does not necessarily mean that all software will support this." How are people normally accessing the Lightning Network ~as of now~ from the Web, from Android and from iOS? Is there something special that needs to be installed on these 3 different types of platforms so that access to the Lightning Network will be successful?

AFAIK the web lightning wallets function in the same way that normal Bitcoin web wallets do - the website communicates with a server that has a lightning software on it. That server handles the LN protocol things while your browser handles the private keys necessary for creating transactions. LN software for phones use some form of SPV.

thanks again! You said : " server handles the LN protocol " - is there a code example of this? You also said: " your browser handles the private keys" - could you point me to a code example of this? Or - provide some kind of hint :) TIA

I don't know of any LN web wallets which are open source.

Was not looking for an open-source wallet. Just sample code on the use of LN Protocol on the server side and the use of public/private keys on client side.

Any such sample code would be an open source wallet. LN is not in enough use nor is the protocol fully finalized yet such that people would provide sample code.

If you want full functionality and have a trustless setup, you need to run your own Bitcoin node to watch the blockchain, and run an Lightning implementation (LND, c-lightning or Eclair) on top.

LND uses either BTCD or Bitcoin Core (since v0.4?)
c-lightning uses Bitcoin Core
Eclair uses Bitcoin Core

But you cannot run a full node on your mobile phone, therefore the mobile Lightning node/wallet needs to connect to a remote Bitcoin node. This could probably be achieved using current SPV, but this is bad for privacy & I am not aware of a mobile app that does this. In the future, a different method (Neutrino) will be used for this purpose, which is being tested on testnet.
You can already use the Eclair Lightning wallet on your (Android 5+) phone. Not sure how they connect to their remote Bitcoin node. This app also supports opening a channel to a random node.
For the full LND node, no config is needed, as this node has an "autopilot" feature that opens and closes channels automatically.


text:
Programming API for Lightning Network
I am trying to learn how to work with Bitcoin and the Lightning Network from a programming perspective.
I saw that bitcoin-core was using: https://bitcoin.org/en/developer-reference#bitcoin-core-apis in order to allow one to work with RPCs (or APIs).
Is there something similar for the Lightning Network? When using an API for the Lightning Network, would Authentication/Authorization be necessary?
TIA

For lnd, it's at https://api.lightning.community
Note that there are additional resources at https://dev.lightning.community


text:
Do you have to be a node to use the Lightning Network
I am new to the use of the Lightning Network and had a question.
To run a bitcoin node, one would need to start btcd. What would one need to run for a Lightning Node? One can specify the port a Lightning Node runs on as well - right?
Also, when a channel is opened, between 2 entities, must each entity be a Lightning Node?
For example, Bob wants to buy a coffee from Starbucks on 9th Street.
To open a channel, Bob's wallet needs to run a Lightning Node - and - the Starbucks on 9th street must also run a Lightning Node?
TIA

Yes.  In order to be a part of a network, you need to be a part of the network.  This is the same as with Bitcoin...or even the internet in general.
However, it is perfectly possible for someone to run a node on your behalf.  This is the same as how a web wallet operates.  Some service runs one (or multiple) nodes on the Bitcoin network, and then provides you with a web interface and an account such that you can interact with the network without running your own node.  This is similar in concept to printing out your grandmother's email for her to read without requiring her to touch a computer.  Yes, she can still send and receive email, but with someone else operating the network node.

thank you for the response! You said "it is perfectly possible for someone to run a node on your behalf..." In this case, the LN Node is acting as a type of "server" while the web interface is acting like a "client"? If so, is there a coding example one can look at that displays this kind relationship (client connecting to LN Node)? Also, with regards to Grandma, you say "someone else operating the network node". Is the "network node" a SMTP server? TIA

I was thinking the node in the Grandma example was whatever computer her grandkid is on, but I suppose that grandkid is using a service where someone operates an SMTP/POP3/IMAP server on their behalf.  When it comes to network protocols, it's turtles all the way down.


text:
Using Docker version of Lightning Network daemon to gain understanding
In trying to understand the Lightning Network further. To do so, I was looking at the link below (it shows how it is being used with Docker):
https://hub.docker.com/r/cdecker/lightning-ci/
Based on the information in the link, I had questions.

STEP 1: # Returns an address  cli/lightning-cli newaddr
^^^^ Ques: Is this a new address created on LIGHTNING NETWORK? Is it on the BITCOIN NETWORK? Is it in "No-Mans-Land/Purgatory"? Where
is the Private-Key? Can many addresses be created?
STEP 2: # Returns a transaction id  bitcoin-cli -testnet sendtoaddress
^^^^ Ques: So, if you forget / lose this address (after sending money to it), you are out-of-luck? Also, "bitcoin-cli" commands are
executed under the "btcd" BITCOIN dameon?
STEP 3: # Retrieves the raw transaction  bitcoin-cli -testnet getrawtransaction
STEP 4: # Notifies lightningd that there are now funds available: cli/lightning-cli addfunds
^^^^ Ques: It is at this point the funds are actually added to the LIGHTNING NETWORK. I take it that the funds are associated with
the  created in STEP 1 above?

TIA

The documentation you are following is using the bitcoind wallet, so the address generated in Step1 will be derived from your bitcoind master private key.
Checking the current c-lightning documentation, this seems outdated, step 3 and 4 are no longer needed.

Q1: this returns an address from the bitcoind wallet
Q2: you send a bitcoin transaction from any wallet you like, here the local bitcoind (Bitcoin Core) node is used.
Q4: here, you notify lightningd to use the funds when opening channels. But no channel is opened and thus no funds are committed to the Lightning Network yet. (as mentioned, seems obsolete now with current release)


text:
What Is 51% Attack In Bitcoin? Can Crypto Mining Pools Invade A Majority Attack On Bitcoin?
As the cryptocurrency mining is becoming less profitable for the small miners with rising electricity costs and a shrunken market due to big commercial miners pool, these pools pose a risk of a 51% attack to the cryptocurrency.
https://coingape.com/what-is-51-attack-in-bitcoin-blockchain/


text:
Automated FIAT Transfer to bank from Exchange
is it possible to trigger an automated wire to a bank account with an exchange platform?
we do want to make a couple of transactions a day depending on the bitcoin trading price based on rule sets.
from our wallet --> send to exchange --> automated process to sell into EUR --> wire send to Bank
any ideas?

This would vary widely by exchange. The BTC transfer and trade can almost definitely be automated through most large exchange's APIs. Fiat withdrawals might be possible, but I don't recall seeing any APIs for that recently. Your best bet would be to contact the exchanges and ask about it, as I suspect it is a feature that is available "on-demand" for larger clients, and they might be willing to support you on it.

Bitstamp provide API for both auto selling (it's called "liquidation address") and withdrawals. Other exchanges may provide similar features.


text:
Lnd - how to open a double-funded channel
I know about lncli openchannel, however this means I will open a (bidirectional) channel with my peer, but initially just my side will have some balance. Let's say I agree with a collegue running another lightning node that we both want to have some initial balance. I'd expect to use something like lncli halfopenchannel on both sides and after we both do it the funding transaction and everything aftewards will get created. Same as TCP SYN this is prone to DoS and there needs to be some timeout. Of course the workaround is that he gives me some funds and I then push some of them directly to his side, but that's not really the most optimal approach. Any other possibility? Does any other lightning implementation address this?

What you ask for is called dual funded channels as both peers would found the channel. All implementations will eventually have this feature as it has been agreed to include it to BOLT 1.1. I think c Lightning has done most progress on dual funded channels (in particular Lisa neigut has a branch working on it)


text:
Can it be exchanged.
Can I send  bitcoins to my main bank account ,and the output is like normal cash in the bank. Rather than selling it online .

No.
Maybe one day some exchange partners with a bank to allow such deposits. But that is very far in time...


text:
How do I find the nodeid of the node issuing a lightning network invoice?
I recently got an BOLT11 invoice from a store. For example this one:
LNBC13934430N1PWJUJEZPP5K7FLC2M5CLHTWMTF3HZDN7VVRCA4RNFAFA89F2D6KGYWW4JCSDAQDZ62PSKJEPQW3HJQ4RGV5SYC6T8DP6XU6TWVUSYXMMWVEJHYETWVDJJQ2Z0WFJX2U3QF9ZR5GZ5F3PNZWFD25UY54Z39YXQZURCQP2RZJQG8ZM9WXGVHZPMUQEWY9J9KDQNRVLZHPCWM6S8CVLC2CKRDNELE42Z9U5VQQY3SQQYQQQQL8QQQQPJQQ9Q6KM84F7KTW6UV574UR2LFNKZLM7VNK8NJRZV20JD8SAW6FZ3TQGSZE6YUT0X2UXGDGYJZGTXEFSQSSEVTKFF97P3TER5K23ZPHDDXHQQ95UT4N

Luckily I my node was able to find a route to the destination but if it wouldn't? How could I find out the nodeid to open and fund a direct payment channel with the store? They did not say anything about their nodeid on their website.

Lightning invoices are signed by the node issuing the invoice in order to verify the signature we need to know the node id. Thus it is included in the invoice. Lightning nodes provide an API to decode invoices.
In c-lightning you can do (lnd below!)
lightning-cli decodepay LNBC13934430N1PWJUJEZPP5K7FLC2M5CLHTWMTF3HZDN7VVRCA4RNFAFA89F2D6KGYWW4JCSDAQDZ62PSKJEPQW3HJQ4RGV5SYC6T8DP6XU6TWVUSYXMMWVEJHYETWVDJJQ2Z0WFJX2U3QF9ZR5GZ5F3PNZWFD25UY54Z39YXQZURCQP2RZJQG8ZM9WXGVHZPMUQEWY9J9KDQNRVLZHPCWM6S8CVLC2CKRDNELE42Z9U5VQQY3SQQYQQQQL8QQQQPJQQ9Q6KM84F7KTW6UV574UR2LFNKZLM7VNK8NJRZV20JD8SAW6FZ3TQGSZE6YUT0X2UXGDGYJZGTXEFSQSSEVTKFF97P3TER5K23ZPHDDXHQQ95UT4N
{
  "currency": "bc",
  "created_at": 1563314978,
  "expiry": 899,
  "payee": "024a8228d764091fce2ed67e1a7404f83e38ea3c7cb42030a2789e73cf3b341365",
  "msatoshi": 1393443000,
  "amount_msat": "1393443000msat",
  "description": "Paid to The Lightning Conference (Order ID: TLC19-U8JTQ)",
  "min_final_cltv_expiry": 10,
  "routes": [
    [
      {
        "pubkey": "020e2d95c6432e20ef80cb885916cd04c6cf8ae1c3b7a81f0cfe158b0db3cff355",
        "short_channel_id": "572579x582x1",
        "fee_base_msat": 999,
        "fee_proportional_millionths": 200,
        "cltv_expiry_delta": 40
      }
    ]
  ],
  "payment_hash": "b793fc2b74c7eeb76d698dc4d9f98c1e3b51cd3d4f4e54a9bab208e75658837a",
  "signature": "3045022100d5b67aa7d65bb5c653d5e0d5f4cec2fefcc9d8f390c4c53e4d3c3aed245158110220016744e2de6570c86a09212166ca6008432c5d9292f8315e474b2a220ddad35c"
}

In particular we find the entry which reveals the node of the store:
 "payee": "024a8228d764091fce2ed67e1a7404f83e38ea3c7cb42030a2789e73cf3b341365"

We can query the gossip store for more information about the node with the following command:
lightning-cli listnodes | grep 024a8228d764091fce2ed67e1a7404f83e38ea3c7cb42030a2789e73cf3b341365 -B 1 -C 13
    {
      "nodeid": "024a8228d764091fce2ed67e1a7404f83e38ea3c7cb42030a2789e73cf3b341365",
      "alias": "www.fulmo.org",
      "color": "f2f3f4",
      "last_timestamp": 1561973177,
      "globalfeatures": "",
      "global_features": "",
      "addresses": [
        {
          "type": "ipv4",
          "address": "95.211.207.81",
          "port": 9735
        }
      ]
    },

This gives us all the information we need to peer with the shop and fund a channel.  (In case the node was private we could use the routes hints from the decoded invoice to connect to a public exit node.)
lnd users
If you are using lnd you can use the command DecodePayReq and instead of invoking listnodes to get the information from the gossip store you can invoke GetNetworkInfo

Also note that the payee field in a BOLT 11 invoice is optional. If omitted, the signing node's public key can be recovered from the signature and the recovery id. I am not sure whether the implementations will automatically fill in the payee field in this case when using decodepay/DecodePayReq.

I was not aware that pub keys can be computed from signatures in ecdsa. If others are surprised too look at  https://crypto.stackexchange.com/questions/18105/how-does-recovering-the-public-key-from-an-ecdsa-signature-work to see why / how this is possible.


text:
How do I calculate the expected shares per minute for my pool
I am currently mining Zencash in a pool, and I'd like to be able to calculate the expected shares per minute for a given pool's hashrate. Let's say the pool's hashrate is 50 KSol/s. I think that the shares would need to be standardized to some share difficulty, let's call it one. The current block difficulty is 764,414.6217 (11.0 block reward). I've been digging around and it's hard to find the math involved in this. Does anyone know how to do this for Zencash?

I was able to find some information from this page...
https://arxiv.org/abs/1112.4980
I think I was able to figure out the formula, but somebody please correct me if I am wrong. The formula is not very difficult, but you must know the number of hashes that correspond to a minimum difficulty of 1 (share difficulty). This is easy to find for some currencies (bitcoin, 2^32), but very difficult to find for others. For Zencash, it is 8192 or 2^13. The formula to determine the expected blocks per minute for Zencash is:

blocks_per_time = (hashrate * seconds) / (8192 * block_difficulty)

In our case, the estimated blocks per minute would be:
(50000 * 60) / (8192 * 764414.6217) = 4.791e-4 blocks/min.
Then if we know it takes an average of 764,414.6217 difficulty-1 shares to solve a block, then we can calculate the pool's shares/min as:

shares_per_min = blocks_per_min * block_difficulty

shares_per_min = 4.791e-4 * 764414.6217 = 366.2.
Or, we can simplify the original equation to estimate the given shares/min in a pool based on a given pool hashrate as:

shares_per_min = (hashrate * 60 seconds) / 8192

shares_per_min = (50000 * 60) / 8192. = 366.2
Based on this (and the reason I wanted to know the answer to this question), you can see that if you stop mining on a PPLNS pool with a steady hashrate of 50 kH/s where n=20000, you're estimated reward should fall to zero after about 54 minutes. Of course, your shares are very likely submitted at a higher difficulty than one, but this shouldn't affect that calculation.


text:
In what circumstances would a block appear in the Leveldb .ldb index but have no data in a .dat file?
I've observed this behavior with Bitcoin Core v0.15 while parsing the LevelDB index in .bitcoin/blocks/index. It is also a situation that is obviously accounted for in the Bitcoin Core source code here (See BLOCK_HAVE_DATA and BLOCK_HAVE_UNDO). I am just looking to understand what would cause this to happen? Can a peer receive the block header before receiving the block itself? I'm finding that only the top few blocks in the chain seem to be missing data; They appear in the index but not the .dat files.

Bitcoin Core may only have just the block headers for some blocks, not the full block data. Such block headers would get an entry into the chainstate and block index as a block that the node knows about. However the node does not have the full block data corresponding to that header so there is no corresponding block data in the blk*.dat files.
This situation happens during syncing (since Bitcoin Core will fetch block headers first and then the blocks), when there are orphan blocks, and when there are chain forks.


text:
Bitcoin Core LevelDB Index saves forked blocks
I've noticed that bitcoind will permanently save unverified blocks that it receives even if they turn out to be small blockchain forks, likely occurring from the relatively simultaneous discovery of two blocks of equal height. This doesn't cause a problem for Bitcoin Core's functionality, as there is surely a process written into the source that ignores such small forks, however, it's curious to me that the data persists in the data saved to the .dat and .ldb files in .bitcoin/blocks/ and .bitcoin/blocks/index nonetheless. Is there a way to prune these invalid blocks that belong to small forks in the blockchain without -reindexing the entire blockchain? Would -reindex even remove them?

There’s a negligible impact of that. Less than 1% of blocks are stale.

They're kept because it is complicated to remove them (it would require rewriting the block storage files), and there is hardly any cost to keeping them (they're a small fraction of the total). Furthermore, close to the tip we want to have forks available, in order to be able to reorganize quickly if one of them ends up becoming the main chain.
More technically, the $DATADIR/blocks directory for Bitcoin Core is not a representation of the blockchain. It's a store of all valid blocks we've ever downloaded. Which out of them end up being considered the main chain is a different issue, and dealt with independently.
Also note that in pruning mode all old blocks are removed - including these small forks.

I cannot think of any convenient way to do this.
The only method that comes to mind is to create for yourself a bootstrap.dat file from your own up to date node, then exit Bitcoin Core and manually clear out all the relevant db files before importing from the bootstrap.dat.

Are these small forks called "orphans"? How come they only seem to appear when you've been actually running a node? Is it because node's don't rebroadcast orphaned blocks, but do save them indefinitely?


text:
What is the difference between bitcoin core vs bitcoinj vs bitcoind vs bitcoinqt?
What is the difference between bitcoin-core vs bitcoinj vs bitcoind vs bitcoin-tx vs bitcoin-cli vs bitcoinqt?
There are many github repo with different names around bitcoin.
Can someone really help me understand differences in between them in terms of what they offer?

bitcoind, and bitcoin-qt are all part of the same program. Bitcoind and bitcoin-qt are both Bitcoin Core and do the same thing except that bitcoind has no GUI. Both are part of the Bitcoin Core project and referred to as Bitcoin Core. Both share a large amount of code and are full nodes.
bitcoin-tx is a utility that is part of the Bitcoin Core project that allows you to create, modify, and sign raw transactions.
bitcoin-cli is a utility that is part of the Bitcoin Core project that allows you to interact with the headless (i.e. no stdin input, no gui) bitcoind and a bitcoin-qt if -server is set.
bitcoinj is a Java library for using Bitcoin. It does not have full node capability and is primarily used for Android apps and lightweight clients.


text:
What happens when a transaction is included in more than one block?
What happens when two miners include same or set of transaction in their block, mine it and then broadcast it at same time? Can the transaction be included in two separate block like this OR a fork happens?

It is possible for two miners find blocks at the same height simultaneously, and for those two blocks to have an overlap of included transactions. Each block is valid in it's own right, but eventually only one can be included in the longest chain, and the other will become an 'stale block'. Lets call these two blocks Block A and Block B.
When the blocks are found, the lucky miners will immediately broadcast them to the network. Block propagation has been optimized very well over the years, so in reality it takes very little time for the entire network to hear about a new block (and miners generally stay very well connected to the network), but lets pretend it takes much longer than normal. In this situation, some nodes may hear about Block A first, and other nodes will hear about Block B first. Many nodes follow the rule of 'first seen is valid', so they will reject the second block they hear about (even though it is valid). This creates a fork, since the network is not all agreeing on a common history.
This situation resolves once the next block is found**. Miners run nodes as well, so each miner will begin mining the next block on top of whichever block they heard about first. So if a miner that saw Block A finds the next block, then Block A will be included in the final blockchain history, and Block B will be made stale (and vice versa). This brings the network back in sync, as all nodes that heard about Block B first would drop Block B in order to stay in sync with what is now the longest chain.
** note that it is possible for the situation to repeat, and for a miner on each side of the fork to again find blocks simultaneously. The resolution is no different, but now the stale blocks will be two deep, instead of just one. In practice it is very rare for this situation to occur (it is equal to the probability of a stale block occurring, squared)

But what would that mean to the remaining transactions within the other block? 1 of the 2 blocks is accepted and appended to the chain, right? Let's assume that 50 similar transactions existed in both blocks. Will these be automatically removed from the block that didn't make it and the miner needs to add 50 different ones?

@Oliver: There can be only one block at every height in the chain. As soon as the winning block emerges, every node reorganizes their view of the chain to the best chain-tip, and the stale block is essentially ignored as if it had never existed.

If both blocks are broadcast at the same time, and we assume both miners are equally connected to the network, a fork may occur.
Since the default mining policy is mining on the top of the longest chain, or the first one heard in case of a fork, the network should mine in two different blocks until a new one is found, moment when all should go back to "normal".


text:
Does copay accepts multiple payments?
I am a bit confused as to whether copay wallet accepts multiple payments to a single address based on the phrasing "wallet generates new addresses automatically once you use a bitcoin address once"...if tracking is not an issue I suppose the same address can be sent out to several payees right?

Yes, once you have generated an address, you can have people send any number of transactions to that address from then on.  I just tried it with Copay on testnet and it works fine.
It automatically generates a new address when a transaction arrives, just to encourage you to use fresh ones if possible, for privacy.  But previous ones continue to work indefinitely.
(Interestingly, it generates a new address whenever a transaction is received, even if that transaction was sent to an older address and the previously displayed address hadn't been used at all.)

Thank you Nate, tried it over testnet too. I was trying to see whether the address generation can be applied irrespective of payment received yet...thanks for the feedback


text:
Javascript library to explore bitcoin blockchain data by connecting to a local running node
I need to get data of a block using the block height.
Is there a javascript library to get block data from the Bitcoin blockchain, by connecting to local running node or a 3rd party blockchain provider?

What exact data do you need from the block? Would a simple API call
to e.g. blockchain.info be enough?

@0xb10c Specifically, I want to pull all the transactions that are included in a block. I don't want to be dependent on a third party api provider. It would be better if I could connect to the blockchain by using the js library and query the blockchain directly.

bitcoin tools library
https://github.com/bitpay/bitcore-lib
Unofficial Node.js client for blockexplorer.com API
https://github.com/barto-/blockexplorer
or you can install bitcore full node and use bitcore-lib with that full node

Thanks. I'm using the Bitcoin-Core library, which makes it easy to execute bitcoin-cli commands by simply formatting them to camel case.


text:
List imported watch-only adresses
I can import addresses without private key just to track incoming transactions as it is provided in RPC calls :
https://bitcoin.org/en/release/v0.10.0#watch-only-wallet-support
But how can i see my previously imported addresses? (With or without RPC)
Bonus : Is there any proper way to delete them?

OK. I've found a workaround by adding label on importing :
bitcoin-cli importaddress <address> my_imported_address_list

Checking my label on listing :
bitcoin-cli getaddressesbyaccount "my_imported_address_list"

This solves my problem for now. But still searching for better answer where i dont need to label my own imported addresses.

listreceivedbyaddress can list all addresses (including watch-only) that haven't received, if used as below:
listreceivedbyaddress 0 true true

While this would exclude watch-only addresses:
listreceivedbyaddress 0 true false

And their outputs could be compared to find watch-only addresses


text:
Is it possible to copy BTC balances of only some addresses while forking
I want to fork a Blockchain for private use, however I want to keep only balance of some known adresses
for example if my adresses are @A(10BTC), @B(20BTC), @C(30BTC)
I want to see same adresses with same amount on the new forked one
@A(10BTC), @B(20BTC), @C(03BTC), @D(5BTC), @E(6BTC)
Many thanks

This is possible, but will require a change to your fork's consensus rules to see the block that alters address balances as valid.
Essentially, you need to drop the utxos of addresses you don't want to have a balance on your new fork at the fork block, and leave the ones you want with a balance alone. This is of course an invalid block for regular Bitcoin clients, since utxos cannot arbitrarily disappear. You can get around this by assuming that the fork block hash is valid explicitly in your fork's code.


text:
How are mined blocks broadcasted in a distributed environment?
I understand that after the golden nonce is found by the miner he broadcasts the block and the other miners verify it. But I don't understand how they broadcast it because bitcoin is completely distributed with no servers to store the ip to distribute it to. How does a new miner know whom to distribute it to?
Any help is really appreciated ...

Miners operate nodes (as do many non-miners). These nodes make connections to each other. When a node receives a block or transaction from another node, it will check the block or transaction and then send it to the other nodes it is connected to. In this way, a block or transaction reaches every node in the network.
The miners do not connect to some central server or service in order to broadcast blocks to the network. They simply send the blocks to the nodes that they are connected to and the blocks are then relayed to all nodes.
As for how a node figures out who to connect to, please see this question: How does the bitcoin client make the initial connection to the bitcoin network?

Actually, many miners do connect to a number of central servers or services to help get their blocks distributed as widely and quickly as possible. Punch "bitcoin relay network" into your favorite search engine and you'll find a number of them. These are not necessary, as simple node relaying is sufficient. But the faster you relay, the more likely you are to get your mining reward. So there has been lots of optimization efforts.


text:
how can you calculate the total XRP transaction fees for an offer?
If I place an offer (say BUY 1.0 BTC, SELL 450 USD), how do I know what my expected transaction fees are?
Is the transaction fee applied to the offer, or is the transaction fee applied to each execution/payment that fills the offer?  If its applied per execution/payment, it seems like the associated fees could be extremely large if somehow I ended up with a billion executions/payments (through no fault of my own).

The transaction fee is specified by the signed transaction itself and is consumed when the transaction is first processed, regardless of the transaction type or amount or its success or failure. For example, an offer than is later cancelled takes one transaction fee to place the offer and another to later cancel it and a properly specified payment transaction that fails (lack of funds, lack of destination trust lines, paths dry, etc, etc) still charges the transaction fee. Only transactions that are outright dropped by the server (e.g. badly formatted, invalid in some way, fee too low, etc) return a local server error and do not charge the fee.
The nominal transaction fee is 0.000010 XRP (10 drops) but this is dynamically increased when the server(s) are under load.
It's up to the client to set a sane amount based on the amount the server(s) are currently requesting.
For example, a client could set the fee to the current amount and re-try with a higher fee if it fails, it could set a slightly higher fee than requested to prevent such retries if the fees are changing. It could have a configurable or hard coded maximum beyond which it stops and asks the user if they want to proceed with the high fee or try again later.
Completely unrelated are transit fees that an issuer may have set.


text:
How do I retrieve the funds on my Core wallet when I can't synchronize the blockchain?
I have about $200 in bitcoin but my wallet (Core) won't synchronize the blockchain (it's been doing this for days) since for some reason it got corrupted (As it tends to do say every other month) and now I'm unable to retrieve my money. I want to transfer it to a lightweight wallet so this won't happen anymore but I don't want to lose the money I have locked in there. How do I transfer this money when I can't synchronize to the current state of the blockchain?

You'll want to use bitcoin-cli to dump the priv key and import it into a lightweight wallet. Here's the command to do this in bitcoin core:
$ bitcoin-cli dumpprivkey <address that contains your coins>

This should dump out the WIF format private key.
You can import this private key into something like Mycelium which will then have access to your funds. Typically, the wallet will sweep the funds from the address containing your coins to an address controlled by Mycelium. In any case, you'll have full access to your funds after you do this, though at the cost of some miner fees (about 10 cents)

I'm  trying to run this on a command line promp from both Program Files and %appdata% bitcoin locations but I'm getting 'bitcoin-cli' is not recognized as an internal or external command,
operable program or batch file

bitcoin-cli was added in version 0.9, so if you are using an earlier version you'll need to do this with Bitcoin-QT instead (easiest from the debug console).   If you are on v0.9 or later just make sure you have done a cd to the Bitcoin directory and run it from there (or if Linux, add prefix to load the program from the current path: ./bitcoin-cli ).

I'm trying to run dumpprivkey myAddress from console but the it asks for walletpassphrase before. I've tried tke phrase I use to make transfers but it gives me the error "parse: have" ("have" is the second word I use in my passphrase for transferring funds) so I'm lost, how can I find this walletpassphrase?

If you have multiple words in your wallet passphrase (or a space somewhere), quote the entire phrase. So c:\> bitcoin-cli.exe walletpassphrase "<your passphrase>".

I am pasting the entire phrase but it gives me this "parse: have" error. "have" happens to be the second words in the phrase. This happens from the Core console. If I use terminal command "bitcoin-cli dumpprivkey myAddress", then it shows me the message "error: couldn't connect to server"

@Tom Jimmy means you need to put your passphrase in quotes after you paste it in. So if your passphrase is I have Bitcoins, then you have to put it in quotes, like "I have Bitcoins". The quotes tell the computer to treat all the words inside as a string of characters or letters, not a command that gets parsed.


text:
Can I run bitcoind + lnd instances on my home-based computer to be publicly accessible?
Technically, I understand, I can start bitcoind and actually already running it on my home-based small server and lnd works with it quite well.
However, I wonder, how reliably this is in regards of speed for users around the world. Does it demand real high speeds or can I run it on my 260mbit/s down and 90mbit/s up connection with 43ms ping? The connection itself is stable.

Could you clarify what exactly you want to make publicly accessible?

@Murch I am building a webapp which makes transactions via Lightning network and uses LND + bitcoind to do so. So, basically, nothing would be accessible publicly besides some API endpoints of the webapp which would be used by the browser client.


text:
Tampering with the last block
In a blockchain, considering that each block stores the hash of the previous block, the hash of the last block is not stored anywhere. How is this block tamperproof?

Proof of work requires that in order for a block to be replaced with an alternate a sufficient PoW is required. Using an alternate mining operation the equivalent power of the rest of the global mining (yes, all of it) it would be possible in theory to do this in an average of ten minutes by which time there would be on average another new block to build upon meaning that the attacker would also have to build an additional two blocks in order, by then, to present a longer chain.
So, that's an average of 30 minutes work using more mining hash power than exists in all of the rest of the network in order to complete the work in less than approximately 20 minutes after first receiving the block to be tampered with.
This is effectively a 51% attack. With a 51% attack, this would, theoretically at least, be possible.
Occasionally, two blocks are found and distributed to the network at the same time. The one that is built on first is the blockchain, the one that is not is orphaned. It is pretty much foolproof.

The hash that identifies a block is calculated from the block header. This hash must fulfil the difficulty requirements. Every node will check any block that they receive for validity. Since it requires a lot of proof of work to create a block whose hash fulfils the difficulty statement, it's infeasible to fake blocks and trivial to verify valid blocks.


text:
How do I know the public key of the person I need to pay?
As per the answer to this question, we cannot retrieve the public key of a user.1)How does my wallet get the value (public key/public key hash) to which I need to make the payment?2)What is the harm in exposing an address's public key?

Second question answered here.  In general, please do not ask multiple questions in a single post.

I read through both questions, so please correct me if im answering the wrong question.

The user who you want to pay must send you their public key (or QR to scan) in order to send money to them. This is, of course, unless their public key (wallet) is listed on a public website. Please see https://en.bitcoin.it/wiki/Help:Introduction#Sending_payments
In general*, there is no harm in exposing a public key. This question on crypto stackexchange goes into more detail Is exposing a public key bad?

That person has to tell you what their public key (or its hash) is.  A Bitcoin address is the most convenient way to encode the hash of a public key, so typically the person will tell you an address and ask you to make a payment to that address.  You can enter that address into your wallet and make the payment.
Related: How can a sender know receiver's public key when creating "Pay To Public Key" tx?

If you are using bitcoind, you can get new receivable address using bitcore-cli getnewaddress command


text:
What is a bitcoin incoming transaction with value = 0.00000546?
I wrote a script that downloads and parses blocks. I see a lot of incoming payments with value = 0.00000546. Any explanation?
Thanks!

This output value corresponds to the dust limit that is imposed on bitcoin transactions.
Many of these outputs may have been generated as a dust attack to de-anonymize users/ addresses.

In addition to Chytrik's mention of a dust attack, dust limit transactions are also used heavily by the Omni Layer to identify the recipient of an omni asset. Transactions such as those will follow pattern of one dust (or near dust) output, an OP_RETURN output, and a change output.
While dust attacks tend to create thousands of dust outputs in a handful of transactions, omni layer transactions will only create a single dust output per transaction.

I would say that a Dust Attack on Bitcoin is the most dangerous system that hackers use to steal Bitcoin. According to this article:
https://dust-attack.blogspot.com/2019/03/blog-post.html?m=1

I don't think dust attack can help steal bitcoin. May only be used to de-anonymize users.

It is in this transaction: https://www.blockchain.com/btc/tx/1d6580dcd979951bd600252b741c22a3ea8e605e43168f8452c68915c3ea2bf3
With the help of a dusty attack, a lot of coins were stolen.

How do you know it's stolen?


text:
Blockchain Payment receive api
While using blockchain receive payment api, what if some one send less then require fund or some one send extra fund on same one time generated address.
does fund will automatically refund to sender or not ?
what to do if i want to refund that amount back to user who send less fund or double fund on that address?
any one have information regarding this?

I dont know which api you are talking about , but i am this much sure that ones your transaction is minned and updated in block you cannot revert it or refund it. Blockchain data is immutable. If you want to restrict your client form sending low or high then you have to use some business logic on backend of your code also using JS logic on front end when user inputs the value will help you alot

I am talking about blockchain.info receive payment API.

I have seen on coinpayments.net it's a wallet and when some one send less payment when you are using coinpayments.net api so it's refund that less send amount and transaction status shows canceled and when some one send extra so it's shows it's also refund extra amount

It depends on what you are taking the payment for. Say it is an ecommerce store accepting btc and the  payment is less than what was asked for, then refund is the only option.
If say this is for some ICO Platform you are making, then you can calculate the corresponding number of tokens for that much amount of btc and complete his transaction. You will obviously have to post a disclaimer for the same when the payment is being initiated.
In case of an exchange, the amount is undecided so this question not relevant there.
As cryptoKTM mentioned it entirely depends on the business logic.


text:
what happens if i sent bitcoin to a clubcoin address (address starts with a C)?
what happens if i sent bitcoin to a clubcoin address (address starts with a C)? will i get refunded and get notified the transaction was invalid? please help me!

There are no Bitcoin addresses defined that start with a C (only P2PKH addresses which start with 1, P2SH addresses which start with 3, and P2WSH or P2WPKH addresses which start with bc1).
As a result, it is impossible to even create a transaction to such an address. There is nothing to refund of nofify - it's simply impossible to perform.

Depending the wallet, it's possible they simply converted the clubcoin address to the hex version and did the tx anyways - OP might be able to retrieve the coins by importing the private key for the clubcoin address into electrum or something, I've helped people in similar situations in the past

in ordee to send BTC, you need a BTC client. How would such software even know how to interpretba clubcoin addrss?

I've seen it happen with exchanges a few times - Some wallets/exchanges will accept any base58 address, since most coins just change the version byte, and interpret it as a regular btc p2pkh. The vertcoin webwallet accidentally burned a bunch of vtc this way when sending to segwit addresses some time ago. Sometimes, if you have the private key, you can get to the same address on the BTC chain (of course, it'll look different because of the version byte, but if the coin didn't change too much from BTC, you can still spend it)

But you're talking about altcoin software that did something crazy. In order to explain OP's behaviour, you need a Bitcoin wallet that somehow accepts a crazy address format. I don't think any wallet software out there is insane enough to act this way.

Yeah, I can't think of a btc wallet that's done that anytime recently off the top of my head. Just pointing out it's possible. If OP has a txid, the tx happened, so something somewhere messed up, and he might be able to get it back. If there's no txid, then the coins are still where they used to be

Thanks so much. I got freakingly worried as I accidentally sent 2600 worth usd to a clubcoin address! It was pending for a day (24hrs) and deducted my funds; and thank enough, funds are back! whew! thanks all.


text:
Not able to use bitcoin/bitcoin.hpp
the libbitcoin library seems to have been installed just fine.
but the following code can't seem to include bitcoin/bitcoin.hpp
#include<bitcoin/bitcoin.hpp>
#include<iostream>
using namespace bc;
int main() {
    block_type blk = genesis_block();
    std::cout<<encode_hex(hash_block_header(blk.header))<<std::endl;
    return 0;
}

Error:
g++ -o test test.cpp $(pkg-config --cflags --libs libbitcoin)
test.cpp: In function ‘int main()’:
test.cpp:5:5: error: ‘block_type’ was not declared in this scope
     block_type blk = genesis_block();
     ^~~~~~~~~~
test.cpp:5:5: note: suggested alternative: ‘clock_t’
     block_type blk = genesis_block();
     ^~~~~~~~~~
     clock_t
test.cpp:6:45: error: ‘blk’ was not declared in this scope
     std::cout<<encode_hex(hash_block_header(blk.header))<<std::endl;
                                             ^~~
test.cpp:6:45: note: suggested alternative: ‘brk’
     std::cout<<encode_hex(hash_block_header(blk.header))<<std::endl;
                                             ^~~
                                             brk
test.cpp:6:27: error: ‘hash_block_header’ was not declared in this scope
     std::cout<<encode_hex(hash_block_header(blk.header))<<std::endl;
                           ^~~~~~~~~~~~~~~~~
test.cpp:6:16: error: ‘encode_hex’ was not declared in this scope
     std::cout<<encode_hex(hash_block_header(blk.header))<<std::endl;
                ^~~~~~~~~~

I was able to run this code:
#include <bitcoin/bitcoin.hpp>

int main()
{
  bc::ec_secret decoded;
  bc::decode_base16(decoded, "038109007313a5807b2eccc082c8c3fbb988a973cacf1a7df9ce725c31b");
}

Follow this https://github.com/libbitcoin/libbitcoin#debianubuntu
After compile it in a /home/x/dir run this:
g++ helloworld.cpp -I/home/dev/bitcoin-lib/include /home/dev/bitcoin-lib/lib/libbitcoin.a /home/dev/bitcoin-lib/lib/libboost_system.a /home/dev/bitcoin-lib/lib/libbitcoin.a /home/dev/bitcoin-lib/lib/libboost_program_options.a /home/dev/bitcoin-lib/lib/libboost_regex.a

we need to tell g++ where are the headers for our #include statement
We need to tell g++ what are the libraries: libbitcoin.a, etc

Don't forget -lgmp when it happens

The book code examples has an accompanying actively maintained github repo here. However, extra care is needed for the libbitcoin library dependency as it has new breaking changes. For example, #include <bitcoin/bitcoin.cpp> has been rename to #include <bitcoin/system.hpp>. And no more bc::ec_point. It is possible to install an older compatible version of libbitcoin library, but I prefer to use the latest version and made some adjustment in addr.cpp example code.
I manually build and installed the libbitcoin library to a custom location as suggested:
mkdir -p /opt/my/libbitcoin/
$ wget https://raw.githubusercontent.com/libbitcoin/libbitcoin/version3/install.sh
$ chmod +x install.sh
$ ./install.sh --prefix=/opt/my/libbitcoin/ --build-boost --disable-shared

The way I now build addr.cpp is thus:
export PKG_CONFIG_PATH=/opt/my/libbitcoin/lib/pkgconfig/
g++ -o addr addr.cpp $(pkg-config --cflags --libs libbitcoin-system) -lgmp

And then, running the executable:
$ ./addr
Public key: 0202a406624211f2abbdc68da3df929f938c3399dd79fac1b51b0e4ad1d26a47aa
Address: 1PRTTaJesdNovgne6Ehcdu1fpEdX7913CK

Try adding the -std=c++11 flag.
I had the same problem and the following worked for me:
g++ -std=c++11 -o test test.cpp $(pkg-config --cflags --libs libbitcoin)

it didn't work!


text:
Do multiple miners contribute to mining the same block even if they are not in the same pool
Assuming that this is reliable, according to an article on investopedia on the 51 percent attack:

51% attack refers to an attack on a blockchain – usually bitcoin's, for which such an attack is still hypothetical – by a group of miners controlling more than 50% of the network's mining hashrate, or computing power. The attackers would be able to prevent new transactions from gaining confirmations, allowing them to halt payments between some or all users.

Does the above bold line imply that all the miners are required to provide their confirmation for a transaction to be added to the blockchain? I was under the impression that even an individual could, if he completes the proof of work first, cause the addition of the block to the blockchain? What am I missing here?

I was under the impression that even an individual could, if he completes the proof of work first, cause the addition of the block to the blockchain? What am I missing here?

That it could happen that 2 blocks are found at the same time. Which usually causes the network to split into two chains, ( One part of the network is going to build/find a new hash based on block y, and the other part of the network is trying to find a new valid hash based on block X.).
Whoever finds Block Z first creates the longest chain and thus invalidating the shorter one. ( Miners who build on block Y find block Z so now block X ( which is the shortest chain) gets orphaned )= now invalid.
The same principle applies to a 51% attack, since you have majority hashpower you will always (hypothetically) find a valid hash faster then the other chain which has 49% hashpower.
In that case, no individual ( unless the individual owns 51% of the hashpower) is going to be able to "mine" a block/(or rather, - get a block added to the longest/valid chain).
Your title is something else then your question,

Do multiple miners contribute to mining the same block even if they are not in the same pool

Well, yes, if there's only 1 chain, every miner is going to be building upon that chain, which means they'll all be looking for a block ( or rather a hash) that is going to be valid on that chain ( basically the next block?)
I wouldn't use contribute though, it's not like they share their hashpower, but they do indeed share the same goal. -> Finding a block with a valid hash that is valid on the current chain.


text:
What is the role of miners in Blockchain?
What is the need of validating and authorizing a BTC transaction.
Say, for eg, A have 5 BTC in its wallet and he wants to send 3 BTC to B. What is the need to validate such a transaction?
And in case A wants to send 7 BTC to B, he simply cannot because he doesnt have the sufficient balance.
What is the need to validate then.
Why cant it be possible that the transactions are automatically added as a next block on the blockcahin on all the nodes?

Welcome to Bitcoin.SE!
It is necessary to validate all transactions since it is possible otherwise for A to just broadcast a transaction sending 7 BTC to B even if he does not have enough. A could easily write his own program to do so.
Validating all transactions and the entire blockchain is necessary for Bitcoin to have any value and to ensure consensus rules are followed.

No, A can't send more money that he/she has in a wallet, and we don't need miners to validate that. All nodes validate that.
Miners don't actually 'validate' anything more than all the other full nodes, this is a common misconception coming from popular news sites. Miners are there to prevent double-spending. When a transaction is included in a block it's called 'confirmed'.

@OsiasJota And, because the mempool is not consensus. Without the blockchain, the order of transactions cannot be agreed upon. The blockchain is the record of all transactions, without the miners we do not have a blockchain and there is no way for nodes to decisively know or agree who has what balance.

Unless you think that the OP wonders in the case if all nodes instantly 'mine' any received transaction into a personal blockchain, but then, still no consensus.

This is not about 'to validate', it's about 'to confirm'.

Only valid transactions can be confirmed.

So who validates the transactions?

@DanielMorritt Under normal circumstances, all full nodes validate the transaction against consensus rules. Without the blockchain it is impossible to know if a transaction meets all consensus rules.

First of all, it must be clear that there isn't just one chain: every node in the network has its own local chain stored locally. Through message exchanges, all the nodes chains would eventually end up synchronized.
Now imagine the situation in which there are no miners and Alice wants to send 1 BTC to Bob. She sends the transaction to a node in the network and the node updates its local chain with the new transaction.
In the meantime, Alice sends the very same Bitcoin to Charlie but this time she notifies a different node of the network with a new transaction, and that node will properly update its local chain with the new transaction. Now both nodes have two different transactions in their local chain. They start sending messages to their neighbour nodes telling that new transaction have been arrived. The nodes will receive updates from both the nodes but will not be able to discriminate which is the legit spending.
Mining is there because in this way all the nodes can agree on the same set of transactions: finding a solution to the hashing problem will guarantee (at least, most of the time) that all the nodes chains agree on the same set of transactions. This happens because only the one who finds the Proof of Work can add and propagate a block in the network. Finding the Proof of Work is a hard task, and due to its randomness nature it is probable that only one miner will  find a Proof of Work.
By the way, this question has been asked several times, i suggest you to take a read from the official Bitcoin Wiki that will enlight you on the features of a blockchain.


text:
Why for creating a new test coin is suggested to use Litecoin code?
I think I'd prefer to use Bitcoin code. Is there a reason that LiteCoin is preferred?
As far as I know, for creating a different coin from BTC source I need to create a genesis block and remove the checkpoints. And obviously change any important parameter that I think should be changed (block rewards, etc.).
Is anything else needed? Maybe it's hard to create a genesis block for Bitcoin?

who said that?!?

where did you get this assumption?

I've found that suggestion in the "BitcoinTalk" forum:
https://bitcointalk.org/index.php?topic=225690.0
But I've just realized is on the "altcoin discussion" forum.
Anyway, I haven't found a good tutorial on forking Bitcoin (specially the genesis block part) but this, related with LiteCoin, was easier to find.


text:
BTC Core not syncing, maybe becacuse it has zero blocks?

I have two BTC Core nodes modified with custom genesis blocks. I'm connecting them using the -connect option and the -listen option, and I can see in debug.log that they see each other:
2018-03-21 19:49:42 New outbound peer connected: version: 70015, blocks=0, peer=5

And:
2018-03-21 19:49:40 New outbound peer connected: version: 70015, blocks=0, peer=0

But, as shown in the image, the client is not able to sync. Do I need to mine at least one block so BTC Core can finish the synchronization, or is this a sympton of other problem?

I think I've found the answer (but I'm not 100% sure). I had to change this:
chainTxData = ChainTxData{
        // Data as of block 0000000000000000002d6cca6761c99b3c2e936f9a0e304b7c7651a993f461de (height 506081).
        // 1521311547, // * UNIX timestamp of last known number of transactions
        // 0,  // * total number of transactions between genesis and that timestamp
        //             //   (the tx=... number in the SetBestChain debug.log lines)
        // 3.5         // * estimated number of transactions per second after that timestamp
    };

To this in chainparams.cpp:
    chainTxData = ChainTxData{
        0,
        0,
        0
    };

That seem to have solved the issue and now I see "Progress 100%". It's the only thing that changed, the other things "Progress increase per hour", "Number of blocks left", "3 days behind" are still the same but the progress seems to have changed after I modified that section of the code.
UPDATE: When trying to use cpuminer-multi to mine, I receive this message:
JSON protocol response:
{
   "error": {
      "code": -10,
      "message": "Bitcoin is downloading blocks..."
   },
   "result": null,
   "id": 0
}

So it looks I was wrong and even if it says 100% it didn't finish the sync. It's because the chain has zero blocks? It doesn't count the genesis block for some reason?