text:
How can one examine the contents of the wallet.dat file using sqlite3?
These two questions (1, 2) discuss what is in wallet.dat. Andrew Chow states in 2017 (when the database was still BDB and not sqlite):

The wallet.dat file contains your private keys, public keys, scripts (which correspond to addresses), key metadata (e.g. labels), and the transactions related to your wallet. If you have an HD wallet, it also includes the HD seed and and the derivation paths for each private key.

I also found a 2011 Bitcointalk post discussing using the bitkeys format for the first time where there were 4 classes of keys:

(1) Used keys, which have unredeemed transaction outputs linked to them (and thus contain funds)
(2) Used keys, which only have redeemed transaction outputs linked to them (worthless, but the addresses may be used still by others)
(3) Unused keys (whose corresponding addresses may have been published)
(4) Reserve keys (whose corresponding addresses have typically never been divulged, but may have been divulged at the time the export is imported again)

I am trying to inspect the contents of wallet.dat using sqlite3. I've generated a (signet) Bitcoin address and sent some (signet) Bitcoin to this address. I cd into ./bitcoin/signet/wallets/insert_wallet_name. Then I open wallet.dat with sqlite3 (I have pre-installed sqlite3).
When bitcoind (-signet) is running I get the error message that the database is locked when I try a .dump.
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
/**** ERROR: (5) database is locked *****/
ROLLBACK; -- due to errors

I initially assumed wallet.dat would only be locked when changes were being made to it e.g. it was updating to append a new transaction that was relevant to the wallet. Why does wallet.dat need to be locked permanently if bitcoind is running?
Then I stop bitcoind (./bitcoin-cli -signet stop)
I can now do a .dump on wallet.dat
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE main(key BLOB PRIMARY KEY NOT NULL, value BLOB NOT NULL);
INSERT INTO main VALUES(X'0776657273696f6e',X'3cf70300');
INSERT INTO main VALUES(X'0a6d696e76657273696f6e',X'ac970200');
INSERT INTO main VALUES(X'1377616c6c657464657363726970746f726b657942acd6e8276af09714363d52c0b9cc0fe11e9754a1434dc946ae5cface386a072103f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad16',X'd63081d3020101042074f01c223562a336a90531c0bf75d3a68c3749675fda4d5047520e31f3651038a08185308182020101302c06072a8648ce3d0101022100fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f300604010004010704210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798022100fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141020101a12403220003f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad165aee6f1ffa219adaa51dbd256a15208a416e511b7e3940185ab86ca3947e664f');
INSERT INTO main VALUES(X'1577616c6c657464657363726970746f72636163686542acd6e8276af09714363d52c0b9cc0fe11e9754a1434dc946ae5cface386a0700000000',X'4a04a71ffc7b0000000048855f27d0303b9d670fdcf6b8f979db9c1298a7cb869b3b587ed18a25e1ab6903eabd3dde71174e1308de895cf5e5497b0e3c96a4613544ee4d8cda58258aa1ad');
INSERT INTO main VALUES(X'1777616c6c657464657363726970746f726c68636163686542acd6e8276af09714363d52c0b9cc0fe11e9754a1434dc946ae5cface386a0700000000',X'4a03f4318f8b800000004e8fb53fef11968a0b85695857eab14f96db62f63bf847a1c4a9bdbd09e69dff02a7b1be2f7be89d1d0f31536d5d9a31254a458c1873a7ddb37927b21bd3dc7b44');
INSERT INTO main VALUES(X'1161637469766565787465726e616c73706b00',X'42acd6e8276af09714363d52c0b9cc0fe11e9754a1434dc946ae5cface386a07');
INSERT INTO main VALUES(X'1377616c6c657464657363726970746f726b6579998efcbca89c9e9a166f185fc9f0290e1c5827fba40f3955dc8f14a6dac833c32103f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad16',X'd63081d3020101042074f01c223562a336a90531c0bf75d3a68c3749675fda4d5047520e31f3651038a08185308182020101302c06072a8648ce3d0101022100fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f300604010004010704210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798022100fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141020101a12403220003f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad165aee6f1ffa219adaa51dbd256a15208a416e511b7e3940185ab86ca3947e664f');
INSERT INTO main VALUES(X'1577616c6c657464657363726970746f726361636865998efcbca89c9e9a166f185fc9f0290e1c5827fba40f3955dc8f14a6dac833c300000000',X'4a0438c061be00000000ecfbb5e86f07637492fae9c970d2a626aa30f06adbeb98c00273bc2fc7619f8902b5d54090c152e636b5d0576dbacd25bafd08e6cb7e6d5701235c451e351e619a');
INSERT INTO main VALUES(X'1777616c6c657464657363726970746f726c686361636865998efcbca89c9e9a166f185fc9f0290e1c5827fba40f3955dc8f14a6dac833c300000000',X'4a037e36c8c080000000f3c0b8f779be115a89ae76d2b2d77598978c5a2816cf3c0ef981c779741df80f034b6f4a99c316475a2e5191dc1f09cc02fe3e9e958094135ecdc167d65e77d634');
INSERT INTO main VALUES(X'1161637469766565787465726e616c73706b01',X'998efcbca89c9e9a166f185fc9f0290e1c5827fba40f3955dc8f14a6dac833c3');
INSERT INTO main VALUES(X'1377616c6c657464657363726970746f726b65792b1b9b48fd74166775b6778d559f8d54d31343e120e6188d195f5c45ea185b642103f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad16',X'd63081d3020101042074f01c223562a336a90531c0bf75d3a68c3749675fda4d5047520e31f3651038a08185308182020101302c06072a8648ce3d0101022100fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f300604010004010704210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798022100fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141020101a12403220003f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad165aee6f1ffa219adaa51dbd256a15208a416e511b7e3940185ab86ca3947e664f');
INSERT INTO main VALUES(X'1577616c6c657464657363726970746f7263616368652b1b9b48fd74166775b6778d559f8d54d31343e120e6188d195f5c45ea185b6400000000',X'4a049cb9df5a00000000b552d1c36ac23c78a730608104693b7915694e5ab03ff34f8d8d706ee73ec7e00285d14224e669b80f6f4149705f8dbc508c29c4dbed94a5ee2c96045c32f0632b');
INSERT INTO main VALUES(X'1777616c6c657464657363726970746f726c6863616368652b1b9b48fd74166775b6778d559f8d54d31343e120e6188d195f5c45ea185b6400000000',X'4a03a4c293f9800000004317022d155cbb10052ff5786e89e5c196f4a1a57bdfdef9511c5507c78764d403b71bc893e32be5177635fd964c70bab9edb84c2240228153010946553a18ad27');
INSERT INTO main VALUES(X'1161637469766565787465726e616c73706b02',X'2b1b9b48fd74166775b6778d559f8d54d31343e120e6188d195f5c45ea185b64');
INSERT INTO main VALUES(X'1377616c6c657464657363726970746f726b6579dfa720e29640a23d214c9d47582725e4e3cb71b54ae9c176e34cbd82b92ab7d32103f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad16',X'd63081d3020101042074f01c223562a336a90531c0bf75d3a68c3749675fda4d5047520e31f3651038a08185308182020101302c06072a8648ce3d0101022100fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f300604010004010704210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798022100fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141020101a12403220003f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad165aee6f1ffa219adaa51dbd256a15208a416e511b7e3940185ab86ca3947e664f');
INSERT INTO main VALUES(X'1577616c6c657464657363726970746f726361636865dfa720e29640a23d214c9d47582725e4e3cb71b54ae9c176e34cbd82b92ab7d300000000',X'4a04032caab500000000d10bc88cb19ac535d760acb590fa6ce507e30f3c08d29cac208d3a9ace855c04030c3d3220cf2e2103cb84f985c7c9b39d5eb59ec898bcc40eb0638a0345579d81');
INSERT INTO main VALUES(X'1777616c6c657464657363726970746f726c686361636865dfa720e29640a23d214c9d47582725e4e3cb71b54ae9c176e34cbd82b92ab7d300000000',X'4a03f70b175e80000000e3069525a4804e1a9d49bdac616a69b60b6be027b3f56a0ff8591687b455c530035ea0a5fa512f2eabd38a435f69a46e29bd207307700f47e97b69a8ed82066105');
INSERT INTO main VALUES(X'1161637469766565787465726e616c73706b03',X'dfa720e29640a23d214c9d47582725e4e3cb71b54ae9c176e34cbd82b92ab7d3');
INSERT INTO main VALUES(X'1377616c6c657464657363726970746f726b65793e8e6fe5fb928e0578c5287c7f5f1cedf633a6bb0bf8c24584d42b6f243bffae2103f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad16',X'd63081d3020101042074f01c223562a336a90531c0bf75d3a68c3749675fda4d5047520e31f3651038a08185308182020101302c06072a8648ce3d0101022100fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f300604010004010704210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798022100fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141020101a12403220003f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad165aee6f1ffa219adaa51dbd256a15208a416e511b7e3940185ab86ca3947e664f');
INSERT INTO main VALUES(X'1577616c6c657464657363726970746f7263616368653e8e6fe5fb928e0578c5287c7f5f1cedf633a6bb0bf8c24584d42b6f243bffae00000000',X'4a04a71ffc7b00000001fa40dc9524fd202c8c9581392d91eea09eb0b6ef79333582d78f0ba57ce1ede4036133839bc4ec08d40d7fde8eaa27dfd28b5912bd960a4a32cc1f0af758fe7d32');
INSERT INTO main VALUES(X'1777616c6c657464657363726970746f726c6863616368653e8e6fe5fb928e0578c5287c7f5f1cedf633a6bb0bf8c24584d42b6f243bffae00000000',X'4a03f4318f8b800000004e8fb53fef11968a0b85695857eab14f96db62f63bf847a1c4a9bdbd09e69dff02a7b1be2f7be89d1d0f31536d5d9a31254a458c1873a7ddb37927b21bd3dc7b44');
INSERT INTO main VALUES(X'11616374697665696e7465726e616c73706b00',X'3e8e6fe5fb928e0578c5287c7f5f1cedf633a6bb0bf8c24584d42b6f243bffae');
INSERT INTO main VALUES(X'1377616c6c657464657363726970746f726b6579d05dd81a6c47fca61e3c361c06268406b5ac627e32f9e01f7a17ecbb5ed237372103f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad16',X'd63081d3020101042074f01c223562a336a90531c0bf75d3a68c3749675fda4d5047520e31f3651038a08185308182020101302c06072a8648ce3d0101022100fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f300604010004010704210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798022100fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141020101a12403220003f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad165aee6f1ffa219adaa51dbd256a15208a416e511b7e3940185ab86ca3947e664f');
INSERT INTO main VALUES(X'1577616c6c657464657363726970746f726361636865d05dd81a6c47fca61e3c361c06268406b5ac627e32f9e01f7a17ecbb5ed2373700000000',X'4a0438c061be000000016b6acfd8ecfe4913307fc51a174be60ad505231bc8ea7a21184377e9feecba67024df1fcddacbe9a87640d9e75dcb871c4aa9891d038a9af294415d2ade6f10a67');
INSERT INTO main VALUES(X'1777616c6c657464657363726970746f726c686361636865d05dd81a6c47fca61e3c361c06268406b5ac627e32f9e01f7a17ecbb5ed2373700000000',X'4a037e36c8c080000000f3c0b8f779be115a89ae76d2b2d77598978c5a2816cf3c0ef981c779741df80f034b6f4a99c316475a2e5191dc1f09cc02fe3e9e958094135ecdc167d65e77d634');
INSERT INTO main VALUES(X'11616374697665696e7465726e616c73706b01',X'd05dd81a6c47fca61e3c361c06268406b5ac627e32f9e01f7a17ecbb5ed23737');
INSERT INTO main VALUES(X'1377616c6c657464657363726970746f726b6579c56da0ec8d553443e27cf2439fd158b8aed74e22606b55476dfc9ea4c5ea64742103f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad16',X'd63081d3020101042074f01c223562a336a90531c0bf75d3a68c3749675fda4d5047520e31f3651038a08185308182020101302c06072a8648ce3d0101022100fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f300604010004010704210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798022100fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141020101a12403220003f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad165aee6f1ffa219adaa51dbd256a15208a416e511b7e3940185ab86ca3947e664f');
INSERT INTO main VALUES(X'1577616c6c657464657363726970746f726361636865c56da0ec8d553443e27cf2439fd158b8aed74e22606b55476dfc9ea4c5ea647400000000',X'4a049cb9df5a0000000122601970b9ba136689c5f57eae67b04d5212bff3cda47da08d4e9830ae25eac1039ee257e4d1d60d2feca52ae5da6e5747d2d58297d7ce4d47501c72f1170e16b2');
INSERT INTO main VALUES(X'1777616c6c657464657363726970746f726c686361636865c56da0ec8d553443e27cf2439fd158b8aed74e22606b55476dfc9ea4c5ea647400000000',X'4a03a4c293f9800000004317022d155cbb10052ff5786e89e5c196f4a1a57bdfdef9511c5507c78764d403b71bc893e32be5177635fd964c70bab9edb84c2240228153010946553a18ad27');
INSERT INTO main VALUES(X'11616374697665696e7465726e616c73706b02',X'c56da0ec8d553443e27cf2439fd158b8aed74e22606b55476dfc9ea4c5ea6474');
INSERT INTO main VALUES(X'1377616c6c657464657363726970746f726b65792cf50eb58604c5e8fc815855e9b1c94f1ed98e0923e8479ed68d30dfcf445f392103f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad16',X'd63081d3020101042074f01c223562a336a90531c0bf75d3a68c3749675fda4d5047520e31f3651038a08185308182020101302c06072a8648ce3d0101022100fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f300604010004010704210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798022100fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141020101a12403220003f41276cb64db241c0db73aa721c18c9878c5425d03f3afe608988ef96796ad165aee6f1ffa219adaa51dbd256a15208a416e511b7e3940185ab86ca3947e664f');
INSERT INTO main VALUES(X'1577616c6c657464657363726970746f7263616368652cf50eb58604c5e8fc815855e9b1c94f1ed98e0923e8479ed68d30dfcf445f3900000000',X'4a04032caab500000001291c53f60dd0f14781b11b650a1a4903e93e3e357163a64620e21f946c7d26ee034250539a23c870d479ac7e51b68c25c836e1205be06230ea3ecc06f89ee4fb9a');
INSERT INTO main VALUES(X'1777616c6c657464657363726970746f726c6863616368652cf50eb58604c5e8fc815855e9b1c94f1ed98e0923e8479ed68d30dfcf445f3900000000',X'4a03f70b175e80000000e3069525a4804e1a9d49bdac616a69b60b6be027b3f56a0ff8591687b455c530035ea0a5fa512f2eabd38a435f69a46e29bd207307700f47e97b69a8ed82066105');
INSERT INTO main VALUES(X'05666c616773',X'0000000004000000');
INSERT INTO main VALUES(X'11616374697665696e7465726e616c73706b03',X'2cf50eb58604c5e8fc815855e9b1c94f1ed98e0923e8479ed68d30dfcf445f39');
INSERT INTO main VALUES(X'1077616c6c657464657363726970746f7242acd6e8276af09714363d52c0b9cc0fe11e9754a1434dc946ae5cface386a07',X'8b706b68287470756244364e7a56626b7259685a34586e426b616a31614a6272674a64623567537545517972717138477562385244564c334a6d3443487471347a664d4e3278654c515158715348425354433966706d58505448586171524d5477555879446e7436574c42676a727365636472382f3434682f31682f30682f302f2a292368346c6370647878537bcb64000000000000000000000000e8030000');
INSERT INTO main VALUES(X'1077616c6c657464657363726970746f723e8e6fe5fb928e0578c5287c7f5f1cedf633a6bb0bf8c24584d42b6f243bffae',X'8b706b68287470756244364e7a56626b7259685a34586e426b616a31614a6272674a64623567537545517972717138477562385244564c334a6d3443487471347a664d4e3278654c515158715348425354433966706d58505448586171524d5477555879446e7436574c42676a727365636472382f3434682f31682f30682f312f2a29237870366575636b37547bcb64000000000000000000000000e8030000');
INSERT INTO main VALUES(X'1077616c6c657464657363726970746f722cf50eb58604c5e8fc815855e9b1c94f1ed98e0923e8479ed68d30dfcf445f39',X'8a7472287470756244364e7a56626b7259685a34586e426b616a31614a6272674a64623567537545517972717138477562385244564c334a6d3443487471347a664d4e3278654c515158715348425354433966706d58505448586171524d5477555879446e7436574c42676a727365636472382f3836682f31682f30682f312f2a292338747a70786c3833547bcb64000000000000000000000000e8030000');
INSERT INTO main VALUES(X'1077616c6c657464657363726970746f72d05dd81a6c47fca61e3c361c06268406b5ac627e32f9e01f7a17ecbb5ed23737',X'9073682877706b68287470756244364e7a56626b7259685a34586e426b616a31614a6272674a64623567537545517972717138477562385244564c334a6d3443487471347a664d4e3278654c515158715348425354433966706d58505448586171524d5477555879446e7436574c42676a727365636472382f3439682f31682f30682f312f2a292923376e36306d307a66547bcb64000000000000000000000000e8030000');
INSERT INTO main VALUES(X'1077616c6c657464657363726970746f72998efcbca89c9e9a166f185fc9f0290e1c5827fba40f3955dc8f14a6dac833c3',X'9073682877706b68287470756244364e7a56626b7259685a34586e426b616a31614a6272674a64623567537545517972717138477562385244564c334a6d3443487471347a664d4e3278654c515158715348425354433966706d58505448586171524d5477555879446e7436574c42676a727365636472382f3439682f31682f30682f302f2a29292363736a32717a6661537bcb64000000000000000000000000e8030000');
INSERT INTO main VALUES(X'07707572706f73652a74623171786a6e71756a3974703036743374656361747a717876796c703667306538326d326b61386c6c',X'0772656365697665');
INSERT INTO main VALUES(X'046e616d652a74623171786a6e71756a3974703036743374656361747a717876796c703667306538326d326b61386c6c',X'00');
INSERT INTO main VALUES(X'07707572706f73653e746231706c78707764343432716d376d68753338676a39396334307761716174356864386b75716a797534796a3273656c6b653773756e716b3072646c67',X'0772656365697665');
INSERT INTO main VALUES(X'046e616d653e746231706c78707764343432716d376d68753338676a39396334307761716174356864386b75716a797534796a3273656c6b653773756e716b3072646c67',X'00');
INSERT INTO main VALUES(X'1077616c6c657464657363726970746f72dfa720e29640a23d214c9d47582725e4e3cb71b54ae9c176e34cbd82b92ab7d3',X'8a7472287470756244364e7a56626b7259685a34586e426b616a31614a6272674a64623567537545517972717138477562385244564c334a6d3443487471347a664d4e3278654c515158715348425354433966706d58505448586171524d5477555879446e7436574c42676a727365636472382f3836682f31682f30682f302f2a29236b6c38716d326866537bcb64000000000100000000000000e9030000');
INSERT INTO main VALUES(X'0274786b746dca2ac39af4038a94a9c166771b9e17c9d520fdfc17172f778582a668e1',X'020000000001013256a3b153e66a96598e041392fd1907e66b4ad447eec90e3867e2508334d3900000000000fdffffff03e538900b1b000000160014447fde1e37d97255b5821d2dee816e8f18f6bac99b9f0c0000000000225120f982e6d6aa06fdbbf227448a5c55eee83aba5da7b7012272a492a19fdb3e872600000000000000000f6a0d6c6561726e20426974636f696e0247304402206d29cea2d22173f4dcab743c7a9e8345d38c45211e2f418028c6132cb1c545ad02205d07f5fc7373200eb0a4a342bf464d2e76ada26056fd706fe6936dbc1d4aca5f012102a8c3fa3dbc022ca7c9a2214c5e673833317b3cff37c0fc170fc347f1a2f6b6e200000000db5acb04e903ed8470b0b14c8669b83cca8e78dade69343323c2f15c1c010000000100000000030b66726f6d6163636f756e7400016e01300974696d65736d6172740a313639313035373530380000000000647dcb640000');
INSERT INTO main VALUES(X'1077616c6c657464657363726970746f722b1b9b48fd74166775b6778d559f8d54d31343e120e6188d195f5c45ea185b64',X'8c77706b68287470756244364e7a56626b7259685a34586e426b616a31614a6272674a64623567537545517972717138477562385244564c334a6d3443487471347a664d4e3278654c515158715348425354433966706d58505448586171524d5477555879446e7436574c42676a727365636472382f3834682f31682f30682f302f2a29233339773973306a34537bcb64000000000200000000000000e9030000');
INSERT INTO main VALUES(X'07707572706f73652a74623171766b6470757339326d366b3037386576687074716c6c336d7573657438397566367036707a65',X'0772656365697665');
INSERT INTO main VALUES(X'046e616d652a74623171766b6470757339326d366b3037386576687074716c6c336d7573657438397566367036707a65',X'00');
INSERT INTO main VALUES(X'0c6f72646572706f736e657874',X'0200000000000000');
INSERT INTO main VALUES(X'1077616c6c657464657363726970746f72c56da0ec8d553443e27cf2439fd158b8aed74e22606b55476dfc9ea4c5ea6474',X'8c77706b68287470756244364e7a56626b7259685a34586e426b616a31614a6272674a64623567537545517972717138477562385244564c334a6d3443487471347a664d4e3278654c515158715348425354433966706d58505448586171524d5477555879446e7436574c42676a727365636472382f3834682f31682f30682f312f2a29237133747964367a64547bcb64000000000100000000000000e9030000');
INSERT INTO main VALUES(X'0274788a42ba28ebea6a32f21698ae39dfc0cf88c145ee8d25c6c87a231f1182c28445',X'020000000001016b746dca2ac39af4038a94a9c166771b9e17c9d520fdfc17172f778582a668e10100000000fdffffff02390b080000000000160014fc8dcae6a94955d00b49166ee72d9b066da41f71e093040000000000160014447fde1e37d97255b5821d2dee816e8f18f6bac90140059be0aba5788132c56d1066808dafbc8109c443bca304b2845a837ec696e620b0e597fdaf2f78ef8b0661d56fbec860a2af96ca5cf040adc11b348a3262e7ef5b5b0200d850a0ace4ecb413d999684ef03e8163ab60e9d3cffb471cddcc0b3e2f010000000300000000030b66726f6d6163636f756e7400016e01310974696d65736d6172740a3136393130363133333800010000005a8ccb640100');
INSERT INTO main VALUES(X'0962657374626c6f636b',X'3cf7030000');
INSERT INTO main VALUES(X'1262657374626c6f636b5f6e6f6d65726b6c65',X'3cf703001ddd15adef25236aa1977490ce19a6a7e534fd97d193bcbebb8bfa2c88bf000000131d64d114facb8f46b68e16fe12ebb4d60150731cfac80756246794460000007af81238abfd623040d8634443b867594e8e75fb2975b63827b5ec41bc00000002047e9106550a8e9539eb38d71442564e9fcf15616a4127479c79beb5000000db037021175293d1508c23c48dc54cb72d7e8e91807b0c8dc79e06fd0601000045bdc5e2b216703e00ddaf69dbdc59a6ffa8f32b2467c1670bd9727f05000000504e7ba80a6e63aae65d95b0f0ab14fcc239e426b37c5f79525494fc6600000000b303ab247eb9c703a51e38b66e6a61e85953a2e0fc02d175d4dd43e8000000300272effaa174f5e3749eca784c9a813af18f03499cb360b3bf34d5cb0000004edcbae37032eacb983c8f9b0bcca15d73139baa2b662a1f892447de88000000a1c55728fcd56898069614b8869b708571fb733ef27625516567d769170100000f0786ccd8d64a001cd87a97dbfd0ca470a4382c932480102ca4920a430000001337f951dd3895cdff5668a5b967e0706cb11c51d04226cc758eb22b22000000cc4a08855bdb2bd633bb1985b9d00e4af8111c5db42cb7ef5ffb6e751e010000793fb041c5b0ca50969819da43fdc409c7c232b17de13eee0ea0c12fd1000000ec226da0ab29c00036e0a5e9375625f4d26c8ce3900ecde5a6b1102a03010000a7956b1c6ed68eec76b46c17e5a4bdc4c702fe8f2e2c1c4f6950e1ff35000000df69bfaa2d50536a7247b1954c2de1142345068302946a6f9c0bf53ac800000008d855cde03522d453c3824da788c385bc710630e664df874caeaa2ba80000008ebc7677430a7118616d94608aabe2d288a5776724a705af7eaa34910600000088728ff409de829256fa94089a042254bb3488294709a49037b8e07d18000000162d72901b16c40d6520766fd0d301da1d944036a68413d7e1a2f101040100007ec427aa72cc655d18cfc723ba5b43b841d2d908b961ea967b9df63f010100005274107a2ae7f427661f034d9d1eb601bddd6267e5cc5c02c4a459d470000000f022e96318bbe6a13c4a924409523b7c589838b718e9a4215378db8bef00000070a46af62689bab6afbf4134b6483c4f26b8e1a55d73d26934ff5cd9020100006ac1f7a814d77735cf4b0a5d60ff83396d587efe7425ea6f18cc38cd970000007e16f2b8ef7ae47a17aa62318bc597bd5c4b931bbe456bb2d5f89c4030000000f61eee3b63a380a477a063af32b2bbc97c9ff9f01f2c4225e973988108000000');
COMMIT;

I'm not sure what is going on here or how to proceed. My guess is it is generating HD child pubkeys and the transaction that sent (signet) Bitcoin to this wallet must be in here somewhere. I'll look at sqlite.cpp and dig around in this dump for data I was expecting to see but any pointers on where to start and what I might be misunderstanding would be appreciated.

Why does wallet.dat need to be locked permanently if bitcoind is running?

bitcoind does not retrieve data from disk when it is running. It only retrieves data on loading, and will occasionally write to the database. If something were to modify the database while it were running, then the state on disk would mismatch the running state which would result in corruption and possible loss of funds. By locking the database when bitcoind has that wallet loaded, it can ensure that nothing else is going to modify the database and thus have better guarantees that the state on disk matches the state in memory.

Although SQLite is a relational database, Bitcoin Core only uses it as a key-value store like BDB. It uses a single table with two columns, one for the key, the other for the value. Each row is then formatted in the same way that key-value records were formatted for BDB. The parsing of each record is shared for both. You can find the code for that in src/wallet/walletdb.cpp.

Thanks for this Andrew. How can I inspect the contents of the wallet.dat then? dumpwallet is only compatible with legacy wallets?

Your sqlite command is inspecting the contents of the wallet.dat, just without any interpretation of the records as loading the wallet is. It would be more helpful if you could explain what you are trying to do that requires inspecting the wallet at such a low level. Note that dumpwallet does not actually dump the contents of the wallet.dat nor provide actual inspection of a wallet.dat's contents.

Just digging around trying to understand how it works from a learning/educational perspective. If it isn't meant to be inspected like this that's fine. I just assumed (perhaps wrongly) that any database including this one could be inspected or at least exported and be made human readable? Essentially view that single table with two columns using some software?


text:
Where does a single script go in the Taproot tree of a script path spend?
If I only have a single script for the script path does the hash of the script get placed at the top of the Merkle tree (instead of the effective Merkle root) or does it get placed in level 1 of the Merkle tree and get hashed up to the Merkle root with another arbitrary hash?

You can do either. Putting the hash of the single script (hash_TapLeaf) instead of the Merkle root makes it cheaper to spend as you don't need to reveal other hashes to prove that script is present in the Merkle tree. However, if you want to obscure the fact that there was only a single script path in the Merkle tree on spending (and are willing to pay a higher transaction fee on spending) you may include it on a branch at level 1 (or even a higher level) of the Merkle tree. To use hash_TapBranch you need at least two branches and so you need to place arbitrary hash(es) on whatever branch(es) you aren't using.
Thanks to Russell O'Connor for answering this on IRC.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How does the Bitcoin Core wallet generate entropy for the HD seed?
How does the Bitcoin Core wallet generate entropy for the HD (Hierarchical Deterministic) seed and ultimately the private keys?
This post asks a related question but doesn't focus on the entropy part, more the BIP 32/39 part.

The Bitcoin Core wallet requires entropy both for generating HD seeds and for generating nonces (number used once) used in digital signatures (handled by the secp256k1 library).
raw_avocado discussed how the Bitcoin Core wallet generates entropy for a HD seed on Twitter. In summary, by mixing entropy from 4 main sources: low level processor instructions, entropy from the OS, dynamic events and static events.

(i) Processor instructions
RDSEED -  access to the built-in True Random Number Generator on the
processor.
RDRAND  - access to the built-in Pseudo-Random Number generator that
gets seeded every time it's called with RDSEED
RDTSC - returns the # of CPU cycles since reset
(ii) OS sources
Most of the Linux distributions have specific functions that call
/dev/urandom, like getrandom,  Where this is not the case it's called
directly.
getentropy used on Open BSD & Mac  sysctl kern.random on Free & Net
BSD
CryptGenRandom on Windows.
(iii) Dynamic Events
Various clocks are called real-time, monotonic, and boot time.
Resource usage is constantly changing and is a great source of
entropy.
On Windows, getrusage is called, and on Linux pseudofiles from /proc
are used or sysctl where available.
(iv) Static Events
Anything from static details about compiler version to bitcoin client
version to CPUID, hostname, network interfaces, kernel info,
filesystem data.
Even though the Events Static & Dynamic are non-cryptographic data,
they still increase the entropy.
The Dynamic and Static events are mixed in(mostly) by using << Left
Shift Bits. Because it's a binary operation, every digit is a power of
2, so shifting the bit n positions ends up multiplying by 2^n. This
adds further confusion when multiple numbers are used at once. The
most common method used to mix in the entropy is the SHA512(its
compression function more precisely). SHA512 works effectively as a
pseudoRNG, as it shows uniformity, meaning each output has the same
probability of occurring. In some cases, SHA256 is used.
Good hash functions have a strong avalanche effect, meaning that when
even a single input bit is incremented, each output bit changes with a
probability of 50%. The bit diffusion is a result of adding, rotating,
and XORing chunks of the input.
In this mixing situation, the results of the processor pseudoRNG
RDRAND are XORed 1024 times. (this is used as a fallback from RDSEED -
the TRNG). Every call should reseed the PRNG differently and XORing is
a great way to obfuscate things.
Another technique for strengthening the entropy is hashing inputs in a
loop for an amount of TIME, not just a number of times. The mixing
techniques are benchmarked and the benchmarks are also mixed in as
sources of entropy. With each entropy source and each mix the chances
of an adversary recreating the entropy decrease exponentially. Even if
a few of the sources are somehow compromised, it will still result in
a safe key in the end.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Why doesn't c-lightning allow you to open multiple channels with the same peer?
This question was asked by various individuals on Telegram.
The spec (BOLT 2) allows for multiple channels between the same two peers:

Note that multiple channels can operate in parallel, as all channel
messages are identified by either a temporary_channel_id (before the
funding transaction is created) or a channel_id (derived from the
funding transaction).

The temporary_channel_id is unique only within a single connection hence implying the possibility of multiple channels per peer/connection.
As far as c-lightning goes, I get the following error when trying to fund a second channel with the same peer:
Peer already CHANNELD_NORMAL

Why doesn't c-lightning follow the spec here?
Update: Core Lightning v0.11.0 (April 2022) introduced support for opening multiple parallel channels to a single peer (channel multiplexing)

This was answered by Christian Decker on Telegram.

Just because the specification allows something doesn't mean every
implementation has to implement it and support it. In c-lightning's
case it is both an architectural choice (multiplexing multiple
channels onto a TCP connection is not trivial) as well as a conviction
(peer diversity beats doubling down on a single peer with multiple
channels, and avoids the creation of single points of failure). After
all dual funding and splicing give the same abilities as channel
multiplexing with a reduced onchain footprint as well as less
complexity and less load on the gossip network.

Rene Pickhardt added in the comments below:

Parallel channels might also be a challenge for
payment flows. First the computation becomes more expensive and second
the assigned liquidity is artificially split between two separate
channels. Thus one should always prefer dual funding, splicing and
rebalancing over parallel channels

I want to add that parallel channels might also be a challenge for payment flows. First the computation becomes more expensive and second the assigned liquidity is artificially split between two separate channels. Thus one should always prefer dual funding, splicing and rebalancing over parallel channels


text:
What escrow system does the official Bitcoin bounty progam use?
There is a list of bounties on the wiki here:
https://en.bitcoin.it/wiki/Active_Bounties
How are the BTC pledges collated and eventually paid? I assume there is some kind of escrow system but I can't find out anything about it.

Usually they're a multisig transaction between a couple of core developers. One recent one was a 2-of-3 between Greg Maxwell and a couple of others. It's mainly just to prevent hit-by-a-bus scenarios, we trust in these people absolutely.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How to use "OP_CHECKSEQUENCEVERIFY" in regtest?
I have created a P2sh includes OP_CHECKSEQUENCEVERIFY and funded the same. I was trying to redeem the coins. But i could not understand the working of OP_CHECKSEQUENCEVERIFY.
Below is the example which I have tried,
Execute the timelock condn:
currently the block count is 152
btcc [OP_IF 155 OP_CHECKSEQUENCEVERIFY OP_DROP OP_SHA256 ef7797e13d3a75526946a3bcf00daec9fc9c9c4d51ddc7cc5df888f74dd434d1 OP_EQUAL OP_ENDIF]
redeemscript:
    2a63029b00b275a820ef7797e13d3a75526946a3bcf00daec9fc9c9c4d51ddc7cc5df888f74dd434d18768

bitcoin-cli decodescript 63029b00b275a820ef7797e13d3a75526946a3bcf00daec9fc9c9c4d51ddc7cc5df888f74dd434d18768

{
  "asm": "OP_IF 155 OP_CHECKSEQUENCEVERIFY OP_DROP OP_SHA256 ef7797e13d3a75526946a3bcf00daec9fc9c9c4d51ddc7cc5df888f74dd434d1 OP_EQUAL OP_ENDIF",
  "type": "nonstandard",
  "p2sh": "2N8grZYRHeBLPy9hE7LoY6MvS53j81UYcpP",
  "segwit": {
    "asm": "0 204808939e268852417e8d2988688fbb7482887aa3ecb10e46e848427976bd77",
    "hex": "0020204808939e268852417e8d2988688fbb7482887aa3ecb10e46e848427976bd77",
    "reqSigs": 1,
    "type": "witness_v0_scripthash",
    "addresses": [
      "bcrt1qypyq3yu7y6y9yst7355cs6y0hd6g9zr650ktzrjxapyyy7tkh4mshxnqjw"
    ],
    "p2sh-segwit": "2NFRd4rwoGkRy13fA4h2DhmHrAUgJ1rx5cr"
  }
}

bitcoin-cli sendtoaddress 2N8grZYRHeBLPy9hE7LoY6MvS53j81UYcpP 0.001
4383cef8bb9026ec5a0a7ee0be4379b5d888bddbbf328ed2c356941fb0e2a4af

create raw txn with below given txid and vout:
bitcoin-cli getrawtransaction 4383cef8bb9026ec5a0a7ee0be4379b5d888bddbbf328ed2c356941fb0e2a4af
02000000000101fc4d29e944c0efb5aad0e437dc13a0bba3374ae885673879b213c6dc80c021350000000017160014544aa5cc9c1377f5bc51240216d0dfbcd0809efdfeffffff02c4bb25050000000017a9145b5337c2a79e2f3c01185eaca7f1322eff343fb087a08601000000000017a914a962d0d582a9ffe4f4ffeef878ced14b381a3daa870247304402206c9c3af6dedd715b21037e53b7212d4228fb75b7e7c9b136511f46a35dadcdda0220499b18337235323bd558cbf0b91ed0a73ff56373c312fccd69e5866ba1662cf1012103ba91e8963545b487403bf7b325598b896dbd6e6e858f9b8fd053576033f114c198000000

Some parts of o/p:
txid: 4383cef8bb9026ec5a0a7ee0be4379b5d888bddbbf328ed2c356941fb0e2a4af
vout:
{
"value": 0.00100000,
  "n": 1,
  "scriptPubKey": {
    "asm": "OP_HASH160 a962d0d582a9ffe4f4ffeef878ced14b381a3daa OP_EQUAL",
    "hex": "a914a962d0d582a9ffe4f4ffeef878ced14b381a3daa87",
    "reqSigs": 1,
    "type": "scripthash",
    "addresses": [
      "2N8grZYRHeBLPy9hE7LoY6MvS53j81UYcpP"
    ]
  }
}
bitcoin-cli createrawtransaction "[{\"txid\":\"4383cef8bb9026ec5a0a7ee0be4379b5d888bddbbf328ed2c356941fb0e2a4af\",\"vout\":1}]" "[{\"mtrNo3QBMwXRV5HdxknBesWBzitqEnfLxM\":0.0003},{\"2N8grZYRHeBLPy9hE7LoY6MvS53j81UYcpP\":0.000688}]"

o/p:
0200000001afa4e2b01f9456c3d28e32bfdbbd88d8b57943bee07e0a5aec2690bbf8ce83430100000000ffffffff0230750000000000001976a9149245fd67a780afdd93ae3e8d39cdc41b40deeeaa88acc00c01000000000017a914a962d0d582a9ffe4f4ffeef878ced14b381a3daa8700000000

adding the signature:
ScriptSig part:
secret hex: 0a626c6f636b636861696e
redeemscript:
2a63029b00b275a820ef7797e13d3a75526946a3bcf00daec9fc9c9c4d51ddc7cc5df888f74dd434d18768

secrethash + if condn+ redeem script
0a626c6f636b636861696e01512a63029b00b275a820ef7797e13d3a75526946a3bcf00daec9fc9c9c4d51ddc7cc5df888f74dd434d18768

manually created signature:
0200000001afa4e2b01f9456c3d28e32bfdbbd88d8b57943bee07e0a5aec2690bbf8ce834301000000380a626c6f636b636861696e01512a63029b00b275a820ef7797e13d3a75526946a3bcf00daec9fc9c9c4d51ddc7cc5df888f74dd434d18768ffffffff0230750000000000001976a9149245fd67a780afdd93ae3e8d39cdc41b40deeeaa88acc00c01000000000017a914a962d0d582a9ffe4f4ffeef878ced14b381a3daa8700000000

bitcoin-cli sendrawtransaction 0200000001afa4e2b01f9456c3d28e32bfdbbd88d8b57943bee07e0a5aec2690bbf8ce834301000000380a626c6f636b636861696e01512a63029b00b275a820ef7797e13d3a75526946a3bcf00daec9fc9c9c4d51ddc7cc5df888f74dd434d18768ffffffff0230750000000000001976a9149245fd67a780afdd93ae3e8d39cdc41b40deeeaa88acc00c01000000000017a914a962d0d582a9ffe4f4ffeef878ced14b381a3daa8700000000

error code: -26
error message:
non-mandatory-script-verify-flag (Locktime requirement not satisfied) (code 64)
the same error , was getting even if the blocksize reached more than 155.
Can you please explain to me what I have missed?

I would like to replicate what you're doing: could you please edit in the result of bitcoin-cli getrawtransaction 4383cef8bb9026ec5a0a7ee0be4379b5d888bddbbf328ed2c356941fb0e2a4af

@AlistairMann edited.

@kitty you have set nSequence of your transaction to default, which is 4294967295. When redeeming OP_CSV, your nSequence should be higher than the blocks locked in OP_CSV, but lower than the number of blocks that have been mined since the transaction was confirmed in the block. You can set your nSequence to 155 in the createrawtransaction step.

Hi @UgamKamat , could not understand properly. Say my block size 155, in the script condition is for 160 then what can be set to sequence value to redeem the coins? When you are mentioning the nsequence should be greater than the blocks locked in CSV (i.e script) and should be less than blocks mined (block size). It sounds contradictory. Can you please give some clarification on this? When I changed the seqeunce value to a specified value, I m getting error as error code: -26
error message:
non-BIP68-final (code 64)

@kitty 1/2 for example, let us assume that you locked the bitcoins in CSV output for 155 blocks and was mined in a block at height N. Now, after some elapsed time the current height of the blockchain is N+166. Since 166 blocks have passed you can spend that locked output. However, you have to indicate that by setting the nSequence value. This value has to be more than 155, but less than 166 in our case. If we set it more than 166, the software will detect that the value of nSequence is higher than the number of blocks have passed and reject the transaction.

2/2 However also note that setting nSequence value 0xffffffff (which is in your case) disables relative locktime. you can refer to my answer for more info on CSV and nSequence

Thank you @UgamKamat. How did you calculate 166 for 155?

@kitty blockheight of N+166 (i assumed 166 blocks have been mined since your transaction was confirmed in a block.)


text:
Installation of Bitcoin Core 0.18.0 on Ubuntu 18.04.2 error
I used "sudo snap install bitcoin-core" and it appeared to install fine except I get the error: "Failed to launch 'Bitcoin Core' Desktop file didn't specify Exec
Field."  Did I do something wrong?  Maybe I am not finished with the install?
This is nearly a pristine installation of Ubuntu.  The only other thing I did was run updates and I installed Konsole prior to installing Bitcoin Core.  I ran the "sudo snap install..." command from Konsole.
Thanks to all.

This must help you.
Open terminal:
$ sudo su
# cd /var/lib/snapd/desktop/applications
# nano bitcoin-core_bitcoin-qt.desktop

and last line must look like that
Exec=env BAMF_DESKTOP_FILE_HINT=/var/lib/snapd/desktop/applications/bitcoin-core_bitcoin-qt.desktop /snap/bin/bitcoin-core.qt %U

Ctrl + X will quit the editor and you will be asked if you want to save your changes. Press Y for Yes.
Now try launching the app.

From the command line, I typed '/snap/bin/bitcoin-core.qt' and now Bitcoin Core is synchronizing with the network. Problem solved.
I still don't know how to repair the shortcut but that is because I'm a Linux newbie.


text:
Deduct transaction fee always from 1 receiver address if sending some BTC amount to 2 different receiver addresses in single transaction
Need help!! anyone there..
How to Deduct transaction fee always from 1 receiver address if sending some BTC amount to 2 different receiver addresses in single transaction

There is no concept of 'which address is paying the fee' in a Bitcoin transaction.
Instead, transactions are constructed as follows:

One or more UTXOs are specified as inputs. These UTXOs are spent in their entirety
New UTXOs are created as outputs. Each one is of a certain value, and pays to a certain address.
The difference (value of inputs - value of outputs) is the implied miner's fee.

So, it is up to whoever crafts the transaction to allocate Bitcoin to each newly created output in a way which satisfies the socially agreed upon transaction conditions (ie, "decide who is paying the fee, and then deduct the fee from the newly created output they will own").


text:
An updated forkcoins tree chart (only blockchains)
Where can I find something conveying this information, but up to date?
digraph {
  # rankdir = LR;
  splines = polyline;

  # Bitcoin is the original chain.
  node0 [label = "Bitcoin\n~29000$US (BTC - Jun 2022)"];

  # Bitcoin Cash was a first successful fork.
  node1 [label = "Bitcoin Cash\n~200$US (BCH - Jun 2022)", color = "#333333",
         fontcolor = "#333333"];
  node0 -> node1 [label = "Aug 2017", color = "#333333",
                  fontcolor = "#333333"];

  # Bitcoin Gold was a second try to make a successful fork.
  node2 [label = "Bitcoin Gold\n~20$US (BTG - Jun 2022)", color = "#999999",
         fontcolor = "#999999"];
  node0 -> node2 [label = "Oct 2017", color = "#999999",
                  fontcolor = "#999999"];

  # Subsequent forks were tried, but most have been low valued.
  node3 [label = "Bitcoin Diamond\n~.5 $ US (BCD / Nov 2020)",
         color = "#bbbbbb", fontcolor = "#bbbbbb"];
  node0 -> node3 [label = "Nov 2017", color = "#bbbbbb",
                  fontcolor = "#bbbbbb"];
  node4 [label = "United Bitcoin\n~.6 $ US (UBTC / Nov 2020)",
         color = "#bbbbbb", fontcolor = "#bbbbbb"];
  node0 -> node4 [label = "Dec 2017", color = "#bbbbbb",
                  fontcolor = "#bbbbbb"];

  #   There were more forks in Dec 2017 but I think they're all low valued.
  # Ignoring Bitcoin X, Super Bitcoin, Bitcoin Hot.

  # Bitcoin SV was forked from Bitcoin Cash.
  node5 [label = "Bitcoin SV\n~150 $ US (BSV / Nov 2020)", color = "#666666",
         fontcolor = "#666666"];
  node1 -> node5 [label = "Nov 2018", color = "#666666",
                  fontcolor = "#666666"];

  # eCash (previously called Bitcoin Cash ABC) was forked from Bitcoin Cash.
  node6 [label = "eCash\n~40$US/MXEC (XEC - Jun 2022)", color = "#999999",
         fontcolor = "#999999"];
  node1 -> node6 [label = "2021?", color = "#999999",
                  fontcolor = "#999999"];
}

Please note this aims discussion on the forks of the original Bitcoin blockchain (forkcoins, specifically), not discussing the codebase forks bootstrapping a different blockchain.

As far as I'm aware, the newest graph mapping forkcoins which spun off from Bitcoin is Bitcoin Magazine's Map of Bitcoin forks infographic. It was published in April 2019, so it will not include newer forks like the upcoming BCH split. It does include the major project forks, but doesn't list all forks (there have been over 40 mostly irrelevant forkcoins).
I looked up a prior project called Map of Coins which was aiming to collect the full pedigree of altcoins that forked from Bitcoin's codebase (and other original projects) but it appears to not have gotten updated since 2017.

what's the upcoming BCH split?

I never meant codebase forks, tried to edit the question accordingly, but that tree of codebase forks is sweet!

On November 15th, BCH has a scheduled hardfork. Looks like there are two groups that are taking the project in two different directions.

SE related facility hints this other page full of good resources: https://bitcoin.stackexchange.com/questions/66054/ :)

I'm reading news that only one branch of that fork is viable. :)

So there is now BCHA, while BCHN will keep the smaller name BCH?


text:
What is the purpose of the script like RETURN PUSHDATA(36)(...)
The example is in here:
https://www.blockchain.com/btc/tx/63910fdfa84db96bd1f28fead4c32f97388d5368c10059d035aac3c5f72fcdc0
Actually, there are two outputs
DUP HASH160 PUSHDATA(20)[78ce48f88c94df3762da89dc8498205373a8ce6f] EQUALVERIFY CHECKSIG and  RETURN PUSHDATA(36)[aa21a9ed9463ee420b10cbad1d997bfa132a89dc9ba3de04e7233e6e6954f1bc339ebb1a]
The strange point is the first output has 12.77303443 BTC, the second is 0 BTC.
There are two questions about this transaction.

How the coinbase could have more than 12.5 BTC?
What is the purpose of the RETURN PUSHDATA(36)(...), actually, in theory, this output can never be consumed by anyone, becuase there is no OP_CHECKSIG or similar OP at the end to return true in the Script stack.

1) How the coinbase could have more than 12.5 BTC?

Because of transaction fees

2) What is the purpose of the RETURN PUSHDATA(36)(...),

The purpose is to embed data in the blockchain. No bitcoins were burned (and no transaction fee), so that's a free way to embed data in the blockchain.

It's technically not free, though because the fee is based on satoshi/kB and the extra data adds some to the size of the tx.

@JBaczuk It's a coinbase tx.

you're right, but it's only free for coinbase not any tx with OP_RETURN

@JBaczuk embed data for what, fun or get the right hash of the block(that should be in the coinbase input)? Will UXTO pool hold this output in memory?

@Capemer It will not store unspendable outputs, there is a check that immediately prunes UTXO that start with OP_RETURN: https://github.com/bitcoin/bitcoin/commit/ec84e81e8383b3b1e1ef4a6dbcb088193d8de5d7

@JBaczuk, then what is the purpose of this?

People abuse it to store any data they want stored "permanently".

In this case, that output is part of segwit. It is for the witness data commitment. See BIP 141: https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Commitment_structure. The reason OP_RETURN is used is so that the UTXO will not be added to the UTXO set as it is provably unspendable.

2) What is the purpose of the RETURN PUSHDATA(36)(...), actually, in
  theory, this output can never be consumed by anyone, because there is
  no OP_CHECKSIG or similar OP at the end to return true in the Script
  stack.

From my understanding, this is mandatory in SegWit enabled blocks.
The second script (starting with OP_RETURN) provides the merkleroot of the witness tree. IIRC it was created this way to prevent a hard fork. Adding an extra field into the block header would defiantly require a hard fork, so the best alternative was to place the data somewhere already available in all current blocks. Since the OP_RETURN script will never be a valid output, it will simply be ignored.
Reference and more information: BIP141 Commitment structure and a similar question

As MCCCS said, the mining reward is composed of block subsidy and transaction fees.
The second output on the coinbase commits to the witness data in the block.
As you might know, the blockheader commits to transactions in the block via the Merkle root field. The Merkle root is produced by building a Merkle tree from the TXIDs (transaction identifiers) of all transactions. Since TXIDs do not commit to the witness data, the Merkle root alone would leave the witness data mutable.
Every block that includes any segwit transactions must therefore the Witness Commitment. The Witness Commitment takes the form of a 36-byte OP_RETURN output on  the coinbase transaction that has the prefix aa21a9ed followed by the 32-byte hash that is the Merkle root of the transactions WTXIDs (witness transaction identifiers). The WTXIDs are produced from the whole transaction, including the witness.


text:
Bitcore-p2p: Peer.connect does not work
I'm trying to run a simple script to connect to a Bitcoin peer, like this one:
  var bitcore = require('bitcore-lib')
  var p2p = require('bitcore-p2p')
  var Peer = p2p.Peer;

  var peer = new Peer({host: 'localhost', port: 18333, network: Networks.testnet})
  peer.on('connect', function () { console.log('connect'); })
  peer.on('ready', function(){  console.log('ready'); });
  peer.on('disconnect', function() {  console.log('connection closed'); });
  peer.connect()

However, the connection seems to never be established.
I only get the 'connect' and the 'connection closed' output, but not 'connected'.
The port I set is the one in the Bitcore config file.
Can someone please help me figure out what is the problem?

It sounds like you're connecting to yourself.

I am connecting to a local node. But this code is separated.

I'm actually not sure how, but now everything works.
I'm writing here a few things to be kept in mind to make it work:
1) the port number is that of the bitcoind server (8333 for mainnet, 18333 for testnet)
2) in case of testnet, the network must be passed as an option to the Peer constructor
3) node version must be 4.x (use nvm use 4), otherwise the code will raise an exception

I'm not sure what exactly you are trying to do with:
peer.on('connected', function(){
 console.log('connected');
});

You can just do on ready instead.
peer.on('ready', function(){
  console.log('connected');
});

Example connecting to a peer and receiving notifications
var bitcore = require('bitcore-lib');
var Peer = require('bitcore-p2p').Peer;

var peer = new Peer({host: "101.201.31.79"});
//IP of one of the btc.com's miners

peer.on('ready', function(){
    console.log("Connected "+peer.version,
    peer.subversion, peer.bestHeight);
});
peer.on('disconnect', function(){
    console.log("disconnected");
});
//loop through peer notifications
peer.on('inv', function(message){
    var nodeMsg = message.inventory;
    var nodeMsgLen = nodeMsg.length;
    for (var i = 0; i < nodeMsgLen; i++){
        console.log(nodeMsg[i].hash);
    };
});
//connect
peer.connect();

I had already tried with every event. Maybe the 'ready' event is more approriate (I will update my question). The problem is that it disconnect as soon as it tried to connect. Furthermore, the same code works for livenet. I did a little debug and found out that, in the testnet case, it sends a wrong (I guess) message to the peer, where the 'network' object is set to livenet instead of testnet. I am updating the issue I opened on github. Hope the developers will help

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Bitcoin: block reward for a tiny transaction
I'm sure this a basic question, but I would like to check my understanding.
At the moment, a miner who grows the blockchain by confirming a block is rewarded with 12.5 bitcoins. This incentivises miners to validate transactions.
Today each bitcoin is worth $13032.00.
It is really correct that if Bob sends Alice some tiny fraction, say one ten-thousandth, of a bitcoin, today worth $1.30, then each confirmation is worth $162,900.00 to the miner?
If this is correct, then it is only economical to offer a transaction fee comparable in value to the miner's reward when the transaction is huge. So as a follow up, I would like to ask:
What are typical transaction fees at the moment, and how is this likely to change as the number of bitcoins approaches its theoretical maximum of 21 million?

Are you aware that the block mining reward goes down over time?

The reward is for a block, and a typical block generally contains around 2,500 separate transactions in it.
Typical transaction fees seem to be around $8 (USD), so transaction fees in a block are typically around $20,000.
The miner who successfully submits a block receives:
   $162,500 (12.5 BTC * $13,000) : reward
 +  $20,000 (2,500 tx * $8 ea.)  : tx fees
--------------------------------
   $182,500 (approximately)

In this case, why should Alice accept Bob's bitcoin, when it will require many further transactions before it is confirmed? Is it simply because she can check that there is no other (unconfirmed) blockchain in which Bob has paid the same bitcoin to Charlie?

I believe you are confusing transactions and blocks.  One transaction does NOT require more transactions to be confirmed.   One block requires more blocks to be confirmed.  One block contains a few thousand transactions.   Alice will accept Bob's bitcoin because it is presumably part of a sale of goods or services.

Furthermore, having a transaction successfully submitted in a block is generally pretty good for most purposes.  There is a tiny chance it could be reversed by a fork, but it is unlikely.  2 or more confirmations means its quite solid.  6 confirmations is generally considered the point where not even a determined state-level actor could reverse a transaction.  That is the level needed for multi-million dollar deals or absolutely critical transactions, and that still usually happens in less than an hour!

@abelensky: I think I follow your answer, but I am still somewhat confused. Is the point that a single block validates many transactions, but these transactions typically involve different bitcoins, so when Alice sends Bob a bitcoin, Bob can sit on this bitcoin, not using it in any further transactions, and still expect Alice's transaction to be confirmed in a 6 block chain? (By the way, I have asked the question in my comment here: https://bitcoin.stackexchange.com/questions/64438/protection-against-double-spending-when-blocks-validate-thousands-of-transaction.)

I agree, I was confusing transactions and blocks, exactly as you said. Thank you for your help. I've deleted the question I linked to since it just reflects my misconceptions.

Block reward and fees are not really related. A miner will get the basic reward of 12.5 bitcoin no matter what. But the miner also has some space available in the block that they can fill with transactions. This does not incur any cost on the miner. So as long as there is space in the block the miner can either mine the block without Bobs tx for a reward of 12.5 btc, or the miner can include Bobs tx, which propbaly pays very little fee, for a reward of 12.500001 btc. A rational miner will always choose 12.500001 over just 12.5 even if the difference is really small. Fees increase if there are more tx than can fit into a single block. In that case a rational miner will pick the tx with the highest fees and not include the tx with very low fee.

https://bitcoin.stackexchange.com/questions/50184/why-does-antpool-mine-so-many-empty-blocks

Hashing in bitcoin is 2-stage. First the transactions are hashed to produce the "merkle root hash". Then this is combined with other information and a nonce and hashed again. Only the second stage hash (which is computed over a fixed-size chunk of data) has to be repeated for each mining attempt.

"A miner will get the basic reward of 12.5 bitcoin no matter what." Not true. It takes 100 blocks for the reward to be given, and orphaned blocks generate no reward.

@Jazimov you are maybe thinking about a different blockchain. In bitcoin the reward is always given immediately. It is in effect not very different from any other transaction in the block. It has a very special input, but it looks and behaves like a regular transaction with an output equal to the reward plus the fees. Orphaned blocks would also generate the reward, but they are not part of the blockchain.

@blues the 100-block freezing is a part of bitcoin.


text:
Why Coinbase hot wallet showing insane price that BTC didn't reach for the day?
I m bit suspicious about Coinbase service. It shows the current BTC is 16900 USD but in Bitfinex is 15 326 USD.
The worst part is I selected buy BTC from Coinbase its showing 17686.50 USD. How on earth such price variance?
I raised the complaint to Coinbase no answer from them, I felt bit dodgy about their service.
I felt discouraged to use mobile wallet they are not providing the correct exchange rate. Could someone can suggest me best way to buy BTC and move to other wallets?

related: https://bitcoin.stackexchange.com/q/9626/5406, https://bitcoin.stackexchange.com/q/2566/5406


text:
Signing a transaction with openSSL
I've read a bit about signing a transaction using openSSL, I've tried implementing it but it seems like the signatures generated are not accepted. So I'm wondering if the following commands provide a valid signature.
Let's create a private key: dd1cd59c4de2fa92e363dac282afe790a5193177d413b38e206b2b86b879ffbf
And generate a testnet public address: mhZQtrhzjqkDmm8yp8jR2D84AagCdL4YzM
And send some testnet btc to it: a05f60a9679a1ad0ef57b092006b6c01886291a2ff696f5e2ac107b349594cf6
We will create a transaction that sends the BTC to 2N8hwP1WmJrFF5QWABn38y63uYLhnJYJYTF.
Okay, so the unsigned raw transaction is:

01000000
01
f64c5949b307c12a5e6f69ffa2916288016c6b0092b057efd01a9a67a9605fa0
00000000
19
166749c16df955750990623397be66e965863611
ffffffff
01
70508e0600000000
1976a914a9974100aeee974a20cda9a2f545704a0ab54fdc88ac
00000000
01000000

We double hash the raw transaction: 372f7f092abc73c2bcd357ec2b0dab4a9fdbb956771c676cf92a899feedcda0a.
And create a PEM key for openSSL (30740201010420 . private_key . a00706052b8104000aa144034200 . public_key and base64 it):

-----BEGIN EC PRIVATE KEY----- MHQCAQEEIN0c1ZxN4vqS42PawoKv55ClGTF31BOzjiBrK4a4ef+/oAcGBSuBBAAK oUQDQgAEnQ9w7thTRcpcHARWO+0ZYIViw31kQGF4WoNujxmRVfKBih4mYfntUB2P kGYRiSCZqd5afqn2Qey03XW7tjwtWg==
-----END EC PRIVATE KEY-----

And then I use the following command to create the DER sig: echo "372f7f092abc73c2bcd357ec2b0dab4a9fdbb956771c676cf92a899feedcda0a" | openssl dgst -sha256 -hex -sign key.pem.
Adding the length byte and public key, we end up with the following scriptSig: 46 304402203c8aaa41ae647149f70ce24f73109e99d59d7e54a5d854239943358d581e9f8302206336cb57c00bb78f46c881b3d6bb2ed76dc652eedf36064ae022ea4ae3ba3b100141049d0f70eed85345ca5c1c 04 563bed19608562c37d644061785a836e8f199155f2818a1e2661f9ed501d8f906611892099a9de5a7ea9f641ecb4dd75bbb63c2d5a.
Great. This leaves us with the following raw transaction:

01000000
01
f64c5949b307c12a5e6f69ffa2916288016c6b0092b057efd01a9a67a9605fa0
00000000
8a
47304402203c8aaa41ae647149f70ce24f73109e99d59d7e54a5d854239943358d581e9f8302206336cb57c00bb78f46c881b3d6bb2ed76dc652eedf36064ae022ea4ae3ba3b100141049d0f70eed85345ca5c1c04563bed19608562c37d644061785a836e8f199155f2818a1e2661f9ed501d8f906611892099a9de5a7ea9f641ecb4dd75bbb63c2d5a
ffffffff
01
70508e0600000000
1976a914a9974100aeee974a20cda9a2f545704a0ab54fdc88ac
00000000

Which is unfortunately not a valid transaction (error received: PUSH TRANSACTION ERROR: 16: MANDATORY-SCRIPT-VERIFY-FLAG-FAILED (OPCODE MISSING OR NOT UNDERSTOOD)).
Using a transaction tool, I get the following (valid) raw transaction, which I was able to redeem:

01000000
01
f64c5949b307c12a5e6f69ffa2916288016c6b0092b057efd01a9a67a9605fa0
00000000
8a
47304402204cdb6499578276106748c4877bf2a3381ba484fb621ce8d3faa2161254bdeec902205e3536aeaa65f8f0f55b3eb31a9bf8959b70e247c0ebd9da485e586370a25b900141 049d0f70eed85345ca5c1c04563bed19608562c37d644061785a836e8f199155f2818a1e2661f9ed501d8f906611892099a9de5a7ea9f641ecb4dd75bbb63c2d5a
ffffffff
01
70508e0600000000
1976a914a9974100aeee974a20cda9a2f545704a0ab54fdc88ac
00000000

All the fields are correct besides the signature. How come the openSSL signing command I used was incorrect?
Is the signature generated valid? Is the openSSL command valid? Can we even use openSSL to sign a transaction?

While I haven't verified the signature, upon inspection of the encoding I think I can see what is wrong. Let's break down the scriptSig briefly and I'll show you:
8a (total number of bytes to follow)
46 (the signature is 0x46 bytes long *THIS IS WHERE THE ISSUE IS*)
30 (compound DER type)
44 (0x44 bytes following)
02 (integer type)
20 (0x20 bytes long)
3c8aaa41ae647149f70ce24f73109e99d59d7e54a5d854239943358d581e9f83 (R value)
02 (integer type)
20 (0x20 bytes long)
6336cb57c00bb78f46c881b3d6bb2ed76dc652eedf36064ae022ea4ae3ba3b10 (S value)
01 (SIGHASH_ALL sighash type)
41 (length of public key is 0x41 bytes)
049d0f70eed85345ca5c1c04563bed19608562c37d644061785a836e8f199155f2818a1e2661f9ed501d8f906611892099a9de5a7ea9f641ecb4dd75bbb63c2d5a (public key)

Now I've highlighted the length of the DER encoded signature as the issue - 46. This should be 47. The reason is that that length is actually supposed to be the length of the signature plus one byte for the sighash type. Unlike normal DER signature produced by OpenSSL, Bitcoin requires the sighash type to be appended, in this case 01 for SIGHASH_ALL. You can see the 01 byte present. But you haven't updated the length. In the valid transaction below it, on the other hand, you can see the length is 47 despite the fields being the same length.
So try updating the 46 to a 47 and see if it works :)

You're correct I did miscalculate the signature length (didn't include the 01). I changed it and try to submit it, but the signature is still not accepted.

What error do you have now? I cannot help without more info :)

Hi, I'm getting an error regarding the inputs (which suggest an issue with the signature because when I try to re-submit the same raw transaction, I get an error that the transaction was already broadcasted).

It could be that it views it as a double spend because you already created a valid transaction afterwards? Make sure the other transaction wasn't broadcast and remove it from your mempool if it is there

When the signature is valid, I get an error that the transaction has already been broadcasted (good), but when it's invalid I get a different error. Therefore I know that when my signature is still invalid. Is there a way to verify my signature (using an online tool or something)?

Unsure, I haven't seen one before that I can recall. Specifically what error message do you get?


text:
Is there a listing of strange or unusual scripts found in transactions?
I'm studying scripts and am looking for strange or unusual scripts that have appeared in any of the *coin networks..(outside the standard ones listed on this page)

Is any one person, or website listing non-standard transactions that are not generated by the default client?

Ideally there would be an analysis of the script and what's going on, but I'm not picky.  I'd even settle for a command line method to extract this data and discover it myself.
My goal is to learn what contracts are occurring in each network and determine the frequency of each. (How popular is multi-sig tx over time)
Alternatively, I can use this as a tool to learn how people are using the scripting language.

Quite the opposite: Which scripts are currently considered standard and therefore actually propagated?

also see https://bitcoin.stackexchange.com/questions/28181/what-new-transactions-are-possible-with-15-operations

Blockchain Info has a list of strange transactions that its nodes received but were unable to verify.

They all look like p2pool transactions.

I have been looking into the same thing lately to build a "Bitcoin Bestiary" with strange transactions. I have downloaded bitcoin tools (
https://github.com/gavinandresen/bitcointools ). It has an option to trace non-standard transactions:
  dbdump.py --search-blocks=NONSTANDARD_CSCRIPTS
I have built on that a python program to study my local blk000x.dat files with the blockchain info. BCDataStream and deserialize come from bitcoin tools.
#!/usr/bin/env python

from BCDataStream import *
import deserialize

def extract_script(d):

 for t in d['transactions']:

     for txOut in t['txOut']:
          print  "Prev hash: "+(d['hashPrev'][::-1]).encode('hex')+" Script: "+deserialize.decode_script(txOut['scriptPubKey'])+'\n'

 return None

def findlock(filename):

    f = open(filename, 'r')

    magic=f.read(4)
    magicref='f9beb4d9'.decode('hex')

    while magic:
        size=f.read(4)[::-1].encode('hex')
        N=int(size,16)

        blk = deserialize.BCDataStream()
        blk.write(f.read(N))
        d=deserialize.parse_Block(blk)
        extract_script(d)

        magic=f.read(4)

    f.close()
    return None

def test():
     findlock('blk0001.dat')
     findlock('blk0002.dat')
     findlock('blk0003.dat')

if __name__ == '__main__':
    test()

The program dumps the Script in all transactions with the previous block number (I have to improve that, but you can later find the right block). The whole file is about 3 GB.
In my Linux terminal I redirect the output to a scripts.txt file and then use grep to find commands, like:
grep 'IF ' scripts.txt > IF.txt
MIN, IF, DROP, SWAP, for instance, show interesting things. Some of them are errors (see https://bitcointalk.org/index.php?topic=67158.0), some I don't know.

Lacking any other resource to track notable transaction, the following answer is made CW so that anyone can edit it and update as needed.

Block 728:  The most common payment type is pubkey-hash which first appeared here. [source]

Local script to find them + preprocessed data
https://bitcoin.stackexchange.com/a/5890/21282 published a local script, and https://bitcoin.stackexchange.com/a/5886/21282 published a list, but here is both together:

list: https://github.com/cirosantilli/bitcoin-strings-with-txids/blob/master/data/known_op_signatures.json
documentation: https://github.com/cirosantilli/bitcoin-strings-with-txids#known_op_signatures
art of the code that stores the transaction: https://github.com/cirosantilli/bitcoin-strings-with-txids/blob/48ed96fd85d8b5d75b8bb1c529a5c6a8a7099059/main.py#L315 p

The script is based on the nice https://github.com/alecalve/python-bitcoin-blockchain-parser Python library which parses blk files without the need for a server.
Then at known_op_signatures.json the output data is organized as follows:
[
  {
    "count": 1122142242,
    "ioidx": 0,
    "sample": "74c1a6dd6e88f73035143f8fc7420b5c395d28300a70bb35b943f7f2eddc656d",
    "sig": [
      null,
      null
    ]
  },
  {
    "count": 1103133457,
    "ioidx": 0,
    "sample": "6f7cf9580f1c2dfb3c4d5d043cdbb128c640e3f20161245aa7372e9666168516",
    "sig": [
      "OP_DUP",
      "OP_HASH160",
      null,
      "OP_EQUALVERIFY",
      "OP_CHECKSIG"
    ]
  },

So you can see we have at toplevel a list of transaction types, sorted by occurrence count.
In the transaction signature sig, null indicates a literal number constant. This way, only operands matter, and the script groups all transactions of a given type by operand alone to keep the data size manageable.
For example, the most common transaction type, with 1122142242 occurrences was:
    "sig": [
      null,
      null
    ]

i.e. two literals, which is the input script of a P2PKH transaction.
The second most common one was:
"sig": [
  "OP_DUP",
  "OP_HASH160",
  null,
  "OP_EQUALVERIFY",
  "OP_CHECKSIG"
]

which is the output scrip of a P2PKH transaction.
By quickly scrolling through the list, you will quickly see that, unsurprisingly, there are extremely few non-standard transactions.
Overviews of the non-standard scripts
https://www.quantabytes.com/articles/a-survey-of-bitcoin-transaction-types provides a nice summary. Other sources:

Useful alternative Bitcoin transaction scripts
Which scripts are currently considered standard and therefore actually propagated?
Non standard Tx with obscure OP codes: examples


text:
Tracing Bitcoins to their original blocks
Can I find (always, without fault) the specific set of blocks from which the Bitcoins that I hold originated?
If yes, would such a trace-back be computationally costly?

It's not really clear what that means. For example, say a transaction pulls in the 25 Bitcoins mined in two different blocks and, among other outputs, pays out 1 Bitcoin. What does it mean to ask which of those two original blocks is the one that Bitcoin "originated" in?

Can I find (always, without fault) the specific set of blocks from which the Bitcoins that I hold originated?

Short answer: No.

Yes in the completely trivial sense that your Bitcoins were created in the transaction in which they appeared as outputs. But this isn't the sort of origin you had in mind.
But no in any real sense because a Bitcoin is  an unspent transaction output (UTXO). A UTXO has no continuity through transactions and so neither does a Bitcoin. You cannot meaningfully or reliably trace a Bitcoin back through multiple transactions to one or more specific mining rewards.
This is mainly because the public journal of transactions we call the blockchain does not associate transaction outputs with specific transaction inputs. Often you can make an inference if there are few inputs but certainly not "always, without fault".
If a transaction takes three of Alice's coins worth 3, 1.5 and 3.5 and pays Bob 4 and Carol 4 there is no way for Eve or Judy to say where Bob's 4 came from.

Yes, you can, every transaction in the bitcoin network is registered in a public ledger called blockchain. You can query this ledger via HTTP interface like https://blockchain.info/api/blockchain_api
I don't think will be costly but it depends on how many hands your bitcoin passed. When you find a transaction without a sender, you have to stop, these are mined (generated) blocks
UPDATE 2022:
HTTP API has 2 major problems:

Trust, you are trusting the service to provide you the right data.
Performance, network calls are expensive.

It is better to query your local copy of the blockchain, using Bitcoin Core with tx index allows you to query the previous outputs of transactions until you find the generating blocks. Note I used plural for blocks because transactions could have multiple inputs, possibly generated by different blocks.

Yes you can do this but I wonder what to do with the outcome. If you receive 1 BTC today you can assume that the path back ends up often in ten thousands of mined bitcoin addresses. Assume the transaction used 3 input addresses and each of the 3 had 3 inputs one step back in history. Then this exponentially grows.
This is similar like going back in your personal origin. You have 2 parents, 4 grandparents, 8 grand-grandparents, 16 grand-grand-grandparents ... . After 10 steps back you're above 1000 and after 20 steps above 1 million. There is of course double counting however the total number gets quickly high.
I think it's not too computationally costly if you do it only for one address. If you do it for all 600+ million you need a powerful machine or a lot of time.

authors: Антон Ковалев


text:
child keys generated addresses reflect in wallet?
I am confused about child HD public keys. Is this scenario correct?

I am using bx (libbitcoin) for all action below.
Generate a seed (bx seed).
create master private key using seed (bx hd-new)
create master public key from private key (bx hd-to-public)
generate several child BIP32 public key using random indexes (bx hd-public)
use generated child keys and generate several addresses
now using generated addresses

will all addresses connected to master private key? or because I used random indexes (step 5), it is disconnected from master private and public key?
In simple word, is it enough to use private key (step 3) in bitcoin clients such as electrum to access to all addresses that generated using child pub keys (not master key)?

It will depend on the wallet, so you should make that explicit in your question.

especialy bitcoin client and electrum. but also if I don't want to use any wallet, what will be needed? is master private key enough or should I generate and store child private key too?

Warning: bx seed generates insecure seeds from a pseudorandom number generator and very low entropy.

Use incremental indexes instead of random
First of all, you should generate child addressess incrementally instead of randomly, as wallets will re-generate them starting from the index 0, incrementing the indexes until they start finding empty addresses.
From the BIP44 Account Discovery chapter:

When the master seed is imported from an external source the software should start to discover the accounts in the following manner:

derive the first account's node (index = 0)
derive the external chain node of this account
scan addresses of the external chain; respect the gap limit described below
if no transactions are found on the external chain, stop discovery
if there are some transactions, increase the account index and go to step 1

Address gap limit
Address gap limit is currently set to 20. If the software hits 20 unused addresses in a row, it expects there are no used addresses beyond this point and stops searching the address chain. We scan just the external chains, because internal chains receive only coins that come from the associated external chains.
  Wallet software should warn when the user is trying to exceed the gap limit on an external chain by generating a new address.

If you generate your child keys at random indexes, most likely the first 20 indexes will be empty, so the wallet will stop the discovery and show you an empty wallet.
Standard derivation paths
The standard derivation path (defined in the aforementioned BIP44 standard) is the following (M denotes the master private key):

m / purpose' / coin_type' / account' / change / address_index

Most wallet implementations (Electrum being a notable exception) follow this standard. That means for a given receiving address, you should use a derivation path like this:
m / 44' / 0' / 0' / 0 / 0
Which translates to bx commands like this:
cat m | bx hd-private --index 44 --hard \
       | bx hd-private --index 0 --hard \
       | bx hd-private --index 0 --hard \
       | bx hd-private --index 0 \
       | bx hd-private --index 0

Since the first three levels involve hardened derivation, you have to use the master private key. The master public key can only be used for non-hardened derivation paths.
Electrum doesn't implement the full BIP44 standard as it only supports single-account wallets. In Electrum, the derivation path looks like the following:
m / change / address_index
Since there is no hardened derivation on the account level, you can also import the master public key into Electrum. This is similar to the chain you described but there is an extra "change" level still.
Electrum can also import a BIP44 master private key (there's a checkbox in the import dialog), by making you select an account index at import time.
I suggest to stick to the BIP44 standard as it will be can be imported by other wallets (like Mycelium) as a HD-wallet, and will also work in Electrum if you remember to import it as a BIP44 key.
However, if you need to work exclusively with master public keys, you can still use the 2-level derivation and import them to Electrum. I'm not sure if other wallets support importing them.


text:
How does this simple script use the OP_CHECKSIG opcode?
I have this Script P2PKH
<A Signature> <A Public Key>
OP_DUP OP_HASH160 <A Public Key Hash> OP_EQUALVERIFY OP_CHECKSIG

The bitcoin.org documentation say this

The entire transaction's outputs, inputs, and script (from the most recently-executed OP_CODESEPARATOR to the end) are hashed. The signature used by OP_CHECKSIG must be a valid signature for this hash and public key. If it is, 1 is returned, 0 otherwise.

Now I want to understand how the code is running inside the stack and how the OP_CHECKSIG work.
My question comes from the fact, why use another operator (OP_CHECKSIG) if is used the operator OP_EQUALVERIFY to does control ?
But I have another question on OP_DUP, the documentation say this

Duplicates the top stack item.

Why when the bitcoin run the script need to duplicate the items inside the stack?
to close, exist other documentation on Bitcoin Script language?

Let's run through the program.

The stack starts as <A sig> <A pubkey>.
You run OP_DUP. The stack is now <A sig> <A pubkey> <A pubkey>.
You run OP_HASH160. The stack is now <A sig> <A pubkey> <A pubkeyhash>.
You push (A expected pubkeyhash). The stack is now <A sig> <A pubkey> <A pubkeyhash> <A expected pubkeyhash>.
You run OP_EQUALVERIFY. If the initial public key on the stack does matches the expected one, the execution aborts and the script fails. If it matches, the resulting stack is <A sig> <A pubkey>.
You run OP_CHECKSIG. If the signature matches, the resulting stack is 1, which means success.

Without the OP_DUP, it would be impossible to verify the public key both against the signature and against the expected public key hash.

This kind of stack-oriented scripting reminds me of Postscript which has operators such as dup, exch and pop. I have often used dup at the start of a Postscript subroutine so that a parameter can be used for multiple operations, each of which consumes items from the stack.

... other documentation on Bitcoin Script language?

Looking at Bitcoin Core's interpreter.cpp can be helpful. It contains a big switch(opcode) ...  statement for all the opcodes.

Now I want to understand how the code is running inside the stack and
how the OP_CHECKSIG work.

The OP_CHECKSIG and OP_CHECKSIGVERIFY cases are handled together:
case OP_CHECKSIG:
case OP_CHECKSIGVERIFY:
{
    // (sig pubkey -- bool)
    if (stack.size() < 2)
        return set_error(serror, SCRIPT_ERR_INVALID_STACK_OPERATION);

    valtype& vchSig    = stacktop(-2);
    valtype& vchPubKey = stacktop(-1);

    bool fSuccess = true;
    if (!EvalChecksig(vchSig, vchPubKey, pbegincodehash, pend, execdata, flags, checker, sigversion, serror, fSuccess)) return false;
    popstack(stack);
    popstack(stack);
    stack.push_back(fSuccess ? vchTrue : vchFalse);
    if (opcode == OP_CHECKSIGVERIFY)
    {
        if (fSuccess)
            popstack(stack);
        else
            return set_error(serror, SCRIPT_ERR_CHECKSIGVERIFY);
    }
}
break;

The code:

Checks there are at least two items on top of the stack
Reads the Signature and PubKey as the top two items
Uses EvalChecksig to check if Signature matches PubKey; fSuccess is false if not
Removes the Signature and PubKey from the stack
Puts fSuccess on the stack
If it's OP_CHECKSIGVERIFY it removes fSuccess and throws an error (causing transaction failure) if fSuccess == False

To go deeper, you can follow the path of function calls from EvalChecksig(.. onwards. E.g.

EvalChecksig(.. Checks if it is a TapRoot Sig
EvalChecksigPreTapscript(.. Gets the script from pbegincodehash - the last OP_CODESEPARATOR (for message hash creation) or the start of the script  if no OP_CODESEPARATOR seen (as in OP's example). Also checks encoding.
CheckECDSASignature Creates hash message for the signature.
VerifyECDSASignature
pubkey.Verify(...)
secp256k12 code.


text:
Some doubts regarding the structure of the Coinbase Transaction?
I was trying to understand the structure of coinbase transaction. I tried a few sources but I still have a few doubts regarding the structure with exactly 1 output. Here is what I know and the doubts are marked by a ?:
[PARAMETER]           [SIZE]   [ENDIANNESS] [ALLOWED VALUES]

 - version            - 4 Bytes   - LE  - 01000000
 - input count        - 1 Bytes   - xx  - 01
 - prev tx            - 32 Bytes  - xx  - 0000000000...0000
 - prev out n         - 4 Bytes   - xx  - ffffffff (outpoint index or fake index)
 - coinbase script    - <?> Bytes - <?> - <what is value range of LCS i.e. min and max Bytes?>
   length (LCS)
 - coinbase script              - LCS Bytes :
                                                1 Bytes       - xx - 03 (Size of Block Height in next 3 Bytes)
                                                3 Bytes       - LE - __ (Valid Block Height in LE for Block being mined)
                                                [LCS-4 Bytes] - xx - xx (ANY binary string is Valid. ANY STRING!!!!)

 - sequence           - 4 Bytes   - xx  - ffffffff
 - output count       - 1 Bytes   - xx  - 01
 - value              - <?> Bytes - LE  - __ (mining reward + transaction fee of all other included transactions in Satoshi)

 - script length (SL) - 1 Bytes   - xx  - (what is value range of SL i.e. min and max bytes, or is it fixed size?)
 - pubkey script      - SL Bytes  - <?> - __ (What is this?! Is it the wallet address or the public key or something else - of the person who recieves the funds in 'value'?)
 - locktime           - 4 Bytes   - xx  - 00000000

Notation:

LE    - Little Endian.
BE    - Big Endian.
xx    - Endianness doesn't matter.
__    - User input.
<..?> - Unclear.

Q1: Can someone please address the doubts marked by ? and validate if rest of the structure is correct?
Q2: Does coinbase script have any additional structure, rules or constraints that I missed including here (other than above) that can make the transaction invalid?

These are the requirements on the coinbase transaction that I am currently aware of.
The transaction is structured like every other transaction, though not all fields have the same meaning.

Version: a 32-bit LE integer. For normal transactions, the version number can only be 1 or 2 (for relative timelock semantics, see BIP 68). For coinbase transactions it has no meaning, and can be anything.
Transaction inputs: for normal transactions, refers to the set of unspent previous transaction outputs that are being spent. For coinbase transactions it is different, there must be exactly 1 input, and it must have specific values.

Prevout hash: must be 0000...000
Prevout index: must be 0xffffffff
scriptSig: for normal (non-segwit, see BIP141) transactions contains the proof that this transaction is authorized by the previous holder of the coin being spent. For coinbase transactions, it has different semantics. Instead it must:

Start with a push of the block height (as specified in BIP34)
Be followed by arbitrary data (which is often called the extraNonce)
The entire scriptSig must be between 2 and 100 bytes exactly (this rule predates BIP34, so its lower bound 2 is now superseded by the fact that the block height push is already larger than that).

nSequence: for normal transactions this field affects BIP125 RBF semantics, and relative locktime semantics (BIP 68), but these don't apply to coinbase transactions. Can be anything.

Transaction outputs: determines which UTXOs are created by this transaction. The semantics are identical for normal transactions and coinbase transactions. In particular, it can contain any number of outputs. Typically it will contain one or more outputs that give mined coins (subsidy + tx fees from other transactions in the block) to the miner, plus some dummy 0-value outputs for various purposes (one of them being the segwit commitment, if this is a segwit block, see BIP141).
nLockTime: This has absolute locktime semantics (stating when the transaction becomes valid), both for normal transactions and coinbase transactions.

So to answer your question directly:
 - version                      - 4 Bytes   - LE  - 01000000

In fact it can be any 4-byte value.
 - input count                  - 1 Bytes   - xx  - 01
 - prev tx                      - 32 Bytes  - xx  - 0000000000...0000
 - prev out n                   - 4 Bytes   - xx  - ffffffff (outpoint index or fake index)

Correct.
 - coinbase script length (LCS) - <?> Bytes - <?> - <what is value range of LCS i.e. min and max Bytes?>
 - coinbase script              - LCS Bytes :
                                                1 Bytes       - xx - 03 (Size of Block Height in next 3 Bytes)
                                                3 Bytes       - LE - __ (Valid Block Height in LE for Block being mined)
                                                [LCS-4 Bytes] - xx - xx (ANY binary string is Valid. ANY STRING!!!!)

The coinbase scriptSig is encoded as 1 byte encoding the length of the script, plus the script itself. That script must be between 2 and 100 bytes. The script has to start with a push of the block height, due to BIP34. For the current range of heights (65536 through 16777215), that's a 0x03 to indicate a 3-byte push, followed by 3 bytes encoding the block height in LE. After that, up to 96 other bytes can follow, whose contents does indeed not matter at all.
 - sequence                     - 4 Bytes   - xx  - ffffffff

This can be anything.
 - output count                 - 1 Bytes   - xx  - 01
 - value                        - <?> Bytes - LE  - __ (mining reward + transaction fee of all other included transactions in Satoshi)

Correct. You can have multiple outputs in fact (and their sum of values has to match the subsidy + tx fees), but 1 works fine. If you want to create a segwit block (see BIP141), there also needs to be another zero-value output with a specific structure; see the BIP for details.
 - script length (SL)           - 1 Bytes   - xx  - (what is value range of SL i.e. min and max bytes, or is it fixed size?)
 - pubkey script                - SL Bytes  - <?> - __ (What is this?! Is it the wallet address or the public key or something else - of the person who recieves the funds in 'value'?)

This is the scriptPubKey corresponding to the address you want to receive the mining reward for. There are many software libraries that can decode addresses to scripts for you, and answers on this site that explain it.
 - locktime                     - 4 Bytes   - xx  - 00000000

Not strictly required, as there are other permitted values, but they must result in a "final" transaction, which depends on the height and timestamp of block. Choosing 00000000 here will always work.

I understand. Since this is extremely technical and requires the knowledge of quiet a few protocols I am a bit lost. Can you please go through the above and see through the doubts and also see if I missed anything? I would be grateful. I don't know how else to understand/create a coinbase transaction rather than by hand like this.

If you're asking how to actually build coinbase transactions in production, that's a very different question. Generally you'd use mining software, which uses the GBT (BIP22) protocol to ask a full node for the set of transactions and other information about the current state of the network, and uses this to build a block candidate, which is then sent to hasher hardware that searches for nonces plus updates the extranonce to find a full block. In my answer I've given the rules (and references to rules) that are relevant, but there simply isn't space to fully explain block building on this site.

Lets say I don't want to build a block candidate with other transactions from mempool but just with exactly 1 coinbase transaction why is it difficult to built it by hand? And can't the block itself be build too that way? What are the hinderences and is it that difficult for such a small block too? I am struggling to see any? if possible can you point to a free/opensource library that can do the same for me in python/C++. P.S. I don't have bitcoin core and I can't afford to download 400 GB for this as of now.

If you mean for documentation purposes, sure - among others, the Bitcoin Core test framework has a full (but obviously incredibly slow) implementation of what is needed to build blocks. If you only want a coinbase transaction it's pretty simple (I'll try to extend my answer later to include some specifics). For production purposes none of that matters, however, you need ASIC mining hardware these days, and it comes with built-in firmware that does a significant part of the block building, including automatically connecting to pools (using Stratum protocol); nothing on your part needed.

I understand. Please ignore the practicality of mining as a single CPU computer since I know its pretty much hopeless without pools. If you can have a look at the doubts and see the structure w.r.t. your knowledge of the protocols that would be enough.

I went over your list, and added comments in my answer.

thanks a lot! it really clarified quiet a bit. thanks again.


text:
P2SH-P2WPKH - has always segwit UTXOs?
I've been reading a lot of docs on BTC transaction structure, and find it really confusing when it comes to SegWit story.
So, the very first thing is, if I've P2SH-P2WPKH address, does it mean that all outputs to this address (UTXOs) are by definition segwit UTXOs ?
In the docs (https://bitcoincore.org/en/segwit_wallet_dev/) it says:

Signature Generation and Verification for P2SH-P2WPKH
For spending of non-segwit UTXO, the signature generation algorithm is  unchanged.
For spending of P2SH-P2WPKH:

The scriptSig MUST ONLY contain a push of the redeemScript

...

So, either all UTXOs for P2SH-P2WPKH address are segwit ones, or they are SegWit if SegWit transaction was made to this address (which does little sense actually for me, as then arises chicken-egg problem).
Why they wrote this "For spending of non-segwit UTXO, the signature generation algorithm is  unchanged" part, what case it covers? If you've P2SH-P2WPKH address but you are taking some foreign UTXOs to spend(that were directed not to your address)?
Another not clear part are scripts for P2SH-P2WPKH address.
So here is my example address:
3MxYPuvEcKDdaEG1xoGyddqir6C4b66MJi
And here is transaction that was made to it:
https://blockchain.info/rawtx/6c2806587d5cdb3f0363e43f60b4a84baf37b3012fbde02f108f4eb1bdcede42
Based on it pubKeyScript of output is a914de523116b281c96ff1e204ea2ff435b75d48f0de87 which translates to:
OP_HASH160 de523116b281c96ff1e204ea2ff435b75d48f0de OP_EQUAL

So, de523116b281c96ff1e204ea2ff435b75d48f0de is hash160 of redeem script.
The only redeem script that matches this hash I've found is following:
0 389f26d8d616cb96df1cfd6d9989248a30b933b9

Which is witness v0 program script(0 sha160sha256(pubkey), so it means I've to create SegWit transactions, right?
I tried making P2SH ones and it says that I've non-nominal pushes in the redeem script, that's cause of "0" push there I believe.
I feel that doing something wrong in this part, and as a result signing either wrong thing or in a wrong way...
Very appreciate any explanations.

if I've P2SH-P2WPKH address, does it mean that all outputs to this
  address (UTXOs) are by definition segwit UTXOs ?

Yes, Bitcoin sent to a P2SH-P2WPKH address will produce a segwit UTXO.

Why they wrote this "For spending of non-segwit UTXO, the signature
  generation algorithm is unchanged" part, what case it covers? If
  you've P2SH-P2WPKH address but you are taking some foreign UTXOs to
  spend(that were directed not to your address)?

I think you're right, it sounds like it's just saying if you want to send to a segwit address, you don't need to sign a non-segwit input any differently than before.
Spending P2SH(P2WPKH)
Here is a good reference example: BIP 141: P2SH(P2WPKH). In order to spend Bitcoin sent to your address 3MxYPuvEcKDdaEG1xoGyddqir6C4b66MJi, you have to provide the following:
witness:      <signature> <pubkey>
scriptSig:    <0 <20-byte-key-hash>>
              (0x160014{389f26d8d616cb96df1cfd6d9989248a30b933b9})

Perhaps you are forgetting the push ops 0x16 and/or 0x14 in scriptSig?

Yes, Bitcoin sent to a P2SH-P2WPKH address will produce a segwit UTXO. Technically, the utxo type is still p2sh. It's only revealed as segwit when it is spent. You could totally spend it as a p2sh if you can brute force a redeem script that satisfies the hash and is solvable by you, although that's rather impossible. Regardless, the output itself is p2sh.

@RaghavSood True, there would be no way to detect that it was a segwit UTXO, because it's the redeem script that is segwit.

Thanks for the answer and comments, now it's more clear what is what :)
I don't have 0x16, here is mine full script:
 "script": "0014389f26d8d616cb96df1cfd6d9989248a30b933b9",

Is 0x16 required also?

Basically here is serialized tx I've
0100000000010142decebdb14e8f102fe0bd2f01b337af4ba8b4603fe463033fdb5c7d5806286c00000000160014389f26d8d616cb96df1cfd6d9989248a30b933b9ffffffff0210270000000000001976a914eeba84c4b62a6b6db8f1e0a1daffa0eccde33adf88acbe2100000000000017a914de523116b281c96ff1e204ea2ff435b75d48f0de8702483045022100fc44e10e8ab2e344c7e90f04f4f21a313850cd8eaebb0320edbc5e2d34c749b70220369da1b85125babd90a067d47e4034bc15ca1b9631ed5c8554148be3a4349d19012102ed991ef73f82488def8031fcccc4d51de840e7f17e3ad5d820781df0b4ee6e0600000000

You need the 0x16 because it tells the interpreter to push the following 22 bytes (redeem script) to the stack.


text:
Can you send a transaction from non-native segwit to native(bech32) segwit address?
Let's say I have a wallet with a non-native segwit address (prefixed with 3- like multisig).
Can I send my utxos from this address to a native segwit (bech32) address?

The wallet needs to be able to decode a bech32 address and construct the appropriate native witness output when sending funds.
Given that your (hypothetical) wallet can spend P2SH(witness) outputs already, it is likely it can or will soon support native bech32 addresses, but there is no guarantee that the latter must be supported given the first.

This hypothetical wallet does not need to support any type of address to be able to send to them, it simply will not be able to benefit from the improvements of later protocols. Basically, you can send a UTXO that came from any address type to any other address type. Whether or not you can benefit from improvements like better transaction fees, change address, or batching—or if the sendTo address is even considered valid—is all dependent on your wallet client.

All script types are compatible
This somewhat depends on the wallet client, as there is nothing in the bitcoin core code that prevents any address type from sending to any other address type. The type of address/UTXO is determined by how it was sent. Coins that were at one point transacted between SegWit addresses, but made their way to a legacy address, will create a legacy UTXO.
Common Scenario
When you receive bitcoin (UTXO), it is from one of the protocols. Your SegWit address can receive coins from a legacy address; but this does not "upgrade" the coins to benefit from SegWit improvements, they are still legacy "coins". Mixed script transactions are a feature that enables you to spend UTXOs in a single transaction that may have come from different script types.
Transaction Example:

You receive 0.5 BTC to a SegWit address from a legacy address.
You already have 0.5 BTC that were sent using SegWit.
You send a 1 BTC transaction, which uses both the legacy and SegWit UTXOs, from a SegWit enabled wallet. The 0.5 BTC that was already SegWit will take advantage of the transaction fee updates, etc. The legacy UTXO will not.
The resulting 1 BTC in the receiving address is a SegWit UTXO, and any further movement of the coins will benefit from SegWit improvements, unless at some point they are sent back to a legacy address, where they again become "legacy coins"

So yes, you can send UTXOs of any protocol type to any other, but your wallet client must be able to recognize the address protocol in order to send to it. Even if you are using a wallet client with only legacy addresses and no SegWit support, the source code is backwards compatible, you will receive the coins, but when spending the UTXO, you cannot take advantage of the SegWit improvements.

If you truly didn't find this helpful, feel free to downvote, it's a useful feature. However, I can not provide more accurate answers without an explanation. An accompanying comment would be greatly appreciated :)


text:
Creating a "listunspent" API call using MongoDB
I used a tutorial online to create my own block explorer (https://www.youtube.com/watch?v=laeV2slJgc8).
In this tutorial, I used "Iquidus" to create the block explorer, which uses MongoDB and Node.js. Now I want to make my own API call that will list all UTXOs for a given address.
Is there a MongoDB query I can just run to get the UTXOs or What would the code even look like to do something like that?
The call I am talking about is the same they have on "https://chainz.cryptoid.info/api.dws":
"unspent becomes available, this API is similar to blockchain.info's, it has a single active parameter in which you can pass multiple addresses separated by '|'. It returns information about unspent outputs of those addresses."
MongoDB is a schema-less database but I managed to get this:
> show collections
addresses
coinstats
markets
peers
richlists
txes
> var col_list = db.addresses.findOne(); for (var col in col_list) { print (col) ; }
_id
a_id
balance
sent
received
txs
__v
> var col_list = db.coinstats.findOne(); for (var col in col_list) {print (col) ; }
_id
coin
last_price
connections
supply
last
count
__v
> var col_list = db.markets.findOne(); for (var col in col_list) {print (col) ; }
_id
market
history
sells
buys
chartdata
__v
> var col_list = db.peers.findOne(); for (var col in col_list) {print (col) ; }
> var col_list = db.richlists.findOne(); for (var col in col_list) {print (col) ; }
_id
coin
balance
received
__v
> var col_list = db.txes.findOne(); for (var col in col_list) {print (col) ; }
_id
txid
blockhash
blockindex
timestamp
total
vout
vin
__v
>

Desired output
{"unspent_outputs":[{"tx_hash":"819d2c6331fecf0bcda79962ed7b66c71b508a4698680b84310f4d2f67203e87","tx_ouput_n":0,"value":100000,"confirmations":63499,"script":"76a9142beec605c9a6512f55fe93ad76753e24fc8579b088ac","addr":"MBuTKxJaHMN3UsRxQqpGRPdA7sCfE1UF7n"},{"tx_hash":"819d2c6331fecf0bcda79962ed7b66c71b508a4698680b84310f4d2f67203e87","tx_ouput_n":1,"value":767048,"confirmations":63499,"script":"76a9142beec605c9a6512f55fe93ad76753e24fc8579b088ac","addr":"MBuTKxJaHMN3UsRxQqpGRPdA7sCfE1UF7n"},{"tx_hash":"7d76cc54c510f1e8f4dd4dbf4d215713245cb3fd7ec1417f852b7d34deee116f","tx_ouput_n":0,"value":1000000000,"confirmations":63439,"script":"76a9142beec605c9a6512f55fe93ad76753e24fc8579b088ac","addr":"MBuTKxJaHMN3UsRxQqpGRPdA7sCfE1UF7n"},{"tx_hash":"03033110cbe81f34c3eede0affd626a135ccea67e783f2beaf02c98ba2e721f3","tx_ouput_n":0,"value":314419,"confirmations":10977,"script":"76a9142beec605c9a6512f55fe93ad76753e24fc8579b088ac","addr":"MBuTKxJaHMN3UsRxQqpGRPdA7sCfE1UF7n"}]}

can you provide the database schema?

I edited my question, I hope it helps. Thank you for the reply. @JBaczuk

Where does the explorer get this data? Does it sync from a Bitcoin Core node? If so, why not just adjust the explorer to sync the latest UTXO set? It will be very difficult to create a query that gets unspent outputs for a given address because it will have to traverse the entire blockchain to determine which outputs are unspent.

yes, the explorer is using the core wallet to get its data and places that data into the database. Not really sure how to adjust the explorer to sync the latest UTXO set, thats why I assumed its possible to make a query of some sort, that would query the database to get that information.

In order to execute that query you have to search the entire blockchain for every output to any address that you own. And for each of those outputs you have to search from and including the block with that output to the latest block to see if it has been spent or not. Bitcoin core does the last part for you by maintaining a database of unspent outputs.