Notes from Day 2 of a Bitcoin bootcamp, written a few weeks afterwards, once I had actually understood most of it.
I attended a Lightning development bootcamp about three weeks ago, and Day 2 covered two things that looked completely unrelated at the time: building a blockchain explorer in Python, and doing real proof-of-work mining on a shared chain instead of the instant, frictionless mining we had done on Day 1.
I am writing this now rather than on the day itself because a lot of what happened in that room went past me at the speed of the person explaining it, and I nodded along to more of it than I would like to admit. In the weeks since, I have rebuilt the explorer from scratch, read through the Bitcoin Core RPC documentation properly, gone and found the actual specification for the network we were mining on, and filled in the pieces I had accepted on trust. So this is not a live diary of a lecture, it is what I understand now, which is quite a lot more than what I understood then.
The connection between the two halves only became obvious once I had spent time with both. An explorer is not particularly interesting when it is pointed at a chain where you personally decide when blocks appear, but it becomes genuinely useful the moment it is pointed at a chain that other people are extending without asking you.
What you will build: a working blockchain explorer that queries your own node, and an understanding of what mining actually costs when the difficulty is not set to zero.
Prerequisites: a running Bitcoin Core node (Day 1 covers this if you need it), Python 3.10 or newer, and enough Python to read a function. You do not need to know anything about APIs.
This is a long one, so get something to drink (or eat).
Part One: your node has been running a web server this whole time
Run this in a terminal, assuming your node is up:
curl --user bootcamp:bootcamp123 \
--data '{"jsonrpc":"1.0","id":"demo","method":"getblockchaininfo","params":[]}' \
-H 'content-type: text/plain;' \
http://127.0.0.1:18443/
You get back the exact same JSON that bitcoin-cli getblockchaininfo gives you, and you will notice that there is no bitcoin-cli involved anywhere, because all that program ever does is build a small JSON document, post it to a web server running on your own machine, print whatever comes back and then exit.
Which means bitcoin-cli is a convenience wrapper, and you are about to write your own version of it. Every Bitcoin application that exists, whether that is an exchange backend, a block explorer, a Lightning node or a wallet service, talks to Bitcoin Core through this same door, so once you can open it yourself you stop being a person typing commands and start being someone who can build things on top of it.
What JSON-RPC actually is
RPC stands for Remote Procedure Call, which is a fancy way of saying "run a function on another machine," and JSON-RPC just means you describe that function call as JSON.
Every request looks like this:
{
"jsonrpc": "1.0",
"id": "myapp",
"method": "getblockchaininfo",
"params": []
}
The method is the command name, identical to whatever you would have typed after bitcoin-cli, and params is an array of arguments in order, left empty when there are none. The jsonrpc field is the protocol version, and Bitcoin Core speaks 1.0.
Then there is id, which confused me for a while because you can put literally anything in it and nothing changes. I tried "myapp", "banana" and 7, and the node just echoed each one back untouched. It seemed pointless, but I assure you it is not, and I will come back to it at the end of this section because the reason turns out to be quite satisfying.
Every response has three fields, always:
{
"result": {"chain": "regtest", "blocks": 101},
"error": null,
"id": "myapp"
}
The result holds the answer and is null on failure, error is null on success and holds an object on failure, and id is your label coming home. When something goes wrong you get this instead:
{
"result": null,
"error": {"code": -18, "message": "Requested wallet does not exist or is not loaded"},
"id": "myapp"
}
The thing to know here is that when there is an error, the HTTP status code is 500, so if your code only checks resp.status_code and gives up at that point, you throw away the actual message and end up staring at "server returned 500" with no idea what you did wrong. The useful information lives in the body of the response, not in the status code. You will meet the same handful of error codes over and over, and it saves time to recognise them. Code -1 means you used a command wrong, -5 means an address or key is invalid, -6 is insufficient funds and -8 is a bad parameter. Code -18 means the wallet is not loaded, which you will see a lot, and -19 means you have multiple wallets loaded and did not say which one you meant. Code -32601 is "method not found," which in my experience is almost always a typo, and -32700 means your JSON itself was malformed.
About that password
The bootcamp had us authenticate like this:
auth=("bootcamp", "bootcamp123")
That is HTTP Basic Auth, and the requests library turns those two strings into a header containing the base64 encoding of bootcamp:bootcamp123. Base64 is encoding rather than encryption, so it is not scrambled and it is not a secret, and anyone who can see that header can decode it in about two seconds. Over localhost this genuinely does not matter, but over a network without TLS it matters a great deal.
When I went back through the Bitcoin Core documentation afterwards, I found the better approach, which is to leave rpcuser and rpcpassword out of your config entirely. Bitcoin Core then generates a random password on every startup and writes it to a .cookie file inside the network directory, and reading it in Python takes four lines:
from pathlib import Path
def load_cookie(network="regtest"):
base = Path.home() / ".bitcoin"
path = base / network / ".cookie" if network != "mainnet" else base / ".cookie"
user, password = path.read_text().strip().split(":", 1)
return user, password
This is better for several reasons at once, because the password is random, it rotates on every restart, it is protected by file permissions, and it never accidentally ends up in a git repository, which is not a hypothetical risk given how many people have leaked node credentials by committing a config file. On regtest with worthless coins, bootcamp:bootcamp123 is completely harmless, but I would rather build the habit now than learn it on a node that holds something.
The helper function, finished properly
The slide showing the rpc() helper was cut off mid-line, which felt like a fitting summary of my note-taking that day. This is the version I ended up writing once I had time to think about it, and four things in it were not in the original at all.
import json
import requests
from decimal import Decimal
RPC_URL = "http://127.0.0.1:18443/"
RPC_AUTH = ("bootcamp", "bootcamp123")
session = requests.Session()
session.auth = RPC_AUTH
class RPCError(Exception):
"""Raised when the node returns an error object."""
def rpc(method, params=None, wallet=None, timeout=30):
url = RPC_URL
if wallet:
url = f"{url}wallet/{wallet}"
payload = json.dumps({
"jsonrpc": "1.0",
"id": "explorer",
"method": method,
"params": params or [],
})
resp = session.post(url, data=payload, timeout=timeout)
try:
body = resp.json(parse_float=Decimal)
except ValueError:
resp.raise_for_status()
raise RPCError(f"Non-JSON response: {resp.text[:200]}")
if body.get("error"):
err = body["error"]
raise RPCError(f"{method} failed [{err.get('code')}]: {err.get('message')}")
return body["result"]
Please do not store money as a float
This is the most important line in the file, and it is the one I would go back and add first if I could only change one thing about the code we wrote on the day.
Open a Python shell and try this:
>>> 0.1 + 0.2 0.30000000000000004
Floating point numbers are binary approximations of decimal values, which you will never notice most of the time, except that money is not most of the time. Imagine you are building a service that tracks balances across a few thousand users, and each balance is stored as a float, so every deposit, withdrawal and fee adds another tiny error eleven decimal places down where nobody is looking. Eventually somebody runs a reconciliation report, the total is off by 0.00000003 BTC, and nobody can work out where it went because it never went anywhere at all, it was rounded into existence and then rounded back out again. That bug is horrible to find, because the code looks correct and the arithmetic is correct and the only thing wrong is the number type.
There are two ways to avoid it, and the first is to use Decimal, which stores exact decimal values:
>>> from decimal import Decimal
>>> Decimal("0.1") + Decimal("0.2") == Decimal("0.3")
True
The second, which is what most serious Bitcoin software does, is to work entirely in satoshis as integers and only convert to BTC when you display something, since one BTC is exactly 100,000,000 satoshis and integers have no rounding problems at all. Passing parse_float=Decimal tells Python's JSON parser to build exact decimals instead of floats as it reads the response, which removes a whole category of bug for the cost of one argument.
The other three
Using requests.Session() matters because without it every call opens a fresh TCP connection, does the handshake, sends one request and then tears the whole thing down, whereas with a session the connection stays open and gets reused, which becomes noticeable the moment you write anything that polls in a loop.
Raising an exception instead of returning None matters because if rpc() fails quietly, your program does not crash where the problem is, it crashes three functions later with a confusing TypeError about NoneType and sends you looking in entirely the wrong place. Raising immediately means the error message points at the actual error, which is unglamorous and saves hours.
And setting a timeout matters because without one, a request to an unresponsive node hangs forever while your program sits there looking thoughtful.
Why the wallet goes in the URL
This line looked strange to me until I understood the reason behind it:
url = f"{url}wallet/{wallet}"
A Bitcoin node can have several wallets loaded at once, and rather than bolting a wallet argument onto every single wallet-related command, Bitcoin Core routes by URL path, so node-level commands go to http://127.0.0.1:18443/ while wallet-level commands go to http://127.0.0.1:18443/wallet/alice. Which means that -rpcwallet=alice on the command line was only ever changing the URL, and that is the entire feature.
The bare except, and why I stopped using it
The version we were given for loading a wallet looked like this:
try:
rpc("loadwallet", [wallet_name])
except:
pass # Already loaded
The intention is reasonable, because loading an already-loaded wallet throws an error and you do not care about it. The problem is that a bare except: catches everything, including the node being down, a typo in your method name, and Ctrl+C when you are trying to quit, so all of that gets caught and thrown in the bin while you spend twenty minutes wondering why your program is doing nothing. I lost a genuine amount of time to exactly this, so now I write it like this instead:
def ensure_wallet(name):
"""Load a wallet if it is not already loaded. Returns True if available."""
if name in rpc("listwallets"):
return True
try:
rpc("loadwallet", [name])
return True
except RPCError as e:
if "already loaded" in str(e):
return True
print(f"Could not load wallet {name}: {e}")
return False
It asks first, catches only the specific thing it expects, and says something out loud when reality surprises it, which costs three extra lines.
Challenge 1: blockchain info
def show_blockchain_info():
info = rpc("getblockchaininfo")
print("=== Blockchain Info ===")
print(f"Chain: {info['chain']}")
print(f"Blocks: {info['blocks']}")
print(f"Difficulty: {info['difficulty']}")
That was the version from the exercise and it works fine, but when I read the RPC documentation afterwards I found that the response carries several other fields that are worth printing. The headers field tells you how many block headers your node knows about, and if headers is higher than blocks then your node knows blocks exist that it has not downloaded yet, which is literally what "syncing" means and is the fastest way to tell whether you are actually caught up. Alongside that, bestblockhash is your current tip, verificationprogress is a number between 0 and 1 estimating how far through you are, initialblockdownload stays true while you are catching up, size_on_disk tells you how much space the chain is eating, and pruned tells you whether old blocks are being discarded.
None of these matter much on regtest where you have eleven blocks and control the universe, but all of them matter the moment you point the same code at a chain that somebody else is extending.
Challenge 2: wallet balance
def show_wallet_balance(wallet_name):
if not ensure_wallet(wallet_name):
return
balances = rpc("getbalances", [], wallet=wallet_name)["mine"]
print(f"=== Wallet: {wallet_name} ===")
print(f"Spendable now: {balances['trusted']} BTC")
print(f"Unconfirmed: {balances['untrusted_pending']} BTC")
print(f"Immature (mining): {balances['immature']} BTC")
Use getbalances in the plural rather than getbalance in the singular, because the singular version only returns the trusted number, which means a wallet full of freshly mined coins shows a flat zero when none of it has matured yet. We were shown a real wallet from a test run the night before, and it looked like this:
"mine": {
"trusted": 0.00000000,
"untrusted_pending": 0.00000000,
"immature": 364.11001803
}
That is 364 BTC and not a single spendable sat, and if your explorer only ever shows one number then it will lie to whoever is reading it in exactly this way. Show all three, because the gap between them is where the actual information lives.
Challenge 3: listing transactions
def list_transactions(wallet_name, count=5):
if not ensure_wallet(wallet_name):
return
txs = rpc("listtransactions", ["*", count], wallet=wallet_name)
for tx in reversed(txs):
cat = tx["category"]
direction = "OUT" if cat == "send" else "IN "
print(f"{direction} {tx['amount']:+.8f} BTC "
f"{cat:9} {tx['confirmations']:>4} conf {tx['txid'][:16]}...")
The arguments here are label, count, skip and include_watchonly, so the "*" in first position means "all labels" and passing an actual label filters down to just that one, while skip exists for paging through longer histories. The ordering caught me out, because the list comes back oldest first, which means that if you ask for the five most recent transactions you do get the five most recent, but printed in ascending order, which is the opposite of what anybody expects from a transaction list. That is what the reversed() is doing.
The category field tells you what kind of transaction each row represents, where receive means coins arrived from someone else and send means coins left, and in the second case the amount is negative, which is why the format string has a + in it to force the sign to display. Beyond those two, generate is a mined coinbase reward that has matured and immature is one that is still locked, while orphan is a coinbase from a block that got reorganised out of existence, which is worth exactly nothing and is a slightly bleak thing to find sitting in your history.
The part that genuinely confused me was that one transaction can show up as several separate rows. When I looked at the mining wallet I saw this:
immature 6.48147661 2 confs a8c1f50e... immature 6.00434331 1 conf afe1dc7d... immature 11.40451117 1 conf afe1dc7d... immature 6.48147661 1 conf afe1dc7d...
Three rows with the same transaction ID, and my first thought was that something had broken. Nothing had broken, because that is one coinbase transaction paying three different addresses that all belong to the same wallet, and listtransactions reports one row per wallet-relevant output rather than one row per transaction, so if a single transaction touches your wallet four times then you get four rows. It makes complete sense once you know it and looks like database corruption until you do.
Challenge 4: decoding a transaction
def decode_transaction(txid):
tx = rpc("getrawtransaction", [txid, True])
print(f"TXID: {tx['txid']}")
print(f"Size: {tx['size']} bytes vsize: {tx['vsize']} weight: {tx['weight']}")
print("\nInputs:")
for vin in tx["vin"]:
if "coinbase" in vin:
print(" COINBASE (new coins, spends nothing)")
else:
print(f" {vin['txid'][:20]}...:{vin['vout']}")
print("\nOutputs:")
for vout in tx["vout"]:
spk = vout["scriptPubKey"]
addr = spk.get("address", f"[{spk.get('type', 'unknown')}]")
print(f" {vout['value']:>16.8f} BTC -> {addr}")
That .get("address") is not a stylistic preference, it is load-bearing, because some outputs have no address at all. OP_RETURN outputs are provably unspendable data carriers and every SegWit block's coinbase contains one holding the witness commitment, while bare multisig and various nonstandard scripts also have no address to report. If you write spk["address"] instead then your explorer works beautifully right up until it meets a coinbase transaction, at which point it dies with a KeyError on a key that a reasonable person would assume always exists.
Reading the documentation later, I also found that getrawtransaction has verbosity levels, so passing nothing returns the raw hex string, passing True returns decoded JSON, and passing 2 on recent versions of Bitcoin Core returns decoded JSON along with the input amounts and the calculated fee. That last one is a gift, because on Day 1 I learned that transaction inputs do not state their own amounts, which normally means calculating a fee requires fetching every parent transaction and adding them up yourself, and verbosity 2 simply does that lookup for you using the block undo data.
It is also worth running the command once with no flag at all so you see the wall of hex, because that wall is the transaction and everything else is presentation.
Challenge 5: block details
from datetime import datetime, timezone
def show_block(blockhash=None):
if blockhash is None:
blockhash = rpc("getbestblockhash")
block = rpc("getblock", [blockhash, 1])
when = datetime.fromtimestamp(block["time"], tz=timezone.utc)
print(f"=== Block #{block['height']} ===")
print(f"Hash: {block['hash']}")
print(f"Time: {when:%Y-%m-%d %H:%M:%S} UTC")
print(f"Transactions: {block['nTx']}")
print(f"Size: {block['size']} bytes weight: {block['weight']} WU")
print(f"Difficulty: {block['difficulty']}")
print(f"Merkle root: {block['merkleroot']}")
print(f"Prev block: {block.get('previousblockhash', 'GENESIS')}")
The block["time"] field is a Unix timestamp, meaning seconds since 1 January 1970 UTC, which is why the raw value looks like 1756072990 rather than like a date.
Something I only picked up while reading around afterwards is that block timestamps are not reliable clocks. The miner picks the timestamp, and the consensus rules only require that it be greater than the median of the previous eleven blocks and not more than two hours ahead of network-adjusted time, which is a surprisingly wide window, so blocks can and do appear slightly out of chronological order. Which means you should never use block time for anything that needs precision, and should reach for block height instead, because height is exact and ordered and unambiguous while time is closer to a suggestion.
Two smaller things are worth noting here as well. The previousblockhash field does not exist on the genesis block, because there is nothing before it, so that one wants a .get() too. And getblock has its own verbosity levels, where 0 gives raw hex, 1 gives header fields plus a list of transaction IDs, 2 gives fully decoded transactions and 3 on recent versions adds input amounts, so it is worth staying on 1 unless you actually need the transaction bodies because 2 is considerably heavier.
One last fact becomes very useful shortly, which is that block["tx"][0] is always the coinbase transaction, on every chain and in every block, guaranteed by consensus.
Turning it into something you would actually use
The starter file is a collection of functions, which is fine for a lesson and annoying afterwards, so ten minutes of extra work turns it into a real tool:
import sys
def main():
commands = {
"info": lambda a: show_blockchain_info(),
"balance": lambda a: show_wallet_balance(a[0] if a else "alice"),
"txs": lambda a: list_transactions(a[0] if a else "alice",
int(a[1]) if len(a) > 1 else 5),
"tx": lambda a: decode_transaction(a[0]),
"block": lambda a: show_block(a[0] if a else None),
}
if len(sys.argv) < 2 or sys.argv[1] not in commands:
print(f"usage: {sys.argv[0]} [{' | '.join(commands)}] [args...]")
return 1
try:
commands[sys.argv[1]](sys.argv[2:])
except RPCError as e:
print(f"RPC error: {e}")
return 1
except requests.exceptions.ConnectionError:
print("Cannot reach the node. Is bitcoind running, and is the port right?")
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
Now python explorer.py block and python explorer.py tx <txid> behave like actual commands. That if __name__ == "__main__": line means "only run this when the file is executed directly rather than when it is imported," which is what lets you write from explorer import rpc in another script without accidentally launching the menu.
And this is why the id field exists
JSON-RPC lets you send an array of requests in a single POST, which is where that seemingly pointless id finally does something:
def rpc_batch(calls):
"""calls: list of (method, params) tuples."""
payload = json.dumps([
{"jsonrpc": "1.0", "id": i, "method": m, "params": p or []}
for i, (m, p) in enumerate(calls)
])
resp = session.post(RPC_URL, data=payload, timeout=60)
results = resp.json(parse_float=Decimal)
results.sort(key=lambda r: r["id"])
return [r["result"] for r in results]
hashes = rpc_batch([("getblockhash", [h]) for h in range(1, 101)])
That fetches one hundred block hashes in a single request instead of one hundred separate ones, and because the responses can come back in any order you need something to match each answer to the question that produced it, which is exactly what the id is for. It sits there looking useless in every single-request example and then earns its entire existence the moment you send more than one, which I found unreasonably satisfying.
Part Two: the mining on Day 1 was pretend
The second half of the day opened with a slide that said "Yesterday you typed generatetoaddress. That was pretend," which was rude and also completely correct. What was pretend about it is that on regtest the difficulty target is set to the easiest possible value, so finding a valid block hash takes a handful of attempts and happens instantly. Running generatetoaddress 101 did not skip proof of work, it did proof of work against a target so easy that the work rounds to nothing. And I was the only participant, so there was no competition, no race, and no possibility of losing a block to somebody faster. All three of those things changed in the second half.
Signet, and a genuinely clever trick
We joined a private signet running on the local network, and understanding why signet exists took me a while and a proper read of the specification afterwards. Signet exists because testnet is chaotic. Anyone can mine testnet, so every so often somebody points serious hashpower at it, mines thousands of blocks in an hour and then wanders off, which means blocks arrive in bursts and droughts and large reorganisations happen regularly. The BIP that introduced signet describes the problem plainly, noting that huge reorgs and sudden bursts of blocks make realistic testing infeasible in practice, especially when several independent parties are running software over an extended period.
Signet fixes this by adding a signature requirement on top of proof of work, so that every signet block must satisfy a signet challenge, which is a script baked into the chain's parameters and satisfied by a solution carried in the coinbase transaction. On the default public signet, that challenge requires a signature from a known coordinator, which is why only they can produce blocks and the schedule stays reliable.
Anyone can run their own signet with a different challenge, which brings me to the line in our config that I think was the cleverest thing I saw all week:
[signet] signetchallenge=51 addnode=192.168.10.208:38333
The 51 is hexadecimal, and byte 0x51 is the Bitcoin Script opcode OP_1, also known as OP_TRUE, which is a script that pushes the value 1 and stops, meaning it always evaluates to true. So the challenge is satisfied by everybody, always, with no signature involved at all, which removes the coordinator entirely and drops the chain back to pure proof of work. The result is a private, disposable, worthless-coin network where mining is genuinely competitive, and it was built with one hex byte.
Two practical things are worth knowing here. The challenge value feeds into the genesis block, so if your signetchallenge is wrong by a single character then you are sitting alone on a completely different chain wondering why nothing syncs. And signet uses port 38332 for RPC and 38333 for peer-to-peer, which means your explorer's RPC_URL needs updating before any of the Part One code will work against it.
You can check whether you actually joined:
bitcoin-cli getconnectioncount # should be 1 or more bitcoin-cli getblockchaininfo # chain should say "signet" bitcoin-cli getblockcount # should match the room and keep climbing bitcoin-cli getpeerinfo # who you are actually talking to
If getconnectioncount says 0 then you are mining a private island and nobody will ever see your blocks.
The thing that caught me out during setup was that when I switched from regtest=1 to signet=1, my wallets vanished. They had not gone anywhere, because wallets live inside the network directory, so alice and bob were sitting happily in ~/.bitcoin/regtest/wallets/ while my node was reading ~/.bitcoin/signet/ and finding nothing at all in it. That is why the exercise has you create a fresh wallet, and switching back to regtest brings everyone straight back.
A word about curl piped into bash
The setup instructions were this:
curl -sSL http://192.168.10.208:8000/join-room.sh | bash
Which means downloading a script from a server and executing it immediately with your own user's permissions without ever looking at what is inside it. In a room where I could physically see the instructor, on a local network, that is a reasonable convenience, but as a general habit it is dangerous enough that I think it is worth saying so even though nothing went wrong.
A script fetched that way can do anything you can do, including reading your files, installing something persistent, copying your SSH keys somewhere, and on a machine holding actual Bitcoin, spending your coins. This exact pattern has been used to steal from people repeatedly, and crypto tooling is a high-value target precisely because the theft cannot be reversed afterwards. The safe version costs about five seconds:
curl -sSL http://192.168.10.208:8000/join-room.sh -o join-room.sh less join-room.sh # actually read it bash join-room.sh
Our materials also included a manual appendix showing exactly what the script does by hand, which exists precisely so that you can avoid the pipe, and it is worth working through at least once so you know what was being done on your behalf. It is also worth knowing what happens when the room server is simply switched off, which I found out on a morning when curl returned (7) Failed to connect and I assumed I had broken something. That error means nothing was listening at the other end, so it is a network problem rather than a Bitcoin problem, and the sequence that gets you an answer fastest is checking whether you can ping the host, checking you are on the right Wi-Fi, and asking whether the person next to you can reach it either.
Hashing, and why mining is a lottery you cannot cheat
Bitcoin hashes block headers with SHA-256 applied twice, which is written SHA-256d. A hash function takes any input and produces a fixed-length fingerprint, and three of its properties matter here: it is deterministic, so the same input always gives the same output; it has avalanche, so changing one bit of input flips roughly half the output bits; and it is one-way, so given an output there is no method for finding an input other than guessing.
We were shown avalanche with two strings that differ by a single character:
"Genesis Workshop 0" -> 5e6fd6e1c7ed4a85c15968870007b5fc96aa6157b2a1d0771b3818e8fe486bbc "Genesis Workshop 1" -> 7b06cc7d121d318448fb0f25be0698084cb663ace5df51dc759b0a5ea6e51b89
The outputs are completely unrelated, and that property is the whole reason mining works the way it does, because you cannot steer toward a target hash. There is no clever technique, no gradient to follow and no partial progress, so you try a value, look at the result, and if it is not small enough you try another one, billions of times per second.
The header is only 80 bytes
What miners actually hash is the block header, which contains six fields in a fixed layout: a 4-byte version, the 32-byte hash of the previous block, the 32-byte merkle root, a 4-byte timestamp, 4 bytes of nBits encoding the difficulty target, and a 4-byte nonce, which comes to eighty bytes in total. This is the elegant part, because it does not matter whether the block contains two transactions or two thousand, the miner still only hashes those eighty bytes, since the merkle root already commits to every transaction inside it. Change any transaction and the merkle root changes, which changes the header, which changes the hash, so miners never rehash the block itself, they rehash a tiny header over and over.
Target, and those leading zeros
A block is valid if SHA256d(header) <= target, and here is a real example from the chain we were mining:
Hash: 00000000fb1cdaf9b2f20deea8227ef0db4206340a83711fc68fd7e5ecf69a62 Target: 00000000ffff0000000000000000000000000000000000000000000000000000
Read both of those as enormous 256-bit numbers, and the hash is smaller than the target, so the block is valid. Everyone notices the leading zeros and assumes the zeros are the point, but they are not, because the zeros are simply what "smaller than a very small number" looks like when written in hexadecimal, and more required zeros just means a smaller target and therefore more attempts.
The nBits field is a compact four-byte encoding of that 256-bit target, working a little like scientific notation, and getdifficulty converts it into a friendlier ratio describing how much harder the current target is than the easiest one the rules allow.
What happens when the nonce runs out
This part was not covered in the session and I went looking for it afterwards, because something about the arithmetic had been bothering me. The nonce is four bytes, which gives about 4.3 billion possible values, and that sounds like plenty until you work out that a modern ASIC running at 100 TH/s burns through all 4.3 billion in well under a millisecond.
So it changes something else in order to get a completely fresh batch of 4.3 billion, and usually that something is the extranonce, a field inside the coinbase transaction's input script. Changing the extranonce changes the coinbase transaction, which changes the merkle root, which produces an entirely new header to search, and this is why the pool has to send you coinbase templates rather than just a header. Miners can also nudge the timestamp within the allowed window, or roll certain version bits, a technique known as ASICBoost.
Which means the real search space is very much larger than the nonce alone, and when a slide says "roll the nonce," that is only the inner loop of a considerably bigger loop.
Pools, shares, and how anyone gets paid
At around 2 MH/s, my laptop's chance of finding a block on mainnet is so close to zero that I would expect to wait a good deal longer than the age of the universe. Even on our small room chain, mining alone means either finding a block and getting everything or finding nothing and getting nothing for hours at a stretch, and that variance is unbearable, which is the entire reason pools exist. A pool turns a lottery into a wage.
What a share actually is
You cannot prove effort by insisting that you tried hard, so the pool sets its own target, much easier than the network's, and any hash you find below that easier target counts as a share. A share is cryptographic evidence that you performed roughly a known number of hash attempts, so if the pool's target is a thousand times easier than the network's then on average one in every thousand shares also happens to be a valid block, which makes your share count a measurement of your work that costs the pool nothing to verify.
This is what "First share accepted" meant in my miner's log, which was the pool confirming that my CPU was really searching and starting to count. Shares are worthless in themselves and function as receipts, and pools also run something called vardiff, which adjusts your personal share target so that you submit shares at a comfortable rate whether you are a laptop or a warehouse full of ASICs.
Stratum, briefly
Stratum is the protocol between your miner and the pool, and the sequence is straightforward once you see a log of it. Your miner subscribes and authorises, using your payout address as the username, then the pool sends a job containing the previous block hash, a coinbase template, merkle branches and a target, and your miner searches and submits any shares it finds while the pool accepts or rejects them and pushes a new job whenever the chain tip moves.
Watching my own log fill up with accepted shares while the hashrate counter sat at 2.2 MH/s was a strange feeling, because 2.2 million hashes per second sounds enormously impressive right up until you put it next to a real ASIC doing trillions, at which point my laptop was proportionally a person shouting numbers at a wall.
PPLNS, and why our pool was unusual
The payout scheme was PPLNS, which stands for Pay Per Last N Shares, and it works by looking back over the last N shares submitted across everyone whenever a block is found and splitting the reward proportionally. The rolling window exists to stop pool hopping, because under a naive scheme that simply splits by shares since the last block, a miner could jump onto a pool right after it found a block, when the round is fresh and each share represents a bigger slice, and then leave again once the round dragged on. Looking back over a fixed window of recent shares makes that strategy pointless.
The part I found genuinely impressive is how the payouts were made. Most pools are custodians, in that they mine to their own address, collect the whole reward, keep an internal ledger of what everyone is owed and pay out later from their own wallet, which means they can be hacked, can go insolvent, can freeze your account and can simply decide not to pay, all of which has happened more than once to real miners with real money.
Our pool paid inside the coinbase transaction itself, and here is an actual block from the chain:
7.02477632 BTC -> tb1q30h... student3 15.70118768 BTC -> tb1qekh... student2 18.79558081 BTC -> tb1qmc3... earlier worker 8.47845519 BTC -> tb1qmxz... student1 0.00000000 BTC -> OP_RETURN (witness commitment)
That is one coinbase transaction with multiple outputs, each paying a miner directly, which means the money never passes through the pool's hands at all. It goes from protocol issuance straight to the miner in the very transaction that creates it, and while the pool computes the split and builds the template, it never holds custody and cannot refuse to pay, because paying and creating the block are the same event.
The amounts being all different is not a rounding artifact, it is the share weighting, so student2 received roughly twice what student3 did because student2 submitted roughly twice the work inside the window. That zero-value OP_RETURN at the bottom is worth understanding too, because every SegWit block's coinbase carries one holding the witness commitment, which is a hash committing to the witness data of every transaction in the block. SegWit needed a way to commit to witness data without changing the eighty-byte header, since that would have been a hard fork, so the commitment was tucked into the coinbase instead. On signet, the block's signet solution is appended to that same commitment output rather than living in a separate one, which is a detail I got wrong in my own notes on the day and only corrected when I went and read the specification.
Neither of those carries an address and neither is spendable, which is exactly why your explorer needs the .get("address") call from Part One.
The best five minutes of the week
At some point the instructor plugged in a Bitaxe, which is a small dedicated ASIC miner doing roughly 1 TH/s, and room hashrate went from something like 25 MH/s to something like 1,000,000 MH/s, which is roughly forty thousand times more powerful, instantly.
Blocks did not speed up. They stayed at one every thirty seconds, and what happened instead was that difficulty climbed until they did.
watch -n 5 bitcoin-cli getdifficulty
Watching that number rise in real time while the block interval refused to move taught me more about Bitcoin's design than any explanation had managed, because the chain does not care how much hashpower exists, it measures how fast blocks are arriving and adjusts the target until the interval matches the goal. Mainnet hashrate has grown by something like a trillion-fold since 2009 and blocks still average ten minutes.
The corollary took me longer to appreciate, which is that more mining does not create more bitcoin, since issuance is fixed by the block schedule, so all that extra hashpower buys you is a larger share of a fixed pie and a chain that is more expensive to attack.
Our materials said the pool "retargets every 10 blocks," while standard Bitcoin Core retargets network difficulty every 2016 blocks, so that either means the room chain was running custom parameters or the slide was referring to the pool's own share difficulty, which is a different thing entirely. Network difficulty affects whether a block is valid at all, while share difficulty only affects payout accounting, and those two are worth keeping separate in your head.
Reading your own mining income
bitcoin-cli -rpcwallet=day2 getbalances
Which brought me straight back to the lesson from Day 1, except this time I was living it rather than reading about it:
"mine": {
"trusted": 0.00000000,
"untrusted_pending": 0.00000000,
"immature": 364.11001803
}
Coinbase outputs are locked for 100 blocks, so at thirty seconds per block that is roughly fifty minutes of waiting on our chain, where on mainnet the same rule works out to about sixteen hours. And then listtransactions produced those three rows sharing a transaction ID, which I have already admitted to finding alarming.
The mining challenges, rewritten in Python
Our challenges for this section were written in Go, but the concepts translate directly, so I redid them in Python afterwards using the same helper from Part One with RPC_URL changed to http://127.0.0.1:38332/.
Who got paid
def show_coinbase_payout(blockhash=None):
if blockhash is None:
blockhash = rpc("getbestblockhash")
block = rpc("getblock", [blockhash, 1])
coinbase_txid = block["tx"][0] # always index 0
cb = rpc("getrawtransaction", [coinbase_txid, True])
print(f"=== Block #{block['height']} coinbase ===")
total = Decimal(0)
for out in cb["vout"]:
addr = out["scriptPubKey"].get("address")
if not addr: # OP_RETURN commitments
continue
total += out["value"]
print(f"{out['value']:>16.8f} BTC -> {addr}")
print(f"{'-' * 40}\n{total:>16.8f} BTC total")
The whole function rests on one guaranteed fact, which is that the coinbase is always the first transaction in the block, so block["tx"][0] works every time.
Your mining income, with a countdown
def show_mining_income(wallet):
if not ensure_wallet(wallet):
return
txs = rpc("listtransactions", ["*", 50], wallet=wallet)
mined = [t for t in txs if t["category"] in ("immature", "generate")]
total = Decimal(0)
spendable = Decimal(0)
for tx in sorted(mined, key=lambda t: -t["confirmations"]):
left = max(0, 100 - tx["confirmations"])
total += tx["amount"]
if left == 0:
spendable += tx["amount"]
eta = "SPENDABLE" if left == 0 else f"{left * 30 // 60}m {left * 30 % 60}s"
print(f"+{tx['amount']:.8f} BTC | {tx['confirmations']:>3} conf | "
f"{left:>3} blocks left | {eta}")
print(f"\nTotal mined: {total:.8f} BTC")
print(f"Spendable now: {spendable:.8f} BTC")
Here 100 - confirmations is the maturity countdown, and multiplying by thirty seconds turns it into a wall-clock estimate on our particular chain. The guard against negatives matters because matured coins keep accumulating confirmations well past 100, and I did not enjoy my explorer telling me a coin would be spendable in negative four hundred blocks.
A live block ticker
import time
from datetime import datetime
def block_ticker(poll=2):
last = None
print("Watching for new blocks. Ctrl+C to stop.\n")
try:
while True:
tip = rpc("getbestblockhash")
if tip != last:
block = rpc("getblock", [tip, 1])
stamp = datetime.now().strftime("%H:%M:%S")
print(f"[{stamp}] NEW BLOCK #{block['height']} | "
f"{block['nTx']} tx | diff {block['difficulty']:.4f}")
last = tip
time.sleep(poll)
except KeyboardInterrupt:
print("\nStopped.")
Leaving this running in a terminal means every line that appears is a block somebody in the room mined, which is oddly compelling to watch. Two design notes are worth making here. It compares the tip hash rather than the height, which matters because during a chain reorganisation the height can stay the same or even go backwards while the tip hash changes, so watching the hash catches that and watching the height does not, and on a chain where laptops are racing at thirty-second intervals you will very likely see a reorg at some point. If you want to catch one explicitly, print a warning whenever the new block's previousblockhash is not the hash you saw last time round.
It also polls, which is slightly rude, because Bitcoin Core can push notifications instead, either through ZeroMQ using something like zmqpubhashblock=tcp://127.0.0.1:28332 in your config, or by running a script on every new block with blocknotify=. Polling every two seconds is fine for a demonstration and wasteful for anything real.
Then multisig, apparently out of nowhere
The session ended with a pivot into 2-of-2 multisig that felt like a non sequitur until the reason landed, and it only fully landed for me during Day 3.
A normal output is locked with the condition "prove you hold the key for this hash," while a 2-of-2 multisig output is locked with "provide valid signatures from both of these two keys," which means neither party can move the coins alone.
Now consider the problem Lightning is solving, which is that every on-chain payment has to be broadcast, validated by every node on earth and stored forever, and that is an absurd amount of ceremony for buying a coffee. The insight is that you only need the blockchain to settle disputes rather than to record every single payment, so two people can lock coins into a 2-of-2 multisig with one on-chain transaction, then exchange signed transactions that would split that balance differently and simply not broadcast any of them. Alice pays Bob and they sign a new split, then another, then a thousand more, all of it instant and free and private and none of it touching the chain, and when they are finished they broadcast the final agreed split as one more on-chain transaction. Two transactions, unlimited payments in between, and the multisig is what makes it safe because neither party can move the funds alone and therefore neither can steal.
This is also where Day 1's SegWit section suddenly mattered, because the second step requires signing transactions that spend the funding transaction before the funding transaction has confirmed. If the funding transaction's ID could still change after broadcast, which it could before SegWit fixed transaction malleability, then every one of those pre-signed transactions would become worthless and the coins could be stuck permanently. I had read the sentence "malleability had to be fixed before Lightning could exist" before that week, but this was the first time I understood what it actually meant.
The homework question I enjoyed most
One of the homework items asked why every coinbase fee on the room chain was zero, and what fills that role on mainnet.
The first half is easy once you look at a block, because our blocks were about 406 bytes, which is roughly the size of a coinbase transaction and nothing else, since nobody was transacting. No user transactions means no fees, so the coinbase paid out the block subsidy on its own.
The second half is much more interesting, and it is the part I have kept thinking about since. On mainnet, block space is scarce and users bid for it, so fees currently add a modest amount on top of the subsidy, but the subsidy halves roughly every four years and is heading toward zero, which means that eventually fees have to become the entire security budget of the network. Whether fee revenue will actually be sufficient to secure the chain once the subsidy is negligible is a genuinely open question with serious people arguing different positions, and that is the real answer to the homework, which is not what I expected to be thinking about after two days of a bootcamp.
If you want to see the difference concretely, go back to regtest, send a few transactions, mine a block and decode that coinbase, because it will exceed the subsidy by exactly the sum of the fees.
Things that broke, and what they meant
A short list, because I hit most of these either on the day or in the weeks afterwards.
If Python throws a ConnectionError then check your port, since 18443 is regtest and 38332 is signet, and I chased that one for longer than I want to admit. If bitcoin-cli works but your script does not, remember that bitcoin-cli reads the cookie file automatically while your script is probably using hardcoded credentials that may well be stale.
Error -19 means multiple wallets are loaded and you need to specify which one, while error -18 means the wallet is not loaded at all, or it is sitting in a different network's directory. A KeyError on address means you hit an OP_RETURN output and a KeyError on previousblockhash means you hit the genesis block, and both of those want .get() instead.
If your balance arithmetic is subtly wrong then you are using floats, and if transactions appear in what feels like the wrong order then that is listtransactions returning oldest first and working as intended. If the same transaction ID appears several times, that is one transaction paying several of your addresses, which is also working as intended.
And if getconnectioncount says 0 while everyone else's block count keeps climbing, you are not actually peered, so check getpeerinfo, check your addnode line, and check your signetchallenge character by character.
What has stayed with me
Three things: the API was always there, because bitcoin-cli is a wrapper around an HTTP endpoint that has been running on my machine the whole time, and once that clicked, "building a Bitcoin application" stopped sounding like something other people did and started sounding like something I could do on a Tuesday.
Difficulty is the mechanism that makes the whole thing work, more than hashing or blocks or even proof of work considered on its own, because the self-adjusting target is what keeps block times steady across a trillion-fold change in hashpower, and watching it respond live to a single ASIC being plugged into a wall was the moment it stopped being an abstraction for me.
And custody is a choice that is usually made badly, because our pool paid every miner directly inside the coinbase transaction and never held anyone's money, while most pools do not work that way at all, and a remarkable number of the horror stories in this industry come down to somebody holding something they did not need to be holding.
Next up: Lightning, which is where channels, invoices and HTLCs come in, along with the discovery that a payment can route through a complete stranger without them being able to steal it.
Where I went to check things
Most of this was taught in a room, but a fair amount of it I only understood after going and reading the following, and I would recommend all of them if you want to go deeper than a bootcamp session allows.
- Bitcoin Core RPC documentation at bitcoincore.org, particularly the entries for
getblockchaininfo,getrawtransaction,getblock,listtransactionsandgetbalances. The per-command help built into your own node (bitcoin-cli help <command>) is the same material and is genuinely excellent. - BIP 325: Signet, by Karl-Johan Alm and Anthony Towns, which explains why testnet is unreliable, what the signet challenge is, and how the signet solution is committed inside the coinbase transaction. Available in the bitcoin/bips repository on GitHub.
- BIP 141: Segregated Witness, for the witness commitment and the reasoning behind putting it in the coinbase.
- Bitcoin Core PR Review Club at bitcoincore.reviews, which has sessions on both signet and the
getrawtransactionverbosity change, and is a surprisingly readable way into how these features were actually designed. - Mastering Bitcoin by Andreas M. Antonopoulos, for the chapters on mining and on transaction structure. The extranonce and the mechanics of pool payouts are covered properly there.
- learnmeabitcoin.com by Greg Walker, which has the clearest explanation of the block header layout, the nonce, and the target that I have found anywhere.
- Bitcoin Optech topic pages, for the ongoing discussion about the long-term fee market and the security budget question.
- The Python
decimalmodule documentation, for why floating point is the wrong tool for money.
If you spot something I have got wrong, I would genuinely like to hear it, because I am still early in this and a correction is worth more to me than a compliment. Leave a comment.
If you found this useful, subscribe to be notified when I publish the next one.