Backups, and what "verified by restore" means

What is copied, and when

Nightly at 03:17 UTC, plus on demand. Two kinds of artifact:

• Every book’s ledger, as a git bundle --all — the full history, not the current state. Every commit, every diff, every message.
• The control-plane database, through SQLite’s online-backup API.

Both go to a private object-storage bucket that is not on either machine, and each artifact carries a manifest with its sha256, where it came from, and — for a book — the bean-check result at the moment it was taken.

The odd minute is deliberate. Scheduled jobs queue, and the top of the hour is the most contended slot, so a job on :00 is the one that gets delayed or dropped.

What “verified by restore” means

It means the nightly job does not stop at “the upload returned 200”.

For a book, the bundle is downloaded, its sha256 checked against the manifest, and then it is cloned — and the resulting HEAD and commit count are checked against what was recorded. Cloning rather than git bundle verify, because cloning is what someone actually recovering their books would do.

For the database, the copy is checked for integrity and its row counts are compared to the manifest. That second check is the whole lesson:

A naive cp of the live database was measured against production and produced a file that lost six of seven users, six of seven memberships and five of six books — and reported itself perfectly healthy. integrity_check asks whether a file is well-formed, and a truncated one is.

An integrity check is not a restore. A checksum is not a restore. A restore is a restore, and it runs every night.

Retention

Nothing is ever pruned. There is no lifecycle rule on the bucket and neither script issues a delete. A book’s backups accumulate from the day it was first taken, at roughly a few hundred kilobytes each.

This is deliberate rather than lazy. Deleting a backup when its book row disappears would mean destroying a customer’s only off-machine copy in the same request that deleted their book — so the rule is: never.

Why deleting a book refuses

The control plane will not destroy a book’s machine unless it can confirm that the ledger exists somewhere else. Today it cannot: the verifier it asks returns “cannot be confirmed” for every book, without exception, so a delete request comes back 409.

That is not the same sentence as “there is no off-machine copy” — the nightly backup demonstrably writes one and has been restore-verified. The control plane simply cannot see it yet, and the refusal says “cannot be confirmed” rather than pretending otherwise.

What there is not

No self-service restore. You cannot roll your own book back from a page. The restore tooling downloads and verifies; putting a file back onto a live volume is a deliberate human act, and today that human is us. Email support@countbean.com.

No point-in-time recovery between snapshots. The nightly bundle is the granularity of the backup — though within a book, every change is a commit and history is far finer than a day.

No backup you can take yourself yet. There is no route that hands you a copy of the repository — #48, and it is the honest gap on this page.

No cross-region replication you can choose, and no per-book retention setting. There is one policy and it applies to everyone.