All projects

Recovery engineering / RestoreProof

The restore finished.
Does the application work?

A recovery-testing project that looks past a successful database restore to the application workflows that depend on it.

Scope
Controlled project workload
Status
Implemented · Public source (MIT)
Focus
Recovery · Python · AWS S3
Result
Healthy and failure paths tested

Why this project

I’m interested in recovery workflows where the backup’s identity, the application checks and the cleanup are all visible in the result. This case study describes what the project actually implements.

The problem

A completed restore is only part of recovery. The data can be back while the application’s essential workflows are still broken. A useful recovery exercise has to make that difference visible and keep failed attempts as evidence.

The approach

The workflow pairs an isolated restore with application-level checks and readable results. The backup is selected and integrity-checked before the restored workload is evaluated. Failures and cleanup are part of the outcome, so an unsuccessful attempt can’t quietly turn into a clean-looking report.

Key decisions

Check behavior after the restore

Restoring data and verifying application behavior are separate stages. That makes it possible to say which stage failed, instead of reducing recovery to a single pass/fail flag.

Keep the input explicit

Version and integrity checks make the backup traceable. A mismatch stops the exercise rather than letting an ambiguous result pass downstream.

Make cleanup observable

Temporary recovery environments have a lifecycle. Cleanup belongs in the report, because leftover resources are also a sign of an unsuccessful exercise.

Evidence & limits

Backup integrity mismatch

If a backup’s version or checksum doesn’t match, the recovery exercise stops. Cleanup still shows up in the outcome.

Implemented checks with recorded results from a controlled workload. Recorded results are in the public repo; not a production recovery guarantee.

The implementation includes lifecycle handling, storage integrity checks and explicit failure outcomes. Existing CI results were confirmed during this review. The recorded recovery suite covers a healthy path and expected failure cases. Separate project records document real S3 retrieval using a temporary AWS session and local containers. 245 automated tests run in public CI on every change.

What this doesn’t prove

This is evidence for a controlled workload, not a production recovery-time guarantee. Scheduled cloud recovery and GitHub-to-AWS role assumption haven’t been validated yet. The test suite was not rerun independently for this case study; its CI runs are public on GitHub.

Evidence last reviewed 7 October 2026. This is a high-level case study: results taken from project records are kept separate from checks rerun for this review, and Chisel's code and internal reports are not published.

Takeaway

Recovery is a chain of conditions. A good tool shows which link failed, keeps the evidence, and leaves the environment in a known state.