My backup system found a bad stick of RAM
A one-person operation cannot afford a two-week rebuild after a dead machine. So both machines rebuild from one encrypted off-site copy, one script, one passphrase. Then the backups started failing on exactly one machine, and that was the point.
Ask any solo operator what happens if their main machine dies tonight and you get a pause, then something about "the important stuff is in the cloud somewhere." I estimated my own honest answer at two to four weeks of part-time archaeology: reinstalling, hunting credentials, reconstructing configurations that existed nowhere but on that disk. For an operation where the automation is the business, that is not an inconvenience. That is an outage measured in weeks.
What the system does
- Nightly encrypted machine backups to off-site storage, deduplicated, with retention tiers going back months. Both machines, every night.
- A repo sweeper that auto-commits roughly fifty working git repositories nightly, so uncommitted work in progress survives too.
- Credential bundles, encrypted and shipped off-site twice a week, so the keys to everything are recoverable without being scattered.
- One bootstrap script that takes a bare fresh machine to a working environment: restores the home directory, unpacks credentials, reinstalls tooling, restores the schedule of automated jobs.
- One passphrase, held in a password manager, unlocks the whole recovery path.
Ongoing cost: a few dollars a month of storage. Build cost: about six hours, against a worst case measured in weeks.
The part nobody plans for
The best thing the backup system ever did had nothing to do with restoring files.
During the initial runs, the backup tool kept aborting with data-integrity errors on one machine, while the identical setup on the second machine ran clean every night. Same software, same configuration, different hardware. A backup tool that checksums everything it touches is, accidentally, a merciless memory tester, and it was pointing at a machine that already had a history of unexplained crashes. The failing backups became the strongest evidence yet of degrading RAM, on a box I might otherwise have kept trusting with production work.
Monitoring you did not plan is still monitoring. Every system that verifies its own work is also testing the machine underneath it.
What this means for your business
If a vendor builds automation for you, ask two questions: what happens when the machine it runs on dies, and how do you know the backups actually work? A good answer names a script and a passphrase, not a shrug toward the cloud. This whole system is boring, cheap, and it converts a multi-week disaster into an afternoon. Boring is what you are paying for.
Storage locations and key details are omitted. The numbers and the RAM story are real. If your business runs on machines nobody could rebuild, start with a roadmap session.
