The production database was gone. What saved it was a decision I made on a random Saturday.

Two weeks ago I logged in to check something small and saw it: every app returning unhealthy at once. The production database was gone. Deleted days earlier during a routine cost cleanup, along with the connection string every app still pointed at.

For about ten minutes I felt something I had not felt in years: real panic. Every product I had built shared that database. All of it, gone, unless a backup existed somewhere.

One did. A manual snapshot from three days earlier, taken on a random Saturday for no urgent reason, because past me decided that a database this load-bearing should have a snapshot that did not depend on any automated policy being configured correctly. That decision, made when nothing was on fire, was the entire difference between a bad afternoon and a total loss.

The recovery itself was hours of unglamorous, careful work. Restore the snapshot to a temporary instance. Dump the real data out. Transform it for its new home, including stripping extensions the new environment did not support. Load it, migrate it, repoint every app, and watch health checks go green one by one. By that evening everything was healthy. No data lost. Nobody but me ever knew it happened.

Two lessons stuck, and neither is "have backups," because everyone already knows that.

First: the outcome was decided before the incident started. Nothing I did during the recovery was heroic. It was mechanical execution of options that existed only because of prior boring discipline. Incident response is mostly a replay of decisions you made weeks earlier.

Second: automated policies are not a safety net until you have verified what they actually cover. The automated backups I assumed existed went down with the instance, because they were tied to its lifecycle. The manual snapshot survived precisely because it was outside the machinery that failed.

What is the boring safety net you are glad you built before you needed it? If you cannot name one, that is the first thing to fix this week.