Rollback

The speed of the fix decides how bad a mistake is. Without a rollback, a bad release means debugging live while customers hit errors. With one, the choice is simple: restore the last good version first, then investigate calmly. This also changes behaviour. Teams that can undo a release in minutes ship smaller changes more often, because a mistake costs little.

• Keep releases small. A small change is quicker to diagnose and safer to reverse.
• Practise a rollback before it is needed, so the steps are known and not invented under pressure.
• Watch error rates and key pages straight after each release, so a problem is spotted by the team first.
• Decide in advance who can trigger a rollback, and agree that rolling back needs no justification.
• Afterwards, write down what went wrong and add a check that would have caught it.

Every release replaces something that worked with something new, and sometimes the new version breaks. A rollback undoes the release. The live product goes back to the last version known to be good, and the faulty change is set aside to be fixed.

On a modern hosting platform this is often a single action. Vercel, for example, keeps previous deployments and can point the live site at an older one in moments. Databases are harder. A change to the data itself may not undo cleanly, so a rollback plan has to say what happens to records created while the bad version was live.

• Treating a rollback as a failure to be avoided, which delays the decision while customers are affected.
• Forgetting the database. Code can be reverted in minutes, but a changed data structure may need its own plan.