How to deal with failed migrations during deployment in Rails?

Question

To evolve a DB schema over time we need to do DB migrations. Rails provide us with great functionality to do that. As we know, each migration in Rails is wrapped in a DDL transaction (at least for Postgres). It means that if some migration is failed in halfway then all changes related to this migration will be rolled back. But, I've noticed that if we have more than one migration and one of them fails then previous migrations will not have been rolled back. And it is a problem in terms of production deployment since a new portion of migrations can be done partially.

Let's imagine we have 5 new migrations that we want to deploy into production.

migration_1.rb
migration_2.rb
migration_3.rb
migration_4.rb
migration_5.rb

If an error occurs for example on migration_4.rb then all previous migrations (migration_1.rb, migration_2.rb, migration_3.rb) will not have been rolled back.

How should I handle this situation?

I have an idea to cancel code deployment and rollback previous migrations by something like this

rails db:rollback STEP=n # n - number of migrations that managed to pass successfully

But then I have another question - what if an error occurs during rolling back?

Answer 1

The best practice I'd recommend: Only have one migration per deploy. Migrations run in a transaction, so if a single one fails, no harm done, assuming the deploy as a whole fails with it (so you don't end up deploying code changes that rely on the failed migration).

But if one succeeds and the next one fails, your database will end up in a state that you didn't intend, and you will have to manually roll back.

If your multiple migrations are related to each other and need to be deployed as one, just refactor them into a single migration file. If they belong to separate features that can be deployed individually, do that instead.

Regarding rollback failures - your migrations should always be able to roll back, and this should be part of your development process. If you migrate data, make sure to migrate it back in the down version, for example.

An easy way to test this is using db:migrate:redo locally. It's good practice to always to that before committing a migration.