Breaking Django ORM migrations and blue-green deployment

• Database Migrations
• Deployment
• Devops
• Django
• Web

Published on 2024-03-28

Backward incompatible migrations

Some migrations may be backward incompatible and will break production for a shot period of time between the migration application and pod rotation. This also will be a problem for a blue/green deployment if you to decide to implement it. To avoid this kind of downtime migration should be done in 2 steps.

Caution

• Use this method only for the listed below operations. This method is dangerous. It can dissync the state of the app from DB. Never use self written SQL. Always copy it from the sqlmigrate command.
• This type of migration is hard to revert for obvious reasons, so be extra careful doing it.
• Doing this operation without using SeparateDatabaseAndState will result into a downtime. Workers will refuse to start if change is made to the model, but not applied to the django internal state.

List of breaking migration operations

• RemoveField
• DeleteModel
• AddField when the field is not nullable (there is no null=True in the field declaration)

Solution

Create 2 PR’s and merge them into 2 separate releases:

Usages removal PR

1. Remove all the usages of the field from the code
2. Remove the field from the model
3. Autogenerate a migration (./manage.py makemigrations your_app)
4. Retrieve the sql of the migration (we’ll need it later) (./manage.py sqlmigrate your_app 0002_auto_20240328_1527)
5. Move the breaking operations to SeparateDatabaseAndStates state_operations (example below).

Database changes PR

1. Create empty migration (./manage.py makemigrations your_app --empty)
2. Add SeparateDatabaseAndState again
3. Put the SQL extracted in step 4 of previous PR creation into database_operations. Without comments and transaction parts.

Example

Autogenerated migration:

from django.db import migrations

class Migration(migrations.Migration):

    dependencies = [
        ('team_management', '0001_auto_20240319_1146'),
    ]

    operations = [
        migrations.RemoveField(
            model_name='organization',
            name='session_expiration_time',
        ),
    ]

PR 1

from django.db import migrations

class Migration(migrations.Migration):

    dependencies = [
        ('team_management', '00001_auto_20240319_1146'),
    ]

    operations = [
        migrations.SeparateDatabaseAndState(
            state_operations=[
                migrations.RemoveField(
                    model_name='organization',
                    name='session_expiration_time',
                ),
            ]
        ),
    ]

PR 2

from django.db import migrations

class Migration(migrations.Migration):

    dependencies = [
        ('team_management', '0002_auto_20240328_1527'),
    ]

    operations = [
        migrations.SeparateDatabaseAndState(
            database_operations=[
                migrations.RunSQL(
                    sql=(
                        'ALTER TABLE "team_management_organization" DROP COLUMN "session_expiration_time" CASCADE;'
                    ),
                ),
            ]
        ),
    ]