Django South - schema and data migration at the same time

Question

Isn't it possible to do something like the following with South in a schemamigration?

def forwards(self, orm):
    ## CREATION
    # Adding model 'Added'
    db.create_table(u'something_added', (
        (u'id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
        ('foo', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['something.Foo'])),
        ('bar', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['something.Bar'])),
    ))
    db.send_create_signal(u'something', ['Added'])

    ## DATA
    # Create Added for every Foo
    for f in orm.Foo.objects.all():
        self.prev_orm.Added.objects.create(foo=f, bar=f.bar)

    ## DELETION
    # Deleting field 'Foo.bar'
    db.delete_column(u'something_foo', 'bar_id')  

See the prev_orm that would allow me to access to f.bar, and do all in one. I find that having to write 3 migrations for that is pretty heavy...

I know this is not the "way to do" but to my mind this would be honestly much cleaner.

Would there be a real problem to do so btw?

Answer 1

I guess your objective is to ensure that deletion does not run before the data-migration. For this you can use the dependency system in South.

You can break the above into three parts:

001_app1_addition_migration (in app 1)

then

001_app2_data_migration (in app 2, where the Foo model belongs)

and then

002_app1_deletion_migration (in app 1) with something like following:

class Migration:

    depends_on = (
        ("app2", "001_app2_data_migration"),
    )

    def forwards(self):
        ## DELETION
        # Deleting field 'Foo.bar'
        db.delete_column(u'something_foo', 'bar_id')

Answer 2

First of all, the orm provided by South is the one that you are migrating to. In other words, it matches the schema after the migration is complete. So you can just write orm.Added instead of self.prev_orm.Added. The other implication of this fact is that you cannot reference foo.bar since it is not present in the final schema.

The way to get around that (and to answer your question) is to skip the ORM and just execute raw SQL directly.

In your case, the create statement that accesses the deleted row would look something like:

cursor.execute('SELECT "id", "bar_id" FROM "something_foo"')
for foo_id, bar_id in cursor.fetchall()
    orm.Added.ojbects.create(foo_id=foo_id, bar_id=bar_id)

Answer 3

South migrations are using transaction management.

When doing several migrations at once, the code is similar to:

for migration in migrations:
    south.db.db.start_transaction()
    try:
        migration.forwards(migration.orm)
        south.db.db.commit_transaction()
    except:
        south.db.db.rollback_transaction()
        raise

so... while it is not recommended to mix schema and data migrations, once you commit the schema with db.commit_transaction() the tables should be available for you to use. Be mindful to provide a backwards() method that does that correct steps backwards.