How to use a relative path for Django FileStorage location in migrations?

Question

I am defining a Model containing a FileField and cannot save the Files this FileField should contain in the media folder for unrelated reasons. Therefore I need to define a FileStorage that saves into a different Path. When defining said FileStorage in my model and passing it the DJANGO_ROOT variable from my django settings to build my location path, this gets resolved into a system specific path once I generate migrations for my models. Because the path is now specific to my development directories, I cannot apply the generated migration file on all of the production servers and generating migrations on all of the production servers is obviously not an option. I have also tried editing the path inside the auto generated migration itself, making it dependent on the django settings in there. Sadly, the manage.py migrate command tells me that it wants to generate new migrations for that model. How can I pass a relative path to my FileStorage location, allowing me to generate migrations for my production servers? My model:

class ModelWithFile(models.Model):
    file = models.FileField(
        storage=FileSystemStorage(location=os.path.join(settings.DJANGO_ROOT, "folder_name"),
        verbose_name=_("FileObject"),
    ) 

Auto generated migration:

from django.db import migrations, models
from django.conf import settings
import django.core.files.storage

class Migration(migrations.Migration):

    dependencies = [
        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
    ]

    operations = [
        migrations.CreateModel(
            name='ModelWithFile',
            fields=[
                ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
                ('file', models.FileField(upload_to=b'', storage=django.core.files.storage.FileSystemStorage(base_url=None, location='/home/myusername/repros/thisrepro/folder_name/'), verbose_name='File')),
            ],
            options={
                'verbose_name': 'Model With File',
            },
        ),
]

What I tried Pass os.path.join(settings.DJANGO_ROOT, "folder_thats_not_media") as my location.

What I was expecting for the autogenerated migration to use the same relative path as defined in the settings

What did I experience? The location path in my auto generated migration was resolved to a system specific path

Answer 1

It sounds like you are dealing with 2 problems here. For the first:

Use upload_to param on django FileField:

https://docs.djangoproject.com/en/4.1/ref/models/fields/#django.db.models.FileField.upload_to

The function inputs the instance and filename, but you can import settings as well for added flexibility. An example:

def file_upload_path(instance, filename):
    prefix = f'{instance.id}:{instance.__class__.__name__}'
    return os.path.join(f'logos/{prefix}', filename)

class ModelWithFile(models.Model):
    file = models.FileField(storage=FileSystemStorage(), upload_to=file_upload_path)

As for reading locations within a migration script, it sounds like you are looking for django.apps:

In [1]: from django.apps import apps as django_apps                                                                                                                                                         
In [2]: app_config = django_apps.get_app_config('appname')                                                                                                                                                 
In [3]: app_path = app_config.path                                                                                                                                                                          

From this point, you can use the os lib to get to where you need to go from the app location.

More documentation found here: https://docs.djangoproject.com/en/4.1/ref/applications/

Answer 2

I got into the same situation because I needed to limit the circle of users who can upload/download files.

If you specify for:

models.FileField(storage=custom_storage) 

then the migration file will contain the full path to the storage.

 ('file', models.FileField(blank=True, null=True, storage=django.core.files.storage.FileSystemStorage(base_url='/archives/', location=pathlib.PurePosixPath('/home/username/.../archives')), upload_to='file_name_foo_bar'))

I solved this issue by simply overriding the default storage.

STORAGES = {
# ...
"default": {
    "BACKEND": "django.core.files.storage.FileSystemStorage",
    "OPTIONS": {
        "location": BASE_DIR / 'archives',
        "base_url": "/archives/",
    },
},
"staticfiles": {
    "BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage",
},}

As a result, the migration will look like if you did not specify the storage argument

('file', models.FileField(blank=True, null=True, upload_to='my_custome_filename'))

According to the documentation "default" and "staticfiles" must be present otherwise you will get an error when trying to load statics in the template.

storages docs