Uploading large file to GCS with django and django-storages -- direct or multi-part uploads?

109 Views Asked by At

I'm creating a django application that uploads multimedia to Google Cloud Storage using django-storages. My application is running on Google App Engine; I'm using the admin interface for admins to upload new media.

Right now I'm having trouble with models that contain a large number of images -- for example, some of my models contain several 4K images, and at 25MB/ea I'm easily hitting hundreds of MB per save operation.

django-storages is working great for smaller operations, but larger operations are causing issues. I'm currently getting an HTTP 413 error on larger save operations -- my app is POSTing about 100 MB, and Google App Engine's limit is 32MB.

So my question is: to get around the limit, is it possible to make multiple POST requests using django-storages? Like, if my model has 4x 4K images, can I upload them each in a sequential upload?

I've also read that I should upload files "directly" to GCS -- is it possible to do a "direct" upload using django? How would I go about doing that?

And either way, I'd have to change how the admin interface uploads file... how can I go about doing that?

thanks!

0

There are 0 best solutions below