Generating thumbnail manually and uploading it in S3 bucket

Question

My web app is image-centric and when a user uploads an image (any size), I need to create a thumbnail and store so that I can use the thumbnail and not the original image. I use AWS S3 bucket, boto3, django-storages. The file upload works flawlessly, my issue is when I generate a thumbnail and upload to the S3 bucket with different file name (it does not throw error, but I cannot see any thumbnail images generated or stored)

This is my settings.py

Media root

MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

AWS Specific settings (I did not override MEDIA_ROOT)

AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_S3_REGION_NAME = os.getenv('AWS_S3_REGION_NAME')
AWS_QUERYSTRING_AUTH = False
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = os.getenv('AWS_DEFAULT_ACL')
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
AWS_S3_FILE_OVERWRITE=False

# s3 static settings
STATIC_LOCATION = 'static'
STATIC_ROOT = f'https://{AWS_S3_CUSTOM_DOMAIN}/{STATIC_LOCATION}/'
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{STATIC_LOCATION}/'
CKEDITOR_BASEPATH = f'https://{AWS_S3_CUSTOM_DOMAIN}/{STATIC_LOCATION}/ckeditor/ckeditor/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

# s3 public media settings
PUBLIC_MEDIA_LOCATION = 'media'
MEDIA_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{PUBLIC_MEDIA_LOCATION}/'
#MEDIA_ROOT = f'https://{AWS_S3_CUSTOM_DOMAIN}/'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

The code where I upload generated thumbnail files (if it ever generates):

I use the Pillow library for generating thumbnail

Version 1 of code:

tfname = os.path.join(os.path.join(settings.BASE_DIR,"media"), file_name)
image.save(tfname)

Version 2 of the code (saw it somewhere on this site and tried):

import boto3
s3 = boto3.resource('s3')
s3.Bucket(settings.AWS_STORAGE_BUCKET_NAME).upload_file('/{0}'.format(file_name),file_name)

Neither of them works, not throw any exception/error

Answer 1

I would recommend you to use something like thumbor (source code: https://github.com/thumbor/thumbor).

Basically the functionality thumbor offers is to store the transformations made to the original image and caching the transformed image. With an approach like the one this package offers you won't need to manually store a thumbnail image from the original. The cache mechanism allows you to fetch thumbnails faster after the first time you transform the original image. As the documentation states it offers support to AWS S3 services. I'll leave some tutorials that might be helpful (https://dev.to/jrgarciadev/how-to-create-your-own-image-cdn-with-thumbor-and-aws-hpg, https://medium.com/@adam_70793/best-thumbnailing-solution-set-up-thumbor-on-aws-c5625c25a564)

If you prefer to follow the current path, we should check how you're handling the original image upload and try to mimic it with the new thumbnail images.