sorl-thumbnail won't delete thumbnails

Question

Having issues with SORL Thumbnail and deleting thumbnails files or refreshing thumbnails when a file is overwritten. The scenario is that I have a file that for each entry is always the same but can be overwritten. Need the thumbnail to be recreated when a new file is uploaded and the old file is overwritten.

This is at the model + form level so I'm using the low level API to generate thumbs.

Have tried using:

from sorl.thumbnail import delete

delete(filename)

But with no success, the thumbnail is never deleted or overwritten.

I have even tried:

from sorl.thumbnail.images import ImageFile
from sorl.thumbnail import default

image_file = ImageFile(filename)
default.kvstore.delete_thumbnails(image_file)

Again with no success.

Please help!

Update:

I found a work around by creating an alternate ThumbnailBackend and a new _get_thumbnail_filename method. The new method uses a file's SHA-1 hash to always have a thumbnail specific to the current file.

Here's the backend for anyone else that might encounter a similar scenario.

class HashThumbnailBackend(ThumbnailBackend):
  
  def _get_thumbnail_filename(self, source, geometry_string, options):
    """
    Computes the destination filename.
    """
    import hashlib
    
    # hash object
    hash = hashlib.sha1()
    
    # open file and read it in as chunks to save memory
    f = source.storage.open(u'%s' % source, 'rb')
    while True:
      chunk = f.read(128)
      if not chunk:
        break
      hash.update(hashlib.sha1(chunk).hexdigest())
    
    # close file
    f.close()
    
    hash.update(geometry_string)
    hash.update(serialize(options))
    key = hash.hexdigest()
    
    # make some subdirs
    path = '%s/%s/%s' % (key[:2], key[2:4], key)
    return '%s%s.%s' % (settings.THUMBNAIL_PREFIX, path,
                        self.extensions[options['format']])

Answer 1

Its a little hard to explain so I made this awesome table. the first column's commands are listed below, the other columns marks wheter it deletes using an X. Original is the original file, thumbnails the thumbnails for the original and KV means the Key Value store reference.

| Command | Original | Thumbnails | KV Original | KV Thumbnails |
| #1      | X        | X          | X           | X             |
| #2      |          | X          |             | X             |
| #3      |          | X          | X           | X             |

1. sorl.thumbnail.delete(filename)
2. sorl.thumbnail.default.kvstore.delete_thumbnails(image_file)
3. sorl.thumbnail.delete(filename, delete_file=False)

As I understand it you really want to do #3. Now, your problem... a guess is that filename does not refer to a filename relative to MEDIA_ROOT (if you are using another storage backend the situation would be similar). But I think I need to know what you are doing besides this to get a better picture, note that ImageFields and FileFields do not overwrite, also note that django changed the deletion behaviour in 1.2.5, see release notes.

Update: Anyone reading this should note that the above way to generate thumbnail filenames is extremely inefficient, please do not use if you care anything at about performance.

Answer 2

I saw this problem. It was happening because Sorl was being used oddly.

All the thumbnail were got in the following style:

sorl.thumbnail.get_thumbnail(self.picture.url, geometry_string, **options)
# picture being a FieldFile

And when deleting the thumbnail (removing them from cache) it was being done like this:

sorl.thumbnail.delete(self.picture.name, delete_files=False)

Shortly, we were using the image's URL to generate and fetch the thumbnails, and when deleting we were using the image's name. Although Sorl didn't complain about it, the KV Store and the FS weren't never cleaned up.

The fix was to just change the get_thumbnail name argument to self.picture.name.

Answer 3

The thing is you cannot use the shortcut delete(file) with a File class that is different to the one you employed to generate that very thumbnail through get_thumbnail() or the {% thumbnail ...%} templatetag.

The reason is ImageFile instances constructed from the file objects will get differents keys (ImageFile.key) and delete() will never be able to retrieve the good thumbnails to remove because the keys don't match.

I'm not sure that it won't works if you use a Python File object and then a Django File object for instance, but in Django, if you generate the thumbnail with a FileField object and try to delete it (and its thumbnails) with a File instance, it will not works for sure.

So, in your templates, don't do:

{% load thumbnail %}
{% thumbnail image "100" as im %}
<img src="{{ im.url }}" width="{{ im.width }}" height="{{ im.height }}">
{% endthumbnail %}

Where image is a models.ImageField instance, but use its file attribute:

{% load thumbnail %}
{% thumbnail image.file "100" as im %}

And to delete it in your Python code (the following is an example of Storage to overwrite the existing file if the name is the same):

from django.core.files.storage import FileSystemStorage
from django.core.files import File
from sorl.thumbnail import delete


class OverwriteStorage(FileSystemStorage):
    def _save(self, name, content):
        if self.exists(name):
            img = File(open(os.path.join(self.location, name), "w"))
            delete(img)
        return super(OverwriteStorage, self)._save(name, content)

Not sure if its a bug in sorl or if there is a good reason to generate different keys.

Answer 4

I'm not completely sure whether this answers your question, but I was having the same problem and this was my solution.

I have a model with a FileField on it, like such:

material = models.FileField(upload_to='materials')

When handling an uploaded file, I use get_thumbnail() to generate the thumbnail, passing the FileField in as the parameter vs the python level file behind it. ie:

thumb = get_thumbnail(modelinstance.material, '%dx%d' % (thumb_width, thumb_height))

As with your issue, I also found that when a file had the same name, sorl would just grab the thumbnail from the cache instead of generating a new one. Aggravating!

What worked was using sorl's delete method and passing the FileField. I first tried passing in the python file behind the FileField object, which is possibly what you were trying? Going from this:

sorl.thumbnail.delete(modelinstance.material.file)

To this:

sorl.thumbnail.delete(modelinstance.material)

Seemed to line up with sorl-thumbnail's KV Store, and would properly get the cached thumbnail out of the way so the new one could be created from the new file. Yay!

This was helpful for me: http://sorl-thumbnail.readthedocs.org/en/latest/operation.html

Also, even after running ./manage.py thumbnail cleanup and ./manage.py thumbnail clear, I couldn't get Django to stop looking for the old thumbnails in the same place. I had to manually clear the Django cache (I'm using memcached). Here's how you can do that:

import os

# Set the DJANGO_SETTINGS_MODULE environment variable.
os.environ['DJANGO_SETTINGS_MODULE'] = "yourproject.settings"

from django.core.cache import cache

# Flush!
cache._cache.flush_all()

This is my first SO answer. Hope it helps someone :)