how to allow a worker to run a ffmpeg command on heroku for my python/django app?

Question

I've been stuck trying to figure this out for weeks. I previously asked a similar question found here but I never got any replies. I really cannot find any good documentation anywhere. All I need to do is use a worker (don't care what worker have django-celery and rq installed) to convert a file to flv when it is uploaded from a form. I was able to get this done easily locally, but after over a week I haven't been able to get it to work no matter what I have tried. I tried adding a tasks.py file for celery, or a worker.py file for rq, and I have no idea what else (if anything) needs to be done, such as in my settings.py or Procfile. My procfile looks like:

web: gunicorn lftv.wsgi -b 0.0.0.0:$PORT
celeryd: celery -A tasks worker --loglevel=info
worker: python worker.py

My requirements.txt showing what I have installed looks like this:

Django==1.4.3
Logbook==0.4.1
amqp==1.0.6
anyjson==0.3.3
billiard==2.7.3.19
boto==2.6.0
celery==3.0.13
celery-with-redis==3.0
distribute==0.6.31
dj-database-url==0.2.1
django-celery==3.0.11
django-s3-folder-storage==0.1
django-storages==1.1.6
gunicorn==0.16.1
kombu==2.5.4
pil==1.1.7
psycopg2==2.4.5
python-dateutil==1.5
pytz==2012j
redis==2.7.2
requests==1.1.0
rq==0.3.2
six==1.2.0
times==0.6

The only thing relevant in my settings.py are as follows:

BROKER_BACKEND = 'django'
BROKER_URL = #For this I copy/pasted the code from my redistogo add-on from heroku. Not sure if correct
BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 1800}

Without trying to take up too much more space, my tasks.py looks like this:

import subprocess

@task
def ffmpeg_conversion(input_file):
    converted_file = subprocess.call(input_file)
    return converted_file

I use S3 to store my static and media files, and the upload works (adding uploads to my bucket), however no matter what I try the conversion never will. Is there a good tutorial for absolute beginners? I followed the heroku redis tutorial, celery docs, rq docs, and whatever else I can find, and got the examples to work, but the worker will not execute the command from my view. For example one of the many things I tried:

...
ffmpeg = "ffmpeg -i %s -acodec mp3 -ar 22050 -f flv -s 320x240 %s" % (sourcefile, targetfile)
ffmpegresult = ffmpeg_conversion.delay(ffmpeg)
...

or using rq

...
q = Queue(connection=conn)
result = q.enqueue(ffmpeg_conversion, ffmpeg)
...

I seems like it should be simple, however I am completely self-taught and have never deployed a project whatsoever, and there just doesn't seem to be any good documentation or tutorial available for what I am trying to do. I can't judge whether I am completely off and completely missing something significant or relatively close to getting this to work. I really do appreciate any input whatsoever, this is driving me nuts. Thanks in advance.

Answer 1

I'm trying to do something similar and I don't think you need to create a worker. Why not use a custom build pack, such as https://github.com/integricho/heroku-buildpack-python-ffmpeg. If you had ffpmeg installed at the OS level, you could then issue video transformation commands from python itself, without the need for a separate worker.

Hope that helps ?