django_crontab is adding my jobs, but they don't seem to execute. Any solution?

Question

I got a django project running with django-crontab (github) on Ubuntu 20.

in <my_django_app> directory I added a cron.py file:

from .models import <my_model>
from datetime import datetime


def remove_stamps():
    for stamp in <my_model>.objects.order_by('-stop_date'):
        if stamp.can_be_removed():
            stamp.delete()
        else:
            break


def close_stamps():
    for stamp in <my_model>.objects.filter(stop_date=None):
        stamp.stop_date = datetime.now()
        stamp.save()

in settings:

CRONJOBS = [
('*/4 * * * *', '<my_django_app>.cron.remove_stamps'),
...
]

CRONTAB_LOCK_JOBS = True

I deployed the project as follows:

First created a <new_user> in the command line with @root like this:

adduser --system --home=/var/opt/<project_name> --no-create-home --disabled-password --group --shell=/bin/bash <new_user>

Using Nginx I ran the virtual environment with this <new_user> using gunicorn like this:

    [Unit]
         Description=<project_name>
    [Service]
         User=<new_user>
         Group=<new_user>
         Environment="PYTHONPATH=/etc/opt/<project_name>:/opt/<project_name>"
         Environment="DJANGO_SETTINGS_MODULE=settings"
         ExecStart=/opt/<project_name>/venv/bin/gunicorn \
           --workers=4 \
           --log-file=/var/log/<project_name>/gunicorn.log \
           --bind=127.0.0.1:8000 --bind=[::1]:8000 \
           <project_name>.wsgi:application
    [Install]
         WantedBy=multi-user.target 

Next I added the django_crontab jobs using:

PYTHONPATH=/etc/opt/<project_name>:/opt/<project_name> DJANGO_SETTINGS_MODULE=settings su <new_user> -c "/opt/<project_name>/venv/bin/python3 /opt/<project_name>/manage.py crontab add"

Checking the crontab jobs with .... crontab show gives:

<HASH KEY> -> ('*/4 * * * *', '<my_django_app>.cron.remove_stamps')
<HASH KEY> -> ('*/5 * * * *', '<my_django_app>.cron.close_stamps')

Using journalctl _COMM=cron --since="2021-5-1 14:00" to check if the job runs, gives following:

...
May 01 17:00:01 ubuntu-2gb-hel1-2 CRON[276942]: pam_unix(cron:session): session opened for user <new_user> by (uid=0)
May 01 17:00:01 ubuntu-2gb-hel1-2 CRON[276940]: pam_unix(cron:session): session opened for user <new_user>by (uid=0)
May 01 17:00:01 ubuntu-2gb-hel1-2 CRON[276946]: (<new_user>) CMD (/opt/<project_name>/venv/bin/python3 /opt/<project_name>/manage.py crontab run <HASH KEY>>
May 01 17:00:01 ubuntu-2gb-hel1-2 CRON[276947]: (<new_user>) CMD (/opt/<project_name>/venv/bin/python3 /opt/<project_name>/manage.py crontab run <HASH KEY>>
May 01 17:00:08 ubuntu-2gb-hel1-2 CRON[276945]: pam_unix(cron:session): session closed for user <new_user>
May 01 17:00:08 ubuntu-2gb-hel1-2 CRON[276942]: pam_unix(cron:session): session closed for user <new_user>
...

I got the idea that the cronjob is running correctly but not executing the script. But I cannot figure out how to fix. I'm completely stuck. All help is welcome.

I also tried to add a cronjob using crontab -e , but then I got stuck on how to use it with the <new_user> in the virtualenv.

Answer 1

I found a solution to add cron jobs to my django project. I used the django commands (django documentation). As you can see below, I created a management/commands/ folder in <my_django_app> (part of <django_project>) and added the command script file.

<my_django_app>/
    __init__.py
    models.py
    management/
        __init__.py
        commands/
            __init__.py
            <my_command_script>.py
    tests.py
    views.py

In de script I used the BaseCommand class as explained in the django documentation:

from django.core.management.base import BaseCommand, CommandError
from <my_django_app>.models import <my_model>
from datetime import datetime


class Command(BaseCommand):
    help = 'Closes stamps that have NO end date'

    def handle(self, *args, **options):
        try:
            stamps = <my_model>.objects.filter(date=None)
        except stamps.DoesNotExist:
            raise CommandError('No stamps available to close')

        for stamp in stamps:
            stamp.date = datetime.now()
            stamp.save()

        self.stdout.write(self.style.SUCCESS('Successfully closed the stamps'))

I could execute the command by using following script:

PYTHONPATH=/etc/opt/<project_name>:/opt/<project_name> DJANGO_SETTINGS_MODULE=settings su <new_user> -c "/opt/<project_name>/venv/bin/python3 /opt/<project_name>/manage.py <my_command_script>"

Now I still had to find a way to add this to cron (not an easy search). I fixed it by adding a <my_cron_file>.sh into a random folder. (I assume I could also use /etc/cron.d/, but I've not yet tried it.)

<my_cron_file>.sh :

#!/bin/bash
export PYTHONPATH=/etc/opt/<project_name>:/opt/ <project_name>
export DJANGO_SETTINGS_MODULE=settings
su <new_user> -c "/opt/<project_name>/venv/bin/python /opt/<project_name>/manage.py <my_command_script>"

Next I made this file executable by using:

chmod 755 /<my_folder>/<my_cron_file>.sh

Then I opened the cron jobs with crontab -e and added following line:

*/2 * * * * /<my_folder>/<my_cron_file>.sh

Now the script executes every 2 minutes. It seems to work.

I'm not sure this is the correct way, so if anyone sees an issues, please let me know.