UWSGI timer and cron decorators running duplicate jobs

Question

I have been trying to make the uwsgi python spooler work properly for quite some time. I have a setup in which I run a django application with two worker processes. I have tried setting a cron spooler (and a timer spooler) to run a task every ten minutes, but no matter what configuration of settings I've tried, it always seems to register the signal multiple times, and running the task multiple times.

This is how I run uwsgi:

#!/bin/bash
sudo uwsgi --emperor /etc/uwsgi/vassals --uid http --gid http --enable-threads --pidfile=/tmp/uwsgi.pid --daemonize=/var/log/uwsgi/uwsgi.log

This is my uwsgi vassal config in /etc/uwsgi/vassals/django.ini:

[uwsgi]

chdir           = /home/user/django
module          = django.wsgi

master          = true
processes       = 2
socket          = /tmp/uwsgi-django.sock
vacuum          = true
pidfile         = /tmp/uwsgi-django.pid
daemonize      = /home/user/django/log.log
env = DJANGO_SETTINGS_MODULE=django.settings
#lazy-apps = false
#lazy = false
spooler = %(chdir)/tasks
#spooler-processes = 1
#import = django-app/spooler.py
#spooler-import = django-app/spooler.py
shared-import = django-app/spooler.py

(I have changed some of the path names for privacy reasons). The lines that are commented out are various attempts at making it not duplicate my signals, but every time it seems to register the signal twice, and sometimes even thrice (presumably in both the workers and the single spooler process).

[uwsgi-signal] signum 0 registered (wid: 0 modifier1: 0 target: default, any worker)
[uwsgi-signal] signum 1 registered (wid: 1 modifier1: 0 target: default, any worker)
[uwsgi-signal] signum 1 registered (wid: 2 modifier1: 0 target: default, any worker)

Does anyone know why this is happening, and how to properly prevent it?

This is the spooler.py file:

@cron(-10, -1, -1, -1, -1)
def periodicUpdate(signal):
    print "Running cron job..."
    _getStats()

also tried

@timer(600)
def periodicUpdate(signal):
    print "Running cron job..."
    _getStats()

I also tried adding target='spooler' to the timer/cron-decorator, but it did not seem to many any difference.

Answer 1

Are you sure you do not have other signals registered in django.wsgi, settings.py or other django-related file ? --shared-import will only load things one time (in the master).

Btw i do not get what you are trying to accomplish. This is not how the spooler is supposed to work, and even if you want to use it as a signal handler target you have to specify it when you register signals (with target='spooler' in the decorator)

Answer 2

While this is an old question, I couldn't find the answer elsewhere. I used this solution with Flask, but it should be similar with Django.

During initialization (prefork mode) you need to register a signal.

uwsgi.register_signal(26, "spooler", periodicUpdate)

Then the timer should look like this:

@timer(600, target='spooler')
def periodicUpdate(signal):
    print "Running cron job..."
    _getStats()

As for the comments:

The error 'only the master and the workers can register signal handlers' is correct because you haven't register any signal.

The issue with the

'whenever I load one of the pages in my django application it re-registers it'

can probably happen because its worker is calling the method (periodicUpdate) once. That's why the signal must be register before the workers are being spawned.