Flask_APScheduler conflicting job error?

Question

Not sure how to store a job on a database. Whenever I run my flask app it runs the first time and then when I shutdown the server and restart the app I get the error:

sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: apscheduler_jobs.id [SQL: 'INSERT INTO apscheduler_jobs (id, next_run_time, job_state) VALUES (?, ?, ?)'] [parameters: ('job1', 1505488120.453826, <memory at 0x00000200DB4A6DC8>)]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run.py", line 4, in <module>
    scheduler.start()
  File "C:\Anaconda3\envs\g2x_flask\lib\site-packages\flask_apscheduler\scheduler.py", line 82, in start
    self._scheduler.start()
  File "C:\Anaconda3\envs\g2x_flask\lib\site-packages\apscheduler\schedulers\background.py", line 33, in start
    BaseScheduler.start(self, *args, **kwargs)
  File "C:\Anaconda3\envs\g2x_flask\lib\site-packages\apscheduler\schedulers\base.py", line 154, in start
    self._real_add_job(job, jobstore_alias, replace_existing)
  File "C:\Anaconda3\envs\g2x_flask\lib\site-packages\apscheduler\schedulers\base.py", line 851, in _real_add_job
    store.add_job(job)
  File "C:\Anaconda3\envs\g2x_flask\lib\site-packages\apscheduler\jobstores\sqlalchemy.py", line 94, in add_job
    raise ConflictingIdError(job.id)
apscheduler.jobstores.base.ConflictingIdError: 'Job identifier (job1) conflicts with an existing job'

My app\__intel__.py files looks like:

from flask import Flask
from flask_apscheduler import APScheduler
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore

from .models import db


app = Flask(__name__)
app.config.from_object('config.BaseConfig')
db.init_app(app)


class APSConfig(object):
    JOBS = [
        {
            'id': 'job1',
            'func': 'intel_app.jobs:job1',
            'args': (1, 2),
            'trigger': 'interval',
            'seconds': 10
        }
    ]

    SCHEDULER_JOBSTORES = {
        'default': SQLAlchemyJobStore(url=app.config['SQLALCHEMY_DATABASE_URI'])
    }

    SCHEDULER_EXECUTORS = {
        'default': {'type': 'threadpool', 'max_workers': 20}
    }

    SCHEDULER_JOB_DEFAULTS = {
        'coalesce': False,
        'max_instances': 3
    }

    SCHEDULER_API_ENABLED = True


app.config.from_object(APSConfig())

scheduler = APScheduler()
scheduler.init_app(app)

It looks like it is trying to recreate the job with the same jobid everytime the app starts, however, I am not sure how to get it to only create the job once and then if the job is there to use it from the database.

EDIT: My eventual goal is to get up a job that pulls data from a website and then stores the data in a database. So, I would like to set this job up as a cron job and deploy it on heroku.

Answer 1

The error you are getting because the job id "job1" is already exists in jobstore (in database) and you can't have the same job id for the different jobs. Job id should be unique. You can assign id as 'None' and the scheduler will automatically assign one id to it. The new job id will be in alphanumeric of length 32. You can also generate your own uuid and assign it to the job id.

Answer 2

If you are using a datastore, you should set replace_existing to True in the job parameters so that APScheduler will replace the existing job when it restarts. Here is an example cron job that works with SQLAlchemyJobStore:

    'id': 'daily_email',
    'func': send_daily_email,
    'replace_existing': True,
    'trigger': 'cron',
    'misfire_grace_time': 600,
    'max_instances': 1,
    'hour': 22,  # every day at 22:05
    'minute': 5,