Since my project has so many moving parts.. probably best to explain the symptom
I have 1 scheduler running on 1 queue. I add scheduled jobs ( to be executed within seconds of the scheduling).
I keep repeating scheduling of jobs with NO rq worker doing anything (in fact, the process is completely off). In another words, the queue should just be piling up.
But ALL of a sudden.. the queue gets chopped off (randomly) and first 70-80% of jobs just disappear.
Does this have anything to do with:
- the "max length" of queue? (but i dont recall seeing any limits)
- does the scheduler automatically "discard" jobs where the start time is BEFORE the current time?
ran my own experiment. RQ scheduler does indeed remove jobs whose start date < now.