How to schedule Sidekiq jobs according to a recurring schedule in database?

Question

I've set up a system where Measurements are run based on MeasurementSettings. Each setting defines a schedule using ice_cube (and some other properties for measurements). I'm able to edit this schedule for each setting and then poll for the next occurrences with something like:

def next_occurrences
  schedule.occurrences(Time.now + 1.day)
end

This gives me a set of timestamps when there should be a Measurement.

Now, I also have Sidekiq installed, and I can successfully run a measurement at a specific time using a MeasurementWorker. To do that, I just create an empty Measurement record, associate it with its settings, and then perform_async (or perform_at(...)) this worker:

class MeasurementWorker
  include Sidekiq::Worker
  sidekiq_options retry: 1, backtrace: true

  def perform(measurement_id)
    Measurement.find(measurement_id).run
  end
end

What's missing is that I somehow need to create the empty measurements based on a setting's schedule. But how do I do this?

Say I have the following data:

• MeasurementSetting with ID 1, with a daily schedule at 12:00 PM
• MeasurementSetting with ID 2, with an hourly schedule at every full hour (12:00 AM, 01:00 AM, etc.)

Now I need to:

• Create a Measurement that uses setting 1, every day at 12:00
• Create a Measurement that uses setting 2, every full hour
• Call the worker for these measurements

But how?

Should I check, say every minute, whether there is a MeasurementSetting that is defined to occur now, and then create the empty Measurement records, then run them with the specific setting using Sidekiq?

Or should I generate the empty Measurement records with their settings in advance, and schedule them this way?

What would be the simplest way to achieve this?

Answer 1

Here's how I successfully did it:

• Update your model that should run on a scheduled basis with a field planned_start_time. This will hold the time when it was planned to start at.

• Use the whenever Gem to run a class method every minute, e.g. Measurement.run_all_scheduled

• In that method, go through each setting (i.e. where the schedule is), and check if it is occurring now:

  setting = MeasurementSetting.find(1) # get some setting, choose whatever
  schedule = setting.schedule
  if not schedule.occurring_at? Time.now
    # skip this measurement, since it's not planned to run now
  

• If that's the case, then check if we can run the measurement by looking in the database if there isn't any previous measurement with the same planned start time. So first, we have to get the planned start time for the current schedule, e.g. when it's now 3:04 PM. the planned start time could have been 3:00 PM.

  this_planned_start_time = schedule.previous_occurrence(Time.now).start_time
  

  Then we check if the last measurement's start time (limit(1) gets just the last one) is the same or not.

  if Measurement.limit(1).last.planned_start_time != this_planned_start_time
    # skip this one, since it was already run
  

• If not, we can continue setting up a measurement.

  measurement = Measurement.create measurement_setting: setting, 
                                   planed_start_time: this_planned_start_time
  

• Then run it:

  measurement.delay.run
  

Answer 2

I would use cron every minute to look for MeasurementSettings that should run NOW. If yes, create a Sidekiq job to run immediately which populates the Measurement.