Ruby Rspec, Finding how many times a worker being called recursively

Question

I need to test a worker on Rspec, to Find how many times the same worker being called again and again in recursive manner. Eg:

Class Myworker
  def perform(id)
    model = Mymodel.find(id)
    associated_records = Mymodel.users.limit(1000)
    associated_records.each(&:destroy)
    if Mymodel.users.exists?
      Myworker.perform_async(id)
    end
  end
end

i need to write rpsec for this to count how many times this iteration happend,

i have tried to stub the worker, and incrementing counter, since i'm stubbing the worker it doesn't execute the same worker again and i'm struck with counter 1 as final value. How to find how many times the worker being called recursively in RSPEC.

Answer 1

I'm going to get in here with a frame challenge: In you specific use case you don't want to call the worker recursively, but you just want to use find_each to iteratively remove all the users from Mymodel, i.e.

Class Myworker
  def perform(id)
    User.where(mymodel_id: id)
        .find_each do |user|
      user.destroy
    end
  end
end

Edit: If you don't like the idea of deleting all association objects within one run of the worker and definitely want to run your worker recursively, then in my opinion your best option would not be to test the actual amount of recursions for your worker, but rather have two test cases:

1. Test that the worker is rescheduled when there are more association records than you delete in one recursion
2. Test that the worker is not rescheduled when all association records can be deleted within a single iteration

That way you also verify that your worker will run as often as needed to delete all records without the need to verify the actual count of recursions used.

(It's a little bit late at my place and I actually do want to get some sleep, so I'm not putting in example code here, let me know if you need it, I can then add some tomorrow)

Answer 2

You are right that when deleting huge amounts of database records, then the job will need more time and ultimately delay other background jobs.

But that should really be an issue because usually you will have multiple workers running at the same time anyway. And when you still have issues with multiple huge jobs delaying more important jobs, then you should think about having multiple queues with different priorities.

When you have multiple priority queues configured in Sidekiq then you can change your job to run on a low priority queue and not block the critical or default queue like this:

class DestroyUsersAssociatedToMyModel
  include Sidekiq::Job
  sidekiq_options queue: 'low'

  def perform(id)
    MyModel.find(id).users.destroy_all
  end
end