How to trace Que jobs in newrelic?

Question

I want to run my script in a cron job. This script creates multiple Que jobs that do the hard work. How can I trace the activity of each job in new relic?

I have read this post https://blog.newrelic.com/2012/05/30/using-new-relic-to-monitor-ruby-background-tasks/ about how to trace cronjobs, but since the Que job is managed in its own worker (i'm using puma), it doesn't get traced.

This is an example script that tries to do it (but it doesn't trace anything):

class NewrelicTest

  def run
    Order.all.find_each do |order|
      Job.enqueue(order.id)
    end
  end

  class Job < Que::Job
    include NewRelic::Agent::Instrumentation::ControllerInstrumentation

    def run(id)
      order = Order.find(id)
      do_heavy_work_here(order)
    end

    add_transaction_tracer :run, :category => "OtherTransaction/cronjobs"
  end
end

NewRelic::Agent.manual_start :app_name => "Background Jobs"
NewrelicTest.new.run

Thank you

Answer 1

Your script looks good to me, however, New Relic doesn't support Que out of the box.

At least it doesn't appear to be on the list of officially supported frameworks listed in their docs. That could explain why you aren't getting the data you need.

I would definitely look into the possibility of "custom instrumentation" to achieve this. New Relic's Ruby Agent API can be customized to collect/trace additional metrics:

https://docs.newrelic.com/docs/agents/ruby-agent/features/ruby-custom-instrumentation

I'll also add (the link below) is a good place to start when using a background framework unsupported by New Relic:

https://docs.newrelic.com/docs/agents/ruby-agent/background-jobs/monitoring-ruby-background-processes-daemons

Hope that gets you closer.