Aborting queries on neo4jrb

Question

I am running something along the lines of the following:

results = queries.map do |query|
  begin
    Neo4j::Session.query(query)
  rescue Faraday::TimeoutError
    nil
  end
end

After a few iterations I get an unrescued Faraday::TimeoutError: too many connection resets (due to Net::ReadTimeout - Net::ReadTimeout) and Neo4j needs switching off and on again.

I believe this is because the queries themselves aren't aborted - i.e. the connection times out but Neo4j carries on trying to run my query. I actually want to time them out, so simply increasing the timeout window won't help me.

I've had a scout around and it looks like I can find my queries and abort them via the Neo4j API, which will be my next move.

Am I right in my diagnosis? If so, is there a recommended way of managing queries (and aborting them) from neo4jrb?

Answer 1

Rebecca is right about managing queries manually. Though if you want Neo4j to automatically stop queries within a certain time period, you can set this in your neo4j conf:

dbms.transaction.timeout=60s

You can find more info in the docs for that setting.

The Ruby gem is using Faraday to connect to Neo4j via HTTP and Faraday has a built-in timeout which is separate from the one in Neo4j. I would suggest setting the Neo4j timeout as a bit longer (5-10 seconds perhaps) than the one in Ruby (here are the docs for configuring the Faraday timeout). If they both have the same timeout, Neo4j might raise a timeout before Ruby, making for a less clear error.

Answer 2

Query management can be done through Cypher. You must be an admin user.

To list all queries, you can use CALL dbms.listQueries;.

To kill a query, you can use CALL dbms.killQuery('ID-OF-QUERY-TO-KILL');, where the ID is obtained from the list of queries.

The previous statements must be executed as a raw query; it does not matter whether you are using an OGM, as long as you can input queries manually. If there is no way to manually input queries, and there is no way of doing this in your framework, then you will have to access the database using some other method in order to execute the queries.

Answer 3

So thanks to Brian and Rebecca for useful tips about query management within Neo4j. Both of these point the way to viable solutions to my problem, and Brian's explicitly lays out steps for achieving one via Neo4jrb so I've marked it correct.

As both answers assume, the diagnosis I made IS correct - i.e. if you run a query from Neo4jrb and the HTTP connection times out, Neo4j will carry on executing the query and Neo4jrb will not issue any instruction for it to stop.

Neo4jrb does not provide a wrapper for any query management functionality, so simply setting a transaction timeout seems most sensible and probably what I'll adopt. Actually intercepting and killing queries is also possible, but this means running your query on one thread so that you can look up its queryId in another. This is the somewhat hacky solution I'm working with atm:

class QueryRunner
  DEFAULT_TIMEOUT=70

  def self.query(query, timeout_limit=DEFAULT_TIMEOUT)
    new(query, timeout_limit).run
  end

  def initialize(query, timeout_limit)
    @query = query
    @timeout_limit = timeout_limit
  end

  def run
    start_time = Time.now.to_i
    Thread.new { @result = Neo4j::Session.query(@query) }
    sleep 0.5

    return @result if @result

    id = if query_ref = Neo4j::Session.query("CALL dbms.listQueries;").to_a.find {|x| x.query == @query }
      query_ref.queryId
    end

    while @result.nil?
      if (Time.now.to_i - start_time) > @timeout_limit
        puts "killing query #{id} due to timeout"
        Neo4j::Session.query("CALL dbms.killQuery('#{id}');")
        @result = []
      else
        sleep 1
      end
    end
    @result
  end
end