Job is visible only from redis cli but not showing in rq dashboard and not executed

Question

I want to build a pipeline using Redis and RQ. I created a worker, server and a job, the worker is running and listening to queue, the server is dispatching a job to a queue, the job is dispatched and I print the job ID, in console, I can see the worker logs sth that receive a job in a queue. The job is never executing and never shows in rq dashboard, but I can see it in Redis CLI.

Verions I am using:

rq==1.7.0

redis==3.5.0

Here is my code:

Worker in run.py

import os

import redis
from rq import Worker, Queue, Connection

listen = ['stance_queue','default']

redis_url = os.getenv('REDIS_URL', 'redis://redis:6379')

conn = redis.from_url(redis_url)

# conn = redis.Redis(host='redis', port=6379)

if __name__ == '__main__':
    with Connection(conn):
        print("Createing worker")
        worker = Worker(map(Queue, listen))
        # worker = Worker([Queue()])
        worker.work()

And here were I dispatch a job

from workers.stance.run import conn

q = Queue('default', connection=conn)

@server.route("/task")
def home():

    if request.args.get("n"):
        print('create a job in default queue')
        job = q.enqueue( background_task,  args=(20,))
        return f"Task ({job.id}) added to queue at {job.enqueued_at}"

    return "No value for count provided"

And here is the background job

def background_task(n):

    """ Function that returns len(n) and simulates a delay """

    delay = 2

    print("Task running", flush=True)
    print(f"Simulating a {delay} second delay", flush=True)

    time.sleep(delay)

    print(len(n))
    print("Task complete")

    return len(n)  

Here is a screenshot for rq-dashboard

And here is the logs in the worker

Attaching to annotators_server_stance_worker_1
stance_worker_1      | Createing worker
stance_worker_1      | 08:33:44 Worker rq:worker:cae161cf792b4c998376cde2c0848291: started, version 1.7.0
stance_worker_1      | 08:33:44 Subscribing to channel rq:pubsub:cae161cf792b4c998376cde2c0848291
stance_worker_1      | 08:33:44 *** Listening on stance_queue, default...
stance_worker_1      | 08:33:44 Cleaning registries for queue: stance_queue
stance_worker_1      | 08:33:44 Cleaning registries for queue: default
stance_worker_1      | 08:33:49 default: home.annotator_server.background_task(20) (9f1f31e0-f465-4019-9dc6-85bc349feab9)

and here is the logs from redis-cli

mpose exec redis redis-cli


127.0.0.1:6379> keys *
1) "rq:workers"
2) "rq:failed:default"
3) "rq:clean_registries:default"
4) "rq:queues"
5) "rq:job:9f1f31e0-f465-4019-9dc6-85bc349feab9"
6) "rq:worker:cae161cf792b4c998376cde2c0848291"
7) "rq:workers:default"
8) "rq:clean_registries:stance_queue"
9) "rq:workers:stance_queue"

And here is my compose

version: '3'
services:
  annotators_server:
    build:  
      context: .
      dockerfile: Dockerfile
    ports:
     - "5000:5000"
    volumes:
     - ./app:/home
    depends_on:
    - redis     
  redis:
    image: "redis:alpine"
  dashboard:
    image: "godber/rq-dashboard"
    ports:
      - 9181:9181
    command: rq-dashboard -H redis
    depends_on:
      - redis      
  stance_worker:
    build:  
      context: ./app/workers/stance
      dockerfile: Dockerfile
    environment: 
      - REDIS_URL=redis://redis:6379
    depends_on:
    - redis             

I never see a logs for the job excution, I tried to add TTL and TIMEOUT but still facing the samething.

Answer 1

Pass the redis database to the connection string, when starting dashboard and worker. Redis url = redis://redis-host:6379/0 ( this refers to db 0 ).