So I would like to have the following architecture:
Flask API (VM 1) -> Redis Server (VM 2) <- Worker (VM 3)
I have seen some examples and the idea I get is that on the API you define the tasks that handle the jobs (workers will perform) and push it to Redis queue. What I am not understanding is, how does a remote worker knows the code of the function handler to execute? All the examples I have seen have the worker on the same directory as the API code... Does the Redis Server stores the byte-codes of the function that the worker will execute? Do I have to spawn a worker and link it to the code which has all the task handlers code?
Please elucidate me...
As I understand from this guide code that you want to execute must be on worker machine when you create a job. Example from this guide:
in app.tasks+'name' you describe path for .py file that stored in Worker.
So Redis store info about jobs and Workers execute code that describes in jobs.