I am looking for advice for the following problem:
- I am working with other people on a microservices architecture where the microservices are distributed on different machines. Resources on the machines are very limited.
- Currently, communication runs through a message broker.
- In my use case, one microservice occasionally needs to run some heavy computation. I would like to perform the computation on a machine with low CPU usage and enough available memory space.
My first idea is that every machine installs a microservice which publishes CPU usage and available memory space in the message broker. Each microservice that needs to distribute their workload is looking for the fittest machines and installs "worker"-microservices on the fly. Results are published in the message broker. Since resources are limited, worker-microservices are uninstalled when not needed anymore.
I haven't found a similar use case yet. Do you guys know a better existing solution?
I am quite new to the topic of microservices and distributed computing, so i would appreciate some advice and help.