How to Autoscale a GCP Managed Instance Group using a Rabbitmq VM outside the group

Question

I am using GCP and I have a specific problem to solve where I want to use the metrics from a RabbitMQ instance to control the autoscaling requirements of a Managed Instance Group. Do note that this RabbitMQ instance is outside this group and is used only to maintain the messages in the queue.

I want to scale up the number of instances in the group, when the number of current messages in the queue exceeds the number of available consumers. I had implemented the same in AWS using Amazon MQ integrated with RabbitMQ to autoscale for an ECS Cluster of Instances.

I have installed an OPS agent on the RabbitMQ Instance so that I can monitor the queue-based stats in a dashboard, but I am not sure how these metrics can be used to scale the instance as no specification on the MIG config page seems to point to the accessibility of these metrics.

My question is that, is it possible to scale the instances in an MIG through the metrics of an external instance like in my case? This question arises because the documentation on GCP seems to point out that autoscaling can be used only on metrics of the instances within the group.

If not, I would like to understand other ways I can implement the same by perhaps monitoring a consumer-based metric.

Answer 1

Custom metrics can be used for triggering the auto scaling feature. This document outlines clearly how to configure custom metrics for triggering auto scaling of MIG instances. The configuration involves three simple steps.

• Create a custom metric for the Rabbitmq queue in cloud monitoring.
• Create a service account and give sufficient permissions for performing auto scaling actions.
• Create a trigger using these custom metrics for scaling the managed instance group.