Here is out scenario. We have around 12 web servers that are active in a target group today. Those servers are always running. We monitor those servers with prometheus / grafana. Each server is configured with static DNS so that we can attach to a single server at a time for testing if we run into something in the logs on one of them (our logs are aggregated).
I would like to implement an auto scaling group with cloudwatch metrics to spin up or down servers based on utilization. However, the following challenges exist.
- Current DHCP options set the private IP domain to IPADDRESS.ec2.local and our core code base is hardcoded for developemnentdomain.com. We do not use AWS resolvers so the ec2.local domain wont work to attach to a single server in the domain.
- Grafana is configured against the specific IP of each ec2 instance. While we can accept that each instance will go down and come back up, the IPs are of course randomly assigned when the launchtemplate spins up a new instance.
My questions would be:
- Can we use multiple AMIs (like a pool) that the auto scaling group uses to increase active instances during heavy load? Or are we stuck with a single AMI as the system template to be instantiated.
- Can we use a custom domain hosted in Route53 and get the devices that are brought online added to the host list as they are instantiated? This way we can attach through our developer VPN in a private subnet for testing on a machine by machine basis.
Yes, it is possible but you need to use multiple launch templates with different AMIs in a single autoscaling group.
See https://aws.amazon.com/about-aws/whats-new/2020/11/amazon-ec2-auto-scaling-announces-support-for-multiple-launch-templates-for-auto-scaling-groups/