I have the following scenario and need to know the best way to achieve it in AWS.
I have an EC2 autoscaling group which will initially be set to 0, after a manually run process an SSM document is triggered which sets the ASG to 3, however I need each instance to be fully up and running before the next of the 3 is started launching.
The current setup launches all 3 instances simultaneously and as they form a cluster, them all launching at the same time is seeming to cause issues, they need to wait for some setup/caching actions to take place before another node is safe to join.
As well as this no node is to be up for more than 20 days, so max instance age is set on the instances and I'm using a lifecycle hook to gracefully terminate the instance, but what I need to achieve is that when each node reaches it's 20 days it doesn't begin its lifecycle hook to terminate until its replacement node is running.
I'm trying to determine how much of this I can rely on ASG for and how much I should be handing out to automation external to the ASG, eg Lambda.
Any advice appreciated,
Maximum Instance Lifetime won't ever replace more than 10% of the total instances in an ASG at once (so in your case, never more than 1 at a time): https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-max-instance-lifetime.html
For the initial launching, you could either increment it by hand a few minutes apart if its a 1 time thing, or make your start up script increment one at a time with a sleep in between (step functions may work well for this)