We are using Istio version 1.18.2 on all our AKS (V1.27.3) based envts (Dev,Stg, Prod, Dr, etc.) setup. We have enabled the Istio sidecars injection with namespace injection for Springboot java-based applications.
During our scalability tests, we noticed that when a service scales out, (i.e. from 3 to 4 instances), the new instance after reporting being healthy immediately gets i.e. 25% of the load. Since the service instance did not have a chance to warm up yet, requests sent to it temporarily time out or show increased response time compared to other existing instances. This state can remain from 3-4 minutes to even 15 minutes, depending on the load, from what we saw in our tastings.
We were looking for a scaling related solution so that a new service instances right after scaling out to receive load that they are able to handle. so that scaling out does not cause temporary response time to increase for the production services.
We searched for a solution from istio side and tried as per the solution provided here for Destination Rule. https://stackoverflow.com/questions/74517652/gradually-warm-java-application-in-k8s-with-istio#:~:text=trafficPolicy%3A%0A%20%20%20%20loadBalancer%3A%0A%20%20%20%20%20%20simple%3A%20ROUND_ROBIN%0A%20%20%20%20%20%20warmupDurationSecs%3A%20180s
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: myapp
namespace: myns
spec:
host: "myapp.myns.svc.cluster.local"
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
warmupDurationSecs: "180s"
subsets:
- name: v1
labels:
app: myapp
version: 1.0
In our destination rule, we haven't configured different versions for subsets, and it will be always v1.0 (we don't require the different versions as of now). But this solution of traffic policy also didn't worked as expected, as we tested the scaling behavior and not anew release.
is there any way to achieve this using istio while scaleout happening. ?
Started running test at X TPS load
Increase load to X*4 TPS in 5 minutes
Kept load at X*4 TPS for 30 minutes