Spring Cloud ZUUL not proxying call to newly added kubernetes service

Question

We've set of microservices built using Spring Cloud stack, I was using Eureka service as service discovery while developing it but when it comes to moving it to Kubenretes we had two option

1. Keep using Eureka as discovery service
2. Start using native Kubernetes service

So, we opted for #2 because it makes more sense to use Kubernetes native service than having another layer of redirection.

We are using Spring Cloud Zuul as API Gatway enabled with @DiscoveryClient to fetch services deployed in namespace so that Zuul can proxy calls to underlying services, the setup works well until we add a new service to namespace, the API Gateway is not able to figure out newly added service thus not able to proxy call to it, however, if API Gateway is restarted then it discovers the services added, so it seems like it's not able to resolve the newly added service on the fly, is there a configuration or jar which is missing from my setup

Here is my API Gateway pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.org.apigw</groupId>
    <artifactId>api-gateway</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <java.version>1.8</java.version>
        <spring-cloud.version>Hoxton.SR9</spring-cloud.version>
    </properties>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.5.RELEASE</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>


    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>com.org.starters</groupId>
                <artifactId>org-starters</artifactId>
                <version>0.0.1-SNAPSHOT</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-zuul</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-kubernetes-all</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
        </dependency>
        <dependency>
            <groupId>com.auth0</groupId>
            <artifactId>java-jwt</artifactId>
            <version>3.11.0</version>
        </dependency>

        <dependency>
            <groupId>com.org.starters</groupId>
            <artifactId>org-observability-starter</artifactId>
        </dependency>
        <!-- enable ribbon auto retries -->
        <dependency>
            <groupId>org.springframework.retry</groupId>
            <artifactId>spring-retry</artifactId>
        </dependency>

        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger-ui</artifactId>
            <version>2.9.2</version>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger2</artifactId>
            <version>2.9.2</version>
        </dependency>

    </dependencies>


    <build>
        <finalName>${project.artifactId}</finalName>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>


</project>

My API Gateway config looks like below :-

spring:
  application:
    name: api-gateway
  cloud.kubernetes.discovery.enabled: false

server:
  port: 9191

countryCode: ""

opa:
   host: http://localhost:8181
   
jwt:
   secret: "this is test secret key for development"
   expiration: 86400000
   tokenValidationEnabled: true

# fix gateway timeouts
hystrix.command.default.execution.timeout.enabled: false

ribbon:
  ReadTimeout: 30000
  ConnectTimeout: 30000
  MaxAutoRetries: 1
   
zuul:
   ribbon:
      eager-load:
         enabled: true
   host:
      connect-timeout-millis: 30000
      socket-timeout-millis: 30000
      # maxTotalConnections: 1700
      # maxPerRouteConnections: 100
   sensitiveHeaders:
   ignoredServices: ""
   #ignoredPatterns:
    #  - /as/*
   routes:
      service1:
         path: /svc1/**
         serviceId: svc1
         stripPrefix: true
         customSensitiveHeaders: false
      service2:
         path: /svc1/**
         serviceId: svc2
         stripPrefix: true
         customSensitiveHeaders: false
   include-debug-header: true
   debug.request: true
   debugFilters.disabled: false
   

logging.level:
   com.netflix.zuul: debug