Getting Exception while Leader Election : Unable to update LeaseLock

66 Views Asked by At

While testing with this leadership election example:
https://github.com/fabric8io/kubernetes-client/blob/main/kubernetes-examples/src/main/java/io/fabric8/kubernetes/examples/LeaderElectionExamples.java

... I am getting an error like below:

      {"@timestamp":"2024-01-23T19:10:59.440984762Z","@version":"1","message":"Exception occurred while acquiring lock 'LeaseLock: default - concurrent-example-lease (Concurrent-22)'","logger_name":"io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector","thread_name":"-727539330-pool-13-thread-23","level":"ERROR","level_value":40000,"stack_trace":"io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.LockException: Unable to update LeaseLock
    at io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.LeaseLock.update(LeaseLock.java:102)
    at io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.tryAcquireOrRenew(LeaderElector.java:209)
    at io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.lambda$acquire$4(LeaderElector.java:151)
    at io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.lambda$loop$8(LeaderElector.java:253)
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PUT at: https://172.20.0.1/apis/coordination.k8s.io/v1/namespaces/default/leases/concurrent-example-lease. Message: Operation cannot be fulfilled on leases.coordination.k8s.io \"concurrent-example-lease\": the object has been modified; please apply your changes to the latest version and try again. Received status: Status(apiVersion=v1, code=409, details=StatusDetails(causes=[], group=coordination.k8s.io, kind=leases, name=concurrent-example-lease, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Operation cannot be fulfilled on leases.coordination.k8s.io \"concurrent-example-lease\": the object has been modified; please apply your changes to the latest version and try again, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Conflict, status=Failure, additionalProperties={}).
    at io.fabric8.kubernetes.client.KubernetesClientException.copyAsCause(KubernetesClientException.java:238)
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.waitForResult(OperationSupport.java:517)
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:551)
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleUpdate(OperationSupport.java:347)
    at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleUpdate(BaseOperation.java:680)
    at io.fabric8.kubernetes.client.dsl.internal.HasMetadataOperation.lambda$replace$0(HasMetadataOperation.java:167)
    at io.fabric8.kubernetes.client.dsl.internal.HasMetadataOperation.replace(HasMetadataOperation.java:172)
    at io.fabric8.kubernetes.client.dsl.internal.HasMetadataOperation.replace(HasMetadataOperation.java:113)
    at io.fabric8.kubernetes.client.dsl.internal.HasMetadataOperation.replace(HasMetadataOperation.java:41)
    at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.replace(BaseOperation.java:1043)
    at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.replace(BaseOperation.java:88)
    at io.fabric8.kubernetes.client.extended.leaderelection.resourcelock.LeaseLock.update(LeaseLock.java:100)
    ... 7 common frames omitted
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PUT at: https://172.20.0.1/apis/coordination.k8s.io/v1/namespaces/default/leases/concurrent-example-lease. Message: Operation cannot be fulfilled on leases.coordination.k8s.io \"concurrent-example-lease\": the object has been modified; please apply your changes to the latest version and try again. Received status: Status(apiVersion=v1, code=409, details=StatusDetails(causes=[], group=coordination.k8s.io, kind=leases, name=concurrent-example-lease, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Operation cannot be fulfilled on leases.coordination.k8s.io \"concurrent-example-lease\": the object has been modified; please apply your changes to the latest version and try again, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Conflict, status=Failure, additionalProperties={}).
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:709)
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:689)
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.assertResponseCode(OperationSupport.java:640)
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.lambda$handleResponse$0(OperationSupport.java:576)
    at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(Unknown Source)
    at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
    at java.base/java.util.concurrent.CompletableFuture.complete(Unknown Source)
    at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.lambda$retryWithExponentialBackoff$2(OperationSupport.java:618)
    at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source)
    at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source)
    at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
    at java.base/java.util.concurrent.CompletableFuture.complete(Unknown Source)
    at io.fabric8.kubernetes.client.okhttp.OkHttpClientImpl$4.onResponse(OkHttpClientImpl.java:277)
    at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
    ... 3 common frames omitted
","appname":"test-test1-connector"}

Stack trace made readable by https://TidyTrace.com

the dependency version used is:

<dependency>
    <groupId>io.fabric8</groupId>
    <artifactId>kubernetes-client</artifactId>
</dependency>

Would anyone be able to explain why I am getting these errors and how to solve this?

Tried:

changing the version to 6.5.0, 6.9.2 and 6.10.0. but with these versions, i am getting below error:

        {"@timestamp":"2024-01-23T17:15:27.912309596Z","@version":"1","message":"Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'testLeaderElectionSampleCode' defined in URL [jar:file:/test-microservice.jar!/BOOT-INF/classes!/io/test/test1/connector/processor/TestLeaderElectionSampleCode.class]: Unsatisfied dependency expressed through constructor parameter 0: No qualifying bean of type 'io.fabric8.kubernetes.client.KubernetesClient' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {}","logger_name":"org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext","thread_name":"main","level":"WARN","level_value":30000,"appname":"test-test1-connector"}
    {"@timestamp":"2024-01-23T17:15:27.914034355Z","@version":"1","message":"destroy jedis pool","logger_name":"io.test.test1.connector.cache.redis.RedisStore","thread_name":"main","level":"INFO","level_value":20000,"appname":"test-test1-connector"}
    {"@timestamp":"2024-01-23T17:15:28.020439885Z","@version":"1","message":"\n\n***************************\nAPPLICATION FAILED TO START\n***************************\n\nDescription:\n\nParameter 0 of constructor in io.test.test1.connector.processor.TestLeaderElectionSampleCode required a bean of type 'io.fabric8.kubernetes.client.KubernetesClient' that could not be found.\n\n\nAction:\n\nConsider defining a bean of type 'io.fabric8.kubernetes.client.KubernetesClient' in your configuration.\n","logger_name":"org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter","thread_name":"main","level":"ERROR","level_value":40000,"appname":"test-test1-connector"}
0

There are 0 best solutions below