Is there a way to end-to-end test a controller-runtime operator in conjunction with a client-go fake kubernetes clientset?

Question

I have an operator written in go using sigs.k8s.io/controller-runtime and I'm trying to write an end to end test for it. The test looks something like:

Feature: Operator deletes pods in a namespace when asked

  Scenario: Operator deletes pods on label updated to deletionRequested
    Given the target K8s cluster has got a namespace called "a525bb67-fda5-42cd-84e4-5d604e9f0156"
    And the namespace "a525bb67-fda5-42cd-84e4-5d604e9f0156" has got the following labels
      | state    | active |
    And the namespace "a525bb67-fda5-42cd-84e4-5d604e9f0156" has got the following releases installed with Helm
      | my-service    | 1.1.1 | https://example.com/artifactory/helm-test |
    And operator is running
    When the state label of "a525bb67-fda5-42cd-84e4-5d604e9f0156" is updated to "toBeDeleted"
    Then the namespace "a525bb67-fda5-42cd-84e4-5d604e9f0156" has the following labels
      | state    | deleted |
    And the namespace "a525bb67-fda5-42cd-84e4-5d604e9f0156" has no releases installed with Helm

What happens is that there is an event where the label is updated to "toBeDeleted" which the operator picks up based on a defined predicate. Then the operator will clear out the pods and update the label to "deleted". This is not expected to be instant so the "Then" steps poll until true with a timeout.

In other parts of the system I have k8s.io/client-go/kubernetes/fake working nicely to do things like setting and getting labels but I'm struggling to find a way to integrate it with the controller-runtime tests.

Is there a way of passing the fake clientset to the controller-runtime manager? (From my digging around in the source it seems to use a rest client and client object to communicate...) Or should I approach setting and updating the label differently for this test?

Thank you!

Answer 1

Unfortunately, as far as I know, the recommended way of testing the manager is via envtest.

I've tried using fake client via ClientBuilder, but this doesn't seem to work.

However, you should be able to structure your code in a way, that e2e tests only test that controller loops are running, leader election is happening etc. Controller loop logic itself can be tested with fake client, by passing whatever you normally pass using manager.GetClient().