Creating a k8s controller that watches all kinds in a single api group

Question

I need to write a k8s controller using kubebuilder that watches objects of all kinds in a api group. It should also support dynamic kinds in the api group. That is if a new CRD is installed that belongs to the same api group and an instance of it is created then the controller should pick it up.

Is this possible at all? What are some of the other alternatives? e.g. hard-code the kinds (or take it from a config) and use the watches function to monitor the resources?

Answer 1

Thanks to some of the ideas from comments, I was able to make it work like the below. The code is part of the SetupWithManager function. The idea is to listen for CRDs and then dynamically create controllers based on GVK

builder.Watches(crd, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []reconcile.Request {

        crd := obj.(*v1.CustomResourceDefinition)
        r.Queue.SetContext(ctx)
        customResource := &unstructured.Unstructured{}
        for _, version := range crd.Spec.Versions {
            gvk := schema.GroupVersionKind{
                Group:   crd.Spec.Group,
                Version: version.Name,
                Kind:    crd.Spec.Names.Kind,
            }
            // set GVK for unstructured object
            // this is needed to add a watch unstructured objects
            customResource.SetGroupVersionKind(gvk)
            // Create a new controller dynamically based on the CRD added within an API group
            err := ctrl.NewControllerManagedBy(mgr).
                For(customResource).
                Owns(crd).
                WithLogConstructor(func(request *reconcile.Request) logr.Logger {
                    return mgr.GetLogger()
                }).
                Complete(&myReconciler{client: mgr.GetClient(), gvk: gvk, Queue: r.Queue})
            if err != nil {
                mgr.GetLogger().Error(err, fmt.Sprintf("Unable to create a controller for %s", gvk))
            }
        }
        return []reconcile.Request{}
    }

I also filtered the CRDs using predicate function (not part of snippet above)

and then need a reconciler function like below

func (r *myReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
// this is invoked when objects of the filtered CRDs are created, updated etc.
}

The key really is to set the GVK on unstructured types. Understand this might be not recommended, because Do one thing and do it good. but it works for my specific use case.