How to Connect knative-eventing to RabbitMQ External Source

Question

I am trying to connect Knative with an existing RabbitMQ queue outside the Kubernetes cluster, but after testing and looking at the documentation I am not sure this can be done, does anyone know how to use Knative-eventing with an external RabbitMQ as a message source ?

Regards!

Answer 1

As listed in the Knative Eventing sources catalog, there is a RabbitMQ source being developed and maintained by the community (largely contributors from VMware and the RabbitMQ team).

Documentation and install instructions are here; in the case of bugs, please report them in the associated repo.

Answer 2

Hey @Juan you were absolutely right about our docs, here I'll leave a PR fixing part of them while it's being reviewed and merged, and with a clear example about using external RabbitMQ instances: https://github.com/knative-sandbox/eventing-rabbitmq/pull/786/files#diff-4fdb9e4eb3a1c9da58e4445d94aa5ce4573b5c8d005f20c41c767b07c09a2418

Hope this helps =), and thanks for the feedback! If you find anything wrong, you can comment on the PR or leave a reply over here!

Answer 3

thank you @sameer for your answer, I think I am not understanding you correctly.

After visiting the link you give me, I have launched:

kubectl apply --filename https://github.com/knative-sandbox/eventing-rabbitmq/releases/latest/download/rabbitmq-source.yaml

To install the rabbitmq source and generated the secret:

kubectl create secret generic rabbitmqc-default-user -n knative-eventing --from-literal=user=root --from-file=password=/tmp/password

And the following resource:

apiVersion: sources.knative.dev/v1alpha1
kind: RabbitmqSource
metadata:
  name: rabbitmq-source
  namespace: knative-eventing
spec:
  broker: "host.external.dns:5671/"
  connectionSecret:
    name: "rabbitmqc-default-user"
  user:
    secretKeyRef:
      name: "rabbitmqc-default-user"
      key: "username"
  password:
    secretKeyRef:
      name: "rabbitmqc-default-user"
      key: "password"
  exchangeConfig:
    name: "logs"
    type: "fanout"
    durable: true
    autoDelete: false
  queueConfig:
    name: "test"
    routingKey: ""
    durable: false
    autoDelete: false
  channelConfig:
    parallelism: 10
  sink:
    ref:
      apiVersion: v1
      kind: Service
      name: recorder

But after generating the resources I get these errors:

knative-sources/rabbitmq-controller-manager

│ W0524 11:45:04.108665       1 reflector.go:324] k8s.io/[email protected]/tools/cache/reflector.go:167: failed to list *v1beta1.Binding: the server could not find the requested resource (get bindings.rabbitmq.com)                                                                   │
│ E0524 11:45:04.108718       1 reflector.go:138] k8s.io/[email protected]/tools/cache/reflector.go:167: Failed to watch *v1beta1.Binding: failed to list *v1beta1.Binding: the server could not find the requested resource (get bindings.rabbitmq.com)                                 │
│ W0524 11:45:04.109915       1 reflector.go:324] k8s.io/[email protected]/tools/cache/reflector.go:167: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com)                                                                       │
│ E0524 11:45:04.109935       1 reflector.go:138] k8s.io/[email protected]/tools/cache/reflector.go:167: Failed to watch *v1beta1.Queue: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com)                                       │
│ W0524 11:45:04.109976       1 reflector.go:324] k8s.io/[email protected]/tools/cache/reflector.go:167: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com)                                                                 │
│ E0524 11:45:04.109985       1 reflector.go:138] k8s.io/[email protected]/tools/cache/reflector.go:167: Failed to watch *v1beta1.Exchange: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com)                              │
│ W0524 11:45:05.253806       1 reflector.go:324] k8s.io/[email protected]/tools/cache/reflector.go:167: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com)                                                                 │
│ E0524 11:45:05.254035       1 reflector.go:138] k8s.io/[email protected]/tools/cache/reflector.go:167: Failed to watch *v1beta1.Exchange: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com)                              │
│ W0524 11:45:05.443628       1 reflector.go:324] k8s.io/[email protected]/tools/cache/reflector.go:167: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com)

The truth is that I have searched and searched and I have not found information on how to connect it to an external RabbitMQ to Kubernetes, maybe I am not understanding the documentation well.

Answer 4

This error message indicates that xxx.rabbitmq.com CRD is not installed in the current k8s cluster.

│ W0524 11:45:04.109915 1 reflector.go:324] k8s.io/[email protected]/tools/cache/reflector.go:167: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com) │

You need to install the RabbitQA messaging-topology-operator in order to add the missing CRDS

as mentioned in the docs:

Note: An external RabbitMQ instance can be used, but if you want to use the Source without predeclared resources (specifically the Exchange and Queue), the RabbitMQ Message Topology Operator needs to be installed in the same Kubernetes Cluster as the Source.

follow this command :

kubectl apply -f https://github.com/rabbitmq/messaging-topology-operator/releases/latest/download/messaging-topology-operator-with-certmanager.yaml