Connect to external Kafka brokers via istio egress gateway

3k Views Asked by At

My app deployed in openshift cluster needs to connect to 2 external kafka brokers. Since the application is on the istio mesh, all outbound traffic must go through the egress gateway. The connection to kafka is via the log4j2 appender over SSL. I made the following istio config:

kind: ServiceEntry
metadata:
  name: se-kafka
spec:
  hosts:
    - kafka1.host.com
    - kafka2.host.com
  addresses:
    - 10.200.200.1
    - 10.200.200.2
  ports:
    - name: kafka-port
      number: 9093
      protocol: TCP
  location: MESH_EXTERNAL
  resolution: NONE
  exportTo:
    - .
=====================
kind: DestinationRule
metadata:
  name: dr-kafka
spec:
  host: egressgateway #name egressgateway deployment
  subnets:
    - name: se-kafka
=====================
kind: Gateway
metadata:
  name: gw-kafka
spec:
  servers:
    - hosts:
        - kafka1.host.com
      port:
        name: kafka1-egress-port
        number: 16001
        protocol: TCP
    - hosts:
        - kafka2.host.com
      port:
        name: kafka2-egress-port
        number: 16002
        protocol: TCP
 selector:
   istio: egressgateway
=======================
kind: VirtualService
metadata:
  name: vs-kafka
spec:
  hosts:
    - kafka1.host.com
    - kafka2.host.com
  gateways:
    - mesh
    - gw-kafka
  tls:
    - match:
        - gateways:
            - mesh
          port: 9093
          sniHosts:
            - kafka1.host.com
      route:
        - destination:
            host: egressgateway
            port:
              number: 16001
    - match:
        - gateways:
            - mesh
          port: 9093
          sniHosts:
            - kafka2.host.com
      route:
        - destination:
            host: egressgateway
            port:
              number: 16002
    - match:
        - gateways:
            - gw-kafka
          port: 16001
          sniHosts:
          - kafka1.host.com
      route:
        - destination:
            host: kafka1.host.com
            port:
              number: 9093
    - match:
        - gateways:
            - gw-kafka
          port: 16002
          sniHosts:
          - kafka2.host.com
      route:
        - destination:
            host: kafka2.host.com
            port:
              number: 9093
========================

It works. But I think that traffic bypasses the istio egressgateway. There is no connection in kiali between ServiceEntry and Egressgateway. And if you look at the egressgateway logs, you can see the following warning:

gRPC config for envoy.api.v2.ClusterLoadAssigment rejected: malformed IP address: kafka1.host.com. Consider setting resolver_name or setting cluster type to 'STRICT_DNS' or 'LOGICAL_DNS'

What is the problem and how to properly configure the egress gateway?

0

There are 0 best solutions below