Connect to kafka cluster from openshift with istio and tls encryption

173 Views Asked by At

Found this articles, but had't found suitable answer:

Connect to external Kafka brokers via istio egress gateway

https://istio.io/latest/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/

I need to connect from OpenShift pod with istio sidecar to external kafka cluster which is not in OpenShift. there is an app inside pod with kafka consumer plaintext protocol setting. i need to catch it's traffic with egressgateway and forward it with TLS to kafka 9093 port. Help me, please, with creating istio items for that (SE, VS, GW, DR).

now i have working solution, but only using same internal\external ports in virtualService. Here is my configuration:

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: kafka-se1
  namespace: someNameSpace
spec:
  exportTo:
    - .
  hosts:
    - somehost1.somedomain.com
  location: MESH_EXTERNAL
  ports:
    - name: tcp-9093
      number: 9093
      protocol: tcp
    - name: tcp-18084
      number: 18084
      protocol: TCP
  resolution: DNS
=======================================
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: kafka-gw1
  namespace: someNameSpace
spec:
  selector:
    istio: egressgateway-someNameSpace  #my istio\app label of egressgateway deployment
  servers:
    - hosts:
        - somehost1.somedomain.com
      port:
        name: tcp-1814
        number: 1814
        protocol: TCP
==========================================================
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: kafka-vs1
  namespace: someNameSpace
spec:
  exportTo:
    - .
  gateways:
    - kafka-gw1
    - mesh
  hosts:
    - somehost1.somedomain.com
  tcp:
    - match:
        - gateways:
            - mesh
          port: 9093
      route:
        - destination:
            host: egressgateway  #my egressgateway deployment name
            port:
              number: 1814
    - match:
        - gateways:
            - kafka-gw1
          port: 1814
      route:
        - destination:
            host: somehost1.somedomain.com
            port:
              number: 9093
==========================================================
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: kafka-dr1
  namespace: someNameSpace
spec:
  exportTo:
    - .
  host: somehost1.somedomain.com
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
    portLevelSettings:
      - port:
          number: 9093
        tls:
          caCertificates: /etc/istio/egressgateway-ca-certs/ca-chain.cert.pem
          clientCertificate: /etc/istio/egressgateway-certs/tls.crt
          mode: MUTUAL
          privateKey: /etc/istio/egressgateway-certs/tls.key
          sni: somehost1.somedomain.com

and there are three of each item (SE, GW, VS, DR) differs in dest.address. and it's working. It connects with tls to kafka cluster.

but if i change port to 18084 in VS like that:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: kafka-vs1
  namespace: someNameSpace
spec:
  exportTo:
    - .
  gateways:
    - kafka-gw1
    - mesh
  hosts:
    - somehost1.somedomain.com
  tcp:
    - match:
        - gateways:
            - mesh
          port: 18084
      route:
        - destination:
            host: egressgateway  #my egressgateway deployment name
            port:
              number: 1814
    - match:
        - gateways:
            - kafka-gw1
          port: 1814
      route:
        - destination:
            host: somehost1.somedomain.com
            port:
              number: 9093

that leads to connection timeout in app and "UX, URX" in egressgateway deployment logs.

tried different solutions but no luck. Only UX URX in sidecar logs or same UX URX in egressgateway logs. My main goal is to get an opportunity to connect different apps in same namespace to one kafka cluster but with different tls certificates.

0

There are 0 best solutions below