How to exposing CockroachDB using Ingress on Google Cloud for external loadtesting

Question

For my current distributed databases project in my studies I should deploy a CockrouchDB Cluster on Google Cloud Kubernetes Engine and run a YCSB Loadtest against it.

The YCSB Client is going to run on another VM so that the results are comparable to other groups results.

Now I need to expose the DB Console on Port 8080 as well as the Database Endpoint on Port 26257.

so far I started changing the cockraochdb-public service to kind: NodePort and exposing its ports using an Ingress. My current Problem is exposing both ports (if possible on their default ports 8080 and 26257) and having them accessible from YCSB.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: cockroachdb-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: cockroachdb-global-ip
    ingress.citrix.com/insecure-service-type: “tcp”
    ingress.citrix.com/insecure-port: “6379”
  labels:
    app: cockroachdb
spec:
  backend:
    serviceName: cockroachdb-public
    servicePort: 26257
  rules:
  - http:
      paths:
      - path: /labs/*
        backend:
          serviceName: cockroachdb-public
          servicePort: 8080
      - path: /*
        backend:
          serviceName: cockroachdb-public
          servicePort: 26257

So far I just managed to route it to different paths. I'm not sure if this may work, because the JDBC driver used by YCSB is using TCP not http.

How do I expose two ports of one service using an Ingress for TCP?

Answer 1

Focusing on:

How do I expose two ports of one service using an Ingress for TCP?

In general when an Ingress resource is referenced it's for HTTP/HTTPS traffic.

You cannot expose the TCP traffic with an Ingress like the one mentioned in your question.

Side note!

There are some options to use an Ingress controller to pass the TCP/UDP traffic (nginx-ingress).

---

You could expose your application with service of type LoadBalancer:

apiVersion: v1
kind: Service
metadata:
  name: cockroach-db-cockroachdb-public
  namespace: default
spec:
  ports:
  - name: grpc
    port: 26257
    protocol: TCP
    targetPort: grpc # (containerPort: 26257) 
  - name: http
    port: 8080
    protocol: TCP
    targetPort: http # (containerPort: 8080) 
  selector:
    selector: INSERT-SELECTOR-FROM-YOUR-APP
  type: LoadBalancer

Disclaimer!

Above example is taken from cockroachdb Helm Chart with modified value:

• service.public.type="LoadBalancer"

By above definition you will expose your Pods to external traffic on ports: 8080 and 26257 with a TCP/UDP LoadBalancer. You can read more about it by following below link:

• Cloud.google.com: Kubernetes Engine: Docs: How to: Exposing apps: Creating a service of type LoadBalancer

---

The YCSB Client is going to run on another VM so that the results are comparable to other groups results.

If this VM is located in GCP infrastructure you could also take a look on Internal TCP/UDP LoadBalancer:

• Cloud.google.com: Kubernetes Engine: Using an internal TCP/UDP load balancer

---

Also I'm not sure about the annotations of your Ingress resource:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: cockroachdb-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: cockroachdb-global-ip
    ingress.citrix.com/insecure-service-type: “tcp”
    ingress.citrix.com/insecure-port: “6379”

In GKE when you are creating an Ingress without specifying the ingress.class you are using: gce controller. The ingress.citrix.com annotations are specific to citrix controller and will not work with gce controller.

---

Additional resources:

• Kubernetes.io: Docs: Ingress
• Kubernetes.io: Docs: Service