How to expose Kubernetes Services to Internet Access?

Question

I just configured a Kubernetes Cluster with this environment

4 VPS nodes with one public IP for each node

K3S Cluster with embebed etcd (k3s version v1.22.7+k3s1)

• 3 master nodes
• 1 Worker node just for testing

MetalLB for Internal Load Balancer (metallb/v0.12.1)

• IP Range 10.10.0.200-10.10.0.250

Traefik as default Kubernetes Ingress Class (Chart v10.19.4 & App v2.6.3)

Every thing is running as expected, I can access all services inside each node in the cluster.

Now, how to finally expose services to Internet Acesss?

• Cloud Provider Firewall already exposing ports 80 and 443
• Internal iptables firewall accept public traffict from those ports

I thought Traefik automatically expose port 80 and 443, but lsof actually is not showing as "LISTEN". and pubic ips not responding anything. I am really confused at this, I am newby in kubernetes world.

I have tried port forwarding private ip to metallb load balancer ip but it actually not solve the route.

iptables -t nat -I PREROUTING -p tcp -d <enp0s3-local-ip> --dport 80 -j DNAT --to-destination <load-balancer-ip>:80
iptables -I FORWARD -m state -d <load-balancer-subnet>/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT

Edit: The nodes and traefik already showing the public ip

But response from outside the cluster still curl: (56) Recv failure: Connection reset by peer

Answer 1

Try using the kubectl expose command:

$ kubectl expose (-f FILENAME | TYPE NAME) [--port=port] [--protocol=TCP|UDP|SCTP] [--target-port=number-or-name] [--name=name] [--external-ip=external-ip-of-service] [--type=type]

--external-ip=Additional external IP address (not managed by Kubernetes) to accept for the service. If this IP is routed to a node, the service can be accessed by this IP in addition to its generated service IP.

Or when you install traefik add this value file (as traefik.yaml in this case):

service:
  externalIPs:
    - <your_external_static_ip_here_without_the_brackets>

and then install it like this:

helm install --values=./traefik.yaml traefik traefik/traefik -n traefik --create-namespace

Refer to the stackpost and a document on Exposing applications using services for more information.

Answer 2

At the end I solved using a simple nginx proxy stream (tcp,udp) to the MetalLB Load Balancer IP. Here is an example if someone else with this case.

stream {

    # Redirect Web Requests
    upstream traefik_http {
        server 10.10.0.200:80;
    }
    server {
    listen [::]:80;
        listen 80;
    proxy_pass traefik_http;
    }

    # Redirect Websecure Requests
    upstream traefik_https {
        server 10.10.0.200:443;
    }
    server {
    listen [::]:443;
        listen 443;
        proxy_pass traefik_https;
    }
}

Answer 3

Deploy nginx ingress controller and make the ingress controller service as Load Balancer. Define ingress rules for each service that you want to access over internet.