After updating EKS cluster to 1.22 all websites are down. Pods are ok but all the networking is not working.
I don't know how to fix ingresses and load balancer.
- I have tried updating deprecated API versions for ingress-kong and internal-ingress-kong.
- I can't find yaml file for alb-ingress-controller, but when I check last applied it is based on new API.
- I have manually updated docker image of alb from 1.1.8 to 2.4.1
Name: alb-ingress-controller
Namespace: default
CreationTimestamp: Thu, 03 Sep 2020 02:05:01 +0000
Labels: app=alb-ingress-controller
app.kubernetes.io/name=alb-ingress-controller
git_version=54709a8bd94f795b1184b0c8336e9a6ec8aee807
name=alb-ingress-controller
version=20200909005829
Annotations: deployment.kubernetes.io/revision: 9
Selector: app.kubernetes.io/name=alb-ingress-controller
Replicas: 1 desired | 1 updated | 1 total | 0 available | 1 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=alb-ingress-controller
app.kubernetes.io/name=alb-ingress-controller
git_version=54709a8bd94f795b1184b0c8336e9a6ec8aee807
name=alb-ingress-controller
version=20200909005829
Annotations: kubectl.kubernetes.io/restartedAt: 2022-04-14T19:19:01Z
Service Account: alb-ingress-controller
Containers:
alb-ingress-controller:
Image: docker.io/amazon/aws-alb-ingress-controller:v2.4.1
Port: <none>
Host Port: <none>
Args:
--watch-namespace=default
--ingress-class=alb-ingress-controller
--cluster-name=staging-trn
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetAvailable
Available False MinimumReplicasUnavailable
OldReplicaSets: <none>
NewReplicaSet: alb-ingress-controller-c46ff7bd9 (1/1 replicas created)
Events: <none>I'm new to kubernetes and aws.
I think I have updated deprecated APIs in all places but errors are still pointing to the old APIs.
Error on ingresses:
E0415 07:54:29.332371 1 reflector.go:153] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105: Failed to list *v1beta1.Ingress: the server could not find the requested resource (get ingresses.extensions)
Error on alb:
{"level":"error","ts":1650009210.0149224,"logger":"setup","msg":"unable to create controller","controller":"TargetGroupBinding","error":"no matches for kind \"TargetGroupBinding\" in version \"elbv2.k8s.aws/v1beta1\""}
I have created missing CRD TargetGroupBindings:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.5.0
creationTimestamp: null
labels:
app.kubernetes.io/name: alb-ingress-controller
name: targetgroupbindings.elbv2.k8s.aws
spec:
group: elbv2.k8s.aws
names:
kind: TargetGroupBinding
listKind: TargetGroupBindingList
plural: targetgroupbindings
singular: targetgroupbinding
scope: Namespaced
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
description: TargetGroupBinding is the Schema for the TargetGroupBinding API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: TargetGroupBindingSpec defines the desired state of TargetGroupBinding
properties:
networking:
description: networking provides the networking setup for ELBV2 LoadBalancer
to access targets in TargetGroup.
properties:
ingress:
description: List of ingress rules to allow ELBV2 LoadBalancer
to access targets in TargetGroup.
items:
properties:
from:
description: List of peers which should be able to access
the targets in TargetGroup. At least one NetworkingPeer
should be specified.
items:
description: NetworkingPeer defines the source/destination
peer for networking rules.
properties:
ipBlock:
description: IPBlock defines an IPBlock peer. If specified,
none of the other fields can be set.
properties:
cidr:
description: CIDR is the network CIDR. Both IPV4
or IPV6 CIDR are accepted.
type: string
required:
- cidr
type: object
securityGroup:
description: SecurityGroup defines a SecurityGroup
peer. If specified, none of the other fields can
be set.
properties:
groupID:
description: GroupID is the EC2 SecurityGroupID.
type: string
required:
- groupID
type: object
type: object
type: array
ports:
description: List of ports which should be made accessible
on the targets in TargetGroup. If ports is empty or unspecified,
it defaults to all ports with TCP.
items:
properties:
port:
anyOf:
- type: integer
- type: string
description: The port which traffic must match. When
NodePort endpoints(instance TargetType) is used,
this must be a numerical port. When Port endpoints(ip
TargetType) is used, this can be either numerical
or named port on pods. if port is unspecified, it
defaults to all ports.
x-kubernetes-int-or-string: true
protocol:
description: The protocol which traffic must match.
If protocol is unspecified, it defaults to TCP.
enum:
- TCP
- UDP
type: string
type: object
type: array
required:
- from
- ports
type: object
type: array
type: object
serviceRef:
description: serviceRef is a reference to a Kubernetes Service and
ServicePort.
properties:
name:
description: Name is the name of the Service.
type: string
port:
anyOf:
- type: integer
- type: string
description: Port is the port of the ServicePort.
x-kubernetes-int-or-string: true
required:
- name
- port
type: object
targetGroupARN:
description: targetGroupARN is the Amazon Resource Name (ARN) for
the TargetGroup.
type: string
targetType:
description: targetType is the TargetType of TargetGroup. If unspecified,
it will be automatically inferred.
enum:
- instance
- ip
type: string
required:
- serviceRef
- targetGroupARN
type: object
status:
description: TargetGroupBindingStatus defines the observed state of TargetGroupBinding
properties:
observedGeneration:
description: The generation observed by the TargetGroupBinding controller.
format: int64
type: integer
type: object
type: object
additionalPrinterColumns:
- jsonPath: .spec.serviceRef.name
description: The Kubernetes Service's name
name: SERVICE-NAME
type: string
- jsonPath: .spec.serviceRef.port
description: The Kubernetes Service's port
name: SERVICE-PORT
type: string
- jsonPath: .spec.targetType
description: The AWS TargetGroup's TargetType
name: TARGET-TYPE
type: string
- jsonPath: .spec.targetGroupARN
description: The AWS TargetGroup's Amazon Resource Name
name: ARN
priority: 1
type: string
- jsonPath: .metadata.creationTimestamp
name: AGE
type: date
Ingress resource should be updated as follows:
apiVersion: networking.k8s.io/v1
pls see examples here: https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource
to find ingress resources type the following: kubectl get ingress --all-namespaces
then do the modification as mentioned above
pls be noted that backend configuration in ingress resource also needs some modification due to api change
also please be noted that from version 1.18 you're able to bind ingress resources using spec.ingressClassName field. If Omitted, ingress will work only if ingressClass that ingress controller implements is set to default.