We have a Tekton pipeline task in Openshift which uses Kaniko to build a container image and then push it to a external registry.
The build step of the task always fails with the error:
error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "xxxxxxx.io/xxxxxxxxxxx/frontend:frontend-secure-pipeline-csd6ur": POST https://xxxxxxx.io/v2/xxxxxxxxxxx/frontend/blobs/uploads/: DENIED: You are not authorized to access the specified resource. See https://cloud.ibm.com/docs/Registry?topic=Registry-troubleshoot-access-denied; [map[Action:pull Class: Name:xxxxxxxxxxx/frontend Type:repository] map[Action:push Class: Name:xxxxxxxxxxx/frontend Type:repository]]
The task yaml is (annotations, comments and later steps removed for brevity)...
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
annotations:
name: source-to-image-cache
namespace: openshift-pipelines
resourceVersion: '259117173'
uid: xxxxxxxxxxxxxxxxxxxxx
spec:
params:
- description: The path to the dockerfile to build
name: pathToDockerfile
type: string
- default: .
description: >-
The build context used by Kaniko
(https://github.com/GoogleContainerTools/kaniko#kaniko-build-contexts)
name: pathToContext
type: string
- default: ''
description: Image tag
name: imageTag
type: string
- default: sbu-pipeline
description: Is the name of the base image registry namespace secret
name: base-reg-secret-name
type: string
- default: sbu-pipeline
description: Is the name of the built image registry namespace secret
name: build-reg-secret-name
type: string
- default: ''
description: >-
This is the URL to save the built image to - used with
imageRespositoryPath below
name: imageRegistryUrl
type: string
- default: ''
description: >-
This is the repository path at the imageRegistryUrl to save the built
image to
name: imageRepositoryPath
type: string
results:
- description: The image SHA code for the built application
name: image-digest
type: string
steps:
- image: 'uk.icr.io/sbu-pipeline/alpine-curl-jq:6'
imagePullPolicy: IfNotPresent
name: merge-json
resources: {}
script: >
#!/usr/bin/env bash
set +x
printf "\nSorting out config.json for Kaniko.\n"
diff /home/.dockerwip/base.dockerconfigjson
/home/.dockerwip/build.dockerconfigjson -q
myDiff=$?
if [ "$myDiff" -gt 0 ]; then
# Hopefully short-term fix until Kaniko supports multiple auths...
cp /home/.dockerwip/base.dockerconfigjson /home/.docker/config.json
else
cp /home/.dockerwip/base.dockerconfigjson /home/.docker/config.json
fi
printf "\nFinished sorting out config.json for Kaniko.\n"
securityContext:
runAsUser: 0
volumeMounts:
- mountPath: /home/.dockerwip/base.dockerconfigjson
name: base-registry-creds
subPath: base.dockerconfigjson
- mountPath: /home/.dockerwip/build.dockerconfigjson
name: build-registry-creds
subPath: build.dockerconfigjson
- mountPath: /home/.docker
name: docker-config
- resources: {}
name: build
command:
- /kaniko/executor
env:
- name: DOCKER_CONFIG
value: /kaniko/.docker/
securityContext:
runAsUser: 0
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /kaniko/.docker/
name: docker-config
image: 'gcr.io/kaniko-project/executor:v1.16.0'
args:
- >-
--dockerfile=$(workspaces.source.path)/$(inputs.params.pathToDockerfile)
- >-
--destination=$(params.imageRegistryUrl)/$(params.imageRepositoryPath):$(inputs.params.imageTag)
- '--context=$(workspaces.source.path)/$(inputs.params.pathToContext)'
- '--single-snapshot=true'
- '--image-name-with-digest-file=$(workspaces.source.path)/image-digest'
- '--cache=true'
- '--cache-copy-layers=true'
- '--use-new-run=true'
volumes:
- emptyDir: {}
name: docker-config
- name: base-registry-creds
secret:
items:
- key: .dockerconfigjson
path: base.dockerconfigjson
secretName: $(params.base-reg-secret-name)
- name: build-registry-creds
secret:
items:
- key: .dockerconfigjson
path: build.dockerconfigjson
secretName: $(params.build-reg-secret-name)
workspaces:
- name: source
I've tested the credentials locally and everything seems fine. I can mount the secret containing the docker credentials in another container and then successfully login, pull and push images. The error only occurs with Kaniko. It seems Kaniko is not reading the DOCKER_CONFIG environment variable. I've tried multiple different Kaniko versions without success.
Any suggestions on possible fixes or next steps in debugging?