DEVHIDE
Login Or Sign up

How to supply registry credential to a lifecycle

662 Views Asked by At

I am taking a look at the buildpacks.io standard and I wanted to make my own platform.

I am encountering issues with exporting my images to the docker hub. I have not written the exporter but I am using one from the buildpacks.io project as follows.

A platform is an executable that given a build image (heroku/buildpacks) will

  1. start a build container from that image and set some environment variables
  2. copy the sources to a predefined directory in the build container
  3. execute the /cnb/lifecycle/creator

A minimal implementation as a shelling out to buildah would then be like this.

#! /usr/bin/env bash    
    
# this script is a buildpacks.io implementation of a platform    
    
readonly SCRIPT_NAME="$(basename "$0")"    
    
run () {    
        local -r builder_image="$1"    
        local -r app_name="$2"    
        local -r new_container=$(buildah from "$builder_image")    
        # here we can set-up all the environment variables that are    
        # requested by the plaftorm API    
        buildah config -e CNB_PLATFORM_API="0.5" "$new_container"

        # this is useful for debugging what is actually happening
        # during the lifecycle    
        buildah config -e CNB_LOG_LEVEL="debug" "$new_container"

        # I have mounted my credentials directory inside this build container
        # this is where it should be found
        # buildah run "$new_container" cat /home/heroku/.docker/config.json
    
        # finally we can copy what we want built and packaged as an image    
        buildah copy "$new_container" '.' '/workspace'    
        buildah run "$new_container" /cnb/lifecycle/creator "$app_name"    
    
        # remove this build container when done
        #buildah rm "$new_container"    
}    
    
run "$@"

The good news is that I am getting somewhere and I can go through the phases described in the buildpacks.io lifecycle but I am getting an error when the lifecycle tries to upload the resulting image to a remote.

From an example directory containing just one file that will print "Hello world" when run I can do.

$ buildpacks docker://docker.io/heroku/buildpacks docker.io/edoput/buildpacks-example
Getting image source signatures
Copying blob 9c080d283a81 skipped: already exists  
...
Copying blob e80fe5117c2a [--------------------------------------] 0.0b / 0.0b
Copying config 451cf0ce8d done  
Writing manifest to image destination
Storing signatures
container ID must be specified
ERRO exit status 125                              
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
CNB_USER_ID=1000
CNB_GROUP_ID=1000
STACK=heroku-18
CNB_STACK_ID=heroku-18
CNB_PLATFORM_API=0.5
CNB_LOG_LEVEL=debug
HOSTNAME=1fca0b1ec107
HOME=/home/heroku
ef90cc8f4caed8241b832e8836c81a8530dada6358e3b2f43fb865ae269352ce
Warning: Not restoring or caching layer data, no cache flag specified.
===> DETECTING
======== Output: heroku/[email protected] ========
no
======== Results ========
err:  heroku/[email protected] (1)
pass: heroku/[email protected]
======== Results ========
fail: heroku/[email protected]
pass: heroku/[email protected]
======== Results ========
fail: heroku/[email protected]
pass: heroku/[email protected]
======== Results ========
fail: heroku/[email protected]
pass: heroku/[email protected]
======== Results ========
fail: heroku/[email protected]
pass: heroku/[email protected]
======== Results ========
pass: heroku/[email protected]
pass: heroku/[email protected]
skip: heroku/[email protected]
fail: heroku/[email protected]
Warning: Warning: buildpack heroku/jvm has a "version" key. This key is deprecated in build plan requirements in buildpack API 0.3. "metadata.version" should be used instead
======== Output: heroku/[email protected] ========
JVM
======== Output: heroku/[email protected] ========
Could not find a pom.xml file! Please check that it exists and is committed to Git.
======== Results ========
pass: heroku/[email protected]
fail: heroku/[email protected]
fail: heroku/[email protected]
======== Results ========
pass: heroku/[email protected]
fail: heroku/[email protected]
skip: heroku/[email protected]
pass: heroku/[email protected]
======== Results ========
pass: heroku/[email protected]
pass: heroku/[email protected]
skip: heroku/[email protected]
pass: heroku/[email protected]
Resolving plan... (try #1)
3 of 4 buildpacks participating
heroku/nodejs-engine 0.7.4
heroku/nodejs-npm    0.4.4
heroku/procfile      0.6.2
===> ANALYZING
Previous image with name "docker.io/edoput/buildpacks-example" not found
Usable cache not provided, using empty cache metadata.
===> RESTORING
Usable cache not provided, using empty cache metadata.
===> BUILDING
Starting build
Running build for buildpack heroku/[email protected]
Looking up buildpack
Finding plan
Running build for buildpack Node Buildpack 0.7.4
Updating buildpack plan entries
Creating plan directory
Preparing paths
Running build command
[INFO] Node.js Buildpack
[INFO] Setting NODE_ENV to production
[INFO] Installing toolbox
[INFO] - yj

[Installing Node]
[INFO] Getting Node version
[INFO] Resolving Node version
[INFO] Downloading and extracting Node v16.3.0

[Parsing package.json]
[INFO] Parsing package.json
Processing layers
Updating environment
Reading output files
Updating buildpack processes
Updating process list
Finished running build for buildpack heroku/[email protected]
Running build for buildpack heroku/[email protected]
Looking up buildpack
Finding plan
Running build for buildpack NPM Buildpack 0.4.4
Updating buildpack plan entries
Creating plan directory
Preparing paths
Running build command
[INFO] Using npm v7.15.1 from Node
[INFO] Installing node modules

up to date, audited 1 package in 778ms

found 0 vulnerabilities
npm notice 
npm notice New minor version of npm available! 7.15.1 -> 7.18.1
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v7.18.1>
npm notice Run `npm install -g [email protected]` to update!
npm notice 

[Warning: Skip pruning because NODE_ENV is not 'production'.]

Processing layers
Updating environment
Reading output files
Updating buildpack processes
Updating process list
Finished running build for buildpack heroku/[email protected]
Running build for buildpack heroku/[email protected]
Looking up buildpack
Finding plan
Running build for buildpack Procfile 0.6.2
Updating buildpack plan entries
Creating plan directory
Preparing paths
Running build command
[INFO] Discovering process types
[INFO] Procfile declares types -> (none)
Processing layers
Updating environment
Reading output files
Updating buildpack processes
Updating process list
Finished running build for buildpack heroku/[email protected]
Listing processes
Finished build
===> EXPORTING
no project metadata found at path '/layers/project-metadata.toml', project metadata will not be exported
Reusing tarball for layer "heroku/nodejs-engine:nodejs" with SHA: sha256:22b14cf125adb65ab87e7dc2dcbfddc9e80e3d1916536be195709424b91be64d
Adding layer 'heroku/nodejs-engine:nodejs'
Layer 'heroku/nodejs-engine:nodejs' SHA: sha256:22b14cf125adb65ab87e7dc2dcbfddc9e80e3d1916536be195709424b91be64d
Layer 'slice-1' SHA: sha256:62f98a22a3bd756b22102aacc338bbc61810fac32149d4a213f74fdb7e64759d
Adding 1/1 app layer(s)
Reusing tarball for layer "launcher" with SHA: sha256:20e1cf6014bd25720eb257f028b876dae49298820951982fa57cc2f64c086e66
Adding layer 'launcher'
Layer 'launcher' SHA: sha256:20e1cf6014bd25720eb257f028b876dae49298820951982fa57cc2f64c086e66
Reusing tarball for layer "config" with SHA: sha256:c8c42576a4717d5b075ea911146c10b8843b45a1f8b59c492eb866e418522a19
Adding layer 'config'
Layer 'config' SHA: sha256:c8c42576a4717d5b075ea911146c10b8843b45a1f8b59c492eb866e418522a19
Reusing tarball for layer "process-types" with SHA: sha256:83d85471d9f8a3834b4e27cf701e3f0aef220cc816d9c173c7d32cd73909a590
Adding layer 'process-types'
Layer 'process-types' SHA: sha256:83d85471d9f8a3834b4e27cf701e3f0aef220cc816d9c173c7d32cd73909a590
Adding label 'io.buildpacks.lifecycle.metadata'
Adding label 'io.buildpacks.build.metadata'
Adding label 'io.buildpacks.project.metadata'
Setting CNB_LAYERS_DIR=/layers
Setting CNB_APP_DIR=/workspace
Setting CNB_PLATFORM_API=0.5
Setting CNB_DEPRECATION_MODE=quiet
Prepending /cnb/process and /cnb/lifecycle to PATH
Setting default process type 'web'
Setting ENTRYPOINT: '/cnb/process/web'
Saving docker.io/edoput/buildpacks-example...
*** Images (sha256:30279532d29d797bc9fd61d705b6701a62ded6e857150f2cabf176b698351d20):
      docker.io/edoput/buildpacks-example - POST https://index.docker.io/v2/edoput/buildpacks-example/blobs/uploads/?from=heroku%2Fpack&mount=sha256%3Ad2e110be24e168b42c1a2ddbc4a476a217b73cccdba69cdcb212b812a88f5726: UNAUTHORIZED: authentication required; [map[Action:pull Class: Name:edoput/buildpacks-example Type:repository] map[Action:push Class: Name:edoput/buildpacks-example Type:repository] map[Action:pull Class: Name:heroku/pack Type:repository]]

*** Digest: sha256:30279532d29d797bc9fd61d705b6701a62ded6e857150f2cabf176b698351d20

*** Manifest Size: 2209
ERROR: failed to export: failed to write image to the following tags: [docker.io/edoput/buildpacks-example: POST https://index.docker.io/v2/edoput/buildpacks-example/blobs/uploads/?from=heroku%2Fpack&mount=sha256%3Ad2e110be24e168b42c1a2ddbc4a476a217b73cccdba69cdcb212b812a88f5726: UNAUTHORIZED: authentication required; [map[Action:pull Class: Name:edoput/buildpacks-example Type:repository] map[Action:push Class: Name:edoput/buildpacks-example Type:repository] map[Action:pull Class: Name:heroku/pack Type:repository]]]
error while running runtime: exit status 246
ERRO exit status 246                              
container ID must be specified
ERRO exit status 125

As the logs points out I am missing the authorization to pull/push to the remote that I have set up and is publicly accessible.

I am providing a token as the access credential and it is available in the build container as a file mounted at /home/heroku/.docker/config.json which is readable by the user running the creator process.

The configuration looks like this with the base64 encoded "$user:$password" value.

{
    "auths": {
        "docker.io": {
            "auth": "REDACTED"
        }
    }
}

The exporting command specification is described here and the authentication for registries.

How can I debug the exporting process without having to package my own image with a custom lifecycle implementation?

I have tested usage of my credentials as follows on another image repository and authentication is working as the docker registry expects.

$ skopeo copy docker://docker.io/heroku/buildpacks docker://docker.io/edoput/buildpacks --debug --authfile .docker/config.json 
DEBU[0000] Returning credentials from .docker/config.json 
DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration 
DEBU[0000]  No signature storage configuration found for docker.io/edoput/buildpacks:latest, using built-in default file:///home/edoput/.local/share/containers/sigstore 
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/docker.io 
DEBU[0000] Loading registries configuration "/home/edoput/.config/containers/registries.conf" 
DEBU[0000] Trying to access "docker.io/heroku/buildpacks:latest" 
DEBU[0000] Trying to access "docker.io/heroku/buildpacks:latest" 
DEBU[0000] Returning credentials from .docker/config.json 
DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration 
DEBU[0000]  No signature storage configuration found for docker.io/heroku/buildpacks:latest, using built-in default file:///home/edoput/.local/share/containers/sigstore 
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/docker.io 
DEBU[0000] GET https://registry-1.docker.io/v2/         
DEBU[0000] Ping https://registry-1.docker.io/v2/ status 401 
DEBU[0000] GET https://auth.docker.io/token?account=edoput&scope=repository%3Aheroku%2Fbuildpacks%3Apull&service=registry.docker.io 
DEBU[0001] GET https://registry-1.docker.io/v2/heroku/buildpacks/manifests/latest 
DEBU[0001] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.v2+json" 
DEBU[0001] Using blob info cache at /home/edoput/.local/share/containers/cache/blob-info-cache-v1.boltdb 
DEBU[0001] IsRunningImageAllowed for image docker:docker.io/heroku/buildpacks:latest 
DEBU[0001]  Using default policy section                
DEBU[0001]  Requirement 0: allowed                      
DEBU[0001] Overall: allowed                             
Getting image source signatures
DEBU[0001] Reading /home/edoput/.local/share/containers/sigstore/heroku/buildpacks@sha256=e30ff30cbabe53acd6e55fb43e831dd0274b318247d681215ec24bf341241ef7/signature-1 
DEBU[0001] Manifest has MIME type application/vnd.docker.distribution.manifest.v2+json, ordered candidate list [application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.v1+prettyjws, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v1+json] 
DEBU[0001] ... will first try using the original manifest unmodified 
DEBU[0001] Checking /v2/edoput/buildpacks/blobs/sha256:9c080d283a816b19233adfe7339c4666d4ad207cc1d88b6523d233c73dfb0240 
DEBU[0001] GET https://registry-1.docker.io/v2/         
DEBU[0001] Checking /v2/edoput/buildpacks/blobs/sha256:889a7173dcfeb409f9d88054a97ab2445f5a799a823f719a5573365ee3662b6f 
DEBU[0001] Checking /v2/edoput/buildpacks/blobs/sha256:d839a7ccb751206cf9d53204ad13da59c9a271f3bb725b632d38b938d97ea961 
DEBU[0001] Checking /v2/edoput/buildpacks/blobs/sha256:4bbfd2c87b7524455f144a03bf387c88b6d4200e5e0df9139a9d5e79110f89ca 
DEBU[0001] Checking /v2/edoput/buildpacks/blobs/sha256:d2e110be24e168b42c1a2ddbc4a476a217b73cccdba69cdcb212b812a88f5726 
DEBU[0001] Checking /v2/edoput/buildpacks/blobs/sha256:1fb371fdf5afa4bc49d73dc64613efa9276525cbb8f95cbfbe94815726089124 
DEBU[0002] Ping https://registry-1.docker.io/v2/ status 401 
DEBU[0002] GET https://auth.docker.io/token?account=edoput&scope=repository%3Aedoput%2Fbuildpacks%3Apull%2Cpush&service=registry.docker.io 
DEBU[0002] GET https://auth.docker.io/token?account=edoput&scope=repository%3Aedoput%2Fbuildpacks%3Apull%2Cpush&service=registry.docker.io 
DEBU[0002] GET https://auth.docker.io/token?account=edoput&scope=repository%3Aedoput%2Fbuildpacks%3Apull%2Cpush&service=registry.docker.io 
DEBU[0002] GET https://auth.docker.io/token?account=edoput&scope=repository%3Aedoput%2Fbuildpacks%3Apull%2Cpush&service=registry.docker.io 
DEBU[0002] GET https://auth.docker.io/token?account=edoput&scope=repository%3Aedoput%2Fbuildpacks%3Apull%2Cpush&service=registry.docker.io 
DEBU[0002] GET https://auth.docker.io/token?account=edoput&scope=repository%3Aedoput%2Fbuildpacks%3Apull%2Cpush&service=registry.docker.io 
DEBU[0002] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:1fb371fdf5afa4bc49d73dc64613efa9276525cbb8f95cbfbe94815726089124 
DEBU[0002] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:9c080d283a816b19233adfe7339c4666d4ad207cc1d88b6523d233c73dfb0240 
DEBU[0002] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:d839a7ccb751206cf9d53204ad13da59c9a271f3bb725b632d38b938d97ea961 
DEBU[0002] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:4bbfd2c87b7524455f144a03bf387c88b6d4200e5e0df9139a9d5e79110f89ca 
DEBU[0002] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:889a7173dcfeb409f9d88054a97ab2445f5a799a823f719a5573365ee3662b6f 
DEBU[0002] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:d2e110be24e168b42c1a2ddbc4a476a217b73cccdba69cdcb212b812a88f5726 
DEBU[0003] ... already exists                           
DEBU[0003] ... already exists                           
DEBU[0003] Skipping blob sha256:d2e110be24e168b42c1a2ddbc4a476a217b73cccdba69cdcb212b812a88f5726 (already present): 
Copying blob d2e110be24e1 skipped: already exists  
...
DEBU[0003] Checking /v2/edoput/buildpacks/blobs/sha256:5da6b0533d17399a58d377f0eaf0e6d6ecd24d0d3a222b483f45fb1cc612774b 
DEBU[0003] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:5da6b0533d17399a58d377f0eaf0e6d6ecd24d0d3a222b483f45fb1cc612774b 
Copying blob d2e110be24e1 skipped: already exists  
...  
DEBU[0003] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:0427fe92c4f9f6578c1ad74d625c1e411d2421e674e4b9c77a01f53d9dd9c834 
DEBU[0003] ... already exists                           
DEBU[0003] ... already exists                           
DEBU[0003] Skipping blob sha256:1fb371fdf5afa4bc49d73dc64613efa9276525cbb8f95cbfbe94815726089124 (already present): 
Copying blob d2e110be24e1 skipped: already exists  
... 
DEBU[0005] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:f18d74887f79510e979837ddd7330678832b0b7e99e9c8bb59de423a5d400f76 
Copying blob d2e110be24e1 skipped: already exists  
...
Copying blob 4f4fb700ef54 [--------------------------------------] 0.0b / 0.0b
DEBU[0006] Downloading /v2/heroku/buildpacks/blobs/sha256:1b0fecc52ded060857a931d07ec4e5b629cd6ce293d981846093cebacfe159d6 
DEBU[0006] GET https://registry-1.docker.io/v2/heroku/buildpacks/blobs/sha256:1b0fecc52ded060857a931d07ec4e5b629cd6ce293d981846093cebacfe159d6 
DEBU[0007] No compression detected                      
DEBU[0007] Using original blob without modification     
DEBU[0007] Checking /v2/edoput/buildpacks/blobs/sha256:1b0fecc52ded060857a931d07ec4e5b629cd6ce293d981846093cebacfe159d6 
DEBU[0007] HEAD https://registry-1.docker.io/v2/edoput/buildpacks/blobs/sha256:1b0fecc52ded060857a931d07ec4e5b629cd6ce293d981846093cebacfe159d6 
Copying config 1b0fecc52d [--------------------------------------] 0.0b / 15.7KiB
DEBU[0007] ... already exists                           
Writing manifest to image destination
DEBU[0007] PUT https://registry-1.docker.io/v2/edoput/buildpacks/manifests/latest 
Storing signatures
buildpack buildah
Original Q&A
1

There are 1 best solutions below

2
On

You might try updating docker.io to https://index.docker.io/v1/ in the config. Are you able to docker push to the remote using the same config.json?

Related Questions in BUILDPACK

Related Questions in BUILDAH

Trending Questions

Popular # Hahtags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs node.js arrays c asp.net json python-3.x ruby-on-rails .net sql-server swift django angular objective-c pandas excel

Popular Questions

Copyright © 2021 Jogjafile Inc.