Is it possible to copy a multi-OS image from one docker registry to another on a linux machine?

Question

I need a way to copy my manifest and all related blobs/etc from a private registry to a public registry, where the images were never pushed to the public registry before.

I am successfully creating a multi-arch manifest via buildah. Note that while the image is built with buildah, I would be satisfied with a docker-based approach for copying the final result between registries. Here is the output from buildah inspect:

{
    "schemaVersion": 2,
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "digest": "sha256:e1c1de1b56adc07e5a97970b3340b1cf71c02796eb4e27c638634b6bcf0e510e",
            "size": 5590,
            "platform": {
                "architecture": "amd64",
                "os": "windows"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "digest": "sha256:c4bf2b94bbedceab3888544f4b966e8c1435231daeff27546acaf3b817485226",
            "size": 511,
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        }
    ]
}

In reality, there will be even more permutations... but this simple manifest demonstrates the problem.

My CI pushes this image to a local registry, for performance (and various other) reasons. Once I have tested the final manifest, I want to push it to the public docker.io registry with the same tags.

Before I added Windows into the mix, I was able to accomplish this by pulling each image, retagging it with the new registry, and pushing. Something like...

buildah manifest create docker.io/img/name:latest

# Retag & add windows...
buildah pull myreg/img/name:windows
buildah tag myreg/img/name:windows docker.io/img/name:windows
buildah push docker.io/img/name:windows
buildah manifest add docker.io/img/name:latest docker.io/img/name:windows

# ... other variants ...

Unfortunately, this does not work with Windows. When the Linux machine tries to pull the Windows image, I receive the following error:

Error committing the finished image: error adding layer with blob "sha256:0363fe57a309a0e39c3dd1bb7be80beed88dcef74b1773ee1a37f6caf81e0fe2": Error processing tar file(exit status 1): link /Files/Program Files/common files/Microsoft Shared/Ink/en-US/micaut.dll.mui /Files/Program Files (x86)/common files/Microsoft Shared/ink/en-US/micaut.dll.mui: no such file or directory

Do I need to use a Windows machine for the retagging bit, where the image is copied from the local to public registry? Or is there a simple way to just copy/mirror an entire manifest without adding all these extra pull/tag/push steps?

Answer 1

For copying between registries, you can hit the registry API without interacting with a docker engine or any other container runtime. The images are json manifests and compressed tar blobs described in the OCI image-spec, and the API's access to those on a registry is now covered by the OCI distribution-spec.

The most complicated part of this is typically handling the auth that may vary by registry server, otherwise it could be fairly easily implemented in a shell script with a variety of curl and jq commands. I ended up making my own tooling to handle this exact use case, available on github.com/regclient/regclient. And for your specific request, you'd run:

regctl image copy myreg/img/name:latest docker.io/img/name:latest

Similar tools that I think implement these features include RedHat's skopeo and Google's crane (part of their go-containerregistry project).

Answer 2

There is an open-source tool skopeo.

Quote:

Copying an image from and to various storage mechanisms. For example you can copy images from one registry to another, without requiring privilege.

To just copy an image from one registry to another (from official examples):

skopeo copy --all docker://quay.io/skopeo/stable:latest docker://registry.example.com/skopeo:latest