Why does `go list -mod=readonly -m all` require access to private repos?

Question

I'm trying to print a list of direct dependencies in a Go project that has a vendor directory with:

go list -mod=readonly -m -f '{{ if not .Indirect }}{{ .Path }}{{ end }}' all

As I understand, I have to use -mod=readonly because go list refuses to report based on the partial information in vendor. This works on my laptop, because I have access to the private repo github.com/monzo/argo-rollouts, but in CI it fails with:

go list -m: github.com/argoproj/[email protected] (replaced by github.com/monzo/[email protected]): version "v0.0.0-20220309162305-84c86ea52e8f" invalid: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /home/circleci/go/pkg/mod/cache/vcs/052bf06be1713582ba9f1144e9982b362ff355fec71675863c108f9cf5a00bb4: exit status 128:
    ERROR: Repository not found.
    fatal: Could not read from remote repository.
    
    Please make sure you have the correct access rights
    and the repository exists.

Why does go list -mod=readonly need access to the private repo? I haven't provided -u.

Also, why can't it get this information locally when in vendor mode?

Edit: I've found a workaround that passes eligible modules to go list -m in vendor-mode, but it involves parsing the unstable vendor/modules.txt file intended for machines only.

% cat modules-that-provide-packages.awk
BEGIN {
        mod = ""
}

# Remember the module
/^# / {
        mod = $2
}

# Print the module if it provides a package
!/^# / {
        print mod
}


% awk -f modules-that-provide-packages.awk vendor/modules.txt /
    | sort -u \
    | xargs go list -m -f '{{ if not .Indirect }}{{ .Path }}{{ end }}'

Answer 1

• make sure you set GOPRIVATE to your private GitHub account link eg github.com/abc* in your CI script
• Add git config to use the GitHub account token to log in to the private repo. On your ci, you would have to do something like this in the CI script

 git config --global url."https://${{ inputs.token }}:[email protected]/<useraccount>".insteadOf "https://github.com/<useraccount>"

• Now when you run the command it will be able to access the private repo or before running the list command make sure to do go mod vendor to get all the dependencies on CI first

Answer 2

I think the answer is: go list -m doesn't require access to private repos, but populating an empty GOMODCACHE does.

go list -m gets the module info from either:

• GOMODCACHE, but this must be already populated. It is empty in CI, so Go will try to download from the private repos. Hence the error.
• The vendor directory, but you must provide a specific module. go list -m all gives up in vendor-mode.