How to use GitHub fine-grained access tokens in production across many repos when the max expiration is a year?

Question

I am trying to transition from the GitHub Personal Access Tokens classic version to the currently beta version fine-grained tokens. My issue is the token expiration max age is 1 year, whereas before the classic tokens could be set to not have any expiration. I know the simplest solution is just use the classic tokens, but if I've learned anything from GitHub's beta programs is that they will deprecate the previous option eventually. The fine-grained tokens are also appealing given they limit access to 1+ specific repositories.

I understand the reasons behind the expiration but am having an issue reconciling how to handle it. My issue is I have python packages hosted on GitHub using private repositories. I have a requirements.txt using the token to install the package(s) in the private repository(s) in multiple applications/scripts being hosted on the cloud or ran locally on multiple machines. If I only had one repository it would be a simple update, but there are many programs consuming these private packages. So, when the token(s) inevitably expires (max age is currently a year) I will then need to update the requirements.txt for all repositories that use the private repository package(s).

I realize I could store the token in a separate file and reference the token. However, this doesn't seem like a proper approach given the syntax appears so convoluted.

-e git+https://github.com/USERNAME/REPOSITORY.git@TAG#egg=PACKAGE_NAME&
subdirectory=PACKAGE_DIRECTORY&oauth_token=$(cat PATH/TO/FILE.TXT)

This solution also doesn't solve the case where I have an CD GitHub Action that deploys an application to AWS that needs to install the private repositories' packages. I would need to store the fine-grained token(s) as a Secret for the repository to be used in the GitHub Action. Then when the token inevitably updates I will have to go to every repository and update it.

I created the fine-grained access tokens and began using them but ran into the issues described above.

Answer 1

The appropriate approach here is not to store the token in the requirements.txt file at all. That file is visible to everyone who has your project, and a GitHub token is a secret. Secrets should not be checked into your repository since it's very easy for them to accidentally leak.

Instead, the thing to do is use a credential helper. On most OSes, there is a built-in credential helper that the user can use to save a suitable token and when Git is invoked, it can prompt for credentials if they're needed.

If you're working in a CI environment, such as GitHub Actions, generate a single token that works for all the repositories you need (which is possible with fine-grained tokens), store it in the appropriate secret store, expose it in the environment, and then use the technique outlined in the Git FAQ to read the token from the environment.

If you don't want to use a fine-grained token because of the expiration, you can create a GitHub App, which can issue tokens on your behalf. These tokens will be short-lived and have fine-grained permissions, but they can be issued on demand in a programmatic way, so you can have your infrastructure issue a token for the duration of your CI/CD job and then it will automatically be expired. For users, the need to regenerate a token from time to time should not be too burdensome.