Automatic deployment of Docker image as last step of CI/CD pipeline

Question

I am using a CI/CD server (self-hosted Woodpecker) to manage my app. The pipeline runs all these steps: clone, build, test, package as container, push to container registry (self-hosted Gitea).

But the final step is missing: "deploy". I don't want to ssh into the app server (a self-managed Debian VPS) and redeploy manually - I want to automatically pull the newest Docker image and redeploy the app. I'd like that to be the pipeline's final step.

I could create a pipeline step that uses ssh to run a script on the app server. But that feels kind of dirty - before using CI/CD I did everything via scripts, but I'd like to do things the modern way now.

I'm not a CI/CD expert, so I don't know how this is typically done?

(I am running a normal Docker environment, not Kubernetes.)

Answer 1

A common approach for your situation, considering you are not using an orchestration system like Kubernetes, is using webhook triggers that tell your server to update whenever a new Docker image is pushed to your container registry.

This process could work as follows:

1. Create a simple application that listens for webhooks on your server. This application should do the following:

   • Authenticate the webhook to ensure it's coming from your trusted source.
   • Pull the new Docker image.
   • Safely bring down your running application (if it's running).
   • Start your application with the new Docker image.

2. Configure your container registry to send a webhook to this application whenever a new image is pushed.

In terms of implementing this:

• If you use Gitea, it supports webhooks natively. In Gitea, you can set up a webhook to be fired whenever a new Docker image is pushed.
• Your listener application could be a simple HTTP server written in a language you're comfortable with. It should listen for POST requests (or whatever method the webhook uses) and then run the necessary commands to update your application.
• For authentication, consider using a secret token that is known to both Gitea and your listener application. Include this token in the webhook and have the listener application verify it.
• For bringing your application down and back up, Docker Compose could be a good tool. If you define your application in a docker-compose.yml file, updating could be as simple as running docker-compose down && docker-compose pull && docker-compose up -d.

This approach keeps your CI/CD pipeline clean and doesn't require SSH-ing into your server. Just remember to secure your listener application because it will have the power to bring your app down and back up. Authentication for the webhooks is a must.

Answer 2

Another few approaches that I tinkered with:

One could create a pipeline step that uses ssh to log into the app server and run a "deployment script". The result would be captured by the pipeline step so that it is part of the output log.

Or a script or cron job on the app server, that polls the docker registry (every minute say) for new images, then redeploys. This decouples the CI/CD and app servers. I think this is cleaner, but the problem is there is always a one minute delay before redeployment.

Or a separate webhook listening "app" on the app server, which listens for a specific webhook from the container registry and then runs a script. There are existing tools for that, so there's no need to write one - one only needs to write the deploy script itself. Seems the most popular tools are webhook and webhookd.

Or one could use watchtower which receives a webhook then upgrades specific containers' images.