Traefik intermittent 404 on docker-compose

Question

I'm running a bunch of services behind a traefik reverse proxy. I have tested those services to death and they work great. The problem is that when traefik is involved I get intermittent 404 errors whhenever I interact with them.

One of those services exposes a nice and simple REST api. Consuming code has to retry all requests. This is managable.

One of thse services exposes a frontend: If I want to use the frontend I have to constantly refresh the page. This is a truely aweful user experience.

Here are some samples from my compose file:

version: "2.1"

services:
  reverse-proxy:
    image: traefik:v2.2
    command:
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
    ports:
      - "8081:80"
      - "8082:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock


  api_service:
    image: its_just_a_flask_app

    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.configrouter1.rule=PathPrefix(`/config_backend/`) && (Method(`GET`) || Method(`POST`))"
      - "traefik.http.routers.configrouter1.middlewares=config-backend-auth@docker"
      - "traefik.http.middlewares.config-backend-auth.basicauth.usersfile=/config/usersfile"
      

  webserver:
    image: puckel/docker-airflow:with_a_few_lil_tweaks
    restart: always
    depends_on:
      - postgres # these exist and work fine
      - redis
    environment:
      - LOAD_EX=n
      - FERNET_KEY=stuff=
      - EXECUTOR=Celery
      - AIRFLOW__WEBSERVER__BASE_URL=http://webserver/airflow

    volumes:
      - ../orchistrator/dags/:/usr/local/airflow/dags
      - ./requirements.txt:/requirements.txt

    command: webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.aurflowrouter.rule=PathPrefix(`/airflow`)"
      - "traefik.http.routers.aurflowrouter.middlewares=airflow-basic-auth@docker"
      - "traefik.http.middlewares.airflow-basic-auth.basicauth.usersfile=/config/usersfile"
      - traefik.http.services.my-service.loadbalancer.server.port=8080

• As you can see both of these use basic auth. removing the auth has no effect
• removing the webserver's healthcheck has no effect
• when making api calls to the api_service, the first call often fails, the second call always succeeds
• when accessing the airflow frontend: the first page load fails, after that it succeeds.
• theres a button on the airflow fronend that triggers a POST. The POST returns a 302 Found, then the redirect always gives me a 404 at first
• sometimes the web frontend oads, but the static resources that it relies on do not load, resulting in an ugly and unusable site. So I find myself refreshing the page a lot

I'm at a loss here. Any help would be very apprectiated.

Traefik tags I've tried:

Sinve the only advice I've recieved or found so far is about using old tags or new tags, here's what I've found

image: traefik:v2.2
image: traefik:v2.2.1
image: traefik:v2.2.5
image: traefik:latest

Answer 1

Turned out it was a bug on v2.2.2. See here for more. Use, for example, v2.2.5 to get rid of this problem

Answer 2

I got similar issues after pulling traefik:latest yesterday. Just noticed the image was updated today and a new pull fixed my issues.