django docker container cannot access redis container when using redis as session backend

Question

I am trying to deploy staging enviroment that mimics my django+postgresql(AWS RDS)+redis(AWS Elasticache).

I use the latest django-redis(4.10) to set up django with redis. My settings are as follows:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            # Mimicing memcache behavior.
            # http://niwinz.github.io/django-redis/latest/#_memcached_exceptions_behavior
            "IGNORE_EXCEPTIONS": True,
        },
    }
}

# django-redis: use redis as session cache
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"

When I run this locally in my pipenv enviroment with postgresql running remotely on AWS RDS & redis server running locally on brew services, everything works fine. But if I docker-compose up the containers, django's session handling seems to break.

My docker-compose as follows: (Please note that because I intend to use AWS RDS and Elasticache, I have not included postgres and redis containers in compose file)

version: '3.7'

services:
  django: &django
    restart: always
    build:
      context: .
      dockerfile: ./compose/staging/django/Dockerfile  # expose 8000
    image: staging_django
    container_name: staging_django
    hostname: staging_django
    networks:
      - backend
    env_file:
      - ./.envs/.staging/.django
    command: /start

  nginx:
    build:
      context: .
      dockerfile: ./compose/staging/nginx/Dockerfile
    image: staging_nginx
    container_name: staging_nginx
    hostname: nginx
    ports:
      - "80:80"
    networks:
      - backend
    restart: on-failure
    links:
      - django
    depends_on:
      - django

networks:
  backend:
    driver: 'bridge'

When I try to login to admin site, django container returns following errors.

staging_django |   File "/usr/local/lib/python3.7/site-packages/django/contrib/sessions/backends/cache.py", line 51, in create
staging_django |     "Unable to create a new session key. "
staging_django | RuntimeError: Unable to create a new session key. It is likely that the cache is unavailable.

I can confirm that django is not accessing the redis server because redis-cli's MONITOR does not show any django-related logs.

If I comment out the two lines in the above settings.py,

SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"

I can login in the the admin no problem, so it is definitely definitely related to using redis as session backends, and possibly due to redis being not accessible from django docker container.

Any help is REALLY appreciated! I have been working two full days to solve this, and now my boss is bullying me for this. :(

Answer 1

I'd put redis in docker-compose.yml as well.

And then in your django settings file, use the service name for redis, like so :

docker-compose.yml

version: '3'
services:
  web_app_server:
    container_name: web_app_server
    depends_on:
      - session_caching

  session_caching:
    container_name: session_caching
    image: redis:latest
    restart: always

settings.py

"LOCATION": "redis://session_caching:6379/1",