NGINX : Static resources missing. Server response code 304

Question

I'm posting here because I'm struggling to get my head around what seems to be a simple issue. The setup is the following :

• Angular application uses static resources (such as fonts).
• The angular app is deployed using docker which includes an nginx server distributing the files.
• The whole container is hosted on Openshift and is accessed via the "classic" openshift networking pattern :
  1. Http call reaches route (openshift CRD for ingress, manages SSL).
  2. call gets redirected to service.
  3. service send the call to a single pod (managed via a DeploymentConfig, which is an openshift CRD similar to deployments).

The issue is the following : When loading the page, some static resources (fonts espacially) do not get loaded. There are no traces of any http call in the network tab of the browser tools, and the fonts used do not display anything (used for icons).

Some fonts are called in a css file as such (generated from angular sources) :

@font-face {
    font-family: FontAwesome;
    src: url(fontawesome-webfont.2b13baa7dd4f54c9.eot?v=4.7.0);
    src: url(fontawesome-webfont.2b13baa7dd4f54c9.eot?#iefix&v=4.7.0) ...;
}

---

Related symptoms :

We have noticed some 304 http response codes in the nginx server :

{"@timestamp": "2024-02-26T10:34:50+01:00","source": { "ip": "" },"http": {"request": { "method": "GET", "line": "GET /fontawesome-webfont.e9955780856cf8aa.woff2?v=4.7.0 HTTP/1.1", "bytes": 1268, "referrer": "https://<base_url>/styles.dc792cbafebe4f47.css" },"response": { "status_code": 304, "bytes": 214 }},"user_agent": { "original": "..." },"processing_time":"0.000"}

When some fonts are missing :

When loading wile deactivating cache :

Example of reponse headers sent by the nginx server :

Local cache seems to be empty and not contain any ressource :

---

Our setup :

The NGINX config we are using is the following

include /opt/rh/rh-nginx118/root/usr/share/nginx/modules/*.conf;

events {
    worker_connections  1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/opt/rh/rh-nginx118/log/nginx/access.log  main;

    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay     on;
    keepalive_timeout  65;
    types_hash_max_size 4096;
    
    include       /etc/opt/rh/rh-nginx118/nginx/mime.types;
    
    default_type  application/octet-stream;

    # The next block is loaded using the following include statement
    # for simplicity purposes, the files have been merged
    #include /opt/app-root/etc/nginx.d/*.conf;

    server_tokens off;

    gzip  on;
    gzip_types  text/css text/xml font/ttf font/woff font/woff2 font/otf image/gif image/jpeg image/png image/svg+xml image/tiff image/vnd.wap.wbmp image/webp image/x-icon image/x-jng image/x-ms-bmp application/javascript;
    
    server {
        listen       8080 default_server;
        listen       [::]:8080 default_server;
        server_name  _;
        root         /opt/app-root/src;

        # The next block is loaded using the following include statement
        # for simplicity purposes, the files have been merged
        #include      /opt/app-root/etc/nginx.default.d/*.conf;
        location ~* \.(ico|css|js|gif|jpe?g|png|svg|ttf|woff2?|otf)$ {
          expires 1d;
          add_header Cache-Control "public, must-revalidate";
        }
        
        location / {
          try_files $uri $uri/ /index.html?$args;
        }
    }

}

Our mime.types does also include the woff and woff2 mime types, as indicated by @VonC.

Disabling browser cache solves the issue, but is not a viable solution.

Any help would be more than appreciated. Thanks.

---

Edit 1 : The NGINX conf has been added to the post. Screenshots have been added to give more context.

Answer 1

Your setup involves an Angular application that uses static resources like fonts, served by NGINX, and the entire setup is containerized and deployed on OpenShift.

Internet
   |
[OpenShift Route]
   | (SSL/TLS)
[OpenShift Service]
   |
[Pod with NGINX + Angular App Container]

The 304 HTTP response code indicates that the resource has not been modified, and can be loaded from the browser's cache. However, if the resources are not loading, and you are seeing 304s without the actual content being displayed, it could be an issue with how caching is handled, or a misconfiguration in NGINX.

---

So first, check your NGINX configuration set up to serve static files. It looks OK, but you might also need to make sure the root or alias directive is correctly pointing to your static files' directory.

server {
    listen 80;
    server_name yourdomain.com;

    location / {
        root /path/to/your/angular/app/dist;
        try_files $uri $uri/ /index.html;
    }

    location ~* \.(ico|css|js|gif|jpe?g|png|svg|ttf|woff2?|otf)$ {
        add_header Cache-Control "public, max-age=259200";
        root /path/to/your/static/files;
    }
}

Make sure the root or alias directive in the location block points correctly to where your static files are stored within the Docker container.

---

Then, see if NGINX is configured to send the correct MIME types for your font files. NGINX needs to be aware of the correct MIME types for WOFF and WOFF2 font files:

http {
    include       mime.types;
    ...
    types {
        font/woff                 woff;
        font/woff2                woff2;
        ...
    }
    ...
}

---

Since disabling the browser cache "solves" the issue, it points towards a caching problem. The 304 status code is generally a good sign, indicating that your browser correctly understands when to fetch resources from the cache versus the server. However, if resources are not displaying correctly, it could be due to the browser not having a cached version of the files upon the first request or a misinterpretation of the cache directives.

To make sure your browser correctly caches and retrieves static resources, consider adding a must-revalidate directive to the Cache-Control header. That forces the browser to revalidate stale resources, making sure that it has the most up-to-date versions:

location ~* \.(ico|css|js|gif|jpe?g|png|svg|ttf|woff2?|otf)$ {
    add_header Cache-Control "public, max-age=259200, must-revalidate";
}

---

We added must-revalidate to the headers, and are currently testing it out.

I was wondering: why must this be explicitly added? Shouldn't this be the default behavior?
I don't see any use case where a server would like to send a cached resource that doesn't revalidate

The must-revalidate directive is not the default behavior, primarily because of the flexibility required to accommodate various use cases across the web. That directive makes sure once a cached resource becomes stale (i.e., it has passed its freshness lifetime), it must be revalidated with the origin server before it can be served again. That is important for resources where freshness is important for functionality or integrity, such as user-specific content or rapidly updating information.

It is not the default because:

• Performance: Automatically requiring revalidation for every stale resource could introduce latency in serving requests, as the browser would need to wait for the server to respond, even if the resource has not changed.
• Offline use: Applications designed to work offline rely on being able to serve cached resources without revalidating them against the server.
• Server load: Constant revalidation increases server load, as more requests must be processed. In scenarios where resources rarely change, this can be unnecessary overhead.

There are several use cases where a server might prefer to serve cached resources without revalidation:

• Unchanging resources: For static resources that do not change, like historical data or versioned files, revalidation might be unnecessary.
• Low-Risk content: For content where staleness is not a critical issue, the overhead of revalidation might not be justified.
• High-availability: In scenarios where high availability is critical, and network conditions are variable, serving from cache without revalidation can make sure content is available even if the server is temporarily unreachable.

---

Adding must-revalidate did not solve the issue. We also updated the config to add the expires field instead of using max-age, which sends an additional header. Still with no success.

From your edits, I see that the response headers for the font files indicate that the correct Cache-Control header is being sent (max-age=86400, public, must-revalidate). The presence of an ETag header suggests that NGINX is configured to support revalidation. That seems to be configured correctly.
The local cache being empty is puzzling, especially if the resources are not loading without a hard refresh. That indicates that the browser may not be storing the files even though NGINX is responding with a 200 status when the cache is disabled.
The NGINX configuration appears to be correct in terms of serving static files and including the necessary MIME types. The expires and Cache-Control headers are also set to allow for caching, while requiring revalidation after one day.

Browsers may interpret Cache-Control headers differently. The must-revalidate directive should cause the browser to revalidate the resource once it becomes stale, but this behavior can sometimes be inconsistent.
The absence of HTTP calls in the browser's network tab might indicate a race condition where the CSS is parsed before the font files are fully loaded or recognized by the browser: double-check that the paths to the font files in your CSS are correct and that the files are indeed present at those locations within your container.

If your Angular application uses service workers, they might be interfering with how resources are cached and served. Service workers can implement their own caching strategies that can override HTTP cache headers.
If service workers are used, inspect them to make sure they are not intercepting network requests and serving resources in an unexpected manner.

Use curl or another tool to directly request the static resources from your NGINX server and inspect the headers and content of the response.
As an experiment, try disabling ETags in NGINX to see if this affects the browser's caching behavior. That can be done by adding etag off; to the NGINX configuration.

And look for errors or warnings in the browser console that might provide additional information about why the resources are not charging. For instance, check there are no CORS issues that might be preventing fonts from loading, especially if they are being accessed from a different domain.

---

As noted by Sergey in the comments:

Judging from the screenshots there is no initiator for fonts. Hence they are not loaded. This looks not like a font cache issue, but like an issue with JS/CSS which does not load the font.
Try debugging your component if it's attached correctly. If the CSS is being attached to the DOM. If it does not – no wonder fonts are not loaded. No one requested for them.

True, confirm that the paths defined in the @font-face src URLs match the actual location of the font files on the server.
verify that the CSS containing the @font-face declarations is correctly loaded into the DOM, using the network tab in developer tools.

Make sure the CSS file is included in the build output and is not being omitted due to a build configuration issue: review the Angular build process to make sure it includes all necessary assets.

And make sure that the components that are supposed to use the fonts are being rendered as expected.

---

We have added the headers expires -1; and cache-control "no-store, no-cache, must-revalidate"; to css files ONLY. That seems to be fixing the issue for the moment.

I think the next step would be to find why my font does not get loaded from my css file, and how to fix this. We will probably be keeping this solution (of not caching css files) for the moment, as this is an acceptable temporary fix for us.

True: setting expires -1; and cache-control "no-store, no-cache, must-revalidate"; on CSS files makes sure these files are fetched fresh from the server on each request, bypassing the cache completely.
That indicates that the caching strategy for the CSS files might have been contributing to the issue, possibly due to stale CSS being served from cache which did not include the updated @font-face definitions.

To understand why the fonts are not loaded from the CSS files when they are cached:

• Make sure the @font-face definitions in the CSS files are correct. Verify that the URLs in the src attribute of @font-face point to the correct locations where the font files are stored on the server.

  /* Make sure the URLs in the CSS are correct */
  @font-face {
    font-family: 'FontAwesome';
    src: url('/path/to/fonts/fontawesome-webfont.eot?v=4.7.0');
    /* More src definitions */
  }
  

• With caching enabled for CSS files, inspect the delivered CSS content in the browser to make sure the @font-face declarations are present. It is possible that when cached, an older version of the CSS without the correct font declarations is served.

  // Use JavaScript in the browser console to check CSS content
  fetch('/path/to/your/styles.css').then(response => response.text()).then(text => console.log(text));
  

• If there is any server-side caching involved (like NGINX's fastcgi_cache, proxy_cache, or any similar feature), clear these caches (for instance, rm -rf /path/to/nginx/cache/*) to make sure they are not serving stale versions of CSS files.

• You might consider implementing a versioning for static files including CSS. By appending a version query string or changing the filename with each deployment, you can make sure browsers request the most recent version of the file.

  <!-- Append version as a query parameter in the HTML -->
  <link rel="stylesheet" href="/path/to/your/styles.css?v=1.0.0">
  

• Make sure CORS policies are not preventing fonts from being loaded due to cross-origin restrictions.

  # Example NGINX configuration for CORS
  location ~* \.(eot|otf|ttf|woff|woff2)$ {
      add_header 'Access-Control-Allow-Origin' '*';
  }
  

• Implement detailed error logging on the client-side to capture any issues during the loading of CSS and font files.

  // Client-side JavaScript to log errors when loading resources
  window.addEventListener('error', function(event) {
      // Log or report error details
      console.error('Resource failed to load:', event);
  }, true);