I think there's a downside with very short-lived containers.
My understanding is that when an application runs inside a container, it sees a virtualised file-system (Docker does copy-on-write). Thus, mutations to the file-system are volatile.
This means that >1 containers on a big VM will not share the HTTP cache. More tangibly, on Windows since that's my domain, any writes to the WinInet cache will be isolated.
If I were to configure the OS to place the cache in a shared location, a data volume, would I now encounter locking issues where >1 HTTP OS subsystems are all trying to write to the same cache?
Probably not...
https://blogs.msdn.microsoft.com/wndp/2006/08/04/a-bit-about-wininets-index-dat/
How else could we design for 1,000s of transient containers to benefit from the same cache?
- Move the Temporary Internet Files to a data volume.
- Reverse proxy with caching.
- Output caching in the web server.
The nice thing about OS caching is that the request never hits the wire.