How does chroot affect dynamic libraries memory use?

Question

Although there is another question with similar topic, it does not cover the memory use by the shared libraries in chrooted jails.

Let's say we have a few similar chroots. To be more specific, exactly the same sets of binary files and shared libraries which are actually hard links to the master copies to conserve the disk space (to prevent the potential possibility of a files alteration the file system is mounted read only).

How is the memory use affected in such a setup?

Answer 1

As described in the chroot system call:

This call changes an ingredient in the pathname resolution process and does nothing else.

So, the shared library will be loaded in the same way as if it were outside the chroot jail (share read only pages, duplicate data, etc.)

http://man7.org/linux/man-pages/man2/chroot.2.html

Answer 2

When I set this up, I copied the shared libraries per chroot instead of linking to a read-only mount. With separate files, the text segments were not shared. It's likely that the same inode will map to the same read-only text segment, but this may vary with available memory management hardware and similar architectural details.

Try this experiment on your system: write a small program that makes some minimal use of a large shared library. Run twenty or thirty chroot jails as you describe, each with a running copy of the program. Check overall memory usage before & during running, and dissect one instance to get a good text/data segment breakdown. If memory use increases by the full size of the map for each instance, the segments are not shared. Conversely, if memory use goes up by a fraction of the map, the segments are shared.

Answer 3

Because hardlinks share the same underlying inode, the kernel treats them as the same item when it comes to caching/mapping.

You'll see filesystem cache savings by using hardlinks, as well as disk-space savings.

The biggest issue I'd have with this is that if someone manages so subvert the read-only nature of one of the chroot environments, then they could subvert all of them by making modifications to any of the hardlinked files.