Clearing OS cache from mem-mapped files without file handle

899 Views Asked by At

I need to force OS to purge the pages used for a mapped file. I don't have the file descriptor, so posix_fadvise cannot be used.

Our application caches a lot of files by mapping them into memory. After the file has been mapped (i.e. we've got the pointer from mmap()), we close the file. When at some later point we have to clean the cache, we want to purge the pages in OS cache as well. That is, we want to unmap the file, and do something like posix_fadvise(POSIX_FADV_DONTNEED), but the file descriptor is not available at this point.

The flow looks like this:

//caching stage
fd = open("file");
data = mmap(fd, <mmap flags>);
close(fd);

//clean-up stage
munmap(data);
// posix_fadvise(???, POSIX_FADV_DONTNEED);

Is there a way to clear the cached pages without file descriptor?

I have thought about following two workarounds:

  • Keeping the files open, so that I have valid descriptors at the time of cleanup. However, there may be tens of thousands files, and keeping them all open may affect OS performance.
  • Keep the file path, and reopen it just to get a descriptor and call posix_fadvise(). But the question is: will the old mapped area be associated with the same file? And will fadvise() purge the cached pages in this scenario?
1

There are 1 best solutions below

0
On

The second option worked. When the file is reopened later, the mapped area is associated with it, and calling posix_fadvise with new file descriptor unloads the mapped pages:

//caching stage
fd = open("file");
data = mmap(fd, <mmap flags>);
close(fd);

//clean-up stage
fd = open("file");
munmap(data);
posix_fadvise(fd, POSIX_FADV_DONTNEED);
close(fd);