What is a reasonable amount of inotify watches with Linux?

Question

I am working on a daemon that monitors file events via inotify to trigger various types of events when files are accessed. I have read that watches are a little expensive, because the Kernel is storing the full path name of every file being watched.

How many watches would be too many?

Edit: Mostly, I'm wondering .. have you ever seen a noticeable performance hit, if so, at how many watches did it happen? Yes, I have to monitor / recursively (however its a minimal bootstrapped system).

Answer 1

/proc/sys/fs/inotify/max_user_watches is the current max number of watches per user.

Historically, the kernel has defaulted this to 8192, but given that many Linux distros customize their kernel builds quite a bit, this may not be true on every Linux system. A recent kernel change [1] dynamically selects a default max_user_watches value in the range [8192, 1048576] based on how much RAM the system has. (5.11 is the first kernel release containing this change.)

AFAICT, root can change max_user_watches to any value that's 2147483647 (2³¹-1) or under, as long as you're confident you have enough RAM to support that number of watches.

[1] https://github.com/torvalds/linux/commit/92890123749bafc317bbfacbe0a62ce08d78efb7

Answer 2

My info:

[foo@caffeine ~]# cat /var/log/lsyncd.status | grep Inotify
Inotify watching 293208 directories

[foo@caffeine ~]# cat /proc/sys/fs/inotify/max_user_watches
1048576

lsyncd uses about 130M of memory.

I use lsyncd to keep some directories in sync with the disaster recovery server.

No performance hit/penalty on the main server.

Answer 3

100 billions trillions gazillions would be too many, probably. Kernel Korner - Intro to inotify mentions “thousands of watches” so at least that number should not be a problem.

Answer 4

You can find the system limits by reading /proc/sys/fs/inotify/max_user_instances (maximum number of inotify "objects") and /proc/sys/fs/inotify/max_user_watches (maximum number of files watched), so if you exceed those numbers, it's too many ;-) The maximum number of watches is usually several tens of thousands or higher - on my system, 262143 - which is probably more than you'd ever need unless you're trying to watch every file in a file system, but you shouldn't be doing that. I would say, just try not to use more inotify watches than you need to, and don't worry about it unless you notice a significant decrease in performance.

Answer 5

AFAIK the kernel isn't storing the pathname, but the inode. Nevertheless, there are 540 bytes per Watch on a 32bit system. Double as much on 64bit.

I know from Lsyncd (maybe you want to check that out?) people who have a million watches. It just eats a Gigabyte of memory.

Answer 6

It depends on how much ram you've got

While 524288 is the maximum number of files that can be watched, if you're in an environment that is particularly memory constrained, you may wish to lower the number. Each file watch takes up 540 bytes (32-bit) or ~1kB (64-bit), so assuming that all 524288 watches are consumed that results in an upper bound of around 256MB (32-bit) or 512MB (64-bit).