C# FileSystemWatcher high cpu load when no change happen to directory

2.1k Views Asked by At

My case is the following:

  1. Files are creating inside a directory by some process (about 10 files per minute with max file size 5MB each). Lets call this folder MYROOT.
  2. That files need to be moved and categorized into sub directories according some specific logic based on the filename and external settings.
  3. There will be 5 sub directories under the MYROOT with some sub directories inside them also, lets name them C1, C2, C3, C4, C5. And some sub directories inside them as A1, A2, A3, An...
  4. So, we will have the following categorization: C:\MYROOT\C1\A1, C:\MYROOT\C1\A2, C:\MYROOT\C2\B1 and so on...
  5. All files are written to C:\MYROOT and have the same file type and same naming convention. No renames, deletes, changes are made into this folder.

The FileSystemWatcher is set as follows:

this._watcher = new FileSystemWatcher();
this._watcher.NotifyFilter = NotifyFilters.CreationTime;
this._watcher.Filter = string.Empty;
this._watcher.Path = @"C:\MYROOT";
this._watcher.IncludeSubdirectories = false;
this._watcher.Created += this._watcher_Created;

The event handler:

private void _watcher_Created(object sender, FileSystemEventArgs e)
{
    string filePath = string.Empty;

    if (e.ChangeType != WatcherChangeTypes.Created)
        return;

    lock (FileOrganizer.Manager.s_LockObject)
    {
        filePath = e.FullPath;

        //Further processing here like creating a custom object that 
        //contains the file name and some other info, add that object inside a queue
        //that is processed async with logic applied to wait the file to be written,
        //validate file contents, etc.. and finally move it where it should be moved.
        //The code here is very fast, exception free and in a fire-and-forget manner.
    }
}

When the MYROOT directory is empty and files are created by another process, they are moved inside the folders as I described. The folders are only created once if they do not exist. The amount of files that exist inside the sub folders are increasing and we are talking about ~200GB and counting.

Now hear this: When no files are created (the "creator" process is not running) nothing should trigger the watcher and I can see that from logs that I enable for debugging before posting this question. But, my process containing the watcher hits a constant 13 to 14% cpu on an octa core processor and increases as the size of the sub directories increase. During processing even if I create (copy-paste) 2000 files at once goes 1% more than that. The funny part is that when I change the monitoring path to an empty one or the same but with less volume inside it, the cpu utilization of my process is 0% and only when files are created goes to a max of 2% - 5%.

Again as said, the observation is clear, when the monitoring path sub directories contain data, that data affect the watcher internals even if you set not to monitor the sub directories. And if that data volume is big, then high cpu resources are needed by the file system watcher. That is the same even if no changes are taking place to trigger any watcher events.

Is this the normal or by design behavior of the FileSystemWatcher?

PS. When I monitor MYROOT folder and move the files outside that folder everything seems OK, but that is not an acceptable solution.

Thanx.

Marios

0

There are 0 best solutions below