Python watchdog: ignoring duplicate events

Question

I'm trying to set up watchdog so I can monitor changes to JavaScript files. But you always get duplicate events when a single file is modified.

I want to set it up so that when a file is modified it looks at the time of the event and if it is the same second as the previous event then it doesn't do anything. This way it can ignore the duplicate events. Is there a way to achieve this and always have the time of the previous event stored in a variable?

import time
from os import path
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler


class MyHandler(FileSystemEventHandler):
    def on_modified(self, event):
        print(time.ctime(), f'path : {event.src_path}')


if __name__ == "__main__":
    event_handler = MyHandler()
    observer = Observer()
    observer.schedule(event_handler, path=path.join(path.dirname(__file__), 'static/js'), recursive=False)
    observer.start()

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

Answer 1

One naive way to do this is to store the files/dates in a dictionary, and then check to see if the time/file source is in the dictionary before handling the event.

class MyHandler(FileSystemEventHandler):
    file_cache = {}

    def on_modified(self, event):
        seconds = int(time.time())
        key = (seconds, event.src_path)
        if key in self.file_cache:
            return
        self.file_cache[key] = True
        # Process the file

If you are worried about the number of events, you could try using cachetools to cache the results so the dictionary stays a bit smaller.

Answer 2

I am not sure anybody else. but I took on_closed event instead of modified as normally file writer will close the file after done with writing the file (at least in my case). Be noted that on_closed event seems not triggered on win32 system.