PermissionError: [Errno 13] Permission denied error while loading the file with pandas

Question

I'm getting this error :

Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\watchdog\observers\api.py", line 199, in run      
    self.dispatch_events(self.event_queue, self.timeout)
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\watchdog\observers\api.py", line 372, in dispatch_events
    handler.dispatch(event)
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\watchdog\events.py", line 403, in dispatch        
    super().dispatch(event)
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\watchdog\events.py", line 278, in dispatch        
    }[event.event_type](event)
  File "d:/Railway_Scenario/DataUpload.py", line 16, in on_created
    pd.read_csv(file)
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 610, in read_csv      
    return _read(filepath_or_buffer, kwds)
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 462, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 819, in __init__      
    self._engine = self._make_engine(self.engine)
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 1050, in _make_engine 
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 1867, in __init__     
    self._open_handles(src, kwds)
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 1368, in _open_handles
    storage_options=kwds.get("storage_options", None),
  File "C:\Users\40005139\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\common.py", line 647, in get_handle     
    newline="",
PermissionError: [Errno 13] Permission denied: 'D:\\Railway_Scenario\\ProfileData.csv'

When running this :

class Handler(watchdog.events.PatternMatchingEventHandler):
    def __init__(self):
        # Set the patterns for PatternMatchingEventHandler
        watchdog.events.PatternMatchingEventHandler.__init__(self, patterns=['*.xls'],
                                                             ignore_directories=True, case_sensitive=False)
      
        def on_created(self, event):
            print("Watchdog received created event - % s." % event.src_path)
            df=pd.read_excel(event.src_path)

can someone please tell how cani read the file on_created event or what is wrong in the snippet Specs : Python 3.7.6 x86 Windows 10 x64 Thanks

Answer 1

Issue here is that you are trying to open a file, which is not yet completely saved/created.

on_created event is triggered immediately when the file/directory creation starts and not when the file is created completely.

So you should wait for the file to be completed created and then do the reading part.

Make a simple check to compare file size repeatedly with a delay. Once the file is created fully, file size remains constant, and you can continue with reading the file.

I found this info from below link. It also mentions other ways. https://www.py4u.net/discuss/187495

def wait_till_file_is_created(self, source_path):
    historicalSize = -1
    while (historicalSize != os.path.getsize(source_path)):
        historicalSize = os.path.getsize(source_path)
        time.sleep(1) # Wait for one second

def on_created(self, event):
    print("Watchdog received created event - % s." % event.src_path)
    self.wait_till_file_is_created(event.src_path)                  
    df=pd.read_excel(event.src_path)