Pandas ExcelWriter workaround for fsspec URLs?

241 Views Asked by At

Is there a workaround for using pandas ExcelWriter to append to a fsspec URL? I am working out of OneDrive and need to automatically append a master xlsx file with each new xlsx file that gets uploaded to the OneDrive folder (a new xlsx file get added to the OneDrive folder daily and need to create a master list without changing previous data) but the append function does not work with fsspec URLs and overwrites the master xlsx file.

This script runs on a trigger automatically and calls on any new .xlsx files that are in the OneDrive folder. The columns are exactly the same, but the rows vary and the file names are not consistent other than the .xlsx format so I do not think I can use the manipulate the start row or call a specific file name.

Is there a workaround for this? Essentially I want a master xlsx file that exists in OneDrive that will grow and update with each xlsx file export that gets uploaded to the OneDrive folder every day.

I tried...

with pd.ExcelWriter(
    "/Users/silby/OneDrive/test/dataTest.xlsx",
    mode='a',
    engine='openpyxl',
    if_sheet_exists='overlay'
) as writer:
    excel_merged.to_excel(writer)

and expected it to append the dataTest.xlsx file but it overwrites the existing data instead.

0

There are 0 best solutions below