I am looking for a more optimized way to move files to folders. Currently I have 600k files and would like to split them into separate folders of 40k sized chunks. Bellow is the method I am currently using however it looks like it will take a few days to complete. Any help you can provide would be greatly appreciated.
import os, glob, shutil
os.chdir('filepath')
list_of_file = os.listdir()
#split list into 40k sized chunks
chunks = [list_of_files[x: x + 40000] for x in range(0, len(list_of_files), 40000)]
#make new folders for files
for x in range(0,16):
os.mkdir('file path' + str(x))
#move files to folders
for x in range(0,16):
for i in chunks[x]:
if i in os.listdir():
shutil.copy(os.path.join(i), 'file path' + str(x))