BufferedWriter and map function: empty files

75 Views Asked by At

I am trying to write a huge output file the quickest possible on two different files. I have created my lists list1 and list2, and I am now trying to write those in a respective file.

I started to increase the buffer size by doing so:

rand_file = io.FileIO("test1.txt", 'w')
    
writer = io.BufferedWriter(rand_file,buffer_size=500000000)
        
for x in mylist1:

    writer.write(x.encode())

writer.flush()

rand_file = io.FileIO("test2.txt", 'w')
    
writer = io.BufferedWriter(rand_file,buffer_size=500000000)

for x in mylist2:

    writer.write(x.encode())

writer.flush()

but now I wanted to change those "for loops" with list comprehension but I cannot make it work.

if I do the following:

writer.write(x for x in mylist1)

I do have the following error "TypeError: a bytes-like object is required, not 'generator'". Note I also tried writer.write(x.encode() for x in mylist1), with the same result.

I then wanted to use the map function like so:

map(writer.write, mylist1)

This time, no error are raised but my output files are empty.

What am I doing wrong here ?

0

There are 0 best solutions below