Writing and reading a new file by passing the files as an argument to a function in a python script

Question

I have a python script that gets input file names from the command prompt. I created a list to store all the input files and pass that to a function to create a new file with all the input files merged at once. Now, I pass this newly written file as an input to another function. I am getting an error message

TypeError: coercing to Unicode: need string or buffer, list found

Code:

file_list = []                                                                                                                                                                                                                                                                                                                        
for arg in range(1,len(sys.argv)-2):    
    file_list.append(sys.argv[arg])
    process_name = sys.argv[len(sys.argv)-1]
integrate_files(file_list,process_name)

def integrate_files(file_list,process_name):
    with open('result.log', 'w' ) as result:        
       for file_ in file_list:
          for line in open( file_, 'r' ):
             result.write( line )
    start_process(result,process_name)

def start_process(result,process_name):
    with open(result,'r') as mainFile:
       content = mainFile.readlines()

I am getting this error highlighted at the lines having the word with.open(). I tried to print the abspath of the result.log file. It printed closed file 'result.log', mode 'w' at 0x000000000227578. Where am I going wrong ? How should I create a new file and pass it to a function?

Answer 1

Your problem is that result is a closed file object:

start_process(result,process_name)

I think you want

start_process('result.log', process_name)

You could clean the script up a bit with

import shutil
                                                                                            file_list = sys.argv[1:-1]
process_name = sys.argv[-1]
integrate_files(file_list,process_name)

def integrate_files(file_list,process_name):
    with open('result.log', 'w' ) as result:        
       for file_ in file_list:
           with open(file_) as infile:
               shutil.copyfileobj(infile, result)
    start_process('result.log',process_name)

def start_process(result,process_name):
    with open(result,'r') as mainFile:
       content = mainFile.readlines()

Answer 2

The issue is here:

with open('result.log', 'w' ) as result:        
   # ...
start_process(result,process_name)

Since you reopen your file in start_process, you should just pass the name:

start_process(result.name, process_name)

Or just be explicit:

start_process('result.log', process_name)

Answer 3

When you write with open('result.log', 'w') as result:, you make result be an object representing the actual file on disk. That is different from the name of the file.

You certainly can pass that result to another function. But since it will be the actual file object, and not a file name, you can't pass that to open - open expects a name of a file, and looks for the file with that name, in order to create a new file object.

You can call methods on that file object, but none of them will actually re-open the file. Instead, the simplest thing is to remember and pass the file name, so that start_process can open it again.

As shown in @matsjoyce's answer, the file object remembers the original file name. So you could pass the object, and have start_process get the name. But that's messy. Really, just pass the name. (You could, like mats showed, pass result.name explicitly instead of making your own name variable first). Passing file objects around is usually not what you want - do it only when you want to split the reading/writing work across functions (and have a good reason for that).

Answer 4

In this:

with open('result.log', 'w' ) as result:

When you define result above, you are only defining it for that single loop, so it won't pass when you call start_process

So either change start_process to:

with open('result.log','r') as mainFile:

Or you could pass the string result.log into start_process instead of the variable result:

file_list = []                                                                                                                                                                                                                                                                                                                        
for arg in range(1,len(sys.argv)-2):    
    file_list.append(sys.argv[arg])
    process_name = sys.argv[len(sys.argv)-1]
integrate_files(file_list,process_name)

def integrate_files(file_list,process_name):
    with open('result.log', 'w' ) as result:        
       for file_ in file_list:
          for line in open( file_, 'r' ):
             result.write( line )
    start_process('result.log',process_name)

def start_process(result,process_name):
    with open(result,'r') as mainFile:
       content = mainFile.readlines()