Close all open files in ipython

Question

Sometimes when using ipython you might hit an exception in a function which has opened a file in write mode. This means that the next time you run the function you get a value error,

ValueError: The file 'filename' is already opened. Please close it before reopening in write mode.

However since the function bugged out, the file handle (which was created inside the function) is lost, so it can't be closed. The only way round it seems to be to close the ipython session, at which point you get the message:

Closing remaining open files: filename... done

Is there a way to instruct ipython to close the files without quitting the session?

Answer 1

You should try to always use the with statement when working with files. For example, use something like

with open("x.txt") as fh:
    ...do something with the file handle fh

This ensures that if something goes wrong during the execution of the with block, and an exception is raised, the file is guaranteed to be closed. See the with documentation for more information on this.

Edit: Following a discussion in the comments, it seems that the OP needs to have a number of files open at the same time and needs to use data from multiple files at once. Clearly having lots of nested with statements, one for each file opened, is not an option and goes against the ideal that "flat is better than nested".

One option would be to wrap the calculation in a try/finally block. For example

file_handles = []
try:
    for file in file_list:
        file_handles.append(open(file))

    # Do some calculations with open files

finally:
    for fh in file_handles:
        fh.close()

The finally block contains code which should be run after any try, except or else block, even if an exception occured. From the documentation:

If finally is present, it specifies a "cleanup" handler. The try clause is executed, including any except and else clauses. If an exception occurs in any of the clauses and is not handled, the exception is temporarily saved. The finally clause is executed. If there is a saved exception, it is re-raised at the end of the finally clause. If the finally clause raises another exception or executes a return or break statement, the saved exception is lost. The exception information is not available to the program during execution of the finally clause.

Answer 2

A few ideas:

• use always finally (or a with block) when working with files, so they are properly closed.
• you can blindly close the non standard file descriptors using os.close(n) where n is a number greater than 2 (this is unix specific, so you might want to peek /proc/ipython_pid/fd/ to see what descriptors the process have opened so far).
• you can inspect the captured stack frames locals to see if you can find the reference to the wayward file and close it... take a look to sys.last_traceback