open file depending on whether it's .gz or not

Question

I'm trying to figure out what the best way of opening a python file is based on its type.

For example, I've got something basic like this but it just doesn't seem 'pythonic' to me and I feel like in some way it can be refactored and written more cleaner;

def openfile(filename):
    if read_file_from_top:
        if not filename.endswith('.gz'):
            with open(filename, 'r') as infile:
                for line in infile:
                   # do something
        else:
            with gzip.open(filename, 'r') as infile:
                for line in infile:
                   # do something
    elif read_file_from_bottom:
        if not filename.endswith('.gz'):
            with open(filename, 'r') as infile:
                for line in infile:
                    # do something
        else:
            with gzip.open(filename, 'r') as infile:
                for line in infile:
                    # do something

Would there be a better way to do this, maybe using a generator? Thanks.

Answer 1

You should separate the opening and the reading:

def openfile(filename, mode='r'):
    if filename.endswith('.gz'):
        return gzip.open(filename, mode) 
    else:
        return open(filename, mode)

with openfile(filename, 'r') as infile:
    for line in infile:
       # do something

Answer 2

I think something like this is at least a little better:

import gzip


def file_line_gen(filename):
    if filename.endswith('.gz'):
        open_fn = gzip.open
    else:
        open_fn = open

    with open_fn(filename, 'r') as f:
        for line in f:
            yield line


for line in file_line_gen('data.gz'):
    # do something here
    print repr(line)

Answer 3

Short solution using a predefined list of crucial functions:

def processFile(filepath):
    with [open, gzip.open][0 if not filepath.endswith('.gz') else 1](filepath, 'r') as fh:
        if read_file_from_top:
            # do something
        elif read_file_from_bottom:
            # do something