Python 3 going through a file until EOF. File is not just a set of similar lines needing processing

Question

The answers to questions of the type "How do I do "while not eof(file)""

do not quite cover my issue

I have a file with a format like

header block

data

another header block

more data (with arbitrary number of data lines in each data block)

...

I do not know how many header-data sets there are

I have successfully read the first block, then a set of data using loops that look for the blank line at the end of the data block.

I can't just use the "for each line in openfile" type approach as I need to read the header-data blocks one at a time and then process them.

How can I detect the last header-data block.

My current approach is to use a try except construction and wait for the exception. Not terribly elegant.

Answer 1

This is some time later but I post this for others who do this search. The following script, suitably adjusted, will read a file and deliver lines until the EOF.

"""

Script to read a file until the EOF

"""
def get_all_lines(the_file):
    for line in the_file:
        if line.endswith('\n'):
            line = line[:-1]
        yield line


line_counter = 1
data_in = open('OAall.txt')
for line in get_all_lines(data_in):
    print(line)
    print(line_counter)
    line_counter += 1

data_in.close()

Answer 2

It's hard to answer without seeing any of your code...

But my guess is that you are reading the file with fp.read():

fp = open("a.txt")
while True:
    data = fp.read()

Instead:

1. try to pass always the length of data you spected
2. Check if the read chunck is a empty string, not None

For example:

fp = open("a.txt")
while True:
    header = fp.read(headerSize)
    if header is '':
        # End of file
        break
    read_dataSize_from_header
    data = fp.read(dataSize)
    if data is '':
        # Error reading file
        raise FileError('Error reading file')
    process_your_data(data)