Time efficient way to skip no of line from very large text file (16gb) using python

Question

I have a very large text file of 16gb . I need to skip no of line .I want to skip those line in time efficient manner. I am using python for code.how to do that ?

Answer 1

Just read the number of lines you want to skip and throw them away:

with open(your_file) as f_in:
    for i in range(number_of_lines_to_skip):
        f_in.readline()
    # your file is now at the line you want...  

You can also use enumerate to have a generator that only yields lines once you have skipped the lines you want to:

with open(your_file) as f_in:
    for line in (line for i, line in enumerate(f_in) if i>lines_to_skip):
        # here only when you have skipped the first lines

The second there is likely faster.

Answer 2

beware, calling next on a file object will raise StopIteration if the end of file is reached.

go_to_line_number = some_line_number

with open(very_large_file) as fp:

    for _ in range(go_to_line_number):
        next(fp)

    for line in fp:
        # start your work from desired line number
        pass