which is faster and efficient between generator expression or itertools.chain for iterating over large list?

Question

I have large list of string and i want to iteratoe over this list. I want to figure out which is the best way to iterate over list. I have tried using the following ways:

• Generator Expression: g = (x for x in list)

• Itertools.chain: ch = itertools.chain(list)

Is there is another approach, better than these two, for list iteration?

Answer 1

The fastest way is just to iterate over the list. If you already have a list, layering more iterators/generators isn't going to speed anything up.

A good old for item in a_list: is going to be just as fast as any other option, and definitely more readable.

---

Iterators and generators are for when you don't already have a list sitting around in memory. itertools.count() for instance just generates a single number at a time; it's not working off of an existing list of numbers.

Another possible use is when you're chaining a number of operations - your intermediate steps can create iterators/generators rather than creating intermediate lists. For instance, if you're wanting to chain a lookup for each item in the list with a sum() call, you could use a generator expression for the output of the lookups, which sum() would then consume:

total_inches_of_snow = sum(inches_of_snow(date) for date in list_of_dates)

This allows you to avoid creating an intermediate list with all of the individual inches of snow and instead just generate them as sum() consumes them, thus saving memory.