How can I find out if a javascript iterator terminates early?

Question

Lets say I have a generator:

function* source() {
  yield "hello"; yield "world";
}

I create the iterable, iterate with a for-loop, and then break out of the loop before the iterator fully completes (returns done).

function run() {
  for (let item of source()) {
    console.log(item);
    break;
  }
}

Question: How can I find out, from the iterable-side, that the iterator terminated early?

There doesn't seem to be any feedback if you try to do this directly in the generator itself:

function* source2() {
  try {
    let result = yield "hello";
    console.log("foo");
  } catch (err) {
    console.log("bar");
  }
}

... neither "foo" nor "bar" is logged.

Answer 1

There's a much simpler way to do this: use a finally block.

function *source() {
  let i;

  try {
    for(i = 0; i < 5; i++)
      yield i;
  }
  finally {
    if(i !== 5)
      console.log('  terminated early');
  }
}

console.log('First:')

for(const val of source()) {
  console.log(`  ${val}`);
}

console.log('Second:')

for(const val of source()) {
  console.log(`  ${val}`);

  if(val > 2)
    break;
}

...yields:

First:
  0
  1
  2
  3
  4
Second:
  0
  1
  2
  3
  terminated early

Answer 2

Edit: See Newer accepted answer. I will keep this as it does/did work, and I was pretty happy at the time I was able to hack a solution. However, as you can see in the accepted answer, the finally solution is so simple now it has been identified.

I noticed that typescript defines Iterator (lib.es2015) as:

interface Iterator<T> {
  next(value?: any): IteratorResult<T>;
  return?(value?: any): IteratorResult<T>;
  throw?(e?: any): IteratorResult<T>;
} 

I intercepted these methods and logged calls and it does appear that if an iterator is terminated early --at least via a for-loop-- then the return method is called. It will also be called if the consumer throws an error. If the loop is allowed to fully iterate the iterator return is not called.

Return hack

So, I made a bit of a hack to allow capturing another iterable - so I don't have to re-implement the iterator.

function terminated(iterable, cb) {
  return {
    [Symbol.iterator]() {
      const it = iterable[Symbol.iterator]();
      it.return = function (value) {
        cb(value);
        return { done: true, value: undefined };
      }
      return it;
    }
  }
}

function* source() {
  yield "hello"; yield "world";
}

function source2(){
  return terminated(source(), () => { console.log("foo") });
}


for (let item of source2()) {
  console.log(item);
  break;
}

and it works!

hello
foo

remove the break and you get:

hello
world

Check after each yield

While typing this answer, I realised a better problem/solution is to find out in the original generator method.

The only way I can see to pass information back to the original iterable is to use next(value). So if we pick some unique value (say Symbol.for("terminated")) to signal the termination, and we alter the above return-hack to call it.next(Symbol.for("terminated")):

function* source() {
  let terminated = yield "hello";
  if (terminated == Symbol.for("terminated")) {
    console.log("FooBar!");
    return;
  }
  yield "world";
}
    
function terminator(iterable) {
  return {
    [Symbol.iterator]() {
      const it = iterable[Symbol.iterator]();
      const $return = it.return;
      it.return = function (value) {
        it.next(Symbol.for("terminated"));
        return $return.call(it)
      }
      return it;
    }
  }
}

for (let item of terminator(source())) {
  console.log(item);
  break;
}

Success!

hello
FooBar!

Chaining Cascades Return

If you chain some extra transform iterators, then the return call cascades through them all:

function* chain(source) {
  for (let item of source) { yield item; }
}

for (let item of chain(chain(terminator(source())))) {
  console.log(item);
  break
}

hello
FooBar!

Package

I've wrapped the above solution as a package. It supports both [Symbol.iterator] and [Symbol.asyncIterator]. The async iterator case is of particular interest to me, especially when some resource needs to be disposed of correctly.

Answer 3

I've run into a similar need to figure out when an iterator terminates early. The accepted answer is really clever and probably the best way to solve the problem generically, but I think this solution could also be helpful for other uses cases.

Say, for example, you have an infinite iterable, such as the fibonacci sequence described in MDN's Iterators and Generators docs.

In any sort of loop there needs to be a condition set to break out of the loop early, like in the solution already given. But what if you want to destructure the iterable in order to create an array of values? In that case, you'd want to limit the number of iterations, essentially setting a maximum length on the iterable.

To do this, I wrote a function called limitIterable, which takes as arguments an iterable, an iteration limit, and an optional callback function to be executed in the event that the iterator terminates early. The return value is a Generator object (which is both an iterator and an iterable) created using an Immediately Invoked (Generator) Function Expression.

When the generator is executed, whether in a for..of loop, with destructuring, or by calling the next() method, it will check to see if iterator.next().done === true or iterationCount < iterationLimit. In the case of an infinite iterable like the fibonacci sequence, the latter will always cause the while loop to be exited. However, note that one could also set an iterationLimit that is greater than the length of some finite iterable, and everything will still work.

In either case, once the while loop is exited the most recent result will be checked to see if the iterator is done. If so, the original iterable's return value will be used. If not, the optional callback function is executed and used as a return value.

Note that this code also allows the user to pass in values to next(), which will in turn be passed to the original iterable (see the example using MDN's fibonacci sequence inside the code snippet attached). It also allows for additional calls to next() beyond the set iterationLimit within the callback function.

Run the code snippet to see the results of a few possible use cases! Here's the limitIterable function code by itself:

function limitIterable(iterable, iterationLimit, callback = (itCount, result, it) => undefined) {
   // callback will be executed if iterator terminates early
   if (!(Symbol.iterator in Object(iterable))) {
      throw new Error('First argument must be iterable');
   }
   if (iterationLimit < 1 || !Number.isInteger(iterationLimit)) {
      throw new Error('Second argument must be an integer greater than or equal to 1');
   }
   if (!(callback instanceof Function)) {
      throw new Error('Third argument must be a function');
   }
   return (function* () {
      const iterator = iterable[Symbol.iterator]();
      // value passed to the first invocation of next() is always ignored, so no need to pass argument to next() outside of while loop
      let result = iterator.next();
      let iterationCount = 0;
      while (!result.done && iterationCount < iterationLimit) {
         const nextArg = yield result.value;
         result = iterator.next(nextArg);
         iterationCount++;
      }
      if (result.done) {
         // iterator has been fully consumed, so result.value will be the iterator's return value (the value present alongside done: true)
         return result.value;
      } else {
         // iteration was terminated before completion (note that iterator will still accept calls to next() inside the callback function)
         return callback(iterationCount, result, iterator);
      }
   })();
}

function limitIterable(iterable, iterationLimit, callback = (itCount, result, it) => undefined) {
   // callback will be executed if iterator terminates early
   if (!(Symbol.iterator in Object(iterable))) {
      throw new Error('First argument must be iterable');
   }
   if (iterationLimit < 1 || !Number.isInteger(iterationLimit)) {
      throw new Error('Second argument must be an integer greater than or equal to 1');
   }
   if (!(callback instanceof Function)) {
      throw new Error('Third argument must be a function');
   }
   return (function* () {
      const iterator = iterable[Symbol.iterator]();
      // value passed to the first invocation of next() is always ignored, so no need to pass argument to next() outside of while loop
      let result = iterator.next();
      let iterationCount = 0;
      while (!result.done && iterationCount < iterationLimit) {
         const nextArg = yield result.value;
         result = iterator.next(nextArg);
         iterationCount++;
      }
      if (result.done) {
         // iterator has been fully consumed, so result.value will be the iterator's return value (the value present alongside done: true)
         return result.value;
      } else {
         // iteration was terminated before completion (note that iterator will still accept calls to next() inside the callback function)
         return callback(iterationCount, result, iterator);
      }
   })();
}

// EXAMPLE USAGE //
// fibonacci function from:
//https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators#Advanced_generators
function* fibonacci() {
   let fn1 = 0;
   let fn2 = 1;
   while (true) {
      let current = fn1;
      fn1 = fn2;
      fn2 = current + fn1;
      let reset = yield current;
      if (reset) {
         fn1 = 0;
         fn2 = 1;
      }
   }
}

console.log('String iterable with 26 characters terminated early after 10 iterations, destructured into an array. Callback reached.');
const itString = limitIterable('abcdefghijklmnopqrstuvwxyz', 10, () => console.log('callback: string terminated early'));
console.log([...itString]);
console.log('Array iterable with length 3 terminates before limit of 4 is reached. Callback not reached.');
const itArray = limitIterable([1,2,3], 4, () => console.log('callback: array terminated early?'));
for (const val of itArray) {
   console.log(val);
}

const fib = fibonacci();
const fibLimited = limitIterable(fibonacci(), 9, (itCount) => console.warn(`Iteration terminated early at fibLimited. ${itCount} iterations completed.`));
console.log('Fibonacci sequences are equivalent up to 9 iterations, as shown in MDN docs linked above.');
console.log('Limited fibonacci: 11 calls to next() but limited to 9 iterations; reset on 8th call')
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next(true).value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log(fibLimited.next().value);
console.log('Original (infinite) fibonacci: 11 calls to next(); reset on 8th call')
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next(true).value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);