Exceptions thrown while generating a sequence

Question

I want to copy a potentially huge file from one place to another (for example, but not limited to, the local filesystem). In order to decouple the reading from the writing, my copy flow has steps to perform each task:

• The reading step returns a sequence of lines (the file content). This step yields every line of the file to avoid having all the file content in memory.
• The writing step writes the sequence of lines received from the previous step.

Well, sometimes exceptions may occur when reading a file and I want, in that case, to catch that exception and return an empty sequence of lines. I've been reading on how to proceed but I only find coroutine-related references. And the sequence generator is not coroutine.

EDIT: As suggested in comments, I avoid the use of Either. Despite this, a IOException could be thrown during file reading.

Finally, I end with this:

import arrow.syntax.function.compose
import arrow.syntax.function.pipe

import java.io.IOException
import java.io.InputStream

fun asSequenceOfLines(stream: InputStream): Sequence<String> =
  sequence {
      stream.bufferedReader().use {
        // readLine() can throw a IOException during file reading
        var line = it.readLine()
        while (line != null) {
          yield(line)
          line = it.readLine()
        }
      }
  }

// A step for reading a file given its path
fun readFile(): FlowStep = { data ->
  val path = data["input"] as String
  val inputStream = File(path).inputStream()
  try {
    val lines = asSequenceOfLines(inputStream)
    data + mapOf("lines" to lines)
  } catch (e: Exception) {
    // [!] NOT catched exception
    println("[WARN] Error when reading file content $path")
    data + mapOf("lines" to emptySequence<String>())
  }
}

When calling this function and an Exception has been thrown, I am not able to catch it as it is thrown when the sequence is consumed (generally, in the writing step). How can I capture the Exception in the asSequenceOfLines() caller function (the reading step)?

You have the complete code here: https://pastebin.com/PCarVGP8. I'm using Kotlin 1.3.50 and arrow-kt 0.10.0.

Thanks for reading :)

Answer 1

And the sequence generator is not coroutine

Actually, it is... But it doesn't matter in this case.

Well, sometimes exceptions may occur when reading a file and I want, in that case, to catch that exception and return an empty sequence of lines

Do you realize how sequences in Kotlin work and what they are? Sequence is an abstraction to make collecion processing lazy. asSequenceOfLines is not a reading step. It just creates this abstraction. Zero lines are read when asSequenceOfLines function is called. So, IOException can't be thrown on that call.

Actual reading happens when sequence is consumed. But Kotlin sequences can't predict future, and say in the very begining that there will be an IOException on 100500-th line. So, by the time IOException happens, some of lines are already consumed, and sequence is already not empty!

If you can't afford to read all file in advance and hold it in memory, you need to handle this exception on the writing step and revert all partly done writing job.

Also, there is a stdlib method for reading (and processing) text files line by line.