Using Ktor HTTP Client (and kotlinx.io?) for resumable downloads directly to a file

Question

I haven't worked with (big) HTTP downloads before, so I really don't know too much about the topic.

So, what I want to do is to download a file from the internet directly into the local filesystem programmatically. It would also be good if the download was resumable. For Java, I found this Baeldung article that basically explains how to do exactly this.

However, I don't work with Java, but with Kotlin. More specifically, I use the Ktor HTTP Client (with the Apache engine, but I could switch that if necessary). I would like to know whether there is a way to use this client in the same way as in the mentioned article. Basically, I'd invoke a GET request, but I'd tell the client to stream the bytes to a specified File location. When the download is interrupted, I'd query the file size for the current status, and resume it at this byte position (PartialContent).

I've found out that

1. The Ktor server supports this: https://ktor.io/docs/partial-content.html
2. The kotlinx-io library might be helpful (because of the analogy to the article): https://github.com/Kotlin/kotlinx-io

However, I don't know how to put everything together and "connect the loose wires" to make it all work. It also seems that my question is too specific for a simple google search, that's why I'm resorting to SO now. If anyone with some experience with Ktor and HTTP Downloads could shed some light on this for me, it would be very appreciated!

PS in the further progress, an answer to this SO question might also be interesting.

Version data:

• Kotlin/JVM 1.4.20
• Ktor 1.4.1
• OpenJDK 11

Answer 1

The basic idea is to get the total content length by making a HEAD request and then iteratively download content with a help of the Range header. Here is an example implementation:

suspend fun main() {
    val client = HttpClient()
    client.download(
        "https://weq714976.live-website.com/wp-content/uploads/2019/03/WeQ-Influencers-Logo-1568x812.png",
        File("result.png")
    )
}

suspend fun HttpClient.download(url: String, outFile: File, chunkSize: Int = 1024) {
    val length = head<HttpResponse>(url).headers[HttpHeaders.ContentLength]?.toLong() as Long
    val lastByte = length - 1

    var start = outFile.length()
    val output = FileOutputStream(outFile, true)

    while (true) {
        val end = min(start + chunkSize - 1, lastByte)

        get<HttpResponse>(url) {
            header("Range", "bytes=${start}-${end}")
        }.content.copyTo(output)

        if (end >= lastByte) break

        start += chunkSize
    }
}