I'm encountering io.ktor.client.plugins.HttpRequestTimeoutException errors when using Ktor to download a file from the public API endpoint https://public.bybit.com/trading/10000000AIDOGEUSDT/10000000AIDOGEUSDT2024-03-05.csv.gz.
Interestingly, I can download the same file successfully using a web browser like Chrome.
Code Snippet:
suspend fun downloadFile(
url: String,
destination: File
): File {
val response = httpClient.get(url).bodyAsChannel()
val sink = destination.writeChannel()
response.copyTo(sink)
sink.close()
return destination
}
HttpClient(CIO) {
this.install(ContentNegotiation) {
json(
Json {
ignoreUnknownKeys = true
prettyPrint = true
isLenient = true
}
)
}
this.install(HttpTimeout) {
requestTimeoutMillis = 20000
}
}
Error:
Exception in thread "DefaultDispatcher-worker-7" io.ktor.client.plugins.HttpRequestTimeoutException: Request timeout has expired [url=https://public.bybit.com/trading/10000000AIDOGEUSDT/10000000AIDOGEUSDT2024-03-05.csv.gz, request_timeout=20000 ms]
at io.ktor.client.plugins.HttpTimeout$Plugin$install$1$1$killer$1.invokeSuspend(HttpTimeout.kt:165)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@75171877, Dispatchers.IO]
Problem:
- The download consistently fails with
HttpRequestTimeoutExceptiondespite having a 20-second timeout set. - Downloading the same file from the same URL works using a web browser.
Question:
- What could be causing the
HttpRequestTimeoutExceptionwhen usingKtorto download the file, even though a web browser can download it successfully? - Are there specific considerations or configurations I need to be aware of when downloading data using
Ktor? - Are there alternative approaches or troubleshooting steps I can try to overcome this timeout issue?