I want to read big excel file and go though sheets and process rows. But I want to use limited memory so even if excel file is so big than my project memory capacity I want to be able to read and process it. (First of all, Is it even possible?)
After some research, I see that reading excel file streamly could be a solution for me so in my kotlin controller I get file in type of StreamingFileUpload according to the documentation for file uploads (6.21 File Uploads).
Then I am creating a temporary file and transfer the file I get as param to temporary file. Then I am sending this temp file to another method where I am trying to process the temp file. (There is no any error here)
@Post("/read-excel-file")
@Consumes(MediaType.MULTIPART_FORM_DATA)
fun importExcelFile(@Part file: StreamingFileUpload): Mono<MutableHttpResponse<String>> {
val tempFile: File = File.createTempFile(file.filename, "temp")
val uploadPublisher: Publisher<Boolean>? = file.transferTo(tempFile);
if (uploadPublisher != null) {
return Mono.from(uploadPublisher)
.map { success ->
if (success) {
ExcelReaderHandler().readExcelFile(tempFile); //sax parser logic is implemented here
HttpResponse.ok("DONE INSIDE")
} else {
HttpResponse.badRequest("Failed to upload file")
}
}
}
return Mono.just(HttpResponse.ok("DONE"))
}
After reading excel file, I want to process it and as far as I can see the best way for that is Sax Parser way. So I implemented Sax Parser logic according to the documentation of poi.apache official page in my readExcelFile method.
But the problem is with OPCPackage.open(file) method which is required to use in sax parser logic. The method just can not handle the temp file I give and throws memory error.
@Throws(Exception::class)
fun readExcelFile(file: File) {
try {
val factory = SAXParserFactory.newInstance()
val saxParser: SAXParser = factory.newSAXParser()
val opcPackage: OPCPackage = OPCPackage.open(file)
val here = "here" // can not reach here
I thought it might be about outdated version thing of poi library but I am using 5.2.3 version of poi (latest is 5.2.5)