I'm trying to download file using Java Spring WebClient. The endpoint is returning application/octet-stream which I process via DataBuffers and write it into the temp File like this.
public <T> Flux<DataBuffer> exchangeFileReceive(ObjectResponseResourceDefinition<T> resource, Map<String, Object> variables) {
final Map<String, Object> vars = variables == null ? new HashMap<>() : variables;
return client.method(resource.getMethod())
.uri(ub -> prepareUri(ub, resource, vars))
.header(HttpHeaders.ACCEPT, MediaType.APPLICATION_OCTET_STREAM_VALUE)
.retrieve()
.bodyToFlux(DataBuffer.class);
}
public Flux<DataBuffer> getContent(String documentId) {
return exchangeFileReceive(
(ObjectResponseResourceDefinition<Object>) GET_CONTENT,
new VariablesBuilder()
.var("documentId", documentId)
.get()
);
}
public File getContent(String documentId) throws IOException {
log.debug("getContent method called:\n documentId: {}", documentId);
Flux<DataBuffer> dataBufferFlux = integrationService.getContent(documentId);
File tmpFile = File.createTempFile("getContent", "");
DataBufferUtils.write(dataBufferFlux, tmpFile.toPath(), StandardOpenOption.CREATE).share().block();
log.debug("getContent method executed with file returned:\n file: {}", tmpFile.getName());
return tmpFile;
}
My problem is, that response from server contains "headers" in body which looks like this:
--XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Content-disposition: form-data; name=file; filename=test.xml
Content-Type: application/octet-stream
Content-Length: 209
<test-assign1-structure>
<test-assign1-data>
<test-assign1-name>Test</test-assign1-name>
<test-assign1-surname>File</test-assign1-surname>
</test-assign1-data>
</test-assign1-structure>
--XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX--
I need to delete those "headers" from the file. My problem is, that I'm not really sure how to do it, while not loading whole file into the memory (do it streaming way). I found some solutions but I'm not really sure, if this does not load whole file into memory and if it is correct. Something like this:
ResponseEntity<Flux<Part>> request = webClient
.get()
.uri("...")
.accept(MediaType.APPLICATION_OCTET_STREAM)
.retrieve()
.toEntityFlux((inputMessage, context) ->
defaultPartHttpMessageReader.read(ResolvableType.forType(byte[].class), inputMessage, Map.of())
)
.block();
}
List<Part> parts = request.getBody().collectList().block();
for (Part part : parts) {
DataBufferUtils.write(part.content(), tmpFile.toPath(), StandardOpenOption.CREATE).share().block();
}
I'm concerned that this approach will not work, because request.getBody().collectList() is returning Mono<List> and as far as I understand as soon as Mono is used, whole object is loaded into memory. Correct me if I'm wrong, I just started to work with WebClient and quiet don't understand everything. Or is it not loading whole file into memory, because part.content() is actually DataBuffer type?
Sorry if this question is duplicate of something, but I tried to google something, but didn't find anything that covers both removing "headers" and not loading whole response into memory at the same time.
Thank you
.......................................................