Jackson ObjectWriter only writes first entry from stream

Question

I want to create a Spring Boot controller which creates a CSV file using data from a stream. I use Jackson CSV (jackson-dataformat-csv 2.12.1) to write the data stream from the DB to a StreamingResponseBody.

To keep it simple, I replaced the actual data from the DB with a list containing 1, 2, 3. I want a CSV file which looks like this:

1
2
3

But it only contains the first entry (1). Can someone help me to identify the problem?

Please not that I don't want to create the file somewhere on the server, I want to stream the content directly to the user.

My code looks like this:

import com.fasterxml.jackson.dataformat.csv.CsvMapper
import org.springframework.http.MediaType
import org.springframework.http.ResponseEntity
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RequestMapping
import org.springframework.web.bind.annotation.RestController
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody
import javax.servlet.http.HttpServletResponse

@RestController
@RequestMapping(value = ["/test"], produces = [MediaType.TEXT_HTML_VALUE])
class TestController {

    @GetMapping("/download", produces = [MediaType.TEXT_EVENT_STREAM_VALUE])
    fun download(response: HttpServletResponse): ResponseEntity<StreamingResponseBody>? {
        response.setHeader("Content-Disposition", "attachment;filename=download.csv")
        response.status = HttpServletResponse.SC_OK

        val mapper = CsvMapper()
        val schema = mapper.schemaFor(Int::class.java)
        val writer = mapper.writer(schema)
        val input = listOf(1, 2, 3).stream()

        val stream = StreamingResponseBody { outputStream ->
            input.forEach { entity ->
                writer.writeValue(outputStream, entity)
            }
        }

        return ResponseEntity
            .ok()
            .header("Content-Disposition", "attachment;filename=download.csv")
            .body(stream)
    }
}

Answer 1

With the help of Andriy's comment I was able to find the cause and the solution. Jackson closes the stream when it's finished writing to it, see: ObjectMapper._writeValueAndClose.

To change this behavior you have to set JsonGenerator.Feature.AUTO_CLOSE_TARGET to false like this:

val jsonFactory = CsvFactory().configure(JsonGenerator.Feature.AUTO_CLOSE_TARGET, false)
val mapper = ObjectMapper(jsonFactory)
val writer = mapper.writer(CsvMapper().schemaFor(Int::class.java))

Note: There is no AUTO_CLOSE_TARGET option for the CsvGeneratorbut using the JsonGenerator setting also works for the CsvFactory.

Answer 2

Didn't see the column formatter assigned while writing to the stream in your code. Can you try the following :-

    CsvMapper mapper = new CsvMapper();
    CsvSchema schema = mapper.schemaFor(Dymmy.class);
    schema = schema.withColumnSeparator('\t');

Answer 3

I had a similar problem with POJOs (only the first record would write to the file).

Here's how my code looked in the POJO:

  @JsonProperty("userName")
  @Nullable
  private Double getUserName() {
    return user().getName();
  }

My issue was that user() was @Nullable, and I wasn't checking for null first in the getter method.

Adding a null check worked and all the records were written to the CSV:

  @JsonProperty("userName")
  @Nullable
  private Double getUserName() {
    if (user() == null) {
      return null;
    }
    return user().getName();
  }

Answer 4

There is a better alternative then setting AUTO_CLOSE_TARGET to false, which is to use SequenceWritter.

val stream = StreamingResponseBody { outputStream ->
        val mapper = CsvMapper()
        val sequenceWriter = mapper.writer(mapper.schemaFor(Int::class.java).withHeader())
            .writeValues(outputStream)
        input.forEach { entity ->
            sequenceWriter.write(entity)
        }
    }