Streaming data over REST

Question

Hello is there any way to read streamed data over REST? I have an application with one microservice. I would like to send request from micreservice to main app and get these data by batch size. I followed https://jvmaware.com/streaming-json-response/

Spring boot framework

Java 1.8

Main app getting data from database:

public Stream<Transaction> getTrx(String id, int batchSize, TrxFilter filter) {
        jdbcTemplate.setFetchSize(batchSize);
        String sql = ...

        log.info("Fetching {} transactions for id: {}", batchSize, id);

        return jdbcTemplate.queryForStream(sql, (ResultSet resultSet, int rowNum) -> {
            return new Transaction(resultSet.getLong("id"));
        });
    }

Main app API:

@PostMapping(value = "/getTransactions/{id}/{batchSize}")
    public ResponseEntity<StreamingResponseBody> getTrx(@PathVariable("id") String id, @PathVariable("batchSize") int batchSize, @RequestBody TrxFilter filter) {
        Stream<Transaction> trxs = service.getTrx(id, batchSize, filter);
        StreamingResponseBody responseBody = outputStream -> {
            try (Writer writer = new BufferedWriter((new OutputStreamWriter(outputStream)))) {
                trxs.forEach(trx -> {
                    try {
                        log.info("WRITING: {}", trx.getId().getSequenceId());
                        writer.write(objectMapper.writeValueAsString(trx));
                        writer.flush();
                    } catch (IOException e) {
                        log.error("Exception occurred while writing object to stream.", e);
                    }
                });
            } catch (Exception exception) {
                log.error("Exception occurred while publishing data.", exception);
            }
            outputStream.close();
        };
        return ResponseEntity.status(HttpStatus.OK).contentType(MediaType.APPLICATION_JSON).body(responseBody);
    }

I tried this, but get nothing, what am I doing wrong?

try {
            URL url = new URL("http://.../repeng/getTransactions/" + id + "/" + batchSize);
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();
            connection.setRequestMethod("POST");
            connection.setRequestProperty("Content-Type", "application/json");
            connection.setRequestProperty("Accept", "application/json");
            connection.setDoOutput(true);
            connection.getOutputStream().write(objectMapper.writeValueAsBytes(filter));

            BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));

            String data;
            while ((data = reader.readLine()) != null) {
                Transaction transaction = objectMapper.readValue(data, Transaction.class);
                log.info("Received transaction {}", transaction);
                transactions.add(transaction);
            }

            reader.close();
        } catch (IOException e) {
            e.printStackTrace();
        }

Is there any way to test this? Can Postman work with streaming REST?

thx <3

Answer 1

Some Points:

• The API endpoint should follow the standards. Something like this: /transaction/{id}/{batchSize}
• getTrx method has a parameter filter, which maps request body to TrxFilter object. So there must be an input JSON that must be sent in the request body.
• Use Postman to test the REST API endpoints. If you have to change the request, you can easily modify it in Postman and test to see the expected response. Testing the APIs by writing Java code is cumbersome unless if there is any requirement for it.
• Please check if it is really required to call flush() method.
• Can we try something like this?

StreamingResponseBody stream = outputStream -> {
     trxs.forEach(trx -> {
        outputStream.write(objectMapper.writeValueAsString(trx).getBytes());
     }
});