InputStream Size unable to get in java when more than 1 gb. needs more buffer exception coming. how to resolve?

1.1k Views Asked by At

I am using inputstream to read data and byteArray to get size. I am getting outOfMemory issue need new buffer in bytearray as file size is 1.5 gb. i have checked in stackoverflow and googled lots of places. i couldn't find the solution to get the size. Kindly help me to get the size from input stream or any other best way . Thanks.

IOUtils package - import org.apache.commons.io.IOUtils;

private void writeIntoResponse(String uri, String id, String fileName, HttpServletResponse response)
       try (InputStream in = /*read(uri,id,fileName)reading from s3*/) { //1.5 gb file

           response.setContentType("application/octet-stream");
           byte[] bytes = IOUtils.toByteArray(in); //getting error out of memory issue           
           response.setHeader("Content-Length", String.valueOf((long)bytes.length)); // we needs to set bytes length as we need must
           OutputStream responseOutputStream = response.getOutputStream();           
           responseOutputStream.write(bytes);
           response.flushBuffer();
        } 
    catch (Exception e) {
        ////error
    }
   }

I am getting error as below.

Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.apache.commons.io.output.ByteArrayOutputStream.needNewBuffer(ByteArrayOutputStream.java:122) ~[commons-io-2.4.jar:2.4]
    at org.apache.commons.io.output.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153) ~[commons-io-2.4.jar:2.4]
1

There are 1 best solutions below

3
On

Don't load the 1.5GB file into memory. Get rid of this line:

byte[] bytes = IOUtils.toByteArray(in);

Find out the length of the external file in bytes and use that instead of bytes.length with:

len = Files.size(Paths.get(fileName))

or

len = new File(fileName).length();

Transfer the input to output by replacing:

responseOutputStream.write(bytes);

by

in.transferTo(out);