Smooks : return an OutputStream

Question

I am currently writing a JAVA application that will input an EDI file and return an OutputStream using Smooks library for this purpose. I am struggling to return the output stream and use it without killing memory. The goal of an output stream is to allow users to convert it into an InputStream and manipulate the stream into object creation, files, push to db, etc, ... I would really appreciate if somebody with any considerable input could give me an insight what am I doing wrong. Thanks in advance ..

public class EdiToXml {

  private static final int headerBufferSize = 100;
  private static final byte[] buf = new byte[headerBufferSize];
  private static Smooks smooks;
  private static final String headerVersion1 = "IFLIRR\u001F15\u001F2\u001F1A";
  private static StreamSource stream;


 protected static ByteArrayOutputStream TransformBifToJava(FileInputStream inputStream) throws IOException, SAXException, SmooksException {

    Locale defaultLocale = Locale.getDefault();
    Locale.setDefault(new Locale("en", "EN"));

    //Creating a bufferedInputStream
    BufferedInputStream bufferedInputStream = new BufferedInputStream(inputStream);

    //Marking the bufferedInputStream
    bufferedInputStream.mark(0);

    //Obtaining the first 100 bytes to detect the file version
    bufferedInputStream.read(buf);

    //Reading first 100 bytes
    String value = new String(buf);

    if(value.indexOf(headerVersion1) > 0) {
      // Instantiate Smooks with the config for 15.2.1A
      smooks = new Smooks("smooks-config.xml");
    }

    bufferedInputStream.reset();

    stream = new StreamSource(bufferedInputStream);

    try {
      return Parse1(defaultLocale, smooks, stream);
    }finally {
      bufferedInputStream.close();
      inputStream.close();
    }

 }

 protected static ByteArrayOutputStream Parse1(Locale locale, Smooks smooks, StreamSource streamSource) throws IOException, SAXException, SmooksException {
    try {
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
    
        // Create an exec context - no profiles....
        ExecutionContext executionContext = smooks.createExecutionContext();
    
        // Filter the input message
        smooks.filterSource(executionContext, streamSource, new StreamResult(byteArrayOutputStream));
    
        Locale.setDefault(locale);

        System.out.println(byteArrayOutputStream.size());
    
        return byteArrayOutputStream;
    } finally {
        smooks.close();
    }
 }

 public static void main(String[] args) throws IOException, SAXException, SmooksException {

    ByteArrayOutputStream byteArrayOutputStream = EdiToXml.TransformBifToJava(new FileInputStream("xxxx/BifInputFile.DATA"));

    InputStream is = new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
    byteArrayOutputStream.close();

    int b = is.read();

    while (b != -1) {
        System.out.printf("%c", b);
        b = is.read();
    }

    is.close();

    System.out.println("======================================\n\n");
    System.out.print("Finished");
    System.out.println("======================================\n\n");

 }

}

Exception in thread "main" org.milyn.SmooksException: Smooks Filtering operation failed.
    at org.milyn.Smooks._filter(Smooks.java:548)
    at org.milyn.Smooks.filterSource(Smooks.java:482)
    at com.maureva.xfunctional.EdiToXml.Parse1(EdiToXml.java:102)
    at com.maureva.xfunctional.EdiToXml.TransformBifToJava(EdiToXml.java:86)
    at com.maureva.xfunctional.EdiToXml.main(EdiToXml.java:173)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3236)
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
    at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
    at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
    at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
    at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
    at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
    at org.milyn.delivery.sax.SAXHandler.flushCurrentWriter(SAXHandler.java:503)
    at org.milyn.delivery.sax.SAXHandler.endElement(SAXHandler.java:234)
    at org.milyn.delivery.SmooksContentHandler.endElement(SmooksContentHandler.java:96)
    at org.milyn.edisax.EDIParser.endElement(EDIParser.java:897)
    at org.milyn.edisax.EDIParser.endElement(EDIParser.java:883)
    at org.milyn.edisax.EDIParser.mapComponent(EDIParser.java:693)
    at org.milyn.edisax.EDIParser.mapField(EDIParser.java:636)
    at org.milyn.edisax.EDIParser.mapFields(EDIParser.java:603)
    at org.milyn.edisax.EDIParser.mapSegment(EDIParser.java:564)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:535)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:453)
    at org.milyn.edisax.EDIParser.mapSegment(EDIParser.java:566)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:535)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:453)
    at org.milyn.edisax.EDIParser.mapSegment(EDIParser.java:566)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:535)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:453)
    at org.milyn.edisax.EDIParser.mapSegment(EDIParser.java:566)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:535)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:453)
    at org.milyn.edisax.EDIParser.mapSegment(EDIParser.java:566)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:535)
    at org.milyn.edisax.EDIParser.mapSegments(EDIParser.java:453)

Process finished with exit code 1

Answer 1

I suggest you upgrade to the latest version of Smooks (v2.0.0-RC1) given that the EDI cartridge has been totally overhauled. The app is running out of memory because you're writing to a java.io.ByteArrayOutputStream which keeps the written bytes in-memory. I haven't understood what you're trying to accomplish. The things you mentioned like, creating objects, writing to files, and saving to a database, can be done from within Smooks.

If you only want to use Smooks for converting the EDI into XML then you should write the result to an output stream that doesn't keep the data in-memory like a FileOutputStream or implement your own OutputStream should you want to do something funky with the result. Having said this, it doesn't make too much sense to me to use Smooks only for transforming the input into XML.