java heap error while reading a file line by line inside a while loop

104 Views Asked by At

i'm using weblogic and java ee. when i try to read a text file (about 100mb) and create an array out of it, java heap size exceeds and weblogic goes down. i use scanner to avoid reading file in memory and also try not to create any new variable inside the loop to prevent java heap error. but unfortunately i get that error and get weblogic exit on panic message in the console and it breaks down.

public static CensusData[] load(String path) throws Exception {
        List<CensusData> list = new ArrayList<>(43);
        String s;
        try (FileInputStream inputStream = new FileInputStream(path); Scanner sc = new Scanner(inputStream, "UTF-8")) {
            while (sc.hasNextLine()) {
                 s=sc.nextLine();
                list.add(new CensusData(Integer.parseInt(s.split(",")[0].trim()),
                        s.split(",")[1].trim(),
                        Integer.parseInt(s.split(",")[2].trim()),
                        Integer.parseInt(s.split(",")[3].trim()),
                        s.split(",")[4].trim(),
                        Integer.parseInt(s.split(",")[5].trim()),
                        s.split(",")[6].trim(),
                        s.split(",")[7].trim(),
                        s.split(",")[8].trim(),
                        s.split(",")[9].trim(),
                        s.split(",")[10].trim(),
                        s.split(",")[11].trim(),
                        s.split(",")[12].trim(),
                        s.split(",")[13].trim(),
                        s.split(",")[14].trim(),
                        s.split(",")[15].trim(),
                        Integer.parseInt(s.split(",")[16].trim()),
                        Integer.parseInt(s.split(",")[17].trim()),
                        Integer.parseInt(s.split(",")[18].trim()),
                        s.split(",")[19].trim(),
                        s.split(",")[20].trim(),
                        s.split(",")[21].trim(),
                        s.split(",")[22].trim(),
                        s.split(",")[24].trim(),
                        s.split(",")[25].trim(),
                        s.split(",")[26].trim(),
                        s.split(",")[27].trim(),
                        s.split(",")[28].trim(),
                        s.split(",")[29].trim(),
                        s.split(",")[30].trim(),
                        s.split(",")[31].trim(),
                        s.split(",")[32].trim(),
                        s.split(",")[33].trim(),
                        s.split(",")[34].trim(),
                        s.split(",")[35].trim(),
                        s.split(",")[36].trim(),
                        s.split(",")[37].trim(),
                        s.split(",")[38].trim(),
                        Integer.parseInt(s.split(",")[39].trim()),
                        s.split(",")[40].trim()));
            }
            // note that Scanner suppresses exceptions
            if (sc.ioException() != null) {
                throw sc.ioException();
            }
        } catch (Exception e) {
           e.printStackTrace();
        }

i know increasing Xmx size will help (which in my case will not), i'm looking for a real solution. why heap size is exceeded while i'm not storing any new instance inside the loop?

also consider the case. i need to get an array of CensusData out of the file. therefore i iterate each line of the text file and split it to get necessary data for creating that array.

0

There are 0 best solutions below