i'm using weblogic and java ee. when i try to read a text file (about 100mb) and create an array out of it, java heap size exceeds and weblogic goes down.
i use scanner
to avoid reading file in memory and also try not to create any new variable inside the loop to prevent java heap error. but unfortunately i get that error and get weblogic exit on panic
message in the console and it breaks down.
public static CensusData[] load(String path) throws Exception {
List<CensusData> list = new ArrayList<>(43);
String s;
try (FileInputStream inputStream = new FileInputStream(path); Scanner sc = new Scanner(inputStream, "UTF-8")) {
while (sc.hasNextLine()) {
s=sc.nextLine();
list.add(new CensusData(Integer.parseInt(s.split(",")[0].trim()),
s.split(",")[1].trim(),
Integer.parseInt(s.split(",")[2].trim()),
Integer.parseInt(s.split(",")[3].trim()),
s.split(",")[4].trim(),
Integer.parseInt(s.split(",")[5].trim()),
s.split(",")[6].trim(),
s.split(",")[7].trim(),
s.split(",")[8].trim(),
s.split(",")[9].trim(),
s.split(",")[10].trim(),
s.split(",")[11].trim(),
s.split(",")[12].trim(),
s.split(",")[13].trim(),
s.split(",")[14].trim(),
s.split(",")[15].trim(),
Integer.parseInt(s.split(",")[16].trim()),
Integer.parseInt(s.split(",")[17].trim()),
Integer.parseInt(s.split(",")[18].trim()),
s.split(",")[19].trim(),
s.split(",")[20].trim(),
s.split(",")[21].trim(),
s.split(",")[22].trim(),
s.split(",")[24].trim(),
s.split(",")[25].trim(),
s.split(",")[26].trim(),
s.split(",")[27].trim(),
s.split(",")[28].trim(),
s.split(",")[29].trim(),
s.split(",")[30].trim(),
s.split(",")[31].trim(),
s.split(",")[32].trim(),
s.split(",")[33].trim(),
s.split(",")[34].trim(),
s.split(",")[35].trim(),
s.split(",")[36].trim(),
s.split(",")[37].trim(),
s.split(",")[38].trim(),
Integer.parseInt(s.split(",")[39].trim()),
s.split(",")[40].trim()));
}
// note that Scanner suppresses exceptions
if (sc.ioException() != null) {
throw sc.ioException();
}
} catch (Exception e) {
e.printStackTrace();
}
i know increasing Xmx size will help (which in my case will not), i'm looking for a real solution. why heap size is exceeded while i'm not storing any new instance inside the loop?
also consider the case. i need to get an array of CensusData
out of the file.
therefore i iterate each line of the text file and split it to get necessary data for creating that array.