LinkedHashMap memory consumption

Question

The user uploads a huge file consisting of 1 million words. I parse the file and put the each line of the file into a LinkedHashMap<Integer, String>.

I need O(1) access and removal by key. Also, I need to preserve the access order, iterate from any position and sort.

The memory consumption is huge. I enabled Strings deduplication feature which appears in Java 8, but it turns out that the LinkedHashMap consumes most of the memory.

I found that LinkedHashMap.Entry consumes 40 bytes, but there are only 2 pointers - one for the next entry and one for the previous entry. I thought 1 pointer should be 64 bits or 32 bits. Buy if I divide 409,405,320(bytes) by 6,823,422(entries count) I have 60 bytes per entry.

I think I don't need the previous pointer, the next pointer should be enough to keep order. Why does LinkedHashMap consume so much memory? How can I reduce memory consumption?

Answer 1

How to reduce memory consumption?

1) Add -XX:+UseCompressedOops flag to your JVM startup.

2) Implement your own version of LinkedHashMap, optimized for your needs. I. e. use primitive int as a key instead of Integer, remove "previous" pointer if you don't need it, etc. Note that copying OpenJDK source might be impossible unless you wish to release your modified hash map implementation under GPLv2 license, because OpenJDK is GPLv2. However you can copy and modify LinkedHashMap implementation from Android Open Source Project, because it is Apache licensed.