How to do Cache Sizing?

41 Views Asked by At

Using Ignite 2.15, i am trying to do some cache sizing. Do we have any documents to do sizing activity?

I found an old document https://apacheignite.readme.io/docs/capacity-planning

Is this still relevant, in the newer documentation i don't see the references to the above document.

Also even in this document, what is the an overhead of one cache minus the data. I was doing some basic tests where i used the example dataregion configuration file and simply added caches with no data and recorded some memory numbers off my mac.

Below are the number's i see,

Config Memory Real Private Compressed Threads
No Caches 365.4 MB 343.4 MB 88.0 MB 45.1 MB 99
5 Caches 456.5 MB 442.9 MB 388.3 MB 36.4 MB 102
10 Caches 625.7 MB 422.9 MB 366.1 MB 225.4 MB 108
15 Caches 687.2 MB 444.7 MB 388.9 MB 264.8 MB 108
20 Caches 687.2 MB 444.7 MB 388.9 MB 264.8 MB 108
40 Caches 971.5 MB 516.4 MB 457.7 MB 476.9 MB 108

So roughly i see ~ 15-20MB per cache. Is this understanding accurate? Any way to reduce this internal memory footprint?

Below are the dataregion and cache configs. Pretty straight forward off the examples configs,

<property name="dataStorageConfiguration">
        <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
            <!--
                Default memory region that grows endlessly. A cache is bound to this memory region
                unless it sets another one in its CacheConfiguration.
            -->
            <property name="defaultDataRegionConfiguration">
                <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                    <property name="name" value="Default_Non_Persistent_Region"/>
                    <!-- 15 MB initial memory region with disabled eviction -->
                    <property name="initialSize" value="#{15 * 1024 * 1024}"/>
                     <!-- 400 MB max memory region -->
                    <property name="maxSize" value="#{400 * 1024 * 1024}"/>
                </bean>
            </property>
        </bean>
    </property>

One cache config, other are more of this only the name would change,

<bean class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="name" value="cache1"/>
                <property name="atomicityMode" value="ATOMIC"/>
                <property name="backups" value="1"/>
            </bean>
1

There are 1 best solutions below

1
On

Not sure where the capacity planning page went to. You can see one on the GridGain side. You can use things like cache groups -- which allow tables to share some of their metadata -- to reduce the memory used.