Can multiple clients of Infinispan replicated cache share the same persistent file store?

Question

Cross posted at https://developer.jboss.org/thread/279735

Suppose we have multiple docker containers (each of which has java webapps, so multiple JVMs essentially), each of which are using the same replicated Infinispan cache either in:

• Embedded mode using jgroups for discovery
• Server mode using Docker Hub as Infinispan server, and the clients connecting via hotrod.

In either case, all the cache members/clients have a file-store from which they preload at startup (sample is for server mode, its similar for embedded mode's xml):

Via docker compose, the path inside each container's file store (lets say /var/tmp/server/OUR_CACHE.dat) is bind-mounted to the same file on the docker host.

<paths>
       <path name="cachestore.root" path="/var/tmp"/>
</paths>
...
    <local-cache name="OUR_CACHE">

         <expiration lifespan="-1"/>

         <locking isolation="SERIALIZABLE" acquire-timeout="30000" concurrency-level="1000" striping="false"/>

         <file-store relative-to="cachestore.root" path="server" max-entries="-1" purge="false" passivation="false" preload="true" fetch-state="true"/>

         <memory>

              <binary size="100000000" eviction="MEMORY"/>

         </memory>

    </local-cache>

My question is that - is the persistence mechanism designed so that multiple replicated cache clients can read/write to/from the same file store without any errors?

My understanding is that in replicated mode, the key values will become eventually consistent across all node's memory. But my goal is to ensure that multiple client containers using the same file-store for persistence and preloading should not adversely affect this replication.

If its not advised to share the same file-store .dat file, then what is the best practice to have a GUID in the filepath for each container's path in the .xml file. Each docker container's hostname (which is containerId) is unique, but it won't be known before its deployed, so it won't be easy to populate the infinispan_server.xml file with the value for "path" statically.

Thanks,

_Prateek

Answer 1

Isn't the point of replicated mode that each node has its own independent copy of the data? I don't fully understand what you're trying to achieve.

To the last point of your question:

If its not advised to share the same file-store .dat file, then what is the best practice to have a GUID in the filepath for each container's path in the .xml file. Each docker container's hostname (which is containerId) is unique, but it won't be known before its deployed, so it won't be easy to populate the infinispan_server.xml file with the value for "path" statically.

Can you put a placeholder in the config xml (e.g.: ${myprop}) and specify it at startup (e.g.: -Dmyprop=hostname01)?