Ehcache thread deadlocks

Question

We have a production app running ehcache for years - currently ehcache 2.10.1. Lately we have noticed deadlocks on all of our servers running ehcache - around 10 in all - every day as shown below.

Maybe our load increased over time and is pushing ehcache too hard. This error has been creating a JVM hung condition after a few hours of the dead lock. Any ideas?

 /home>  ps -eLo pid,lwp,nlwp,ruser,pcpu,stime,etime,args|grep 89366 | awk ' $5 >= 2 ' (Wildfly PID 89366, find Linux lightweight process (LWP) with CPU usage greater than 2%) 
 89366  81979  118 prddepl+ 98.2 09:45    03:57:44 /apps/jdk1.8.0_92/bin/java -server -Xms1024m -Xmx5120m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=512m -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+ScavengeBeforeFullGC -XX:+CMSScavengeBeforeRemark -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/apps/wildfly/standalone/log -Dsun.net.inetaddr.ttl=60 -Djava.awt.headless=true -Djboss.modules.system.pkgs=org.jboss.byteman -Dorg.tanukisoftware.wrapper.WrapperManager.mbean=false -Dlogging.configuration=file:/apps/wildfly/standalone/configuration/logging.properties -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dorg.jboss.logging.Logger.pluginClass=org.jboss.logging.logmanager.LoggerPluginImpl -Djava.io.tmpdir=/tmp -Djava.net.preferIPv4Stack=true -javaagent:/apps/wildfly/standalone/lib/notsoserial-1.0-SNAPSHOT.jar -Dnotsoserial.whitelist=/apps/wildfly/bin/is-deserialized.txt -XX:-OmitStackTraceInFastThrow -Djava.library.path=/apps/wildfly/standalone/lib -classpath /apps/wildfly/standalone/lib/wrapper.jar:/apps/wildfly/standalone/lib/notsoserial-1.0-SNAPSHOT.jar:/apps/wildfly/standalone/lib/wrapper.jar -Dwrapper.key=u02jJraVjSs0PwNd -Dwrapper.port=32000 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.disable_console_input=TRUE -Dwrapper.pid=89364 -Dwrapper.version=3.5.13 -Dwrapper.native_library=wrapper -Dwrapper.service=TRUE -Dwrapper.cpu.timeout=10 -Dwrapper.jvmid=1 org.tanukisoftware.wrapper.WrapperJarApp /apps/wildfly/jboss-modules.jar -mp /apps/wildfly/modules -logmodule org.jboss.logmanager -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -Djboss.home.dir=/apps/wildfly -b=0.0.0.0

[my-web1.idm.local(myuser)] 
 /home> printf "%x\n" 81979 (convert LWP to hex) 
1403b

[my-web1.idm.local(myuser)] 
 /home> jstack -l  89366 | grep -A 10 1403b
"EventsThread <3096>" #248148 prio=5 os_prio=0 tid=0x00007f815c076000 nid=0x1403b runnable [0x00007f81326e5000]
   java.lang.Thread.State: RUNNABLE
        at net.sf.ehcache.store.disk.Segment.clearFaultedBit(Segment.java:1051)
        at net.sf.ehcache.store.disk.DiskStore.clearFaultedBit(DiskStore.java:675)
        at net.sf.ehcache.store.CacheStore.flush(CacheStore.java:436)
        at net.sf.ehcache.Cache.flush(Cache.java:2650)
        - locked <0x000000069dd8d428> (a net.sf.ehcache.Cache)
        at com.mycompany.ticker.cache.ExternalTickerCacheManager.onUpdate(ExternalTickerCacheManager.java:945)

Edit: see ehcache config below.

<ehcache name="mycompany-tickerEhcache-dev" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="ehcahce.xsd">
     <!--Default Cache configuration. These will applied to caches programmatically created through
    the CacheManager.

    The following attributes are required for defaultCache:

    maxInMemory       - Sets the maximum number of objects that will be created in memory
    eternal           - Sets whether elements are eternal. If eternal,  timeouts are ignored and the element
                        is never expired.
    timeToIdleSeconds - Sets the time to idle for an element before it expires.
                        i.e. The maximum amount of time between accesses before an element expires
                        Is only used if the element is not eternal.
                        Optional attribute. A value of 0 means that an Element can idle for infinity
    timeToLiveSeconds - Sets the time to live for an element before it expires.
                        i.e. The maximum time between creation time and when an element expires.
                        Is only used if the element is not eternal.
    overflowToDisk    - Sets whether elements can overflow to disk when the in-memory cache
                        has reached the maxInMemory limit.
    timeToIdleSeconds="500"
    timeToLiveSeconds="700"
    -->

<diskStore path="/apps/wildfly/conf/tickerEhcache-dev"/>

<defaultCache maxElementsInMemory="0" eternal="true" memoryStoreEvictionPolicy="LRU" diskPersistent="true" >
</defaultCache>

<cache name="com.mycompany.ticker.ehcache"
  maxElementsInMemory="0"
  eternal="true"
  overflowToDisk="false"
  memoryStoreEvictionPolicy="LFU"
  diskPersistent="true" >

  <bootstrapCacheLoaderFactory class="com.mycompany.ticker.cache.MyCompanyDiskStoreBootstrapCacheLoaderFactory"
                               properties="bootstrapAsynchronously=true, interval=5"/>
</cache>

<cache name="com.mycompany.ticker.history.ehcache" eternal="false" maxElementsInMemory="0" overflowToDisk="false" diskPersistent="true" >

<cacheWriter writeMode="write_behind" notifyListenersOnException="true" maxWriteDelay="0" 
                   writeCoalescing="false" writeBatching="true" writeBatchSize="25" 
                   retryAttempts="1" retryAttemptDelaySeconds="1" rateLimitPerSecond="0">
</cacheWriter>
</cache>
</ehcache>