MemoryCache with regions support?

Question

I need to add cache functionality and found a new shiny class called MemoryCache. However, I find MemoryCache a little bit crippled as it is (I'm in need of regions functionality). Among other things I need to add something like ClearAll(region). Authors made a great effort to keep this class without regions support, code like:

if (regionName != null)
{
throw new NotSupportedException(R.RegionName_not_supported);
}

flies in almost every method. I don't see an easy way to override this behaviour. The only way to add region support that I can think of is to add a new class as a wrapper of MemoryCache rather then as a class that inherits from MemoryCache. Then in this new class create a Dictionary and let each method "buffer" region calls. Sounds nasty and wrong, but eventually...

Do you know of better ways to add regions to MemoryCache?

Answer 1

You can create more than one just one MemoryCache instance, one for each partition of your data.

http://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache.aspx :

you can create multiple instances of the MemoryCache class for use in the same application and in the same AppDomain instance

Answer 2

I know it is a long time since you asked this question, so this is not really an answer to you, but rather an addition for future readers.

I was also surprised to find that the standard implementation of MemoryCache does NOT support regions. It would have been so easy to provide right away. I therefore decided to wrap the MemoryCache in my own simple class to provide the functionality I often need.

I enclose my code it here to save time for others having the same need!

/// <summary>
/// =================================================================================================================
/// This is a static encapsulation of the Framework provided MemoryCache to make it easier to use.
/// - Keys can be of any type, not just strings.
/// - A typed Get method is provided for the common case where type of retrieved item actually is known.
/// - Exists method is provided.
/// - Except for the Set method with custom policy, some specific Set methods are also provided for convenience.
/// - One SetAbsolute method with remove callback is provided as an example.
///   The Set method can also be used for custom remove/update monitoring.
/// - Domain (or "region") functionality missing in default MemoryCache is provided.
///   This is very useful when adding items with identical keys but belonging to different domains.
///   Example: "Customer" with Id=1, and "Product" with Id=1
/// =================================================================================================================
/// </summary>
public static class MyCache
{
    private const string KeySeparator = "_";
    private const string DefaultDomain = "DefaultDomain";


    private static MemoryCache Cache
    {
        get { return MemoryCache.Default; }
    }

    // -----------------------------------------------------------------------------------------------------------------------------
    // The default instance of the MemoryCache is used.
    // Memory usage can be configured in standard config file.
    // -----------------------------------------------------------------------------------------------------------------------------
    // cacheMemoryLimitMegabytes:   The amount of maximum memory size to be used. Specified in megabytes. 
    //                              The default is zero, which indicates that the MemoryCache instance manages its own memory
    //                              based on the amount of memory that is installed on the computer. 
    // physicalMemoryPercentage:    The percentage of physical memory that the cache can use. It is specified as an integer value from 1 to 100. 
    //                              The default is zero, which indicates that the MemoryCache instance manages its own memory 
    //                              based on the amount of memory that is installed on the computer. 
    // pollingInterval:             The time interval after which the cache implementation compares the current memory load with the 
    //                              absolute and percentage-based memory limits that are set for the cache instance.
    //                              The default is two minutes.
    // -----------------------------------------------------------------------------------------------------------------------------
    //  <configuration>
    //    <system.runtime.caching>
    //      <memoryCache>
    //        <namedCaches>
    //          <add name="default" cacheMemoryLimitMegabytes="0" physicalMemoryPercentage="0" pollingInterval="00:02:00" />
    //        </namedCaches>
    //      </memoryCache>
    //    </system.runtime.caching>
    //  </configuration>
    // -----------------------------------------------------------------------------------------------------------------------------



    /// <summary>
    /// Store an object and let it stay in cache until manually removed.
    /// </summary>
    public static void SetPermanent(string key, object data, string domain = null)
    {
        CacheItemPolicy policy = new CacheItemPolicy { };
        Set(key, data, policy, domain);
    }

    /// <summary>
    /// Store an object and let it stay in cache x minutes from write.
    /// </summary>
    public static void SetAbsolute(string key, object data, double minutes, string domain = null)
    {
        CacheItemPolicy policy = new CacheItemPolicy { AbsoluteExpiration = DateTime.Now + TimeSpan.FromMinutes(minutes) };
        Set(key, data, policy, domain);
    }

    /// <summary>
    /// Store an object and let it stay in cache x minutes from write.
    /// callback is a method to be triggered when item is removed
    /// </summary>
    public static void SetAbsolute(string key, object data, double minutes, CacheEntryRemovedCallback callback, string domain = null)
    {
        CacheItemPolicy policy = new CacheItemPolicy { AbsoluteExpiration = DateTime.Now + TimeSpan.FromMinutes(minutes), RemovedCallback = callback };
        Set(key, data, policy, domain);
    }

    /// <summary>
    /// Store an object and let it stay in cache x minutes from last write or read.
    /// </summary>
    public static void SetSliding(object key, object data, double minutes, string domain = null)
    {
        CacheItemPolicy policy = new CacheItemPolicy { SlidingExpiration = TimeSpan.FromMinutes(minutes) };
        Set(key, data, policy, domain);
    }

    /// <summary>
    /// Store an item and let it stay in cache according to specified policy.
    /// </summary>
    /// <param name="key">Key within specified domain</param>
    /// <param name="data">Object to store</param>
    /// <param name="policy">CacheItemPolicy</param>
    /// <param name="domain">NULL will fallback to default domain</param>
    public static void Set(object key, object data, CacheItemPolicy policy, string domain = null)
    {
        Cache.Add(CombinedKey(key, domain), data, policy);
    }




    /// <summary>
    /// Get typed item from cache.
    /// </summary>
    /// <param name="key">Key within specified domain</param>
    /// <param name="domain">NULL will fallback to default domain</param>
    public static T Get<T>(object key, string domain = null)
    {
        return (T)Get(key, domain);
    }

    /// <summary>
    /// Get item from cache.
    /// </summary>
    /// <param name="key">Key within specified domain</param>
    /// <param name="domain">NULL will fallback to default domain</param>
    public static object Get(object key, string domain = null)
    {
        return Cache.Get(CombinedKey(key, domain));
    }

    /// <summary>
    /// Check if item exists in cache.
    /// </summary>
    /// <param name="key">Key within specified domain</param>
    /// <param name="domain">NULL will fallback to default domain</param>
    public static bool Exists(object key, string domain = null)
    {
        return Cache[CombinedKey(key, domain)] != null;
    }

    /// <summary>
    /// Remove item from cache.
    /// </summary>
    /// <param name="key">Key within specified domain</param>
    /// <param name="domain">NULL will fallback to default domain</param>
    public static void Remove(object key, string domain = null)
    {
        Cache.Remove(CombinedKey(key, domain));
    }



    #region Support Methods

    /// <summary>
    /// Parse domain from combinedKey.
    /// This method is exposed publicly because it can be useful in callback methods.
    /// The key property of the callback argument will in our case be the combinedKey.
    /// To be interpreted, it needs to be split into domain and key with these parse methods.
    /// </summary>
    public static string ParseDomain(string combinedKey)
    {
        return combinedKey.Substring(0, combinedKey.IndexOf(KeySeparator));
    }

    /// <summary>
    /// Parse key from combinedKey.
    /// This method is exposed publicly because it can be useful in callback methods.
    /// The key property of the callback argument will in our case be the combinedKey.
    /// To be interpreted, it needs to be split into domain and key with these parse methods.
    /// </summary>
    public static string ParseKey(string combinedKey)
    {
        return combinedKey.Substring(combinedKey.IndexOf(KeySeparator) + KeySeparator.Length);
    }

    /// <summary>
    /// Create a combined key from given values.
    /// The combined key is used when storing and retrieving from the inner MemoryCache instance.
    /// Example: Product_76
    /// </summary>
    /// <param name="key">Key within specified domain</param>
    /// <param name="domain">NULL will fallback to default domain</param>
    private static string CombinedKey(object key, string domain)
    {
        return string.Format("{0}{1}{2}", string.IsNullOrEmpty(domain) ? DefaultDomain : domain, KeySeparator, key);
    }

    #endregion

}

Answer 3

Another approach is to implement a wrapper around MemoryCache that implements regions by composing the key and region name e.g.

public interface ICache 
{
...
    object Get(string key, string regionName = null);
...
}

public class MyCache : ICache
{
    private readonly MemoryCache cache

    public MyCache(MemoryCache cache)
    {
        this.cache = cache.
    }
...
    public object Get(string key, string regionName = null)
    {
        var regionKey = RegionKey(key, regionName);

        return cache.Get(regionKey);
    }   

    private string RegionKey(string key, string regionName)
    {
       // NB Implements region as a suffix, for prefix, swap order in the format
       return string.IsNullOrEmpty(regionName) ? key : string.Format("{0}{1}{2}", key, "::", regionName);
    }
...
}

It's not perfect but it works for most use cases.

I've implemented this and it's available as a NuGet package: Meerkat.Caching

Answer 4

I just recently came across this problem. I know this is an old question but maybe this might be useful for some folks. Here is my iteration of the solution by Thomas F. Abraham

namespace CLRTest
{
    using System;
    using System.Collections.Concurrent;
    using System.Diagnostics;
    using System.Globalization;
    using System.Linq;
    using System.Runtime.Caching;

    class Program
    {
        static void Main(string[] args)
        {
            CacheTester.TestCache();
        }
    }

    public class SignaledChangeEventArgs : EventArgs
    {
        public string Name { get; private set; }
        public SignaledChangeEventArgs(string name = null) { this.Name = name; }
    }

    /// <summary>
    /// Cache change monitor that allows an app to fire a change notification
    /// to all associated cache items.
    /// </summary>
    public class SignaledChangeMonitor : ChangeMonitor
    {
        // Shared across all SignaledChangeMonitors in the AppDomain
        private static ConcurrentDictionary<string, EventHandler<SignaledChangeEventArgs>> ListenerLookup = 
            new ConcurrentDictionary<string, EventHandler<SignaledChangeEventArgs>>();

        private string _name;
        private string _key;
        private string _uniqueId = Guid.NewGuid().ToString("N", CultureInfo.InvariantCulture);

        public override string UniqueId
        {
            get { return _uniqueId; }
        }

        public SignaledChangeMonitor(string key, string name)
        {
            _key = key;
            _name = name;
            // Register instance with the shared event
            ListenerLookup[_uniqueId] = OnSignalRaised;
            base.InitializationComplete();
        }


        public static void Signal(string name = null)
        {
            // Raise shared event to notify all subscribers
            foreach (var subscriber in ListenerLookup.ToList())
            {
                subscriber.Value?.Invoke(null, new SignaledChangeEventArgs(name));
            }
        }

        protected override void Dispose(bool disposing)
        {
            // Set delegate to null so it can't be accidentally called in Signal() while being disposed
            ListenerLookup[_uniqueId] = null;
            EventHandler<SignaledChangeEventArgs> outValue = null;
            ListenerLookup.TryRemove(_uniqueId, out outValue);
        }

        private void OnSignalRaised(object sender, SignaledChangeEventArgs e)
        {
            if (string.IsNullOrWhiteSpace(e.Name) || string.Compare(e.Name, _name, true) == 0)
            {
                // Cache objects are obligated to remove entry upon change notification.
                base.OnChanged(null);
            }
        }
    }

    public static class CacheTester
    {
        private static Stopwatch _timer = new Stopwatch();

        public static void TestCache()
        {
            MemoryCache cache = MemoryCache.Default;
            int size = (int)1e6;

            Start();
            for (int idx = 0; idx < size; idx++)
            {
                cache.Add(idx.ToString(), "Value" + idx.ToString(), GetPolicy(idx, cache));
            }
            long prevCnt = cache.GetCount();
            Stop($"Added    {prevCnt} items");

            Start();
            SignaledChangeMonitor.Signal("NamedData");
            Stop($"Removed  {prevCnt - cache.GetCount()} entries");
            prevCnt = cache.GetCount();

            Start();
            SignaledChangeMonitor.Signal();
            Stop($"Removed  {prevCnt - cache.GetCount()} entries");
        }

        private static CacheItemPolicy GetPolicy(int idx, MemoryCache cache)
        {
            string name = (idx % 10 == 0) ? "NamedData" : null;

            CacheItemPolicy cip = new CacheItemPolicy();
            cip.AbsoluteExpiration = System.DateTimeOffset.UtcNow.AddHours(1);
            var monitor = new SignaledChangeMonitor(idx.ToString(), name);
            cip.ChangeMonitors.Add(monitor);
            return cip;
        }

        private static void Start()
        {
            _timer.Start();
        }

        private static void Stop(string msg = null)
        {
            _timer.Stop();
            Console.WriteLine($"{msg} | {_timer.Elapsed.TotalSeconds} sec");
            _timer.Reset();
        }
    }
}

His solution involved using an event to keep track of ChangeMonitors. But the dispose method was working slow when the number of entries were more than 10k. My guess is that this code SignaledChangeMonitor.Signaled -= OnSignalRaised removes a delegate from invocation list by doing a linear search. So when you remove a lot of entries it becomes slow. I decided to use ConcurrentDictionary instead of an event. In hope that dispose becomes faster. I ran some basic performance tests and here are the results:

Added    10000 items | 0.027697 sec
Removed  1000 entries | 0.0040669 sec
Removed  9000 entries | 0.0105687 sec

Added    100000 items | 0.5065736 sec
Removed  10000 entries | 0.0338991 sec
Removed  90000 entries | 0.1418357 sec

Added    1000000 items | 6.5994546 sec
Removed  100000 entries | 0.4176233 sec
Removed  900000 entries | 1.2514225 sec

I am not sure if my code does not have some critical flaws. I would like to know if that is the case.