Membase server And Enyim -- Integrating LINQ and/or Collections

Question

I just installed membase and the enyim client for .NET, and came across an article that mentions this technique for integrating linq:

    public static IEnumerable<T> CachedQuery<T>
        (this IQueryable<T> query, MembaseClient cache, string key) where T : class
    {
        object result; 
        if (cache.TryGet(key, out result))
        {
            return (IEnumerable<T>)result;
        }
        else
        {
            IEnumerable<T> items = query.ToList();
            cache.Store(StoreMode.Set, key, items);
            return items;
        }
    }

It will check if the required data is in cache first, and if not cache it then return it.

Currently I am using a Dictionary<'String, List'> in my application and want to replace this with a membase/memcached type approach.

What about a similar pattern for adding items to a List<'T'> or using Linq operators on a cached list? It seems to me that it could be a bad idea to store an entire List<'T'> in cache under a single key and have to retrieve it, add to it, and then re-set it each time you want to add an element. Or is this an acceptable practice?

    public bool Add(T item)
    {
        object list;
        if (cache.TryGet(this.Key, out list))
        {
            var _list = list as List<T>;
            _list.Add(item);
            return cache.Store(StoreMode.Set, this.Key, _list); 
        }
        else
        {
            var _list = new List<T>(new T[] { item });
            return cache.Store(StoreMode.Set, this.Key, _list); 
        }
    }

How are collections usually handled in a caching situation like this? Are hashing algorithms usually used instead, or some sort of key-prefixing system to identify 'Lists' of type T within the key-value store of the cache?

Answer 1

It depends on several factors: Is this supposed to be scalable? Is this list user-specific and you can be certain that "Add" won't be called twice at the same time for the same list? - Race conditions are a risk.

I did implement such a thing where I stored a generic list in membase, but it's user-specific, so I can be pretty certain that there will be no race condition.

You should also consider the volume of the serialized list, which may be large. I my case the lists were pretty small.

Not sure if it helps, but I implemented a very basic iterateable list with random access over membase (via double indirection). Random access is done via a composite key (which is composed of several fields).

You need to:

1. Have a key that holds the list's length.
2. Have the ability to build the composite key (e.g one or more fields from your object).
3. Have the value that you'd like save (e.g. another field).

E.g:

list_length = 3

prefix1_0-> prefix2_[field1.value][field2.value][field3.value] -> field4.value

prefix1_1-> prefix2_[field1.value][field2.value][field3.value] -> field4.value

prefix1_2-> prefix2_[field1.value][field2.value][field3.value] -> field4.value

To perform serial access you iterate over the keys with "prefix1". To perform random access you use the keys with "prefix2" ans the fields that compose the key.

I hope it's clear enough.