Distributed Cache - Simplified

Following Google search engine or advertising in technology sites or magazines we can easily find a lot of sophisticated solutions for caching.
I found/know: MemCached, NCache, ScaleOut StateServer,
Shared Cache & even a Microsoft's implementation named Velocity.

Sometimes these solutions are the best thing for your project, they have a lot of bells and whistles, they were written by professionals who dedicated a lot of thought in all sort of cache situations (or states..) & some of them already proved themselves in production environment - so why invent the wheel?
But...
If you think about it...cache is really simple - at least the basics, so why not take full control?

Cache mechanism is one of the basic infrastructure in every medium & above project, either web application or winForm application.
The main goal of caching is to save roundtrips, from client to server, from application server to database server & in some cases from web server to application server.
The last one (web server to application server) is mainly used in web-sites to save html result of common queries instead of redirecting these again & again to the application.
In this article I will concentrate on the other two.

In every application there are lookup tables (mainly to fill combo-boxes with list of values), decision tables and other static or semi-static tables, these tables are read again & again whenever a form in the application containing them is opened and/or the application flow needs its values - these roundtrips from client to server & from application server to database are a waste of resources that could be saved if you save these in the client's & application server's memory.

As first step you need to decide how far you can go with this.

1. Which static tables are commonly used?

2. How big are they? (memory on server & client are not endless...).

3. What kind of access is needed to these tables? (if you search a lot in these tables using joins it won't be very efficient to cache them).

4. If these tables are tables that the user can update, how critical it is to refresh them & if so - how often is sufficient?

The answers to these questions are different in every application, but the main guidelines are to cache most (if not all) small static tables that are used in the application & never to cache big tables, tables that are updated often or tables that you can't 'live' with the fact that you're querying an old snapshot of the real one (I'm talking about a few seconds old..).
You are left with the question what is the limit of medium tables (in size) & what is often updated - as said before these changes from application to application.

So let's start building...

We'll start with the database:

1. sysApplicationServers table: this table will function as registration table, every application server on load will 'register' itself here and will unregister itself when unloaded. Columns: IpAddress, FromDate.

2. CacheItemQueue table: this table will contain tables needed for refresh.

3. trig[table name]Cache trigger: every dynamic & cached table will have a trigger, this trigger will add a row in CacheItemQueue for every sysApplicationServers row (on insert, update & delete).

For example: we have two application server registered in sysApplicationServers, when we update users table the result of the trigger is a simple insert with the result:
Table, IpAddress, FromDate
-------------------------------------------
Users, appServer1IP, now
Users, appServer2IP, now

So this simple mechanism will 'let us know' when a table is modified so we can refresh its in memory snapshot (the insert will insert a new row only if there is no existing row for the same table+server combination).

Next we'll write a thread that will sample CacheItemQueue table in required interval, this thread will run in endless loop from application load.
When it identifies new table for it to load, it loads and deletes this row from cacheItemQueue.

            ...
            CacheListenerThread cacheListenerThread = new CacheListenerThread();
            thread = new Thread(cacheListenerThread.RunListener);
            thread.Start();

            while (true)
            {
                Thread.Sleep(Convert.ToInt32(AppConfigManager.GetAppSettingsValue("CacheRefreshInterval")));
                RefreshCache();
            }

        public void RefreshCache()
        {
            string ipAddress = BasicUtil.GetLocalIp();
            SqlCommand command = new SqlCommand("spCache_AsyncTablesLoader");
            DatabaseManager.AddSqlParameter(command, "@ipAddress", ipAddress);

            RefreshCacheInnerImpl(command);
        }
        ...

The actual cache can be built using asp.net which has nice implementation, I chose Microsoft's Enterprise library to allow the cache to work also under a non web application server (in my case windows service).

Cache manager interface explains itself:

    public interface ICacheManager
    {
        object Get(string key);

        bool Add(string key, object value);

        bool Contains(string key);

        void LoadDataTable(string tableName);
    }

LoadDataTable method will allow us to maintain cache tables that are loaded only on first use or reloaded if needed.

I mainly use this infrastructure to contain all sorts of dataTables but as you can see it's built to contain any object & also cache stuff with expiration date/time.
The Server implementation of cache manager:

..
using Microsoft.Practices.EnterpriseLibrary.Caching;
using Microsoft.Practices.EnterpriseLibrary.Caching.Expirations;
..

    public class ServerCache : ICacheManager
    {
        public delegate bool LoadDataTableDelegate(string tableName);

        private static ServerCache serverCacheManager;
        private CacheManager cacheManager;

        private event LoadDataTableDelegate loadDataTableEvent;

        private ServerCache()
        {
            try
            {
                cacheManager = CacheFactory.GetCacheManager();
            }
            catch (Exception ex)
            {
                throw new ApplicationException("Failed to initilize cache manager", ex);
            }
        }

        public static void InitCache(LoadDataTableDelegate loadDataTable)
        {
            serverCacheManager = new ServerCache();
            serverCacheManager.loadDataTableEvent += loadDataTable;
        }

        /// 

        /// get the server cache in a lazy fashion.
        /// 
        /// 
        public static ServerCache GetServerCache()
        {
            if (serverCacheManager == null)
            {
                string message = "cache was not loaded (should call InitCache)";
                throw new ApplicationException(message);
            }
            return serverCacheManager;
        }

        /// 

        /// get value from the cache by the given key
        /// 
        /// 
        /// 
        public object Get(string key)
        {
            return cacheManager.GetData(key);
        }

        /// 

        /// check if object with given key exists in cache
        /// 
        /// 
        /// 
        public bool Contains(string key)
        {
            return cacheManager.Contains(key);
        }

        /// 

        /// add item to cache
        /// 
        /// 
        /// 
        /// true if the key was overriden
        public bool Add(string key, object value)
        {
            //if the key already exist - run the value over and return false
            bool result = (cacheManager.Contains(key));
            cacheManager.Add(key, value);
            return result;
        }

        /// 

        /// add item to cache with timeout
        /// 
        /// 
        /// 
        /// 
        /// true if the key was overriden
        public bool Add(string key, object value, TimeSpan expirationTime)
        {
            //check if there is an object which is already cached with the same key
            bool result = (cacheManager.Contains(key));
            cacheManager.Add(key, value, CacheItemPriority.Normal, null, new SlidingTime(expirationTime));
            return result;
        }

        public void LoadDataTable(string tableName)
        {
            loadDataTableEvent(tableName);
        }
    }

The client implementation of ICacheManager is even simpler, it holds a static dictionary of objects, the LoadDataTable method can point to the server's gateway delegate or can be left alone if you download only static tables to client side.

    public class ClientCache : ICacheManager
    {
        private static ClientCache clientCacheManager;

        private static Dictionary cacheMap;

        private ClientCache()
        {
            cacheMap = new Dictionary();
        }

        public static ClientCache GetClientCache()
        {
            if (clientCacheManager == null)
            {
                clientCacheManager = new ClientCache();
            }
            return clientCacheManager;
        }

        public object Get(string key)
        {
            object result;
            cacheMap.TryGetValue(key, out result);
            return result;
        }

        /// 

        /// check if object with given key exists in cache
        /// 
        /// 
        /// 
        public bool Contains(string key)
        {
            return cacheMap.ContainsKey(key);
        }

        public bool Add(string key, object value)
        {
            bool overrideKey = cacheMap.ContainsKey(key);
            if (overrideKey)
            {
                lock (cacheMap)
                {
                    cacheMap.Remove(key);
                    cacheMap.Add(key, value);
                }
            }
            else
            {
                cacheMap.Add(key, value);
            }
            return overrideKey;
        }

        public void LoadDataTable(string tableName)
        {
            string message = string.Format("table {0} was not loaded to client cache", tableName);
            throw new ValidationException(message);
        }

    }

To allow the cache to be configured except the interval we sample the cacheItemQueue we use a simple xml that contains the list of tables to be cached.
Every element in this xml contains three attributes (except the name of the table of course):
1. loadOnStart: load on application load or on first call.
2. loadToClient: include table in the response to client's "getCache" method on client's load.
3. refreshOnUpdate: if a cache table can be updated this will be true (to be sure all tables that "refreshOnUpdate" have a trigger we have a deployment utility that uses this same xml to automaticly create these triggers and we check this matches on application load).

If I summarize the main idea:

1. On application load the cache data is retrieved to application server's memory, the server registers itself to receive updates & starts the thread checking for updates.

2. Each dynamic table in cache has a trigger that is used to eventually notify the application server about the update and force it to refresh, the refresh could be made to the whole table (usually small tables with small number of writes) or you can use a timestamp column to identify which rows were updated and selectively refresh the cache.

3. Every client on load retrieves a snapshot of the static cached tables to save roundtrips to server.

That's it for now, till next time...

Diego