Tag Archives: Testable Design

A Multilevel Cache Retrieval Design for .NET

Caching data is vital for high-performance web applications. However, cache retrieval code can get messy and hard to test without the proper abstractions. In this article, we’ll start an ugly multilevel cache and progressively refine it into something maintainable and testable.

The source code for this article is available at the Gigi Labs BitBucket repository.

Naïve Multilevel Cache Retrieval

A multilevel cache is just a collection of separate caches, listed in order of speed. We typically try to retrieve from the fastest cache first, and failing that, we try the second fastest; and so on.

For the example in this article we’ll use a simple two-level cache where:

We’re going to build a Web API method that retrieves a list of supported languages. We’ll prepare this data in Redis (e.g. using the command SADD languages en mt) but will leave the MemoryCache empty (so it will have to fall back to the Redis cache).

A simple implementation looks something like this:

    public class LanguagesController : ApiController
    {
        // GET api/languages
        public async Task<IEnumerable<string>> GetAsync()
        {
            // retrieve from MemoryCache

            var valueObj = MemoryCache.Default.Get("languages");

            if (valueObj != null)
                return valueObj as List<string>;
            else
            {
                // retrieve from Redis

                var conn = await ConnectionMultiplexer.ConnectAsync("localhost:6379");
                var db = conn.GetDatabase(0);
                var redisSet = await db.SetMembersAsync("languages");

                if (redisSet == null)
                    return null;
                else
                    return redisSet.Select(item => item.ToString()).ToList();
            }
        }
    }

Note: this is not the best way to create a Redis client connection, but is presented this way for the sake of simplicity.

Data Access Repositories and Multilevel Cache Abstraction

The controller method in the previous section is having to deal with cache fallback logic as well as data access logic that isn’t really its job (see Single Responsibility Principle). This results in bloated controllers, especially if we add additional cache levels (e.g. fall back to database for third-level cache).

To alleviate this, the first thing we should do is move data access logic into repositories (this is called the Repository pattern). So for MemoryCache we do this:

    public class MemoryCacheRepository : IMemoryCacheRepository
    {
        public Task<List<string>> GetLanguagesAsync()
        {
            var valueObj = MemoryCache.Default.Get("languages");
            var value = valueObj as List<string>;
            return Task.FromResult(value);
        }
    }

…and for Redis we have this instead:

    public class RedisCacheRepository : IRedisCacheRepository
    {
        public async Task<List<string>> GetLanguagesAsync()
        {
            var conn = await ConnectionMultiplexer.ConnectAsync("localhost:6379");
            var db = conn.GetDatabase(0);
            var redisSet = await db.SetMembersAsync("languages");

            if (redisSet == null)
                return null;
            else
                return redisSet.Select(item => item.ToString()).ToList();
        }
    }

The repositories each implement their own interfaces, to prepare for dependency injection which is one of our end goals (we’ll get to that later):

    public interface IMemoryCacheRepository
    {
        Task<List<string>> GetLanguagesAsync();
    }

    public interface IRedisCacheRepository
    {
        Task<List<string>> GetLanguagesAsync();
    }

For this simple example, the interfaces look almost identical. If your caches are going to be identical then you can take this article further and simplify things even more. However, I’m not assuming that this is true in general; you might not want to have a multilevel cache everywhere.

Let’s also add a new class to abstract the fallback logic:

    public class MultiLevelCache
    {
        public async Task<T> GetAsync<T>(params Task<T>[] tasks) where T : class
        {
            foreach(var task in tasks)
            {
                var retrievedValue = await task;

                if (retrievedValue != null)
                    return retrievedValue;
            }

            return null;
        }
    }

Basically this allows us to pass in a number of tasks, each corresponding to a cache lookup. Whenever a cache lookup returns null, we know it’s a cache miss, which is why we need the where T : class restriction. In that case we try the next cache level, until we finally run out of options and just return null to the calling code.

This class is async-only to encourage asynchronous retrieval where possible. Synchronous retrieval can use Task.FromResult() (as the MemoryCache retrieval shown earlier does) to conform with this interface.

We can now refactor our controller method into something much simpler:

        public async Task<IEnumerable<string>> GetAsync()
        {
            var memoryCacheRepository = new MemoryCacheRepository();
            var redisCacheRepository = new RedisCacheRepository();
            var cache = new MultiLevelCache();

            var languages = await cache.GetAsync(
                memoryCacheRepository.GetLanguagesAsync(),
                redisCacheRepository.GetLanguagesAsync()
            );

            return languages;
        }

The variable declarations will go away once we introduce dependency injection.

Multilevel Cache Repository

The code looks a lot neater now, but it is still not testable. We’re still technically calling cache retrieval logic from the controller. A cache depends on external resources (e.g. databases) and also potentially on time (if expiry is used), and that’s not good for unit tests.

A cache is not very different from the more tangible data sources (such as Redis or a database). With them it shares the function of retrieving data and the nature of relying on resources external to the application, which makes it incompatible with unit testing. A multilevel cache has the additional property that it is an abstraction for the underlying data sources, and is thus itself a good candidate for the repository pattern:

multilevel-cache-repository

We can now move all our cache retrieval logic into a new MultiLevelCacheRepository class:

    public class MultiLevelCacheRepository : IMultiLevelCacheRepository
    {
        public async Task<List<string>> GetLanguagesAsync()
        {
            var memoryCacheRepository = new MemoryCacheRepository();
            var redisCacheRepository = new RedisCacheRepository();
            var cache = new MultiLevelCache();

            var languages = await cache.GetAsync(
                memoryCacheRepository.GetLanguagesAsync(),
                redisCacheRepository.GetLanguagesAsync()
            );

            return languages;
        }
    }

Our controller now needs only talk to this repository, and carry out any necessary logic after retrieval (in this case we don’t have any):

        public async Task<IEnumerable<string>> GetAsync()
        {
            var repo = new MultiLevelCacheRepository();
            var languages = await repo.GetLanguagesAsync();
            return languages;
        }

Dependency Injection

Our end goal is to be able to write unit tests for our controller methods. A prerequisite for that is to introduce dependency injection.

Follow the instructions in “ASP .NET Web API Dependency Injection with Ninject” to set up Ninject, or use any other dependency injection framework you prefer.

In your dependency injection configuration class (NinjectWebCommon if you’re using Ninject), set up the classes and interfaces you need:

        private static void RegisterServices(IKernel kernel)
        {
            kernel.Bind<IMemoryCacheRepository>().To<MemoryCacheRepository>()
                .InSingletonScope();
            kernel.Bind<IRedisCacheRepository>().To<RedisCacheRepository>()
                .InSingletonScope();
            kernel.Bind<IMultiLevelCacheRepository>().To<MultiLevelCacheRepository>()
                .InSingletonScope();
            kernel.Bind<MultiLevelCache>().To<MultiLevelCache>()
                .InSingletonScope();
        }

Note: you can also set up an interface for MultiLevelCache if you want. I didn’t do that out of pure laziness.

Next, refactor MultiLevelCacheRepository to get the classes it needs via constructor injection:

    public class MultiLevelCacheRepository : IMultiLevelCacheRepository
    {
        private IMemoryCacheRepository memoryCacheRepository;
        private IRedisCacheRepository redisCacheRepository;
        private MultiLevelCache cache;

        public MultiLevelCacheRepository(
            IMemoryCacheRepository memoryCacheRepository,
            IRedisCacheRepository redisCacheRepository,
            MultiLevelCache cache)
        {
            this.memoryCacheRepository = memoryCacheRepository;
            this.redisCacheRepository = redisCacheRepository;
            this.cache = cache;
        }

        public async Task<List<string>> GetLanguagesAsync()
        {
            var languages = await cache.GetAsync(
                memoryCacheRepository.GetLanguagesAsync(),
                redisCacheRepository.GetLanguagesAsync()
            );

            return languages;
        }
    }

Do the same with the controller:

    public class LanguagesController : ApiController
    {
        private IMultiLevelCacheRepository repo;

        public LanguagesController(IMultiLevelCacheRepository repo)
        {
            this.repo = repo;
        }

        // GET api/languages
        public async Task<IEnumerable<string>> GetAsync()
        {
            var languages = await repo.GetLanguagesAsync();
            return languages;
        }
    }

…and make sure it actually works:

multilevel-cache-verify

Unit Test

Thanks to this design, we can now write unit tests. There is not much to test for this simple example, but we can write a simple (!) test to verify that the languages are indeed retrieved and returned:

        [TestMethod]
        public async Task GetLanguagesAsync_LanguagesAvailable_Returned()
        {
            // arrange

            var languagesList = new List<string>() { "mt", "en" };

            var memCacheRepo = new Mock<MemoryCacheRepository>();
            var redisRepo = new Mock<RedisCacheRepository>();
            var cache = new MultiLevelCache();
            var multiLevelCacheRepo = new MultiLevelCacheRepository(
                memCacheRepo.Object, redisRepo.Object, cache);
            var controller = new LanguagesController(multiLevelCacheRepo);

            memCacheRepo.Setup(repo => repo.GetLanguagesAsync())
                        .ReturnsAsync(null);
            redisRepo.Setup(repo => repo.GetLanguagesAsync())
                        .ReturnsAsync(languagesList);

            // act

            var languages = await controller.GetAsync();
            var actualLanguages = new List<string>(languages);

            // assert

            CollectionAssert.AreEqual(languagesList, actualLanguages);
        }

Over here we’re using Moq’s Mock objects to help us with setting up the unit test. In order for this to work, we need to make our GetLanguagesAsync() method virtual in the data repositories:

public virtual Task<List<string>> GetLanguagesAsync()

Conclusion

Caching makes unit testing tricky. However, in this article we have seen how we can treat a cache just like any other repository and hide its retrieval implementation details in order to keep our code testable. We have also seen an abstraction for a multilevel cache, which makes cache fallback straightforward. Where cache levels are identical in terms of data, this approach can probably be simplified even further.

A Framework for Application Settings

It is a very common practice to store settings in config keys within the AppSettings section of an App.config file. These settings then need to be read and converted to the appropriate type. One must also take care to cater for situations where the key is not found, or the value is invalid. This article provides a structured approach to this practice. Feel free to review and use the accompanying source code.

Update 2015-02-28: I made a minor improvement to ReadAsync() suggested by Stephen Cleary, who I thank for the code review.

Update 2015-03-03: Some people have asked why we actually need AppSettings any more, given that there are alternatives such as .NET Settings or custom configuration sections. They are correct. However I still see a lot of projects using AppSettings, and this article is intended to provide a better way to deal with those AppSettings.

Update 2015-11-12: If you want to use this in your own code, check out my .NET Settings Framework project which is based on this article and provides NuGet packages that you can just drop into your projects.

The Problem

I’ve seen a lot of production code that reads values from config keys in App.config that looks something like this:

            // set a default, just in case the key is not found or the conversion fails

            int timeout = 3000;

            // retrieve the value for the desired key

            string timeoutStr = ConfigurationManager.AppSettings["timeoutInMilliseconds"];

            // check whether the key was actually found; if not, the default value is retained

            if (timeoutStr != null)
            {
                // attempt to convert to the desired type
                //   -> if it succeeds, the default value is replaced with the retrieved value
                //   -> if it fails, the default value is retained

                bool converted = int.TryParse(timeoutStr, out timeout);
            }

Aside from the bloat due to comments and braces (which were both necessary to make this example clear), you can see that we essentially have four lines of code just to read an integer setting from App.config.

What’s really bad is that there will essentially be four lines of code for every setting, all doing essentially the same thing for different settings. That isn’t very DRY.

A Basic Solution

One of my earlier attempts at solving this problem involved a utility class which took care of reading the settings and converting them to the appropriate type, using a specific method per type:

    public class ConfigKey
    {
        private string key;

        public ConfigKey(string key)
        {
            this.key = key;
        }

        public int GetAsInt(int defaultValue = 0)
        {
            int value = defaultValue;

            string valueStr = ConfigurationManager.AppSettings[this.key];

            if (valueStr != null)
            {
                bool converted = int.TryParse(valueStr, out value);
            }

            return value;
        }

        public bool GetAsBool(bool defaultValue = false)
        {
            bool value = defaultValue;

            string valueStr = ConfigurationManager.AppSettings[this.key];

            if (valueStr != null)
            {
                bool converted = bool.TryParse(valueStr, out value);
            }

            return value;
        }

        // ...
    }

This approach was pretty decent, as it made it very easy to read settings and specify optional default values:

            int timeout = new ConfigKey("timeoutInMilliseconds").GetAsInt(3000);
            bool enabled = new ConfigKey("enabled").GetAsBool(false);

The only problem with this class is that while removes the bloat and duplication from the actual logic, it is full of duplication itself: you need a method per type to perform the type-specific conversion.

A Generic Approach

The duplication in the ConfigKey class is solved by using a generic conversion method:

        public T Get<T>(T defaultValue = default(T)) where T : IConvertible
        {
            T value = defaultValue;

            string valueStr = ConfigurationManager.AppSettings[this.key];

            if (valueStr != null)
            {
                try
                {
                    value = (T)Convert.ChangeType(valueStr, typeof(T));
                }
                catch(Exception)
                {
                    return defaultValue;
                }
            }

            return value;
        }

The usage changes as follows:

            int timeout = new ConfigKey("timeoutInMilliseconds").Get<int>(3000);
            bool enabled = new ConfigKey("enabled").Get<bool>(false);

That’s good enough for reading settings from App.config.

Dependency injection

In order to unit test our ConfigKey class, it’s best if we abstract out the dependency on App.config. In particular, we want to separate the part that reads the settings (reader) from the part that does the conversion and returns the value (provider).

For this, we need two interfaces. First, IConfigKeyReader is responsible to read the value of a setting from a source (e.g. App.config):

    public interface IConfigKeyReader
    {
        string Read(string key);
    }

Secondly, IConfigKeyProvider does all the rest: given a key, it returns a value (by internally using the IConfigKeyReader, which is not immediately evident from the interface):

    public interface IConfigKeyProvider
    {
        T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible;
    }

The IConfigKeyReader implementation for reading from App.config is extremely simple:

    public class AppSettingReader : IConfigKeyReader
    {
        public string Read(string key)
        {
            return ConfigurationManager.AppSettings[key];
        }
    }

The IConfigKeyProvider for App.config settings is almost the same as the code we had in the previous section, with one important exception: it no longer depends directly on ConfigurationManager. Instead, it depends on the IConfigKeyReader which is injected in the constructor. This reader can be mocked in unit tests.

    public class ConfigKeyProvider: IConfigKeyProvider
    {
        private IConfigKeyReader reader;

        public ConfigKeyProvider(IConfigKeyReader reader)
        {
            this.reader = reader;
        }

        public T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible
        {
            T value = defaultValue;

            string valueStr = reader.Read(key);

            if (valueStr != null)
            {
                try
                {
                    value = (T)Convert.ChangeType(valueStr, typeof(T));
                }
                catch (Exception)
                {
                    return defaultValue;
                }
            }

            return value;
        }
    }

You’ll also notice that we can now use a single instance of this AppSettingProvider to retrieve all our settings, rather than create a different ConfigKey for each setting. This approach is pretty handy if you’re using an IoC container to inject utility classes into your class constructors.

At this point we can throw away our old ConfigKey class, and instead use the new classes as follows:

            var reader = new AppSettingReader();
            var provider = new ConfigKeyProvider(reader);

            int timeout = provider.Get<int>("timeoutInMilliseconds", 3000);
            bool enabled = provider.Get<bool>("enabled", false);

Unit tests

Thanks to the separation between reader and provider, it is now easy to unit test our provider code while mocking our reader code. The reader will be source-specific and depends on external factors (e.g. files or databases) so it doesn’t make sense to unit test that. But we can unit test our provider, which handles the conversion and default values, and which will be reused whatever the reader (in fact notice the names used in the code above: AppSettingReader is specific to App.config AppSettings, but ConfigKeyProvider is used for any config key).

In the example unit test below, I’m using Moq to create a mock IConfigKeyReader, and thus test that the provider code works as expected:

        [TestMethod]
        public void Get_IntAvailableWithDefault_ValueReturned()
        {
            // arrange

            var key = "timeoutInMilliseconds";

            var reader = new Mock<IConfigKeyReader>();
            reader.Setup(r => r.Read(key)).Returns("5000");

            var provider = new ConfigKeyProvider(reader.Object);

            // act

            var expected = 5000;
            var actual = provider.Get<int>(key, 3000);

            // assert

            Assert.AreEqual(expected, actual);
        }

For the sake of brevity I won’t include the other unit tests here, but you can find them in the source code accompanying this article.

Database settings

The separation between reader and provider that we achieved in the previous section means that we can reuse the provider code (responsible for conversion and default values) regardless of the source of the settings. This means that anyone can write, for example, a DbSettingReader class which implements IConfigKeyReader and retrieves settings from a database. Its implementation would depend on the database structure so there won’t be any single standard implementation.

However, there is one improvement to our framework that we can make to facilitate reading settings from external sources such as databases. In particular, nowadays it is quite easy to query a database asynchronously without having to block the application. So it makes sense to add support for async methods in our interfaces so that anyone writing a DbSettingReader can then provide an asynchronous implementation.

IConfigKeyReader now becomes:

    public interface IConfigKeyReader
    {
        string Read(string key);

        Task<string> ReadAsync(string key);
    }

We now need to update our AppSettingReader implementation accordingly. Since reading AppSettings from App.config isn’t asynchronous, we can use Task.FromResult() to help satisfy the contract:

    public class AppSettingReader : IConfigKeyReader
    {
        public string Read(string key)
        {
            return ConfigurationManager.AppSettings[key];
        }

        public Task<string> ReadAsync(string key)
        {
            var value = this.Read(key);
            return Task.FromResult(value);
        }
    }

The provider code also needs to be updated to support asynchrony. First the interface:

    public interface IConfigKeyProvider
    {
        T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible;

        Task<T> GetAsync<T>(string key, T defaultValue = default(T)) where T : IConvertible;
    }

The changes necessary to ConfigKeyProvider are a little more radical:

    public class ConfigKeyProvider : IConfigKeyProvider
    {
        private IConfigKeyReader reader;

        public ConfigKeyProvider(IConfigKeyReader reader)
        {
            this.reader = reader;
        }

        public T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible
        {
            string valueStr = reader.Read(key);

            return this.ConvertValue<T>(valueStr, defaultValue);
        }

        public async Task<T> GetAsync<T>(string key, T defaultValue = default(T)) where T : IConvertible
        {
            string valueStr = await reader.ReadAsync(key).ConfigureAwait(false);

            return this.ConvertValue<T>(valueStr, defaultValue);
        }

        private T ConvertValue<T>(string valueStr, T defaultValue)
        {
            if (valueStr != null)
            {
                try
                {
                    return (T)Convert.ChangeType(valueStr, typeof(T));
                }
                catch (Exception)
                {
                    return defaultValue;
                }
            }
            else
                return defaultValue;
        }
    }

I opted to move the conversion code to a method shared by the async and non-async methods, and then call separate reader code in them. I intentionally avoided having Get() call GetAsync().Result as it can result in deadlocks.

Technically the best approach would have been to drop the synchronous Get() method altogether and force the use of the asynchronous version. However, I realise there are times when people actually want to call the synchronous version, such as in Console applications or in constructors (although there are workarounds for both – see Async Console Programs and “Can constructors be async?“).

Conclusion and Source Code

This article has presented a simple framework that can be used to read application settings without having to bloat actual program logic. It supports reading AppSettings from an App.config file out of the box, and can easily be extended to support other sources (e.g. databases). It makes it easy to provide default values, works nicely with dependency injection, and can also be used asynchronously.

Check out the source code at the Gigi Labs Bitbucket repository. Feel free to use this code as you like, and let me know if you think it can be improved.