Tag Archives: async

The Dangers of async void Event Handlers

When using async/await, you’ll want to use async Task methods most of the time, and use async void methods only for event handlers (see “Async/Await – Best Practices in Asynchronous Programming“, by Stephen Cleary, MSDN Magazine, March 2013).

This conventional wisdom works great if you’re building something like a WPF (GUI) application, and event handlers are invoked occasionally as a result of user interaction (e.g. user presses a button, and an event fires). However, there is another class of event handlers that are invoked as part of a dispatcher loop in a third-party library. async void can be pretty dangerous in these cases.

async void Event Handlers in RabbitMQ

Let’s take the example of RabbitMQ. We’ll set up a basic publisher and consumer. A fundamental property of message queues is that messages are delivered in order, and that’s what we expect to happen.

First, install the RabbitMQ Client library via NuGet:

Install-Package RabbitMQ.Client

Then, we can set up a basic publisher and consumer:

        static void Main(string[] args)
        {
            Console.Title = "async RabbitMQ";

            var factory = new ConnectionFactory();

            using (var connection = factory.CreateConnection())
            using (var channel = connection.CreateModel())
            {
                const string queueName = "testqueue";

                // create queue if not already there

                channel.QueueDeclare(queueName, true, false, false, null);

                // publish

                var props = channel.CreateBasicProperties();
                    
                for (int i = 0; i < 5; i++)
                {
                    var msgBytes = Encoding.UTF8.GetBytes("Message " + i);
                    channel.BasicPublish("", queueName, props, msgBytes);
                }

                // set up consumer

                var consumer = new EventingBasicConsumer(channel);
                consumer.Received += Consumer_Received;
                channel.BasicConsume("testqueue", true, consumer);

                Console.ReadLine();
            }
        }

Our consumer will call the Consumer_Received event handler whenever a message is received. This is the first version of the event handler:

        private static void Consumer_Received(object sender, BasicDeliverEventArgs e)
        {
            var body = e.Body;
            var content = Encoding.UTF8.GetString(body);

            Console.WriteLine("Began handling " + content);

            Thread.Sleep(1000);

            Console.WriteLine("Finished handling " + content);
        }

If we run this now, the messages are processed one at a time and in order just a we expect:

rabbitmq-async-sync

Now, let’s change the event handler to an asynchronous one:

        private static async void Consumer_Received(object sender, BasicDeliverEventArgs e)
        {
            var body = e.Body;
            var content = Encoding.UTF8.GetString(body);

            Console.WriteLine("Began handling " + content);

            await Task.Delay(1000);

            Console.WriteLine("Finished handling " + content);
        }

If we run this now…

rabbitmq-async-async

…we see that our concurrency and ordering guarantees have just gone out the window.

Understanding async void, Revisited

In my recent article, “In-Depth Async in Akka .NET: Why We Need PipeTo()“, I explained what happens when you call async void methods. Let’s recap that.

Say we have this program. We’re calling an async void method in a loop.

        static void Main(string[] args)
        {
            Console.Title = "async void";

            for (int i = 0; i < 5; i++)
                RunJob("Job " + i);

            Console.ReadLine();
        }

        static async void RunJob(string str)
        {
            Console.WriteLine("Start " + str);

            await Task.Delay(1000);

            Console.WriteLine("End " + str);
        }

When you call an async void method, it’s done in a fire-and-forget manner. The caller has no way of knowing whether or when the operation ended, so it just resumes execution immediately, rather than waiting for the async void method to finish. So you end up with parallel and interleaved execution such as this:

rabbitmq-async-asyncvoid

If we change RunJob() to be synchronous instead…

        static void RunJob(string str)
        {
            Console.WriteLine("Start " + str);

            Thread.Sleep(1000);

            Console.WriteLine("End " + str);
        }

…you’ll see that everything happens one at a time and in order:

rabbitmq-async-sync2

So you have to be really careful when using async void:

  1. There is no way for the caller to await completion of the method.
  2. As a result of this, async void calls are fire-and-forget.
  3. Thus it follows that async void methods (including event handlers) will execute in parallel if called in a loop.
  4. Exceptions can cause the application to crash (see the aforementioned article by Stephen Cleary for more on this).

Fixing async void event handlers

Despite these problems, if you want to await in your event handler, you have to make it async void. To prevent parallel and interleaved execution, you have to lock. However, you can’t await in a lock block, so you need to use a different synchronisation mechanism such as a semaphore.

My own Dandago.Utilities provides a ScopedAsyncLock that allows you to neatly wrap the critical section in a using block:

        private static ScopedAsyncLockFactory factory = new ScopedAsyncLockFactory();

        private static async void Consumer_Received(object sender, BasicDeliverEventArgs e)
        {
            using (var scopedLock = await factory.CreateLockAsync())
            {
                var body = e.Body;
                var content = Encoding.UTF8.GetString(body);

                Console.WriteLine("Began handling " + content);

                await Task.Delay(1000);

                Console.WriteLine("Finished handling " + content);
            }
        }

Like this, messages are consumed one at a time, and in order:

rabbitmq-async-scopedasynclock

ScopedAsyncLockFactory uses a semaphore underneath, so don’t forget to dispose it!

Asynchronous and Concurrent Processing in Akka .NET Actors

Yesterday, I wrote a long article about asynchronous processing in Akka .NET actors, the dangers of async void methods, and PipeTo().

The article was written as a result of outdated Akka .NET documentation which claimed that async/await was evil in Akka .NET actors and recommended using PipeTo() instead (the documentation has since been updated). This was further exacerbated by the confusion between asynchronous and concurrent processing in the documentation.

I would like to thank Aaron Stannard and Roger Alsing from the Akka .NET team who, via separate channels, clarified a lot of the confusion. This new article covers asynchronous and concurrent processing in Akka .NET actors as a result.

Asynchronous vs Concurrent Processing

A lot of people confuse asynchronous and concurrent processing, more so given that you can do both with .NET Tasks.

This is asynchronous:

        static async Task RunAsync()
        {
            await Task.Delay(1000);
        }

RunAsync begins executing, but is suspended waiting for some external operation to occur (typically I/O such as reading from a database or a REST service). It can resume execution when the operation completes, but can’t continue doing other things in the meantime.

This, on the other hand, is concurrent:

        static async Task RunAsync()
        {
            Task.Delay(1000);
        }

Because the task is not awaited, the method proceeds with its execution while the task is running. In fact, we get a nice big warning when we do this in a method marked as async:

akkanet-async-not-awaited

If we extend our example just a little bit, we can understand the behaviour better:

        static Task RunAsync()
        {
            Task.Delay(1000)
                .ContinueWith(x => Console.WriteLine("After Delay"));
            Console.WriteLine("End of RunAsync()");
        }

The output is as follows:

End of RunAsync()
After Delay

Now, I could go on about definitions of synchrony and asynchrony, concurrency and parallelism, interleaving, and task-switching. But that would bore you to tears, and it’s really not the point here.

The important thing is to realise that despite using very similar C# syntax, we’re doing two very different things here. And I need to make this clear because PipeTo() is really targeted at concurrent processing, although it is described within the context of asynchrony in Akka .NET documentation.

async/await in actors

You can do async/await in an actor by using the ReceiveActor‘s ReceiveAsync() method:

    public class MyActor : ReceiveActor
    {
        public MyActor()
        {
            this.ReceiveAsync<string>(HandleAsync);
        }

        public async Task HandleAsync(string str)
        {
            Console.WriteLine($"Begin {str}");

            await Task.Delay(2000);

            Console.WriteLine($"End {str}");
        }
    }

This is perfectly valid, and you will most likely resort to this when you need a guarantee on message order. In such situations, you don’t want to start processing the next message while waiting for an I/O operation to complete, as it is possible for the results of an older message to overwrite those of a newer message.

It is also very useful for sequential steps in I/O operations that depend directly on each other, such as when talking to a database:

        public async Task HandleAsync(string str)
        {
            const string connStr = @"Data Source=.\SQLEXPRESS;Database=test;Integrated Security=true";
            using (var conn = new SqlConnection(connStr))
            {
                await conn.OpenAsync();

                const string sql = "select * from person;";

                using (var command = new SqlCommand(sql, conn))
                using (var reader = await command.ExecuteReaderAsync())
                {
                    while (await reader.ReadAsync())
                    {
                        string id = reader["id"].ToString();
                        string name = reader["name"].ToString();

                        Console.WriteLine($"{id} {name}");
                    }
                }
            }
        }

However, this comes at a cost. First, there is a performance impact when compared to synchronous execution, because the Akka .NET pipeline needs to carry the current context across asynchronous steps.

Secondly, the actor will not be able to process the next message until the current one is finished. Sometimes, this is exactly what you want, such as when you need a guarantee on message order. In such situations, you don’t want to start processing the next message while waiting for an I/O operation to complete, as it is possible for the results of an older message to overwrite those of a newer message. But if your message processing does not depend on prior state, you will get much more throughput if you run tasks concurrently and use PipeTo() to collect the results (more on this later).

Using Ask()

Ask() lets you do request/response between actors:

    public class ServiceActor : ReceiveActor
    {
        public ServiceActor()
        {
            this.ReceiveAsync<string>(HandleAsync);
        }

        public async Task HandleAsync(string str)
        {
            await Task.Delay(2000);
            Sender.Tell(str + " done");
        }
    }

    public class MyActor : ReceiveActor
    {
        private IActorRef serviceActor;

        public MyActor(IActorRef serviceActor)
        {
            this.serviceActor = serviceActor;

            this.ReceiveAsync<string>(HandleAsync);
        }

        public async Task HandleAsync(string str)
        {
            Console.WriteLine($"Begin {str}");

            var result = await this.serviceActor.Ask(str);

            Console.WriteLine($"End {result}");
        }
    }

Here is the output for this:

akkanet-async-ask

The approach above is something you typically want to avoid, for the same reason outlined in the previous section. If your actor is waiting for a response, it can’t process other messages in the meantime. Most of the time you should spawn a concurrent task as shown in the documentation, unless you have a good reason for not wanting to process the next message before the current one has finished. Try to design your system in a push fashion, rather than request/response.

Concurrent Execution and PipeTo()

If you have no reason to process messages in a strictly sequential manner, then you can do long-running tasks and I/O operations in a spawned task.

    public class MyActor : ReceiveActor
    {
        public MyActor()
        {
            this.Receive<string>(x => Handle(x));
        }

        public void Handle(string str)
        {
            Task.Run(async () =>
            {
                Console.WriteLine($"Begin {str}");

                await Task.Delay(2000);

                Console.WriteLine($"End {str}");
            });
        }
    }

Because of this, the actual processing you do within the task will be interleaved, as in yesterday’s article:

akkanet-async-concurrent

But this is okay, because we’re working on the assumption that the messages don’t need to be processed strictly in sequence.

Now if you want to send the result of your concurrent tasks somewhere, you can do that with PipeTo():

    public class MyActor : ReceiveActor
    {
        public MyActor()
        {
            this.Receive<string>(x => Handle(x));
            this.Receive<int>(x => Handle(x));
        }

        public void Handle(string str)
        {
            Task.Run(async () =>
            {
                Console.WriteLine($"Begin {str}");

                await Task.Delay(2000);

                Console.WriteLine($"End {str}");

                return 42;
            }).PipeTo(Self);
        }

        public void Handle(int result)
        {
            Console.WriteLine($"Got result: {result}");
        }
    }

The result of the concurrent operation is sent to the actor you specify (in this case to itself) and processed as any other message in its mailbox. You can also do post-processing (e.g. check HTTP status code after an HTTP GET operation) by adding a ContinueWith(); see the PipeTo() article on the Petabridge blog for an example.

More Advanced Concurrent Operations

Given that you can use both tasks and async/await in your actors, you can use all of the typical patterns you would normally use with the Task Parallel Library (TPL).

Here’s an example representing when you’d aggregate data from multiple external resources:

    public class MyActor : ReceiveActor
    {
        public MyActor()
        {
            this.ReceiveAsync<string>(x => HandleAsync(x));
        }

        public async Task HandleAsync(string str)
        {
            var task1 = Task.Delay(1000).ContinueWith(x => { return 1; });
            var task2 = Task.Delay(2000).ContinueWith(x => { return 2; });
            var task3 = Task.Delay(3000).ContinueWith(x => { return 3; });

            var results = await Task.WhenAll<int>(task1, task2, task3);
            var sum = results.Sum();
            Console.WriteLine(sum);
        }
    }

WhenAll() will wait for all the tasks to complete before the method can proceed with its execution. Here’s the output:

akkanet-async-whenall

Here’s another example which takes the result of whichever task completes first:

        public async Task HandleAsync(string str)
        {
            var task1 = Task.Delay(1000).ContinueWith(x => { return 1; });
            var task2 = Task.Delay(2000).ContinueWith(x => { return 2; });
            var task3 = Task.Delay(3000).ContinueWith(x => { return 3; });

            var result = await await Task.WhenAny<int>(task1, task2, task3);
            Console.WriteLine(result);
        }

In this example, WhenAny() suspends execution of the method until any of the tasks completes. The result from the fastest task is taken.

akkanet-async-whenany

Note: if you’re looking to do this kind of concurrent fastest-query operation, you might want to look at Akka .NET Routers with Routing Strategies such as ScatterGatherFirstCompleted.

In-Depth Async in Akka .NET: Why We Need PipeTo()

Update 21st August 2016: I wrote this article based on outdated Akka .NET documentation that discouraged async/await within actors and suggested using PipeTo() instead. Akka .NET now does support async/await (thanks to the ReceiveAsync() method), and PipeTo() is not a replacement for it. Aaron Stannard (in a comment on this post) and Roger Alsing (on Reddit) from the Akka .NET team were very prompt in correcting various misconceptions, and Aaron Stannard has since updated the Petabridge blog post about PipeTo(). See my followup post for the latest best practices.

Tasks and the more recent async/await syntactic sugar have been a blessing for .NET developers aiming to keep their applications responsive despite increasing requirements for I/O and CPU-intensive requests.

Thus it was really odd for me to learn that Akka .NET, an emergent framework for distributed computing, not only does not support async/await directly within actors, but actually discourages its use (going as far as calling them “code smell”).

In fact, they implemented this PipeTo() workaround that you need to use, sending the result of a task to an actor for processing. You can’t use async/await; you have to resort to the old ContinueWith() way of chaining tasks if you want to do any post-execution logic. If you’ve worked with ContinueWith() in the past, you’ll know it can get ugly really fast.

Why is it such a problem to have elegant asynchrony in our actors, seeing how competitor Microsoft Orleans has no problem with it? As Natan Vivo said in the comments of The Top 7 Mistakes Newbies Make with Akka.NET:

“The fact I decided to use DbCommand.ExecuteNonQueryAsync() instead of DbCommand.ExecuteNonQuery() shouldn’t force me to break a single message into multiple messages with PipeTo.”

Update 20th August 2016: Thanks to the Reddit user who brought to my attention that there actually is proper async support (though apparently not yet documented anywhere). Use the ReceiveActor’s ReceiveAsync() method.

Why Akka .NET Discourages async/await

To learn why awaiting in an actor is bad, let’s break the rules and do it.

    public class BusyActor : UntypedActor
    {
        protected override void OnReceive(object message)
        {
            Console.WriteLine($"Begin processing {message.ToString()}");

            Thread.Sleep(2000);

            Console.WriteLine($"End processing {message.ToString()}");
        }
    }

I have this example actor. For now it’s doing synchronous stuff, sleeping for a couple of seconds and writing something before and after so we can see the behaviour.

        static void Main(string[] args)
        {
            using (var actorSystem = ActorSystem.Create("MyActorSystem"))
            {
                var actor = actorSystem.ActorOf(Props.Create<BusyActor>(), "BusyActor");

                actor.Tell("Task 1");
                actor.Tell("Task 2");
                actor.Tell("Task 3");

                Console.ReadLine();
            }
        }

The main program simply creates the actor system and the actor, and then sends it three messages in succession.

akkanet-async-synchronous-output

As you can see, the messages are handled sequentially and there is no overlap.

Now let’s change the actor to work asynchronously instead:

    public class BusyActor : UntypedActor
    {
        protected override async void OnReceive(object message)
        {
            Console.WriteLine($"Begin processing {message.ToString()}");

            await Task.Delay(2000);

            Console.WriteLine($"End processing {message.ToString()}");
        }
    }

Run it again…

akkanet-async-async-output

What happened here? All three messages were processed in quick succession, and they have been interleaved. This is very bad, and in fact we were warned about it. Quoting the questions on the official PipeTo() sample:

“Await breaks the “actors process one message at a time” guarantee, and suddenly your actor’s context might be different. Variables such as the Sender of the previous message may be different, or the actor might even be shutting down when the await call returns to the previous context.”

Why Processing Messages Asynchronously Causes Interleaving

We can learn a lot about how actors process messages by investigating the Akka .NET source code. This method in Mailbox.cs seems to be more or less where actors begin to process their messages:

        private void ProcessMailbox(int left, long deadlineTicks)
        {
            while (ShouldProcessMessage())
            {
                Envelope next;
                if (!TryDequeue(out next)) return;

                DebugPrint("{0} processing message {1}", Actor.Self, next);

                // not going to bother catching ThreadAbortExceptions here, since they'll get rethrown anyway
                Actor.Invoke(next);
                ProcessAllSystemMessages();
                if (left > 1 && (Dispatcher.ThroughputDeadlineTime.HasValue == false || (MonotonicClock.GetTicks() - deadlineTicks) < 0))
                {
                    left = left - 1;
                    continue;
                }
                break;
            }
        }

From Actor.Invoke(), there is a succession of method calls that ends in a method called Receive() in UntypedActor.cs:

        protected sealed override bool Receive(object message)
        {
            OnReceive(message);
            return true;
        }

Our OnReceive() method, where implement our message-handling logic for our actors, is subsequently called.

Now, the code above may look confusing, but the point here is not to understand what it’s doing exactly. Take a closer look. The methods in the call stack are mostly void (or otherwise returning simple types). There are no Tasks to be seen anywhere.

What does this mean for us? It means that we’re doing something very bad when we declare our message handler as async void.

Understanding async void

In order to better understand why the approach we took earlier will never work, it’s best to look at a much simpler example:

    class Program
    {
        static void Main(string[] args)
        {
            RunAll();
            Console.ReadLine();
        }

        static void RunAll()
        {
            RunJob("Job 1");
            RunJob("Job 2");
            RunJob("Job 3");
        }

        static void RunJob(string str)
        {
            Console.WriteLine("Start " + str);

            Thread.Sleep(2000);

            Console.WriteLine("End " + str);
        }
    }

Here we’ve reproduced the earlier scenario, but with no Akka .NET. And with the synchronous implementation, it works just fine:

akkanet-async-taskasync-output

Let’s change RunJob() to run asynchronously:

        static async void RunJob(string str)
        {
            Console.WriteLine("Start " + str);

            await Task.Delay(2000);

            Console.WriteLine("End " + str);
        }

When we run it, the following happens:

akkanet-async-taskasync2-output

This is exactly the same interleaving problem we had with Akka .NET, except that this time we have no Akka .NET.

The real reason why we have this problem is due to an incorrect use of asynchrony. As you can read in Stephen Cleary’s MSDN Magazine article, “Async/Await – Best Practices in Asynchronous Programming” (March 2013), async void methods can be pretty dangerous to work with. When you call an async void method, you have two main problems: you have no way of awaiting completion of the method, and exceptions can bring the whole application down.

But here, we have also seen a third problem: that the method effectively exits when you await, returning execution control to the caller. In Akka .NET, this means that the next message will begin processing while the current one hasn’t finished yet.

async void methods should be restricted to methods at the beginning of the call chain (such as event handlers and WPF command handlers). You can’t sneak asynchrony into an otherwise synchronous call stack by introducing an async void. If you do async, it has to be all the way.

So it really seems that the problem with having asynchronous actor logic is simply that Akka .NET was never really designed to work with asynchronous methods.

Asynchrony in Akka .NET with PipeTo()

It should be clear by now that doing async/await in actors is not an option. So how do we go about doing our asynchronous work? We do that by using the PipeTo() pattern (because in Akka .NET, everything is called a pattern).

Let’s go back to our original example with the BusyActor. We left off with this code:

    public class BusyActor : UntypedActor
    {
        protected override async void OnReceive(object message)
        {
            Console.WriteLine($"Begin processing {message.ToString()}");

            await Task.Delay(2000);

            Console.WriteLine($"End processing {message.ToString()}");
        }
    }

Now, we need to refactor this to do the asynchronous operation (in this case Task.Delay()) in a fire-and-forget manner, and send the result as a separate message to an actor. We’re going to need separate messages for this:

    public class TaskMessage
    {
        public string Message { get; }

        public TaskMessage(string message)
        {
            this.Message = message;
        }

        public override string ToString()
        {
            return this.Message;
        }
    }

    public class ResultMessage
    {
        public string Message { get; }

        public ResultMessage(string message)
        {
            this.Message = message;
        }

        public override string ToString()
        {
            return this.Message;
        }
    }

Since our message handling is going to grow a little, UntypedActor is no longer suitable for what we need. Instead, we’ll refactor BusyActor as follows:

    public class BusyActor : ReceiveActor
    {
        public BusyActor()
        {
            this.Receive<TaskMessage>(m => Handle(m));
            this.Receive<ResultMessage>(m => Handle(m));
        }

        public void Handle(TaskMessage message)
        {
            Console.WriteLine($"Begin processing {message.ToString()}");

            Task.Delay(2000)
                .ContinueWith(x => new ResultMessage(message.Message),
                    TaskContinuationOptions.AttachedToParent
                    & TaskContinuationOptions.ExecuteSynchronously)
                .PipeTo(Self);
        }

        public void Handle(ResultMessage message)
        {
            Console.WriteLine($"End processing {message.ToString()}");
        }
    }

Similarly to the official example (which shows how to do an HTTP GET request within an actor), we are firing off an asynchronous request but not awaiting it. This happens in fire-and-forget manner as far as the actor is concerned. When the asynchronous operation is done, we create a new message and send it to the same actor so that he can log the end of the task.

Note that we have those two TaskContinuationOptions settings. You can read more about them in the official PipeTo() blog post, but the point I want to make here is that you need to remember to include them, and this makes this approach pretty error-prone.

Back in our main program, we need to send a TaskMessage instead of a simple string now:

        static void Main(string[] args)
        {
            using (var actorSystem = ActorSystem.Create("MyActorSystem"))
            {
                var actor = actorSystem.ActorOf(Props.Create<BusyActor>(), "BusyActor");

                actor.Tell(new TaskMessage("Task 1"));
                actor.Tell(new TaskMessage("Task 2"));
                actor.Tell(new TaskMessage("Task 3"));

                Console.ReadLine();
            }
        }

Let us now run this code:

akkanet-async-pipeto-interleaving

This is bad. Even with PipeTo(), we still have the same interleaving problem as before. If you think about it, it makes sense.

What we are doing is firing off a fire-and-forget task, and the method can return immediately, thus allowing the next message to be processed before the asynchronous operation has completed. This is exactly the same problem we had when using async void.

If you’re firing off an asynchronous operation that doesn’t touch anything else and you just want to take its result, then the suggested PipeTo() approach will work. But if you need a guarantee on message order because your message processing is touching some state (thus an older message might overwrite the results of a newer message), then this is going to be a problem.

Coupling and Cohesion

Another problem with using PipeTo() is that it… complicates things. You can already see how our original example has been bloated into something a lot less easy to work with, just for the sake of not using async/await. But there’s more.

One common pitfall I see when developers begin to understand the importance of decoupled software is that they go to the other extreme: they split up components into extremely granular classes. In doing so, they breaking the companion principle of coupling: cohesion. While coupling dictates that software components should have minimal dependencies between themselves, cohesion suggests that components with strong direct interrelations should work closely together. Making classes too granular, for instance, is another way to end up with messy software.

At the beginning of this article, I quoted Natan Vivo’s comment about having to break a database operation into multiple operations. Typically, in ADO .NET, a database operation would look something like this:

  1. Open a connection to the database.
  2. Execute a command (query, nonquery, etc) against the database.
  3. In case of a query, iterate over the rows and do something with them.

Each of the three operations above can be done asynchronously in sequence. They are meant to be together because they are part of the same cohesive operation. But if you break each of these operations into different messages and different message handlers, you’re going to scatter this otherwise contiguous operation all over the place. And that makes software a lot harder to maintain.

So when I see something like (to again quote the questions from the official Akka .NET PipeTo() sample) this:

“So just don’t do it. Await is evil inside an actor. Await is just syntactic sugar anyway. Use ContinueWith and PipeTo instead.”

…I feel the need to remind people that syntactic sugar is really important to make our software easier to write, but more importantly, easier to maintain.

For the reasons outlined above, I believe that the PipeTo() ‘pattern’ is really an anti-pattern, and I appeal for native asynchronous support in Akka .NET rather than quirky workarounds.

A Framework for Application Settings

It is a very common practice to store settings in config keys within the AppSettings section of an App.config file. These settings then need to be read and converted to the appropriate type. One must also take care to cater for situations where the key is not found, or the value is invalid. This article provides a structured approach to this practice. Feel free to review and use the accompanying source code.

Update 2015-02-28: I made a minor improvement to ReadAsync() suggested by Stephen Cleary, who I thank for the code review.

Update 2015-03-03: Some people have asked why we actually need AppSettings any more, given that there are alternatives such as .NET Settings or custom configuration sections. They are correct. However I still see a lot of projects using AppSettings, and this article is intended to provide a better way to deal with those AppSettings.

Update 2015-11-12: If you want to use this in your own code, check out my .NET Settings Framework project which is based on this article and provides NuGet packages that you can just drop into your projects.

The Problem

I’ve seen a lot of production code that reads values from config keys in App.config that looks something like this:

            // set a default, just in case the key is not found or the conversion fails

            int timeout = 3000;

            // retrieve the value for the desired key

            string timeoutStr = ConfigurationManager.AppSettings["timeoutInMilliseconds"];

            // check whether the key was actually found; if not, the default value is retained

            if (timeoutStr != null)
            {
                // attempt to convert to the desired type
                //   -> if it succeeds, the default value is replaced with the retrieved value
                //   -> if it fails, the default value is retained

                bool converted = int.TryParse(timeoutStr, out timeout);
            }

Aside from the bloat due to comments and braces (which were both necessary to make this example clear), you can see that we essentially have four lines of code just to read an integer setting from App.config.

What’s really bad is that there will essentially be four lines of code for every setting, all doing essentially the same thing for different settings. That isn’t very DRY.

A Basic Solution

One of my earlier attempts at solving this problem involved a utility class which took care of reading the settings and converting them to the appropriate type, using a specific method per type:

    public class ConfigKey
    {
        private string key;

        public ConfigKey(string key)
        {
            this.key = key;
        }

        public int GetAsInt(int defaultValue = 0)
        {
            int value = defaultValue;

            string valueStr = ConfigurationManager.AppSettings[this.key];

            if (valueStr != null)
            {
                bool converted = int.TryParse(valueStr, out value);
            }

            return value;
        }

        public bool GetAsBool(bool defaultValue = false)
        {
            bool value = defaultValue;

            string valueStr = ConfigurationManager.AppSettings[this.key];

            if (valueStr != null)
            {
                bool converted = bool.TryParse(valueStr, out value);
            }

            return value;
        }

        // ...
    }

This approach was pretty decent, as it made it very easy to read settings and specify optional default values:

            int timeout = new ConfigKey("timeoutInMilliseconds").GetAsInt(3000);
            bool enabled = new ConfigKey("enabled").GetAsBool(false);

The only problem with this class is that while removes the bloat and duplication from the actual logic, it is full of duplication itself: you need a method per type to perform the type-specific conversion.

A Generic Approach

The duplication in the ConfigKey class is solved by using a generic conversion method:

        public T Get<T>(T defaultValue = default(T)) where T : IConvertible
        {
            T value = defaultValue;

            string valueStr = ConfigurationManager.AppSettings[this.key];

            if (valueStr != null)
            {
                try
                {
                    value = (T)Convert.ChangeType(valueStr, typeof(T));
                }
                catch(Exception)
                {
                    return defaultValue;
                }
            }

            return value;
        }

The usage changes as follows:

            int timeout = new ConfigKey("timeoutInMilliseconds").Get<int>(3000);
            bool enabled = new ConfigKey("enabled").Get<bool>(false);

That’s good enough for reading settings from App.config.

Dependency injection

In order to unit test our ConfigKey class, it’s best if we abstract out the dependency on App.config. In particular, we want to separate the part that reads the settings (reader) from the part that does the conversion and returns the value (provider).

For this, we need two interfaces. First, IConfigKeyReader is responsible to read the value of a setting from a source (e.g. App.config):

    public interface IConfigKeyReader
    {
        string Read(string key);
    }

Secondly, IConfigKeyProvider does all the rest: given a key, it returns a value (by internally using the IConfigKeyReader, which is not immediately evident from the interface):

    public interface IConfigKeyProvider
    {
        T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible;
    }

The IConfigKeyReader implementation for reading from App.config is extremely simple:

    public class AppSettingReader : IConfigKeyReader
    {
        public string Read(string key)
        {
            return ConfigurationManager.AppSettings[key];
        }
    }

The IConfigKeyProvider for App.config settings is almost the same as the code we had in the previous section, with one important exception: it no longer depends directly on ConfigurationManager. Instead, it depends on the IConfigKeyReader which is injected in the constructor. This reader can be mocked in unit tests.

    public class ConfigKeyProvider: IConfigKeyProvider
    {
        private IConfigKeyReader reader;

        public ConfigKeyProvider(IConfigKeyReader reader)
        {
            this.reader = reader;
        }

        public T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible
        {
            T value = defaultValue;

            string valueStr = reader.Read(key);

            if (valueStr != null)
            {
                try
                {
                    value = (T)Convert.ChangeType(valueStr, typeof(T));
                }
                catch (Exception)
                {
                    return defaultValue;
                }
            }

            return value;
        }
    }

You’ll also notice that we can now use a single instance of this AppSettingProvider to retrieve all our settings, rather than create a different ConfigKey for each setting. This approach is pretty handy if you’re using an IoC container to inject utility classes into your class constructors.

At this point we can throw away our old ConfigKey class, and instead use the new classes as follows:

            var reader = new AppSettingReader();
            var provider = new ConfigKeyProvider(reader);

            int timeout = provider.Get<int>("timeoutInMilliseconds", 3000);
            bool enabled = provider.Get<bool>("enabled", false);

Unit tests

Thanks to the separation between reader and provider, it is now easy to unit test our provider code while mocking our reader code. The reader will be source-specific and depends on external factors (e.g. files or databases) so it doesn’t make sense to unit test that. But we can unit test our provider, which handles the conversion and default values, and which will be reused whatever the reader (in fact notice the names used in the code above: AppSettingReader is specific to App.config AppSettings, but ConfigKeyProvider is used for any config key).

In the example unit test below, I’m using Moq to create a mock IConfigKeyReader, and thus test that the provider code works as expected:

        [TestMethod]
        public void Get_IntAvailableWithDefault_ValueReturned()
        {
            // arrange

            var key = "timeoutInMilliseconds";

            var reader = new Mock<IConfigKeyReader>();
            reader.Setup(r => r.Read(key)).Returns("5000");

            var provider = new ConfigKeyProvider(reader.Object);

            // act

            var expected = 5000;
            var actual = provider.Get<int>(key, 3000);

            // assert

            Assert.AreEqual(expected, actual);
        }

For the sake of brevity I won’t include the other unit tests here, but you can find them in the source code accompanying this article.

Database settings

The separation between reader and provider that we achieved in the previous section means that we can reuse the provider code (responsible for conversion and default values) regardless of the source of the settings. This means that anyone can write, for example, a DbSettingReader class which implements IConfigKeyReader and retrieves settings from a database. Its implementation would depend on the database structure so there won’t be any single standard implementation.

However, there is one improvement to our framework that we can make to facilitate reading settings from external sources such as databases. In particular, nowadays it is quite easy to query a database asynchronously without having to block the application. So it makes sense to add support for async methods in our interfaces so that anyone writing a DbSettingReader can then provide an asynchronous implementation.

IConfigKeyReader now becomes:

    public interface IConfigKeyReader
    {
        string Read(string key);

        Task<string> ReadAsync(string key);
    }

We now need to update our AppSettingReader implementation accordingly. Since reading AppSettings from App.config isn’t asynchronous, we can use Task.FromResult() to help satisfy the contract:

    public class AppSettingReader : IConfigKeyReader
    {
        public string Read(string key)
        {
            return ConfigurationManager.AppSettings[key];
        }

        public Task<string> ReadAsync(string key)
        {
            var value = this.Read(key);
            return Task.FromResult(value);
        }
    }

The provider code also needs to be updated to support asynchrony. First the interface:

    public interface IConfigKeyProvider
    {
        T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible;

        Task<T> GetAsync<T>(string key, T defaultValue = default(T)) where T : IConvertible;
    }

The changes necessary to ConfigKeyProvider are a little more radical:

    public class ConfigKeyProvider : IConfigKeyProvider
    {
        private IConfigKeyReader reader;

        public ConfigKeyProvider(IConfigKeyReader reader)
        {
            this.reader = reader;
        }

        public T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible
        {
            string valueStr = reader.Read(key);

            return this.ConvertValue<T>(valueStr, defaultValue);
        }

        public async Task<T> GetAsync<T>(string key, T defaultValue = default(T)) where T : IConvertible
        {
            string valueStr = await reader.ReadAsync(key).ConfigureAwait(false);

            return this.ConvertValue<T>(valueStr, defaultValue);
        }

        private T ConvertValue<T>(string valueStr, T defaultValue)
        {
            if (valueStr != null)
            {
                try
                {
                    return (T)Convert.ChangeType(valueStr, typeof(T));
                }
                catch (Exception)
                {
                    return defaultValue;
                }
            }
            else
                return defaultValue;
        }
    }

I opted to move the conversion code to a method shared by the async and non-async methods, and then call separate reader code in them. I intentionally avoided having Get() call GetAsync().Result as it can result in deadlocks.

Technically the best approach would have been to drop the synchronous Get() method altogether and force the use of the asynchronous version. However, I realise there are times when people actually want to call the synchronous version, such as in Console applications or in constructors (although there are workarounds for both – see Async Console Programs and “Can constructors be async?“).

Conclusion and Source Code

This article has presented a simple framework that can be used to read application settings without having to bloat actual program logic. It supports reading AppSettings from an App.config file out of the box, and can easily be extended to support other sources (e.g. databases). It makes it easy to provide default values, works nicely with dependency injection, and can also be used asynchronously.

Check out the source code at the Gigi Labs Bitbucket repository. Feel free to use this code as you like, and let me know if you think it can be improved.