Tag Archives: AppSettings

Transforming Config Files and XML Documents

When you’re deploying an application to different servers representing different environments, manually updating configuration settings can get very messy. In .NET applications, one of the ways to keep configuration settings under control is by using configuration transforms.

These transforms are a set of rules that define how the App.config or Web.config is to change (e.g. add appsettings, replace connection strings, etc) for the particular environment.

Syntax

Typically, when you create a new web application in Visual Studio, you’ll get a little more than just a Web.config file. In fact, if you expand the Web.config node in Solution Explorer, you’ll find that there are another two files: Web.Debug.config and Web.Release.config.

config-transform-web-default

As shown in the screenshot above, the Web.Release.config has a similar structure to Web.config itself; however it defines rules that work on the Web.config to produce a different result. In this case, there is a transform that will remove the debug attribute from the compilation node. You can see the result of this if you right click on Web.Release.config and select Preview Transform:

config-transform-preview

One of the most common places where transforms come in handy is to change the value of appsettings when deploying. Let’s say, for example, that we have the following setting in our App.config or Web.config file:

  <appSettings>
    <add key="SomeFilePath" value="myfile.txt" />
  </appSettings>

In our transform file (which so far has been Web.Release.config), we could define the following transform:

  <appSettings>
    <add key="SomeFilePath" value="C:\data\productionfile.txt"
         xdt:Transform="Replace"
         xdt:Locator="Match(key)" />
  </appSettings>

This tells the transform engine to look for the matching key (in this case “SomeFilePath”), and replace it with what is specified in the transform file:

config-transform-preview2

Similarly, it is very common to replace database connection strings depending on the environment. In fact, the default Web.Debug.config and Web.Release.config (shown earlier) explain in comments how to do this.

Other than removing attributes and replacing values, transforms support various other operations which allow insertion and removal of nodes and attributes. See Web.config Transformation Syntax for Web Project Deployment Using Visual Studio (official documentation) for detail.

Despite the title of that article, transforms may also be used with App.config files (for non-web projects) and just about any arbitrary XML file (although not quite in the same way as shown above). For web applications, Visual Studio provides some support for creating and previewing transforms. For other application types and XML documents, it is necessary to resort to other tools.

Tools

One of the simplest ways to apply config transforms (in particular, the Release transform) is to Publish your web application. This will compile your web application in Release mode, apply config transforms, and put it in the destination you specify.

Unfortunately, this approach has several drawbacks:

  • It is only suitable for web applications (not e.g. console applications).
  • It can’t be used for arbitrary XML files.
  • It can’t be extended to various different environments.
  • Being embedded in Visual Studio, it can’t be automated or integrated with a continuous integration system.

In recent years, Microsoft released an XML Document Transformation (XDT) library that enabled a variety of tools to run these transforms. This is available as the Microsoft.Web.Xdt NuGet package, and the source code is available on CodePlex.

The Web.config Transformation Tester is a web application that is useful for testing transforms. It uses XDT internally (see its source code) and can transform web and application configs as well as arbitrary XML files.

Config Transformation Tool (ctt) or XDT Transformation Tool is a command-line utility launched in 2010 also based on Microsoft’s XDT library. Although its Codeplex site says it’s still Alpha, I’ve used it to transform Web.config, App.config and XML files.

Say we have this Web.config file:

<?xml version="1.0"?>
<configuration>
  <appsettings></appsettings>
  <connectionStrings>
    <add name="foo" connectionString="value"/>
  </connectionStrings>
  <system.web>
    <customErrors mode="Off"/>
  </system.web>
</configuration>

…and we have this transform file called Web.Debug.config:

<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <appsettings>
    <add key="EnvironmentName" value="Debug"
        xdt:Transform="Insert" />
  </appsettings>
  <connectionStrings>
    <add name="foo" connectionString="differentValue"
        xdt:Transform="Replace" xdt:Locator="Match(name)" />
  </connectionStrings>
</configuration>

Then we can run ctt as follows:

ctt s:Web.config t:Web.Debug.config d:Web.Out.config i

I’m using the shorthand parameters here:

  • s: marks the source file.
  • t: marks the transform file.
  • d: marks the destination file.
  • i causes the output to be indented, because by default it’s minified and unreadable.

The output is as follows:

<?xml version="1.0"?><configuration>
    <appsettings>
        <add key="EnvironmentName" value="Debug" />
    </appsettings>
    <connectionStrings>
        <add name="foo" connectionString="differentValue" />
    </connectionStrings>
    <system.web>
        <customErrors mode="Off" />
    </system.web>
</configuration>

This also works on XML files. In fact, I’ve taken Microsoft’s sample books.xml file and applied the following transform file (books.transform.xml):

<?xml version="1.0"?>
<catalog xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
   <book id="bk113" xdt:Transform="Insert">
      <author>David Kushner</author>
      <title>Masters of Doom</title>
      <genre>Non-Fiction</genre>
      <price>10.95</price>
      <publish_date>2003-01-01</publish_date>
      <description>How two guys created an empire and transformed pop culture</description>
   </book>
</catalog>

This command applies the transformation:

ctt s:books.xml t:books.transform.xml d:books.out.xml i

As you can see, the insertion transform actually works:

config-transform-ctt-xml

Additionally, as a command line tool, ctt is great to automate transforms as part of a deploy script (e.g. in a batch file).

Aside from ctt, XDT’s open sourcing announcement lists a couple of other tools that use Microsoft’s XDT library. One is a Web Config Transform Runner which I haven’t tried but which is presumably similar to ctt. The other is SlowCheetah, a Visual Studio Extension which allows transforms to be previewed and added. The author discontinued development on SlowCheetah (see also Github thread) out of frustration that it wouldn’t become a first class citizen in Visual Studio, which ironically is what happened because the preview shown at the beginning of this article was created with Visual Studio 2015 without SlowCheetah installed.

Finally, this article would not be complete without a mention of TeamCity. Automating transformation of configuration settings is only one step in what should be a complete continuous integration solution running automated builds. TeamCity is just that, and it has support for transforms.

A Framework for Application Settings

It is a very common practice to store settings in config keys within the AppSettings section of an App.config file. These settings then need to be read and converted to the appropriate type. One must also take care to cater for situations where the key is not found, or the value is invalid. This article provides a structured approach to this practice. Feel free to review and use the accompanying source code.

Update 2015-02-28: I made a minor improvement to ReadAsync() suggested by Stephen Cleary, who I thank for the code review.

Update 2015-03-03: Some people have asked why we actually need AppSettings any more, given that there are alternatives such as .NET Settings or custom configuration sections. They are correct. However I still see a lot of projects using AppSettings, and this article is intended to provide a better way to deal with those AppSettings.

Update 2015-11-12: If you want to use this in your own code, check out my .NET Settings Framework project which is based on this article and provides NuGet packages that you can just drop into your projects.

The Problem

I’ve seen a lot of production code that reads values from config keys in App.config that looks something like this:

            // set a default, just in case the key is not found or the conversion fails

            int timeout = 3000;

            // retrieve the value for the desired key

            string timeoutStr = ConfigurationManager.AppSettings["timeoutInMilliseconds"];

            // check whether the key was actually found; if not, the default value is retained

            if (timeoutStr != null)
            {
                // attempt to convert to the desired type
                //   -> if it succeeds, the default value is replaced with the retrieved value
                //   -> if it fails, the default value is retained

                bool converted = int.TryParse(timeoutStr, out timeout);
            }

Aside from the bloat due to comments and braces (which were both necessary to make this example clear), you can see that we essentially have four lines of code just to read an integer setting from App.config.

What’s really bad is that there will essentially be four lines of code for every setting, all doing essentially the same thing for different settings. That isn’t very DRY.

A Basic Solution

One of my earlier attempts at solving this problem involved a utility class which took care of reading the settings and converting them to the appropriate type, using a specific method per type:

    public class ConfigKey
    {
        private string key;

        public ConfigKey(string key)
        {
            this.key = key;
        }

        public int GetAsInt(int defaultValue = 0)
        {
            int value = defaultValue;

            string valueStr = ConfigurationManager.AppSettings[this.key];

            if (valueStr != null)
            {
                bool converted = int.TryParse(valueStr, out value);
            }

            return value;
        }

        public bool GetAsBool(bool defaultValue = false)
        {
            bool value = defaultValue;

            string valueStr = ConfigurationManager.AppSettings[this.key];

            if (valueStr != null)
            {
                bool converted = bool.TryParse(valueStr, out value);
            }

            return value;
        }

        // ...
    }

This approach was pretty decent, as it made it very easy to read settings and specify optional default values:

            int timeout = new ConfigKey("timeoutInMilliseconds").GetAsInt(3000);
            bool enabled = new ConfigKey("enabled").GetAsBool(false);

The only problem with this class is that while removes the bloat and duplication from the actual logic, it is full of duplication itself: you need a method per type to perform the type-specific conversion.

A Generic Approach

The duplication in the ConfigKey class is solved by using a generic conversion method:

        public T Get<T>(T defaultValue = default(T)) where T : IConvertible
        {
            T value = defaultValue;

            string valueStr = ConfigurationManager.AppSettings[this.key];

            if (valueStr != null)
            {
                try
                {
                    value = (T)Convert.ChangeType(valueStr, typeof(T));
                }
                catch(Exception)
                {
                    return defaultValue;
                }
            }

            return value;
        }

The usage changes as follows:

            int timeout = new ConfigKey("timeoutInMilliseconds").Get<int>(3000);
            bool enabled = new ConfigKey("enabled").Get<bool>(false);

That’s good enough for reading settings from App.config.

Dependency injection

In order to unit test our ConfigKey class, it’s best if we abstract out the dependency on App.config. In particular, we want to separate the part that reads the settings (reader) from the part that does the conversion and returns the value (provider).

For this, we need two interfaces. First, IConfigKeyReader is responsible to read the value of a setting from a source (e.g. App.config):

    public interface IConfigKeyReader
    {
        string Read(string key);
    }

Secondly, IConfigKeyProvider does all the rest: given a key, it returns a value (by internally using the IConfigKeyReader, which is not immediately evident from the interface):

    public interface IConfigKeyProvider
    {
        T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible;
    }

The IConfigKeyReader implementation for reading from App.config is extremely simple:

    public class AppSettingReader : IConfigKeyReader
    {
        public string Read(string key)
        {
            return ConfigurationManager.AppSettings[key];
        }
    }

The IConfigKeyProvider for App.config settings is almost the same as the code we had in the previous section, with one important exception: it no longer depends directly on ConfigurationManager. Instead, it depends on the IConfigKeyReader which is injected in the constructor. This reader can be mocked in unit tests.

    public class ConfigKeyProvider: IConfigKeyProvider
    {
        private IConfigKeyReader reader;

        public ConfigKeyProvider(IConfigKeyReader reader)
        {
            this.reader = reader;
        }

        public T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible
        {
            T value = defaultValue;

            string valueStr = reader.Read(key);

            if (valueStr != null)
            {
                try
                {
                    value = (T)Convert.ChangeType(valueStr, typeof(T));
                }
                catch (Exception)
                {
                    return defaultValue;
                }
            }

            return value;
        }
    }

You’ll also notice that we can now use a single instance of this AppSettingProvider to retrieve all our settings, rather than create a different ConfigKey for each setting. This approach is pretty handy if you’re using an IoC container to inject utility classes into your class constructors.

At this point we can throw away our old ConfigKey class, and instead use the new classes as follows:

            var reader = new AppSettingReader();
            var provider = new ConfigKeyProvider(reader);

            int timeout = provider.Get<int>("timeoutInMilliseconds", 3000);
            bool enabled = provider.Get<bool>("enabled", false);

Unit tests

Thanks to the separation between reader and provider, it is now easy to unit test our provider code while mocking our reader code. The reader will be source-specific and depends on external factors (e.g. files or databases) so it doesn’t make sense to unit test that. But we can unit test our provider, which handles the conversion and default values, and which will be reused whatever the reader (in fact notice the names used in the code above: AppSettingReader is specific to App.config AppSettings, but ConfigKeyProvider is used for any config key).

In the example unit test below, I’m using Moq to create a mock IConfigKeyReader, and thus test that the provider code works as expected:

        [TestMethod]
        public void Get_IntAvailableWithDefault_ValueReturned()
        {
            // arrange

            var key = "timeoutInMilliseconds";

            var reader = new Mock<IConfigKeyReader>();
            reader.Setup(r => r.Read(key)).Returns("5000");

            var provider = new ConfigKeyProvider(reader.Object);

            // act

            var expected = 5000;
            var actual = provider.Get<int>(key, 3000);

            // assert

            Assert.AreEqual(expected, actual);
        }

For the sake of brevity I won’t include the other unit tests here, but you can find them in the source code accompanying this article.

Database settings

The separation between reader and provider that we achieved in the previous section means that we can reuse the provider code (responsible for conversion and default values) regardless of the source of the settings. This means that anyone can write, for example, a DbSettingReader class which implements IConfigKeyReader and retrieves settings from a database. Its implementation would depend on the database structure so there won’t be any single standard implementation.

However, there is one improvement to our framework that we can make to facilitate reading settings from external sources such as databases. In particular, nowadays it is quite easy to query a database asynchronously without having to block the application. So it makes sense to add support for async methods in our interfaces so that anyone writing a DbSettingReader can then provide an asynchronous implementation.

IConfigKeyReader now becomes:

    public interface IConfigKeyReader
    {
        string Read(string key);

        Task<string> ReadAsync(string key);
    }

We now need to update our AppSettingReader implementation accordingly. Since reading AppSettings from App.config isn’t asynchronous, we can use Task.FromResult() to help satisfy the contract:

    public class AppSettingReader : IConfigKeyReader
    {
        public string Read(string key)
        {
            return ConfigurationManager.AppSettings[key];
        }

        public Task<string> ReadAsync(string key)
        {
            var value = this.Read(key);
            return Task.FromResult(value);
        }
    }

The provider code also needs to be updated to support asynchrony. First the interface:

    public interface IConfigKeyProvider
    {
        T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible;

        Task<T> GetAsync<T>(string key, T defaultValue = default(T)) where T : IConvertible;
    }

The changes necessary to ConfigKeyProvider are a little more radical:

    public class ConfigKeyProvider : IConfigKeyProvider
    {
        private IConfigKeyReader reader;

        public ConfigKeyProvider(IConfigKeyReader reader)
        {
            this.reader = reader;
        }

        public T Get<T>(string key, T defaultValue = default(T)) where T : IConvertible
        {
            string valueStr = reader.Read(key);

            return this.ConvertValue<T>(valueStr, defaultValue);
        }

        public async Task<T> GetAsync<T>(string key, T defaultValue = default(T)) where T : IConvertible
        {
            string valueStr = await reader.ReadAsync(key).ConfigureAwait(false);

            return this.ConvertValue<T>(valueStr, defaultValue);
        }

        private T ConvertValue<T>(string valueStr, T defaultValue)
        {
            if (valueStr != null)
            {
                try
                {
                    return (T)Convert.ChangeType(valueStr, typeof(T));
                }
                catch (Exception)
                {
                    return defaultValue;
                }
            }
            else
                return defaultValue;
        }
    }

I opted to move the conversion code to a method shared by the async and non-async methods, and then call separate reader code in them. I intentionally avoided having Get() call GetAsync().Result as it can result in deadlocks.

Technically the best approach would have been to drop the synchronous Get() method altogether and force the use of the asynchronous version. However, I realise there are times when people actually want to call the synchronous version, such as in Console applications or in constructors (although there are workarounds for both – see Async Console Programs and “Can constructors be async?“).

Conclusion and Source Code

This article has presented a simple framework that can be used to read application settings without having to bloat actual program logic. It supports reading AppSettings from an App.config file out of the box, and can easily be extended to support other sources (e.g. databases). It makes it easy to provide default values, works nicely with dependency injection, and can also be used asynchronously.

Check out the source code at the Gigi Labs Bitbucket repository. Feel free to use this code as you like, and let me know if you think it can be improved.