Tag Archives: C#

Software development

Monitoring the Size of a Folder

Leave a comment

In this article, we’ll develop a simple application that can check the total size taken by a folder (including the files directly inside it, and those in its subfolders), and monitor it periodically to send alerts if it exceeds a certain threshold. This can be useful, for example, to ensure that automatic backups aren’t taking too much space on disk. The source code is available at the Gigi Labs BitBucket repository.

The techniques we’ll use for this are nothing new. We’ll use recursive directory traversal to calculate the total size of the folder, and we’ll use a timer to check this periodically. To send alerts, we’d typically use SmtpClient to send email, but since you’d need an actual SMTP server to test this, we’ll just write something to the console instead.

We’ll start off by calculating the total size of a folder:

        static void Main(string[] args)
        {
            string directoryPath = ConfigurationManager.AppSettings["DirectoryPath"];

            var dir = new DirectoryInfo(directoryPath);
            var size = CalculateDirectorySize(dir);
            Console.WriteLine(size);

            Console.ReadKey(true);
        }

We’re reading the path to the directory to monitor from an application setting (remember to add a reference to System.Configuration). Then we pass the resulting DirectoryInfo object to a recursive CalculateDirectorySize() method, which we’ll define next:

        static long CalculateDirectorySize(DirectoryInfo dir)
        {
            long totalSize = 0L;

            foreach (var file in dir.EnumerateFiles())
                totalSize += file.Length;

            foreach (var subdir in dir.EnumerateDirectories())
                totalSize += CalculateDirectorySize(subdir);

            return totalSize;
        }

This recursive method simply adds up the sizes of the files it contains directly, then adds the total size after recursing into its subdirectories. We’re using the long data type because int will overflow if the folder contains several gigabytes of data.

Note: you might get an UnauthorizedAccessException if you run this in something like C:. It’s easiest to just run it on a folder within your user folder.

This is the result of running the above code against my desktop (which I seem to need to clean up):

dirsize-desktop

We can now refactor our Main() method to run a timer and periodically check the size of the folder:

        private static string directoryPath;

        static void Main(string[] args)
        {
            directoryPath = ConfigurationManager.AppSettings["DirectoryPath"];

            var timer = new Timer();
            timer.Interval = 5000;
            timer.Elapsed += Timer_Elapsed;
            timer.Enabled = true;
            timer.Start();

            Console.ReadKey(true);
        }

Every 5 seconds, the timer will run the following event handler:

        private static void Timer_Elapsed(object sender, ElapsedEventArgs e)
        {
            var dir = new DirectoryInfo(directoryPath);
            var size = CalculateDirectorySize(dir);
            Console.WriteLine(size);
        }

It is now very easy to extend this with a warning mechanism:

        private static string directoryPath;
        private static long threshold;

        static void Main(string[] args)
        {
            directoryPath = ConfigurationManager.AppSettings["DirectoryPath"];
            threshold = Convert.ToInt64(ConfigurationManager.AppSettings["Threshold"]);

            var timer = new Timer();
            timer.Interval = 5000;
            timer.Elapsed += Timer_Elapsed;
            timer.Enabled = true;
            timer.Start();

            Console.ReadKey(true);
        }

        private static void Timer_Elapsed(object sender, ElapsedEventArgs e)
        {
            var dir = new DirectoryInfo(directoryPath);
            var size = CalculateDirectorySize(dir);
            string warning = size > threshold ? " WARNING - THRESHOLD EXCEEDED" : null;
            Console.WriteLine("{0}{1}", size, warning);
        }

Here’s an example run of this application:

dirsize-warnings

Add to that some exception handling and console title code, and you get the source code available in the Gigi Labs BitBucket repository.

Software development

Understanding Recursion with Directory Traversal

1 Comment

This article was originally posted as “C#: Understanding Recursion with Directory Traversal” at Programmer’s Ranch on 27th September 2013. It has undergone minor modifications.

In this article, we’re going to talk about recursion. This technique is often considered an alternative to iteration (i.e. loops), and is useful for a wide variety of situations ranging from computing factorials to clearing empty areas in Minesweeper:

csrecursion-minesweeper

Since factorials are boring, and Minesweeper is a bit complex for this easy tutorial, we’re going to look at the filesystem in order to learn about recursion. For example, take a look at the folder for the solution I just created in SharpDevelop:

csrecursion-cmd-files-folders

See, the filesystem is actually a tree data structure. Each folder can contain other files and folders, which can in turn contain other files and folders, and so on. It isn’t easy to work with things like trees using loops, but with recursion it just comes natural. Let’s see how.

After creating a new console application, add the following at the top, which we need to interact with the filesystem:

using System.IO;

The first thing we want to do is get the current directory where the executable will be running. We do this by using Directory.GetCurrentDirectory(). Let’s try that out:

string currentDir = Directory.GetCurrentDirectory();
Console.WriteLine(currentDir);
Console.ReadKey(true);

…and here’s what we get:

csrecursion-current-dir

Now, we want to navigate up to the first CsRecursion folder, which is the solution folder. From there we’ll be able to list the contents of all the subfolder. To do this, we create an instance of DirectoryInfo:

var dir = new DirectoryInfo(currentDir);

This allows us to get to the parent folder:

dir = dir.Parent.Parent.Parent;

…and this is what we have so far:

csrecursion-parent

Right, now about listing the folder and subfolder contents. Let’s add a method to do that:

public static void ListContents(DirectoryInfo dir)
{

}

In this method, we first want to list all the files in that folder. We can do this using DirectoryInfo.GetFiles(), or else using the static Directory.GetFiles() method, which is easier and works directly with file paths (strings):

foreach (string file in Directory.GetFiles(dir.FullName))
    Console.WriteLine(file);

Okay, now all we need is to do the same thing for all subfolders. It turns out that our ListContents() method can actually call itself, and pass in each subdirectory as a parameter:

foreach (DirectoryInfo subdir in dir.GetDirectories())
    ListContents(subdir);

When a method calls itself, it’s called recursion. In this case we say we are recursing into subdirectories.

Let’s just change Main() so that it calls our ListContents() method:

        public static void Main(string[] args)
        {
            string currentDir = Directory.GetCurrentDirectory();
            var dir = new DirectoryInfo(currentDir);
            dir = dir.Parent.Parent.Parent;
            ListContents(dir);
           
            Console.ReadKey(true);
        }

…and voilà:

csrecursion-listing

As you can see, there’s a very small amount of code, and recursion is a perfect fit for this kind of thing, because the same method can work on folders at different levels of the filesystem.

There’s an important concept about recursion you need to be aware of, that might not be so evident in this example: the stopping condition. If a method calls itself and has no way to stop calling itself, you get a sort of infinite loop which actually ends in a stack overflow (in short, there’s a limit to the number of times a method can call another method). Therefore, a recursive function always needs a way to stop calling itself.

If you’re doing factorials, recursion stops when n=1. If you’re computing a Fibonacci sequence, n=0 and n=1 are the stopping conditions. In our case, recursion stops when a folder has no further subfolders. In Minesweeper, recursion stops when there are no more adjacent blank squares (you either hit an edge or a number).

Anyhow, as we have seen in this article, recursion is a great technique to use when your data has divergent paths (most notably when dealing with trees).

Software development

Scope Bound Resource Management in C#

6 Comments

This article explains how we can use scope to localize resource lifetime as well as other arbitrary user code effects within a method. The article starts with background from C++, because that’s where this technique comes from.

Update 5th November 2016: Some of the utility classes described here have been incorporated in my Dandago.Utilities NuGet package.

The RAII Pattern

Despite their power, pointers in C/C++ have been the cause of much outrage. Using them incorrectly typically leads to disastrous effects, from memory leaks to segmentation faults. While newer languages such as Java and C# have imposed damage control by taking control of the lifetime of objects allocated on the heap (via garbage collection), there are techniques even in C++ that make pointers a lot less messy to use.

In fact, the problem here is not even about pointers. Pointers belong to a family of resources, along with file handles, sockets, and many other things. These are typically allocated on the heap and must be released after use; otherwise bad things happen.

Scott Meyers’ excellent book Effective C++: 55 Specific Ways to Improve Your Programs and Designs has an entire section dedicated to safely working with resources. One of the first things he suggests in that section is to encapsulate a resource within an object. For example:

class MyResource
{
public:
    MyResource() // constructor
    {
        this->ptr = new int[1000]; // allocate memory on heap
    }

    ~MyResource() // destructor
    {
        delete[] this->ptr; // free memory
    }

private:
    int * ptr; // pointer encapsulated within class
};

This encapsulation is called Resource Acquisition Is Initialization (RAII), or Scope-Bound Resource Management. Apart from constructors, C++ classes can have destructors. These get called either explicitly from application code, or automatically when an object allocated on the stack goes out of scope. Let’s see how this works in practice:

int main(int argc, char ** argv)
{
    // create instance of MyResource on stack
    MyResource resource;

    return 0;
} // we went out of scope; destructor gets called

Just like declaring an int variable allocates it on the stack, the same thing happens with class types. When that object goes out of scope (i.e. reaches the next } brace), its destructor gets called. This is great because you can’t really forget to dispose of the encapsulated resource. If you return early, throw an exception, etc, the destructor will get called when control leaves the current execution block.

The IDisposable Pattern

In C#, objects allocated on the heap (via the new keyword) are typically killed by the garbage collector when it determines that they are no longer in use. While C# code typically doesn’t face the problems C++ code has with pointers, managing resources is no less important. Just like C++, C# has to deal with file handles, databases, sockets, unmanaged libraries and other stuff that must be disposed as carefully as they are initialized.

For this reason, C# provides two tools to essentially do the work of C++ destructors: the IDisposable pattern, and finalizers. Using these correctly is non-trivial and depends on the situation, but they are also pretty standard and well-documented. Check out this CodeProject article for an in-depth treatment of the topic.

For convenience, C# also provides an overloaded using keyword that works hand-in-hand with the IDisposable pattern:

            using (var fs = File.OpenWrite("file.txt"))
            using (var sw = new StreamWriter(fs))
            {
                sw.WriteLine("Hello!");
            } // automatically calls Dispose() for both

A using block, wrapping an object that implements IDisposable, will automatically call that object’s Dispose() method when it goes out of scope (and using blocks can be stacked, as above). This is essentially equivapent to:

            FileStream fs = null;
            StreamWriter sw = null;
            try
            {
                fs = File.OpenWrite("file.txt");
                sw = new StreamWriter(fs);

                sw.WriteLine("Hello!");
            }
            finally
            {
                sw.Dispose();
                fs.Dispose();
            }

Abusing the IDisposable Pattern

While IDisposable is meant to be used to safely dispose of resources, we can extend its usefulness to other scenarios that relate to a particular scope.

One example where I’ve used this idea before is to log the beginning and end of a method:

    public class ScopedLog : IDisposable
    {
        private string name;

        public ScopedLog(string name)
        {
            this.name = name;
            Console.WriteLine("Begin {0}", name);
        }

        public void Dispose()
        {
            Console.WriteLine("End {0}", this.name);
        }
    }

This pattern doesn’t really add anything, but gives you an elegant way to do scope-related stuff without that getting in the way of your application logic:

        static void Main(string[] args)
        {
            using (var log = new ScopedLog("Main"))
            {
                Console.WriteLine("Hello!");
            }

            Console.ReadLine();
        }

Here’s the output:

scope-logging

Another example is when you want to benchmark your code. Code can get really messy when you’re doing logging, benchmarking and other stuff in the same method. Instead, we can make a dedicated class:

    public class ScopedTimer : IDisposable
    {
        private string name;
        private DateTime startTime;

        public ScopedTimer(string name)
        {
            this.name = name;
            this.startTime = DateTime.Now;
        }

        public void Dispose()
        {
            var endTime = DateTime.Now;
            var elapsed = endTime - this.startTime;

            Console.WriteLine("{0} took {1}", this.name, elapsed);
        }
    }

…and then put it neatly in a using block:

        static void Main(string[] args)
        {
            using (var timer = new ScopedTimer("Main"))
            using (var log = new ScopedLog("Main"))
            {
                Console.WriteLine("Hello!");
            }

            Console.ReadLine();
        }

Here’s the output:

scope-benchmarking

A final example is changing the console colour. It’s very easy to save the old colour, set a new one, and then revert back to the old colour when going out of scope:

    public class ScopedConsoleColour : IDisposable
    {
        private ConsoleColor oldColour;

        public ScopedConsoleColour(ConsoleColor newColour)
        {
            this.oldColour = Console.ForegroundColor;

            Console.ForegroundColor = newColour;
        }

        public void Dispose()
        {
            Console.ForegroundColor = this.oldColour;
        }
    }

Note: you could use Console.ResetColor(), but then you can’t nest colour changes.

Here’s the updated Main() code for this example:

        static void Main(string[] args)
        {
            using (var timer = new ScopedTimer("Main"))
            using (var log = new ScopedLog("Main"))
            {
                Console.WriteLine("Hello!");

                using (var colour = new ScopedConsoleColour(ConsoleColor.Yellow))
                {
                    Console.WriteLine("Howdy?");
                }

                Console.WriteLine("Bye!");
            }

            Console.ReadLine();
        }

Here’s the output:

scope-colour

Other Applications in C++

The RAII pattern is awesome. Not only does it allow you to safely manage the lifetime of resources, but it enables a whole class of scope-based applications. Aside from the C# examples in the previous section, RAII enables things like smart pointers in C++ (e.g. unique_ptr) and scoped locks.

Software development

Visual Studio bug with long const strings

Leave a comment

It turns out that constants aren’t only problematic in ASP .NET 5.

If you have a pretty long const string with a null character (\0) somewhere towards the beginning, then you’ll be pretty surprised to get the following error when you try to compile:

Unexpected error writing debug information — ‘Error HRESULT E_FAIL has been returned from a call to a COM component.’

Say you have a 3,000-character string that looks something like this:

const string s = "aaaaaaaaaaaa\0aa...aaaaa";

Although this is perfectly valid, Visual Studio fails to compile it:

vsconststringbug

I created a Stack Overflow question and a Visual Studio bug report for this.

Comments on the Stack Overflow question confirmed this to be a bug in the PDB writer (Eric Lippert). It’s Visual Studio specific, and doesn’t happen when compiling from the command line (Jon Skeet). Apparently it happens if the null character is before the 2033rd index in the string (Tommy).

One workaround suggested by Tommy is to use the string literal notation, which actually works:

const string s = @"aaaaaaaaaaaa\0aa...aaaaa";

Another workaround is to simply drop the const qualifier.

Software development

XML Serialization in C#

Leave a comment

This article was originally posted here at Programmer’s Ranch on 6th November 2013.

Hi guys and gals! 🙂

In this article, we’re going to see how to do XML serialization, which is really just a fancy way of saying we want to load and save objects as files in XML format.

So you see, I’ve been a big fan of Warcraft 2 for a long time… probably about 15 years. In this game, you command different units (e.g. peasants, knights, etc), and each of them has different attributes such as Damage, Speed, etc. For instance, this is the mage:

csxmlser-war2-mage

Don’t mess with that guy – he can turn you into a sheep! And here’s the knight… not exactly a mild-mannered fellow, either:

csxmlser-war2-knight

If we represent those units and their attributes in XML, we might end up with something like this (I took out some of the extra crap that appears at the beginning when you actually save a file using XML serialization):

<UnitDatabase>
  <units>
    <Unit>
      <Name>Mage</Name>
      <Armor>0</Armor>
      <MinDamage>5</MinDamage>
      <MaxDamage>9</MaxDamage>
      <Range>2</Range>
      <Sight>9</Sight>
      <Speed>8</Speed>
    </Unit>
    <Unit>
      <Name>Knight</Name>
      <Armor>4</Armor>
      <MinDamage>2</MinDamage>
      <MaxDamage>12</MaxDamage>
      <Range>1</Range>
      <Sight>4</Sight>
      <Speed>13</Speed>
    </Unit>
  </units>
</UnitDatabase>

So, let’s see how we can actually read and write a file like this in C#. Create a new Console Application in your favourite IDE.

We first need to create a class to represent our units with their attributes. Create a class and call it Unit. For convenience, we can implement the attributes as auto-implemented properties as follows:

public String Name { get; set; }
public int Armor { get; set; }
public int MinDamage { get; set; }
public int MaxDamage { get; set; }
public int Range { get; set; }
public int Sight { get; set; }
public int Speed { get; set; }

This is just a quick alternative to declaring a member variable and a corresponding read-write property (available from .NET 3.0 onwards). For example, the Name property above is more or less equivalent to the following (just for demonstration – don’t actually add it to your code):

private String name;

public String Name
{
    get
    {
        return this.name;
    }
    set
    {
        this.name = value;
    }
}

Next, add a constructor to set the attributes, so we can easily create Unit instances from our main program code:

public Unit(String name, int armor, int minDamage, int maxDamage,
    int range, int sight, int speed)
{
    this.Name = name;
    this.Armor = armor;
    this.MinDamage = minDamage;
    this.MaxDamage = maxDamage;
    this.Range = range;
    this.Sight = sight;
    this.Speed = speed;
}

Now we need to create another class to hold an array of these units. Create a new class and call it UnitDatabase (admittedly a bit of a poor choice of a name, since it’s not actually a database, but anyway). Give it a Units property as follows:

public Unit[] Units { get; set; }

A constructor to assign this directly can also be pretty convenient. Add the following:

public UnitDatabase(Unit[] units)
{
    this.Units = units;
}

Now we can implement our loading and saving code in UnitDatabase itself. Start by adding the code to save the UnitDatabase to a file:

public void Save(String filename)
{
    XmlSerializer ser = new XmlSerializer(typeof(UnitDatabase));

    using (StreamWriter sw = File.CreateText(filename))
        ser.Serialize(sw, this);
}

You can see that we’re making use of the XmlSerializer class. The file is saved by using its Serialize() method, which takes a TextWriter and the object to serialize. The StreamWriter returned by File.CreateText() quite conveniently is a subclass of TextWriter, so we can pass it as the first parameter to Serialize(). The second parameter is this: the UnitDatabase itself.

To get this code to compile, you’ll have to add the following using statements at the top:

using System.Xml.Serialization;
using System.IO;

Loading an XML file as a UnitDatabase is just as easy. In this case we make the method static since it isn’t tied to any particular UnitDatabase instance:

public static UnitDatabase Load(String filename)
{
    XmlSerializer ser = new XmlSerializer(typeof(UnitDatabase));
    
    using (StreamReader sr = File.OpenText(filename))
        return ser.Deserialize(sr) as UnitDatabase;
}

You can see that we’re still using the XmlSerializer, but this time we use the Deserialize() method to read the file from disk and create a UnitDatabase from it. Deserialize() takes a TextReader, which again is a base class of the StreamReader that we get by calling File.OpenText(), so everything fits like magic. Deserialize() returns an Object, so as a last touch we cast this to a UnitDatabase using the as keyword. It’s just the same as writing it like this:

return (UnitDatabase) ser.Deserialize(sr);

That’s all we need! Now, let’s add some functionality to make it easy to write our units to the console output. All classes inherit from Object, and Object defines this ToString() method which we can use to return a string representation of our objects. This is very convenient in our case, so we can implement Unit‘s ToString() method as follows:

public override string ToString()
{
    StringBuilder sb = new StringBuilder();
    sb.AppendFormat("Name:      {0}", this.Name);
    sb.AppendLine();
    sb.AppendFormat("Armor:     {0}", this.Armor);
    sb.AppendLine();
    sb.AppendFormat("MinDamage: {0}", this.MinDamage);
    sb.AppendLine();
    sb.AppendFormat("MaxDamage: {0}", this.MaxDamage);
    sb.AppendLine();
    sb.AppendFormat("Range:     {0}", this.Range);
    sb.AppendLine();
    sb.AppendFormat("Sight:     {0}", this.Sight);
    sb.AppendLine();
    sb.AppendFormat("Speed:     {0}", this.Speed);
    sb.AppendLine();
    
    return sb.ToString();
}

Note the override keyword in the method’s signature. This means that we are replacing ToString()‘s default functionality (which usually just returns the name of the class) with our own, in this case showing the unit’s name and attributes.

Let’s do the same for UnitDatabase. In this case we return a concatenation of all the units’ string representations:

public override string ToString()
{
    StringBuilder sb = new StringBuilder();
    foreach (Unit unit in this.Units)
        sb.AppendLine(unit.ToString());
    return sb.ToString();
}

To compile this code, you’ll need to add the following line at the top of both files (because of the StringBuilder):

using System.Text;

Now all we have left to do is write code in Main() that actually uses these classes. We can start by creating our two units:

Unit mage = new Unit("Mage", 0, 5, 9, 2, 9, 8);
Unit knight = new Unit("Knight", 4, 2, 12, 1, 4, 13);

We can then combine these into an array using collection initializer syntax (see “C# Basics: Morse Code Converter Using Dictionaries” if you forgot what that is):

Unit[] units = new Unit[] { mage, knight };

Then, we create a UnitDatabase out of this array:

UnitDatabase db = new UnitDatabase(units);

…and finally save it to a file called units.xml:

db.Save("units.xml");

You can now press F5 to run the program and see that it works. If you’re using Visual Studio, you might have run into this error:

csxmlser-ctor-error

That’s because XML serialization needs classes to have an empty constructor. SharpDevelop creates one for you when you create a new class, but Visual Studio does not. So if you’re missing those, add them in. One for Unit:

public Unit()
{

}

…and one for UnitDatabase:

public UnitDatabase()
{

}

Good. Now press F5 to run the program, and then go to the project’s bin\Debug folder to check that the units.xml file has been created. When you open it, it should look like this:

<?xml version="1.0" encoding="utf-8"?>
<UnitDatabase xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <Units>
    <Unit>
      <Name>Mage</Name>
      <Armor>0</Armor>
      <MinDamage>5</MinDamage>
      <MaxDamage>9</MaxDamage>
      <Range>2</Range>
      <Sight>9</Sight>
      <Speed>8</Speed>
    </Unit>
    <Unit>
      <Name>Knight</Name>
      <Armor>4</Armor>
      <MinDamage>2</MinDamage>
      <MaxDamage>12</MaxDamage>
      <Range>1</Range>
      <Sight>4</Sight>
      <Speed>13</Speed>
    </Unit>
  </Units>
</UnitDatabase>

It’s got some more stuff in the first two lines than I showed you at the beginning, but that’s just added there to make it a valid XML document and you can just ignore it.

At the end of Main(), let us now add code to load the file and display the unit data:

UnitDatabase loadedDb = UnitDatabase.Load("units.xml");
Console.WriteLine(loadedDb.ToString());
Console.ReadLine();

Press F5 to see the result:

csxmlser-result

If you omit the ToString() as follows:

Console.WriteLine(loadedDb);

…then the program works all the same, because Console.WriteLine() uses the ToString() method of the objects it is meant to write.

Great! 🙂 In this article, we have seen how we can very easily save (serialize) objects as XML files, and load (deserialize) them back from XML files. To do this we need classes that match the XML structure, as well as the handy XmlSerializer class. Classes to be serialized must have a parameterless constructor. It is possible to do a lot more with XML serialization – there are several attributes that allow you to control the actual XML nodes and attributes that are written to the file.

We have also seen other aspects of C#, such as the ToString() method available in every object; how to override inherited methods; the as keyword which is an elegant alias for type casting; and auto-implemented properties.

Thanks for reading, and come visit Programmer’s Ranch Gigi Labs again in future! 🙂