Yesterday, we introduced grain persistence in Microsoft Orleans, using the volatile MemoryStorage provider to quickly and easily learn how to set up and work with storage providers.

Today, we will use the ADO .NET Storage Provider in order to save our grain state to a database. Currently, we can use this with either SQL Server or MySQL.

Note: MariaDB is a drop-in replacement for MySQL. Thus, the ADO .NET Storage Provider works with it too.

Note: While there is a partial script for PostgreSQL, it is not yet supported.

Update 9th June 2017: Some material from this article was contributed towards the official Grain Persistence documentation.

Example Scenario

Like in yesterday’s article, we’ll use a very simple setup based on the Orleans Dev/Test Host project. We’ll use the same classes and interface:

    public interface IPersonGrain : IGrainWithStringKey
    {
        Task SayHelloAsync();
    }

    public class PersonGrainState
    {
        public bool SaidHello { get; set; }
    }

    [StorageProvider(ProviderName = "OrleansStorage")]
    public class PersonGrain : Grain<PersonGrainState>, IPersonGrain
    {
        public async Task SayHelloAsync()
        {
            string primaryKey = this.GetPrimaryKeyString();

            bool saidHelloBefore = this.State.SaidHello;
            string saidHelloBeforeStr = saidHelloBefore ? " already" : null;

            Console.WriteLine($"{primaryKey}{saidHelloBeforeStr} said hello!");

            this.State.SaidHello = true;
            await this.WriteStateAsync();
        }
    }

If we give our PersonGrain the name of “Joe” and call SayHelloAsync(), then it will write “Joe said hello!” to the console. After that, it sets the SaidHello property in its grain state to true, and saves the updated state to the database via the configured storage provider. The next time SayHelloAsync() is called, it will find that the value of SaidHello is true, so the output will be “Joe already said hello!” instead.

In the main program, let’s add the following code (same as in yesterday’s article) to invoke SayHelloAsync() on a grain. This needs to go in the place of the TODO comment that was generated as part of the project template.

            var joe = GrainClient.GrainFactory.GetGrain<IPersonGrain>("Joe");
            joe.SayHelloAsync();

Now, let’s move on to configure our storage provider.

Setting up the ADO .NET Storage Provider

Install the following package:

Install-Package Microsoft.Orleans.OrleansSqlUtils

Navigate to the folder where the package was installed alongside the project, and you’ll find folders in there for different databases.

In there you’ll find scripts you’ll need to run in order to set up the Orleans database, for different database vendors. Each script contains a table for storage and several others for cluster membership and other internal Orleans operations. For the scope of this article, we’re only concerned with storage.

Note: while there is a script for PostgreSQL, at the time of writing this article, it does not support storage.

The steps to configure an ADO .NET Storage Provider are as follows:

1. Create the database specific for the database vendor you want.
2. Install a NuGet package specific for the database vendor you want.
3. Set up the storage provider in code or XML configuration. Set the AdoInvariant setting (optional only if you’re using SQL Server).

Using SQL Server for Grain Persistence

Setting Up the Database

When using SQL Server, you’ll want to use the SQLServer\CreateOrleansTables_SqlServer.sql script.

Open SQL Server Management Studio. Right click on Databases, and create a new one.

Call it whatever you like, e.g. OrleansStorage.

Open a query window based on your new database. Copy in the aforementioned script, and run it.

Installing the NuGet Package

Next, install the following package:

Install-Package System.Data.SqlClient

Configuring via Code

Finally, we need to configure the storage provider itself. We can do this in code by replacing the following line in OrleansHostWrapper.cs:

config.AddMemoryStorageProvider();

…with code that sets up an ADO .NET storage provider. First, we declare a connection string:

const string connStr = "Server=.\SQLEXPRESS;Database=OrleansStorage;Integrated Security=True";

Then, we can set up the ADO .NET storage provider using either a registration call that takes the storage provider’s type name as a string:

            var typeName = "Orleans.Storage.AdoNetStorageProvider";
            var properties = new Dictionary<string, string>()
            {
                ["DataConnectionString"] = connStr,
                ["AdoInvariant"] = "System.Data.SqlClient"
            };

            config.Globals.RegisterStorageProvider(typeName, "OrleansStorage", properties);

…or its generic equivalent:

config.Globals.RegisterStorageProvider<AdoNetStorageProvider>("OrleansStorage", properties);

Note that there is also a generic extension method we could use as a shortcut:

config.AddAdoNetStorageProvider("OrleansStorage", connStr);

However, I recommend against using this method. As per an issue I just opened, there is no way to set the AdoInvariant (which is optional for SQL Server but necessary when working with other database vendors), and the default serialization format is XML (whereas the default is usually binary). Thus, if you switch between this method and the earlier ones while relying on defaults, you will get deserialization errors.

Configuring via XML

On the other hand, to configure the storage provider using an XML configuration file (which is generally a better approach), first we have to add a file named OrleansConfiguration.xml and set it to copy to the output directory.

Add the following to the file:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <StorageProviders>
      <Provider Type="Orleans.Storage.AdoNetStorageProvider"
                Name="OrleansStorage"
                AdoInvariant="System.Data.SqlClient"
                DataConnectionString="Server=.\SQLEXPRESS;Database=OrleansStorage;Integrated Security=True"/>
    </StorageProviders>
    <SeedNode Address="localhost" Port="22222" />
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="22222" />
    <ProxyingGateway Address="localhost" Port="40000" />
  </Defaults>
</OrleansConfiguration>

Then, replace the following code in OrleansHostWrapper.cs:

            var config = ClusterConfiguration.LocalhostPrimarySilo();
            config.AddMemoryStorageProvider();
            siloHost = new SiloHost(siloName, config);

…with this:

            siloHost = new SiloHost(siloName);
            siloHost.ConfigFileName = "OrleansConfiguration.xml";

Testing It Out

Once you have configured everything, run the program:

It says “Joe said hello!” Remember that the state should have been updated at the end of the method that writes that message. Let’s verify it by closing the program, and running it again:

This time, it says “Joe already said hello!” That means that the state was correctly read from the database.

Using MySQL for Grain Persistence

Note: this also works for MariaDB, which is a drop-in replacement for MySQL.

Setting Up the Database

Deep beneath the folder where the Microsoft.Orleans.SqlUtils package gets installed, you’ll find a MySql\CreateOrleansTables_MySql.sql script. We’ll use this for MySQL or MariaDB.

Use your favourite administrative tool to create a new database, and give it a name (e.g. “orleansstorage”). Paste the script in a query window, and run it against the new database.

Installing the NuGet Package

Install the following package:

Install-Package MySql.Data

Configuring via Code

We can replace the AddMemoryStorageProvider() call in OrleansHostWrapper.cs with code to set up our ADO .NET provider. First, let’s put our connection string in a variable to keep the rest of the setup code concise:

const string connStr = "Server=localhost;Database=orleansstorage;uid=xxx;pwd=xxx";

We can now configure the storage provider, using either a named type:

            var typeName = "Orleans.Storage.AdoNetStorageProvider";
            var properties = new Dictionary<string, string>()
            {
                ["DataConnectionString"] = connStr,
                ["AdoInvariant"] = "MySql.Data.MySqlClient"
            };

            config.Globals.RegisterStorageProvider(typeName, "OrleansStorage", properties);

…or a generic method based on the type itself:

config.Globals.RegisterStorageProvider<AdoNetStorageProvider>("OrleansStorage", properties);

As you can see, this is almost identical to the configuration for SQL Server. We’re using the exact same ADO .NET storage provider, and changing just the connection string and the AdoInvariant which identifies the underlying database vendor.

Configuring via XML

Instead of hardcoding our configuration, we have the option to read it from an XML file. Let’s create a file named OrleansConfiguration.xml, set it to copy to the output directory, and give it the following content:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <StorageProviders>
      <Provider Type="Orleans.Storage.AdoNetStorageProvider"
                Name="OrleansStorage"
                AdoInvariant="MySql.Data.MySqlClient"
                DataConnectionString="Server=localhost;Database=orleansstorage;Uid=xxx;Pwd=xxx"/>
    </StorageProviders>
    <SeedNode Address="localhost" Port="22222" />
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="22222" />
    <ProxyingGateway Address="localhost" Port="40000" />
  </Defaults>
</OrleansConfiguration>

Then, as we did before in the SQL Server section, replace the following code in OrleansHostWrapper.cs:

            var config = ClusterConfiguration.LocalhostPrimarySilo();
            config.AddMemoryStorageProvider();
            siloHost = new SiloHost(siloName, config);

…with this:

            siloHost = new SiloHost(siloName);
            siloHost.ConfigFileName = "OrleansConfiguration.xml";

This allows us to bypass the default localhost silo configuration and read the configuration from our file instead.

A Word On Formats

ADO .NET Storage providers can save data in one of three formats: JSON, XML, or a compact binary format.

The default format is binary, and while it is compact, it means the data is opaque and you can’t really work with it directly. This could be a problem if you need to patch, migrate, or troubleshoot the data. XML is bloated, so I generally recommend using JSON. It’s lightweight (although not as much as the binary format) and is easy to work with.

You can switch between them by setting the appropriate property in either code or XML configuration:

• UseJsonFormat="true" uses JSON.
• UseXmlFormat="true" uses XML.
• UseBinaryFormat="true" uses binary, which is the default setting.

Ideally you should set this only the first time. If you change this setting when you already have data, you’ll have a mess of formats and will not be able to read old data.

Summary

Install the base package for the ADO .NET Storage Provider:

Install-Package Microsoft.Orleans.OrleansSqlUtils

Then set up as follows depending on the database vendor:

	SQL Server	MySQL
Script	SQLServer\CreateOrleansTables_SqlServer.sql	MySql\CreateOrleansTables_MySql.sql
NuGet Package	System.Data.SqlClient	MySql.Data
AdoInvariant	System.Data.SqlClient	MySql.Data.MySqlClient

You can set the following propeties:

Name	Type	Description
Name	String	Arbitrary name that persistent grains will use to refer to this storage provider
Type	String	Set to Orleans.Storage.AdoNetStorageProvider
AdoInvariant	String	Identifies the database vendor (see above table for values; default is System.Data.SqlClient)
DataConnectionString	String	Vendor-specific database connection string (required)
UseJsonFormat	Boolean	Use JSON format
UseXmlFormat	Boolean	Use XML format
UseBinaryFormat	Boolean	Use compact binary format (default)

When configuring via code, prefer the generic registration call, as it avoids magic strings and potential runtime errors by using a type provided at compile time. Avoid the shortcut extension method since it does not let you configure the AdoInvariant and has a different default format from the other API methods.

In this article, we’re going to take a look how the state of virtual grains can be saved to and loaded from a data store. Since there are various storage providers available for Microsoft Orleans, we will use the in-memory storage provider to introduce the concepts in a simple manner. This is not a real storage provider, and its state is lost if the silo goes down, but it gives us an easy way to learn to set up and interact with storage providers.

The source code for this article is available in the Gigi Labs BitBucket repository. Look for the OrleansMemoryStorage folder.

Update 9th June 2017: Some material from this article was contributed towards the official Grain Persistence documentation.

A Simple Sample

Before we can see how grain persistence works, we need a minimal example. Let’s create a new Orleans Dev/Test Host project using the project templates from the Microsoft Orleans Tools for Visual Studio extension.

Let’s add a new interface that our grain will implement:

    public interface IPersonGrain : IGrainWithStringKey
    {
        Task SayHelloAsync();
    }

Next, we’ll implement a really simple grain:

    public class PersonGrain : Grain, IPersonGrain
    {
        public Task SayHelloAsync()
        {
            string primaryKey = this.GetPrimaryKeyString();

            Console.WriteLine($"{primaryKey} said hello!");

            return Task.CompletedTask;
        }
    }

Note: if you’re using a .NET Framework version prior to 4.6, use TaskDone.Done instead of Task.CompletedTask.

All we need to do now is actually use our grain. Locate the following comment in the template code generated in Main():

            // TODO: once the previous call returns, the silo is up and running.
            //       This is the place your custom logic, for example calling client logic
            //       or initializing an HTTP front end for accepting incoming requests.

…and replace it with the following:

            var joe = GrainClient.GrainFactory.GetGrain<IPersonGrain>("Joe");
            joe.SayHelloAsync();

We can now run the program to ensure that the grain’s SayHelloAsync() is actually called:

Adding State

While it is perfectly valid to have grain state in member variables within the grain itself, if we want to persist that state, we need something a little different.

The first thing we need to do is create a class that holds the state:

    public class PersonGrainState
    {
        public bool SaidHello { get; set; }
    }

Then, we need to change the definition of our grain class to inherit not from Grain, but from Grain<T>, where T is the grain state class:

    public class PersonGrain : Grain<PersonGrainState>, IPersonGrain

We can then change the code within our grain to save its state and do something a little more interesting:

    public class PersonGrain : Grain<PersonGrainState>, IPersonGrain
    {
        public async Task SayHelloAsync()
        {
            string primaryKey = this.GetPrimaryKeyString();

            bool saidHelloBefore = this.State.SaidHello;
            string saidHelloBeforeStr = saidHelloBefore ? " already" : null;

            Console.WriteLine($"{primaryKey}{saidHelloBeforeStr} said hello!");

            this.State.SaidHello = true;
            await this.WriteStateAsync();
        }
    }

All we’re doing here is that if the SaidHello property was set, then we write “Joe already said hello!” instead of “Joe said hello!”. However, note the last two lines. We’re setting the value of SaidHello to true, and then calling WriteStateAsync().

Orleans provides a very simple API for dealing with persistent state, as you can see above. You can access your grain’s state object using the State property. The WriteStateAsync(), ReadStateAsync() and ClearStateAsync() methods allow you to save, load, and clear the grain’s state from the data store respectively.

Note: we haven’t yet set up our data store, but we’ll do that shortly.

The logic for saving state is entirely up to you. You can save state with every message you receive (i.e. with every method call), or periodically, or in whatever manner makes most sense for your application. For instance, in our example above, we are saving state every time the SayHello() method is called. However, it would save unnecessary writes to the data store to do a simple check on the value of SaidHello, and write to the data store only if the value was previously false.

State is loaded automatically from the data store when the grain is activated, so you don’t actually need to call ReadStateAsync() unless it makes logical sense to do so, e.g. if your grain has ended up in an inconsistent state due to errors and you need to reload its original state from the data store.

Configuring a Persistence Store

Let’s run the application as it is now:

It is failing because we haven’t configured our storage provider yet. A grain having persistent state will by default look for a storage provider configured with the name “Default”. We can use an arbitrarily named storage provider by using an attribute:

    [StorageProvider(ProviderName = "OrleansStorage")]
    public class Person : Grain<PersonState>, IPerson

We now have to configure that OrleansStorage data store so that Orleans will know where to find it and how to interact with it. There are various different storage providers, as you can see from the Grain Persistence documentation. All of these can be configured either in code or using XML configuration.

In this example, we’ll use the MemoryStorage provider. This is not a real persistent data store, in the sense that it holds data in memory which is lost when the silo stops. While this does not allow for very interesting storage-and-retrieval scenarios, it allows us to experiment with storage providers without the hassle of setting up a real database. We’ll see how to set up proper storage providers with an underlying database in an upcoming article.

Configuring MemoryStorage in Code

You should have an OrleansHostWrapper class that was automatically created as part of the project template. In the Init() method, we can add the configuration code we need:

        private void Init()
        {
            siloHost.LoadOrleansConfig();

            siloHost.Config.AddMemoryStorageProvider("OrleansStorage");
        }

If you run it now, there will be no error:

The AddMemoryStorageProvider() is just a shortcut method. It takes a name and an optional number of grains used to store the state, which we can leave well enough alone with its default of 10:

The real way to set up any configuration provider is to use RegisterStorageProvider(), and pass in the type of the provider (either as a typename string or as a generic parameter), along with the storage provider name (in our case, “OrleansStorage”) and any additional parameters (in the case of MemoryStorage, we can set that NumStorageGrains here).

Using generics, it looks like this:

        private void Init()
        {
            siloHost.LoadOrleansConfig();

            siloHost.Config.Globals.RegisterStorageProvider<MemoryStorage>("OrleansStorage");
        }

On the other hand, this is how you’d do it if you specified the type name as a string:

        private void Init()
        {
            siloHost.LoadOrleansConfig();

            siloHost.Config.Globals.RegisterStorageProvider("Orleans.Storage.MemoryStorage",
                "OrleansStorage");
        }

Configuring MemoryStorage in XML

Instead of hardcoding your configuration, you can set it up in an OrleansConfiguration.xml file that your silo will then pick up.

Let’s add an OrleansConfiguration.xml file to the project. We also need to set the file’s Copy to Output Directory property to either Copy always or Copy if newer. Then, we can add the following to it:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <StorageProviders>
      <Provider Type="Orleans.Storage.MemoryStorage"
                Name="OrleansStorage"
                NumStorageGrains="10" />
    </StorageProviders>
    <SeedNode Address="localhost" Port="22222" />
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="22222" />
    <ProxyingGateway Address="localhost" Port="40000" />
  </Defaults>
</OrleansConfiguration>

You can see how we’re configuring pretty much the same things that we did earlier in code. Any optional parameters, such as NumStorageGrains in this case, can also be added.

The StorageProviders section is all we need to set up a storage provider, but you’ll notice we have a lot more configuration in the file. The Dev/Test Host project template uses LocalhostPrimarySilo() configuration, which is good to set something up quickly, but it’s a bit of a pain to get to work with configuration files. Thus, we’ll remove it and replace its default configuration with that in the file. With a proper Orleans project setup, you won’t need to do the following steps.

Locate the following code in OrleansHostWrapper.cs:

            var config = ClusterConfiguration.LocalhostPrimarySilo();
            config.AddMemoryStorageProvider();
            siloHost = new SiloHost(siloName, config);

Replace it with the following:

            siloHost = new SiloHost(siloName);
            siloHost.ConfigFileName = "OrleansConfiguration.xml";

The Init() method should be left as it originally was:

        private void Init()
        {
            siloHost.LoadOrleansConfig();
        }

You can now run the program. It should work without problems, using the configuration from the file to set up the storage provider.

Summary

In order to persist a grain’s state, it is necessary to:

1. Create a class that holds the grain’s state.
2. Have the grain inherit from Grain<T>, where T is the grain state class.
3. Access the grain’s state object using the State property.
4. Use WriteStateAsync(), ReadStateAsync() and ClearStateAsync() to write, read or clear the grain’s state in the data store respectively.
5. A grain’s state is automatically loaded from the data store when it is activated.
6. A grain’s storage provider name is set using the StorageProvider attribute, in absence of which it uses one named “Default”.
7. There are various storage providers available that work with different data stores. We have so far used only MemoryStorage, which is a volatile implementation to be used only for testing.
8. Storage providers may be configured either in code or using an XML configuration file.