Category Archives: Software development

Your First Microsoft Orleans Cluster

If you’re planning to use Microsoft Orleans in production, you need to look beyond the lonely silos that we’ve built in the articles thus far. Orleans is designed to work in a cluster, such that a large number of grains can be distributed among multiple silos. In case of silo failure, its grains are reactivated at other silos that are still alive.

In this article, we’ll create a simple silo and run multiple interconnected instances of it in order to set up a cluster.

For the scope of this article, we’ll use the default cluster membership provider: MembershipTableGrain. This is not intended to be used in production, but will allow us to focus on getting a simple cluster up and running. Setting up different cluster membership providers is non-trivial and requires separate articles for each.

Note: this article is based on Orleans 1.4.2 using .NET Framework 4.6.2.

Note: the source code for this article is in the OrleansFirstCluster folder within the Gigi Labs BitBucket repository.

Setting Up An Example

To set up a cluster, the Dev/Test Host project template we’ve been using so far is no longer suitable. Instead, we have to set up the full project structure. This is covered by the latter part of “Getting Started with Microsoft Orleans” and there is no point in repeating it here.

Don’t write any code yet though. I recently learned that all that AppDomain stuff is not necessary unless you’re planning to run Silo and Client in the same application, so we’ll go for a cleaner approach.

We’ll also install the Orleans Dashboard (see: “A Dashboard for Microsoft Orleans“) in the Silo project. This will give us an idea how grains are spread across the cluster later.

Install-Package OrleansDashboard

Hence, when setting up the Silo configuration, remember to include the configuration for the Dashboard:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <SeedNode Address="localhost" Port="11111" />
    <BootstrapProviders>
      <Provider Type="OrleansDashboard.Dashboard" Name="Dashboard" />
    </BootstrapProviders>
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="11111" />
    <ProxyingGateway Address="localhost" Port="30000" />
  </Defaults>
</OrleansConfiguration>

We can now start adding some code. First, we need a grain in our Grains project. What the grain actually does doesn’t matter. We just want to create a large number of grains to see them spread out over the cluster.

    public class UselessGrain : Grain, IUselessGrain
    {
        public Task DoNothingAsync()
        {
            return Task.CompletedTask;
        }
    }

Note: if you’re using a .NET Framework version prior to 4.6, then you’ll need to use TaskDone.Done instead of Task.CompletedTask.

The corresponding interface goes in the Interfaces project:

    public interface IUselessGrain : IGrainWithIntegerKey
    {
        Task DoNothingAsync();
    }

Doing away with all the AppDomain junk, the following code should be enough for a simple Silo:

        static void Main(string[] args)
        {
            Console.Title = "Silo";

            try
            {
                using (var siloHost = new SiloHost("Silo"))
                {
                    siloHost.LoadOrleansConfig();
                    siloHost.InitializeOrleansSilo();
                    var startedOk = siloHost.StartOrleansSilo(catchExceptions: false);
                    Console.WriteLine("Silo started successfully!");

                    Console.WriteLine("Press ENTER to exit...");
                    Console.ReadLine();
                    siloHost.ShutdownOrleansSilo();
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }

Apart from the AppDomain logic, another thing we’re doing differently from usual here is that we’re calling StartOrleansSilo() with catchExceptions set to false. In case the silo fails to initialise, this gives us the ability to inspect the details of the failure within the exception, rather than have Orleans silently swallow it and simply return false.

On the client side, we can use an adaptation of the client code from “Getting Started with Microsoft Orleans“:

        static void Main(string[] args)
        {
            Console.Title = "Client";

            var random = new Random();
            var config = ClientConfiguration.LoadFromFile("ClientConfiguration.xml");

            while (true)
            {
                try
                {
                    GrainClient.Initialize(config);
                    Console.WriteLine("Connected to silo!");

                    while (true)
                    {
                        var grainId = random.Next();
                        var grain = GrainClient.GrainFactory.GetGrain<IUselessGrain>(grainId);
                        grain.DoNothingAsync();
                    }
                }
                catch (SiloUnavailableException)
                {
                    Console.WriteLine("Silo not available! Retrying in 3 seconds.");
                    Thread.Sleep(3000);
                }
            }
        }

In the inner infinite-while-loop we’re taking random grain IDs and bombarding them with messages. The idea is to create a lot of grain instances that we can visualise. Since this is very heavy, you’ll see Orleans giving warnings, and high latencies from the Dashboard at times.

Running a 3-Node Cluster

We will now run 3 instances of the same silo. Each instance must have different ports configured. This is the configuration for the first silo, which we already set up earlier:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <SeedNode Address="localhost" Port="11111" />
    <BootstrapProviders>
      <Provider Type="OrleansDashboard.Dashboard" Name="Dashboard" />
    </BootstrapProviders>
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="11111" />
    <ProxyingGateway Address="localhost" Port="30000" />
  </Defaults>
</OrleansConfiguration>

A silo needs 2 ports: one to communicate with other silos (the Networking endpoint) and one for clients to connect to it (the ProxyingGateway endpoint). The client can connect to any node on the cluster.

In production scenarios, Orleans silos are all equal, and there is no concept of a primary and secondary silo. However, when you use the default MembershipTableGrain cluster membership, then all information regarding the silos on the cluster is stored within a grain in one of the silos. As a result, the silo containing the MembershipTableGrain is denoted as the Primary Silo. It must be started before the others, and the entire cluster is messed up if it goes down. Naturally, this is not good, and you should look into other cluster membership providers.

In such a setup, the SeedNode configuration specified in all silos must be the endpoint of the Primary silo. Let’s see what the configuration for our second silo instance looks like:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <SeedNode Address="localhost" Port="11111" />
    <BootstrapProviders>
      <Provider Type="OrleansDashboard.Dashboard" Name="Dashboard" Port="8081" />
    </BootstrapProviders>
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="11112" />
    <ProxyingGateway Address="localhost" Port="30001" />
  </Defaults>
</OrleansConfiguration>

Aside from changing the Networking and ProxyingGateway ports, we are also using a different port for the Dashboard (default is 8080). Each silo has its own Dashboard (although they all show the same information), and they cannot all run from the same port.

Similarly, the configuration for our third silo instance is just a matter of changing ports:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <SeedNode Address="localhost" Port="11111" />
    <BootstrapProviders>
      <Provider Type="OrleansDashboard.Dashboard" Name="Dashboard" Port="8082" />
    </BootstrapProviders>
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="11113" />
    <ProxyingGateway Address="localhost" Port="30002" />
  </Defaults>
</OrleansConfiguration>

We can then start the 3 silo instances and the client:

On my system, the load is just too much and Orleans just dies after around 64k activations. So let’s add a little delay in the random message loop to give Orleans some room to breathe:

                    while (true)
                    {
                        var grainId = random.Next();
                        var grain = GrainClient.GrainFactory.GetGrain<IUselessGrain>(grainId);
                        Thread.Sleep(50);
                        grain.DoNothingAsync();
                    }

After running it again, what I see is that grains are allocated mainly to the primary silo initially, but they are distributed more evenly across the other silos after around 1,000 activations:

I am not sure why they are not evenly distributed from the start. My guess is that either it is more efficient to have them all in one place if the number of activations is small, or the silos need time to coordinate between themselves before this happens (which would explain why, without a delay, all activations are allocated on the primary node).

Single Point of Failure

Close the primary silo.

Since the Primary silo contains the MembershipTableGrain, all information about the cluster dies with it. The remaining silos and clients will not recover automatically even if the Primary silo is brought up again. They in turn will have to be restarted. This is because, as we saw earlier, Secondary silos must start after the primary one. When the Primary silo is brought back, it effectively starts a fresh cluster and does not know about any other silos until they join.

Conclusion

We have seen how to get a very basic Orleans cluster working with multiple silos sharing the burden of holding the grains. However, this is hardly an ideal setup because (a) cluster membership information is held in memory and represents a single point of failure, and (b) the fact that I ran all silos on the same machine made them subject to the same physical resource constraints as if I were running a standalone silo.

For better results, run different silos on different machines, and use a decent cluster membership provider. Orleans supports the following:

  • MembershipTableGrain (not realiable, use for testing only)
  • SQL Server
  • Azure Table Storage
  • Apache ZooKeeper
  • Consul
  • DynamoDB

Update 20th June 2017: I am told that Azure Service Fabric should also be supported. As for database implementations of cluster membership, these are not limited to just SQL Server. You may use any supported ADO .NET provider, which at the moment includes SQL Server, MySQL/MariaDB, or PostgreSQL. To clarify: while the PostgreSQL storage provider for grain persistence is not yet available, its use as a cluster membership provider is supported.

Saving Screenshots in SDL2

Saving screenshots is a simple and common feature in many games. It allows us to capture the image of the game being rendered on the screen at any given time.

While this might sound easy, let us remember that the image rendered to the screen might have a lot of different overlaid surfaces, meaning that it would be a pain to recompose that image on the side of the CPU in the same way that we’re composing the image to the texture that eventually ends up in video memory.

Fortunately, this Stack Overflow answer provides a simple solution. SDL2 provides the SDL_RenderReadPixels() function, which can be used to read pixel data back from video memory. This is generally discouraged, because it is very costly to do such a thing, but taking screenshots is a one-off operation where it makes perfect sense.

The following code shows how screenshot capture was implemented in Ultima 1 Revenge. It is only slightly different from the code in the aforementioned Stack Overflow answer:

void SosariaInputController::SaveScreenshot()
{
	const Uint32 format = SDL_PIXELFORMAT_ARGB8888;
	const int width = 640;
	const int height = 400;
	auto renderer = sdl2Core->GetRenderer();

	SDL_Surface *surface = SDL_CreateRGBSurfaceWithFormat(0, width, height, 32, format);
	SDL_RenderReadPixels(renderer, NULL, format, surface->pixels, surface->pitch);
	SDL_SaveBMP(surface, "screenshot.bmp");
	SDL_FreeSurface(surface);
}

While I am not showing where the sdl2Core object is coming from, just assume that it stores an instance of SDL_Renderer. With that available, all that is needed is to create a surface using one of the functions that SDL2 provides, call SDL_RenderReadPixels() to transfer the pixels from video memory to the surface, and then actually do something with the surface (in this case, we are saving it to a bitmap file). If you wanted to read back only a portion of the screen, you would pass in an SDL_Rect instead of NULL as the second parameter to SDL_RenderReadPixels().

This simple code is used to save screenshots in Ultima 1 Revenge, such as the one shown earlier.

The Sorry State of The Web: 3 Group Special

It’s been around 15 years since I first came across Web Pages That Suck. Coming from a time when flashy Geocities-style websites were the order of the day, it was a web nitpicker’s paradise. This is where the term Mystery Meat Navigation (which I have written about in the past) was actually invented.

The very premise behind Web Pages That Suck, “learn good web design by looking at bad web design”, is something that has fascinated me back then, and still does to this day (in fact, it is one of the main reasons behind the Sorry State of the Web series).

Today, we will look at a family of related websites (belonging to a single group of companies) which I’m sure would qualify as first class citizens of Web Pages That Suck.

Enter MyKrypto

I first heard about MyKrypto on the radio. The ad described Bitcoin as a currency just like any other – and said that you could produce it! An old version of their website, which I obtained via the Google Web Cache, is along the same lines of the radio ad:

“Malta has the Euro, UK has the Pound and USA has the Dollar, the Internet has the Bitcoin. Bitcoin is digital and produced by computers..start producing money today!!”

While it’s true you can produce Bitcoins, this feels a lot like a scam in that it’s urging people to print their own money (in a way) without telling them about the risk or the difficulty involved in actually mining Bitcoins. In fact, the website also used to say that Bitcoin mining is a secure investment:

Whether Bitcoin mining is really a secure investment is debatable (although one can get an idea by looking at market crashes that have occurred in the past). In any case, while I’m not a lawyer, I don’t believe a company can legally give investment advice unless it is an authorised financial institution, especially without evaluating the risk portfolio of potential investors.

MyKrypto Home Page

The above selling points were removed, and the site transitioned into a different realm of madness. The site’s homepage had this image with Comic Sans text:

This was eventually replaced by the text image we see today:

Although the text changed, the link behind it remained the same. It’s basically a Google link (notice the URL) that takes you to this Satasoshi graphic on Deviantart:

So, in case it’s not clear, let’s summarise the fails that occur just within that little text image:

  • Using an image to show text with a particular font.
  • Using Comic Sans on what is supposed to be a serious website.
  • Linking to a Google search result rather than to an actual webpage.
  • Completely failing to understand what you’re selling (it’s a Satoshi, not Satasoshi, and the horse has nothing to do with it).

MyKrypto Mining Page

Let’s move over to MyKrypto’s Mining page. It now looks something like this:

While the image is totally out of place – and we’ll get to it – this is on the whole not too bad. Let’s take a look at what the previous version of this page (from just a couple of weeks ago) used to look like:

First, the title.

“What is minning?”

I don’t know. Perhaps this lovely lady might be able to answer that.

Image credit: taken from Wikipedia’s Minnie Mouse page

Next, we can take a look at the text.

“Do you look up for information on the internet? Did you ever google for a product or information? Did you ever wondered how google is so intelligent by search for the right information? Google is based on mathematical calculations. FIND X. Let’s say I want to google for ‘this week top 20 music list’ as you type in the google box and press search google, google will do all the calculations to find ‘this week top 20 music list ‘. this is called Math (Algebra) when we were at school we used to have the same problem solving; – FIND X.”

It is simply beyond belief how much crap these guys have managed to fit into a single paragraph. But beyond that, take a look at the image below that paragraph in the screenshot. It’s a popular joke that has been circulating for many years: a clueless student answered a mathematical problem in a witty manner. Of course, whoever built the MyKrypto website didn’t get the joke, and put the image there as an example of mathematics. Go figure.

List of fails in this section:

  • Terrible use of English (if it can be called that).
  • Google does not solve algebraic problems to give you your search results.
  • Don’t lie about what you used to do at school, if you evidently know nothing about English, mathematics, or computing.
  • Try to understand what an image actually means, before ripping it off.
  • Try to understand what you’re talking about in the first place.

MyKrypto Audio

MyKrypto automatically plays audio.

That’s something really annoying, especially if you happen to already be playing music. It’s also of questionable legality depending on whether the site has the right to distribute that music.

Besides, using some cheesy 70s disco background music – reminiscent of Earth, Wind & Fire – is totally not appropriate on a company website.

MyKrypto Mobile

Mobile users will be disappointed to find out that they can’t really browse the entirety of the site because the navigation is simply not available:

MyKrypto Plagiarism

To be fair, messed up paragraphs like the one we saw earlier are a rare sight on this site. In fact, a lot of MyKrypto’s content is blatantly stolen from other websites.

Let’s see some examples:

I think they should plagiarise more. It would make them look a lot less silly.

CEO Plagiarism

Of course, the plagiarism on MyKrypto makes perfect sense if we look at the LinkedIn profile of 3 Group CEO Dario Azzopardi (MyKrypto is part of 3 Group):

When I first read this, I thought it was really weird as a job description. So I Googled part of it.

Google did its algebra (!), and what do you know

…and further down…

3 Group: Questionable Stuff

Having seen all this, I thought it was just as well to check what else 3 Group actually do.

3 Group do IT Services, IPTV, and E-Money. That’s a nice name for the Bitcoin stuff we’ve seen above. They actually got the link wrong, and E-Money points to IPTV.

If we take a look at IT Services, we get to this horrendous page with a background animation driving you nuts while you try to read text with very bad contrast:

Further down that page, 3 Group are trying to convince people that free antiviruses are bad, and that they should instead pay 3 Group to install McAfee for them:

Towards the bottom, you can see some grey text representing a link to Intel’s homepage. Of course, they didn’t bother to actually make it a link that you can just click on. What’s even worse is that the superhero on the left is an image overlaid onto the text where the link is, so you can’t even select and copy it.

Right, what else do 3 Group do? Ah yes, IPTV. It’s interesting how they have this “Legal” page under the IPTV section, claiming that “IPTV is 100% legal”, and quoting some court case from the European Court of Justice.

This is noteworthy because:

  • Naturally, a company encourages trust by stressing that its services are 100% legal.
  • This company knows a lot about copyright, given the aforementioned plagiarism.
  • It claims that “watching streams even those which are illegal is not an act of copyright infringement”. So it’s ok if it’s illegal, as long as it’s not copyright infringement, right?

Well, they say IPTV is legal, so it must be true.

Image credit: taken from here

That’s curious, because I could swear I recently read an article about this Kodi TV streaming service being declared illegal across the EU:

Conclusion

While 3 Group’s web design is appalling at best, this is not nearly as worrying as their questionable business practices. As an exception in this series, I hope not only that web designers/developers learn from the mistakes we have seen here, but also that potential customers do some proper research and understand what these guys are actually trying to sell to them.

The Pitiful State of the Web in May 2017 (Part 2)

This article is a continuation of The Pitiful State of the Web in May 2017 (Part 1) and a part of the Sorry State of the Web series. I and the others who contributed to the content of this series hope that web developers will learn from the mistakes of others and produce better quality websites.

Dakar: Language Issues and Insecure Login

Dakar Software Solutions is a well-known name locally, especially in the realm of payroll systems.

They had this little glitch with the language of dates in the news of their Dakinet product (which might be fixed now):

Also, Dakar joins the long list of websites that offer insecure login:

Insecure Login Galore

As you can imagine, Dakar is not alone in failing to transmit user credentials securely. We’ve seen a lot of these before, and we have a lot more to show here.

For starters, we have Freelance Malta. Since all of the site is based on insecure HTTP, the login form and both registration forms transmit credentials insecurely:

Then we have Gizmodo, the popular tech website:

KeepMePosted is a similar offender:

And then we have MyMoneyBox (part of the MFSA family), which given its name should know better about security. In fact, it seems to have now gone HTTPS, so the login is now secure. But as you can see below, it wasn’t until recently:

Couchsurfing: Invalid SSL Certificate

I recently caught the Couchsurfing blog giving invalid certificate warnings:

Oops. Looks like the certificate had expired.

Needless to say, it is useless to use HTTPS if it is not trusted. Fortunately, this issue has since been fixed.

Malta Police Force: Passwords In Email, Freedom of Information Link

The Malta Police Force website offers a number of services including filing a police report online. At the bottom of the declaration where you’re about to file a report, there’s a link to the Freedom of Information Act:

Unfortunately, it doesn’t work:

That error is actually coming from elsewhere on the government network (looks like it’s the Department of Information). Either the Malta Police Force need to fix their link to point elsewhere, or the DOI needs to fix a problem in their SharePoint system.

There’s something a lot worse, though. Some people have reported that when you file a police report, you choose a password, which is then sent to you via email.

This image was contributed by someone who actually filed a real police report. Aside from various spelling issues in the email, you can see that the password (obfuscated here for obvious reasons) is included.

This is something you always want to avoid because you can never assume that email is a secure channel on which to send sensitive information such as credentials or credit card information. Good practice is to let users choose their password over a secure channel (which the system reportedly already does), store it securely using a one-way hash, and provide the means to reset it using limited-time tokens in case the password is forgotten.

Rizzo Farrugia – Broken Link For New Equity

When new shares under the symbol “PG” were listed on the Malta Stock Exchange, Rizzo Farrugia were quick to add it to their own list:

However, they were not nearly as fast at creating the detail page that the listing links to:

No big deal there. It was fixed the next day.

PWC Refresh Form

PricewaterhouseCoopers has this newsletter signup form. It has a reset button. Something pretty normal, you’d think, until you see that it refreshes the entire page!

Form resetting functionality has been built into browsers since long before I started creating websites (15 years and counting). In this case, I see they wanted to reset the CAPTCHA. But they already have functionality to reset the CAPTCHA without reloading the page (the orange round arrow next to the CAPTCHA), so why reload the whole page just to reset a form?

IDPC: Line Spacing

The Office of the Information and Data Protection Commissioner has a form where you can submit complaints:

What I’d like to call out here is the questionable design choice of using massive line spacing, which is especially noticeable in the Complaint text area since about half of the tiny box is wasted with empty space.

Line Separator Characters

JobsPlus, whose encoding issues we have already visited in “The Broken Web of March 2017“, is now also exhibiting these weird LSEP characters:

It’s okay though. Perhaps they can’t sanitise their data, but they still get to keep their eBusiness Award!

I have also spotted the same problem at Creative Jobs:

Summary

Transmitting credentials insecurely remains one of the most common issues on websites today, and it is completely unacceptable. Depending on the nature of the user account, this might not be as risky as transmitting credit card details insecurely (something we’ve also seen in abundance over the past few months), but that does not relieve websites from their duty as data controllers to transmit sensitive data securely.

It is also important to test websites properly in order to identify broken links and data-related issues as we have seen.

Finally, secure transmission of sensitive data does not stop at using HTTPS. SSL certificates must be integral and trusted, otherwise it is just as good as not having HTTPS. Email is not a secure channel, so don’t use it to send sensitive data, especially if there exist alternative data flows where you don’t have to.

How to Update All Orleans Packages

Sometimes, you will want to upgrade to a newer Microsoft Orleans version. Like today, since Orleans 1.4.2 has just been released.

Thing is, it can be a bit of a pain to update all these packages manually across different projects. It’s also quite error-prone. It has happened in the past that some of those packages ended up on one version, and others ended up on another, causing runtime mayhem.

Fortunately, there’s a simple way to get them all in sync with minimal effort. Just use the following command (taken from the Grain Persistence documentation) in the Package Manager Console:

Get-Package | where Id -like 'Microsoft.Orleans.*' | foreach { update-package $_.Id }

This will go through all the packages that start with “Microsoft.Orleans.” and update them.

Orleans Grain Persistence with the ADO .NET Storage Provider

Yesterday, we introduced grain persistence in Microsoft Orleans, using the volatile MemoryStorage provider to quickly and easily learn how to set up and work with storage providers.

Today, we will use the ADO .NET Storage Provider in order to save our grain state to a database. Currently, we can use this with either SQL Server or MySQL.

Note: MariaDB is a drop-in replacement for MySQL. Thus, the ADO .NET Storage Provider works with it too.

Note: While there is a partial script for PostgreSQL, it is not yet supported.

Update 9th June 2017: Some material from this article was contributed towards the official Grain Persistence documentation.

Example Scenario

Like in yesterday’s article, we’ll use a very simple setup based on the Orleans Dev/Test Host project. We’ll use the same classes and interface:

    public interface IPersonGrain : IGrainWithStringKey
    {
        Task SayHelloAsync();
    }

    public class PersonGrainState
    {
        public bool SaidHello { get; set; }
    }

    [StorageProvider(ProviderName = "OrleansStorage")]
    public class PersonGrain : Grain<PersonGrainState>, IPersonGrain
    {
        public async Task SayHelloAsync()
        {
            string primaryKey = this.GetPrimaryKeyString();

            bool saidHelloBefore = this.State.SaidHello;
            string saidHelloBeforeStr = saidHelloBefore ? " already" : null;

            Console.WriteLine($"{primaryKey}{saidHelloBeforeStr} said hello!");

            this.State.SaidHello = true;
            await this.WriteStateAsync();
        }
    }

If we give our PersonGrain the name of “Joe” and call SayHelloAsync(), then it will write “Joe said hello!” to the console. After that, it sets the SaidHello property in its grain state to true, and saves the updated state to the database via the configured storage provider. The next time SayHelloAsync() is called, it will find that the value of SaidHello is true, so the output will be “Joe already said hello!” instead.

In the main program, let’s add the following code (same as in yesterday’s article) to invoke SayHelloAsync() on a grain. This needs to go in the place of the TODO comment that was generated as part of the project template.

            var joe = GrainClient.GrainFactory.GetGrain<IPersonGrain>("Joe");
            joe.SayHelloAsync();

Now, let’s move on to configure our storage provider.

Setting up the ADO .NET Storage Provider

Install the following package:

Install-Package Microsoft.Orleans.OrleansSqlUtils

Navigate to the folder where the package was installed alongside the project, and you’ll find folders in there for different databases.

In there you’ll find scripts you’ll need to run in order to set up the Orleans database, for different database vendors. Each script contains a table for storage and several others for cluster membership and other internal Orleans operations. For the scope of this article, we’re only concerned with storage.

Note: while there is a script for PostgreSQL, at the time of writing this article, it does not support storage.

The steps to configure an ADO .NET Storage Provider are as follows:

  1. Create the database specific for the database vendor you want.
  2. Install a NuGet package specific for the database vendor you want.
  3. Set up the storage provider in code or XML configuration. Set the AdoInvariant setting (optional only if you’re using SQL Server).

Using SQL Server for Grain Persistence

Setting Up the Database

When using SQL Server, you’ll want to use the SQLServer\CreateOrleansTables_SqlServer.sql script.

Open SQL Server Management Studio. Right click on Databases, and create a new one.

Call it whatever you like, e.g. OrleansStorage.

Open a query window based on your new database. Copy in the aforementioned script, and run it.

Installing the NuGet Package

Next, install the following package:

Install-Package System.Data.SqlClient

Configuring via Code

Finally, we need to configure the storage provider itself. We can do this in code by replacing the following line in OrleansHostWrapper.cs:

config.AddMemoryStorageProvider();

…with code that sets up an ADO .NET storage provider. First, we declare a connection string:

const string connStr = "Server=.\SQLEXPRESS;Database=OrleansStorage;Integrated Security=True";

Then, we can set up the ADO .NET storage provider using either a registration call that takes the storage provider’s type name as a string:

            var typeName = "Orleans.Storage.AdoNetStorageProvider";
            var properties = new Dictionary<string, string>()
            {
                ["DataConnectionString"] = connStr,
                ["AdoInvariant"] = "System.Data.SqlClient"
            };

            config.Globals.RegisterStorageProvider(typeName, "OrleansStorage", properties);

…or its generic equivalent:

config.Globals.RegisterStorageProvider<AdoNetStorageProvider>("OrleansStorage", properties);

Note that there is also a generic extension method we could use as a shortcut:

config.AddAdoNetStorageProvider("OrleansStorage", connStr);

However, I recommend against using this method. As per an issue I just opened, there is no way to set the AdoInvariant (which is optional for SQL Server but necessary when working with other database vendors), and the default serialization format is XML (whereas the default is usually binary). Thus, if you switch between this method and the earlier ones while relying on defaults, you will get deserialization errors.

Configuring via XML

On the other hand, to configure the storage provider using an XML configuration file (which is generally a better approach), first we have to add a file named OrleansConfiguration.xml and set it to copy to the output directory.

Add the following to the file:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <StorageProviders>
      <Provider Type="Orleans.Storage.AdoNetStorageProvider"
                Name="OrleansStorage"
                AdoInvariant="System.Data.SqlClient"
                DataConnectionString="Server=.\SQLEXPRESS;Database=OrleansStorage;Integrated Security=True"/>
    </StorageProviders>
    <SeedNode Address="localhost" Port="22222" />
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="22222" />
    <ProxyingGateway Address="localhost" Port="40000" />
  </Defaults>
</OrleansConfiguration>

Then, replace the following code in OrleansHostWrapper.cs:

            var config = ClusterConfiguration.LocalhostPrimarySilo();
            config.AddMemoryStorageProvider();
            siloHost = new SiloHost(siloName, config);

…with this:

            siloHost = new SiloHost(siloName);
            siloHost.ConfigFileName = "OrleansConfiguration.xml";

Testing It Out

Once you have configured everything, run the program:

It says “Joe said hello!” Remember that the state should have been updated at the end of the method that writes that message. Let’s verify it by closing the program, and running it again:

This time, it says “Joe already said hello!” That means that the state was correctly read from the database.

Using MySQL for Grain Persistence

Note: this also works for MariaDB, which is a drop-in replacement for MySQL.

Setting Up the Database

Deep beneath the folder where the Microsoft.Orleans.SqlUtils package gets installed, you’ll find a MySql\CreateOrleansTables_MySql.sql script. We’ll use this for MySQL or MariaDB.

Use your favourite administrative tool to create a new database, and give it a name (e.g. “orleansstorage”). Paste the script in a query window, and run it against the new database.

Installing the NuGet Package

Install the following package:

Install-Package MySql.Data

Configuring via Code

We can replace the AddMemoryStorageProvider() call in OrleansHostWrapper.cs with code to set up our ADO .NET provider. First, let’s put our connection string in a variable to keep the rest of the setup code concise:

const string connStr = "Server=localhost;Database=orleansstorage;uid=xxx;pwd=xxx";

We can now configure the storage provider, using either a named type:

            var typeName = "Orleans.Storage.AdoNetStorageProvider";
            var properties = new Dictionary<string, string>()
            {
                ["DataConnectionString"] = connStr,
                ["AdoInvariant"] = "MySql.Data.MySqlClient"
            };

            config.Globals.RegisterStorageProvider(typeName, "OrleansStorage", properties);

…or a generic method based on the type itself:

config.Globals.RegisterStorageProvider<AdoNetStorageProvider>("OrleansStorage", properties);

As you can see, this is almost identical to the configuration for SQL Server. We’re using the exact same ADO .NET storage provider, and changing just the connection string and the AdoInvariant which identifies the underlying database vendor.

Configuring via XML

Instead of hardcoding our configuration, we have the option to read it from an XML file. Let’s create a file named OrleansConfiguration.xml, set it to copy to the output directory, and give it the following content:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <StorageProviders>
      <Provider Type="Orleans.Storage.AdoNetStorageProvider"
                Name="OrleansStorage"
                AdoInvariant="MySql.Data.MySqlClient"
                DataConnectionString="Server=localhost;Database=orleansstorage;Uid=xxx;Pwd=xxx"/>
    </StorageProviders>
    <SeedNode Address="localhost" Port="22222" />
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="22222" />
    <ProxyingGateway Address="localhost" Port="40000" />
  </Defaults>
</OrleansConfiguration>

Then, as we did before in the SQL Server section, replace the following code in OrleansHostWrapper.cs:

            var config = ClusterConfiguration.LocalhostPrimarySilo();
            config.AddMemoryStorageProvider();
            siloHost = new SiloHost(siloName, config);

…with this:

            siloHost = new SiloHost(siloName);
            siloHost.ConfigFileName = "OrleansConfiguration.xml";

This allows us to bypass the default localhost silo configuration and read the configuration from our file instead.

A Word On Formats

ADO .NET Storage providers can save data in one of three formats: JSON, XML, or a compact binary format.

The default format is binary, and while it is compact, it means the data is opaque and you can’t really work with it directly. This could be a problem if you need to patch, migrate, or troubleshoot the data. XML is bloated, so I generally recommend using JSON. It’s lightweight (although not as much as the binary format) and is easy to work with.

You can switch between them by setting the appropriate property in either code or XML configuration:

  • UseJsonFormat="true" uses JSON.
  • UseXmlFormat="true" uses XML.
  • UseBinaryFormat="true" uses binary, which is the default setting.

Ideally you should set this only the first time. If you change this setting when you already have data, you’ll have a mess of formats and will not be able to read old data.

Summary

Install the base package for the ADO .NET Storage Provider:

Install-Package Microsoft.Orleans.OrleansSqlUtils

Then set up as follows depending on the database vendor:

SQL Server MySQL
Script SQLServer\CreateOrleansTables_SqlServer.sql MySql\CreateOrleansTables_MySql.sql
NuGet Package System.Data.SqlClient MySql.Data
AdoInvariant System.Data.SqlClient MySql.Data.MySqlClient

You can set the following propeties:

Name Type Description
Name String Arbitrary name that persistent grains will use to refer to this storage provider
Type String Set to Orleans.Storage.AdoNetStorageProvider
AdoInvariant String Identifies the database vendor (see above table for values; default is System.Data.SqlClient)
DataConnectionString String Vendor-specific database connection string (required)
UseJsonFormat Boolean Use JSON format
UseXmlFormat Boolean Use XML format
UseBinaryFormat Boolean Use compact binary format (default)

When configuring via code, prefer the generic registration call, as it avoids magic strings and potential runtime errors by using a type provided at compile time. Avoid the shortcut extension method since it does not let you configure the AdoInvariant and has a different default format from the other API methods.

Introduction to Grain Persistence with Microsoft Orleans

In this article, we’re going to take a look how the state of virtual grains can be saved to and loaded from a data store. Since there are various storage providers available for Microsoft Orleans, we will use the in-memory storage provider to introduce the concepts in a simple manner. This is not a real storage provider, and its state is lost if the silo goes down, but it gives us an easy way to learn to set up and interact with storage providers.

The source code for this article is available in the Gigi Labs BitBucket repository. Look for the OrleansMemoryStorage folder.

Update 9th June 2017: Some material from this article was contributed towards the official Grain Persistence documentation.

A Simple Sample

Before we can see how grain persistence works, we need a minimal example. Let’s create a new Orleans Dev/Test Host project using the project templates from the Microsoft Orleans Tools for Visual Studio extension.

Let’s add a new interface that our grain will implement:

    public interface IPersonGrain : IGrainWithStringKey
    {
        Task SayHelloAsync();
    }

Next, we’ll implement a really simple grain:

    public class PersonGrain : Grain, IPersonGrain
    {
        public Task SayHelloAsync()
        {
            string primaryKey = this.GetPrimaryKeyString();

            Console.WriteLine($"{primaryKey} said hello!");

            return Task.CompletedTask;
        }
    }

Note: if you’re using a .NET Framework version prior to 4.6, use TaskDone.Done instead of Task.CompletedTask.

All we need to do now is actually use our grain. Locate the following comment in the template code generated in Main():

            // TODO: once the previous call returns, the silo is up and running.
            //       This is the place your custom logic, for example calling client logic
            //       or initializing an HTTP front end for accepting incoming requests.

…and replace it with the following:

            var joe = GrainClient.GrainFactory.GetGrain<IPersonGrain>("Joe");
            joe.SayHelloAsync();

We can now run the program to ensure that the grain’s SayHelloAsync() is actually called:

Adding State

While it is perfectly valid to have grain state in member variables within the grain itself, if we want to persist that state, we need something a little different.

The first thing we need to do is create a class that holds the state:

    public class PersonGrainState
    {
        public bool SaidHello { get; set; }
    }

Then, we need to change the definition of our grain class to inherit not from Grain, but from Grain<T>, where T is the grain state class:

    public class PersonGrain : Grain<PersonGrainState>, IPersonGrain

We can then change the code within our grain to save its state and do something a little more interesting:

    public class PersonGrain : Grain<PersonGrainState>, IPersonGrain
    {
        public async Task SayHelloAsync()
        {
            string primaryKey = this.GetPrimaryKeyString();

            bool saidHelloBefore = this.State.SaidHello;
            string saidHelloBeforeStr = saidHelloBefore ? " already" : null;

            Console.WriteLine($"{primaryKey}{saidHelloBeforeStr} said hello!");

            this.State.SaidHello = true;
            await this.WriteStateAsync();
        }
    }

All we’re doing here is that if the SaidHello property was set, then we write “Joe already said hello!” instead of “Joe said hello!”. However, note the last two lines. We’re setting the value of SaidHello to true, and then calling WriteStateAsync().

Orleans provides a very simple API for dealing with persistent state, as you can see above. You can access your grain’s state object using the State property. The WriteStateAsync(), ReadStateAsync() and ClearStateAsync() methods allow you to save, load, and clear the grain’s state from the data store respectively.

Note: we haven’t yet set up our data store, but we’ll do that shortly.

The logic for saving state is entirely up to you. You can save state with every message you receive (i.e. with every method call), or periodically, or in whatever manner makes most sense for your application. For instance, in our example above, we are saving state every time the SayHello() method is called. However, it would save unnecessary writes to the data store to do a simple check on the value of SaidHello, and write to the data store only if the value was previously false.

State is loaded automatically from the data store when the grain is activated, so you don’t actually need to call ReadStateAsync() unless it makes logical sense to do so, e.g. if your grain has ended up in an inconsistent state due to errors and you need to reload its original state from the data store.

Configuring a Persistence Store

Let’s run the application as it is now:

It is failing because we haven’t configured our storage provider yet. A grain having persistent state will by default look for a storage provider configured with the name “Default”. We can use an arbitrarily named storage provider by using an attribute:

    [StorageProvider(ProviderName = "OrleansStorage")]
    public class Person : Grain<PersonState>, IPerson

We now have to configure that OrleansStorage data store so that Orleans will know where to find it and how to interact with it. There are various different storage providers, as you can see from the Grain Persistence documentation. All of these can be configured either in code or using XML configuration.

In this example, we’ll use the MemoryStorage provider. This is not a real persistent data store, in the sense that it holds data in memory which is lost when the silo stops. While this does not allow for very interesting storage-and-retrieval scenarios, it allows us to experiment with storage providers without the hassle of setting up a real database. We’ll see how to set up proper storage providers with an underlying database in an upcoming article.

Configuring MemoryStorage in Code

You should have an OrleansHostWrapper class that was automatically created as part of the project template. In the Init() method, we can add the configuration code we need:

        private void Init()
        {
            siloHost.LoadOrleansConfig();

            siloHost.Config.AddMemoryStorageProvider("OrleansStorage");
        }

If you run it now, there will be no error:

The AddMemoryStorageProvider() is just a shortcut method. It takes a name and an optional number of grains used to store the state, which we can leave well enough alone with its default of 10:

The real way to set up any configuration provider is to use RegisterStorageProvider(), and pass in the type of the provider (either as a typename string or as a generic parameter), along with the storage provider name (in our case, “OrleansStorage”) and any additional parameters (in the case of MemoryStorage, we can set that NumStorageGrains here).

Using generics, it looks like this:

        private void Init()
        {
            siloHost.LoadOrleansConfig();

            siloHost.Config.Globals.RegisterStorageProvider<MemoryStorage>("OrleansStorage");
        }

On the other hand, this is how you’d do it if you specified the type name as a string:

        private void Init()
        {
            siloHost.LoadOrleansConfig();

            siloHost.Config.Globals.RegisterStorageProvider("Orleans.Storage.MemoryStorage",
                "OrleansStorage");
        }

Configuring MemoryStorage in XML

Instead of hardcoding your configuration, you can set it up in an OrleansConfiguration.xml file that your silo will then pick up.

Let’s add an OrleansConfiguration.xml file to the project. We also need to set the file’s Copy to Output Directory property to either Copy always or Copy if newer. Then, we can add the following to it:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <StorageProviders>
      <Provider Type="Orleans.Storage.MemoryStorage"
                Name="OrleansStorage"
                NumStorageGrains="10" />
    </StorageProviders>
    <SeedNode Address="localhost" Port="22222" />
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="22222" />
    <ProxyingGateway Address="localhost" Port="40000" />
  </Defaults>
</OrleansConfiguration>

You can see how we’re configuring pretty much the same things that we did earlier in code. Any optional parameters, such as NumStorageGrains in this case, can also be added.

The StorageProviders section is all we need to set up a storage provider, but you’ll notice we have a lot more configuration in the file. The Dev/Test Host project template uses LocalhostPrimarySilo() configuration, which is good to set something up quickly, but it’s a bit of a pain to get to work with configuration files. Thus, we’ll remove it and replace its default configuration with that in the file. With a proper Orleans project setup, you won’t need to do the following steps.

Locate the following code in OrleansHostWrapper.cs:

            var config = ClusterConfiguration.LocalhostPrimarySilo();
            config.AddMemoryStorageProvider();
            siloHost = new SiloHost(siloName, config);

Replace it with the following:

            siloHost = new SiloHost(siloName);
            siloHost.ConfigFileName = "OrleansConfiguration.xml";

The Init() method should be left as it originally was:

        private void Init()
        {
            siloHost.LoadOrleansConfig();
        }

You can now run the program. It should work without problems, using the configuration from the file to set up the storage provider.

Summary

In order to persist a grain’s state, it is necessary to:

  1. Create a class that holds the grain’s state.
  2. Have the grain inherit from Grain<T>, where T is the grain state class.
  3. Access the grain’s state object using the State property.
  4. Use WriteStateAsync(), ReadStateAsync() and ClearStateAsync() to write, read or clear the grain’s state in the data store respectively.
  5. A grain’s state is automatically loaded from the data store when it is activated.
  6. A grain’s storage provider name is set using the StorageProvider attribute, in absence of which it uses one named “Default”.
  7. There are various storage providers available that work with different data stores. We have so far used only MemoryStorage, which is a volatile implementation to be used only for testing.
  8. Storage providers may be configured either in code or using an XML configuration file.

The Pitiful State of the Web in May 2017 (Part 1)

Welcome back to the Sorry State of the Web series! This is a collection of bad stuff found on so-called professional websites, contributed by both myself and others who have submitted entries. It is sad to see so many fundamental mistakes being repeated over and over again, and by calling them out, we hope to promote better quality work in web development, and as a result, a better experience on the web.

Unfortunately, this month we are once again about to see a lot of security-related violations, including insecure login and credit card processing. We will also see a lot of negligence. Thus, without further ado…

Deal: Insecure Login

deal.com.mt, like many other websites we have mentioned and will mention, support registration and login over insecure HTTP:

You will also notice the strangely superimposed text saying “Please log into this app” below the Facebook button. Certainly not an artistic style I would want to imitate.

Careers in Finance: A Different Kind of Education

Careers in Finance, a pathetically designed website that seems to be part of MFSA, has this Warnings page.

The warnings page presumably takes you to a list of unrecognised training institutions. So when you follow the link, you get…

…this. Aside from the error page, you’ll notice a hilarious misspelling of the word “Universities” in the filename. Whoever named the file was evidently alienated by more… interesting stuff at the time.

Microsoft: Runtime Error Page

I noticed a similar runtime error when accessing a webpage on Microsoft’s own website. They could have handled this better.

The Malta Independent: Sneaky Advertising

The Malta Independent had this really invasive ad covering the whole site as you load it:

If you click the link at the top-right of the ad that says “Skip and Visit Site”, you are actually taken to the website that the ad is promoting, rather than just closing the ad and letting you read the online newspaper. What a sneaky way of raising advertising revenue!

If you wanted to just close the ad, you actually had to click the “X” at the top left, which is very easy to miss.

This shameful advertising mechanism seems to be gone now, thankfully.

Mediterranean Bank: Out With the Old, In With The Crap

Last weekend, Mediterranean Bank launched their shiny new internet banking platform, after a whole weekend of planned downtime for the changeover.

Existing users have to undergo a migration process, and this is fraught with flaws.

The first thing you see in this new system is a field requesting a “Client number”. The problem is, nobody has any idea what this client number is. In the old system, we used to use a username and various other fields, but no client number. And sure enough, if you enter something invalid, an error appears, telling you to enter your old username if you are using the new platform for the first time.

That would have been useful to have before you try to login.

After that, you have to enter your surname. So they made a whole webpage just for you to enter your surname (yes, full page reload).

To migrate your account, you have to enter all the stuff you used to have in the old system (understandably, because you have to be authenticated). That includes a secret question:

Now, using secret questions is already arguably very stupid in the first place. But not obfuscating the answer (which the old system did properly, by the way), is just terrible from a security standpoint. Security answers, while not passwords in themselves, are password-like material. You do not want someone looking over your shoulder to be able to read them just because you are typing them in.

Moving on to the less serious and more silly flaws, it seems like Mediterranean Bank have taken inspiration from JobsPlus (see the March issue) and put in a language selector with just English in it:

You can choose between English… and English.

What about that cookie policy at the bottom? They ask you to read their cookie policy, but there is no link. It looks like they just forgot to include it, because their main website (i.e. not the online banking part) has it:

Sport Malta: Insecure All The Way

Sport Malta, another website by Cyberspace Solutions Ltd. (a company well-known to this series – see “Lost in Cyberspace in February 2017“), was caught processing credit cards and login insecurely:

It seems like they now have HTTPS, but it doesn’t quite work because of mixed content:

Poor guys. They can’t seem to get one thing right.

EUROPA: Cobwebs and Such

Like Sport Malta, the website of the European Union has a bit of a mixed content issue that invalidates its HTTPS setup:

So like any good citizen would do, I decided to report the issue. In their contact form, you can specify what browser you’re using. Well, the browser versions in the list are ancient (I was using Chrome 58, and the latest one in their list is 40; likewise, although I was using Firefox 53, I could only choose up to Firefox 34. They even managed to misspell the Konqueror web browser.

Anyway, I reported the HTTPS problem, and also asked them nicely to update the browser versions on the contact page. When you write to them, they tell you that it can take about 3 days for them to get back to you.

And that’s exactly what happened. Today, I received a reply, which said:

“Would you kindly clarify if you are referring to some specific webpages?
You may contact us again in any of the 24 EU official languages via our webform which is available here:
https://europa.eu/european-union/contact/write-to-us_en
This clarification would enable us to forward your message to the relevant department of the European Commission for information purposes.”

So basically, having taken 3 days to reply, these guys didn’t even bother to browse their own website’s homepage. And contacting me back through a no-reply email address, they expect me to fill in that form again, just so that I can tell them what they could already have determined themselves, and then forward it to some department where it would then get lost in a bureaucratic hole.

No thanks.

Spotlancer: Insecure Login

Just more of the same from Spotlancer:

TicketArena: Insecure Credit Card & Login

Be careful where you buy your tickets from! Ticket Arena is served over insecure HTTP, yet it processes credit card info:

…and credentials:

“Your credit Card is 100% Safe and Secure,” they said. “We use the latest standards for security with Comodo,” they said.

Image credit: taken from here

Summary

As I’ve repeated ad nauseam over the past articles, you simply cannot process sensitive data (including passwords and credit card details) over an insecure channel. It doesn’t matter if you’re using an HTTPS iframe inside an insecure HTTP-served page. It’s simply not enough.

Websites also need to be tested better. Several websites that we have seen in this article have various problems of different severity levels that could have easily been caught earlier with a little more attention.

We’ll see more issues along these lines in Part 2. In the meantime, I would like to thank all those who sent reports for entries that were included in this article, and I welcome submissions for the June issue.

The Shameful Web of April 2017 (Part 2)

This article is a continuation of The Shameful Web of April 2017 (Part 1) and a part of the Sorry State of the Web series, in which I and various contributors show various blunders in supposedly professionally made websites in order to promote a better web.

The Hive: Mixed Content

At the time of writing this article, The Hive still has an issue with its HTTPS connectivity in that it is considered insecure because it’s using a resource that isn’t coming over HTTPS.

If you want your site to be served over HTTPS, then any images, scripts, and any other resources that it uses must also be served over HTTPS.

Malta Stock Exchange: Content Should Come First

Think of this: if I trade on the stock exchange, I would like to be able to see stock and share prices quickly.

So let’s go to the Malta Stock Exchange website:

(By the way, until a few days ago, there was a nice big photo of Fort St. Angelo instead of this Latest News section. It still gets in the way of finding the information you want, but it looked a lot more silly with a nice picture of the Fort, and I wish I had grabbed a screenshot back then.)

Now, we have to scroll halfway down the page:

Then, we need to expand “Regular Market”…

…and finally, we can see the prices we were looking for. Unfortunately, this is not very intuitive if you’re visiting the site for the first time, and it is a real pain in the ass to have to do this every time you want to check the share/stock prices. This is the information that people want to see most of the time, and it should be the first thing presented on the site, not buried somewhere far down the page.

There is nothing intrinsically ‘wrong’ with this in the sense of many other serious flaws that I usually write about in these articles. However, from a usability point of view, it really sucks.

MTA: Load Times and Insecure Login

The Malta Tourism Authority website is a terrible failure in terms of load time: it usually takes over 20 seconds to load.

As if that wasn’t enough, it offers an insecure login facility, which you’ll know to be a serious Data Protection violation if you’ve read previous articles in this series.

Olimpus Music: Insecure Login

Another offender in the category of insecure logins is Olimpus Music.

Basically, don’t use their online checkout facility until they use an encrypted connection.

Owner’s Best – A Real Mess

In “The Broken Web of March 2017 “, we covered some issues with the Owner’s Best website. I see they still haven’t fixed the “Error : Rows Not Set” bug that you can still see if you scroll to the bottom of the page, and neither did they fix the property detail links scrolling down to the contact form and confusing people as a result.

But there’s more. And worse.

For starters, they have a “Property TV” link in the navigation.

Sounds interesting! Let’s see what it does.

Boom. Dead link.

Okay. Let’s try searching for something from the homepage. Oops, I forgot to enter a budget – my bad.

But what the hell is this Fulcrum Alert? And what is wrong with the close buttom? That was a rhetorical question actually. Image 404s in console:

Oh dear. Okay. Let’s put in a budget then.

I put in 10,000. Hey, I’m broke. Obviously, nothing matched, and I got a sad message saying “None properties found”. Yes, you has very good England.

Now I put in a budget of 10 million. That means that I’m super rich, and I’m ready to spend anything up to 10 million on a single property. I got 3 results. Wow. These guys must deal in some real luxury stuff. In fact, two of the results are over budget.

The above search results are based on a 5-million-Euro budget. It gave me this one 4.3-million-Euro bungalow in Dingli. Why didn’t I get this when I searched with 10 million Euros as a budget? 4.3 million is less than 10 million, right?

Now I searched with a budget of 100,000 Euros. Not only do we get all these nice results that would have fitted quite nicely within the several-million-Euro budgets we pretended to have earlier, but we also get properties that are beyond budget, like the one at the top right and the one at the bottom right.

In summary, let’s just say that the search functionality at the Owner’s Best website works in mysterious ways, whether that is intentional or not.

Seasus – Insecure Login

Let’s welcome Seasus among the ranks of the websites that offer an insecure login form:

It is touching to see how much they care about their clients.

Something Different – Various Issues

Let’s take a look at Something Different, a website by Untangled Media (we’ve covered some more of their work in the past).

First, they accept credit card details over an insecure connection. That’s bad. Very bad.

Of course, the credit card iframe itself uses HTTPS, but it’s an HTTPS iframe embeded in an HTTP page, which is still insecure (and illegal – see “The Sorry State of the Web in 2016“), and there is no padlock icon necessary to provide the user with the trust guarantees s/he needs in order to give out his/her sensitive information on the web.

Login is also served insecurely, as you can see above.

We can see another instance of this, as well a lack of a lot of basic validation, in the user registration process:

As you can see above, you can fill in bogus data for most fields. There isn’t even a simple check on the structure of the email address.

In the second step of user registration, you choose a password. Insecurely, of course.

And that’s it! Congratulations for registering your invalid account insecurely!

In this section, we took a look at Something Different. Or rather, more of the same.

Untangled Media / Winit

In Untangled Media‘s Web Publishing section, you’ll find references to various sites including Something Different (see previous section) and something called winit.com.mt:

As they say in the summary, “Everybody loves winning things.” So do I! Let’s follow the link and check out the site.

Oops. Let’s try going to the root of the domain instead.

Win it indeed! It’s more like Untangled Media have lost it.

Summary

April has been a very busy month for spotting issues on websites. We’ve seen a lot of serious security flaws (e.g. insecure login and credit card processing) that have been covered extensively throughout this series.

However, we’ve also spotted a number of issues including high loading times (on one occasion due to the use of large images without thumbnails) and various usability problems. Always keep in mind that websites need to deliver information (whether to sell or otherwise), and thus, information needs to be delivered in a timely, clear, and intuitive manner.

Let’s hope that this article makes some people chuckle, and makes others do a better job of building websites!

Thanks for reading, and stay tuned for the May edition of The Sorry State of the Web! If you find any issue that you would like to include in this series, we would love to hear about it.

The Shameful Web of April 2017 (Part 1)

This article is part of the Sorry State of the Web series, which aims to raise awareness about common and fundamental issues in supposedly professional websites in order to push web developers and designers to raise the bar and deliver at least decent user experience. Since a lot of issues were noted in April 2017, the April issue will be split into two parts. I would like to thank those readers of Gigi Labs who contributed several of the entries in this article.

JobsPlus Receives e-Business Award

In the March 2017 issue of the Sorry State of the Web series, I had pointed out some really basic flaws in the JobsPlus website. That didn’t keep it from receiving an award for “best technology in the e-Government sector”.

Image credit: taken from here

Facebook’s Intrusive Login Prompt

If you view a video on Facebook and you aren’t logged in, you get this login prompt that practically takes up the entire window:

There’s a tiny “Not Now” link at the bottom that you can click. This doesn’t actually remove the prompt, but makes it smaller and moves it to the bottom:

Unfortunately, there seems to be no way to close it, and it still takes up a significant portion of the screen, especially if you are on a laptop. Not very nice!

Don’t Send Passwords via Email

I got this email from a web hosting company:

They never learn. You should never send passwords via email. There is absolutely no guarantee that emails are transmitted via secure channels, so you should assume that it is insecure by default. Instead, let the user choose a password on your website, when the content is served over HTTPS.

Links Should Actually Work

We all know how annoying broken links are, but RightBrain have found a way to match that frustration using links that actually work:

The social media icons at the bottom-right actually point to the website’s homepage, rather than to the social media portrayed by the icons.

It’s not enough that links aren’t broken. Make sure they actually go to the right place!

Microsoft .NET Core Documentation

If you want to learn a little C#&lrm; (whatever that is supposed to mean), you’re in luck. Microsoft has some tutorials about it:

Seriously though, the .NET Core documentation had some funny HTML entities running around in its sidebar, as you can see above. Very careless, but it looks like they’ve noticed, because this has now been fixed.

Another area where .NET Core documentation is still lacking is in printer-friendliness:

I have written in the past how making webpages printer-friendly is really easy yet very often overlooked. In fact, in the example above, you get around 10 pages of printed content, and the rest of 67 pages which are blank. I have raised this with Microsoft. It seems to be fixed in some browsers (e.g. Edge), and mitigated in Chrome. It’s no longer 67 pages, but at the time of writing this article, you still get quite a few blanks.

Finally, I noticed an issue with their HTTPS. As you can see, you don’t get the padlock indicating that the connection is properly encrypted:

Apparently it’s due to mixed content:

This only happened to me using Firefox on Linux though.

Dear Steve

High up on the list of biggest fails ever in this series is the MySmile dental clinic. There is this contact page with instructions from the dental clinic to a certain “Steve” (presumably from Just Some Coding Ltd, who developed the website) on improvements to make:

Although some pages and links seem to have been renamed, the old “Contact” page shown above is still online!

In any case, Steve didn’t really give a shit, because the map point that he was asked to change still points to the exact same place.

Language Confusion

Unlike JobsPlus, DR Gaming Technology‘s website is really multilingual. In fact, it supports so many languages that one of the language flags actually ended up sitting over the search box:

Despite the language selection, The Latest News box to the right includes many languages at the same time, including English, German and Spanish:

Timely CORS Issue

A friend noted that one of the fonts (Times New Roman) used on the login form of Timely (a web app that I love to hate) looked very out of place.

In fact, the developers never intended to use Times New Roman. They wanted a font called Avenir, but the browser defaulted to Times New Roman due to a CORS issue:

Timely fixed this issue within hours, but it wasn’t timely enough to keep me from taking screenshots.

Use Thumbnails

On some articles at Forbes, the images take ages to load. For instance:

What is more depressing than the job ads mentioned in the article? The fact that the image embedded in that page is actually a really large image:

It should be common knowledge now, in 2017, that you should embed a small version of the image (a thumbnail), and link to a larger version. This way, the image won’t impact page loading time, but the people who want to see the detail can opt to do so. This is especially important in galleries with lots of images.

To Be Continued…

More to follow in Part 2.