Category Archives: Software

Software

Setting Up Elasticsearch on Linux Ubuntu

1 Comment

Elasticsearch is a lightning-fast and highly scalable search engine built on top of Apache Lucene. In this article, we’re going to see how we can quickly set it up on an Ubuntu Linux environment (using Ubuntu 16.10 here) to be able to play around with it. We do not cover configuring Elasticsearch or setting up a cluster. To set up Elasticsearch on Windows, see “Setting Up Elasticsearch and Kibana on Windows” instead.

Before we can set up Elasticsearch itself, we need Java. We can follow these instructions to set up Java on Ubuntu. Before proceeding, verify that the JAVA_HOME environment variable is set:

echo $JAVA_HOME

It is likely that you won’t see anything as a result of this command. That’s because while the Java setup instructions do set this environment variable, it does not get applied to your current session. Try opening a new terminal window or reboot the machine, and chances are that your JAVA_HOME will be set correctly. If not, you may have to set JAVA_HOME manually.

Once Java is correctly set up (complete with the JAVA_HOME environment variable), we can proceed to set up Elasticsearch. By going to the Elasticsearch downloads page, we can download (among other things) the Debian package containing Elasticsearch:

We can now install the Debian package using dpkg. At the time of writing this article, the latest version of Elasticsearch is 5.4, so after opening a Terminal window based in the Downloads folder, we can use the following command to install Elasticsearch:

sudo dpkg -i elasticsearch-5.4.0.deb

Elasticsearch is now installed, but it is not yet running! So first, we’ll enable the Elasticsearch service so that it will start automatically when the machine is rebooted:

sudo systemctl enable elasticsearch.service

We can now start the Elasticsearch service.

sudo systemctl start elasticsearch.service

The Elasticsearch HTTP endpoint will need a few seconds before it is reachable. After that, we can verify that Elasticsearch is running either by going to localhost:9200 from a web browser, or by hitting that same endpoint using curl in the command line:

curl -X GET http://localhost:9200/

In either case, you should get a response with some JSON data about the Elasticsearch instance you’re running:

We are now all set up to play around with Elasticsearch! Since we didn’t configure anything, we have a single instance with all default settings. If you’re planning to use Elasticsearch in a production environment, you will of course want to read up on configuring it properly and setting up a cluster to ensure that it can handle the use cases you need and that it can survive failure scenarios.

Software, Software development

Split Code Window in Visual Studio

Leave a comment

This is another of those little things that are right there and yet many people seem to not know about them.

There’s a little handle at the top-right corner of the code window:

When you drag it down, it will split your code window into two parts:

This is very useful when you want to look at two different places within the same code file (e.g. while examining one method, check another one declared earlier in the same file).

At any time, you can drag the splitter in the middle all the way to the top to go back to single view.

Software, Software development

Exception Detail Without A Variable

Leave a comment

Here’s a scenario just about everyone has run into: you had this try/catch block:

try
{
    // ...
}
catch (Exception)
{
    // ...
}

You didn’t need the exception variable, for whatever reason. Although you’d typically leave it there in case you needed to dig deeper into the exception, you got tired of Visual Studio nagging about the unused variable, and removed it.

And then, it happened: an exception occurred, and you actually needed the detail.

As it turns out, there’s a way you can see the exception detail without adding the exception variable back and reproducing the issue a second time. There’s a special $exception variable that you can use in the Locals, Watches, or Immediate windows:

An additional benefit is that since $exception is local to the scope in which your instruction pointer is, you can check the exception detail even if you’re looking at code elsewhere in the project, without having to go back and find the exact place where your exception was thrown.

A big thanks goes to this Stack Overflow answer for this handy little tip.

Software, Software development

A Dashboard for Microsoft Orleans

3 Comments

Introduction

While Microsoft Orleans has developed into a robust and scalable product over the years, the engagement between the Orleans team and the community around it is a spectacular example of the collaboration that open source projects should foster. On the one hand, the Orleans team members make themselves very available to help out with questions and issues that developers have when using Orleans. On the other hand, developers using Orleans have together built OrleansContrib, a collection of repositories adding unofficial functionality on top of what Orleans provides.

These repositories cover a variety of different areas: storage providers, logging and telemetry, documentation on design patterns, virtual meetups… and, of course, the Orleans Dashboard. This dashboard is a great way to monitor the activity of your silos and the grains within them.

Dashboard Overview

At a glance, the dashboard gives you a brief summary of your active silos, and the grain activations within them.

In the Grains section, you get an overview of the total grain activations in your silos, and a breakdown of each grain type. For each type of grain, you can see statistics on the number of activations, the rate of exceptions, throughput, and latency. In this case I haven’t actually created any grains, so all you can see are the system grains and the ones created by the Dashboard itself; however the data you see here will be a lot more interesting once you use it to monitor the activity and behaviour of your actual grains.

You can home in on an individual grain type. Aside from the statistics mentioned earlier, you also get detailed statistics on throughput, latency and failed requests per method.

At the bottom of the same page detailing a single type of grain, you can see a list of activations of that grain by silo.

Moving on, in the Silos section, you can see a summary of your active silos.

When you click on a silo, it gives you a more detailed view. At the top, you can see a graphical view showing resource utilisation: CPU, memory, and grains.

At the bottom, there are sections showing Silo Counters (number of clients, and messages sent and received), Silo Properties (information about the silo and its configuration), and a list of grain activations by type in the silo.

Adding the Dashboard to an Orleans silo

The Orleans Dashboard GitHub page explains how to set up and configure the dashboard. The first thing you need to do is install a NuGet package:

Install-Package OrleansDashboard

Then you will need to add an entry in the silo configuration to enable the dashboard. This can be done either using the XML configuration, or programmatically in code.

If you’re using a Dev/Test Host project to play with Orleans, it’s probably easier to do this in code. Find the file OrleansHostWrapper.cs, and after adding using OrleansDashboard; at the top, add the highlighted line below to register the dashboard:

            var config = ClusterConfiguration.LocalhostPrimarySilo();
            config.AddMemoryStorageProvider();
            config.Globals.RegisterDashboard();
            siloHost = new SiloHost(siloName, config);

If, on the other hand, you have a properly partitioned set of projects (as in “Getting Started with Microsoft Orleans“) and are using an OrleansConfiguration.xml file for your silo’s configuration, then just add an entry for the dashboard under the <Globals> node:

<?xml version="1.0" encoding="utf-8"?>
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <SeedNode Address="localhost" Port="11111" />
    <BootstrapProviders>
      <Provider Type="OrleansDashboard.Dashboard" Name="Dashboard" />
    </BootstrapProviders>
  </Globals>
  <Defaults>
    <Networking Address="localhost" Port="11111" />
    <ProxyingGateway Address="localhost" Port="30000" />
  </Defaults>
</OrleansConfiguration>

The dashboard runs by default at localhost:8080. If you want, you can change the port, or add basic username/password security. See the Orleans Dashboard page for an example showing how to configure these.

Summary

The Orleans dashboard is a great way to get detailed information about the silos in an Orleans cluster, and the grains within those silos. With detailed statistics like throughput, latency and failed requests, it is an invaluable tool to not only monitor the smooth operation of the cluster, but also to troubleshoot errors and performance bottlenecks.

Setting up the dashboard involves simply installing a NuGet package and then adding some very simple configuration to enable it. Additional configuration to change the port or add username/password protection is also possible.

The Orleans Dashboard homepage claims that:

“This project is alpha quality, and is published to collect community feedback.”

I’d argue that it’s pretty damn impressive for something that claims to be alpha quality.

See also: Orleans Virtual Meetup #11: A monitoring and visualisation show with Richard Astbury, Dan Vanderboom and Roger Creyke (13th October 2016).

Software

Setting Up Elasticsearch and Kibana on Windows

2 Comments

Elasticsearch is fantastic to index your data so that it can be searched by its lightning-fast search engine. With Kibana, you also get the ability to analyse and visualise that data. Both of these products are provided for free by Elastic.

Installing Java Runtime Environment

Elastic products are developed in Java, so you’ll need the Java Runtime Environment (JRE) to run them. Get the latest JRE from the relevant ugly Oracle downloads page. Either use the .exe installer, or download the .zip file and then extract the folder inside.

Either way, take note of the JRE folder location and add it as an environment variable. To do this, hit the start menu and type “environment variables“:

In the window that comes up, go on Environment Variables…:

You will now see the user and system environment variables. Hit New… under the System variables:

Name it JAVA_HOME, and in the value put in the path to the JRE folder (not its bin folder):

You can now OK out of the various dialog windows.

Setting up Elasticsearch

Go to the Elasticsearch product page, and hit Download:

In the next page, download the ZIP file:

Extract the folder in the .zip file somewhere.

You can now run elasticsearch.bat. If you get “The syntax of the command is incorrect”, you probably didn’t set the JAVA_HOME environment variable as explained in the previous section.

elasticsearch.bat

Running this command, you should see a bunch of initialisation output:

…and if you browse to localhost:9200, you should see some JSON returned:

Now that we know it’s working, we can install it as a Windows service. So press Ctrl+C to kill the instance of Elasticsearch you just ran, and instead run:

elasticsearch-service.bat install

This should install it as a service:

This installs it as Manual startup type, and does not start it. You probably want to change that to Automatic (Delayed Start), from the Services window in Microsoft Windows, and also Start it. Once you have done that, give it a few seconds to start, and then verify again that you get a response from localhost:9200.

Setting up Kibana

Next, we will set up Kibana. Grab the Windows .zip file from the Kibana downloads page:

Extract it wherever your heart desires.

Make sure Elasticsearch is running. Then, in Kibana’s bin folder, run kibana.bat:

kibana.bat

Some text will be written to the console as Kibana is initialised, and then you should be able to go to localhost:5601 and actually get a webpage:

Now we know that it works. Let’s set it up as a service. Kill the instance we just ran using Ctrl+C first.

Oh crap, Kibana does not come with a service installer! What are we gonna do?

Enter NSSM, the Non-Sucking Service Manager, which we can use to install just about any application as a Windows service, using either the command line or an interactive GUI. After downloading NSSM, we can install Kibana as a Windows service with a command like the following from NSSM’s win64 folder:

nssm install "Kibana 5.2.2" C:\[...]\kibana-5.2.2-windows-x86\bin\kibana.bat

With an elevated command prompt, we can also configure the Windows service, such as setting the startup type and the description:

nssm set "Kibana 5.2.2" Start "SERVICE_DELAYED_AUTO_START"
nssm set "Kibana 5.2.2" Description "Kibana lets you visualize your Elasticsearch data"

Finally, we start the service:

nssm start "Kibana 5.2.2"

If all goes well:

…then we can go back to localhost:5601 and verify that it’s really running.

With that, it’s all set up. All that’s needed is an index with some data that you can use Kibana to visualise, but that’s beyond the scope of this article.