Category Archives: Software development

ADO .NET Part 2: Basic Queries

In this article, we’ll cover the most important aspects of running queries using ADO .NET. This continues from Part 1, in which we discussed ADO .NET and saw how to connect to a database.

A Stellar Example

To demonstrate the capabilities of ADO .NET, we’ll need a database with at least one table. We’ll focus on SQL Server for the time being, and talk about how to connect to different database engines in Part 3.

Therefore, in SQL Server Management Studio, create a database called Cosmos and run the following query to create a table called Star and populate it with a few rows:

create table Star(
    Id int not null primary key identity(1, 1),
	Name nvarchar(50) not null
);

insert into Star (Name) values
    ('Aldebaran'),
	('Sirius'),
	('Betelgeuse');

Running A Query

In ADO .NET, there are three things we need to do to execute a query against a database:

  1. Open a connection
  2. Create an SQL command (i.e. the query)
  3. Execute that command and get the results

We’ve actually already done the first step in Part 1, so we can start with the same code, just with an updated connection string to specify that we will actually be connecting to the Cosmos database that we created in the previous section:

            const string connStr = @"Server=.\SQLEXPRESS;
                Initial Catalog=Cosmos; Integrated Security=True";

            using (var conn = new SqlConnection(connStr))
            {
                conn.Open();
                Console.WriteLine("Connected!");
            }

After opening the connection, we create our query. Here’s one way to do it:

            using (var conn = new SqlConnection(connStr))
            {
                conn.Open();

                const string query = "select * from Star";

                using (var command = new SqlCommand())
                {
                    command.Connection = conn;
                    command.CommandText = query;
                    command.CommandType = System.Data.CommandType.Text;

                    // TODO execute query
                }
            }

We specify the connection that the command will use to execute the query. We also provide the SQL query in the CommandText property, given that CommandType is Text (which applies for regular SQL queries). If you wanted to execute a stored procedure (SP), you would set CommandType to StoredProcedure and put the name of the SP in the CommandText property instead.

But since Text is actually the default value of CommandType, and we can provide the CommandText and Connection in SqlCommand‘s constructor, then we can write this much more concisely as follows:

            using (var conn = new SqlConnection(connStr))
            {
                conn.Open();

                const string query = "select * from Star";

                using (var command = new SqlCommand(query, conn))
                {
                    // TODO execute query
                }
            }

There is also a third way to create the SqlCommand which we’ll talk about in Part 3.

At this point, we can execute the query and get the results. For this particular query, this means calling ExecuteReader() and iterating over the results as follows:

            using (var conn = new SqlConnection(connStr))
            {
                conn.Open();

                const string query = "select * from Star";

                using (var command = new SqlCommand(query, conn))
                using (var reader = command.ExecuteReader())
                {
                    while (reader.Read())
                    {
                        int id = (int) reader["Id"];
                        string name = (string) reader["Name"];

                        Console.WriteLine($"{id}: {name}");
                    }
                }
            }

Each call to SqlDataReader.Read() will move to the next row, and return false when there are no more, breaking the loop. With each iteration, you can extract column values in a few different ways, including:

  1. By column name, as shown above. I favour this method because I find it to be more readable, although there is a little more conversion work involved.
  2. By column number, via reader[columnNo], which returns an object and therefore also requires conversion.
  3. Using helper methods such as reader.GetInt32(columnNo). These handle data conversion for you, but they only take the column number as input, making them less readable.

In Part 4, we’ll see how to avoid this nonsense altogether by using Dapper.

The output of the code we have so far would be:

1: Aldebaran
2: Sirius
3: Betelgeuse

Asynchronous Queries

Queries against a database are a type of Input/Output (I/O) operation (like reading a file, making an HTTP request, etc), and as such they should be executed asynchronously. Fortunately, asynchronous versions of ADO .NET methods are available, and we only need to update the code in three places (other than Main() itself, of course):

        static async Task Main(string[] args)
        {
            const string connStr = @"Server=.\SQLEXPRESS;
                Initial Catalog=Cosmos; Integrated Security=True";

            using (var conn = new SqlConnection(connStr))
            {
                await conn.OpenAsync();

                const string query = "select * from Star";

                using (var command = new SqlCommand(query, conn))
                using (var reader = await command.ExecuteReaderAsync())
                {
                    while (await reader.ReadAsync())
                    {
                        int id = (int) reader["Id"];
                        string name = (string) reader["Name"];

                        Console.WriteLine($"{id}: {name}");
                    }
                }
            }
        }

Scalar Queries

Not all queries require you to iterate over multiple rows. For instance, the following query simply gives us back the value ‘Sirius‘:

select top 1 Name
from Star
order by Name desc;

The execution of such queries is simpler, and uses ExecuteScalarAsync() instead of ExecuteReaderAsync(). Here’s an example:

            using (var conn = new SqlConnection(connStr))
            {
                await conn.OpenAsync();

                const string query = @"select top 1 Name
                                       from Star
                                       order by Name desc; ";

                using (var command = new SqlCommand(query, conn))
                {
                    string name = (string) await command.ExecuteScalarAsync();
                    Console.WriteLine(name);
                }
            }

We essentially threw away the third using block because it’s not necessary for a query that returns a single scalar value. However, ExecuteScalarAsync() returns Task<object>, which means that a cast is necessary (especially if the data returned is a numeric type — for a string you could get away with just calling ToString()).

Non Queries

There is another category of queries that return nothing at all: inserts, updates and deletes. Technically, although we tend to use the term “query” for any SQL statement, these aren’t really queries at all, because they aren’t giving us back any data. In fact, we execute them in ADO .NET by calling ExecuteNonQuery(). Take a look:

            using (var conn = new SqlConnection(connStr))
            {
                await conn.OpenAsync();

                const string query = @"insert into Star (Name) values ('Vega');";

                using (var command = new SqlCommand(query, conn))
                {
                    int rows = await command.ExecuteNonQueryAsync();
                    Console.WriteLine($"{rows} row(s) affected.");
                }
            }

Calling ExecuteNonQueryAsync() gives us back a Task<int>, representing the number of rows inserted, updated or deleted. This can be useful to determine whether the query was effective, but is not strictly necessary, so you could also ignore the return value if you don’t need it.

Summary and Next Steps

After setting up a very simple example of a database table, we saw how to execute different types of queries: those giving us rows of data, those giving us back a single scalar value, and those that manipulate the table data and do not give us any data back. We’ve also seen how to run these queries asynchronously.

However, we haven’t yet talked about parameterised queries, transactions, or how to connect to databases that aren’t SQL Server. In fact, we’re about to cover these in Part 3.

Also, it’s worth mentioning that while we haven’t seen any exception handling in the code samples for the sake of brevity, it needs to be present in your production code.

ADO .NET Part 1: Introduction

Hi everyone, and welcome to this mini-series on ADO .NET, a fundamental API that lets us connect to and work with relational databases from our .NET code. It’s been around since the earliest versions of the .NET Framework.

What is ADO .NET?

The meaning of “ADO” is not really important, since it comes from a bygone era when dinosaurs ruled the land. What’s important is that it lets us connect to a database, run queries, update data, execute stored procedures, manage transactions, and all the things you would expect to do when working with a relational database. The part we’ll be covering is called the connected layer.

There is another aspect called the disconnected layer which we won’t be covering here. Using things like DataSets and DataTables, it essentially allows you to work on an in-memory copy of part of your database, and handles connections for you. Since it needs to model database schema constructs, it ends up using a lot more memory than simply retrieving your data using the connected layer.

Why Bother?

Over the course of my career, I occasionally found myself teaching ADO .NET to colleagues who fell into one of the following categories:

  • They just graduated, so they haven’t had the chance to learn ADO .NET yet
  • They’ve been using Entity Framework all along, and never needed ADO .NET
  • They’ve been living under a rock

ADO .NET is worth learning because it is the core database connectivity technology for .NET. Object Relational Mappers (ORMs) like Entity Framework (EF) or NHibernate use ADO .NET underneath. ADO .NET is very easy to use, as we shall see in this mini-series, and gives you all the flexibility and performance you can get. This means you can resort to it even for things that ORMs can’t handle.

As it happens, nowadays, it seems like many people automatically resort to Entity Framework by default, without considering the tradeoffs or alternatives. Entity Framework has quite a learning curve and there is a lot you might be paying in terms of performance if you don’t learn to tame it and work in the way it expects. This might be overkill if I just want to run some queries against a database.

Consider this: with Entity Framework I have to at a minimum translate the query from SQL to LINQ or lambda syntax, and then translate it back to SQL every time I need to troubleshoot or optimise it. I don’t see what we gain by representing SQL in OOP style. If anything, it’s a leaky abstraction (try using string.IsNullOrEmpty() with Entity Framework 6) with a lot of baggage.

There are two main things that Entity Framework can help us with: keeping hardcoded SQL strings out of our logic, and mapping result data to our OOP models. The former can easily be achieved by using a repository pattern (which you should be doing anyway even with EF), and the latter can be facilitated by a micro-ORM called Dapper, as we’ll see in the last article in this mini-series.

So while Entity Framework or other ORMs might have their appeal, and ADO .NET might sound like something ancient and low-level, it’s actually helpful to understand this core technology, add it to your toolkit, and understand how it compares to similar technologies.

Opening a Connection

Enough talk! Let’s open a connection against a database. Assuming we’re working with SQL Server (we’ll see how to work with other databases later in this mini-series), we’ll need the following using statement:

using System.Data.SqlClient;

The above namespace is part of the .NET Framework (so no extra steps necessary there), but if you’re using .NET Core, you’ll need to install the System.Data.SqlClient package first.

            const string connStr = @"Server=.\SQLEXPRESS; Integrated Security=True";

            using (var conn = new SqlConnection(connStr))
            {
                conn.Open();
                Console.WriteLine("Connected!");
            }

The above code (actually, just the two highlighted lines) is all you need to establish a connection to a local SQL Server Express instance (no specific database) using the default Windows Authentication. If it’s successful, you’ll see “Connected” written to the console window. Otherwise, it will throw an exception. We’re not doing any exception handling for the sake of brevity, but obviously, you should have that in real production code. Also, Open() has an asynchronous counterpart (OpenAsync()) which we’ll talk about later in this mini-series.

This trivial example is not only a starting point that we will build on in the next articles. It also stands on its own as a simple application to test basic database connectivity when no other tools are available.

Connection Pooling

In the connected layer of ADO .NET, we obviously need to manage connections, and it might not be immediately evident how this should be done.

The good news is that it’s easier than you might think. ADO .NET internally has a connection pooling mechanism. This means that connections you close aren’t necessarily closed (at least not immediately) and can internally be reused, avoiding certain overheads with opening and closing connections.

This means that you’re encouraged to open and close connections as much as you need, especially because ADO .NET objects such as SqlConnection (and others we’ll see later) are not thread-safe; thus reusing instances can lead to severe problems in environments such as web applications where multiple requests may need to access the database concurrently.

Summary and Next Steps

In this article, we’ve discussed why it is useful to learn ADO .NET if you don’t know it already, and we’ve limited the scope to the connected layer only. This mini-series is meant to teach you the essentials as concisely a possible, and is not intended to be a comprehensive reference.

In the second half of the article, we saw how easy it is to open a connection, and discussed the connection pooling mechanism that ADO .NET automatically applies whenever we create or dispose of a connection object.

This lays the foundation for the next article, in which we’ll learn how to run different types of queries.

Visual Studio Bug with Spaces in Path

About a year ago, I ran into a weird bug. After having cloned a repository from Azure DevOps, I was baffled to see that the Solution Explorer in Visual Studio was not showing any files, even though there were plenty:

This problem still annoys my colleagues to this day. What happened in every case was that there was a space in the name of the project in Azure DevOps, so cloning the repo resulted in a folder that included %20 in the name. As I’m sure you’re aware, that’s the URI-encoded version of a space.

Unfortunately, for whatever reason, Visual Studo does not like this very much. It’s not a big deal though. Simply change that %20 to an actual space, and the files suddenly show up in Solution Explorer:

This problem seems to apply only to .NET Core projects. I haven’t been able to replicate it for older .NET Framework projects.

SDL2 Drag and Drop

Hi everyone! It’s been a while since my last SDL2 article (see all SDL2 articles). Today, I’m going to show how you can use the mouse to drag an object across an SDL2 window.

Drag and drop is nowadays a standard feature of any Multiple Document Interface (MDI). Although MDI as a user interface layout has fallen out of fashion in mainstream applications (even GNOME seems to think we don’t need it), it has enabled so many things ranging from windows in operating system GUIs to inventory containers in games.

The source code for this article is available in the SDL2DragDrop folder at the Gigi Labs BitBucket repository.

Displaying an Empty Window

Let’s start out by displaying an empty window. We can use the code below for this:

#include <SDL2/SDL.h>
 
int main(int argc, char ** argv)
{
    // variables
    
    bool quit = false;
    SDL_Event event;
    
    // init SDL
    
    SDL_Init(SDL_INIT_VIDEO);
    SDL_Window * window = SDL_CreateWindow("SDL2 Drag and Drop",
        SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 640, 480, 0);
    SDL_Renderer * renderer = SDL_CreateRenderer(window, -1, 0);
    
    // handle events
    
    while (!quit)
    {
        SDL_Delay(10);
        SDL_PollEvent(&event);
    
        switch (event.type)
        {
            case SDL_QUIT:
                quit = true;
                break;
        }
    
        SDL_SetRenderDrawColor(renderer, 242, 242, 242, 255);
        SDL_RenderClear(renderer);
        
        SDL_RenderPresent(renderer);
    }
    
    // cleanup SDL
    
    SDL_DestroyRenderer(renderer);
    SDL_DestroyWindow(window);
    SDL_Quit();
    
    return 0;
}

Most of this should be familiar from “Showing an Empty Window in SDL2“, but there are a few differences:

  • The #include on the first line is different from that of many of my previous SDL2 articles. That’s because I’m now using SDL2 on Linux (see “How to Set Up SDL2 on Linux“).
  • We’re using SDL_RenderClear() to give the window a background colour (as we’ve done in a few earlier articles).
  • We’re using SDL_PollEvent() to check for events all the time (combined with SDL_Delay() to occasionally give the CPU a break). This is also something we’ve done a few times before.

On Linux, assuming my file is called main.cpp, I can compile this from the command-line as follows:

g++ main.cpp -lSDL2 -lSDL2main -o sdl2exe

Just to be super clear (as this has been a point of confusion in earlier articles), I’m using the C++ compiler. We’ll be using a couple of C++ features, so please don’t try to compile this article’s code as-is with a C compiler.

Once this compiles successfully, I can run the executable that was produced as a result:

./sdl2exe

…and I get an empty window:

Adding Rectangles

We’re going to need something we can drag around, so let’s add a couple of rectangles. Again, we’ve done this before (see “SDL2 Bounding Box Collision Detection“), but we’ll improve a little by storing our rectangles in a C++ STL list.

First, add the necessary #include at the top of the file:

#include <list>

With this in place, we can now create a couple of SDL_Rects and put them in a list. The following code goes before the event loop.

    SDL_Rect rect1 = { 288, 208, 100, 100 };
    SDL_Rect rect2 = { 50, 50, 100, 80 };
    
    std::list<SDL_Rect *> rectangles;
    rectangles.push_back(&rect1);
    rectangles.push_back(&rect2);

Towards the end of the event loop, we can add code to draw those rectangles. Since we’re using a list, we can add more rectangles if we want, and this part of the code isn’t going to change. Note that I’m also using a little C++11 shortcut to iterate through the list.

        SDL_SetRenderDrawColor(renderer, 242, 242, 242, 255);
        SDL_RenderClear(renderer);
        
        for (auto const& rect : rectangles)
        {
            SDL_SetRenderDrawColor(renderer, 0, 255, 0, 255);
            SDL_RenderFillRect(renderer, rect);
        }
        
        SDL_RenderPresent(renderer);

If we compile and run this now, we get a couple of green rectangles:

Changing Colour on Click

Before we get to actually dragging those rectangles, we need a way to select one when it gets clicked, both in code and visually. For this, we’ll add a selectedRect variable somewhere before the event loop:

SDL_Rect * selectedRect = NULL;

By default, no rectangle is selected, which is why this is set to NULL.

We also need a couple more variables somewhere at the beginning of the program to help us keep track of mouse events:

    bool leftMouseButtonDown = false;
    SDL_Point mousePos;

In the switch statement within the event loop, we can now start adding mouse event handlers. We’ll start with one for SDL_MOUSEMOTION, which keeps track of the mouse coordinates in the mousePos variable we just declared:

            case SDL_MOUSEMOTION:
                mousePos = { event.motion.x, event.motion.y };
                break;

Adding another event handler for SDL_MOUSEBUTTONUP, we clear the leftMouseButtonDown flag and the selected rectangle when the left mouse button is released:

            case SDL_MOUSEBUTTONUP:
                if (leftMouseButtonDown && event.button.button == SDL_BUTTON_LEFT)
                {
                    leftMouseButtonDown = false;
                    selectedRect = NULL;
                }
                break;

Finally, we add another event handler for SDL_MOUSEBUTTONDOWN, which uses SDL_PointInRect() to identify the rectangle being clicked (if any), and sets it as the selected rectangle:

            case SDL_MOUSEBUTTONDOWN:
                if (!leftMouseButtonDown && event.button.button == SDL_BUTTON_LEFT)
                {
                    leftMouseButtonDown = true;
                    
                    for (auto rect : rectangles)
                    {
                        if (SDL_PointInRect(&mousePos, rect))
                        {
                            selectedRect = rect;
                            break;
                        }
                    }
                }
                break;

At this point, we can add some conditional logic in the rendering code to draw the selected rectangle (i.e. the one being clicked) in blue instead of green:

        SDL_SetRenderDrawColor(renderer, 242, 242, 242, 255);
        SDL_RenderClear(renderer);
        
        for (auto const& rect : rectangles)
        {
            if (rect == selectedRect)
                SDL_SetRenderDrawColor(renderer, 0, 0, 255, 255);
            else
                SDL_SetRenderDrawColor(renderer, 0, 255, 0, 255);
            
            SDL_RenderFillRect(renderer, rect);
        }
        
        SDL_RenderPresent(renderer);

If we compile and run this now, we find that rectangles become blue when clicked:

Drag and Drop

That might have seemed like a lot of work just to highlight a rectangle when clicked, but in fact we have already implemented much of what we need for drag and drop. To finish the job, we’ll start by adding a new variable near the beginning of the program:

SDL_Point clickOffset;

This clickOffset variable will store the point within the rectangle (relative to the rectangle’s boundary) where you clicked, so that as we move the rectangle by dragging, we can keep that same spot under the mouse pointer.

In fact, when the left mouse button is pressed, we will now store this location so that we can use it later in the code:

            case SDL_MOUSEBUTTONDOWN:
                if (!leftMouseButtonDown && event.button.button == SDL_BUTTON_LEFT)
                {
                    leftMouseButtonDown = true;
                    
                    for (auto rect : rectangles)
                    {
                        if (SDL_PointInRect(&mousePos, rect))
                        {
                            selectedRect = rect;
                            clickOffset.x = mousePos.x - rect->x;
                            clickOffset.y = mousePos.y - rect->y;
                            
                            break;
                        }
                    }
                }
                break;

Then, while the mouse is moving, we update the selected rectangle’s position accordingly:

            case SDL_MOUSEMOTION:
                {
                    mousePos = { event.motion.x, event.motion.y };
                    
                    if (leftMouseButtonDown && selectedRect != NULL)
                    {
                        selectedRect->x = mousePos.x - clickOffset.x;
                        selectedRect->y = mousePos.y - clickOffset.y;
                    }
                }
                break;

…and that is all that is needed to allow the user to click and drag those rectangles:

Wrapping Up

In this article, we’ve mostly built on concepts covered in earlier articles. By manipulating mouse events, we were able to add interactivity to rectangles. They change colour when clicked, and can be dragged around using the mouse.

There are a few things you’ll notice if you’re diligent enough:

  • If you drag the rectangles around really fast, they lag a little behind the mouse pointer. I’m not really sure how to fix that.
  • There isn’t really a proper z-index implementation, so it looks bizarre when you can drag a rectangle underneath another. This can probably be fixed by changing the order in which the rectangles are rendered, so that the selected one always appears on top of the rest.
  • Using a list is okay for a few items, but if you have a lot of objects in your window, you might want to use a more efficient data structure such as a quadtree.

OData Session at Virtual APIdays Helsinki 2020

With RedisConf2020 done and dusted, my next stop is API Days Helsinki 2020. Like RedisConf, this is a conference that got its plans a little messed up by COVID19. Instead of the originally planned conference, they’re doing a virtual event in June, and (hopefully) a physical one in September.

I’m going to be presenting a session on OData on 2nd June, during the virtual event. If you haven’t heard about it before, you can think of it as something that tried to solve similar problems to what GraphQL did, but much earlier. GraphQL is not OData, as some will be quite willing to point out, and I’ll be explaining some of the tradeoffs between the two.