Category Archives: Software development

From .NET to GoLang: Here We Go Again

Although I’ve been working with Go for less than a year, there are a number of things I’ve learned or noticed since I wrote “From .NET to GoLang: Where Did Everything Go?“, my critique of the Go programming language and ecosystem, three and a half months ago.

As a result, I’d like to use this followup article to discuss some new things that bother me as well as revisit points I already mentioned in the original article.

This is also the 300th article published at Gigi Labs. Hooray!

Productivity

I ended “From .NET to GoLang: Where Did Everything Go?” on a somewhat negative note:

“Coming from C#, learning my way around Go has been a fun diversion, but also something of a disappointment. Given the simplistic, verbose and sometimes confusing language, and the limited standard library, there’s no reason I can think of to choose Go for a new project over something richer and more mature like C#, which nowadays runs on any platform.”

As a result, it is somewhat surprising that I actually feel a lot more productive with Go than I ever did with C#, despite C# being the superior language in my opinion. I think the main reason for this is that .NET software tends to be extremely overcomplicated, owing to a combination of excessive OOP, layers of abstraction, design patterns, and obscenely exploited third party libraries (see “Pitfalls of AutoMapper“) that overzealous software engineers anxious to prove their mettle tend to sprinkle all over the place. On the other hand, Go’s simplicity – the very thing I was complaining about – does not provide many opportunities for this kind of madness.

Stability

I also mentioned that Go’s development seems to be somewhat slow, with particular reference to generics having taken 12 years to appear in the language. But this is also in a way a good thing. Go is a very stable language and upgrading to newer versions is a breeze. I can’t say the same about C# any more, given that every year it’s become a hassle to keep up with the latest unnecessary language ‘features’ as well as breaking changes to ASP .NET Core.

Exception Handling

I talked about Go’s lack of exception handling and the need to have heaps of if statements to check errors returned by function calls.

Go does in fact have a mechanism similar to exception handling, in the form of panic and recover. But since there are no exception objects, this mechanism operates with basic types, just like how the error interface returns a string and does not otherwise carry structured error information. This is unfortunate because it means it’s impossible for Go to have first chance exceptions, which make the debugger pause as soon as an exception is thrown – an essential troubleshooting tool that I sorely miss from .NET.

Defer

The same blog post I linked about panic and recover also talks about defer, which is the closest thing Go has to .NET’s using block (to clean up resources that need disposing). In “Scope Bound Resource Management in C#“, I discuss many of the things you can do by using and abusing the cleanup mechanisms in C++ and C#, and you can do most if not all of the same things in Go with defer.

Exported Names

Go code is organised in packages (like namespaces), but it doesn’t have public or private accessors. Variables and functions can be shared between packages if they start with a capital letter.

Wait, what? Let’s see this in action. I made a really simple program with two packages. I call a function from the money package from main():

Make() can be called from main() because it starts with a capital letter.

But if we change the name to start with a lowercase letter, it’s no longer visible:

make(), on the other hand, can only be used by code inside the money package.

I’m not at all a fan of such magic conventions, since they are not obvious at all to newcomers (unlike a keyword such as public would be), in the same way that I’m also not a fan of convention over configuration (used for a long time in ASP .NET Web API, MVC and Core) and content before chrome (used by the disastrous Windows 8 “Metro” theme).

LINQ and Sets

In “From .NET to GoLang: Where Did Everything Go?“, I complained about the standard library’s lack of support for common data structures such as sets, as well as the lack of anything like LINQ, making Go rely a lot on loops and elementary data structures.

However, in both cases, the community has stepped in and filled the gaps. I recently wrote the article “GoLang Set Data Structure“, which describes how to use the third party golang-set library. There is also a go-linq library on GitHub which I haven’t tried yet. Although it’s likely very useful, it’s somewhat unfortunate that Go’s syntax makes its usage a lot less elegant than in C#.

Slices

Go has arrays and slices (the latter being an array-like data structure). You can get a sub-range (also called “slice”) of an array, slice or string just like you can in Python:

package main

import "fmt"

func main() {
	numbers := []int{1, 2, 3, 4, 5, 6}
	jonathan := "Jonathan"

	fmt.Println(numbers[1:3]) // [2 3]
	fmt.Println(jonathan[:3]) // Jon
}

Go, however, doesn’t allow negative indices:

package main

import "fmt"

func main() {
	numbers := []int{1, 2, 3, 4, 5, 6}

	fmt.Println(numbers[1:-3]) // compiler error
}

Whereas with Python you can use negative indices to denote an index starting from the end (handy when you want to e.g. trim off the last character of a string):

>>> numbers = [1, 2, 3, 4, 5, 6]
>>> numbers[1:-3]
[2, 3]

Achieving the same thing with Go is a little more verbose, which is not very surprising at this point:

package main

import "fmt"

func main() {
	numbers := []int{1, 2, 3, 4, 5, 6}

	fmt.Println(numbers[1 : len(numbers)-3]) // [2 3]
}

Else Braces

I did mention in the original article that Go is a little fussy about braces used with an if statement, both in terms of their necessity and placement. Where it gets really annoying, though, is when it fails to build because you put the else on the next line, like this:

package main

import "fmt"

func main() {
	if ("sky" == "blue") {
		fmt.Println("Not in Ireland")
	}
	else { // compiler error
		fmt.Println("Probably Ireland")
	}
}

The pedantic bastard requires you to put the else on the same line as the preceding brace, like this:

package main

import "fmt"

func main() {
	if "sky" == "blue" {
		fmt.Println("Not in Ireland")
	} else { // compiler is happy
		fmt.Println("Probably Ireland")
	}
}

Whatever makes you happy, dear Go compiler.

Syntax (Revisited)

I wrote at length in the original article about how I find Go’s variable declaration syntax to be very confusing at best. There’s an article on the Go blog explaining the reasoning behind Go’s Declaration Syntax, which is informative but frankly not very convincing.

I also wrote about my aversion to the := syntax. My feelings are not just a matter of style but also come from having been screwed a couple of times by this fancy operator. Let’s see a simple if somewhat absurd example:

package main

import "fmt"

func uptown() (string, error) {
	return "Uptown Funk!", nil
}

func main() {
	lyrics := "Eye of the Tiger"

	if "sky" != "fuchsia" {
		lyrics, err := uptown()
		if err != nil {
			// nobody cares, Bruno
		}
		fmt.Println(lyrics)
	}

	fmt.Println(lyrics)
}

What do you think the output will be? OK, I’ll tell you. It’s:

Uptown Funk!
Eye of the Tiger

I used the := operator because the err variable is new and needs declaring. But because of that same operator, the lyrics variable inside the if statement is an entirely new variable that shadows the one outside. So once we’re back outside the if statement, thinking we updated the value of the lyrics variable, we find that it still has its original value.

I’m not even sure what is the best way to deal with this situation, though an easy way to update the outer lyrics variable is to use an intermediate variable to capture the output of uptown() and then assign that to lyrics:

package main

import "fmt"

func uptown() (string, error) {
	return "Uptown Funk!", nil
}

func main() {
	lyrics := "Eye of the Tiger"

	if "sky" != "fuchsia" {
		innerLyrics, err := uptown()
		if err != nil {
			// nobody cares, Bruno
		}
		fmt.Println(innerLyrics)
		lyrics = innerLyrics
	}

	fmt.Println(lyrics)
}

Addressable Values

Let’s say I have a map whose value is a struct. For instance:

package main

type Person struct {
	Hobby string
}

func main() {
	people := map[string]Person{"James": {Hobby: "Drinking"}}

	people[i].Hobby = "Farting"
}

I create a map with a single value which maps the key “James” to a struct, and then I try to update a property on that struct. Seems simple enough? And yet, it doesn’t compile:

You cannot simply assign a value to a struct’s property in a map in Go. Sorry.

What?! Why? It seems like this boils down to this concept of Addressable values in Go, WHICH FRANKLY I DON’T CARE ABOUT BECAUSE NO OTHER LANGUAGE EVER HAD A PROBLEM UPDATING A PROPERTY LIKE THIS!

To update that value, as far as I can tell, you instead have to take the detour of replacing the entire object in the map:

func main() {
	people := map[string]Person{"James": {Hobby: "Drinking"}}

	updatedJames := Person{Hobby: "Drinking"}
	people["James"] = updatedJames
}

Dates & Times

In “GoLang Timestamps: A Cross-Platform Nightmare“, I wrote about a very specific problem I encountered with Go time. But why don’t we talk about something more common? Parsing timestamps.

Let’s try to parse a date/time in a way that any reasonable language would allow, i.e. using placeholders like “yyyy-MM-dd“:

You cannot simply parse a date in an intuitive manner either.

No, Go doesn’t use such placeholders. Why would it? Instead, it uses this crazy mnemonic device to represent the various parts of the date:

Mon Jan 2 15:04:05 MST 2006

If you look closely, you’ll notice that these are actually the numbers 1 through 7:

  • Jan (1) is the month
  • 2 is the day
  • 15 (3) is the hour
  • 4 is the minutes
  • 5 is the seconds
  • 2006 (6) is the year
  • MST (-7) is the time zone relative to UTC

So, the equivalent of parsing as “yyyy-MM-dd” would be something like:

You have to use the weird mnemonic thingy. But you have to chop off the extra text too.

Oops, that didn’t work either. I have to chop off the extra bits for it to work:

package main

import (
	"fmt"
	"time"
)

func main() {
	fmt.Println(time.Now()) // 2022-11-16 22:23:39.354436468 +0100 CET m=+0.000014317
	parsedTime, err := time.Parse("2006-01-02", "2022-11-16")
	if err != nil {
		fmt.Println("Failed to parse time: ", err)
	} else {
		fmt.Println("Parsed time: ", parsedTime)
	}
}
Finally. That was a little bit complicated for such a basic operation.

Fortunately, there are a few predefined layouts you can use, such as time.RFC3339. But the ubiquitous RFC 8601 is nowhere to be found, so you’ll likely have to beg Stack Overflow for that.

Conclusion

It’s certainly been educational, entertaining and occasionally frustrating to learn about Go’s quirks. And considering I haven’t been working with it that long, I’m sure there will be lots more to come!

GoLang Timestamps: A Cross-Platform Nightmare

Have you ever had one of those bugs where your tests failed in your Continuous Integration (CI) pipeline, but they worked just fine locally, and the only way to troubleshoot the problem was to add logs and wait significant amounts of time for each run to fail? Well, I had one of those this week, and I learned a thing or two in the process. I’m hoping to share that here for anyone else who might run into the same problem.

Time Precision Across Operating Systems

Let’s create a really simple Go program.

  1. Create a new folder.
  2. In that folder, run go mod init main
  3. Create a file called main.go and paste the following code:
package main

import (
	"fmt"
	"time"
)

func main() {
	fmt.Println(time.Now())
}

Nothing special, right? It just prints out the current time. Let’s run it then.

The following is the output on my personal laptop running Linux:

$ go run main.go
2022-10-27 17:34:43.471537065 +0200 CEST m=+0.000016185

And the following is the output on my work machine, which is a Mac:

$ go run main.go
2022-10-27 17:35:27.349114 +0200 CEST m=+0.000181459

See the difference? The Linux output was precise to the nanosecond (9 fractional digits of a second), whereas the Mac output was precise to the microsecond (6 fractional digits of a second). This subtle difference was the root cause of my problem.

How Did It Happen?

I had written automated tests to verify some simple CRUD functionality. I would send the data to an API, and the API would store it in a PostgreSQL database. Then, I would retrieve the data and verify that it matched what I had sent. Sounds trivial? Yes, but it failed. It failed on the CI server where I couldn’t debug the problem as easily as I could on my machine.

The reason it worked on my Mac was that a timestamp field in PostgreSQL has microsecond precision. Coincidentally, that’s the same precision that I get for Go time on my Mac.

However, on the CI system as well as on a colleague’s WSL system, things worked differently:

  1. The test data, as well as the expected values to be compared later, are generated in Go. Timestamps have nanosecond precision.
  2. The data is sent via the API and stored in the database. There’s a loss of precision and the timestamps are stored with microsecond precision.
  3. The test retrieves, via the API, the same data that has just been saved. The timestamps come back with microsecond precision.
  4. The automated test compares the expected values (nanosecond precision) with the actual values (microsecond precision) and finds that they are different. The test fails.

How Did I Resolve It?

Honestly, I don’t know what the right way to solve this problem is. It’s a pretty messy situation reminiscent of floating-point error, and that in fact influenced my solution.

The first thing I did was strip off the extra precision. Go time has this Truncate() function you can use for exactly this reason. For instance, let’s try the following:

func main() {
	fmt.Println(time.Now())
	fmt.Println(time.Now().Truncate(time.Microsecond))
}

The output on my Linux laptop then looks like:

$ go run main.go
2022-10-27 19:06:52.427287572 +0200 CEST m=+0.000013810
2022-10-27 19:06:52.427347 +0200 CEST

Doing this on my Mac would, I suppose, mean that the Truncate() is a no-op and just returns the same thing – at least that’s the behaviour I observed when testing it out.

However, in practice, Truncate() alone didn’t solve the problem. Some tests kept failing intermittently because the last digit (the microsecond) was off by one. I’m not sure what was causing this – perhaps some kind of rounding, e.g. when the data goes into the database.

So I used the same approach as with floating-point equality comparisons and decided to allow an error of one microsecond, i.e. if two timestamps are one microsecond apart, we assume they’re still equal. This is as simple as using Sub() to calculate the interval between two timestamps, and consider them equal if it’s not greater than one microsecond:

func timestampsEqual(a, b time.Time) bool {
	return b.Sub(a) <= time.Microsecond
}

Let’s test it out with some code:

func main() {
	t1 := time.Now().Truncate(time.Microsecond)
	t2 := t1.Add(time.Microsecond)
	t3 := t2.Add(time.Microsecond)

	fmt.Println("t1 == t2: ", timestampsEqual(t1, t2))
	fmt.Println("t1 == t3: ", timestampsEqual(t1, t3))
}

And the output would be:

$ go run main.go
t1 == t2:  true
t1 == t3:  false

So, it’s not great, and I’m sure there are ways to improve, but it seems to have solved my problem at least. One thing I’d like to do is to use some equivalent of math.abs() so that the order of the timestamps doesn’t matter. But, I have no idea how to do that with Go time with out-of-the-box functionality.

Calculating Circle Area the Monte Carlo Way

Ten years ago, I was in the middle of my MSc degree, titled “Distributed High-Fidelity Graphics Using P2P”. My research revolved around ray tracing – a family of techniques used to produce highly photorealistic images by simulating various aspects of Physics, especially the way light interacts with the various surfaces and materials it comes in contact with.

Ray tracing is very complex and computationally intensive, and it would take an entire book to write anything substantial about it. But, at the heart of modern ray tracing is an approach called Monte Carlo simulation, or stochastic simulation. It’s a fancy way of saying: this problem is too complex and I don’t have an easy way to calculate a solution, so I’ll take a number of random samples and approximate it instead.

In this article, I’ll illustrate the concept by calculating the area of a circle. Since we have an easy formula to calculate the area of a circle (πr2), there’s normally no need to do this via Monte Carlo simulation. However, it’s an easy way to explain the concept, which you can then use to solve more complex problems.

Approximating the Area of a Circle in Theory

Let’s assume that we don’t know how to calculate the area of a circle – either because a formula does not exist, or because it is too complex to derive.

However, we do have a way to determine whether a point lies inside a circle. If the point is represented by the coordinates (x, y), then the point lies inside the circle of radius r if (x2 + y2) <= r2, assuming that the circle’s centre is located at (0, 0).

A visualisation of a Monte Carlo approximation of a circle.

In this case, we can take a number of random points in space and compare how many fall inside and outside of the circle. You can see a simple visualisation of this above. Imagine we were throwing darts at a dartboard while blindfolded. The more darts we throw, the more we are filling in the circle, and the ratio of darts inside vs outside the dartboard tells us the overall area we covered.

It’s a little counterintuitive to imagine how a random process leads towards a deterministic solution. In fact, it’s not really deterministic, and the actual result changes with every run; but the approximation does work very well, and the more samples you take, the closer you get to the real solution. Let’s test this out in practice.

Approximating the Area of a Circle in Python 3.8

The code to implement the Monte Carlo approximation of the area of a circle is quite straightforward, as you can see below.

import random

def get_random_coord(radius):
    return random.uniform(-radius, radius)

def calculate_area_monte_carlo(radius):

    total_in_circle = 0

    for i in range(0, 10000000):
        (x, y) = get_random_coord(radius), get_random_coord(radius)
        in_circle = x ** 2 + y ** 2 <= radius ** 2
        
        if in_circle:
            total_in_circle += 1

        if i in [9999, 99999, 150000, 999999, 1500000, 9999999]:
            approximate_area = (2 * radius) ** 2 * total_in_circle / i
            print(f"Area ({i} samples): {approximate_area}")

Essentially, all I’m doing is:

  • Defining a helper function get_random_coord() to generate a random coordinate, just so that I don’t have to repeat the same code twice.
  • Generating an (x, y) coordinate pair within the square containing the circle.
  • Determining whether this (x, y) falls within the circle, using the aforementioned formula (x2 + y2) <= r2, in which case I increment total_in_circle.
  • At specific (arbitrary) intervals (numbers of samples), I calculate the approximate area of the circle and print it out to show how the calculated area gets more accurate, the more samples you take.

The approximate area is calculated as a ratio of samples in circle (total_in_circle) vs total samples both inside and outside of the circle (i). However, we have to adjust that by the area of a square, which is 2r2, as explained in this Stack Overflow answer.

In reality, we do know a formula to accurately calculate the area of a circle (πr2), so let’s toss that in as well in order to compare how accurate the results are:

import random
import math

def get_random_coord(radius):
    return random.uniform(-radius, radius)

def calculate_area_monte_carlo(radius):

    total_in_circle = 0

    for i in range(0, 10000000):
        (x, y) = get_random_coord(radius), get_random_coord(radius)
        in_circle = x ** 2 + y ** 2 <= radius ** 2
        
        if in_circle:
            total_in_circle += 1

        if i in [9999, 99999, 150000, 999999, 1500000, 9999999]:
            approximate_area = (2 * radius) ** 2 * total_in_circle / i
            print(f"Area ({i} samples): {approximate_area}")

def calculate_area_formula(radius):
    area = math.pi * radius ** 2
    print("Area (reference):", area)

def main():
    radius = 400
    calculate_area_monte_carlo(radius)
    calculate_area_formula(radius)

if __name__ == "__main__":
    main()

We can now run this program, and after a little patience, we get some results:

$ python3 main2.py
Area (9999 samples): 501874.1874187419
Area (99999 samples): 503793.8379383794
Area (150000 samples): 503940.26666666666
Area (999999 samples): 502511.2225112225
Area (1500000 samples): 502637.2266666667
Area (9999999 samples): 502743.410274341
Area (reference): 502654.8245743669

If we run it again, the results are a little different, but the trend is still there:

$ python3 main2.py
Area (9999 samples): 506930.69306930696
Area (99999 samples): 501861.0186101861
Area (150000 samples): 502101.3333333333
Area (999999 samples): 502941.30294130294
Area (1500000 samples): 502856.1066666667
Area (9999999 samples): 502779.44227794424
Area (reference): 502654.8245743669

This is the essence of the Monte Carlo approach: the more random samples you take, the more you’re filling in the problem space. However, this comes at a cost: the more samples you take, the more time it takes to compute. With enough samples, the error eventually becomes small enough that the approximation is indistinguishable from an accurate solution. Let’s visualise this.

Visualising the Area of a Circle Approximation

We can use the PIL library to quickly generate progressive images of the circle coverage. All we need is to add a few lines to our existing code:

import random
import math
from PIL import Image

def get_random_coord(radius):
    return random.uniform(-radius, radius)

def calculate_area_monte_carlo(radius):
    diameter = radius * 2
    image = Image.new("RGB", (diameter, diameter), (255, 255, 255))
    in_colour = (255, 0, 0) # red
    out_colour = (0, 0, 255) # blue

    total_in_circle = 0

    for i in range(0, 10000000):
        (x, y) = get_random_coord(radius), get_random_coord(radius)
        in_circle = x ** 2 + y ** 2 <= radius ** 2
        
        if in_circle:
            total_in_circle += 1
            colour = in_colour
        else:
            colour = out_colour

        image.putpixel((math.floor(x + radius), math.floor(y + radius)), colour)

        if i in [9999, 99999, 150000, 999999, 1500000, 9999999]:
            approximate_area = (2 * radius) ** 2 * total_in_circle / i
            print(f"Area ({i} samples): {approximate_area}")
            image.save(f'circle_{i}.png')

def calculate_area_formula(radius):
    area = math.pi * radius ** 2
    print("Area (reference):", area)

def main():
    radius = 400
    calculate_area_monte_carlo(radius)
    calculate_area_formula(radius)

if __name__ == "__main__":
    main()

What we’re doing here is:

  • Creating an image at the beginning.
  • For each sample, we colour the corresponding pixel either red (if it’s in the circle) or blue (if it’s outside the circle). Note that we have to add the radius to x and y because in the image, (0, 0) is not the centre of the circle, but the upper-left corner of the image.
  • Saving an image whenever we print an approximate area.

Earlier, we saw how the numbers seemed to be converging towards the accurate solution. If we now run the updated code above, we can verify the progressive coverage of the circle from actual images (click to enlarge):

Applications

As I said earlier, the Monte Carlo approach is not necessary for problems where we have a simple formula, like calculating the area of a circle. However, not all problems have a simple formula.

If you’ve studied calculus, you’ll know that you can integrate a function to calculate the area under its graph. You can do that for a simple function like 5x2, but there are more complex functions for which it is impossible to derive an analytical solution. Even if you never studied calculus, you can imagine how hard it must be to calculate the area of an irregular shape, such as your favourite politician. Randomly throwing darts at them and counting how many hit might be one way of working that out.

And if you ever work with ray tracing, you’ll eventually realise that ray tracing is really just another integration problem – one where you have to calculate the area (or contribution of light towards a surface) of something very irregular. A Monte Carlo approximation is a very effective – if computationally expensive – way of solving this otherwise intractable problem.

GoLang Set Data Structure

One of my complaints about Go in my recent article, “From .NET to GoLang: Where Did Everything Go?“, was the lack of built-in data structures, such as a set.

Many people suggest writing your own, and there are websites that show you how to do this. Of course, you can use a map to represent a set, but you’d also have to implement common operations such as union, intersection and difference.

Fortunately, there’s no need to reinvent the wheel like this, because the community has stepped in to provide a good set implementation. The golang-set package provides all the functionality expected of a set data structure, and its v2 release makes full use of the (recently released) Go generics functionality, allowing you to use it with virtually any data type.

I’m expecting the reader to know what a set is (i.e. an unordered, deduplicated collection) and what the common operations are. I’ll be focusing on how to work with sets in Go using the aforementioned package.

Installing golang-set

First, as usual, set up a Go module:

go mod init main

Then, to install golang-set in your Go project, make sure to get the v2 release, as follows:

go get github.com/deckarep/golang-set/v2

Then in your Go file (e.g. main.go) be sure to import the package as in the following example:

package main

import (
    mapset "github.com/deckarep/golang-set/v2"
)

func main() {
	// test code will Go here
}

Initialising a Set

You create a new Set by calling the NewSet() function. For instance, for a set of strings:

fruit := mapset.NewSet[string]()

Since the Set type supports generics, you can use a different type instead of a string, for instance, we could have a set of integers:

oddNumbers := mapset.NewSet[int]()

While we’ll soon see how to add values to our set, it’s possible to initialise a set with values when you create it. For instance, you can pass integer values directly:

oddNumbers := mapset.NewSet(1, 3, 5, 7, 9)

Or else unpack an array of integer values using the ... spread operator:

	oddNumbersArray := []int{1, 3, 5, 7, 9}
	oddNumbers := mapset.NewSet(oddNumbersArray...)

Note that in these cases, it’s not necessary to specify the generic type parameter when calling NewSet() because Go is smart enough to figure out the type from the values you pass in.

String Representation of a Set

The Set type has a convenient string representation. You can see this by printing the Set directly, e.g.:

fmt.Println(oddNumbers)

…gives…

Set{1, 3, 5, 7, 9}

Alternatively, you can obtain the same string representation programmatically. The following should provide the same output:

	oddNumbersStr := oddNumbers.String()
	fmt.Println(oddNumbersStr)

Actually, it won’t necessarily give exactly the same output. Because the Set is unordered, the order of the values in the string representation may change. For instance, running it again, you might get this:

Set{9, 1, 3, 5, 7}

Converting a Set to a Slice

You can use the ToSlice() function to convert a Set to a slice:

	oddNumbersSlice := oddNumbers.ToSlice()
	fmt.Println(oddNumbersSlice) // prints "[5 7 9 1 3]"

Update 9th August 2022: Like String(), the result of ToSlice() is non-deterministic and can give different results each time it’s run. This is particularly annoying when you want to compare sets in a test (the set data might be coming from an API, so it’s not just a matter of calling Equal()).

Adding and Removing Items

Use the Add() function to add items to a set. For example:

	evenNumbers := mapset.NewSet[int]()
	evenNumbers.Add(2)
	evenNumbers.Add(4)
	evenNumbers.Add(6)
	evenNumbers.Add(8)
	evenNumbers.Add(10)

	fmt.Println(evenNumbers)

It seems like this needs to be called for each item you want to add. It would be nice to have something like C#’s AddRange() method, by which you could append an entire array, list, or other collection of items.

You can remove a specific item from a Set by calling Remove():

	evenNumbers.Remove(10)

	fmt.Println(evenNumbers)

The combined output of these two examples would be something like:

Set{2, 4, 6, 8, 10}
Set{2, 4, 6, 8}

Another way to remove items from a set is to call Pop(), which “removes and returns an arbitrary item from the set.” I suppose this could be useful when you want to process all items in the set until it’s empty.

You can also call Clear() to remove all items from the set.

Counting Items

Use the Cardinality() function to get the number of items in the set. The following example outputs a value of 4.

fmt.Println(evenNumbers.Cardinality())

Set Membership

You can test basic set membership using Contains():

	primeNumbers := mapset.NewSet(2, 3, 5, 7, 11)

	fmt.Println(primeNumbers.Contains(3)) // prints "true"
	fmt.Println(primeNumbers.Contains(4)) // prints "false"

Set Comparisons

The following functions are available to compare sets:

  • Equal() returns true if the two sets have the same elements
  • IsSubset() returns true if the first set is a subset of the second set, or if they’re equal
  • IsProperSubset() returns true if the first set is a subset of the second set and they aren’t equal
  • IsSuperset() returns true if the first set is a superset of the second set (i.e. the second set is a subset of the first), or if they’re equal
  • IsProperSuperset() returns true if the first set is a superset of the second set (i.e. the second set is a subset of the first), and they aren’t equal

The following example shows what to expect from these operations:

	primeNumbers := mapset.NewSet(2, 3, 5, 7, 11)
	primeNumbers2 := mapset.NewSet(2, 3, 5, 7, 11)
	primeNumbersSubset := mapset.NewSet(2, 3, 5)

	// set equality

	fmt.Println(primeNumbers.Equal(primeNumbers2))      // prints "true"
	fmt.Println(primeNumbers.Equal(primeNumbersSubset)) // prints "false"

	// subset

	fmt.Println(primeNumbersSubset.IsSubset(primeNumbers)) // prints "true"
	fmt.Println(primeNumbers.IsSubset(primeNumbersSubset)) // prints "false"
	fmt.Println(primeNumbers2.IsSubset(primeNumbers))      // prints "true"

	// proper subset

	fmt.Println(primeNumbersSubset.IsProperSubset(primeNumbers)) // prints "true"
	fmt.Println(primeNumbers.IsProperSubset(primeNumbersSubset)) // prints "false"
	fmt.Println(primeNumbers2.IsProperSubset(primeNumbers))      // prints "false"

	// superset

	fmt.Println(primeNumbersSubset.IsSuperset(primeNumbers)) // prints "false"
	fmt.Println(primeNumbers.IsSuperset(primeNumbersSubset)) // prints "true"
	fmt.Println(primeNumbers2.IsSuperset(primeNumbers))      // prints "true"

	// proper superset

	fmt.Println(primeNumbers.IsProperSuperset(primeNumbersSubset)) // prints "true"
	fmt.Println(primeNumbersSubset.IsProperSuperset(primeNumbers)) // prints "false"
	fmt.Println(primeNumbers.IsProperSuperset(primeNumbers2))      // prints "false"

Set Operations

Naturally, the golang-set package provides the typical set operations you would expect, including:

  • Union() – combines all elements from both sets, eliminating duplicates
  • Intersection() – obtains only those elements that exist in both sets
  • Difference() – gets those elements in the first set that aren’t in the second set
  • SymmetricDifference() – the union minus the intersection

Here are a few examples showing these operations in action:

	fibonacciNumbers := mapset.NewSet(0, 1, 2, 3, 5)
	triangularNumbers := mapset.NewSet(1, 3, 6, 10, 15)

	fmt.Println(fibonacciNumbers.Union(triangularNumbers))     // prints "Set{0, 1, 2, 3, 5, 6, 10, 15}"
	fmt.Println(fibonacciNumbers.Intersect(triangularNumbers)) // prints "Set{1, 3}"

	fmt.Println(fibonacciNumbers.Difference(triangularNumbers)) // prints "Set{0, 2, 5}"
	fmt.Println(triangularNumbers.Difference(fibonacciNumbers)) // prints "Set{6, 10, 15}"

	fmt.Println(fibonacciNumbers.SymmetricDifference(triangularNumbers)) // prints "Set{5, 6, 10, 15, 0, 2}"

JSON Functions

golang-set seems to have functions to serialise and deserialise a set to/from JSON. I’m not sure where these would be useful, but I decided to give them a try

Use MarshalJSON() to serialise a set to JSON, which ends up looking just like the slice representation:

	evenNumbers := mapset.NewSet(2, 4, 6, 8, 10)

	jsonBytes, err := evenNumbers.MarshalJSON()
	if err == nil {
		fmt.Println(string(jsonBytes)) // prints "[2,4,6,8,10]"
	}

UnmarshalJSON() is supposed to deserialise JSON back to a set, but it doesn’t seem to work:

	evenNumbers2 := mapset.NewSet[int]()
	err = evenNumbers2.UnmarshalJSON(jsonBytes)
	fmt.Println(evenNumbers2) // prints "Set{}"

I have no idea what’s the problem with this. The JSON functions are neither documented in the readme nor covered by tests, but they were easy enough to discover via Intellisense in Visual Studio Code.

Conclusion

Hopefully this tour of the golang-set package has shown you enough existing Set functionality that you won’t have to write your own set data structure in Go ever again.

From .NET to GoLang: Where Did Everything Go?

Today marks six months since I started working with Go (also known as GoLang). Before that, I worked for about a decade using C#, with which I became quite comfortable over the years. It’s been fun to learn a new programming language professionally, but it does take some adjustment. After six months, I don’t expect to be an expert, or even know the language well, but I’d like to share the candid experience of a newcomer to programming in Go.

The general feeling I have about Go is that it is somewhat tedious to work with (see also my Twitter thread from 3 months ago). This is down to a lack of (a) language features that make development more productive, and (b) standard library functionality that provides common things that everybody uses. Perhaps some of this might be down to my own inexperience, and I welcome feedback as long as it’s constructive. However, my understanding (e.g. based on common Stack Overflow answers) is due to Go’s nature as a “simple language”.

I don’t buy the “simple language” argument (we’ll see more of this later). Why would Google bother to create a new language that offered less features than existing languages? Thinking about it, it’s probably down to historical reasons. Go seems to have appeared late in 2009. At the time, there weren’t a lot of options in terms of robust, productive, general-purpose programming languages that were both open-source and cross-platform. Node.js was still in its infancy; Rust would only be released the following year; and .NET Core was still several years away, which means C# was still restricted to Microsoft platforms. Python had been around much longer, but it has some very well-known limitations (e.g. in terms of performance).

In the rest of this article, we’ll take a tour of some of the things that stood out as I’ve been learning and using Go. I’m just hoping this will help illustrate why I think Go is tedious, and perhaps help other people thinking about picking up the language.

Update 16th November 2022: see also the followup article: From .NET to GoLang: Here We Go Again.

OOP and Classes

Let’s get this out of the way first: Go is a procedural language, not unlike C or Pascal. You write a bunch of statements, control the flow with loops and conditional statements, organise them into functions, and that’s about it. There are no classes, objects, methods and all that (although there is some concept of interfaces, and receiver functions seem very much like extension methods).

To be honest, this is the one thing that doesn’t bother me at all. I’ve seen countless developers overcomplicate life unnecessarily with OOP (e.g. layers upon layers of inheritance) in C#. I’m not all-out against OOP as Zed Shaw is (see “Object Oriented Programming in Python“). However, the vast of majority of work I’ve done with C# was working with data (whether that’s building an API, working with a cache, using queues etc, it’s almost always a matter of getting data, transforming it, and passing it somewhere else) which doesn’t seem to need abstraction, so a procedural approach fits. OOP is better suited for modelling more complex things like GUI elements, games, etc.

It’s interesting to note that while Go doesn’t have OOP, this didn’t quite spare it from the horror of ORMs (e.g. GORM). (See ADO .NET Part 1: Introduction for why I’m not a fan of ORMs.)

Generics

One of the things developers missed most since Go’s inception were generics. While Go does have some generic data structures (e.g. the map), it didn’t allow developers to create their own generic data structures until support for generics was added to the language in Go 1.18 in March this year. This means that, for instance, if you wanted to create your own stack data structure, you’d have to create one stack for integers, another for strings, etc.

Even now that generics are available, the fact that they haven’t been around long means there are limitations. A couple I’ve run into include:

If you’re familiar with the history of .NET, you might recognise that even C# initially shipped without generics in 2001, and they only made it into the language in C# 2.0 (2005). However, it’s taken Go 12 years to get generics, and we’re now in 2022.

Standard Library

Sets

One thing I use a lot is a set data structure. In C#, Python or JavaScript, you get this out of the box. But Go doesn’t have it. Why not?

Well, someone asked this on Stack Overflow back in 2015. As is typical for Stack Overflow (see “On Stack Overflow“), the question got closed. The top answers are variants of “it doesn’t have a set data structure because you can write it yourself”.

This attitude infuriates me. Software development is complex enough, and I’d like to focus on whatever problem I need to solve, instead of reinventing the wheel and going on a yak shaving spree every time I need some common dependency that the standard library doesn’t provide.

Besides, it’s not as simple as writing a set data structure. You also need to write implementations for the operations you need (e.g. intersection, union, difference, symmetric difference, etc), test them thoroughly, make sure they’re efficient (from a performance perspective), etc. This is something that takes time to get right, but at the same time it’s also something basic that’s already been solved to death, and there’s no reason why every Go developer should have to reimplement it, when other languages provide battle-tested implementations out of the box.

In fact, there’s a comment on one of the answers that echoes my frustration:

“The usual answer for golang question: “Why provide a feature when you can rewrite it in just a few lines?”. This is why something that can be done in 3 explicit lines in python (or many other languages) takes 50+ obscure lines in go. This is one of the reasons (along with single letter variables) why I hate reading go code. It’s uselessly long, just doing with for loops what should be done by a clear, efficient and well tested properly named function. Go “spirit” is just throwing away 50 years of good software engineering practice with dubious justifications.”

Colin Pitrat, Jul 8, 2021 at 11:34

Go doesn’t come with much other than arrays, slices and maps. However, in 2017, with the release of Go 1.9, it did get sync.Map, which I understand is similar to the ConcurrentDictionary in .NET. For anything else, you’ll likely have to find an implementation on GitHub or write it yourself.

LINQ

I already said Go is a procedural language. You’ll feel it a lot. For everything you need to do, you’ll have to write lots and lots of loops, making the code a lot more verbose and error-prone compared to other languages where you can use a more functional approach (e.g. C# LINQ Select(), map() in JavaScript or Python, or list comprehensions in Python).

Mathematical Functions

If I want to find the smallest number in an array in C#, I just call Math.Min().

Does Go have a built-in function to get the smallest number in an array? No, you have to write it yourself. Here we go again.

Update 30th October 2023: min() and max() built-in functions were finally added recently in Go 1.21.

Exception Handling

I never really liked exception handling in OOP languages. I felt that checking return values of functions as in C was a lot more clear that what seemed to be a wrapper for a goto-like construct where code could suddenly jump elsewhere unpredictably on a whim.

Go doesn’t have exception handling, and so most logic looks something like this:

func doSomething() error {
	foo, err := doSomethingElse(1)
	if err != nil {
		logrus.Error("Step 1 failed", err)
		return err
	}

	bar, err := doSomethingElse(5)
	if err != nil {
		logrus.Error("Step 2 failed", err)
		return err
	}

	chicken, err := doSomethingElse(10)
	if err != nil {
		logrus.Error("Step 3 failed", err)
		return err
	}

	// ...

	return nil
}

I changed my mind. I want exception handling back.

Unused Variables

The code in the previous section doesn’t actually compile. Why not?

$ go run main.go
# command-line-arguments
./main.go:14:2: foo declared but not used
./main.go:20:2: bar declared but not used
./main.go:26:2: chicken declared but not used

Go actually fails to build if you have unused variables.

While I totally understand the benefit of keeping code clean, this is simply extreme, and very irritating. It’s very common for me to need to add a temporary variable to capture the output of some computation (or an HTTP request) and see what’s in the data, but in Go, I have to resort to a redundant fmt.Println() just to mark the variable as in-use and keep the compiler happy. It’s much more suitable to issue a warning than to fail the build.

Syntax

Function Overloading

Go doesn’t have function overloading, so you can’t have different functions with the same name and different parameters. You’ll instead have to come up with silly variants of functions that do the same thing, e.g. doSomething() and doSomething2(). Feels like going back in time, doesn’t it?

if Statements, Braces and Semicolons

if statements in Go don’t need brackets around the condition, but do mandate braces around the statements, even if there is only one statement:

	if chicken < 5 {
		logrus.Info("Chicken is less than 5")
	}

I’ve already written at length in “To Always Use Braces for Conditionals and Loops… or not” why I’m in favour of omitting braces for single-line statements. And since that’s not possible in Go, it only serves to add further verbosity to the language.

Braces must also be “Egyptian-style” (as shown above). The reason for this is explained in the FAQ and is down to semicolon insertion, just like JavaScript (which is very sad):

Why are there braces but no semicolons? And why can’t I put the opening brace on the next line?

“Go uses brace brackets for statement grouping, a syntax familiar to programmers who have worked with any language in the C family. Semicolons, however, are for parsers, not for people, and we wanted to eliminate them as much as possible. To achieve this goal, Go borrows a trick from BCPL: the semicolons that separate statements are in the formal grammar but are injected automatically, without lookahead, by the lexer at the end of any line that could be the end of a statement. This works very well in practice but has the effect that it forces a brace style. For instance, the opening brace of a function cannot appear on a line by itself.

“Some have argued that the lexer should do lookahead to permit the brace to live on the next line. We disagree. Since Go code is meant to be formatted automatically by gofmt, some style must be chosen. That style may differ from what you’ve used in C or Java, but Go is a different language and gofmt’s style is as good as any other. More important—much more important—the advantages of a single, programmatically mandated format for all Go programs greatly outweigh any perceived disadvantages of the particular style. Note too that Go’s style means that an interactive implementation of Go can use the standard syntax one line at a time without special rules.”

Go FAQ

Ternary Operator

Go doesn’t have the ?: ternary operator, often used in other languages as a concise replacement for an if statement. Why not? Once again, the question that asks this on Stack Overflow has been closed, but the answer quotes the Go FAQ to shed some light. The reason is a variant of “some developers have made messes with the ternary operator, and that’s why you can’t have nice things”. Come. On.

Loops

When it comes to loops, Go has just the for loop, so it doesn’t have the usual while and do..while loops you’d normally find in C-style programming languages. This doesn’t bother me, as I almost always use just for loops anyway.

Go’s for loop does, however, support a foreach-style way of iterating over objects in an array. Let’s try a simple iteration over an array of odd numbers:

	odds := []int{1, 3, 5, 7, 9}
	for n := range odds {
		fmt.Println(n)
	}

There are a few things that bother me here.

  1. The syntax of the array declaration. We’ll get to this later.
  2. The for n := range odds part looks like it’s assigning an entire range to the variable, whereas what it’s really doing is something like foreach (n in odds) in C#.
  3. It doesn’t print what you think it does! The first variable from a range assignment is the index, not the element, so the above code gives the output:
0
1
2
3
4

In order to print the elements, you have to introduce a second variable:

	odds := []int{1, 3, 5, 7, 9}
	for _, n := range odds {
		fmt.Println(n)
	}

Since the point of a foreach is typically to work with an element in a collection, I much prefer C#’s orthogonal way of giving the element by default and having the index as optional.

Variable Declarations

Go’s syntax seems to be loosely based on C-style languages. It uses braces and a lot of syntax and operators are familiar, but it does make some very strange deviations. We’ve already mentioned earlier the lack of semicolons, but there are a couple of other differences that make the language more reminiscent of Pascal than anything else.

The first of these is the fact that type declarations go after the variable name in a variable declaration, e.g.:

var age int

This is strange both because of the redundant var keyword and because it gets very confusing when you switch between Go and another language.

As with many languages today (including older ones like C++), Go can infer the type if you initialise it to a value, e.g.:

age := 5

This := syntax is the other thing that reminds me of Pascal. I don’t really get why it’s beneficial (I assume the reason is mostly academic), but on the other hand I have found it very annoying, as I often have to change between := and = while I’m moving code around. It’s also quite tricky given the fact that many functions return multiple values, and you’ll typically assign the results to a combination of new and reused variables.

Where variable declarations get really confusing is when the data types are data structures. We’ve already seen the initialisation of an array… where the square brackets come before the type:

odds := []int{1, 3, 5, 7, 9}

What about a map? This is a map of string to int:

mapping := map[string]int{}

It’s really, really strange that the type of the value is not delimited by any operator. So this gets weird when the value can be of a complex type itself. For instance, how would you make a map of string to slice of string? I suppose it would be something like this:

mapping := map[string][]string{}

What about a map of string to another map of string to string? I’m guessing that would be:

mapping := map[string]map[string]string{}

This is really weird. I think C#’s syntax is much more readable, even for complex generic data structures.

Race Conditions

Most of the things I’ve talked about are things I find annoying in Go, but I want to wrap up with one feature I find really great.

Go has a way to automatically detect race conditions when you run a program with the -race parameter. This is particularly nice because multithreaded programming is very tricky to get right precisely because of race conditions. Go does provide goroutines and channels as an alternative way of inter-thread communication, but they don’t fit every situation. And since Go does provide basic locking mechanisms but not much in the way of concurrent collections (as opposed to C#), synchronising access to critical sections of code is often necessary. When that happens, having -race handy is a nice feature.

Conclusion

Coming from C#, learning my way around Go has been a fun diversion, but also something of a disappointment. Given the simplistic, verbose and sometimes confusing language, and the limited standard library, there’s no reason I can think of to choose Go for a new project over something richer and more mature like C#, which nowadays runs on any platform.