The Tenth Anniversary and Second Hiatus

Today, it’s been no less than ten years since I launched Gigi Labs as a successor to Programmer’s Ranch. The new tech blog is based on WordPress instead of Blogger, providing a somewhat better platform, most notably due to a plugin supporting proper syntax highlighting.

The past year for Gigi Labs has been a rather slow one, owing both to a focus on my Ravenloft: Strahd’s Possession Walkthrough (published last month) and a declining interest in tech writing vs other hobbies. I managed to publish six articles, including a combination of in-depth tutorials and more basic stuff:

It’s funny how, three years ago, I announced that I was retiring from tech blogging, and yet I kept writing occasionally at whatever pace I felt was right. Well, I think the circumstances now are quite similar. I’m again retiring from tech blogging, and again, I’m not excluding returning to it in future.

There are a number of reasons for this. The first and most obvious thing is that we live in a chaotic world where life priorities and circumstances change, and it’s necessary to focus on the most important things and at the same time get enough rest to avoid burning out. Although I consider writing and sharing knowledge as a very important thing, I’ve already written in “10 Years of Programmer’s Ranch” that the realities of tech blogging (that it’s a lot of work for no tangible reward) make it hard to sustain, especially in the long term.

Another factor is the interest in these articles. While I don’t write primarily to grow my site’s stats, they do tell me a lot about which articles are most popular. I have observed, for instance, that there is huge interest in VS Code. It’s also pretty clear to me that the most interesting and detailed articles get little attention, whereas the more basic ones are hugely popular. I’ve also mentioned that in “10 Years of Programmer’s Ranch“, and I understand why it’s the case. However, this doesn’t really align with how and why I write. If I’m going to write a lot of basic articles to achieve popularity, then I would just be adding more of the same to the infinite pool of mediocre articles on the internet, whereas if I write higher-quality ones (as I decided I’d do in “The Sixth Anniversary“), then it’s a lot of effort to put into articles that few people will ever see.

Speaking of statistics, I’ve seen Gigi Labs’ page views drop significantly over the past year or so. I can only speculate why this happened. Perhaps it’s because my articles are gradually getting outdated, or because Google has updated its algorithm, or maybe due to people shifting more and more towards ChatGPT instead of traditional search engines. Whatever the reasons, this decline doesn’t quite inspire me to write more.

And since we mentioned ChatGPT, let’s talk about AI. There are a couple of aspects about the recent AI hype that discourage me from continuing to write. One is that I’m not keen on investing all that effort in creating high-quality content only to have some company absorb it and use it for commercial interests. On the other hand, I’m happy for those people who find ChatGPT useful to solve their day-to-day coding problems. And, assuming that ChatGPT answers arbitrary questions so well (which is not always the case), then there’s little reason for me to write tutorials on how to do specific things. One can simply ask ChatGPT instead!

I’m also really, really tired of WordPress. Sure, it was a step up from Blogger, but it’s still a slow, insecure and clunky mess. Unfortunately I haven’t yet found a better solution for syntax highlighting in articles, otherwise I’d have moved off WordPress a long time ago.

Finally, it’s also simply a matter of hobbies and where I’d rather spend my free time. I started writing about games on the web 22 years ago, gradually shifted towards tech blogging, and have now come full circle. Nowadays I find it a lot more fun to write about games than about tech.

I’m really happy with what I’ve achieved here at Gigi Labs. Realistically, I don’t think I’ll have the time or inclination to continue writing here for the foreseeable future, but things may change, as they always do. Whether I continue writing or not, I’m really happy with what I’ve built and achieved here at Gigi Labs, and the site continues to be available for those who need the articles within.

On the other hand, my Sirius Planner task planner side project has reached the end of its life, after 4 years, and will be taken down in November. Take a look if you want – it’s your last chance!

Investigating Memory Leaks in Go with pprof

Memory leaks are among the toughest problems a software engineer may need to deal with. A program consumes huge amounts of memory, possibly crashes as a result, and it’s not immediately apparent why. Different languages have different tools to deal with resource problems. In Go, we can use the built-in profiler called pprof.

To see it in action, we’ll consider a simple program that allocates some memory. It’s not the best example, but real situations with memory leaks are both hard to troubleshoot and to make up. So instead of trying to come up with a really good example, I’ll show you how to use the tooling on a simple one, and then you can apply the same steps when you encounter a real problem situation.

package main

import (
	"fmt"
)

func main() {
	const size = 1000000
	waste := make([]int, size)
	for i := 0; i < size; i++ {
		waste[i] = i
	}

	fmt.Println("Done.")
}

The above code is nothing special, right? It’s clear where we’re allocating memory. However, real situations aren’t always so simple, and so we’ll use pprof to help us understand what’s going on under the hood. We do this by adding a couple of imports and setting up an endpoint as follows:

package main

import (
	"fmt"
	"net/http"
	_ "net/http/pprof"
)

func main() {
	const size = 1000000
	waste := make([]int, size)
	for i := 0; i < size; i++ {
		waste[i] = i
	}

	http.ListenAndServe("localhost:8090", nil)

	fmt.Println("Done.")
}

Before we go on, there are a few things to note at this point:

  • The port is arbitrary. You can use any you want, as long as it doesn’t conflict with something else.
  • If you want to access the endpoint from another machine, use 0.0.0.0 instead of localhost.
  • It’s generally not a good idea to include profiling code with a program by default, because its operation can cause nontrivial resource consumption, and can also expose internal details about the code that can have security and intellectual property implications. So, do this only in case of necessity and in a controlled environment.
  • The ListenAndServe() is blocking, so in this case you won’t see “Done.” printed afterwards. If you need code to resume afterwards (e.g. you’re using a web framework such as Gin), run ListenAndServe() at the beginning in a goroutine like:
	go func() {
		err := http.ListenAndServe("localhost:8090", nil)
		if err != nil {
			fmt.Println(err)
		}
	}()

So, once you’ve got pprof set up, you can run the program and, from a separate terminal, access /debug/pprof relative to the endpoint you specified:

Accessing the pprof endpoint

From here, if we click on “heap”, we get a dump of heap (memory) information:

The program’s heap info from pprof

Because this output is rather cryptic, our problem now shifts from obtaining memory data to interpreting it. Grab a copy of this heap dump either by saving directly from the browser, or using the following command. If it’s on a server, copy it over to your local system using scp or similar. We’ll inspect the memory using this dump, but hang onto it so that you can compare the state before and after a possible fix later.

curl http://localhost:8090/debug/pprof/heap > heap.dump

Once you have the heap dump on your local system, there are different ways to inspect it. The easiest way I found is to use pprof’s -http switch to run a local web server that gives you a few different views. Let’s try that:

go tool pprof -http=localhost:8091 heap.dump

Again, the port here is arbitrary but it needs to be different from the one you specified in the code. That was for the pprof endpoint, whereas this one is for pprof’s analysis tool. Once this is running, we can open localhost:8091 to understand a little more:

The graph view and menu of viewing options
The flame graph view
The source view
The top view

This tool has a few different ways you can use to inspect memory allocation in the program’s heap. I find the graph, flame graph, top and source views most useful, but there are others. Because a program’s memory structures can be very complex, it often helps to use several of these views to look at it from different angles. For instance, the graph view gives us an idea of the extent of memory allocation as functions call each other, but the source view actually tells us the exact line that is allocating memory.

Keep in mind that the fact that memory is being allocated does not necessarily imply that there is a problem. So, in practice you’ll need to diligently spend time inspecting several heap dumps of the same program at different times or with different changes. Every program is different, so there’s no general-purpose advice that can be offered to solve memory leaks. pprof can help you gather and view data about the program’s memory, but it’s up to you to understand the nature of how your program behaves and be able to spot bad behaviour in the heap dumps.

While I prefer the above approach, some people like inspecting heap dumps directly in the terminal using pprof. For instance, you can run pprof without the -http switch to enter interactive mode and then use the svg command to generate the graph view as an image:

$ go tool pprof heap.dump
File: __debug_bin2191375990
Type: inuse_space
Time: Jun 16, 2024 at 12:36pm (CEST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) svg
Generating report in profile001.svg
(pprof) 

Personally, I think this is a little more cumbersome because you have to learn pprof’s specific commands, it requires graphviz to be installed in some situations (for the image generation), and it takes more effort to see the different views that the web-based approach offers trivially. But, if you prefer it this way, the option is there.

So, hopefully the steps above should make pprof really easy for you to use. I’d like to express my gratitude to the following sources that helped me figure out how to use it originally, and contain some more detail if you need it:

Go: Gin Returns Misleading Response When JSON Fails

There’s a thing with Go where a float containing special values such as NaN, +Inf or -Inf will fail to serialize, as is documented in several GitHub issues (e.g. this one and this other one). We can see this easily as follows.

First, create a module:

go mod init main

Then, try serialising a special float value:

package main

import (
	"encoding/json"
	"fmt"
	"math"
)

func main() {
	if _, err := json.Marshal(math.NaN()); err != nil {
		fmt.Println(err)
	}
}

The output then shows:

json: unsupported value: NaN

Okay, so the serialiser doesn’t play well with special float values. Now, let’s talk about what happens when you do this in the Gin Web Framework.

We’ll get right to the point if we copy Gin’s “Getting Started” sample:

package main

import (
  "net/http"

  "github.com/gin-gonic/gin"
)

func main() {
  r := gin.Default()
  r.GET("/ping", func(c *gin.Context) {
    c.JSON(http.StatusOK, gin.H{
      "message": "pong",
    })
  })
  r.Run() // listen and serve on 0.0.0.0:8080 (for windows "localhost:8080")
}

Then, we tweak the struct to contain a special float value:

package main

import (
	"math"
	"net/http"

	"github.com/gin-gonic/gin"
)

func main() {
  r := gin.Default()
  r.GET("/ping", func(c *gin.Context) {
    c.JSON(http.StatusOK, gin.H{
      "message": math.Inf(1),
    })
  })
  r.Run() // listen and serve on 0.0.0.0:8080 (for windows "localhost:8080")
}

After running a go mod tidy (or the go get command per Gin’s setup instructions) to retrieve the Gin dependency, we can run the program. This results in some pretty odd behaviour:

There’s an error in the output, but Gin returns a 200 response with no data in it.

By hitting the http://localhost:8080/ping endpoint (e.g. in a web browser), we get back a 200 OK response, with Content-Length: 0 and an empty body. Gin’s output clearly shows an error:

Error #01: json: unsupported value: +Inf

The same behaviour occurs with any JSON that can’t be parsed, not just floats. For instance, we could also put a function value into the JSON and Gin will half-fail the same way:

		c.JSON(http.StatusOK, gin.H{
			"message": func() {},
		})

It’s pretty misleading for the API to return a 200 OK as if everything’s fine, but with no data. We had this problem at my work, earlier this week, and in that case we didn’t even get the error in the output, so it was a tricky data issue to troubleshoot. It would be a lot better if Gin returned a response with an appropriate status code and body so that the API’s client could at least get an idea of what went wrong.

Padding Thumbnails to Fix Aspect Ratio

Creating thumbnails for images on a website is key to performant and responsive page loads. This is easy to do for things like screenshots that tend to be all the same size; however it can be problematic for images with varying sizes, especially elongated ones. In this article we’ll discuss a technique to generate thumbnails that preserve a consistent aspect ratio, by padding them as needed.

Generating Thumbnails with ImageMagick

A 712×180 image from my upcoming Ravenloft: Strahd’s Possession walkthrough

In “Resizing Images and Creating Thumbnails with ImageMagick“, I demonstrated a simple way to resize images using ImageMagick:

convert input.jpg -resize 175x109 output.jpg
175×44 result

This works, and it preserves the aspect ratio, but disregards the desired size (in this case 175×109). For instance, applying this to a 712×180 image produces a thumbnail of size 175×44, which is undesirable in a page where your other thumbnails will be 175×109.

An alternative is to force the output size by adding an exclamation mark to the size:

convert input.jpg -resize 175x109! output.jpg
175×109 result

This produces an image with the desired size, but throws the aspect ratio out of the window. Elongated images end up with thumbnails that look squished.

Fortunately, ImageMagick already has the ability to pad out thumbnails, and so it is easy to produce a thumbnail that respects both the desired output size and the aspect ratio, by having it pad out the extra space in the thumbnail:

convert input.jpg -thumbnail 175x109 -background cyan -gravity center -extent 175x109 output.jpg
Padded thumbnail

Note that I’ve used cyan only to demonstrate the effect of the padding; in practice you will want to use a colour that blends with the rest of the image, in this case white.

The resulting thumbnail preserves the proportions of the features of the original image, while fitting nicely into the 175×109 dimensions that will be used by all other thumbnails on the page.

Mathematical Foundation

By the time I discovered how to generate padded thumbnails using ImageMagick, I had already worked out the mathematical calculations to do it myself. While this is now unnecessary, it’s an interesting exercise and can be useful if you ever need to do this kind of thing yourself programmatically without the help of ImageMagick.

Elongated images need to be resized into a smaller space (white) and padded (blue) to fit fixed thumbnail dimensions

So, imagine we have an elongated image and we need to generate a thumbnail that fits a specific size and is padded out to keep the original image’s aspect ratio, same as we did with ImageMagick. One way to do this is to pad the original image with extra space to achieve the desired aspect ratio of the thumbnail, and then just resize to the desired thumbnail dimensions. This works out slightly differently depending on whether the image is horizontally or vertically elongated.

Fitting a 600×200 image into a higher space

Let’s start with the first case: say we have an image of original size 600×200, and we want to fit it into a thumbnail of size 175×109. The first thing we need to do is calculate the aspect ratio of the original image and that of the thumbnail.

Calculating aspect ratio

The aspect ratio is calculated simply by dividing width by height. When the aspect ratio of the original image is larger than that of the thumbnail, as in this case, it means that the image is horizontally elongated and needs to be padded vertically. Conversely, if the original image’s aspect ratio were smaller than that of the thumbnail, we would be considering the second case, i.e. a vertically elongated image that needs to be padded horizontally.

Now, we need to figure out the dimensions of the padded image. We already know that our 600×200 image needs to be padded vertically, so the width remains the same at 600, but how do we calculate the new height (new_h)? As it turns out, the Law of Similar Triangles also applies to rectangles, and since we want to keep a constant aspect ratio, then it becomes just a matter of comparing ratios:

Finding the new height after padding

To double-check the result, calculate its aspect ratio again. Dividing 600 by 373.71 does in fact roughly give us 1.6055, the aspect ratio we were hoping to obtain.

Fitting a 300×700 image into wider space

The second case, i.e. when we’re dealing with vertically elongated images, works out similarly. In this case the original image’s aspect ratio is less than that of the thumbnail, and we need to find out the padded image height instead of the width. Assuming we’re dealing with a 300×700 image, then:

Finding the new width after padding

Dividing the new height, 481.65, by 300 roughly gives us the aspect ratio we wanted.

For both cases, once we manage to fit the original image onto a bigger canvas with the right aspect ratio, then it can be resized right down to the thumbnail dimensions without losing quality.

PIL Proof of Concept

To see the above concepts in action, let’s implement them using the Python Image Library (PIL). First, make sure you have it installed:

pip3 install pillow

Then, the following code generates thumbnails for horizontally elongated images:

from PIL import Image

thumb_w = 175
thumb_h = 109

with Image.open('input.jpg') as input_image:
    orig_w, orig_h = input_image.size

    orig_aspect = (orig_w / orig_h)
    thumb_aspect = (thumb_w / thumb_h)

    if orig_aspect > thumb_aspect: # horizontal elongation - pad vertically
        new_w = orig_w
        new_h = int((orig_w * thumb_h) / thumb_w)

        with Image.new( 'RGB', (new_w, new_h), (0, 255, 255)) as output_image: # cyan background
            # y-position of original image over padded image
            orig_y = int((new_h  / 2) - (orig_h / 2))
            # copy original image onto padded image
            output_image.paste(input_image, (0, orig_y))
            # resize padded image to thumbnail size
            output_image = output_image.resize((thumb_w, thumb_h), resample=Image.LANCZOS)
            # save final image to disk
            output_image.save('output.jpg')
        
    else: # vertical elongation - pad horizontally
        pass # ...

Based on the calculations in the previous section, the code compares the aspect ratio of the original image to that of the desired thumbnail dimensions to determine whether it needs to pad vertically or horizontally. For the first case (pad vertically), it calculates the padded image height (new_h) and creates a new image to accommodate it (again, the cyan background is just to demonstrate the effect). It then copies the original image into the middle of the new image. Finally, it resizes the new image to thumbnail size, and saves it to disk:

Vertically padded thumbnail generated with PIL

For the second case (pad horizontally), the code is mostly the same, except that we calculate the padded image width (new_w) instead of the height, and we calculate the x-position (orig_x) when placing the original image in the middle of the new image:

    else: # vertical elongation - pad horizontally
        new_w = int((thumb_w * orig_h) / thumb_h)
        new_h = orig_h

        with Image.new( 'RGB', (new_w, new_h), (0, 255, 255)) as output_image: # cyan background
            # x-position of original image over padded image
            orig_x = int((new_w  / 2) - (orig_w / 2))
            # copy original image onto padded image
            output_image.paste(input_image, (orig_x, 0))
            # resize padded image to thumbnail size
            output_image = output_image.resize((thumb_w, thumb_h), resample=Image.LANCZOS)
            # save final image to disk
            output_image.save('output.jpg')

Applying this to a vertically-elongated image, we get something like this:

Horizontally padded thumbnail generated with PIL

This code is just a quick-and-dirty proof of concept. It can be simplified and may need adjusting to account for off-by-one errors, cases where the aspect ratio already matches, images that aren’t JPG, etc. But it shows that the calculations we’ve seen actually work in practice and produce the desired result.

Conclusion

In this article we’ve discussed the need to produce thumbnails that both conform to a desired size and retain the original image’s aspect ratio. In cases where resizing breaks the aspect ratio, we can pad the original image before resizing in order to maintain the aspect ratio.

We’ve seen how to generate padded image thumbnails using ImageMagick, and then delved into how we could do the same thing ourselves. After demonstrating the mathematical calculations necessary to create the right padding to preserve the aspect ratio, we then applied them in practice using PIL.

I learned the above techniques while trying to find a way to automatically generate decent-looking thumbnails for maps in my upcoming Ravenloft: Strahd’s Possession walkthrough, where the maps come in all shapes and sizes and some of them needed a little more attention due to elongation. Hopefully this will be useful to other people as well.

Filling Go Structs in VS Code

If you program in Go, then you work with structs all the time. There’s a handy little tool in VS Code that you can use to quickly populate an empty struct with all its fields instead of writing them by hand. With your cursor over the name of the struct, bring up the context menu by doing one of the following:

  • Clicking on the light bulb to the left
  • Pressing Ctrl+. (Windows/Linux)
  • Pressing Cmd+. (Max)
This context menu lets you fill a struct’s fields.

Then, click on the “Fill” option or press ENTER to accept it, and the struct’s fields will be added along with default values for each according to their type:

The struct’s fields have been added with default values.

This little productivity tool is great, especially when you’re mapping data across structs in different parts of your application.

"You don't learn to walk by following rules. You learn by doing, and by falling over." — Richard Branson