Tag Archives: Python

Padding Thumbnails to Fix Aspect Ratio

Creating thumbnails for images on a website is key to performant and responsive page loads. This is easy to do for things like screenshots that tend to be all the same size; however it can be problematic for images with varying sizes, especially elongated ones. In this article we’ll discuss a technique to generate thumbnails that preserve a consistent aspect ratio, by padding them as needed.

Generating Thumbnails with ImageMagick

A 712×180 image from my upcoming Ravenloft: Strahd’s Possession walkthrough

In “Resizing Images and Creating Thumbnails with ImageMagick“, I demonstrated a simple way to resize images using ImageMagick:

convert input.jpg -resize 175x109 output.jpg
175×44 result

This works, and it preserves the aspect ratio, but disregards the desired size (in this case 175×109). For instance, applying this to a 712×180 image produces a thumbnail of size 175×44, which is undesirable in a page where your other thumbnails will be 175×109.

An alternative is to force the output size by adding an exclamation mark to the size:

convert input.jpg -resize 175x109! output.jpg
175×109 result

This produces an image with the desired size, but throws the aspect ratio out of the window. Elongated images end up with thumbnails that look squished.

Fortunately, ImageMagick already has the ability to pad out thumbnails, and so it is easy to produce a thumbnail that respects both the desired output size and the aspect ratio, by having it pad out the extra space in the thumbnail:

convert input.jpg -thumbnail 175x109 -background cyan -gravity center -extent 175x109 output.jpg
Padded thumbnail

Note that I’ve used cyan only to demonstrate the effect of the padding; in practice you will want to use a colour that blends with the rest of the image, in this case white.

The resulting thumbnail preserves the proportions of the features of the original image, while fitting nicely into the 175×109 dimensions that will be used by all other thumbnails on the page.

Mathematical Foundation

By the time I discovered how to generate padded thumbnails using ImageMagick, I had already worked out the mathematical calculations to do it myself. While this is now unnecessary, it’s an interesting exercise and can be useful if you ever need to do this kind of thing yourself programmatically without the help of ImageMagick.

Elongated images need to be resized into a smaller space (white) and padded (blue) to fit fixed thumbnail dimensions

So, imagine we have an elongated image and we need to generate a thumbnail that fits a specific size and is padded out to keep the original image’s aspect ratio, same as we did with ImageMagick. One way to do this is to pad the original image with extra space to achieve the desired aspect ratio of the thumbnail, and then just resize to the desired thumbnail dimensions. This works out slightly differently depending on whether the image is horizontally or vertically elongated.

Fitting a 600×200 image into a higher space

Let’s start with the first case: say we have an image of original size 600×200, and we want to fit it into a thumbnail of size 175×109. The first thing we need to do is calculate the aspect ratio of the original image and that of the thumbnail.

Calculating aspect ratio

The aspect ratio is calculated simply by dividing width by height. When the aspect ratio of the original image is larger than that of the thumbnail, as in this case, it means that the image is horizontally elongated and needs to be padded vertically. Conversely, if the original image’s aspect ratio were smaller than that of the thumbnail, we would be considering the second case, i.e. a vertically elongated image that needs to be padded horizontally.

Now, we need to figure out the dimensions of the padded image. We already know that our 600×200 image needs to be padded vertically, so the width remains the same at 600, but how do we calculate the new height (new_h)? As it turns out, the Law of Similar Triangles also applies to rectangles, and since we want to keep a constant aspect ratio, then it becomes just a matter of comparing ratios:

Finding the new height after padding

To double-check the result, calculate its aspect ratio again. Dividing 600 by 373.71 does in fact roughly give us 1.6055, the aspect ratio we were hoping to obtain.

Fitting a 300×700 image into wider space

The second case, i.e. when we’re dealing with vertically elongated images, works out similarly. In this case the original image’s aspect ratio is less than that of the thumbnail, and we need to find out the padded image height instead of the width. Assuming we’re dealing with a 300×700 image, then:

Finding the new width after padding

Dividing the new height, 481.65, by 300 roughly gives us the aspect ratio we wanted.

For both cases, once we manage to fit the original image onto a bigger canvas with the right aspect ratio, then it can be resized right down to the thumbnail dimensions without losing quality.

PIL Proof of Concept

To see the above concepts in action, let’s implement them using the Python Image Library (PIL). First, make sure you have it installed:

pip3 install pillow

Then, the following code generates thumbnails for horizontally elongated images:

from PIL import Image

thumb_w = 175
thumb_h = 109

with Image.open('input.jpg') as input_image:
    orig_w, orig_h = input_image.size

    orig_aspect = (orig_w / orig_h)
    thumb_aspect = (thumb_w / thumb_h)

    if orig_aspect > thumb_aspect: # horizontal elongation - pad vertically
        new_w = orig_w
        new_h = int((orig_w * thumb_h) / thumb_w)

        with Image.new( 'RGB', (new_w, new_h), (0, 255, 255)) as output_image: # cyan background
            # y-position of original image over padded image
            orig_y = int((new_h  / 2) - (orig_h / 2))
            # copy original image onto padded image
            output_image.paste(input_image, (0, orig_y))
            # resize padded image to thumbnail size
            output_image = output_image.resize((thumb_w, thumb_h), resample=Image.LANCZOS)
            # save final image to disk
            output_image.save('output.jpg')
        
    else: # vertical elongation - pad horizontally
        pass # ...

Based on the calculations in the previous section, the code compares the aspect ratio of the original image to that of the desired thumbnail dimensions to determine whether it needs to pad vertically or horizontally. For the first case (pad vertically), it calculates the padded image height (new_h) and creates a new image to accommodate it (again, the cyan background is just to demonstrate the effect). It then copies the original image into the middle of the new image. Finally, it resizes the new image to thumbnail size, and saves it to disk:

Vertically padded thumbnail generated with PIL

For the second case (pad horizontally), the code is mostly the same, except that we calculate the padded image width (new_w) instead of the height, and we calculate the x-position (orig_x) when placing the original image in the middle of the new image:

    else: # vertical elongation - pad horizontally
        new_w = int((thumb_w * orig_h) / thumb_h)
        new_h = orig_h

        with Image.new( 'RGB', (new_w, new_h), (0, 255, 255)) as output_image: # cyan background
            # x-position of original image over padded image
            orig_x = int((new_w  / 2) - (orig_w / 2))
            # copy original image onto padded image
            output_image.paste(input_image, (orig_x, 0))
            # resize padded image to thumbnail size
            output_image = output_image.resize((thumb_w, thumb_h), resample=Image.LANCZOS)
            # save final image to disk
            output_image.save('output.jpg')

Applying this to a vertically-elongated image, we get something like this:

Horizontally padded thumbnail generated with PIL

This code is just a quick-and-dirty proof of concept. It can be simplified and may need adjusting to account for off-by-one errors, cases where the aspect ratio already matches, images that aren’t JPG, etc. But it shows that the calculations we’ve seen actually work in practice and produce the desired result.

Conclusion

In this article we’ve discussed the need to produce thumbnails that both conform to a desired size and retain the original image’s aspect ratio. In cases where resizing breaks the aspect ratio, we can pad the original image before resizing in order to maintain the aspect ratio.

We’ve seen how to generate padded image thumbnails using ImageMagick, and then delved into how we could do the same thing ourselves. After demonstrating the mathematical calculations necessary to create the right padding to preserve the aspect ratio, we then applied them in practice using PIL.

I learned the above techniques while trying to find a way to automatically generate decent-looking thumbnails for maps in my upcoming Ravenloft: Strahd’s Possession walkthrough, where the maps come in all shapes and sizes and some of them needed a little more attention due to elongation. Hopefully this will be useful to other people as well.

Migrating Cartography to Memgraph

I’ve recently written about how to use Cartography to collect infrastructural data from both AWS and Okta into a Neo4j graph database for security analysis.

Neo4j is a long-standing player in the graph database market, with a robust product, great documentation, and a massive following. However, its long legacy is in a way also a disadvantage, as it can be costly, slow, and resource-hungry (due in no small part to its reliance on the JVM). Sometimes people would like to use an alternative for any of these reasons.

Memgraph, on the other hand, is a relatively young graph database, and certainly not as fully-featured as Neo4j. A key difference is that it is written in C++, meaning it’s designed to be faster and more lightweight than Neo4j (whether it lives up to this is something you’ll need to evaluate for your own use cases). Memgraph also made a very wise decision to support the Bolt protocol and the Cypher language – both of which Neo4j uses – meaning that it’s compatible with existing Neo4j clients and queries. Although there are variations in Cypher dialect, the incompatibilities are few, and moving from Neo4j to Memgraph is significantly less painful than, say, transitioning to a graph database that uses Gremlin as its query language.

At the time of writing this article, Cartography requires Neo4j 4.x, and does not work with Memgraph. However, I’m going to show you how to make at least part of it (the Okta intel module) work with minor alterations to the Cartography codebase. This serves as a demonstration of how to get started migrating an existing application from Neo4j to Memgraph.

Running Memgraph

Before we start looking at Cartography, let’s run an instance of Memgraph. To do this, we’ll take a tip from my earlier article, “Using the Neo4j Bolt Driver for Python with Memgraph“, and run it under Docker as follows (drop the sudo if you’re on Mac or Windows):

sudo docker run --rm -it -p 7687:7687 -p 3000:3000 -e MEMGRAPH="--bolt-server-name-for-init=Neo4j/" memgraph/memgraph-platform

That --bolt-server-name-for-init=Neo4j/ is a first critical step in Neo4j compatibility. As explained in that same article, the Neo4j Bolt Driver (i.e. client) for Python (which Cartography uses) checks whether the server sends an “agent” value that starts with “Neo4j/”. By setting this, Memgraph is effectively posing as a Neo4j server, and the Neo4j Bolt Driver for Python can’t tell the difference.

Update 19th September 2023: as of Memgraph v2.11, --bolt-server-name-for-init has a default value compatible with the Neo4j Bolt Driver, and therefore no longer needs to be provided.

If it’s successful, you should see output such as the following:

Memgraph is running. You can also execute queries directly from here.

Cloning the Cartography Repo

The next thing to do is grab a copy of the Cartography source code from the Cartography GitHub repo:

git clone https://github.com/lyft/cartography.git

Next, run the following command to install the necessary dependencies:

pip3 install -e .

Note: in the past, I’ve usually had to upgrade the Neo4j Bolt Driver for Python to 5.2.1 to get anything working, but as I try this again, it seems to work even with the default 4.4.x that Cartography uses. If you have problems, try changing setup.py to require neo4j>=5.2.1 and run the above command again.

Creating a Launch Configuration in Visual Studio Code

In order to run Cartography from its source code, you could run it directly from the terminal, for instance:

cd cartography/cartography
python3 __main__.py

However, as I’ve recently been using Visual Studio Code for all my polyglot software development needs, I find it much more convenient to set up a launch configuration that allows me to easily debug Cartography and pass whatever command-line arguments and environment variables I want.

The following launch.json is handy to run Cartography with an Okta configuration as described in “Getting Started with Cartography for Okta“:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Run Cartography",
            "type": "python",
            "request": "launch",
            "program": "cartography/__main__.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": [
                "--neo4j-user",
                "ignore",
                "--neo4j-password-env-var",
                "NEO4J_PASS",
                "--okta-org-id",
                "dev-xxxxxxxx",
                "--okta-api-key-env-var",
                "OKTA_API_TOKEN"
            ],
            "env": {
                "NEO4J_PASS": "ignore",
                "OKTA_API_TOKEN": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
            }
        }
    ]
}

You might notice we’re telling Cartography to connect to Neo4j (Memgraph actually) with username and password both set to “ignore”. The reason for this is that while the Community Edition of Memgraph does not require (or support) authentication, the Neo4j Bolt Driver for Python (i.e. Neo4j client) does require a username and password to be provided. So, as a second critical compatibility step, we pass any arbitrary value for the Neo4j username and password so long as they are not left empty.

As for the Okta configuration, remember to replace the Organisation ID and API Token with real ones.

Incompatible Index Creation Cypher

Pressing F5, we can now run Cartography from inside Visual Studio Code, and we immediately run into the first problem:

The error says: “no viable alternative at input ‘CREATEINDEXIF'”.

Memgraph is choking on the index creation step in indexes.cypher (in VS Code, use Ctrl+P / Command+P to quickly locate the file) because the index creation syntax is one aspect of Memgraph’s Cypher implementation that is not compatible with that of Neo4j. If we take the first line in the file, the Neo4j-compatible syntax is:

CREATE INDEX IF NOT EXISTS FOR (n:AWSConfigurationRecorder) ON (n.id);

…whereas the equivalent on Memgraph would be:

CREATE INDEX ON :AWSConfigurationRecorder(id);

Note: in Memgraph, the “IF NOT EXISTS” bit is implicit: an index is created if it doesn’t exist; if it does, the operation is a no-op that does not cause any error.

Fortunately, this syntactic difference is easily resolved by replacing (using VS Code search & replace syntax, with regex enabled) this:

CREATE INDEX IF NOT EXISTS FOR \(n:(.*?)\) ON \(n.(.*?)\);

…with this:

CREATE INDEX ON :$1($2);

Tip: although not in scope here, you’ll need to make a similar change also in querybuilder.py and tx.py if you also want to get other intel modules (e.g. AWS) working.

Neo4j Result Consumption

After fixing the index creation syntax and rerunning Cartography, we run into another problem:

The error says: “The result is out of scope. The associated transaction has been closed. Results can only be used while the transaction is open.”

I’m told that consume() is used to fix a problem in which Neo4j connections hang in situations where internal buffers fill up, although the Cartography team is re-evaluating whether this is necessary. In practice, I have seen that removing this doesn’t seem to cause problems with datasets I’ve tested with, although your mileage may vary. Let’s fix this problem by removing usage of consume() in statement.py.

First, we drop the .consume() at the end of line 76 inside the run() function:

    def run(self, session: neo4j.Session) -> None:
        """
        Run the statement. This will execute the query against the graph.
        """
        if self.iterative:
            self._run_iterative(session)
        else:
            session.write_transaction(self._run_noniterative)
        logger.info(f"Completed {self.parent_job_name} statement #{self.parent_job_sequence_num}")

Then, in the _run_iterative() function, we remove the entire while loop (lines 120-128) except for line 121, which we de-indent:

        # while True:
        result: neo4j.Result = session.write_transaction(self._run_noniterative)

            # Exit if we have finished processing all items
            # if not result.consume().counters.contains_updates:
            #     # Ensure network buffers are cleared
            #     result.consume()
            #     break
            # result.consume()

When we run it again, it should finish the run without problems and return control of the terminal with the prompt showing:

...
INFO:cartography.sync:Finishing sync stage 'duo'
INFO:cartography.sync:Starting sync stage 'analysis'
INFO:cartography.intel.analysis:Skipping analysis because no job path was provided.
INFO:cartography.sync:Finishing sync stage 'analysis'
INFO:cartography.sync:Finishing sync with update tag '1689401212'
daniel@andromeda:~/git/cartography$

Querying the Graph

The terminal we’re using to run Memgraph has the mgconsole client running (that’s the memgraph> prompt you see in the earlier screenshot), meaning we can try running queries directly there. For starters, we can try the ubiquitous “get everything” Cypher query:

memgraph> match (n) return n;

Note: if you get a “mg_raw_transport_send: Broken pipe”, just run the query again and it should reconnect.

This gives us some data back:

Querying Memgraph using mgconsole.

As you can see, this is not great to visualise results. Fortunately, Memgraph has its own web client (similar to Neo4j Browser) called Memgraph Lab, that you can access on http://localhost:3000/:

Memgraph Lab: Quick Connect page.

On the Quick Connect page, click the “Connect now” button. Then, switch to the “Query Execution” page using the left navigation sidebar, and you can run queries and view results more comfortably:

Seeing some nodes in Memgraph Lab.

Unlike Neo4j Browser, Memgraph Lab does not return relationships by default when you run this query. If you want to see them as well, you can run this instead:

match (a)-[r]->(b)
return a, r, b
Nodes and relationships in Memgraph Lab.

If the graph looks too cluttered, just drag the nodes around to rearrange them in a way that is more pleasant.

More Cartography with Memgraph

Cartography is a huge project that gathers data from a variety of data sources including AWS, Azure, GitHub, Okta, and others.

I’ve intentionally only covered the Okta intel module in this article because it’s small in scope and easy to digest. To use Cartography with other data sources, additional effort is required to address other problems with incompatible Cypher queries. For instance, at the time of writing this article, there are at least 9 outstanding issues that need to be fixed before Cartography can be used with Memgraph for AWS (that’s quite impressive considering that the AWS intel module is the biggest). Other intel modules may have other problems that need solving; nobody has explored them with Memgraph yet.

Summary

In this article, I’ve shown how one could go about taking an existing application that depends on Neo4j and migrating it to Memgraph. I’ve used Cartography with its Okta intel module to keep things relatively straightforward. The steps involved include:

  1. Running Memgraph with --bolt-server-name-for-init=Neo4j/
  2. Using the same Bolt-compatible Neo4j client, providing arbitrary Neo4j username and password values
  3. Fixing any incompatible Neo4j client code (in this case, consume()), if applicable
  4. Adjusting any incompatible Cypher queries

Minimal Web Application with Flask and AlpineJS

I recently needed to develop a small application for personal use. Since I had a specific need to use Python, I decided to build the backend with Flask. For the frontend, I only needed (literally) a single page to visualise some data, so I resisted the urge to go with something like Angular or React, and instead decided to try this minimal Alpine.js thing I had heard about a few days earlier.

In this article, I’m going to show you how you can really quickly create a minimal web application (REST API, static files and web UI) like I did. This can be handy for putting a simple tool together in very little time, but please consider it as a quick and dirty thing rather than any kind of best practice. I’m new to both Flask and Alpine.js, so I’m certainly not going to be giving advice on how best to use them.

Getting Started with Flask

After creating a new folder for your project, create a file called main.py (or whatever you want, really). We can kick off our API development by using the Flask Quickstart example:

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello_world():
    return "<p>Hello, World!</p>"

The same page also explains how to run the application so that it can listen for connections and serve requests:

“To run the application, use the flask command or python -m flask. You need to tell the Flask where your application is with the --app option.”

— Flask Quickstart

This is nice and all, but I like to be able to debug my code from Visual Studio Code, and for that it’s necessary to create a launch.json file with the right settings. This can be done as illustrated by the following images and their captions. If you need to do anything more advanced, see “Working with VS Code Launch Configurations“.

First, select the Debug tab on the left, and click “create a launch.json” file.
From the list that appears, select “Flask”.
Change the default suggested filename of app.py to the one you’re using, e.g. main.py.

Once that is done, it generates a launch.json file with the following contents:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Flask",
            "type": "python",
            "request": "launch",
            "module": "flask",
            "env": {
                "FLASK_APP": "main.py",
                "FLASK_DEBUG": "1"
            },
            "args": [
                "run",
                "--no-debugger",
                "--no-reload"
            ],
            "jinja": true,
            "justMyCode": true
        }
    ]
}

You can now press F5 to start the application in debug mode. The integrated terminal informs you that the application is running at http://localhost:5000/, so you can open a browser window to see that it’s working. By running it in this way, you can also add breakpoints and debug your Python code as needed.

The Flask “Hello World” is definitely working.

A Simple REST API

To build a reasonable example, we need to return some data from our API. We can do this quite easily by returning a list of dictionary objects from a function annotated with the route “/products”:

@app.route("/products")
def get_products():
    return [
        {
            "name": "Rope",
            "description": "Used to climb heights or swing across chasms.",
            "price": 15
        },
        {
            "name": "Whip",
            "description": "An old and trusty friend.",
            "price": 20
        },
        {
            "name": "Notebook",
            "description": "Contains lots of valuable information!",
            "price": 80
        }
    ]

After restarting the application and visiting http://localhost:5000/products, we can see that the list gets translated quite nicely to JSON:

The response of the /products endpoint.

Serving a Static Webpage

It is now time to build a simple user interface. Before we do that though, we need to figure out how to serve static files so that we can return an HTML file and then use Alpine.js to dynamically populate it using the API.

It turns out that this is quite simple: Flask automatically serves any files you put under the static folder. So let’s create a static folder and add an index.html file with some minimal HTML in it:

<!doctype html>
<html lang="en">
<meta charset="utf-8">
<title>My Products</title>
<body>

<h1>My Products</h1>

</body>
</html>

We can now see this page if we visit http://localhost:5000/static/index.html in a browser:

The HTML page is served from the static folder.

It works, but it’s not great. I’d like this to come up by default when I visit http://localhost:5000/, instead of having such a long URL. Fortunately, this is easy to achieve. All we need to do is replace our “Hello world” code in the default endpoint with the following:

from flask import Flask, send_from_directory

app = Flask(__name__)

@app.route('/')
def serve_index():
    return send_from_directory('static', 'index.html')

Displaying the Products

Now that we have an API and an easily accessible webpage, we can display the data after retrieving it from the API. We can now add Alpine.js to our website by adding a <script> tag, and then build out a table that retrieves data from the API:

<!doctype html>
<html lang="en">
<meta charset="utf-8">
<title>My Products</title>
<script defer src="https://cdn.jsdelivr.net/npm/alpinejs@3.x.x/dist/cdn.min.js"></script>
<style>
    body {
        font-family: sans-serif;
    }

    table, th, td {
        border: solid 1px #555;
        border-collapse: collapse;
    }
</style>
<body>

<h1>My Products</h1>

<div
    x-data="{ products: [] }"
    x-init="products = await (await fetch('/products')).json()"
>
    <table>
        <tr>
            <th>Name</th>
            <th>Description</th>
            <th>Price</th>
        </tr>
        <template x-for="product in products">
            <tr>
                <td x-text="product.name"></td>
                <td x-text="product.description"></td>
                <td x-text="product.price"></td>
            </tr>
        </template>
    </table>
</div>

</body>
</html>

In the above HTML, the main thing to look at is the stuff inside the <div>. Outside of that, the new things are just the <script> tag that imports Alpine.js, and a little CSS to make the webpage look almost bearable.

The <div> tag’s attributes set up the data model that the table is based on. Using this x-data attribute, we’re saying that we have an array called “products”. The x-init attribute is where we use the Fetch API to retrieve data from the /products endpoint and assign the result to the “products” array.

Once this data is retrieved, all that’s left to do is display it. In the <table>, we use an x-for attribute in a <template> to loop over the “products” array. Then, we bind the content of each cell (<td> element) to the properties of each product using the x-text attribute, effectively displaying them.

The result is simple but effective:

Indiana Jones would be proud of this inventory.

Conclusion

As you can see, it’s pretty quick and easy to put something simple together and then build whatever you need on top of that. For instance, you might want to filter the data you display, apply conditional highlighting, or add other functionality.

Honestly, I’m not quite sure how far you can take Alpine.js. For instance, it doesn’t seem possible to create reusable components, which somewhat limits its usefulness. And I don’t yet know how to avoid having heaps of JavaScript within HTML attributes. I’ve only started using this a few days ago, after all.

But it’s great to be able to use something like this to create simple tools without having to incur the learning curves of things like Angular, React or Vue.

The Wonderful World of Barcodes

There are lots of different ways to encode data in software. For instance, text encodings such as ASCII or UTF-8 allow us to represent strings of arbitrarily complex characters, while base64 allows us to represent even binary data as text and transmit it over channels that are more suited to deal with text (such as many internet protocols).

A barcode scanner next to a stack of books, each with its own barcode.

Barcodes, on the other hand, allow us to label physical objects and read them efficiently, providing a bridge between software and the physical world.

Although barcodes could represent any data, they are usually used to represent numbers. They are most commonly used in retail for fast processing of items being sold, especially by supermarkets, but also by pharmacies, bookshops, electronics stores, and other shops.

Many items we buy, from everyday groceries to books, are labelled with numbers (which I’ll call “product codes”) that form part of one or more standards, such as EAN-13, UPC-A, or ISBN. These numbers are encoded as a barcode on the product itself or its packaging. Several websites exist where you can look up these numbers. Other products, such as fruit, do not have standard identifiers, but that doesn’t prevent shops from applying their own.

Barcode Scanners

Barcode scanners can be used to read these product codes. They are quite cheap and can connect via USB to any system, making them widely compatible with Windows, Linux, MacOS, and various cash registers. A barcode scanner is held like a gun and, when the trigger is pulled, it emits a red light that reads the information on the barcode. This is immediately written to any text field that currently has focus, followed by a newline. For instance, if I open a text editor and quickly scan the barcodes on the four books shown in the above picture, I get the following:

9780201633542
9780201633467
9780201433074
9780201633610

Scanning these numbers into a text file is useful for testing or to later bulk-process them, but we can also use this handy input method in other ways. For instance, if we open ISBN Search and click on the search field, scanning a book’s barcode will not only populate the search field with the ISBN number, but also execute the search:

The ISBN Search homepage.
ISBN Search result for 9780201633542 is “TCP/IP Illustrated: The Implementation, Vol. 2”.

Barcodes in Retail

Shops that sell products with barcodes on them can use those barcodes as primary keys to identify each product. This makes it very easy to create a database with information about the various products (including name, price, taxes, discounts, etc) and then have the cashier look this up by just entering the barcode. We can simulate this by writing a simple program, e.g. in Python.

First, we define a simple mapping between each product’s barcode and its price:

from decimal import Decimal

prices = {
    "9780201633542": Decimal(35.75),
    "9780201633467": Decimal(28.90),
    "9780201433074": Decimal(42.30),
    "9780201633610": Decimal(0.50)
} 

Then, we write a simple loop that reads product codes one by one. When an empty input is received, it means there are no more products, and the total price is calculated:

total_price = Decimal(0)

while True:
    product_code = input("Product code: ")

    if product_code == "":
        break
    else:
        price = prices.get(product_code)
        if price == None:
            print('Unknown product code')
        else:
            total_price += price

print(f'Total: {total_price:.2f}')

Running this, we can scan a couple of barcodes and process a sale in a jiffy:

Product code: 9780201633542
Product code: 9780201633467
Product code: 
Total: 64.65

Building a Book Catalogue

Just like a shop can build a database by scanning barcodes and enhancing them with additional information, anyone can. In fact, you can find several online databases where you can look up barcodes, such as the aforementioned ISBN Search, EAN-Search, Barcode Lookup, and others. Some even offer APIs so that you can automate the lookup and build a fully-featured database very quickly by just scanning a bunch of barcodes.

Scanning a book’s ISBN number with the Handy Library mobile app.

If you run a library or are just a private individual with a lot of books, you can use this approach to build a catalogue of your books. The easiest way is to use the Handy Library mobile app, which allows you to scan a book’s ISBN number and through it acquire all the book’s metadata too. If you prefer to have your data on a computer, you can export data from Handy Library, or else build your database yourself by scanning barcodes and looking up the information afterwards.

QR Codes

Barcodes may be great in retail, but most people don’t carry barcode scanners with them wherever they go. Now that everybody and his dog has a smartphone with a powerful camera, QR codes have emerged as a more recent standard to encode data that smartphones can easily scan with their default camera app. The most common use case is to encode a URL, allowing people to visit it by scanning it with their phone’s camera rather than typing it in. The following are a few creative uses I’ve come across in my travels:

  • Exhibitions: putting a QR code beside the label of an exhibit in an art gallery or museum allows people to scan it and visit a webpage with more information. It’s also possible to have this for multiple languages.
  • Paying bills: in Switzerland, the new standard way of paying bills is to scan a QR code containing all payment details with a bank’s mobile app.
  • Contact details: A long time ago, I had an idea that people could easily exchange contact details by scanning a QR code. Someone has probably done this already. Similar applications I’ve seen in the wild are physical business cards with QR codes that lead to the company or owner’s website, and the LinkedIn app which displays a QR code leading to the user’s profile.
  • Queue ticket info: when you take a ticket and wait your turn in some customer service departments, the ticket contains a QR code that tells you how many people are ahead of you and how long you can expect to wait. You could potentially run a quick errand and keep track of your place while you’re away.
  • Two-Factor Authentication: QR codes are the easiest way to initially exchange the secret in Time-Based One-Time Passwords (TOTP), as shown in “Using Time-Based One-Time Passwords for Two-Factor Authentication“.

Conclusion

Nowadays, we talk a lot about artificial intelligence, augmented reality, robotics the internet of things, and so on. Many of these fields are awe-inspiring, yet not very accessible to the man in the street.

On the other hand, barcodes have provided a very simple way for software to interact with physical objects for decades. The advantages in efficiency are hard to dispute, as can be seen in the retail industry.

The ubiquity of smartphones and the more recent widespread use of QR codes has enabled the same concept to extend to a lot of other use cases, allowing us to interact with some very traditional settings in novel ways.

Working with VS Code Launch Configurations

Visual Studio Code (VS Code) is a wonderful IDE. I’m not generally known to praise Microsoft’s products, but VS Code lets me develop and debug code productively in all the languages I’ve needed to work with, on any operating system. I don’t even use Visual Studio any more.

Launch configurations are at the heart of debugging with VS Code. In this article, I’ll explain how you can debug code using different languages, even at the same time. I’ll also show you how you can customise these launch configurations to pass command-line arguments, set environment variables, run pre-launch tasks, and more.

Getting Started with Launch Configurations

Anytime you want to debug something, you just press F5 (like in Visual Studio). Initially, you probably won’t have any launch configurations set up, in which case you’ll be prompted to choose what kind of launch configuration you want to create, as shown in the screenshot below. This list might vary depending on the language runtimes you have installed. When you select one, you’ll be guided towards creating a sample launch configuration.

Pressing F5 brings up a list of languages for which you can create launch configurations.

Another way is to click on the Debug tab on the left, which looks like a play button. You can then click the link to “create a launch.json” file, shown in the screenshot below. We’ll see this in practice in the next sections as we create launch configurations for various languages.

The Debug tab allows you to “create a launch.json file”.

Debugging Python

Before we debug anything, we need some code. Let’s add a folder with a Python file in it, and add the following code:

import time
import datetime

while True:
    time.sleep(1)
    print(f"Swiss church bells say it's {datetime.datetime.now()}")

If I press F5, this actually works for me out of the box:

Pressing F5 runs the Python program and lets us debug it.

Clicking next to a line number adds a breakpoint. When the program hits that point, it pauses and allows us to inspect variables and other things. You can press F10 to go to the next statement, F11 to step into a function call, and Shift+F11 to step out. While a tutorial on how to debug code is outside the scope of this article, if you’re unfamiliar with debugging, this should at least be enough to get you started.

After hitting a breakpoint in the Python program, we can advance step by step and see the state of local variables.

A Launch Configuration for Python

Follow either of the methods in the earlier “Getting Started” section to create your first Python launch configuration. This creates a launch.json file under a .vscode folder with the following contents. As launch.json may contain personally customised configurations for different developers (e.g. with different input parameters), it’s best not to commit it to source control.

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": true
        }
    ]
}

There isn’t much to it: this will simply run whatever file you have open in VS Code (which is what ${file} means). This is useful if you want to run specific files (I do this with tests in Go for instance), but not so much if you have a program with a single entry point and want to run that regardless of what you have open in VS Code. In that case, it’s easy to change the program value:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Main File",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/pyticker/main.py",
            "console": "integratedTerminal",
            "justMyCode": true
        }
    ]
}

${workspaceFolder} represents the folder we have open in VS Code, so this makes sure we’re relative to that. ${workspaceFolder}, ${file} and other such variables are documented in VS Code’s Variables Reference.

Command Line Arguments

Let’s modify our Python program as follows:

import time
import datetime
import sys

while True:
    time.sleep(int(sys.argv[2]))
    print(f"{sys.argv[1]} church bells say it's {datetime.datetime.now()}")

The program now takes the nationality of the church bells as well as the sleep interval from command-line arguments. We’re not doing validation or error-handling for the sake of brevity. The following is an example of how to execute this successfully from a terminal:

$ python3 main.py Maltese 5
Maltese church bells say it's 2023-02-15 19:04:28.311805
Maltese church bells say it's 2023-02-15 19:04:33.315787
Maltese church bells say it's 2023-02-15 19:04:38.319811

To pass the same command-line arguments when debugging with VS Code, we add args to the launch configuration:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Main File",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/pyticker/main.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": ["Maltese", "5"]
        }
    ]
}

Multiple Launch Configurations

As you no doubt noticed, launch.json contains a JSON array of configurations. That means it’s very easy to add more of these configurations, for instance, when you need to provide different inputs:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python Ticker (Maltese)",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/pyticker/main.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": ["Maltese", "5"]
        },
        {
            "name": "Python Ticker (Swiss)",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/pyticker/main.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": ["Swiss", "1"]
        }
    ]
}

From the Debug tab, you can then select the configuration you want to run from the drop-down before hitting F5 to debug with that configuration:

The drop-down in the Debug tab lets you select which Launch Configuration you want to debug.

A Launch Configuration for Node.js

Let’s add a new folder as a sibling to our Python folder, and add the following code in a new app.js file, completely disregarding cleanup for the sake of brevity and YOLO:

const fs = require('fs');

setInterval(() => {
    message = `Swiss church bells say it's ${new Date()}`;
    console.log(message)
    fs.appendFileSync('ding-dong.txt', message + '\n');
}, 1000);

We can set up a simple launch configuration to run this in launch.json:

...
    "configurations": [
        {
            "name": "Node.js Ticker",
            "program": "${workspaceFolder}/jsticker/app.js",
            "request": "launch",
            "type": "node"
        },
...

It works, writing output every second both to standard output and a file:

The Node.js program’s output can be seen in the Debug Console as well as the output file. The overall folder structure is also shown on the left.

The only problem is that the file was created top-level, and since we only specified a filename in the code (not a folder or path), then this must mean that Node.js is running with the top-level folder in VS Code as the current working directory. We can change that in the launch configuration using cwd:

...
    "configurations": [
        {
            "name": "Node.js Ticker",
            "program": "${workspaceFolder}/jsticker/app.js",
            "request": "launch",
            "cwd": "${workspaceFolder}/jsticker",
            "type": "node"
        },
...

When we run this again, the file is created under the jsticker folder.

Pre-Launch Tasks

Sometimes you want to run something before starting your program. For instance, in my work setup, I generate Swagger docs and perform other prerequisite tasks. But since we’re not doing anything that fancy, we’ll just delete the ding-dong.txt file every time we run the “Node.js Ticker” launch configuration.

To do this, we first need to add a tasks.json file inside the .vscode folder, next to launch.json, and add the following to it:

{
    "version": "2.0.0",
    "tasks": [
      {
        "label": "rmfile",
        "command": "rm",
        "args": ["ding-dong.txt"],
        "options":{
            "cwd": "${workspaceFolder}/jsticker"
        }
      }
    ]
  }

This defines a task called rmfile that will run rm ding-dong.txt from the jsticker folder. We then refer to this task in the relevant launch configuration using preLaunchTask:

...
        {
            "name": "Node.js Ticker",
            "program": "${workspaceFolder}/jsticker/app.js",
            "request": "launch",
            "cwd": "${workspaceFolder}/jsticker",
            "preLaunchTask": "rmfile",
            "type": "node"
        },
...

A Launch Configuration for Go (GoLang)

For Go, we need to:

  1. Run go work init from the top-level folder opened in VS Code
  2. Create a new folder alongside pyticker and jsticker
  3. Run go mod init main in it
  4. Add the following code to a new main.go file:
package main

import (
	"fmt"
	"time"
)

func main() {
	for {
		now := time.Now()
		fmt.Printf("Swiss bells won't stop ringing! It's %s!\n", now)
		time.Sleep(time.Second)
	}
}

We can now add a launch configuration for this Go program:

...
        {
            "name": "Go Ticker",
            "type": "go",
            "request": "launch",
            "mode": "debug",
            "program": "${workspaceFolder}/goticker/main.go"
        },
...

And we can have lots of fun running and debugging it, as a reminder that Swiss church bells need to tell you the time all the time. It’s not like they sell watches in Switzerland or anything.

Our Go program runs via the “Go Ticker” launch configuration.

Environment Variables

As we’ve seen with the Python example, however, other countries also have church bells. Instead of using command-line arguments to customise the output, we’ll instead pass environment variables. First, we need to modify the code a little:

package main

import (
	"fmt"
	"os"
	"strconv"
	"time"
)

func main() {
	for {
		country := os.Getenv("COUNTRY")
		intervalSecsStr := os.Getenv("INTERVAL_SECS")
		intervalSecs, _ := strconv.Atoi(intervalSecsStr)
		now := time.Now()
		fmt.Printf("%s bells won't stop ringing! It's %s!\n", country, now)
		time.Sleep(time.Duration(intervalSecs) * time.Second)
	}
}

Then we add the relevant environment variables as key-value pairs in an env block in the launch configuration:

...
        {
            "name": "Go Ticker",
            "type": "go",
            "request": "launch",
            "mode": "debug",
            "program": "${workspaceFolder}/goticker/main.go",
            "env": {
                "COUNTRY": "Maltese",
                "INTERVAL_SECS": "3"
            }
        },
...

When you run it again, it makes all the difference:

The Go program runs with environment variables set.

Go Build Flags

Go has some specific build flags that can’t be passed as regular command-line parameters, such as build tags or the Data Race Detector. If you want to use these, you’ll have to pass them via buildFlags instead:

...
            "buildFlags": "-race",
...

Running Multiple Programs with Compounds

It’s common to need to run multiple programs at once, especially in a microservices architecture, or if there are separate backend and frontend applications. This can be done in VS Code using compound launch configurations. For instance, if we wanted to run both the Python and Go programs, we could define a compound as follows (compounds go after configurations in launch.json):

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Go Ticker",
            "type": "go",
            "request": "launch",
            "mode": "debug",
            "program": "${workspaceFolder}/goticker/main.go",
            "env": {
                "COUNTRY": "Maltese",
                "INTERVAL_SECS": "3"
            }
        },
        {
            "name": "Node.js Ticker",
            "program": "${workspaceFolder}/jsticker/app.js",
            "request": "launch",
            "cwd": "${workspaceFolder}/jsticker",
            "preLaunchTask": "rmfile",
            "type": "node"
        },
        {
            "name": "Python Ticker (Maltese)",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/pyticker/main.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": ["Maltese", "5"]
        },
        {
            "name": "Python Ticker (Swiss)",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/pyticker/main.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": ["Swiss", "1"]
        }
    ],
    "compounds": [
        {
            "name": "Go and Python Tickers",
            "configurations": ["Go Ticker", "Python Ticker (Swiss)"]
        }
    ]
}

The compound simply refers to the individual launch configurations by name, and has a name of its own. It can then be selected from the drop-down in the Debug tab just like any other launch configuration. By running a compound, you can debug and hit breakpoints in any application that is part of that compound.

In fact, you’ll notice there is a new drop-down next to the debugging buttons towards the central-top part of the screen. This allows you to switch certain views (e.g. the Debug Console) between running applications.

A compound can be selected from the list of launch configurations in the Debug tab. When it is run, a dropdown appears next to the debugging buttons in the top part of the screen.

Conclusion

We’ve seen that Launch Configurations allow you to run and debug applications in VS Code with great flexibility, including:

  • Run the current file or a specific one
  • Set the current working directory
  • Run different programs
  • Use different programming languages
  • Set up different configurations even for the same program
  • Pass command-line arguments
  • Set environment variables
  • Run/debug multiple programs at the same time

This provides a great IDE experience even for more complex application architectures in a monorepo.