Go is a relatively young programming language, launched in 2009, with simplicity at its heart. Whereas it might not have all the bells and whistles you might expect from a modern programming language (see “From .NET to GoLang: Where Did Everything Go?” and “From .NET to GoLang: Here We Go Again” for my critique), it has some features that make it unique, such as its first-class concurrency model using channels and goroutines, based on Tony Hoare’s 1978 Communicating Sequential Processes (CSP).
You can set up Go and write your first program easily in a couple of minutes, but in order to program comfortably, I recommend using an Integrated Development Environment (IDE) that you can use to debug your code. You can use whatever you like, but I’ll show you how to use Visual Studio Code (VS Code) to write and debug Go code.
Download and Install Go
First, you need to download and install Go. This link has all the instructions you need for Linux, Mac and Windows, and they’re pretty straightforward, so I won’t bother repeating all of them here.
If you’re on Linux, though, do note that the first command is actually two commands combined with an && operator, so you need to add sudo before both of them. To make it super clear, you need to do the following steps on Linux:
Run the following to remove any previous Go installation (skip this if you’ve never installed Go before):
sudo rm -rf /usr/local/go
cd into the directory where you downloaded the Go package.
Run the following to extract the Go package and put it in the right folder:
sudo tar -C /usr/local -xzf go1.20.3.linux-amd64.tar.gz
Be sure to change the filename to the one you actually downloaded.
Add the following to the end of ~/.profile:
export PATH=$PATH:/usr/local/go/bin
Restart your computer for the change to take effect.
Run go version to ensure that the go command is now accessible in your terminal.
A Minimal Example
Create a new folder, and name it whatever you like (e.g. “hellogo”). In it, add a file called main.go, and add the following code in it:
package main
import "fmt"
func main() {
fmt.Println("Hello Go!")
}
Using Go from the Command Line
You can run the above program in your favourite terminal by cd’ing into the directory where you added main.go, and running the following command:
go run main.go
The output, as you would expect, is:
Hello Go!
If you want to compile the code into an executable, you can run the following command:
go build main.go
This generates an executable named main. If you want to specify the name of the executable, you can add a -o switch and provide the name you want. For instance, the following command generates an executable named myapp:
go build -o myapp main.go
It’s important to note that the file being compiled must be at the end of the command, so the -o switch goes before it.
To run the executable:
On Windows, run: myapp
On Linux or Mac, run: ./myapp
Setting up VS Code for Go
If you don’t have it already, download and install Visual Studio Code. Use it to open the folder where main.go is. At this point, we need to install 3 things:
In my experience, VS Code automatically prompts you to install these when you open a .go file in it, so the easiest way is to just accept. In case it doesn’t, you can install the Go extension yourself from the Extensions tab on the left side of VS Code, and you can install the other two by following their respective instructions.
VS Code asks whether you want to install the Go extension. Click “Install”.
You can also find the Go extension from the Extensions tab on the left. After installing it, VS Code helpfully asks you to install dlv (Delve) and gopls, which you also need.
When you accept to install Delve and gopls, you can see their progress and status in the Output window at the bottom of VS Code.
Debugging Go
Once you’ve installed all three items from the previous section, VS Code becomes a handy tool to write, run and debug Go code with ease. In fact, you get things like syntax highlighting, code completion, and the ability to debug your code.
Code completion for Go code.
To run/debug your code directly from VS Code, you can press F5, or do it via the “Run and Debug” tab (which looks like a Play button with a beetle) on the left side of VS Code. Doing this immediately gives us an error:
The error says: “go mod file not found in current directory or any parent directory”.
That’s because we missed a step. In the bottom part of VS Code, switch to the integrated terminal (“Terminal” tab) or, if you don’t have one, open a new one via the Terminal menu at the top of VS Code. Then, run the following command:
go mod init main
This creates a go.mod file:
Running go mod init main creates a go.mod file, initialising the main module.
You can now press F5 again to debug Go. You can set a breakpoint by clicking next to a line number; this will show the breakpoint as a red circle, and the code will stop when it reaches that point, allowing you to step through the code, inspect variables, and see other parts of the state of the program:
Hitting a breakpoint while debugging Go code.
There isn’t much to debug in this simple program, but at least this gives you what you need to be able to debug more interesting programs as you write them. Debugging is an entire topic in itself that’s out of scope for this article, but it’s an essential skill for a competent developer. You can learn more about it from:
Working with VS Code Launch Configurations – another article here at Gigi Labs showing how you can configure the way you run your programs in VS Code (e.g. passing command line parameters).
One other tip: while you are in a debugging session, VS Code creates a temporary executable called __debug_bin. If you’re using source control, be sure to exclude it (e.g. add it to your .gitignore file when using Git).
Conclusion
Go is fairly straightforward to set up and use:
Download Go and follow the official instructions to set it up.
Use Go from the command-line if you need to (e.g. for Continuous Integration).
Set up the Go extension, Delve debugger and gopls language server in VS Code.
I recently needed to develop a small application for personal use. Since I had a specific need to use Python, I decided to build the backend with Flask. For the frontend, I only needed (literally) a single page to visualise some data, so I resisted the urge to go with something like Angular or React, and instead decided to try this minimal Alpine.js thing I had heard about a few days earlier.
In this article, I’m going to show you how you can really quickly create a minimal web application (REST API, static files and web UI) like I did. This can be handy for putting a simple tool together in very little time, but please consider it as a quick and dirty thing rather than any kind of best practice. I’m new to both Flask and Alpine.js, so I’m certainly not going to be giving advice on how best to use them.
Getting Started with Flask
After creating a new folder for your project, create a file called main.py (or whatever you want, really). We can kick off our API development by using the Flask Quickstart example:
This is nice and all, but I like to be able to debug my code from Visual Studio Code, and for that it’s necessary to create a launch.json file with the right settings. This can be done as illustrated by the following images and their captions. If you need to do anything more advanced, see “Working with VS Code Launch Configurations“.
First, select the Debug tab on the left, and click “create a launch.json” file.
From the list that appears, select “Flask”.
Change the default suggested filename of app.py to the one you’re using, e.g. main.py.
Once that is done, it generates a launch.json file with the following contents:
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python: Flask",
"type": "python",
"request": "launch",
"module": "flask",
"env": {
"FLASK_APP": "main.py",
"FLASK_DEBUG": "1"
},
"args": [
"run",
"--no-debugger",
"--no-reload"
],
"jinja": true,
"justMyCode": true
}
]
}
You can now press F5 to start the application in debug mode. The integrated terminal informs you that the application is running at http://localhost:5000/, so you can open a browser window to see that it’s working. By running it in this way, you can also add breakpoints and debug your Python code as needed.
The Flask “Hello World” is definitely working.
A Simple REST API
To build a reasonable example, we need to return some data from our API. We can do this quite easily by returning a list of dictionary objects from a function annotated with the route “/products”:
@app.route("/products")
def get_products():
return [
{
"name": "Rope",
"description": "Used to climb heights or swing across chasms.",
"price": 15
},
{
"name": "Whip",
"description": "An old and trusty friend.",
"price": 20
},
{
"name": "Notebook",
"description": "Contains lots of valuable information!",
"price": 80
}
]
After restarting the application and visiting http://localhost:5000/products, we can see that the list gets translated quite nicely to JSON:
The response of the /products endpoint.
Serving a Static Webpage
It is now time to build a simple user interface. Before we do that though, we need to figure out how to serve static files so that we can return an HTML file and then use Alpine.js to dynamically populate it using the API.
We can now see this page if we visit http://localhost:5000/static/index.html in a browser:
The HTML page is served from the static folder.
It works, but it’s not great. I’d like this to come up by default when I visit http://localhost:5000/, instead of having such a long URL. Fortunately, this is easy to achieve. All we need to do is replace our “Hello world” code in the default endpoint with the following:
Now that we have an API and an easily accessible webpage, we can display the data after retrieving it from the API. We can now add Alpine.js to our website by adding a <script> tag, and then build out a table that retrieves data from the API:
In the above HTML, the main thing to look at is the stuff inside the <div>. Outside of that, the new things are just the <script> tag that imports Alpine.js, and a little CSS to make the webpage look almost bearable.
The <div> tag’s attributes set up the data model that the table is based on. Using this x-data attribute, we’re saying that we have an array called “products”. The x-init attribute is where we use the Fetch API to retrieve data from the /products endpoint and assign the result to the “products” array.
Once this data is retrieved, all that’s left to do is display it. In the <table>, we use an x-for attribute in a <template> to loop over the “products” array. Then, we bind the content of each cell (<td> element) to the properties of each product using the x-text attribute, effectively displaying them.
The result is simple but effective:
Indiana Jones would be proud of this inventory.
Conclusion
As you can see, it’s pretty quick and easy to put something simple together and then build whatever you need on top of that. For instance, you might want to filter the data you display, apply conditional highlighting, or add other functionality.
Honestly, I’m not quite sure how far you can take Alpine.js. For instance, it doesn’t seem possible to create reusable components, which somewhat limits its usefulness. And I don’t yet know how to avoid having heaps of JavaScript within HTML attributes. I’ve only started using this a few days ago, after all.
But it’s great to be able to use something like this to create simple tools without having to incur the learning curves of things like Angular, React or Vue.
There are lots of different ways to encode data in software. For instance, text encodings such as ASCII or UTF-8 allow us to represent strings of arbitrarily complex characters, while base64 allows us to represent even binary data as text and transmit it over channels that are more suited to deal with text (such as many internet protocols).
A barcode scanner next to a stack of books, each with its own barcode.
Barcodes, on the other hand, allow us to label physical objects and read them efficiently, providing a bridge between software and the physical world.
Although barcodes could represent any data, they are usually used to represent numbers. They are most commonly used in retail for fast processing of items being sold, especially by supermarkets, but also by pharmacies, bookshops, electronics stores, and other shops.
Many items we buy, from everyday groceries to books, are labelled with numbers (which I’ll call “product codes”) that form part of one or more standards, such as EAN-13, UPC-A, or ISBN. These numbers are encoded as a barcode on the product itself or its packaging. Several websites exist where you can look up these numbers. Other products, such as fruit, do not have standard identifiers, but that doesn’t prevent shops from applying their own.
Barcode Scanners
Barcode scanners can be used to read these product codes. They are quite cheap and can connect via USB to any system, making them widely compatible with Windows, Linux, MacOS, and various cash registers. A barcode scanner is held like a gun and, when the trigger is pulled, it emits a red light that reads the information on the barcode. This is immediately written to any text field that currently has focus, followed by a newline. For instance, if I open a text editor and quickly scan the barcodes on the four books shown in the above picture, I get the following:
Scanning these numbers into a text file is useful for testing or to later bulk-process them, but we can also use this handy input method in other ways. For instance, if we open ISBN Search and click on the search field, scanning a book’s barcode will not only populate the search field with the ISBN number, but also execute the search:
ISBN Search result for 9780201633542 is “TCP/IP Illustrated: The Implementation, Vol. 2”.
Barcodes in Retail
Shops that sell products with barcodes on them can use those barcodes as primary keys to identify each product. This makes it very easy to create a database with information about the various products (including name, price, taxes, discounts, etc) and then have the cashier look this up by just entering the barcode. We can simulate this by writing a simple program, e.g. in Python.
First, we define a simple mapping between each product’s barcode and its price:
Then, we write a simple loop that reads product codes one by one. When an empty input is received, it means there are no more products, and the total price is calculated:
Just like a shop can build a database by scanning barcodes and enhancing them with additional information, anyone can. In fact, you can find several online databases where you can look up barcodes, such as the aforementioned ISBN Search, EAN-Search, Barcode Lookup, and others. Some even offer APIs so that you can automate the lookup and build a fully-featured database very quickly by just scanning a bunch of barcodes.
Scanning a book’s ISBN number with the Handy Library mobile app.
If you run a library or are just a private individual with a lot of books, you can use this approach to build a catalogue of your books. The easiest way is to use the Handy Library mobile app, which allows you to scan a book’s ISBN number and through it acquire all the book’s metadata too. If you prefer to have your data on a computer, you can export data from Handy Library, or else build your database yourself by scanning barcodes and looking up the information afterwards.
QR Codes
Barcodes may be great in retail, but most people don’t carry barcode scanners with them wherever they go. Now that everybody and his dog has a smartphone with a powerful camera, QR codes have emerged as a more recent standard to encode data that smartphones can easily scan with their default camera app. The most common use case is to encode a URL, allowing people to visit it by scanning it with their phone’s camera rather than typing it in. The following are a few creative uses I’ve come across in my travels:
Exhibitions: putting a QR code beside the label of an exhibit in an art gallery or museum allows people to scan it and visit a webpage with more information. It’s also possible to have this for multiple languages.
Paying bills: in Switzerland, the new standard way of paying bills is to scan a QR code containing all payment details with a bank’s mobile app.
Contact details: A long time ago, I had an idea that people could easily exchange contact details by scanning a QR code. Someone has probably done this already. Similar applications I’ve seen in the wild are physical business cards with QR codes that lead to the company or owner’s website, and the LinkedIn app which displays a QR code leading to the user’s profile.
Queue ticket info: when you take a ticket and wait your turn in some customer service departments, the ticket contains a QR code that tells you how many people are ahead of you and how long you can expect to wait. You could potentially run a quick errand and keep track of your place while you’re away.
Nowadays, we talk a lot about artificial intelligence, augmented reality, robotics the internet of things, and so on. Many of these fields are awe-inspiring, yet not very accessible to the man in the street.
On the other hand, barcodes have provided a very simple way for software to interact with physical objects for decades. The advantages in efficiency are hard to dispute, as can be seen in the retail industry.
The ubiquity of smartphones and the more recent widespread use of QR codes has enabled the same concept to extend to a lot of other use cases, allowing us to interact with some very traditional settings in novel ways.
Puppeteer is an API enabling browser automation via Chrome/Chromium. Quoting its homepage:
“Puppeteer is a Node.js library which provides a high-level API to control Chrome/Chromium over the DevTools Protocol. Puppeteer runs in headless mode by default, but can be configured to run in full (non-headless) Chrome/Chromium.”
I’ve already shown it in action a couple of years ago in my article, “Gathering Net Salary Data with Puppeteer“, where I used it for web scraping. Browser automation is also very common for automated testing of web applications, and may also be used for a lot of other things.
As with any other piece of software, it is sometimes convenient to package a Puppeteer script in a Docker container. However, since deploying a browser is fundamentally more complicated than your average API, Puppeteer and Docker are a little tricky to get working together. In this article, we’ll see why this combination is problematic and how to solve it.
A Minimal Puppeteer Example
Before we embark upon our Docker journey, we need a simple Puppeteer program we can test with. The easiest thing we can do is use Puppeteer to open a webpage and take a screenshot of it. We can do this quite easily as follows.
First, we need to create a folder, install prerequisites, and create a file in which to put our JavaScript code. The following bash script takes care of all this, assuming you already have Node.js installed. Please note that I am working on Linux Kubuntu 22.04, so if you’re using a radically different operating system, the steps may vary a little.
mkdir pupdock
cd pupdock
npm install puppeteer
touch main.js
To see that it works, run this script using the following command:
node main.js
A file called screenshot.png should be saved in the same folder:
A screenshot of my older tech blog, Programmer’s Ranch, complete with Blogger’s stupid cookie banner.
An Initial Attempt with Docker
Now that we know that the script works, let’s try and make a Docker image out of it. Add a Dockerfile with the following contents:
FROM node:19-alpine3.16
WORKDIR /puppeteer
COPY main.js package.json package-lock.json ./
RUN npm install
CMD ["node", "main.js"]
What we’re doing here is starting from a recent (at the time of writing this article) Node Docker image. Then we set the current working directory to a folder called /puppeteer. We copy the script along with the list of package dependencies (Puppeteer, basically) into the image, install those dependencies, and set up the image to execute node main.js when it is run.
We can build the Docker image using the following command, giving it the tag of “pupdock” so that we can easily find it later. The sudo command is only necessary if you’re running on Linux.
sudo docker build -t pupdock .
Once the image is built, we can run a container based on this image using the following command:
sudo docker run pupdock
Unfortunately, we hit a brick wall right away:
/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/node/BrowserRunner.js:300
reject(new Error([
^
Error: Failed to launch the browser process! spawn /root/.cache/puppeteer/chrome/linux-1108766/chrome-linux/chrome ENOENT
TROUBLESHOOTING: https://pptr.dev/troubleshooting
at onClose (/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/node/BrowserRunner.js:300:20)
at ChildProcess.<anonymous> (/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/node/BrowserRunner.js:294:24)
at ChildProcess.emit (node:events:512:28)
at ChildProcess._handle.onexit (node:internal/child_process:291:12)
at onErrorNT (node:internal/child_process:483:16)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Node.js v19.8.1
The error and stack trace are both pretty cryptic, and it’s not clear why this failed. There isn’t much you can do other than beg Stack Overflow for help.
Fortunately, I’ve done this already myself, and I can tell you that it fails because Chrome (which Puppeteer is trying to launch) needs more security permissions than Docker provides by default. We’ll need to learn a bit more about Docker security to make this work.
Dealing with Security Restrictions
So how do we give Chrome the permissions it needs, without compromising the security of the system it is running on? We have a few options we could try.
Disable the sandbox. Chrome uses a sandbox to isolate potentially harmful web content and prevent it from gaining access to the underlying operating system (see Sandbox and Linux Sandboxing docs to learn more). Many Stack Overflow answers suggest getting around errors by disabling this entirely. Unless you know what you’re doing, this is probably a terrible idea. It’s far better to relax security a little to allow exactly the permissions you need than to disable it entirely.
Use the Puppeteer Docker image. Puppeteer’s documentation on Docker explains how to use a Puppeteer’s own Docker images (available on the GitHub Container Registry) to run arbitrary Puppeteer scripts. While there’s not much info on how to work with these (e.g. which folder to mount as a volume in order to grab the generated screenshot), what stands out is that this approach requires the SYS_ADMIN capability, which exposes more permissions than we need.
Our initial attempt with a simple Dockerfile didn’t go very well, but now we have a number of other Dockerfiles we could start with, including the Puppeteer Docker image’s Dockerfile, a couple in the aforementioned doc section on running Puppeteer in Docker (one built on a Debian-based Node image, and another based on an Alpine image), and several other random ones scattered across Stack Overflow answers.
My preference is the Alpine one, not only because Alpine images tend to be smaller than their Debian counterparts, but also because I had more luck getting it to work across Linux, Windows Subsystem for Linux and an M1 Mac than I did with the Debian one. So let’s replace our Dockerfile with the one in the Running on Alpine section, with a few additions at the end:
FROM alpine
# Installs latest Chromium (100) package.
RUN apk add --no-cache \
chromium \
nss \
freetype \
harfbuzz \
ca-certificates \
ttf-freefont \
nodejs \
yarn
# Tell Puppeteer to skip installing Chrome. We'll be using the installed package.
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser
# Puppeteer v13.5.0 works with Chromium 100.
RUN yarn add puppeteer@13.5.0
# Add user so we don't need --no-sandbox.
RUN addgroup -S pptruser && adduser -S -G pptruser pptruser \
&& mkdir -p /home/pptruser/Downloads /app \
&& chown -R pptruser:pptruser /home/pptruser \
&& chown -R pptruser:pptruser /app
# Run everything after as non-privileged user.
USER pptruser
WORKDIR /puppeteer
COPY main.js ./
CMD ["node", "main.js"]
Because the given Dockerfile already installs the puppeteer dependency and that’s all we need here, I didn’t even bother to do the usual npm install here, although a more complex script might possibly have additional dependencies to install.
At this point, we can build the Dockerfile and run the resulting image as before:
sudo docker build -t pupdock .
sudo docker run pupdock
The result is that it still doesn’t work, but the error is something that is more easily Googled than the one we have before:
/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:237
reject(new Error([
^
Error: Failed to launch the browser process!
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
at onClose (/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:237:20)
at ChildProcess.<anonymous> (/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:228:79)
at ChildProcess.emit (node:events:525:35)
at ChildProcess._handle.onexit (node:internal/child_process:291:12)
Node.js v18.14.2
A Little Security Lesson
Googling that error about PID namespaces is what led me to the solution, but it still took a while, because I had to piece together many clues scattered in several places, including:
Answer by usethe4ce suggests downloading some chrome.json file and passing it to Docker, but it’s not immediately clear what this is/does.
Answer by Riccardo Manzan, based on the one by usethe4ce, provides an example Dockerfile based on Node (not Alpine) and also shows how to pass the chrome.json file both in docker run and docker-compose.
Answer by hidev lists the five system calls that Chrome needs.
To understand all this confusion, we first need to take a step back and understand something about Docker security. Like the Chrome sandbox, Docker has its ways of restricting the extent to which a running Docker container can interact with the host.
Like any Linux process, a container requests whatever it needs from the operating system kernel using system calls. However, a container could be used to wreak havoc on the host if it is allowed to run whatever system calls it wants and then is successfully breached by an attacker. Fortunately, Linux provides a feature called seccomp that can restrict system calls to only the ones that are required, minimising the attack surface.
In Docker, this restriction is applied by means of a seccomp profile, basically a JSON file whitelisting the system calls to be allowed. Docker’s default seccomp profile restricts access to system calls enough that it prevents many known exploits, but this also prevents more complex applications that need additional system calls – such as Chrome – from working under Docker.
That chrome.json I mentioned earlier is a custom seccomp profile painstakingly created by one Jess Frazelle, intended to allow the system calls that Chrome needs but no more than necessary. This should be more secure than disabling the sandbox or running Chrome with the SYS_ADMIN capability.
A Third Attempt with chrome.json
Let’s give it a try. Download chrome.json and place it in the same folder as main.js, the Dockerfile and everything else. Then, run the container as follows:
sudo docker run --security-opt seccomp=path/to/chrome.json pupdock
This time, there’s no output at all – which is good, because it means the errors are gone. To ensure that it really worked, we’ll grab the screenshot from inside the stopped container. First, get the container ID by running:
sudo docker container ls -a
Then, copy the screenshot from the container to the current working directory as follows, taking care to replace 7af0d705a751 with the actual ID of the container:
Puppeteer worked: the screenshot contains the full length of the page.
Additional Notes
I omitted a few things to avoid breaking the flow of this article, so I’ll mention them here briefly:
That SYS_ADMIN capability we saw mentioned earlier belongs to another Linux security feature: capabilities, which are also related to seccomp profiles.
If you run into other issues, use the dumpio setting to get more verbose output. It’s a little hard to separate the real errors from the noise, but it does help. Be sure to run in headless mode (it doesn’t make sense to run the GUI browser under Docker), and disable the GPU (--disable-gpu) if you see related errors.
Conclusion
Running Puppeteer under Docker might sound like an unusual requirement, perhaps overkill, but it does open up a window onto the interesting world of Docker security. Chrome’s complexity requires that it be granted more permissions than the typical Docker container.
It is unfortunate that the intricacies around getting Puppeteer to work under Docker are so poorly documented. However, once we learn a little about Docker security – and the Linux security features that it builds on – the solution of using a custom seccomp profile begins to make sense.
"You don't learn to walk by following rules. You learn by doing, and by falling over." — Richard Branson
