Tag Archives: Security

TOTP and Authenticator Apps for 2FA in Go

Back in 2019, I wrote “Using Time-Based One-Time Passwords for Two-Factor Authentication“, explaining how to allow a server written in C# to generate and verify codes in sync with authenticator apps. In this article, I’ll show how to do the same thing in Go.

How It Works

Logins that use Two-Factor Authentication (2FA) typically rely on the user’s password as the first factor, as well as a second factor that could be anything from an email/SMS to something biometric. Among all these options, the Time-Based One-Time Password (TOTP) protocol is a simple way to provide the benefits of 2FA without the disadvantages of other options (such as SMS gateways being unreliable and expensive, and biometrics being complex to implement).

The way TOTP works is that the client and server each generate a code, and these have to match. The codes are generated based on a shared secret and the current time, so as long as the client and server are using the same secret and generate the codes at roughly the same time, they will match.

The client usually takes the form of an authenticator app – such as Google Authenticator, Microsoft Authenticator, or Authy – on a mobile device. Once it acquires the shared secret from the server – typically by scanning a QR code – it then generates codes every 30 seconds. Usually the server will accept that code during these 30 seconds and also for an additional 30 seconds while the next code is displayed.

This might all sound a little abstract, but it will become clear shortly once we go through implementing TOTP on the server side and testing it out.

TOTP in Go – Getting Started

Before looking at TOTP, let’s first do the usual steps to create a simple program in Go:

  • Create a new folder, e.g. go-otp
  • Create a new file in it, e.g. main.go
  • Run go mod init main
  • Add a basic program stub:
package main

func main() {

}

In this article, we’ll be using the excellent otp library by Paul Querna. Install it as follows:

go get github.com/pquerna/otp

Now we can start adding code in that main() function.

Generating a Shared Secret (Server Side)

To generate a shared secret, you call totp.Generate(), passing in a totp.GenerateOpts object. The Issuer and AccountName are required, and you can set other options if you need to.

	// generate shared secret

	options := totp.GenerateOpts{
		Issuer:      "My Web Application",
		AccountName: "me@example.com",
	}

	key, err := totp.Generate(options)
	if err != nil {
		fmt.Println("Failed to generate shared secret: ", err)
		return
	}

In order to do something with that key object, you have to convert it to base32, and you do that by simply calling its Secret() method:

	secret := key.Secret()
	fmt.Println(secret)

The output will change every time you run this, but the following is an example of what it looks like. In a real implementation, this would be saved in a database for each user.

QS4MXEE5AKT5NAIS74Z2AT3JPAGNDW7V

Generating QR Codes (Server Side)

The next step is to transmit the shared secret to the user. Normally, at some point during the registration process, the server generates the shared secret for the user, and sends it to the frontend, which displays it as a QR code. The user then scans the QR code with their authenticator app.

Although the server has no need to generate QR codes, it’s useful to do this for testing purposes, and it’s certainly faster than typing in the shared secret manually (which apps usually allow as a fallback option).

The otp library’s example code shows how to generate a QR code from the shared secret. Adapting this example a little, we get:

	// generate QR code

	var buf bytes.Buffer
	img, err := key.Image(200, 200)
	if err != nil {
		fmt.Println("Failed to save QR code: ", err)
		return
	}
	png.Encode(&buf, img)
	os.WriteFile("qr-code.png", buf.Bytes(), 0644)

Running this code generates a neat little QR code:

QR code for the shared secret.

Acquiring the Shared Secret (Client Side)

We can now pass the QR-encoded secret to the client. For this, we’ll need an authenticator app. Google Authenticator, Microsoft Authenticator and Authy should all work, but I personally prefer the latter because of its cloud backup feature, which is handy if your phone dies or goes missing.

Once the app is installed, locate the option to add a new account, after which you’ll be able to add a shared secret either by scanning a QR code or typing it manually. Excuse the quality of the next two pictures; none of these authenticator apps allow screenshots to be taken, so the closest thing I could do was take a photo with another phone.

Adding an account in Authy.

Generating Authenticator Codes (Client Side)

Once your app has acquired the shared secret, it will automatically start generating 2FA codes, typically six digits long at 30-second intervals:

Authy generating 2FA codes.

Alternatively, for testing purposes or if you need to develop a client with similar functionality, you can generate similar 2FA codes using the otp library itself. To do this, simply call totp.GenerateCode(), passing in the shared secret you generated earlier, and the current time:

	// generate 2FA code

	secondFactorCode, err := totp.GenerateCode(secret, time.Now())
	if err != nil {
		fmt.Println("Failed to generate 2FA code: ", err)
		return
	}
	fmt.Println(secondFactorCode)

The output is the same kind of six-digit 2FA code that you’d get with an authenticator app, for instance:

605157

In fact, if you run this while you also have the authenticator app set up, you’ll see the exact same codes.

Validating 2FA Codes (Server Side)

To complete the second factor part of the login process, a user grabs a 2FA code from their authenticator app and enters it in the prompt in the web application’s frontend, essentially sending it over to the server. The server’s job is to ensure that the code is valid. The otp library allows us to do this validation using the totp.Validate() function, which takes the input 2FA code, and the shared secret from earlier.

In order to test how this behaves over a period of time, we can do a little infinite loop that requests 2FA codes from the user and then validates them:

	for {
		fmt.Print("Enter 2FA code: ")

		var input2faCode string
		fmt.Scanln(&input2faCode)

		codeIsValid := totp.Validate(input2faCode, secret)
		fmt.Println("Code is valid: ", codeIsValid)
	}

While playing with this, you’ll notice that:

  • Entering a code shown by the authenticator app returns true
  • Entering the same code twice returns true
  • Entering the same code still returns true within 30 seconds after it has stopped displaying in the authenticator app
  • Entering the same code after that returns false
  • Entering anything else (other than the current or previous code) returns false

In theory, you should prevent the application from accepting the same code twice, because they’re supposed to be one-time-passwords. To do this, you’ll need to add additional state and logic to your application. Or, you can accept the security tradeoff knowing that the 2FA codes are only valid for a minute, giving little opportunity for an attacker to exploit them.

Summary

My earlier article showed how easy it is to work with 2FA codes using a TOTP library in C#, and in Go it’s no different. After generating a secret from the otp library, you can either share it with your authenticator app (e.g. by generating a QR code) to generate 2FA codes, or use it to generate 2FA codes directly. The library also allows you to validate the generated 2FA codes, assuming they are valid for two 30-second windows.

In the interest of simplicity, I’ve only shown basic usage of the library. If you have more complex requirements, you can customise the generation of secrets, generation of 2FA codes, and also the validation of 2FA codes.

Finally, one thing to keep in mind is that 2FA becomes a major headache when one of the factors goes missing (e.g. phone is lost or stolen). So you’ll need to consider how to handle this situation (e.g. recovery codes, reset shared secret, etc.), and how to handle normal password resets (you may want to reset the shared secret too).

Migrating Cartography to Memgraph

I’ve recently written about how to use Cartography to collect infrastructural data from both AWS and Okta into a Neo4j graph database for security analysis.

Neo4j is a long-standing player in the graph database market, with a robust product, great documentation, and a massive following. However, its long legacy is in a way also a disadvantage, as it can be costly, slow, and resource-hungry (due in no small part to its reliance on the JVM). Sometimes people would like to use an alternative for any of these reasons.

Memgraph, on the other hand, is a relatively young graph database, and certainly not as fully-featured as Neo4j. A key difference is that it is written in C++, meaning it’s designed to be faster and more lightweight than Neo4j (whether it lives up to this is something you’ll need to evaluate for your own use cases). Memgraph also made a very wise decision to support the Bolt protocol and the Cypher language – both of which Neo4j uses – meaning that it’s compatible with existing Neo4j clients and queries. Although there are variations in Cypher dialect, the incompatibilities are few, and moving from Neo4j to Memgraph is significantly less painful than, say, transitioning to a graph database that uses Gremlin as its query language.

At the time of writing this article, Cartography requires Neo4j 4.x, and does not work with Memgraph. However, I’m going to show you how to make at least part of it (the Okta intel module) work with minor alterations to the Cartography codebase. This serves as a demonstration of how to get started migrating an existing application from Neo4j to Memgraph.

Running Memgraph

Before we start looking at Cartography, let’s run an instance of Memgraph. To do this, we’ll take a tip from my earlier article, “Using the Neo4j Bolt Driver for Python with Memgraph“, and run it under Docker as follows (drop the sudo if you’re on Mac or Windows):

sudo docker run --rm -it -p 7687:7687 -p 3000:3000 -e MEMGRAPH="--bolt-server-name-for-init=Neo4j/" memgraph/memgraph-platform

That --bolt-server-name-for-init=Neo4j/ is a first critical step in Neo4j compatibility. As explained in that same article, the Neo4j Bolt Driver (i.e. client) for Python (which Cartography uses) checks whether the server sends an “agent” value that starts with “Neo4j/”. By setting this, Memgraph is effectively posing as a Neo4j server, and the Neo4j Bolt Driver for Python can’t tell the difference.

Update 19th September 2023: as of Memgraph v2.11, --bolt-server-name-for-init has a default value compatible with the Neo4j Bolt Driver, and therefore no longer needs to be provided.

If it’s successful, you should see output such as the following:

Memgraph is running. You can also execute queries directly from here.

Cloning the Cartography Repo

The next thing to do is grab a copy of the Cartography source code from the Cartography GitHub repo:

git clone https://github.com/lyft/cartography.git

Next, run the following command to install the necessary dependencies:

pip3 install -e .

Note: in the past, I’ve usually had to upgrade the Neo4j Bolt Driver for Python to 5.2.1 to get anything working, but as I try this again, it seems to work even with the default 4.4.x that Cartography uses. If you have problems, try changing setup.py to require neo4j>=5.2.1 and run the above command again.

Creating a Launch Configuration in Visual Studio Code

In order to run Cartography from its source code, you could run it directly from the terminal, for instance:

cd cartography/cartography
python3 __main__.py

However, as I’ve recently been using Visual Studio Code for all my polyglot software development needs, I find it much more convenient to set up a launch configuration that allows me to easily debug Cartography and pass whatever command-line arguments and environment variables I want.

The following launch.json is handy to run Cartography with an Okta configuration as described in “Getting Started with Cartography for Okta“:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Run Cartography",
            "type": "python",
            "request": "launch",
            "program": "cartography/__main__.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": [
                "--neo4j-user",
                "ignore",
                "--neo4j-password-env-var",
                "NEO4J_PASS",
                "--okta-org-id",
                "dev-xxxxxxxx",
                "--okta-api-key-env-var",
                "OKTA_API_TOKEN"
            ],
            "env": {
                "NEO4J_PASS": "ignore",
                "OKTA_API_TOKEN": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
            }
        }
    ]
}

You might notice we’re telling Cartography to connect to Neo4j (Memgraph actually) with username and password both set to “ignore”. The reason for this is that while the Community Edition of Memgraph does not require (or support) authentication, the Neo4j Bolt Driver for Python (i.e. Neo4j client) does require a username and password to be provided. So, as a second critical compatibility step, we pass any arbitrary value for the Neo4j username and password so long as they are not left empty.

As for the Okta configuration, remember to replace the Organisation ID and API Token with real ones.

Incompatible Index Creation Cypher

Pressing F5, we can now run Cartography from inside Visual Studio Code, and we immediately run into the first problem:

The error says: “no viable alternative at input ‘CREATEINDEXIF'”.

Memgraph is choking on the index creation step in indexes.cypher (in VS Code, use Ctrl+P / Command+P to quickly locate the file) because the index creation syntax is one aspect of Memgraph’s Cypher implementation that is not compatible with that of Neo4j. If we take the first line in the file, the Neo4j-compatible syntax is:

CREATE INDEX IF NOT EXISTS FOR (n:AWSConfigurationRecorder) ON (n.id);

…whereas the equivalent on Memgraph would be:

CREATE INDEX ON :AWSConfigurationRecorder(id);

Note: in Memgraph, the “IF NOT EXISTS” bit is implicit: an index is created if it doesn’t exist; if it does, the operation is a no-op that does not cause any error.

Fortunately, this syntactic difference is easily resolved by replacing (using VS Code search & replace syntax, with regex enabled) this:

CREATE INDEX IF NOT EXISTS FOR \(n:(.*?)\) ON \(n.(.*?)\);

…with this:

CREATE INDEX ON :$1($2);

Tip: although not in scope here, you’ll need to make a similar change also in querybuilder.py and tx.py if you also want to get other intel modules (e.g. AWS) working.

Neo4j Result Consumption

After fixing the index creation syntax and rerunning Cartography, we run into another problem:

The error says: “The result is out of scope. The associated transaction has been closed. Results can only be used while the transaction is open.”

I’m told that consume() is used to fix a problem in which Neo4j connections hang in situations where internal buffers fill up, although the Cartography team is re-evaluating whether this is necessary. In practice, I have seen that removing this doesn’t seem to cause problems with datasets I’ve tested with, although your mileage may vary. Let’s fix this problem by removing usage of consume() in statement.py.

First, we drop the .consume() at the end of line 76 inside the run() function:

    def run(self, session: neo4j.Session) -> None:
        """
        Run the statement. This will execute the query against the graph.
        """
        if self.iterative:
            self._run_iterative(session)
        else:
            session.write_transaction(self._run_noniterative)
        logger.info(f"Completed {self.parent_job_name} statement #{self.parent_job_sequence_num}")

Then, in the _run_iterative() function, we remove the entire while loop (lines 120-128) except for line 121, which we de-indent:

        # while True:
        result: neo4j.Result = session.write_transaction(self._run_noniterative)

            # Exit if we have finished processing all items
            # if not result.consume().counters.contains_updates:
            #     # Ensure network buffers are cleared
            #     result.consume()
            #     break
            # result.consume()

When we run it again, it should finish the run without problems and return control of the terminal with the prompt showing:

...
INFO:cartography.sync:Finishing sync stage 'duo'
INFO:cartography.sync:Starting sync stage 'analysis'
INFO:cartography.intel.analysis:Skipping analysis because no job path was provided.
INFO:cartography.sync:Finishing sync stage 'analysis'
INFO:cartography.sync:Finishing sync with update tag '1689401212'
daniel@andromeda:~/git/cartography$

Querying the Graph

The terminal we’re using to run Memgraph has the mgconsole client running (that’s the memgraph> prompt you see in the earlier screenshot), meaning we can try running queries directly there. For starters, we can try the ubiquitous “get everything” Cypher query:

memgraph> match (n) return n;

Note: if you get a “mg_raw_transport_send: Broken pipe”, just run the query again and it should reconnect.

This gives us some data back:

Querying Memgraph using mgconsole.

As you can see, this is not great to visualise results. Fortunately, Memgraph has its own web client (similar to Neo4j Browser) called Memgraph Lab, that you can access on http://localhost:3000/:

Memgraph Lab: Quick Connect page.

On the Quick Connect page, click the “Connect now” button. Then, switch to the “Query Execution” page using the left navigation sidebar, and you can run queries and view results more comfortably:

Seeing some nodes in Memgraph Lab.

Unlike Neo4j Browser, Memgraph Lab does not return relationships by default when you run this query. If you want to see them as well, you can run this instead:

match (a)-[r]->(b)
return a, r, b
Nodes and relationships in Memgraph Lab.

If the graph looks too cluttered, just drag the nodes around to rearrange them in a way that is more pleasant.

More Cartography with Memgraph

Cartography is a huge project that gathers data from a variety of data sources including AWS, Azure, GitHub, Okta, and others.

I’ve intentionally only covered the Okta intel module in this article because it’s small in scope and easy to digest. To use Cartography with other data sources, additional effort is required to address other problems with incompatible Cypher queries. For instance, at the time of writing this article, there are at least 9 outstanding issues that need to be fixed before Cartography can be used with Memgraph for AWS (that’s quite impressive considering that the AWS intel module is the biggest). Other intel modules may have other problems that need solving; nobody has explored them with Memgraph yet.

Summary

In this article, I’ve shown how one could go about taking an existing application that depends on Neo4j and migrating it to Memgraph. I’ve used Cartography with its Okta intel module to keep things relatively straightforward. The steps involved include:

  1. Running Memgraph with --bolt-server-name-for-init=Neo4j/
  2. Using the same Bolt-compatible Neo4j client, providing arbitrary Neo4j username and password values
  3. Fixing any incompatible Neo4j client code (in this case, consume()), if applicable
  4. Adjusting any incompatible Cypher queries

Getting Started with Cartography for Okta

Cartography is a great security tool that gathers infrastructure and security data from various sources for subsequent analysis. Last year, I wrote an article about Getting Started with Cartography for AWS. Although Cartography focuses mostly on AWS, it also gathers data from several other sources including major cloud and SaaS providers.

In this article, we’ll use Cartography to ingest Okta data. For the unfamiliar, Okta is an enterprise identity management tool that is great for its Single Sign On (SSO) capability. From a single dashboard, it provides seamless access to many different services (e.g. AWS, Gmail, and many others), without having to login every time. See also: What is Okta and What Does Okta Do?

It’s worth noting before we start this journey that Cartography’s support for Okta isn’t great. It only supports a handful of types, and it uses a retired version of the Okta SDK for Python. Nonetheless, it retrieves the most important types, and they enable analysis of some more interesting attack paths (e.g. an Okta user gaining unauthorised access to resources in AWS).

Creating an Okta Developer Account

We’ll first need an Okta account. There are a few different options including a trial, but for development, the best is to sign up for an Okta Developer account as follows.

Click on the Sign up button in the top-right.
In this confusing selection screen, go for the Developer Edition on the right.
Fill the sign-up form and proceed.

Once you get to the sign-up form, fill in the four required fields, and then either sign-up via email or use your GitHub or Google account. Note that Okta demands a “business email”, so you can’t use a Gmail account for this.

After signing up, you’ll get an email to activate your account. Follow its instructions to choose a password, and then you will be logged in and redirected to your Okta dashboard.

The Okta dashboard.

Creating an Okta API Token

Cartography’s Okta Configuration documentation says it’s necessary to set up an Okta API token, so let’s do that. From the Okta Dashboard:

  1. Go to Security -> API via the left navigation menu.
  2. Switch to the “Tokens” tab.
  3. Click the “Create token” button.
Security -> API, Tokens tab, Create token button.

You will then be prompted to enter a name for the API token, and subsequently given the token itself. Copy the token and keep it handy. Take note also of your organisation ID, which you can find either in the URL, or in the top-right under your name (but remove the “okta-” prefix). The organisation ID for a developer account looks like “dev-12345678”.

Running Neo4j

Before we run Cartography, we need a running instance of the Neo4j graph database, because that’s where the data gets stored after being retrieved from the configured data sources (in this case Okta). When I wrote “Getting Started with Cartography for AWS“, Cartography only supported up to Neo4j 3.5. Thankfully, that has changed. The Cartography Installation documentation specifically asks for Neo4j 4.x, further remarking that “Neo4j 5.x will probably work but Cartography does not explicitly support it yet.” The latest Neo4j Docker image at the time of writing this article seems to be 5.9, and I’m feeling adventurous, so let’s give it a try.

I did explain in “Getting Started with Cartography for AWS” how to run Neo4j under Docker, but we’ll do it a little better this time. Use the following command:

sudo docker run --rm -p 7474:7474 -p 7473:7473 -p 7687:7687 -e NEO4J_AUTH=neo4j/password neo4j:5.9

Here’s a brief explanation of what all this means:

  • sudo: I’m on Linux, so I need to run Docker with elevated privileges. If you’re on Windows or Mac, omit this.
  • docker run: runs a new Docker container with the image specified at the end.
  • --rm: destroys the container after you shut it down. This is because we’re just doing a quick test and don’t want to keep containers around. If you want to keep the container, remove this.
  • -p 7474:7474 -p 7473:7473 -p 7687:7687: maps ports 7473, 7474 and 7687 from the Docker container to the host, so that we can access Neo4j from the host machine. 7474 in particular lets us access the Neo4j Browser, which we’ll see in a moment.
  • -e NEO4J_AUTH=neo4j/password: sets up the initial username and password to “neo4j” and “password” respectively. This bypasses the need to reset the password from the Neo4j Browser as I did in the earlier article. Remember it’s just a quick test, so excuse the silly “password” and choose a better one in production.
  • neo4j:5.9: This is the image we’re going to run – neo4j with tag 5.9.
  • Note that any data will be lost when you stop the container, regardless of the --rm argument. You’ll need to use Docker volumes if you want to retain the data.

Once the container has started, you can access the Neo4j Browser at http://localhost:7474/, and login using the username “neo4j” and password “password”. We’ll use this later to run Cypher queries, but for now it is a sign that Neo4j is running properly.

The Neo4j Browser’s login screen.

Running Cartography

Following the Cartography Installation documentation, run the following to install Cartography:

pip3 install cartography

As per Cartography’s Okta Configuration documentation, assign the Okta API token you created earlier to an environment variable (the following will set it only for your current terminal session):

export OKTA_API_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Then, run Cartography with the following command:

cartography --neo4j-uri bolt://localhost:7687 --neo4j-password-prompt --neo4j-user neo4j --okta-org-id dev-xxxxxxxx --okta-api-key-env-var OKTA_API_TOKEN

Here’s a brief summary of the parameters:

  • --neo4j-uri bolt://localhost:7687: specifies the Neo4j URI to connect to
  • --neo4j-user neo4j: will login with the username “neo4j”
  • --neo4j-password-prompt: means that you will be prompted for the Neo4j password and will have to type it in
  • --okta-org-id dev-xxxxxxxx: will connect to Okta using the organisation ID “dev-xxxxxxxx” (replace this with yours)
  • --okta-api-key-env-var OKTA_API_TOKEN: will use the value of the OKTA_API_TOKEN environment variable as the API token when connecting to Okta

If you see “cartography: command not found” when you run this (especially on Linux), there’s a very good Stack Overflow answer that explains why this happens and offers a simple solution:

export PATH="$HOME/.local/bin:$PATH"

When you manage to run Cartography with the earlier command, enter the Neo4j password (it’s “password” in this example). It will take some time to collect the data from Okta and will write to the terminal periodically as it makes progress. You’ll know it’s done because you’ll see your terminal’s prompt again, and hopefully won’t see any errors.

Querying the Graph

You should now have data in Neo4j, so open your Neo4j Browser at http://localhost:7474/ and run some queries to look at the data. The easiest to start with is the typical “get everything” query:

match (n) return n

On a fresh new account, this gives you back a handful of nodes and the relationships between them:

Okta data in the Neo4j Browser.

Although this is not great for analysis, it’s all you need to get started using Cartography for Okta. You can get more data to play with by either building out your directory (users, groups, etc) via the Okta Dashboard, or else connecting to a real production account with real data.

If you want to analyse attack paths from Okta to AWS, then do the necessary AWS setup (see my earlier article, “Getting Started with Cartography for AWS“), and follow Cartography’s Okta Configuration documentation to set up the bridge between Okta and AWS.

Summary

To get Cartography to collect your Okta data:

  1. Sign up for an Okta account if you don’t have one already.
  2. Create an Okta API Token, and take note of your Okta Organisation ID
  3. Run Neo4j
  4. Run Cartography, providing settings to access Neo4j and Okta

Once the data is in Neo4j, you can analyse it and visualise how the nodes are connected. This can help you understand the paths that an attacker could take to breach the critical parts of your infrastructure. In the case of Okta, this is particularly useful when considering how an attacker could exploit the privileges of an Okta user to access resources in other cloud or SaaS providers.

Running Puppeteer under Docker

Puppeteer is an API enabling browser automation via Chrome/Chromium. Quoting its homepage:

“Puppeteer is a Node.js library which provides a high-level API to control Chrome/Chromium over the DevTools Protocol. Puppeteer runs in headless mode by default, but can be configured to run in full (non-headless) Chrome/Chromium.”

I’ve already shown it in action a couple of years ago in my article, “Gathering Net Salary Data with Puppeteer“, where I used it for web scraping. Browser automation is also very common for automated testing of web applications, and may also be used for a lot of other things.

As with any other piece of software, it is sometimes convenient to package a Puppeteer script in a Docker container. However, since deploying a browser is fundamentally more complicated than your average API, Puppeteer and Docker are a little tricky to get working together. In this article, we’ll see why this combination is problematic and how to solve it.

A Minimal Puppeteer Example

Before we embark upon our Docker journey, we need a simple Puppeteer program we can test with. The easiest thing we can do is use Puppeteer to open a webpage and take a screenshot of it. We can do this quite easily as follows.

First, we need to create a folder, install prerequisites, and create a file in which to put our JavaScript code. The following bash script takes care of all this, assuming you already have Node.js installed. Please note that I am working on Linux Kubuntu 22.04, so if you’re using a radically different operating system, the steps may vary a little.

mkdir pupdock
cd pupdock
npm install puppeteer
touch main.js

Next, open main.js with your favourite text editor or IDE and use the following code (adapted from “Gathering Net Salary Data with Puppeteer“):

const puppeteer = require('puppeteer');
 
(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://www.programmersranch.com');
  await page.screenshot({ path: 'screenshot.png' });
  await browser.close();
})();

To see that it works, run this script using the following command:

node main.js

A file called screenshot.png should be saved in the same folder:

A screenshot of my older tech blog, Programmer’s Ranch, complete with Blogger’s stupid cookie banner.

An Initial Attempt with Docker

Now that we know that the script works, let’s try and make a Docker image out of it. Add a Dockerfile with the following contents:

FROM node:19-alpine3.16
WORKDIR /puppeteer
COPY main.js package.json package-lock.json ./
RUN npm install
CMD ["node", "main.js"]

What we’re doing here is starting from a recent (at the time of writing this article) Node Docker image. Then we set the current working directory to a folder called /puppeteer. We copy the script along with the list of package dependencies (Puppeteer, basically) into the image, install those dependencies, and set up the image to execute node main.js when it is run.

We can build the Docker image using the following command, giving it the tag of “pupdock” so that we can easily find it later. The sudo command is only necessary if you’re running on Linux.

sudo docker build -t pupdock .

Once the image is built, we can run a container based on this image using the following command:

sudo docker run pupdock

Unfortunately, we hit a brick wall right away:

/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/node/BrowserRunner.js:300
            reject(new Error([
                   ^

Error: Failed to launch the browser process! spawn /root/.cache/puppeteer/chrome/linux-1108766/chrome-linux/chrome ENOENT


TROUBLESHOOTING: https://pptr.dev/troubleshooting

    at onClose (/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/node/BrowserRunner.js:300:20)
    at ChildProcess.<anonymous> (/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/node/BrowserRunner.js:294:24)
    at ChildProcess.emit (node:events:512:28)
    at ChildProcess._handle.onexit (node:internal/child_process:291:12)
    at onErrorNT (node:internal/child_process:483:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

Node.js v19.8.1

The error and stack trace are both pretty cryptic, and it’s not clear why this failed. There isn’t much you can do other than beg Stack Overflow for help.

Fortunately, I’ve done this already myself, and I can tell you that it fails because Chrome (which Puppeteer is trying to launch) needs more security permissions than Docker provides by default. We’ll need to learn a bit more about Docker security to make this work.

Dealing with Security Restrictions

So how do we give Chrome the permissions it needs, without compromising the security of the system it is running on? We have a few options we could try.

  • Disable the sandbox. Chrome uses a sandbox to isolate potentially harmful web content and prevent it from gaining access to the underlying operating system (see Sandbox and Linux Sandboxing docs to learn more). Many Stack Overflow answers suggest getting around errors by disabling this entirely. Unless you know what you’re doing, this is probably a terrible idea. It’s far better to relax security a little to allow exactly the permissions you need than to disable it entirely.
  • Use the Puppeteer Docker image. Puppeteer’s documentation on Docker explains how to use a Puppeteer’s own Docker images (available on the GitHub Container Registry) to run arbitrary Puppeteer scripts. While there’s not much info on how to work with these (e.g. which folder to mount as a volume in order to grab the generated screenshot), what stands out is that this approach requires the SYS_ADMIN capability, which exposes more permissions than we need.
  • Build your own Docker image. Puppeteer’s Troubleshooting documentation (also available under Chrome Developers docs and Puppeteer docs) has a section on running Puppeteer in Docker. We’ll follow this method, as it is the one that worked best for me.

A Second Attempt Based on the Docs

Our initial attempt with a simple Dockerfile didn’t go very well, but now we have a number of other Dockerfiles we could start with, including the Puppeteer Docker image’s Dockerfile, a couple in the aforementioned doc section on running Puppeteer in Docker (one built on a Debian-based Node image, and another based on an Alpine image), and several other random ones scattered across Stack Overflow answers.

My preference is the Alpine one, not only because Alpine images tend to be smaller than their Debian counterparts, but also because I had more luck getting it to work across Linux, Windows Subsystem for Linux and an M1 Mac than I did with the Debian one. So let’s replace our Dockerfile with the one in the Running on Alpine section, with a few additions at the end:

FROM alpine

# Installs latest Chromium (100) package.
RUN apk add --no-cache \
      chromium \
      nss \
      freetype \
      harfbuzz \
      ca-certificates \
      ttf-freefont \
      nodejs \
      yarn

# Tell Puppeteer to skip installing Chrome. We'll be using the installed package.
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser

# Puppeteer v13.5.0 works with Chromium 100.
RUN yarn add puppeteer@13.5.0

# Add user so we don't need --no-sandbox.
RUN addgroup -S pptruser && adduser -S -G pptruser pptruser \
    && mkdir -p /home/pptruser/Downloads /app \
    && chown -R pptruser:pptruser /home/pptruser \
    && chown -R pptruser:pptruser /app

# Run everything after as non-privileged user.
USER pptruser

WORKDIR /puppeteer
COPY main.js ./
CMD ["node", "main.js"]

Because the given Dockerfile already installs the puppeteer dependency and that’s all we need here, I didn’t even bother to do the usual npm install here, although a more complex script might possibly have additional dependencies to install.

At this point, we can build the Dockerfile and run the resulting image as before:

sudo docker build -t pupdock .
sudo docker run pupdock

The result is that it still doesn’t work, but the error is something that is more easily Googled than the one we have before:

/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:237
            reject(new Error([
                   ^

Error: Failed to launch the browser process!
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted


TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md

    at onClose (/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:237:20)
    at ChildProcess.<anonymous> (/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:228:79)
    at ChildProcess.emit (node:events:525:35)
    at ChildProcess._handle.onexit (node:internal/child_process:291:12)

Node.js v18.14.2

A Little Security Lesson

Googling that error about PID namespaces is what led me to the solution, but it still took a while, because I had to piece together many clues scattered in several places, including:

  • Answer by usethe4ce suggests downloading some chrome.json file and passing it to Docker, but it’s not immediately clear what this is/does.
  • Answer by Riccardo Manzan, based on the one by usethe4ce, provides an example Dockerfile based on Node (not Alpine) and also shows how to pass the chrome.json file both in docker run and docker-compose.
  • GitHub Issue by WhisperingChaos explains what that chrome.json is about.
  • Answer by hidev lists the five system calls that Chrome needs.

To understand all this confusion, we first need to take a step back and understand something about Docker security. Like the Chrome sandbox, Docker has its ways of restricting the extent to which a running Docker container can interact with the host.

Like any Linux process, a container requests whatever it needs from the operating system kernel using system calls. However, a container could be used to wreak havoc on the host if it is allowed to run whatever system calls it wants and then is successfully breached by an attacker. Fortunately, Linux provides a feature called seccomp that can restrict system calls to only the ones that are required, minimising the attack surface.

In Docker, this restriction is applied by means of a seccomp profile, basically a JSON file whitelisting the system calls to be allowed. Docker’s default seccomp profile restricts access to system calls enough that it prevents many known exploits, but this also prevents more complex applications that need additional system calls – such as Chrome – from working under Docker.

That chrome.json I mentioned earlier is a custom seccomp profile painstakingly created by one Jess Frazelle, intended to allow the system calls that Chrome needs but no more than necessary. This should be more secure than disabling the sandbox or running Chrome with the SYS_ADMIN capability.

A Third Attempt with chrome.json

Let’s give it a try. Download chrome.json and place it in the same folder as main.js, the Dockerfile and everything else. Then, run the container as follows:

sudo docker run --security-opt seccomp=path/to/chrome.json pupdock

This time, there’s no output at all – which is good, because it means the errors are gone. To ensure that it really worked, we’ll grab the screenshot from inside the stopped container. First, get the container ID by running:

sudo docker container ls -a

Then, copy the screenshot from the container to the current working directory as follows, taking care to replace 7af0d705a751 with the actual ID of the container:

sudo docker cp 7af0d705a751:/puppeteer/screenshot.png ./screenshot.png
Puppeteer worked: the screenshot contains the full length of the page.

Additional Notes

I omitted a few things to avoid breaking the flow of this article, so I’ll mention them here briefly:

  • That SYS_ADMIN capability we saw mentioned earlier belongs to another Linux security feature: capabilities, which are also related to seccomp profiles.
  • If you need to pass a custom seccomp profile in a docker-compose.yaml file, see Riccardo Manzan’s Stack Overflow answer for an example.
  • If you run into other issues, use the dumpio setting to get more verbose output. It’s a little hard to separate the real errors from the noise, but it does help. Be sure to run in headless mode (it doesn’t make sense to run the GUI browser under Docker), and disable the GPU (--disable-gpu) if you see related errors.

Conclusion

Running Puppeteer under Docker might sound like an unusual requirement, perhaps overkill, but it does open up a window onto the interesting world of Docker security. Chrome’s complexity requires that it be granted more permissions than the typical Docker container.

It is unfortunate that the intricacies around getting Puppeteer to work under Docker are so poorly documented. However, once we learn a little about Docker security – and the Linux security features that it builds on – the solution of using a custom seccomp profile begins to make sense.

Information Disclosure Vulnerability in WordPress REST API

WordPress is insanely popular. Around 43% of websites on the internet use WordPress. I don’t know how it’s come to this, as WordPress is not that great. But, aside from being a relatively easy writing platform for hobbyist bloggers like me, it’s also very prevalent among digital marketing companies, who build their own websites and those of their customers on WordPress.

As a result, bugs and security flaws in WordPress are not to be taken lightly, no matter how small.

Listing Users

I came across the following in the book Hacking APIs: Breaking Web Application Programming Interfaces by Corey J. Ball:

“Sensitive data can include any information that attackers can leverage to their advantage. For example, a site that is using the WordPress API may unknowingly be sharing user information with anyone who navigates to the API path /wp-json/wp/v2/users, which returns all the WordPress usernames, or “slugs”.”

— Hacking APIs: Breaking Web Application Programming Interfaces by Corey J. Ball (2022), page 54

Indeed, I was able to simply append /wp-json/wp/v2/users to the URL of several WordPress sites and see a list of users. For instance, this is from a fresh WordPress install:

A list of users shown in a fresh install of WordPress.

The following, on the other hand, is a list I got from a company website:

A list of users from a company website, obfuscated to protect their identity. You can also see they use the Yoast SEO plugin.

The following is from another company website, which has been protected with a plugin, but reveals the name of the plugin used:

The list of users from this other website is restricted by the iThemes Security plugin.

We’ll discuss in the next section why revealing the names of plugins you use, like Yoast SEO or iThemes Security, is probably not a good idea.

Meanwhile, as we have seen, it is very easy to get a list of users, including their full name and username, for a WordPress website. Unfortunately, the WordPress REST API, of which wp-json is the top-level endpoint, is enabled by default.

To be fair, it’s also possible to find out the users of a WordPress site by just checking its blog posts and taking note of the author. The link to an author exposes the username, instead of using something more generic like an id.

Either way, once an attacker knows the usernames pertaining to a website, all that’s left is to figure out a password. They could brute force a password for one of the users or else attempt a set of weaker passwords for several users and see if any fit (a technique known as password spraying). It also doesn’t help that it’s universally known that WordPress sites have their login page at /wp-admin. Having strong passwords is more important than ever.

Listing Plugins

There is also a wp-json/wp/v2/plugins endpoint, which presumably could give us a list of plugins, but it does seem protected by default:

The plugins endpoint doesn’t tell us anything.

However, we can still get a lot of information about plugins by just expanding routes at the top-level wp-json endpoint. For instance, the screenshot below shows that this website uses Elementor, Gravity Forms, and Google Site Kit among other things. It is also possible to find out plugin names from other places, as shown in the previous section.

Expanding routes exposes the names of many plugins used by a WordPress website.

Knowing the plugins used in a website, an attacker could look up known vulnerabilities, e.g. on WPScan, and attempt to exploit them.

Disabling the REST API

Just because an attacker knows your users’ usernames or plugins, it doesn’t mean they will manage to compromise your website, or that they couldn’t do it without that knowledge anyway. But it’s conventional wisdom in IT security that we can sleep more comfortably at night if we don’t make it easy for bad actors to destroy what we worked so hard to build.

An easy way to stop sharing all this information is to disable the REST API altogether. As with everything else in WordPress, this can be done with a plugin.

The Disable REST API plugin.

Simply find, install and activate the Disable REST API plugin. This will protect the REST API:

With the REST API disabled, we can’t see the list of users any more.

Some notes:

  • This protects the entire API, not just the users endpoint shown above.
  • In the plugin’s settings, you can configure any endpoints you want to remain accessible.
  • “DRA” does expose the name of the plugin. It’s not perfect but reduces risk considerably.
  • You should test this from an incognito browser window. If you access the API from a browser where you’re logged into WordPress, you’ll likely see the regular API response simply because you’re authorised.
  • There’s another way to disable the REST API using the WPCode plugin instead.

Conclusion

Given how much information the REST API provides to basically everyone, it’s a little shocking that it’s enabled by default. By disabling it, we can make attackers’ lives a little harder and reduce the security risk of our WordPress websites.