Run GitHub Actions locally with DinD and nektos/act

2021-01-01 1178 words 6 minutes

Contents

A few days ago I came across a situation where we wanted to execute our CI pipeline locally. This article details the steps to run GitHub actions locally using nektos/act and Docker-in-Docker(dind).

Why even bother?

While running your CI pipeline locally may not seem so essential, it is definitely helpful for various reasons. The two very valid reasons listed by nektos/act are:

Fast Feedback: Rather than having to commit/push every time you want to test out the changes you are making to your .github/workflows/ files (or for any changes to embedded GitHub actions), you can use act to run the actions locally. The environment variables and filesystem are all configured to match what GitHub provides.
Local Task Runner: I love make. However, I also hate repeating myself. With act, you can use the GitHub Actions defined in your .github/workflows/ to replace your Makefile!

For me, personally, it was helpful because now I could avoid having to force push my branch in case I mess something up and CI fails. Simple misses like spelling mistakes, unused variables, ineffectual assignments, and others can force you to fix, rebase, and push the branch over and over again.

The Setup

In order to set up and run GitHub actions locally, all you need is Docker. act accepts a -P flag using which we can define a custom image to use per platform.

In order to make things simple, act also works with a ~/.actrc or ./.actrc file. Depending upon the runner to be used per platform, we can define the .actrc file.

The repository that we will be using to test our pipeline runs-on ubuntu-latest runner. So, we will create the file as:

1
2
3


cat << EOF > .actrc
-P ubuntu-latest=node:12.20.1-buster-slim
EOF

Note

Note: Please refer to [Configuration][5] for more information about .actrc and to [Runners][6] for information about used/available Docker images.

The .actrc file contains the default configuration flags to act. Any flags in the file will be applied before any flags provided directly on the command line.

Now, let’s write a Dockerfile that gives us the CI pipeline.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


FROM docker:dind

RUN apk add curl
RUN curl https://raw.githubusercontent.com/nektos/act/master/install.sh | sh

COPY .actrc /
RUN mv /.actrc ~/.actrc

WORKDIR /project

CMD /bin/sh -c "act -n > /logs/dry-run.log; act > /logs/run.log"

Here is what we are doing in the Dockerfile:

we start with docker:dind as the base image
then we install curl to install act
next, we create a directory /project and set it as the working directory
and finally, we set the command to be executed as the github-actions-pipeline container starts
the command runs act and writes the logs to logs directory

Let’s create a Docker image for our pipeline with:

1

docker build -t github-actions-pipeline .

GitHub Actions in action

It’s time to put our setup to test. For this purpose, let’s clone the cplee/github-actions-demo repository first:

1

git clone https://github.com/cplee/github-actions-demo.git

Now, we can run our GitHub Actions pipeline to test this repository:

1
2
3
4
5
6


docker run \
    -d --rm \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v $(pwd)/github-actions-demo:/project \
    -v $(pwd)/ci-logs:/logs \
    github-actions-pipeline

Note that we have mounted the cloned directory to /project inside the container. act reads the /project/.github/workflows and executes the actions.

The pipeline writes the logs to dry-run.log and run.log which can be found in the $(pwd)/ci-logs directory. You can also watch the logs as the pipeline progresses using the tail command:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61


  $ tail -f ci-logs/run.log
  [CI/test] 🚀  Start image=node:12.6-buster-slim
  [CI/test]   🐳  docker run image=node:12.6-buster-slim entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
  [CI/test]   🐳  docker cp src=/project/. dst=/github/workspace
  [CI/test] ⭐  Run actions/checkout@v2
  [CI/test]   ✅  Success - actions/checkout@v2
  [CI/test] ⭐  Run actions/setup-node@v1
  [CI/test]   ☁  git clone 'https://github.com/actions/setup-node' # ref=v1
  [CI/test]   🐳  docker cp src=/root/.cache/act/actions-setup-node@v1 dst=/actions/
  [CI/test]   💬  ::debug::isExplicit: 
  [CI/test]   💬  ::debug::explicit? false
  [CI/test]   💬  ::debug::evaluating 0 versions
  [CI/test]   💬  ::debug::match not found
  [CI/test]   💬  ::debug::evaluating 430 versions
  [CI/test]   💬  ::debug::matched: v10.23.0
  [CI/test]   💬  ::debug::isExplicit: 10.23.0
  [CI/test]   💬  ::debug::explicit? true
  [CI/test]   💬  ::debug::checking cache: /opt/hostedtoolcache/node/10.23.0/x64
  [CI/test]   💬  ::debug::not found
  [CI/test]   💬  ::debug::Downloading https://nodejs.org/dist/v10.23.0/node-v10.23.0-linux-x64.tar.gz
  [CI/test]   💬  ::debug::Destination /tmp/14890b9f-821d-425f-ae5d-215aabfd3919
  [CI/test]   💬  ::debug::download complete
  [CI/test]   💬  ::debug::Checking tar --version
  [CI/test]   💬  ::debug::tar (GNU tar) 1.30%0ACopyright (C) 2017 Free Software Foundation, Inc.%0ALicense GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.%0AThis is free software: you are free to change and redistribute it.%0AThere is NO WARRANTY, to the extent permitted by law.%0A%0AWritten by John Gilmore and Jay Fenlason.
  [CI/test]   | [command]/bin/tar xz --warning=no-unknown-keyword -C /tmp/d706df03-0627-4b41-9a9e-9fd2a1de7e2e -f /tmp/14890b9f-821d-425f-ae5d-215aabfd3919
  [CI/test]   💬  ::debug::Caching tool node 10.23.0 x64
  [CI/test]   💬  ::debug::source dir: /tmp/d706df03-0627-4b41-9a9e-9fd2a1de7e2e/node-v10.23.0-linux-x64
  [CI/test]   💬  ::debug::destination /opt/hostedtoolcache/node/10.23.0/x64
  [CI/test]   💬  ::debug::finished caching tool
  [CI/test]   ⚙  ::add-path:: /opt/hostedtoolcache/node/10.23.0/x64/bin
  [CI/test]   | [command]/opt/hostedtoolcache/node/10.23.0/x64/bin/node --version
  [CI/test]   | v10.23.0
  [CI/test]   | [command]/opt/hostedtoolcache/node/10.23.0/x64/bin/npm --version
  [CI/test]   | 6.14.8
  [CI/test]   ❓  ##[add-matcher]/actions/actions-setup-node@v1/.github/tsc.json
  [CI/test]   ❓  ##[add-matcher]/actions/actions-setup-node@v1/.github/eslint-stylish.json
  [CI/test]   ❓  ##[add-matcher]/actions/actions-setup-node@v1/.github/eslint-compact.json
  [CI/test]   ✅  Success - actions/setup-node@v1
  [CI/test] ⭐  Run npm install
  [CI/test]   | added 280 packages from 643 contributors and audited 280 packages in 2.35s
  [CI/test]   | 
  [CI/test]   | 24 packages are looking for funding
  [CI/test]   |   run `npm fund` for details
  [CI/test]   | 
  [CI/test]   | found 1 low severity vulnerability
  [CI/test]   |   run `npm audit fix` to fix them, or `npm audit` for details
  [CI/test]   ✅  Success - npm install
  [CI/test] ⭐  Run npm test
  [CI/test]   | 
  [CI/test]   | > [email protected] test /github/workspace
  [CI/test]   | > mocha ./tests --recursive
  [CI/test]   | 
  [CI/test]   | 
  [CI/test]   | 
  [CI/test]   |   GET /
  [CI/test]   |     ✓ should respond with hello world
  [CI/test]   | 
  [CI/test]   | 
  [CI/test]   |   1 passing (14ms)
  [CI/test]   | 
  [CI/test]   ✅  Success - npm test

Troubleshooting

If the logs in dry-run.log or run.log looks like below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


➜ cat run.log    
? Please choose the default image you want to use with act:

  - Large size image: +20GB Docker image, includes almost all tools used on GitHub Actions (only ubuntu-latest/ubuntu-18.04 platform is available)
  - Medium size image: ~500MB, includes only necessary tools to bootstrap actions and aims to be compatible with all actions
  - Micro size image: <200MB, contains only NodeJS required to bootstrap actions, doesn't work with all actions

Default image and other options can be changed manually in ~/.actrc (please refer to https://github.com/nektos/act#configuration for additional information about file structure)  [Use arrows to move, type to filter, ? for more help]
  Large
> Medium
  Micro

Then this is because either:

the ~/.actrc or ./.actrc file is not present
you have not defined the runner for your platform in the .actrc file (check first act run)

Conclusion

We have successfully set up a pipeline to run GitHub Actions locally. However, there are a few actions, like cachix/install-nix-action, that you may not be able to run locally. I believe there would be a workaround or an alternative to such actions. All you may need is to do some research.