New job; new tech stack. One of the new technologies is GitLab and in this post I am scratching the surface of GitLab’s continuous integration facilities.

My use case is that I want to run tests of R packages as part of merge requests. Unlike my previous posts about testing R I now have a virtual machine with Windows to execute the tests.

Preparing a runner

GitLab’s CI tools interact with the computer executing the tests through its runners.

Setting up a runner is pretty straightforward following the instructions. The setup saves a file config.toml with the configuration.

The only thing I needed to tweak after the setup was to specify the shell in config.toml to use PowerShell:

[[runners]]
  name = "shell executor runner"
  executor = "shell"
  shell = "powershell"

CI configuration

The configuration of the tests in a repository happens through the file .gitlab-ci.yml. Because my primary usecase is to run tests as part of merge requests, I want to merge the two branches on the VM before running the tests.

This feature requires a GitLab plan that I do not have access to – so I have to perform the merge manually.

For an R package I use the following .gitlab-ci.yml:

Test:
  stage: test
  tags:
    - rlang
  only:
    - merge_requests
  before_script:
    - Write-Output "Target branch $env:CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
    - Write-Output "Source branch $env:CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"
    - git remote set-branches origin '*'
    - git fetch -v
    - git checkout "origin/$env:CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
    - git merge "origin/$env:CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"
  script:
    - '& "path\to\Rscript.exe" path\to\run_test.R'
  artifacts:
    paths:
      - .\test-results.xml
      - .\coverage.xml
    reports:
      junit: .\test-results.xml

Before executing the actual script running tests, I merge the source branch into the target branch. These are available as predefined environment variables through the GitLab runner. The PowerShell commands with Write-Output is just for logging purposes.

One thing to be mindful of that caused me quite a lot of investigation is how the repository is cloned in the test. On the repository’s web page, go to “Settings” > “CI / CD”. Expanding “General pipelines” we see a “Git strategy for pipelines”:

If we use “git fetch”, Git performs a so-called shallow clone that does not contain any branch names. In this case it is not possible to checkout the target branch nor the source branch. I circumvent this by explicitly fetching the branches with the two remote/fetch lines.

If we perform a full Git clone, we just leave out these lines. In that case a vanillabefore_script would be:

before_script:
  - git checkout "origin/$env:CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
  - git merge "origin/$env:CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"

Tests

This is the only R specific part. To execute the tests I use the run_test.R script that I have introduced before. Here it is again – with one more line added in the end:

devtools::install()

# Run tests
options("testthat.output_file" = "test-results.xml")
devtools::test(reporter = testthat::JunitReporter$new())

# Compute code coverage
cov <- covr::package_coverage()
covr::to_cobertura(cov, "coverage.xml")
print(cov)

A summary of the code coverage is printed in the log as this is the way GitLab is able to include the code coverage in the test summary.

Parsing results

In .gitlab-ci.yml the two xml files with test results and detailed code coverage are uploaded from the VM to GitLab in the artifacts section. They are not of much interest on their own in this format, but they can be downloaded from GitLab for further inspection.

The test results are also uploaded as a report that GitLab parses as part of the test results in the merge request.

The code coverage report is not parsed by GitLab, but we can capture the overall code coverage from the log using a regular expression in the the “Settings” > “CI / CD” (same tab as the “Git strategy for pipelines”). The command print(cov) in run_test.R prints the code coverage for the package in the format

<package name> Coverage: 12.34%

This number can be captured with the regular expression <package name> Coverage: (\d+\.\d+\%)

With a setup like this the test results appear on a merge request: