Robert's Data Science Blog

Test R in VSTS

I recently wrote about how to test Python on VSTS. Now the time has come to R. The fundamental difference between Python and R on VSTS is that Python is available on VSTS’s “hosted” computers whereas we need to bring our own environment for testing R.

The solution I present here use a Docker image to provide such an environment. I build on the Docker images introduced in an earlier post.

The big lines are as follows:

Making an R package

Just as in my post about testing Python I make a small package called VSTS with two simple function add and subtract, that fully live up to their names. My knowledge about R packages stem from Hadley Wickham’s book “R Packages”. In particular, I use the suggested devtools, roxygen2 and testthat packages. Furthermore, I use the covr package for computing code coverage.

Create the new package with e.g. RStudio through File > New Project... or with devtools::create. My package has the following files: 

VSTS
├── DESCRIPTION
├── man
│   ├── add.Rd
│   └── subtract.Rd
├── NAMESPACE
├── R
│   └── arithmetic.R
├── tests
│   ├── testthat
│   │   └── test_arithmetic.R
│   └── testthat.R
└── VSTS.Rproj

The two announced functions are in arithmetic.R: 

#' Add two numbers
#'
#' @param x A number
#' @param y A number
#'
#' @return The sum x + y.
#'
#' @export
add <- function(x, y) {
  x + y
}

#' Subtract two numbers
#'
#' @inheritParams add
#'
#' @return The difference x - y.
#'
#' @export
subtract <- function(x, y) {
  x - y
}

The functions are tested in test_arithmetic.R: 

context("Arithmetic")

test_that("Add", {
    expect_equal(add(1,1), 2)
})

test_that("Subtract", {
    expect_equal(subtract(1,1), 0)
})

The generated NAMESPACE and Rd files are generated by devtools::document. In RStudio I usually go to Tools > Project Options... > Build Tools and configure Generate documentation using Roxygen to roxygenize when running Build & Reload.

The latter will install the package and generate documentation when running Install and Restart in the Build tab.

Dockerfile for testing

Building on the r-devtools Docker image from my post about Dockerfiles I create a Docker image called r-test that contains the R packages needed for testing:

ARG R_VERSION
FROM r-devtools:${R_VERSION}

USER root

RUN apt-get update \
	&& apt-get -y install \
		libxml2-dev \
	&& rm -rf /var/lib/apt/lists/*

USER shiny

RUN Rscript -e 'install.packages(c("covr", "roxygen2", "testthat"))'

COPY --chown=shiny:shiny run_tests.R /home/shiny/package/

WORKDIR /home/shiny/package

CMD ["Rscript", "run_tests.R"]

The directory /home/shiny/package is going to have the source of the package to be tested. The run_tests.R script that executes the tests is introduced in the next section. Note that the CMD in r-test will not execute successfully – it needs the source code for a package in /home/shiny/package.

In the directory of the package to be tested I include the following Dockerfile: 

FROM r-test

COPY --chown=shiny:shiny . /home/shiny/package

Since no CMD is provided it is inherited from r-test. If the package has system requirements these are installed just like libxml2-dev is in r-test.

I have updated my repository on GitHub with the r-test Dockerfile.

Testing a package

In the usual “interactive” use of the testthat and covr packages the test results are written directly in the REPL. As mentioned in my post about testing Python on VSTS we must export the test results in the JUnit format and the code coverage in the Cobertura format for VSTS to utilize them.

The run_tests.R script I use look like this: 

devtools::install()

# Run tests
options("testthat.output_file" = "test-results.xml")
devtools::test(reporter = testthat::JunitReporter$new())

# Compute code coverage
cov <- covr::package_coverage()
covr::to_cobertura(cov, "coverage.xml")

Here we assume that run_tests.R is located in the package source directory and we carry out three tasks: Install the package; run the tests and export the results to test-results.xml; compute the code coverage and export the results to coverage.xml.

The benefit of including run_tests.R in the r-test image is that I only have to update this script once to use it in all R packages.

Build in VSTS

My build definition in VSTS is as follows:

As an agent queue (not seen in the picture) I use a “Hosted Linux Preview”. I use Azure Container Registry to host the images I use. There is a nice tutorial on Microsoft’s docs on how to to this using Azure CLI.

The first two are built-in tasks: Pull an image from a container registry and build a Docker image based on the Dockerfile included in the repository.

The “Bash Script” executes the image just built and copies the resulting xml files from the container to the host. 

docker run test:latest

CONTAINERID=`docker ps -alq`

docker container cp $CONTAINERID:/tmp/VSTS/test-results.xml .
docker container cp $CONTAINERID:/tmp/VSTS/coverage.xml .

In “Publish Test Results”:

In “Publish Code Coverage Results”:

Alternatives

As mentioned in a previous post there is an official Microsoft guide to run tests in containers using Kubenetes.

I have also found an interesting alternative that use Azure Container Instances to provide the test environment. Just as the author of that article I see the following pros and cons: