Testing Strategies
Testing code can be performed in two different ways: manual and automated. The first approach means that every time we write a new function, or refactor code, or add some new arguments to a function call, we would need to figure out what behaviour we expect. Then we would need to write some ephemeral code and check the outputs. This way it is not only time consuming but also prone to bias (it is easy to omit some edge cases) and not reproducible.
Automated testing on the other hand allows us to formalize any number of checks, and lets the computer do the hard work. While it is also possible to omit some checks, this can be quickly caught by other developers during code review. The downside of this approach is that the development process takes more time, however, that time is later saved during code maintenance.
There are two main code testing strategies – Test Driven Development and Test After Development. The first one is the golden standard for many developers. However, since our core activities are related to data science, and also in many cases is hard to predict the output calculations, we lean more towards the latter strategy. This means, that we first write code having in mind the big picture, and after we are happy with the function performance we write tests.
Our testing framework is based on the
testhat package. To make things easy, after developing a function you can take advantage of the
usethis package, which creates test files with a simple function call usethis::use_test("my_function"). Now you can see a new file test-my_function.R in tests/testhat folder. In that file you can specify your assumption and expectations.
Lets assume we have a simple function that helps us with mapping a continuous variable to a new range.
#` @param x A numeric vector.
#` @param new_range A numeric vector length (2). First element is new minimum,
#` second element is new maximum.
#` @return Numeric vector mapped to new range.
scale_range <- function(x, new_range) {
old_min <- min(x)
old_max <- max(x)
new_x <- new_range[1] + ((new_range[2] - new_range[1]) * (x - old_min)) / (old_max - old_min)
new_x
}
The body of the test file might look like this:
test_that("scale_range returns correct vector", {
#example vectors
test_vector_1 <- c(1, 2, 3, 4, 5)
test_vector_2 <- c(1.5, 3, 5.5, 11, 20)
#check assumptions
expect_equal(
scale_range(test_vector_1, c(-2, 2)),
c(-2, -1, 0, 1, 2) #expected output
)
expect_equal(
# matching numerics can be tricky, so lets round the values to the 3 decimals
round(scale_range(test_vector_2, c(-30, 30)), 3),
c(-30, -25.135, -17.027, 0.811, 30) #expected output
)
})
This way we tested our function with 2 different vectors, and two different ranges. So we can be pretty sure that the calculations are working as expected. However this is just the beginning. There is much more to test.
To write tests that are indeed helpful you need to think about as many possible ways to break the function as possible. What would happen if we pass other than numeric vector to argument x? What will happen if there is NA in the vector. What if new_range vector length is not equal to two? How to proceed when the first element of new_range is bigger than the second element?
The function is quite simple and we would probably be O.K. just using built in error messages. With more complicated functions this might not be the case and we could get different behaviour with different input classes. Also, some built in errors or warnings are not very informative and user’s life would be less miserable if we use custom messages.
We can now add some code to the function: This example needs adjustments after we come with full specification of internal function tests
scale_range <- function(x, new_range) {
if (!inherits(x, "numeric")) {
stop("x needs to be numeric vector\n", call. = FALSE)
}
if (!inherits(new_range, "numeric") | length(numeric) != 2) {
stop("new_range must be numeric vector of length exactly 2\n", call. = FALSE)
}
if (new_range[1] > new_range[2]) {
message(
paste0(
"The first element of new_range is bigger then the second element.\n",
"The output vector will be reversed. Please assure that this is correct."
)
}
old_min <- min(x)
old_max <- max(x)
new_x <- new_range[1] + ((new_range[2] - new_range[1]) * (x - old_min)) / (old_max - old_min)
new_x
}
And now we can add some test to check if our sanity checks in the function are working as expected.
test_that("scale_range returns correct vector", {
#example vectors
#...
})
test_that("scale_range throws errors and messages", {
wrong_vector <- c("a", "b")
short_range <- c(1)
long_range <- c(-2, 3, -4, 5)
wrong_range <- c("a", "B")
reverse_range <- c(-30, 30)
expect_error(scale_range(wrong_vector, c(-2, 2)))
expect_error(scale_range(c(1, 2, 3, 4, 5), short_range))
expect_error(scale_range(c(1, 2, 3, 4, 5), long_range))
expect_error(scale_range(c(1, 2, 3, 4, 5), wrong_range))
expect_message(scale_range(c(1, 2, 3, 4, 5), reverse_range))
expect_equal(
round(scale_range(test_vector_2, c(30, -30)), 3),
c(30, 25.135, 17.027, -0.811, -30)
)
})
Of course the above test does not exhaust the possibilities of all testing errors and messages. This is just a simple example what you should think about when writing tests. It also shows how the process of writing automated tests might help you figure out how to make your code more robust.
Our test case is simple. We pass two vectors and expect a vector as an output. So the test would look like:
test_that("output is a vector", {
expect_type(scale_range(c(1, 2, 3, 4, 5), new_range(-3, 3)), "double")
expect_length(scale_range(c(1, 2, 3, 4, 5, 6), new_range(-3, 3)), 6)
})
With more complicated functions, e.g. where we expect lists or data.frames as an output we should check if the output has
- correct class (e.g. is it tibble or data.frame, consider
test_s3_class()) - proper number of dimensions, rows, columns or number of list elements(you can safely use
dim(),nrow(), etc. combined withexpect_equal()) - names of list elements or names of columns
- types/classes of particular elements or columns
- proper column labels or other attributes
- etc.