Handbook
Gradient Handbook / Product Team / Coding / Package Development / Code Testing / Code Testing Protocol Edit this page

Code Testing Protocol

What makes code close to perfect?

Any code that we write should fulfill four requirements:

  1. It should be safe.
  2. It should be efficient.
  3. It should be proper.
  4. It should be elegant.

While the last point is subjective, the first three are indeed objective and crucial in any development process. And from all this characteristics safe is probably the most important.

We test our code to assure it is safe

Testing is a vital part of software development. It ensures that your code does what you want it to dohttp://r-pkgs.had.co.nz/tests.html

Automated testing allows us to assure that our code is working properly and catch errors early in development process. While some things might be hard or even impossible to test in automated way, the are still the foundation of development process.

It is also true that the process of writing automated tests is time consuming. However, the time is saved later during maintenance. One hour spent on development saves 5 hours on maintenance (author’s estimate). Automated testing also can be treated as a documentation for developers – since all the test are telling us what can be expected and was is not expected from function behavior. Taking this into account, test should not only cover the correctness of calculations or data wrangling processes. It should also cover structure, naming and classes of output.

What do we use to test our code?

Package Name Description
testthat Using testthat R package we can create expressions that ensures the function’s value is the desired object. For example, you can use test_that("str_length is number of characters", { expect_equal(str_length("a"), 1) or expect_equal(str_length("ab"), 2) expect_equal(str_length("abc"), 3)}). Additionally, we should keep functions in the R directory and tests in tests directory. We can use testthat workflow to compare results of our functions on example data.
roxygen2 roxygen2 is a package that helps creating documentation for functions. During the R CMD CHECK, it for example verifies whether all argument of functions are actually documented and whether each function has at least the required documentation fields filled, e.g. the title and description.
covr The covr package provides a framework for measuring unit test coverage. Unit testing is one of the cornerstones of software development. Any piece of R code can be thought of as a software application with a certain set of behaviors. Unit testing means creating examples of how the code should behave with a definition of the expected output. Measuring code coverage allows developers to asses their progress in quality checking their own (or their collaborators) code. Measuring code coverage allows code consumers to have confidence in the measures taken by the package authors to verify high code quality. covr provides three functions to calculate test coverage.
R CMD Check Sources your R directory and runs all possible examples included in the package. This will also run tests in your /tests directory. You can also check the package vignettes. Checks if we don’t use non-ascii STRINGS and similar stuff like that. This is designed for R packages but we can move a bit from R projects and pipelines into R packages and executions of functions.

What do we use to verify whether code is properly documented?

We perform R CMD CHECK with devtools::check() call which makes several tests. They include checks on documentation: it’s structure, if the examples are not throwing errors, if the functions from dependencies are properly imported etc. However there are many other things that are impossible to verify automatically, e.g. if the function arguments have sensible descriptions or if the code comments make sense. During development process we assign our work for review to others. The review process is crucial for finding errors, bugs and code refactoring that cannot be automated.

Sometimes we are using internal package functions that are not exported. It is good practice however to add some (brief) documentation for these functions. At minimum the should have @title and @description tag. Also the documentation for the internal functions should end with @noRd tag.

When is our code tested sufficiently?

  • You wrote tests for:
    • checking behavior with different inputs
    • checking errors, warnings and messages
    • checking output structure and classes
    • checking if output values are as expected
  • It passes all the tests
  • If it is better to test manually (e.g. function has side effects) you wrote a protocol covering what was/should be tested manually.
  • It has proper documentation
  • It has a proper amount of comments
  • Package can be installed without errors
  • The code passed peer review process

Overall recommendations

  1. We follow rules from Hadley’s R Packages http://r-pkgs.had.co.nz/
  2. We run analyses in the vignettes directory. Each client project is a part of g6tr.projects package and each Trendlines edition is a part of g6tr.voice package. Particular projects/trendlines editions have separate folders withing vignettes folder.
  3. We use sanity checks (e.g. if the arguments are of certain type, or if data.frame passed to function has all the needed columns) inside functions we develop.
  4. We create tests in tests directory with testthat.
  5. We create examples and documentation in R directory with roxygen2.
  6. We use R CMD CHECK to:
    • check examples and tests
    • check whether vignettes can be parsed
    • check whether functions are properly documented
  7. We verify the score of the coverage for the package with the covr tool
  8. After ensuring that tests are passing and the package can be installed we use code peer review process to check for any mistakes that we were unable to catch with automated tests