Skip to content

Testing

Test specific dependencies

Test dependencies now need to be added to the Project.toml file in ./test:

pkg> activate ./test
pkg> add MyPkg
pkg> activate .

All dependencies used in tests now have to be manually added. They do not "carry over" from the main package.

Developing test set dependencies seems to cause problems ("error: cannot merge projects"). They need to be added.

Important note: For now (1.5) test dependencies are still considered "beta" and buggy. Do not use. Instead, use the 1.0 method of extras in Project.toml.

Structuring tests

The goal is to be able to run subsets of tests and to encapsulate the code of each test. One way of achieving this:

# runtests.jl


@testset "All" begin
    include("test_one.jl");
end


# test_one.jl

using Test

function test_one()
    @testset "A" begin
        @test 1 == 1
    end
end

@testset "One" begin
    test_one();
end

Now Pkg.test() runs everything, but include("test/test_one.jl") only runs the subset. SafeTests.jl goes further by wrapping tests in modules.

Test helpers can be put into a separate file and conditionally included, as in (taken from MPVerify.jl):

@isdefined(TestHelpers) || include("../TestHelpers.jl")

This checks whether the module TestHelpers exists. But one may not need a module for the helpers.

Useful packages

  • Aqua.jl
    • checks for method ambiguities, invalid exports, stale dependencies, and more.
  • TestSetExtensions.jl
    • mainly provides nicer display of test progress and failures
  • UnitTestDesign
    • generates combinations of arguments that are passed to tests
    • the goal is to generate coverage without having to run all parameter combinations