Automated tests

From MidasWiki
Jump to navigation Jump to search

make test

After compiling midas with cmake, you can run an automated test script that will:

  • create a new midas experiment
  • run a few programs and commands
  • check that the output of the script matches what is expected.

From the main midas directory (not your build directory!) run:

unset MIDAS_EXPT_NAME
make test

At the end of the tests you should see

MIDAS self test completed with no errors

Code coverage

As of summer 2022, midas supports generating code coverage reports for the C++ code.

Pre-requisites

Install the lcov package from apt/yum/brew/macports depending on your OS preference.

Compiling with support for code coverage

Build midas with the "Coverage" build type, e.g.

mkdir build_cov
cd build_cov
cmake .. -DCMAKE_BUILD_TYPE=Coverage
make install

Generating coverage report

From the main midas directory (not your build directory!) run:

unset MIDAS_EXPT_NAME
make coverage

This will generate HTML files with annotated source code coverage based on the same tests as make test. It will also print an overall summary to screen.

The starting point is ~14% code coverage in summer 2022 before any dedicated effort to increase the coverage.

Reverting to the non-coverage build

Note that the default build type for midas is RelWithDebInfo. The coverage build disables optimizations and enables code coverage instrumentation, so is only suitable for running the code coverage tests!

To revert back to your normal build, do one of the following:

If you compiled the coverage version in a separate build directory (e.g. build_cov), run:

cd build
make install

If you compiled the coverage version in your regular build directory, run:

cd build
cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo
make install

BitBucket pipeline

Midas source code is hosted on BitBucket, and a CI/CD pipeline is configured to build on Ubuntu and run make test.

The pipeline is scheduled to run against the develop branch weekly (not on every commit), but can also be triggered manually by developers using the BitBucket pipelines page.