I've been working on updating MIDAS' testing suite. It is yet incomplete, but the basic structure is there. Once the code coverage reaches a more acceptable level and it is decided that the new tests are good replacements to the existing tests, it will be merged into develop.
Tests are located in $MIDASSYS/tests directory in the automated_testing branch.
ENABLE
For now changes are relegated to the automated_testing branch:
git pull
git checkout automated_testing
make cmake
STRUCTURE
Because MIDAS is implemented in three main languages (cxx, python, javascript), tests must be written in three languages. So each language has its own set of tests and frameworks in which they run. This introduces some new dependencies needed to run the tests:
| Toolchain |
New required deps |
New optional deps |
| C++ |
GoogleTest 1.15.2, CMake ≥ 3.24 |
|
| Python |
pytest, pytest-cov |
numpy, lz4 |
| JS |
Node ≥ 22, jsdom 25, @playwright/test 1, Chromium |
|
These dependencies are ONLY needed to run the test suites and do not affect the main build of MIDAS. Compiling the tests is controlled via the flag MIDAS_BUILD_TESTS, which is off by default, unless make test is called. Tests are located in the $MIDASSYS/tests directory, which should share roughly the same directory structure as the MDIAS source code.
USAGE
1. Make commands:
- "make test": run all tests
- "make coverage": run all tests and generate coverage reports. Reports are located as indicated in the below table
- "make coverage-recapture": regenerate coverage report without re-running tests.
Coverage reports locations:
| Suite |
Location |
Format |
| C++ |
cov_html/index.html |
HTML (lcov/genhtml) |
| Python |
cov_html_python/index.html |
HTML (pytest-cov), plus a term summary printed to stdout |
| JS |
tests/resources/coverage/lcov.info |
lcov info (V8 via npm run coverage) |
2. Run executables directly:
$MIDASSYS/tests/README.md has the full instructions on how to do this, as well as detailed descriptions on the tests and frameworks used. In general each cxx subdirectory has a binary executable to run the tests for that subdirectory group, whereas python and javascript have their own executables related to the frameworks running the tests. |