|
| 1 | +This document describes the testing done in the OSIPI TF2.4 IVIM repository |
| 2 | + |
| 3 | + |
| 4 | +-- Outline -- |
| 5 | +1. Testing philosophy |
| 6 | +2. Testing structure |
| 7 | +3. Testing results |
| 8 | + |
| 9 | + |
| 10 | + |
| 11 | +-- Testing philosophy -- |
| 12 | +Testing is integral to the repository. |
| 13 | +There are many different contributions from many different people and only through diligent testing can they all be ensured to work correctly together. |
| 14 | +There are 3 major types of tests we conduct. |
| 15 | +1. Requirements |
| 16 | + - Which are required to pass |
| 17 | + - All algorithms have the same requirements |
| 18 | + - E.g. bounds honored, code runs reasonably |
| 19 | +2. Expectations |
| 20 | + - Considered warnings |
| 21 | + - Should not necessarily prevent a merge |
| 22 | + - Should cover performance changes |
| 23 | + - E.g. performance changes |
| 24 | +3. Performance |
| 25 | + - The accuracy of the results |
| 26 | + - The speed of the generated results |
| 27 | + |
| 28 | +-- Testing structure -- |
| 29 | +* The testing is controlled in several places. |
| 30 | +* The testing itself is done with pytest which parses files for "test_" and runs the appropriate tests. |
| 31 | +* The pytest testing can be done on your own machine by running "python -m pytest". |
| 32 | +** This is configured with pytests.ini and conftest.py |
| 33 | +* Testing on github is controlled by the github actions which are in the workflows folder. |
| 34 | +* Each workflow performs a series of tests, and is defined by the yml file. |
| 35 | +* Each workflow can run at specified times and with specified outputs. |
| 36 | +* Currently the major testing workflows are unit_test.yml and analysis.yml. |
| 37 | +* The unit_test workflow is done frequently and is relativly fast and does some basic algorithm testing. |
| 38 | +* The analysis workflow is done more infrequently with merges to the main branch and does more Expecation testing. |
| 39 | + |
| 40 | + |
| 41 | +-- Testing results -- |
| 42 | +The test results are written to several files, most notably an xml file for machine parsing, and a web page for code coverage. |
| 43 | +The "analysis" tests are written to a csv file as well. |
0 commit comments