Report to the National Measurement System Policy Unit, Department of Trade and Industry, from the UK Software Support for Metrology Programme

ABSTRACT A general methodology for testing the numerical accuracy of scientific software is presented. The basis of the approach is the design and use of reference data sets and corresponding reference results to undertake black-box testing. Reference data sets and results are generated in a manner consistent with the functional specification of the problem addressed by the test software. In order to make the testing representative of a particular metrology application, and to consider "fitness for purpose" of the test software, data sets are produced with application-specific properties and various "degrees of difficulty". The results returned by the software for the reference data are compared objectively with the reference results. Quality metrics are used for this purpose that account for the key aspects of the problem. Complementary tests that do not require the availability of reference data are also described. These include consistency and continuity checks, spot checks against tabulated values, and checks of solution characterisations. Crow