A Runtime Testability Metric for Dynamic High-Availability Component-based Systems

Runtime testing is emerging as the solution for the integration and assessment of highly dynamic, high availability software systems where traditional development-time integration testing cannot be performed. A prerequisite for runtime testing is the knowledge about to which extent the system can be tested safely while it is operational, i.e., the system’s runtime testability. This article evaluates Runtime Testability Metric (RTM), a cost-based metric for estimating runtime testability. It is used to assist system engineers in directing the implementation of remedial measures, by providing an action plan which considers the trade-off between testability and cost. We perform a theoretical and empirical validation of RTM, showing that RTM is indeed a valid, and reasonably accurate measurement with ratio scale. Two testability case studies are performed on two different component-based systems, assessing RTM’s ability to identify runtime testability problems.Software Computer TechnologyElectrical Engineering, Mathematics and Computer Scienc