www.isr.uci.edu/tech-reports.html Scenario-based and State Machine-based Testing: An Evaluation of Automated Approaches

Abstract: Software testing is regarded by practitioners as the central activity used for ensuring that a system behaves as expected. It consists of the following activities: test generation, execution, evaluation, coverage analysis and regression testing. Testing is often a neglected activity that has the potential and the need for benefiting from automation. In fact, testing solutions currently available automate some testing activities. However, the majority of the automated solutions are code-based. Code-based testing uses the implementation to derive test cases. It can be used to test the system. It cannot, however, be used to test that the system built is what was intended. Testing activities that compare the system’s implementation to their expected behavior are more informative because they show the system satisfies its original requirements. This kind of activity is characteristic of specification-based testing. In practice, specification-based testing has not yet achieved the level of adoption achieved by code-based testing. This can be explained in part by the lack of support for its activities. These activities require support for managing relationships between specifications and other artifacts. Relationships include mapping from specification t