Automated Selective Test Case Generation Methods for Real-Time Systems

This work aims to investigate the state of the art in test case generation for real-time systems, to analyze existing methods, and to propose future research directions in this area. We believe that a combination of design for testability, automation, and sensible test case selection is the key for verifying modern real-time systems. Existing methods for system-level test case generation for real-time systems are presented, classified, and evaluated against a real-time system model. Significant for real-time systems is that timeliness is crucial for their correctness. Our system model of the testing target adopts the event-triggered design paradigm for maximum flexibility. This paradigm results in target systems that are harder to test than its time-triggered counterpart, but the model improves testability by adopting previously proposed constraints on application behavior. This work investigates how time constraints can be tested using current methods and reveals problems relating to test-case generation for verifying such constraints. Further, approaches for automating the test-case generation process are investigated, paying special attention to methods aimed for real-time systems. We also note a need for special test-coverage criteria for concurrent and real-time systems to select test cases that increase confidence in such systems. We analyze some existing criteria from the perspective of our target model. The results of this dissertation are a classification of methods for generating test cases for real-time systems, an identification of contradictory terminology, and an increased body of knowledge about problems and open issues in this area. We conclude that the test-case generation process often neglects the internal behavior of the tested system and the properties of its execution environment as well as the effects of these on timeliness. Further, we note that most of the surveyed articles on testing methods incorporate automatic test-case generation in some form, but few consider the issues of automated execution of test cases. Four high-level future research directions are proposed that aim to remedy one or more of the identified problems