EVALUATING CHALLENGES, BENEFITS, AND DEPENDABILITY OF VIRTUAL AND PHYSICAL TESTING OF EMBEDDED SYSTEMS SOFTWARE

Software testing is a widely used quality assurance activity and often starts from the early development stages. However, starting early in the development process raises difficulties and challenges practitioners must deal with; most typically, hardware is not available in the required quantities, and there is the risk of damaging the hardware while testing. Emulating the physical hardware into virtual versions is a popular approach to overcome the mentioned obstacles. This master thesis, carried out at Westermo AB, investigates the differences between the physical and virtual hardware used for embedded system software testing and the possible benefits of combining both hardware versions in a hybrid system. Investigating differences between the hardware options helps identify which type of tests are more suitable in physical versus virtual hardware. The selected method for this thesis is a case study, starting with a pre-study phase investigating how other industries tackle the difficulties and challenges mentioned. Further, data were collected from two sources, historical test reports, and a questionnaire. The historical test reports showed a timing difference between virtual and physical hardware. The reset process of physical hardware is often faster than virtual hardware, but the link-up/link-down time is slower in physical hardware. The questionnaire also confirmed the timing differences as a significant challenge often experienced by engineers. Another challenge highlighted by the questionnaire answers is that "false positives" are typically caused by virtual hardware, where issues do not always turn up due to the virtual nature of communication. Another difference proven from this thesis is that virtual hardware is more failure-prone during the early stages of testing than physical hardware. The hybrid system could be advantageous in various ways, such as quantitatively expanding the current test systems and increasing test coverage. Future work could contribute with a proof-of-concept implementation of the hybrid system to confirm the advantages and demonstrate the third option of a test system.