On the Design and Testing of Dependable Autonomous Systems

Designing software-intensive embedded systems for dependable autonomous applications is challenging. In addition to fulfilling complex functional requirements, the system must be safe under all operating conditions, even in the presence of faults. The key to achieving this is by simulating and testing the system enough, including possible faults that can be expected, to be confident that it reaches an acceptable level of performance with preserved safety. However, as the complexity of an autonomous system and its application grows, it becomes exponentially more difficult to perform exhaustive testing and explore the full state space, which makes the task a significant challenge. Property-Based Testing (PBT) is a software testing technique where tests and input stimuli for a system are automatically generated based on specified properties of the system, and it is normally used for testing software libraries. PBT is not a formal proof that the system fulfills the specified properties, but an effective way to find deviations from them. Safety-critical systems that must be able to deal with hardware faults are often tested using Fault Injection (FI) at several abstraction levels. The purpose of FI is to inject faults into a system in order to exercise and evaluate fault handling mechanisms. In this thesis, we utilize techniques from PBT and FI, for automatically testing functional and safety requirements of autonomous system simultaneously. We have done this on both simulations of hardware, and on real-time hardware for autonomous systems. This has been done in the process of developing a quadcopter system with collision avoidance, as well as when developing a self-driving model car. With this work we explore how tests can be auto-generated with techniques from PBT and FI, and how this approach can be used at several abstraction levels during the development of these systems. We also explore which details and design choices have to be considered while developing our simulators and embedded software, to ease testing with our proposed methods