Defeating barriers for resource usage testing for autonomous driving frameworks

The software used to implement advanced functionalities in critical domains (e.g. autonomous operation) impairs providing evidence that the software has enough resources to correctly execute (e.g. time and memory). This is not only due to the complexity of the underlying high-performance hardware deployed to provide the required levels of computing performance, but also due to the complexity, non-deterministic nature, and huge input space of the Artificial Intelligence (AI) algorithms used. In this Thesis, we focus on Apollo, an industrial-quality Autonomous Driving (AD) software framework. AD systems, similar to other automotive safety-critical systems, must undergo a development process with exhaustive Verification and Validation (V&V) steps. Both steps are challenged by the inherent complexity of AD systems. Our work can be divided into two contributions. First, we statistically characterise Apollo's observed execution time variability and reason on the sources behind it, aiming the verification step. We discuss the main challenges and limitations in finding a satisfactory software timing analysis solution for Apollo. While providing a consolidated solution for the software timing analysis of Apollo is a huge effort far beyond the scope of a single master Thesis, our work aims to set the basis for future and more elaborated techniques for the timing analysis of AD software. Second, we enable software resource usage testing, including execution time bounds and memory, on Apollo. Resource usage testing is a mandatory validation step during the integration of safety-related real-time systems. This Thesis exposes the difficulties to perform resource usage testing for AD frameworks by analysing a complex and critical module of Apollo. Then, it provides some guidelines and practical evidence on how resource usage testing can be effectively performed, thus enabling end-users to validate their safety-related real-time AD frameworks