Autonomous Software-in-the-Loop Modelling, Validation and Assessment

Software-in-the-Loop simulation has become an inevitable part of testing in the process of autonomous vehicle development. Simulation enables to test the system safely, allowing for further development of the AI system. It solves real-world problems safely and efficiently. This project develops an open-sourced Software-in-the-Loop (SIL) modelling, validation and assessment, that enables the testing of racing autonomous vehicles in a controlled environment. It contributes with an open-source solution to a widely extended necessity on the automotive industry of simulation testing. The simulator is developed with Gazebo and ROS, for real-time dynamic simulation of the autonomous vehicle and associated sensors. A performance evaluation tool has been developed with Matlab and Python, enabling the critical analysis and validation on vehicle and sensor behaviour. Performance metrics enhances the quantitative analysis, its tuning and the optimisation of the autonomous control system algorithms. Sensors are validated with available experimental data, with a selection of appropriate noise models. The stereo camera sensor is validated in different lighting and weather conditions. The effect of lighting is quantified from experimental tests. With the analysis of different weather conditions, it has been demonstrated the need of a Gaussian noise model to mimic sensor accuracy in the conditions of shadowing and raining. GPS and IMU sensors are validated with different kinds of noise, its modelling is developed with Matlab. Vehicles are modelled following provider specifications, while the FS electric vehicle OBR20 is also validated with available data from FS Austria 2018. Results are favourable when comparing simulation against experimental data. A design study demonstrates the abilities of the SIL to assist in control system tuning - resulting in just 8% lap-time difference from the optimum racing path, with an average velocity of 33.2km/h, reaching a maximum velocity of 40km/h on the AutoCross event - indicating a great performance and the value of using simulation for the optimisation of multiple parameters on the autonomous control system. The accurate, certain and reliable SIL, allows testing and development of new concepts on the Autonomous vehicle system in an Open-Source and safely environment. Furthermore, the author suggests new experimental tests and the collection of data in order to be able to model more scenarios that have an important effect on the sensors performance