Data Representativeness in the Context of Pilot State Monitoring: A Case Study on Sleep and Drowsiness Detection and Certification Challenges

With the greater emphasis on flight safety, efficiency, cockpit automation and the inevitability of the changing role of a pilot, the aviation industry is becoming more open towards pilot monitoring technologies. If designed and used properly, they can detect, prevent, or mitigate undesirable and/or potentially dangerous states of a pilot, such as fatigue, sleep, loss of consciousness and others. Data-driven decision-making model, harnessing the capabilities of machine learning (as opposed to rule-based algorithms) is the essential element of the proposed pilot state monitoring system. However, leveraging artificial intelligence-based algorithms in aviation domain is a challenging task and the efforts to create unified guidelines for certifying such algorithms by aviation authorities are still in an early stage. Nevertheless, one could state that the machine learning model generalization capability, which is defined not only by the appropriate design of the model itself, but also by the amount of variance represented in the training data, is of the essence for the future certifiability. From the pilot state monitoring perspective, there are several sources of variability: state variability, environmental conditions variability and subject variability. Addressing the state variability means making sure that all the signatures of the state of interest (in this instance, sleep and drowsiness) are captured in the training data set. The signatures admitted into the model will depend on the chosen sensor set. As the ability of the sensors to extract signals may be affected by the properties of the environment, such as light, vibrations, etc., these properties must be considered to cover the environmental variability. Subject variability could be subdivided into the proper subject variability (e.g., sex, age, ethnicity) and situational subject variability (e.g., behaviors, worn garments and accessories). The pilot monitoring model for sleep and drowsiness detection with a camera-based system was trained/tested with the above-mentioned types of variability in mind. Validation of model was performed in the lab as well as in the real environment. This project has received funding from the Clean Sky 2 Joint Undertaking (JU) under grant agreement No 945583. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and the Clean Sky 2 JU members other than the Union. The results, opinions, conclusions, etc. presented in this work are those of the author(s) only and do not necessarily represent the position of the JU; the JU is not responsible for any use made of the information contained herein