Video Processing Analysis For Non-Invasive Fatigue Detection And Quantification

Fatigue is a common symptom of weakness either physically or mentally. These symptoms may led to a drop in motivation, weakened sensitivity, slowing of responsiveness and inability to give full attention. All of these problems can cause adverse effects, such as accidents, especially those that require full attention as drivers of vehicles, and rail operators, the pilot of an aircraft or ship operators. This research investigates systems to detect and quantify the signs of fatigue using non-invasive facial analytics. There are four main algorithms that represent the major contribution from the PhD research. These algorithms encompass facial fatigue detection and quantification system as a whole. Firstly, a new technique to detect the face is introduced. This face detection algorithm is an affiliation of colour skin segmentation technique, connected component of binary image usage, and learning machine algorithm. The introduced face detection algorithm is able to reduce the false positive detection rate by a very significant margin. For the facial fatigue detection and quantification, the major fatigue signs features are from the eye activity. A new algorithm called the , Interdependence and Adaptive Scale Mean Shift (IASMS) is presented. The IASMS is able to quantify the state of eye as well as to track non-rigid eye movement. IASMS integrates the mean shift tracking algorithm with an adaptive scale scheme, which is used to track the iris and quantify the iris size. The IASMS is associated with face detection algorithm, image enhanced scheme, eye open detection technique and iris detection method in the initialisation process. This proposed method is able to quantify the eye activities that represent the blink rate and the duration of eye closure. The third contribution is yawning analysis algorithm. Commonly yawning is detected based on a wide mouth opening. Frequently however this approach is thwarted by the common human reaction to hand-cover the mouth during yawning. In this research, a new approach to analyse yawning which takes into account the covered mouth is introduced. This algorithm combines with a new technique of mouth opening measurements, covered mouth detection, and facial distortion (wrinkles) detection. By using this proposed method, yawning is still able to detect even though the mouth is covered. In order to have reliable results from the testing and evaluating of the developed fatigue detection algorithm, the real signs of fatigue are required. This research develops a recorded face activities database of the people that experience fatigue. This fatigue database is called as the Strathclyde Fatigue Facial (SFF). To induce the fatigue signs, ethically approved sleep deprivation experiments were carried out. In these experiments twenty participants, and four sessions were undertaken, which the participant has to deprive their sleep in 0, 3, 5, and 8 hours. The participants were subsequently requested to carry out 5 cognitive tasks that are related to the sleep loss. The last contribution of this research is a technique to recognise the fatigue signs. The existing fatigue detection system is based on single classification. However, this work presents a new approach for fatigue recognition which the fatigue is classified into levels. The levels of fatigue are justified based on the sleep deprivation stages where the SFF database is fully used for training, testing and evaluation of the developed fatigue recognition algorithm. This fatigue recognition algorithm is then integrated into a Fatigue Monitoring Tool (FMT) platform. This FMT has been used to test the participant that carried out the tasks as ship crew in shipping bridge simulator