Semantic Analysis of Driver Behavior by Data Fusion

Behavioral signal processing and data-fusion have been two important components of the analytical toolbox that is used to understand driver behavior and implement advanced driver assistance systems (ADAS). The recent need for quantitative analysis of driver behavior is now driven by a new revelation that incorporating human-like behavior and control strategies in the autonomous vehicles can increase their safety and acceptability in a mixed-fleet traffic environment. In addition to that, the overall safety and efficiency of the driver-vehicle system in a conditional or partial automation (Level 2–4) can be leveraged if the perception, cognition, and action capabilities of driver are enhanced based on driving-task or traffic-scenario. Motivated by this new interest, this work attempts to define a highlevel semantic analysis framework incorporating eye-motion, road-scene, and vehicle dynamics data. The study aims to identify general trends or patterns in driver behavior, especially concerning focus of attention (FoA), based on two categories: traffic scenario and complexity. To perform semantic analysis, open database from DR(eye)VE Project is used. First, the road-scene video and vehicle dynamics data are used together to obtain a complexity measure in addition to automatic recognition of the traffic-scenario. Next, the raw eye-movement data is processed to obtain gaze distribution maps and metrics. Then, a support vector machine (SVM) is trained using gaze metrics to infer the complexity level or the traffic-scenario. To obtain better separation between two classes (i.e., low vs high complexity or urban vs highway scenarios), the SVM is trained using Bayesian optimization. The results showed that based on the gaze distribution, it is possible to distinguish between urban and highway scenarios (85% accuracy), while this distinction between complexity levels can be even stronger (98% accuracy). The framework can be used as a high-level analysis and inference tool to discover behavioral characteristics of drivers and their relation to FoA patterns