Robust Vehicle and Surrounding Environment Dynamic Analysis for Assistive Driving Using Visual-Inertial Measurements

Vehicle and surrounding environment dynamic analysis (VSEDA) is an indispensable component of modern assistive drivings. A robust and accurate VSEDA could ensure the driving system reliability in presence of highly dynamic environments. This paper proposes a novel VSEDA framework by fusing the measurements from an inertial sensor and a monocular camera. Compared to traditional visualinertial-based assistive driving methods, the proposed approach can analyze both the vehicle dynamics and the surrounding environment. Even in the scenario that moving objects occupy a majority area of the scene captured in the image, the proposed method can still robustly analyze the surrounding environment by identifying the static inliers and dynamic inliers, which lie on stationary objects and moving objects, respectively. The theoretical framework consists of three steps. First, the vehicle nonholonomic constraint is applied to pairwise feature matching. For vehicle dynamic analysis, the static inliers are selected by choosing the features with their histogram bins consistent with inertial orientations. Second, for the surrounding environment dynamic analysis, the dynamic inliers are matched through histogram voting, together with the developed part-based vehicle detection model that can segment and match the vehicle regions from the background in image pairs. Finally, both the vehicle dynamics and surrounding environments are analyzed with static and dynamic inliers respectively. The proposed method has been evaluated on the challenging datasets, part of which was collected during rush hours in downtown areas. The experimental results prove the effectiveness and accuracy of the proposed VSEDA