IMU-RGBD camera 3d pose estimation and extrinsic calibration: Observability analysis and consistency improvement

Abstract — In this paper, we address the problem of ex-trinsically calibrating an inertial measurement unit (IMU) with respect to an RGBD sensor. In particular, we study the observability of the nonlinear IMU-RGBD calibration system and prove that the calibration parameters are observable given observations to a single point feature. Moreover, we show that the system has four unobservable directions corresponding to the global translation and rotations about the gravity vector. Based on the results of the observability analysis, we design a consistency-improved, observability constrained (OC) extended Kalman filter (EKF)-based estimator for calibrating the sensor pair while at the same time tracking its pose and creating a 3D map of the environment. Finally, we validate the key findings of the observability analysis and assess the performance of the OC-EKF estimator both in simulation and experimentally