Automated three-dimensional body orientation reconstruction and motion tracking with two views during avoidance maneuver of bumblebee

The bumblebee has excellent performance for flying through complex natural environments and avoiding obstacles. So, it is of interest for engineering and biology to analyze how its flight behaviors are modified during avoidance maneuvers. Thus, we developed a behavior analysis pipeline to auto-track bumblebees and 3D reconstruct its flight motion only in two cameras. The cameras used in this study are calibrated and their poses are linearly and non-linearly estimated under M-estimator Sample and Consensus (MSAC) outliers rejection. The 3D reconstruction is realized by triangulation with root mean square error (r.m.s error) around 1mm. The tracking algorithm is constructed based on different brightness values of the wing and body. Under image arithmetic and Morphological transformation (like erosion, dilation,etc.), wings and body can be separated from each other. For body orientation estimation, unlikely the traditional way to reconstruct 3D body from 2D images, the bumblebee's body is simplified as a 3D ellipsoid and projected to the image planes. With a defined error function, the projection of ellipsoid will eventually be fitted into the body contours of bumblebee extracted from videos by adjusting the orientation of the ellipsoid in 3D. In this case, only pitch and yaw are considered. The average root mean square error is 3.58±1.21 deg in pitch and 2.84±0.90 deg in yaw across five analyzed videos. Thus, we show that two views are sufficient for this study with a simplified algorithm, comparing to three or more views. The pipeline we developed could accelerate the process of unveiling the flight strategies and biomechanics of insects flight through complex, structured aerial environments