Improving vision based pose estimation using LSTM neural networks

This paper deals with the development of a machine vision based pose estimation system for industrial robots and improving accuracy of the estimated pose using Long Short Term Memory (LSTM) neural networks. To this end, an LSTM network is proposed in order to improve the accuracy obtained from the Levenberg-Marquardt (LM) based pose estimation algorithm during trajectory tracking of the robot's end effector. The proposed method utilizes an LSTM network to extract dynamic features from the pose estimated by the LM algorithm and then feeding it to a regression layer to estimate the correct pose. Moreover, a target object trackable with a monocular camera with ± 90° in all directions was designed and fitted with fiducial markers. The designed placement of these fiducial markers guarantees the detection of at least two non-planar markers thus preventing ambiguities in pose estimation. The effectiveness of the proposed method is validated by an experimental study performed using a KUKA KR240 R2900 ultra robot while following sixteen distinct trajectories based on ISO 9238. The obtained results show that the proposed method significantly improves the pose estimation accuracy and precision of the vision based system during trajectory tracking of industrial robots' end effector