Real-world, real-time robotic grasping with convolutional neural networks

© Springer International Publishing AG 2017. Adapting to uncertain environments is a key obstacle in the development of robust robotic object manipulation systems, as there is a trade-off between the computationally expensive methods of handling the surrounding complexity, and the real-time requirement for practical operation. We investigate the use of Deep Learning to develop a real-time scheme on a physical robot. Using a Baxter Research Robot and Kinect sensor, a convolutional neural network (CNN) was trained in a supervised manner to regress grasping coordinates from RGB-D data. Compared to existing methods, regression via deep learning offered an efficient process that learnt generalised grasping features and processed the scene in real-time. The system achieved a successful grasp rate of 62% and a successful detection rate of 78% on a diverse set of physical objects across varying position and orientation, executing grasp detection in 1.8 s on a CPU machine and a complete physical grasp and move in 60 s on the robot