Human detection and tracking in robot workspace using RGB-D camera and YOLO algorithm

Robotski sustavi općenito bi trebali biti sigurni za okolinu u kojoj rade. Ako se u toj okolini nalaze i ljudi, tada se javlja potreba za sigurnosnim sustavima koji će osigurati zaštitu čovjeka i spriječiti koliziju čovjeka i robota koji se kreće. U ovome radu prikazan je prijedlog rješenja jednog sigurnosnog sustava. Sigurnosni sustav sastoji se od RGB-D kamere i robotskog manipulatora. Pomoću RGB-D kamere detektiraju se ljudi koji se nalaze u okolini robotskog sustava te se estimira njihova pozicija, smjer i brzina kretanja. S obzirom na poziciju, smjer i brzinu kretanja čovjeka, upravlja se radom robotskom manipulatora. Sustav je implementiran u ROS-u te je integriran s Microsoft Kinect RGB-D kamerom i ABB IRB 2400L robotskim manipulatorom. Uspješnost razvijenog sigurnosnog sustava ispitana je na posebno izgrađenom skupu podataka kojim se ispituju zahtjevi prema kojima je izrađen sigurnosni sustav. Prikazani su rezultati testiranja te mogućnosti poboljšanja rezultata i samog sigurnosnog sustava.In general, robotic systems should be safe for the environment they are working in. If humans are working in such environments, safety systems are needed which will ensure human protection and prevent collision between human and moving robot. In this paper, a solution proposal of a safety system is represented. The safety system includes an RGB-D camera and a robot manipulator. With the RGB-D camera, the safety system can detect humans located near the robot manipulator and estimate their position, walking speed and direction. With respect to the human position, speed and direction, working mode of the robot manipulator is regulated. The safety system is implemented in ROS and integrated with Microsoft Kinect RGB-D camera and ABB IRB 2400L robot manipulator. Performance of the developed safety system is tested on a dataset designed for testing the conditions according to which the safety system is created. The results of the developed safety system are shown with adduced possibilities for improvement of both the results and the safety system