The control of 3D model in unity engine based on the user's pose in real time

U ovom radu je realizirano upravljanje 3D modela u Unity-u na temelju poze korisnika u realnom vremenu. Za upravljanje modelom korišten je Unity u kojemu je implementiran programski jezik C#, za detekciju poze korisnika u realnom vremenu koriste se biblioteke Mediapipe i OpenCV u Python-u koje služe za pozivanje funkcija koje obuhvaćaju područja računalnog vida te razvojno okruženje PyCharm i Microsoft Visual Studio. Model korišten za realizaciju paralelne animacije jest SMPL-X 3D model. U prvom dijelu rada opisan je sustav koji koristi algoritam za detekciju ljudskog kostura u RGB videu koristeći Mediapipe te OpenCV biblioteke. Detekcijom zglobova računaju se kutovi oko z osi koje se potom šalju u Unity putem Socket komunikacije. U drugom dijelu rada na temelju dobivenih podataka prikazana je realizacija paralelne animacije 3D modela u Unity-u.In this paper, the control of 3D model in Unity engine based on the user 's pose in real time is realized.. Unity, in which the C # programming language is implemented was used to manage the model, and Mediapipe and OpenCV libraries in Python were used to detect user poses in real time, which are used to call functions that include areas of computer vision and the PyCharm and Microsoft Visual Studio development environment. We used the SMPL-X 3D model. The first part of the paper describes a system that uses an algorithm for human skeleton detection in RGB video using Mediapipe and OpenCV libraries. Angle detection calculates angles around the z axis which are then sent to Unity via Socket communication. In the second part of the paper, based on the obtained data, the realization of parallel animation of a 3D model in Unity is presented