Interaktion in 3D durch Gesten mittels eines optischen Motion Capturing Verfahrens

Immersion is one of the most important issues, when talking about virtual and augmented reality. Actual technics of visual representation make it easy to afford the user a realistic feeling of being inside the virtual world. But not only visuals are important for a good immersion. The kind of interaction plays also a big role. Making inputs on keyboard or mouse, reduces the illusion of "real" interaction with the virtual objects. So some other method has to be found. An easy way is to use your fingers. In reality you grab an object with your hand. Why don't we do so in virtual reality? Aim of my diploma thesis is to test different possibilities of interacting with virtual objects by doing hand gestures. Several questions have to be answered during this work. First of all, how can the motion of the hand be tracked? Two webcams respectively take images of three illuminating diodes, which are attached to a black glove. For easy identifying these diodes, I use the colours red, green and blue. To make sure, only the light of the diodes is visible in the images, the shutters of the webcams are closed. So you get a black image with a little red, a little green and a little blue area of pixel. After averaging the x and y positions we have three points in each image, which can be used for triangulation. The results are three threedimensional points. Now there is a 3D representation of the users hand. In a little sample application should be shown, how you can interact with gestures. A puzzle with 16 parts and numbers on them is a good base to analyse different sets of gestures. Functions like translation, exchanging and rotating have to be realised by handmotions. Three sets of gestures were implemented and tested. The main idea of the gestures came from the internet browser Firefox and one of its plug-ins. You can go back and forward by gestures. Just push the right mouse button and "draw" a line right (forward) or left (back). This idea was implemented in my sample application. Because it is 3D, we have vectors we can compare to the drawn gesture. These vectors are X, Y and Z axis, respectively positive and negative direction. When "drawing" a gesture, this gesture is compared to the unit vectors. The lowest angle between two vectors indicates in which direction the gesture was drawn. This is very secure, and the rate of wrong recognition is very low. But it's not really intuitive. Another gesture set makes it possible to grab pieces of the puzzle and bring them to another position. In order to do this you have to do a grab gesture. Like in reality just bring the thump and the index finger together nearby a piece of the puzzle. As long as the two fingers stay together the piece follows each motion of the hand. Bringing the piece nearby another puzzlepiece and by releasing the grab gesture the two pieces are swabbed. To access several other functions like quit, or a training program for new users, there are also two possibilities, first by getting access to a Pie menu. This is a menu, whose functions are positioned in a circle, so the distance to each function is the same. Pie Menus are a very popular technic in VR/AR. The second method is the access to functions again by gestures. Drawing a circle, an arrow or the letter M activates the functions. After drawing one of these gestures several features are calculated to separate the gestures from each other. Dent, total length, distance between start and endpoint are very good features to ensure a secure and good recognition. All gesture sets were tested by several participants. This shows which gestures are the easiest one to learn and which gesture reduces the rate of wrong input by the user