Ultrasound 3D gesture recognition

Thesis (MEng)--Stellenbosch University, 2018.ENGLISH ABSTRACT: Interest in gesture recognition systems have grown with the recent advancements in the field of virtual and augmented reality. Gesture recognition provides a flexible interface that is not bound by the hardware as traditional mouse and keyboard combination do. This flexibility is important in virtual environments where the user has to interact with 3 dimensional objects. Having a reliable gesture recognition system will allow development of an intuitive user interface where the user can work with virtual objects as they would do in the real life. This research presents the development of a 2D beamforming microphone array for capturing 3D images, and the processing of the captured images for gesture recognition. The developed hardware consists of an 8x8 square array of MEMS microphones that capture pulsed sinusoids emitted by an ultrasonic transducer. The data from the microphone array is demodulated, filtered, and beamformed using appropriate methods to produce 3D images of the scene. The captured data is then processed, extracting only the relevant features, to a set of time-series vectors that represents the movement of a hand - i.e. a gesture. Using dynamic time warping (DTW) and k-nearest neighbours, the presented gestures are matched with previously captured templates, thereby recognising the type of gesture that was presented. The result showed very promising outcome with 97.5% accuracy in identifying correct gestures when the gestures are presented using a reflector, and 88.2% when the gestures are presented with a bare hand.AFRIKAANSE OPSOMMING: Geen opsomming beskikbaa