Vision based natural assistive technologies with gesture recognition using Kinect

Assistive technologies for disabled persons are widely studied and the demand for them is rapidly growing. Assistive technologies enhance independence by enabling disabled persons to perform tasks that they are otherwise unable to accomplish, or have great difficulty accomplishing, by providing the necessary technology needed in the form of assistive or rehabilitative devices. Despite their great performance, assistive technologies that require disabled persons to wear or mount devices on themselves place undue restrictions on users.This thesis presents two different attempts of a solution to the problem, proposing two different novel approaches to achieve vision-based Human-Computer Interaction (HCI) for assisting persons with different disabilities to live more independently in a more natural way, in other words, without the need for additional equipment or wearable devices. The first approach aims to build a system that assists visually impaired persons to adjust to a new indoor environment. Assistants record important objects that are in fixed positions in the new environment using pointing gestures, and visually impaired persons may then use pointing gestures as a virtual cane to search for the objects. The second approach aims to build a system that can assist disabled persons who have difficulties in moving their arms or legs and cannot drive vehicles normally. In the proposed system, the user only needs one hand to perform steering, switching between gears, acceleration, brake and neutral (no acceleration, no brake).The technical contributions include a new method for pointing gesture extracted from forearm and a new algorithm for measuring the steering angle from the movement of a single hand. The proposed approaches were experimented with and both quantitative and qualitative analyses are presented. The experimental results show that the general performance of the implemented systems is satisfactory, though the effectiveness of the systems in a real-life situation for disabled persons is currently unknown. However, it is believed that these approaches can potentially be used in real life in future, and will encourage development of more natural assistive technologies