Exploration of programming by demonstration approaches for smart environments

The number of smart electronic devices like smartphones, tablet computers and embedded sensors/actuators in our domestic and work environment is constantly growing. Some of them work as a stand along devices while others already collaborate with each other. It is apparent that once a common layer for device intercommunication between major consumer device manufactures has been agreed upon, a new class of networked smart applications will rise. These applications will dynamically utilise required sensors and actuators of a smart environment to optimally achieve tasks for us human users. Inhabitants of such environments are already interacting with dozens of computers per day. A lot of research has addressed many issues in hardware and software for the future smart environments But few have focused on the users. An important research topic lies in finding simple, intuitive yet powerful enough approaches to allow end-users to create and modify the behaviour of smart environments in which they live and work according to their needs. I believe that for the ubiquitous computing environments to reach its full potential, enabling end-user programming is one of the important criteria. This thesis describes the exploration of various approaches for "Do It Yourself" philosophy in smart environment applications by providing inhabitants with the appropriate tools which empower them to build their environments in accordance to their needs and with enough room for personal creativity. To this end, I choose speech as the main input by the end users along with demonstration of certain parts of over all approach in building applications for smart environments. The resulting application is built on top of the meSchup platform developed during meSchup FP7 EU project at the VIS institute in Stuttgart which provides a middleware for seamlessly interconnecting heterogeneous devices. The resulting web application is called "Speechweaver" which combines speech, programming by demonstration and automatic code generation into usable and intuitive approach for creating and modifying the rule based behaviour of smart environments in place