Controlling smart home environments with semantic connections : a tangible and an AR approach

In the transition from a device-oriented paradigm towards a more task-oriented paradigm with increased interoperability, people are struggling with inappropriate user interfaces, competing standards, technical incompatibilities and other difficulties. The current handles for users to explore, make and break connections between devices seem to disappear in overly complex menu structures displayed on small screens. This paper tackles the problem of establishing connections between devices in a smart home environment, by introducing an interaction model that we call semantic connections. Two prototypes are demonstrated that introduce both a tangible and an augmented reality approach towards exploring, making and breaking connections. In the augmented reality approach, connections between real-world objects are visualised by displaying visible lines and icons from a mobile device containing a pico-projector. In the tangible approach, objects are tagged and can be scanned to explore connection possibilities and manipulate the connections