Making Sense of Interaction Using a Model-Based Approach.

This thesis provides a theoretical method for developing and designing human computer interaction based on a continuous control process on mobile computing devices. This view provides a tight coupling between the user and system based on a continuous exchange of input/output dynamic information over a period of time, where continuous feedback from the display (visual/audio/haptic)influences the user's actions as more information becomes available and changes the user's perception. The proper representation and modeling of conceptual models in the interaction -via state-space model- and the explicit analysis of human behaviour and adaptability of the system to human behaviour -in the form of dynamic systems and probability theory are inherent to this framework. This framework supports continuous interaction techniques based on tilt inputs and multimodal outputs with handheld devices because one-handed control requires less visual attention and multimodality in the interaction can compensate for the lack of the screen space. The dynamic systems approach to the design of such continuous interactive interfaces allows the incorporation of analytical tools and constructive techniques from manual and automatic control theory, probabilistic models{and thus many of the techniques of machine learning{into the interface and integrating multimodality in a principled manner. Methods are presented for displaying the state of a system(visual/audio/haptic) with appropriate representation of a pseudo-physical model, via state-space model. Specifically, the use of predictive audio/visual-feedback for auditory/graphical display in a period of interaction is described, and it is shown how predictive elements can be introduced into goal directed displays, considering gains and delays present in the interaction loop. The use of these techniques in simulating the system behaviour before the actual implementation, and tuning and testing the system parameters are illustrated. Viewing human behaviour as a control process, a general framework for supporting human behaviour is developed, which supports intermittent interaction by smooth and natural dynamic mode switching. This is a probabilistic approach and not only applicable on small screen devices but also in many range of computing appliances. It provides general design guidelines for dynamic interactive systems based on models for the dynamic system, probabilistic language model and a probabilistic audio feedback