Adaptation of multimodal outputs

This thesis focuses on Adaptive Multimodal Systems and their applications on improving user interface accessibility. Disabled and/or elderly people are a group at high risk of social exclusion. The access to the opportunities offered by society is obviously limited if these cannot be reached by persons with impairments or restricted mobility. A more subtle way of exclusion results from the sensory modalities in which they are presented. Therefore, if the presentation of information has only one modality it will exclude people with impairments in that particular sensory modality. As a solution to these problems, this document presents GUIDE, an European funded project which intends to develop a software framework which allows developers to efficiently integrate accessibility features into their applications. To perform adaptation the system must know the users, their characteristics and preferences. Thus, a prototype was implemented to assist in user trials. These trials had the goal to understand users’ interaction patterns as well as to group users with common characteristics. In order to match a user with its cluster, it was implemented a prototype named User Initialisation Application (UIA) that besides of tutoring the user on how to interact with the system, it asks the user to perform some tasks and answer some questions. When finished the UIA is able to decide which group the user identifies with. This thesis takes special focus on Dialogue Manager as it is the core component of the system architecture. It coordinates the activity of several subcomponents in a dialogue system and its main goal is to maintain a representation of the current state of the ongoing dialogue. This document presents the design of the Dialogue Manager that will run in the GUIDE framework. Additionally, in order to the Dialogue Manager and the other components understand the applications’ user interface it was implemented a tool that extracts a User Interface Markup Language (UIML) from a Web based application