Usability Testing Methodology of Proactive HMIs for Virtual Control Room

The aim of this thesis is to develop methodology for usability testing of proactive HMIs. Usability is an important aspect of functionality of any Human-Machine Interface (HMI) since users directly interact with interfaces. Usability testing is, therefore, an important part of the development of HMIs. Emerging technologies, such as mobile devices and context-aware systems, pose new challenges for HMI developers due to new features and functionality. Since context of use is essential for HMI usability, HMI usability testing becomes especially important in context-aware systems. Both, the user and the HMI, need to have the same interpretation of context of use to ensure the usability of the HMI. At the same time, the context interpretation of a system should not conflict with the common sense of a user. A proactive HMI can be run on top of a context-aware system and it tries to predict next feasible action based on the context. There are two challenges for testing usability in this kind of HMI. Firstly, the proactive HMI is a new technology and there are no previous studies made on proactive HMI usability testing. It has been conjectured that new features like proactivity, adaptivity and multimodality may challenge the results of measurements for earlier deffined attributes of usability (SFS-EN ISO 1998) and return the misleading results. Therefore, the usability testing methodology should be further elaborated in order to capture the usability of the proactive features. Secondly, the HMI is ment to run on mobile devices. Mobile devices have many constrains, such as limited computational resources, connectivity issues and varying display resolutions. Despite of these constrains, information should be accessible at any place and any time with mobile devices. This thesis proposes a usability testing methodology of proactive HMI developed for Virtual Control Room (VCR). VCR is a proactive content-managing context-aware system; which was developed to increase the usability of embedded systems and human decision-making in data intensive environment. The methodology proposed in this thesis is based on Human-Centered Design (HCD) principles and consists of two stages: alpha (α) and beta (β) stage. In the α stage, traditional usability testing techniques are used with the addition of questions focused on getting information about what kind of proactive functionality the users would like to have on the HMI. In the β stage, a comparison of two different versions of the HMI, static and proactive, is conducted. This was done in order to test that the proactive functionality works well. It was found that the standard attributes of usability are not enough to capture the usability of the proactive HMIs: thus the rates for efficiency (which is measured as a function of time) were low despite of positive verbal feedbacks from the participants. Assuming the importance of context of use, a new usability attribute – transparency – was created to capture usability parameters related to proactivity. The transparency was measured via the participants’ reports on how easier they understand the HMIs elements and are capable to link them with a goal. In order to illustrate the methodology in use, two use cases were selected to develop the usability testing methodology. In the first one, an HMI was being developed for Building Management (BM) domain. The HMI was developed for an automated five building complex to assist the living conditions in the buildings. In total this encompassed 25 apartments that were controlled via the VCR system. Another use case was an HMI that was being developed for Production Management (PM) domain. The HMI was generic software to monitor and control production line systems. The approach selected in this thesis, allows an efficient way to receive feedback from usability tests and use this feedback to improve the proactive HMI. It is expected that the methodology developed in this thesis can be used to test other proactive HMI developed for automation systems