Accounting for Domain Context in Evaluation

Work is situated activity. Taking into account human factors in evaluation involves considering not only users but also their contexts of use. Consequently, the evaluation of systems — from video-games to safetycritical interfaces — requires analysis of context to understand not only the effect of context on usability but also the impact of artifacts\u27 usability on users\u27 environments. In the case of safety-critical systems (SCS), errors (by users or designers) may threaten human lives. To assess the degree to which interface evaluation methods currently account for context, we have used the research strategy taxonomy of McGrath as a framework for classifying existing evaluation methods of aviation domain and general HCI interactive systems. This framework enabled us to describe common grounds and key differences of methods used in HCI and SCS, and to highlight aspects of context that could be analyzed using each strategy. For instance, characteristics of SCS, such as time-criticality, unpredictability and dynamics, emphasize the leading role of operational context on the remaining work context including physical or technical constraints defined by organizational, social, cultural and technical contexts which is not the case for general HCI