Evaluating user-computer interaction: a framework

A framework is described, which classifies usability evaluations in terms of three dimensions; the approach to evaluation, the type of evaluation and the time of evaluation in the context of the product life cycle. The approaches described are user-based, theory-based and expert-based. The approach to evaluation reflects the source of the data which forms the basis of the evaluation. The types of evaluation are diagnostic, summative and metrication. These reflect the purpose of the evaluation and therefore the nature of the data and likely use of the results. The time of testing reflects the temporal location in the product life cycle at which the evaluation is conducted. This dictates the representation of the product which is available for evaluation. The paper describes the relationship between these three framework dimensions. It also relates the methods of data capture, measurements and criteria which may be appropriately applied in various evaluation contexts. The latter part of the paper focuses on a more detailed review of methods which are associated with the most commonly applied and often most effective approach, i.e. the user-centred diagnostic evaluation. Finally the paper considers the need to perform evaluations more effectively in the design of products and systems in the commercial world. The discussion addresses the need for computer support tools to facilitate the handling of resulting data from user trials