Employing a comparative evaluation of Heuristic evaluations with end-users and usability experts as evaluators

There is scarce research that implement a formal framework when evaluating usability evaluation methods such as heuristicevaluation. This paper aimed to explore and compare the results of a heuristic evaluation performed by end-users and a heuristicevaluation performed by experts. Both heuristic evaluations took place in the context of forestry industry where a mobileapplication developed to give harvest operators performance feedback was evaluated. A thorough literature review for researchregarding evaluation of UEM was a crucial first step. The outcome of this produced an evaluation framework that included threecriteria, Relevance, Frequency and Timeliness. These criteria were used to analyse the results from the heuristics evaluationsperformed by the two groups, using mixed methods. The quantitative analysis concluded that the evaluation performed by theend-users had a higher frequency and relevance value, and that the evaluation performed by the expert group had higher valuefor their solution rate in the timeliness criteria. Furthermore, the qualitative analysis held within the criteria timeliness concludedthat the two groups identified different types of usability problems, confirming previous research performed on different types ofheuristic evaluators