Interface design recommendations for computerised clinical audit and feedback: Hybrid usability evidence from a research-led system

AbstractBackgroundAudit and Feedback (A&F) is a widely used quality improvement technique that measures clinicians’ clinical performance and reports it back to them. Computerised A&F (e-A&F) system interfaces may consist of four key components: (1) Summaries of clinical performance; (2) Patient lists; (3) Patient-level data; (4) Recommended actions. There is a lack of evidence regarding how to best design e-A&F interfaces; establishing such evidence is key to maximising usability, and in turn improving patient safety.AimTo evaluate the usability of a novel theoretically-informed and research-led e-A&F system for primary care (the Performance Improvement plaN GeneratoR: PINGR).Objectives(1) Describe PINGR’s design, rationale and theoretical basis; (2) Identify usability issues with PINGR; (3) Understand how these issues may interfere with the cognitive goals of end-users; (4) Translate the issues into recommendations for the user-centred design of e-A&F systems.MethodsEight experienced health system evaluators performed a usability inspection using an innovative hybrid approach consisting of five stages: (1) Development of representative user tasks, Goals, and Actions; (2) Combining Heuristic Evaluation and Cognitive Walkthrough methods into a single protocol to identify usability issues; (3) Consolidation of issues; (4) Severity rating of consolidated issues; (5) Analysis of issues according to usability heuristics, interface components, and Goal-Action structure.ResultsA final list of 47 issues were categorised into 8 heuristic themes. The most error-prone heuristics were ‘Consistency and standards’ (13 usability issues; 28% of the total) and ‘Match between system and real world’ (n=10, 21%). The recommended actions component of the PINGR interface had the most usability issues (n=21, 45%), followed by patient-level data (n=5, 11%), patient lists (n=4, 9%), and summaries of clinical performance (n=4, 9%). The most error-prone Actions across all user Goals were: (1) Patient selection from a list; (2) Data identification from a figure (both population-level and patient-level); (3) Disagreement with a system recommendation.ConclusionsBy contextualising our findings within the wider literature on health information system usability, we provide recommendations for the design of e-A&F system interfaces relating to their four key components, in addition to how they may be integrated within a system