The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

ABSTRACT Objective To evaluate the accuracy of digital and online symptom checkers in providing diagnoses and appropriate triage advice. Design Systematic review. Data sources Medline and Web of Science were searched up to 15 February 2021. Eligibility criteria for study selection Prospective and retrospective cohort, vignette, or audit studies that utilised an online or application-based service designed to input symptoms and biodata in order to generate diagnoses, health advice and direct patients to appropriate services were included. Main outcome measures The primary outcomes were (1) the accuracy of symptom checkers for providing the correct diagnosis and (2) the accuracy of subsequent triage advice given. Data extraction and synthesis Data extraction and quality assessment (using the QUADAS-2 tool) were performed by two independent reviewers. Owing to heterogeneity of the studies, meta-analysis was not possible. A narrative synthesis of the included studies and pre-specified outcomes was completed. Results Of the 177 studies retrieved, nine cohort studies and one cross-sectional study met the inclusion criteria. Symptom checkers evaluated a variety of medical conditions including ophthalmological conditions, inflammatory arthritides and HIV. 50% of the studies recruited real patients, while the remainder used simulated cases. The diagnostic accuracy of the primary diagnosis was low (range: 19% to 36%) and varied between individual symptom checkers, despite consistent symptom data input. Triage accuracy (range: 48.8% to 90.1%) was typically higher than diagnostic accuracy. Of note, one study found that 78.6% of emergency ophthalmic cases were under-triaged. Conclusions The diagnostic and triage accuracy of symptom checkers are variable and of low accuracy. Given the increasing push towards population-wide digital health technology adoption, reliance upon symptom checkers in lieu of traditional assessment models, poses the potential for clinical risk. Further primary studies, utilising improved study reporting, core outcome sets and subgroup analyses, are warranted to demonstrate equitable and non-inferior performance of these technologies to that of current best practice. PROSPERO registration number CRD42021271022. SUMMARY BOXES What is already known on this topic Chambers et al. (2019) have previously examined the evidence underpinning digital and online symptom checkers, including the accuracy of the diagnostic and triage information, for urgent health problems and found that diagnostic accuracy was generally low and varied depending on the symptom checker used. Given the increased reliance upon digital health technologies by health systems in light of the ongoing COVID-19 pandemic, in addition to the marked increase in availability of similarly themed digital health products since the last systematic review, a contemporary and comprehensive reassessment of this class of technologies to ascertain their diagnostic and triage accuracy is warranted. What this study adds Our systematic review demonstrates that the diagnostic accuracy of symptom checkers remains low and varies significantly depending on the pathology or symptom checker used. The findings of this systematic review suggests that this class of technologies, in their current state, poses significant risk for patient safety, particularly if utilised in isolation