What do healthcare professionals need to turn risk models for type 2 diabetes into usable computerized clinical decision support systems? Lessons learned from the MOSAIC project

[EN] Background To understand user needs, system requirements and organizational conditions towards successful design and adoption of Clinical Decision Support Systems for Type 2 Diabetes (T2D) care built on top of computerized risk models. Methods The holistic and evidence-based CEHRES Roadmap, used to create eHealth solutions through participatory development approach, persuasive design techniques and business modelling, was adopted in the MOSAIC project to define the sequence of multidisciplinary methods organized in three phases, user needs, implementation and evaluation. The research was qualitative, the total number of participants was ninety, about five-seventeen involved in each round of experiment. Results Prediction models for the onset of T2D are built on clinical studies, while for T2D care are derived from healthcare registries. Accordingly, two set of DSSs were defined: the first, T2D Screening, introduces a novel routine; in the second case, T2D Care, DSSs can support managers at population level, and daily practitioners at individual level. In the user needs phase, T2D Screening and solution T2D Care at population level share similar priorities, as both deal with risk-stratification. End-users of T2D Screening and solution T2D Care at individual level prioritize easiness of use and satisfaction, while managers prefer the tools to be available every time and everywhere. In the implementation phase, three Use Cases were defined for T2D Screening, adapting the tool to different settings and granularity of information. Two Use Cases were defined around solutions T2D Care at population and T2D Care at individual, to be used in primary or secondary care. Suitable filtering options were equipped with "attractive" visual analytics to focus the attention of end-users on specific parameters and events. In the evaluation phase, good levels of user experience versus bad level of usability suggest that end-users of T2D Screening perceived the potential, but they are worried about complexity. Usability and user experience were above acceptable thresholds for T2D Care at population and T2D Care at individual. Conclusions By using a holistic approach, we have been able to understand user needs, behaviours and interactions and give new insights in the definition of effective Decision Support Systems to deal with the complexity of T2D care.The research leading to these results has received funding from the European Commission under the European Union's Seventh Framework Programme (FP7/2007-2013) grant agreement no 600914.Fico, G.; Hernandez, L.; Cancela, J.; Dagliati, A.; Sacchi, L.; Martinez-Millana, A.; Posada, J.... (2019). What do healthcare professionals need to turn risk models for type 2 diabetes into usable computerized clinical decision support systems? Lessons learned from the MOSAIC project. BMC Medical Informatics and Decision Making. 19(1):1-16. https://doi.org/10.1186/s12911-019-0887-8116191World Health Statistics 2018, Monitoring health for the SDGs, World Health Organization. Available at: https://www.who.int/gho/publications/world_health_statistics/en/ , last Accessed 09 Aug 2019.Kane R, Priester R, Totten A. Meeting the challenge of chronic illness. Baltimore: The Johns Hopkins University Press; 2005.Colagiuri, S., Kent, J., Kainu, T., Sutherland, S., Vuik, S. Rising to the challenge: preventing and managing type 2 diabetes, report of the WISH diabetes forum. 2015. Available from: http://www.wish.org.qa/wp-content/uploads/.../WISH_Diabetes_Forum_08.01.15_WEB-1.pdf . Accessed 09 Aug 2019.IDFD Atlas. 2017. Available from: http://www.diabetesatlas.org/resources/2017-atlas.html . Accessed 11 Feb 2018.American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27(9):2262–5.Noble D, Mathur R, Dent T, Meads C, Greenhalgh T. Risk models and scores for type 2 diabetes: systematic review. BMJ. 2011;343:d7163.Abbasi A, Peelen LM, Corpeleijn E, van der Schouw YT, Stolk RP, Spijkerman AM, et al. Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ. 2012;345:e5900.Zarkogianni K, Litsa E, Mitsis K, Wu P, Kaddi CD, Cheng C, Wang MD, Nikita KS. A review of emerging technologies for the management of diabetes mellitus. IEEE Trans Biomed Eng. 2015;62(12):2735–49.Garg AX, Adhikari NK, McDonald H, Rosas-Arellano M, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38.Roshanov PS, et al. Computerized clinical decision support systems for chronic disease management: a decision-maker-researcher partnership systematic review. Implement Sc. 2011;6(1):92.Roshanov PS, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013;346:f657.Miller A, Moon B, Anders S, Walden R, Brown S, Montella D. Integrating computerized clinical decision support systems into clinical work: a meta-synthesis of qualitative research. Int J Med Inform. 2015;84(12):1009–18.Patel VL, Kannampallil TG. Cognitive informatics in biomedicine and healthcare. J Biomed Inform. 2015;53:3–14.Zhang J. Human-centered computing in health information systems part 1: analysis and design. J Biomed Inform. 2005;38(1):1–3.Rinkus S, Walji M, Johnson-Throop KA, Malin M, Turley JP, Smith JW, Zhang J. Human-centered design of a distributed knowledge management system. J Biomed Inform. 2005;38:4–17.Nemeth CP, Nunnally M, O’Connor M, Klock PA, Cook R. Getting to the point: developing IT for the sharp end of healthcare. J Biomed Inform. 2005;38:18–25.Xiao Y. Artifacts and collaborative work in healthcare: methodological, theoretical and technological implications of the tangible. J Biomed Inform. 2005;38:26–33.Malhotra S, Laxmisan A, Keselman A, Zhang J, Patel VL. Designing the design phase of critical care devices: a cognitive approach. J Biomed Inform. 2005;38:34–50.Samaras GM, Horst RL. A systems engineering perspective on the human-centered design of health information systems. J Biomed Inform. 2005;38:61–74.Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform. 2005;38:75–87.Patterson ES, Boebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. J Biomed Inform. 2005;38:189–99.Laxmisan A, Malhotra S, Keselman A, Johnson TR, Patel VL. Decisions about critical events in device-related scenarios as a function of expertise. J Biomed Inform. 2005;38:200–12.Ginsburg GE. Human factors engineering: a tool for medical device evaluation in hospital procurement decision-making. J Biomed Inform. 2005;38:213–9.Reddy M, McDonald DW, Pratt W, Shabot MM. Technology, work, and information flows: lessons from the implementation of a wireless alert pager system. J Biomed Inform. 2005;38:229–38.Despont-Gros C, Mueller H, Lovis C. Evaluating user interactions with clinical information systems: a model based on human–computer interaction models. J Biomed Inform. 2005;38:244–55.World Health Organization, 2009. Practical guidance for scaling up health service innovations.European Commission, 2015. European scaling up strategy on active and healthy ageing.van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, Seydel ER. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13(4):e111.Sacchi L, Dagliati A, Segagni D, Leporati P, Chiovato L, Bellazzi R. Improving risk-stratification of diabetes complications using temporal data mining. In: 2015 37th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC). United States: IEEE; 2015. p. 2131–4.Sambo F, Di Camillo B, Franzin A, Facchinetti A, Hakaste L, Kravic J, Fico G, et al. A Bayesian Network analysis of the probabilistic relations between risk factors in the predisposition to type 2 diabetes. In: 2015 37th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC). United States: IEEE; 2015. p. 2119–22.Van Velsen L, van Gemert-Pijnen L, Nijland N, Beaujean D, Van Steenbergen J. Personas: the linking pin in holistic design for eHealth. proc. eTELEMED; 2012.Van Velsen L, Wentzel J, Van Gemert-Pijnen JE. Designing eHealth that matters via a multidisciplinary requirements development approach. JMIR Res Protoc. 2013;2(1):e21.Maurya A. Running lean: iterate from plan A to a plan that works. United States: O’Reilly Media, Inc.; 2012.Wentzel J, Van Limburg M, Karreman J, Hendrix R, Van Gemert-Pijnen L. Co-creation with stakeholders: a Web 2.0 Antibiotic Stewardship Program. Proceedings of The Fourth International Conference on eHealth, Telemedicine, and Social Medicine: January 30, 2012 to February 4, 2012; Valencia. 2012:196–202.Morgan DL. Focus groups as qualitative research. Thousand Oaks: Sage; 1997.Saaty T. How to structure and make choices in complex problems. Hum Syst Manag. 1982;3:255–61.Saaty TL. A scaling method for priorities in hierarchical structures. J Math Psychol. 1977;15:234–81.Pecchia L, Bath PA, Pendleton N, Bracale M: Web-based system for assessing risk factors for falls in community-dwelling elderly people using the analytic hierarchy process. International Journal of the Analytic Hierarchy Process. 2010;2(2):135–57.Fico G, Gaeta E, Arredondo MT, Pecchia L. Analytic hierarchy process to define the most important factors and related technologies for empowering elderly people in taking an active role in their health. J Med Syst. 2015;39(9):1–7.Goepel KD. Implementation of an online software tool for the Analytic Hierarchy Process (AHP-OS). Int J Anal Hierarchy Process. 2018;10(3):469–87. https://doi.org/10.13033/ijahp.v10i3.590 .Nielsen J. Ten usability heuristics. United States: Nielsen Norman Group; 2005.Brooke J. SUS-A quick and dirty usability scale. Usability Eval Ind. 1996;189(194):4–7.Hassenzahl M, Burmester M, Koller F. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Mensch & computer. Germany: Vieweg+ Teubner Verlag; 2003, 2003. p. 187–96.International Organization for Standardization. Ergonomics of human-system interaction: part 210: human-centred design for interactive systems. United States: ISO; 2010.Sauro J, Lewis JR. Quantifying the user experience: practical statistics for user research. Burlington: Morgan Kaufmann; 2012.Gülcü C. The complete log4j manual. QOS. ch; 2003.Nantz B. Open source. NET development: programming with NAnt, NUnit, NDoc, and More. United States: Addison-Wesley Professional; 2004.Borsci S, Federici S, Lauriola M. On the dimensionality of the system usability scale: a test of alternative measurement models. Cogn Process. 2009;10(3):193–7.Borsci S, Federici S, Bacci S, Gnaldi M, Bartolucci F. Assessing user satisfaction in the era of user experience: comparison of the SUS, UMUX, and UMUX-LITE as a function of product experience. Int J Hum Comput Interact. 2015;31(8):484–95.Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems. 1993. pp. 206–13. ACM.Dagliati A, Sacchi L, Tibollo V, Cogni G, Teliti M, Martinez-Millana A, et al. A dashboard-based system for supporting diabetes care. J Am Med Inform Assoc. 2018;25(5):538–47.Fico G, et al. User requirements for incorporating diabetes modeling techniques in disease management tools. In: 6th European conference of the international federation for medical and biological engineering. Switzerland: Springer International Publishing; 2015.Cancela J, Hernandez L, Fico G, Waldmeyer MTA. Heuristic evaluation of a toolset for type 2 diabetes mellitus management. In: XIV Mediterranean conference on medical and biological engineering and computing. Switzerland: Springer International Publishing; 2016, 2016. p. 982–7.Borsci S, Uchegbu I, Buckle P, Ni Z, Walne S, Hanna GB. Designing medical technology for resilience: integrating health economics and human factors approaches. Expert Rev Med Devices. 2018;15(1):15–26