A Systematic Method for Evaluating Implementations of Electronic Medical Records Systems in Low- and Medium-Income Countries

Background: Implementations of Electronic Medical Record systems (EMRs) in healthcare industry in Low- and Middle- Income Countries (LMICs) have been on the rise in recent years driven by the promise of improvements in quality of patient care, patient safety, and cost reduction. Several LMICs, including Kenya, Uganda, and Mozambique have rolled out EMRs within public facilities at a national level. By nature, EMRs implementations are complex, disruptive, costly, and have a potential to restructure workflow processes. Successful EMRs implementation depends on numerous factors, often going beyond a simple consideration of the technology used. While LMICs have embraced national-level EMRs implementations, little evidence exists on how to systematically evaluate success of these implementations. Hence, the aim of this research was to develop and apply a systematic method for evaluating the state of EMRs implementations based on Information Systems (IS) use construct. Methods: This research was anchored on Design Science Research (DSR) methodology. We employed DSR process model by Vaishnavi et al. five steps in the development of the artefact, namely: awareness of problem, suggestion, development, evaluation, and conclusion. The development step comprised of three sub-cycles that included development of EMRs use indicators utilizing Nominal Group Technique (NGT) process, application of the resultant EMRs use indicators to assess the actual state of EMRs implementations in the case study, and qualitative inquiry to identify barriers and facilitators to EMRs use through Focus Group Discussions (FGD) technique. The application of the quantitative and qualitative phases of the study were synthesized to give the overall result of the state of EMRs implementations in real environment settings. Results: A systematic method of evaluating the actual state EMRs implementations comprising of three sub-cycles was developed. The sub-cycles outputs were as follows: a set of fifteen indicators and metrics for monitoring and evaluating actual EMRs use, hinged on four dimensions namely: system use, data quality, interoperability, and reporting (sub-cycle 1); low system usage levels which pointed to salient challenges such as poor management of EMRs user accounts (sub-cycle 2); and EMRs use facilitators such as ease of use, ease of learn, system functionalities, adequate training, timely technical support, and collegial support were identified. However, factors related to infrastructure such as lack of power backup plan and insufficient hardware, as well as EMRs operation mode remain significant barriers to EMRs use (sub-cycle 3). The findings of these sub-cycles were used to validate the proposed methods and framework. Conclusion: Continuous monitoring and evaluation of information systems implementations for successful scale-up and sustainable initiatives remains paramount. The developed method to systematically evaluate the state of EMRs post-implementations can be applied in similar settings in developing countries. The study results provide relevant and actionable information that can benefit EMRs users, ministries of health, EMRs implementing partners, and funding agencies