Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop

Devices and programs using digital technology to foster or support behavior change (digital interventions) are increasingly ubiquitous, being adopted for use in patient diagnosis and treatment, self-management of chronic diseases, and in primary prevention. They have been heralded as potentially revolutionizing the ways in which individuals can monitor and improve their health behaviors and health care by improving outcomes, reducing costs, and improving the patient experience. However, we are still mainly in the age of promise rather than delivery. Developing and evaluating these digital interventions presents new challenges and new versions of old challenges that require use of improved and perhaps entirely new methods for research and evaluation. This article discusses these challenges and provides recommendations aimed at accelerating the rate of progress in digital behavior intervention research and practice. Areas addressed include intervention development in a rapidly changing technological landscape, promoting user engagement, advancing the underpinning science and theory, evaluating effectiveness and cost-effectiveness, and addressing issues of regulatory, ethical, and information governance. This article is the result of a two-day international workshop on how to create, evaluate, and implement effective digital interventions in relation to health behaviors. It was held in London in September 2015 and was supported by the United Kingdom’s Medical Research Council (MRC), the National Institute for Health Research (NIHR), the Methodology Research Programme (PI Susan Michie), and the Robert Wood Johnson Foundation of the United States (PI Kevin Patrick). Important recommendations to manage the rapid pace of change include considering using emerging techniques from data science, machine learning, and Bayesian approaches and learning from other disciplines including computer science and engineering. With regard to assessing and promoting engagement, a key conclusion was that sustained engagement is not always required and that for each intervention it is useful to establish what constitutes “effective engagement,” that is, sufficient engagement to achieve the intended outcomes. The potential of digital interventions for testing and advancing theories of behavior change by generating ecologically valid, real-time objective data was recognized. Evaluations should include all phases of the development cycle, designed for generalizability, and consider new experimental designs to make the best use of rich data streams. Future health economics analyses need to recognize and model the complex and potentially far-reaching costs and benefits of digital interventions. In terms of governance, developers of digital behavior interventions should comply with existing regulatory frameworks, but with consideration for emerging standards around information governance, ethics, and interoperability