Evaluating the impact of integrated development: are we asking the right questions? A systematic review [version 2; referees: 2 approved, 1 approved with reservations]

Background: Emerging global transformations - including a new Sustainable Development Agenda - are revealing increasingly interrelated goals and challenges, poised to be addressed by similarly integrated, multi-faceted solutions. Research to date has focused on determining the effectiveness of these approaches, yet a key question remains: are synergistic effects produced by integrating two or more sectors? We systematically reviewed impact evaluations on integrated development interventions to assess whether synergistic, amplified impacts are being measured and evaluated. Methods: The International Initiative for Impact Evaluation’s (3ie) Impact Evaluation Repository comprised our sampling frame (n = 4,339). Following PRISMA guidelines, we employed a three-stage screening and review process. Results: We identified 601 journal articles that evaluated integrated interventions. Seventy percent used a randomized design to assess impact with regard to whether the intervention achieved its desired outcomes. Only 26 of these evaluations, however, used a full factorial design to statistically detect any synergistic effects produced by integrating sectors. Of those, seven showed synergistic effects. Conclusions: To date, evaluations of integrated development approaches have demonstrated positive impacts in numerous contexts, but gaps remain with regard to documenting whether integrated programming produces synergistic, amplified outcomes. Research on these program models needs to extend beyond impact only, and more explicitly examine and measure the synergies and efficiencies associated with linking two or more sectors. Doing so will be critical for identifying effective integrated development strategies that will help achieve the multi-sector SDG agenda