Conceptual and statistical analysis of complex interventions in the presence of confounding variables: An example from public health

Background: Meta-analyses of complex interventions are challenging because causality operates through multiple paths and confounding variables can be difficult to distinguish. Objectives: To meta-analyse public health interventions that engage members of the community in their conception, design, or delivery. To disentangle intervention complexity by analysing according to their theories of change. Study selection criteria: Published after 1990; outcome or process evaluation; community engagement intervention; written in English; reported health or community outcomes; study populations or differential impacts reported according to social determinants of health. Analysis: Intervention complexity was examined by conceptualising, operationalising, and mapping their theories of change; and through random effects subgroup analyses. Main results: 131 studies were included in the synthesis. Three main theories of change were identified, which were useful in describing trends in intervention effectiveness. Statistically significant between-group differences were not detected, since there were likely to have been too many confounding variables. Conclusions: Intervention complexity in systematic reviews can be addressed through examining theories of change and trends in effect size estimates. Such complexity appears to defy current meta-analytical methods when confounding variables undermine analysis of variance