Assessing intervention fidelity in a randomized field experiment: Illuminating the black box

The inner workings of an intervention can be thought of as a black box when they are not investigated directly. Typically researchers do not fully illuminate the black box and make assumptions about the presence of underlying processes instead. The current study utilizes an adapted five-step framework for the systematic intervention fidelity assessment of a motivation-based field experiment. The five-step framework is applied to a previously implemented study of student motivation in an introductory psychology course. Data irregularities necessitated the introduction of a new method for calculating differences in achieved relative strength indices. The intervention was found to be implemented with a high degree of fidelity in both of the treatment conditions at micro and macro levels of analysis. Based on the results, it appears that the intervention was implemented effectively, which allows for conclusions about the intervention’s impact on outcomes without the typical black box assumptions. Ultimately the effects of the intervention on outcomes were minimal but consistent with prior research. Based on these findings, future implementations should consider more or stronger doses of the intervention to increase the intervention’s effect size