Factor structure and measurement invariance of the Health Education Impact Questionnaire: does the subjectivity of the response perspective threaten the contextual validity of inferences?

Objective: On-going evidence is required to support the validity of inferences about change and group differences in the evaluation of health programs, particularly when self-report scales requiring substantial subjectivity in response generation are used as outcome measures. Following this reasoning, the aim of this study was to replicate the factor structure and investigate the measurement invariance of the latest version of the Health Education Impact Questionnaire, a widely used health program evaluation measure.Methods: An archived dataset of responses to the most recent version of the English-language Health Education Impact Questionnaire that uses four rather than six response options (N=3221) was analysed using exploratory structural equation modelling and confirmatory factor analysis appropriate for ordered categorical data. Metric and scalar invariance were studied following recent recommendations in the literature to apply fully invariant unconditional models with minimum constraints necessary for model identification.Results: The original eight-factor structure was replicated and all but one of the scales (Self Monitoring and Insight) was found to consist of unifactorial items with reliability of ⩾0.8 and satisfactory discriminant validity. Configural, metric and scalar invariance were established across pre-test to post-test and population sub-groups (sex, age, education, ethnic background).Conclusion: The results support the high level of interest in the Health Education Impact Questionnaire, particularly for use as a pre-test/post-test measure in experimental studies, other pre–post evaluation designs and system-level monitoring and evaluation