Approaches to the measurement of outcomes of chronic disease self-management interventions using a self-report inventory

Background Health education programs that are aimed at improving individuals' skills to self-manage are increasingly recognised as a critical component of chronic disease management. Despite the apparent need for such interventions, current studies show inconsistent results regarding program effectiveness, with meta-analyses indicating only marginal effects for some disease groups. A closer examination of these studies however suggests that the magnitude and inconsistency of the findings may be related to the types of outcomes that were assessed rather than specific disease groups. Where self-report measures were used, results tended to be smaller and inconsistent. It is therefore possible that current studies do not adequately reflect program effects because self-report outcomes have a high risk to be confounded by a range of potential biases. Objective The aim of this thesis was to identify and quantify the potential influence of biases in the measurement of change in chronic disease self-management interventions using self-report. Methods The research design targeted the processes that individuals undergo when filling out questionnaires and whether this has an influence on their self-report outcomes. This was achieved by developing a three-group research design. The Health Education Impact Questionnaire (heiQ) was used to collect outcomes data. While pretest questionnaires were identical across groups, three questionnaire versions were randomly distributed at posttest. One of the groups filled out traditional posttest questions (n=331), whereas the other two groups were asked to provide data in addition to posttest questions, with one group providing transition questions (n=304) and one providing retrospective pretest data (n=314). Resulting datasets were further examined for possible confounding effects through response shift and social desirability bias. Through the random allocation of the heiQs it was ensured that data were not influenced by potential intra-group effects. Results The thesis revealed that the design of the posttest questionnaire significantly influenced people's ratings of their posttest levels. In particular, when participants were asked to provide ratings of their retrospective pretest levels in addition to their posttest levels, the latter scores were significantly higher than those of participants who did not perform this additional task. Subsequent analyses however suggested that these differences could neither be explained by response shift nor by social desirability bias. Conclusions This research has provided important insight into the measurement of outcomes of chronic disease self-management interventions. While the threat to the validity of traditional pretest-posttest data due to confounding effects through response shift and social desirability biases could not be supported, the thesis has highlighted that the cognitive task that subjects are asked to perform when providing data at posttest significantly influenced their self-reported outcomes. Given that previous research has predominantly focused on other aspects of validity - such as applying control group designs to circumvent common threats to internal and external validity - this study suggests that more attention must be paid to the design of questionnaires. The thesis concludes that further research, in particular into the influence of cognitive tasks on obtained scores, is important to improve the interpretation of self-report outcomes data derived from participants of self-management interventions