Implications of the pre/post/then design for evaluating social group work

Pre/post designs that rely on self-report measures are potentially contaminated by response shift bias, a change in respondents ’ understanding of the phenomena being measured between the pre- and posttests. Retrospective pretests, or then tests, have been shown to be useful in identifying response shifts. Pre/post/then designs are highly utilitarian designs that have the potential of providing more accurate assessments of respondent change. This article reviews the literature on response shift bias and the utility of retrospective pretests m detecting such bias. Three studies that used the pre/post/then design to evaluate self-reported changes in caseworker abilities (knowledge or skill) relative to practice evaluation, basic interviewing skills, and/or assessment are described. The most typical design used in analyzing the effectiveness of training and intervention groups is the pre/post design, which obtains information from participants before and after the training or intervention and assumes that changes in the two measures reflect the impact of the training or intervention. The more sophisticated, and fortunate, researcher may add a control group that also provides pre/post data on the issue of concern (Campbell & Stanley