Dialogue Negotiating Measurement: Methodological and Interpersonal Considerations in the Choice and Interpretation of Instruments

Sound evaluation planning requires numerous decisions about how constructs in a program theory will be translated into measures and instruments that produce evaluation data. This article, the first in a dialogue exchange, examines how decisions about measurement are (and should be) made, especially in the context of small-scale local program settings. Rigorous measurement strategies will increase the credibility of a study’s conclusions, but they usually entail various kinds of costs. In mak-ing measurement decisions, evaluators must establish standards for strength of evidence that a given measure produces, weigh alternative measurement options, and communicate carefully with clients and other stakeholders about the measurement requirements in a given evaluation. Keywords evaluation methods, evaluation planning, measurement, impact evaluation Social programs are called upon to produce evidence of their impacts, to a wide variety of interested parties that may include funders, agency partners, target audience members, legislators, internal administrators, and many others. When a program evaluation results in claims that desired impacts have been achieved, stakeholders ’ acceptance of those claims will depend, in part, on their being confident that the evaluation study was well-planned, well-reasoned, well-executed, and focused on essential program outcome variables.1 Experienced evaluators can anticipate these eventual sta-keholder judgments early on, during the evaluation planning process, and address them through fea-tures of the evaluation design. In this article, I examine one aspect of that planning process: the measurement-related decisions that will have a bearing on the strength of the conclusions about program effectiveness and success