An Empirical, Evaluative Comparison of the Goal-Based and Goal-Free Approaches to Educational Program Evaluation

341 p.Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 1980.Theoretically, the use of different evaluation approaches yields different information. This information often leads to different conclusions, but it is not necessarily inaccurate. It is different information, sometimes relevant, sometimes irrelevant, depending on the audience. A comparative research study of evaluation approaches might provide data on which to base more reasoned decisions concerning which approach to select in a particular evaluation context.This study chose two outcome-oriented approaches to educational program evaluation (i.e., goal-based and goal-free) and implemented them concurrently on a short-term continuing education workshop for health care professionals. The purpose of the inquiry was to empirically compare in an evaluative mode the goal-based and goal-free approaches to evaluation.The methodology for the evaluative comparison was modeled after a process described by Taylor (1961). The process entailed developing standard dimensions, each of which was elaborated into operational statements. The standards and their associated statements represented criteria for high quality evaluation. The standards used in the study were adopted from Grotelueschen et al. (1974): purpose, audiences, issues, resources, evidence, data-gathering, analysis, and reporting.The study produced two types of data: (a) a narrative description or verbatim reproduction of what took place during the implementation of each evaluation approach related to each of the standard dimensions and its associated operational statements, and (b) a verbal and numerical rating indicating the degree to which each evaluation approach fulfilled or failed to fulfill the standard and its operational statements.The goal-based and goal-free approaches were found to differ substantively on virtually every dimension of comparison. Differences were found in: the types of value considered by each approach (goal-based, instrumental; goal-free, instrumental, contributive, and intrinsic), the role of the evaluator (goal-based, technician; goal-free, archeologist/anthropologist), the resources required to implement each (different expertise demanded of the evaluator, more time for goal-free, goal-free also required a person to act as a screen to filter goal/objective information from the evaluator), the types of evidence considered, the methods by which data were gathered and analyzed, as well as the manner in which the results were reported.The goal-based approach satisfied the requirements of the evaluation task more successfully than did the goal-free approach. This outcome was attributed to the internal consistency found in the goal-based approach with regard to the purpose, issues, evidence, data-gathering, analysis, and reporting of the evaluation, and the match between what transpired and what was needed. Such consistency and match was not found in the goal-free approach.The goal-free approach was found to excel in surfacing issues to consider in an evaluation and also in providing insights regarding perspectives which might be taken in viewing a program. The methodology used to compare the two approaches was found deficient in a number of areas. Suggestions were made to improve each shortcoming.The similarity between Stufflebeam's (1974) definition of meta-evaluation as a "procedure for describing an evaluation activity and judging it against a set of ideas concerning what constitutes good evaluation," and the current study's method was discussed.Finally, a number of uses of the method for future studies were suggested. These uses ranged from identifying the critical attributes of various evaluation approaches to utilizing the criteria/standards developed for the study as a prescriptive model for performing evaluations.U of I OnlyRestricted to the U of I community idenfinitely during batch ingest of legacy ETD