Factors Influencing Program Impact Evaluation in Cooperative Extension

Impact evaluation (IE) has become a major focus of Extension program evaluation during the past three decades, yet Extension professionals continue to struggle to produce well-documented evidence of program impacts (Lamm, 2011; Workman, 2010). This research was primarily a quantitative, non-experimental correlational study of factors associated with Extension educators? IE behaviors and practices in the North Central Region of the Cooperative Extension service. The study employed a comprehensive, systems approach to explore interrelated individual and organizational factors that affect IE in Extension. A major feature of this research design was the use Structural Equation Modeling (SEM) statistical analysis methodology. The use of SEM to organize and implement this study facilitated simultaneous exploration of many constructs theorized to be involved in IE behaviors in the context of Extension educational program evaluation. Specification of a theoretical, conceptual model to be used to frame the potential relationships among the many constructs and factors was necessary prior to data collection in order to be able to use SEM for data analysis. Data were analyzed using SEM path analysis to determine relationships among the factors. Significant findings included the identification of the most influential factors on the dependent variable of actual IE behaviors. These were: competency by perceived skill level, behavioral intention, number of roles in IE, education level, and attitude. Factors exhibiting noteworthy influence on factors other than the dependent variable within the model included training, culture, and proportion of teamwork. Qualitative data themes most frequently mentioned by participants included: the need to know how to measure change; a need for training in general; a need for better planning of programs to achieve impact; conflicting priorities, lack of time and timing/coordination concerns; changing expectations (?moving target?) regarding IE practices and goals; and diverseness in contexts, competencies, and schedules. Study findings identified a number of influential factors not previously cited in literature, including number of roles in IE; proportion of teamwork employed in IE; a lack of understanding of how to measure change; and issues of diverseness regarding competencies, context (including stakeholder expectations), and scheduling within the Extension organization