An exploratory study of the factors influencing the relationship between program evaluators and their youth grantee clients

This study was designed to explore factors influencing the evaluator-client relationship in recent evaluations of programs serving transition age youth (ages 14-25) from the perspective of professional evaluators working with youth grantees. Study data were collected by a researcher-developed electronic survey of 396 members of the American Evaluation Association who self-identified as providing evaluation services to youth programs serving the targeted age group. Both quantitative and qualitative were collected. Descriptive data were collected on the youth organization, the evaluator or evaluation consultant, and factors that were identified in the evaluation literature as characteristics, challenges, or threats influencing evaluation implementation or validity. The study found that survey respondents worked with a variety of organizations providing youth services and had multiple roles as evaluators. Most respondents reported some level of staff or stakeholder anxiety related to the evaluation. Outcome/impact evaluation was the most common type of evaluation. A majority of respondents reported evaluation challenges related to inadequate or problematic data; lack of staff resources such as time or funding; lack of staff cooperation, collaboration, or communication; rigid organizational procedures or settings; limited methodological capacity of staff; or threats to validity related to mutating interventions and outside forces. The majority of respondents also reported evidence of successful evaluation implementation and usefulness. Qualitative responses linked identified challenges to the evaluator-client relationship. Resources used by evaluators spanned a wide range of content and delivery methods. Examination of the data generated a number of questions regarding the evaluator-client relationship which, when investigated, will be instrumental in filling gaps in the literature on this important relationship