Contingency questioned. Understanding the dynamics of policy evaluation practices in public sector organisations

Policy evaluation practice, as an applied policy analysis tool, has in general gained considerable terrain in the public arena, in Europe and abroad. Notwithstanding this trend, it does not seem appropriate to talk in terms of “a” European evaluation tradition, or “a” European evaluation culture. Mapping different countries’ evaluation traditions, would reveal an ‘evaluation tree with different trunks and branches’. Both the trajectories (‘roots’) of its development, as well as its resulting evaluation discourse, practices and institutions, largely differ throughout Europe. In explaining the similarities/differences in evaluation traditions an almost general consensus exists on the importance of societal, legal, institutional and historical contexts. Yet, systematic research evidence on this phenomenon is generally lacking. The only certainty remaining is the apparent highly contingent nature of evaluation practices. Yet, its inherent contingency might not discourage more systematic research to understand “why evaluation practices look like they do in the public sector”, and to get more grips on the role of various contextual factors herein. Remarkably, this is an area in which systematic research about evaluation is almost non-existent. Moreover, it is striking to notice that the majority of available analytical schemes concentrate at a single level of analysis (biased towards the country level) without taking into account possible interaction with other levels (e.g. sectoral; organisational). Conducting systematic multilevel research in search of possible combinations of contextual factors clarifying evaluation practices would be of high relevance. Not only for mere theoretical purposes, but also to indicate possible levers for evaluation capacity building. The paper presents a (PhD) research design, set up to tackle (part of) this research agenda. It approaches the general label ‘evaluation practices’ from two angles: first, from a developmental perspective. The ambition here is to explain the maturity of the evaluation capacity of a public sector organisation on a “plan-do-check-act” management scale. Secondly, evaluation practices are studied from an evaluation cycle outlook. We focus on explaining the loci of evaluation initiation, implementation and the number of evaluations undertaken by public sector organisations. Variables are inductively generated and ordered by an actor centered institutionalist framework into actor; institution; policy issue related conditions and evaluation policy legacies. 32 organisations of the Flemish administration constitute the area of investigation. In accordance with the contextual nature of evaluation practices and the theoretical framework applied, the methodology considered most suitable for the analysis is quali-quantitative comparative analysis (QCA). This involves a particular understanding of causality, known as “multiple conjunctural causation”.status: publishe