Evaluating Transformation - what can we learn from the literature?

The last decade’s rise of the so-called “third frame” (or third generation) of Transformative Innovation Policies (TIP) has shifted focus of research and innovation investments from economic growth and competitiveness to also tackling societal challenges and generating broader environmental and societal impact. The evolution in rationale and aims for policy action also implies a need to adapt and evolve evaluative strategies and practices. As policymakers begin to develop new transformative innovation programmes, a key question arises as to how monitoring, evaluation and learning practices (currently framed around 1st and 2nd generation innovation policies) can be adapted in order to meet 3rd generation innovation policy needs? In shaping a response, one can learn from both theory and practice. This brief (produced within the GReaTr initiative ) aims to provide a synthesis of what recent academic research tells us about evaluating transformation, leveraging a set of 11 seminal articles (and other complementary literature) to answer four questions: For whom and why? What to evaluate? How to evaluate? What unit of analysis? The synthesis points to a relative consensus in the academic literature on the main purposes and uses, the recommended principles and approaches, as well as possibilities for delineating and dealing with multiple scopes and units of analysis in evaluating transformation – yet highlights different conceptual framings of system change. The summary provides inputs to planning an evaluative strategy for TIP and highlights the need to consider new questions related to approaches to reporting across funding agencies, and more active roles for funding and policymaking agencies in dialogues about strategic direction and prioritisation of investments