Policy-Oriented Research Impact Assessment (PORIA) case study on the International Food Policy Research Institute (IFPRI) and the Mexican PROGRESA anti-poverty and human resource investment conditional cash transfer program

"The objective of this study is to attempt to characterize the influence and impact of IFPRI in relation to the Mexican PROGRESA/Oportunidades anti-poverty and human resource program with its conditional cash transfers (CCT)—conditional on specific investments in education, health, and nutrition. The paper first describes PROGRESA/Oportunidades and estimates of the impact and benefits-to-costs of this program; then discusses the challenges in assessing the influence and impact of IPRRI on and through PROGRESA/Oportunidades; and then presents the information sources used in this study to attempt to identify the influence and impact of IFPRI on PROGRESA/Oportunidades including interviews with 39 key informants as well as various published and unpublished studies and memos, publications in the popular media and on the internet and press releases and other documents. With this foundation it next explores the apparent influence and impact of IFPRI on PROGRESA/Oportunidades by considering four questions: Was the PROGRESA program design influenced by prior IFPRI research? Why was IFPRI chosen to undertake the initial impact evaluation of PROGRESA? How did the IFPRI evaluation of PROGRESA contribute to the program? Were there spillovers of the IFPRI evaluation of PROGRESA? The conclusions that come out of this investigation include that: (1) IFPRI did not directly participate in the PROGRESA design. So it would be misleading to claim that a major part of the PROGRESA program benefits should be attributed to IFPRI's critical role in the design process. Nevertheless prior IFPRI research and research by IFPRI evaluation team members apparently directly (in the sense of direct citations) and indirectly (in the sense of contributions to evolving conventional wisdom) on topics including food subsidy programs, nutrition, health, gender, intrahousehold allocations, and related policies played some role in the design process. Therefore, it might not be misleading to attribute a very small part of the PROGRESA benefits to better design based on IFPRI's prior research contributions on a number of related topics. (2) IFPRI presented advantages in terms of (i) type of organization (neither appearing to be a captive of national interests nor identified with structural adjustment or multilateral lending); (ii) reputation and credibility for solid scientific analysis of relevant policy-related issues enhanced by the addition of some prominent academic researchers; and (iii) aspects of management including flexibility, reliability, and relatively low costs. These were perceived to offset the disadvantages in the eyes of some Mexican entities of not being Mexican and the logistic and communication difficulties of not having a Mexican base. The advantages built, in part on IFPRI's prior contribution to the knowledge on which the designers of PROGRESA built (see Question 1) and underlay some of the benefits of IFPRI's involvement in the evaluation (Questions 3 and 4). The perceived package of advantages apparently dominated whatever PROGRESA considered the next best option, so early on in the conversation between IFPRI and PROGRESA about the evaluation, the focus seemed to be primarily on making mutual adjustments so that the evaluation contract could be worked out rather than strong debates about whether it should be worked out. (3) Key persons in the development and implementation of PROGRESA, as well as most of those interviewed, international organizations and the more general media seem to agree that the IFPRI evaluation team did make a significant contribution to the short-run and longer-run sustainability and expansion of the program as well as to details of the evaluation and program modifications—probably beyond what might have occurred with some alternative evaluator for reasons that were anticipated in the selection of IFPRI (see Question 2 above). While it undoubtedly remains the case that the basic success of PROGRESA is due primarily to those in Mexico who developed, nurtured, and implemented the program, it also seems plausible that some small share of the credit might be attributed to the IFPRI evaluation. (4) There have been substantial spillover effects, both in Mexico and internationally, of the evaluation of PROGRESA on the culture of policy evaluation in general and on CCT programs in particular. And it seems that some significant share of that is due to IFPRI being engaged in the evaluation because that involvement probably added considerably to both the Mexican domestic and the international awareness of and receptivity to what had been learned about evaluation processes and what had been learned about CCTs through the IFPRI evaluation of PROGRESA. Thus, the evidence summarized in this study suggests that IFPRI probably did have an important significant impact on the direct and indirect outcomes of PROGRESA/Oportunidades, even if this impact was a small percentage of the total impact of PROGRESA/Oportunidades. There apparently were significant gains from having an international organization undertake the initial PROGRESA evaluation that was not viewed as captive to national or international interests and that had a reputation of undertaking objective policy-related research that may have been enhanced by the involvement of several fairly visible academics. Simulations of the benefit-to-cost ratio for IFPRI's involvement, even under conservative assumptions, suggest that the benefits outweighed the costs substantially—and probably by much more with more moderate and more plausible assumptions." -- from Author's AbstractPRIFPRI1; IA; GRP28; Capacity StrengtheningDG