Orphans in Zambia: program monitoring and evaluation practices and the association of external support with education status and psychosocial wellbeing

Thesis (D.P.H.)--Boston UniversityProblem: The Government of the Republic of Zambia (GRZ) adopted a community-based strategy to support a growing number of orphans and other vulnerable children (OVC). However, the impact of community-based support programs remains unclear. This dissertation examines one OVC program to answer three questions: 1) are orphans disadvantaged compared to non-orphans in educational outcomes and psychosocial wellbeing, 2) are differences associated with receipt of external support, and 3) what can current programs learn from one project's monitoring and evaluation (M&E) experience? Methods: This study used mixed-methods. We administered quantitative household surveys to 204 households at the close of a community-based project and again one year later. Additionally, we conducted 4 focus group discussions (FGDs) with caregivers and 4 FGDs with adolescents. Finally, we conducted in depth interviews (IDIs) with 26 project staff and reviewed project documents. Bivariate and multivariate regressions were used to analyze the quantitative data. Grounded theory analysis was conducted on the FGD transcripts and content analysis was conducted on the IDIs. Results: Orphanhood was not a significant predictor of worse educational outcomes or psychosocial wellbeing. However, loss of external psychosocial support was associated with worse psychosocial outcomes as measured by the Strengths and Difficulties Questionnaire. Project staff had varied perceptions, priorities and capacity regarding 1) Quality of M&E Systems, 2) Project Evaluation, and 3) Data Analysis and Use, resulting in an M&E system that could not adequately capture complexities and measure success. Conclusions: In future programs, implementers should: 1) develop strong M&E systems that are responsive to donor mandates and inclusive of community-defined measures of success, 2) prioritize baseline capacity assessment of all partners and community needs assessment to inform program design, 3) consider alternative targeting strategies with less emphasis on orphan status, and 4) be cognizant of potential negative impacts on a child, particularly psychosocial wellbeing, from the removal of external support. Funders and policymakers should: 1) increase implementer accountability to project plans, 2) invest in integrating measures of quality into reporting frameworks, 3) generate an evidence-base by encouraging the development of strong program M&E systems 4) consider allowing implementers to select realistic targets that are responsive to community needs