A study on reporting and learning in three natural resource management programmes in South Africa

This is a case study that focuses on three Natural Resource Management programmes, namely: the Department of Environmental Affair’s Working for Water Programme, the Kruger to Canyon SANParks Biodiversity Social Projects as well as AWARD’s programme for Resilience in the Limpopo-Olifants Basin. All three programmes are being implemented in the Limpopo and Mpumalanga Provinces of South Africa. As a monitoring and evaluation officer at AWARD, I was part of the team that developed a strategy to enable learning through monitoring, evaluation and reporting, and this sparked my interest in understanding how other programmes might be approaching this. The focus of the study is on the role of reporting, as part of programme monitoring and evaluation, as a means to support learning of both programme implementers and organisations. An in-depth description of each case has been provided to enable an understanding of the overall reporting system, how reporting may play a role in supporting individual and organisational learning, and what factors enable or constrain learning in association with reporting. The research was descriptive and interpretive as the first phase of a possible extended study engaging organisations in exploring how to strengthen reporting for learning. The participants and organisations were sampled using both purposive and convenience sampling. Data was collected through 17 key informant interviews; observations during two learning events; and, the collection and review of numerous documents (guides, reports, templates etc.). The data was analysed through qualitative content analysis using two frameworks: Argyris’ three loops of learning (1974;1978) and the Cultural-Historical Activity Theory (CHAT) lens (Engeström, 1987). The former was useful for bounding the types of learning to identify, and the latter for surfacing contradictions that can be explored in organisational development processes. Through the analysis of the data and literature, the following insights were gained: All three programmes have been ‘thinking’ about learning, but only in the case of AWARD was reporting associated with learning; in all three programmes, additional spaces have been designed to facilitate learning. Although programmes might not necessarily focus on reporting for learning, their reports do indicate a potential for facilitating reflection and thus learning. The organisational culture in government-based reporting systems is, however, strongly geared to performance management, accounting for resources spent, meeting targets and eliminating mistakes. Reports are used to guide planning, to account to the funder and to release further funding.. In this context only single and double loop learning is in evidence. Programme participants did raise ‘triple loop’ questions about whether the programme or system overall was working, but there did not seem to be space for such questions, in the reporting or the learning events, and there was no evidence that they were addressed. In the case of AWARD, spaces were created through reporting to encourage learning. This worked sometimes to enable learning but not always due to, for example, time constraints and competing responsibilities. There are certain aspects to consider when designing reporting that seem to support learning (see below). The process of using reporting for learning takes time and organisations might not get this right the first time around. Reflection opportunities are only experienced as enabling if participants have the space and power to act on the reflections, or see others acting on them to address issues, otherwise they get frustrated. Steps that could assist programmes in improving reporting to enable learning are: Adding to reporting templates a space for reflections to ensure that while the programme reports on the quantitative data needed for accountability purposes, it also creates additional space (without unduly increasing reporting responsibilities) for implementers’ learning. Encouraging discussion about ‘mistakes’ such as failures to meet targets, why they occur, and what we can learn from them. Allowing programme implementers to give feedback on their reporting challenges and take measures to support them, e.g. make adjustments to templates. Understanding the programme’s definition of learning, and the diverse learning needs that may be involved, which can contribute to a working strategy that the organisation can build on when initiating learning spaces. When learning events are organised, taking the inputs of participants seriously and developing ways in which these could be taken further into actions from which further learning can stem; reflecting on and raising issues without an opportunity for action is frustrating. The process of enabling reporting for learning takes time and organisations might not get it right the first time around (trial and error) and a strategy that worked for one organisation might not necessarily work for the next. When designing a learning strategy as part of a monitoring, evaluation and reporting framework, it is therefore important to consider the needs of the organisation and to work with implementers to develop and refine this over time. The thesis concludes with comments on the limitations of the study, the methodology and the analytical frameworks used, and finally, provides recommendations for further research