The development and implementation of an evaluation for rural ICT projects in developing countries: an exploration of the Siyakhulu Living Lab, South Africa

Rural development is a priority for poverty alleviation and development in developing countries, as the majority of the poor live in rural areas. Information and knowledge are key strategic resources for social and economic development as they empower rural communities with the ability to expand their choices through knowing what works best in their communities. Information and communication technologies (ICT) play a significant role in supporting rural development activities through providing supportive development information and creating essential interconnectivities between rural areas and more developed regions. However, rural ICT for development (ICT4D) is still at best a ‘working hypothesis’, faced with barriers and challenges associated with implementation and use in the rural environment; which threaten the success, sustainability or relevance of an ICT intervention. Many key questions remain largely unanswered, with no concrete or credible data to support a wide range of claims concerning the use of ICT for development. The evaluation of rural ICT projects is indispensable as it determines the need, effectiveness, impact, sustainability and extent of the awareness of the contribution such projects or programmes can make in poverty alleviation and development. Even so, existing ICT4D evaluations are confrontedwith shortcomings and challenges which influence the accuracy and reliability of evaluation conclusions. These shortcomings highlight the need to embark on a more comprehensive evaluation approach, sensitive to the rural environment. This research study was aimed at developing a comprehensive rural ICT evaluation framework to assess ICT projects and interventions that work toward supporting poverty eradication in rural communities. A multi-method approach was used to determine the multiple variables and components associated with rural ICT evaluation, and then to determine how these variables interrelate. The approach is founded on programme evaluation, ICT for development evaluation, and information systems evaluation. Firstly, key domains of programme evaluations combined with an exploration of the need and shortcomings of ICT4D evaluation, contributed to the development of a template to analyse existing ICT4D evaluation frameworks and information systems frameworks, based on a selection of criteria. The combined analysis of the two groups of frameworks compares and contrasts key characteristics that form the structure of a comprehensive evaluation. This analysis and a review of programme evaluation enabled the development of a Rural ICT Comprehensive Evaluation Framework (RICT-CEF) that encompasses the key components essential for a comprehensive evaluation of rural ICT projects. The theoretical framework aims to inform ICT intervention to improve and support rural development, through the application of fundamental and interconnected evaluation domains sensitive to the rural environment, throughout the project’s lifecycle. In order to obtain a better understanding and application of the RICT-CEF, a real-life case study investigation of the Siyakhula Living Lab reveals the lessons learned (shortcomings and suitability) from applying a prototype of the framework in a rural environment. The study is characteristically a rich case study, as the investigation occurs at two levels: 1) The actual evaluation of the project to obtain results to improve or guide the project, through applying domains of the RICT-CEF, and 2) Observing and investigating the application of the RICT-CEF framework to learn lessons from its evaluation process in a real-life context. The research study reveals the compatibility of the RICT-CEF framework in a real-life rural ICT intervention case, and builds lessons learned for enhancing the framework and guiding future evaluations in ICT4D. The RICT-CEF can possibly be viewed as a platform for the key domains and processes essential for the evaluation of ICT4D interventions; which can be customised for a variety of ICT projects, such that a comparative assessment of projects can provide measurement and further awareness of the impact of rural ICT in developing countries