Developing a Practical Framework for Information Literacy Program Evaluation

This paper was presented at the Library Assessment Conference held in Arlington, VA October 31–November 2, 2016. Purpose. The Purdue University Libraries, like many academic libraries, face increased expectations for demonstrating their value and impact. This has not only led to an expectation of the increased use of metrics to demonstrate impact, but also a more fundamental imperative that libraries more clearly articulate their contributions to educational and research outcomes of their campus communities (value). At Purdue, the Provost implemented a new program review process in July 2015, while the Libraries were simultaneously going through the process of developing a new mission statement for its information literacy program. This statement was developed through a broad collaborative process within the libraries and with external campus stakeholders. These two developments led the Libraries to launch a project to advance an outcomes-based, mission-centric framework for evaluating its information literacy programing that can be sustained over time. The project to develop this framework was predicated on being able to answer the following question, derived from the program mission statement: “Does the Purdue University Libraries’ information literacy programming empower diverse learners to use information to learn in transformative ways; lead to the discovery of new knowledge; and foster academic, personal and professional success?” This question not only needs to be answered, but needs to be answered on an ongoing basis to communicate the programming impact to external stakeholders. To be effective, sustainable, and practical, it also needs to be uncomplicated and integrated into regular workflows. Methods. The methods for developing this framework consists of four steps: 1) focus groups with librarians to collaborate on gaining a more comprehensive understanding of existing assessment practices, as well as their perceptions of challenges and opportunities in assessing information literacy programs, 2) analysis of focus group findings, characterizing current assessment practices and identifying where outcomes-based assessment is already occurring, 3) a gap analysis, comparing focus group findings to the information literacy mission statement, and 4) development of recommendations with measures/indicators to address gaps and develop a comprehensive framework for program evaluation. This paper reports on the first three steps, concluding with suggestions for further development of the evolving framework. Findings. The assessment practices identified in the analysis of the focus group discussions suggested that librarians assessed how students critically used information to learn more than the other dimensions of the Purdue Libraries’ information literacy mission statement: research-based programming, empowerment of diverse learners, enabling the creation of new knowledge, and fostering academic, personal, and professional success. The findings suggest next steps in the development of the framework, including: 1) developing guidelines for collecting assessment data gathered by librarians for use in programmatic assessment, 2) determining assessment strategies for the Libraries and allocate resources, and 3) providing professional development and incentives for librarians to create assessment strategies related to all aspects the mission statement. Practical Implications/Value. We expect that the results of this project will contribute to the body of knowledge in library assessment by presenting a framework for the outcomes-based evaluation of information literacy program evaluation that is based on a strategic perspective on the program, but that also builds upon existing practices and capacity within the organization