Evaluating an embedded program: Increasing awareness, expanding services, and fulfilling patron needs

Objectives: To provide a multifaceted, generalizable toolkit for evaluating the effectiveness of an embedded program. An embedded librarian program was assessed using a toolkit which includes traditional activity data and a survey aimed at measuring use of services, awareness of the embedded program, and the perceived value of embedded services. Methods: A national survey of librarians in the health sciences and a literature review identified few evaluation tools for embedded activities. This lack of resources led librarians to envision developing an evaluation toolkit to assess the success of their embedded program. The evaluation toolkit consists of locally collected data on multiple service transactions and a survey to measure patron awareness, usage, and perceptions of embedded services. Local data was provided by tracking patron interactions, teaching opportunities, and interprofessional collaborations in LibAnalytics. The survey included customized question sets for different patron groups and was reviewed for content and internal structure elements to increase its validity. Qualtrics software was used for survey distribution to students, residents, and faculty in April 2015. Survey results were analyzed to identify patterns of patron usage, awareness gaps, and desired services. Results: The survey response rate was 10% with 381 completed forms from 4,408 survey recipients. Over 58% of participants who completed the survey reported being aware of their embedded librarian. The majority of faculty strongly agreed that embedded librarians saved them time and were an integral part of their group. Analysis of locally collected data revealed that a high number of reference transactions occur in person which corresponds with survey results. Additional review of the data reflected an increasing trend toward librarian collaborations on grants, publications, and presentations. Conclusions: The evaluation results are guiding future planning for the embedded librarian program by indicating areas of strength and opportunities for growth. Embedded librarians were contacted primarily through in-person exchanges, demonstrating the strength of having a physical presence in colleges and departments. Lack of contact through online courseware shows an opportunity to expand into virtual areas. Literature searching was the primary use of embedded librarian assistance, illustrating a possible need for greater marketing of other services. The evaluation toolkit has helped provide the library a broad analysis of the program