Coding into the Great Unknown: Analyzing Instant Messaging Session Transcripts to Identify User Behaviors and Measure Quality of Service

After one year of providing virtual reference service through an instant messaging (IM) service, Binghamton University (BU) Libraries, under the purview of its Digital Reference Committee (DRC), undertook a study of collected session transcripts. The goals of this work were to determine who was using the IM service and why; if staffing for the service was adequate and met our in-person reference standards; and if improvements to the Libraries' existing reference services were needed. The findings revealed that 31% of identifiable users were students and 5% of users were campus community members. The analyses also revealed that many used the service for complex questions and not just ready reference, policy, and directional questions as had been expected. The most common question types were website navigation help (29% of all sessions), research assistance (22%), and instructional questions (23%). The American Library Association Reference & User Services Association (RUSA) Guidelines for the behavioral performance of reference and information service providers were used to measure quality of service. The findings reveled that approachability, showing interest, and listening were each demonstrated in over 80% of sessions, indicating these activities can be demonstrated effectively in a virtual environment. The study also found that questions were correctly answered 84% of the time. The study provided valuable insight into how patrons approach and locate information on our website and demonstrated a need for additional training and future discussions of staffing alternatives for the IM service