Comparing Apples and Bananas? A/B Testing for Discovery System Optimization

Discovery systems have changed the ways in which students are able to search academic library resources by providing a simplified, customizable user interface. Librarians often make discovery system customization decisions based on instincts about what will work best for students or on small usability studies. A/B testing, commonly used by commercial enterprise but infrequently by libraries, employs two simultaneous, live versions of a web interface to gauge the effects of changing variables. This method has the advantage of reaching a large number of users performing authentic search tasks. The authors combined A/B testing with scenario-based usability testing to explore variables such as facet labels, facet order, and placement of search options. The results provide insight into the extent to which interface changes prompt users to employ available search options, allowing libraries to make decisions about discovery system customization driven by user data