Comparing User Approach to Complex Information Needs in Traditional Web Search and Conversational Search

Conversational search systems have recently gained popularity due to their effectiveness in performing small tasks and answering factoid questions. However, under complex search scenarios, these systems fail and fall back to techniques used by traditional search engines. As tasks requiring user learning are inherently complex, user knowledge is a powerful indicator of system performance under complex search and user ability to successfully interact with the information content. We present a study in which user knowledge gain and query formulation is explored in both the traditional web search and conversational search formats. Through crowd sourcing, 50 participants were recruited and conducted complex search sessions on either the traditional or conversational search medium. Through the use of a knowledge test given to the participant both before and after the search session, knowledge gain was determined. Furthermore, session queries and timestamps were tracked. It is found that participants have a significantly higher knowledge gain in traditional search, while conversational search sessions tended to have a higher number of queries and average query length.CSE3000 Research ProjectComputer Science and Engineerin