Interactive Query Expansion: A User-based Evaluation in a Relevance Feedback Environment

A user-centered investigation of interactive query ex-pansion within the context of a relevance feedback sys-tem is presented in this article. Data were collected from 25 searches using the INSPEC database. The data col-lection mechanisms included questionnaires, transac-tion logs, and relevance evaluations. The results discuss issues that relate to query expansion, retrieval effective-ness, the correspondence of the on-line-to-off-line rele-vance judgments, and the selection of terms for query expansion by users (interactive query expansion). The main conclusions drawn from the results of the study are that: (1) one-third of the terms presented to users in a list of candidate terms for query expansion was identified by the users as potentially useful for query expansion. (2) These terms were mainly judged as either variant ex-pressions (synonyms) or alternative (related) terms to the initial query terms. However, a substantial portion of the selected terms were identified as representing new ideas. (3) The relationships identified between the five best terms selected by the users for query expansion and the initial query terms were that: (a) 34 % of the query expansion terms have no relationship or other type of correspondence with a query term; (b) 66 % of the re-maining query expansion terms have a relationship to the query terms. These relationships were: narrower term (46%), broader term (3%), related term (17%). (4) The results provide evidence for the effectiveness of interactive query expansion. The initial search produced on average three highly relevant documents; the query expansion search produced on average nine further highly relevant documents. The conclusions highlight the need for more research on: interactive query expan-sion, the comparative evaluation of automatic vs. inter-active query expansion, the study of weighted Web-based or Web-accessible retrieval systems in opera-tional environments, and for user studies in searching ranked retrieval systems in general