Query-Performance Prediction Using Minimal Relevance Feedback

There has been much work on devising query-performance prediction approaches that estimate search effectiveness with-out relevance judgments (i.e., zero feedback). Specifically, post-retrieval predictors analyze the result list of top-retrieved documents. Departing from the zero-feedback approach, in this paper we show that relevance feedback for even very few top ranked documents can be exploited to dramatically im-prove prediction quality. Specifically, applying state-of-the-art zero-feedback-based predictors to only a very few rele-vant documents, rather than to the entire result list as orig-inally designed, substantially improves prediction quality. This novel form of prediction is based on quantifying prop-erties of relevant documents that can attest to query perfor-mance. We also show that integrating prediction based on relevant documents with zero-feedback-based prediction is highly effective; specifically, with respect to utilizing state-of-the-art direct estimates of retrieval effectiveness when min-imal feedback is available