Bullseye: Structured Passage Retrieval and Document Highlighting for Scholarly Search

We present the Bullseye system for scholarly search. Given a collection of research papers, Bullseye: 1) identifies rel-evant passages using any off-the-shelf algorithm; 2) auto-matically detects document structure and restricts retrieved passages to user-specified sections; and 3) highlights those passages for each PDF document retrieved. We evaluate Bullseye with regard to three aspects: system effectiveness, user effectiveness, and user effort. In a system-blind evalu-ation, users were asked to compare passage retrieval using Bullseye vs. a baseline which ignores document structure, in regard to four types of graded assessments. Results show modest improvement in system effectiveness while both user effectiveness and user effort show substantial improvement. Users also report very strong demand for passage highlight-ing in scholarly search across both systems considered