Beyond comparing image pairs: Setwise active learning for relative attributes

It is useful to automatically compare images based on their visual properties—to predict which image is brighter, more feminine, more blurry, etc. However, comparative models are inherently more costly to train than their classi-fication counterparts. Manually labeling all pairwise com-parisons is intractable, so which pairs should a human su-pervisor compare? We explore active learning strategies for training relative attribute ranking functions, with the goal of requesting human comparisons only where they are most informative. We introduce a novel criterion that requests a partial ordering for a set of examples that minimizes the to-tal rank margin in attribute space, subject to a visual diver-sity constraint. The setwise criterion helps amortize effort by identifying mutually informative comparisons, and the diversity requirement safeguards against requests a human viewer will find ambiguous. We develop an efficient strategy to search for sets that meet this criterion. On three challeng-ing datasets and experiments with “live ” online annotators, the proposed method outperforms both traditional passive learning as well as existing active rank learning methods. 1