Multi-Label Active Learning: Query Type Matters

Active learning reduces the labeling cost by selec-tively querying the most valuable information from the annotator. It is essentially important for multi-label learning, where the labeling cost is rather high because each object may be associated with mul-tiple labels. Existing multi-label active learning (MLAL) research mainly focuses on the task of se-lecting instances to be queried. In this paper, we disclose for the first time that the query type, which decides what information to query for the selected instance, is more important. Based on this obser-vation, we propose a novel MLAL framework to query the relevance ordering of label pairs, which gets richer information from each query and re-quires less expertise of the annotator. By incorpo-rating a simple selection strategy and a label rank-ing model into our framework, the proposed ap-proach can reduce the labeling effort of annota-tors significantly. Experiments on 20 benchmark datasets and a manually labeled real data validate that our approach not only achieves superior perfor-mance on classification, but also provides accurate ranking for relevant labels.