Designing Category-Level Attributes for Discriminative Visual Recognition∗

Attribute-based representation has shown great promis-es for visual recognition due to its intuitive interpretation and cross-category generalization property. However, hu-man efforts are usually involved in the attribute designing process, making the representation costly to obtain. In this paper, we propose a novel formulation to automatically de-sign discriminative “category-level attributes”, which can be efficiently encoded by a compact category-attribute ma-trix. The formulation allows us to achieve intuitive and crit-ical design criteria (category-separability, learnability) in a principled way. The designed attributes can be used for tasks of cross-category knowledge transfer, achieving su-perior performance over well-known attribute dataset An-imals with Attributes (AwA) and a large-scale ILSVRC2010 dataset (1.2M images). This approach also leads to state-of-the-art performance on the zero-shot learning task on AwA. 1