Text Classification with Evolving Label-sets

In this paper, we introduce the evolving label-set prob-lem encountered in building real-world text classification systems. This problem arises when a text classification sys-tem trained on a label-set encounters documents of unseen classes at deployment time. We design a Class-Detector module attached to text classification systems that moni-tors unlabeled data, detects new classes, and suggests them to the human administrator for deployment in the original label-set. A central notion in our algorithms is the use of abstrac-tions that group together tokens under human understand-able concepts and also provide a mechanism of assigning importance to unseen terms. We present algorithms for se-lecting documents for a new class based on state-of-the-art generative models and high performance discrimina-tive classifiers. Experiments on three real world taxonomies show that the generative models can select documents com-prising a new class with high precision, and also automat-ically trigger the emergence of new classes with more than 85 % precision. 1