Multi-conditional learning: Generative/discriminative training for clustering and classification

This paper presents multi-conditional learning (MCL), a training criterion based on a product of multiple conditional likelihoods. When combining the traditional conditional probability of “label given input ” with a generative proba-bility of “input given label ” the later acts as a surprisingly ef-fective regularizer. When applied to models with latent vari-ables, MCL combines the structure-discovery capabilities of generative topic models, such as latent Dirichlet allocation and the exponential family harmonium, with the accuracy and robustness of discriminative classifiers, such as logistic re-gression and conditional random fields. We present results on several standard text data sets showing significant reductions in classification error due to MCL regularization, and sub-stantial gains in precision and recall due to the latent structure discovered under MCL