Training a naive Bayes classifier via the EM algorithm with a class distribution constraint

Combining a naive Bayes classifier with the EM algorithm is one of the promising ap-proaches for making use of unlabeled data for disambiguation tasks when using local con-text features including word sense disambigua-tion and spelling correction. However, the use of unlabeled data via the basic EM algorithm often causes disastrous performance degrada-tion instead of improving classification perfor-mance, resulting in poor classification perfor-mance on average. In this study, we introduce a class distribution constraint into the iteration process of the EM algorithm. This constraint keeps the class distribution of unlabeled data consistent with the class distribution estimated from labeled data, preventing the EM algorithm from converging into an undesirable state. Ex-perimental results from using 26 confusion sets and a large amount of unlabeled data show that our proposed method for using unlabeled data considerably improves classification per-formance when the amount of labeled data is small.