Abstract
Semi-supervised classifier design that simultaneously utilizes both a small number of labeled samples and a large number of unlabeled samples is a major research issue in machine learning. Existing semi-supervised learning methods for probabilistic classifiers belong to either generative or discriminative approaches. This paper focuses on a semi-supervised probabilistic classifier design for multiclass and single-labeled classification problems and first presents a hybrid approach to take advantage of the generative and discriminative approaches. Our formulation considers a generative model trained on labeled samples and a newly introduced bias correction model, whose belongs to the same model family as the generative model, but whose parameters are different from the generative model. A hybrid classifier is constructed by combining both the generative and bias correction models based on the maximum entropy principle, where the combination weights of these models are determined so that the class labels of labeled samples are as correctly predicted as possible. We apply the hybrid approach to text classification problems by employing naive Bayes as the generative and bias correction models. In our experimental results on three English and one Japanese text data sets, we confirmed that the hybrid classifier significantly outperformed conventional probabilistic generative and discriminative classifiers when the classification performance of the generative classifier was comparable to the discriminative classifier.
