Transfer Discriminant Softmax Regression with Weighted MMD

Abstract

As an efficient classical machine learning classifier, the Softmax regression uses cross-entropy as the loss function. Therefore, it has high accuracy in classification. However, when there is inconsistency between the distribution of training samples and test samples, the performance of traditional Softmax regression models will degrade. A transfer discriminant Softmax regression model called Transfer Discriminant Softmax Regression with Weighted MMD (TDS-WMMD) is proposed in this paper. With this method, the Weighted Maximum Mean Divergence (WMMD) is introduced into the objective function to reduce the marginal distribution and conditional distribution between domains both locally and globally, realizing the cross domain transfer of knowledge. In addition, to further improve the classification performance of the model, Linear Discriminant Analysis (LDA) is added to the label iteration refinement process to improve the class separability of the designed method by keeping the same kind of samples together and the different kinds of samples repeling each other. Finally, after conducting classification experiments on several commonly used public transfer learning datasets, the results verify that the designed method can enhance the knowledge transfer ability of the Softmax regression model, and deliver higher classification performance compared with other current transfer learning classifiers.