Improving the Performance of the Decision Boundary Making Algorithm via Outlier Detection

Abstract

Outlier detection is one of the methods for improving the performance of machine learning models. Since outliers often affect the performance of the learning models negatively, it is desired to detect and remove outliers before model construction. In this paper, we try to improve the performance of the decision boundary making (DBM) algorithm via outlier detection. DBM has been proposed by us for inducing compact and high performance learning models that are suitable for implementation in portable computing devices. The basic idea of DBM is to generate data that can fit the decision boundary (DB) of a high performance model, and then induce a compact model based on the generated data. In our study, a support vector machine (SVM) is used as the high performance model, and a single hidden layer multilayer perceptron (MLP) is used as the compact model. Experimental results obtained so far show that DBM performs well in many cases, but its performance still is not good enough for some applications. In this paper, we use SVM not only for obtaining the DB, but also for detecting the outliers, so that better MLP can be induced using cleaner data. We use a threshold δ_outlier to control the number of outliers to remove. Experimental results show that, if we select δ_outlier properly, the DBM incorporated with outlier detection outperforms the original DBM, and it is better than or comparable to SVM for all databases used in the experiments.