Semantic Interpretation of SVM Nonlinear Discriminators

Abstract

We are going to propose a framework for extracting informative or redundant features associated with the nonlinear discriminators generated by support vector machine. We use the idea of decision boundary analysis introduced by Lee and Landgrebe. The quality of this analysis depends on the calculations of the points, or the associated gradient vectors, which are lying on the decision boundary. Exploiting the special structure of the kernel induced feature space, we show that the gradients on the boundary are easily obtained. Numerical experiments for some artificial datasets demonstrate that both informative and redundant features can be identified clearly. Also, we show that the performance of the discriminator can be improved by discarding the redundant features of a real world dataset.