Feature Similarity: Geometrical Modeling and Discriminative Kernel Learning

Abstract

We propose geometrical models of features of a learning problem with the assumption that features are not independent. The key idea is to model feature similarity by means of non-orthogonality of a basic in the space. We show that there are two alternative ways to interpret similarities between features within kernel method framework. One follows a projection model while the other follows a reconstruction model. This shed a light on the relations among previous feature similarity methods. We also discuss the use of label information, which has been missing in previous works, for classification within the feature similarity learning step. It turns out that we have discriminative counterparts of previous feature similarity methods.