Local Fisher Discriminant Analysis for Dimensionality Reduction

Abstract

Dimensionality reduction is one of the important preprocessing steps in high-dimensional data analysis. In this paper, we consider the supervised dimensionality reduction problem, i.e., samples are accompanied with class labels. Fisher discriminant analysis (FDA) is a traditional but powerful technique for linear supervised dimensionality reduction. However, FDA tends to give undesired results if samples in a class are multimodal. Locality-preserving projection (LPP) allows us to reduce the dimensionality of multimodal data without losing the local structure. However, LPP is an unsupervised method and is not necessarily effective in supervised learning scenarios. In this paper, we propose a new linear supervised dimensionality reduction method called local Fisher discriminant analysis (LFDA). LFDA effectively combines the ideas of FDA and LPP and works well for dimensionality reduction of multimodal labeled data. LFDA has an analytic form of the embedding transformation and the solution can be easily computed just by solving a generalized eigenvalue problem. This is an advantage over recently proposed supervised dimensionality reduction methods. We demonstrate the practical usefulness and high scalability of the LFDA method in data visualization and classification tasks through extensive simulation studies. We also show that LFDA can be extended to non-linear dimensionality reduction scenarios by applying the kernel trick.