Learning Theory of Multiple Kernel Learning

Abstract

We review convergence rate analyses about multiple kernel leaning (MKL) by regularization methods and a Bayes method. As for regularization methods, we show convergence rates for ℓ₁ and elastic-net regularizations. The elastic-net regularization is a composition of an ℓ₁-regularizer for inducing the sparsity and an ℓ₂-regularizer for controlling the smoothness. We focus on a sparse setting where the total number of kernels is large but the number of non-zero components of the ground truth is relatively small, and show sharper convergence rates than the learning rates ever shown for both ℓ₁ and elastic-net regularizations. Moreover, we show that, using a Bayesian method with Gaussian process priors, we don't need a strong conditions on the design to achieve a fast learning rate.