Parallel parameter optimization in Support Vector Regression by Particle Swarm Optimization

Abstract

A parallel parameter optimization for Support Vector Regression (SVR) is proposed. A central of this technique is the use of population-based search, called Particle Swarm Optimization (PSO), for the simultaneous optimization of multiple parameters in SVR modeling. PSO is a biologically inspired search algorithm motivated by a social analogy. It is an iterative method based on the search behavior of a swarm of particles "flying" through a multidimensional search space. The particles have two biological properties as follows; they can remember the position (i.e. candidate for the solution) they have ever searched and can also share the information related to the solution to be solved and, consequently, these biological properties make it possible to reach global solution of the problem. In the present study, optimization of SVR parameter such as loss function and parameter of Gaussian kernel function is demonstrated with a QSAR data set. An optimization of robust kernel based on Huber's M-estimator is also shown.