An acceleration of proximal diagonal Newton method

Abstract

Recently, proximal Newton-type methods with metrics restricted to diagonal matrices have been proposed for solving composite optimization problems whose objective function is the sum of a smooth function and a possibly nonsmooth function. Although the effectiveness of one of them, the proximal diagonal Newton method (PDNM), has been reported theoretically and numerically, only 𝒪(1/k) sublinear convergence rate has been obtained at best for non-strongly convex problems. We propose an accelerated variant of the PDNM, which achieves the convergence rate of 𝒪(1/k²).