Deriving efficient optimization methods based on stable explicit numerical methods

Abstract

Some continuous optimization methods can be regarded as numerical methods applied to ordinary differential equations, and attempts to derive and analyze optimization methods from this perspective have been made for quite a long time. In this study, along this line of research we point out a possibly overlooked simple fact that by carefully observing the relationship between the convergence of optimization methods and the stability of numerical methods, we can construct an efficient optimization method based on certain numerical methods with extremely wide stability domain.