A Robust Quasi-Newton Training with Adaptive Momentum for Microwave Circuit Models in Neural Networks

Abstract

In this paper, we describe a robust technique based on the quasi-Newton method (QN) using an adaptive momentum term to train of neural networks. Microwave circuit models have strong nonlinearities and need a robust training algorithm for their neural network models. The robustness here means that practical solutions can be obtained regardless of the initial values. QN-based algorithms are commonly used for these purposes. Nesterov's accelerated quasi-Newton method (NAQ) proposed a way to accelerate of the QN using a fixed momentum coefficient. In this research, we verify the effectiveness of NAQ for microwave circuit modeling with high nonlinearities and propose a robust QN-based training algorithm with an adaptive momentum coefficient. The proposed algorithm is demonstrated through the modeling of a function and two microwave circuit modeling problems.