Activation functions of artificial-neural-network-based nonlinear equalizers for optical nonlinearity compensation

Abstract

We investigated the performance of artificial neural network (ANN)-based nonlinear equalizers for optical nonlinearity compensation by comparing activation functions, including a sigmoid function, ReLU, and Leaky ReLU. We compared the learning speeds and compensation performances by evaluating the resulting error vector magnitudes of the compensated signals. The performance was investigated using simulated 100-km optical fiber transmission of 10-GSymbol/s 16QAM signals. When the number of hidden-layer units in the ANN was small, the sigmoid function showed better performance in learning speed than ReLU and Leaky ReLU. This point is important because the number of ANN units has to be reduced in order to improve the computational complexity of the ANN-based nonlinear equalizer.