Intelligent Neural Network Design for Nonlinear Control using Simultaneous Perturbation Stochastic Approximation ( SPSA ) Optimization

Simultaneous Perturbation Stochastic Approximation (SPSA) Optimization Adrienn Dineva*, Annamária R. Várkonyi-Kóczy**, József K. Tar*** and Vincenzo Piuri**** *Doctoral School of Applied Informatics and Applied Mathematics, Óbuda University, Budapest, Hungary *Doctoral School of Computer Science, Universita’ degli Studi di Milano, Crema, Italy **Institute of Mechatronics & Vehicle Engineering, Óbuda University, Budapest Hungary ** Department of Mathematics and Informatics, J. Selye University, Komarno, Slovakia ***Institute of Applied Mathematics, Óbuda University, Budapest, Hungary **** Department of Computer Science, Universita’ degli Studi di Milano, Crema, Italy E-mail: * dineva.adrienn@bgk.uni-obuda.hu, ** varkonyi-koczy@uni-obuda.hu, ***tar.jozsef@nik.uni-obuda.hu, ****vincenzo.piuri@unimi.it


Introduction
Neural Networks are widely applied in various fields of intelligent systems.For instance in nonlinear control tasks which contain strong non-linearities, characterized by high dimensionalities, etc., complicated mathematical functions can be well approximated with several types of neural networks.Their main capability lies in learning a nonlinear model without a priori knowledge of the parameters and structure of the model.These problems are usually computationally complex and often evolutionary programming techniques are required.It is known, that the Simultaneous Perturbation Stochastic Approximation (SPSA) techniques serve as an alternative solution for a variety of optimization problems [1] that ensure satisfying performance with less computation time even in the case of high-dimensional parameter tuning and also well suited to the NN's learning problem.Further, these methods can be especially efficient tools for multi-agent based optimization problems (see, for e.g.[2]).However, in the literature there are few examples of the application of SPSA in nonlinear control [3][4].For instance, recent studies address a wide range of possible application of NN-based controllers for robot manipulators highlighting its advantages in inverse kinematics problems [5].It has been also shown that neural networks trained by the simultaneous perturbation stochastic approximation (SPSA) method guarantee closed-loop stability of the estimation in the control problem considered in [6].This paper addresses the performance analysis of the SPSA-based neural networks in the adaptive control of the TORA (Translational Oscillator with a Rotational Proof Mass Actuator) system [7] [8].Simulation results demonstrate the potential of the proposed method.

SPSA Techniques for Training Neural Networks
Large amount of work on the estimation of weights in neural networks has been carried out.The SPSA-based techniques are found to be efficient in the minimization of the error criterion regardless of the dimension of the optimization problem and require only the measurement of the objective function.In contrast to the standard training algorithms both the first-order stochastic approximation and the second-order SPSA is found to be satisfactory [1] [3].

The SPSA Algorithm
In this section we give a brief outline on the mathematical background on the Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm which was devised by Spall in [1].Let's consider the following mean square error minimizing problem Where y is the target value and y ˆ is the actual output of the neural network according to parameters w.For the first-order simultaneous perturbation stochastic approximation (commonly denoted as 1SPSA) algorithm the parameter update law is given by [1] ) (  is a scalar gain coefficient which should satisfy certain conditions [1] and ) ( ˆ1 j w g denotes the approximation of the gradient that can be calculated as follows in which the elements of ( 1 j ˆ w ) are varied simultaneously, j  is a random perturbation vector generated independently while cj is a positive scalar number.Based on this recursion, the stochastic gradient algorithm can be obtained by applying a smoothed gradient approximation as The second-order or 2SPSA algorithm is based on two-measurements defined by the formulas below the evaluation (or measurement) of the objective functions independent of the number of unknown parameters.Further variations of the core theory exist, such as the global (GSPSA) and the adaptive (ASPSA) techniques are also well established.

Performance Evaluation
In order to demonstrate the feasibility and performance of the application of the neural network trained by SPSAs in adaptive controllers a simulation study has been carried out.The Translational Oscillator with a Rotational Proof Mass Actuator (TORA) system has been investigated as a nonlinear benchmark problem [7] in its modified fully driven form.There have been a growing number of literature focusing on the control design of underactuated systems [8] [9].In [10] an adaptive backstepping control strategy with online approximation of uncertainties has been introduced for a class of underactuated systems with functional uncertainties.The model contains a cart restricted to horizontal movement (translational displacement q3[m]) with Q3 force.A pendulum is attached to the device with a rotary joint with angular position q1[rad] є [-π/3;π/3]) Q1[Nm] torque, and at the end of it a rotatable dial can be found (q2 [rad]) with the driving torque Q2[Nm].In the underactuated model Q3=0 is assumed.For q3 and q2 a reference signal is defined with the Q1 and Q2 control variables.According to [7], the equation of motion is given by For adaptive control strategy the SGFPT (Sigmoid Generated Fixed Point Transformation) been applied, that is found to be highly efficient in the control of the underactuated TORA system discussed in a previous work of the authors [11].The applied method is based on a special fixed point transformation (see, for e.g.[12]) and assumes the existence of the approximate model of the system.Detailed discussion of this alternative method which has been originally introduced as an alternative of Lyapunov's method in adaptive control of nonlinear systems can be found in, for e.g.[12][13] [14].When failures or critical events occur in the system it can become difficult to compute or access the necessary values thus an application of a NN can be useful for predicting the data in order to ensure continuous operation.In the present sequel we investigate the applicability of a neural network trained with SPSA technique in this task.For approximating the nonlinear functions in the control algorithm a feedforward time-delayed neural network has been applied using the model expressed as follows are the measured or known values for the training database for the NN approximation.The training set consists of three input sequence of 2000 data and also the Q1 and Q2 appropriate torque data of 1000 values for target class obtained by numerical simulations [11].The network has been trained off-line with 1SPSA as well as 2SPSA and for comparison a conventional Levenberg-Marquardt (LM) algorithm.Twenty percent of the database has been reserved for validation while 15 % for test set.For a comparative assessment, the applicability of NNs with different configurations in the controller has been tested.The simulations have been implemented in Matlab7.

Simulation Results
The exact model parameters were set in m=20   Results collected in Table 1.exemplify that SPSAs can provide improved training performance over the standard methods.The tracking performance of some configuration is illustrated in Fig. 1. while the tracking error is depicted in Fig. 2. It can be observed that in the here considered task the application of NN trained with 2SPSA and with 20 hidden neurons serves the best trade-off between computational expense and performance.

Conclusions
This paper has given an account of intelligent neural network design using SPSA techniques for nonlinear control.It has been shown by the comparative analysis, that the proposed approach leads to a fast and simpler design methodology in case of different NN configurations.The proposed scheme is applied in the adaptive control of the TORA system.Results of numerical simulations validate that the application of the neural network trained with SPSA technique in adaptive control ensures satisfactory performance with less computation time need and appropriately suites to the RFPT-based control strategy.The results also highlight that the 2SPSA provides the best trade-off between performance and computation need in contrast to the other training methods.With the spacious examination of different analyze our results prove to be promising.

Table I .
Simulation Results