The Open Loop Feedback Optimal Control for Unknown Linear Systems with Finite Parameter Set

Abstract

The optimal control problem of linear stochastic systems with unknown parameters is essentially nonlinear, so that an explicit solution of dynamic programming is not available. Thus, various suboptimal approaches have been suggested in treating this class of problems.
In this paper, a suboptimal algorithm based on the use of the open-loop-feedback-optimal (OLFO) method is derived on the assumption that unknown parameters take finite values. It is shown that the resulting control system consists of
1) an optimal state-parameter estimator formed by the parallel operation of Kalman filters;
2) an, OLFO adaptive feedback gain which is computed on-line based on the a posteriori probabilities of unknown parameters.
The OLFO algorithm is compared with existing algorithms by using the Monte Carlo method. The results are summarized as follows.
1) In the average performance, the OLFO control is a little superior to others for stable systems but inferior for unstable systems.
2) For the both stable and unstable systems, the standard deviation of the performance of the OLFO control is smaller than that of others.
3) The convergence rate of the a posteriori probabilities of unknown parameters is primarily dependent on the stability of the system rather than the choice of the algorithm. For unstable systems, the convergence rate is faster than that of stable systems.