Approximate Solution of an LQ Problem with Random Parameter

Abstract

This paper deals with an LQ problem in which the plant and/or the cost functional contains a finite Markov chain as a random parameter. Although the existence of the optimal solution has not been completely cleared, this problem has been almost solved under the assumption that the value of the parameter can be measured as well as the state variables of the plant. The optimal control signal is generated by a linear state feedback law with a gain matrix dependent on the parameter. In this paper, we are interested in the case where the parameter has a nominal value and stays at the other values for very short time in average. We prove the existence of the optimal solution, and consider the property of an approximate solution constructed from the solution of the matrix Riccati equation with the nominal value of the parameter. For the particular problem in which all the coefficients on the control variables are constant, the approximate solution consists with the optimal solution of the usual LQ problem with the parameter fixed to the nominal value. It is shown that the loss caused by the approximation vanishes faster than the average staying time of the parameter at the non-nominal values as the latter is made to approach zero.