Sequential Decision Making Problem with Uncertainty and Failure Probability

Abstract

The stochastic shortest path problem (SSP) is a standard model for sequential decision making under uncertain environments. In addition, a method was proposed to explicitly handle failure scenarios by introducing dead-ends[4]. On the other hand, due to modeling errors with the real environment, the policy constructed for the model may not achieve the desired performance. In this paper, we derive a policy that works well by keeping the failure probability within a desired range even when the model has uncertainty. This problem is formulated as a partially observable stochastic dynamic game (POSDG) by introducing a label that indicate either the objective function or the constraints. Numerical examples are used to demonstrate the effectiveness of the obtained optimal policy and the property that action selection depends not only on the initial state and current state but also on the history.