k-確実探査法とFuzzy-ARTを用いた連続状態環境のセミマルコフ決定過程モデル構築手法の提案

抄録

This paper describes a new exploration and modeling method for an unknown continuous state space environment. The purpose of our study is to divide a continuous state space and to construct a discrete Semi-Markov Decision Processes (SMDPs) model so that the agent can perform the task well. In our method, a hierarchical Fuzzy-ART (Adaptive Resonance Theory) network structure enables to construct the optimal discrete state space for the task by using SMDPs model information about state values and probabilities of state transition. If there are great differences between values of the next states and the highest probability of state transition is very low value about a state-action pair, the state is divided into several smaller states. The SMDPs model is constructed by using kcertainty exploration method efficiently. The result of mobile robot simulation showed that our system could construct useful SMDPs model to perform the task well.