Transactions of the Society of Instrument and Control Engineers
Online ISSN : 1883-8189
Print ISSN : 0453-4654
ISSN-L : 0453-4654
Systems and Information
A State Space Filter for Reinforcement Learning in Partially Observable Markov Decision Processes
Masato NagayoshiHajime MuraoHisashi Tamaki
Author information
JOURNAL FREE ACCESS

2009 Volume 45 Issue 1 Pages 41-50

Details
Abstract
This paper presents a technique for reinforcement learning to deal with both discrete and continuous state space systems in POMDPs while keeping the state space of an agent compact. First, in our computational model for MDP environments, a concept of “ state space filtering ” has been introduced and constructed to make the state space of an agent smaller properly by referring to “ entropy ” calculated based on the state-action mapping. The model is extended to be applicable in POMDP environments by introducing the mechanism of utilizing effectively of history information. The extended model is capable of being dealt with a continuous state space as well as a discrete state space by the extended model. Here, the mechanism of adjusting the amount of history information is also introduced so that the state space of an agent should be compact. Moreover, some computational experiments with a robot navigation problem with a continuous state space have been carried out. The potential and the effectiveness of the extended approach have been confirmed through these experiments.
Content from these authors
© 2009 The Society of Instrument and Control Engineers
Previous article Next article
feedback
Top