Agent Reinforcement Learning by Using a Finite State Machine with Deep Neural Network

Abstract

An agent design by using reinforcement learning has been made progress, and there is a need for more efficient and flexible methods to control reinforcement learning. Therefore, the combination of classical decision-making model, the state machine, and deep neural network (DNN) reinforcement learning is supposed and examined. A state with DNN, in which one trained deep neural network (DNN) is installed per state, and a neural net is switched by state transition to control the movement of character. A state is a set of symbolistically defined state and connectionism NN. It makes character AI creation more flexible. In this study, a state machine with four states was built in the Unity3D environment, and the movement of a character was implemented by using reinforcement learning such that the character takes a ball in the stage right side area and transports it to the goal in the left side area. The performance, flexibility, etc. of this method are evaluated by comparing a model in which the training is split for each state of the method with a model trained by a single DNN.