Learning Membership Functions in a Control System that Combines Fuzzy Inference with the Policy Gradient Reinforcement Learning Method

Abstract

A method has been proposed that combines policy gradient reinforcement learning with a fuzzy controller to create decision-making policies. The method directly learns the weights of fuzzy rules and the outputs of membership functions in a policy to maximize the expected value of the reward per episode. The advantages of the method include high flexibility in terms of fuzzy representation. Using the same framework for learning, it can automatically adjust both the shape of each membership function and the rule weight, which represents the reliability of each control rule. A study applied the method to the task of controlling the speed of an automobile and obtained appropriate policies by learning the rule weights. However, membership functions were not learned; they were designed in advance based on a priori human knowledge. Therefore, in this paper, we propose applying a neural network to the membership function and conducting reinforcement learning on its weight parameters using the combination method. Additionally, we illustrate the learning process with an example of controlling the speed of an automobile. As a result of computational experiments on automobile speed control, we confirmed that the proposed method is capable of learning appropriate membership functions. Consequently, it is believed that the proposed method can be generally applied to automatically acquire concepts using fuzzy representations, such as ‘long/short’ and ‘fast/slow’.