Applicability of Inverse Reinforcement Learning to Behavior Analysis of Drosophila melanogaster

Abstract

Understanding the behavioral strategies and decision-making processes of animals and humans is an important issue for environmental conservation and disaster prevention planning. Recently, some studies have reported the introduction of inverse reinforcement learning(IRL) to reveal reward-based behavioral strategies using their behavioral trajectories. Various algorithms have been proposed for IRL, which is a method for learning reward functions and strategies from the action trajectories of experts with optimal strategies. Each of them assumes different situations, such as the availability of prior knowledge, and the necessity of an environment model. Therefore, it is necessary to select an algorithm that fits the characteristics of the target problem. Most of the existing studies apply MaxEntIRL with an environmental model as a given, and the results depend on the validity of the model. In addition, the influence of the number of behavioral trajectories used as input, incompleteness, the pros and cons of assuming a common reward function, as well as differences in the settings of state inputs, may have a significant impact on the results of behavioral analysis. In this study, we compare the rewards obtained by two IRL algorithms, model-free and model-based, for the behavioral trajectories of Drosophila with visual stimuli, and examine the effects of the design of the state input.