Generalization of the state space for cooperation in similar situations of a heterogeneous multi-agent environment

Abstract

In a multi-agent environment, agents should often be required to choose cooperative behavior to solve tasks. The agents usually owes such behavior to special rules designed by humans, especially in an environment consisting of heterogeneous agents, but it is impossible to design such rules for myriad situations. Thus, it has been proposed to learn such cooperative behavior with reinforcement learning in such a hetero-agent environment where they have to collect targets. That method, however, highly depends on the environment; the learned policies do not work in other environments at all, even in similar ones. This work alleviates the problem by defining the state space relatively, i.e., the state space is defined by the relation between the agents and the target. The experimental results show that the policies obtained by the proposed method work well in other, similar environments, as well as in the identical one.