The Implicit Reward of the Entropy Regularizer in Signaling Games

Abstract

This paper focuses on the auxiliary objective function, the entropy regularizer, used in signaling game optimization, and to show its implicit reward function. The signaling game is a very simple communication model used in the field of language emergence. The entropy regularizer is used to aid the agents' search when optimizing signaling games via reinforcement learning techniques. However, this auxiliary function is introduced ad hoc, and thus the reward function implicitly assumed therein is unclear. It may also hinder mathematical discussions in this research field. We clarify the implicit reward function of the entropy regularization term to make the agent's optimization target more explicit. In addition, we discuss the entropy maximizer which is a similar auxiliary objective to the entropy regularizer. We hope that our paper will trigger mathematical discussions in the field of language emergence.