A Sequential Decision-Making Model in Contextual Foraging Behavior

Abstract

In order efficiently to explore and exploit the environment humans and animals (i) generalize experience through trial-and-error in similar states and (ii) focus on a certain level of achievement to switch risk attitudes in an appropriate way. In that process they build up and draw on a model of the environment that enable them to deal with contextual information such as the features of each option and the current state of the environment. In this paper, we propose the Regional Linear Risk-sensitive Satisficing (RegLinRS) model, which is a decision-making algorithm that incorporates the exploratory strategy and risk attitudes of humans and animals. Based on the simulations of the contextual bandit problems, we evaluate the performance and exploratory efficiency of RegLinRS. We then analyze RegLinRS’s adaptive rationality, that is, in what manner and to what degree the algorithm works to promote the survival of agents. The results show that RegLinRS enables the agent to quickly secure a satisfactory goal while minimizing expected loss.