IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
Online ISSN : 1745-1337
Print ISSN : 0916-8508
Regular Section
KL-UCB-Based Policy for Budgeted Multi-Armed Bandits with Stochastic Action Costs
Ryo WATANABEJunpei KOMIYAMAAtsuyoshi NAKAMURAMineichi KUDO
Author information
JOURNALS RESTRICTED ACCESS

2017 Volume E100.A Issue 11 Pages 2470-2486

Details
Abstract

We study the budgeted multi-armed bandit problem with stochastic action costs. In this problem, a player not only receives a reward but also pays a cost for an action of his/her choice. The goal of the player is to maximize the cumulative reward he/she receives before the total cost exceeds the budget. In the classical multi-armed bandit problem, a policy called KL-UCB is known to perform well. We propose KL-UCB-SC, an extension of this policy for the budgeted bandit problem. We prove that KL-UCB-SC is asymptotically optimal for the case of Bernoulli costs and rewards. To the best of our knowledge, this is the first result that shows asymptotic optimality in the study of the budgeted bandit problem. In fact, our regret upper bound is at least four times better than that of BTS, the best known upper bound for the budgeted bandit problem. Moreover, an empirical simulation we conducted shows that the performance of a tuned variant of KL-UCB-SC is comparable to that of state-of-the-art policies such as PD-BwK and BTS.

Information related to the author
© 2017 The Institute of Electronics, Information and Communication Engineers
Previous article Next article
feedback
Top