A UNIFORM TWO-ARMED BANDIT PROBLEM: THE PARAMETER OF ONE DISTRIBUTION IS KNOWN

Abstract

The sequential design procedure which selects one of two experiments e₀ and e₁ at each stage in order to maximize the sum of n observations is considered, where the distributions of e₀ and e₁ are uniform distributions on the intervals (0, u) and (0, 1) respectively. The value of u is unknown and is assumed to have a Pareto distribution with parameters w and α as a prior distribution. This problem is formulated by the dynamic programming and is solved recursively. The expected sum of n observations is given as function of w, α and n, and the optimal strategy is to perform e₀ if and only if w_??_w_n (α). w_n (α) is strictly increasing in α and non-decreasing in n, and the value of w_n (α) is calculated for several values of w and α and tabulated.