Thinning-out: A Method to Reduce Trials in Skill Discovery of a Robot

Abstract

In skill discovery of a robot, the number of trials (i.e., evaluations of a score function) is highly limited since each trial takes much time and cost. In this case, memory-based learning, which retains and utilizes the history of trials, is efficient. There are mainly two approaches in studies of memory-based learning. One is an approach to estimate scores by using an approximation model of an original score function despite evaluating the score function. The other is an approach to estimate proper scores in a noisy score function. In this paper, we take another approach to find unpromising search points and skip over the evaluations by characterizing a function class which a score function belongs to. We call this approach thinning-out of search points in contrast to pruning of search trees. The main advantage of thinning-out is to make correct judgments definitely, which means that thinning-out skips over only unpromising search points, as long as the defined function class is proper. We show the properties of thinning-out by addressing the maximization problems of several test functions. In addition, we apply thinning-out to the problem of discovering of physical motions of virtual legged robots and show that the virtual robots can discover sophisticated motions that are much different from the initial motion in a reasonable amount of trials.