分散Q学習を用いた多目的最適化アルゴリズムのMNK-Landscapesでの検討

抄録

In this paper, we study a distributed Q-learning approach for multi-objective optimization of epistatic binary problems and investigate its performance. The Q-learning based method assigns an agent per objective function, specifies a state as a solution and a position where an agent can act and defines an action as a 1-bit mutation operator. The method also introduces conditional state transitions, where an agent moves to a new state only if the chosen action improves the best solution found in the current episode. Otherwise, the agent discards previously chosen actions and continues sampling from the same state until a threshold is reached. We investigate the proposed method solving MNK-landscapes with 100 bits varying the number of objectives from 2 to 4 and the number of epistatic interactions from 1 to 20. We also compare with a multi-objective random bit climber moRBC that also implements a 1-bit neighborhood search and NSGA-II and MOEA/D, two well known multi-objective algorithms representatives of Pareto dominance and decomposition based approaches, to better understand the algorithm’s search behavior and performance on epistactic problems of increased difficulty.