Abstract
Model-based reinforcement learning includes two steps, estimation of a plant and planning. Planning is formulated as dynamic programming (DP) problem, which is solved by a DP method. This DP problem has an equivalent linear programming (LP) problem that can be solved by LP method, but it is generally less efficient than typical DP method. However, numerical examples show linear programming is more efficient than the typical DP method in problems whose self-transition probabilities are large. The reason is clarified by geometrical discussion of each solution of method approaches to optimal solution.
