Hybrid quantum-classical Ulam-von Neumann linear solver-based quantum dynamic programing algorithm

Abstract

Solving large-scale systems of linear equations is an important part in many artificial intelligence applications, especially for dynamic programing which is heavily used in the reinforcement learning field. The arrival of noisy-intermediate-scale-quantum computers provides new opportunities to solve linear systems at larger scales. The hybrid quantum-classical linear solver using Ulam-von Neumann method was demonstrated previously. In this work, we apply the hybrid quantum-classical Ulam-von Neumann linear solver to the dynamic programming where the state value function or action state value function V(or Q)= (1-γP)^-1 R (where is γ is discount rate, P is state transition matrix and R is reward) to be solved. Systematic circuit extensions beyond unistochastic matrices are developed based on the idea of linear combination of unitarizes and quantum random walks. A generative adversarial networks training method for matrix construction is also developed. Numerical examples for some benchmark reinforcement learning tasks are demonstrated.