We propose a new approach to obtain the solutions of HJB equations. Firstly, based on a simple but unique idea, we transform the conventional HJB equations into the virtual-time HJB equations (VT-HJB). As the virtual time goes by, the solution of the VT-HJB equations becomes that of the conventional ones. Then, we solve the VT-HJB equations with respect to the virtual time by means of existing differential equation solvers. Lastly, we demonstrate the effectiveness of our method through numeric experiments with linear and nonlinear control problems.
