Neural 3D Mesh Renderer

Abstract

We introduce our paper ``Neural 3D Mesh Render" presented at CVPR and MIRU last year. In this work, we proposed a novel renderer that takes a 3D mesh, light, and camera setting and outputs an image. Because ``back-propagation" is defined in our renderer, it can be used as a layer of deep neural networks. By using it, we can pass the gradient of a loss into a 3D space through renderer and optimize components there. In experiments, we demonstrated the effectiveness of our renderer by applying it to view-based training of single-view 3D reconstruction, 2D-to-3D style transfer, and 3D DeepDream. We also introduce some papers that use our renderer for other problems.