A Meta-Learning Perspective on Generative Query Network

Abstract

Generative Query Network (GQN) is a novel deep generative model for three-dimentional modelling, making it possible to render images from unknown viewpoints. However, properly training GQN requires huge learning costs (including both computational time and hardware), and it is severally sensitive to hyper-parameters. Moreover, the lack of studies on GQN from a probabilistic view makes it difficult to interpret the neural network architecture. In this paper, we formulate the probabilistic model of GQN using meta-learning framework, and based on the formulation, we propose a new method that significantly reduces learning costs, and improves the accuracy of generated images, learning efficiency and robustness against hyper-parameters. We show the effectiveness through experiments using the Shepard Metzler dataset.