A Versatile Approach to Shape from Shading by Learning with Inclusion of Image-Formation Models

Abstract

In this paper, we present a versatile method of solving the problem of shape from shading in computer vision by neural networks. The proposed method is versatile in the following sense. (i) It can deal with all the surface-reflection models which have been proposed so far. (ii) It can handle the case where not only a single image of an object but also a few or more images taken by different illumination and shooting conditions are available for surface recovering. (iii) It can handle the case where not only image data but also depth and/or gradient data of an object taken by appropriate sensors are available.
We first introduce a mathematical model, which we call ‘image-formation model’, expressing the process that the image is formed from an object in some illumination and shooting condition. We formulate the problem of surface recovery into the learning problem of neural networks with inclusion of the image-formation model in the learning loop and derive its learning algorithm. The proposed method can be used to solve various problems in all the above situations by changing the image-formation model according to the situations.