Abstract
We propose a new textured 3D reconstruction method that uses image sequences output by projection-based display systems. So that image sequences can recreate convex objects on a planar display, we extend the photometric model used in photometric stereo to handle full color images. According to the photometric model, we generate multiple shading images from surface normals on the original 3D object with varying light directions. We show that an image data matrix created from the multiple images can be decomposed into a surface normal matrix scaled up to the color intensity of three channels and a light-source directional matrix, and that its rank is at most three. In order to ensure robust and unique matrix decomposition, we introduce new metric constraints by using pairs of the directional lights that correspond to odd and even frames. Based on matrix decomposition, we describe a method that reconstructs the surface normals for 3D shape and surface texture. We examine the proposed method by challenging it with both computer-simulated data and real images. These results demonstrate that our method yields good 3D reconstruction and surface texture.
