On-line Lighting and Reflectance Estimation for AR Systems Based on Inverse Rendering

Abstract

Photometric registration is one of the more challenging problems related to augmented reality (AR) because the simultaneous estimations of both lighting and reflectance are especially difficult problems due to large number of parameters and ill-posed problems. As a result, most currently utilized lighting and reflectance estimation methods employ light probes such as mirror spheres, omnidirectional cameras, or require preliminary scanning of the target object. However, these light probe types are not fully suitable for AR systems. In this paper, we introduce an in-situ lighting and reflectance estimation method that does not require specific light probes and/or preliminary scanning. Our method uses images taken from multiple viewpoints while data accumulation and lighting and reflectance estimations run in the background of the primary AR system. As a result, our method requires little manipulations for image collection.