Efficient method to evaluate misalignment between virtual and real objects for AR-based assembly assistance system

Abstract

For complex assembly, it is important to give a user concise and effective assembly instructions for efficient assembly operation and training. Using an AR HMD is a possible way to improve it by showing virtual objects at installing locations. Moreover, it is important to give a warning when the user does not install the object well. In this research, we propose a methodology to efficiently evaluate if the user set the real object as instructed for an AR-based assembly assistance system using only an AR HMD, Microsoft HoloLens. It is evaluated by computing the distance between the virtual object (CAD model) and the digital information of the real object that is the points scanned by a depth camera embedded in HoloLens. For the efficiency, instead of performing the calculation in 3D, including determining the correspondences and computing the distances, we address it in a 2D depth image to achieve efficient computation without the need of searching for the correspondences. With the preliminary development, without other computing devices, the proposed method that is implemented on HoloLens can achieve the fast assessment of the misalignment and is expected to be used in various situations.