Estimating the Viewing Position of a Castle Using An Image of Its Lower Structure Only

Abstract

In recent years, Augmented Reality (AR) technology has been widely applied and actively researched in the digital reconstruction and interactive presentation of cultural heritage. However, many historical sites have lost their upper structures, making image-based spatial registration and viewpoint estimation using only the remaining lower structures a major technical challenge. This study proposes a lightweight monocular image-based angle estimation system specifically designed for such partially preserved historical architecture scenarios. The system uses an enhanced ResNet18 network to output the cosine and sine of the viewing angle, while prediction stability is improved through the integration of a Channel and Spatial Reliability Tracker (CSRT) and a moving average filter. The estimated angles are transmitted to Unity in real time via UDP (User Datagram Protocol) to achieve synchronized rotation and AR of the virtual model. Experiments conducted on a dataset constructed using an Osaka Castle model demonstrate a Mean Absolute Error (MAE) of 3.1[deg], Root Mean Squared Error (RMSE) of 8.7[deg]. The proposed method significantly outperforms baseline methods including PoseNet [8], which utilizes Convolutional Neural Networks (CNNs) for camera pose estimation, as well as Scene-Agnostic regression [11]. The proposed method offers an effective and practical digital presentation solution for structurally incomplete heritage sites, highlighting its potential for use in educational settings and preservation-oriented AR applications.