Action Recognition of Public Spaces Using Multi-Modal Model

Abstract

In promoting smart cities, there is a demand for the evaluation of the quantity and quality of activities in public spaces. Research on labor-saving measures through AI for assessing the quantity of activities is progressing, but research on labor-saving measures for quality assessment is just beginning. Traditional research on AI models for labor-saving qualitative evaluation of public spaces faced issues such as 1) high model creation costs, and 2) low model versatility, which did not lead to sufficient labor-saving. In response to this problem, this study proposes a method for recognizing actions in public spaces using a multimodal model. A multimodal model is one that integrates multiple data sources and has strengths such as 1) zero model creation cost, and 2) high model versatility. By quantitatively evaluating the performance of the multimodal model for qualitative evaluation using small-scale video data, this study demonstrates the potential for public labor-saving through multimodal models.