Trials of Night-Time 4K-Camera-based Human Action Recognition in Riverine Environments with Multimodal and Object Detection Technologies

Abstract

At present, the Ministry of Land, Infrastructure, Transport, and Tourism (i.e., MLIT) has installed about 3,000 CCTV cameras in rivers throughout Japan. Initially, these devices were installed to monitor water levels during floods and visually inspect damage after earthquakes. Moreover, the riverine environment management researchers (i.e., “the researchers” in the following content) have been trying to survey the situation of rivers and embankments through patrolling. In addition, night-time patrols are carried out only in certain areas at certain times of the year for certain purpose. It is difficult for the researchers to monitor the status of riverine areas 24 hours a day. In recent years, researchers have conducted river space utilization surveys (i.e., RSUS) using camera images and machine learning. However, in the case of nocturnal RSUS, there is no detailed research on this topic. Therefore, the authors of this study (i.e., “the authors” in the following content) aim to understand the number and behavior of humans using river space at night, and confirm the performance of the models depending on the brightness difference in the images.