Automated Generation of a Meta-Video Dataset for Human Action Recognition using Commonsense Knowledge Graph

Abstract

Constructing meta-datasets that capture the relationships between action labels across different datasets typically requires extensive human annotations. To address this challenge, we propose an approach that automates this process by leveraging common sense knowledge graphs. Specifically, our method aims to extract equivalence relationships between action labels by linking labels from multiple action recognition datasets to the corresponding nodes in common sense knowledge graphs. In this study, we focus on six widely used datasets. To evaluate the effectiveness of our approach, we use precision, recall, and F-measure with the manually constructed MetaVD serving as the ground truth. Additionally, we introduce a simplified evaluation metric, the Average Transferred Precision Gain to assess the performance. Our preliminary results demonstrate the potential of the proposed method for automating the equivalence relationship extraction. Future work will extend this approach to identify additional relationships, such as similarity and hierarchical (is-a) relationships, to enable a more comprehensive automated meta-dataset construction.