Event-centric Knowledge Graph Representation to Transcribe Human Activity into the Cyber-Physical System

Abstract

Expectations are rising for human-centered AI embodied in the real world. Nevertheless, applications such as older adult support, child monitoring, and general-purpose robots for home use require event-centric knowledge of what happened in addition to observed data and external factual knowledge. In previous work, we have modeled event-centric knowledge graphs for mystery novels, using events as units to represent the whole scene as a sequence of events. We have also developed VirtualHome2KG, representing human behavior in cyberspace as an event-centric knowledge graph, including living environments and furniture and rooms. On the other hand, we are also developing an inference system that uses event-centric knowledge graphs to infer and explain dangers in daily life and derive safer alternatives. In this study, we discuss the schema of event-centric knowledge graphs, which enables us to infer risks that are difficult to detect directly in daily life and improve planning accuracy for generous-purpose home robots. Furthermore, we aim to emulate human daily living activity represented by knowledge graphs in cyberspace using VirtualHome2KG to provide a high-quality data set that serves to improve video recognition technology.