Environment-specific knowledge acquisition system for home service robots via brain-inspired memory and commonsense knowledge

Abstract

Home-service robots are expected to enhance the quality of daily life. Two types of knowledge, commonsense and environment-specific knowledge, are required for home-service robots. Current robots can handle audio-visual-based common knowledge, task planning, and action generation. Additionally, several systems that handle environment-specific knowledge have been developed. Previously proposed brain-inspired models can integrate visual and location information to represent episodes; however, it is difficult to obtain novel environment-specific knowledge. In this study, we propose a system that efficiently acquires novel environment-specific knowledge by combining environment-specific knowledge stored in a brain-inspired memory system with commonsense knowledge inferred by large language models (LLMs). We verified the performance of common-sense retrieval from LLMs and evaluated the effectiveness of combining environment-specific knowledge and commonsense knowledge in the home environment.