Proceedings of the Annual Conference of JSAI
Online ISSN : 2758-7347
37th (2023)
Session ID : 3G5-OS-24b-02
Conference information

Automatic Generation of Action Scripts from Descriptions of Daily Life Activities in VirtualHome
*Jin AOYAMAMotoki YATSUTakeshi MORITA
Author information
CONFERENCE PROCEEDINGS FREE ACCESS

Details
Abstract

Agents in the household simulator named VirtualHome can interact with the environment based on action scripts that describe a sequence of instructions. Each instruction consists of an action and an object used by the action. There is a dataset containing activity labels, activity descriptions, and action scripts that correspond to each activity description built through crowdsourcing. However, the construction cost is high, and there is an issue that action scripts that cannot be executed in VirtualHome are included in the dataset. Since actions that can be performed on VirtualHome have preconditions for execution, the list of actions generated by existing methods often cannot be executed on VirtualHome. To solve the issues, we propose a method to generate action scripts from descriptions of daily life activities by natural language processing technologies and home environment knowledge. In addition, we evaluated the proposed method using the above dataset.

Content from these authors
© 2023 The Japanese Society for Artificial Intelligence
Previous article Next article
feedback
Top