Validation of ChatGPT's object co-occurrence information using a 3D scene-space dataset

Abstract

For a robot to operate in a human workspace, information about the co-occurrence of objects in space and the associated concept of location is essential. Recently, large-scale language models such as ChatGPT know object co-occurrence and can elicit information about the co-occurrence between objects in a dialogue. However, the validity of the co-occurrence answers provided by LLMs has not yet been verified. Therefore, we use ScanNet v2, a spatial dataset with object annotations, to validate the co-occurrence information generated by ChatGPT. This paper determines the co-occurrence based on the object location domain in ScanNet v2. After ChatGPT had provided preliminary information about Sacnnet V2, questions about co-occurrence between objects were asked, and co-occurrence information was generated. The verification results showed that the F value of the co-occurrence of ChatGPT was 0.634 when the co-occurrence by ScanNet v2 was the actual value. It was also found that ChatGPT tended to have more false positive predictions but fewer false negatives.