Generation of Consistent Linguistic Expressions of Pictures

Abstract

This paper describes the construction of the system that outputs consistent linguistic expressions explaining some pictures given in random order. Input to this system is objective information on some pictures and this system outputs linguistic expression explaining the contents of pictures and the connective relationships between pictures. The present system consists of the basic contents explanatory part, the connective relationship explanatory part and the linguistic composition part. The basic contents explanatory part outputs behavior description explaining behaviors of objects drawn in pictures. The connective relationship explanatory part infers event description explaining the matters imagined to be happened between two consecutive pictures and outputs the combination of event descriptions evaluated to be consistent. The linguistic composition part composes the descriptions obtained from other two parts. In order to confirm the usefulness of the present approach, simulation experiments are performed. In the experiments several subjects interact with the system to construct their own database and the linguistic expressions obtained by the system are evaluated.