Generation of Picture Cards to Support Emotional Understanding Using Generative AI

Abstract

Picture cards illustrating emotions and actions with words are used to support children with developmental disorders and communication difficulties. Recently, AI image generators are used for different purposes in several applications, but the production of picture cards still relies on manual work and takes time and cost. This study aims at supporting emotional awareness and communication in therapy using generative AI in two aspects. First, we propose a method of inferring emotions and communication represented by picture card illustrations using a Large Language Model (LLM) and improving the accuracy of word inference through fine-tuning. We evaluate whether the generated words correctly represent emotion and communication described in illustrations. Second, we introduce a method of generating picture card illustrations using an image generator Stable Diffusion. We verify whether the generated illustrations express emotions properly.