Recent studies have demonstrated the usefulness of contextualized word embeddings in unsupervised semantic frame induction. However, they have also revealed that generic contextualized embeddings are not always consistent with human intuitions about semantic frames, which causes unsatisfactory performance for frame induction based on contextualized embeddings. In this paper, we tackle supervised semantic frame induction, which assumes the existence of frame-annotated data for a subset of verbs in a corpus and propose to fine-tune contextualized word embedding models using deep metric learning for high-performance semantic frame induction methods. Our experiments on FrameNet show that fine-tuning with deep metric learning considerably improves the clustering evaluation scores by about 8 points or more. We also demonstrate that our approach is effective even when the number of training instances is small.

The ability of natural language processing models to handle numerical values is of practical and scientific interest. For a deeper understanding of these models, it is important to see how they handle math problems requiring multiple inference steps. We introduce a method for analyzing how a Transformer model handles these inputs by focusing on simple arithmetic problems and their intermediate results. To trace where information about intermediate results is encoded, we measure the correlation between intermediate values and the activations of the model using principal component analysis (PCA). Then, we perform a causal intervention by manipulating model weights. This intervention shows that the weights identified via tracing are not merely correlated with intermediate results, but causally related to model predictions. Our findings provide a deeper understanding of how Transformer models handle arithmetic problems, which has implications for improving their interpretability.

We developed a chatbot that imitates a specific person by combining a large amount of dialogue data of that person with a large language model. Furthermore, we investigated the current performance and problems of the chatbot by conducting an open experiment and error analysis of the chatbot. The experiment confirmed the high naturalness and characterness of the chatbot. In addition, our investigation revealed that the errors specific to a person can be divided into two types: errors in attributes and errors in relations, each of which can be divided into two levels: self and other.

We propose a method for building a commonsense inference dataset based on basic events. Specifically, we automatically extract contingent pairs of basic event expressions such as “I'm hungry, so I have a meal” from text, verify by crowdsourcing, and automatically generate commonsense inference problems regarding the contingent relation between basic events. We built a commonsense inference dataset of 100k problems by the proposed method and conducted experiments to investigate the model performance. The results showed that there is a performance gap between high-performance language models and humans. In addition, we automatically generated large-scale pseudo problems by utilizing the scalability of the proposed method and investigated the effects by the data augmentation on the commonsense inference task and the related tasks. The results demonstrated the effectiveness of learning extensive contingent knowledge for both the commonsense inference task and the related tasks, which suggests the importance of contingent reasoning.