Verification of Chain-of-Thought Prompting in Japanese

Abstract

Foundation models can be adapted to various tasks by Few-Shot learning, which uses a small number of examples as a prompt. To improve Few-Shot learning, Chain-of-Thought (CoT) prompting has been proposed, which divides the process of thinking into steps. Although the effectiveness of CoT has been proved in English tasks requiring logical reasoning, it has not been verified in Japanese. We examine the effectiveness of CoT in Japanese using a Japanese foundation model, HyperCLOVA JP. We first construct Japanese datasets for the following three tasks: arithmetic, commonsense, and symbolic reasoning. Then, we conduct experiments using HyperCLOVA models of four different sizes. The results showed that CoT prompts were more accurate than standard prompts, and that the performance of CoT prompts was correlated with model size.