Automatic Generation and Tuning Method of Definition Statements for Binary Classification of Financial Texts Using Large Language Models

Abstract

Text classification using large language models (LLMs) often has low interpretability and is hard to adjust manually. On the other hand, zero-shot learning, which uses definition statements in LLMs for classification, is more interpretable but creating good definition statements is a challenging task. Therfore, we propose a method to automatically generate definition statements using LLMs to improve classification accuracy and interpretability in (binary) text classification. The proposed method first randomly splits the labeled data and generates (initial) definition statements based on sampled data. Then, it classifies the labeled data using these statements and updates them by inputting misclassified data back into LLMs, repeating this process to improve the definition statements. Experiments with real-world texts show that the proposed method performs well compared to fine-tuned BERT model and LLM few-shot learning and creates appropriate definition sentences.