Classification of Treatment Discontinuation Reasons in Japanese Electronic Health Records Using Large Language Models

Abstract

A large volume of free-text data in electronic health records (EHRs) describes treatment discontinuations, including those caused by adverse events. However, because this information is insufficiently structured in existing databases and thus difficult to extract, it remains underutilized despite its significant value. In this study, we combined automated labeling using Large Language Models (LLMs) with a small amount of manual annotation to efficiently classify treatment discontinuations due to adverse events. We integrated approximately 6,256 LLM-labeled records with 200 manually annotated samples, then fine-tuned JMedRoBERTa and T5. When evaluated on a 100-record test set, the T5 model demonstrated high precision (0.83) but was limited to a recall of 0.25. Missing adverse events is a critical concern in clinical practice, underscoring the need for more extensive training data. In the future, we plan to expand our approach to other discontinuation reasons (e.g., patient preferences or insufficient therapeutic effect) and strive for practical clinical implementation.