Machine Translation of English Medical Device Adverse Event Terminology using Deep Learning

Abstract

　We are building a system to map efficiently between medical device adverse event terminology in the Japanese Federation of Medical Device Associations (JFMDA) and that in the International Medical Device Regulatory Forum (IMDRF) for international harmonization. The purpose of this study was to evaluate the accuracy of machine translation in the IMDRF terminology using deep learning as the one of the steps of constructing the system. We obtained nine models as follows: mBART and m2m-100 (418M parameters and 1.2B parameters) which are multilingual translation models developed by Facebook, GPT-3 developed by Open AI, googletrans developed by Google, our original model created by parallel corpus using IMDRF terminology and Japanese Medical Device Nomenclature code and the three fine-tuned mBART and m2m-100. We evaluated the accuracy of translation in each model using test data extracted from the parallel corpus. Googletrans was the most accurate model for our task, with the highest BLEU score and manual confirmation. GPT-3 was the second accurate model in manual confirmation. The fine-tuned mBART had a slightly higher BLEU score, however after manual confirmation, the quality of translation was significantly reduced. Fine-tuned m2m-100 had lower BLEU score with deterioration in the quality of translation. Our original model had the lowest BLEU due to insufficient amount of data used for training.