Japanese Journal of Medical Technology
Online ISSN : 2188-5346
Print ISSN : 0915-8669
ISSN-L : 0915-8669
Materials
Performance of generative pretrained transformer on the national licensing examination for medical technologist in Japan
Hiroki DOIHidekazu ISHIDAHiroki NAGASAWAYoshiki TSUBOIRyosuke KIKUCHINaohiro ICHINOHidehiko AKIYAMAKuniaki SAITO
Author information
JOURNAL FREE ACCESS FULL-TEXT HTML

2024 Volume 73 Issue 2 Pages 323-331

Details
Abstract

In recent years, Large Language Models (LLM) have gained worldwide attention in various fields. LLM are language models built using extensive datasets and deep learning techniques. LLM have garnered global attention due to their ability to exhibit human-like fluency in speech and achieve high accuracy in various natural language-based processes. In this study, we examined whether LLM could correctly solve the National Clinical Laboratory Technician Examination for the past three years. We used ChatGPT (GPT-3.5 and GPT-4), one of the LLM developed by OpenAI. The results showed that GPT-3.5 had an average correct response rate of 51.4% over the past three years, which did not reach the passing level of 60%. On the other hand, GPT-4 had an average correct response rate of 79.8%. These findings indicate that ChatGPT has potential to evolve as an effective advisor in the field of clinical laboratory science. However, the 20% of incorrect answers in this study included answers that could lead to misdiagnosis when diagnosing patients, suggesting that further improvement of the accuracy of the ChatGPT is essential. We believe that this validation will contribute to the development of various applications of ChatGPT in LLM in the clinical laboratory field, and we look forward to its further development.

Content from these authors
© 2024 Japanese Association of Medical Technologists
Previous article Next article
feedback
Top