Development of Automated Grade Application for Processing Description in Programming Exercise Reports

Abstract

This paper describes an attempt to automatically grade a Japanese C language programming exercise report by summarizing its processing explanation in three lines. We fine-tuned the natural language processing model Text-To-Text Transfer Transformer (T5) to generate a model for a three-line summary task that outputs a summary sentence from the processing explanation of the programming exercise reports. The proposed automatic grading process, grades to the description, are based on the cosine similarity between the summary sentence output by the three-line summary task and the ground truth three-line summary. The performance evaluation uses two learning models: a livedoor model that fine-tunes live-door news articles and a C-lang model that fine-tunes C language programming exercise reports. As a result of the experiment, the livedoor model, fine-tuned using training data from news articles with many common expressions in Japanese sentences, was sufficient for the three-line summary description.