The Impact of Academic Writing Course Attendance on University Students' Evaluation of Text Generation Capabilities of Generative AI

Abstract

Recently, the application of generative AI in higher education, particularly writing instruction, has received significant attention. However, limited research has explored how university students evaluate the Japanese text-generation capabilities of generative AI and whether these evaluations change after receiving writing instruction. This study examined the impact of writing courses on university students’ evaluations of generative AI’s report-writing skills. A survey of 222 students enrolled in a writing class at a private university revealed that their evaluations of generative AI-generated texts improved in alignment with course content, particularly in areas such as “warrant” and “logic.” However, evaluations in certain areas, such as “data,” showed limited improvement, highlighting the need for curricular enhancements to deepen students’ understanding of generative AI’s characteristics.