Creating a Move Corpus of Three-Minute Thesis Presentations with the Assistance of ChatGPT-o1: Toward a Move-Based Phraseological List

抄録

This study investigates whether large language models (LLMs) can expedite reliable move annotation-where “moves” are functional discourse units performing distinct communicative purposes-for spoken academic discourse, with the goal of deriving a pedagogically useful, move-based phraseological list from Three-Minute Thesis (3MT) presentations. A corpus of 160 finalist/winner 3MT talks from 12 universities (69,705 tokens; 7,846 types) was transcribed from YouTube captions and segmented into eight moves (Orientation, Rationale, Framework, Purpose, Methods, Results, Implication, and Termination) using ChatGPT-o1 with a fixed prompt. Two experienced EAP raters independently verified sentence-level labels; residual disagreements were adjudicated by two additional instructors. Agreement with human labels was near-expert (Cohen’s κ = 0.953 vs. Rater 1; 0.924 vs. Rater 2), matching or exceeding human-human alignment (κ = 0.905); three-coder reliability was strong (Krippendorff’s α = 0.927). Only 1.4% of sentences (55/3,914) required reconsideration, and accuracy remained ≥ 0.968 across moves, with the main difficulty at Purpose boundaries. Using the validated move corpus, we extracted four-word phrase-frames (one open slot) per move via text dispersion keyness (log-likelihood based on text frequency) with a file-frequency cutoff (≥ 3 talks). The resulting high-dispersion phrase-frames are strongly related to the functions of moves-for example, Orientation launchers (e.g., my name is *, I’m going to *), Purpose aim frames (e.g., my research * to), Methods chaining (e.g., to do this *, and then we *), Results report shells (e.g., we found that *), and Implication outlooks (e.g., in the future *), while generic function-word skeletons are de-emphasized. These findings demonstrate that an LLM-assisted, human-verified workflow can efficiently produce a reliable move corpus for 3MT and yield pedagogically oriented, text-dispersion-based phraseological resources. Limitations and avenues for refinement of prompt design, broader sentence-level improvements, and application to other genres are discussed.