Abstract
This downstream case study tests whether Whisper transcripts can be used as inputs for construction-based proficiency research on L2 spontaneous speech using the Constructional Diversity Analyzer (CDA). We analyzed one-minute monologues from 107 Japanese learners in the International Corpus Network of Asian Learners of English (ICNALE) and their associated TOEIC scores. After AI-based pruning removed disfluencies from both Whisper transcripts and the corresponding human transcripts (Manual), CDA yielded a constructional diversity index and arcsine-transformed proportions for 11 constructions. Manual–Whisper agreement was strong for constructional diversity (r = .793; mean absolute difference = .0279) and analyzable construction-specific proportions (r = .648–.837), although absolute deviation varied by construction. Constructional diversity correlated positively with TOEIC in both conditions (Manual r = .370; Whisper r = .335), and an interaction model showed no evidence of a slope difference per 1 SD increase in constructional diversity (b = −0.758, 95% CI [−34.233, 32.718], p = .964). Conversely, AIC-based stepwise models using construction proportions showed modest fit (Manual R² = .098; Whisper R² = .071), with the passive proportion as the only consistent significant predictor. Overall, Whisper appears viable for scalable CDA-based profiling via constructional diversity, whereas construction-level proficiency claims warrant caution in this dataset.
