Exploring Sampling Methodology for Obtaining Reliable Text Coverage

Abstract

Increasingly, "text coverage" is being used in studies to measure the intelligibility of word lists and second language learning material, and because, to date, there are no established standards for calculating text coverage, the reliability of resulting data and its practical application to language learning is called into question. This study addresses this issue by applying empirical analyses (distribution of mean score and standard deviation) to text coverage samples using variations of text length (both with and without proper nouns), vocabulary size and sample size in order to determine how these variables might affect the calculation of text coverage. Twenty-six different text lengths taken from CNN news transcripts and 22 lists of graded vocabulary range taken from high frequency words of the British National Corpus were analyzed using 10 djfferent sample sizes in 1,000 iterations. The results of the study clearly demonstrate that text coverage is more stable when the text length is longer when more samples are used, and when proper nouns are excluded When proper nouns were retained, the coverage figures were 10-12% less than when they were excluded As a practical guideline for educators, a table showing minimum parameters is included for reference in computing text coverage calculations.