The purpose of this study is to investigate the relationship between word order knowledge and EFL reading comprehension among high school students. The participants were seventy-four second year senior high school students. A written recall test, and three types of sentence anagram tests (basic, intra-phrasal and inter-phrasal anagram tests) were administered to all 70 students. There were significant correlations between word order tests and reading comprehension. Moreover, a 3 (the reading comprehension group) × 3 (types of anagram tests) two-way ANOVA revealed significant interactions. The results indicate that regardless of the levels of reading comprehension, inter-phrasal anagram task is the most difficult. However, the high-level group also shows the significant difference in intra-phrasal anagram task. Therefore, it is concluded that both intra-phrasal word order instructions and inter-phrasal word order instructions should be focused on more for better reading comprehension.

View full abstract

This study examines the effects of reactivity in a vocabulary test in sentential context by comparing the performance of two groups: one using thinking aloud and the other not using thinking aloud. Although using thinking aloud in investigating the learning process is a widespread practice, there is a possibility that conducting thinking aloud might change the learner's cognitive process; hence, the data might not reflect the true process. The results of comparing two groups, one using thinking aloud and the other not using it, showed that the thinking-aloud group outperformed the control group. There were two possible interpretations: the thinking aloud itself affected the test performance, or other variables accompanying the thinking aloud were factors that caused the improved test score. The results of further analysis showed that the difference of the time allotment between the two groups rather than the characteristics of thinking aloud could influence the test performance. From these results, this study adds further proof of the validity of using thinking aloud as a methodology for collecting data about test-taking processes.

View full abstract

This paper presents a preliminary portion of the reports on our research project. In May 2006, the TOEIC(R) was revised in order to make it a more suitable test battery to measure English communicative language ability. The purpose of the project was to compare the older version of the TOEIC(R) test and the new one from the perspective of established testing theories. University students learning English as a foreign language (N=136) in Japanese universities made up the participants in this study. In November and December 2008, the two versions of the TOEIC(R) were administered to the participants of the study with an interval of one to two weeks. Based on the reliability estimation method, the two-way ANOVA (TOEIC(R) Version (2) × Subtest (2)), and item analyses of the participants' test scores, the following research findings were obtained. (1) Both versions of the TOEIC(R) test indicated high reliability coefficients (old: α=0.928; new: α=0.933); (2) the participants performed better in the new TOEIC(R) than the old one; (3) The participants performed identically in both TOEIC(R) Reading Comprehension sections, but performed better in the new Listening Comprehension section than the old one; (4) the new TOEIC(R) Listening Comprehension section contained more satisfactory test items than the old one in terms of item difficulty; and (5) the old TOEIC(R) Listening Comprehension section contained more satisfactory items in terms of actual equivalent number of options. The directions for further research on TOEIC(R) as a placement test in the language classroom setting are also discussed.

View full abstract

Laufer and Hulstijn (2001) indicated that incidental vocabulary learning would be effective in fostering higher vocabulary retention if the task induced higher involvement load that comprises of one motivational perspective (i.e., need) and two cognitive perspectives (i.e., search and evaluation). This hypothesis was examined empirically by Hulstijn and Laufer (2001) and Kim (2008). However, the extent to which the level of the involvement load should be adjusted to foster more effective vocabulary retention remains unknown. In addition, in the previous studies, participants were limited to university students. This study, thus, examines the effect of various levels of involvement load on Japanese high school EFL learners' vocabulary retention. In order to collect data, reading and sentence writing tasks with different levels of involvement load were created and implemented during class time. The learners' vocabulary retention was measured on a vocabulary knowledge scale (Paribakht & Wesche, 1996; 1997), immediately after the tasks were performed and again after 10 days. The results demonstrated that sentence writing task with a moderate level of involvement load (i.e., moderate need, no search, and moderate evaluation) seemed to more successfully promote vocabulary retention of the high school learners than the reading tasks. The study also argued that the evaluation component is more likely to contribute to effective vocabulary retention than the search component. In addition, it was suggested that for learners, higher involvement load may not always induce deeper processing of vocabulary, which in turn would lead to retention.

View full abstract

This study introduced course grading criteria that value self-focused opinionated writings as opposed to using conventional evaluation methods based on quizzes and examinations from textbooks. These writings were aimed at developing self-identity for a successful college life. With the mission of college education in mind (i.e., the aim of fostering good citizenship-minded adolescents through successful character forming), the study proposed a new pedagogical paradigm in English as a foreign language (EFL) classes. This new approach includes the finding, choosing, and acquiring of critical target language and the necessary communicative abilities needed to pursue the development of true self-image and self-identity in real life contexts beyond college. Hence, a number of self-focused tasks, materials, and evaluation methods were exercised in order to encourage college learners to better describe themselves in a critical manner. Within such a learning environment, they were urged to develop their personal life values, beliefs, philosophy of life, concrete academic purposes, and their envisaged professional life style. Over a one semester period, first-year college learners (N = 55) participated in a classroom-based study using a core weekly EFL course. Data were collected through several types of surveys including a course evaluation administered at the end of the study. The results suggested that the introduction of self-focused written assignments, follow-up oral presentations, and the related course grading were readily accepted by the participants. It was surmised that a self-focused EFL course may change the learners' attitudes and behaviors toward a solid language learning purpose: developing successful self-orienting and authentic selfhood for the maturity of spirit.

View full abstract

This research describes a part of the development process of a task-based writing performance assessment for Japanese learners of English. In the previous study, two assessment tasks to elicit the underlying constructs and rating scales were developed. The pre-testing was conducted on the participants comprised of 15 university students and five experienced teacher raters, who provided scores for two sets of 15 texts. The correlational analysis showed that the teacher raters could produce relatively high scores with each rating scale The results of FACETS analysis, however, suggested that rating scales for task 1 (accuracy) and 2 (communicability) should be recommended to be optimized. In this research, we explored how rating scales can be developed and improved based on the, results of the pre-testing. First, we optimized the rating scales on the basis of the criteria that Linacre (2002) proposed. Then, the scoring guide was revised in each section. As a result, the new rating scales are comprised of clearer descriptions of each construct and of 5-point Likert scales. The descriptors of each category are also provided with written samples as an explanatory part of the scale in order to give raters a shared understanding of the constructs and to make them more self-consistent in scoring.

View full abstract

The relationship between reading comprehension and vocabulary knowledge has been examined in many studies. However, few studies so far have distinguished the vocabulary knowledge in isolation and in context. Addressing this point, this study considered two types of vocabulary tests: Mochizuki (1998) Vocabulary Size Test (VST) and TOEFL vocabulary items, and tried to clarify the relationship further. Adding to this, we classified question types in the reading test and examined each relationship with two types of vocabulary knowledge in order to differentiate reading items by the level of reading comprehension. The results of Experiment I suggested that regardless of the presence or absence of context, there was no significant difference between the correlation coefficients of the two vocabulary tests and TOEFL Reading scores. In addition, no correlation coefficients were significantly different in any question type, remaining the issue about whether the vocabulary tests in context and without context really differ or not. For the sake of investigating this point further, the word inference test was newly introduced in Experiment II to examine what knowledge or ability the two vocabulary tests measure. The results revealed a large overlap between VST and TOEFL vocabulary items. It suggests not only VST but also TOEFL vocabulary items which present target items in context tend to measure isolated vocabulary knowledge. Pedagogical implications for vocabulary testing and recommendations for further research are also discussed.

View full abstract

This paper reports on the comparison of two types of scale descriptors in a questionnaire on language learning strategies. The main purpose of this study was to investigate which type of the two scale descriptors, frequency-based or trueness-based, is better for language learning strategy research. With a few weeks' interval, a questionnaire on learning strategies was administered to 408 EFL learners twice with frequency-based scale descriptors and trueness-based scale descriptors alternately. First, mean differences in the responses obtained from the two different scale descriptors were examined. Second, confirmatory factor analysis was applied to see which scale descriptor of the two shows better fit to the hypothesized model. Finally, equidistance between the categories was checked. Results show that trueness-based scale descriptors elicited slightly higher mean values of responses, which resulted in a better fit to the model. The distances between the categories were almost identical for both scale descriptors. The findings of the current study partially provide supportive evidence for the claim made by Dornyei and his colleagues that trueness-based scale descriptors are preferable to frequency-based counterparts in a questionnaire on learning strategies. But they also show that the latter descriptors are not totally 'flawed' as was claimed.

View full abstract