When students are involved in student-centered language learning activities such as group projects or when the instruction is content-based, it is difficult for teachers to evaluate the student performance using only traditional multiple- choice examinations. Alternative assessment is needed to evaluate student socio-cultural competence, critical thinking skills, and problem solving skills. This paper examines and rethinks the concept of evaluation, how to learn and teach languages, and the power relations that exist between teachers and students. It pays particular attention to portfolio assessment. A portfolio is a systematic self-collection of student work, including self-evaluations and reflections on certain periods during learning activities. Portfolio assessment is the data for both the student and the teacher to show or catalog the student's efforts, progress and future directions as well as the summative results of routine class activities. In a survey conducted in three college-level classes, students had a positive reaction towards portfolio assessment as a means of developing student intrinsic motivation, fulfillment, self-esteem, and social interaction among peers. The paper offers some advice on how to overcome the difficulties of the portfolio assessment, including developing relevant criteria and designing a score sheet.

View full abstract

In Japan and Taiwan, English is one of the main subjects included in the entrance examinations, which a majority of high school students are required to take to apply for admission to a higher level of education. And the national unified exam (NUE) is administered once a year to all or many of those planning to enter the universities. This paper is concerned with probing into the peculiar nature inherent in each nation's latest NUE English test with a special focus on the reading comprehension sections. For this purpose, the readability of the English passages in the reading section of each NUE English test will be compared. The 'Flesch Score' is a powerful estimate of the level of difficulty in comprehending that the test takers have experienced, and measures readability based on the average numbers of words per sentence and of syllables per 100 words. 'JACET Basic Vocabulary 4000' can also provide other useful information to analyze vocabulary in each reading passage. Utilizing these two, we find that there are not any statistically significant differences in either the average number of syllables per 100 words or the difficulty of the vocabulary used. However, the average numbers of words per sentence are clearly different between the two nations.

View full abstract

English reading ability has conventionally been measured by learner's performance of how they respond to questions that reflect the content of a reading passage. This is a rather indirect way of measuring one's ability when it is compared with how learner's speech or writing is directly evaluated in measuring these abilities. The present study has given a focus on learner's reading speed and tested its validity as a more direct and valid index of reading ability. Furthermore, it has sought factors that account for learner's reading speed. The effects of text type and topic, along with learner's interest, have also been considered as the factors that influence learner's reading speed. The study involved 239 university students (out of which the results of 160 students were used) in taking three different tests of reading speed. The results revealed that reading speed could be considered as a valid index of one's reading ability. The skills such as skimming and scanning highly accounted for learner's reading speed. As for the influence of text type and topic on reading speed, it was found that the text topic affected the reading speed. This means that, when measuring reading speed, it is necessary to prepare at least one passage from a humanity genre and another from a scientific genre. The learner's interest and motivation to read the given passages revealed little influence on their reading speed. This is more evidence that reading speed could be a valid index of a learner's reading ability. Although there were some limitations to the present study, it demonstrated that the reading speed along with the factors that affect it cast a new light in assessing learner's ability and in considering a new way to consider text readability.

View full abstract

The purposes of this paper are a) to investigate whether the Lunic Language Marathon sorts the test-takers into a normal distribution; b) to detect inappropriate items, which are misfitting items, the items with low discrimination power values, or the items with low or high item difficulty values; and c) to investigate whether the results of the LLM show the selection effects of college entrance examinations. A total of 420 college students from four groups participated in this study. The first group consists of 78 students majoring in information technology at a national university. The second group has 105 students majoring in English language at a co-ed private university. Other 122 students, which fall into the third group, are majoring in social science or humanities at the same university as the second group. The last group contains 115 students majoring in English language at a women's private university. The results of this study show that a) Part 1 and 3 of the LLM sort the college students into a normal distribution, but as for Part 2 and 4, the ceiling effect has been observed; b) three items in the LLM are judged as inappropriate items; and c) there is some effect of group difference on the performance of the participants (Wilks' Lamda=.82, F(15, 1137.75)=5.79, p .00, η^2=.07, Power=1.00), but apparent effects of college entrance examinations have not been observed. This paper further discusses the use of the LLM as a diagnostic test at a college, and points out that specifically the scores on Part 1 will tell the instructors much about the learners' variety.

View full abstract

Although L2 listening construct has been hypothesized to consist of the ability to understand literal messages and the ability to make inferences, its structure has not beenfully investigated empirically. One exception is Shimada (2000), which has explored the psychometric dimensionality of L2 listening ability and found that L2 listening ability is psychometrically more bidimensional. However, his study seems to suffer from a flaw in its methods of dimensionality analysis. The present research partially replicates Shimada and reexamines the psychometric dimensionality of L2 listening ability. The CELT was taken by 171 Japanese EFL college students and analyzed using item-level confirmatory factor analysis and testlet-based confirmatory factor analysis, which were used to overcome the problem with the analyses and differed from Shimada's methods. The results showed that L2 listening ability was psychometrically more bidimensional, which was congruent with Shimada. The results suggested that (a) when we want to measure the whole range of listening ability, two types of items should be put into a test, and that (b) it may be important to see whether data is psychometrically more unidimensional or bidimensional before using item response theory. The fact that the same conclusion as Shimada was reached using different methods reconfirms that L2 listening ability is psychometrically more bidimensional at least among Japanese EFL students.

View full abstract

In Katagiri (2001a), three Forms of a new vocabulary test based on word difficulty order with only 32 items which can be conducted in less than 10 minutes were developed for quick and approximate estimates of examinees' general English ability. Through Katagiri (2001b, 2002b), the concurrent validity and the reliability of the three Forms were confirmed among senior high school students, and the three Forms were confirmed as parallel tests with one another. In this study, the author, first, examines the concurrent validity and the reliability of the three Forms among university students, using TOEIC as a target of concurrent validity. He, then, attempts to create mathematical formulas to convert the scores of each Form into TOEIC scores. One hundred seventy-five freshmen at university took the three Forms of the new vocabulary tests and TOEIC IP. Significant middle correlations were narrowly confirmed between each Form and TOEIC; a middle-high IRT reliability was also narrowly affirmed on each Form. Using simple regression analysis, mathematical formulas which convert the raw scores of each Form into TOEIC scores was created. It was concluded that (1) the concurrent validity and the reliability of each Form were also confirmed among university students as tests for quick and approximate estimates of general English ability, and (2) the conversion formulas from the scores of each Form into TOEIC scores were calculated and shown with the means of errors, which must be taken into consideration when they are utilized.

View full abstract