JLTA Journal
Online ISSN : 2189-9746
Print ISSN : 2189-5341
ISSN-L : 2189-5341
Volume 19
JLTA Journal
Displaying 1-8 of 8 articles from this issue
  • 2016 Volume 19 Pages Cover_1-
    Published: 2016
    Released on J-STAGE: December 13, 2016
    JOURNAL OPEN ACCESS
    Download PDF (1195K)
  • Jessica R W Wu
    2016 Volume 19 Pages 3-11
    Published: 2016
    Released on J-STAGE: December 13, 2016
    JOURNAL OPEN ACCESS
    The General English Proficiency Test (GEPT) is a 5-level, criterion-referenced EFL testing system implemented in Taiwan to assess the general English proficiency of EFL learners at all levels. The GEPT was designed as a skills-based test battery assessing both receptive (listening and reading) and productive (speaking and writing) skills. Since its first administration in 2000, the GEPT has been taken by more than 6.5 million learners, and has become the largest-scale standardized English test in Taiwan. In the wake of the introduction of productive skills to the university entrance examination system in Japan, this talk aims to share the GEPT experience. Several key issues about speaking and writing tests will be presented in relation to the socio-cognitive framework for validation (Weir, 2005). A number of examples about GEPT validation will be illustrated to demonstrate that both a priori and a posteriori validity evidence are required to establish test quality. The paper also emphasizes the importance of facilitating communication between test developers and stakeholders when introducing new assessment.
    Download PDF (7101K)
  • Su Yon YIM
    2016 Volume 19 Pages 15-23
    Published: 2016
    Released on J-STAGE: December 13, 2016
    JOURNAL OPEN ACCESS
    The concept of assessment for learning in classrooms emphasizes the supportive role of assessment for learning in the sense that assessment occurs while learning rather than at the end of the learning unit. Peer assessment is one of the methods which realize the concept of assessment for learning since it provides students with opportunities to participate in the assessment process and consequently become autonomous in learning. The current study examines factors influencing primary school teachers’ intention to use peer assessment in English writing. A total of 82 primary school teachers (79 females and 3 male) in Korea responded to the theory of planned behaviour (TPB) survey. A seven –point Likert scale numbered from 1 (strongly disagree) to 7 (strongly agree) was used. Results show that Korean primary school teachers on average intended to use peer assessment for teaching writing in English classes (M = 4.00, SD = 1.28). The analysis of open-ended responses also shows that teachers are aware of the pedagogical benefits of peer assessment in English writing. The perceived constraints are centred on who to involve and how to implement peer assessment in classrooms. More studies are needed to gain more information on the way of implementing peer assessment for young EFL learners.
    Download PDF (6280K)
  • Yuzuru UTSUNOMIYA, Masazumi MARUYAMA, Shinji OGASAWARA
    2016 Volume 19 Pages 27-45
    Published: 2016
    Released on J-STAGE: December 13, 2016
    JOURNAL OPEN ACCESS
    The purpose of this study is to examine (a) whether students’ TOEIC scores will increase as they gain higher English proficiency measured by G-TELP, a surrogate English proficiency test and (b) how and what kind of factors (such as individual, faculty, and year) affect their scores most. The target group is students in a Japanese national university from nine departments, sampled from school year 2011 to 2015. To explore the objectives, we generated hierarchical Bayesian models to estimate the tendency and differences together. We obtained the following three results. First, students’ TOEIC scores rise as their English proficiency improves. Second, the three factors affect the TOEIC scores. Finally, the similarly-proficient students with lower TOEIC scores raise their TOEIC scores more than those with higher scores if the students improve their own proficiency. We conclude that the students’ TOEIC scores will improve if the students raise their proficiency. Moreover, if we judge their English proficiency in terms of their TOEIC scores, we should consider the three factors carefully.
    Download PDF (18202K)
  • Hideki IIMURA
    2016 Volume 19 Pages 46-65
    Published: 2016
    Released on J-STAGE: December 13, 2016
    JOURNAL OPEN ACCESS
    This study reports on a replication of Iimura’s (2014) study on the attractiveness of distractors in multiple-choice listening tests. Sixty-eight Japanese university students were assessed on their correct responses in a picture-description task of an MC listening test (15 questions, four options each). During the listening test, the participants were asked to judge each of the four options as correct or incorrect and report the degree of confidence they had in their judgment. On the basis of the confidence level and correctness of response, confidence and attractiveness scores were generated. To assess how listening ability affected test-takers’ confidence and distractors’ attractiveness, three groups were developed on the basis of correct scores obtained on the listening test. The results of the replication study have confirmed the original study, suggesting that (a) the least frequently chosen distractors were not always the least attractive, (b) upper-level listeners were less attracted to distractors, and (c) upper-level listeners had greater confidence when responding with the correct answers. This article concludes that the conventional item analyses (i.e., response frequency and discriminatory power) are insufficient in evaluating the effectiveness of distractors, and a new kind of survey, in which test-takers can evaluate each distractor independently, should be incorporated in future MC test development.
    Download PDF (9436K)
  • Tetsuhito SHIZUKA
    2016 Volume 19 Pages 66-85
    Published: 2016
    Released on J-STAGE: December 13, 2016
    JOURNAL OPEN ACCESS
    Segmenting a stream of continuous speech into recognizable words presents one of the biggest challenges to novice EFL listeners. One straightforward method of checking how good a learner is at this task is dictation. Two members of the dictation family are partial dictation and multiple-choice (MC) dictation. Combining these two techniques, the author and his colleagues devised multiple-choice 1-blank partial dictation (MC1PD). An earlier study that examined the nature of MC1PD produced partly expected and partly unexpected results. Comparison of MC1PD and its open-ended (OE) counterpart indicated that success on an OE item usually entails success on its MC counterpart but not vice versa, which had been expected. Comparison of MC1PD and TOEIC® scores indicated that MC1PD correlated more strongly with TOEIC® reading than with TOEIC® listening, which had been unexpected. This unexpected pattern had to be attributed to random noise caused by the sample for which TOEIC® scores were available being small. In addition, a number of distractors of MC1PD items used in the earlier study were subsequently found to be non-functional. The present study addressed the following research questions: (a) Will replacing non-functional distractors of MC1PD with new options better the items’ psychometric performances? (b) Can the expected relationship between MC1PD items and their OE counterparts observed again using the revised MC1PD test? (c) Can reasonable and interpretable relationships be found between MC1PD scores and TOEIC® scores, using the revised MC1PD test? A revised MC1PD test was prepared by updating 14 distractors in the MC1PD test used in the earlier study. A total of 115 Japanese EFL university students took an OE partial dictation test and the revised MC1PD test, with a one-week interval between them. A subsample (n = 85) also sat for institutional TOEIC® tests. Classical and Rasch analyses of the data indicated that the answers to the three research questions were all affirmative.
    Download PDF (16031K)
  • Wakana ONOZUKA
    2016 Volume 19 Pages 86-108
    Published: 2016
    Released on J-STAGE: December 13, 2016
    JOURNAL OPEN ACCESS
    The purpose of this study is to show differential item functioning (DIF) of gender and resident (in Japan or not) on the Business Japanese proficiency Test. In the first study, 2,444 examinees, the students of Japanese as a foreign language class, took 160 item test of business Japanese proficiency. Three empirical methods were used to detect DIF: namely, the Mantel-Haenszel method, ANOVA method and item response theory method. As a result, 20 gender DIF items and 67 residential DIF items were found by at least one of the methods mentioned above. Three gender DIF items and 14 residential DIF (in Japan) items were detected by all the three methods. In the second study, 60 items consisting of DIF and DIF free items were shown to nine experts (5 females and 4 males) of item writing and their subjective impression of items concerning DIF were analyzed. It was found that the experts used five beliefs about gender DIF and five beliefs about residential DIF when they rated items. 12 items were judged to have gender DIF and 54 items were judged to have residential DIF by at least one expert. The comparative study showed that the result of the statistical DIF analysis and the subjective prediction did not coincide much: The experts overlooked gender DIF items and they were very sensitive to residential DIF items.
    Download PDF (11462K)
  • 2016 Volume 19 Pages 111-124
    Published: 2016
    Released on J-STAGE: December 13, 2016
    JOURNAL OPEN ACCESS
    Download PDF (9031K)
feedback
Top