A historical progress of educational and psychological measurement in the 20th century is reviewed by summarizing theoretical and technological developments which appeared in each decade. The 20th century can be viewed as a century that attaining objective measurement was the primary concern of testing. The reliability and validity theory was the good example and the advent of multiple-choice test and the machine scoring were as well. The more use of advanced technology will appear in the field of testing in the 21st century under the rapid progress of IT. Computer-based testing will be popularized in school and the use of constructed-response format will increase for performance assessment. Highly technological testing will be necessary in distance education through electronic communication. The complex but challenging research tasks to be solved will be forced to psychometric researchers who have never seen before. Some examples are presented in this article.

View full abstract

[in Japanese]

View full abstract

This paper proposes a new test theory, which is called the graphical test theory, from Bayesian approach. The unique features of this method are as follows ; 1. the joint probability distribution of test data is represented by the probabilistic network model, 2. the test construction algorithm by using the amount of test information is proposed from the decision theory, and 3. the predictive test score and predictive learner's knowledge states can be provided. From the mathematical view point, we assume the Dirichlet-multinomial model for the network model, which is more moderate assumption than one of the traditional modern test theories. From this, we derive a maximum a posterior (MAP) estimator of parameters and the exact solutions of the predictive distributions of the structures for the networks. Furthermore, we provide the following results : When the hyper-parameter is 1 (the prior is uniform), this model asymptotically becomes close to the Item Response Theory as the number of items becomes large. The amount of Test Information of this model becomes close to the amount of Fischer Information as the number of items becomes large. In addition, the estimation of a network structure given data in this theory is equivalent to one of the Item Relational Structure Analysis (IRS Analysis) when the hyper-parameter is 1/(2 log n) (the asymptotic likelihood form).

View full abstract

Answer-Until-Correct (AUC) item response model (HiROSE 1999) for the multiple-choice test items was examined in the case where the number of responses was restricted. By using 10,000 generated data of four-category items, the ability estimates from the completed AUC responses were compared with that from the uncompleted AUC responses in which the answering was terminated up to twice. The estimates from the uncompleted AUC responses correlated more than 0.9 with those from the complete AUC responses and the increase of the root mean square error (RMSE) was less than 6%. Furthermore, the estimates from the uncompleted AUC responses indicated much superior, in terms of RMSE, to other estimates based on nominal response model, 2-parameter logistic (2PL) model, and the number right score. This result demonstrated that the AUC item response model was useful even when the AUC procedure was uncompleted.

View full abstract

This paper reports a development of a computerized testing system for practical use which can assist the pre-mid-post test execution process. The system design is consistent to unify the function of CATC (Computer Assisted Test Construction), CAT (Computer Assisted Testing) and CMI (Computer Managed Instruction). The advantages of the system are an interactive test construction with an estimation function using a theoretical model, the newly gathered data like time required for answering and answering histories of examinees and the automatized process of data input through test execution, marking and analysis. The empirical knowledge of problems and attention points getting through the system development process is mentioned too.

View full abstract

New engineering design courses called Engineering Design I and II have already been implemented on approximately 2000 sophomore students since 1996, at the Kanazawa Institute of Technology. In these courses, students carry out their own team project, faculties do not have lectures and written test. So faculties have to know student's individual performance well. We intend to establish the method that grade individual student's contribution to the team achievement. In other word, we intended to establish the method that grade individual student precisely, accurately and fairly. In this paper, we investigate and analyze the relationship between student's evaluations or faculty's one and individual activities that are observed objectively.

View full abstract