This paper provides an overview of formative evaluation in the context of the systematic design and development of stand-alone instructional materials with a special emphasis on the application of various techniques of formative evaluation to CAI courseware development. The historical advancement of formative evaluation concepts and techniques is first described from the introduction of the term by Scriven in 1967 to later works by Markle, Baker, Burkman, Palmer, Komoski, Nathenson and Henderson, and Dick and Carey. Next, the reasons for conducting formative evaluation are listed: (1) to improve the effectiveness of the materials using student data, (2) to contribute to the formation of instructional theories and models, (3) to keep the materials consistent with the entry behavior and characteristics of the target audience, (4) to provide a training opportunity for instructional designers, and (5) to facilitate later adoption of the materials. After the three-stage procedure of formative evaluation is briefly introduced using the model by Dick and Carey (1985), potential problems with instructional materials and methods of possible revisions are provided in order to link formative evaluation data and consequent revisions. It is pointed out that although the general model of formative evaluation can be applied to CAI development, deliberate attention should be paid to the unique capabilities of the computer as an instructional medium. The discussion includes: (1) timing for conducting formtive evaluation in relation to cost effectiveness of the production as well as the prediction of the effectiveness of the final product, and (2) techniques for evaluating CAI courseware that has adaptive structure to individual learner differences. Finally, an alternative 3 - step approach to develop CAI courseware is proposed for the practitioners with limited resources.

View full abstract

In ordedr to develop an effective CAI program, it is crucial to obtain the knowledge on the conditions, such as construction methods of the program and conditions of learners. In case of developing CAI programs for foreign language teaching, the relations between the "egrul" (inductive) method and the "ruleg" (deductive) method can be said as a problem of constructing the effective CAI program. The former method is thought to correspond to the Audio-Lingual Approach, while the latter to the Cognitive-Code Learning Theory. Moreover, it is important to compare audio-only presentation (sound) with audiovisual one (sound and letters), since it is now possible to include sound in the whole system. In the Experiment I , four methods of ruleg, egrul, dispersed ruleg and dispersed egrul were tested with 181 high school students. The content of the instructional material was about the usage of "come" and "go". In the experiment, a pertest consisting of 10 items was conducted first, followed by the explanation by tapes, and finally a posttest consisting of 20 items was practiced. The scores were calculated and an analysis of variance, an item analysis and a t-test were applied. An a result of this, the presenting method of "ruleg" was found to be most effective, that is, the Cognitive-Code Learning method. In the Experiment II, another visual condition of letters was added. Three methods of ruleg, dispersed ruleg and dispersed egrul in audio-only presentation and the same three methods in audiovisual presentation methods were tested with 254 university students. The content of the instructional material and the procedure of the experiment was the same as the Experiment I. The scores were computed and to which the item analysis and the analysis of variance were applied. From this result, it was clear that the audiovisual presenting method was superior to the audio-only presenting method.

View full abstract

The authors have been conducting formative research aimed at developing a personal computer-based system for training in speed reading Japanese. Reducing both frequency and duration of eye fixation points is basic to speed reading. In order to be able to achieve these goals it is necessary to improve the subject's reading concentration and rhythm. The authors have been developing a personal computer-based Japanese language speed reading system, adaptable to individual learners, which (1) will present appropriate practice materials, (2) uses such presentation as a pacemaker for promoting reading speed, and (3) will keep the student informed of his or her reading accuracy. Using students at Tezukayama Gakuin (Women's) University, the authors have already reported the results of their pre-survey, a preliminary experiment and a first attempt at formative research (Segawa, 1984). From the results of this work, the authors realized that making the CRT display coefficient (the length of time 10 characters are displayed) faster than the reading coefficient (the length of time it takes the subject to read 10 characters) results in increased speed with the reading coefficient evidencing an exponential distribution. For the second attempt at formative research, a display coefficient was established by drawing a learning curve on dual logarithmic coordinate graph paper which leveled out slightly sooner than did the learning curve in the first study. A ten-day study course for a group of Tezukayama Gakuin University students, using the same methods and materials as used in the first study, showed positive mastery from the first day of the course. The reading coefficient rose from 1.24 to 0.86 ; analysis of variance showed a 1 % level of significance. The rate of correct responses also rose from 5696 to 8596 ; a mastery coefficient of 0.0815. The third formative research project was designed to consider to what extent plotted straight line reading and display coefficients could deviate from one another. Because, during the first experiment, it had been found that the learning curve for the reading coefficient followed an exponential distribution it was possible to predict the succeeding day's reading coefficient. Each of the subjects used his or her own personal computer and program for computing the display coefficient was made a part of each learning data file. The results showed that the average reading coefficient increased from 1.56 to 0.66 ; again using analysis of variance, significant at the 1 96 level. The rate of correct response was stable at between 71.4 and 74.98%. From the first day of the experiment the reading coefficient evidenced an exponential distribution with a learning coefficient of 0.1619. A relative high degree of correlation was also found between the daily "phrase reading training" (phrases of 5 or more characters and syllabary combined) and the reading coefficient for that same day.

View full abstract

The problem of how students perceived teacher behavior, from the point of view of Non-Verbal Communication, is discussed in this study. In the classroom, a teacher exibits many different types of behavior, of which illustrators, classified by Ekman and Friesen, were effective to student learning of Japanese. Illustrators help students to understand the content of educational materials. It is interpreted that this role of behavior can repeat, illustrate, and emphasize the words of the teacher. The cognitive process of behavior perception, however, has not been clarified, so far, in the educational practice settings. In order to clarify this cognitive process, two types of questionnaires were developed. One is concerned with how many types of behavior the students recognize in their own teacher, the other with the degree of perception of the students. In total, two hundred and twenty nine students in the 4th, 5th, and 6th grades were questioned in the classroom settings. The results were arranged in accordance with the score of the Japanese Achievement Test and with that of the Intelligence Test. Two practical and useful things were found, as follows : (1) there is a high correlational coefficient greater than .667 between the number of students' perception of their parents' behavior and that of their own teacher. (2) there are two types of student's perceptions of behavior, as shown in Fig. 6. [figure] Fig. 6 Two Types of Perception of Behavior. Fig. 6 shows that the higher intelligent group imagined something through their teacher's behavior when they saw their own teacher's behavior, but the lower intelligent group imagined nothing even with the behavior. This latter tendency would seem to be greater in the younger grades, and would disappear when one matures.

View full abstract