2007 Volume 30 Issue 4 Pages 275-282
A tool to reduce rating errors due to the recency effect in human report evaluation is developed and evaluated. The recency effect is a cognitive bias due to the quality of sentences read up to that point. This tool visualizes similarity of contents in reports obtained through Latent Semantic Analysis (LSA) by mapping using Multidimensional Scaling (MDS). The rater can refer to the map and read or evaluate similar contents. To investigate its effectiveness, rating scale dispersion for each evaluation item was compared between groups that did and did not use the tool for report evaluation. The results indicate that in 3 items-[The report includes characteristic vocabulary of the theme], [there is consistency in the content], and [comprehensive evaluation]-the variance was significantly lower in the group that did vs. the groups that did not use the tool. This indicates an aspect of report evaluation in which this tool is effective.
