Abstract
In evaluating the effectiveness of information retrieval (IR) and extraction system, the most common method is to compare two retrieval methods and decide if one system measurably achieves better results than the other. However, it is difficult for researchers to compare more than two retrieval methods because there are many participants in IR task in IREX workshop. In this paper, we evaluate the characteristics and the effectiveness of the IR systems using a statistical method based on the results of the IR formal run and questionnaires of systems. Comparisons of systems deal with effects on the performance such as indexing, querying and retrieval model. The results confirm the effectiveness of this evaluation method because phrases relates to the performance better than words. There is a trade-off relation between the precision value at 0.0 and decrease rate in many systems and this result indicates the difficulty of the choice of techniques in system. We also evaluate correlations between the efficiency and the characteristics of the systems with both a short and long versions of the topics. A result of this evaluation shows that it is important to select effective methods for the long version of topics.
