A Comparison of Adaptation and Conditional Probability for Accuracy Improvement of Term Extraction

Abstract

“Adaptation” is a statistic to measure importance of words, and assumes that keywords tend to occur more than twice in a document. Given target information (such as some keywords and their definition) as a hint, we can use other statistics that are more effective than adaptation, but such information is often unavailable. If adaptation can extract keywords similar to the statistics, it will be an evidence to enhance the usefulness of adaptation which does not require the target information. To demonstrate the usefulness, we conduct an experiment on term extraction with two statistics: adaptation and conditional probability which is given the target information in advance. As a result, we show that these statistics tend to extract common keywords.