Keyword Extraction by Keyword-Fitness Optimization

Abstract

Nowadays there are very many document databases. However, people often find it difficult to retrieve target documents from those databases. One reason for the difficulty is that keywords assigned to documents are not adequate. This paper presents a novel method for automatic keyword extraction from Japanese documents in a database. Conventionally, keywords have been extracted, based on various heuristics, with which the importance of individual words is measured. This paper proposes objective criteria for extracting keywords from a mass of candidate-words. They are efficiency criterion and recall criterion. The efficiency criterion concerns the efficiency involved in utilizing a word for retrieving a document from a database. The recall criterion for a word concerns the likelihood that that word is used as a keyword for database retrieval. Those two criteria are quantified statistically using distribution pattern of documents in a database. A product of the quantified criteria supplies a keyword-fitness measure for a word. Keyword extraction is implemented as an optimization of the keyword-fitness by Genetic Algorithm. An experimental result shows the validness of the keyword-fitness and suggets the complementarity of the authors' keyword-fitness and heuristics, when conventionally used.