自然言語処理
Online ISSN : 2185-8314
Print ISSN : 1340-7619
Clustering Words with the MDL Principle
Hang LiNaoki Abe
著者情報
ジャーナル フリー

1997 年 4 巻 2 号 p. 71-88

詳細
抄録

We address the problem of automatically constructing a thesaurus (hierarchically clustering words) based on corpus data. We view the problem of clustering words as that of estimating a joint distribution over the Cartesian product of a partition of a set of nouns and a partition of a set of verbs, and propose an estimation algorithm using simulated annealing with an energy function based on the Minimum Description Length (MDL) Principle. We empirically compared the performance of our method based on the MDL Principle against a method based on the Maximum Likelihood Estimator, and found that the former outperforms the latter. We also evaluated the method by conducting pp-attachment disambiguation experiments using an automatically constructed thesaurus. Our experimental results indicate that we can improve accuracy in disambiguation by using such a thesaurus.

著者関連情報
© The Association for Natural Language Processing
前の記事 次の記事
feedback
Top