Journal of Natural Language Processing
Online ISSN : 2185-8314
Print ISSN : 1340-7619
ISSN-L : 1340-7619
Estimation of Nativeness of Documents Based on Skew Divergence
HIROSHI FUJIIYOICHI TOMIURASHOSAKU TANAKA
Author information
JOURNAL FREE ACCESS

2005 Volume 12 Issue 4 Pages 79-96

Details
Abstract
The automatic discrimination between documents written by native speakers andones by non-native speakers is an important technique to construct a high-qualitycorpus, to help native speakers with writing, and to gather useful knowledge in Sec-ond Language Acquisition.This paper proposes the method of such a discriminationbased on the similarity of part-of-speech trigram distributions.The distributionalsimilarity is given by Skew Divergence.Skew Divergence is an improved functionof KL Divergence, and it does not suffer from the zero-frequency problem.To use Skew Divergence, it needs to decide the value of the parameter α in Skew Divergence.However, there have not been any sufficient discussions on how to decide it.This pa-per also proposes one of the methods how to set the parameter αThe experimentalresult shows the effectiveness of the proposed method.
Content from these authors
© The Association for Natural Language Processing
Previous article Next article
feedback
Top