Automatic Evaluation of the Fluency of English-to-Japanese Machine Translation Using Word Alignment

Abstract

This paper presents a method of automatically evaluating the fluency of machine-translated sentences. We constructed a classifier that would distinguish machine translations from human translations, using Support Vector Machines as machine learning algorithms. In order to obtain a clue to the distinction, we focused on literal translations (word-for-word translations). The classifier was constructed based on features derived from word alignment distributions between source sentences and human/machine translations. Our method employed parallel corpora to construct the classifier but required neither manually labeled training examples nor multiple reference translations to evaluate new sentences. We confirmed that our method could assist evaluation on system level. We also found that this method could identify the qualitative characteristics of machine translations, which would greatly help improve the translation quality.