2022 Volume 29 Issue 2 Pages 669-687
Supervised quality estimation methods require a corpus that manually annotates qualities of translation outputs. To avoid such costly annotation process, previous studies have proposed unsupervised quality estimation methods based on machine translation models trained on a large-scale parallel corpora. However, these methods are not applicable to low-resource or zero-resource language pairs. This study addresses this problem by utilising a pre-trained multilingual denoising autoencoder. Specifically, the proposed method constructs a machine translation model by fine-tuning the multilingual denoising autoencoder with parallel corpora. It then estimates the translation quality as a forced-decoding probability of a translation output given its source sentence. The pre-trained denoising autoencoder captures linguistic characteristics across languages, which allows our method to evaluate translation quality of low-resource and zero-resource language pairs. Evaluation results on the WMT20 quality estimation task confirm that the proposed method achieves the best unsupervised quality estimation performance for five language pairs under the black box settings. Detailed analysis shows that the proposed method also performs well on under the zero-shot setting.
