Improving Neural Machine Translation by Incorporating Hierarchical Subword Features

Abstract

This paper focuses on the subword-based neural machine translation (NMT). We hypothesize that the appropriate subword-units for three modules in the NMT model, namely, (1) encoder embedding layer, (2) decoder embedding layer, and (3) decoder output layer, can differ each other. We empirically investigate our assumption and find that incorporating several different subword-units for input and output embedding layers can consistently improve the BLEU score on the IWSLT 2012, 2013 and 2014 evaluation datasets for De-En, En-De, Fr-En, and En-Fr translation tasks.