Abstract
This paper proposes a novel word alignment model based on a recurrent neural network (RNN), in which an unlimited alignment history is represented by recurrently connected hidden layers. In addition, we perform unsupervised learning inspired by (Dyer et al. 2011), which utilizes artificially generated negative samples. Our alignment model is directional, like the generative IBM models (Brown et al. 1993). To overcome this limitation, we encourage an agreement between the two directional models by introducing a penalty function, which ensures word embedding consistency across two directional models during training. The RNN-based model outperforms both the feed-forward NN-based model (Yang et al. 2013) and the IBM Model 4 under Japanese-English and French-English word alignment tasks, and achieves comparable translation performance to those baselines under Japanese-English and Chinese-English translation tasks.
