In this paper, we propose a new method for calculating the output layer in neural machine translation systems with largely reduced computation cost based on binary code. The method is performed by predicting a bit array instead of actual output symbols to obtain word probabilities, and can reduce computation time/memory requirements of the output layer to be logarithmic in vocabulary size in the best case. In addition, since learning proposed model is more difficult than softmax models, we also introduce two approaches to improve translation quality of the proposed model: combining softmax and our models and using error-correcting codes. Experiments on English-Japanese bidirectional translation tasks show proposed models achieve that their BLEU approach the softmax, while reducing memory usage on the order of one tenths, and also improving decoding speed on CPUs by x5 to x10.

This paper presents a method for improving semantic role labeling (SRL) using a large amount of automatically acquired knowledge. We acquire two varieties of knowledge, which we call surface case frames and deep case frames. Although the surface case frames are compiled from syntactic parses and can be used as rich syntactic knowledge, they have limited capability for resolving semantic ambiguity. To compensate for the deficiency of the surface case frames, we compile deep case frames from automatic semantic roles. We also consider quality management for both types of knowledge in order to get rid of the noise brought from the automatic analyses. The experimental results show that Chinese SRL can be improved using automatically acquired knowledge and the quality management shows a positive effect on this task.

Several studies on automated text simplification are based on a large-scale monolingual parallel corpus constructed from a comparable corpus comprising complex text and simple text. However, constructing a parallel corpus for text simplification is expensive as large-scale simplified corpora are not available in many languages other than English. Therefore, we propose an unsupervised method that automatically builds a pseudo-parallel corpus to train a text simplification model. Our framework combines readability assessment and sentence alignment and automatically constructs a text simplification corpus from only a raw corpus. Experimental results show that a statistical machine translation model trained using our corpus can generate simpler synonymous sentences performing comparably to models trained using a large-scale simplified corpus.