This paper proposes the recognition of semantically equivalent predicate phrases such as “consumes” and “eats” in “it consumes/eats a lot of memory.” Differences in predicate expressions pose a serious problem in natural language processing applications such as text mining, which extracts text data according to a user’s needs and wants. We propose a novel technique that uses various linguistic clues ranging from abstract semantic features to contextual features in order to detect a semantic similarity in different predicates. The results indicate that our proposed method achieved the highest f-score compared with baseline methods.

View full abstract

Recent research on multilingual statistical machine translation (SMT) focuses on the usage of pivot languages in order to overcome resource limitations for certain language pairs. This paper proposes a new method to translate a dialect language into a foreign language by integrating transliteration approaches based on Bayesian alignment (BA) models with pivot-based SMT approaches. The advantages of the proposed method with respect to standard SMT approaches are threefold: (1) it uses a standard language as the pivot language and acquires knowledge about the relation between dialects and a standard language automatically, (2) it avoids segmentation mismatches between the input and the translation model by mapping the character sequences of the dialect language to the word segmentation of the standard language, and (3) it reduces the translation task complexity by using monotone decoding techniques. Experiment results translating five Japanese dialects (Kumamoto, Kyoto, Nagoya, Okinawa, Osaka) into four Indo-European languages (English, German, Russian, Hindi) and two Asian languages (Chinese, Korean) revealed that the proposed method improves the translation quality of dialect translation tasks and outperforms standard pivot translation approaches concatenating SMT engines for the majority of the investigated language pairs.

View full abstract

In this study, we regard multi-document summarization as a redundancy-constrained knapsack problem. The summarization model based on this formulation is obtained by adding a constraint that curbs redundancy in the summary to a summarization model based on the Knapsack problem. As the redundancy-constrained knapsack problem is an NP-hard problem and its computational cost is high, we propose a fast decoding method based on the Lagrange heuristic to quickly locate an approximate solution. Experiments based on ROUGE evaluation show that our proposed model outperforms the state-of-the-art text summarization model, the maximum coverage model, in finding the optimal solution. We also show that our decoding method finds a good approximate solution, which is comparable to the optimal solution of the maximum coverage model, more than 100 times faster than an integer linear programming solver.

View full abstract