Sentence Embeddings using Definition Sentences

Abstract

Sentence embeddings, which represent sentences as dense vectors, have been actively studied as a fundamental technique for natural language processing using deep learning. In particular, sentence embedding methods based on Natural Language Inference (NLI) tasks have been successful. However, these methods heavily rely on large NLI datasets and thus cannot be expected to produce adequate sentence embedding for languages for which large NLI datasets are not available. In this paper, we propose a sentence embedding method using definition sentences from a word dictionary, which is available in many languages. Experimental results on standard benchmarks demonstrate that our method performs comparably to NLI-based methods. Furthermore, we demonstrate differences in performance depending on the properties of the evaluation task and data, and even higher performance can be achieved by combining the two methods.