Optimization of Rank Order of Weighted City-block Distance

Toward Supervised Learning for Analogy Task using Word Vectors

Abstract

Making analogical inference in vector space has become a standard method to test the quality of word vectors. Typically the operator for the analogical inference is manually optimized for the task. In this study, we consider a systematic optimization of the metric-based rank order function for the analogical inference. If we directly evaluate the rank order function, one needs to process a few millions of word vectors every step of optimization. This causes a considerably large computational cost which makes a systematic optimization of such analogical inference intractable. In this study, we propose a theoretical approximation for this rank-order evaluation, and demonstrate an optimization of the analogical inference using the approximated evaluation. Lastly, we discuss about the ``parallelogram'' relationship, which may or may not have a deep connection with the well known ``analogy parallelogram'', revealed by the mathematical analysis of the probability of the distance-based rank order.