Multilingual Code Search Dataset Using Neural Machine Translation

Abstract

Code search is a task to find programming codes that semantically match the given natural language queries. Even though some of the existing datasets for this task are multilingual on the programming language side, their query data are only in English. In this research, we create a multilingual code search dataset in four natural and four programming languages using a neural machine translation model. Using our dataset, we pre-train and fine-tune the transformer-based models, and then evaluate them on multiple code search test sets. Our results showed that the model pre-trained with all natural and programming language data has achieved the best performance in most cases. Exceptionally, the model pre-trained only with Python for programming language data performed better when tested on Python data.