Searching for Distress of Similar Situation from CQA Content

Abstract

In this study, we tackle the problem of retrieving questions from a corpus archived in a Community Question Answering service that a consultant having distress can feel empathy with them. We hypothesize that the consultant feels empathy with the questions having a similar situation with that of the consultant’s distress, and propose a method of retrieving similar sentences focusing on the situation of the distress. Specifically, we propose two approaches to fine-tuning the pre-trained BERT model so that the learned model better captures the similarity of the situation between distress. One tries to extract only the words representing the situation of the distress, the other tries to predict whether the two sentences show the same situation. The data for training the models are gathered by the crowdsourcing task where the workers are asked to gather the sentences whose situation is similar to the given sentence and to annotate the words in the sentences that represent the situation. The data is then used to fine-tune the BERT model. The effectiveness of the proposed methods is evaluated with the baselines such as TF-IDF, Okapi BM25, and the pre-trained BERT. The results of the experiment with 20 queries showed that one of our methods achieved the highest nDCG@5 while we could not observe any significant differences among the methods.