A Multi-Domain Framework for Textual Similarity. A Case Study on Question-to-Question and Question-Answering Similarity Tasks

A Multi-Domain Framework for Textual Similarity. A Case Study on Question-to-Question and Question-Answering Similarity Tasks Amir Hazem Basma El Amal Boussaha Nicolas Hernandez Laboratoire des Sciences du Num´erique de Nantes (LS2N) Universit´e de Nantes, 44322 Nantes Cedex 3, France {Amir.Hazem,Basma.Boussaha,Nicolas.Hernandez}@univ-nantes.fr Abstract Community Question Answering (CQA) websites have become a very popular and useful source of information, which helps users to find out answers to their corresponding questions. On one hand, if a user’s question does not exist in the forum, a new post is created so that other users can contribute and provide answers or comments. On the other hand, if similar or related questions already exist in the forum, the system should be able to detect them and redirect the user towards the corresponding threads. This procedure of detecting similar questions is also known as question-to-question similarity task in the NLP research community. Once the correct posts have been detected, it is important to provide the correct answer since some posts can contain tens or hundreds of answers/comments which make the user’s research more difficult. This procedure is also known as the question-answering similarity task. In this paper, we address both tasks and aim at providing the first framework on the evaluation of similar questions and question-answering detection on a multi-domain corpora. For that purpose, we use the community question answering forum Stack-Exchange to extract posts and pairs of questions and answers from multiple domains. We evaluate two baseline approaches over 19 domains and provide preliminary results on multiple annotated question-answering datasets to deal with question-answering similarity task. Keywords: Community question answering, textual similarity, word embeddings 1. Introduction The increasing popularity of question answering websites has led to the emergence of a new area of research called community question answering, which has to deal with two distinct but complementary tasks. The first task, called question-to-question similarity, has to provide related ques- tions to a given original question. The identification of similar question pairs aims at preventing duplicate posts in the forums and to redirect users towards posts that might contain an appropriate answer. The second task, called question-answering similarity, aims at providing a correct answer to a given original question. If several users con- tribute to a given post, it is important to automatically ex- tract the correct answers among tens or hundreds of an- swers since a manual exploration becomes hard to achieve. These tasks offer a key challenge while they have to deal with textual similarity not only in terms of lexical similarity but also in terms of reformulation, paraphrasing, duplicates and near duplicates, textual entailment, semantics, etc. Over the past years, there have been several studies on com- munity question answering (Qiu and Huang, 2015; Filice et al., 2016; Barr´on-Cede˜no et al., 2016; Franco-Salvador et al., 2016; Nakov et al., 2016; Nakov et al., 2017; Pa- tra, 2017), etc. Most of them addressed this task through specific datasets such as the programming Q&A website Stackoverflow 1, Quora dataset for duplicate extraction 2, Yahoo!Answers dataset (Qiu and …

Content truncated to 500 words