SimBow at SemEval-2017 Task 3: Soft-Cosine Semantic Similarity between Questions for Community Question Answering

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 315–319, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics SimBow at SemEval-2017 Task 3: Soft-Cosine Semantic Similarity between Questions for Community Question Answering Delphine Charlet and G´eraldine Damnati Orange Labs Lannion, France delphine.charlet,geraldine.damnati@orange.com Abstract This paper describes the SimBow sys- tem submitted at SemEval2017-Task3, for the question-question similarity subtask B. The proposed approach is a supervised combination of different unsupervised tex- tual similarities. These textual similarities rely on the introduction of a relation ma- trix in the classical cosine similarity be- tween bag-of-words, so as to get a soft- cosine that takes into account relations be- tween words. According to the type of re- lation matrix embedded in the soft-cosine, semantic or lexical relations can be con- sidered. Our system ranked first among the official submissions of subtask B. 1 Introduction Social networks enable people to post questions, and to interact with other people to obtain relevant answers. The popularity of forums show that they are able to propose reliable answers. Due to this tremendous popularity, forums are growing fast, and the first reflex for an internet user is to check with his favorite search engine if a similar ques- tion has already been posted.Community Question Answering at SemEval focuses on this task, with 3 different subtasks. SubtaskA (resp. subtaskC) aims at re-ranking the comments of one original question (resp. the comments of a set of 10 re- lated questions), regarding the relevancy to the original questions. SubtaskB aims at re-ranking 10 related questions proposed by a search en- gine, regarding the relevancy to the original ques- tion. Subtasks A and C are question-answering tasks. SubtaskB can be viewed as a pure seman- tic textual similarity task applied on community questions, with noisy user-generated texts, mak- ing it different from SemEval-Task1 (Agirre et al., 2016), which focuses on semantic similarity be- tween short well-formed sentences. In this paper, we only focus on subtaskB, with the purpose of developing semantic textual sim- ilarity measures for such noisy texts. Question- question similarity appeared in SemEval2016 (Nakov et al., 2016), and is pursued in Se- mEval2017 (Nakov et al., 2017). The approaches explored last year were mostly supervised fusion of different similarity measures, some being un- supervised, others supervised. Among the un- supervised measures, many were based on over- lap count between components (from n-grams of words or characters to knowledge-based compo- nents such as named entities, frame representa- tions, knowledge graphs, e.g. (Franco-Salvador et al., 2016)...). Much attention was also paid for the use of word embeddings (e.g. (Mihaylov and Nakov, 2016)), with question-level averaged vec- tors used directly with a cosine similarity or as in- put of a neural classifier. Finally, fusion was often performed with SVMs (Filice et al., 2016) Our motivation in this work was slightly dif- ferent: we considered that forum data were too noisy to get reliable outputs from linguistic anal- ysis and we wanted to focus on core textual se- mantic …

Content truncated to 500 words