Semantic Equivalence Detection: Are Interrogatives Harder than Declaratives?

Semantic Equivalence Detection: Are Interrogatives Harder than Declaratives? Jo˜ao Rodrigues, Chakaveh Saedi, Ant´onio Branco and Jo˜ao Silva University of Lisbon NLX-Natural Language and Speech Group, Department of Informatics Faculdade de Ciˆencias Campo Grande, 1749-016 Lisboa, Portugal {joao.rodrigues, chakaveh.saedi, antonio.branco, jsilva}@di.fc.ul.pt Abstract Duplicate Question Detection (DQD) is a Natural Language Processing task under active research, with applications to fields like Community Question Answering and Information Retrieval. While DQD falls under the umbrella of Semantic Text Similarity (STS), these are often not seen as similar tasks of semantic equivalence detection, with STS being implicitly understood as concerning only declarative sentences. Nevertheless, approaches to STS have been applied to DQD and paraphrase detection, that is to interrogatives and declaratives, alike. We present a study that seeks to assess, under conditions of comparability, the possible different performance of state-of-the-art approaches to STS over different types of textual segments, including most notably declaratives and interrogatives. This paper contributes to a better understanding of current mainstream methods for semantic equivalence detection, and to a better appreciation of the different results reported in the literature when these are obtained from different data sets with different types of textual segments. Importantly, it contributes also with results concerning how data sets containing textual segments of a certain type can be used to leverage the performance of resolvers for segments of other types. Keywords: semantic text similarity, paraphrase detection, duplicate question detection 1. Introduction Semantic Text Similarity (STS) is a Natural Language Pro- cessing (NLP) task whereby a system, given two input text segments, assigns to them a similarity score in a dis- crete or continuous scale that ranges from representing total similarity—for semantically equivalent segments—to rep- resenting total dissimilarity—for segments that are seman- tically independent. The STS task has been part of the SemEval competitive shared tasks since 2012 (Agirre et al., 2012), together with other challenges for a wide variety of other tasks, such as plagiarism detection, sentiment analysis or relation extrac- tion, to name but a few. More recently, SemEval embraced STS challenges that concern more focused tasks, like para- phrase detection, which consists of a binary decision on whether two input sentences are paraphrases of each other and, starting in 2016, a task on Duplicate Question Detec- tion (DQD) (Nakov et al., 2016). DQD appears as a special case of paraphrase detection, where the focus is on interrogative sentences: this task con- sists of a binary decision on whether two input interroga- tives sentences are a duplicate of each other. The motivation for the increasing interest in DQD, and the inclusion in SemEval of challenges dedicated to DQD, comes from the increasing popularity of on-line Commu- nity Question Answering (CQA) forums, such as Stack Ex- change1 or Quora2. These forums are quite open in allow- ing any user to post questions (and answer questions from other users) but from this arises a potential problem that may eventually affect the effectiveness of these on-line ser- vices, namely that many posted questions are duplicates of questions already answered. …

Content truncated to 500 words