ECNU: One Stone Two Birds: Ensemble of Heterogenous Measures for Semantic Relatedness and Textual Entailment

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 271–277, Dublin, Ireland, August 23-24, 2014. ECNU: One Stone Two Birds: Ensemble of Heterogenous Measures for Semantic Relatedness and Textual Entailment Jiang Zhao, Tian Tian Zhu, Man Lan∗ Department of Computer Science and Technology East China Normal University 51121201042,51111201046@ecnu.cn; mlan@cs.ecnu.edu.cn∗ Abstract This paper presents our approach to se- mantic relatedness and textual entailment subtasks organized as task 1 in SemEval 2014. Specifically, we address two ques- tions: (1) Can we solve these two sub- tasks together? (2) Are features proposed for textual entailment task still effective for semantic relatedness task? To address them, we extracted seven types of features including text difference measures pro- posed in entailment judgement subtask, as well as common text similarity measures used in both subtasks. Then we exploited the same feature set to solve the both sub- tasks by considering them as a regression and a classification task respectively and performed a study of influence of differ- ent features. We achieved the first and the second rank for relatedness and entailment task respectively. 1 Introduction Distributional Semantic Models (DSMs)(surveyed in (Turney et al., 2010)) exploit the co-occurrences of other words with the word being modeled to compute the semantic meaning of the word un- der the distributional hypothesis: “similar words share similar contexts” (Harris, 1954). Despite their success, DSMs are severely limited to model the semantic of long phrases or sentences since they ignore grammatical structures and logical words. Compositional Distributional Semantic Models (CDSMs)(Zanzotto et al., 2010; Socher et This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ al., 2012) extend DSMs to sentence level to cap- ture the compositionality in the semantic vector space, which has seen a rapidly growing interest in recent years. Although several CDSMs have been proposed, benchmarks are lagging behind. Previous work (Grefenstette and Sadrzadeh, 2011; Socher et al., 2012) performed experiments on their own datasets or on the same datasets which are limited to a few hundred instances of very short sentences with a fixed structure. To provide a benchmark so as to compare dif- ferent CDSMs, the sentences involving composi- tional knowledge task in SemEval 2014 (Marelli et al., 2014) develops a large dataset which is full of lexical, syntactic and semantic phenomena. It con- sists of two subtasks: semantic relatedness task, which measures the degree of semantic relatedness of a sentence pair by assigning a relatedness score ranging from 1 (completely unrelated) to 5 (very related); and textual entailment (TE) task, which determines whether one of the following three re- lationships holds between two given sentences A and B: (1) entailment: the meaning of B can be inferred from A; (2) contradiction: A contradicts B; (3) neutral: the truth of B cannot be inferred on the basis of A. Semantic textual similarity (STS) (Lintean and Rus, 2012) and semantic relatedness are closely related and interchangeably used in …

Content truncated to 500 words