FBK-TR: SVM for Semantic Relatedeness and Corpus Patterns for RTE

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 289–293, Dublin, Ireland, August 23-24, 2014. FBK-TR: SVM for Semantic Relatedness and Corpus Patterns for RTE Ngoc Phuoc An Vo Fondazione Bruno Kessler University of Trento Trento, Italy ngoc@fbk.eu Octavian Popescu Fondazione Bruno Kessler Trento, Italy popescu@fbk.eu Tommaso Caselli TrentoRISE Trento, Italy t.caselli@trentorise.eu Abstract This paper reports the description and scores of our system, FBK-TR, which participated at the SemEval 2014 task #1 "Evaluation of Compositional Distribu- tional Semantic Models on Full Sentences through Semantic Relatedness and Entail- ment". The system consists of two parts: one for computing semantic relatedness, based on SVM, and the other for identi- fying the entailment values on the basis of both semantic relatedness scores and entailment patterns based on verb-specific semantic frames. The system ranked 11th on both tasks with competitive results. 1 Introduction In the Natural Language Processing community, meaning related tasks have gained an increasing popularity. These tasks focus, in general, on a couple of short pieces of text, like pair of sen- tences, and the systems are required to infer a cer- tain meaning relationship that exists between these texts. Two of the most popular meaning related tasks are the identification of Semantic Text Sim- ilarity (STS) and Recognizing Textual Entailment (RTE). The STS tasks require to identify the de- gree of similarity (or relatedness) that exists be- tween two text fragments (sentences, paragraphs, . . . ), where similarity is a broad concept and its value is normally obtained by averaging the opin- ion of several annotators. The RTE task requires the identification of a directional relation between a pair of text fragments, namely a text (T) and a hypothesis (H). The relation (T → H) holds when- ever the truth of H follows from T. This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ At SemEval 2014, the Task #1 "Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Entailment" (Marelli et al., 2014a) primarily aimed at evaluating Compositional Distributional Semantic Models (CDSMs) of meaning over two subtasks, namely semantic relatedness and tex- tual entailment (ENTAILMENT, CONTRADIC- TION and NEUTRAL), over pairs of sentences (Marelli et al., 2014b). Concerning the relatedness subtask, the system outputs are evaluated against gold standard ratings in two ways, using Pearson correlation and Spearman’s rank correlation (rho). The Pearson correlation is used for evaluating and ranking the participating systems. Similarly, for the textual entailment subtask, system outputs are evaluated against a gold standard rating with re- spect to accuracy. Our team, FBK-TR, participated in both sub- tasks with five different runs. In this paper, we present a comprehensive description of our system which obtained competitive results in both tasks and which is not based on CDSMs. Our approach for the relatedness task is based on machine learn- ing techniques to learn models from different lexi- cal and semantic features from …

Content truncated to 500 words