SAGAN: An approach to Semantic Textual Similarity based on Textual Entailment

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 667–672, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics SAGAN: An approach to Semantic Textual Similarity based on Textual Entailment Julio Castillo†‡ Paula Estrella‡ ‡FaMAF, UNC, Argentina †UTN-FRC, Argentina jotacastillo@gmail.com pestrella@famaf.unc.edu.ar Abstract In this paper we report the results obtained in the Semantic Textual Similarity (STS) task, with a system primarily developed for textual entailment. Our results are quite promising, getting a run ranked 39 in the official results with overall Pearson, and ranking 29 with the Mean metric. 1 Introduction For the last couple of years the research com- munity has focused on a deeper analysis of natural languages, seeking to capture the meaning of the text in different contexts: in machine translation preserving the meaning of the translations is cru- cial to determine whether a translation is useful or not, in question-answering understanding the ques- tion leads to the desired answers (while the oppo- site case makes a system rather frustrating to the user) and the examples could continue. In this newly defined task, Semantic Textual Similarity, there is hope that efforts in different areas will be shared and united towards the goal of identifying meaning and recognizing equivalent, similar or unrelated texts. Our contribution to the task, is from a textual entailment point of view, as will be described below. The paper is organized as follows: Section 2 de- scribes the relevant tasks, Section 3 describes the architecture of the system, then Section 4 shows the experiments carried out and the results ob- tained, and Section 5 presents some conclusions and future work. 2 Related work In this section we briefly describe two different tasks that are closely related and in which our sys- tem has participated with very promising results. 2.1 Textual Entailment Textual Entailment (TE) is defined as a generic framework for applied semantic inference, where the core task is to determine whether the meaning of a target textual assertion (hypothesis, H) can be inferred from a given text (T). For example, given the pair (T,H): T: Fire bombs were thrown at the Tunisian embas- sy in Bern H: The Tunisian embassy in Switzerland was at- tacked we can conclude that T entails H. The recently created challenge “Recognising Textual Entailment” (RTE) started in 2005 with the goal of providing a binary answer for each pair (H,T), namely whether there is entailment or not (Dagan et al., 2006). The RTE challenge has mu- tated over the years, aiming at accomplishing more 667 accurate and specific solutions; for example, in 2008 a three-way decision was proposed (instead of the original binary decision) consisting of “en- tailment”, “contradiction” and “unknown”; in 2009 the organizers proposed a pilot task, the Textual Entailment Search (Bentivogli et al, 2009), consist- ing in finding all the sentences in a set of docu- ments that entail a given Hypothesis and since 2010 there is a Novelty Detection Task, which means that RTE systems are required to judge whether the information …

Content truncated to 500 words