Predicting Correlations Between Lexical Alignments and Semantic Inferences

Proceedings of Recent Advances in Natural Language Processing, pages 388–397, Hissar, Bulgaria, Sep 7–9 2015. Predicting Correlations Between Lexical Alignments and Semantic Inferences Simone Magnolini University of Brescia FBK, Trento, Italy magnolini@fbk.eu Bernardo Magnini FBK, Trento, Italy magnini@fbk.eu Abstract While there is a strong intuition that word alignments (e.g. synonymy, hyperonymy) play a relevant role in recognizing text- to-text semantic inferences (e.g. textual entailment, semantic similarity), this in- tuition is often not reflected in the sys- tem performances and there is a general need of a deeper comprehension of the role of lexical resources. This paper provides an empirical analysis of the dependen- cies between data-sets, lexical resources and algorithms that are commonly used in text-to-text inference tasks. We define a resource impact index, based on lexi- cal alignments between pairs of texts, and show that such index is significantly cor- related with the performance of different textual entailment algorithms. The result is an operational, algorithm-independent, procedure for predicting the performance of a class of available RTE algorithms. 1 Introduction In the last decade text-to-text semantic inference has been a relevant topic in Computational Lin- guistics. Driven by the assumption that language understanding crucially depends on the ability to recognize semantic relations among portions of text, several text-to-text inference tasks have been proposed, including recognizing paraphrasing (Dolan and Brockett., 2005), recognizing textual entailment (RTE) (Dagan et al., 2005), and se- mantic similarity (Agirre et al., 2012). A common characteristic of such tasks is that the input are two portions of text, let’s call them Text1 and Text2, and the output is a semantic relation between the two texts, possibly with a degree of confidence of the system. For instance, given the following text fragments: Text1: George Clooneys longest relationship ever might have been with a pig. The actor owned Max, a 300-pound pig. Text2: Max is an animal. a system should be able to recognize that there is an ”entailment” relation among Text1 and Text2. While the task is very complex, requiring in principle to consider syntax, semantics and also pragmatics, current systems adopt rather sim- plified techniques, based on available linguistic resources. For instance, many RTE systems (Da- gan et al., 2012) would attempt to take advantage of the fact that, according to WordNet, the word animal in Text2 is a hypernym of the word pig in Text1. A relevant aspect in text-to-text tasks is that data-sets are usually composed of textual pairs for positive cases, where a certain relation (e.g. entailment) holds, and negative pairs, where the semantic relation does not hold. For instance, the following pair: Text1: John has a cat, named Felix, in his farm, it’s a Maine Coon, it’s the largest domesticated breed of cat. Text2: Felix is the largest domesticated animal in John’s farm. shows a case of ”non-entailment”. It is worth to notice that in both the examples, although the en- tailment judgment is different, still there is an high degree of lexical alignments between words in Text1 and Text2 (e.g. Max −→ …

Content truncated to 500 words