A Probabilistic Lexical Model for Ranking Textual Inferences

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 237–245, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics A Probabilistic Lexical Model for Ranking Textual Inferences Eyal Shnarch and Ido Dagan Computer Science Department Bar-Ilan University Ramat-Gan 52900, Israel {shey,dagan}@cs.biu.ac.il Jacob Goldberger Faculty of Engineering Bar-Ilan University Ramat-Gan 52900, Israel goldbej@eng.biu.ac.il Abstract Identifying textual inferences, where the meaning of one text follows from another, is a general underlying task within many natu- ral language applications. Commonly, it is ap- proached either by generative syntactic-based methods or by “lightweight” heuristic lexical models. We suggest a model which is confined to simple lexical information, but is formu- lated as a principled generative probabilistic model. We focus our attention on the task of ranking textual inferences and show substan- tially improved results on a recently investi- gated question answering data set. 1 Introduction The task of identifying texts which share semantic content arises as a general need in many natural lan- guage processing applications. For instance, a para- phrasing application has to recognize texts which convey roughly the same content, and a summariza- tion application needs to single out texts which con- tain the content stated by other texts. We refer to this general task as textual inference similar to prior use of this term (Raina et al., 2005; Schoenmackers et al., 2008; Haghighi et al., 2005). In many textual inference scenarios the setting re- quires a classification decision of whether the infer- ence relation holds or not. But in other scenarios ranking according to inference likelihood would be the natural task. In this work we focus on ranking textual inferences; given a sentence and a corpus, the task is to rank the corpus passages by their plau- sibility to imply as much of the sentence meaning as possible. Most naturally, this is the case in question answering (QA), where systems search for passages that cover the semantic components of the question. A recent line of research was dedicated to this task (Wang et al., 2007; Heilman and Smith, 2010; Wang and Manning, 2010). A related scenario is the task of Recognizing Tex- tual Entailment (RTE) within a corpus (Bentivogli et al., 2010)1. In this task, inference systems should identify, for a given hypothesis, the sentences which entail it in a given corpus. Even though RTE was presented as a classification task, it has an appeal- ing potential as a ranking task as well. For instance, one may want to find texts that validate a claim such as cellular radiation is dangerous for children, or to learn more about it from a newswire corpus. To that end, one should look for additional mentions of this claim such as extensive usage of cell phones may be harmful for youngsters. This can be done by rank- ing the corpus passages by their likelihood to entail the claim, where the top ranked passages are likely to contain additional relevant information. Two main approaches have been used to address textual inference (for either ranking …

Content truncated to 500 words