A Phrase-Based Alignment Model for Natural Language Inference

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 802–811, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics A Phrase-Based Alignment Model for Natural Language Inference Bill MacCartney, Michel Galley, Christopher D. Manning Natural Language Processing Group, Stanford University {wcmac,mgalley,manning}@stanford.edu Abstract The alignment problem—establishing links between corresponding phrases in two related sentences—is as important in natural language inference (NLI) as it is in machine transla- tion (MT). But the tools and techniques of MT alignment do not readily transfer to NLI, where one cannot assume semantic equiva- lence, and for which large volumes of bitext are lacking. We present a new NLI aligner, the MANLI system, designed to address these challenges. It uses a phrase-based alignment representation, exploits external lexical re- sources, and capitalizes on a new set of su- pervised training data. We compare the per- formance of MANLI to existing NLI and MT aligners on an NLI alignment task over the well-known Recognizing Textual Entailment data. We show that MANLI significantly out- performs existing aligners, achieving gains of 6.2% in F1 over a representative NLI aligner and 10.5% over GIZA++. 1 Introduction The problem of natural language inference (NLI) is to determine whether a natural-language hypothesis H can reasonably be inferred from a given premise text P. In order to recognize that Kennedy was killed can be inferred from JFK was assassinated, one must first recognize the correspondence between Kennedy and JFK, and between killed and assas- sinated. Consequently, most current approaches to NLI rely, implicitly or explicitly, on a facility for alignment—that is, establishing links between cor- responding entities and predicates in P and H. Re- cent entries in the annual Recognizing Textual En- tailment (RTE) competition (Dagan et al., 2005) have addressed the alignment problem in a variety of ways, though often without distinguishing it as a separate subproblem. Glickman et al. (2005) and Jijkoun and de Rijke (2005), among others, have ex- plored approaches based on measuring the degree of lexical overlap between bags of words. While ig- noring structure, such methods depend on matching each word in H to the word in P with which it is most similar—in effect, an alignment. At the other extreme, Tatu and Moldovan (2007) and Bar-Haim et al. (2007) have formulated the inference problem as analogous to proof search, using inferential rules which encode (among other things) knowledge of lexical relatedness. In such approaches, the corre- spondence between the words of P and H is implicit in the steps of the proof. Increasingly, however, the most successful RTE systems have made the alignment problem explicit. Marsi and Krahmer (2005) and MacCartney et al. (2006) first advocated pipelined system architec- tures containing a distinct alignment component, a strategy crucial to the top-performing systems of Hickl et al. (2006) and Hickl and Bensley (2007). However, each of these systems has pursued align- ment in idiosyncratic and poorly-documented ways, often using proprietary data, making comparisons and further development difficult. In this paper we undertake the first systematic …

Content truncated to 500 words