Modeling Semantic Containment and Exclusion in Natural Language Inference

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 521–528 Manchester, August 2008 Modeling Semantic Containment and Exclusion in Natural Language Inference Bill MacCartney Stanford University wcmac@cs.stanford.edu Christopher D. Manning Stanford University manning@cs.stanford.edu Abstract We propose an approach to natural lan- guage inference based on a model of nat- ural logic, which identifies valid infer- ences by their lexical and syntactic fea- tures, without full semantic interpretation. We greatly extend past work in natural logic, which has focused solely on seman- tic containment and monotonicity, to in- corporate both semantic exclusion and im- plicativity. Our system decomposes an in- ference problem into a sequence of atomic edits linking premise to hypothesis; pre- dicts a lexical entailment relation for each edit using a statistical classifier; propagates these relations upward through a syntax tree according to semantic properties of in- termediate nodes; and composes the result- ing entailment relations across the edit se- quence. We evaluate our system on the FraCaS test suite, and achieve a 27% re- duction in error from previous work. We also show that hybridizing an existing RTE system with our natural logic system yields significant gains on the RTE3 test suite. 1 Introduction A necessary (if not sufficient) condition for true natural language understanding is a mastery of open-domain natural language inference (NLI): the task of determining whether a natural-language hypothesis can be inferred from a given premise. Indeed, NLI can enable more immediate applica- tions, such as semantic search and question an- c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. swering (Harabagiu and Hickl, 2006). In recent years a spectrum of approaches to robust, open- domain NLI have been explored within the con- text of the Recognizing Textual Entailment chal- lenge (Dagan et al., 2005). Up to now, the most successful approaches have used fairly shallow semantic representations, relying on measures of lexical or semantic overlap (Jijkoun and de Ri- jke, 2005), pattern-based relation extraction (Ro- mano et al., 2006), or approximate matching of predicate-argument structure (Hickl et al., 2006). Such methods, while robust and often effective, are at best partial solutions, unable to explain even simple forms of logical inference. For example, most shallow approaches would fail to license the introduction of large in the following example: (1) Every firm saw costs grow more than expected, even after adjusting for inflation. Every large firm saw costs grow. At the other extreme, some researchers have ap- proached NLI as logical deduction, building on work in theoretical semantics to translate sentences into first-order logic (FOL), and then applying a theorem prover or model builder (Akhmatova, 2005; Fowler et al., 2005). Regrettably, such ap- proaches tend to founder on the myriad complexi- ties of full semantic interpretation, including tense, aspect, causality, intensionality, modality, vague- ness, idioms, indexicals, ellipsis, and many other issues. (What is the right FOL representation of (1), for example?) FOL-based systems that have attained high precision (Bos and Markert, 2006) have done so …

Content truncated to 500 words