Combining Natural Logic and Shallow Reasoning for Question Answering

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 442–452, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Combining Natural Logic and Shallow Reasoning for Question Answering Gabor Angeli Stanford University Stanford, CA 94305 Neha Nayak Stanford University Stanford, CA 94305 Christopher D. Manning Stanford University Stanford, CA 94305 {angeli, nayakne, manning}@cs.stanford.edu Abstract Broad domain question answering is of- ten difficult in the absence of structured knowledge bases, and can benefit from shallow lexical methods (broad coverage) and logical reasoning (high precision). We propose an approach for incorporating both of these signals in a unified frame- work based on natural logic. We extend the breadth of inferences afforded by nat- ural logic to include relational entailment (e.g., buy → own) and meronymy (e.g., a person born in a city is born the city’s country). Furthermore, we train an eval- uation function – akin to gameplaying – to evaluate the expected truth of candidate premises on the fly. We evaluate our ap- proach on answering multiple choice sci- ence questions, achieving strong results on the dataset. 1 Introduction Question answering is an important task in NLP, and becomes both more important and more diffi- cult when the answers are not supported by hand- curated knowledge bases. In these cases, view- ing question answering as textual entailment over a very large premise set can offer a means of gen- eralizing reliably to open domain questions. A natural approach to textual entailment is to treat it as a logical entailment problem. How- ever, this high-precision approach is not feasible in cases where a formal proof is difficult or impossi- ble. For example, consider the following hypothe- sis (H) and its supporting premise (P) for the ques- tion Which part of a plant produces the seeds?: P: Ovaries are the female part of the flower, which pro- duces eggs that are needed for making seeds. H: A flower produces the seeds. This requires a relatively large amount of infer- ence: the most natural atomic fact in the sentence is that ovaries produce eggs. These inferences are feasible in a limited domain, but become difficult the more open-domain reasoning they require. In contrast, even a simple lexical overlap classifier could correctly predict the entailment. In fact, such a bag-of-words entailment model has been shown to be surprisingly effective on the Recog- nizing Textual Entailment (RTE) challenges (Mac- Cartney, 2009). On the other hand, such methods are also notorious for ignoring even trivial cases of nonentailment that are easy for natural logic, e.g., recognizing negation in the example below: P: Eating candy for dinner is an example of a poor health habit. H: Eating candy is an example of a good health habit. We present an approach to leverage the bene- fits of both methods. Natural logic – a proof the- ory over the syntax of natural language – offers a framework for logical inference which is already familiar to lexical methods. As an inference sys- tem searches for a …

Content truncated to 500 words