Natural Language Inference from Multiple Premises

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 100–109, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Natural Language Inference from Multiple Premises Alice Lai1 Yonatan Bisk2 Julia Hockenmaier1 1Department of Computer Science, University of Illinois at Urbana-Champaign 2Paul G. Allen School of Computer Science & Engineering, Univ. of Washington aylai2@illinois.edu, ybisk@cs.washington.edu, juliahmr@illinois.edu Abstract We define a novel textual entailment task that requires inference over mul- tiple premise sentences. We present a new dataset for this task that mini- mizes trivial lexical inferences, empha- sizes knowledge of everyday events, and presents a more challenging setting for textual entailment. We evaluate several strong neural baselines and analyze how the multiple premise task differs from standard textual entailment. 1 Introduction Standard textual entailment recognition is con- cerned with deciding whether one statement (the hypothesis) follows from another statement (the premise). However, in some situations, multiple independent descriptions of the same event are available, e.g. multiple news articles describing the same story, social media posts by different people about a single event, or multiple witness reports for a crime. In these cases, we want to use multiple independent reports to infer what really happened. We therefore introduce a variant of the standard textual entailment task in which the premise text consists of multiple independently written sen- tences, all describing the same scene (see exam- ples in Figure 1). The task is to decide whether the hypothesis sentence 1) can be used to describe the same scene (entailment), 2) cannot be used to describe the same scene (contradiction), or 3) may or may not describe the same scene (neutral). The main challenge is to infer what happened in the scene from the multiple premise statements, in some cases aggregating information across multi- ple sentences into a coherent whole. Premises: 1. Two girls sitting down and looking at a book. 2. A couple laughs together as they read a book on a train. 3. Two travelers on a train or bus reading a book together. 4. A woman wearing glasses and a brown beanie next to a girl with long brown hair holding a book. Hypothesis: Women smiling. ⇒ENTAILMENT Premises: 1. Three men are working construction on top of a building. 2. Three male construction workers on a roof working in the sun. 3. One man is shirtless while the other two men work on construction. 4. Two construction workers working on infrastructure, while one worker takes a break. Hypothesis: A man smoking a cigarette. ⇒NEUTRAL Premises: 1. A group of individuals performed in front of a seated crowd. 2. Woman standing in front of group with black folders in hand. 3. A group of women with black binders stand in front of a group of people. 4. A group of people are standing at the front of the room, preparing to sing. Hypothesis: A group having a meeting. ⇒CONTRADICTION Figure 1: The Multiple Premise Entailment Task Similar to the SICK and SNLI datasets (Marelli et al., 2014; …

Content truncated to 500 words