NOPE: A Corpus of Naturally-Occurring Presuppositions in English

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 349–366 November 10–11, 2021. ©2021 Association for Computational Linguistics 349 1 NOPE: A Corpus of Naturally-Occurring Presuppositions in English Alicia Parrish∗ Sebastian Schuster∗ Alex Warstadt∗ Omar Agha Soo-Hwan Lee Zhuoye Zhao Samuel R. Bowman Tal Linzen Department of Linguistics & Center for Data Science, New York University {alicia.v.parrish,schuster,warstadt}@nyu.edu Abstract Understanding language requires grasping not only the overtly stated content, but also mak- ing inferences about things that were left unsaid. These inferences include presuppo- sitions, a phenomenon by which a listener learns about new information through reason- ing about what a speaker takes as given. Pre- suppositions require complex understanding of the lexical and syntactic properties that trigger them as well as the broader conver- sational context. In this work, we intro- duce the Naturally-Occurring Presuppositions in English (NOPE) Corpus to investigate the context-sensitivity of 10 different types of pre- supposition triggers and to evaluate machine learning models’ ability to predict human in- ferences. We fnd that most of the trig- gers we investigate exhibit moderate variabil- ity. We further fnd that transformer-based models draw correct inferences in simple cases involving presuppositions, but they fail to cap- ture the minority of exceptional cases in which human judgments reveal complex interactions between context and triggers. Introduction In every statement, certain facts are taken for granted by the speaker; such facts, while left un- said, can often be inferred by the listener. A speaker who says Chet fnished law school, for example, asserts that Chet reached the end of law school, and presupposes that he attended law school in the frst place. While such presuppositions are usually not the main information that a speaker intends to convey, listeners can still learn new information from them by drawing inferences about what the speaker takes as a given. One type of signal that helps human listeners draw such inferences is presupposition triggers, such as the verb fnish. Triggers have long been ∗ Equal contribution. Context Sensitivity / Cancelability ...He didn’t finish law school. Gradience Chet didn’t finish the last year of law school. Unsupportive context Negated trigger sent. Projection out of negation Chet finished law school. Trigger sent. Negated trigger sent. Chet didn’t finish law school. Presupposition Supportive context Presupposition Soft presupposition Chet attended law school. Relatively weak trigger Chet attended law school. presupposes presupposes Followed by Followed by presupposes Doesn’t presuppose Presupposes relatively weakly Chet never became a lawyer... Chet just finished med school... Relatively strong trigger Chet didn’t finish law school. Presupposes relatively strongly Chet attended law school. Figure 1: Inferences involving presuppositions can be challenging because presuppositions project out of negation (frst box), they are context-sensitive and can be canceled (second box), and they can differ in how easily they can be cancelled and how likely a listener is to make the inference (third box). known in linguistics to be associated with presuppo- sitions, but the nature of this relationship is a matter of much debate (e.g., Heim, 1983; …

Content truncated to 500 words