Scoring and Classifying Implicit Positive Interpretations: A Challenge of Class Imbalance

Proceedings of the 27th International Conference on Computational Linguistics, pages 2253–2264 Santa Fe, New Mexico, USA, August 20-26, 2018. 2253 Scoring and Classifying Implicit Positive Interpretations: A Challenge of Class Imbalance Chantal van Son, Roser Morante, Lora Aroyo, Piek Vossen Vrije Universiteit Amsterdam De Boelelaan 1105, 1081 HV Amsterdam The Netherlands {c.m.van.son, r.morantevallejo, lora.aroyo, piek.vossen}@vu.nl Abstract This paper reports on a reimplementation of a system on detecting implicit positive meaning from negated statements. In the original regression experiment, different positive interpretations per negation are scored according to their likelihood. We convert the scores to classes and report our results on both the regression and classification tasks. We show that a baseline taking the mean score or most frequent class is hard to beat because of class imbalance in the dataset. Our error analysis indicates that an approach that takes the information structure into account (i.e. which information is new or contrastive) may be promising, which requires looking beyond the syntactic and semantic characteristics of negated statements. 1 Introduction Modeling inference is one of the most challenging tasks in computational linguistics, yet crucial for true natural language understanding. It requires looking beyond the literal meaning of statements and determining what exactly the author intends to convey. The position of focus (the new or contrastive information) in a sentence, for instance, is one pragmatic phenomenon that can affect its interpretation. The theory of alternative semantics (Rooth, 1992) assumes that the general function of focus is to evoke alternatives, i.e. a set of propositions potentially contrasting with the intended meaning of the sentence. A number of lexical items and constructions have been identified as focus-sensitive in English, such as only, even, too, counterfactual conditionals (if), frequency adverbs (always, sometimes, often) and negation. Their semantics depend on the information structure of the sentence; different choices of focus can result in different truth values. For example, Kawamura (2007) illustrates how focus contributes to the interpretation of negation in Sentence 1 below. In English, focus can be explicitly signaled prosodically, e.g. in the form of a strong pitch accent, or syntactically, e.g. by moving focused phrases to a special position in the sentence (Stevens, 2017). If no explicit signal is present, one has to rely on the context to determine which part of the sentence is in focus and, hence, what is its correct interpretation. This is a notoriously difficult task that so far has received little attention in the field. 1. (a) We do not cover PRAGMATICS in the lecture this semester. the lecture of this semester covers other fields of linguistics, but not pragmatics (b) We do not cover pragmatics IN THE LECTURE this semester. pragmatics is covered in the discussion sessions, homework, and exams, but not in the lecture (c) We do not cover pragmatics in the lecture THIS SEMESTER. the lecture usually covers pragmatics, but not this semester This paper reports on a reproduction study for the purpose of understanding the performance of a system on detecting implicit positive meaning from negated …

Content truncated to 500 words