We Understand Elliptical Sentences, and Language Models should Too: A New Dataset for Studying Ellipsis and its Interaction with Thematic Fit

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3340–3353 July 9-14, 2023 ©2023 Association for Computational Linguistics We Understand Elliptical Sentences, and Language Models Should Too: A New Dataset for Studying Ellipsis and its Interaction with Thematic Fit Davide Testa University of Pisa d.testa6@studenti.unipi.it Emmanuele Chersoni The Hong Kong Polytechnic University emmanuele.chersoni@polyu.edu.hk Alessandro Lenci University of Pisa alessandro.lenci@unipi.it Abstract Ellipsis is a linguistic phenomenon character- ized by the omission of one or more sentence el- ements. Solving such a linguistic construction is not a trivial issue in natural language process- ing since it involves the retrieval of non-overtly expressed verbal material, which might in turn require the model to integrate human-like syn- tactic and semantic knowledge. In this paper, we explored the issue of how the prototypicality of event participants affects the ability of Lan- guage Models (LMs) to handle elliptical sen- tences, and to identify the omitted arguments at different degrees of thematic fit, ranging from highly typical participants to semantically anomalous ones. With this purpose in mind, we built ELLie, the first dataset composed entirely of utterances containing different types of ellip- tical constructions, and structurally suited for evaluating the effect of argument thematic fit in solving ellipsis and reconstructing the miss- ing element. Our tests demonstrated that the probability scores assigned by the models are higher for typical events than for atypical and impossible ones in different elliptical contexts, confirming the influence of prototypicality of the event participants in interpreting such lin- guistic structures. Finally, we conducted a re- trieval task of the elided verb in the sentence in which the low performance of LMs highlighted a considerable difficulty in reconstructing the correct event. 1 Introduction A key phenomenon of natural languages is ellipsis, the omission of a word or phrase that is expected to occupy a place in the syntactic structure of a sentence (McShane, 2005).1 Elliptical sentences are usually composed of a standard sentence (aka antecedent clause) and an elliptical clause, which is not fully propositional and apparently not well- formed from a syntactic point of view (Culicover 1Literature tends to distinguish between syntactic and se- mantic ellipsis. Here we focus on the former type. and Jackendoff, 2005). Consider the following ex- ample, where the antecedent is underlined and the elliptical one is characterized by the verb omission: (1) The engineer completed the project, but the student didn’t. Since ellipsis represents a deviation from the sim- ple compositional mapping between form and meaning, elliptical sentences have been the focus of many studies that seek to investigate how ellipsis is mentally represented, how the interpretation of the elided material is recovered, and consequently, how meaning can arise in the absence of form (Ginzburg and Sag, 2000; Schwabe and Winkler, 2003; Culi- cover and Jackendoff, 2006; Jacobson, 2012; Mer- chant, 2013, 2018; van Craenenbroeck and Tem- merman, 2018). Over the years, such theoretical discussions have proven the presence of a structural parallelism between the two sentence components through …

Content truncated to 500 words