Comparing Probabilistic, Distributional and Transformer-Based Models on Logical Metonymy Interpretation

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 224–234 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 224 Comparing Probabilistic, Distributional and Transformer-Based Models on Logical Metonymy Interpretation Giulia Rambelli University of Pisa giulia.rambelli@phd.unipi.it Emmanuele Chersoni Hong Kong Polytechnic University emmanuelechersoni@gmail.com Alessandro Lenci University of Pisa alessandro.lenci@unipi.it Philippe Blache Aix-Marseille University blache@lpl-aix.fr Chu-Ren Huang Hong Kong Polytechnic University churen.huang@polyu.edu.hk Abstract In linguistics and cognitive science, Logical metonymies are defined as type clashes be- tween an event-selecting verb and an entity- denoting noun (e.g. The editor finished the ar- ticle), which are typically interpreted by infer- ring a hidden event (e.g. reading) on the basis of contextual cues. This paper tackles the problem of logical metonymy interpretation, that is, the retrieval of the covert event via computational methods. We compare different types of models, includ- ing the probabilistic and the distributional ones previously introduced in the literature on the topic. For the first time, we also tested on this task some of the recent Transformer-based models, such as BERT, RoBERTa, XLNet, and GPT-2. Our results show a complex scenario, in which the best Transformer-based models and some traditional distributional models perform very similarly. However, the low performance on some of the testing datasets suggests that logical metonymy is still a challenging phe- nomenon for computational modeling. 1 Introduction The phenomenon of logical metonymy is defined as a type clash between an event-selecting metonymic verb (e.g., begin) and an entity-denoting nominal object (e.g., the book), which triggers the recov- ery of a hidden event (e.g., reading). Logical metonymies have been widely studied, on the one hand, in theoretical linguistics as they represent a challenge to traditional theories of compositionality (Asher, 2015; Pustejovsky and Batiukova, 2019). On the other hand, they received extensive atten- tion in cognitive research on human sentence pro- cessing as they determine extra processing costs during online sentence comprehension (McElree et al., 2001; Traxler et al., 2002), apparently related to “the deployment of operations to construct a se- mantic representation of the event” (Frisson and McElree, 2008).1 Logical metonymy has also been explained in terms of the words-as-cues hypothesis proposed by Jeffrey Elman (Elman, 2009, 2014). This hy- pothesis relies on the experimental evidence that human semantic memory stores knowledge about events and their typical participants (see McRae and Matsuki (2009) for an overview) and claims that words act like cues to access event knowledge, incrementally modulating sentence comprehension. The results obtained in a probe recognition exper- iment by Zarcone et al. (2014), in line with this explanation, suggest that speakers interpret logical metonymies by inferring the most likely event the sentences could refer to, given the contextual cues. Previous research in NLP on logical metonymy has often been influenced by such theoretical explana- tion (Zarcone and Pad´o, 2011; Zarcone et al., 2012; Chersoni et al., 2017). In our contribution, we propose a general com- parison of different classes …

Content truncated to 500 words