Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers

Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, pages 89 - 100 July 14-15, 2022 ©2022 Association for Computational Linguistics Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers Luis Espinosa-Anke† Alexander Shvets♡ Alireza Mohammadshahi♢♠ James Henderson♢ Leo Wanner♣♡ †CardiffNLP (Cardiff University) - AMPLYFI ♡TALN Group, Universitat Pompeu Fabra ♢Idiap Research Institute ♠EPFL ♣ICREA espinosa-ankel@cardiff.ac.uk {alexander.shvets,leo.wanner}@upf.edu {alireza.mohammadshahi,james.henderson}@idiap.ch Abstract Recognizing and categorizing lexical colloca- tions in context is useful for language learning, dictionary compilation and downstream NLP. However, it is a challenging task due to the varying degrees of frozenness lexical colloca- tions exhibit. In this paper, we put forward a sequence tagging BERT-based model enhanced with a graph-aware transformer architecture, which we evaluate on the task of collocation recognition in context. Our results suggest that explicitly encoding syntactic dependencies in the model architecture is helpful, and provide insights on differences in collocation typifica- tion in English, Spanish and French.1 1 Introduction Native speech is idiosyncratic. Of special promi- nence are syntactically-bound restricted binary co- occurrences of lexical items, in which one of the items conditions the selection of the other item. Consider a CNN sports headline from 02/15/2021: Rafael Nadal eases into Australian Open quarterfinals, remains on course for record-breaking grand slam (cnn.com). In this short headline, we see already three of such co-occurrences: ease [into] quarterfinals, re- main [on] course, and record-breaking grand slam. Quarterfinals conditions the selection of [to] ease [into], course of remain [on], and grand slam of record-breaking. The idiosyncrasy of these co- occurrences becomes obvious when we look at them from a multilingual angle. Thus, in French, instead of the literal translation of ease [into], we would use se qualifier ‘qualify [oneself]’, in Span- ish, remain [on] will be translated as seguir [en] ‘continue in’, and in Italian record-breaking will be da record, lit. ‘of record’ – while the translation of 1Data and code are available at https://github.com/TalnUPF/ graph-aware-collocation-recognition. quarterfinals, course, and grand slam will be literal. In lexicology, such binary co-occurrences are re- ferred to as collocations (Hausmann, 1985; Cowie, 1994; Mel’ˇcuk, 1995; Kilgarriff, 2006), with the conditioning item called the base and the condi- tioned item the collocate. Collocations in this sense are of high relevance to second language learning, lexicography and NLP alike, and constitute a chal- lenge for computational models because of their heterogeneity in terms of idiosyncrasy and degree of semantic composition (Mel’ˇcuk, 1995). Research in NLP has already addressed a num- ber of collocation-related tasks, in particular: (1) collocation error detection, categorization, and cor- rection in writings of second language learners (Ferraro et al., 2011; Wanner et al., 2013; Ferraro et al., 2014; Rodríguez-Fernández et al., 2015); (2) creation of collocation-enriched lexical resources (Espinosa-Anke et al., 2016; Maru et al., 2019; Di Fabio et al., 2019); (3) use of knowledge on collocations in downstream NLP tasks, among them, e.g., machine translation (Seretan, 2014), word sense disambiguation (Maru et al., 2019), nat- ural language generation (Wanner and Bateman, 1990), or semantic role labeling (Scozzafava et al., …

Content truncated to 500 words