ANDI at SemEval-2021 Task 1: Predicting complexity in context using distributional models, behavioural norms, and lexical resources

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 655–660 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 655 Abstract In this paper we describe our participation in the Lexical Complexity Prediction (LCP) shared task of SemEval 2021, which involved predicting subjective ratings of complexity for English single words and multi-word expressions, presented in context. Our approach relies on a combination of distributional models, both context-dependent and context- independent, together with behavioural norms and lexical resources. 1 Introduction In our day-to-day life, outside the laboratory, we almost never come across single words or pairs of words, in isolation. Instead, such verbal stimuli are typically embedded within sentences or phrases, and our understanding of individual words and word pairs is influenced by their linguistic contexts (e.g., by disambiguating their intended meaning). Hoewever, almost all behavioural norms collected so far focus only on single words or word pairs (Johns, Jamieson, & Jones, 2020). Therefore, the Lexical Complexity Prediction (LCP) shared task (Shardlow, Evans, Paetzold, & Zampieri, 2021), hosted at SemEval 2021, constitutes a timely and valuable contribution to the study of context-dependent semantics. The task requires competitors to predict subjective ratings of complexity for words or pairs of words, presented within sentences. As mentioned by the organisers, being able to automatically estimate contextualised complexity ratings would have several practical applications, such as detecting and simplifying portions of text that might be particularly difficult 1 https://github.com/armandrotaru/TeamAndi-LCP to understand for second language learners, and people with low literacy levels (e.g., as a result of suffering from a reading impairment). The dataset for the competition is CompLex 2.0 (Shardlow, Cooper, & Zampieri, 2020; Shardlow, Evans, & Zampieri, 2021), consisting of passages from the Bible, the proceedings of the European Parliament, and biomedical journal articles. The training data covers 7,662 single words (2,574 bible, 2,512 europarl, and 2,576 biomed), and 1,517 multi-word expressions (505 bible, 498 europarl, and 514 biomed). The test data covers 917 single words (283 bible, 345 europarl, and 289 biomed), and 184 multi-word expressions (66 bible, 65 europarl, and 53 biomed). In this paper we describe our submission to the competition, based on distributional models, both context-dependent and context-independent, as well as behavioural norms/lexical resources1. The best results are obtained by combining the three classes of predictors. However, the improvement in performance over using just context-independent models is small, and, in practice, might be compensated by their impressive vocabulary size and ease of use. 2 General Description In order to predict word complexity in context, we combined information from three type of sources, namely behavioural norms/lexical resources, and distributional models. With respect to the latter, we included two distinct classes of models: • context-independent models, which output the same vector representation for a given word, regardless of the context in which the word is encountered; ANDI at SemEval-2021 Task 1: Predicting complexity in context using distributional models, behavioural norms, and lexical resources Armand Rotaru Independent researcher armand.rotaru@gmail.com 656 • context-dependent models, which output a potentially different representations …

Content truncated to 500 words