CLULEX at SemEval-2021 Task 1: A Simple System Goes a Long Way

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 632–639 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 632 CLULEX at SemEval-2021 Task 1: A Simple System Goes a Long Way Greta Smolenska, Peter Kolb, Sinan Tang, Mironas Bitinis, H´ector Hern´andez, Elin Askl¨ov Babbel|Lesson Nine GmbH {gsmolenska, pkolb, stang, mbitinis, hhernandez, easkloev}@babbel.com Abstract This paper presents the system we submit- ted to the first Lexical Complexity Prediction (LCP) Shared Task 2021. The Shared Task pro- vides participants with a new English dataset that includes context of the target word. We participate in the single-word complexity pre- diction sub-task and focus on feature engineer- ing. Our best system is trained on linguis- tic features and word embeddings (Pearson’s score of 0.7942). We demonstrate, however, that a simpler feature set achieves comparable results and submit a model trained on 36 lin- guistic features (Pearson’s score of 0.7925). 1 Introduction Lexical complexity relates to complexity of words. Its assessment can be beneficial in a number of fields, ranging from education to communication. For instance, lexical complexity studies can assist in providing language learners with learning ma- terials suitable for their proficiency level or aid in text simplification (Siddharthan, 2014). These studies are also a central part of reading compre- hension, as lexical complexity can predict which words might be difficult to understand and could hinder the readability of the text. Lexical complex- ity studies typically make use of Natural Language Processing and Machine Learning methods (Paet- zold and Specia, 2016). Previous similar studies focus on Complex Word Identification (CWI), which is a process of identi- fying complex words in a text (Shardlow, 2013). In this case, lexical complexity is assumed to be binary - words are either complex or not. LCP Shared Task 2021 addresses this limitation by in- troducing a new dataset designed for continuous rather than binary complexity prediction (Shardlow et al., 2021). In this paper, we describe a single-word lexi- cal complexity prediction system. Our goal is to demonstrate that a simple system can achieve re- sults comparable to more complex ones. Therefore, we focus on feature engineering rather than model tuning. 2 Related Work 2.1 Lexical Complexity Over the years, studies on lexical complexity have ranged from research on the overall readability enhancement and text simplification to studies fo- cusing specifically on lexical complexity. Some of the earlier work on lexical complexity targeted communication enhancement of medical documents by assessing the familiarity of medical terminology (Zeng et al., 2005). Paetzold and Spe- cia (2013) showed that the absence of lexical sim- plification in Automatic Text Simplification (ATS) systems yielded texts that readers might still find too complex to understand. CWI has then gained more interest, and two Shared Tasks have been organised with the goal of establishing state-of-the-art performance in the field. SemEval-2016 Task 11 approached CWI as a binary classification task and collected a dataset for English which was annotated by non-native speakers (Paetzold and Specia, 2016). Zampieri et al. (2017) showed that …

Content truncated to 500 words