Manchester Metropolitan at SemEval-2021 Task 1: Convolutional Networks for Complex Word Identification

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 603–608 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 603 Manchester Metropolitan at SemEval-2021 Task 1: Convolutional Networks for Complex Word Identification Robert Flynn School of Computing, Mathematics and Digital Technology Manchester Metropolitan University robert.flynn@stu.mmu.ac.uk Matthew Shardlow School of Computing, Mathematics and Digital Technology Manchester Metropolitan University m.shardlow@mmu.ac.uk Abstract We present two convolutional neural networks for predicting the complexity of words and phrases in context on a continuous scale. Both models utilize word and character embeddings alongside lexical features as inputs. Our sys- tem displays reasonable results with a Pearson correlation of 0.7754 on the task as a whole. We highlight the limitations of this method in properly assessing the context of the target text, and explore the effectiveness of both sys- tems across a range of genres. Both models were submitted as part of LCP 2021, which fo- cuses on the identification of complex words and phrases as a context dependent, regression based task. 1 Introduction Complex Word Identification (CWI) involves iden- tifying words that the reader may find difficult to understand. A word’s complexity can depend on many factors and differ according to context. Fur- ther, assessment of the complexity of named enti- ties can require some degree of general knowledge, making CWI a challenging task (Shardlow, 2013). Accurately identifying complex words is important for many downstream simplification tasks, mak- ing literature more accessible for people with con- ditions such as dyslexia (Rello et al., 2013), and the assessment of a text’s readability as a whole (Dubay, 2004). Our methodology plans to extend on previous convolutional network based approaches to CWI (Aroyehun et al., 2018; Sheang, 2019). With the goal of producing a system that is able to better model the complexities of phrases and unfamiliar words, within the English language. Previous shared tasks on CWI addressed the problem as a binary and probabilistic classification type task, although human judgements on word complexity are not binary and exist on a continuous scale. Lexical Complexity Prediction (LCP) 2021 tries to address this and uses an augmented version of CompLex (Shardlow et al., 2020), a dataset an- notated with a 5-point Likert scale. CompLex also features context-specific annotation, with words receiving different annotations depending on their context. The dataset provides annotations from three different domains: Bible, Biomed and Eu- roparl (Shardlow et al., 2021). The code for this task is available on GitHub1. 2 Related Work Word frequency is a commonly used feature in CWI (Gooding and Kochmar, 2018; Kajiwara and Komachi, 2018); words that appear frequently in language are more likely to be recognised and un- derstood by the reader (Carroll et al., 1998). For the purpose of identifying medical terminology that may be unfamiliar to the lay reader, Elhadad (2006) leveraged lexical frequencies while also explor- ing the potential of other features such as word familiarity ratings from the MRC Psycholinguistic Database (Coltheart, 1981). More recently lexical and psycholinguistic fea- tures have been utilized …

Content truncated to 500 words