Cambridge at SemEval-2021 Task 1: An Ensemble of Feature-Based and Neural Models for Lexical Complexity Prediction

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 590–597 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 590 Cambridge at SemEval-2021 Task 1: An Ensemble of Feature-Based and Neural Models for Lexical Complexity Prediction Zheng Yuan Gladys Tyen David Strohmaier ALTA Institute, University of Cambridge, United Kingdom Department of Computer Science and Technology, University of Cambridge, United Kingdom {firstname.lastname}@cl.cam.ac.uk Abstract This paper describes our submission to the SemEval-2021 shared task on Lexical Com- plexity Prediction. We approached it as a regression problem and present an ensemble combining four systems, one feature-based and three neural with fine-tuning, frequency pre-training and multi-task learning, achieving Pearson scores of 0.8264 and 0.7556 on the trial and test sets respectively (sub-task 1). We further present our analysis of the results and discuss our findings. 1 Introduction Predicting which words are considered hard to un- derstand for a given target population has many ap- plications. For example, it can be used to identify texts appropriate for language learners or included in a pipeline for automatic text simplification for people with low literacy skills or reading disabil- ities (Xia et al., 2016; Shardlow, 2014; Gooding and Kochmar, 2019b). In this paper, we describe our submission to the SemEval-2021 shared task on Lexical Complexity Prediction (LCP) (sub-task 1), where participating teams are expected to pre- dict the complexity score of single words in con- text (Shardlow et al., 2021). Compared to previ- ous shared tasks on Complex Word Identification (CWI), which have primarily focused on binary classification as systems were expected to iden- tify words as complex or not (Paetzold and Specia, 2016a; Yimam et al., 2018); a new multi-domain English dataset was used for the purpose, which was annotated using a 5-point Likert scale (Shard- low et al., 2020). We approached LCP as a regres- sion problem and proposed a traditional feature- based model, as well as three neural models explor- ing fine-tuning, frequency pre-training and multi- task learning (MTL). The remainder of this paper is organised as fol- lows. Section 2 presents related work in the area. In Section 3, we describe our approach to the task and detail the four models included in our final en- semble system. In Section 4, we turn to the experi- ments, describing the data and evaluation metrics used, and presenting our results on the shared task trial set. Section 5 presents our official results on the shared task test set, and offers a discussion of the results and the performance of our submitted system. Finally, we conclude the paper and provide an overview of our findings in Section 6. 2 Related work The SemEval-2016 shared task on CWI (Paetzold and Specia, 2016a) was framed as a binary clas- sification problem, where complexity was defined as whether or not a word is difficult to understand for non-native English speakers. A set of 400 non- native speakers annotated the data in a binary fash- ion and a word was labelled as complex if it was …

Content truncated to 500 words