DeepBlueAI at SemEval-2021 Task 1: Lexical Complexity Prediction with A Deep Ensemble Approach

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 578–584 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 578 DeepBlueAI at SemEval-2021 Task 1: Lexical Complexity Prediction with A Deep Ensemble Approach Chunguang Pan Bingyan Song Shengguang Wang Zhipeng Luo DeepBlue Technology (Shanghai) Co., Ltd {songby, panchg, wangshg, luozp}@deepblueai.com Abstract Lexical complexity plays an important role in reading comprehension. lexical complex- ity prediction (LCP) can not only be used as a part of Lexical Simplification systems, but also as a stand-alone application to help peo- ple better reading. This paper presents the win- ning system we submitted to the LCP Shared Task of SemEval 2021 that capable of deal- ing with both two subtasks. We first per- form fine-tuning on numbers of pre-trained language models (PLMs) with various hyper- parameters and different training strategies such as pseudo-labelling and data augmenta- tion. Then an effective stacking mechanism is applied on top of the fine-tuned PLMs to ob- tain the final prediction. Experimental results on the Complex dataset show the validity of our method and we rank first and second for subtask 2 and 1. 1 Introduction Lexical complexity is one of the main reasons lead- ing to overall text complexity and thus result in poor reading comprehension for readers (DuBay, 2004). Different from the Complex Word Identifi- cation (CWI) (Shardlow, 2014) task, which aims to predict whether a given word is complex or not, the goal of lexical complexity prediction (LCP) is to predict the complexity value of the given parts from contexts as shown in Figure 1. The under- lined parts of the sentence are the words that need to be predicted and the same words in different con- texts may have different complexity scores. LCP plays an important role in the usual Lexical Sim- plification (LS) (Bott et al., 2012) pipeline since it can help simplifiers find the challenging words and replace them with appropriate alternatives that easy to understand. Either LCP or CWI can not only be used as a component of LS systems but also as a stand-alone application within intelligent Multi-words Context1: SEM confirmed many of the observations made by confocal microscopy. Complexity score: 0.64473 Context2: SJ and SVJ carried out confocal microscopy on whole-mounts of stria vascularis. Complexity score: 0.7750 Single word Context1: They shall be to you for a refuge from the avenger of blood. Complexity score: 0.3475 Context2: There will be a pavilion for a shade in the daytime from the heat, and for a refuge and for a shelter from storm and from rain. Complexity score: 0.075 Figure 1: Examples of LCP including single words and multi-words. The complexity score is the score for the underlined words. tutoring systems for second language learners or in reading devices for people with low literacy skills (Gooding and Kochmar, 2018). In this paper, we introduce our system for the lexical complexity prediction task of the SemEval- 2021 (Matthew et al., 2021). We fulfill this task by leveraging multiple pre-trained language …

Content truncated to 500 words