CSECU-DSG at SemEval-2021 Task 1: Fusion of Transformer Models for Lexical Complexity Prediction

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 627–631 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 627 CSECU-DSG at SemEval-2021 Task 1: Fusion of Transformer Models for Lexical Complexity Prediction Abdul Aziz, MD. Akram Hossain, and Abu Nowshed Chy Department of Computer Science and Engineering University of Chittagong, Chattogram-4331, Bangladesh {aziz.abdul.cu, akram.hossain.cse.cu}@gmail.com, and nowshed@cu.ac.bd Abstract Lexical complexity prediction (LCP) conveys the anticipation of the complexity level of a to- ken or a set of tokens in a sentence. It plays a vital role in the improvement of various NLP tasks including lexical simplification, transla- tions, and text generation. However, multiple- meaning of a word in multiple circumstances, grammatical complex structure, and the mu- tual dependency of words in a sentence make it difficult to estimate the lexical complex- ity. To address these challenges, SemEval- 2021 Task 1 introduced a shared task focusing on LCP and this paper presents our participa- tion in this task. We proposed a transformer- based approach with sentence pair regression. We employed two fine-tuned transformer mod- els including BERT and RoBERTa to train our model and fuse their predicted score to the complexity estimation. Experimental re- sults demonstrate that our proposed method achieved competitive performance compared to the participants’ systems. 1 Introduction Lexical complexity prediction (LCP) has become an important task in this globalization age, espe- cially for second language learners (Przybyła and Shardlow, 2020). LCP is a little bit expansion of complex word identification (CWI) task (Paet- zold and Specia, 2016; ˇStajner et al., 2018), where CWI is a binary classification of a word that is complex or not and LCP is finding the complex- ity level of a word in continuous labelling in a sentence (Shardlow et al., 2020). LCP plays a vital role in many NLP applications such as lex- ical simplification (Paetzold, 2016; Paetzold and Specia, 2017; Qiang et al., 2020), text generation, and machine translation (Wang et al., 2016). Be- sides, it helps those people who are suffering from **The first two authors have equal contributions. Dyslexia (Rello et al., 2013a), Aphasia (Rello et al., 2013b), and those with low literacy levels (Aluisio and Gasperin, 2010). LCP is a very challenging task (Zampieri et al., 2017), especially because the non-identical target audiences will have distinct needs. For example, speakers of one language usually less familiar with different subsets of the vocabulary of a second language. Besides, the grammatical shape of a sentence and the ambiguous meaning of a word in different places make this task more challeng- ing and important to explore. A single word may portray different lexical complexity because of its non-identical usage, position, tense form, and re- dundancy in different sentences or in the same sen- tence. To estimate multi-word complexity, we need to consider the dependency between tokens. Sentence Token Complexity Sub-task 1 His head is like the purest gold. gold 0.210 Sub-task 2 They shall eat it with bitter herbs. bitter herbs 0.25 Table 1: Example of sub-task 1 …

Content truncated to 500 words