IITK@LCP at SemEval-2021 Task 1: Classification for Lexical Complexity Regression Task

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 541–547 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 541 IITK@LCP at SemEval-2021 Task 1: Classification for Lexical Complexity Regression Task Neil Shirude∗ Sagnik Mukherjee∗ Tushar Shandhilya Ananta Mukherjee Ashutosh Modi Indian Institute of Technology Kanpur (IIT Kanpur) {neilrs, sagnikm, anantam, stushar}@iitk.ac.in ashutoshm@cse.iitk.ac.in Abstract This paper describes our contribution to Se- mEval 2021 Task 1: Lexical Complexity Pre- diction. In our approach, we leverage the ELECTRA model and attempt to mirror the data annotation scheme. Although the task is a regression task, we show that we can treat it as an aggregation of several classification and regression models. This somewhat counter- intuitive approach achieved an MAE score of 0.0654 for Sub-Task 1 and MAE of 0.0811 on Sub-Task 2. Additionally, we used the concept of weak supervision signals from Gloss-BERT in our work, and it significantly improved the MAE score in Sub-Task 1. 1 Introduction With the rapid growth in digital pedagogy, English has become an extremely popular language. Al- though English is considered an easy language to learn and grasp, a person’s choice of words often affects texts’ readability. The use of difficult words can potentially lead to a communication gap, thus hampering language efficiency. Keeping these is- sues in mind, many Natural Language Processing tasks for text simplification have been recently pro- posed (Paetzold and Specia, 2017; Sikka and Mago, 2020). Our task of lexical complexity prediction is an important step in the process of simplifying texts. The SemEval 2021 Task 1 (Shardlow et al., 2021) focuses on lexical complexity prediction in English. Given a sentence and a token from it, we have to predict the complexity score of the token. The task has two Sub-Tasks- Sub-Task 1: complexity prediction of single words Sub-Task 2: complexity prediction of multi word expressions (MWEs). A word might seem complex because of 2 major ∗ Authors equally contributed to this work. factors- a) The word is less common or complex in itself. b) The context in which the word is used makes it hard to comprehend. Observing the orthogonality of these two reasons, we captured the context-dependent features and in- dependent features separately, trained models on them individually, and then combined the two us- ing ensemble methods. We used the ELECTRA (Clark et al., 2020) model for extracting context- dependent features and GloVe embeddings (Pen- nington et al., 2014) for representing the word-level features. Additionally, we propose a classification pipeline that is trained on GloVe embeddings of the tokens. This pipeline can be interpreted as a model for capturing different annotators’ thought processes: overconfidence, under-confidence and randomness. We are making our code available for our models and experiments via GitHub1. 2 Background This task uses the CompLex dataset (Shardlow et al., 2020), which is a lexical complexity predic- tion dataset in English for single and multi word ex- pressions (2-grams). Sentences in this task consists of sentences taken from 3 corpora- Bible, Biomed and Europarl. The train, …

Content truncated to 500 words