UPB at SemEval-2021 Task 1: Combining Deep Learning and Hand-Crafted Features for Lexical Complexity Prediction

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 609–616 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 609 UPB at SemEval-2021 Task 1: Combining Deep Learning and Hand-Crafted Features for Lexical Complexity Prediction George-Eduard Zaharia, Dumitru-Clementin Cercel, Mihai Dascalu University Politehnica of Bucharest, Faculty of Automatic Control and Computers george.zaharia0806@stud.acs.upb.ro {dumitru.cercel, mihai.dascalu}@upb.ro Abstract Reading is a complex process which requires proper understanding of texts in order to cre- ate coherent mental representations. However, comprehension problems may arise due to hard-to-understand sections, which can prove troublesome for readers, while accounting for their specific language skills. As such, steps towards simplifying these sections can be per- formed, by accurately identifying and evalu- ating difficult structures. In this paper, we describe our approach for the SemEval-2021 Task 1: Lexical Complexity Prediction compe- tition that consists of a mixture of advanced NLP techniques, namely Transformer-based language models, pre-trained word embed- dings, Graph Convolutional Networks, Cap- sule Networks, as well as a series of hand- crafted textual complexity features. Our mod- els are applicable on both subtasks and achieve good performance results, with a MAE below 0.07 and a Person correlation of .73 for single word identification, as well as a MAE below 0.08 and a Person correlation of .79 for mul- tiple word targets. Our results are just 5.46% and 6.5% lower than the top scores obtained in the competition on the first and the second subtasks, respectively. 1 Introduction Reading is a complex process due to the mental exercise readers are challenged to perform, since a coherent representation of the text needs to be pro- jected into their mind in order to grasp the underly- ing content (Van den Broek, 2010). For non-native speakers, the lack of text understanding hinders knowledge assimilation, thus becoming the main obstacle that readers need to overcome. Complex words can impose serious difficulties, considering that their meaning is often dependant on their con- text and cannot be easily inferred. In order to facil- itate text understanding or to perform text simpli- fication, complex tokens first need to be detected. This can be performed by developing systems ca- pable of identifying them by individual analysis, as well as contextual analysis. There are two main approaches regarding the complexity task. Tokens can be binary classified as complex or non-complex, a procedure that helps users separate problematic words from the others. Words can be also labeled with a probabilistic com- plexity value, which in return can be used to sim- plify the text. Words with lower degrees of com- plexity can be easily explained, whereas more com- plex tokens can be replaced with simpler equivalent concepts. The Lexical Complexity Prediction (LCP) shared task, organized as the SemEval-2021 Task 1 (Shardlow et al., 2021a), challenged the research community to develop robust systems that identify the complexity of a token, given its context. Sys- tems were required to be easily adaptable, consider- ing that the dataset entries originated from multiple domains. At the same time, the target …

Content truncated to 500 words