C3SL at SemEval-2021 Task 1: Predicting Lexical Complexity of Words in Specific Contexts with Sentence Embeddings

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 683–687 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 683 C3SL at SemEval-2021 Task 1: Predicting Lexical Complexity of Words in Specific Contexts with Sentence Embeddings Raul Gomes Pimentel de Almeida C3SL Labs Universidade Federal do Paran´a Curitiba - Brazil rgpa18@inf.ufpr.br Hegler Tissot Cognitive Computation Group University of Pennsylvania Philadelphia - USA hegler@seas.upenn.edu Marcos Didonet Del Fabro C3SL Labs Universidade Federal do Paran´a Curitiba - Brazil marcos.ddf@inf.ufpr.br Abstract We present our approach to predicting lexi- cal complexity of words in specific contexts, as entered LCP Shared Task 1 at SemEval 2021. The approach consists of separating sen- tences into smaller chunks, embedding them with Sent2Vec, and reducing the embeddings into a simpler vector used as input to a neural network, the latter for predicting the complex- ity of words and expressions. Results show that the pre-trained sentence embeddings are not able to capture lexical complexity from the language when applied in cross-domain appli- cations. 1 Introduction Lexical complexity plays a crucial role in reading comprehension. Predicting lexical complexity of words within sentences in specific textual context can enable systems to better perform certain NLP tasks, such as simplifying texts, and favoring less fortunate readers in a giving target language. The SemEval 2021 proposes a Lexical Complexity Pre- diction (LCP) shared task (Task 1) (Shardlow et al., 2021) based on a new annotated English dataset with a Likert scale (Shardlow et al., 2020). Word embeddings (Mikolov et al., 2013; Devlin et al., 2019) are very important resources that sup- port several NLP tasks by providing a semantic latent representation that heuristically captures re- lationships in language that are very difficult to ob- serve otherwise. Furthermore, such semantic repre- sentation can be used to embed language structures other than just words. Sent2Vec (Pagliardini et al., 2018) is an unsuper- vised model designed to compose sentence embed- dings using word vectors along with n-gram em- beddings, simultaneously training composition and the embedding vectors themselves. Sent2Vec has been used in several NLP tasks, such as analysing semantic properties of sentences (Zhu et al., 2018), classification of sentences in the biomedical do- main (Agibetov et al., 2018), automatic detection of incoherent speech (Iter et al., 2018), and measur- ing sentence similarity (Quan et al., 2019), In this work, we aim to test the ability of Sent2Vec to detect the complexity of English words and expressions in specific contexts. We evaluate in what extent the semantic information captured when learning the embeding representation is able to incorporate word complexity. Our approach uses pretrained Sent2Vec models and aims to validate in what extent such models are able to predict complexity of words. Results show strong evidence that pretrained sentence em- beddings do not capture complexity features from language, specially when applied in cross-domain applications. 2 Method The proposed shared task consists of determin- ing the complexity of token words in the con- text of given input sentences (Lexical Complexity Prediction-LCP). Training input sentences …

Content truncated to 500 words