Lexical Complexity Prediction using Word Embeddings

Abstract Lexical complexity is crucial for reading comprehension. In the past, research work of lexical complexity prediction mainly focuses on differentiating the complexity difference between two words. Moreover, most of the previous lexical complexity prediction approaches only consider traditional lexically relevant features. In this paper, we propose a novel supervised approach using word embeddings features to tackle the lexical complexity prediction problem as a single-label multi- classification problem. We discuss four word embeddings techniques including Word2Vec, fastText, GloVe, and BERT. We also discuss five classification models including k-Nearest Neighbors, Support Vector Machines, Multilayer Perception, Random Forest, and XGBoost. The prediction models are evaluated with three datasets in English, Traditional Chinese, and Japanese. The results show that SVM with fastText can achieve the highest accuracy of 66.23% for the English dataset. SVM with GloVe can achieve the highest accuracy of 53.84% for the Traditional Chinese dataset. SVM with Word2Vec can achieve the highest accuracy of 49.96% for the Japanese dataset. Keywords: Lexical Complexity, Word Embeddings, Classification Models 1 Introduction In many human reading activities, lexical complexity plays an important role for reading comprehension (North et al., 2023). Lexical complexity prediction, therefore, becomes a crucial sub-task for various natural language understanding (NLU) tasks. For example, in the lexical simplification task (Specia et al., 2012) aiming to replace words that are difficult for readers to completely understand with alternative words that can be more easily understood, identifying complex words by estimating their lexical complexity degree enables the simplification systems to find complex words for the following replacements (Shardlow, 2013, 2014). To predict lexical complexity levels of words, various approaches (Elhadad & Sutaria, 2007; Keskisärkkä, 2012; Maddela & Xu, 2018; North et al., 2023; Paetzold & Specia, 2010; Shardlow, 2013; Zeng et al., 2005) have been proposed to identify complex words. For example, word frequency thresholds are used to identify complex words in the work of Shardlow (Shardlow, 2013) and the work of Keskisärkkä (Keskisärkkä, 2012). Elhadad et al. proposed a lexicon-based approach to recognize complex words (Elhadad & Sutaria, 2007). The identification performance of approaches based on Support Vector Machines (SVM) (Vapnik, 1996) are evaluated in the work of Zeng et al. (Zeng et al., 2005) and the work of Shardlow (Shardlow, 2013). However, these lexical complexity prediction approaches only consider traditional lexically relevant features, such as word frequencies and Part-of-Speech attributes, to construct the prediction models. Since past studies show that word embeddings features can effectively represent the semantic space to capture semantic relations (Incitti et al., 2023), lexical complexity of the context is latently embedded in the lexical co-occurrence space. As shown in (Shardlow et al., 2022), collocations are an important feature to assess lexical complexity. In this paper, we propose a novel supervised approach using word embeddings features to facilitate the construction of lexical complexity prediction models. In this paper, four word embeddings techniques are investigated to study their effectiveness: Lexical Complexity Prediction using Word Embeddings Cheng-Zen Yang1, Jin-Jian Li1, and Shu-Chang Lin2 1 Department of Computer Science and Engineering …

Content truncated to 500 words