Stanford MLab at SemEval-2021 Task 1: Tree-Based Modelling of Lexical Complexity using Word Embeddings

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 688–693 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 688 Stanford MLab at SemEval-2021 Task 1: Tree-Based Modelling of Lexical Complexity Using Word Embeddings Erik Rozi∗, Niveditha Iyer∗, Gordon Chi, Enok Choe, Kathy Lee, Kevin Liu, Patrick Liu, Zander Lack, Jillian Tang†, and Ethan A. Chi† Stanford University {erikrozi, nivsiyer}@stanford.edu {jiltang, ethanchi}@cs.stanford.edu Abstract This paper presents our system for the single- and multi-word lexical complexity prediction tasks of SemEval Task 1: Lexical Complex- ity Prediction. Text comprehension depends on the reader’s ability to understand the words present in it; evaluating the lexical complex- ity of such texts can enable readers to find an appropriate text and systems to tailor a text to an audience’s needs. We present our model pipeline, which applies a combination of embedding-based and manual features to predict lexical complexity on the CompLex English dataset using various tree-based and linear models. Our method is ranked 27 / 54 on single-word prediction and 14 / 37 on multi- word prediction. 1 Introduction The rapid expansion of social media and other on- line channels has made readable information avail- able at an astounding rate. However, the accessi- bility of this information is often limited by the complexity of this information, especially among readers with low literacy levels in the language of the text and those with reading disabilities. Fur- thermore, even to the average reader, specialized jargon found in governmental documents and sci- entific fields is often difficult to decipher. Systems to guide these users may redirect read- ers to more easily comprehensible sources, convert the text to simpler wording, or provide additional information about what difficult words mean. The development of such systems is benefited by the ability to evaluate the complexity of sections of the text. While there is currently a large amount of available text data, very little of it is labeled with word complexity; automating the labelling process would make much more data available to aid the ∗Co-first authors. †Co-senior authors. development of NLP systems in tasks such as text simplification. Multiple features of a word can affect lexical complexity. In addition to a word’s frequency, length and syllable count, the context in which a word is found is likely to affect its understand- ability. The additional factor of the reader’s profi- ciency in a language makes this task complex as many words have a highly variable complexity. In this paper, we describe our model that predicts single- and multi-word lexical complexity scores. 2 Background 2.1 Task Overview All data was provided through SemEval Task 1 (Shardlow et al., 2021). Our dataset consists of an augmented version of the CompLex Corpus (Shard- low et al., 2020), which contains English sentences from three genres of corpora: the Bible, Europarl, and biomedical writing. From each sentence, both single- and multi-word tokens were selected and annotated by approximately 7 annotators. Each token was annotated on complexity from a scale of 1-5, though for this …

Content truncated to 500 words