USAAR at SemEval-2016 Task 11: Complex Word Identification with Sense Entropy and Sentence Perplexity

Proceedings of SemEval-2016, pages 958–962, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics USAAR at SemEval-2016 Task 11: Complex Word Identification with Sense Entropy and Sentence Perplexity Jos´e Manuel Mart´ınez Mart´ınez and Liling Tan Universit¨at des Saarlandes Campus, Saarbr¨ucken, Germany j.martinez@mx.uni-saarland.de liling.tan@uni-saarland.de Abstract This paper describes an information-theoretic approach to complex word identification using a classifier based on an entropy based mea- sure based on word senses and sentence-level perplexity features. We describe the motiva- tion behind these features based on informa- tion density and demonstrate that they perform modestly well in the complex word identifica- tion task in SemEval-2016. We also discuss the possible improvements that can be made to future work by exploring the subjectivity of word complexity and more robust evalua- tion metrics for the complex word identifica- tion task. 1 Introduction Complex Word Identification (CWI) is the task of automatically identifying difficult words in a sen- tence. It is an important subtask prior to the textual/lexical simplification task that pertains to the substitution of abstruse words with lucid variants which can be apprehended by a wider gamut of readers (Siddharthan, 2006; Specia et al., 2012; Shardlow, 2013). The aim of the CWI task is to annotate the difficult words as shown in the underlined examples in the previous paragraph, such that a lexical simplification system can produce the following sentence: It is an important subtask before the textual/lexical simplification task that concerns the replacement of difficult words with simpler variants which can be understood by a wider range of readers. Lexical simplification is a specific case of lexical substitution where the complex words in a sentence are replaced with simpler words. Historically, lexical substitution was conceived as a means to examine the issue of the appropriate- ness of a fixed word sense inventory in the word sense disambiguation task the “sense” of a polyse- mous word is correctly identified given a context sentence (Kilgarriff, 1997; Palmer, 2000; Hanks, 2000; Ide and Wilks, 2007; McCarthy and Navigli, 2009). By allowing fluidity in the “sense” inventory and by quantifying how much the systems were able to generate good substitutes, these lexical substitutes would have built a word sense cluster of words that may not be covered by a set of pre-defined words in a sense inventory, e.g. Princeton WordNet (Miller, 1995) and Open Multilingual WordNet (Bond and Paik, 2012). 2 Entropy and Perplexity Entropy is an information-theoretical measure of the degree of indeterminacy of a random variable1. In simpler words, entropy measures how unpredictable an event is likely to occur (Shannon, 1951). For the case of complex word , we can also as- sume that the degree of word ambiguity contributes to its level of unpredictability which determines its complexity. We define the degree of word ambigu- ity as the number of possible senses a word can have, 1https://www.encyclopediaofmath.org/index.php/Entropy 958 more specially the number of synsets of a lemma of the target word as recorded in the Princeton Word- Net. Formally, we define the sense …

Content truncated to 500 words