Pomona at SemEval-2016 Task 11: Predicting Word Complexity Based on Corpus Frequency

Proceedings of SemEval-2016, pages 1047–1051, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics Pomona at SemEval-2016 Task 11: Predicting Word Complexity Based on Corpus Frequency David Kauchak Computer Science Department Pomona College Claremont, CA david.kauchak@pomona.edu Abstract We introduce a word frequency-based classi- fier for the SemEval 2016 complex word iden- tification task (#11). Words with lower fre- quency are predicted as complex based on a threshold optimized for G-score. We exam- ine three different corpora for calculating fre- quencies and find English Wikipedia to per- form best (ranked 13th on the SemEval task), followed by the Google Web Corpus and lastly Simple English Wikipedia. Bagging is also shown to slightly improve the performance of the classifier. Overall, we find word frequency to be a strong predictor of complexity. On the SemEval “test” set, a frequency classifier that uses the optimal frequency threshold performs on-par with the best submitted system and a system trained using only 500 labeled exam- ples split from the test set achieves results that are only slightly below the best submitted sys- tem. 1 Introduction Text simplification aims to transform text into more accessible versions while retaining the origi- nal meaning. A frequent subproblem of the general simplification problem is complex word identifica- tion: identify words in a text that are difficult to un- derstand for the reader. Complex word identification is critical in lexical simplification algorithms where the simplification process is done a word at a time; frequently, simplification is broken into two steps, first identifying the complex words that need sim- plifying and, second, determining substitutions for these words (Shardlow, 2014). Even for simplifi- cation systems that make sentence-level transforma- tions (Siddharthan, 2014) complex word identifica- tion can be used as an additional feature function in the model and as a development tool to help measure progress. Additionally, in some domains such as health and medicine, accuracy is critical and semi- automated simplification tools are common (Leroy et al., 2012). In these domains, complex word iden- tification is useful to help guide the simplification process by both identifying which words need to be simplified and filtering/ranking possible candidate substitutions (Leroy et al., 2013). In this paper, we explore the use of word fre- quency as a predictor of the complexity of that word. Corpus studies have shown that simpler texts contain more frequent words than more complicated texts (Breland, 1996; Pitler and Nenkova, 2008; Leroy et al., 2012). User studies have also shown a cor- relation between word frequency and whether users know the definition of a word (Leroy and Kauchak, 2013). In semi-automated text simplification ap- proaches, replacing less frequent words with higher frequency synonyms has been shown to produce text that people view as simpler and is easier to under- stand (Leroy et al., 2013). 2 Bagged Frequency Classifier Given a sentence S = s1s2...sm and a word in that sentence, sj, the complex word identification task is to predict whether that word is complex (1) or not (0). Labeled …

Content truncated to 500 words