AI-KU at SemEval-2016 Task 11: Word Embeddings and Substring Features for Complex Word Identification

Proceedings of SemEval-2016, pages 1042–1046, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics AI-KU at SemEval-2016 Task 11: Word Embeddings and Substring Features for Complex Word Identification Onur Kuru Artificial Intelligence Laboratory Koc University Istanbul, Turkey okuru13@ku.edu.tr Abstract We investigate the usage of word embeddings, namely Glove and SCODE, along with sub- string features on Complex Word Identifica- tion task. We introduce two systems: the first system utilizes the word embeddings of the target word and its substrings as features while the other considers the context infor- mation by using the embeddings of the sur- rounding words as well. Although the pro- posed representations perform below the aver- age with nonlinear models, we show that word embeddings with substring features is an ef- fective representation choice when employed with linear classifiers. 1 Introduction Complex Word Identification (CWI) is the task of determining which words in a given sentence should be simplified. CWI is often considered as the first step in Lexical Simplification (LS) where the goal is to identify and replace complex words with sim- pler substitutes. Although CWI is considered as a vital component of Lexical Simplification pipeline, there is not a lot of work done for identifying word complexity in the context of Lexical Simplification. Horn et al. (2014) describes an LS model in which they employ a Complex Word identifier implicitly. However, their results show that the method is not able to capture complex words very accurately. Paetzold and Specia (2016) organizes the Com- plex Word Identification task of SemEval 2016 with the goal of attracting systems that are able to de- tect complex words in given a sentence. This work investigates the usage of word embeddings along with substrings as features to build a Complex Word Identifier. Word embeddings are unsupervised word representations which map each word to a dense, real valued, low dimensional vector. These vectors are able to capture semantic and syntactic similar- ities between words and proven useful as features in a variety of applications, such as, document clas- sification (Sebastiani, 2002), named entity recogni- tion (Turian et al., 2010), and parsing (Socher et al., 2013). GloVe (Pennington et al., 2014) is an unsuper- vised learning algorithm for obtaining vector rep- resentations for words which is essentially a log- bilinear model with a weighted least-squares ob- jective. Pennington et al. (2014) shows that the word embeddings produced by the model achieves state-of-the-art performance in Word Analogy task. Moreover, they also illustrate that Glove embed- dings outperform embeddings induced by other methods on several word similarity tasks. Another work (Yatbaz et al., 2012) represents the context of a word by its probable substitutes. Words with their probable substitutes are fed to a co- occurrence modeling framework (SCODE) (Maron et al., 2010). Words co-occurring in similar context are closely embedded on a sphere. These word em- beddings are effective at modeling syntactic features and they achieve state-of-the-art results in inducing part-of-speech. We conducted several experiments to examine the usage of word embeddings. …

Content truncated to 500 words