SV000gg at SemEval-2016 Task 11: Heavy Gauge Complex Word Identification with System Voting

Proceedings of SemEval-2016, pages 969–974, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics SV000gg at SemEval-2016 Task 11: Heavy Gauge Complex Word Identification with System Voting Gustavo Henrique Paetzold and Lucia Specia Department of Computer Science University of Sheffield, UK {ghpaetzold1,l.specia}@sheffield.ac.uk Abstract We introduce the SV000gg systems: two En- semble Methods for the Complex Word Iden- tification task of SemEval 2016. While the SV000gg-Hard system exploits basic Hard Voting, the SV000gg-Soft system employs Performance-Oriented Soft Voting, which weights votes according to the voter’s perfor- mance rather than its prediction confidence, allowing for completely heterogeneous sys- tems to be combined. Our performance com- parison shows that our voting techniques out- perform traditional Soft Voting, as well as other systems submitted to the shared task, ranking first and second overall. 1 Introduction In Complex Word Identification (CWI), the goal is to find which words in a given text may challenge the members of a given target audience. It is part of the usual Lexical Simplification pipeline, which is illustrated in Figure 1. As shown by the results ob- tained by (Paetzold and Specia, 2013) and (Shard- low, 2014), ignoring the step of Complex Word Identification in Lexical Simplification can lead sim- plifiers to neglect challenging words, as well as to re- place simple words with inappropriate alternatives. Various strategies have been devised to address CWI and most of them are very simple in na- ture. For example, to identify complex words, the lexical simplifier for the medical domain in (El- hadad and Sutaria, 2007) uses a Lexicon-Based ap- proach that exploits the UMLS (Bodenreider, 2004) database: if a medical expression is among the tech- nical terms registered in UMLS, then it is complex. Figure 1: Lexical Simplification pipeline The complexity identifier for the lexical simplifier in (Keskis¨arkk¨a, 2012), for Swedish, uses a thresh- old over word frequencies to distinguish complex from simple words. Recently, however, more so- phisticated approaches have been used. (Shardlow, 2013) presents a CWI benchmarking that compares the performance of a Threshold-Based strategy, a Support Vector Machine (SVM) model trained over various features, and a “simplify everything” base- line. (Shardlow, 2013)’s SVM model has shown promising results, but CWI approaches do not tend to explore Machine Learning techniques and, in par- ticular, their combination. As an effort to fill this gap, in this paper we describe our contributions to the Complex Word Identification task of SemEval 2016. We introduce two systems, SV000gg-Hard and SV000gg-Soft, both of which use straightfor- ward Ensemble Methods to combine different pre- dictions for CWI. These come from a variety of models, ranging from simple Lexicon-Based ap- proaches to more elaborate Machine Learning clas- sifiers. 969 2 Dataset and Evaluation In the CWI task of SemEval 2016, participants were asked to submit predictions on the complexity of words based on the needs of non-native English speakers. The setup of the task is as follows: given a target word in a sentence, predict whether or not a non-native English speaker would be able …

Content truncated to 500 words