Strong Baselines for Complex Word Identification across Multiple Languages

Proceedings of NAACL-HLT 2019, pages 970–977 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 970 Strong Baselines for Complex Word Identification across Multiple Languages Pierre Finnimore1, Elisabeth Fritzsch1, Daniel King1, Alison Sneyd1, Aneeq Ur Rehman1, Fernando Alva-Manchego1 and Andreas Vlachos2 1Department of Computer Science, University of Sheffield 2Department of Computer Science and Technology, University of Cambridge {pmfinnimore,fritzsch.elisabeth,danielking1903}@gmail.com, a.sneyd@shef.ac.uk,a.neeq8394@gmail.com,f.alva@shef.ac.uk, andreas.vlachos@cst.cam.ac.uk Abstract Complex Word Identification (CWI) is the task of identifying which words or phrases in a sentence are difficult to understand by a tar- get audience. The latest CWI Shared Task re- leased data for two settings: monolingual (i.e. train and test in the same language) and cross- lingual (i.e. test in a language not seen dur- ing training). The best monolingual models relied on language-dependent features, which do not generalise in the cross-lingual setting, while the best cross-lingual model used neural networks with multi-task learning. In this pa- per, we present monolingual and cross-lingual CWI models that perform as well as (or bet- ter than) most models submitted to the latest CWI Shared Task. We show that carefully se- lected features and simple learning models can achieve state-of-the-art performance, and re- sult in strong baselines for future development in this area. Finally, we discuss how incon- sistencies in the annotation of the data can ex- plain some of the results obtained. 1 Introduction Complex Word Identification (CWI) consists of deciding which words (or phrases) in a text could be difficult to understand by a specific type of reader. In this work, we follow the CWI Shared Tasks (Paetzold and Specia, 2016; Yimam et al., 2018) and assume that a target word or multi-word expression (MWE1) in a sentence is given, and our goal is to determine if it is complex or not (an ex- ample is shown in Table 1). Under this setting, CWI is normally treated using supervised learn- ing and feature engineering to build monolingual models (Paetzold and Specia, 2016; Yimam et al., 2018). Unfortunately, this approach is infeasible for languages with scarce resources of annotated 1We consider n-grams with n ≥ 2 as MWEs, while Yi- mam et al. (2018) used n ≥ 3. Sentence Target word/MWE Complex? Both China and the Philippines flexed their muscles on Wednesday. flexed Yes flexed their muscles Yes muscles No Table 1: An annotated sentence in the English dataset of the Second CWI Shared Task. data. In this paper, we are interested in both mono- lingual and cross-lingual CWI; in the latter, we build models to make predictions for languages not seen during training. While monolingual CWI has been studied ex- tensively (see a survey in Paetzold and Specia (2017)), the cross-lingual setup of the task was in- troduced only recently by Yimam et al. (2017b), who collected human annotations from native and non-native speakers of Spanish and German, and integrated them with similar data previously pro- duced for three English domains (Yimam et al., 2017a): News, WikiNews and Wikipedia. For the Second CWI Shared Task …

Content truncated to 500 words