Phonologically Aware Neural Model for Named Entity Recognition in Low Resource Transfer Settings

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1462–1472, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Phonologically Aware Neural Model for Named Entity Recognition in Low Resource Transfer Settings Akash Bharadwaj David Mortensen Chris Dyer Jaime G. Carbonell {akashb, dmortens, cdyer, jgc}@cs.cmu.edu Language Technologies Institute Carnegie Mellon University Abstract Named Entity Recognition is a well estab- lished information extraction task with many state of the art systems existing for a va- riety of languages. Most systems rely on language specific resources, large annotated corpora, gazetteers and feature engineering to perform well monolingually. In this pa- per, we introduce an attentional neural model which only uses language universal phonolog- ical character representations with word em- beddings to achieve state of the art perfor- mance in a monolingual setting using super- vision and which can quickly adapt to a new language with minimal or no data. We demon- strate that phonological character represen- tations facilitate cross-lingual transfer, out- perform orthographic representations and in- corporating both attention and phonological features improves statistical efficiency of the model in 0-shot and low data transfer settings with no task specific feature engineering in the source or target language. 1 Introduction Named Entity Recognition (NER) (Nadeau and Sekine, 2007; Marrero et al., 2013) is an informa- tion extraction task that deals with finding and clas- sifying entities in text into a fixed set of types of interest. It is challenging for a variety of reasons. Named Entities (NEs) can be arbitrarily synthesized (eg. people’s/organization’s names). NEs are often not subject to uniform cross-linguistic spelling con- ventions: compare France (English) and Frantsiya (Uzbek). NEs occur rarely in data which makes gen- eralization difficult. Skewed class statistics necessi- tate measures to prevent models from merely favor- ing a majority class. Named entities must also be annotated in con- text (eg. “[New York Times]ORG” vs. “[New York]LOC”). Lexical ambiguity (Turkey—country vs. bird), semantic ambiguity (“I work at the [New York Times]ORG” vs. “I read the New York Times”) and sparsity induced by morphology add complex- ity. Despite the challenges mentioned above, compe- tent monolingual systems that rely on having suffi- cient annotated data, knowledge and resources avail- able for engineering features have been developed. A more challenging task is to design a model that retains competence in monolingual scenarios and can easily be transferred to a low resource language with minimum overhead in terms of data annotation requirements and feature engineering. This trans- fer setting introduces additional challenges such as varying character usage conventions across lan- guages with same script, differing scripts, lack of NE transliteration, varying morphology, different lexicons and mutual non-intelligibility to name a few. We propose the following changes over prior work (Lample et al., 2016) to address the challenges of the low-resource transfer setting. We use: 1. Language universal phonological character representations instead of orthographic ones. 2. Attention over characters of a word while labeling it with an NE category. 1462 Figure 1: Attentional LSTM-CRF architecture. li denotes …

Content truncated to 500 words