Embracing Non-Traditional Linguistic Resources for Low-resource Language Name Tagging

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 362–372, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Embracing Non-Traditional Linguistic Resources for Low-resource Language Name Tagging Boliang Zhang1, Di Lu1, Xiaoman Pan1, Ying Lin1, Halidanmu Abudukelimu2, Heng Ji1, Kevin Knight3 1 Computer Science Department, Rensselaer Polytechnic Institute {zhangb8,lud2,panx2,liny9,jih}@rpi.edu 2 Computer Science Department, Tsinghua University abdklmhldm@gmail.com 3 Information Sciences Institute, University of Southern California knight@isi.edu Abstract Current supervised name tagging ap- proaches are inadequate for most low- resource languages due to the lack of anno- tated data and actionable linguistic knowl- edge. All supervised learning methods (including deep neural networks (DNN)) are sensitive to noise and thus they are not quite portable without massive clean annotations. We found that the F-scores of DNN-based name taggers drop rapidly (20%-30%) when we replace clean man- ual annotations with noisy annotations in the training data. We propose a new so- lution to incorporate many non-traditional language universal resources that are read- ily available but rarely explored in the Nat- ural Language Processing (NLP) commu- nity, such as the World Atlas of Linguis- tic Structure, CIA names, PanLex and sur- vival guides. We acquire and encode vari- ous types of non-traditional linguistic re- sources into a DNN name tagger. Ex- periments on three low-resource languages show that feeding linguistic knowledge can make DNN significantly more robust to noise, achieving 8%-22% absolute F- score gains on name tagging without using any human annotation 1. 1 Introduction There is a general agreement that Deep Neural Networks provides a general, powerful underlying model for Information Extraction (IE), confirmed by improved state-of-the-art performance on many tasks such as name tagging (Chiu and Nichols, 2016; Lample et al., 2016), relation classifica- tion (Zeng et al., 2014; Liu et al., 2015; Nguyen 1We make all cleaned resources and converted linguistic features publicly available at http://nlp.cs.rpi.edu/denoise and Grishman, 2015b; Yang et al., 2016) and event detection (Nguyen and Grishman, 2015b; Chen et al., 2015; Nguyen and Grishman, 2015a, 2016; Feng et al., 2016). For example, our experiments on several languages show that a DNN-based name tagger generally outperforms (up to 6% F-score gain) a Conditional Random Fields (CRFs) model trained from the same labeled data and feature set. DNN architecture is attractive to couple with char- acter/word embeddings for IE tasks because it is easy to learn and usually effective enough to elim- inate the need of explicit linguistic feature design. However, training general models like DNN usually requires a massive amount of clean an- notated data, which is often not available for low-resource languages and difficult to obtain during emergent settings (Zhang et al., 2016a). In order to compensate this data requirement, various automatic annotation generation methods have been proposed, including knowledge base driven distant supervision (An et al., 2003; Mintz et al., 2009; Ren et al., 2015), cross-lingual pro- jection (Li et al., 2012; Kim et al., 2012; Che et al., 2013; Wang et al., 2013; Wang and Man- ning, 2014; Zhang et …

Content truncated to 500 words