Dual Adversarial Neural Transfer for Low-Resource Named Entity Recognition

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3461–3471 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3461 Dual Adversarial Neural Transfer for Low-Resource Named Entity Recognition Joey Tianyi Zhou1,†, Hao Zhang2,†, Di Jin3, Hongyuan Zhu4,‡, Meng Fang5, Rick Siow Mong Goh1, Kenneth Kwok1 1IHPC, A*STAR 2A*AI, A*STAR 3CSAIL, MIT 4I2R, A*STAR 5Tencent Robotics X {zhouty,gohsm,kenkwok}@ihpc.a-star.edu.sg, {zhang hao@scei,zhuh@i2r}.a-star.edu.sg, jindi15@mit.edu, mfang@tencent.com Abstract We propose a new neural transfer method termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER). Specifically, two variants of DATNet, i.e., DATNet-F and DATNet-P, are investigated to explore effec- tive feature fusion between high and low re- source. To address the noisy and imbal- anced training data, we propose a novel Gen- eralized Resource-Adversarial Discriminator (GRAD). Additionally, adversarial training is adopted to boost model generalization. In ex- periments, we examine the effects of differ- ent components in DATNet across domains and languages, and show that significant im- provement can be obtained especially for low- resource data, without augmenting any ad- ditional hand-crafted features and pre-trained language model. 1 Introduction Named entity recognition (NER) is an important step in most natural language processing (NLP) applications. It detects not only the type of named entity, but also the entity boundaries, which re- quires deep understanding of the contextual se- mantics to disambiguate the different entity types of same tokens. To tackle this challenging prob- lem, most early studies were based on hand- crafted rules, which suffered from limited per- formance in practice. Current methods are de- voted to developing learning based algorithms, es- pecially neural network based methods, and have been advancing the state-of-the-art progressively (Collobert et al., 2011; Huang et al., 2015; Lam- ple et al., 2016; Chiu and Nichols, 2016; Ma and Hovy, 2016). These end-to-end models general- ize well on new entities based on features auto- matically learned from the data. However, when † The first two authors contributed equally. ‡ Corresponding author. the annotated corpora is small, especially in the low resource scenario (Zhang et al., 2016), the per- formance of these methods degrades significantly since the hidden feature representations cannot be learned adequately. Recently, more and more approaches have been proposed to address low-resource NER. Early works (Chen et al., 2010; Li et al., 2012) primar- ily assumed a large parallel corpus and focused on exploiting them to project information from high- to low-resource. Unfortunately, such a large parallel corpus may not be available for many low-resource languages. More recently, cross- resource word embedding (Fang and Cohn, 2017; Adams et al., 2017; Yang et al., 2017) was pro- posed to bridge the low- and high-resources and enable knowledge transfer. Although the afore- mentioned transfer-based methods show promis- ing performance in low-resource NER, there are two issues remain further study: 1) Representa- tion Difference - they did not consider the repre- sentation difference across resources and enforced the feature representation to be shared across lan- guages/domains; 2) Resource Data Imbalance - the …

Content truncated to 500 words