Cross-Domain NER using Cross-Domain Language Modeling

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2464–2474 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2464 Cross-Domain NER using Cross-Domain Language Modeling Chen Jia†‡ , Xiaobo Liang3∗and Yue Zhang‡§ †Fudan University, China ‡School of Engineering, Westlake University, China 3Natural Language Processing Lab, Northeastern University, China §Institute of Advanced Technology, Westlake Institute for Advanced Study {jiachen,zhangyue}@westlake.edu.cn, liangxiaobo0309@gmail.com Abstract Due to limitation of labeled resources, cross- domain named entity recognition (NER) has been a challenging task. Most existing work considers a supervised setting, making use of labeled data for both the source and target do- mains. A disadvantage of such methods is that they cannot train for domains without NER data. To address this issue, we consider using cross-domain LM as a bridge cross-domains for NER domain adaptation, performing cross- domain and cross-task knowledge transfer by designing a novel parameter generation net- work. Results show that our method can effec- tively extract domain differences from cross- domain LM contrast, allowing unsupervised domain adaptation while also giving state-of- the-art results among supervised domain adap- tation methods. 1 Introduction Named entity recognition (NER) is a fundamen- tal task in information extraction and text under- standing. Due to large variations in entity names and flexibility in entity mentions, NER has been a challenging task in NLP. Cross-domain NER adds to the difficulty of modeling due to the difference in text genre and entity names. Existing meth- ods make use of feature transfer (Daum´e III, 2009; Kim et al., 2015; Obeidat et al., 2016; Wang et al., 2018) and parameters sharing (Lee et al., 2017; Sachan et al., 2018; Yang et al., 2017; Lin and Lu, 2018) for supervised NER domain adaptation. Language modeling (LM) has been shown use- ful for NER, both via multi-task learning (Rei, 2017) and via pre-training (Peters et al., 2018). In- tuitively, both noun entities and context patterns can be captured during LM training, which ben- efits the recognition of named entities. A natu- ral question that arises is whether cross-domain ∗Work done when visiting Westlake University. W News Domain Target Domain NER Task LM Task src, ner tgt ,ner  tgt ,lm src, lm  Vertical Transfer Vertical Transfer Horizontal Transfer T ner I T lm I D Isrc D tgt I Figure 1: Overview of the proposed model. LM training can benefit cross-domain NER. Fig- ure 1 shows one example, where there are rela- tively large training data in the news domain but no data or a small amount of data in a target domain. We are interested in transferring NER knowledge from the news domain to the target domain by con- trasting large raw data in both domains through cross-domain LM training. Naive multi-task learning by parameter sharing (Collobert and Weston, 2008) does not work ef- fectively in this multi-task, multi-domain setting due to potential conflict of information. To achieve cross-domain information transfer as shown in the red arrow, two types of connections must be made: …

Content truncated to 500 words