Fine-grained Knowledge Fusion for Sequence Labeling Domain Adaptation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4197–4206, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4197 Fine-grained Knowledge Fusion for Sequence Labeling Domain Adaptation Huiyun Yang1,2 Shujian Huang1,2 Xinyu Dai1,2 Jiajun Chen1,2 National Key Laboratory for Novel Software Technology, Nanjing, China1 Nanjing University, Nanjing, China2 yanghy@smail.nju.edu.cn {huangsj, daixinyu, chenjj}@nju.edu.cn Abstract In sequence labeling, previous domain adap- tation methods focus on the adaptation from the source domain to the entire target domain without considering the diversity of individ- ual target domain samples, which may lead to negative transfer results for certain sam- ples. Besides, an important characteristic of sequence labeling tasks is that different ele- ments within a given sample may also have di- verse domain relevance, which requires further consideration. To take the multi-level domain relevance discrepancy into account, in this pa- per, we propose a fine-grained knowledge fu- sion model with the domain relevance mod- eling scheme to control the balance between learning from the target domain data and learn- ing from the source domain model. Experi- ments on three sequence labeling tasks show that our fine-grained knowledge fusion model outperforms strong baselines and other state- of-the-art sequence labeling domain adapta- tion methods. 1 1 Introduction Sequence labeling tasks, such as Chinese word segmentation (CWS), POS tagging (POS) and named entity recognition (NER), are fundamen- tal tasks in natural language processing. Recently, with the development of deep learning, neural se- quence labeling approaches have achieved pretty high accuracy (Chen et al., 2017; Zhang and Yang, 2018), relying on large-scale annotated corpora. However, most of the standard annotated corpora belong to the news domain, and models trained on these corpora will get sharp declines in per- formance when applied to other domains like so- cial media, forum, literature or patents (Daume III, 2007; Blitzer et al., 2007), which limits their ap- plication in the real world. Domain adaptation 1Our code is available at https://github.com/yhy1117/ FGKF-DA. Types Cases Strongly Ops Steve Jobs resigned as CEO of Apple. Share prices are rising soooo fast! Weakly Alas as time goes by, hair’s gone. Rock to 204 Section next week! Table 1: Tweets from the social media domain have different degrees of relevance to the source domain (news). Within each case, the bold part is strongly rel- evant and the italic part is weakly relevant. aims to exploit the abundant information of well- studied source domains to improve the perfor- mance in target domains (Pan and Yang, 2010), which is suitable to handle this issue. Following Daume III (2007), we focus on the supervised do- main adaptation setting, which utilizes large-scale annotated data from the source domain and small- scale annotated data from the target domain. For sequence labeling tasks, each sample is usu- ally a sentence, which consists of a sequence of words/Chinese characters, denoted as the element. We notice an interesting phenomenon: different target domain samples may have varying degrees of domain …

Content truncated to 500 words