Multi-Cell Compositional LSTM for NER Domain Adaptation

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5906–5917 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5906 Multi-Cell Compositional LSTM for NER Domain Adaptation Chen Jia†‡ and Yue Zhang‡§ †Fudan University, China ‡School of Engineering, Westlake University, China §Institute of Advanced Technology, Westlake Institute for Advanced Study, China {jiachen,zhangyue}@westlake.edu.cn Abstract Cross-domain NER is a challenging yet prac- tical problem. Entity mentions can be highly different across domains. However, the corre- lations between entity types can be relatively more stable across domains. We investigate a multi-cell compositional LSTM structure for multi-task learning, modeling each entity type using a separate cell state. With the help of entity typed units, cross-domain knowledge transfer can be made in an entity type level. Theoretically, the resulting distinct feature dis- tributions for each entity type make it more powerful for cross-domain transfer. Empiri- cally, experiments on four few-shot and zero- shot datasets show our method significantly outperforms a series of multi-task learning methods and achieves the best results. 1 Introduction Named entity recognition (NER) is a fundamental task in information extraction, providing necessary information for relation classification (Mooney and Bunescu, 2006), event detection (Popescu et al., 2011), sentiment classification (Mitchell et al., 2013), etc. NER is challenging because entity men- tions are an open set and can be ambiguous in the context of a sentence. Due to relatively high cost in manual labeling, cross-domain NER has received increasing research attention. Recently, multi-task learning methods (Yang et al., 2017; Wang et al., 2018, 2019; Zhou et al., 2019; Jia et al., 2019) have achieved great success for cross-domain NER. Other methods such as fine-tuning (Rodriguez et al., 2018), share-private (Cao et al., 2018; Lin and Lu, 2018) and knowledge distill (Yang et al., 2019) also show effectivenesses for cross-domain NER. There are three main source of challenges in cross-domain NER. First, instances of the same type entities can be different across domains. For example, typical person names can include “Trump” and “Clinton” in the political news domain, but “James” and “Trout” in the sports domain. Sec- ond, different types of entities can exhibit different degrees of dissimilarities across domains. For ex- ample, a large number of location names are shared in the political news domain and the sports domain, such as “Barcelona” and “Los Angeles”, but the case is very different for organization names across these domains. Third, even types of entities can be different across domains. For example, while disease names are a type of entities in the medical domain, it is not so in the biochemistry domain. We investigate a multi-cell compositional LSTM structure to deal with the above challenges by sep- arately and simultaneously considering the pos- sibilities of all entity types for each word when processing a sentence. As shown in Figure 1, the main idea is to extend a standard LSTM structure by using a separate LSTM cell to model the state for each entity type in a recurrent step. Intuitively, the model differs …

Content truncated to 500 words