Target-oriented Fine-tuning for Zero-Resource Named Entity Recognition

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1603–1615 August 1–6, 2021. ©2021 Association for Computational Linguistics 1603 Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition Ying Zhang1∗, Fandong Meng2, Yufeng Chen1†, Jinan Xu1 and Jie Zhou2 1Beijing Key Lab of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing, China 2Pattern Recognition Center, WeChat AI, Tencent Inc, China {zhying,chenyf,jaxu}@bjtu.edu.cn {fandongmeng,withtomzhou}@tencent.com Abstract Zero-resource named entity recognition (NER) severely suffers from data scarcity in a spe- cific domain or language. Most studies on zero-resource NER transfer knowledge from various data by fine-tuning on different aux- iliary tasks. However, how to properly se- lect training data and fine-tuning tasks is still an open problem. In this paper, we tackle the problem by transferring knowledge from three aspects, i.e., domain, language and task, and strengthening connections among them. Specifically, we propose four practical guide- lines to guide knowledge transfer and task fine- tuning. Based on these guidelines, we design a target-oriented fine-tuning (TOF) framework to exploit various data from three aspects in a unified training manner. Experimental re- sults on six benchmarks show that our method yields consistent improvements over baselines in both cross-domain and cross-lingual scenar- ios. Particularly, we achieve new state-of-the- art performance on five benchmarks. 1 Introduction Named Entity Recognition (NER) is one of the fundamental tasks in natural language processing. Recently, zero-resource NER draws more and more attention in recent studies (T¨ackstr¨om et al., 2012; Jia et al., 2019; Bari et al., 2020; Liu et al., 2020b; Wu et al., 2020a). This task describes that, in a specific domain or language, there is no labeled training data for NER. Therefore, zero-resource NER severely suffers from data scarcity. As shown in Figure 1, the ideal training data for zero-resource NER is regarded as the Targets, which should satisfy two conditions at the same time: a) in the target domain or language, and b) annotated with NER labels. Thus it is intuitive to ∗ Work was done when Ying Zhang was interning at Pattern Recognition Center, WeChat AI, Tencent Inc, China. †Yufeng Chen is the corresponding author. Spanish: Sao Paulo (Brasil), 23 may (EFECOM). NER: <Sao Paulo, LOC>, <Brasil, LOC>, <EFECOM, LOC> Spanish: Como se conoce popularmente en Brasil al tenista. English: Brazil to use hovercrafts for Amazon travel. News: Santander, 23 may (EFE) Twitter: RT @Gabriele_Corno: Beach by Josh Adamski #meditation #inspiration #CGE http://t.co/ParMW4CG4X NER: <Brazil, LOC>, <Amazon, ORG> MRC: Who has the most to lose? I apologize. W NJ has the most to lose. Domain Language Task Targets Figure 1: The example of the Targets and essential knowledge from three aspects, i.e., Task, Language, and Domain. The middle rectangle denotes an Span- ish NER example in news domain, which is referred as the Targets. The rounded rectangle above the Tar- gets denotes knowledge from different tasks. The bot- tom left one denotes essential knowledge in Spanish and English languages. The bottom right one denotes knowledge in News and Twitter domains. augment training data or transfer knowledge from three …

Content truncated to 500 words