An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5012 - 5024 July 10-15, 2022 ©2022 Association for Computational Linguistics An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling Peiyi Wang1∗‡, Runxin Xu1∗, Tianyu Liu2, Qingyu Zhou2, Yunbo Cao2, Baobao Chang1, Zhifang Sui1† 1 Key Laboratory of Computational Linguistics, Peking University, MOE, China 2 Tencent Cloud Xiaowei {wangpeiyi9979, runxinxu}@gmail.com {rogertyliu, qingyuzhou, yunbocao}@tencent.com {chbb, szf}@pku.edu.cn Abstract Few-Shot Sequence Labeling (FSSL) is a canonical paradigm for the tagging models, e.g., named entity recognition and slot fill- ing, to generalize on an emerging, resource- scarce domain. Recently, the metric-based meta-learning framework has been recognized as a promising approach for FSSL. However, most prior works assign a label to each token based on the token-level similarities, which ig- nores the integrality of named entities or slots. To this end, in this paper, we propose ESD, an Enhanced Span-based Decomposition method for FSSL. ESD formulates FSSL as a span-level matching problem between test query and sup- porting instances. Specifically, ESD decom- poses the span matching problem into a se- ries of span-level procedures, mainly includ- ing enhanced span representation, class pro- totype aggregation and span conflicts resolu- tion. Extensive experiments show that ESD achieves the new state-of-the-art results on two popular FSSL benchmarks, FewNERD and SNIPS, and is proven to be more ro- bust in the nested and noisy tagging scenarios. Our code is available at https://github. com/Wangpeiyi9979/ESD. 1 Introduction Many natural language understanding tasks can be formulated as sequence labeling tasks, such as Named Entity Recognition (NER) and Slot Filling (SF) tasks. Most prior works on sequence labeling follow the supervised learning paradigm, which requires large-scale annotated data and is limited to pre-defined classes. In order to generalize on the emerging, resource-scare domains, Few-Shot Se- quence Labeling (FSSL) has been proposed (Hou et al., 2020; Yang and Katiyar, 2020). In FSSL, the model (typically trained on the source domain *Equal contribution. †Corresponding author. ‡Contribution during internship in Tencent. query : Albert Einstein was born in Germany S1: [Steve Jobs]PER created [Apple]ORG S2: [Isaac Newton]PER studied at [Cambridge University]ORG Model (never accesses “PER” and “ORG” data) query : [Albert Einstein]PER was born in Germany Figure 1: A 2-way 2-shot few-shot NER task. The model needs to learn new entities with few examples. data) needs to solve the N-way (N unseen classes) K-shot (only K annotated examples for each class) task in the target domain. Figure 1 shows a 2-way 2-shot target domain few-shot NER task, where ‘PER’ and ‘ORG’ are 2 unseen entity types, and in the training set S = {S1, S2}, both of them have only 2 annotated entities. The tagging models should annotate ‘Albert Einstein’ in the test sen- tence q as a ‘PER’ according to S. Recently, metric-based meta-learning (MBML) methods have become the mainstream and state-of- the-art methods in FSSL (Hou et al., 2020; Ding et al., 2021), which train the models on the tasks sampled from the source …

Content truncated to 500 words