A Two-phase Prototypical Network Model for Incremental Few-shot Relation Classification

Proceedings of the 28th International Conference on Computational Linguistics, pages 1618–1629 Barcelona, Spain (Online), December 8-13, 2020 1618 A Two-phase Prototypical Network Model for Incremental Few-shot Relation Classification Haopeng Ren1,2, Yi Cai1,2∗, Xiaofeng Chen1,2, Guohua Wang1,2, Qing Li3 1School of Software Engineering, South China University of Technology, Guangzhou, China 2Key Laboratory of Big Data and Intelligent Robot (South China University of Technology), Ministry of Education 3Department of Computing, The Hong Kong Polytechnic University , Hong Kong, China ycai@scut.edu.cn Abstract Relation Classification (RC) plays an important role in natural language processing (NLP). Cur- rent conventional supervised and distantly supervised RC models always make a closed-world assumption which ignores the emergence of novel relations in an open environment. To in- crementally recognize the novel relations, current two solutions (i.e, re-training and lifelong learning) are designed but suffer from the lack of large-scale labeled data for novel relations. Meanwhile, prototypical network enjoys better performance on both fields of deep supervised learning and few-shot learning. However, it still suffers from the incompatible feature embed- ding problem when the novel relations come in. Motivated by them, we propose a two-phase prototypical network with prototype attention alignment and triplet loss to dynamically recog- nize the novel relations with a few support instances meanwhile without catastrophic forgetting. Extensive experiments are conducted to evaluate the effectiveness of our proposed model. 1 Introduction Relation Classification (RC) is a fundamental task in natural language processing (NLP), aiming to as- sign semantic relations to the entity pairs mentioned in sentences. Currently, conventional supervised (Zeng et al., 2014; Chen et al., 2020) or distantly supervised (Mintz et al., 2009; Zhang et al., 2019) RC models are widely used and achieve remarkable performance. They are always based on a closed-world assumption that the relations expressed in query instances must have appeared in the pre-defined relation set. It is clearly limited in many realistic scenarios, especially in a dynamic or open environment. As shown in Table 1, the query instance Q expresses the relation father which is out of the pre-defined rela- tion set. However, current supervised RC models ignore the novel relations (i.e., out of the pre-defined relations) and incorrectly classify this query instance into one of the pre-defined relations. To incremen- tally recognize the novel relations, two kinds of solutions, i.e. re-training (Gidaris and Komodakis, 2018) and lifelong learning (Wang et al., 2019; Han et al., 2020), are proposed. However, these two solutions still suffer from the lack of large-scale labeled data for novel relations. They are prone to overfitting on novel relations and may even lead to catastrophic forgetting on base ones (i.e., previous pre-defined relations) when given insufficient training data for novel relations (Xiang et al., 2019). In contrast, it is intuitive that the human can learn new knowledge after being taught just few instances (Snell et al., 2017). Based on this intuition, a series of few-shot RC models (Mishra et al., 2017; Gao et al., 2019a; Soares et al., 2019; Gao et al., 2020) are proposed and can effectively …

Content truncated to 500 words