Knowledge Guided Metric Learning for Few-Shot Text Classification

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3266–3271 June 6–11, 2021. ©2021 Association for Computational Linguistics 3266 Knowledge Guided Metric Learning for Few-Shot Text Classification Dianbo Sui1,2, Yubo Chen1,2, Binjie Mao1,2, Delai Qiu3, Kang Liu1,2, Jun Zhao1,2 1 National Laboratory of Pattern Recognition, Institute of Automation, CAS 2 School of Artificial Intelligence, University of Chinese Academy of Sciences 3 Beijing Unisound Information Technology Co., Ltd. {dianbo.sui, yubo.chen, binjie.mao, kliu, jzhao}@nlpr.ia.ac.cn qiudelai@unisound.com Abstract Humans can distinguish new categories very efficiently with few examples, largely due to the fact that human beings can leverage knowl- edge obtained from relevant tasks. However, deep learning based text classification model tends to struggle to achieve satisfactory perfor- mance when labeled data are scarce. Inspired by human intelligence, we propose to intro- duce external knowledge into few-shot learn- ing to imitate human knowledge. A novel pa- rameter generator network is investigated to this end, which is able to use the external knowledge to generate different metrics for dif- ferent tasks. Armed with this network, simi- lar tasks can use similar metrics while differ- ent tasks use different metrics. Through exper- iments, we demonstrate that our method out- performs the SoTA few-shot text classification models. 1 Introduction Humans are adept at quickly learning from a small number of examples. This motivates research of few-shot learning (Vinyals et al., 2016; Finn et al., 2017), which aims to recognize novel categories from very few labeled examples. The key challenge in few-shot learning is to make full use of the limited labeled examples to find the “right” generalizations. Metric-based ap- proaches (Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018; Li et al., 2019; Zhang et al., 2020) are effective ways to address this challenge, which learn to represent examples in an appropriate feature space and use a distance metric to predict labels. However, directly employing metric-based approaches in text classification faces a problem that tasks are diverse and significantly different from each other, since words that are highly infor- mative for one task may not be relevant for other tasks (Bao et al., 2019). Therefore, a single met- ric is insufficient to cope with all these tasks in few-shot text classification (Yu et al., 2018). To adapt metric learning to significantly diverse tasks, we propose a knowledge guided metric learn- ing method. This method is inspired by the fact that human beings approach diverse tasks armed with knowledge obtained from relevant tasks (Lake et al., 2017). We use external knowledge from the knowledge base (KB) to imitate human knowledge, whereas the role of external knowledge has been ignored in previous methods (Yu et al., 2018; Bao et al., 2019; Geng et al., 2019, 2020). In detail, we resort to distributed representations of the KB instead of symbolic facts, since symbolic facts face the issues of poor generalization and data sparsity. Based on such KB embeddings, we investigate a novel parameter generator network (Ha …

Content truncated to 500 words