Dynamic Memory Induction Networks for Few-Shot Text Classification

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1087–1094 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 1087 Dynamic Memory Induction Networks for Few-Shot Text Classification Ruiying Geng1, Binhua Li1, Yongbin Li1∗, Jian Sun1, Xiaodan Zhu2 1 Alibaba Group, Beijing 2 Ingenuity Labs Research Institute & ECE, Queen’s University {ruiying.gry,binhua.lbh,shuide.lyb,jian.sun}@alibaba-inc.com zhu2048@gmail.com Abstract This paper proposes Dynamic Memory Induc- tion Networks (DMIN) for few-shot text clas- sification. The model utilizes dynamic routing to provide more flexibility to memory-based few-shot learning in order to better adapt the support sets, which is a critical capacity of few- shot classification models. Based on that, we further develop induction models with query information, aiming to enhance the generaliza- tion ability of meta-learning. The proposed model achieves new state-of-the-art results on the miniRCV1 and ODIC dataset, improving the best performance (accuracy) by 2∼4%. Detailed analysis is further performed to show the effectiveness of each component. 1 Introduction Few-shot text classification, which requires mod- els to perform classification with a limited number of training instances, is important for many appli- cations but yet remains to be a challenging task. Early studies on few-shot learning (Salamon and Bello, 2017) employ data augmentation and regu- larization techniques to alleviate overfitting caused by data sparseness. More recent research leverages meta-learning (Finn et al., 2017; Zhang et al., 2018; Sun et al., 2019) to extract transferable knowledge among meta-tasks in meta episodes. A key challenge for few-shot text classification is inducing class-level representation from support sets (Gao et al., 2019), in which key information is often lost when switching between meta-tasks. Recent solutions (Gidaris and Komodakis, 2018) leverage a memory component to maintain mod- els’ learning experience, e.g., by finding from a supervised stage the content that is similar to the unseen classes, leading to the state-of-the-art per- formance. However, the memory weights are static ∗Corresponding author. during inference and the capability of the model is still limited when adapted to new classes. Another prominent challenge is the instance-level diversity caused by various reasons (Gao et al., 2019; Geng et al., 2019), resulting in the difficulty of finding a fixed prototype for a class (Allen et al., 2019). Recent research has shown that models can benefit from query-aware methods (Gao et al., 2019). In this paper we propose Dynamic Memory In- duction Networks (DMIN) to further tackle the above challenges. DMIN utilizes dynamic routing (Sabour et al., 2017; Geng et al., 2019) to render more flexibility to memory-based few-shot learning (Gidaris and Komodakis, 2018) in order to better adapt the support sets, by leveraging the routing component’s capacity in automatically adjusting the coupling coefficients during and after training. Based on that, we further develop induction models with query information to identify, among diverse instances in support sets, the sample vectors that are more relevant to the query. These two modules are jointly learned in DMIN. The proposed model achieves new state-of-the- art results on the miniRCV1 and ODIC datasets, improving the best performance by …

Content truncated to 500 words