Induction Networks for Few-Shot Text Classification

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3904–3913, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3904 Induction Networks for Few-Shot Text Classification Ruiying Geng1,2, Binhua Li2, Yongbin Li2∗ , Xiaodan Zhu3, Ping Jian1∗, Jian Sun2 1School of Computer Science and Technology, Beijing Institute of Technology 2 Alibaba Group, Beijing 3 ECE, Queen’s University {ruiying.gry,binhua.lbh,shuide.lyb,jian.sun}@alibaba-inc.com zhu2048@gmail.com pjian@bit.edu.cn Abstract Text classification tends to struggle when data is deficient or when it needs to adapt to unseen classes. In such challenging scenarios, recent studies have used meta-learning to simulate the few-shot task, in which new queries are compared to a small support set at the sample- wise level. However, this sample-wise com- parison may be severely disturbed by the var- ious expressions in the same class. Therefore, we should be able to learn a general repre- sentation of each class in the support set and then compare it to new queries. In this pa- per, we propose a novel Induction Network to learn such a generalized class-wise repre- sentation, by innovatively leveraging the dy- namic routing algorithm in meta-learning. In this way, we find the model is able to induce and generalize better. We evaluate the pro- posed model on a well-studied sentiment clas- sification dataset (English) and a real-world di- alogue intent classification dataset (Chinese). Experiment results show that on both datasets, the proposed model significantly outperforms the existing state-of-the-art approaches, prov- ing the effectiveness of class-wise generaliza- tion in few-shot text classification. 1 Introduction Deep learning has achieved a great success in many fields such as computer vision, speech recognition and natural language processing (Kuang et al., 2018). However, supervised deep learning is notoriously greedy for large labeled datasets, which limits the generalizability of deep models to new classes due to annotation cost. Hu- mans on the other hand are readily capable of rapidly learning new classes of concepts with few examples or stimuli. This notable gap provides a fertile ground for further research. ∗Corresponding authors: Y.Li and P.Jian. Few-shot learning is devoted to resolving the data deficiency problem by recognizing novel classes from very few labeled examples. The limi- tation of only one or very few examples challenges the standard fine-tuning method in deep learning. Early studies (Salamon and Bello, 2017) applied data augmentation and regularization techniques to alleviate the overfitting problem caused by data sparseness, only to a limited extent. Instead, re- searchers have explored meta-learning (Finn et al., 2017) to leverage the distribution over similar tasks, inspired by human learning. Contempo- rary approaches to few-shot learning often decom- pose the training procedure into an auxiliary meta- learning phase, which includes many meta-tasks, following the principle that the testing and train- ing conditions must match. They extract some transferable knowledge by switching the meta- task from mini-batch to mini-batch. As such, few- shot models can classify new classes with just a small labeled support set. However, existing approaches for …

Content truncated to 500 words