HSCNN: A Hybrid-Siamese Convolutional Neural Network for Extremely Imbalanced Multi-label Text Classification

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6716–6722, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6716 HSCNN: A Hybrid-Siamese Convolutional Neural Network for Extremely Imbalanced Multi-label Text Classification Wenshuo Yang1,2, Jiyi Li2∗, Fumiyo Fukumoto2 and Yanming Ye1 Hangzhou Dianzi University, Hangzhou, China1 University of Yamanashi, Kofu, Japan2 yangwen7873@gmail.com, {jyli,fukumoto}@yamanashi.ac.jp, yeym@hdu.edu.cn Abstract The data imbalance problem is a crucial issue for the multi-label text classification. Some existing works tackle it by proposing imbal- anced loss objectives instead of the vanilla cross-entropy loss, but their performances re- main limited in the cases of extremely imbal- anced data. We propose a hybrid solution which adapts general networks for the head categories, and few-shot techniques for the tail categories. We propose a Hybrid-Siamese Con- volutional Neural Network (HSCNN) with ad- ditional technical attributes, i.e., a multi-task architecture based on Single and Siamese net- works; a category-specific similarity in the Siamese structure; a specific sampling method for training HSCNN. The results using two benchmark datasets and three loss objectives show that our method can improve the perfor- mance of Single networks with diverse loss ob- jectives on the tail or entire categories. 1 Introduction The data imbalance problem is a crucial issue for the multi-label text classification. In many cor- pora for the classification tasks, the number of in- stances of a category follows the long tail distribu- tion, where many tail categories has only a small number of instances. To handle this problem, some works sample hard examples for training (Shrivas- tava et al., 2016); some works address the prob- lem by proposing imbalance loss objectives, e.g., weighted cross-entropy loss and Focal loss (Lin et al., 2017), in place of the vanilla cross-entropy loss (Kim, 2014). Although the imbalanced loss objectives are better than the vanilla one, their per- formances remain limited in the cases of extremely imbalanced data because they are not designed for it, i.e., tail (head) categories have extremely small (large) numbers of instances. On the one hand, the recent few-shot learning techniques (e.g., optimization-based methods (Finn et al., 2017; Munkhdalai and Yu, 2017; Mishra et al., 2018), metric-based methods (Koch et al., 2015; Vinyals et al., 2016; Snell et al., 2017)) have become popular for various NLP tasks (Yu et al., 2018; Han et al., 2018). They have already shown the capability for few-shot classifications. They thus may also perform well for tail category clas- sification (some of the tail categories are few-shot, all have relatively small numbers of instances). On the other hand, for the head categories with many instances, general approaches such as the single CNN model (Kim, 2014; Liu et al., 2017) may be more effective in terms of performance and more efficient in terms of complexity. Therefore, our basic idea for tackling the problem of extremely imbalanced multi-label text classification is a hy- brid solution that adapts a general approach (i.e., a Single network) for head categories and a few-shot approach for tail categories, so that we can take …

Content truncated to 500 words