Imbalanced Chinese Multi-label Text Classification Based on Alternating Attention

Imbalanced Chinese Multi-label Text Classification Based on Alternating Attention Hongliang Bi, Han Hu, Pengyuan Liu∗ Beijing Language and Culture University, Beijing 100083, China Language Resources Monitoring & Research Center, Beijing 100083, China {201821198617, 201821198609}@stu.blcu.edu.cn, liupengyuan@blcu.edu.cn Abstract In this work, we construct an imbalanced Chinese multi-label text classification dataset, IMCM. The imbalance is mainly reflected in: (1) The degree of discrimination among labels is different. (2) The distribution of labels is moderately imbalanced. Then, we adopt sev- eral methods for multi-label classification and conduct thorough evaluation of them, which show that even the most competitive models struggle on this dataset. Therefore, to tackle these imbalanced problems, we proposed an alternating attention model, AltXML. Two at- tention heads which alternately reading se- quence enable the model capture different parts of the document rather than one point. Experimental results show that our proposed model significantly outperforms the state-of- the-art baselines in our IMCM dataset, and also achieves quite good results in several pub- lic datasets. 1 Introduction Multi-label classification (MLC) is an important task in natural language processing (NLP) due to the increasing number of fields where it can be ap- plied, such as text classification, tag suggestion, in- formation retrieval, and so on. Compared to single- label classification task, multi-label classification task aims to assign a set of labels to a single instance simultaneously. However, the number of label sets grows exponentially as the number of class labels in- creases and the uncertainty in the number of labels ∗ Corresponding Author. per instance inevitably makes the MLC task much more difficult to solve. Therefore, the key challenge of this task lies in the overwhelming and uncontrol- lable size of output space. Large amount of efforts have been done to- wards MLC task, including Binary Relevance (BR) (Boutell et al., 2004), Classifier Chains (CC) (Read et al., 2011), Label Powerset (LP) (Tsoumakas and Vlahavas, 2007), PD-Spare (Yen et al., 2016), SLEEC (Bhatia et al., 2015), AnnexML (Tagami, 2017), PfastreXML (Jain et al., 2016), Parabel (Prabhu et al., 2018).. In addition to the above meth- ods, neural networks provide some new approaches: CNN (Kim, 2014), CNN-RNN (Chen et al., 2017), SGM (Yang et al., 2018), etc. These methods have made great progress in capturing label correlations to cope with the exponential-sized output space, but still face the problem of high computational com- plexity and poor scalability. While utilizing correlations among labels is es- sential for MLC task, in real-word scenarios, there are no obvious semantic boundaries among some la- bels and some seemingly distinct labels may appear together, especially for text. Moreover, the distribu- tion of labels may be imbalanced. On the one hand, the number of instance belonging to a certain label may outnumber other labels. On the other hand, there may be a relatively high number of examples associated with the most common labels or infre- quent labels (Gibaja and Ventura, 2015). These may affect the performance of models utilizing correla- tions of labels. Therefore, it is important to explore the balance …

Content truncated to 500 words