Ranking-Based Autoencoder for Extreme Multi-label Classification

Proceedings of NAACL-HLT 2019, pages 2820–2830 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2820 Ranking-Based Autoencoder for Extreme Multi-label Classification Bingyu Wang Khoury College of CS Northeastern University rainicy@ccs.neu.edu Li Chen JD.com Inc. li.chen@jd.com Wei Sun Department of ECE North Carolina State University wsun12@ncsu.edu Kechen Qin Khoury College of CS Northeastern University qin.ke@husky.neu.edu Kefeng Li JD.com Inc. likefeng@jd.com Hui Zhou JD.com Inc. hui.zhou@jd.com Abstract Extreme Multi-label classification (XML) is an important yet challenging machine learn- ing task, that assigns to each instance its most relevant candidate labels from an extremely large label collection, where the numbers of la- bels, features and instances could be thousands or millions. XML is more and more on de- mand in the Internet industries, accompanied with the increasing business scale / scope and data accumulation. The extremely large label collections yield challenges such as computa- tional complexity, inter-label dependency and noisy labeling. Many methods have been pro- posed to tackle these challenges, based on dif- ferent mathematical formulations. In this pa- per, we propose a deep learning XML method, with a word-vector-based self-attention, fol- lowed by a ranking-based AutoEncoder archi- tecture. The proposed method has three ma- jor advantages: 1) the autoencoder simulta- neously considers the inter-label dependencies and the feature-label dependencies, by project- ing labels and features onto a common em- bedding space; 2) the ranking loss not only improves the training efficiency and accuracy but also can be extended to handle noisy la- beled data; 3) the efficient attention mecha- nism improves feature representation by high- lighting feature importance. Experimental re- sults on benchmark datasets show the pro- posed method is competitive to state-of-the-art methods. 1 Introduction and Related Work In multi-label classification (Tsoumakas and Katakis, 2007; Zhang and Zhou, 2014), one as- signs multiple labels to each instance. Multi-label classification has many real-word applications: for example, a movie may be associated with multi- ple genres, a web page may contain several top- ics, and an image can be tagged with a few ob- jects. In these classification tasks, labels often ex- hibit complex dependencies: for example, Doc- umentary and Sci-Fi are usually mutually exclu- sive movie genres, while Horror and Thriller are typically highly correlated. Predicting labels inde- pendently fails to capture these dependencies and suffers suboptimal performance (Tsoumakas and Katakis, 2007; Ghamrawi and McCallum, 2005; Li et al., 2016). Several methods that capture la- bel dependencies have been proposed, including Conditional Random Fields (CRF) (Lafferty et al., 2001; Ghamrawi and McCallum, 2005), Classi- fier Chains (CC) (Read et al., 2011; Dembczyn- ski et al., 2010), Conditional Bernoulli Mixtures (CBM) (Li et al., 2016), and Canonical Correlated AutoEncoder (C2AE) (Yeh et al., 2017). However, these methods typically only work well on small- to-medium scale datasets. Extreme multi-label classification (XML) is a multi-label classification task in which the num- ber of instances, features and labels are very large, often on the order of thousands to millions (Zubi- aga, 2012; Bhatia et al., 2015). It has numerous real-world …

Content truncated to 500 words