Efficient Strategies for Hierarchical Text Classification: External Knowledge and Auxiliary Tasks

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2252–2257 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2252 Efficient Strategies for Hierarchical Text Classification: External Knowledge and Auxiliary Tasks Kervy Rivas Rojas, Gina Bustamante, Arturo Oncevay‡, Marco A. Sobrevilla Cabezudo† Research Group on Artificial Intelligence, Pontificia Universidad Cat´olica del Per´u, Peru ‡School of Informatics, University of Edinburgh, Scotland †Instituto de Ciˆencias Matem´aticas e de Computac¸˜ao, Universidade de S˜ao Paulo, Brazil k.rivas@pucp.pe, gina.bustamante@pucp.edu.pe, a.oncevay@ed.ac.uk, msobrevillac@usp.br Abstract In hierarchical text classification, we perform a sequence of inference steps to predict the cat- egory of a document from top to bottom of a given class taxonomy. Most of the studies have focused on developing novels neural net- work architectures to deal with the hierarchi- cal structure, but we prefer to look for efficient ways to strengthen a baseline model. We first define the task as a sequence-to-sequence prob- lem. Afterwards, we propose an auxiliary syn- thetic task of bottom-up-classification. Then, from external dictionaries, we retrieve textual definitions for the classes of all the hierarchy’s layers, and map them into the word vector space. We use the class-definition embeddings as an additional input to condition the predic- tion of the next layer and in an adapted beam search. Whereas the modified search did not provide large gains, the combination of the auxiliary task and the additional input of class- definitions significantly enhance the classifica- tion accuracy. With our efficient approaches, we outperform previous studies, using a dras- tically reduced number of parameters, in two well-known English datasets. 1 Introduction Hierarchical text classification (HTC) aims to cat- egorise a textual description within a set of la- bels that are organized in a structured class hierar- chy (Silla and Freitas, 2011). The task is perceived as a more challenging problem than flat text classi- fication, since we need to consider the relationships of the nodes from different levels in the class tax- onomy (Liu et al., 2019). Both flat text classification and HTC have been tackled using traditional machine learning classi- fiers (Liu et al., 2005; Kim et al., 2006) or deep neural networks (Peng et al., 2018; Conneau et al., 2017). Nevertheless, the majority of the latest ap- proaches consider models with a large number of parameters that require extended training time. In the flat-classification scenario, some studies have addressed the problem of efficiency by proposing methods that do not focus on the model architec- ture, but in external ways of improving the results (Joulin et al., 2017; Howard and Ruder, 2018). However, the listed strategies are still underdevel- oped for HTC, and the most recent and effective methods are still computationally expensive (Yang et al., 2019; Banerjee et al., 2019). The described context opens our research ques- tion: How can we improve HTC at a lower com- putational cost? Therefore, our focus and main contributions are: • A robust model for HTC, with few parame- ters and short training time, that follows the paradigm of sequence-to-sequence learning. …

Content truncated to 500 words