Open Vocabulary Extreme Classification Using Generative Models

Findings of the Association for Computational Linguistics: ACL 2022, pages 1561 - 1583 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Open Vocabulary Extreme Classification Using Generative Models Daniel Simig, Fabio Petroni Pouya Yanki, Kashyap Popat, Christina Du, Sebastian Riedel, Majid Yazdani Meta AI {danielsimig,fabiopetroni,pya,kpopat,xiaodu, sriedel,myazdani}@fb.com Abstract The extreme multi-label classification (XMC) task aims at tagging content with a subset of labels from an extremely large label set. The label vocabulary is typically defined in advance by domain experts and assumed to capture all necessary tags. However in real world scenar- ios this label set, although large, is often in- complete and experts frequently need to refine it. To develop systems that simplify this pro- cess, we introduce the task of open vocabulary XMC (OXMC): given a piece of content, pre- dict a set of labels, some of which may be out- side of the known tag set. Hence, in addition to not having training data for some labels–as is the case in zero-shot classification–models need to invent some labels on-the-fly. We pro- pose GROOV, a fine-tuned seq2seq model for OXMC that generates the set of labels as a flat sequence and is trained using a novel loss inde- pendent of predicted label order. We show the efficacy of the approach, experimenting with popular XMC datasets for which GROOV is able to predict meaningful labels outside the given vocabulary while performing on par with state-of-the-art solutions for known labels. 1 Introduction Extreme multi-label classification (XMC) aims at predicting a set of labels for a given input instance from an extremely large labels set (Yen et al., 2016, 2017; Babbar and Sch¨olkopf, 2017, 2019). Exam- ples for applying extreme classification are labeling a new article with Wikipedia’s categories, classi- fying a product with catalog labels, classifying a resume into a collection of pertinent job titles. Despite the the scale of the label space, it is chal- lenging to a priori capture all the possible ways in which an input instance can be categorized, especially at the industrial scale. Real-world e- Commerce platforms, for instance, contain billions of products from thousands of different categories that are continuously updated by human curators for all sort of reasons (e.g., see category changes in eBay (2021)). In this work we introduce the open vocabulary XMC task, where we measure the ability of models to go beyond the given vocabulary and automati- cally propose new labels that might complement the existing ones and fill gaps in the vocabulary. Note that this differs from a zero-shot formulation of the XMC problem (Gupta et al., 2021) where, although no training instance is available for some labels, they are still present in the given vocabulary. To tackle the problem we propose GROOV, an autoregressive model that maps input sequences to a set of sequences. Inputs are documents/text, and outputs are collections of textual labels from an open vocabulary. We investigate multiple sequence- to-set-of-sequences instantiations, in particular an off-the shelf approach based on a encoder-decoder language model (T5, Raffel et …

Content truncated to 500 words