AGRank: Augmented Graph-based Unsupervised Keyphrase Extraction

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1–11 November 20–23, 2022. ©2022 Association for Computational Linguistics 1 Chasing the Tail with Domain Generalization: A Case Study on Frequency-Enriched Datasets Manoj Kumar1, Anna Rumshisky1,2, Rahul Gupta1 1Alexa AI, Amazon 2Department of Computer Science, University of Massachusetts Lowell {abithm,gupra}@amazon.com arum@cs.uml.edu Abstract Natural language understanding (NLU) tasks are typically defined by creating an annotated dataset in which each utterance is encountered once. Such data does not resemble real-world natural language interactions in which certain utterances are encountered frequently, others rarely. For deployed NLU systems, this is a vital problem, since the underlying machine learning (ML) models are often fine-tuned on typical NLU data, in which utterance fre- quency is never factored in, and then applied to real-world data with a very different distri- bution. Such systems need to maintain inter- pretation consistency for the high-frequency (head) utterances, while also doing well on low-frequency (tail) utterances. We propose an alternative strategy that explicitly uses utter- ance frequency in training data to learn mod- els that are more robust to unknown distri- butions. We present a methodology to simu- late utterance usage in two public corpora and create two new corpora with head, body and tail segments. We evaluate several methods for joint intent classification and named entity recognition (referred to as IC-NER), and pro- pose to use two domain generalization (DG) approaches that we adapt to sequence label- ing task. The DG approaches demonstrate up to 7.02% relative improvement in semantic ac- curacy over baselines on the tail data. We provide insights as to why the proposed ap- proaches work and show that the reasons for the observed improvements do not align with those reported in previous work. 1 Introduction In academic research, natural language understand- ing (NLU) tasks are typically defined by creating annotated data, and then that data is used to train and evaluate machine learning models designed to solve that task. In such datasets, each utterance is typically encountered only once. But real-world natural language interactions do not look like that – in the real world, frequency matters. When people interact with each other “in the wild”, some things are said often ("Time to go to bed!"), others are infrequent to the point of being unique. The same holds for how people interact with digital assistants such as Alexa, Siri, or Google Assistant, which we use as the case study in this pa- per. The backbone of such commercial systems is the task of joint intent classification and named en- tity recognition (IC-NER) (Su et al., 2018; Coucke et al., 2018; Anantha et al., 2021). The goal of this task is to identify the intended action (play music, open calendar, etc) and actionable slots (names, places, objects, etc) from a user utterance. The underlying joint IC-NER models must cor- rectly handle both the frequently occurring requests and a …

Content truncated to 500 words