Supervised Neural Clustering via Latent Structured Output Learning: Application to Question Intents

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3364–3374 June 6–11, 2021. ©2021 Association for Computational Linguistics 3364 Supervised Neural Clustering via Latent Structured Output Learning: Application to Question Intents Iryna Haponchyk DISI, University of Trento Povo (TN), Italy gaponchik.irina@gmail.com Alessandro Moschitti Amazon Alexa AI Manhattan Beach, CA, USA amosch@amazon.com Abstract Previous pre-neural work on structured predic- tion has produced very effective supervised clustering algorithms using linear classifiers, e.g., structured SVM or perceptron. However, these cannot exploit the representation learn- ing ability of neural networks, which would make supervised clustering even more pow- erful, i.e., general clustering patterns can be learned automatically. In this paper, we de- sign neural networks based on latent structured prediction loss and Transformer models to ap- proach supervised clustering. We tested our methods on the task of automatically recreat- ing categories of intents from publicly avail- able question intent corpora. The results show that our approach delivers 95.65% of F1, out- performing the state of the art by 17.24%. 1 Introduction Recent years have witnessed a vast spread of virtual assistants, such as Google Home, Siri and Alexa, which are based on the research areas of Conver- sational Agents and Question Answering. When designing such systems, the creation of classes of expected questions, aka intents, is essential for building the main states of a dialog manager. In par- ticular, when an assistant is designed for a specific domain, a knowledge engineer needs to analyze typical user’s questions, answered by human oper- ators. This work would be greatly sped up, if the engineers could have questions clustered according to the different topics they ask for. For example, the following questions/requests from the intent dataset by Larson et al. (2019): i want to switch to direct deposit set up direct deposit for me how do i go about setting up direct deposit all have a common intent of making a direct deposit. Thus, the dialog designer will create this intent, if the cluster captures a large number of requests. However, for being effective, the clustering al- gorithm must demonstrate a sufficient accuracy, which is often not the case for completely unsuper- vised methods. Thus, supervised clustering (Finley and Joachims, 2005), which exploits some training data of the target domain, e.g., previously designed clusters, to discover new clusters, is a viable ap- proach. A seminal work on structured prediction was Latent Structural Support Vector Machines (LSSVM) by Yu and Joachims (2009). Recently, Haponchyk et al. (2018) have shown that LSSVM as well as the Latent Structured Perceptron (LSP) by Fernandes et al. (2014), originally designed for coreference resolution, were also effective, when provided with the appropriate node similarity func- tion, for clustering questions into intents. These ap- proaches used traditional feature engineering (ques- tion similarity) and a linear classifier, i.e., SVM, which can be highly improved by neural networks, and pre-trained Transformers, e.g., Devlin et al. (2019). Indeed, neural models enable representa- tion learning, which can …

Content truncated to 500 words