Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 313–327 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 313 Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding Guokan Shang1,2, Antoine J.-P. Tixier1, Michalis Vazirgiannis1,3, Jean-Pierre Lorr´e2 1 ´Ecole Polytechnique, 2Linagora, 3AUEB Abstract Abstractive community detection is an im- portant spoken language understanding task, whose goal is to group utterances in a con- versation according to whether they can be jointly summarized by a common abstractive sentence. This paper provides a novel ap- proach to this task. We first introduce a neural contextual utterance encoder featuring three types of self-attention mechanisms. We then train it using the siamese and triplet energy- based meta-architectures. Experiments on the AMI corpus show that our system outperforms multiple energy-based and non-energy based baselines from the state-of-the-art. Code and data are publicly available1. 1 Introduction Today, large amounts of digital text are generated by spoken or written conversations, let them be human-human (customer service, multi-party meet- ings) or human-machine (chatbots, virtual assis- tants). Such text comes in the form of transcrip- tions. A transcription is a list of time-ordered text fragments called utterances. Abstractive summa- rization of conversations is an open problem in NLP. Previous work (Mehdad et al., 2013; Oya et al., 2014; Banerjee et al., 2015; Shang et al., 2018) decomposes this task into two subtasks a and b as shown in Fig. 1. Subtask a, or Abstractive Community Detection (ACD), is the focus of this paper. It consists in grouping utterances according to whether they can be jointly summarized by a common abstractive sentence (Murray et al., 2012). Such groups of ut- terances are called abstractive communities. Once they are obtained, an abstractive sentence is gen- erated for each group (subtask b), thus forming the final summary. ACD includes, but is a more 1https://bitbucket.org/guokan_shang/ abscomm transcription abstractive communities abstractive summary extracted utterances a1 a a2 b Figure 1: Abstractive summarization of conversations. general problem than, topic clustering. Indeed, as shown in Fig. 2, communities should capture more complex relationship than simple semantic similar- ity. Also, two utterances may be part of the same community even if they are not close to each other in the transcription. Finally, a given utterance may belong to more than one community, which results in overlapping groupings (e.g., A and D in Fig. 2), or be a singleton community (B in Fig. 2). In this paper, we depart from previous work and argue that the ACD subtask should be bro- ken down into two steps, a1 and a2 in Fig. 1. That is, summary-worthy utterances should first be extracted from the transcription (a1), and then, grouped into abstractive communities (a2). This process is more consistent with the way humans treat the summarization task. E.g., during the cre- ation of the AMI corpus (McCowan et al., 2005), annotators were first asked to extract summary- worthy utterances from the …

Content truncated to 500 words