What Helps Transformers Recognize Conversational Structure? Importance of Context, Punctuation, and Labels in Dialog Act Recognition

Augmenting Transformers with KNN-Based Composite Memory for Dialog Angela Fan Facebook AI Research Universit´e de Lorraine LORIA angelafan@fb.com Claire Gardent CNRS/LORIA claire.gardent@loria.fr Chlo´e Braud CNRS/IRIT chloe.braud@irit.fr Antoine Bordes Facebook AI Research abordes@fb.com Abstract Various machine learning tasks can benefit from access to external information of different modalities, such as text and images. Recent work has focused on learning architectures with large memories capable of storing this knowledge. We propose augmenting genera- tive Transformer neural networks with KNN- based Information Fetching (KIF) modules. Each KIF module learns a read operation to access fixed external knowledge. We apply these modules to generative dialog modeling, a challenging task where information must be flexibly retrieved and incorporated to maintain the topic and flow of conversation. We demon- strate the effectiveness of our approach by identifying relevant knowledge required for knowledgeable but engaging dialog from Wikipedia, images, and human-written dialog utterances, and show that leveraging this retrieved information improves model perfor- mance, measured by automatic and human evaluation. 1 Introduction Machine learning approaches to various tasks, such as game-playing or dialog, are often depen- dent on external information. This information can take multimodal forms, including structured knowledge bases, free text, and images, and also comes in overwhelmingly large quantities. A pressing challenge is to create models that can identify which specific elements of multiple information sources are relevant in a particular context, and incorporate them into standard archi- tectures on each task. In this work, we focus on human–machine dialog and how to efficiently retrieve external knowledge that is relevant to the dialog. We consider two scenarios and for each scenario, retrieve two types of knowledge: (i) knowledge about similar dialog contexts and (ii) external knowledge used to ground the conversation into real world information. Knowledge about similar dialog contexts allows for a hybrid retrieval/generative approachto dialog where the system response is generated based not only on a representation of the current dialog context and of the relevant world knowledge, but also based on a response retrieved from a similar dialog context. The retrieved knowledge can be viewed as providing information about structure and dialog sentences, or utterances: which response is likely given a similar context? External knowledge is also retrieved to improve the semantic content of the dialog model. In one scenario, Wizard of Wikipedia (Dinan et al. 2018), general topics are provided to crowdwor- kers, who are asked to have in-depth and specific conversations about these topics by referencing specific Wikipedia sentences as knowledge. In this scenario, external knowledge is retrieved from a pre-selected set of Wikipedia sentences associated with the current dialog topic. Retrieval aims to select the sentence that is most relevant at each step of the dialog and thereby to ground system responses in relevant world knowledge (e.g., by referring to Star Wars when talking about science fiction). In the other scenario, Engaging ImageChat (Shuster et al., 2020), crowdworkers are provided with images and asked to have a conversation 82 Transactions of the Association for Computational Linguistics, vol. 9, pp. …

Content truncated to 500 words