More Like This: Semantic Retrieval with Linguistic Information

Data Augmentation for Intent Classification of German Conversational Agents in the Finance Domain Sophie Rentschler, Martin Riedl, Christian Stab, Martin R¨uckert Diamant Software GmbH, KI Kompetenzzentrum Robert-Bosch-Str. 7, 64293 Darmstadt {s.rentschler,m.riedl,c.stab,m.rueckert}@diamant-software.de Abstract In this paper, we focus on improving the intent recognition for a conversational agent. For lan- guages other than English, labeled data needed for training is often limited. Limitations rise even more when moving to specific domains. Here, our goal is to improve the intent recog- nition for a German conversational agent de- ployed in the financial sector. We treat this problem as a classification task. Using sev- eral augmentation techniques we expand the seed data used for training and compare the performance of the intent classifier. Applying a backtranslation approach using a commercial Machine Translation (MT) engine yields signif- icant improvement (p < 0.01) over a baseline system. 1 Introduction Conversational agents are becoming ubiquitous as lots of companies employ such agents for sup- porting and extending their services. Based on the applied domain, their languages – specifically, their vocabulary – constantly expand depending on the range of their services as well as the domain they are applied in. Machine learning methods are mainly used to teach conversational agents to react to user requests, called intents. For recognizing the intent, usually a natural language understanding (NLU) component is used. In order to train the NLU, for each intent various user utterances are required to understand the user and to discriminate between different intents. Due to the efforts required to manually create sufficient amounts of training data, we investigate if augmen- tation methods for enriching the training data helps to improve the performance. In this paper, we tackle various research ques- tions: Is it beneficial to add noise to the data by randomly replacing words or do we really need to have ”human”-readable paraphrases? Also, we will investigate which methods are suitable for au- tomatic paraphrase generation for intents. Most of the previous paraphrasing approaches for dialogue agents focus on training data from the open domain (e.g. booking a hotel, booking a table in a restau- rant, calling the police) written in English (Kumar et al., 2019; Quan and Xiong, 2019). In this pa- per, we research the applicability of augmentation approaches for German for the finance domain. We present results for a manually created dataset for the finance domain. Using paraphrasing meth- ods to augment training data used for machine learning differs from the typical paraphrasing sce- nario. Whereas for e.g. text simplification the goal is to generate sentences that can be read by hu- mans, here our goal is to teach the machine learning method to be more robust against textual variations when understanding natural language. In order to extend the data we use methods based on lexical resources (PPDB (Ganitkevitch et al., 2013), GermaNet (Hamp and Feldweg, 1997)), em- beddings and contextual embeddings (BERT (De- vlin et al., 2019)) as well as backtranslation using an out-of-the-box machine translation (MT) system. Based on our experiments we …

Content truncated to 500 words