Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4114–4125 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4114 Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System Eda Okur, Saurav Sahay, Lama Nachman Intel Labs USA {eda.okur, saurav.sahay, lama.nachman}@intel.com Abstract Contextually aware intelligent agents are often required to understand the users and their surroundings in real-time. Our goal is to build Artificial Intelligence (AI) systems that can assist children in their learning process. Within such complex frameworks, Spoken Dialogue Systems (SDS) are crucial building blocks to handle efficient task-oriented communication with children in game-based learning settings. We are working towards a multimodal dialogue system for younger kids learning basic math concepts. Our focus is on improving the Natural Language Understanding (NLU) module of the task-oriented SDS pipeline with limited datasets. This work explores the potential benefits of data augmentation with paraphrase generation for the NLU models trained on small task-specific datasets. We also investigate the effects of extracting entities for conceivably further data expansion. We have shown that paraphrasing with model-in-the-loop (MITL) strategies using small seed data is a promising approach yielding improved performance results for the Intent Recognition task. Keywords: Spoken Dialogue System, Natural Language Understanding, Intent Classification, Entity Recognition, Para- phrase Generation, Data Augmentation 1. Introduction Building Artificial Intelligence (AI) systems that can assist students in the math learning process has been a challenging yet exciting area of research. Play-based learning systems can offer significant advantages in teaching fundamental mathematical concepts interac- tively, especially for younger kids (Skene et al., 2021). Such intelligent systems are often required to handle multimodal understanding of the children and their sur- roundings in real-time. Within these complex archi- tectures, Spoken Dialogue Systems (SDS) are crucial building blocks for carrying out efficient task-oriented communication with kids in game-based learning set- tings. This work presents our multimodal dialogue sys- tem for children learning basic math concepts. This di- alogue system technology needs to be constructed and evaluated carefully to handle goal-oriented interactions between the kids and a virtual conversational agent. This study primarily focuses on creating and improv- ing the Natural Language Understanding (NLU) mod- ule of a task-oriented SDS pipeline, especially with limited dataset resources. Building the NLU module of such goal-oriented SDS often involves the defini- tion of intents (and entities if necessary), creation of domain-specific and task-relevant datasets, annotation of the data with intents and entities, iterative training and evaluation of NLU models, and repeating this pro- cess for each new or updated use-cases. This work ex- plores the potential benefits of data augmentation with paraphrase generation for the NLU models trained on small-size task-specific datasets. The main NLU task we concentrate on improving is the Intent Recognition (IR) from possible user utterances. The ultimate goal of IR is to predict the user’s intent (i.e., what the user wants to accomplish within a task-oriented SDS) given an input utterance. In addition to paraphrasing the pos- sible user utterances …

Content truncated to 500 words