Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5798–5810, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 5798 Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space Dayiheng Liu♠∗ Yeyun Gong† Jie Fu♦ Yu Yan‡ Jiusheng Chen‡ Jiancheng Lv♠ Nan Duan† Ming Zhou† ♠College of Computer Science, Sichuan University †Microsoft Research Asia ♦ Mila ‡Microsoft losinuris@gmail.com Abstract In this paper, we propose a novel data aug- mentation method, referred to as Control- lable Rewriting based Question Data Aug- mentation (CRQDA), for machine reading comprehension (MRC), question generation, and question-answering natural language in- ference tasks. We treat the question data augmentation task as a constrained question rewriting problem to generate context-relevant, high-quality, and diverse question data sam- ples. CRQDA utilizes a Transformer autoen- coder to map the original discrete question into a continuous embedding space. It then uses a pre-trained MRC model to revise the ques- tion representation iteratively with gradient- based optimization. Finally, the revised ques- tion representations are mapped back into the discrete space, which serve as additional ques- tion data. Comprehensive experiments on SQuAD 2.0, SQuAD 1.1 question generation, and QNLI tasks demonstrate the effectiveness of CRQDA1. 1 Introduction Data augmentation (DA) is commonly used to improve the generalization ability and robustness of models by generating more training examples. Compared with the DA used in the fields of com- puter vision (Krizhevsky et al., 2012; Szegedy et al., 2015; Cubuk et al., 2019) and speech process- ing (Ko et al., 2015), how to design effective DA tailored to natural language processing (NLP) tasks remains a challenging problem. Unlike the general image DA techniques such as rotation and cropping, it is more difficult to synthesize new high-quality and diverse text. Asia.∗ Work is done during internship at Microsoft Research 1The source code and dataset will be available at https: //github.com/dayihengliu/CRQDA. Recently, some textual DA techniques have been proposed for NLP, which mainly focus on text clas- sification and machine translation tasks. One way is directly modifying the text data locally with word deleting, word order changing, and word replace- ment (Fadaee et al., 2017; Kobayashi, 2018; Wei and Zou, 2019; Wu et al., 2019). Another popular way is to utilize the generative model to generate new text data, such as back-translation (Sennrich et al., 2016; Yu et al., 2018), data noising tech- nique (Xie et al., 2017), and utilizing pre-trained language generation model (Kumar et al., 2020; Anaby-Tavor et al., 2020). Machine reading comprehension (MRC) (Ra- jpurkar et al., 2018), question generation (QG) (Du et al., 2017; Zhao et al., 2018) and, question-answering natural language inference (QNLI) (Demszky et al., 2018; Wang et al., 2018) are receiving attention in NLP community. MRC requires the model to find the answer given a paragraph2 and a question, while QG aims to generate the question for a given paragraph with or without a given answer. Given a question and a sentence in the relevant paragraph, QNLI requires the model to infer …

Content truncated to 500 words