Submodular Optimization-based Diverse Paraphrasing and its Effectiveness in Data Augmentation

Proceedings of NAACL-HLT 2019, pages 3609–3619 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3609 Submodular Optimization-based Diverse Paraphrasing and its Effectiveness in Data Augmentation Ashutosh Kumar∗1 Satwik Bhattamishra∗2 † Manik Bhandari1 Partha Talukdar1 1 Indian Institute of Science, Bangalore, India 2 Birla Institute of Technology and Science, Pilani, India {ashutosh, ppt}@iisc.ac.in, {satwik55, mbbhandarimanik}@gmail.com Abstract Inducing diversity in the task of paraphras- ing is an important problem in NLP with ap- plications in data augmentation and conver- sational agents. Previous paraphrasing ap- proaches have mainly focused on the issue of generating semantically similar paraphrases, while paying little attention towards diversity. In fact, most of the methods rely solely on top-k beam search sequences to obtain a set of paraphrases. The resulting set, however, con- tains many structurally similar sentences. In this work, we focus on the task of obtaining highly diverse paraphrases while not compro- mising on paraphrasing quality. We provide a novel formulation of the problem in terms of monotone submodular function maximization, specifically targeted towards the task of para- phrasing. Additionally, we demonstrate the ef- fectiveness of our method for data augmenta- tion on multiple tasks such as intent classifi- cation and paraphrase recognition. In order to drive further research, we have made the source code available. 1 Introduction Paraphrasing is the task of rephrasing a given text in multiple ways such that the semantics of the generated sentences remain unaltered. Paraphras- ing Quality can be attributed to two key character- istics - fidelity which measures the semantic sim- ilarity between the input text and generated text, and diversity, which measures the lexical dissimi- larity between generated sentences. Many previous works (Prakash et al., 2016; Gupta et al., 2018; Li et al., 2018) address the task of obtaining semantically similar paraphrases. While it is essential to produce paraphrases with high fidelity, it is equally important, and in many ∗Equal Contribution †This research was conducted during the author’s intern- ship at the Indian Institute of Science, Bangalore. SOURCE – how do i increase body height ? REFERENCE – what do i do to increase my height ? BEAM SEARCH – how do i increase my height ? – how do i increase my body height ? – how do i increase the height ? – how would i increase my body height ? DIPS (OURS) – how could i increase my height ? – what should i do to increase my height ? – what are the fastest ways to increase my height ? – is there any proven method to increase height ? Table 1: Sample paraphrases generated by beam search and DiPS (our method). DiPS offers lexically diverse paraphrases without compromising on fidelity. cases desirable, to produce lexically diverse ones. Diversity in paraphrase generation finds applica- tions in text simplification (Nisioi et al., 2017; Xu et al., 2015), document summarization (Li et al., 2009; Nema et al., 2017), QA systems (Fader et al., 2013; Bernhard and Gurevych, 2008), data augmentation (Zhang et …

Content truncated to 500 words