BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 238–252 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 238 Neural Syntactic Preordering for Controlled Paraphrase Generation Tanya Goyal and Greg Durrett Department of Computer Science The University of Texas at Austin tanyagoyal@utexas.edu, gdurrett@cs.utexas.edu Abstract Paraphrasing natural language sentences is a multifaceted process: it might involve replac- ing individual words or short phrases, local re- arrangement of content, or high-level restruc- turing like topicalization or passivization. Past approaches struggle to cover this space of para- phrase possibilities in an interpretable manner. Our work, inspired by pre-ordering literature in machine translation, uses syntactic trans- formations to softly “reorder” the source sen- tence and guide our neural paraphrasing model. First, given an input sentence, we derive a set of feasible syntactic rearrangements using an encoder-decoder model. This model operates over a partially lexical, partially syntactic view of the sentence and can reorder big chunks. Next, we use each proposed rearrangement to produce a sequence of position embeddings, which encourages our final encoder-decoder paraphrase model to attend to the source words in a particular order. Our evaluation, both au- tomatic and human, shows that the proposed system retains the quality of the baseline ap- proaches while giving a substantial increase in the diversity of the generated paraphrases.1 1 Introduction Paraphrase generation (McKeown, 1983; Barzilay and Lee, 2003) has seen a recent surge of inter- est, both with large-scale dataset collection and curation (Lan et al., 2017; Wieting and Gimpel, 2018) and with modeling advances such as deep generative models (Gupta et al., 2018; Li et al., 2019). Paraphrasing models have proven to be es- pecially useful if they expose control mechanisms that can be manipulated to produce diverse para- phrases (Iyyer et al., 2018; Chen et al., 2019b; Park et al., 2019), which allows these models to be em- ployed for data augmentation (Yu et al., 2018) and 1Data and code are available at https://github. com/tagoyal/sow-reap-paraphrasing Rearrangement Aware Paraphrasing Source Order Rewriting S Clippers won the game NP VP NP VBD XNP won YNP YNP won by XNP 4 3 1 2 Clippers won the game The game was won by the Clippers. Source order encoding Transformer seq2seq Figure 1: Overview of our paraphrase model. First, we choose various pairs of constituents to abstract away in the source sentence, then use a neural transducer to gen- erate possible reorderings of the abstracted sentences. From these, we construct a guide reordering of the in- put sentence which then informs the generation of out- put paraphrases. adversarial example generation (Iyyer et al., 2018). However, prior methods involving syntactic control mechanisms do not effectively cover the space of paraphrase possibilities. Using syntactic templates covering the top of the parse tree (Iyyer et al., 2018) is inflexible, and using fully-specified exemplar sentences (Chen et al., 2019b) poses the problem of how to effectively retrieve such sentences. For a particular input sentence, it is challenging to use these past approaches to enumerate the …

Content truncated to 500 words