Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9986 - 9999 December 7-11, 2022 ©2022 Association for Computational Linguistics Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks Zae Myung Kim1,3∗ , Wanyu Du2 , Vipul Raheja3 , Dhruv Kumar3 , Dongyeop Kang1 1University of Minnesota, 2University of Virginia, 3Grammarly {kim01756,dongyeop}@umn.edu wd5jq@virginia.edu, {vipul.raheja,dhruv.kumar}@grammarly.com Abstract Iterative text revision improves text quality by fixing grammatical errors, rephrasing for bet- ter readability or contextual appropriateness, or reorganizing sentence structures through- out a document. Most recent research has fo- cused on understanding and classifying differ- ent types of edits in the iterative revision pro- cess from human-written text instead of build- ing accurate and robust systems for iterative text revision. In this work, we aim to build an end-to-end text revision system that can it- eratively generate helpful edits by explicitly detecting editable spans (where-to-edit) with their corresponding edit intents and then in- structing a revision model to revise the de- tected edit spans. Leveraging datasets from other related text editing NLP tasks, combined with the specification of editable spans, leads our system to more accurately model the pro- cess of iterative text refinement, as evidenced by empirical results and human evaluations. Our system significantly outperforms previous baselines on our text revision tasks and other standard text revision tasks, including gram- matical error correction, text simplification, sentence fusion, and style transfer. Through extensive qualitative and quantitative analysis, we make vital connections between edit inten- tions and writing quality, and better computa- tional modeling of iterative text revisions. 1 Introduction Text revision, naturally, is an iterative process. Writers are required to simultaneously and re- peatedly comprehend multiple requirements, such as covering the content, and following linguistic norms and discourse conventions, when producing well-written texts (Flower, 1980; Collins and Gen- tner, 1980; Vaughan and McDonald, 1986). Most recent text editing studies have either focused on general-purpose text revision (Malmi et al., 2019; *The work was done while Zae Myung Kim was interning at Grammarly. Tag Gran. Iter. LASERTAGGER (Malmi et al., 2019) O S  FELIX (Mallinson et al., 2020) O S&P  SEQ2EDITS (Stahlberg and Kumar, 2020) O S  ITERATER (Du et al., 2022b)  S&P  DELITERATER (Ours) I S&P  Table 1: Comparison with previous works. Gran. for Granularity: S for sentence-level and P for paragraph- level. Iter. for Iterativeness. Tag for the type of Edit Tagging: O for Edit Operations, I for Edit Intentions. Mallinson et al., 2020; Stahlberg and Kumar, 2020; Li et al., 2022), or targeted monolingual sequence transduction tasks individually, such as grammat- ical error correction (GEC) (Awasthi et al., 2019; Omelianchuk et al., 2020; Chen et al., 2020), text simplification (Dong et al., 2019; Kumar et al., 2020; Omelianchuk et al., 2021; Agrawal et al., 2021), and text style transfer (Madaan et al., 2020; Malmi et al., 2020; Reid and Zhong, 2021), among others. Despite their progress, these works are quite re- stricted in their generalizability to practical …

Content truncated to 500 words