Sentence-level Revision with Neural Reinforcement Learning

Sentence-level Revision with Neural Reinforcement Learning Zhendong Du Waseda University Kitakyushu, Japan zhendong@fuji.waseda.jp Kenji Hashimoto Waseda University Kitakyushu, Japan kenji.hashimoto@waseda.jp Abstract The objective of Sentence-level Revision (SentRev) is to enhance the fluency of En- glish writing; however, the performance of the three baseline methods is notably suboptimal. In this study, we propose a method utilizing neural reinforcement learning, tailored to the specific character- istics of this task, which has resulted in su- perior performance over the baseline meth- ods, surpassing them in multiple evalua- tion metrics. Moreover, we have identified conspicuous bottlenecks in SentRev’s effi- cacy in improving the fluency of English writing. Keywords: English Writing Assistant, SentRev, NRL 1 Introduction The inadequate English writing proficiency of many non-native English speakers renders their academic English writing a challenging task, hence academic writing assistant has be- come a popular downstream task in the field of Natural Language Processing (NLP). How- ever, much of the previous work has predom- inantly concentrated on English Grammati- cal correction (GEC), with scarce results pub- lished concerning the more challenging aspect of English writing fluency enhancement. (Ito et al., 2019) has introduced Sentence- level-revision (SentRev), a task dedicated to enhancing the fluency of English writing. The authors have established baseline performance for the task at hand by employing methodolo- gies from a variety of other Natural Language Processing tasks. However, significant room for improvement in baseline performance re- mains. In pursuit of an optimized approach for the task at hand, we conducted a comprehen- sive analysis of its characteristics. Our inves- tigation revealed that the task inherently in- volves iterative sentence-level revisions aimed at enhancing English writing fluency. This as- pect aligns closely with the self-improving na- ture of reinforcement learning, which contin- uously refines its performance to achieve su- perior outcomes. Consequently, we adopted a reinforcement learning paradigm tailored to the unique requirements of this task and em- ployed the GLUE (Wang et al., 2018) as the reward function to drive the optimization pro- cess. An evaluation was conducted on the SMITH dataset (Ito et al., 2019), and the re- sults substantiated that our proposed method exhibits a significant improvement over the baseline performance. Additionally, our ex- perimental findings have revealed limitations within SentRev, resulting in conspicuous bot- tlenecks in the enhancement of English writing fluency. 2 Related Works 2.1 Grammatical Error Correction (GEC) The objective of Grammatical Error Correc- tion (GEC) is to transform a sentence S with grammatical errors into a corrected version, denoted as S’. Given its nature of transforming a sequence output into a new sequence, mod- ern approaches to this task commonly treat it as a machine translation problem. In essence, it involves ′′translating′′ a sentence with gram- matical errors into a corrected sentence. With the introduction of the Transformer (Vaswani et al., 2017), significant advance- ments have been made in the GEC task over the past few years, particularly in the do- The 35th Conference on Computational Linguistics and Speech Processing (ROCLING 2023) Taipei City, Taiwan, October 20-21, 2023. …

Content truncated to 500 words