On the Helpfulness of Document Context to Sentence Simplification

Proceedings of the 28th International Conference on Computational Linguistics, pages 1411–1423 Barcelona, Spain (Online), December 8-13, 2020 1411 On the Helpfulness of Document Context to Sentence Simplification Renliang Sun, Zhe Lin, Xiaojun Wan Wangxuan Institute of Computer Technology, Peking University Center for Data Science, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University sunrenliangpku@gmail.com {linzhe, wanxiaojun}@pku.edu.cn Abstract Most of the research on text simplification is limited to sentence level nowadays. In this paper, we are the first to investigate the helpfulness of document context on sentence simplification and apply it to the sequence-to-sequence model. We firstly construct a sentence simplification dataset in which the contexts for the original sentence are provided by Wikipedia corpus. The new dataset contains approximately 116K sentence pairs with context. We then propose a new model that makes full use of the context information. Our model uses neural networks to learn the different effects of the preceding sentences and the following sentences on the current sentence and applies them to the improved transformer model. Evaluated on the newly constructed dataset, our model achieves 36.52 on SARI value, which outperforms the best performing model in the baselines by 2.46 (7.22%), indicating that context indeed helps improve sentence simplification. In the ablation experiment, we show that using either the preceding sentences or the following sentences as context can significantly improve simplification. 1 Introduction Text simplification is a hot issue in the field of natural language generation (NLG). It is also one of the critical needs of society (Woodsend and Lapata, 2011). Text simplification aims to adapt a complex text into a more readable version with the same meaning (Sulem et al., 2018b), which will benefit young children (Kajiwara et al., 2013) and non-native English speakers (Paetzold, 2015; Paetzold and Specia, 2016). It includes many ways to deal with the input text, such as deletion, reordering, paraphrase, and sentence separation (Saggion, 2017). Besides, text simplification is closely related to many natural language processing (NLP) tasks, such as machine translation (ˇStajner and Popovi´c, 2016; Hasler et al., 2017), paraphrase generation (Pavlick and Callison-Burch, 2016; Cao et al., 2017; Zhao et al., 2018) and text summarization (Li et al., 2017; Ma and Sun, 2017; Jin et al., 2020). In recent years, many researchers have proposed various models to improve the performance of text simplification. However, few people have explored the impact of document context (i.e., the preceding and following sentences of the original sentence to be simplified in a document) on text simplification, let alone establish a large-scale dataset containing context. Many examples like the one in Table 1 arouse our interest in exploring the influence of context. In the simplified sentence, “played and sang” is the simplification of “in her performances” in the original sentence. The phrase “sing as well as play” in the context may provide additional information to help the simplification. The phrase “at the bar” is retained because of the presence of “at the Midtown Bar” in the context. Correspondingly, there is no mention of …

Content truncated to 500 words