Neural CRF Model for Sentence Alignment in Text Simplification

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7943–7960 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7943 Neural CRF Model for Sentence Alignment in Text Simplification Chao Jiang, Mounica Maddela, Wuwei Lan, Yang Zhong, Wei Xu Department of Computer Science and Engineering The Ohio State University {jiang.1530, maddela.4, lan.105,zhong.536, xu.1265}@osu.edu Abstract The success of a text simplification system heavily depends on the quality and quan- tity of complex-simple sentence pairs in the training corpus, which are extracted by aligning sentences between parallel articles. To evaluate and improve sentence alignment quality, we create two manually annotated sentence-aligned datasets from two commonly used text simplification corpora, Newsela and Wikipedia. We propose a novel neural CRF alignment model which not only leverages the sequential nature of sentences in parallel doc- uments but also utilizes a neural sentence pair model to capture semantic similarity. Experi- ments demonstrate that our proposed approach outperforms all the previous work on monolin- gual sentence alignment task by more than 5 points in F1. We apply our CRF aligner to construct two new text simplification datasets, NEWSELA-AUTO and WIKI-AUTO, which are much larger and of better quality compared to the existing datasets. A Transformer-based seq2seq model trained on our datasets estab- lishes a new state-of-the-art for text simplifica- tion in both automatic and human evaluation.1 1 Introduction Text simplification aims to rewrite complex text into simpler language while retaining its original meaning (Saggion, 2017). Text simplification can provide reading assistance for children (Kajiwara et al., 2013), non-native speakers (Petersen and Ostendorf, 2007; Pellow and Eskenazi, 2014), non- expert readers (Elhadad and Sutaria, 2007; Sid- dharthan and Katsos, 2010), and people with lan- guage disorders (Rello et al., 2013). As a prepro- cessing step, text simplification can also improve 1Code and data are available at: https://github. com/chaojiang06/wiki-auto. Newsela data need to be requested at: https://newsela.com/data/. the performance of many natural language process- ing (NLP) tasks, such as parsing (Chandrasekar et al., 1996), semantic role labelling (Vickrey and Koller, 2008), information extraction (Miwa et al., 2010) , summarization (Vanderwende et al., 2007; Xu and Grishman, 2009), and machine translation (Chen et al., 2012; ˇStajner and Popovic, 2016). Automatic text simplification is primarily ad- dressed by sequence-to-sequence (seq2seq) models whose success largely depends on the quality and quantity of the training corpus, which consists of pairs of complex-simple sentences. Two widely used corpora, NEWSELA (Xu et al., 2015) and WIK- ILARGE (Zhang and Lapata, 2017), were created by automatically aligning sentences between compa- rable articles. However, due to the lack of reliable annotated data,2 sentence pairs are often aligned using surface-level similarity metrics, such as Jac- card coefficient (Xu et al., 2015) or cosine distance of TF-IDF vectors (Paetzold et al., 2017), which fails to capture paraphrases and the context of sur- rounding sentences. A common drawback of text simplification models trained on such datasets is that they behave conservatively, performing mostly deletion, and rarely paraphrase (Alva-Manchego et al., 2017). Moreover, WIKILARGE is the …

Content truncated to 500 words