Split-and-Rephrase in a Cross-Lingual Manner: A Complete Pipeline

Proceedings of Recent Advances in Natural Language Processing, pages 155–164 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_019 155 Split-and-Rephrase in a Cross-Lingual Manner: a Complete Pipeline Paulo Berlanga Neto and Evandro Eduardo Seron Ruiz Department of Computer Science and Mathematics Faculty of Philosophy, Sciences and Letters at Ribeir˜ao Preto University of S˜ao Paulo (USP) {pauloberlanga,evandro}@usp.br Abstract Split-and-rephrase is a challenging task that promotes the transformation of a given com- plex input sentence into multiple shorter sen- tences retaining equivalent meaning. This rewriting approach conceptualizes that shorter sentences benefit human readers and improve NLP downstream tasks attending as a prepro- cessing step. This work presents a complete pipeline capable of performing the split-and- rephrase method in a cross-lingual manner. We trained sequence-to-sequence neural mod- els as from English corpora and applied them to predict the transformations in English and Brazilian Portuguese sentences jointly with BERT’s masked language modeling. Con- trary to traditional approaches that seek train- ing models with extensive vocabularies, we present a non-trivial way to construct sym- bolic ones generalized solely by grammatical classes (POS tags) and their respective recur- rences, reducing the amount of necessary train- ing data. This pipeline contribution showed competitive results encouraging the expansion of the method to languages other than English. 1 Introduction Text Simplification (TS) is the process of mod- ifying natural language to reduce complexity and improve both readability and understandabil- ity (Shardlow, 2014). A simplified vocabulary or a simplified text structure can benefit people with limited language skills, such as those with low education levels, children, non-native speakers, and individuals with learning impairments (e.g., autism, dyslexia, or aphasia) (ˇStajner et al., 2015; Guo et al., 2018). Furthermore, when applied as a preprocessing step, TS may also improve the per- formance of several natural language processing (NLP) tasks, such as parsing, machine translation, semantic role labeling, text summarization, infor- mation extraction, among others (Niklaus et al., This bottle was used until 2002 when it was dropped in favor of a traditional bottle . This bottle was used until 2002 . It was dropped in favor of a traditional bottle . He sought medical care in Rome , but it was unsuccessful , and he died at the age of 42 . He sought medical care in Rome . It was unsuccessful . He died at the age of 42 . Figure 1: Basic split-and-rephrase examples highlight- ing the transformations promoted by the split action. 2019a; ˇStajner and Popovi´c, 2019). Most work on TS has concentrated on analyz- ing specific characteristics at the sentence level, fashioning the task of sentence simplification (SS) (Alva-Manchego et al., 2020). SS applica- tions aim to identify and solve two main aspects: lexical complexity, which refers to difficult words or expressions in the text (e.g., non-frequent words, specific terminologies, foreign words, etc.) (ˇStajner et al., 2020; Narayan and Gardent, 2014); and syn- tactic complexity, which refers to the length of the sentences and their grammatical complexities (e.g., number of subordinate or coordinate clauses, un- usual sentence structures, depth of the syntactic …

Content truncated to 500 words