Multilingual AMR-to-Text Generation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2889–2901, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2889 Multilingual AMR-to-Text Generation Angela Fan FAIR/LORIA Universit´e de Lorraine, Nancy, France angelafan@fb.com Claire Gardent CNRS/LORIA Nancy, France cgardent@loria.fr Abstract Generating text from structured data is chal- lenging because it requires bridging the gap be- tween (i) structure and natural language (NL) and (ii) semantically underspecified input and fully specified NL output. Multilingual gener- ation brings in an additional challenge: that of generating into languages with varied word or- der and morphological properties. In this work, we focus on Abstract Meaning Representa- tions (AMRs) as structured input, where previ- ous research has overwhelmingly focused on generating only into English. We leverage ad- vances in cross-lingual embeddings, pretrain- ing, and multilingual models to create multi- lingual AMR-to-text models that generate in twenty one different languages. For eighteen languages, based on automatic metrics, our multilingual models surpass baselines that gen- erate into a single language. We analyse the ability of our multilingual models to accurately capture morphology and word order using hu- man evaluation, and find that native speakers judge our generations to be fluent. 1 Introduction Generating text from structured data has a vari- ety of applications in natural language processing. Tasks such as decoding from tables (Lebret et al., 2016; Sha et al., 2018), question answering from knowledge bases (Fan et al., 2019a), and generation from RDF Triples (Gardent et al., 2017), knowl- edge graphs (Marcheggiani and Perez-Beltrachini, 2018) and linguistic meaning representations (Kon- stas et al., 2017) face similar challenges: interpret- ing structured input and writing fluent output. We focus on generating from graph structures in the form of Abstract Meaning Representations (AMR) (Banarescu et al., 2013). Previous work has largely focused on generating from AMR into English, but we propose a multilingual approach that can decode into twenty one different languages. Compared to multilingual translation, decod- ing from structured input has distinct challenges. Translation models take natural language input and must faithfully decode into natural language output. However, as shown in Zhao et al. (2020), bridging the gap between structured input and linear output is a difficult task. In addition, in structured input such as graphs, the input is usually semantically under-specified. For example, in AMRs, function words are missing and tense and number are not given. Thus, generation from structured input must bridge the gap between (i) structure and string and (ii) underspecified input and fully specified output. Multilinguality brings a third challenge — that of generating in languages that have varied morpho- logical and word order properties. Annotating natural language with AMR is a complex task and training datasets only exist for English1, so previous work on AMR-to-text gen- eration has overwhelmingly focused on English. We create training data for multilingual AMR-to- Text models, by taking the EUROPARL multilin- gual corpus and automatically annotating the En- glish data with AMRs using the jamr semantic parser. We then use the English AMRs as the in- …

Content truncated to 500 words