Neural data-to-text generation: A comparison between pipeline and end-to-end architectures

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 552–562, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 552 Neural data-to-text generation: A comparison between pipeline and end-to-end architectures Thiago Castro Ferreira1,2 Chris van der Lee1 Emiel van Miltenburg1 Emiel Krahmer1 1Tilburg center for Cognition and Communication (TiCC), Tilburg University, The Netherlands 2Federal University of Minas Gerais (UFMG), Brazil {tcastrof,c.vdrlee}@tilburguniversity.edu {c.w.j.vanmiltenburg,e.j.krahmer}@tilburguniversity.edu Abstract Traditionally, most data-to-text applications have been designed using a modular pipeline architecture, in which non-linguistic input data is converted into natural language through sev- eral intermediate transformations. By contrast, recent neural models for data-to-text gener- ation have been proposed as end-to-end ap- proaches, where the non-linguistic input is rendered in natural language with much less explicit intermediate representations in be- tween. This study introduces a systematic comparison between neural pipeline and end- to-end data-to-text approaches for the gen- eration of text from RDF triples. Both ar- chitectures were implemented making use of the encoder-decoder Gated-Recurrent Units (GRU) and Transformer, two state-of-the art deep learning methods. Automatic and human evaluations together with a qualitative analy- sis suggest that having explicit intermediate steps in the generation process results in better texts than the ones generated by end-to-end ap- proaches. Moreover, the pipeline models gen- eralize better to unseen inputs. Data and code are publicly available.1 1 Introduction Data-to-text Natural Language Generation (NLG) is the computational process of generating mean- ingful and coherent natural language text to de- scribe non-linguistic input data (Gatt and Krah- mer, 2018). Practical applications can be found in domains such as weather forecasts (Mei et al., 2016), health care (Portet et al., 2009), feedback for car drivers (Braun et al., 2018), diet manage- ment (Anselma and Mazzei, 2018), election re- sults (Lepp¨anen et al., 2017) and sportscasting news (van der Lee et al., 2017). 1https://github.com/ThiagoCF05/ DeepNLG/ Traditionally, most of data-to-text applications have been designed in a modular fashion, in which the non-linguistic input data (be it, say, numerical weather information or game statistics) are con- verted into natural language (e.g., weather fore- cast, game report) through several explicit inter- mediate transformations. The most prominent ex- ample is the ‘traditional’ pipeline architecture (Re- iter and Dale, 2000) that performs tasks related to document planning, sentence planning and lin- guistic realization in sequence. Many of the tradi- tional, rule-based NLG systems relied on modules because (a) these modules could be more easily reused across applications, and (b) because going directly from input to output using rules was sim- ply too complex in general (see Gatt and Krahmer 2018 for a discussion of different architectures). The emergence of neural methods changed this: provided there is enough training data, it does be- come possible to learn a direct mapping from in- put to output, as has also been shown in, for ex- ample, neural machine translation. As a result, in NLG more recently, neural end-to-end data-to- text models have been proposed, which directly …

Content truncated to 500 words