Bootstrapping Generators from Noisy Data

Proceedings of NAACL-HLT 2018, pages 1516–1527 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Bootstrapping Generators from Noisy Data Laura Perez-Beltrachini and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB {lperez,mlap}@inf.ed.ac.uk Abstract A core step in statistical data-to-text gen- eration concerns learning correspondences between structured data representations (e.g., facts in a database) and associated texts. In this paper we aim to bootstrap generators from large scale datasets where the data (e.g., DBPedia facts) and related texts (e.g., Wikipedia abstracts) are loosely aligned. We tackle this challenging task by in- troducing a special-purpose content selection mechanism.1 We use multi-instance learning to automatically discover correspondences between data and text pairs and show how these can be used to enhance the content signal while training an encoder-decoder architec- ture. Experimental results demonstrate that models trained with content-specific objec- tives improve upon a vanilla encoder-decoder which solely relies on soft attention. 1 Introduction A core step in statistical data-to-text genera- tion concerns learning correspondences between structured data representations (e.g., facts in a database) and paired texts (Barzilay and Lapata, 2005; Kim and Mooney, 2010; Liang et al., 2009). These correspondences describe how data repre- sentations are expressed in natural language (con- tent realisation) but also indicate which subset of the data is verbalised in the text (content selec- tion). Although content selection is traditionally per- formed by domain experts, recent advances in generation using neural networks (Bahdanau et al., 2015; Ranzato et al., 2016) have led to the use of large scale datasets containing loosely related data and text pairs. A prime example are on- line data sources like DBPedia (Auer et al., 2007) and Wikipedia and their associated texts which 1Our code and data are available at https://github.com/EdinburghNLP/wikigen. are often independently edited. Another example are sports databases and related textual resources. Wiseman et al. (2017) recently define a generation task relating statistics of basketball games with commentaries and a blog written by fans. In this paper, we focus on short text generation from such loosely aligned data-text resources. We work with the biographical subset of the DBPe- dia and Wikipedia resources where the data cor- responds to DBPedia facts and texts are Wikipedia abstracts about people. Figure 1 shows an example for the film-maker Robert Flaherty, the Wikipedia infobox, and the corresponding abstract. We wish to bootstrap a data-to-text generator that learns to verbalise properties about an entity from a loosely related example text. Given the set of properties in Figure (1a) and the related text in Figure (1b), we want to learn verbalisations for those proper- ties that are mentioned in the text and produce a short description like the one in Figure (1c). In common with previous work (Mei et al., 2016; Lebret et al., 2016; Wiseman et al., 2017) our model draws on insights from neural machine translation (Bahdanau et al., 2015; Sutskever et al., 2014) using an encoder-decoder architecture as its backbone. …

Content truncated to 500 words