Neural Text Generation from Rich Semantic Representations

Proceedings of NAACL-HLT 2019, pages 2259–2266 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2259 Neural Text Generation from Rich Semantic Representations Valerie Hajdik1, Jan Buys2, Michael W. Goodman1 and Emily M. Bender1 1Department of Linguistics, University of Washington 2Paul G. Allen School of Computer Science and Engineering, University of Washington vhajdik@uw.edu, jbuys@cs.washington.edu, {goodmami, ebender}@uw.edu Abstract We propose neural models to generate high-quality text from structured representa- tions based on Minimal Recursion Semantics (MRS). MRS is a rich semantic representation that encodes more precise semantic detail than other representations such as Abstract Mean- ing Representation (AMR). We show that a sequence-to-sequence model that maps a lin- earization of Dependency MRS, a graph-based representation of MRS, to English text can achieve a BLEU score of 66.11 when trained on gold data. The performance can be im- proved further using a high-precision, broad coverage grammar-based parser to generate a large silver training corpus, achieving a final BLEU score of 77.17 on the full test set, and 83.37 on the subset of test data most closely matching the silver data domain. Our results suggest that MRS-based representations are a good choice for applications that need both structured semantics and the ability to produce natural language text as output. 1 Introduction Text generation systems often generate their out- put from an intermediate semantic representation (Yao et al., 2012; Takase et al., 2016). How- ever many semantic representations are task- or domain-specific (He and Young, 2003; Wong and Mooney, 2007), while rule-based text gen- eration systems often have incomplete coverage (Langkilde-Geary, 2002; Oepen et al., 2007). In this work we combine the advantages of Min- imal Recursion Semantics (MRS; Copestake et al., 2005) with the robustness and fluency of neu- ral sequence-to-sequence models trained on large datasets. We hypothesize that MRS is particu- larly well-suited for text generation, as it is explic- itly compositional, capturing the contribution to sentence meaning of all parts of the surface form (Bender et al., 2015). In contrast, semantic representations such as Abstract Meaning Representation (AMR; Ba- narescu et al., 2013) seek to abstract away from the syntax of a sentence as much as possible. Therefore MRS captures meaning distinctions that AMR fails to represent (see Fig. 1). Our approach (§2) uses neural sequence-to- sequence models (Sutskever et al., 2014; Bah- danau et al., 2014) to map linearizations of di- rected acyclic graphs (DAGs) to text, similar to the approach proposed by Konstas et al. (2017) to generate text from AMR. We use Dependency MRS (DMRS; Copestake, 2009), a graph-based representation in which nodes are MRS predicates (annotated with additional attributes) and edges represent relations between predicates. MRS and DMRS are interconvertible and the graph-based representation enables more convenient lineariza- tion and manipulation than MRS’s variable-based representation (Copestake et al., 2016). Results (§3) show that neural DMRS to English text generation can obtain up to 83.37 BLEU and 32% exact match, substantially higher than previ- ous work. In particular, we obtain an 11.6 BLEU improvement …

Content truncated to 500 words