Generating Logical Forms from Graph Representations of Text and Entities

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 95–106 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 95 Generating Logical Forms from Graph Representations of Text and Entities Peter Shaw1, Philip Massey1, Angelica Chen1, Francesco Piccinno2, Yasemin Altun2 1Google 2Google Research {petershaw,pmassey,angelicachen,piccinno,altun}@google.com Abstract Structured information about entities is critical for many semantic parsing tasks. We present an approach that uses a Graph Neural Net- work (GNN) architecture to incorporate infor- mation about relevant entities and their rela- tions during parsing. Combined with a de- coder copy mechanism, this approach provides a conceptually simple mechanism to generate logical forms with entities. We demonstrate that this approach is competitive with the state- of-the-art across several tasks without pre- training, and outperforms existing approaches when combined with BERT pre-training. 1 Introduction Semantic parsing maps natural language utter- ances into structured meaning representations. The representation languages vary between tasks, but typically provide a precise, machine inter- pretable logical form suitable for applications such as question answering (Zelle and Mooney, 1996; Zettlemoyer and Collins, 2007; Liang et al., 2013; Berant et al., 2013). The logical forms typically consist of two types of symbols: a vocabulary of operators and domain-specific predicates or func- tions, and entities grounded to some knowledge base or domain. Recent approaches to semantic parsing have cast it as a sequence-to-sequence task (Dong and Lapata, 2016; Jia and Liang, 2016; Ling et al., 2016), employing methods similar to those de- veloped for neural machine translation (Bahdanau et al., 2014), with strong results. However, spe- cial consideration is typically given to handling of entities. This is important to improve generaliza- tion and computational efficiency, as most tasks require handling entities unseen during training, and the set of unique entities can be large. Some recent approaches have replaced surface forms of entities in the utterance with placehold- ers (Dong and Lapata, 2016). This requires a pre- processing step to completely disambiguate enti- ties and replace their spans in the utterance. Ad- ditionally, for some tasks it may be beneficial to leverage relations between entities, multiple entity candidates per span, or entity candidates without a corresponding span in the utterance, while gener- ating logical forms. Other approaches identify only types and sur- face forms of entities while constructing the log- ical form (Jia and Liang, 2016), using a separate post-processing step to generate the final logical form with grounded entities. This ignores poten- tially useful knowledge about relevant entities. Meanwhile, there has been considerable re- cent interest in Graph Neural Networks (GNNs) (Scarselli et al., 2009; Li et al., 2016; Kipf and Welling, 2017; Gilmer et al., 2017; Veliˇckovi´c et al., 2018) for effectively learning representa- tions for graph structures. We propose a GNN architecture based on extending the self-attention mechanism of the Transformer (Vaswani et al., 2017) to make use of relations between input el- ements. We present an application of this GNN ar- chitecture to semantic parsing, conditioning on a graph representation of the …

Content truncated to 500 words