Grammar-Constrained Neural Semantic Parsing with LR Parsers

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1275–1279 August 1–6, 2021. ©2021 Association for Computational Linguistics 1275 Grammar-Constrained Neural Semantic Parsing with LR Parsers Artur Baranowski Technische Hochschule Köln Institut für Nachrichtentechnik German Aerospace Center (DLR) Institute for Software Technology artur.baranowski@mailbox.org Nico Hochgeschwender Hochschule Bonn-Rhein-Sieg Department of Computer Science German Aerospace Center (DLR) Institute for Software Technology nico.hochgeschwender@h-brs.de Abstract Target meaning representations for semantic parsing tasks are often based on programming or query languages, such as SQL, and can be formalized by a context-free grammar. As- suming a priori knowledge of the target do- main, such grammars can be exploited to en- force syntactical constraints when predicting logical forms. To that end, we assess how syntactical parsers can be integrated into mod- ern encoder-decoder frameworks. Specifically, we implement an attentional SEQ2SEQ model that uses an LR parser to maintain syntacti- cally valid sequences throughout the decod- ing procedure. Compared to other approaches to grammar-guided decoding that modify the underlying neural network architecture or at- tempt to derive full parse trees, our approach is conceptually simpler, adds less computational overhead during inference and integrates seam- lessly with current SEQ2SEQ frameworks. We present preliminary evaluation results against a recurrent SEQ2SEQ baseline on GEOQUERY and ATIS and demonstrate improved per- formance while enforcing grammatical con- straints. 1 Introduction Semantic parsing aims at delivering granular, struc- tured representations of natural language utter- ances, referred to as meaning representations or logical forms. Thus, it goes beyond shallow seman- tic analysis involving argument identification and role labeling (Collobert et al., 2011; Roth and Lap- ata, 2016). Meaning representations based on pro- gramming or query languages (PYTHON, SQL) are describable by (deterministic) context-free gram- mars and used for general purpose code genera- tion (Xiao et al., 2016; Yin and Neubig, 2017). Our work targets this particular subset of logical forms. A context-free grammar may be exploited to constrain a semantic parser to only produce token sequences derivable from the grammar. Specifi- cally, we investigate how syntax constraints can be enforced in semantic parsers based on mod- ern encoder-decoder frameworks in a non-intrusive, computationally inexpensive way at inference time. We show that enforcing grammatical constraints with LR parsers is particularly well suited for mod- ern autoregressive neural network architectures used in neural machine translation (Sutskever et al., 2014; Vaswani et al., 2017). We do not require any modifications to standard SEQ2SEQ neural net- work architectures and make very little assump- tions about the inputs and outputs of such models. In contrast, most grammar-constrained decoders attempt to model the grammar explicitly within the neural network, complicating the architecture. Moreover, they predict complete syntax trees or derivation sequences. Our approach predicts source code token streams, preserving syntactic validity throughout the decoding procedure. Enforcing syn- tactical constraints relieves neural networks mod- els from having to learn the syntactic structure of the target language, which is particularly benefi- cial for ensuring balanced expressions over long ranges (Bahdanau et al., 2014; Ling et al., 2016). Also, when integrating our models …

Content truncated to 500 words