Jointly Learning Semantic Parser and Natural Language Generator via Dual Information Maximization

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2090–2101 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2090 Jointly Learning Semantic Parser and Natural Language Generator via Dual Information Maximization Hai Ye♭ Wenjie Li♮ Lu Wang♯ ♭ Information Sciences Institute, University of Southern California ♮ Department of Computing, The Hong Kong Polytechnic University ♯ College of Computer and Information Science, Northeastern University hye@usc.edu, cswjli@comp.polyu.edu.hk, luwang@ccs.neu.edu Abstract Semantic parsing aims to transform natural language (NL) utterances into formal meaning representations (MRs), whereas an NL gener- ator achieves the reverse: producing a NL de- scription for some given MRs. Despite this intrinsic connection, the two tasks are often studied separately in prior work. In this paper, we model the duality of these two tasks via a joint learning framework, and demonstrate its effectiveness of boosting the performance on both tasks. Concretely, we propose the method of dual information maximization (DIM) to regularize the learning process, where DIM empirically maximizes the variational lower bounds of expected joint distributions of NL and MRs. We further extend DIM to a semi- supervision setup (SEMIDIM), which lever- ages unlabeled data of both tasks. Experi- ments on three datasets of dialogue manage- ment and code generation (and summariza- tion) show that performance on both seman- tic parsing and NL generation can be consis- tently improved by DIM, in both supervised and semi-supervised setups1. 1 Introduction Semantic parsing studies the task of translat- ing natural language (NL) utterances into for- mal meaning representations (MRs) (Zelle and Mooney, 1996; Tang and Mooney, 2000). NL generation models can be designed to learn the reverse: mapping MRs to their NL descrip- tions (Wong and Mooney, 2007). Generally speaking, MR often takes a logical form that captures the semantic meaning, including λ- calculus (Zettlemoyer and Collins, 2005, 2007), Abstract Meaning Representation (AMR) (Ba- narescu et al., 2013; Misra and Artzi, 2016), and general-purpose computer programs, such as 1Code for this paper is available at: https:// github.com/oceanypt/DIM x ỹ pθ → (x,y) = p(x) (y|x) pe pθ y x̃ qϕ −→ (x,y) = q(y) (x|y) pd qϕ x y ⟷ (x,y) Figure 1: Illustration of our joint learning model. x: NL; y: MRs. pθ(y|x): semantic parser; qφ(x|y): NL generator. We model the duality of the two tasks by matching the joint distributions of pe(x, y) (learned from semantic parser) and pd(x, y) (learned from NL generator) to an underlying unknown distri- bution P(x, y). Python (Yin and Neubig, 2017) or SQL (Zhong et al., 2017). Recently, NL generation models have been proposed to automatically construct human- readable descriptions from MRs, for code summa- rization (Hu et al., 2018; Allamanis et al., 2016; Iyer et al., 2016) that predicts the function of code snippets, and for AMR-to-text generation (Song et al., 2018; Konstas et al., 2017; Flanigan et al., 2016). Specifically, a common objective that semantic parsers aim to estimate is pθ(y|x), the conditional distribution between NL input x and the corre- sponding MR …

Content truncated to 500 words