Iterative Search for Weakly Supervised Semantic Parsing

Proceedings of NAACL-HLT 2019, pages 2669–2680 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2669 Iterative Search for Weakly Supervised Semantic Parsing Pradeep Dasigi♥, Matt Gardner♥, Shikhar Murty♦, Luke Zettlemoyer♣, and Eduard Hovy♠ ♥Allen Institute for Artificial Intelligence, Seattle, Washington ♥Allen Institute for Artificial Intelligence, Irvine, California ♦Mila, Universit´e de Montr´eal ♣University of Washington ♠Carnegie Mellon University pradeepd@allenai.org Abstract Training semantic parsers from question- answer pairs typically involves searching over an exponentially large space of logical forms, and an unguided search can easily be misled by spurious logical forms that coincidentally evaluate to the correct answer. We propose a novel iterative training algorithm that alter- nates between searching for consistent logical forms and maximizing the marginal likelihood of the retrieved ones. This training scheme lets us iteratively train models that provide guid- ance to subsequent ones to search for logical forms of increasing complexity, thus dealing with the problem of spuriousness. We eval- uate these techniques on two hard datasets: WIKITABLEQUESTIONS (WTQ) and Cornell Natural Language Visual Reasoning (NLVR), and show that our training algorithm outper- forms the previous best systems, on WTQ in a comparable setting, and on NLVR with signif- icantly less supervision. 1 Introduction Semantic parsing is the task of translating natu- ral language utterances into machine-executable meaning representations, often called programs or logical forms. These logical forms can be exe- cuted against some representation of the context in which the utterance occurs, to produce a de- notation. This setup allows for complex reason- ing over contextual knowledge, and it has been successfully used in several natural language un- derstanding problems such as question answer- ing (Berant et al., 2013), program synthesis (Yin and Neubig, 2017) and building natural language interfaces (Suhr et al., 2018). Recent work has focused on training seman- tic parses via weak supervision from denotations alone (Liang et al., 2011; Berant et al., 2013). This is because obtaining logical form annotations is generally expensive (although recent work has addressed this issue to some extent (Yih et al., 2016)), and not assuming full supervision lets us be agnostic about the logical form language. The second reason is more important in open-domain semantic parsing tasks where it may not be pos- sible to arrive at a complete set of operators re- quired by the task. However, training semantic parsers with weak supervision requires not only searching over an exponentially large space of log- ical forms (Berant et al., 2013; Artzi and Zettle- moyer, 2013; Pasupat and Liang, 2015; Guu et al., 2017, inter alia) but also dealing with spurious logical forms that evaluate to the correct denota- tion while not being semantically equivalent to the utterance. For example, if the denotations are bi- nary, 50% of all syntactically valid logical forms evaluate to the correct answer, regardless of their semantics. This problem renders the training sig- nal extremely noisy, making it hard for the model to learn anything without some additional guid- ance during search. We introduce two innovations to …

Content truncated to 500 words