Maximum Margin Reward Networks for Learning from Explicit and Implicit Supervision

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2368–2378 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Maximum Margin Reward Networks for Learning from Explicit and Implicit Supervision Haoruo Peng1 Ming-Wei Chang2 Wen-tau Yih2 1University of Illinois, Urbana-Champaign 2Microsoft Research, Redmond 1hpeng7@illinois.edu 2{minchang,scottyih}@microsoft.com Abstract Neural networks have achieved state-of- the-art performance on several structured- output prediction tasks, trained in a fully supervised fashion. However, annotated examples in structured domains are of- ten costly to obtain, which thus limits the applications of neural networks. In this work, we propose Maximum Mar- gin Reward Networks, a neural network- based framework that aims to learn from both explicit (full structures) and implicit supervision signals (delayed feedback on the correctness of the predicted structure). On named entity recognition and seman- tic parsing, our model outperforms previ- ous systems on the benchmark datasets, CoNLL-2003 and WebQuestionsSP. 1 Introduction Structured-output prediction problems, where the goal is to determine values of a set of inter- dependent variables, are ubiquitous in NLP. Struc- tures of such problems can range from simple se- quences like part-of-speech tagging (Ling et al., 2015) and named entity recognition (Lample et al., 2016), to complex syntactic or semantic analysis such as dependency parsing (Dyer et al., 2015) and semantic parsing (Dong and Lapata, 2016). State- of-the-art methods of these tasks are often neu- ral network models trained using fully annotated structures, which can be costly or time-consuming to obtain. Weakly supervised learning settings, where the algorithm assumes only the existence of implicit signals on whether a prediction is correct, are thus more appealing in many scenarios. For example, Figure 1 shows a weakly super- vised setting of learning semantic parsers using only question–answer pairs. When the system generates a candidate semantic parse during train- ing, the quality needs to be indirectly measured by Q: Who played Meg in Season 1 of Family Guy? 𝜆𝑥. ∃𝑥���. 𝑐𝑎𝑟���𝑟��� FamilyGuySeason1, 𝑥��� ∧ 𝑎𝑐𝑟���𝑜𝑟 𝑥���, 𝑥 𝜆𝑥. ∃𝑥���. 𝑐𝑎𝑟���𝑟��� FamilyGuySeason1, 𝑥��� ∧ 𝑎𝑐𝑟���𝑜𝑟 𝑥���, 𝑥 KB KB Lacey Chabert, Seth MacFarlane, Alex Borstein, Seth Green, John Viener, Alec Sulkin Lacey Chabert, Seth MacFarlane, Alex Borstein, Seth Green, John Viener, Alec Sulkin A: Lacey Chabert Figure 1: Learning a semantic parser using im- plicit supervision signals (labeled answers). Since there are no gold parses, a model needs to explore different parses, where their quality can only be indirectly verified by comparing retrieved answers and the labeled answers. comparing the derived answers from the knowl- edge base and the provided labeled answers. This setting of implicit supervision increases the difficulty of learning a neural model, not only because the signals are vague and noisy, but also delayed. For instance, among different semantic parses that result in the same answers, typically only few of them correctly represent the meaning of the question. Moreover, the correctness of an- swers corresponding to a parse can only be eval- uated through an external oracle (e.g., executing the query on the knowledge base) after the parse is …

Content truncated to 500 words