Empirically-motivated Generalizations of CCG Semantic Parsing Learning Algorithms

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 348–357, Gothenburg, Sweden, April 26-30 2014. c⃝2014 Association for Computational Linguistics Empirically-motivated Generalizations of CCG Semantic Parsing Learning Algorithms Jesse Glass Temple University 1801 N Broad Street Philadelphia, PA 19122, USA jglassemc2@gmail.com Alexander Yates Temple University 1801 N Broad Street Philadelphia, PA 19122, USA ayates@gmail.com Abstract Learning algorithms for semantic parsing have improved drastically over the past decade, as steady improvements on bench- mark datasets have shown. In this pa- per we investigate whether they can gen- eralize to a novel biomedical dataset that differs in important respects from the tra- ditional geography and air travel bench- mark datasets. Empirical results for two state-of-the-art PCCG semantic parsers in- dicates that learning algorithms are sensi- tive to the kinds of semantic and syntac- tic constructions used in a domain. In re- sponse, we develop a novel learning algo- rithm that can produce an effective seman- tic parser for geography, as well as a much better semantic parser for the biomedical dataset. 1 Introduction Semantic parsing is the task of converting nat- ural language utterances into formal representa- tions of their meaning. In this paper, we consider in particular a grounded form of semantic pars- ing, in which the meaning representation language takes its logical constants from a given, fixed on- tology. Several recent systems have demonstrated the ability to learn semantic parsers for domains like the GeoQuery database containing geography relations, or the ATIS database of air travel infor- mation. In these settings, existing systems can produce correct meaning representations with F1 scores approaching 0.9 (Wong and Mooney, 2007; Kwiatkowski et al., 2011). These benchmark datasets have supported a di- verse and influential line of research into semantic parsing learning algorithms for sophisticated se- mantic constructions, with continuing advances in accuracy. However, the focus on these datasets leads to a natural question — do other natural datasets have similar syntax and semantics, and if not, can existing algorithms handle the variability in syntax and semantics? In an effort to investigate and improve the generalization capacity of existing learning algo- rithms for semantic parsing, we develop a novel, natural experimental setting, and we test whether current semantic parsers generalize to the new set- ting. For our datset, we use descriptions of clin- ical trials of experimental drugs in the United States, available from the U.S. National Insti- tutes of Health1. Much of the text in the de- scription of these clinical trials can be mapped neatly onto biomedical ontologies, thus permitting grounded semantic analysis. Crucially, the dataset was not designed specifically with semantic pars- ing or question-answering in mind, and as a re- sult, it provides a natural source for the variety and complexity of utterances that humans use in this domain. As an added benefit, a successful semantic parser in this domain could yield a va- riety of useful bioinformatics applications by per- mitting comparisons between and across clinical trials using structured representations of the …

Content truncated to 500 words