A Supertag-Context Model for Weakly-Supervised CCG Parser Learning

Proceedings of the 19th Conference on Computational Language Learning, pages 22–31, Beijing, China, July 30-31, 2015. c⃝2015 Association for Computational Linguistics A Supertag-Context Model for Weakly-Supervised CCG Parser Learning Dan Garrette∗ Chris Dyer† Jason Baldridge‡ Noah A. Smith† ∗Computer Science & Engineering, University of Washington, dhg@cs.washington.edu †School of Computer Science, Carnegie Mellon University, {cdyer,nasmith}@cs.cmu.edu ‡Department of Linguistics, University of Texas at Austin, jbaldrid@utexas.edu Abstract Combinatory Categorial Grammar (CCG) is a lexicalized grammar formalism in which words are associated with cate- gories that specify the syntactic configura- tions in which they may occur. We present a novel parsing model with the capacity to capture the associative adjacent-category relationships intrinsic to CCG by param- eterizing the relationships between each constituent label and the preterminal cat- egories directly to its left and right, bi- asing the model toward constituent cate- gories that can combine with their con- texts. This builds on the intuitions of Klein and Manning’s (2002) “constituent- context” model, which demonstrated the value of modeling context, but has the ad- vantage of being able to exploit the prop- erties of CCG. Our experiments show that our model outperforms a baseline in which this context information is not captured. 1 Introduction Learning parsers from incomplete or indirect su- pervision is an important component of moving NLP research toward new domains and languages. But with less information, it becomes necessary to devise ways of making better use of the informa- tion that is available. In general, this means con- structing inductive biases that take advantage of unannotated data to train probabilistic models. One important example is the constituent- context model (CCM) of Klein and Manning (2002), which was specifically designed to cap- ture the linguistic observation made by Radford (1988) that there are regularities to the contexts in which constituents appear. This phenomenon, known as substitutability, says that phrases of the same type appear in similar contexts. For example, the part-of-speech (POS) sequence ADJ NOUN fre- quently occurs between the tags DET and VERB. This DET—VERB context also frequently applies to the single-word sequence NOUN and to ADJ ADJ NOUN. From this, we might deduce that DET— VERB is a likely context for a noun phrase. CCM is able to learn which POS contexts are likely, and does so via a probabilistic generative model, providing a statistical, data-driven take on substi- tutability. However, since there is nothing intrin- sic about the POS pair DET—VERB that indicates a priori that it is a likely constituent context, this fact must be inferred entirely from the data. Baldridge (2008) observed that unlike opaque, atomic POS labels, the rich structures of Combina- tory Categorial Grammar (CCG) (Steedman, 2000; Steedman and Baldridge, 2011) categories reflect universal grammatical properties. CCG is a lexi- calized grammar formalism in which every con- stituent in a sentence is associated with a struc- tured category that specifies its syntactic relation- ship to other constituents. For example, a cate- gory might encode that “this constituent can com- bine with a noun phrase to the …

Content truncated to 500 words