Weakly Supervised Supertagging with Grammar-Informed Initialization

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 57–64 Manchester, August 2008 Weakly supervised supertagging with grammar-informed initialization Jason Baldridge Department of Linguistics The University of Texas at Austin jbaldrid@mail.utexas.edu Abstract Much previous work has investigated weak supervision with HMMs and tag dictionar- ies for part-of-speech tagging, but there have been no similar investigations for the harder problem of supertagging. Here, I show that weak supervision for supertag- ging does work, but that it is subject to severe performance degradation when the tag dictionary is highly ambiguous. I show that lexical category complexity and infor- mation about how supertags may combine syntactically can be used to initialize the transition distributions of a first-order Hid- den Markov Model for weakly supervised learning. This initialization proves more effective than starting with uniform tran- sitions, especially when the tag dictionary is highly ambiguous. 1 Introduction Supertagging involves assigning words lexical en- tries based on a lexicalized grammatical theory, such as Combinatory Categorial Grammar (CCG) (Steedman, 2000) Tree-adjoining Grammar (Joshi, 1988), or Head-driven Phrase Structure Grammar (Pollard and Sag, 1994). Supertag sets are larger than part-of-speech (POS) tag sets and their ele- ments are generally far more articulated. For ex- ample, the English verb join has the POS VB and the CCG category ((Sb\NP)/PP)/NP in CCG- bank (Hockenmaier and Steedman, 2007). This category indicates that join requires a noun phrase c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. to its left, another to its right, and a prepositional phrase to the right of that. Supertags convey such detailed syntactic sub- categorization information that supertag disam- biguation is referred to as almost parsing (Banga- lore and Joshi, 1999). Standard sequence predic- tion models are highly effective for supertagging, including Hidden Markov Models (Bangalore and Joshi, 1999; Nielsen, 2002), Maximum Entropy Markov Models (Clark, 2002; Hockenmaier et al., 2004; Clark and Curran, 2007), and Conditional Random Fields (Blunsom and Baldwin, 2006). The original motivation for supertags–parse pre- filtering for lexicalized grammars–of Bangalore and Joshi (1999) has been realized to good effect: the supertagger of Clark and Curran (2007) pro- vides staged n-best lists of multi-tags that dramat- ically improve parsing speed and coverage without much loss in accuracy. Espinosa et al. (2008) have shown that hypertagging (predicting the supertag associated with a logical form) can improve both speed and accuracy of wide-coverage sentence re- alization with CCG. Supertags have gained fur- ther relevance as they are increasingly used as fea- tures for other tasks, including machine translation (Birch et al., 2007; Hassan et al., 2007). Supertaggers typically rely on a significant amount of carefully annotated sentences. As with many problems, there is pressing need to find strategies for reducing the amount of supervision required for producing accurate supertaggers, but as yet, no one has explored the use of weak super- vision for the task. In particular, there are many di- alog systems which rely on hand-crafted lexicons that both provide a starting point …

Content truncated to 500 words