Improved Combinatory Categorial Grammar Induction with Boundary Words and Bayesian Inference

Proceedings of COLING 2012: Technical Papers, pages 1257–1274, COLING 2012, Mumbai, December 2012. Improved Combinatory Categorial Grammar Induction with Boundary Words and Bayesian Inference Yun HUANG1,2 Min ZHANG2 Chew Lim TAN1 1Department of Computer Science 2Human Language Department National University of Singapore Institute for Infocomm Research 13 Computing Drive, Singapore 1 Fusionopolis Way, Singapore {huangyun,tancl}@comp.nus.edu.sg mzhang@i2r.a-star.edu.sg ABSTRACT Combinatory Categorial Grammar (CCG) is an expressive grammar formalism which is able to capture long-range dependencies. However, building large and wide-coverage treebanks for CCG is expensive and time-consuming. In this paper, we focus on the problem of unsupervised CCG induction from plain texts. Based on the baseline model in (Bisk and Hockenmaier, 2012), we propose following two improvements: (1) we utilize boundary part-of-speech (POS) tags to capture lexical information; (2) we perform nonparametric Bayesian inference based on the Pitman-Yor process to learn compact grammars. Experiments on English Penn treebank demonstrate the effectiveness of our boundary model and Bayesian learning. TITLE AND ABSTRACT IN ANOTHER LANGUAGE (CHINESE) 基 基 基于 于 于边 边 边界 界 界词 词 词和 和 和贝 贝 贝叶 叶 叶斯 斯 斯模 模 模型 型 型改 改 改进 进 进的 的 的组 组 组合 合 合范 范 范畴 畴 畴语 语 语法 法 法推 推 推导 导 导 组合范畴语法(CCG)是一种具有丰富表达能力的语法形式，它可以捕获长距离的依 赖关系。但是，构建大规模、覆盖面广的组合范畴语法语料库既昂贵又耗时。在本 文中，我们研究如何从普通文本中无监督地推导出组合范畴语法。基于现有的工 作(Bisk and Hockenmaier, 2012)，我们提出以下两个改进：(1) 我们使用边界词的词性标 记把词汇化信息引入模型中；(2) 我们使用基于Pitman-Yor过程的非参数贝叶斯模型来学习 简洁的文法。在英语宾州树库上的实验结果显示了我们提出的边界词模型和贝叶斯模型的 有效性。 KEYWORDS: Grammar Induction, Combinatory Categorial Grammar, Boundary Words, Bayesian Model. KEYWORDS IN CHINESE: 语法推导, 组合范畴语法, 边界词, 贝叶斯模型. 1257 1 Introduction Unsupervised grammar induction has attracted research interests for a long time. The in- duced grammars can be used to construct large treebanks (van Zaanen, 2000), study lan- guage acquisition (Jones et al., 2010), etc. In recent years, numerous approaches have been introduced to automatically induce hierarchical structures from plain strings. Some approaches focus on the constituency grammar induction: the constituent-context model (Klein and Manning, 2002), the data-oriented parsing (Bod, 2006), the common cover link model (Seginer, 2007), and the tree-substitution grammars (TSG) (Cohn et al., 2009). The other mainstream is the dependency grammar induction: the dependency model with va- lence (DMV) (Klein and Manning, 2004; Headden III et al., 2009; Cohen and Smith, 2009), TSG model for dependency (Blunsom and Cohn, 2010), etc. Among these grammar formalisms, the Combinatorial Categorial Grammar (CCG) is a lexical- ized, mildly-context-sensitive model (Steedman, 2000). In the formal grammar theory, CCGs are known to be weekly equivalent to Linear Indexed Grammars, Tree-adjoining Grammars, and Head Grammars (Vijay-Shanker and Weir, 1994). The CCG formalism provides a trans- parent interface between syntax and semantics, such that the underlying semantics could be naturally defined over syntactical derivations, including the long-range dependencies, the coor- dination structure, and the extraction phenomenon (Bos et al., 2004; Zettlemoyer and Collins, 2007). As a mildly context-sensitive grammar, CCG can be efficiently parsed in polyno- mial time, which makes them practical in real parsing tasks1. The wide-coverage com- binatorial categorial grammars have been used in many NLP tasks, such as the lexical acquisition (Blunsom and Baldwin, 2006), the parsing tasks (Hockenmaier and Steedman, 2002; …

Content truncated to 500 words