Unsupervised Grammar Induction with Depth-bounded PCFG

Unsupervised Grammar Induction with Depth-bounded PCFG Lifeng Jin Department of Linguistics The Ohio State University jin.544@osu.edu Finale Doshi-Velez Harvard University finale@seas.harvard.edu Timothy Miller Boston Children’s Hospital & Harvard Medical School timothy.miller@childrens.harvard.edu William Schuler Department of Linguistics The Ohio State University schuler@ling.osu.edu Lane Schwartz Department of Linguistics University of Illinois at Urbana-Champaign lanes@illinois.edu Abstract There has been recent interest in applying cognitively- or empirically-motivated bounds on recursion depth to limit the search space of grammar induction models (Ponvert et al., 2011; Noji and Johnson, 2016; Shain et al., 2016). This work extends this depth- bounding approach to probabilistic context- free grammar induction (DB-PCFG), which has a smaller parameter space than hierar- chical sequence models, and therefore more fully exploits the space reductions of depth- bounding. Results for this model on grammar acquisition from transcribed child-directed speech and newswire text exceed or are com- petitive with those of other models when eval- uated on parse accuracy. Moreover, gram- mars acquired from this model demonstrate a consistent use of category labels, something which has not been demonstrated by other ac- quisition models. 1 Introduction Grammar acquisition or grammar induction (Car- roll and Charniak, 1992) has been of interest to lin- guists and cognitive scientists for decades. This task is interesting because a well-performing acquisition model can serve as a good baseline for examin- ing factors of grounding (Zettlemoyer and Collins, 2005; Kwiatkowski et al., 2010), or as a piece of evidence (Clark, 2001; Zuidema, 2003) about the Distributional Hypothesis (Harris, 1954) against the poverty of the stimulus (Chomsky, 1965). Unfor- tunately, previous attempts at inducing unbounded context-free grammars (Johnson et al., 2007; Liang et al., 2009) converged to weak modes of a very multimodal distribution of grammars. There has been recent interest in applying cognitively- or empirically-motivated bounds on recursion depth to limit the search space of grammar induction mod- els (Ponvert et al., 2011; Noji and Johnson, 2016; Shain et al., 2016). Ponvert et al. (2011) and Shain et al. (2016) in particular report benefits for depth bounds on grammar acquisition using hierarchical sequence models, but either without the capacity to learn full grammar rules (e.g. that a noun phrase may consist of a noun phrase followed by a preposi- tional phrase), or with a very large parameter space that may offset the gains of depth-bounding. This work extends the depth-bounding approach to di- rectly induce probabilistic context-free grammars,1 which have a smaller parameter space than hier- archical sequence models, and therefore arguably make better use of the space reductions of depth- bounding. This approach employs a procedure for deriving a sequence model from a PCFG (van Schi- jndel et al., 2013), developed in the context of a su- pervised learning model, and adapts it to an unsu- pervised setting. Results for this model on grammar acquisi- tion from transcribed child-directed speech and newswire text exceed or are competitive with those of other models when evaluated on parse accu- racy. Moreover, grammars acquired from this model demonstrate a consistent use of category …

Content truncated to 500 words