Unsupervised Dependency Parsing: Let’s Use Supervised Parsers

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 651–661, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Unsupervised Dependency Parsing: Let’s Use Supervised Parsers Phong Le and Willem Zuidema Institute for Logic, Language, and Computation University of Amsterdam, the Netherlands {p.le,zuidema}@uva.nl Abstract We present a self-training approach to unsu- pervised dependency parsing that reuses exist- ing supervised and unsupervised parsing algo- rithms. Our approach, called ‘iterated rerank- ing’ (IR), starts with dependency trees gener- ated by an unsupervised parser, and iteratively improves these trees using the richer proba- bility models used in supervised parsing that are in turn trained on these trees. Our system achieves 1.8% accuracy higher than the state- of-the-part parser of Spitkovsky et al. (2013) on the WSJ corpus. 1 Introduction Unsupervised dependency parsing and its super- vised counterpart have many characteristics in com- mon: they take as input raw sentences, produce dependency structures as output, and often use the same evaluation metric (DDA, or UAS, the percent- age of tokens for which the system predicts the cor- rect head). Unsurprisingly, there has been much more research on supervised parsing – producing a wealth of models, datasets and training techniques – than on unsupervised parsing, which is more dif- ficult, much less accurate and generally uses very simple probability models. Surprisingly, however, there have been no reported attempts to reuse super- vised approaches to tackle the unsupervised parsing problem (an idea briefly mentioned in Spitkovsky et al. (2010b)). There are, nevertheless, two aspects of supervised parsers that we would like to exploit in an unsuper- vised setting. First, we can increase the model ex- pressiveness in order to capture more linguistic reg- ularities. Many recent supervised parsers use third- order (or higher order) features (Koo and Collins, 2010; Martins et al., 2013; Le and Zuidema, 2014) to reach state-of-the-art (SOTA) performance. In contrast, existing models for unsupervised parsing limit themselves to using simple features (e.g., con- ditioning on heads and valency variables) in order to reduce the computational cost, to identify consis- tent patterns in data (Naseem, 2014, page 23), and to avoid overfitting (Blunsom and Cohn, 2010). Al- though this makes learning easier and more efficient, the disadvantage is that many useful linguistic regu- larities are missed: an upper bound on the perfor- mance of such simple models – estimated by us- ing annotated data – is 76.3% on the WSJ corpus (Spitkovsky et al., 2013), compared to over 93% ac- tual performance of the SOTA supervised parsers. Second, we would like to make use of informa- tion available from lexical semantics, as in Bansal et al. (2014), Le and Zuidema (2014), and Chen and Manning (2014). Lexical semantics is a source for handling rare words and syntactic ambiguities. For instance, if a parser can identify that “he” is a depen- dent of “walks” in the sentence “He walks”, then, even if “she” and “runs” do not appear in the train- ing data, …

Content truncated to 500 words