Ensemble Models for Dependency Parsing: Cheap and Good?

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 649–652, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Ensemble Models for Dependency Parsing: Cheap and Good? Mihai Surdeanu and Christopher D. Manning Computer Science Department Stanford University, Stanford, CA 94305 {mihais,manning}@stanford.edu Abstract Previous work on dependency parsing used various kinds of combination models but a systematic analysis and comparison of these approaches is lacking. In this paper we imple- mented such a study for English dependency parsing and find several non-obvious facts: (a) the diversity of base parsers is more important than complex models for learning (e.g., stack- ing, supervised meta-classification), (b) ap- proximate, linear-time re-parsing algorithms guarantee well-formed dependency trees with- out significant performance loss, and (c) the simplest scoring model for re-parsing (un- weighted voting) performs essentially as well as other more complex models. This study proves that fast and accurate ensemble parsers can be built with minimal effort. 1 Introduction Several ensemble models have been proposed for the parsing of syntactic dependencies. These ap- proaches can generally be classified in two cate- gories: models that integrate base parsers at learn- ing time, e.g., using stacking (Nivre and McDon- ald, 2008; Attardi and Dell’Orletta, 2009), and ap- proaches that combine independently-trained mod- els only at parsing time (Sagae and Lavie, 2006; Hall et al., 2007; Attardi and Dell’Orletta, 2009). In the latter case, the correctness of the final dependency tree is ensured by: (a) selecting entire trees proposed by the base parsers (Henderson and Brill, 1999); or (b) re-parsing the pool of dependencies proposed by the base models (Sagae and Lavie, 2006). The lat- ter approach was shown to perform better for con- stituent parsing (Henderson and Brill, 1999). While all these models achieved good perfor- mance, the previous work has left several questions Devel In domain Out of domain LAS LAS UAS LAS UAS MST 85.36 87.07 89.95 80.48 86.08 Malt→ AE 84.24 85.96 88.64 78.74 84.18 Malt→ CN 83.75 85.61 88.14 78.55 83.68 Malt→ AS 83.74 85.36 88.06 77.23 82.39 Malt← AS 82.43 83.90 86.70 76.69 82.57 Malt← CN 81.75 83.53 86.17 77.29 83.02 Malt← AE 80.76 82.51 85.35 76.18 82.02 Table 1: Labeled attachment scores (LAS) and unlabeled at- tachment scores (UAS) for the base models. The parsers are listed in descending order of LAS in the development partition. unanswered. Here we answer the following ques- tions, in the context of English dependency parsing: 1. When combining models at parsing time, what is the best scoring model for candidate depen- dencies during re-parsing? Can a meta classi- fier improve over unsupervised voting? 2. Are (potentially-expensive) re-parsing strate- gies justified for English? What percentage of trees are not well-formed if one switches to a light word-by-word voting scheme? 3. How important is the integration of base parsers at learning time? 4. How do ensemble models compare against state-of-the-art supervised parsers? 2 Setup In our experiments we used the syntactic dependen- cies from the CoNLL 2008 shared task …

Content truncated to 500 words