Combining Generative and Discriminative Approaches to Unsupervised Dependency Parsing via Dual Decomposition

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1689–1694 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Combining Generative and Discriminative Approaches to Unsupervised Dependency Parsing via Dual Decomposition∗ Yong Jiang, Wenjuan Han and Kewei Tu {jiangyong,hanwj,tukw}@shanghaitech.edu.cn School of Information Science and Technology ShanghaiTech University, Shanghai, China Abstract Unsupervised dependency parsing aims to learn a dependency parser from unanno- tated sentences. Existing work focus- es on either learning generative model- s using the expectation-maximization al- gorithm and its variants, or learning dis- criminative models using the discrimina- tive clustering algorithm. In this paper, we propose a new learning strategy that learn- s a generative model and a discriminative model jointly based on the dual decom- position method. Our method is simple and general, yet effective to capture the ad- vantages of both models and improve their learning results. We tested our method on the UD treebank and achieved a state-of- the-art performance on thirty languages. 1 Introduction Dependency parsing is an important task in nat- ural language processing. It identifies dependen- cies between words in a sentence, which have been shown to benefit other tasks such as semantic role labeling (Lei et al., 2015) and sentence classifica- tion (Ma et al., 2015). Supervised learning of a dependency parser requires annotation of a train- ing corpus by linguistic experts, which can be time and resource consuming. Unsupervised dependen- cy parsing eliminates the need for dependency an- notation by directly learning from unparsed text. Previous work on unsupervised dependency parsing mainly focuses on learning generative models, such as the dependency model with va- lence (DMV) (Klein and Manning, 2004) and combinatory categorial grammars (CCG) (Bisk and Hockenmaier, 2012). Generative models have ∗This work was supported by the National Natural Sci- ence Foundation of China (61503248). many advantages. For example, the learning ob- jective function can be defined as the marginal likelihood of the training data, which is typical- ly easy to compute in a generative model. In addition, many types of inductive bias, such as those favoring short dependency arcs (Smith and Eisner, 2006), encouraging correlations between POS tags (Cohen et al., 2008; Cohen and Smith, 2009; Berg-Kirkpatrick et al., 2010; Jiang et al., 2016), and limiting center embedding (Noji et al., 2016), can be incorporated into generative models to achieve better parsing accuracy. However, due to the strong independence assumption in most generative models, it is difficult for these models to utilize context information that has been shown to benefit supervised parsing. Recently, a feature-rich discriminative model for unsupervised parsing is proposed that captures the global context information of sentences (Grave and Elhadad, 2015). Inspired by discriminative clustering, learning of the model is formulated as convex optimization of both the model parameters and the parses of training sentences. By utilizing language-independent rules between pairs of POS tags to guide learning, the model achieves state-of- the-art performance on the UD treebank dataset. In this paper we propose to jointly train two state-of-the-art models of unsupervised …

Content truncated to 500 words