From neighborhood to parenthood: the advantages of dependency representation over bigrams in Brown clustering

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1382–1391, Dublin, Ireland, August 23-29 2014. From neighborhood to parenthood: the advantages of dependency representation over bigrams in Brown clustering Simon ˇSuster University of Groningen Netherlands s.suster@rug.nl Gertjan van Noord University of Groningen Netherlands g.j.m.van.noord@rug.nl Abstract We present an effective modification of the popular Brown et al. 1992 word clustering algorithm, using a dependency language model. By leveraging syntax-based context, resulting clusters are better when evaluated against a wordnet for Dutch. The improvements are stable across parameters such as number of clusters, minimum frequency and granularity. Further refinement is possible through dependency relation selection. Our approach achieves a desired clustering quality with less data, resulting in a decrease in cluster creation times. 1 Introduction Semi-supervised approaches have been successful in various areas of natural language processing. Among a plethora of clustering techniques, Brown clustering (Brown et al., 1992) is popular for its conceptual simplicity, available implementations (Liang, 2005; Stolcke, 2002), and because the resulting word clusters can be helpful for several tasks. Clusters are used as syntactic and semantic generalizations of words, requiring fewer model parameters. Brown clustering (section 2) groups words based on shared context. However, only immediately adjacent words are taken into account as recognized e.g. by Koo et al. (2008), Sagae and Gordon (2009), and Grave et al. (2013). For example, even though verbs constitute an informative context for object nouns, they are rarely considered in Brown clustering, unlike in dependency-based clustering. The difference between the contexts can be illustrated with the following example: The method repeatedly samples the data bigram contexts dependency contexts The bigram context thus fails to capture the relation between the object data and the predicate samples, as well as the one between the subject method and the predicate. Furthermore, the dependency representation rightly ignores some of the less informative contexts coming from immediately adjacent words. For example, there is no relation between the predicate samples and the article the to the right. It might be preferable therefore to induce word clusters based on the dependency relations in which the words occur. In section 3, we present how this relates to Brown clustering, and we modify the code by Percy Liang, so that dependency clustering can be used. We evaluate clusters in a wordnet-based similarity experiment. Dependency clustering yields superior clusters for Dutch across different settings of parameters such as number of clusters, frequency threshold and level of granularity. Selecting specific dependency relation labels and using data obtained from them as input to clustering further improves the clustering quality. The proposed adaptation of Brown clustering does not change the complexity of the This work is licenced under a Creative Commons Attribution 4.0 International License. Page numbers and proceedings footer are added by the organizers. License details: http://creativecommons.org/licenses/by/4.0/ 1382 algorithm, and—although we assume that syntactically parsed text is available—it requires much less data for a desired level of clustering quality. 2 The Brown clustering algorithm Brown clustering (Brown et al., …

Content truncated to 500 words