An Empirical Evaluation of Automatic Conversion from Constituency to Dependency in Hungarian

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1392–1401, Dublin, Ireland, August 23-29 2014. An Empirical Evaluation of Automatic Conversion from Constituency to Dependency in Hungarian Katalin Ilona Simk´o1, Veronika Vincze1,2, Zsolt Sz´ant´o1, Rich´ard Farkas1 1University of Szeged Department of Informatics 2MTA-SZTE Research Group on Artificial Intelligence kata.simko@gmail.com {vinczev,szantozs,rfarkas}@inf.u-szeged.hu Abstract In this paper, we investigate the differences between Hungarian sentence parses based on auto- matically converted and manually annotated dependency trees. We also train constituency parsers on the manually annotated constituency treebank and then convert their output to dependency trees. We argue for the importance of training on gold standard corpora, and we also demon- strate that although the results obtained by training on the constituency treebank and converting the output to dependency format and those obtained by training on the automatically converted dependency treebank are similar in terms of accuracy scores, the typical errors made by different systems differ from each other. 1 Introduction Nowadays, two popular approaches to data-driven syntactic parsing are based on constituency grammar on the one hand and dependency grammar on the other hand. There exist constituency-based treebanks for many languages and dependency treebanks for most of these languages are converted automatically from constituent trees with the help of conversion rules, which is the case for e.g. the languages used in the SPMRL-2013 Shared Task (Seddah et al., 2013) with the exception of Basque, where constituency trees are converted from manually annotated dependency trees (Aduriz et al., 2003), and Hungarian, where both treebanks are manually annotated (Csendes et al., 2005; Vincze et al., 2010). However, the quality of automatic dependency conversion is hardly investigated. Hungarian is one of those rare examples where there exist manual annotations for both constituency and dependency syntax on the same bunch of texts, the Szeged (Dependency) Treebank (Csendes et al., 2005; Vincze et al., 2010), which makes it possible to evaluate the quality of a rule-based automatic con- version from constituency to dependency trees, to compare the two sets of manual annotations and also the output of constituency and dependency parsers trained on converted and gold standard dependency trees. We investigate the effect of automatic conversions related to the two parsing paradigms as well. It is well known that for English, the automatic conversion of a constituency parser’s output to dependency format can achieve competitive unlabeled attachment scores (ULA) to a dependency parser’s output trained on automatically converted trees1 (cf. Petrov et al. (2010)). One of the possible explanations for this is that English is a configurational language, hence constituency parsers have advantages over depen- dency parsers here. We check whether this hypothesis holds for Hungarian too, which is the prototype of free word order languages. In this paper, we compare three pairs of dependency analyses in order to evaluate the usefulness of converted trees. First, we examine the errors of the conversion itself by comparing the converted dependency trees with the manually annotated gold standard ones. Second, we argue for the importance of …

Content truncated to 500 words