Is it Really that Difficult to Parse German?

Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 111–119, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Is it Really that Difficult to Parse German? Sandra K¨ubler, Erhard W. Hinrichs, Wolfgang Maier SfS-CL, SFB 441, University of T¨ubingen Wilhelmstr. 19 72074 T¨ubingen, Germany �kuebler,eh,wmaier �@sfs.uni-tuebingen.de Abstract This paper presents a comparative study of probabilistic treebank parsing of Ger- man, using the Negra and T¨uBa-D/Z tree- banks. Experiments with the Stanford parser, which uses a factored PCFG and dependency model, show that, contrary to previous claims for other parsers, lexical- ization of PCFG models boosts parsing performance for both treebanks. The ex- periments also show that there is a big difference in parsing performance, when trained on the Negra and on the T¨uBa- D/Z treebanks. Parser performance for the models trained on T¨uBa-D/Z are compara- ble to parsing results for English with the Stanford parser, when trained on the Penn treebank. This comparison at least sug- gests that German is not harder to parse than its West-Germanic neighbor language English. 1 Introduction There have been a number of recent studies on probabilistic treebank parsing of German (Dubey, 2005; Dubey and Keller, 2003; Schiehlen, 2004; Schulte im Walde, 2003), using the Negra tree- bank (Skut et al., 1997) as their underlying data source. A common theme that has emerged from this research is the claim that lexicalization of PCFGs, which has been proven highly beneficial for other languages1, is detrimental for parsing accuracy of German. In fact, this assumption is by now so widely held that Schiehlen (2004) does not even consider lexicalization as a possible 1For English, see Collins (1999). parameter and concentrates instead only on tree- bank transformations of various sorts in his exper- iments. Another striking feature of all studies men- tioned above are the relatively low parsing F- scores achieved for German by comparison to the scores reported for English, its West-Germanic neighbor, using similar parsers. This naturally raises the question whether German is just harder to parse or whether it is just hard to parse the Ne- gra treebank.2 The purpose of this paper is to address pre- cisely this question by training the Stanford parser (Klein and Manning, 2003b) and the LoPar parser (Schmid, 2000) on the two major treebanks available for German, Negra and T¨uBa-D/Z, the T¨ubingen treebank of written German (Telljohann et al., 2005). A series of comparative parsing experiments that utilize different parameter set- tings of the parsers is conducted, including lexi- calization and markovization. These experiments show striking differences in performance between the two treebanks. What makes this comparison interesting is that the treebanks are of compara- ble size and are both based on a newspaper cor- pus. However, both treebanks differ significantly in their syntactic annotation scheme. Note, how- ever, that our experiments concentrate on the orig- inal (context-free) annotations of the treebank. The structure of this paper is as follows: sec- tion 2 discusses three characteristic grammatical features of German that need to …

Content truncated to 500 words