Finding Dependency Parsing Limits over a Large Spanish Corpus

International Joint Conference on Natural Language Processing, pages 942–946, Nagoya, Japan, 14-18 October 2013. Finding Dependency Parsing Limits over a Large Spanish Corpus Muntsa Padr´o1 Miguel Ballesteros2 H´ector Mart´ınez3 Bernd Bohnet4 1Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, Brazil 2Natural Language Processing Group, Universitat Pompeu Fabra, Barcelona, Spain 3Centre for Language Technology, University of Copenhaguen, Denmark 4School of Computer Science, University of Birmingham, United Kingdom muntsa.padro@inf.ufrgs.br, miguel.ballesteros@upf.edu alonso@hum.ku.dk, bohnetb@cs.bham.ac.uk Abstract This paper studies the performance of different parsers over a large Spanish tree- bank. The aim of this work is to assess the limitations of state-of-the-art parsers. We want to select the most appropriate parser for Subcategorization Frame acqui- sition, and we focus our analysis on two aspects: the accuracy drop when parsing out-of-domain data, and the performance over specific labels relevant to our task. 1 Introduction Dependency parsing has been addressed from different perspectives, improving performance as better techniques are developed. Nevertheless, we may wonder whether those results are good enough to be useful for tasks that need parsed sen- tences as input. Depending on the task we want to tackle, we will prefer some labels to be cor- rect with respect to others. For example, in tasks related to extraction of verb complements such as verb Subcategorization Frame (SCF) or Selec- tional Preference acquisition, we are specially in- terested in a parser that correctly detects and labels the arguments of the verbs, while we do not need to have a high accuracy in other kind of relations, such as specifiers or modifiers. In this work, we present a study of the per- formance of different parsers, following the trend started by McDonald and Nivre (2007) and Hara et al. (2009). We want to maximize the perfor- mance of different systems, for Spanish, with the final goal of applying them to concrete tasks, in our case SCF acquisition. For this task, we need to develop a parser that performs well not only in terms of Labeled Attachment Score (LAS), but also that labels verb complements correctly and that performs well when annotating data that is substantially different from the training corpus. 2 Motivation This work was motivated by the intention of build- ing a SCF acquisition system for Spanish (Padr´o et al., 2013). SCF acquisition consists of acquir- ing from data the kind of complements which a verb can appear with (Direct Object, Indirect Ob- ject, etc.) and how this complements are ful- filled (Noun Phrase, Clause, etc.). To perform this task, state-of-the-art systems (Briscoe and Car- roll, 1997; Korhonen, 2002; Messiant, 2008) use parsed data, where the complements of the verbs are already detected. Thus, the first requirement to develop a SCF acquisition system is to have a parser to annotate the input data. We started by training MaltParser (Nivre and Hall, 2005; Nivre et al., 2007b) for Spanish1 us- ing IULA Spanish LSP Treebank (Marimon et al., 2012), which is built from technical text. The re- sults obtained in terms of LAS …

Content truncated to 500 words