Deep Linguistic Analysis for the Accurate Identification of Predicate-Argument Relations

Deep Linguistic Analysis for the Accurate Identification of Predicate-Argument Relations Yusuke Miyao Department of Computer Science University of Tokyo yusuke@is.s.u-tokyo.ac.jp Jun’ichi Tsujii Department of Computer Science University of Tokyo CREST, JST tsujii@is.s.u-tokyo.ac.jp Abstract This paper evaluates the accuracy of HPSG parsing in terms of the identification of predicate-argument relations. We could directly compare the output of HPSG parsing with Prop- Bank annotations, by assuming a unique map- ping from HPSG semantic representation into PropBank annotation. Even though PropBank was not used for the training of a disambigua- tion model, an HPSG parser achieved the ac- curacy competitive with existing studies on the task of identifying PropBank annotations. 1 Introduction Recently, deep linguistic analysis has successfully been applied to real-world texts. Several parsers have been implemented in various grammar for- malisms and empirical evaluation has been re- ported: LFG (Riezler et al., 2002; Cahill et al., 2002; Burke et al., 2004), LTAG (Chiang, 2000), CCG (Hockenmaier and Steedman, 2002b; Clark et al., 2002; Hockenmaier, 2003), and HPSG (Miyao et al., 2003; Malouf and van Noord, 2004). How- ever, their accuracy was still below the state-of-the- art PCFG parsers (Collins, 1999; Charniak, 2000) in terms of the PARSEVAL score. Since deep parsers can output deeper representation of the structure of a sentence, such as predicate argument structures, several studies reported the accuracy of predicate- argument relations using a treebank developed for each formalism. However, resources used for the evaluation were not available for other formalisms, and the results cannot be compared with each other. In this paper, we employ PropBank (Kingsbury and Palmer, 2002) for the evaluation of the accu- racy of HPSG parsing. In the PropBank, semantic arguments of a predicate and their semantic roles are manually annotated. Since the PropBank has been developed independently of any grammar for- malisms, the results are comparable with other pub- lished results using the same test data. Interestingly, several studies suggested that the identification of PropBank annotations would re- quire linguistically-motivated features that can be obtained by deep linguistic analysis (Gildea and Hockenmaier, 2003; Chen and Rambow, 2003). They employed a CCG (Steedman, 2000) or LTAG (Schabes et al., 1988) parser to acquire syntac- tic/semantic structures, which would be passed to statistical classifier as features. That is, they used deep analysis as a preprocessor to obtain useful fea- tures for training a probabilistic model or statistical classifier of a semantic argument identifier. These results imply the superiority of deep linguistic anal- ysis for this task. Although the statistical approach seems a reason- able way for developing an accurate identifier of PropBank annotations, this study aims at establish- ing a method of directly comparing the outputs of HPSG parsing with the PropBank annotation in or- der to explicitly demonstrate the availability of deep parsers. That is, we do not apply statistical model nor machine learning to the post-processing of the output of HPSG parsing. By eliminating the effect of post-processing, we can directly evaluate the ac- curacy of deep linguistic analysis. Section 2 introduces recent …

Content truncated to 500 words