Object-Extraction and Question-Parsing using CCG

Object-Extraction and Question-Parsing using CCG Stephen Clark and Mark Steedman School of Informatics University of Edinburgh 2 Buccleuch Place, Edinburgh, UK � stevec,steedman � @inf.ed.ac.uk James R. Curran School of Information Technologies University of Sydney NSW 2006, Australia james@it.usyd.edu.au Abstract Accurate dependency recovery has recently been reported for a number of wide-coverage statistical parsers using Combinatory Categorial Grammar (CCG). However, overall figures give no indication of a parser’s performance on spe- cific constructions, nor how suitable a parser is for specific applications. In this paper we give a detailed evaluation of a CCG parser on ob- ject extraction dependencies found in WSJ text. We also show how the parser can be used to parse questions for Question Answering. The accuracy of the original parser on questions is very poor, and we propose a novel technique for porting the parser to a new domain, by creating new labelled data at the lexical category level only. Using a supertagger to assign categories to words, trained on the new data, leads to a dra- matic increase in question parsing accuracy. 1 Introduction Several wide-coverage statistical parsers have re- cently been developed for Combinatory Categorial Grammar (CCG; Steedman, 2000) and applied to the WSJ Penn Treebank (Clark et al., 2002; Hock- enmaier and Steedman, 2002; Hockenmaier, 2003b; Clark and Curran, 2004b). One motivation for using CCG is the recovery of the long-range dependencies inherent in phenomena such as coordination and ex- traction. Recovery of these dependencies is impor- tant for NLP tasks which require semantic interpre- tation and for processing text which contains a high frequency of such cases, e.g. Wh-questions fed to a Question Answering (QA) system. One shortcoming of treebank parsers such as Collins (1999) and Charniak (2000) is that they typi- cally produce phrase-structure trees containing only local syntactic information. Johnson (2002) uses post-processing methods to insert “empty” nodes into the trees, and Dienes and Dubey (2003) use pre- processing methods to determine where discontinu- ities are likely to appear in the sentence. In contrast, the CCG parsers detect long-range dependencies as an integral part of the parsing process. The CCG parser used here (Clark and Curran, 2004b) is highly accurate and efficient, recovering labelled dependencies with an overall F-score of over 84% on WSJ text, and parsing up to 50 sen- tences per second. Thus the parser should be useful for large-scale NLP tasks. However, the overall ac- curacy figures give no indication of the parser’s per- formance on specific constructions, nor how suit- able the parser is for specific applications. In this paper we give a detailed evaluation for object ex- traction dependencies and show how the parser can be used to parse questions for QA. We find that the parser performs well on the ob- ject extraction cases found in the Penn Treebank, given the difficulty of the task. In contrast, the parser performs poorly on questions from TREC, due to the small number of questions in the Penn Treebank. This motivates the remainder of the pa- per, …

Content truncated to 500 words