Exploring Difficulties in Parsing Imperatives and Questions

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 749–757, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Exploring Difficulties in Parsing Imperatives and Questions Tadayoshi Hara National Institute of Informatics, Japan harasan@nii.ac.jp Takuya Matsuzaki University of Tokyo, Japan matuzaki@is.s.u-tokyo.ac.jp Yusuke Miyao National Institute of Informatics, Japan yusuke@nii.ac.jp Jun’ichi Tsujii Microsoft Research Asia, China jtsujii@microsoft.com Abstract This paper analyzes the effect of the struc- tural variation of sentences on parsing per- formance. We examine the performance of both shallow and deep parsers for two sen- tence constructions: imperatives and ques- tions. We first prepare an annotated cor- pus for each of these sentence construc- tions by extracting sentences from a fiction domain that cover various types of impera- tives and questions. The target parsers are then adapted to each of the obtained cor- pora as well as the existing query-focused corpus. Analysis of the experimental re- sults reveals that the current mainstream parsing technologies and adaptation tech- niques cannot cope with different sentence constructions even with much in-domain data. 1 Introduction Parsing is a fundamental natural language pro- cessing task and essential to various NLP appli- cations. Recent research on parsing technologies has achieved high parsing accuracy in the same domain as the training data, but once we move to unfamiliar domains, the performance decreases to unignorable levels. To address this problem, previous work has fo- cused mainly on adapting lexical or syntactic pref- erences to the target domain, that is, on adding lexical knowledge or adjusting probabilistic mod- els for the target domain using available resources in the target domain (see Section 2). Underlying the previous approaches, there seems to be the as- sumption that grammatical constructions are not largely different between domains or do not affect parsing systems, and therefore the same parsing system can be applied to a novel domain. However, there are some cases where we can- not achieve such high parsing accuracy as parsing the Penn Treebank (PTB) merely by re-training or adaptation. For example, the parsing accu- racy for the Brown Corpus is significantly lower than that for the Wall Street Journal (WSJ) por- tion of the Penn Treebank, even when re-training the parser with much more in-domain training data than other successful domains. This research attempts to identify the cause of these difficulties, and focuses on two types of sen- tence constructions: imperatives and questions. In these constructions, words in certain syntac- tic positions disappear or the order of the words changes. Although some recent works have dis- cussed the effect of these sentence constructions on parsing, they have focused mainly on more well-formed or style-restricted constructions such as QA queries, etc. This research broadens the target scope to include various types of impera- tives and questions. We analyze how such sen- tences affect the parsing behavior and then attempt to clarify the difficulties in parsing imperatives and questions. To do so, we first prepare an annotated corpus for each of the two sentence constructions by borrowing sentences …

Content truncated to 500 words