Automatic Domain Adaptation for Parsing

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 28–36, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Automatic Domain Adaptation for Parsing David McCloskya,b aStanford University Stanford, CA, USA mcclosky@stanford.edu Eugene Charniakb bBrown University Providence, RI, USA ec@cs.brown.edu Mark Johnsonc,b cMacquarie University Sydney, NSW, Australia mjohnson@science.mq.edu.au Abstract Current statistical parsers tend to perform well only on their training domain and nearby gen- res. While strong performance on a few re- lated domains is sufficient for many situations, it is advantageous for parsers to be able to gen- eralize to a wide variety of domains. When parsing document collections involving het- erogeneous domains (e.g. the web), the op- timal parsing model for each document is typ- ically not obvious. We study this problem as a new task — multiple source parser adapta- tion. Our system trains on corpora from many different domains. It learns not only statistics of those domains but quantitative measures of domain differences and how those differences affect parsing accuracy. Given a specific tar- get text, the resulting system proposes linear combinations of parsing models trained on the source corpora. Tested across six domains, our system outperforms all non-oracle base- lines including the best domain-independent parsing model. Thus, we are able to demon- strate the value of customizing parsing models to specific domains. 1 Introduction In statistical parsing literature, it is common to see parsers trained and tested on the same textual do- main (Charniak and Johnson, 2005; McClosky et al., 2006a; Petrov and Klein, 2007; Carreras et al., 2008; Suzuki et al., 2009, among others). Unfor- tunately, the performance of these systems degrades on sentences drawn from a different domain. This issue can be seen across different parsing models (Sekine, 1997; Gildea, 2001; Bacchiani et al., 2006; McClosky et al., 2006b). Given that some aspects of syntax are domain dependent (typically at the lexi- cal level), single parsing models tend to not perform well across all domains (see Table 1). Thus, statis- tical parsers inevitably learn some domain-specific properties in addition to the more general properties of a language’s syntax. Recently, Daum´e III (2007) and Finkel and Manning (2009) showed techniques for training models that attempt to separate domain- specific and general properties. However, even when given models for multiple training domains, it is not straightforward to determine which model performs best on an arbitrary piece of novel text. This problem comes to the fore when one wants to parse document collections where each document is potentially its own domain. This shows up par- ticularly when parsing the web. Recently, there has been much interest in applying parsers to the web for the purposes of information extraction and other forms of analysis (c.f. the CLSP 2009 summer workshop “Parsing the Web: Large-Scale Syntactic Processing”). The scale of the web demands an au- tomatic solution to the domain detection and adap- tation problems. Furthermore, it is not obvious that human annotators can determine the optimal parsing models for …

Content truncated to 500 words