Effective Measures of Domain Similarity for Parsing

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 1566–1576, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Effective Measures of Domain Similarity for Parsing Barbara Plank University of Groningen The Netherlands b.plank@rug.nl Gertjan van Noord University of Groningen The Netherlands G.J.M.van.Noord@rug.nl Abstract It is well known that parsing accuracy suf- fers when a model is applied to out-of-domain data. It is also known that the most benefi- cial data to parse a given domain is data that matches the domain (Sekine, 1997; Gildea, 2001). Hence, an important task is to select appropriate domains. However, most previ- ous work on domain adaptation relied on the implicit assumption that domains are some- how given. As more and more data becomes available, automatic ways to select data that is beneficial for a new (unknown) target domain are becoming attractive. This paper evaluates various ways to automatically acquire related training data for a given test set. The results show that an unsupervised technique based on topic models is effective – it outperforms ran- dom data selection on both languages exam- ined, English and Dutch. Moreover, the tech- nique works better than manually assigned la- bels gathered from meta-data that is available for English. 1 Introduction and Motivation Previous research on domain adaptation has focused on the task of adapting a system trained on one do- main, say newspaper text, to a particular new do- main, say biomedical data. Usually, some amount of (labeled or unlabeled) data from the new domain was given – which has been determined by a human. However, with the growth of the web, more and more data is becoming available, where each doc- ument “is potentially its own domain” (McClosky et al., 2010). It is not straightforward to determine which data or model (in case we have several source domain models) will perform best on a new (un- known) target domain. Therefore, an important is- sue that arises is how to measure domain similar- ity, i.e. whether we can find a simple yet effective method to determine which model or data is most beneficial for an arbitrary piece of new text. More- over, if we had such a measure, a related question is whether it can tell us something more about what is actually meant by “domain”. So far, it was mostly arbitrarily used to refer to some kind of coherent unit (related to topic, style or genre), e.g.: newspa- per text, biomedical abstracts, questions, fiction. Most previous work on domain adaptation, for in- stance Hara et al. (2005), McClosky et al. (2006), Blitzer et al. (2006), Daum´e III (2007), sidestepped this problem of automatic domain selection and adaptation. For parsing, to our knowledge only one recent study has started to examine this issue (Mc- Closky et al., 2010) – we will discuss their approach in Section 2. Rather, an implicit assumption of all of these studies is that domains are given, i.e. that they are represented by the respective corpora. Thus, a …

Content truncated to 500 words