Treeblazing: Using External Treebanks to Filter Parse Forests for Parse Selection and Treebanking

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 246–254, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Treeblazing: Using External Treebanks to Filter Parse Forests for Parse Selection and Treebanking Andrew MacKinlay♦♥, Rebecca Dridan♦♥, Dan Flickinger♣, Stephan Oepen♠ and Timothy Baldwin♦♥ ♦ Dept of Computer Science and Software Engineering, University of Melbourne, Australia ♥ NICTA Victoria Research Laboratories, University of Melbourne, Australia ♣ Center for Study of Language and Information, Stanford University, USA ♠ Department of Informatics, Universitetet i Oslo, Norway amack@csse.unimelb.edu.au, rdridan@csse.unimelb.edu.au, danf@stanford.edu, oe@ifi.uio.no, tb@ldwin.net Abstract We describe “treeblazing”, a method of us- ing annotations from the GENIA treebank to constrain a parse forest from an HPSG parser. Combining this with self-training, we show significant dependency score im- provements in a task of adaptation to the biomedical domain, reducing error rate by 9% compared to out-of-domain gold data and 6% compared to self-training. We also demonstrate improvements in treebanking efficiency, requiring 25% fewer decisions, and 17% less annotation time. 1 Introduction Computational linguistic research is driven by the development of reference resources for specific tasks and languages. The advent of services such as Amazon’s Mechanical Turk has driven down the cost of annotation considerably, assuming a given task can be broken down into piecemeal units which are intuitive and manageable for non- experts. This is not an option, however, for fine- grained tasks which require an expert understand- ing of a theory or domain, such as syntactic tree- banking or discourse annotation. Two main approaches have been adopted to ef- ficiently create new resources: (1) domain adap- tation, where a trained model from one domain is stochastically adapted to a new domain, us- ing unlabelled data from the new domain (Daum´e III and Marcu, 2006); and (2) annotation projec- tion, where the labels in a pre-existing resource are semi-automatically translated into an indepen- dent formalism, e.g. in translating the PTB into the CCG formalism (Hockenmaier and Steedman, 2002). This paper looks at both of these ap- proaches: domain adaptation from unannotated data in the form of self-training combined with re- source translation over the GENIA treebank (Yuka et al., 2005), in the context of training an HPSG parse selection model for biomedical text, and us- ing the GENIA treebank annotations and retrained parse selection model to accelerate treebanking. Our contributions are: (1) we propose a series of methods for transferring annotation from a tra- ditional phrase structure treebank to constrain the parse forest of a precision grammar; (2) we show that this constrained forest can be used to domain- adapt a parse selection model; (3) we demonstrate improvements in treebanking performance using the constrained forest; and (4) we develop a small- scale HPSG treebank for the biomedical domain. 2 Related Work Domain adaptation is an active research area, trig- gered by the observation that parsers trained on one domain show decreased performance when used in other domains (Gildea, 2001). Much domain-adaptation work involves some small amount of in-domain data to tune a model, …

Content truncated to 500 words