Hierarchical Joint Learning: Improving Joint Parsing and Named Entity Recognition with Non-Jointly Labeled Data

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 720–728, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Hierarchical Joint Learning: Improving Joint Parsing and Named Entity Recognition with Non-Jointly Labeled Data Jenny Rose Finkel and Christopher D. Manning Computer Science Department Stanford University Stanford, CA 94305 {jrfinkel|manning}@cs.stanford.edu Abstract One of the main obstacles to produc- ing high quality joint models is the lack of jointly annotated data. Joint model- ing of multiple natural language process- ing tasks outperforms single-task models learned from the same data, but still under- performs compared to single-task models learned on the more abundant quantities of available single-task annotated data. In this paper we present a novel model which makes use of additional single-task anno- tated data to improve the performance of a joint model. Our model utilizes a hier- archical prior to link the feature weights for shared features in several single-task models and the joint model. Experiments on joint parsing and named entity recog- nition, using the OntoNotes corpus, show that our hierarchical joint model can pro- duce substantial gains over a joint model trained on only the jointly annotated data. 1 Introduction Joint learning of multiple types of linguistic struc- ture results in models which produce more consis- tent outputs, and for which performance improves across all aspects of the joint structure. Joint models can be particularly useful for producing analyses of sentences which are used as input for higher-level, more semantically-oriented systems, such as question answering and machine trans- lation. These high-level systems typically com- bine the outputs from many low-level systems, such as parsing, named entity recognition (NER) and coreference resolution. When trained sepa- rately, these single-task models can produce out- puts which are inconsistent with one another, such as named entities which do not correspond to any nodes in the parse tree (see Figure 1 for an ex- ample). Moreover, one expects that the different types of annotations should provide useful infor- mation to one another, and that modeling them jointly should improve performance. Because a named entity should correspond to a node in the parse tree, strong evidence about either aspect of the model should positively impact the other as- pect. However, designing joint models which actu- ally improve performance has proven challeng- ing. The CoNLL 2008 shared task (Surdeanu et al., 2008) was on joint parsing and semantic role labeling, but the best systems (Johansson and Nugues, 2008) were the ones which completely decoupled the tasks. While negative results are rarely published, this was not the first failed at- tempt at joint parsing and semantic role label- ing (Sutton and McCallum, 2005). There have been some recent successes with joint modeling. Zhang and Clark (2008) built a perceptron-based joint segmenter and part-of-speech (POS) tagger for Chinese, and Toutanova and Cherry (2009) learned a joint model of lemmatization and POS tagging which outperformed a pipelined model. Adler and Elhadad (2006) presented an HMM- based approach for unsupervised joint morpho- logical segmentation and …

Content truncated to 500 words