Improved Relation Extraction with Feature-Rich Compositional Embedding Models

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1774–1784, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Improved Relation Extraction with Feature-Rich Compositional Embedding Models Matthew R. Gormley1⇤ and Mo Yu2⇤and Mark Dredze1 1Human Language Technology Center of Excellence Center for Language and Speech Processing Johns Hopkins University, Baltimore, MD, 21218 2Machine Intelligence and Translation Lab Harbin Institute of Technology, Harbin, China gflfof@gmail.com, {mrg, mdredze}@cs.jhu.edu Abstract Compositional embedding models build a representation (or embedding) for a linguistic structure based on its compo- nent word embeddings. We propose a Feature-rich Compositional Embedding Model (FCM) for relation extraction that is expressive, generalizes to new domains, and is easy-to-implement. The key idea is to combine both (unlexicalized) hand- crafted features with learned word em- beddings. The model is able to directly tackle the difficulties met by traditional compositional embeddings models, such as handling arbitrary types of sentence an- notations and utilizing global information for composition. We test the proposed model on two relation extraction tasks, and demonstrate that our model outper- forms both previous compositional models and traditional feature rich models on the ACE 2005 relation extraction task, and the SemEval 2010 relation classification task. The combination of our model and a log- linear classifier with hand-crafted features gives state-of-the-art results. We made our implementation available for general use1. 1 Introduction Two common NLP feature types are lexical properties of words and unlexicalized linguis- tic/structural interactions between words. Prior work on relation extraction has extensively stud- ied how to design such features by combining dis- crete lexical properties (e.g. the identity of a word, ⇤⇤Gormley and Yu contributed equally. 1https://github.com/mgormley/pacaya its lemma, its morphological features) with as- pects of a word’s linguistic context (e.g. whether it lies between two entities or on a dependency path between them). While these help learning, they make generalization to unseen words difficult. An alternative approach to capturing lexical informa- tion relies on continuous word embeddings2 as representative of words but generalizable to new words. Embedding features have improved many tasks, including NER, chunking, dependency pars- ing, semantic role labeling, and relation extrac- tion (Miller et al., 2004; Turian et al., 2010; Koo et al., 2008; Roth and Woodsend, 2014; Sun et al., 2011; Plank and Moschitti, 2013; Nguyen and Grishman, 2014). Embeddings can capture lexi- cal information, but alone they are insufficient: in state-of-the-art systems, they are used alongside features of the broader linguistic context. In this paper, we introduce a compositional model that combines unlexicalized linguistic con- text and word embeddings for relation extraction, a task in which contextual feature construction plays a major role in generalizing to unseen data. Our model allows for the composition of embed- dings with arbitrary linguistic structure, as ex- pressed by hand crafted features. In the follow- ing sections, we begin with a precise construction of compositional embeddings using word embed- dings in conjunction with unlexicalized features. Various feature sets used in prior work (Turian et al., 2010; Nguyen and Grishman, 2014; Hermann …

Content truncated to 500 words