Position-aware Attention and Supervised Data Improve Slot Filling

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 35–45 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Position-aware Attention and Supervised Data Improve Slot Filling Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, Christopher D. Manning Stanford University Stanford, CA 94305 {yuhao, vzhong, danqi}@cs.stanford.edu {angeli, manning}@cs.stanford.edu Abstract Organized relational knowledge in the form of “knowledge graphs” is important for many applications. However, the abil- ity to populate knowledge bases with facts automatically extracted from documents has improved frustratingly slowly. This paper simultaneously addresses two issues that have held back prior work. We first propose an effective new model, which combines an LSTM sequence model with a form of entity position-aware attention that is better suited to relation extraction. Then we build TACRED, a large (119,474 examples) supervised relation extraction dataset, obtained via crowdsourcing and targeted towards TAC KBP relations. The combination of better supervised data and a more appropriate high-capacity model enables much better relation extraction performance. When the model trained on this new dataset replaces the previous rela- tion extraction component of the best TAC KBP 2015 slot filling system, its F1 score increases markedly from 22.2% to 26.7%. 1 Introduction A basic but highly important challenge in natu- ral language understanding is being able to pop- ulate a knowledge base with relational facts con- tained in a piece of text. For the text shown in Fig- ure 1, the system should extract triples, or equiv- alently, knowledge graph edges, such as hPenner, per:spouse, Lisa Dillmani. Combining such ex- tractions, a system can produce a knowledge graph of relational facts between persons, organizations, and locations in the text. This task involves en- tity recognition, mention coreference and/or entity linking, and relation extraction; we focus on the Penner is survived by his brother, John, a copy editor at the Times, and his former wife, Times sportswriter Lisa Dillman. Subject Relation Object Mike Penner per:spouse Lisa Dillman Mike Penner per:siblings John Penner Lisa Dillman per:title Sportswriter Lisa Dillman per:employee of Los Angeles Times John Penner per:title Copy Editor John Penner per:employee of Los Angeles Times Figure 1: An example of relation extraction from the TAC KBP corpus. most challenging “slot filling” task of filling in the relations between entities in the text. Organized relational knowledge in the form of “knowledge graphs” has become an important knowledge resource. These graphs are now exten- sively used by search engine companies, both to provide information to end-users and internally to the system, as a way to understand relationships. However, up until now, automatic knowledge ex- traction has proven sufficiently difficult that most of the facts in these knowledge graphs have been built up by hand. It is therefore a key challenge to show that NLP technology can effectively con- tribute to this important problem. Existing work on relation extraction (e.g., Ze- lenko et al., 2003; Mintz et al., 2009; Adel et al., 2016) has been unable to achieve sufficient re- call or precision for the results to …

Content truncated to 500 words