Bridge Text and Knowledge by Learning Multi-Prototype Entity Mention Embedding

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1623–1633 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1149 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1623–1633 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1149 Bridging Text and Knowledge by Learning Multi-Prototype Entity Mention Embedding Yixin Cao1, Lifu Huang2, Heng Ji2, Xu Chen1, Juanzi Li1∗ 1 Tsinghua National Laboratory for Information Science and Technology Dept. of Computer Science and Technology, Tsinghua University, China 100084 {caoyixin2011,successcx,lijuanzi2008}@gmail.com 2 Dept. of Computer Science, Rensselaer Polytechnic Institute, USA 12180 {huangl7,jih}@rpi.edu Abstract Integrating text and knowledge into a uni- fied semantic space has attracted signifi- cant research interests recently. However, the ambiguity in the common space re- mains a challenge, namely that the same mention phrase usually refers to various entities. In this paper, to deal with the ambiguity of entity mentions, we propose a novel Multi-Prototype Mention Embed- ding model, which learns multiple sense embeddings for each mention by jointly modeling words from textual contexts and entities derived from a knowledge base. In addition, we further design an efficient language model based approach to disam- biguate each mention to a specific sense. In experiments, both qualitative and quan- titative analysis demonstrate the high qual- ity of the word, entity and multi-prototype mention embeddings. Using entity linking as a study case, we apply our disambigua- tion method as well as the multi-prototype mention embeddings on the benchmark dataset, and achieve state-of-the-art per- formance. 1 Introduction Jointly learning text and knowledge representa- tions in a unified vector space greatly benefits many Natural Language Processing (NLP) tasks, such as knowledge graph completion (Han et al., 2016; Wang and Li, 2016), relation extraction (Weston et al., 2013), word sense disambiguation (Mancini et al., 2016), entity classification (Huang et al., 2017) and linking (Huang et al., 2015). Existing work can be roughly divided into two categories. One is encoding words and entities into a unified vector space using Deep Neural ∗Corresponding author. Networks (DNN). These methods suffer from the problems of expensive training and great limita- tions on the size of word and entity vocabulary (Han et al., 2016; Toutanova et al., 2015; Wu et al., 2016). The other is to learn word and entity em- beddings separately, and then align similar words and entities into a common space with the help of Wikipedia hyperlinks, so that they share similar representations (Wang et al., 2014; Yamada et al., 2016). m1 m1 m2 … action film ''Independence Day'', the United States military uses alien technology… … holds annual Independence Day celebrations and other festivals … … bands played it during public events, such as July 4th celebrations. Text d1 d2 d3 Independence Day (film) Independence Day (US) Entity e2 e1 Knowledge Base Independence Day July 4th Mention m2 m1 Figure 1: Examples. However, there are two major problems arising from directly integrating word and entity embed- dings into a …

Content truncated to 500 words