Learning Transferable Representation for Bilingual Relation Extraction via Convolutional Neural Networks

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 674–684, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Learning Transferable Representation for Bilingual Relation Extraction via Convolutional Neural Networks Bonan Min∗, Zhuolin Jiang∗ Raytheon BBN Technologies 10 Moulton St Cambridge, MA 02138 {bonan.min,zhuolin.jiang}@raytheon.com Marjorie Freedman†, Ralph Weischedel† USC/Information Sciences Institute 4676 Admiralty Way Marina del Rey, CA 90292 {mrf,weisched}@isi.edu Abstract Typically, relation extraction models are trained to extract instances of a relation ontology using only training data from a single language. However, the concepts represented by the relation ontology (e.g. ResidesIn, EmployeeOf) are language in- dependent. The numbers of annotated ex- amples available for a given ontology vary between languages. For example, there are far fewer annotated examples in Spanish and Japanese than English and Chinese. Furthermore, using only language-specific training data results in the need to man- ually annotate equivalently large amounts of training for each new language a sys- tem encounters. We propose a deep neu- ral network to learn transferable, discrim- inative bilingual representation. Experi- ments on the ACE 2005 multilingual train- ing corpus demonstrate that the joint train- ing process results in significant improve- ment in relation classification performance over the monolingual counterparts. The learnt representation is discriminative and transferable between languages. When us- ing 10% (25K English words, or 30K Chi- nese characters) of the training data, our approach results in doubling F1 compared to a monolingual baseline. We achieve comparable performance to the monolin- gual system trained with 250K English words (or 300K Chinese characters) With 50% of training data. ∗indicates co-first authors. These two authors made equal contribution. †This work was done while the author was at Raytheon BBN Technologies. 1 Introduction Semantic relation extraction is critical to many applications including knowledge base popula- tion and question answering. The problem is well-studied when relation-specific annotations are available in a single target language. How- ever, the same relations can be represented us- ing a variety of languages. While the evidence of the relation in context is language specific (e.g. John spent several years living in Beijing vs 约翰在北京生活了几年), the definition of re- lation itself is often language independent (e.g. ResidesIn) and the meaning should be preserved across languages. We hypothesize that common, shared represen- tation can be learnt when annotations are avail- able in multiple languages and propose a bilin- gual relation extraction algorithm for this purpose. Our basic building blocks are Convolutional Neu- ral Networks (CNN) with cross-lingual word em- beddings (Ammar et al., 2016). This allows the system to capture lexical similarities across lan- guages as well as phrase-level semantics. Build- ing on CNNs with cross-lingual embeddings, the algorithm is a joint training algorithm which trains a model from annotated datasets in a pair of lan- guages. We require that the annotated classes be consistent across languages, but do not require an- notations over parallel (or comparable) text. The base system combines two objectives: an objec- tive that predicts the correct relation labels in each dataset …

Content truncated to 500 words