A Deep Neural Information Fusion Architecture for Textual Network Embeddings

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4698–4706, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4698 A Deep Neural Information Fusion Architecture for Textual Network Embeddings Zenan Xu1,2, Qinliang Su1,2∗, Xiaojun Quan1, Weijia Zhang3 1School of Data and Computer Science, Sun Yat-sen University 2Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, China 3School of Electronics and Information Technology, Sun Yat-sen University xuzn@mail2.sysu.edu.cn, suqliang@mail.sysu.edu.cn quanxj3@mail.sysu.edu.cn, zhangwj39@mail2.sysu.edu.cn Abstract Textual network embeddings aim to learn a low-dimensional representation for every node in the network so that both the structural and textual information from the networks can be well preserved in the representations. Tradi- tionally, the structural and textual embeddings were learned by models that rarely take the mutual influences between them into account. In this paper, a deep neural architecture is pro- posed to effectively fuse the two kinds of in- formations into one representation. The nov- elties of the proposed architecture are mani- fested in the aspects of a newly defined objec- tive function, the complementary information fusion method for structural and textual fea- tures, and the mutual gate mechanism for tex- tual feature extraction. Experimental results show that the proposed model outperforms the comparing methods on all three datasets. 1 Introduction Networks provide an effective way to organize heterogeneous relevant data, which can often be leveraged to facilitate downstream applications. For example, the huge amount of textual and rela- tionship data in social networks contains abundant information on people’s preferences, and thus can be used for personalized advertising and recom- mendation. To this end, traditionally a matrix rep- resenting the network structure is often built first, then subsequent tasks proceed. However, matrix methods are computationally expensive and can- not be applied to large-scale networks. Network embedding (NE) maps every node of a network into a low-dimensional vector, while seeking to retain the original network informa- tion. Subsequent tasks (e.g. similar vertices search, linkage prediction) can proceed by lever- aging these low-dimensional features. To obtain ∗Corresponding author. network embeddings, (Perozzi et al., 2014) pro- posed to first generate sequences of nodes by ran- domly walking along connected nodes. Word em- bedding methods are then employed to produce the embeddings for nodes by noting the analogies between node sequences and sentences in natural languages. Second-order proximity information is further taken into account in (Tang et al., 2015). To gather the network connection information more efficiently, the random walking strategy in (Per- ozzi et al., 2014) is modified to favor the important nodes in (Grover and Leskovec, 2016). However, all these methods only took the network structure into account, ignoring the huge amount of tex- tual data. In Twitter social network, for exam- ple, tweets posted by a user contain valuable infor- mation on the user’s preferences, political stand- points, and so on (Bandyopadhyay et al., 2018). To include textual information into the embed- dings, (Tu et al., 2017) proposed to first …

Content truncated to 500 words