MetaFill: Text Infilling for Meta-Path Generation on Heterogeneous Information Networks

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5110 - 5122 December 7-11, 2022 ©2022 Association for Computational Linguistics MetaFill: Text Infilling for Meta-Path Generation on Heterogeneous Information Networks Zequn Liu1, Kefei Duan2, Junwei Yang1, Hanwen Xu3, Ming Zhang1∗, Sheng Wang3∗ 1School of Computer Science, Peking University, Beijing, China 2School of EECS, Peking University, Beijing, China 3Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA zequnliu,dkf4116,yjwtheonly,mzhang_cs@pku.edu.cn xuhw,swang@cs.washington.edu Abstract Heterogeneous Information Network (HIN) is essential to study complicated networks con- taining multiple edge types and node types. Meta-path, a sequence of node types and edge types, is the core technique to embed HINs. Since manually curating meta-paths is time- consuming, there is a pressing need to develop automated meta-path generation approaches. Existing meta-path generation approaches can- not fully exploit the rich textual information in HINs, such as node names and edge type names. To address this problem, we propose MetaFill, a text-infilling-based approach for meta-path generation. The key idea of MetaFill is to for- mulate meta-path identification problem as a word sequence infilling problem, which can be advanced by Pretrained Language Models (PLMs). We observed the superior performance of MetaFill against existing meta-path gen- eration methods and graph embedding meth- ods that do not leverage meta-paths in both link prediction and node classification on two real-world HIN datasets. We further demon- strated how MetaFill can accurately classify edges in the zero-shot setting, where existing approaches cannot generate any meta-paths. MetaFill exploits PLMs to generate meta-paths for graph embedding, opening up new avenues for language model applications in graph anal- ysis.1 1 Introduction Heterogeneous Information Network (HIN) is an effective framework to model complicated real- world network data (Sun et al., 2011; Wang et al., 2019; Dong et al., 2017; Shi et al., 2016; Yang et al., 2020; Fu et al., 2017; Yun et al., 2019; Hu et al., 2020). In contrast to a conventional net- work (Tang et al., 2015; Grover and Leskovec, 2016; Perozzi et al., 2014; Hamilton et al., 2017; ∗Corresponding author 1Our code is available at https://github.com/zequnl/ MetaFill Kipf and Welling, 2016; Veliˇckovi´c et al., 2018), HIN supports multiple node types and edge types, thus facilitating the integrative analysis of multi- ple datasets (Chen et al., 2012; Himmelstein and Baranzini, 2015; Zhao et al., 2020). One of the most important applications on HIN is to discover interactions between different node types by fram- ing it as a link prediction problem (Fu et al., 2020; Zhang et al., 2014; Cao et al., 2017). Link predic- tion is particularly challenging on heterogeneous and long-distance node pairs, which often do not share any neighbors, thus presenting substantial false-negative predictions by conventional network- based approaches (Shi et al., 2014; Fu et al., 2016; Daud et al., 2020). HIN exploits meta-paths to address link pre- diction, especially for heterogeneous and long- distance node pairs. A meta-path is a sequence of node types and edge types. A good meta-path often consists of paths that …

Content truncated to 500 words