Disentangled Code Representation Learning for Multiple Programming Languages

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4454–4466 August 1–6, 2021. ©2021 Association for Computational Linguistics 4454 Disentangled Code Representation Learning for Multiple Programming Languages Jingfeng Zhang1 Haiwen Hong1 Yin Zhang1∗ Yao Wan2 Ye Liu3 Yulei Sui4 1 College of Computer Science and Technology, Zhejiang University, Hangzhou, China 2School of Computer Sci. & Tech., Huazhong University of Science and Technology, China 3 Department of Computer Science, University of Illinois at Chicago, IL, USA 4 School of Computer Science, University of Technology Sydney, Australia {zhjf, honghaiwen, zhangyin98}@zju.edu.cn, wanyao@hust.edu.cn, yliu279@uic.edu, yulei.sui@uts.edu.au Abstract Developing effective distributed representa- tions of source code is fundamental yet chal- lenging for many software engineering tasks such as code clone detection, code search, code translation and transformation. However, current code embedding approaches that rep- resent the semantic and syntax of code in a mixed way are less interpretable and the re- sulting embedding can not be easily general- ized across programming languages. In this paper, we propose a disentangled code repre- sentation learning approach to separate the se- mantic from the syntax of source code under a multi-programming-language setting, obtain- ing better interpretability and generalizability. Specially, we design three losses dedicated to the characteristics of source code to enforce the disentanglement effectively. We conduct comprehensive experiments on a real-world dataset composed of programming exercises implemented by multiple solutions that are se- mantically identical but grammatically distin- guished. The experimental results validate the superiority of our proposed disentangled code representation, compared to several base- lines, across three types of downstream tasks, i.e., code clone detection, code translation, and code-to-code search. 1 Introduction Code representation learning has become an essen- tial technique to support various software engineer- ing tasks. Most of previous code representation learning approaches (Chen and Zhou, 2018; Jain et al., 2020; Nie et al., 2020) focus on a particular programming language, while learning code rep- resentations for multiple programming languages, though challenging, is an important step towards more generalizable and interpretable code embed- dings. In principle, code snippets can be seen as ∗Corresponding author: Yin Zhang Figure 1: We disentangle the code representation into semantic and syntactic parts. The semantic part, which is relevant to code functionality but is often indepen- dent of a specific language, can be reused for semantic- related tasks across programming languages. The syn- tax part, which is related to a particular language but does not represent the code functionality, can be reused to control syntactic transformations for cross- programming language generation tasks. code token sequences where their structural infor- mation often manifests as tree or graph data struc- tures like AST (Abstract Syntax Tree). The down- stream tasks often take full advantage of different code modalities (DQ et al., 2019) (e.g., structural information and textual tokens in the form of natu- ral languages) to achieve better performance. It is noteworthy that syntax-level noise is an im- portant issue in cross-language semantic-related tasks. Simply mixing textual token information and structural information of code (e.g., ASTs) often can not boost the …

Content truncated to 500 words