A Neural Network Architecture for Program Understanding Inspired by Human Behaviors

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5142 - 5153 May 22-27, 2022 c⃝2022 Association for Computational Linguistics A Neural Network Architecture for Program Understanding Inspired by Human Behaviors Renyu Zhu1 Lei Yuan1 Xiang Li1∗ Ming Gao1 Wenyuan Cai2 1School of Data Science and Engineering, East China Normal University, Shanghai, China 2Shanghai Hypers Data Technology Inc., Shanghai, China {52175100003, 51205903063}@stu.ecnu.edu.cn {xiangli, mgao}@dase.ecnu.edu.cn wenyuan.cai@hypers.com Abstract Program understanding is a fundamental task in program language processing. Despite the success, existing works fail to take human be- haviors as reference in understanding programs. In this paper, we consider human behaviors and propose the PGNN-EK model that consists of two main components. On the one hand, inspired by the “divide-and-conquer” reading behaviors of humans, we present a partitioning- based graph neural network model PGNN on the upgraded AST of codes. On the other hand, to characterize human behaviors of resorting to other resources to help code comprehension, we transform raw codes with external knowl- edge and apply pre-training techniques for in- formation extraction. Finally, we combine the two embeddings generated from the two com- ponents to output code embeddings. We con- duct extensive experiments to show the supe- rior performance of PGNN-EK on the code summarization and code clone detection tasks. In particular, to show the generalization abil- ity of our model, we release a new dataset that is more challenging for code clone de- tection and could advance the development of the community. Our codes and data are pub- licly available at https://github.com/ RecklessRonan/PGNN-EK. 1 Introduction The past decades have witnessed the prosperity of programming platforms, such as Github and Stack Overflow. These platforms generate mas- sive open-source code1 data that is named as “Big Code” in (Allamanis et al., 2018a). To automate the software development and maintenance, based on the “Software Naturalness” hypothesis (Hindle et al., 2016), natural language processing (NLP) techniques have been applied in program under- standing. After that, a series of downstream pro- gramming language processing (PLP) tasks can be ∗ Corresponding Author 1We interchangeably use code and program in this paper. performed, including code summarization (Zhang et al., 2020; Ahmad et al., 2020; Liu et al., 2021) and code clone detection (Zhang et al., 2019; Wang et al., 2020). Existing works for understanding programs mainly utilize three types of information: code con- text, code structure and external knowledge. Specif- ically, code context refers to the token sequence in the code. For code structure, each code can be parsed into various types of intermediate represen- tations, such as AST (Abstract Syntax Tree), CFG (Control Flow Graph) and PDG (Program Depen- dence Graph). These representations capture the structural information of codes. Further, there also exists external knowledge associated with codes, such as API documentation and other exemplary codes. Despite the success, all these models ignore considering human behaviors in reading programs. Recently, (Bengio et al., 2021) suggest the potential futures of deep learning by comparing current AI methods with human …

Content truncated to 500 words