Learning Sequential and Structural Information for Source Code Summarization

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2842–2851 August 1–6, 2021. ©2021 Association for Computational Linguistics 2842 Learning Sequential and Structural Information for Source Code Summarization YunSeok Choi, JinYeong Bak, CheolWon Na, Jee-Hyong Lee College of Computing and Informatics Sungkyunkwan University Suwon, South Korea {ys.choi, jy.bak, ncw0034, john}@skku.edu Abstract We propose a model that learns both the se- quential and the structural features of code for source code summarization. We adopt the abstract syntax tree (AST) and graph convolution to model the structural informa- tion and the Transformer to model the se- quential information. We convert code snip- pets into ASTs and apply graph convolution to obtain structurally-encoded node represen- tations. Then, the sequences of the graph- convolutioned AST nodes are processed by the Transformer layers. Since structurally- neighboring nodes will have similar represen- tations in graph-convolutioned trees, the Trans- former layers can effectively capture not only the sequential information but also the struc- tural information such as sentences or blocks of source code. We show that our model out- performs the state-of-the-art for source code summarization by experiments and human evaluations. 1 Introduction Descriptions of source code are very important documents for programmers. Good descriptions help programmers understand the meaning of code quickly and easily. Source code has sequential information (code to- kens) and structural information (dependency and structure). To understand the content of the code and generate a good summary, both pieces of in- formation are essential to understand the summary. However, most previous works used only one kind of information. Iyer et al. (2016); Liang and Zhu (2018); Hu et al. (2018b); Allamanis et al. (2016) simply converted the source code into sequences of tokens and tried to extract the features of the source code from the sequential information of the source code. They rarely considered the structure information about the relationship between tokens. On the other hand, Shido et al. (2019); Harer et al. (2019); LeClair et al. (2020); Scarselli et al. (2008) proposed tree-based models to capture the features of the source code. They used the structural infor- mation from parse trees but hardly considered the sequence information of code tokens. In order to accurately understand and represent the source code, it is necessary to encode the struc- tural information as well as the sequential infor- mation. Recently, Ahmad et al. (2020) tried to rep- resent both the sequential and the structural infor- mation using the Transformer model with relative encoding. Since relative encoding clipped the max- imum distance for attention without considering the parse tree, it is limited to represent the structure of the code. In this work, we propose a model that learns both the structural and sequential information of source code. We represent code snippets as abstract syntax trees (ASTs), and apply graph convolution (Kipf and Welling, 2017) to the ASTs to obtain the node representation reflecting the tree structure such as parents, children, and siblings. Nodes that are close to each other in a tree, such as parent …

Content truncated to 500 words