Boosting Code Summarization by Embedding Code Structures

Proceedings of the 29th International Conference on Computational Linguistics, pages 5966–5977 October 12–17, 2022. 5966 Boosting Code Summarization by Embedding Code Structures Jikyeong Son, Joonghyuk Hahn, Hyeon-Tae Seo and Yo-Sub Han Department of Computer Science, Yonsei University Seoul, Republic of Korea {jikyeong.son, greghahn, dchs504, emmous}@yonsei.ac.kr Abstract Recent research on code summarization relies on the structural information from the abstract syntax tree (AST) of source codes. It is, how- ever, questionable whether it is the most effec- tive to use AST for expressing the structural information. We find that a program depen- dency graph (PDG) can represent the structure of a code more effectively. We propose PDG Boosting Module (PBM) that encodes PDG into graph embedding and the framework to implement the proposed PBM with the existing models. PBM achieves improvements of 6.67% (BLEU) and 7.47% (ROUGE) on average. We then analyze the experimental results, and examine how PBM helps the training of base- line models and its performance robustness. For the validation of robustness, we measure the performance of an out-of-domain benchmark dataset, and confirm its robustness. In addition, we apply a new evaluation measure, SBERT score, to evaluate the semantic performance. The models implemented with PBM improve the performance of SBERT score. This implies that they generate summaries that are semanti- cally more similar to the reference summary. 1 Introduction In the early stage of generating code summaries, researchers adopted information retrieval tech- niques (Marcus et al., 2004; Poshyvanyk and Mar- cus, 2007; Haiduc et al., 2010) to capture source code semantics. However, code summaries pro- duced by such techniques are often inaccurate to use in practice (Wong et al., 2015). With the help of deep learning, researchers proposed the neural ma- chine translation (NMT) frameworks that automat- ically produce summaries from source code (Iyer et al., 2016; LeClair et al., 2019; Liang and Zhu, 2018; Sridhara et al., 2010). Rather than using code sequence as a sole input, several models built upon the Transformer (Vaswani et al., 2017) used addi- tional data structures and information (Lin et al., 2021; Shi et al., 2021; Choi et al., 2021) to learn obscure features that otherwise would be discarded. Recent studies use pretrained models and im- plementations of graph structures to improve the performance of code summarization. Pretrained models are built upon training a huge quantity of benchmark datasets within a long period. Code- BERT (Feng et al., 2020) and CodeT5 (Wang et al., 2021) are popular pretrained models for various code-related tasks. The graph embedding is re- garded as effective on providing code semantics; especially abstract syntax tree (AST) is the most popular supplement types for reflecting the hierar- chical structure of codes. Several researchers used ASTs and improved the performance of source code summarization (Alon et al., 2019; Zhang et al., 2019; Shido et al., 2019; LeClair et al., 2020). Furthermore, combining with the Trans- former, there are a few improved models including mAST+GCN (Choi et al., 2021), BASTS (Lin et al., 2021), CAST (Shi et al., 2021) and …

Content truncated to 500 words