RACE: Retrieval-augmented Commit Message Generation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5520 - 5530 December 7-11, 2022 ©2022 Association for Computational Linguistics RACE: Retrieval-Augmented Commit Message Generation Ensheng Shia Yanlin Wangb,§,† Wei Taoc Lun Dud Hongyu Zhange Shi Hand Dongmei Zhangd Hongbin Suna,§ aXi’an Jiaotong University bSchool of Software Engineering, Sun Yat-sen University cFudan University dMicrosoft Research eThe University of Newcastle s1530129650@stu.xjtu.edu.cn, hsun@mail.xjtu.edu.cn wangylin36@mail.sysu.edu.cn, wtao18@fudan.edu.cn {lun.du, shihan, dongmeiz}@microsoft.com hongyu.zhang@newcastle.edu.au Abstract Commit messages are important for software development and maintenance. Many neu- ral network-based approaches have been pro- posed and shown promising results on auto- matic commit message generation. However, the generated commit messages could be repet- itive or redundant. In this paper, we pro- pose RACE, a new retrieval-augmented neu- ral commit message generation method, which treats the retrieved similar commit as an ex- emplar and leverages it to generate an accu- rate commit message. As the retrieved com- mit message may not always accurately de- scribe the content/intent of the current code diff, we also propose an exemplar guider, which learns the semantic similarity between the re- trieved and current code diff and then guides the generation of commit message based on the similarity. We conduct extensive experi- ments on a large public dataset with five pro- gramming languages. Experimental results show that RACE can outperform all base- lines. Furthermore, RACE can boost the per- formance of existing Seq2Seq models in com- mit message generation. Our data and source code are available at https://github.com/ DeepSoftwareAnalytics/RACE. 1 Introduction In software development and maintenance, source code is frequently changed. In practice, code changes are often documented as natural language commit messages, which summarize what (con- tent) the code changes are or why (intent) the code is changed (Buse and Weimer, 2010; Cortes-Coy et al., 2014). High-quality commit messages are essential to help developers understand the evo- lution of software without diving into implemen- tation details, which can save a large amount of §Yanlin Wang and Hongbin Sun are the corresponding authors. †Work done during the author’s employment at Microsoft Research Asia time and effort in software development and main- tenance (Dias et al., 2015; Barnett et al., 2015). However, it is difficult to write high-quality com- mit messages due to lack of time, clear motivation, or experienced skills. Even for seasoned develop- ers, it still poses a considerable amount of extra workload to write a concise and informative com- mit message for massive code changes (Nie et al., 2021). It is also reported that around 14% of com- mit messages over 23,000 projects in SourceForge are left empty (Dyer et al., 2013). Thus, automat- ically generating commit messages becomes an important task. Over the years, many approaches have been proposed to automatically generate commit mes- sages. Early studies (Shen et al., 2016; Cortes-Coy et al., 2014) are mainly based on predefined rules or templates, which may not cover all situations or comprehensively infer the intentions behind code changes. Later, some studies (Liu et al., 2018; Huang et al., 2017, …

Content truncated to 500 words