Fix-Filter-Fix: Intuitively Connect Any Models for Effective Bug Fixing

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3495–3504 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3495 Fix-Filter-Fix: Intuitively Connect Any Models for Effective Bug Fixing Haiwen Hong1 Jingfeng Zhang1 Yin Zhang1∗ Yao Wan2 Yulei Sui3 1 College of Computer Science and Technology, Zhejiang University, Hangzhou, China 2School of Computer Sci. & Tech., Huazhong University of Science and Technology, China 3 School of Computer Science, University of Technology Sydney, Australia {honghaiwen96, zhjf, zhangyin98}@zju.edu.cn, wanyao@hust.edu.cn, yulei.sui@uts.edu.au Abstract Locating and fixing bugs is a time-consuming task. Most neural machine translation (NMT) based approaches for automatically bug fixing lack generality and do not make full use of the rich information in the source code. In NMT- based bug fixing, we find some predicted code identical to the input buggy code (called un- changed fix) in NMT-based approaches due to high similarity between buggy and fixed code (e.g., the difference may only appear in one particular line). Obviously, unchanged fix is not the correct fix because it is the same as the buggy code that needs to be fixed. Based on these, we propose an intuitive yet effective gen- eral framework (called Fix-Filter-Fix or F3) for bug fixing. F3 connects models with our filter mechanism to filter out the last model’s unchanged fix to the next. We propose an F3 theory that can quantitatively and accurately calculate the F3 lifting effect. To evaluate, we implement the Seq2Seq Transformer (ST) and the AST2Seq Transformer (AT) to form some basic F3 instances, called F3 ST +AT and F3 AT +ST . Comparing them with single model approaches and many model connection base- lines across four datasets validates the effec- tiveness and generality of F3 and corroborates our findings and methodology. 1 Introduction Locating and repairing bugs of programs automati- cally is important in software engineering. Many approaches (Tufano et al., 2019; Chen et al., 2019; Chakraborty et al., 2020) based on the Neural Ma- chine Translation (NMT) have achieved promising performance for semantic bug fixing. The basic idea is to automatically translate a buggy code frag- ment into a fixed patch. However, there still exist some limitations. Most of them (1) do not fully ex- ploit the information of source code and only partly using the textual or structured information; (2) are single model architectures with poor generality. ∗Corresponding author: Yin Zhang. Model public void METHOD_1 ( ) { VAR_1.METHOD_2() ; VAR_1 = false ; VAR_1.METHOD_3() ; } Buggy code Test Set Identical Different Unchanged fix public void METHOD_1 ( ) { VAR_1.METHOD_2() ; VAR_1 = false ; VAR_1.METHOD_3() ; } Predicted code public void METHOD_1 ( ) { VAR_1.METHOD_2() ; VAR_1 .set(false) ; VAR_1.METHOD_3() ; } Ground truth fixed code Fail to fix Figure 1: A motivating example. We find that many current NMT-based bug fix- ing models often predict exactly the same output code as the input buggy codes as Figure 1 shows (see red lines), which we call unchanged fix. The input code is buggy, while the …

Content truncated to 500 words