Learning to Describe Solutions for Bug Reports Based on Developer Discussions

Findings of the Association for Computational Linguistics: ACL 2022, pages 2935 - 2952 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Learning to Describe Solutions for Bug Reports Based on Developer Discussions Sheena Panthaplackel1, Junyi Jessy Li2, Milos Gligoric3, Raymond J. Mooney1 1Department of Computer Science 2Department of Linguistics 3Department of Electrical and Computer Engineering The University of Texas at Austin spantha@cs.utexas.edu, jessy@austin.utexas.edu gligoric@utexas.edu, mooney@cs.utexas.edu Abstract When a software bug is reported, developers engage in a discussion to collaboratively re- solve it. While the solution is likely formulated within the discussion, it is often buried in a large amount of text, making it difficult to com- prehend and delaying its implementation. To expedite bug resolution, we propose generating a concise natural language description of the so- lution by synthesizing relevant content within the discussion, which encompasses both natural language and source code. We build a corpus for this task using a novel technique for obtain- ing noisy supervision from repository changes linked to bug reports, with which we establish benchmarks. We also design two systems for generating a description during an ongoing dis- cussion by classifying when sufficient context for performing the task emerges in real-time. With automated and human evaluation, we find this task to form an ideal testbed for complex reasoning in long, bimodal dialogue context. 1 Introduction Software bugs in open-source projects are reported through issue tracking systems like GitHub Issues. When a bug is reported, a discussion is initiated among developers to collectively resolve it (Noyori et al., 2019). The bug resolution process is often strenuous and time-consuming, involving extended deliberations (Liu et al., 2020b) among multiple participants (Kavaler et al., 2017), spanning long periods of time (Kikas et al., 2015). Although a solution often emerges within the discussion (Arya et al., 2019), this can easily get lost in a large amount of text (Liu et al., 2020b). Wading through a long discussion to determine whether a solution has been suggested, comprehending it, and then implementing it can be daunting, especially for Figure 1: ExoPlayer bug report discussion with user-written and system-generated solution descriptions. developers who are not closely following the dis- cussion (Arya et al., 2019; Tan et al., 2020). Con- sequently, the resolution can be delayed. As developers scan through the long discussion, it is desirable to have an automated system that guides them to more easily absorb information rel- evant towards implementing the solution. We pro- pose automatically generating a concise natural lan- guage description of the solution by synthesizing relevant content as it emerges in the discussion. For example, as the discussion in Figure 1 progresses, the cause of the bug is identified as the shutter get- ting closed “when seeking to an unprepared period” and a solution emerges: “suppress closing the shut- ter in this case, provided the old and new periods belong to the same window.” Our task aims to describe this solution: Prevent shutter closing for within-window seeks to unprepared periods. To study this task, we build a corpus …

Content truncated to 500 words