Repo4QA: Answering Coding Questions via Dense Retrieval on GitHub Repositories

Proceedings of the 29th International Conference on Computational Linguistics, pages 1580–1592 October 12–17, 2022. 1580 Repo4QA: Answering Coding Questions via Dense Retrieval on GitHub Repositories Minyu Chen1, Guoqiang Li1∗, Chen Ma2, Jingyang Li1, Hongfei Fu1 1Shanghai Jiao Tong University, Shanghai, China 2City University of Hong Kong, Hong Kong S.A.R. {minkow,li.g,lijjjjj}@sjtu.edu.cn chenma@cityu.edu.hk, fuhf@cs.sjtu.edu.cn Abstract Open-source platforms such as GitHub and Stack Overflow both play significant roles in current software ecosystems. It is crucial but time-consuming for developers to raise pro- gramming questions in coding forums such as Stack Overflow and be navigated to actual solutions on GitHub repositories. In this pa- per, we dedicate to accelerating this activity. We find that traditional information retrieval- based methods fail to handle the long and complex questions in coding forums, and thus cannot find suitable coding repositories. To effectively and efficiently bridge the seman- tic gap between repositories and real-world coding questions, we introduce a specialized dataset named Repo4QA, which includes over 12,000 question-repository pairs constructed from Stack Overflow and GitHub. Further- more, we propose QuRep, a CodeBERT-based model that jointly learns the representation of both questions and repositories. Experimental results demonstrate that our model simultane- ously captures the semantic features in both questions and repositories through supervised contrastive loss and hard negative sampling. We report that our approach outperforms exist- ing state-of-art methods by 3%-8% on MRR and 5%-8% on P@1. 1 1 Introduction With the increasing popularity of software develop- ment, Stack Overflow and GitHub (two large-scale communities widely recognized in open source ecosystems) have attracted growing research in- terests. As the idiom goes, “Don’t reinvent the wheel”. Tackling programming problems with ex- isting codes and documents is more effective and economical, since various resources in repositories can provide more helpful information than text- formed answers. Specifically, developers can get ∗Corresponding author. 1Our dataset and code are available at https:// github.com/Minkow/Repo4QA help and advice to solve the technical challenges they face, and be provided a variety of solutions and tools in repositories on GitHub. As a vast num- ber of challenges have already been considered and settled by the community, sophisticated schemes posted on GitHub can help satisfy requirements or solve problems discussed in Stack Overflow. Nat- urally, many answers in Stack Overflow provide links to GitHub repositories, and a large amount of these answers are acknowledged to be high-quality and helpful by the community. This phenomenon is worth investigating to determine its contribution to the efficiency improvement and the code reuse of the open source ecosystem. Motivated by the interplay between Stack Over- flow and GitHub, we introduce a novel question- repository matching task. Given a natural- language-formed question in the programming do- main, we aim to search for the most relevant GitHub repository from all candidate repositories as the answer. Figure 1 illustrates the interaction between a question and a repository, where the key information for problem-solving is framed. To this end, we introduce Repo4QA, a dataset consisting of 12,995 question-repository pairs2 for complex coding question solving. The questions are …

Content truncated to 500 words