Adversarial Training for Code Retrieval with Question-Description Relevance Regularization

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4049–4059 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 4049 Adversarial Training for Code Retrieval with Question-Description Relevance Regularization Jie Zhao The Ohio State University zhao.1359@osu.edu Huan Sun The Ohio State University sun.397@osu.edu Abstract Code retrieval is a key task aiming to match natural and programming languages. In this work, we propose adversarial learning for code retrieval, that is regularized by question- description relevance. First, we adapt a sim- ple adversarial learning technique to generate difficult code snippets given the input ques- tion, which can help the learning of code retrieval that faces bi-modal and data-scarce challenges. Second, we propose to lever- age question-description relevance to regular- ize adversarial learning, such that a generated code snippet should contribute more to the code retrieval training loss, only if its paired natural language description is predicted to be less relevant to the user given question. Exper- iments on large-scale code retrieval datasets of two programming languages show that our adversarial learning method is able to im- prove the performance of state-of-the-art mod- els. Moreover, using an additional duplicate question prediction model to regularize ad- versarial learning further improves the perfor- mance, and this is more effective than using the duplicated questions in strong multi-task learning baselines.1 1 Introduction Recently there has been a growing research interest in the intersection of natural language (NL) and programming language (PL), with exemplar tasks including code generation (Agashe et al., 2019; Bi et al., 2019), code summarizing (LeClair and McMillan, 2019; Panthaplackel et al., 2020), and code retrieval (Gu et al., 2018). In this paper, we study code retrieval, which aims to retrieve code snippets for a given NL question such as “Flatten a shallow list in Python.” Advanced code retrieval tools can save programmers tremendous time in 1Source code and dataset are available at https://github.com/jiez-osu/QQC. various scenarios, such as how to fix a bug, how to implement a function, which API to use, etc. Moreover, even if the retrieved code snippets do not perfectly match the NL question, editing them is often much easier than generating a code snippet from scratch. For example, the retrieve-and-edit paradigm (Hayati et al., 2018; Hashimoto et al., 2018; Guo et al., 2019) for code generation has attracted growing attention recently, which first em- ploys a code retriever to find the most relevant code snippets for a given question, and then edit them via a code generation model. Previous work has shown that code retrieval performance can signif- icantly affect the final generated results (Huang et al., 2018) in such scenarios. There have been two groups of work on code re- trieval: (1) One group of work (e.g., the recent retrieve-and-edit work (Hashimoto et al., 2018; Guo et al., 2019)) assumes each code snippet is associated with NL descriptions and retrieves code snippets by measuring the relevance between such descriptions and a given question. (2) The other group of work (e.g., CODENN (Iyer et al., 2016) and Deep …

Content truncated to 500 words