Contrastive Learning with Keyword-based Data Augmentation for Code Search and Code Question Answering

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3609–3619 May 2-6, 2023 ©2023 Association for Computational Linguistics Contrastive Learning with Keyword-based Data Augmentation for Code Search and Code Question Answering Shinwoo Park1, Youngwook Kim2 and Yo-Sub Han2 1Department of Artificial Intelligence, Yonsei University, Seoul, Republic of Korea 2Department of Computer Science, Yonsei University, Seoul, Republic of Korea {pshkhh, youngwook, emmous}@yonsei.ac.kr Abstract The semantic code search is to find code snip- pets from the collection of candidate code snip- pets with respect to a user query that describes functionality. Recent work on code search proposes data augmentation of queries for contrastive learning. This data augmentation approach modifies random words in queries. When a user web query for searching code snip- pet is too brief, the important word that repre- sents the search intent of the query could be undesirably modified. A code snippet has infor- mative components such as function name and documentation that describe its functionality. We propose to utilize these code components to identify important words and preserve them in the data augmentation step. We present Key- DAC (Keyword-based Data Augmentation for Contrastive learning) that identifies important words for code search from queries and code components based on term matching. KeyDAC augments query-code pairs while preserving keywords, and then leverages generated train- ing instances for contrastive learning. We use KeyDAC to fine-tune various pre-trained lan- guage models and evaluate the performance of code search and code question answering via CoSQA and WebQueryTest. The experimen- tal results confirm that KeyDAC substantially outperforms the current state-of-the-art perfor- mance, and achieves the new state-of-the-arts for both tasks. 1 Introduction Software developers or students who major in com- puter science often write natural language queries to search for code snippets with desired function- ality from the web search engine. The retrieved code snippets are reused or referred to improve pro- ductivity of software development. Semantic code search is a well-known code-related downstream task that measures the semantic relevance between a given natural language query and a collection of code snippets to retrieve the most relevant code snippet. CodeXGLUE (Lu et al., 2021), a bench- mark for 10 code tasks, provides two test sets— CodeSearchNet AdvTest and WebQueryTest—for an open challenge code search. CodeSearchNet AdvTest from the CodeSearchNet (Husain et al., 2019) corpus is for a retrieval scenario in which one needs to find a Python code function that matches the search intent of the query best. The queries in CodeSearchNet AdvTest are not real user queries. Instead, they are documentations such as comments or summaries written by developers. WebQuery- Test is a code question answering task that asks whether a Python code function has functionality described by the real user web query collected from the search logs of a commercial search engine. Recently, Huang et al. (2021) introduce CoSQA that consists of pairs of real user web queries and Python code snippets. Then, they use CoSQA as a benchmark for code search to resemble the …

Content truncated to 500 words