Addressing Leakage in Self-Supervised Contextualized Code Retrieval

Proceedings of the 29th International Conference on Computational Linguistics, pages 1006–1013 October 12–17, 2022. 1006 Addressing Leakage in Self-Supervised Contextualized Code Retrieval Johannes Villmow and Viola Campos and Adrian Ulges and Ulrich Schwanecke RheinMain University of Applied Sciences Wiesbaden, Germany firstname.lastname@hs-rm.de Abstract We address contextualized code retrieval, the search for code snippets, helpful to fill gaps in a partial input program. Our approach facilitates a large-scale self-supervised contrastive train- ing by splitting source code randomly into con- texts and targets. To combat leakage between the two, we suggest a novel approach based on mutual identifier masking, dedentation, and the selection of syntax-aligned targets. Our second contribution is a new dataset for direct evalu- ation of contextualized code retrieval, based on a dataset of manually aligned subpassages of code clones. Our experiments demonstrate that the proposed approach improves retrieval substantially, and yields new state-of-the-art results for code clone and defect detection. 1 Introduction AI-supported software development has recently experienced growing interest (Lu et al., 2021), ad- dressing various code understanding tasks such as code auto-completion (Svyatkovskiy et al., 2020), natural language code search (Husain et al., 2019), and code clone detection (Svajlenko and Roy, 2015). Our focus is on a related task called con- textualized code search (Mukherjee et al., 2020; Dahal et al., 2022): Given an incomplete piece of code and a certain position of interest (e.g., the cur- rent cursor position), a retriever searches for code fragments that are relevant for filling in the missing piece. This setting aligns well with programmers’ workflow, and differs substantially from the three tasks mentioned above as follows: (1) In contrast to natural language code search, contextualized code search can exploit local code context. (2) While code generated by autocompletion systems such as GitHub’s CodEx (Chen et al., 2021) is prone to subtle programming errors, contextualized search leaves the developer in charge, and the origin of a solution remains transparent. (3) In contrast to clone detection, contextualized code search is not targeted at semantically similar code but code that complements the query. A key challenge with contextualized code search is that supervised labels for relevant code pas- sages are missing. Therefore, we bootstrap a self- supervised learning process by drawing inspira- tion from Cloze Tasks in natural language process- ing (Lee et al., 2019): Given a large-scale dataset containing pieces of code in 16 programming lan- guages, we erase random blocks. We refer to these blocks as targets, and to their surrounding as con- texts. Together, these pairs form samples for con- trastive learning. Unfortunately, as Figure 1 shows, this approach suffers from leakage between context and target, as the two share (1) common identifiers, (2) a match- ing indentation level, and (3) in some languages – if dividing a syntactic primitive such as for-loops – matching brackets. Retrievers might exploit these effects and bypass semantic similarity. To this end, our first contribution is a novel approach towards self-supervised code retrieval, which avoids the above bias through de-leaking steps such as mutual identifier …

Content truncated to 500 words