Neural Generation of Regular Expressions from Natural Language with Minimal Domain Knowledge

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1918–1923, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Neural Generation of Regular Expressions from Natural Language with Minimal Domain Knowledge Nicholas Locascio CSAIL, MIT njl@mit.edu Karthik Narasimhan CSAIL, MIT karthikn@mit.edu Eduardo DeLeon CSAIL, MIT edeleon4@mit.edu Nate Kushman Microsoft nate@kushman.org Regina Barzilay CSAIL, MIT regina@csail.mit.edu Abstract This paper explores the task of translating nat- ural language queries into regular expressions which embody their meaning. In contrast to prior work, the proposed neural model does not utilize domain-specific crafting, learning to translate directly from a parallel corpus. To fully explore the potential of neural mod- els, we propose a methodology for collecting a large corpus1 of regular expression, natural language pairs. Our resulting model achieves a performance gain of 19.6% over previous state-of-the-art models. 1 Introduction This paper explores the task of translating natu- ral language text queries into regular expressions which embody their meaning. Regular expressions are built into many application interfaces, yet most users of these applications have difficulty writing them (Friedl, 2002). Thus a system for automat- ically generating regular expressions from natural language would be useful in many contexts. Fur- thermore, such technologies can ultimately scale to translate into other formal representations, such as program scripts (Raza et al., 2015). Prior work has demonstrated the feasibility of this task. Kushman and Barzilay (2013) proposed a model that learns to perform the task from a parallel corpus of regular expressions and the text descrip- tions. To account for the given representational dis- parity between formal regular expressions and natu- ral language, their model utilizes a domain specific 1The corpus and code used in this paper is available at https: //github.com/nicholaslocascio/deep-regex component which computes the semantic equiva- lence between two regular expressions. Since their model relies heavily on this component, it cannot be readily applied to other formal representations where such semantic equivalence calculations are not possible. In this paper, we reexamine the need for such spe- cialized domain knowledge for this task. Given the same parallel corpus used in Kushman and Barzi- lay (2013), we use an LSTM-based sequence to se- quence neural network to perform the mapping. Our model does not utilize semantic equivalence in any form, or make any other special assumptions about the formalism. Despite this and the relatively small size of the original dataset (824 examples), our neu- ral model exhibits a small 0.1% boost in accuracy. To further explore the power of neural networks, we created a much larger public dataset, NL-RX. Since creation of regular expressions requires spe- cialized knowledge, standard crowd-sourcing meth- ods are not applicable here. Instead, we employ a two-step generate-and-paraphrase procedure that circumvents this problem. During the generate step, we use a small but expansive manually-crafted grammar that translates regular expression into nat- ural language. In the paraphrase step, we rely on crowd-sourcing to paraphrase these rigid descrip- tions into more natural and fluid descriptions. Using this methodology, we have constructed a corpus …

Content truncated to 500 words