PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 2169–2181 August 1–6, 2021. ©2021 Association for Computational Linguistics 2169 PLOTCODER: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context Xinyun Chen, Linyuan Gong, Alvin Cheung and Dawn Song UC Berkeley {xinyun.chen, gly, akcheung, dawnsong}@berkeley.edu Abstract Creating effective visualization is an impor- tant part of data analytics. While there are many libraries for creating visualizations, writ- ing such code remains difficult given the myr- iad of parameters that users need to provide. In this paper, we propose the new task of syn- thesizing visualization programs from a com- bination of natural language utterances and code context. To tackle the learning prob- lem, we introduce PLOTCODER, a new hierar- chical encoder-decoder architecture that mod- els both the code context and the input ut- terance. We use PLOTCODER to first deter- mine the template of the visualization code, followed by predicting the data to be plotted. We use Jupyter notebooks containing visual- ization programs crawled from GitHub to train PLOTCODER. On a comprehensive set of test samples from those notebooks, we show that PLOTCODER correctly predicts the plot type of about 70% samples, and synthesizes the cor- rect programs for 35% samples, performing 3- 4.5% better than the baselines.1 1 Introduction Visualizations play a crucial role in obtaining insights from data. While a number of li- braries (Hunter, 2007; Seaborn, 2020; Bostock et al., 2011) have been developed for creating vi- sualizations that range from simple scatter plots to complex 3D bar charts, writing visualization code remains a difficult task. For instance, drawing a scatter plot using the Python matplotlib library can be done using both the scatter and plot methods, and the scatter method (Matplotlib, 2020) takes in 2 required parameters (the values to plot) along with 11 other optional parameters (marker type, color, etc), with some parameters having numeric types (e.g., the size of each marker) and some being 1Our code and data are available at https://github. com/jungyhuk/plotcoder. arrays (e.g., the list of colors for each collection of the plotted data, where each color is specified as a string or another array of RGB values). Looking up each parameter’s meaning and its valid values re- mains tedious and error-prone, and the multitude of libraries available further compounds the difficulty for developers to create effective visualizations. In this paper, we propose to automatically syn- thesize visualization programs using a combina- tion of natural language utterances and the pro- grammatic context that the visualization program will reside (e.g., code written in the same file as the visualization program to load the plotted data), focusing on programs that create static visualiza- tions (e.g., line charts, scatter plots, etc). While there has been prior work on synthesizing code from natural language (Zettlemoyer and Collins, 2012; Oda et al., 2015; Wang et al., 2015; Yin et al., 2018), and with addition information such as database schemas (Zhong et al., 2017; Yu et …

Content truncated to 500 words