Barch: an English Dataset of Bar Chart Summaries

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3552–3560 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3552 Barch: an English Dataset of Bar Chart Summaries Iza ˇSkrjanec, Muhammad Salman Edhi, Vera Demberg Saarland University Saarbr¨ucken, Saarland, 66123, Germany {skrjanec, vera}@coli.uni-saarland.de, salman.edhi@gmail.com Abstract We present Barch, a new English dataset of human-written summaries describing bar charts. This dataset contains 47 charts based on a selection of 18 topics. Each chart is associated with one of the four intended messages expressed in the chart title. Using crowdsourcing, we collected around 20 summaries per chart, or one thousand in total. The text of the summaries is aligned with the chart data as well as with analytical inferences about the data drawn by humans. Our datasets is one of the first to explore the effect of intended messages on the data descriptions in chart summaries. Additionally, it lends itself well to the task of training data-driven systems for chart-to-text generation. We provide results on the performance of state-of-the-art neural generation models trained on this dataset and discuss the strengths and shortcomings of different models. Keywords: chart summary, crowdsourcing, natural language generation 1. Introduction Complex quantitative data is often visually presented in a chart and textually described in a summary in nat- ural language. The general communicative goal of the summary is to inform the readers about the content of the chart and help them draw inferences about the data. When writing a summary of a chart such as that in (a) of Table 1 several choices have to be made in terms of the order and content: whether to state the height of each bar or of a selection, describe a general trend or include numerical inferences about the data and so on. Table 1 shows a bar chart, its underlying data table and a summary, the latter providing basic information: the height of each bar. The final sentence provides an infer- ence comparing two bars, stating that one bar is nearly twice as high as the other one. This kind of information is analytical: it puts multiple data points in a relation (arithmetic operation) instead of simply describing sin- gle data points. Our data collection shows that human speakers frequently include such statements into chart summaries. However, many of existing datasets with chart and summaries do not consider such analytical information. In fact, recent years have shown a growing interest in developing datasets with pairs of charts and their sum- maries with two prominent applications. On the one hand, chart-summary datasets can be used to study the production of summaries in humans, and can provide a data basis for studies which compare different descrip- tions in terms of how well they meet users’ information needs. The chart design as well as the summary content can lead to variance in what inferences a comprehender draws from it. For example, Talbot et al. (2014) find that even the slightest changes in the chart design …

Content truncated to 500 words