ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10893 - 10906 December 7-11, 2022 ©2022 Association for Computational Linguistics ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts Rajdeep Mukherjee1∗ Abhinav Bohra1 Akash Banerjee1 Soumya Sharma1 Manjunath Hegde2 Afreen Shaikh2 Shivani Shrivastava2 Koustuv Dasgupta2 Niloy Ganguly1,3 Saptarshi Ghosh1 Pawan Goyal1 1 Department of Computer Science and Engineering, IIT Kharagpur, India 2 Goldman Sachs Data Science and Machine Learning Group, India 3 Leibniz University of Hannover, Germany Abstract Despite tremendous progress in automatic sum- marization, state-of-the-art methods are pre- dominantly trained to excel in summarizing short newswire articles, or documents with strong layout biases such as scientific articles or government reports. Efficient techniques to summarize financial documents, discussing facts and figures, have largely been unexplored, majorly due to the unavailability of suitable datasets. In this work, we present ECTSum, a new dataset with transcripts of earnings calls (ECTs), hosted by publicly traded compa- nies, as documents, and experts-written short telegram-style bullet point summaries derived from corresponding Reuters articles. ECTs are long unstructured documents without any pre- scribed length limit or format. We benchmark our dataset with state-of-the-art summarization methods across various metrics evaluating the content quality and factual consistency of the generated summaries. Finally, we present a simple yet effective approach, ECT-BPS, to generate a set of bullet points that precisely cap- ture the important facts discussed in the calls. 1 Introduction Earnings Calls, typically a teleconference or a we- bcast, are hosted by publicly traded companies to discuss important aspects of their quarterly (10- Q), or annual (10-K) earnings reports, along with current trends and future goals that help financial analysts and investors to review their price targets and trade decisions (Givoly and Lakonishok, 1980; Richard Frankel and Skinner, 1999; Bowen et al., 2002; Keith and Stent, 2019). The corresponding call transcripts (called Earnings Call Transcripts, abbreviated as ECTs) are typically in the form of long unstructured documents consisting of thou- sands of words. Hence, it requires a great deal of time and effort, even on the part of trained analysts, to quickly summarize the key facts covered in these ∗Corresponding author: rajdeep1989@iitkgp.ac.in • QUARTERLY EARNINGS PER SHARE $1.52. • QUARTERLY TOTAL NET SALES $97.28 BILLION VERSUS $89.58 BILLION REPORTED LAST YEAR. • BOARD OF DIRECTORS AUTHORIZED AN IN- CREASE OF $90 BILLION TO THE EXISTING SHARE REPURCHASE PROGRAM. • QUARTERLY IPHONE REVENUE $50.57 BILLION VERSUS $47.94 BILLION REPORTED LAST YEAR. Table 1: ECTSum: Excerpt from the Reuters article1 corresponding to the ECT2 for Apple Q2 2022. transcripts. Given the importance of these calls, they are often summarized by media houses such as Reuters and BusinessWire. The scale of such effort, however, calls for the development of effi- cient methods to automate this task which in turn necessitates the creation of a benchmark dataset. Towards this goal, we present ECTSum, a new benchmark dataset for bullet-point summarization of long ECTs. As discussed in Section 3.2, first we crawled around 7.4K …

Content truncated to 500 words