StreamHover: Livestream Transcript Summarization and Annotation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6457–6474 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6457 StreamHover: Livestream Transcript Summarization and Annotation Sangwoo Cho,† Franck Dernoncourt,‡ Tim Ganter,‡ Trung Bui,‡ Nedim Lipka,‡ Walter Chang,‡ Hailin Jin,‡ Jonathan Brandt,‡ Hassan Foroosh,† Fei Liu† †University of Central Florida ‡Adobe Research swcho@knights.ucf.edu {foroosh,feiliu}@cs.ucf.edu {dernonco,timganter,bui,lipka,wachang,hljin,jbrandt}@adobe.com Abstract With the explosive growth of livestream broad- casting, there is an urgent need for new sum- marization technology that enables us to create a preview of streamed content and tap into this wealth of knowledge. However, the problem is nontrivial due to the informal nature of spoken language. Further, there has been a shortage of annotated datasets that are necessary for tran- script summarization. In this paper, we present StreamHover, a framework for annotating and summarizing livestream transcripts. With a to- tal of over 500 hours of videos annotated with both extractive and abstractive summaries, our benchmark dataset is significantly larger than currently existing annotated corpora. We ex- plore a neural extractive summarization model that leverages vector-quantized variational au- toencoder to learn latent vector representations of spoken utterances and identify salient utter- ances from the transcripts to form summaries. We show that our model generalizes better and improves performance over strong baselines. The results of this study provide an avenue for future research to improve summarization so- lutions for efficient browsing of livestreams. 1 Introduction One of the most powerful communication mediums is livestreaming. New platforms such as YouTube Live, Twitch, Instagram Live and TikTok encom- pass a variety of topics, ranging from video games to social media to professional sports. We are par- ticularly interested in livestreams that are distin- guished by three characteristics: Excessive length, the recordings could last from several minutes to several hours; Verbal communication, the use of natural language is the primary means of communi- cation, in contrast to gestures or facial expressions; Informal nature, the streamers’ language is mostly informal and unplanned, unlike news broadcasts. Without an effective mechanism to summarize such streamed content, livestreaming platforms may not fully meet the needs of their customers. Figure 1: An example of streamed content on B¯ehance, a streaming platform for artists and designers to show- case creative work related to Adobe Photoshop, Illustra- tor, Fresco, UI/UX, photography and more. TOP: The videos are each 27 minutes long. BOTTOM: One video is being broadcast live, the other is >2 hours long. Our goal in this work is to create a text preview of the streamed content. When a user hovers over the thumbnail or scrolls past a video, they are shown a preview of the content. We present a dataset of over 500 hours of video footage, which were streamed live on a social media platform (behance.net) cre- ated to showcase and discover creative work. Fig- ure 1 shows an example of the streams, where the artists showcase the use of Adobe Photoshop and Illustrator in designing holiday cards and posters. It is necessary to point out that …

Content truncated to 500 words