BehanceQA: A New Dataset for Identifying Question-Answer Pairs in Video Transcripts

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7321–7327 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7321 BehanceQA: A New Dataset for Identifying Question-Answer Pairs in Video Transcripts Amir Pouran Ben Veyseh1, Viet Dac Lai1, Franck Dernoncourt2, Thien Huu Nguyen1 1Dept. of Computer and Information Science, University of Oregon, Eugene, OR, USA 2Adobe Research, Seattle, WA, USA {vietl,apouranb,thien}@cs.uoregon.edu, franck.dernoncourt@adobe.com Abstract Question-Answer (QA) is one of the effective methods for storing knowledge which can be used for future retrieval. As such, identifying mentions of questions and their answers in text is necessary for a knowledge construction and retrieval systems. In the literature, QA identification has been well studied in the NLP community. However, most of the prior works are restricted to formal written documents such as papers or websites. As such, Questions and Answers that are presented in informal/noisy documents have not been adequately studied. One of the domains that can significantly benefit from QA identification is the domain of livestreaming video transcripts that involve abundant QA pairs to provide valuable knowledge for future users and services. Since video transcripts are often transcribed automatically for scale, they are prone to errors. Combined with the informal nature of discussion in a video, prior QA identification systems might not be able to perform well in this domain. To enable comprehensive research in this domain, we present a large-scale QA identification dataset annotated by human over transcripts of 500 hours of streamed videos. We employ Behance.net to collect the videos and their automatically obtained transcripts. Furthermore, we conduct extensive analysis on the annotated dataset to understand the complexity of QA identification for livestreaming video transcripts. Our experiments show that the annotated dataset presents unique challenges for existing methods and more research is necessary to explore more effective methods. The dataset and the models developed in this work will be publicly released for future research. Dataset is available from https://github.com/amirveyseh/BehanceQA/tree/main. Keywords: Question Answering, Video Transcripts 1. Introduction One of the efficient methods to acquire knowledge about a new topic is via Questions. Due to their appli- cability, preparing high quality Question-Answer (QA) pairs has been used as an effective method for stor- ing the knowledge to be used in future retrieval. To this end, identifying QA pairs from raw text is a neces- sary pre-processing step. This task is called Question- Answer Identification (QAI) that aims to find all im- portant questions along with their answers provided in text. For instance, in the text excerpt “Now, we will explore the rotation techniques. How can you rotate the image by certain degrees? For that you can use CTR+R shortcut key.”, a QAI system needs to recog- nize the question “How can you rotate the image by certain degrees?” along with its answer “For that you can use CTR+R shortcut key.”. Note that questions and answers (if available) should be matched with each other in QAI. A QAI system can be used for construct- ing a …

Content truncated to 500 words