LiveQA: A Question Answering Dataset over Sports Live

LiveQA: A Question Answering Dataset over Sports Live Qianying Liu∗12, Sicong Jiang∗1, Yizhong Wang13 and Sujian Li1 1 Key Laboratory of Computational Linguistics, MOE, Peking University 2 Graduate School of Informatics, Kyoto University 3 University of Washington ying@nlp.ist.i.kyoto-u.ac.jp; 512580728@qq.com; yizhongw@cs.washington.edu; lisujian@pku.edu.cn Abstract In this paper, we introduce LiveQA, a new question answering dataset constructed from play- by-play live broadcast. It contains 117k multiple-choice questions written by human commen- tators for over 1,670 NBA games, which are collected from the Chinese Hupu1 website. De- rived from the characteristics of sports games, LiveQA can potentially test the reasoning ability across timeline-based live broadcasts, which is challenging compared to the existing datasets. In LiveQA, the questions require understanding the timeline, tracking events or doing mathemati- cal computations. Our preliminary experiments show that the dataset introduces a challenging problem for question answering models, and a strong baseline model only achieves the accuracy of 53.1% and cannot beat the dominant option rule. We release the code and data of this paper for future research.23 1 Introduction The research of question answering (QA), where a system needs to understand a piece of reading material and answer corresponding questions, has drawn considerable attention in recent years. While various QA datasets have been constructed to study how a QA system can understand a specific passage, the common sense knowledge and so on (Rajpurkar et al., 2016; Lai et al., 2017; Dunn et al., 2017; Rajpurkar et al., 2018), most questions in these datasets could be given their answers by extracting from a few relevant sentences so that the model only needs to find a small set of supporting evidences,whose temporal order- ing does not effect the final answer. In other words, these questions are raised only considering a fixed document. However, in the real-life question answering, a question could have its timelines. To infer the answer, a good model needs to understand series of timeline information. For example, the question “how many points did Lebron James have?” would have different answers based on the time when the question was asked during a basketball game, and the answer would continuously change during the game. The other question “Which team would first earn 10 points?” would require a system to track down information of scoring points along the timeline until one team achieves 10 points. According to the analysis above, we consider the timeline-based question answering problem as a gap which has not been covered by existing datasets. Thus, in this work we hope to construct a dataset where passages and questions both have timelines and question respondents are required to judge what information should be gathered for the questions involved in a timeline. Such a timeline inference- involved QA dataset introduces a new research line of reading comprehension, that evaluates the ability of understanding temporal information of a QA model. Additionally, the real-world questions are often involved in some math calculation, such as addition, subtraction and counting. To answer the questions correctly, one not only needs to locate some …

Content truncated to 500 words