Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5878–5882, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5878 Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering Zhiguo Wang, Patrick Ng, Xiaofei Ma, Ramesh Nallapati, Bing Xiang AWS AI Labs {zhiguow, patricng, xiaofeim, rnallapa, bxiang}@amazon.com Abstract BERT model has been successfully applied to open-domain QA tasks. However, previous work trains BERT by viewing passages cor- responding to the same question as indepen- dent training instances, which may cause in- comparable scores for answers from different passages. To tackle this issue, we propose a multi-passage BERT model to globally nor- malize answer scores across all passages of the same question, and this change enables our QA model find better answers by utiliz- ing more passages. In addition, we find that splitting articles into passages with the length of 100 words by sliding window improves per- formance by 4%. By leveraging a passage ranker to select high-quality passages, multi- passage BERT gains additional 2%. Exper- iments on four standard benchmarks showed that our multi-passage BERT outperforms all state-of-the-art models on all benchmarks. In particular, on the OpenSQuAD dataset, our model gains 21.4% EM and 21.5% F1 over all non-BERT models, and 5.8% EM and 6.5% F1 over BERT-based models. 1 Introduction BERT model (Devlin et al., 2018) has achieved significant improvements on a variety of NLP tasks. For question answering (QA), it has domi- nated the leaderboards of several machine reading comprehension (RC) datasets. However, the RC task is only a simplified version of the QA task, where a model only needs to find an answer from a given passage/paragraph. Whereas, in reality, an open-domain QA system is required to pinpoint answers from a massive article collection, such as Wikipedia or the entire web. Recent studies directly applied the BERT-RC model to open-domain QA (Yang et al., 2019; Nogueira et al., 2018; Alberti et al., 2019). They firstly leverage a passage retriever to retrieve mul- tiple passages for each question. During training, passages corresponding to the same question are taken as independent training instances. During inference, the BERT-RC model is applied to each passage individually to predict an answer span, and then the highest scoring span is selected as the final answer. Although this method achieves sig- nificant improvements on several datasets, there are still several unaddressed issues. First, view- ing passages of the same question as indepen- dent training instances may result in incompara- ble answer scores across passages. Thus, globally normalizing scores over all passages of the same question (Clark and Gardner, 2018) may be help- ful. Second, previous work defines passages as articles, paragraphs, or sentences. However, the question of proper granularity of passages is still underexplored. Third, passage ranker for select- ing high-quality passages has been shown to be very useful in previous open-domain QA systems (Wang et al., 2018a; Lin et al., 2018; Pang et al., 2019). …

Content truncated to 500 words