PACRR: A Position-Aware Neural IR Model for Relevance Matching

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1049–1058 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics PACRR: A Position-Aware Neural IR Model for Relevance Matching Kai Hui MPI for Informatics Saarbr¨ucken Graduate School of Computer Science Andrew Yates MPI for Informatics Klaus Berberich htw saar MPI for Informatics Gerard de Melo Rutgers University New Brunswick Abstract In order to adopt deep learning for infor- mation retrieval, models are needed that can capture all relevant information re- quired to assess the relevance of a docu- ment to a given user query. While pre- vious works have successfully captured unigram term matches, how to fully em- ploy position-dependent information such as proximity and term dependencies has been insufficiently explored. In this work, we propose a novel neural IR model named PACRR aiming at better modeling position-dependent interactions between a query and a document. Extensive experi- ments on six years’ TREC Web Track data confirm that the proposed model yields better results under multiple benchmarks. 1 Introduction Despite the widespread use of deep neural mod- els across a range of linguistic tasks, to what extent such models can improve information re- trieval (IR) and which components a deep neural model for IR should include remain open ques- tions. In ad-hoc IR, the goal is to produce a rank- ing of relevant documents given an open-domain (“ad hoc”) query and a document collection. A ranking model thus aims at evaluating the inter- actions between different documents and a query, assigning higher scores to documents that better match the query. Learning to rank models, like the recent IRGAN model (Wang et al., 2017), rely on handcrafted features to encode query docu- ment interactions, e.g., the relevance scores from unsupervised ranking models. Neural IR mod- els differ in that they extract interactions directly based on the queries and documents. Many early neural IR models can be categorized as seman- tic matching models, as they embed both queries and documents into a low-dimensional space, and then assess their similarity based on such dense representations. Examples in this regard include DSSM (Huang et al., 2013) and DESM (Mitra et al., 2016). The notion of relevance is inher- ently asymmetric, however, making it different from well-studied semantic matching tasks such as semantic relatedness and paraphrase detection. In- stead, relevance matching models such as Match- Pyramid (Pang et al., 2016), DRMM (Guo et al., 2016) and the recent K-NRM (Xiong et al., 2017) resemble traditional IR retrieval measures in that they directly consider the relevance of documents’ contents with respect to the query. The DUET model (Mitra et al., 2017) is a hybrid approach that combines signals from a local model for rel- evance matching and a distributed model for se- mantic matching. The two classes of models are fairly distinct. In this work, we focus on relevance matching models. Given that relevance matching approaches mir- ror ideas from traditional retrieval models, the decades of research on ad-hoc IR can guide us …

Content truncated to 500 words