Incorporating Semantic Textual Similarity and Lexical Matching for Information Retrieval

Incorporating Semantic Textual Similarity and Lexical Matching for Information Retrieval Hiroki Iida and Naoaki Okazaki Department of Computer Science, School of Computing, Tokyo Institute of Technology 2-12-1 Ookayama, Meguro-ku, Tokyo, Japan {hiroki.iida@nlp.c., okazaki@c.}titech.ac.jp Abstract BERT, which has been successfully applied to many types of natural language process- ing (NLP) tasks, is also effective with vari- ous information retrieval (IR) tasks. However, it is not easy to obtain appropriate data for fine-tuning a BERT model. This paper pro- poses a method that can improve IR perfor- mance without fine-tuning the model on the target IR data. Focusing on words appearing in both a query and a document, we intro- duce local-similarity (LS). LS calculates the similarity of contextualized representations of the common words, encoded using a pre- trained model for the semantic textual simi- larity task. To incorporate the LS into the IR scoring, we propose local-similarity scoring (LSS) functions. Experimental results show that LSS outperforms BM25 on several rep- resentative benchmarks. We also demonstrate that LSS reflects improving the pre-trained model of LS to the higher IR performance. Our code is available at https://github. com/nlp-titech/rerank_by_sts. 1 Introduction Semantic matching and relevance matching are closely related tasks with different goals. Mainly studied in natural language processing (NLP), se- mantic matching infers the similarity/dissimilarity of two pieces of text, which are usually of the same length. By contrast, relevance matching aims at ranking documents for a given query in information retrieval (IR), where a query is mostly much shorter than documents (Guo et al., 2016). The recent advance of deep learning has achieved success in semantic matching. In addition, the advent of BERT (Devlin et al., 2019) allowed us to solve various semantic matching tasks, for ex- ample, paraphrasing and similarity, with the same model. BERT has also been successfully applied to IR tasks (Yang et al., 2019; Nogueira and Cho, 2019). However, it is not easy to obtain appropri- ate data for fine-tuning. Nogueira and Cho (2019) reported that BERT needs about 10,000 data to out- perform BM25 (Robertson and Walker, 1994): a tra- ditional scoring function based on lexical matching. In this paper, to alleviate the difficulty, we study a method improving the performance without fine- tuning a model on the target IR data. The method incorporates semantic textual similarity (STS) based on words appearing in both a query and a document based on Guo et al. (2016), who reported that match- ing such words is one of the important factors for rel- evance matching. Concretely, we propose the local- similarity (LS) method. In essence, LS considers contextualized representations of tokens appearing in the query tokens rather than those of all tokens in a retrieved document. Incorporating LS into scoring functions, we present local-similarity scoring (LSS) functions. We confirm the effectiveness of the LSS func- tions on three well-known IR benchmarks: MS MARCO Passage, MS MARCO Document (Bajaj et al., 2016), and Robust04 (Voorhees, 2004). A pre- trained model for LSS is based on Sentence-BERT (SBERT) (Reimers and Gurevych, 2019). …

Content truncated to 500 words