A Study of Latent Structured Prediction Approaches to Passage Reranking

Proceedings of NAACL-HLT 2019, pages 1847–1857 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1847 A Study of Latent Structured Prediction Approaches to Passage Reranking Iryna Haponchyk DISI, University of Trento 38123 Povo (TN), Italy iryna.haponchyk@unitn.it Alessandro Moschitti∗ Amazon Manhattan Beach, CA, USA amosch@amazon.com Abstract The structured output framework provides a helpful tool for learning to rank problems. In this paper, we propose a structured out- put approach which regards rankings as latent variables. Our approach addresses the com- plex optimization of Mean Average Precision (MAP) ranking metric. We provide an infer- ence procedure to find the max-violating rank- ing based on the decomposition of the corre- sponding loss. The results of our experiments on WikiQA and TREC13 datasets show that our reranking based on structured prediction is a promising research direction. 1 Introduction The current state-of-the-art learning approaches for answer sentence reranking in question answer- ing (QA) are mostly based on learning pairwise ranking signals or simple binary classification (rel- evant versus irrelevant labels). Intuitively, global information over a rank should improve the ranker accuracy. Thus, there have been promising at- tempts to learn global ranking functions which en- compass the signals of all the candidates for a given query (Chapelle et al., 2007; Weston and Blitzer, 2012; Le et al., 2018). These works em- ploy the structured output learning framework to represent a ranking as a structured object, with re- spect to which it is possible to directly optimize a ranking measure. Direct optimization of the target ranking mea- sures is affordable when they are factorizable, e.g., the structural SVM of Chapelle et al. (2007) makes use of the factorization properties of the Normal- ized Discounted Cumulative Gain (NDCG) rank- ing score. In contrast, MAP is rather complex making its treatment harder. Yue et al. (2007) ∗Most of this work was carried out before joining Ama- zon. could still find an exact solution to the hinge-loss relaxation of Average Precision (AP) for the struc- tural SVM approach. It is found for the partic- ular case of a combined feature mapping of in- puts and structured outputs. Such a mapping ac- counts for respective orderings of the pairs of can- didate items, where one item is relevant and the other is not, without explicitly encoding the or- der of all the items in the rank. Encoding such order (Chapelle et al., 2007; Weston and Blitzer, 2012), i.e., adding yet more complexity to the structural feature space, might lead to intractabil- ity of the previous exact max-violating inference with respect to MAP. Furthermore, the feature rep- resentation of the gold standard rankings, which could be many for a given candidate list of items, is not unique anymore. In this work, we study the effect of using structured ranking representations (Chapelle et al., 2007) within the large-margin structured predic- tion framework versus direct MAP optimization on the most representative task of QA, i.e., pas- sage reranking. To make two ends meet, we have to tackle the …

Content truncated to 500 words