PaRe: A Paper-Reviewer Matching Approach Using a Common Topic Space

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 518–528, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 518 PaRe: A Paper-Reviewer Matching Approach Using a Common Topic Space Omer Anjum∗,1, Hongyu Gong∗,1, Suma Bhat1, Jinjun Xiong2, Wen-Mei Hwu1 1University of Illinois at Urbana-Champaign, USA 2 IBM Thomas J. Watson Research Center, USA {oanjum, hgong6, spbhat2, w-hwu}@illinois.edu, jinjun@us.ibm.com Abstract Finding the right reviewers to assess the quality of conference submissions is a time consuming process for conference organiz- ers. Given the importance of this step, var- ious automated reviewer-paper matching so- lutions have been proposed to alleviate the burden. Prior approaches, including bag-of- words models and probabilistic topic mod- els have been inadequate to deal with the vocabulary mismatch and partial topic over- lap between a paper submission and the re- viewer’s expertise. Our approach, the com- mon topic model, jointly models the topics common to the submission and the reviewer’s profile while relying on abstract topic vectors. Experiments and insightful evaluations on two datasets demonstrate that the proposed method achieves consistent improvements compared to available state-of-the-art implementations of paper-reviewer matching. 1 Introduction The peer review mechanism constitutes the bedrock of today’s academic research landscape spanning submissions to conferences, journals, and funding bodies across numerous disciplines. Matching a paper (or a proposal) to an expert in the topic presented in the paper requires the knowl- edge of diverse topics of both the submission as well as that of the reviewer’s expertise in addition to knowing recent affiliations and co-authorship to resolve conflict of interest. Considering the scale of current conference submissions, performing the task of paper-reviewer matching manually incurs significant overheads to the program committee (PC) and calls for automating the process. Faced with record number of paper submissions essen- tially interdisciplinary in nature, the inadequacy *Omer Anjum and Hongyu Gong have equal contribu- tion. of available reviewer matching systems to scale to the current needs is being expressed by many conference program committees. It is also no- table that the approaches to address the challenges seem ad-hoc and non-scalable, as described in a few of the PC blogs; “Looking at the abstracts for many of the submissions it also quickly became clear that there was disparity in how authors chose topic keywords for submissions with many only using a single keyword and others using over half a dozen keywords. As such relying on the key- words for submissions became difficult. The com- bined effect of these problems made any automatic or semi-automatic assignment using HotCRP sub- optimal...So, we chose to hand assign the papers.” (Falsafi et al., 2018), and again in “Our plan was to rely on the Toronto Paper Matching System (TPMS) in allocating papers to reviewers. Unfor- tunately, this system didnt prove as useful as we had hoped for (it requires more extensive reviewer profiles for optimal performance than what we had available) and the work had to rely largely …

Content truncated to 500 words