Original Content Is All You Need! an Empirical Study on Leveraging Answer Summary for WikiHowQA Answer Selection Task

Proceedings of the 29th International Conference on Computational Linguistics, pages 1546–1555 October 12–17, 2022. 1546 Original Content Is All You Need! An Empirical Study on Leveraging Answer Summary for WikiHowQA Answer Selection Task Liang Wen1,2 Juan Li4 Houfeng Wang1 Yingwei Luo1,2 Xiaolin Wang1,2 Xiaodong Zhang3 Zhicong Cheng3 Dawei Yin3 1School of Computer Science, Peking University, China 2Peng Cheng Laboratory, Shenzhen, China 3Baidu Inc., Beijing, China 4School of Chinese Language and Literature, Beijing Normal University, China {yuco,wanghf,lyw,wxl}@pku.edu.cn {lijuan}@mail.bnu.edu.cn {zhangxiaodong11,chengzhicong01}@baidu.com {yindawei}@acm.org Abstract Answer selection task requires finding appro- priate answers to questions from informative but crowdsourced candidates. A key factor im- peding its solution by current answer selection approaches is the redundancy and lengthiness issues of crowdsourced answers. Recently, Deng et al. (2020) constructed a new dataset, WikiHowQA, which contains a corresponding reference summary for each original lengthy answer. And their experiments show that lever- aging the answer summaries helps to attend the essential information in original lengthy an- swers and improve the answer selection perfor- mance under certain circumstances. However, when given a question and a set of long candi- date answers, human beings could effortlessly identify the correct answer without the aid of additional answer summaries since the origi- nal answers contain all the information volume that answer summaries contain. In addition, pretrained language models have been shown superior or comparable to human beings on many natural language processing tasks. Mo- tivated by those, we design a series of neu- ral models, either pretraining-based or non- pretraining-based, to check wether the addi- tional answer summaries are helpful for rank- ing the relevancy degrees of question-answer pairs on WikiHowQA dataset. Extensive au- tomated experiments and hand analysis show that the additional answer summaries are not useful for achieving the best performance. 1 Introduction Answer selection task in community question an- swering (cQA) has been a popular research topic in both academy and industry due to its practical importance. In recent years, neural attention-based approaches for this task can be roughly catego- rized into two primary types. One type of them (Han et al., 2019; Rücklé et al., 2019) attempts to enhance the interactions of different granular- ity between question and candidate answer using the widely-adopted compare-aggregate framework (Wang and Jiang, 2017). The another focuses on incorporating additional input information, such as user metadata information (Xie et al., 2020), the subject-body relationship of community ques- tions (Wu et al., 2018), etc. However, real-life cQA datasets that contain open-domain and non- factoid questions usually come along with long multi-sentence answer texts and noise. As a result, many previous neural answer selection approaches that were primarily designed to retrieve short an- swers fall short of expectations in such cases (Co- hen et al., 2018; Rücklé et al., 2019). Recently, Deng et al. (2020) propose to leverage answer summaries to tackle the redundancy and lengthiness issues of original answers in long an- swer selection task. To this end, they created Wik- iHowQA, the first large-scale open-domain cQA dataset that contains lengthy answers coupled with its …

Content truncated to 500 words