PhotoshopQuiA: A Corpus of Non-Factoid Questions and Answers for Why-Question Answering

PhotoshopQuiA: A Corpus of Non-Factoid Questions and Answers for Why-Question Answering Andrei Dulceanu†∗, Thang Le Dinh‡, Walter Chang∗, Trung Bui∗, Doo Soon Kim∗, Manh Chien Vu‡, Seokhwan Kim∗ †Universitatea Politehnica Bucures,ti, Romˆania ∗ Adobe Research, California, United States ‡Universit´e du Qu´ebec `a Trois-Rivi`eres, Qu´ebec, Canada andrei.dulceanu@cs.pub.ro, {wachang, bui, dkim, seokim}@adobe.com, {thang.ledinh, manh.chien.vu}@uqtr.ca Abstract Recent years have witnessed a high interest in non-factoid question answering using Community Question Answering (CQA) web sites. Despite ongoing research using state-of-the-art methods, there is a scarcity of available datasets for this task. Why-questions, which play an important role in open-domain and domain-specific applications, are difficult to answer automatically since the answers need to be constructed based on different information extracted from multiple knowledge sources. We introduce the PhotoshopQuiA dataset, a new publicly available set of 2,854 why-question and answer(s) (WhyQ, A) pairs related to Adobe Photoshop usage collected from five CQA web sites. We chose Adobe Photoshop because it is a popular and well-known product, with a lively, knowledgeable and sizable community. To the best of our knowledge, this is the first English dataset for Why-QA that focuses on a product, as opposed to previous open-domain datasets. The corpus is stored in JSON format and contains detailed data about questions and questioners as well as answers and answerers. The dataset can be used to build Why-QA systems, to evaluate current approaches for answering why-questions, and to develop new models for future QA systems research. Keywords: question answering, community question answering, non-factoid question, Why-QA 1. Introduction The success of IBM’s Watson in the Jeopardy! TV game- show in 2011 and the significant investments of large tech companies in building personal assistants (e.g., Microsoft Cortana, Apple Siri, Amazon Alexa or Google Assistant) have strengthened the interest in the Question Answering field. These systems have in common the fact that they mostly tackle factoid questions. These are questions that “can be answered with simple facts expressed in short text answers”; usually, their answers include “short strings expressing a personal name, temporal expression, or location” (Jurafsky and Martin, 2017). An example of a factoid question and its answer is: Q: Who is Canada’s prime minister? A: Justin Trudeau. By contrast, non-factoid questions ask for “opinions, suggestions, interpretations and the like”(Tomasoni and Huang, 2010). Answering and evaluating the quality of the provided answers for non-factoid questions have proved to be non-trivial due to the difficulty of the task complexity as well as the lack of training data. To address the latter issue, numerous researchers have tried to take advantage of user-generated content on Community Question Answer- ing (CQA) web sites such as Yahoo! Answers 1, Stack Overflow 2 or Quora 3. These web forums allow users to post their own questions, answer others’ questions, com- ment on others’ replies, and upvote or downvote answers. 1https://answers.yahoo.com 2https://stackoverflow.com 3https://www.quora.com Usually, if a user is the original questioner, he/she is al- lowed to select the most relevant answer to his/her question. Although CQA web sites have lots of experts, it still takes their …

Content truncated to 500 words