Leveraging Structured Metadata for Improving Question Answering on the Web

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 551–556 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 551 Leveraging Structured Metadata for Improving Question Answering on the Web Xinya Du1∗ Adam Fourney2 Robert Sim2 Claire Cardie1 Paul N. Bennett2 Ahmed Hassan Awadallah2 1Department of Computer Science, Cornell University, Ithaca, NY {xdu, cardie}@cs.cornell.edu 2Microsoft Research, Redmond, WA {adamfo, rsim, paul.n.bennett, hassanam}@microsoft.com Abstract We show that leveraging metadata information from web pages can improve the performance of models for answer passage selection/re- ranking. We propose a neural passage se- lection model that leverages metadata infor- mation with a fine-grained encoding strategy, which learns the representation for metadata predicates in a hierarchical way. The models are evaluated on the MS MARCO (Nguyen et al., 2016) and Recipe-MARCO datasets. Re- sults show that our models significantly out- perform baseline models, which do not incor- porate metadata. We also show that the fine- grained encoding’s advantage over other strate- gies for encoding the metadata. 1 Introduction Question answering (QA) is a long-standing task in NLP and IR. Having QA systems that perform well on real-world questions is of significant value for search engines and intelligent assistants. While some of the earliest work tackled the task of answer- ing questions based on a large corpus (Voorhees and Tice, 2000; Voorhees, 2003; Wang et al., 2007) (albeit mostly focusing on simple fact-oriented questions), much of the recent work on QA has focused on answering questions in a less realis- tic setting – drawing the answer from a paragraph of text (Rajpurkar et al., 2016; Joshi et al., 2017), which is commonly referred to as machine reading comprehension (MRC). In this work, we tackle the more realistic prob- lem — candidate answers passages selection/re- ranking for real-world questions on the web. In contrast to both MRC and early work on QA from a large corpus, web pages often provide an additional source of knowledge. In particular, and thanks in part to the Semantic Web initiative (Berners-Lee ∗Work conducted during internship at Microsoft Re- search. … Textual Object Predicate Classic Meatloaf /recipe/name 20 minutes /recipe/preptime 1 hour, 10 minutes /recipe/cookTime 1 celery rib, … /recipe/ingredients Metadata Object-predicate Pairs: Figure 1: Metadata Example from SimplyRecipes. Recipe cookTime prepTime recipeInstructions recipeyield etc. Figure 2: Hierarchy diagram showing properties of “recipe” from schema.org/recipe. et al., 2001), it is estimated that a non-trivial por- tion of web pages contain metadata annotations that provide a deeper understanding of the website content. The Web Data Commons project (Müh- leisen and Bizer, 2012) estimates that 0.9 billion HTML pages out of the 2.5 billion pages (37.1%) in the Common Crawl web corpus1 contain struc- tured metadata. Figure 1 shows an example of this metadata which comes in the form of object- predicate pairs annotated with schema.org tags – a set of tags/predicates defined in the schema.org2 hierarchy. In the example, the hierarchical meta- data is used …

Content truncated to 500 words