Improving Semantic Parsing via Answer Type Inference

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 149–159, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Improving Semantic Parsing via Answer Type Inference Semih Yavuz1, Izzeddin Gur1, Yu Su1, Mudhakar Srivatsa2 and Xifeng Yan1 1University of California, Santa Barbara, Department of Computer Science 2IBM Research {syavuz,izzeddingur,ysu,xyan}@cs.ucsb.edu msrivats@us.ibm.com Abstract In this work, we show the possibility of infer- ring the answer type before solving a factoid question and leveraging the type information to improve semantic parsing. By replacing the topic entity in a question with its type, we are able to generate an abstract form of the ques- tion, whose answer corresponds to the answer type of the original question. A bidirectional LSTM model is built to train over the abstract form of questions and infer their answer types. It is also observed that if we convert a ques- tion into a statement form, our LSTM model achieves better accuracy. Using the predicted type information to rerank the logical forms returned by AgendaIL, one of the leading se- mantic parsers, we are able to improve the F1-score from 49.7% to 52.6% on the WE- BQUESTIONS data. 1 Introduction Large scale knowledge bases (KB) like Freebase (Bollacker et al., 2008), DBpedia (Auer et al., 2007), and YAGO (Suchanek et al., 2007) that store the world’s factual information in a structured fash- ion have become substantial resources for people to solve questions. KB-based factoid question answer- ing (KB-QA) that attempts to find exact answers to natural language questions has gained much atten- tion recently. KB-QA is a challenging task due to the representation variety between natural language and structural knowledge in KBs. As one of the promising KB-QA techniques, se- mantic parsing maps a natural language question into its semantic representation (e.g., logical forms). Ranking F1 # Improved Qs AgendaIL 49.7 - w/ Oracle Types@10 57.3 +234 w/ Oracle Types@20 58.7 +282 w/ Oracle Types@50 60.1 +331 w/ Oracle Types@All 60.5 +345 Table 1: What if the correct answer type is enforced? On We- bQuestions, we remove those with incorrect answer types in the top-k logical forms returned by AgendaIL (Berant and Liang, 2015), a leading semantic parsing system, and report the new average F1 score as well as the number of questions with an improved F1 score. It uses a logical language with predicates closely re- lated to KB schema, and constructs a dictionary that maps relations to KB predicates. The problem then reduces to generating candidate logical forms, rank- ing them, and selecting one to derive the final an- swer. In this work, we propose an answer type pre- diction model that can improve the ranking of the candidate logical forms generated by seman- tic parsing. The type of an entity, e.g., person, organization, location, carries very useful information for various down-stream natural lan- guage processing tasks such as co-reference resolu- tion (Recasens et al., 2013), knowledge base popu- lation (Carlson et al., 2010), relation extraction (Yao et al., 2012), and question answering …

Content truncated to 500 words