Schema Aware Semantic Reasoning for Interpreting Natural Language Queries in Enterprise Settings

Proceedings of the 28th International Conference on Computational Linguistics, pages 1334–1345 Barcelona, Spain (Online), December 8-13, 2020 1334 Schema Aware Semantic Reasoning for Interpreting Natural Language Queries in Enterprise Settings Jaydeep Sen IBM Research AI, India jaydesen@in.ibm.com Tanaya Babtiwale NMIMS, India tanbab98@gmail.com Kanishk Saxena SMVIT, India kanishksaxena5@gmail.com Yash Butala IIT Kharagpur, India yashbutala@iitkgp.ac.in Sumit Bhatia IBM Research AI, India sumitbhatia@in.ibm.com Karthik Sankaranarayanan IBM Research AI, India kartsank@in.ibm.com Abstract Natural Language Query interfaces allow the end-users to access the desired information without the need to know any specialized query language, data storage, or schema details. Even with the recent advances in NLP research space, the state-of-the-art QA systems fall short of under- standing implicit intents of real-world Business Intelligence (BI) queries in enterprise systems as Natural Language Understanding remains an AI-hard problem. We posit that deploying ontology reasoning over domain semantics can help in achieving better natural language understanding for QA systems. In this paper, we specifically focus on building a Schema Aware Semantic Reason- ing Framework that translates natural language interpretation as a sequence of solvable tasks by an ontology reasoner. We apply our framework on top of an ontology-based, state-of-the-art nat- ural language question-answering system ATHENA, and experiment with 4 benchmarks focused on BI queries. Our experimental numbers empirically show that the Schema Aware Semantic Reasoning indeed helps in achieving significantly better results for handling BI queries with an average accuracy improvement of 30% 1 Introduction Natural Language Query (NLQ) interfaces for information access (e.g., Alexa, Google Assistant, and Siri) have gained popularity in recent times as they allow the end-users to specify their information needs in natural language. In enterprise settings, systems with NLQ interfaces (Li and Jagadish, 2014; Saha et al., 2016; Sen et al., 2020) are especially desirable as they enable the business analysts to access domain-specific knowledge without having to learn specialized query languages such as SQL, SPARQL, or other customized query languages. NLQ interfaces also allow the end-user to explore and analyze the data without the need to learn about the storage mechanism and schema-specific details of the underlying system. Thus such systems can help boost analyst productivity and minimize training costs and time. Current State of NLQ Systems: To retrieve required results from the underlying database, NLQ based systems need to translate the input natural language query to a structured query (e.g., SQL, SPARQL) that is then used to query the database. Recent advances in NLP research has produced many machine learn- ing based systems which try to model the QA pipeline as a sequence-to-sequence architecture (Zhong et al., 2017a; Xu et al., 2017; Guo et al., 2019). However, they focus on rather simple queries on a single table or on a relatively simple schema with a few tables. For example, RAT-SQL (Wang et al., 2020) is a state-of-the-art on Spider (Yu et al., 2018) dataset, where Spider dev set has an average of only 4 tables per Database and less complex queries than typical BI queries seen with complex nesting (Sen et …

Content truncated to 500 words