Are Neural Networks Extracting Linguistic Properties or Memorizing Training Data? An Observation with a Multilingual Probe for Predicting Tense

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 347–357 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 347 Retrieval, Re-ranking and Multi-task Learning for Knowledge-Base Question Answering Zhiguo Wang, Patrick Ng, Ramesh Nallapati, Bing Xiang AWS AI Labs {zhiguow, patricng, rnallapa, bxiang}@amazon.com Abstract Question answering over knowledge bases (KBQA) usually involves three sub-tasks, namely topic entity detection, entity linking and relation detection. Due to the large num- ber of entities and relations inside knowledge bases (KB), previous work usually utilized so- phisticated rules to narrow down the search space and managed only a subset of KBs in memory. In this work, we leverage a retrieve- and-rerank framework to access KBs via tradi- tional information retrieval (IR) method, and re-rank retrieved candidates with more pow- erful neural networks such as the pre-trained BERT model. Considering the fact that di- rectly assigning a different BERT model for each sub-task may incur prohibitive costs, we propose to share a BERT encoder across all three sub-tasks and define task-specific layers on top of the shared layer. The unified model is then trained under a multi-task learning frame- work. Experiments show that: (1) Our IR- based retrieval method is able to collect high- quality candidates efficiently, thus enables our method adapt to large-scale KBs easily; (2) the BERT model improves the accuracy across all three sub-tasks; and (3) benefiting from multi- task learning, the unified model obtains fur- ther improvements with only 1/3 of the origi- nal parameters. Our final model achieves com- petitive results on the SimpleQuestions dataset and superior performance on the FreebaseQA dataset. 1 Introduction Answering natural language questions by search- ing over large-scale knowledge bases (KBQA) is highly demanded by real-life applications, such as Google Assistant, Siri, and Alexa. Owing to the availability of large-scale KBs, significant ad- vancements have been made over the years. One main research direction views KBQA as a seman- tic matching task (Bordes et al., 2014; Dong et al., 2015; Dai et al., 2016; Hao et al., 2017; Mohammed et al., 2018; Yu et al., 2018; Wu et al., 2019; Chen et al., 2019a; Petrochuk and Zettlemoyer, 2018), and finds a relation-chain within KBs that is most similar to the question in a common semantic space, where the relation-chain can be 1-hop, 2-hop or multi-hop (Chen et al., 2019b). Another research direction formulates KBQA as a semantic parsing task (Berant et al., 2013; Bao et al., 2016; Luo et al., 2018), and tackles questions that involve complex reasoning, such as ordinal (e.g. What is the sec- ond largest fulfillment center of Amazon?), and aggregation (e.g. How many fulfillment centers does Amazon have?). Most recently, some stud- ies proposed to derive answers from both KBs and free-text corpus to deal with the low-coverage issue of KBs (Xu et al., 2016; Sun et al., 2018; Xiong et al., 2019; Sun et al., 2019). In this paper, we follow the first research direction since the relation- chain type of …

Content truncated to 500 words