Leveraging Knowledge in Multilingual Commonsense Reasoning

Findings of the Association for Computational Linguistics: ACL 2022, pages 3237 - 3246 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Leveraging Knowledge in Multilingual Commonsense Reasoning Yuwei Fang, Shuohang Wang, Yichong Xu, Ruochen Xu, Siqi Sun Chenguang Zhu, Michael Zeng Microsoft Cognitive Services Research Group {yuwfan, shuowa, yicxu, ruox, siqi.sun, chezhu, nzeng}@microsoft.com Abstract Commonsense reasoning (CSR) requires mod- els to be equipped with general world knowl- edge. While CSR is a language-agnostic pro- cess, most comprehensive knowledge sources are restricted to a small number of languages, especially English. Thus, it remains unclear how to effectively conduct multilingual com- monsense reasoning (XCSR) for various lan- guages. In this work, we propose to use En- glish as a pivot language, utilizing English knowledge sources for our our commonsense reasoning framework via a translate-retrieve- translate (TRT) strategy. For multilingual com- monsense questions and answer candidates, we collect related knowledge via translation and retrieval from the knowledge in the source language. The retrieved knowledge is then translated into the target language and inte- grated into a pre-trained multilingual language model via visible knowledge attention. Then we utilize a diverse of four English knowledge sources to provide more comprehensive cover- age of knowledge in different formats. Exten- sive results on the XCSR benchmark demon- strate that TRT with external knowledge can significantly improve multilingual common- sense reasoning in both zero-shot and translate- train settings, consistently outperforming the state-of-the-art by more than 3% on the mul- tilingual commonsense reasoning benchmark X-CSQA and X-CODAH. 1 Introduction Commonsense reasoning (CSR) is one of the key challenges in natural language understanding. It requires a model to integrate world knowledge into language modeling to produce answers. A large number of tasks have been proposed to eval- uate commonsense reasoning in English, such as COPA (Roemmele et al., 2011a) and CSQA (Tal- mor et al., 2019). Most recently, multilingual commonsense rea- soning (XCSR) extends a model’s commonsense en zh it fi ja es fr de pl pt ru sh ro nl ko cs hu mk ar el 0 100000 200000 300000 400000 500000 600000 700000 800000 Figure 1: Number of total definitions per language. The statistics are generated from Wiktionary 2021-10- 01 dump. There are 55 languages with 10,000 or more definitions and we list top 20 languages by the defini- tion count here. capability beyond language barriers and has be- gun to draw attention from the community. number of multilingual datasets have emerged forA this challenging task, for example, X-CSQA (Lin et al., 2021), X-CODAH (Lin et al., 2021), XCOPA (Ponti et al., 2020). To solve commonsense reasoning tasks, it is essential to fuse human created knowledge into pre-trained language models (PLM) (Lin et al., 2019; Feng et al., 2020; Yu et al., 2020; Xu et al., 2021b). For example, DEKCOR (Xu et al., 2021b) integrates knowledge from ConceptNet (Speer et al., 2017) and Wiktionary 1 into the ALBERT model (Lan et al., 2020) for commonsense ques- tion answering. However, most existing knowledge sources are crafted in a few …

Content truncated to 500 words