Can Explanations Be Useful for Calibrating Black Box Models?

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 561 - 570 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Neural Label Search for Zero-Shot Multi-Lingual Extractive Summarization Ruipeng Jia1,2∗, Xingxing Zhang3†, Yanan Cao1,2†, Shi Wang4, Zheng Lin1,2 and Furu Wei3 1Institute of Information Engineering, Chinese Academy of Sciences 2School of Cyber Security, University of Chinese Academy of Sciences 3Microsoft Research Asia 4Institute of Computing Technology, Chinese Academy of Sciences 1,2{jiaruipeng,caoyanan,linzheng}@iie.ac.cn 3{xizhang,fuwei}@microsoft.com, 4wangshi@ict.ac.cn Abstract In zero-shot multilingual extractive text sum- marization, a model is typically trained on English summarization dataset and then ap- plied on summarization datasets of other lan- guages. Given English gold summaries and documents, sentence-level labels for extrac- tive summarization are usually generated us- ing heuristics. However, these monolingual labels created on English datasets may not be optimal on datasets of other languages, for that there is the syntactic or semantic discrepancy between different languages. In this way, it is possible to translate the English dataset to other languages and obtain different sets of la- bels again using heuristics. To fully leverage the information of these different sets of labels, we propose NLSSum (Neural Label Search for Summarization), which jointly learns hierar- chical weights for these different sets of la- bels together with our summarization model. We conduct multilingual zero-shot summariza- tion experiments on MLSUM and WikiLingua datasets, and we achieve state-of-the-art results using both human and automatic evaluations across these two datasets. 1 Introduction The zero-shot multilingual tasks, which aim to transfer models learned on a high-resource lan- guage (e.g., English) to a relatively low-resource language (e.g., Turkish) without further training, are challenging (Ruder et al., 2019). Recently, large pre-trained multilingual transformers such as M-BERT (Devlin et al., 2019), XLM (Lample and Conneau, 2019), and XLM-R (Conneau et al., 2020) have shown remarkable performance on zero- shot multilingual natural language understanding tasks. During pre-training, these transformer mod- els project representations of different languages ∗ Work done during the first author’s internship at Mi- crosoft Research Asia. †Corresponding authors Sentence (English, Label 1): He was never charged in that Caribbean Nation. Reference Summary (English): He was arrested twice, but never charged in Natalee Holloway’s disappearance. Translated Sentence (German, Label 0): Er wurde jedoch nie in dieser karibischen Nation angeklagt. Translated Reference Summary (German): Beim Verschwinden von Natalee Holloway wurde er zweimal verhaftet, aber nie angeklagt. Table 1: Monolingual Bias for Different Languages. into the same vector space, which makes the trans- fer learning across different languages easier dur- ing fine-tuning (Gong et al., 2021). In zero-shot extractive summarization, we train an extractive model (based on a pre-trained multilingual trans- former) on English summarization dataset, which selects important sentences in English documents. Then, we apply this trained model to documents of a different language (i.e., extracting sentences of documents in another language). In this paper, we aim to enhance the zero-shot capabilities of multi- lingual sentence-level extractive summarization. In text summarization, most datasets only contain human-written abstractive summaries as ground …

Content truncated to 500 words