Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 180 - 189 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Discriminative Marginalized Probabilistic Neural Method for Multi-Document Summarization of Medical Literature Gianluca Moro ,∗†, Luca Ragazzi ,∗, Lorenzo Valgimigli ,∗, Davide Freddi ∗Department of Computer Science and Engineering (DISI) University of Bologna, Cesena Campus, †CNIT Via dell’Università 50, I-47522 Cesena, Italy {gianluca.moro, l.ragazzi, lorenzo.valgimigli}@unibo.it - davide.freddi3@studio.unibo.it Abstract Although current state-of-the-art Transformer- based solutions succeeded in a wide range for single-document NLP tasks, they still strug- gle to address multi-input tasks such as multi- document summarization. Many solutions truncate the inputs, thus ignoring potential summary-relevant contents, which is unaccept- able in the medical domain where each informa- tion can be vital. Others leverage linear model approximations to apply multi-input concate- nation, worsening the results because all in- formation is considered, even if it is conflict- ing or noisy with respect to a shared back- ground. Despite the importance and social impact of medicine, there are no ad-hoc so- lutions for multi-document summarization. For this reason, we propose a novel discriminative marginalized probabilistic method (DAMEN) trained to discriminate critical information from a cluster of topic-related medical docu- ments and generate a multi-document summary via token probability marginalization. Results prove we outperform the previous state-of-the- art on a biomedical dataset for multi-document summarization of systematic literature reviews. Moreover, we perform extensive ablation stud- ies to motivate the design choices and prove the importance of each module of our method.1 1 Introduction The task of multi-document summarization aims to generate a compact and informative summary from a cluster of topic-related documents, which represents a very challenging natural language pro- cessing (NLP) application due to the presence of redundant and sometimes conflicting information among documents (Radev, 2000). In the medi- cal domain, in which machine learning plays an increasingly significant role (Domeniconi et al., 2014a; di Lena et al., 2015), multi-document sum- marization finds application in the generation of 1The solution of this paper is available at https:// disi-unibo-nlp.github.io/projects/damen systematic literature reviews, a biomedical paper that summarizes results across many studies (Khan et al., 2003). DeYoung et al. (2021) are the first that address this task, showing the related issues. State-of-the-art approaches leverage two leading solutions: hierarchical networks that capture cross- document relations via graph encodings (Wan and Yang, 2006; Liao et al., 2018; Li et al., 2020; Pa- sunuru et al., 2021) or hidden states aggregation (Fabbri et al., 2019; Liu and Lapata, 2019a; Jin et al., 2020), and long-range neural models that apply multi-input concatenation (Xiao et al., 2021). While effective, these solutions struggle to pro- cess clusters of many topic-related documents in low computational resource scenarios (Moro and Ragazzi, 2022) because they need to truncate the in- puts. Moreover, pre-trained state-of-the-art Trans- formers are not leveraged despite showing strong performance when fine-tuned in downstream tasks such as single-document summarization (Liu and Lapata, 2019b; Lewis et al., 2020a; Raffel et al., 2020; Beltagy et al., …

Content truncated to 500 words