Articles

Pre-trained language models induce dense entity representations that offer strong per- formance on entity-centric NLP tasks, but such representations are not immediately inter- pretable. This can be a barrier to model uptake in important domains such as biomedicine. There has been recent work on general in- terpretable representation learning (Onoe …...

Commonsense reasoning is one of the key problems in natural language processing, but the relative scarcity of labeled data holds back the progress for languages other than English. Pretrained cross-lingual models are a source of powerful language-agnostic representations, yet their inherent reasoning capabilities are still actively studied. In this work, …...

ive Multi-Document Summarization Hao Zhou1 , Weidong Ren1 , Gongshen Liu1∗, Bo Su1 , Wei Lu2 1 School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University 2 StatNLP Research Group, Singapore University of Technology and Design {zhou1998,renweidong1997,lgshen,subo}@sjtu.edu.cn luwei@sutd.edu.sg Abstract Entities and their mentions convey significant semantic information in …...

In challenging economic times, obtaining value for money by ensuring financial integrity and fairer distribution of services are among the top priorities for social and health-care systems globally. However, healthcare billing policies are complex and identifying non-compliance is often narrow-scope, manual and expensive. Maintaining ‘integrity’ is a challenge - ensuring …...

We study a family of data augmentation meth- ods, substructure substitution (SUB2), that gen- eralizes prior methods. SUB2 generates new examples by substituting substructures (e.g., subtrees or subsequences) with others hav- ing the same label. This idea can be ap- plied to many structured NLP tasks such as part-of-speech tagging …...

The purpose of an argumentative text is to sup- port a certain conclusion. Yet, they are often omitted, expecting readers to infer them rather. While appropriate when reading an individual text, this rhetorical device limits accessibility when browsing many texts (e.g., on a search engine or on social media). In …...

Using multiple treebanks to improve parsing performance has shown positive results. How- ever, to what extent similar, yet competing an- notation decisions play in parser behavior is unclear. We investigate this within a multi-task learning (MTL) dependency parser setup on two parallel treebanks, UD and SUD, which, while possessing similar …...

Recent work demonstrates the potential of training one model for multilingual machine translation. In parallel, denoising pretraining using unlabeled monolingual data as a starting point for finetuning bitext machine translation systems has demonstrated strong performance gains. However, little has been explored on the potential to combine denoising pretraining with multilingual …...

State-of-the-art summarization systems are trained and evaluated on massive datasets scraped from the web. Despite their preva- lence, we know very little about the underly- ing characteristics (data noise, summarization complexity, etc.) of these datasets, and how these affect system performance and the reli- ability of automatic metrics like ROUGE. …...

Methodologies for training visual question an- swering (VQA) models assume the availabil- ity of datasets with human-annotated Image- Question-Answer (I-Q-A) triplets. This has led to heavy reliance on datasets and a lack of generalization to new types of questions and scenes. Linguistic priors along with biases and errors due to …...

In an election campaign, political parties pledge to implement various projects–should they be elected. But do they follow through? To track election pledges from parties’ election manifestos, we need to distinguish between pledges and general statements. In this paper, we use election manifestos of Swedish and In- dian political parties …...

Multiple studies have shown that Transform- ers are remarkably robust to pruning. Con- trary to this received wisdom, we demon- strate that pre-trained Transformer encoders are surprisingly fragile to the removal of a very small number of features in the layer out- puts (<0.0001% of model weights). In case of …...

Current open-domain question answering sys- tems often follow a Retriever-Reader architec- ture, where the retriever first retrieves rele- vant passages and the reader then reads the retrieved passages to form an answer. In this paper, we propose a simple and effec- tive passage reranking method, named Reader- guIDEd Reranker (RIDER), …...

The exploitation of syntactic graphs (SyGs) as a word’s context has been shown to be beneficial for distributional semantic models (DSMs), both at the level of individual word representations and in deriving phrasal repre- sentations via composition. However, notwith- standing the potential performance benefit, the syntactically-aware DSMs proposed to date …...

Journalists usually organize and present the contents of a news article following a well- defined structure. In this paper, we propose a novel joint model for structure-based news genre classification that simultaneously iden- tifies one of four commonly used news struc- tures (including Inverted Pyramid and three other structures) for …...

Existing bias mitigation methods to reduce disparities in model outcomes across cohorts have focused on data augmentation, debiasing model embeddings, or adding fairness-based optimization objectives during training. Sep- arately, certified word substitution robustness methods have been developed to decrease the impact of spurious features and synonym sub- stitutions on model …...

Disfluencies is an under-studied topic in NLP, even though it is ubiquitous in human con- versation. This is largely due to the lack of datasets containing disfluencies. In this paper, we present a new challenge question answering dataset, DISFL-QA, a derivative of SQUAD, where humans introduce con- textual disfluencies in …...

Knowledge Graph (KG) completion has been excessively studied with a massive number of models proposed for the Link Prediction (LP) task. The main limitation of such models is their insensitivity to time. Indeed, the tem- poral aspect of stored facts is often ignored. To this end, more and more works …...

We aim to renew interest in a particular multi- document summarization (MDS) task which we call AgreeSum: agreement-oriented multi- document summarization. Given a cluster of articles, the goal is to provide abstractive summaries that represent information common and faithful to all input articles. Given the lack of existing datasets, we …...

Incorporating external knowledge into Named Entity Recognition (NER) systems has been widely studied in the generic domain. In this paper, we focus on clinical domain where only limited data is accessible and interpretability is important. Recent advancement in technol- ogy and the acceleration of clinical trials has resulted in the …...