Articles

Recent studies report that many machine read- ing comprehension (MRC) models can per- form closely to or even better than humans on benchmark datasets. However, existing works indicate that many MRC models may learn shortcuts to outwit these benchmarks, but the performance is unsatisfactory in real-world ap- plications. In this …...

Data augmentation has recently seen increased interest in NLP due to more work in low- resource domains, new tasks, and the popu- larity of large-scale neural networks that re- quire large amounts of training data. De- spite this recent upsurge, this area is still rel- atively underexplored, perhaps due to …...

Languages vary in their placement of multi- ple adjectives before, after, or surrounding the noun, but they typically exhibit strong intra- language tendencies on the relative order of those adjectives (e.g., the preference for ‘big blue box’ in English, ‘grande boîte bleue’ in French, and ‘als.und¯uq al’azraq alkab¯ır’ in Ara- …...

Generating text from structured inputs, such as meaning representations or RDF triples, has often involved the use of specialized graph- encoding neural networks. However, recent applications of pretrained transformers to lin- earizations of graph inputs have yielded state- of-the-art generation results on graph-to-text tasks. Here, we explore the ability of …...

Transformer is the backbone of modern NLP models. In this paper, we propose Real- Former, a simple and generic technique to create Residual Attention Layer Transformer networks that significantly outperform the canonical Transformer and its variants (BERT, ETC, etc.) on a wide spectrum of tasks including Masked Language Modeling, GLUE, …...

The widespread use of offensive content in so- cial media has led to an abundance of research in detecting language such as hate speech, cyberbullying, and cyber-aggression. Recent work presented the OLID dataset, which fol- lows a taxonomy for offensive language identi- fication that provides meaningful information for understanding the …...

Tackling online hatred using informed textual responses – called counter narratives – has been brought under the spotlight recently. Ac- cordingly, a research line has emerged to au- tomatically generate counter narratives in or- der to facilitate the direct intervention in the hate discussion and to prevent hate content from …...

Commonsense reasoning is intuitive for hu- mans but has been a long-term challenge for artificial intelligence (AI). Recent advance- ments in pretrained language models have shown promising results on several common- sense benchmark datasets. However, the relia- bility and comprehensiveness of these bench- marks towards assessing model’s common- sense reasoning …...

Modern text simplification (TS) heavily relies on the availability of gold standard data to build machine learning models. However, ex- isting studies show that parallel TS corpora contain inaccurate simplifications and incor- rect alignments. Additionally, evaluation is usually performed by using metrics such as BLEU or SARI to compare system …...

Neural language models exhibit impressive performance on a variety of tasks, but their in- ternal reasoning may be difficult to understand. Prior art aims to uncover meaningful proper- ties within model representations via probes, but it is unclear how faithfully such probes portray information that the models actually use. To …...

Cognate prediction is the task of generating, in a given language, the likely cognates of words in a related language, where cognates are words in related languages that have evolved from a common ancestor word. It is a task for which little data exists and which can aid linguists in …...

Large generative language models have been very successful for English, but other lan- guages lag behind, in part due to data and com- putational limitations. We propose a method that may overcome these problems by adapt- ing existing pre-trained models to new lan- guages. Specifically, we describe the adapta- tion …...

The capacity of empathy is crucial to the suc- cess of open-domain dialog systems. Due to its nature of multi-dimensionality, there are various factors that relate to empathy expres- sion, such as communication mechanism, dia- log act and emotion. However, existing meth- ods for empathetic response generation usu- ally either …...

Keyphrase Prediction (KP) task aims at pre- dicting several keyphrases that can summarize the main idea of the given document. Main- stream KP methods can be categorized into purely generative approaches and integrated models with extraction and generation. How- ever, these methods either ignore the diver- sity among keyphrases or …...

Symbolic music understanding, which refers to the understanding of music from the sym- bolic data (e.g., MIDI format, but not audio), covers many music applications such as genre classification, emotion classification, and mu- sic pieces matching. While good music rep- resentations are beneficial for these applica- tions, the lack of …...

While automatic summarization evaluation methods developed for English are routinely applied to other languages, this is the first at- tempt to systematically quantify their panlin- guistic efficacy. We take a summarization cor- pus for eight different languages, and manu- ally annotate generated summaries for focus (precision) and coverage (recall). Based …...

Given a video, video grounding aims to re- trieve a temporal moment that semantically corresponds to a language query. In this work, we propose a Parallel Attention Network with Sequence matching (SeqPAN) to address the challenges in this task: multi-modal represen- tation learning, and target moment boundary prediction. We design …...

Automated predictions require explanations to be interpretable by humans. One type of ex- planation is a rationale, i.e., a selection of in- put features such as relevant text snippets from which the model computes the outcome. How- ever, a single overall selection does not pro- vide a complete explanation, e.g., …...

We investigate if, given a simple symbol mask- ing strategy, self-attention models are capable of learning nested structures and generalise over their depth. We do so in the simplest set- ting possible, namely languages consisting of nested parentheses of several kinds. We use encoder-only models, which we train to pre- …...

Image-text matching has been a popular re- search topic which bridges vision and lan- guage through semantic understanding. Re- cent works mainly focus on exploring the in- teractions between images and sentences to im- prove the performance without considering in- ference efficiency. Specifically, for the large scale databases, it is …...