Articles

Recently, unsupervised parsing of syntactic trees has gained considerable attention. prototypical approach to such unsupervisedA parsing employs reinforcement learning and auto-encoders. However, no mechanism en- sures that the learnt model leverages the well- understood language grammar. We propose an approach that utilizes very generic linguistic knowledge of the language present …...

Multilingual language models have been a crucial breakthrough as they considerably re- duce the need of data for under-resourced languages. Nevertheless, the superiority of language-specific models has already been proven for languages having access to large amounts of data. In this work, we focus on Catalan with the aim to …...

Natural language inference (NLI) is formu- lated as a unified framework for solving vari- ous NLP problems such as relation extraction, question answering, summarization, etc. It has been studied intensively in the past few years thanks to the availability of large-scale labeled datasets. However, most existing studies fo- cus on …...

Usually, we train a neural system on a se- quence of mini-batches of labeled instances. Each mini-batch is composed of k samples, and each sample will learn a representation vector. MIXUP implicitly generates synthetic samples through linearly interpolating inputs and their corresponding labels of random sam- ple pairs in the …...

For many (minority) languages, the resources needed to train large models are not available. We investigate the performance of zero-shot transfer learning with as little data as possi- ble, and the influence of language similarity in this process. We retrain the lexical layers of four BERT-based models using data from …...

The notorious one-to-many nature of open- domain dialogues poses huge challenges for automatic evaluation methods. Recent stud- ies attempt to mitigate this issue by consid- ering the similarity of the generated response with the conversational context and design dis- criminative models to learn from multiple pos- itive responses. Despite the …...

Recent work on multilingual dependency pars- ing focused on developing highly multilingual parsers that can be applied to a wide range of low-resource languages. In this work, we sub- stantially outperform such “one model to rule them all” approach with a heuristic selection of languages and treebanks on which to …...

Existing matching models for response selec- tion adopt the independent matching (IM) ap- proach. To complete a prediction, they have to perform N independent matches, where N is the number of response options. In this paper, we explore a joint matching (JM) approach which performs matching only once regardless of …...

Conventional autoregressive models have achieved great success in text generation but suffer from the exposure bias problem in that token sequences in the training and in the gen- eration stages are mismatched. While gener- ative adversarial networks (GANs) can rem- edy this problem, existing implementations of GANs directly on discrete …...

Some interpersonal verbs can implicitly at- tribute causality to either their subject or their object and are therefore said to carry an im- plicit causality (IC) bias. Through this bias, causal links can be inferred from a narrative, aiding language comprehension. We inves- tigate whether pre-trained language models (PLMs) encode …...

Table-to-text generation refers to generating a descriptive text from a key-value table. Traditional autoregressive methods, though can generate text with high fluency, suffer from low coverage and poor faithfulness prob- lems. To mitigate these problems, we pro- pose a novel Skeleton-based two-stage method that combines both Autoregressive and Non- Autoregressive …...

How to combine several pieces of evidence to verify a claim is an interesting semantic task. Very complex methods have been proposed, combining different evidence vectors using an evidence interaction graph. In this paper, we show that in case of inference based on trans- former models, two effective approaches use …...

Neural biomedical named entity recognition (BioNER) methods usually require a large amount of annotated data, while the anno- tated BioNER datasets are often difficult to obtain and small in scale due to the limita- tions of privacy, ethics and high degree of specialization. To alleviate the lack of train- ing …...

Named entity recognition (NER) is a funda- mental task in natural language processing, these is a long held belief that datasets benefit the model. However, not all the data help with generalization, and some samples may contain ambiguous entities or noisy labels. The ex- isting methods can not distinguish hard …...

Language coverage bias, which indicates the content-dependent differences between sen- tence pairs originating from the source and tar- get languages, is important for neural machine translation (NMT) because the target-original training data is not well exploited in current practice. By carefully designing experiments, we provide comprehensive analyses of the lan- …...

Supervised learning methods have proven to be effective for Aspect-Based Sentiment Anal- ysis (ABSA). However, the lack of fine- grained labeled data hinders their effective- ness in many domains. To address this is- sue, unsupervised domain adaptation methods are desired to transfer knowledge from a la- beled source domain to …...

Simultaneous machine translation (SIMT) in- volves translating source utterances to the tar- get language in real-time before the speaker utterance completes. This paper proposes the multilingual approach to SIMT, where a sin- gle model simultaneously translates between multiple language-pairs. This not only results in more efficiency in terms of the …...

Capturing the salient information from an in- put article has been a long-standing challenge for summarization. On Wikipedia, most of the wiki pages about people contain a factual ta- ble that lists the basic properties of the peo- ple. Illuminatingly, a factual table can be re- garded as a natural …...

Recent advances in natural language process- ing (NLP) have the ability to transform how classroom learning takes place. Combined with the increasing integration of technology in today’s classrooms, NLP systems leverag- ing question answering and dialog processing techniques can serve as private tutors or par- ticipants in classroom discussions to …...

Pruning has been demonstrated as an effective way of reducing computational complexity for deep networks, especially CNNs for computer vision tasks. In this paper, we investigate the opportunity to accelerate the inference of large-scale pre-trained language model via pruning. We propose EBERT, a dynamic struc- tured pruning algorithm for efficient …...