Articles

Relational fact extraction aims to extract se- mantic triplets from unstructured text. In this work, we show that all of the relational fact extraction models can be organized accord- ing to a graph-oriented analytical perspective. An efficient model, aDjacency lIst oRiented rElational faCT (DIRECT), is proposed based on this analytical …...

Transfer learning has become the dominant paradigm for many natural language process- ing tasks. In addition to models being pre- trained on large datasets, they can be further trained on intermediate (supervised) tasks that are similar to the target task. For small Nat- ural Language Inference (NLI) datasets, lan- guage …...

Generative linguistic steganography mainly utilized language models and applied stegano- graphic sampling (stegosampling) to gener- ate high-security steganographic text (stego- text). However, previous methods generally lead to statistical differences between the con- ditional probability distributions of stegotext and natural text, which brings about security risks. In this paper, to further …...

The success of Neural Machine Translation (NMT) largely depends on the availability of large bitext training corpora. Due to the lack of such large corpora in low-resource language pairs, NMT systems often exhibit poor perfor- mance. Extra relevant monolingual data of- ten helps, but acquiring it could be quite ex- …...

Pre-trained language models such as Clini- calBERT have achieved impressive results on tasks such as medical Natural Language In- ference. At first glance, this may suggest that these models are able to perform medi- cal reasoning tasks, such as mapping symp- toms to diseases. However, we find that stan- dard …...

Multi-hop question generation requires com- plex reasoning and coherent language realiza- tion. Learning a generation model for the problem requires extensive multi-hop question answering (QA) data, which are limited due to the manual collection effort. A two-phase strategy addresses the insufficiency of multi- hop QA data by first generating and …...

Multilingual Neural Machine Translation (MNMT) has aroused widespread interest due to its efficiency. An exciting advantage of MNMT models is that they could also translate between unsupervised (zero-shot) language directions. Language tag (LT) strategies are often adopted to indicate the translation directions in MNMT. In this paper, we demonstrate that …...

Recently, Text-to-SQL for multi-turn dialogue has attracted great interest. Here, the user input of the current turn is parsed into the corresponding SQL query of the appropriate database, given all previous dialogue history. Current approaches mostly employ end-to-end models and consequently face two challenges. First, dialogue history modeling and Text-to- …...

The field of natural language understanding has experienced exponential progress in the last few years, with impressive results in sev- eral tasks. This success has motivated re- searchers to study the underlying knowledge encoded by these models. Despite this, at- tempts to understand their semantic capabil- ities have not been …...

Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT), by drastically reducing the need for large parallel data. Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by mask- ing parts of the input and reconstructing them in the decoder. In this work, we systemati- cally compare …...

Bilingual Lexicon Induction (BLI) aims to map words in one language to their transla- tions in another, and are typically through learning linear projections to align monolin- gual word representation spaces. Two classes of word representations have been explored for BLI: static word embeddings and contextual representations, but there is …...

AbstractWe adopt an evolutionary view on language change in which cognitive factors (in addition to social ones) affect the fitness of words and their success in the linguistic ecosystem. Specifically, we propose a variety of psycholinguistic factors—semantic, distributional, and phonological—that we hypothesize are predictive of lexical decline, in which words …...

AbstractDebugging a machine learning model is hard since the bug usually involves the training data and the learning process. This becomes even harder for an opaque deep learning model if we have no clue about how the model actually works. In this survey, we review papers that exploit explanations to …...

AbstractInterpretable rationales for model predictions are crucial in practical applications. We develop neural models that possess an interpretable inference process for dependency parsing. Our models adopt instance-based inference, where dependency edges are extracted and labeled by comparing them to edges in a training set. The training edges are explicitly used …...

AbstractWe introduce a simple but flexible mechanism to learn an intermediate plan to ground the generation of abstractive summaries. Specifically, we prepend (or prompt) target summaries with entity chains—ordered sequences of entities mentioned in the summary. Transformer-based sequence-to-sequence models are then trained to generate the entity chain and then continue …...

AbstractHuman evaluation of modern high-quality machine translation systems is a difficult problem, and there is increasing evidence that inadequate evaluation procedures can lead to erroneous conclusions. While there has been considerable research on human evaluation, the field still lacks a commonly accepted standard procedure. As a step toward this goal, …...

AbstractSeveral metrics have been proposed for assessing the similarity of (abstract) meaning representations (AMRs), but little is known about how they relate to human similarity ratings. Moreover, the current metrics have complementary strengths and weaknesses: Some emphasize speed, while others make the alignment of graph structures explicit, at the price …...

AbstractThis paper contains prompts and model outputs that are offensive in nature. When trained on large, unfiltered crawls from the Internet, language models pick up and reproduce all kinds of undesirable biases that can be found in the data: They often generate racist, sexist, violent, or otherwise toxic language. As …...

AbstractProgress in cross-lingual modeling depends on challenging, realistic, and diverse evaluation sets. We introduce Multilingual Knowledge Questions and Answers (MKQA), an open- domain question answering evaluation set comprising 10k question-answer pairs aligned across 26 typologically diverse languages (260k question-answer pairs in total). Answers are based on heavily curated, language- independent …...

AbstractWe describe a parser of English effectuated by biologically plausible neurons and synapses, and implemented through the Assembly Calculus, a recently proposed computational framework for cognitive function. We demonstrate that this device is capable of correctly parsing reasonably nontrivial sentences.1 While our experiments entail rather simple sentences in English, our …...