Encoder Decoder Architecture

Neural Machine Translation (NMT) based on the encoder-decoder architecture has recently become a new paradigm. Researchers have proven that the target-side monolingual data can greatly enhance the decoder model of …...

In this paper we address the question of how to render sequence-level networks better at handling structured input. We propose a ma- chine reading simulator which processes text incrementally from …...

AbstractNatural language generation (NLG) is a critical component in a spoken dialogue system. This paper presents a Recurrent Neural Network based Encoder-Decoder architecture, in which an LSTM-based decoder is introduced …...

We describe the Sogou neural machine translation systems for the WMT 2017 Chinese↔English news translation tasks. Our systems are based on a multi- layer encoder-decoder architecture with attention mechanism. The …...

AbstractA core step in statistical data-to-text generation concerns learning correspondences between structured data representations (e.g., facts in a database) and associated texts. In this paper we aim to bootstrap generators …...

While extractive summarization is a well stud- ied problem, it is far from solved. In recent years a large number of interesting and com- plex models have been used to …...

AbstractWe present deep communicating agents in an encoder-decoder architecture to address the challenges of representing a long document for abstractive summarization. With deep communicating agents, the task of encoding a …...

Today’s machine translation systems out- put the same translation for a given in- put, despite important differences between users. In practice, translations should be customized for each reader, for instance …...

The paper proposes a system that compensates most of the noise in a text in natural language caused by technical imperfection of the input device such as keyboard or scanner …...

AbstractLarge pre-trained language models (LMs) are known to encode substantial amounts of linguistic information. However, high-level reasoning skills, such as numerical reasoning, are difficult to learn from a language-modeling objective …...

AbstractMultilingual neural machine translation (NMT) has led to impressive accuracy improvements in low-resource scenarios by sharing common linguistic information across languages. However, the traditional multilingual model fails to capture the …...

We have seen significant improvements in machine translation due to the usage of deep learn- ing. While the improvements in translation quality are impressive, the encoder-decoder archi- tecture enables many …...

AbstractRecent approaches to data-to-text generation have adopted the very successful encoder-decoder architecture or variants thereof. These models generate text that is fluent (but often imprecise) and perform quite poorly at …...

. This paper describes a study on gender bias in French/English neural machine trans- lation (MT) systems. We introduce a controlled corpus to measure the intensity of such biases in …...

AbstractThe introduction of immensely large Causal Language Models (CLMs) has rejuvenated the interest in open-ended text generation. However, controlling the generative process for these Transformer-based models is at large an …...

AbstractEncoder-decoder architecture is widely adopted for sequence-to-sequence modeling tasks. For machine translation, despite the evolution from long short-term memory networks to Transformer networks, plus the introduction and development of attention …...

AbstractLarge multilingual Transformer-based machine translation models have had a pivotal role in making translation systems available for hundreds of languages with good zero-shot translation performance. One such example is the …...

AbstractMany machine learning-based low-code or no-code applications involve generating code that interacts with structured knowledge. For example, one of the most studied tasks in this area is generating SQL code …...