Articles

AbstractWe investigate the extent to which the behavior of neural network language models reflects incremental representations of syntactic state. To do so, we employ experimental methodologies which were originally developed in the field of psycholinguistics to study syntactic representation in the human mind. We examine neural network model behavior on …...

AbstractRecent work has shown that LSTMs trained on a generic language modeling objective capture syntax-sensitive generalizations such as long-distance number agreement. We have however no mechanistic understanding of how they accomplish this remarkable feat. Some have conjectured it depends on heuristics that do not truly take hierarchical structure into account. …...

AbstractSelf-training is a semi-supervised learning approach for utilizing unlabeled data to create better learners. The efficacy of self-training algorithms depends on their data sampling techniques. The majority of current sampling techniques are based on predetermined policies which may not effectively explore the data space or improve model generalizability. In this …...

AbstractPrevious research shows that eye-tracking data contains information about the lexical and syntactic properties of text, which can be used to improve natural language processing models. In this work, we leverage eye movement features from three corpora with recorded gaze information to augment a state-of-the-art neural model for named entity …...

Post-editing (PE) machine translation (MT) is widely used for dissemination because it leads to higher productivity than human translation from scratch (HT). In addition, PE translations are found to be of equal or better quality than HTs. However, most such studies measure quality solely as the number of errors. We …...

Two of the more predominant technologies that professional translators have at their disposal for improving productivity are machine translation (MT) and computer- aided translation (CAT) tools based on translation memories (TM). When transla- tors use MT, they can use automatic post- editing (APE) systems to automate part of the post-editing …...

Neural machine translation (NMT) sys- tems operate primarily on words (or sub- words), ignoring lower-level patterns of morphology. We present a character- aware decoder designed to capture such patterns when translating into morpho- logically rich languages. We achieve character-awareness by augmenting both the softmax and embedding layers of an attention-based …...

With the impressive fluency of modern ma- chine translation output, systems may pro- duce output that is fluent but not adequate (fluently inadequate). We seek to identify these errors and quantify their frequency in MT output of varying quality. To that end, we introduce a method for automat- ically predicting …...

This work presents an empirical approach to quantifying the loss of lexical rich- ness in Machine Translation (MT) systems compared to Human Translation (HT). Our experiments show how current MT sys- tems indeed fail to render the lexical di- versity of human generated or translated text. The inability of MT …...

Neural networks have become the state- of-the-art approach for machine transla- tion (MT) in many languages. While linguistically-motivated tokenization tech- niques were shown to have significant ef- fects on the performance of statistical MT, it remains unclear if those techniques are well suited for neural MT. In this pa- per, …...

Most neural machine translation systems are built upon subword units extracted by methods such as Byte-Pair Encoding (BPE) or wordpiece. However, the choice of num- ber of merge operations is generally made by following existing recipes. In this pa- per, we conduct a systematic exploration on different numbers of BPE …...

Neural machine translation (NMT) has set new quality standards in automatic transla- tion, yet its effect on post-editing produc- tivity is still pending thorough investiga- tion. We empirically test how the inclusion of NMT, in addition to domain-specific translation memories and termbases, im- pacts speed and quality in professional translation …...

Today’s machine translation systems out- put the same translation for a given in- put, despite important differences between users. In practice, translations should be customized for each reader, for instance when translating for children versus in a business setting. In this paper, we in- troduce the task of reading level …...

Phrase-based machine translation provides the system developer with controls that enable fine-grained control over machine translation output. One approach to provide similar control in neural machine transla- tion is placeholding (herein called mask- ing), which replaces input tokens with masks which are replaced with the origi- nal input text in …...

Memory-augmented neural networks (MANNs) have been shown to outper- form other recurrent neural network architectures on a series of artificial sequence learning tasks, yet they have had limited application to real-world tasks. We evaluate direct application of Neural Turing Machines (NTM) and Differentiable Neural Computers (DNC) to machine translation. We …...

As one of the contributions of this paper, this paper first explores the upper bound of context-based neural machine transla- tion and attempt to utilize previously un- used context information. We found that, if we could appropriately select the most in- formative context sentence for a given in- put source …...

There is a need for real-time communica- tion between the deaf and hearing with- out the aid of an interpreter. Develop- ing a machine translation (MT) system be- tween sign and spoken languages is a mul- timodal task since sign language is a vi- sual language, which involves the auto- …...

Modeling anaphora resolution is critical for proper pronoun translation in neural machine translation. Recently it has been addressed by context-aware models with varying success. In this work, we pro- pose a carefully designed training curricu- lum that facilitates better anaphora resolu- tion in context-aware NMT. As a baseline, we train …...

This paper proposes a novel multilin- gual multistage fine-tuning approach for low-resource neural machine translation (NMT), taking a challenging Japanese– Russian pair for benchmarking. Al- though there are many solutions for low- resource scenarios, such as multilingual NMT and back-translation, we have em- pirically confirmed their limited success when restricted …...

Earlier approaches indirectly studied the information captured by the hidden states of recurrent and non-recurrent neural ma- chine translation models by feeding them into different classifiers. In this paper, we look at the encoder hidden states of both transformer and recurrent machine trans- lation models from the nearest neighbors perspective. …...