Articles

AbstractFigurative language understanding has been recently framed as a recognizing textual entailment (RTE) task (a.k.a. natural language inference (NLI)). However, similar to classical RTE/NLI datasets they suffer from spurious correlations and annotation artifacts. To tackle this problem, work on NLI has built explanation-based datasets such as eSNLI, allowing us to …...

AbstractLarge-scale pre-trained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks. This paper presents mPLUG, a new vision-language foundation model for both cross-modal understanding and generation. Most existing pre-trained models suffer from inefficiency and …...

AbstractRadiology report generation systems have the potential to reduce the workload of radiologists by automatically describing the findings in medical images. To broaden the application of the report generation system, the system should generate reports that are not only factually accurate but also chronologically consistent, describing images that are presented …...

AbstractAbstractive summarization models often produce factually inconsistent summaries that are not supported by the original article. Recently, a number of fact-consistent evaluation techniques have been proposed to address this issue; however, a detailed analysis of how these metrics agree with one another has yet to be conducted. In this paper, …...

AbstractRelational triple extraction is challenging for its difficulty in capturing rich correlations between entities and relations. Existing works suffer from 1) heterogeneous representations of entities and relations, and 2) heterogeneous modeling of entity-entity interactions and entity-relation interactions. Therefore, the rich correlations are not fully exploited by existing works. In this …...

AbstractReal-world politically-opinionated memes often rely on figurative language to cloak propaganda and radical ideas to help them spread. It is not only a scientific challenge to develop machine learning models to recognize them in memes, but also sociologically beneficial to understand hidden meanings at scale and raise awareness. These memes …...

AbstractContrastive explanations for understanding the behavior of black box models has gained a lot of attention recently as they provide potential for recourse. In this paper, we propose a method Contrastive Attributed explanations for Text (CAT) which provides contrastive explanations for natural language text data with a novel twist as …...

AbstractTransformer has been demonstrated effective in Neural Machine Translation (NMT). However, it is memory-consuming and time-consuming in edge devices, resulting in some difficulties for real-time feedback. To compress and accelerate Transformer, we propose a Hybrid Tensor-Train (HTT) decomposition, which retains full rank and meanwhile reduces operations and parameters. A Transformer …...

AbstractLinear transformers aim to reduce the quadratic space-time complexity of vanilla transformers. However, they usually suffer from degraded performances on various tasks and corpus. In this paper, we examine existing kernel-based linear transformers and identify two key issues that lead to such performance gaps: 1) unbounded gradients in the attention …...

This study discusses the curve-fitting results of the frequency distributions of dependency distances in sentences from a multi-lingual parallel corpus. It assumes that these distributions fit a mathematically well-defined distribution (the right truncated modified Zipf- Alekseev distribution) quite well, which indicates that these distributions of dependency distances reflect an aspect …...

This paper compares three classroom- based measurement methods of the general proficiency of English as a second language (GEP): one measures the GEP by dictation performance; another by read-aloud performance; and the other by both dictation performance and read-aloud performance. A learner’s GEP has been measured by commercially tests because …...

Pretraining models such as BERT, have achieved high accuracy in various natu- ral language processing tasks by pretrain- ing on a large corpus and fine-tuning on downstream task data. However, BERT trains token-level inferences, which make it difficult to train unknown or compound words that are split by byte-pair encod- …...

In this paper, we define a sentence re- generation task: re-generate a sentence given a set of sentences that cover it. Due to the absence of a dataset to perform this task, we firstly build three language resources of a new type containing more than 4 million anno- tated sentences. …...

Crosslinguistic comparison of VOT has indicated linguistic transfer of voicing and aspiration contrasts in many languages. Mandarin has clear aspiration contrasts for voiceless stops, while Min presents another complicated VOT pattern, where voicing and aspiration contrasts are involved. The present study makes a crosslinguistic comparison between languages with voicing and …...

This paper introduces the Forgotten Realms Wiki (FRW) data set and domain specific natu- ral language generation using FRW along with related analyses. Forgotten Realms is the de- facto default setting of the popular open ended tabletop fantasy role playing game, Dungeons & Dragons. The data set was extracted from …...

The technological advancement and the expansion of using the internet has made it possible to communicate between people and machines. Many exchanged textual resources have been given simultaneously make users have difficulty in choosing the most appropriate items that will improve their knowledge. To deal with data sparsity problem, review-based …...

In this study, we propose a method to construct Japanese BERT using fixed Token Embedding to reduce BERT model construction time. This method constructs word embeddings in advance using word2vec and the corpus, which is used in learning BERT models. The token embeddings in the BERT model are set to …...

Most of the available resources for low resource languages are crawled from the web. In order to obtain reasonable ma- chine translation performance with such datasets, it is important to filter low qual- ity samples from the training data. In this paper we explore the use of lan- guage agnostic …...

A corpus-based study is conducted to investigate verbs collocating with present-tensed will and be going to. Data from Corpus of Contemporary American English (COCA) are addressed, especially for lexical verb-collocates (V- collocates). Data-driven results show that lexical V-collocates of will and be going to are sensitive to semantic and pragmatic …...

In the process of numerically modeling natural languages, developing language embeddings is a vital step. However, it is challenging to develop functional embeddings for resource- poor languages such as Sinhala, for which sufficiently large corpora, effective language parsers, and any other required resources are difficult to find. In such conditions, …...