Error Analysis

The problem of manually modifying the lexicon appears with any natural language processing program. Ideally, a program should be able to acquire new lexieal entries from context, the way people …...

\Ve are proposing a new framework of statisti- cal language modeling which integrates lexical .association statistics with syntactic preference, while maintaining the modularity of those differ- ent statistics types, facilitating …...

AbstractATR-MATRIX is a multi-lingual speech-to-speech translation system designed to facilitate communications between two parties of different languages engaged in a spontaneous conversation in a travel arrangement domain. In this paper, …...

The paper describes a similarity-based model to present the morphological rules for Chinese com- pound nouns. This representation model serves functions of 1) as the morphological rules of the compounds, …...

AbstractLexical mappings (word translations) between languages are an invaluable resource for multilingual processing. While the problem of extracting lexical mappings from parallel corpora is well-studied, the task is more challenging …...

We use SVM classifiers to predict the next action of a deterministic parser that builds labeled projective dependency graphs in an incremental fashion. Non-projective dependencies are captured indirectly by projectivizing …...

We present a two-stage multilingual de- pendency parser and evaluate it on 13 diverse languages. The first stage is based on the unlabeled dependency pars- ing models described by McDonald …...

This paper presents an online algorithm for dependency parsing problems. We propose an adaptation of the passive and aggressive online learning algorithm to the dependency parsing domain. We evaluate the …...

This paper describes SYSTRAN submissions for the shared task of the third Workshop on Statistical Machine Translation at ACL. Our main contribution consists in a French-English statistical model trained without …...

. We investigated the effects of processing Swedish compounds for phrase-based SMT between Swedish and English. Compounds were split in a pre-processing step using an unsupervised empirical method. After translation …...

In this paper, we provide descriptive and empirical approaches to effectively ex- tracting underlying dependencies among parsing errors. In the descriptive ap- proach, we define some combinations of error patterns …...

We present an implicit discourse relation classifier in the Penn Discourse Treebank (PDTB). Our classifier considers the con- text of the two arguments, word pair infor- mation, as well as …...

AbstractIn this paper, we present a detailed and critical analysis of the behaviour of the CasEN named entity recognition system during the French Ester2 evaluation campaign. In this project, CasEN …...

AbstractIn the paper we investigate the impact of data size on a Word Sense Disambiguation task (WSD). We question the assumption that the knowledge acquisition bottleneck, which is known as …...

AbstractOne problem in statistical machine translation (SMT) is that the output often is ungrammatical. To address this issue, we have investigated the use of a grammar checker for two purposes …...

AbstractThis paper presents work on the evaluation of online available machine translation (MT) service, i.e. Google Translate, for English-Croatian language pair in the domain of legislation. The total set of …...

AbstractThis article summarizes the evaluation process of an interface under development to consult an oral corpus of learners of Spanish as a Foreign Language. The databank comprises 40 interviews with …...

AbstractWe present the ongoing development of MCG, a linguistically deep and precise grammar for Mandarin Chinese together with its accompanying treebank, both based on the linguistic framework of HPSG, and …...

AbstractError analysis is a means to assess machine translation output in qualitative terms, which can be used as a basis for the generation of error profiles for different systems. As …...

Many NLP tasks rely on accurate statis- tics from large corpora. Tracking com- plete statistics is memory intensive, so recent work has proposed using compact approx- imate “sketches” of frequency …...

We describe TerrorCat, a submission to this year’s metrics shared task. It is a ma- chine learning-based metric that is trained on manual ranking data from WMT shared tasks 2008–2012. …...

Learning from errors is a crucial aspect of improving expertise. Based on this no- tion, we discuss a robust statistical frame- work for analysing the impact of different error types …...

We present an architecture and implemen- tation of a system that builds structured test suites for concept recognition systems. The system applies provided test case def- initions to a target …...

In this paper, we propose a new method for effective error analysis of machine translation (MT) systems. In previous work on error analysis of MT, error trends are often shown …...

AbstractCurrent supervised parsers are limited by the size of their labelled training data, making improving them with unlabelled data an important goal. We show how a state-of-the-art CCG parser can …...

AbstractIn order to improve the symbiosis between machine translation (MT) system and post-editor, it is not enough to know that the output of one system is better than the output …...

Part-of-Speech (POS) tagging is a key step in many NLP algorithms. However, tweets are difficult to POS tag because there are many phenomena that frequently appear in Twitter that are …...

Discourse deixis is a linguistic phenomenon in which pronouns have verbal or clausal, rather than nominal, antecedents. Studies have esti- mated that between 5% and 10% of pronouns in non-conversational …...

Unstructured clinical notes are rich sources for valuable patient information. Information extraction techniques can be employed to extract this valuable information, which in turn can be used to discover new …...

Argumentation mining obviously involves finding support relations between statements, but many interesting instances of argumen- tation also contain counter-considerations, which the author mentions in order to preempt possible objections by …...

In this work, we explore how learners can infer second-language noun meanings in the context of their native language. Mo- tivated by an interest in building interac- tive tools for …...

AbstractPart-of-Speech(POS) tagging is a key step in many NLP algorithms. However, tweets are difficult to POS tag because they are short, are not always written maintaining formal grammar and proper …...

AbstractIn this paper we present our work on the usage of lexical resources for the Machine Translation English and Malayalam. We describe a comparative performance between different Statistical Machine Translation …...

AbstractThe present article reports on efforts to improve the translation accuracy of a corpus―based Machine Translation (MT) system. In order to achieve that, an error analysis performed on past translation …...

AbstractWe extend classic review mining work by building a binary classifier that predicts whether a review of a documentary film was written by an expert or a layman with 90.70% …...

AbstractIn this paper, we address the problem of Machine Translation (MT) for a specialised domain in a language pair for which only a very small domain-specific parallel corpus is available. …...

Understanding the nature of the errors of a machine learning system is often difficult for multiclass classification problems with a large number of classes. This is true even more so …...

AbstractWe present a new framework to induce an in-domain phrase table from in-domain monolingual data that can be used to adapt a general-domain statistical machine translation system to the targeted …...

Named Entity Linking (NEL) and relation extraction forms the backbone of Knowledge Base Population tasks. The recent rise of large open source Knowledge Bases and the continuous focus on improving …...

In this paper we present Multilingual IsA (MIsA), which is a collection of hypernymy relations in five languages (i.e., English, Spanish, French, Italian and Dutch) extracted from the corresponding full …...

AbstractThe most approaches to Knowledge Base Question Answering are based on semantic parsing. In this paper, we address the problem of learning vector representations for complex semantic parses that consist …...

AbstractIn this paper, we present a new approach for the evaluation, error analysis, and interpretation of supervised and unsupervised Paraphrase Identification (PI) systems. Our evaluation framework makes use of a …...

AbstractAutomated fact-checking based on machine learning is a promising approach to identify false information distributed on the web. In order to achieve satisfactory performance, machine learning methods require a large …...

AbstractThis study is an attempt to contribute to documentation and revitalization efforts of endangered Laz language, a member of South Caucasian language family mainly spoken on northeastern coastline of Turkey. …...

AbstractThough error analysis is crucial to understanding and improving NLP models, the common practice of manual, subjective categorization of a small sample of errors can yield biased and incomplete conclusions. …...

The Tibetan language, despite being spoken by 8 million people, is a low- resource language in NLP terms, and research to develop NLP tools and re- sources for the language …...

AbstractWe conduct a manual error analysis of the CoNLL-SIGMORPHON Shared Task on Morphological Reinflection. This task involves natural language generation: systems are given a word in citation form (e.g., hug) …...

AbstractRecently, pre-training contextualized encoders with language model (LM) objectives has been shown an effective semi-supervised method for structured prediction. In this work, we empirically explore an alternative pre-training method for …...

AbstractThis paper introduces the first corpus for Automatic Post-Editing of English and a low-resource language, Brazilian Portuguese. The source English texts were extracted from the WebNLG corpus and automatically translated …...

AbstractIn this paper, we present a modified version of the CBOW algorithm implemented in the fastText framework. Our modified algorithm, CBOW-tag builds a vector space model that includes the representation …...

AbstractInformational bias is bias conveyed through sentences or clauses that provide tangential, speculative or background information that can sway readers’ opinions towards entities. By nature, informational bias is context-dependent, but …...

AbstractWe performed a detailed error analysis in domain-specific neural machine translation (NMT) for the English and Japanese language pair with fine-grained manual annotation. Despite its importance for advancing NMT technologies, …...

AbstractPunning is a creative way to make conversation enjoyable and literary writing elegant. In this paper, we focus on the task of generating a pun sentence given a pair of …...

AbstractThe Slovene language department of the European Commission Directorate-General for Translation has always been an early adopter of new developments in the area of machine translation. In 2018, the department …...

AbstractMachine Translation (MT) is one of the most important natural language processing applications. Independently of the applied MT approach, a MT system automatically generates an equivalent version (in some target …...

AbstractThe focus of a negation is the set of tokens intended to be negated, and a key component for revealing affirmative alternatives to negated utterances. In this paper, we experiment …...

Due to a lack of annotated data, theories of historical syntax are often based on very small, manually compiled data sets. To enable the empirical evaluation of existing hypotheses, the …...

AbstractWe evaluate two common conjectures in error analysis of NLP models: (i) Morphology is predictive of errors; and (ii) the importance of morphology increases with the morphological complexity of a …...

AbstractWhile counterfactual examples are useful for analysis and training of NLP models, current generation methods either rely on manual labor to create very few counterfactuals, or only instantiate limited types …...

AbstractThere has been significant progress in the field of sentiment analysis. However, aspect-based sentiment analysis (ABSA) has not been explored in the Japanese language even though it has a huge …...

AbstractNeural coreference resolution models trained on one dataset may not transfer to new, low-resource domains. Active learning mitigates this problem by sampling a small subset of data for annotators to …...

AbstractWe present the first fully trainable semantic parser for English, German, Italian, and Dutch discourse representation structures (DRSs) that is competitive in accuracy with recent sequence-to-sequence models and at the …...

AbstractSocial media often serves as a breeding ground for various hateful and offensive content. Identifying such content on social media is crucial due to its impact on the race, gender, …...

AbstractThis paper describes the University of Helsinki submission to the SemEval 2022 task on multilingual idiomaticity detection. Our system utilizes several models made available by HuggingFace, along with the baseline …...

AbstractWith the rapid development of deep learning, Seq2Seq paradigm has become prevalent for end-to-end data-to-text generation, and the BLEU scores have been increasing in recent years. However, it is widely …...

AbstractNeural machine translation (NMT) systems are nowadays essential components of professional translation workflows. Consequently, human translators are increasingly working as post-editors for machine-translated content. The NWO-funded InDeep project aims to …...

ive Summaries Tim Fischer Steffen Remus Chris Biemann Language Technology Group Department of Informatics Universität Hamburg, Germany {firstname.lastname}@uni-hamburg.de Abstract Recent abstractive summarization systems fail to generate factually consistent – faithful …...

AbstractWe present a corrected version of a subset of the FactBank data set. Previously published results on FactBank are no longer valid. We perform experiments on FactBank using multiple training …...

AbstractWe present our systems and findings for the iSarcasmEval: Intended Sarcasm Detection In English and Arabic at SEMEVAL 2022. Specifically we take part in Subtask A for the English language. …...

AbstractIn this study, we propose a morpheme-based scheme for Korean dependency parsing and adopt the proposed scheme to Universal Dependencies. We present the linguistic rationale that illustrates the motivation and …...

AbstractIn this paper, we focus on how current Machine Translation (MT) engines perform on the translation of emotion-loaded texts by evaluating outputs from Google Translate according to a framework proposed …...

AbstractIn recent years, deep neural networks (DNNs) have achieved state-of-the-art performance on a wide range of tasks. However, limitations in interpretability have hindered their applications in the real world. This …...

AbstractIn the context of an epidemiological study involving multilingual social media, this paper reports on the ability of machine translation systems to preserve content relevant for a document classification task …...

AbstractAs 3rd-person pronoun usage shifts to include novel forms, e.g., neopronouns, we need more research on identity-inclusive NLP. Exclusion is particularly harmful in one of the most popular NLP applications, …...