Лемматизация

Accurate lemmatization of German nouns mandates the use of a lexicon. Compre- hensive lexicons, however, are expensive to build and maintain. We present a self- learning lemmatizer capable of automati- …...

We present a set of experiments involving sentence classification, addressing issues of representation and feature selection, and we compare our findings with simi- lar results from work on the more …...

We describe a word alignment platform which ensures text pre-processing (to- kenization, POS-tagging, lemmatization, chunking, sentence alignment) as re- quired by an accurate word alignment. The platform combines two different …...

String transformation, which maps a source string s into its desirable form t∗, is related to various applications including stemming, lemmatization, and spelling correction. The essential and important step for …...

Annotation of training data for machine learning is often a laborious and costly process. In Active Learning (AL), crite- ria are investigated that allow ordering the unannotated data in such …...

This paper explores the influence of text pre- processing techniques on plagiarism detection. We examine stop-word removal, lemmatization, number replacement, synonymy recognition, and word generalization. We also look into the …...

AbstractThis paper reports on the annotation of a corpus of 1 million words with four semantic annotation layers, including named entities, co- reference relations, semantic roles and spatial and temporal …...

AbstractWe present work in progress aiming to build tools for the normalization of User-Generated Content (UGC). As we will see, the task requires the revisiting of the initial steps of …...

AbstractThis paper presents a linguistic processing pipeline for Bulgarian including morphological analysis, lemmatization and syntactic analysis of Bulgarian texts. The morphological analysis is performed by three modules ― two statistical-based …...

AbstractPOS Taggers typically fail to correctly tag grammatical neologisms: for known words, a tagger will only take known tags into account, and hence discard any possibility that the word is …...

AbstractThis article describes the collecting, processing and validation of a large balanced corpus for Romanian. The annotation types and structure of the corpus are briefly reviewed. It was constructed at …...

AbstractAlthough in recent years numerous forms of Internet communication ― such as e-mail, blogs, chat rooms and social network environments ― have emerged, balanced corpora of Internet speech with trustworthy …...

The many differences between Dialectal Ara- bic and Modern Standard Arabic (MSA) pose a challenge to the majority of Arabic natural language processing tools, which are designed for MSA. In …...

We present PurePos, an open-source HMM-based automatic morphological an- notation tool. PurePos can perform tag- ging and lemmatization at the same time, it is very fast to train, with the …...

AbstractThis paper introduces a set of freely available, open-source tools for Turkish that are built around TRmorph, a morphological analyzer introduced earlier in Coltekin (2010). The article first provides an …...

AbstractIn this paper a Moses SMT toolkit-based language-independent complete morphological annotation tool is presented called HuLaPos2. Our system performs PoS tagging and lemmatization simultaneously. Amongst others, the algorithm used is …...

AbstractThe task of corpus-dictionary linkage (CDL) is to annotate each word in a corpus with a link to an appropriate dictionary entry that documents the sense and usage of the …...

AbstractThis paper attempts a preliminary interpretation of the occurrence of different types of linguistic constructs in the manually-annotated Polish Coreference Corpus by providing analyses of various statistical properties related to …...

AbstractWe present SETimes.HR ― the first linguistically annotated corpus of Croatian that is freely available for all purposes. The corpus is built on top of the SETimes parallel corpus of …...

AbstractThe paper overviews the SYN series of synchronic corpora of written Czech compiled within the framework of the Czech National Corpus project. It describes their design and processing with a …...

Lemmatization for the Sumerian language, compared to the modern languages, is much more challenging due to that it is a long dead language, highly skilled language experts are extremely scarce …...

We present LEMMING, a modular log- linear model that jointly models lemmati- zation and tagging and supports the inte- gration of arbitrary global features. It is trainable on corpora annotated …...

AbstractPreprocessing is a preliminary step in many fields including IR and NLP. The effect of basic preprocessing settings on English for text summarization is well-studied. However, there is no such …...

AbstractCEPLEXicon (version 1.1) is a child lexicon resulting from the automatic tagging of two child corpora: the corpus Santos (Santos, 2006; Santos et al. 2014) and the corpus Child ― …...

This paper presents an extension of neu- ral machine translation (NMT) model to incorporate additional word-level linguis- tic factors. Adding such linguistic fac- tors may be of great benefits to …...

We present several methods for stemming and lemmatization based on discrimina- tive string transduction. We exploit the paradigmatic regularity of semi-structured inflection tables to identify stems in an un- supervised …...

AbstractIn this paper we present a rule-based method for multi-word term extraction that relies on extensive lexical resources in the form of electronic dictionaries and finite-state transducers for modelling various …...

AbstractThe article describes the current status of a large national project, CoRoLa, aiming at building a reference corpus for the contemporary Romanian language. Unlike many other national corpora, CoRoLa contains …...

AbstractAutomatic natural language processing of large texts often presents recurring challenges in multiple languages: even for most advanced tasks, the texts are first processed by basic processing steps – from …...

AbstractLanguage resources, such as corpora, are important for various natural language processing tasks. Urdu has millions of speakers around the world but it is under-resourced in terms of standard evaluation …...

In this paper we describe the complexity of building a lemmatizer for Arabic which has a rich and complex morphology, and show some differences between lemmatization and surface stemming, i.e. …...

AbstractDistributional word representations (often referred to as word embeddings) are omnipresent in modern NLP. Early work has focused on building representations for word types, and recent studies show that lemmatization …...

We present TIGER 2.2-doc – a new set of annotations for the German TIGER corpus. The set moves the corpus to a document level. It includes a full mapping of …...

AbstractWe present a neural transition-based model that uses a simple set of edit actions (copy, delete, insert) for morphological transduction tasks such as inflection generation, lemmatization, and reinflection. In a …...

Tokenization, or morphological analysis, is a fundamental and important technology for processing a Japanese text, especially for in- dustrial applications. However, we often face many obstacles, such as the inconsistency …...

AbstractEnglish verbs have multiple forms. For instance, talk may also appear as talks, talked or talking, depending on the context. The NLP task of lemmatization seeks to map these diverse …...

AbstractA large percentage of computational tools are concentrated in a very small subset of the planet’s languages. Compounding the issue, many languages lack the high-quality linguistic annotation necessary for the …...

AbstractThis paper describes Charles University sub-mission for Terminology translation shared task at WMT21. The objective of this task is to design a system which translates certain terms based on a …...

AbstractLemmatization aims to reduce the sparse data problem by relating the inflected forms of a word to its dictionary form. Most prior work on ML based lemmatization has focused on …...

AbstractTexts written in Old Literary Finnish represent the first literary work ever written in Finnish starting from the 16th century. There have been several projects in Finland that have digitized …...

AbstractTo facilitate corpus searches by classicists as well as to reduce data sparsity when training models, we focus on the automatic lemmatization of ancient Greek inscriptions, which have not received …...

AbstractIn this paper we present the Serbian part of the ELTeC multilingual corpus of novels written in the time period 1840-1920. The corpus is being built in order to test …...

Document Clustering aims to group similar documents based on the distance among them. A wide range of distance measures are available in the literature, and selecting an appropriate distance function …...

AbstractIn this paper we present the final result of a project focused on Tunisian Arabic encoded in Arabizi, the Latin-based writing system for digital conversations. The project led to the …...

AbstractWe introduce Textstar, a graph-based summarization and keyphrase extraction system that builds a document graph using only lemmatization and POS tagging. The document graph aggregates connections between lemma and sentence …...

AbstractWe present ZAEBUC, an annotated Arabic-English bilingual writer corpus comprising short essays by first-year university students at Zayed University in the United Arab Emirates. We describe and discuss the various …...

AbstractRecent advances in NLP have led to the creation of powerful language models for many languages including Ancient Greek and Latin. While prior work on Classical languages unanimously uses BERT, …...