FastText
Related Papers
Advances in Pre-Training Distributed Word Representations
Many Natural Language Processing applications nowadays rely on pre-trained word representations estimated from large text corpora such as news collections, Wikipedia and Web Crawl. In this paper, we show how …...
Evaluation of Croatian Word Embeddings
Croatian is poorly resourced and highly inflected language from Slavic language family. Nowadays, research is focusing mostly on English. We created a new word analogy dataset based on the original …...
Evaluation of vector embedding models in clustering of text documents
AbstractThe paper presents an evaluation of word embedding models in clustering of texts in the Polish language. Authors verified six different embedding models, starting from widely used word2vec, across fastText …...
INF-HatEval at SemEval-2019 Task 5: Convolutional Neural Networks for Hate Speech Detection Against Women and Immigrants on Twitter
AbstractIn this paper, we describe our approach to detect hate speech against women and immigrants on Twitter in a multilingual context, English and Spanish. This challenge was proposed by the …...
INGEOTEC at SemEval-2019 Task 5 and Task 6: A Genetic Programming Approach for Text Classification
AbstractThis paper describes our participation in HatEval and OffensEval challenges for English and Spanish languages. We used several approaches, B4MSA, FastText, and EvoMSA. Best results were achieved with EvoMSA, which …...
Multilingual Constituency Parsing with Self-Attention and Pre-Training
AbstractWe show that constituency parsing benefits from unsupervised pre-training across a variety of languages and a range of pre-training conditions. We first compare the benefits of no pre-training, fastText, ELMo, …...
NLPR@SRPOL at SemEval-2019 Task 6 and Task 5: Linguistically enhanced deep learning offensive sentence classifier
AbstractThe paper presents a system developed for the SemEval-2019 competition Task 5 hat- Eval Basile et al. (2019) (team name: LU Team) and Task 6 OffensEval Zampieri et al. (2019b) …...
ParallelDots at SemEval-2019 Task 3: Domain Adaptation with feature embeddings for Contextual Emotion Analysis
AbstractThis paper describes our proposed system & experiments performed to detect contextual emotion in texts for SemEval 2019 Task 3. We exploit sentiment information, syntactic patterns & semantic relatedness to …...
Semantic Role Labeling with Pretrained Language Models for Known and Unknown Predicates
AbstractWe build the first full pipeline for semantic role labelling of Russian texts. The pipeline implements predicate identification, argument extraction, argument classification (labeling), and global scoring via integer linear programming. …...
Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation
AbstractPretrained contextual and non-contextual subword embeddings have become available in over 250 languages, allowing massively multilingual NLP. However, while there is no dearth of pretrained embeddings, the distinct lack of …...
Tagger for Polish Computer Mediated Communication Texts
AbstractIn this paper we present a morpho-syntactic tagger dedicated to Computer-mediated Communication texts in Polish. Its construction is based on an expanded RNN-based neural network adapted to the work on …...
CBOW-tag: a Modified CBOW Algorithm for Generating Embedding Models from Annotated Corpora
AbstractIn this paper, we present a modified version of the CBOW algorithm implemented in the fastText framework. Our modified algorithm, CBOW-tag builds a vector space model that includes the representation …...
DeftPunk at SemEval-2020 Task 6: Using RNN-ensemble for the Sentence Classification.
AbstractThis paper describes participation in DeftEval 2020 (part of SemEval sharing task competition), and is focused on the sentence classification. Our approach to the task was to create an ensemble …...
DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks with Self Attention for Arabic Offensive Language Detection
AbstractThis paper describes our team work and submission for the SemEval 2020 (Sub-Task A) “Offensive Eval: Identifying and Categorizing Offensive Arabic Language in Arabic Social Media”. Our two baseline models …...
Go Simple and Pre-Train on Domain-Specific Corpora: On the Role of Training Data for Text Classification
AbstractPre-trained language models provide the foundations for state-of-the-art performance across a wide range of natural language processing tasks, including text classification. However, most classification datasets assume a large amount labeled …...
HyperText: Endowing FastText with Hyperbolic Geometry
AbstractNatural language data exhibit tree-like hierarchical structures such as the hypernym-hyponym hierarchy in WordNet. FastText, as the state-of-the-art text classifier based on shallow neural network in Euclidean space, may not …...
LT3 at SemEval-2020 Task 9: Cross-lingual Embeddings for Sentiment Analysis of Hinglish Social Media Text
AbstractThis paper describes our contribution to the SemEval-2020 Task 9 on Sentiment Analysis for Code-mixed Social Media Text. We investigated two approaches to solve the task of Hinglish sentiment analysis. …...
NLP_Passau at SemEval-2020 Task 12: Multilingual Neural Network for Offensive Language Detection in English, Danish and Turkish
AbstractThis paper describes a neural network (NN) model that was used for participating in the OffensEval, Task 12 of the SemEval 2020 workshop. The aim of this task is to …...
Word Sense Disambiguation for 158 Languages using Word Embeddings Only
AbstractDisambiguation of word senses in context is easy for humans, but is a major challenge for automatic approaches. Sophisticated supervised and knowledge-based models were developed to solve this task. However, …...
A Study on Using Semantic Word Associations to Predict the Success of a Novel
AbstractMany new books get published every year, and only a fraction of them become popular among the readers. So the prediction of a book success can be a very useful …...
Dependency Enhanced Contextual Representations for Japanese Temporal Relation Classification
Recently, quite a few studies have been progressive for temporal relation extraction, which is an important work used in sev- eral natural language processing applications. However, less concentration has been …...
Multi-Label Classification of Chinese Humor Texts Using Hypergraph Attention Networks
AbstractWe use Hypergraph Attention Networks (HyperGAT) to recognize multiple labels of Chinese humor texts. We firstly represent a joke as a hypergraph. The sequential hyperedge and semantic hyperedge structures are …...
Revisiting Tri-training of Dependency Parsers
AbstractWe compare two orthogonal semi-supervised learning techniques, namely tri-training and pretrained word embeddings, in the task of dependency parsing. We explore language-specific FastText and ELMo embeddings and multilingual BERT embeddings. …...
Task-adaptive Pre-training of Language Models with Word Embedding Regularization
Pre-trained language models (PTLMs) ac- quire domain-independent linguistic knowl- edge through pre-training with massive textual resources. Additional pre-training is effective in adapting PTLMs to domains that are not well covered …...
Embeddings models for Buddhist Sanskrit
AbstractThe paper presents novel resources and experiments for Buddhist Sanskrit, broadly defined here including all the varieties of Sanskrit in which Buddhist texts have been transmitted. We release a novel …...
Finnish Hate-Speech Detection on Social Media Using CNN and FinBERT
AbstractThere has been a lot of research in identifying hate posts from social media because of their detrimental effects on both individuals and society. The majority of this research has …...
HeLI-OTS, Off-the-shelf Language Identifier for Text
AbstractThis paper introduces HeLI-OTS, an off-the-shelf text language identification tool using the HeLI language identification method. The HeLI-OTS language identifier is equipped with language models for 200 languages and licensed …...
Imputing Out-of-Vocabulary Embeddings with LOVE Makes LanguageModels Robust with Little Cost
AbstractState-of-the-art NLP systems represent inputs with word embeddings, but these are brittle when faced with Out-of-Vocabulary (OOV) words. To address this issue, we follow the principle of mimick-like models to …...
Task-dependent Optimal Weight Combinations for Static Embeddings
AbstractA variety of NLP applications use word2vec skip-gram, GloVe, and fastText word embeddings. These models learn two sets of embedding vectors, but most practitioners use only one of them, or …...
Leverage Points in Modality Shifts: Comparing Language-only and Multimodal Word Representations
AbstractMultimodal embeddings aim to enrich the semantic information in neural representations of language compared to text-only models. While different embeddings exhibit different applicability and performance on downstream tasks, little is …...
Lexical Complexity Prediction using Word Embeddings
Lexical complexity is crucial for reading comprehension. In the past, research work of lexical complexity prediction mainly focuses on differentiating the complexity difference between two words. Moreover, most of the …...
Word Sense Disambiguation for Automatic Translation of Medical Dialogues into Pictographs
AbstractWord sense disambiguation is an NLP task embedded in different applications. We propose to evaluate its contribution to the automatic translation of French texts into pictographs, in the context of …...