TF IDF
Related Papers
A Simple Probabilistic Approach to Classification and Routing
Several classification and routing methods were im- plemented and compared. The experiments used FBIS documents from four categories, and the measures used were the ff.idf and Cosine similarity measures, and …...
Extending the BLEU MT Evaluation Method with Frequency Weightings
We present the results of an experiment on extending the automatic method of Machine Translation evaluation BLUE with statistical weights for lexical items, such as tf.idf scores. We show that …...
Discretization Based Learning for Information Retrieval
We approached the problem as learning how to order documents by estimated relevance with respect to a user query. Our support vector machines based classifier learns from the relevance judgments …...
Low Cost Portability for Statistical Machine Translation based on N-gram Frequency and TF-IDF
Statistical machine translation relies heavily on the available training data. In some cases it is necessary to limit the amount of training data that can be created for or actually …...
PLSI Utilization for Automatic Thesaurus Construction
. When acquiring synonyms from large corpora, it is important to deal not only with such surface information as the context of the words but also their latent semantics. This …...
Robust Textual Inference via Graph Matching
We present a system for deciding whether a given sentence can be inferred from text. Each sentence is represented as a directed graph (extracted from a depen- dency parser) in …...
Statistical Machine Translation for Query Expansion in Answer Retrieval
We present an approach to query expan- sion in answer retrieval that uses Statisti- cal Machine Translation (SMT) techniques to bridge the lexical gap between ques- tions and answers. SMT-based …...
Learning Term-weighting Functions for Similarity Measures
Measuring the similarity between two texts is a fundamental problem in many NLP and IR applications. Among the ex- isting approaches, the cosine measure of the term vectors representing the …...
Matching Reviews to Objects using a Language Model
We develop a general method to match un- structured text reviews to a structured list of objects. For this, we propose a lan- guage model for generating reviews that incorporates …...
Multi-Document Summarisation Using Generic Relation Extraction
Experiments are reported that investi- gate the effect of various source docu- ment representations on the accuracy of the sentence extraction phase of a multi- document summarisation task. A novel …...
Unsupervised Approaches for Automatic Keyword Extraction Using Meeting Transcripts
This paper explores several unsupervised ap- proaches to automatic keyword extraction using meeting transcripts. In the TFIDF (term frequency, inverse document frequency) weighting framework, we incorporated part- of-speech (POS) information, …...
A Study of Information Retrieval Weighting Schemes for Sentiment Analysis
Most sentiment analysis approaches use as baseline a support vector machines (SVM) classifier with binary unigram weights. In this paper, we explore whether more sophisticated feature weighting schemes from Information …...
BUAP: An Unsupervised Approach to Automatic Keyphrase Extraction from Scientific Articles
In this paper, it is presented an unsuper- vised approach to automatically discover the latent keyphrases contained in scien- tific articles. The proposed technique is constructed on the basis of …...
Effective Use of Linguistic Features for Sentiment Analysis of Korean
. In this paper, we propose a new linguistic approach for sentiment analysis of Korean. In order to overcome shortcomings of previous works confined to statistical methods, we make effective …...
Fairy Tale Corpus Organization Using Latent Semantic Mapping and an Item-to-item Top-n Recommendation Algorithm
AbstractIn this paper we present a fairy tale corpus that was semantically organized and tagged. The proposed method uses latent semantic mapping to represent the stories and a top-n item-to-item …...
Using NLP Methods for the Analysis of Rituals
AbstractThis paper gives an overview of an interdisciplinary research project that is concerned with the application of computational linguistics methods to the analysis of the structure and variance of rituals, …...
A Minimally Supervised Approach for Detecting and Ranking Document Translation Pairs
We describe an approach for generating a ranked list of candidate document transla- tion pairs without the use of bilingual dic- tionary or machine translation system. We developed this approach …...
An Improvement in Cross-Language Document Retrieval Based on Statistical Models
This paper presents a proposed method integrated with three statistical models including Translation model, Query generation model and Document retrieval model for cross-language document retrieval. Given a certain document in …...
DSim, a Danish Parallel Corpus for Text Simplification
AbstractWe present DSim, a new sentence aligned Danish monolingual parallel corpus extracted from 3701 pairs of news telegrams and corresponding professionally simplified short news articles. The corpus is intended for …...
In Your Eyes: Identifying Clichés in Song Lyrics
We investigated methods for the discov- ery of clich´es from song lyrics. Trigrams and rhyme features were extracted from a collection of lyrics and ranked using term-weighting techniques such as …...
Investigating Verbal Intelligence Using the TF-IDF Approach
AbstractIn this paper we investigated differences in language use of speakers yielding different verbal intelligence when they describe the same event. The work is based on a corpus containing descriptions …...
Measuring the Structural Importance through Rhetorical Structure Index
In this paper, we propose a novel Rhetorical Structure Index (RSI) to measure the struc- tural importance of a word or a phrase. Un- like TF-IDF and other content-driven mea- …...
A Keyword-based Monolingual Sentence Aligner in Text Simplification
We introduce a method for learning to align sentences in monolingual parallel articles for text simplification. In our approach, word keyness is integrated to prefer aligning essential words in sentences. …...
A LDA-Based Topic Classification Approach From Highly Imperfect Automatic Transcriptions
AbstractAlthough the current transcription systems could achieve high recognition performance, they still have a lot of difficulties to transcribe speech in very noisy environments. The transcription quality has a direct …...
SAIL-GRS: Grammar Induction for Spoken Dialogue Systems using CF-IRF Rule Similarity
The SAIL-GRS system is based on a widely used approach originating from in- formation retrieval and document index- ing, the TF-IDF measure. In this im- plementation for spoken dialogue system …...
A Graph-based Readability Assessment Method using Word Coupling
This paper proposes a graph-based read- ability assessment method using word coupling. Compared to the state-of-the- art methods such as the readability for- mulae, the word-based and feature-based methods, our …...
Semi-Supervised Bootstrapping of Relationship Extractors with Distributional Semantics
Semi-supervised bootstrapping techniques for relationship extraction from text iter- atively expand a set of initial seed rela- tionships while limiting the semantic drift. We research bootstrapping for relationship extraction using …...
Similarity Metrics for Clustering PubMed Abstracts for Evidence Based Medicine
s for Evidence Based Medicine Hamed Hassanzadeh† Diego Moll´a♦ Tudor Groza‡ Anthony Nguyen♮ Jane Hunter† †School of ITEE, The University of Queensland, Brisbane, QLD, Australia ♦Department of Computing, Macquarie University, …...
Ambient Search: A Document Retrieval System for Speech Streams
AbstractWe present Ambient Search, an open source system for displaying and retrieving relevant documents in real time for speech input. The system works ambiently, that is, it unobstructively listens to …...
Deftor at SemEval-2016 Task 14: Taxonomy enrichment using definition vectors
In this paper we describe the participation of the Joint Research Centre, EC, in task 14 - Semantic Taxonomy Enrichment at SemEval 2016. The algorithm which we propose trans- forms …...
Developing a Dataset for Evaluating Approaches for Document Expansion with Images
AbstractMotivated by the adage that a “picture is worth a thousand words” it can be reasoned that automatically enriching the textual content of a document with relevant images can increase …...
Forecasting Emerging Trends from Scientific Literature
AbstractText analysis methods for the automatic identification of emerging technologies by analyzing the scientific publications, are gaining attention because of their socio-economic impact. The approaches so far have been mainly …...
Automatic Categorization of Tagalog Documents Using Support Vector Machines
Automatic document classification is now a growing research topic in Natural Language Processing. Several techniques were incorporated to build a classifier that can categorize documents written in specific languages into …...
Classifying Illegal Activities on Tor Network Based on Web Textual Contents
AbstractThe freedom of the Deep Web offers a safe place where people can express themselves anonymously but they also can conduct illegal activities. In this paper, we present and make …...
UdL at SemEval-2017 Task 1: Semantic Textual Similarity Estimation of English Sentence Pairs Using Regression Model over Pairwise Features
AbstractThis paper describes the model UdL we proposed to solve the semantic textual similarity task of SemEval 2017 workshop. The track we participated in was estimating the semantics relatedness of …...
UW-FinSent at SemEval-2017 Task 5: Sentiment Analysis on Financial News Headlines using Training Dataset Augmentation
AbstractThis paper discusses the approach taken by the UWaterloo team to arrive at a solution for the Fine-Grained Sentiment Analysis problem posed by Task 5 of SemEval 2017. The paper …...
Tw-StAR at SemEval-2018 Task 1: Preprocessing Impact on Multi-label Emotion Classification
AbstractIn this paper, we describe our contribution in SemEval-2018 contest. We tackled task 1 “Affect in Tweets”, subtask E-c “Detecting Emotions (multi-label classification)”. A multilabel classification system Tw-StAR was developed …...
Using Statistical and Semantic Models for Multi-Document Summarization
. We report a series of experiments with different semantic models on top of various statistical models for extractive text summarization. Though statistical models may better cap- ture word co-occurrences …...
A Semantic Cover Approach for Topic Modeling
AbstractWe introduce a novel topic modeling approach based on constructing a semantic set cover for clusters of similar documents. Specifically, our approach first clusters documents using their Tf-Idf representation, and …...
GSI-UPM at SemEval-2019 Task 5: Semantic Similarity and Word Embeddings for Multilingual Detection of Hate Speech Against Immigrants and Women on Twitter
AbstractThis paper describes the GSI-UPM system for SemEval-2019 Task 5, which tackles multilingual detection of hate speech on Twitter. The main contribution of the paper is the use of a …...
JCTICOL at SemEval-2019 Task 6: Classifying Offensive Language in Social Media using Deep Learning Methods, Word/Character N-gram Features, and Preprocessing Methods
AbstractIn this paper, we describe our submissions to SemEval-2019 task 6 contest. We tackled all three sub-tasks in this task “OffensEval - Identifying and Categorizing Offensive Language in Social Media”. …...
SSN_NLP at SemEval-2019 Task 6: Offensive Language Identification in Social Media using Traditional and Deep Machine Learning Approaches
AbstractOffensive language identification (OLI) in user generated text is automatic detection of any profanity, insult, obscenity, racism or vulgarity that degrades an individual or a group. It is helpful for …...
A Corpus of Spanish Political Speeches from 1937 to 2019
AbstractThis paper documents a corpus of political speeches in Spanish. The documents in the corpus belong to the Christmas speeches that have been delivered yearly by the head of state …...
A French Corpus for Event Detection on Twitter
AbstractWe present Event2018, a corpus annotated for event detection tasks, consisting of 38 million tweets in French (retweets excluded) including more than 130,000 tweets manually annotated by three annotators as …...
Data Selection for Bilingual Lexicon Induction from Specialized Comparable Corpora
AbstractNarrow specialized comparable corpora are often small in size. This particularity makes it difficult to build efficient models to acquire translation equivalents, especially for less frequent and rare words. One …...
Dense Passage Retrieval for Open-Domain Question Answering
AbstractOpen-domain question answering relies on efficient passage retrieval to select candidate contexts, where traditional sparse vector space models, such as TF-IDF or BM25, are the de facto method. In this …...
Ferryman as SemEval-2020 Task 5: Optimized BERT for Detecting Counterfactuals
AbstractThe main purpose of this article is to state the effect of using different methods and models for counterfactual determination and detection of causal knowledge. Nowadays, counterfactual reasoning has been …...
Ferryman at SemEval-2020 Task 3: Bert with TFIDF-Weighting for Predicting the Effect of Context in Word Similarity
AbstractWord similarity is widely used in machine learning applications like searching engine and recommendation. Measuring the changing meaning of the same word between two different sentences is not only a …...
Funny3 at SemEval-2020 Task 7: Humor Detection of Edited Headlines with LSTM and TFIDF Neural Network System
AbstractThis paper presents a neural network system where we participate in the first task of SemEval-2020 shared task 7 “Assessing the Funniness of Edited News Headlines”. Our target is to …...
IDS at SemEval-2020 Task 10: Does Pre-trained Language Model Know What to Emphasize?
AbstractWe propose a novel method that enables us to determine words that deserve to be emphasized from written text in visual media, relying only on the information from the self-attention …...
The Devil is in the Details: Evaluating Limitations of Transformer-based Methods for Granular Tasks
AbstractContextual embeddings derived from transformer-based neural language models have shown state-of-the-art performance for various tasks such as question answering, sentiment analysis, and textual similarity in recent years. Extensive work shows …...
UAIC1860 at SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles
AbstractThe “Detection of Propaganda Techniques in News Articles” task at the SemEval 2020 competition focuses on detecting and classifying propaganda, pervasive in news article. In this paper, we present a …...
UoB at SemEval-2020 Task 12: Boosting BERT with Corpus Level Information
AbstractPre-trained language model word representation, such as BERT, have been extremely successful in several Natural Language Processing tasks significantly improving on the state-of-the-art. This can largely be attributed to their …...
Back to the Basics: A Quantitative Analysis of Statistical and Graph-Based Term Weighting Schemes for Keyword Extraction
AbstractTerm weighting schemes are widely used in Natural Language Processing and Information Retrieval. In particular, term weighting is the basis for keyword extraction. However, there are relatively few evaluation studies …...
CDA: a Cost Efficient Content-based Multilingual Web Document Aligner
AbstractWe introduce a Content-based Document Alignment approach (CDA), an efficient method to align multilingual web documents based on content in creating parallel training data for machine translation (MT) systems operating …...
Confiscation Detection of Criminal Judgment Using Text Classification Approach
AbstractAs the system of confiscation becomes more and more perfect, grasping the distribution of the types of confiscations actually announced by the courts will enable you to understand changing of …...
Game-theoretic Vocabulary Selection via the Shapley Value and Banzhaf Index
AbstractThe input vocabulary and the representations learned are crucial to the performance of neural NLP models. Using the full vocabulary results in less explainable and more memory intensive models, with …...
Afaan Oromo Hate Speech Detection and Classification on Social Media
AbstractHate and offensive speech on social media is targeted to attack an individual or group of community based on protected characteristics such as gender, ethnicity, and religion. Hate and offensive …...
BFCAI at SemEval-2022 Task 6: Multi-Layer Perceptron for Sarcasm Detection in Arabic Texts
AbstractThis paper describes the systems submitted to iSarcasm shared task. The aim of iSarcasm is to identify the sarcastic contents in Arabic and English text. Our team participated in iSarcasm …...
Developing and Evaluating a Dataset for How-to Tip Machine Reading at Scale
In this paper, we focus on the task of machine reading at scale within how-to tip machine reading comprehension (MRC). We propose a method for developing a context dataset using …...
Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization
AbstractEfficient k-nearest neighbor search is a fundamental task, foundational for many problems in NLP. When the similarity is measured by dot-product between dual-encoder vectors or L2-distance, there already exist many …...
GetSmartMSEC at SemEval-2022 Task 6: Sarcasm Detection using Contextual Word Embedding with Gaussian model for Irony Type Identification
AbstractSarcasm refers to the use of words that have different literal and intended meanings. It represents the usage of words that are opposite of what is literally said, especially in …...
Predicting Document Coverage for Relation Extraction
AbstractThis paper presents a new task of predicting the coverage of a text document for relation extraction (RE): Does the document contain many relational tuples for a given entity? Coverage …...
Web-API-Based Chatbot Generation with Analysis and Expansion for Training Sentences
AbstractWith Web API technology becoming increasingly mature, how to integrate Web API and Chatbot technology has become an issue of great interest. This study plans to build a semi-automatic method …...