Pearson Correlation
Related Papers
Feature-Based Method for Document Alignment in Comparable News Corpora
In this paper, we present a feature-based me- thod to align documents with similar content across two sets of bilingual comparable cor- pora from daily news texts. We evaluate the …...
Computing and Evaluating Syntactic Complexity Features for Automated Scoring of Spontaneous Non-Native Speech
This paper focuses on identifying, extracting and evaluating features related to syntactic complexity of spontaneous spoken responses as part of an effort to expand the current feature set of an …...
Goodness: A Method for Measuring Machine Translation Confidence
State-of-the-art statistical machine translation (MT) systems have made significant progress towards producing user-acceptable translation output. However, there is still no efficient way for MT systems to inform users which words …...
FCC: Three Approaches for Semantic Textual Similarity
In this paper we describe the three approaches we submitted to the Semantic Textual Similar- ity task of SemEval 2012. The first approach considers to calculate the semantic similar- ity …...
SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity
Semantic Textual Similarity (STS) measures the degree of semantic equivalence between two texts. This paper presents the results of the STS pilot task in Semeval. The training data contained 2000 …...
Weiwei: A Simple Unsupervised Latent Semantics based Approach for Sentence Similarity
The Semantic Textual Similarity (STS) shared task (Agirre et al., 2012) computes the degree of semantic equivalence between two sen- tences.1 We show that a simple unsupervised latent semantics based …...
SSMT:A Machine Translation Evaluation View To Paragraph-to-Sentence Semantic Similarity
This paper presents the system SSMT measuring the semantic similarity between a paragraph and a sentence submitted to the SemEval 2014 task3: Cross-level Se- mantic Similarity. The special difficulty of …...
UoW: NLP techniques developed at the University of Wolverhampton for Semantic Similarity and Textual Entailment
This paper presents the system submit- ted by University of Wolverhampton for SemEval-2014 task 1. We proposed a ma- chine learning approach which is based on features extracted using Typed …...
AMRITA_CEN@SemEval-2015: Paraphrase Detection for Twitter using Unsupervised Feature Learning with Recursive Autoencoders
We explore using recursive autoencoders for SemEval 2015 Task 1: Paraphrase and Seman- tic Similarity in Twitter. Our paraphrase de- tection system makes use of phrase-structure parse tree embeddings that …...
Computing Semantic Text Similarity Using Rich Features
Semantic text similarity (STS) is an essential problem in many Natural Language Pro- cessing tasks, which has drawn a considerable amount of attention by research community in recent years. In …...
Improving Evaluation of Machine Translation Quality Estimation
Quality estimation evaluation commonly takes the form of measurement of the error that exists between predictions and gold standard labels for a particular test set of translations. Issues can arise …...
ROB: Using Semantic Meaning to Recognize Paraphrases
Paraphrase recognition is the task of iden- tifying whether two pieces of natural lan- guage represent similar meanings. This pa- per describes a system participating in the shared task 1 …...
UQeResearch: Semantic Textual Similarity Quantification
This paper presents an approach for estimat- ing the Semantic Textual Similarity of full English sentences as specified in Shared Task 2 of SemEval-2015. The semantic similarity of sentence pairs …...
GWU NLP at SemEval-2016 Shared Task 1: Matrix Factorization for Crosslingual STS
We present a matrix factorization model for learning cross-lingual representations for sentences. Using sentence-aligned corpora, the proposed model learns distributed repre- sentations by factoring the given data into language-dependent factors …...
ISCAS_NLP at SemEval-2016 Task 1: Sentence Similarity Based on Support Vector Regression using Multiple Features
This paper describes our system developed for English Monolingual subtask (STS Core) of SemEval-2016 Task 1: “Semantic Textual Similarity: A Unified Framework for Seman- tic Processing and Evaluation”. We measure …...
SimiHawk at SemEval-2016 Task 1: A Deep Ensemble System for Semantic Textual Similarity
This paper describes the SimiHawk system submission from UMass Lowell for the core Semantic Textual Similarity task at SemEval- 2016. We built four systems: a small feature- based system that …...
Task-Oriented Intrinsic Evaluation of Semantic Textual Similarity
AbstractSemantic Textual Similarity (STS) is a foundational NLP task and can be used in a wide range of tasks. To determine the STS of two texts, hundreds of different STS …...
UWB at SemEval-2016 Task 1: Semantic Textual Similarity using Lexical, Syntactic, and Semantic Information
We present our UWB system for Seman- tic Textual Similarity (STS) task at SemEval 2016. Given two sentences, the system esti- mates the degree of their semantic similarity. We use …...
ECNU at SemEval-2017 Task 1: Leverage Kernel-based Traditional NLP features and Neural Networks to Build a Universal Model for Multilingual and Cross-lingual Semantic Textual Similarity
AbstractTo address semantic similarity on multilingual and cross-lingual sentences, we firstly translate other foreign languages into English, and then feed our monolingual English system with various interactive features. Our system …...
LIM-LIG at SemEval-2017 Task1: Enhancing the Semantic Similarity for Arabic Sentences with Vectors Weighting
AbstractThis article describes our proposed system named LIM-LIG. This system is designed for SemEval 2017 Task1: Semantic Textual Similarity (Track1). LIM-LIG proposes an innovative enhancement to word embedding-based model devoted …...
Pushing the Limits of Translation Quality Estimation
AbstractTranslation quality estimation is a task of growing importance in NLP, due to its potential to reduce post-editing human effort in disruptive ways. However, this potential is currently limited by …...
QLUT at SemEval-2017 Task 1: Semantic Textual Similarity Based on Word Embeddings
AbstractThis paper reports the details of our submissions in the task 1 of SemEval 2017. This task aims at assessing the semantic textual similarity of two sentences or texts. We …...
QLUT at SemEval-2017 Task 2: Word Similarity Based on Word Embedding and Knowledge Base
AbstractThis paper shows the details of our system submissions in the task 2 of SemEval 2017. We take part in the subtask 1 of this task, which is an English …...
UdL at SemEval-2017 Task 1: Semantic Textual Similarity Estimation of English Sentence Pairs Using Regression Model over Pairwise Features
AbstractThis paper describes the model UdL we proposed to solve the semantic textual similarity task of SemEval 2017 workshop. The track we participated in was estimating the semantics relatedness of …...
CrystalFeel at SemEval-2018 Task 1: Understanding and Detecting Emotion Intensity using Affective Lexicons
AbstractWhile sentiment and emotion analysis has received a considerable amount of research attention, the notion of understanding and detecting the intensity of emotions is relatively less explored. This paper describes …...
How emotional are you? Neural Architectures for Emotion Intensity Prediction in Microblogs
AbstractSocial media based micro-blogging sites like Twitter have become a common source of real-time information (impacting organizations and their strategies, and are used for expressing emotions and opinions. Automated analysis …...
MOCCA: Measure of Confidence for Corpus Analysis - Automatic Reliability Check of Transcript and Automatic Segmentation
The production of speech corpora typically involves manual labor to verify and correct the output of automatic transcription/segmentation processes. This study investigates the possibility of speeding up this correction process …...
THU_NGN at SemEval-2018 Task 1: Fine-grained Tweet Sentiment Intensity Analysis with Attention CNN-LSTM
AbstractTraditional sentiment analysis approaches mainly focus on classifying the sentiment polarities or emotion categories of texts. However, they can’t exploit the sentiment intensity information. Therefore, the SemEval-2018 Task 1 is …...
Embedding Imputation with Grounded Language Information
AbstractDue to the ubiquitous use of embeddings as input representations for a wide range of natural language tasks, imputation of embeddings for rare and unseen words is a critical problem …...
GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems
AbstractAutomatically evaluating dialogue coherence is a challenging but high-demand ability for developing high-quality open-domain dialogue systems. However, current evaluation metrics consider only surface features or utterance-level semantics, without explicitly considering …...
Speakers Fill Lexical Semantic Gaps with Context
AbstractLexical ambiguity is widespread in language, allowing for the reuse of economical word forms and therefore making language more efficient. If ambiguous words cannot be disambiguated from context, however, this …...
A Prompt-independent and Interpretable Automated Essay Scoring Method for Chinese Second Language Writing
AbstractWith the increasing popularity of learning Chinese as a second language (L2) the development of an automatic essay scoring (AES) method specially for Chinese L2 essays has become animportant task. …...
cs60075_team2 at SemEval-2021 Task 1 : Lexical Complexity Prediction using Transformer-based Language Models pre-trained on various text corpora
AbstractThe main contribution of this paper is to fine-tune transformer-based language models pre-trained on several text corpora, some being general (E.g., Wikipedia, BooksCorpus), some being the corpora from which the …...
Ensemble Fine-tuned mBERT for Translation Quality Estimation
AbstractQuality Estimation (QE) is an important component of the machine translation workflow as it assesses the quality of the translated output without consulting reference translations. In this paper, we discuss …...
IAPUCP at SemEval-2021 Task 1: Stacking Fine-Tuned Transformers is Almost All You Need for Lexical Complexity Prediction
AbstractThis paper describes our submission to SemEval-2021 Task 1: predicting the complexity score for single words. Our model leverages standard morphosyntactic and frequency-based features that proved helpful for Complex Word …...
LangResearchLab NC at SemEval-2021 Task 1: Linguistic Feature Based Modelling for Lexical Complexity
AbstractThe present work aims at assigning a complexity score between 0 and 1 to a target word or phrase in a given sentence. For each Single Word Target, a Random …...
Manchester Metropolitan at SemEval-2021 Task 1: Convolutional Networks for Complex Word Identification
AbstractWe present two convolutional neural networks for predicting the complexity of words and phrases in context on a continuous scale. Both models utilize word and character embeddings alongside lexical features …...
Multilingual Machine Translation Evaluation Metrics Fine-tuned on Pseudo-Negative Examples for WMT 2021 Metrics Task
AbstractThis paper describes our submission to the WMT2021 shared metrics task. Our metric is operative to segment-level and system-level translations. Our belief toward a better metric is to detect a …...
Readability and Linearity
This paper investigates automatic assessments of text readability for second language learn- ers. The goodness of text readability as- sessment methods has been evaluated using the correlation coefficients between their …...
BL.Research at SemEval-2022 Task 8: Using various Semantic Information to evaluate document-level Semantic Textual Similarity
AbstractThis paper presents our system for document-level semantic textual similarity (STS) evaluation at SemEval-2022 Task 8: “Multilingual News Article Similarity”. The semantic information used is obtained by using different semantic …...
Impact of Distance Measures on Urdu Document Clustering
Document Clustering aims to group similar documents based on the distance among them. A wide range of distance measures are available in the literature, and selecting an appropriate distance function …...
Noisy Label Regularisation for Textual Regression
AbstractTraining with noisy labelled data is known to be detrimental to model performance, especially for high-capacity neural network models in low-resource domains. Our experiments suggest that standard regularisation strategies, such …...
UZH_CLyp at SemEval-2023 Task 9: Head-First Fine-Tuning and ChatGPT Data Generation for Cross-Lingual Learning in Tweet Intimacy Prediction
AbstractThis paper describes the submission of UZH_CLyp for the SemEval 2023 Task 9 “Multilingual Tweet Intimacy Analysis. We achieved second-best results in all 10 languages according to the official Pearson’s …...