Аннотации человека
Related Papers
Comparative Quality Estimation: Automatic Sentence-Level Ranking of Multiple Machine Translation Outputs
A machine learning mechanism is learned from human annotations in order to perform preference ranking. The mechanism operates on a sentence level and ranks the alternative machine translations of each …...
TruthTeller: Annotating Predicate Truth
We propose a novel semantic anno- tation type of assigning truth values to predicate occurrences, and present TruthTeller, a standalone publicly- available tool that produces such annota- tions. TruthTeller integrates …...
See No Evil, Say No Evil: Description Generation from Densely Labeled Images
This paper studies generation of descrip- tive sentences from densely annotated im- ages. Previous work studied generation from automatically detected visual infor- mation but produced a limited class of sen- …...
Phrase Level Segmentation and Labelling of Machine Translation Errors
AbstractThis paper presents our work towards a novel approach for Quality Estimation (QE) of machine translation based on sequences of adjacent words, the so-called phrases. This new level of QE …...
Emotion Representation Mapping for Automatic Lexicon Construction (Mostly) Performs on Human Level
AbstractEmotion Representation Mapping (ERM) has the goal to convert existing emotion ratings from one representation format into another one, e.g., mapping Valence-Arousal-Dominance annotations for words or sentences into Ekman’s Basic …...
SimLex-999 for Polish
The paper addresses the Polish version of SimLex-999 which we extended to contain not only measurement of similarity but also relatedness. The data was translated by three independent linguists; discrepancies …...
Exploring Human Gender Stereotypes with Word Association Test
AbstractWord embeddings have been widely used to study gender stereotypes in texts. One key problem regarding existing bias scores is to evaluate their validities: do they really reflect true bias …...
Predicting Malware Attributes from Cybersecurity Texts
AbstractText analytics is a useful tool for studying malware behavior and tracking emerging threats. The task of automated malware attribute identification based on cybersecurity texts is very challenging due to …...
A Diagnostic Study of Explainability Techniques for Text Classification
AbstractRecent developments in machine learning have introduced models that approach human performance at the cost of increased architectural complexity. Efforts to make the rationales behind the models’ predictions transparent have …...
FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization
AbstractNeural abstractive summarization models are prone to generate content inconsistent with the source document, i.e. unfaithful. Existing automatic metrics do not capture such mistakes effectively. We tackle the problem of …...
Evaluation of Thematic Coherence in Microblogs
AbstractCollecting together microblogs representing opinions about the same topics within the same timeframe is useful to a number of different tasks and practitioners. A major question is how to evaluate …...
Semantic Overlap Summarization among Multiple Alternative Narratives: An Exploratory Study
AbstractIn this paper, we introduce an important yet relatively unexplored NLP task called Semantic Overlap Summarization (SOS), which entails generating a single summary from multiple alternative narratives which can convey …...