Модель BERT
Related Papers
Combating Adversarial Misspellings with Robust Word Recognition
AbstractTo combat adversarial spelling mistakes, we propose placing a word recognition model in front of the downstream classifier. Our word recognition models build upon the RNN semi-character architecture, introducing several …...
Combining Unsupervised Pre-training and Annotator Rationales to Improve Low-shot Text Classification
AbstractSupervised learning models often perform poorly at low-shot tasks, i.e. tasks for which little labeled data is available for training. One prominent approach for improving low-shot learning is to use …...
Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark
AbstractThe GLUE benchmark (Wang et al., 2019b) is a suite of language understanding tasks which has seen dramatic progress in the past year, with average performance moving from 70.0 at …...
Affective Event Classification with Discourse-enhanced Self-training
AbstractPrior research has recognized the need to associate affective polarities with events and has produced several techniques and lexical resources for identifying affective events. Our research introduces new classification models …...
BYteam at SemEval-2020 Task 5: Detecting Counterfactual Statements with BERT and Ensembles
AbstractWe participate in the classification tasks of SemEval-2020 Task: Subtask1: Detecting counterfactual statements of semeval-2020 task5(Detecting Counterfactuals). This paper examines different approaches and models towards detecting counterfactual statements classification. We …...
CancerEmo: A Dataset for Fine-Grained Emotion Detection
AbstractEmotions are an important element of human nature, often affecting the overall wellbeing of a person. Therefore, it is no surprise that the health domain is a valuable area of …...
GoEmotions: A Dataset of Fine-Grained Emotions
AbstractUnderstanding emotion expressed in language has a wide range of applications, from building empathetic chatbots to detecting harmful online behavior. Advancement in this area can be improved using large-scale datasets …...
Gorynych Transformer at SemEval-2020 Task 6: Multi-task Learning for Definition Extraction
AbstractThis paper describes our approach to “DeftEval: Extracting Definitions from Free Text in Textbooks” competition held as a part of Semeval 2020. The task was devoted to finding and labeling …...
On the weak link between importance and prunability of attention heads
AbstractGiven the success of Transformer-based models, two directions of study have emerged: interpreting role of individual attention heads and down-sizing the models for efficiency. Our work straddles these two streams: …...
Profile Consistency Identification for Open-domain Dialogue Agents
AbstractMaintaining a consistent attribute profile is crucial for dialogue agents to naturally converse with humans. Existing studies on improving attribute consistency mainly explored how to incorporate attribute information in the …...
Understanding User Utterances in a Dialog System for Caregiving
AbstractA dialog system that can monitor the health status of seniors has a huge potential for solving the labor force shortage in the caregiving industry in aging societies. As a …...
Unsupervised FAQ Retrieval with Question Generation and BERT
AbstractWe focus on the task of Frequently Asked Questions (FAQ) retrieval. A given user query can be matched against the questions and/or the answers in the FAQ. We present a …...
Will_go at SemEval-2020 Task 9: An Accurate Approach for Sentiment Analysis on Hindi-English Tweets Based on Bert and Pesudo Label Strategy
AbstractMixing languages are widely used in social media, especially in multilingual societies like India. Detecting the emotions contained in these languages, which is of great significance to the development of …...
You May Like This Hotel Because ...: Identifying Evidence for Explainable Recommendations
AbstractExplainable recommendation is a good way to improve user satisfaction. However, explainable recommendation in dialogue is challenging since it has to handle natural language as both input and output. To …...
Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset
AbstractWe introduce the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP to assess how European solidarity …...
Comparing Feature-Engineering and Feature-Learning Approaches for Multilingual Translationese Classification
AbstractTraditional hand-crafted linguistically-informed features have often been used for distinguishing between translated and original non-translated texts. By contrast, to date, neural architectures without manual feature engineering have been less explored …...
Detecting Health Advice in Medical Research Literature
AbstractHealth and medical researchers often give clinical and policy recommendations to inform health practice and public health policy. However, no current health information system supports the direct retrieval of health …...
Editing Factual Knowledge in Language Models
AbstractThe factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual inference). However, some facts …...
HOMADOS at SemEval-2021 Task 6: Multi-Task Learning for Propaganda Detection
AbstractAmong the tasks motivated by the proliferation of misinformation, propaganda detection is particularly challenging due to the deficit of fine-grained manual annotations required to train machine learning models. Here we …...
Improve Chit-Chat and QA Sentence Classification in User Messages of Dialogue System using Dialogue Act Embedding
AbstractIn recent years, dialogue system is booming and widely used in customer service system, and has achieved good results. Viewing the conversation records between users and real customer service, we …...
Self Promotion in US Congressional Tweets
AbstractPrior studies have found that women self-promote less than men due to gender stereotypes. In this study we built a BERT-based NLP model to predict whether a Congressional tweet shows …...
BERT in Plutarch’s Shadows
AbstractThe extensive surviving corpus of the ancient scholar Plutarch of Chaeronea (ca. 45-120 CE) also contains several texts which, according to current scholarly opinion, did not originate with him and …...
Building Large-Scale Japanese Pronunciation-Annotated Corpora for Reading Heteronymous Logograms
AbstractAlthough screen readers enable visually impaired people to read written text via speech, the ambiguities in pronunciations of heteronyms cause wrong reading, which has a serious impact on the text …...
Constructing A Dataset of Support and Attack Relations in Legal Arguments in Court Judgements using Linguistic Rules
AbstractArgumentation mining is a growing area of research and has several interesting practical applications of mining legal arguments. Support and Attack relations are the backbone of any legal argument. However, …...
I2C at SemEval-2022 Task 6: Intended Sarcasm in English using Deep Learning Techniques
AbstractSarcasm is often expressed through several verbal and non-verbal cues, e.g., a change of tone, overemphasis in a word, a drawn-out syllable, or a straight looking face. Most of the …...
IITR CodeBusters at SemEval-2022 Task 5: Misogyny Identification using Transformers
AbstractThis paper presents our submission to task 5 ( Multimedia Automatic Misogyny Identification) of the SemEval 2022 competition. The purpose of the task is to identify given memes as misogynistic …...
KaMiKla at SemEval-2022 Task 3: AlBERTo, BERT, and CamemBERT—Be(r)tween Taxonomy Detection and Prediction
AbstractThis paper describes our system submitted for SemEval Task 3: Presupposed Taxonomies: Evaluating Neural Network Semantics (Zamparelli et al., 2022). We participated in both the binary classification and the regression …...
NERVE at ROCLING 2022 Shared Task: A Comparison of Three Named Entity Recognition Frameworks Based on Language Model and Lexicon Approach
AbstractROCLING 2022 shared task is to design a method that can tag medical entities in sentences and then classify them into categories through an algorithm. This paper proposes three models …...