Өзін-өзі назар аудару механизмі
Related Papers
Zero Pronoun Resolution with Attention-based Neural Network
AbstractRecent neural network methods for zero pronoun resolution explore multiple models for generating representation vectors for zero pronouns and their candidate antecedents. Typically, contextual information is utilized to encode the …...
Adaptive Attention Span in Transformers
AbstractWe propose a novel self-attention mechanism that can learn its optimal attention span. This allows us to extend significantly the maximum context size used in Transformer, while maintaining control over …...
Fine-tune BERT with Sparse Self-Attention Mechanism
AbstractIn this paper, we develop a novel Sparse Self-Attention Fine-tuning model (referred as SSAF) which integrates sparsity into self-attention mechanism to enhance the fine-tuning performance of BERT. In particular, sparsity …...
ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation
AbstractIn multi-turn dialogue generation, response is usually related with only a few contexts. Therefore, an ideal model should be able to detect these relevant contexts and produce a suitable response …...
A Transformer-based Approach for Source Code Summarization
AbstractGenerating a readable summary that describes the functionality of a program is known as source code summarization. In this task, learning code representation by modeling the pairwise relationship between code …...
An Element-aware Multi-representation Model for Law Article Prediction
AbstractExisting works have proved that using law articles as external knowledge can improve the performance of the Legal Judgment Prediction. However, they do not fully use law article information and …...
Improve Transformer Models with Better Relative Position Embeddings
AbstractThe transformer model has demonstrated superior results on NLP tasks including machine translation and question answering. In this paper, we argue that the position information is not fully utilized in …...
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
AbstractEntity representations are useful in natural language tasks involving entities. In this paper, we propose new pretrained contextualized representations of words and entities based on the bidirectional transformer. The proposed …...
Modelling Context and Syntactical Features for Aspect-based Sentiment Analysis
AbstractThe aspect-based sentiment analysis (ABSA) consists of two conceptual tasks, namely an aspect extraction and an aspect sentiment classification. Rather than considering the tasks separately, we build an end-to-end ABSA …...
NLFIIT at SemEval-2020 Task 11: Neural Network Architectures for Detection of Propaganda Techniques in News Articles
AbstractSince propaganda became more common technique in news, it is very important to look for possibilities of its automatic detection. In this paper, we present neural model architecture submitted to …...
Syntactically Aware Cross-Domain Aspect and Opinion Terms Extraction
AbstractA fundamental task of fine-grained sentiment analysis is aspect and opinion terms extraction. Supervised-learning approaches have shown good results for this task; however, they fail to scale across domains where …...
應用多模式特徵融合的深度注意力網路進行謠言檢測 (Rumor Detection Using Deep Attention Networks With Multimodal Feature Fusion)
With the rapid growth of information, browsing social media on the Internet is becoming a part of people’s daily lives. Social platforms give us the latest information in real time, …...
Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation
AbstractUnderstanding voluminous historical records provides clues on the past in various aspects, such as social and political issues and even natural science facts. However, it is generally difficult to fully …...
Time-aware Graph Neural Network for Entity Alignment between Temporal Knowledge Graphs
AbstractEntity alignment aims to identify equivalent entity pairs between different knowledge graphs (KGs). Recently, the availability of temporal KGs (TKGs) that contain time information created the need for reasoning over …...
ClusterFormer: Neural Clustering Attention for Efficient and Effective Transformer
AbstractRecently, a lot of research has been carried out to improve the efficiency of Transformer. Among them, the sparse pattern-based method is an important branch of efficient Transformers. However, some …...
Entropy- and Distance-Based Predictors From GPT-2 Attention Patterns Predict Reading Times Over and Above GPT-2 Surprisal
AbstractTransformer-based large language models are trained to make predictions about the next word by aggregating representations of previous tokens through their self-attention mechanism. In the field of cognitive modeling, such …...
Nonparametric Forest-Structured Neural Topic Modeling
AbstractNeural topic models have been widely used in discovering the latent semantics from a corpus. Recently, there are several researches on hierarchical neural topic models since the relationships among topics …...
TAGPRIME: A Unified Framework for Relational Structure Extraction
AbstractMany tasks in natural language processing require the extraction of relationship information for a given condition, such as event argument extraction, relation extraction, and task-oriented semantic parsing. Recent works usually …...
Whisper Model Adaptation for FSR-2023 Hakka Speech Recognition Challenge
This study focuses on the adaptation and training of the Whisper model using the provided data and text annotations from the FSR-2023 Hakka Speech Recognition Challenge. The Whisper model is …...