Hyper Parameters
Related Papers
LoonyBin: Keeping Language Technologists Sane through Automated Management of Experimental (Hyper)Workflows
AbstractMany contemporary language technology systems are characterized by long pipelines of tools with complex dependencies. Too often, these workflows are implemented by ad hoc scripts; or, worse, tools are run …...
Comparison of scheduling methods for the learning rate of neural network language models (Modèles de langue neuronaux: une comparaison de plusieurs stratégies d’apprentissage) [in French]
. If neural networks play an increasingly important role in natural language processing, training issues still hinder their dissemination in the community. This paper studies different learning strategies for neural …...
Massive Exploration of Neural Machine Translation Architectures
AbstractNeural Machine Translation (NMT) has shown remarkable progress over the past few years, with production systems now being deployed to end-users. As the field is moving rapidly, it has become …...
Analyzing Word Embedding Through Structural Equation Modeling
AbstractMany researchers have tried to predict the accuracies of extrinsic evaluation by using intrinsic evaluation to evaluate word embedding. The relationship between intrinsic and extrinsic evaluation, however, has only been …...
Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change
AbstractThe choice of hyper-parameters affects the performance of neural models. While much previous research (Sutskever et al., 2013; Duchi et al., 2011; Kingma and Ba, 2015) focuses on accelerating convergence …...
Empirical Studies of Institutional Federated Learning For Natural Language Processing
AbstractFederated learning has sparkled new interests in the deep learning society to make use of isolated data sources from independent institutes. With the development of novel training tools, we have …...
English WordNet Random Walk Pseudo-Corpora
AbstractThis is a resource description paper that describes the creation and properties of a set of pseudo-corpora generated artificially from a random walk over the English WordNet taxonomy. Our WordNet …...
JUST System for WMT20 Chat Translation Task
AbstractMachine Translation (MT) is a sub-field of Artificial Intelligence and Natural Language Processing that investigates and studies the ways of automatically translating a text from one language to another. In …...
DuluthNLP at SemEval-2021 Task 7: Fine-Tuning RoBERTa Model for Humor Detection and Offense Rating
AbstractThis paper presents the DuluthNLP submission to Task 7 of the SemEval 2021 competition on Detecting and Rating Humor and Offense. In it, we explain the approach used to train …...
Exploring Mode Connectivity for Pre-trained Language Models
AbstractRecent years have witnessed the prevalent application of pre-trained language models (PLMs) in NLP. From the perspective of parameter space, PLMs provide generic initialization, starting from which high-performance minima could …...
Unsupervised Domain Adaptation for Text Classification via Meta Self-Paced Learning
AbstractA shift in data distribution can have a significant impact on performance of a text classification model. Recent methods addressing unsupervised domain adaptation for textual tasks typically extracted domain-invariant representations …...