Neural Architecture Search
Related Papers
Automated Concatenation of Embeddings for Structured Prediction
AbstractPretrained contextualized embeddings are powerful word representations for structured prediction tasks. Recent work found that better word representations can be obtained by concatenating different types of embeddings. However, the selection …...
AUTOSUMM: Automatic Model Creation for Text Summarization
AbstractRecent efforts to develop deep learning models for text generation tasks such as extractive and abstractive summarization have resulted in state-of-the-art performances on various datasets. However, obtaining the best model …...
AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models
AbstractPre-trained language models (PLMs) have achieved great success in natural language processing. Most of PLMs follow the default setting of architecture hyper-parameters (e.g., the hidden dimension is a quarter of …...
RankNAS: Efficient Neural Architecture Search by Pairwise Ranking
AbstractThis paper addresses the efficiency challenge of Neural Architecture Search (NAS) by formulating the task as a ranking problem. Previous methods require numerous training examples to estimate the accurate performance …...
Training-free Neural Architecture Search for RNNs and Transformers
AbstractNeural architecture search (NAS) has allowed for the automatic creation of new and effective neural network architectures, offering an alternative to the laborious process of manually designing complex architectures. However, …...