Софтмакс қоспасы
method
ID: mixtureofsoftmax
Related Papers
Softmax Bottleneck Makes Language Models Unable to Represent Multi-mode Word Distributions
AbstractNeural language models (LMs) such as GPT-2 estimate the probability distribution over the next word by a softmax over the vocabulary. The softmax layer produces the distribution based on the …...