Gradient Compression
method
ID: gradientcompression
Related Papers
Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training
AbstractOne way to reduce network traffic in multi-node data-parallel stochastic gradient descent is to only exchange the largest gradients. However, doing so damages the gradient and degrades the model’s performance. …...