With the vigorous development of artificial intelligence technology, various
engineering technology applications have been implemented one after another.
The gradient descent method plays an important role in solving various
optimization problems, due to its simple structure, good stability and easy
implementation. In multi-node machine learning system, the gradients usually
need to be shared. Shared gradients are generally unsafe. Attackers can obtain
training data simply by knowing the gradient information. In this paper, to
prevent gradient leakage while keeping the accuracy of model, we propose the
super stochastic gradient descent approach to update parameters by concealing
the modulus length of gradient vectors and converting it or them into a unit
vector. Furthermore, we analyze the security of super stochastic gradient
descent approach. Our algorithm can defend against attacks on the gradient.
Experiment results show that our approach is obviously superior to prevalent
gradient descent approaches in terms of accuracy, robustness, and adaptability
to large-scale batches.