Natural Gradient Descent for Training Stochastic Complex-Valued Neural Networks

Abstract—In this paper, the natural gradient descent method for the multilayer stochastic complex-valued neural networks is considered, and the natural gradient is given for a single stochastic complex-valued neuron as an example. Since the space of the learnable parameters of stochastic complex-valued neural networks is not the Euclidean space but a curved manifold, the complex-valued natural gradient method is expected to exhibit excellent learning performance. Keywords—Neural network; Complex number; Learning; Sin-gular poin