We review recent works (Sarao Mannelli et al 2018 arXiv:1812.09066, 2019 Int. Conf. on Machine Learning 4333–42, 2019 Adv. Neural Information Processing Systems 8676–86) on analyzing the dynamics of gradient-based algorithms in a prototypical statistical inference problem. Using methods and insights from the physics of glassy systems, these works showed how to understand quantitatively and qualitatively the performance of gradient-based algorithms. Here we review the key results and their interpretation in non-technical terms accessible to a wide audience of physicists in the context of related works
International audienceAn algorithmically hard phase was described in a range of inference problems: ...
International audienceStochastic approximation (SA) is a classical algorithm that has had since the ...
When a parameter space has a certain underlying structure, the ordinary gradient of a function does ...
International audienceIn this work we analyse quantitatively the interplay between the loss landscap...
Gradient-descent-based algorithms and their stochastic versions have widespread applications in mach...
Gradient-descent-based algorithms and their stochastic versions have widespread applications in mach...
Is Stochastic Gradient Descent (SGD) substantially different from Glauber dynamics? This is a fundam...
The goal of this paper is to debunk and dispel the magic behind black-box optimizers and stochastic ...
An algorithmically hard phase is described in a range of inference problems: Even if the signal can ...
Modern machine learning models are complex, hierarchical, and large-scale and are trained using non-...
Stochastic Gradient Descent (SGD) is an out-of-equilibrium algorithm used extensively to train artif...
We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical ...
The asymptotic behavior of the stochastic gradient algorithm using biased gradient estimates is anal...
In this paper we investigate how gradient-based algorithms such as gradient descent (GD), (multi-pas...
In this paper, we present algorithms that perform gradient ascent of the average reward in a partial...
International audienceAn algorithmically hard phase was described in a range of inference problems: ...
International audienceStochastic approximation (SA) is a classical algorithm that has had since the ...
When a parameter space has a certain underlying structure, the ordinary gradient of a function does ...
International audienceIn this work we analyse quantitatively the interplay between the loss landscap...
Gradient-descent-based algorithms and their stochastic versions have widespread applications in mach...
Gradient-descent-based algorithms and their stochastic versions have widespread applications in mach...
Is Stochastic Gradient Descent (SGD) substantially different from Glauber dynamics? This is a fundam...
The goal of this paper is to debunk and dispel the magic behind black-box optimizers and stochastic ...
An algorithmically hard phase is described in a range of inference problems: Even if the signal can ...
Modern machine learning models are complex, hierarchical, and large-scale and are trained using non-...
Stochastic Gradient Descent (SGD) is an out-of-equilibrium algorithm used extensively to train artif...
We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical ...
The asymptotic behavior of the stochastic gradient algorithm using biased gradient estimates is anal...
In this paper we investigate how gradient-based algorithms such as gradient descent (GD), (multi-pas...
In this paper, we present algorithms that perform gradient ascent of the average reward in a partial...
International audienceAn algorithmically hard phase was described in a range of inference problems: ...
International audienceStochastic approximation (SA) is a classical algorithm that has had since the ...
When a parameter space has a certain underlying structure, the ordinary gradient of a function does ...