We consider two high-order tuners that have been shown to have accelerated performance, one based on Polyak's heavy ball method and another based on Nesterov's acceleration method. We show that parameter estimates are bounded and converge to the true values exponentially fast when the regressors are persistently exciting. Simulation results corroborate the accelerated performance and accelerated learning properties of these high-order tuners in comparison to algorithms based on normalized gradient descent.Comment: 18 page
Physical networks, such as biological neural networks, can learn desired functions without a central...
The most difficult part in the design and analysis of Learning Automata (LA) consists of the formal ...
Nonconvex optimization with great demand of fast solvers is ubiquitous in modern machine learning. T...
Thesis: S.M., Massachusetts Institute of Technology, Department of Mechanical Engineering, February,...
This paper presents a framework to solve constrained optimization problems in an accelerated manner ...
This paper studies an intriguing phenomenon related to the good generalization performance of estima...
Injecting noise within gradient descent has several desirable features. In this paper, we explore no...
Due to the simplicity and efficiency of the first-order gradient method, it has been widely used in ...
Nesterov's accelerated gradient (AG) is a popular technique to optimize objective functions comprisi...
A recent line of works, initiated by Russo and Xu, has shown that the generalization error of a lear...
Stochastic Gradient Descent (SGD) is an out-of-equilibrium algorithm used extensively to train artif...
Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achiev...
The grokking phenomenon as reported by Power et al. ( arXiv:2201.02177 ) refers to a regime where a ...
In this paper, we provide new results and algorithms (including backtracking versions of Nesterov ac...
We analyze the convergence rates of stochastic gradient algorithms for smooth finite-sum minimax opt...
Physical networks, such as biological neural networks, can learn desired functions without a central...
The most difficult part in the design and analysis of Learning Automata (LA) consists of the formal ...
Nonconvex optimization with great demand of fast solvers is ubiquitous in modern machine learning. T...
Thesis: S.M., Massachusetts Institute of Technology, Department of Mechanical Engineering, February,...
This paper presents a framework to solve constrained optimization problems in an accelerated manner ...
This paper studies an intriguing phenomenon related to the good generalization performance of estima...
Injecting noise within gradient descent has several desirable features. In this paper, we explore no...
Due to the simplicity and efficiency of the first-order gradient method, it has been widely used in ...
Nesterov's accelerated gradient (AG) is a popular technique to optimize objective functions comprisi...
A recent line of works, initiated by Russo and Xu, has shown that the generalization error of a lear...
Stochastic Gradient Descent (SGD) is an out-of-equilibrium algorithm used extensively to train artif...
Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achiev...
The grokking phenomenon as reported by Power et al. ( arXiv:2201.02177 ) refers to a regime where a ...
In this paper, we provide new results and algorithms (including backtracking versions of Nesterov ac...
We analyze the convergence rates of stochastic gradient algorithms for smooth finite-sum minimax opt...
Physical networks, such as biological neural networks, can learn desired functions without a central...
The most difficult part in the design and analysis of Learning Automata (LA) consists of the formal ...
Nonconvex optimization with great demand of fast solvers is ubiquitous in modern machine learning. T...