Add to ORKGWe develop fast algorithms and robust software for convex optimization of two-layer neural networks with ReLU activation functions. Our work leverages a convex reformulation of the standard weight-decay penalized training problem as a set of group-$\ell_1$-regularized data-local models, where locality is enforced by polyhedral cone constraints. In the special case of zero-regularization, we show that this problem is exactly equivalent to unconstrained optimization of a convex "gated ReLU" network. For problems with non-zero regularization, we show that convex gated ReLU models obtain data-dependent approximation bounds for the ReLU training problem. To optimize the convex reformulations, we develop an accelerated proximal gradient method an...
Abstract — Regularization is useful for extending learning models to be effective for classification...
In supervised learning, the regularization path is sometimes used as a convenient theoretical proxy ...
Neural networks (NNs) have seen a surge in popularity due to their unprecedented practical success i...
Understanding the fundamental principles behind the success of deep neural networks is one of the mo...
Training deep neural networks is a well-known highly non-convex problem. In recent works, it is show...
The success of deep neural networks is in part due to the use of normalization layers. Normalization...
Deep neural networks often lack the safety and robustness guarantees needed to be deployed in safety...
We give the first dimension-efficient algorithms for learning Rectified Linear Units (ReLUs), which ...
The practice of deep learning has shown that neural networks generalize remarkably well even with an...
International audienceGiven a training set, a loss function, and a neural network architecture, it i...
International audienceWe introduce a general framework for designing and training neural network lay...
It has been shown that neural network classifiers are not robust. This raises concerns about their u...
We present a novel layerwise optimization algorithm for the learning objective of Piecewise-Linear C...
Convexity has recently received a lot of attention in the machine learning community, and the lack o...
Convex $\ell_1$ regularization using an infinite dictionary of neurons has been suggested for constr...
Abstract — Regularization is useful for extending learning models to be effective for classification...
In supervised learning, the regularization path is sometimes used as a convenient theoretical proxy ...
Neural networks (NNs) have seen a surge in popularity due to their unprecedented practical success i...
Understanding the fundamental principles behind the success of deep neural networks is one of the mo...
Training deep neural networks is a well-known highly non-convex problem. In recent works, it is show...
The success of deep neural networks is in part due to the use of normalization layers. Normalization...
Deep neural networks often lack the safety and robustness guarantees needed to be deployed in safety...
We give the first dimension-efficient algorithms for learning Rectified Linear Units (ReLUs), which ...
The practice of deep learning has shown that neural networks generalize remarkably well even with an...
International audienceGiven a training set, a loss function, and a neural network architecture, it i...
International audienceWe introduce a general framework for designing and training neural network lay...
It has been shown that neural network classifiers are not robust. This raises concerns about their u...
We present a novel layerwise optimization algorithm for the learning objective of Piecewise-Linear C...
Convexity has recently received a lot of attention in the machine learning community, and the lack o...
Convex $\ell_1$ regularization using an infinite dictionary of neurons has been suggested for constr...
Abstract — Regularization is useful for extending learning models to be effective for classification...
In supervised learning, the regularization path is sometimes used as a convenient theoretical proxy ...
Neural networks (NNs) have seen a surge in popularity due to their unprecedented practical success i...