Add to ORKGThe aim of this paper is to introduce two widely applicable regularization methods based on the direct modification of weight matrices. The first method, Weight Reinitialization, utilizes a simplified Bayesian assumption with partially resetting a sparse subset of the parameters. The second one, Weight Shuffling, introduces an entropy- and weight distribution-invariant non-white noise to the parameters. The latter can also be interpreted as an ensemble approach. The proposed methods are evaluated on benchmark datasets, such as MNIST, CIFAR-10 or the JSB Chorales database, and also on time series modeling masks. We report gains both regarding performance and entropy of the analyzed networks. We also made our code available as a GitHub repos...
Unsupervised neural networks, such as restricted Boltzmann machines (RBMs) and deep belief networks ...
Regularization is essential for avoiding over-fitting to training data in network optimization, lead...
Abstract. In this paper we address the important problem of optimizing regularization parameters in ...
The aim of this paper is to introduce two widely applicable regularization methods based on the dire...
We introduce a simple and effective method for regularizing large convolutional neural networks. We ...
Abstract We present weight normalization: a reparameterization of the weight vectors in a neural net...
For flexible and overparameterised models like neural networks, overfitting can be a notorious probl...
© Copyright 2016, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rig...
Recent years have witnessed the success of deep neural networks in dealing with a plenty of practica...
Regularization is essential when training large neural networks. As deep neural networks can be math...
Ensemble methods of machine learning combine neural networks or other machine learning models in ord...
Rannen Ep Triki A., Blaschko M., ''Stochastic function norm regularization of DNNs'', 9th NIPS works...
Deep neural language models like GPT-2 is undoubtedly strong at text generation, but often requires ...
Multiplicative stochasticity such as Dropout improves the robustness and generalizability of deep ne...
Over-parametrization of deep neural networks has recently been shown to be key to their successful t...
Unsupervised neural networks, such as restricted Boltzmann machines (RBMs) and deep belief networks ...
Regularization is essential for avoiding over-fitting to training data in network optimization, lead...
Abstract. In this paper we address the important problem of optimizing regularization parameters in ...
The aim of this paper is to introduce two widely applicable regularization methods based on the dire...
We introduce a simple and effective method for regularizing large convolutional neural networks. We ...
Abstract We present weight normalization: a reparameterization of the weight vectors in a neural net...
For flexible and overparameterised models like neural networks, overfitting can be a notorious probl...
© Copyright 2016, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rig...
Recent years have witnessed the success of deep neural networks in dealing with a plenty of practica...
Regularization is essential when training large neural networks. As deep neural networks can be math...
Ensemble methods of machine learning combine neural networks or other machine learning models in ord...
Rannen Ep Triki A., Blaschko M., ''Stochastic function norm regularization of DNNs'', 9th NIPS works...
Deep neural language models like GPT-2 is undoubtedly strong at text generation, but often requires ...
Multiplicative stochasticity such as Dropout improves the robustness and generalizability of deep ne...
Over-parametrization of deep neural networks has recently been shown to be key to their successful t...
Unsupervised neural networks, such as restricted Boltzmann machines (RBMs) and deep belief networks ...
Regularization is essential for avoiding over-fitting to training data in network optimization, lead...
Abstract. In this paper we address the important problem of optimizing regularization parameters in ...