The recent success of large and deep neural network models has motivated the training of even larger and deeper networks with millions of parameters. Training these models usually requires parallel training methods where communicating large number of parameters becomes one of the main bottlenecks. We show that many deep learning models are over-parameterized and their learned features can be predicted given only a small fraction of their parameters. We then propose a method which exploits this fact during the training to reduce the number of parameters that need to be learned. Our method is orthogonal to the choice of network architecture and can be applied in a wide variety of neural network architectures and application areas. We evaluate...
Recent work on neural networks with probabilistic parameters has shown that parameter uncertainty im...
Deep neural networks have achieved state-of-the-art performance in many artificial intelligence area...
While Deep Neural Networks (DNNs) have achieved tremen-dous success for large vocabulary continuous ...
The recent success of large and deep neural network models has motivated the training of even larger...
We demonstrate that there is significant redundancy in the parameterization of several deep learning...
In my thesis I explored several techniques to improve how to efficiently model signal representation...
Deep learning has attracted tremendous attention from researchers in various fields of information e...
Deep neural networks are highly expressive models that have recently achieved state of the art perfo...
Deep learning has achieved great performance in various areas, such as computer vision, natural lang...
The remarkable practical success of deep learning has revealed some major surprises from a theoretic...
When a large feedforward neural network is trained on a small training set, it typically performs po...
In recent years the performance of deep learning algorithms has been demon-strated in a variety of a...
© Published under licence by IOP Publishing Ltd. Deep neural networks with a large number of paramet...
Deep learning neural networks, or, more precisely, Convolutional Neural Networks (CNNs), have demons...
Thesis (Ph.D.)--University of Washington, 2016-06The choice of feature representation can have a lar...
Recent work on neural networks with probabilistic parameters has shown that parameter uncertainty im...
Deep neural networks have achieved state-of-the-art performance in many artificial intelligence area...
While Deep Neural Networks (DNNs) have achieved tremen-dous success for large vocabulary continuous ...
The recent success of large and deep neural network models has motivated the training of even larger...
We demonstrate that there is significant redundancy in the parameterization of several deep learning...
In my thesis I explored several techniques to improve how to efficiently model signal representation...
Deep learning has attracted tremendous attention from researchers in various fields of information e...
Deep neural networks are highly expressive models that have recently achieved state of the art perfo...
Deep learning has achieved great performance in various areas, such as computer vision, natural lang...
The remarkable practical success of deep learning has revealed some major surprises from a theoretic...
When a large feedforward neural network is trained on a small training set, it typically performs po...
In recent years the performance of deep learning algorithms has been demon-strated in a variety of a...
© Published under licence by IOP Publishing Ltd. Deep neural networks with a large number of paramet...
Deep learning neural networks, or, more precisely, Convolutional Neural Networks (CNNs), have demons...
Thesis (Ph.D.)--University of Washington, 2016-06The choice of feature representation can have a lar...
Recent work on neural networks with probabilistic parameters has shown that parameter uncertainty im...
Deep neural networks have achieved state-of-the-art performance in many artificial intelligence area...
While Deep Neural Networks (DNNs) have achieved tremen-dous success for large vocabulary continuous ...
Add to ORKG