While Deep Neural Networks (DNNs) have achieved tremen-dous success for large vocabulary continuous speech recognition (LVCSR) tasks, training of these networks is slow. One reason is that DNNs are trained with a large number of training parameters (i.e., 10-50 million). Because networks are trained with a large number of output targets to achieve good performance, the majority of these parameters are in the final weight layer. In this paper, we propose a low-rank matrix factorization of the final weight layer. We apply this low-rank technique to DNNs for both acoustic modeling and lan-guage modeling. We show on three different LVCSR tasks ranging between 50-400 hrs, that a low-rank factorization reduces the num-ber of parameters of the net...
In the last years, deep neural networks have revolutionized machine learning tasks. However, the des...
We investigate two strategies to improve the context-dependent deep neural network hidden Markov mod...
We propose a novel low-rank initialization framework for training low-rank deep neural networks -- n...
Abstract—Deep neural networks (DNNs) have proven very successful for automatic speech recognition bu...
We propose a new technique for training deep neural networks (DNNs) as data-driven feature front-end...
Targeting the on-device speech-To-Text application for streaming inputs, this paper presents an effi...
© 2016 IEEE. Multilingual Deep Neural Networks (DNNs) have been successfully used to exploit out-of-...
Recently, the deep neural network (DNN) has become one of the most advanced and powerful methods use...
Deep neural networks (DNNs) are now a central component of nearly all state-of-the-art speech recogn...
The recent success of large and deep neural network models has motivated the training of even larger...
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Comp...
As a feed-forward architecture, the recently proposed maxout networks integrate dropout naturally an...
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Com...
Abstract—Recently, deep neural network (DNN) based a-coustic modeling has been successfully applied ...
In this paper, we propose a rank‐weighted reconstruction feature to improve the robustness of a feed...
In the last years, deep neural networks have revolutionized machine learning tasks. However, the des...
We investigate two strategies to improve the context-dependent deep neural network hidden Markov mod...
We propose a novel low-rank initialization framework for training low-rank deep neural networks -- n...
Abstract—Deep neural networks (DNNs) have proven very successful for automatic speech recognition bu...
We propose a new technique for training deep neural networks (DNNs) as data-driven feature front-end...
Targeting the on-device speech-To-Text application for streaming inputs, this paper presents an effi...
© 2016 IEEE. Multilingual Deep Neural Networks (DNNs) have been successfully used to exploit out-of-...
Recently, the deep neural network (DNN) has become one of the most advanced and powerful methods use...
Deep neural networks (DNNs) are now a central component of nearly all state-of-the-art speech recogn...
The recent success of large and deep neural network models has motivated the training of even larger...
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Comp...
As a feed-forward architecture, the recently proposed maxout networks integrate dropout naturally an...
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Com...
Abstract—Recently, deep neural network (DNN) based a-coustic modeling has been successfully applied ...
In this paper, we propose a rank‐weighted reconstruction feature to improve the robustness of a feed...
In the last years, deep neural networks have revolutionized machine learning tasks. However, the des...
We investigate two strategies to improve the context-dependent deep neural network hidden Markov mod...
We propose a novel low-rank initialization framework for training low-rank deep neural networks -- n...