Add to ORKGEstimating the Generalization Error (GE) of Deep Neural Networks (DNNs) is an important task that often relies on availability of held-out data. The ability to better predict GE based on a single training set may yield overarching DNN design principles to reduce a reliance on trial-and-error, along with other performance assessment advantages. In search of a quantity relevant to GE, we investigate the Mutual Information (MI) between the input and final layer representations, using the infinite-width DNN limit to bound MI. An existing input compression-based GE bound is used to link MI and GE. To the best of our knowledge, this represents the first empirical study of this bound. In our attempt to empirically falsify the theoretical bound, we...
Deep Neural Networks (DNNs) are increasingly being used in software engineering and code intelligenc...
Increasing the size of overparameterized neural networks has been shown to improve their generalizat...
The search for effective and robust metrics has been the focus of recent theoretical and empirical w...
While there has been progress in developing non-vacuous generalization bounds for deep neural networ...
Deep learning has transformed computer vision, natural language processing, and speech recognition. ...
Over-parameterized deep neural networks (DNNs) with sufficient capacity to memorize random noise can...
This paper provides theoretical insights into why and how deep learning can generalize well, despite...
Modern deep neural networks (DNNs) represent a formidable challenge for theorists: according to the ...
Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to c...
This is the final version. Available from ICLR via the link in this recordDeep neural networks (DNNs...
Existing generalization bounds fail to explain crucial factors that drive generalization of modern n...
Deep networks are usually trained and tested in a regime in which the training classification error ...
The Information Bottleneck theory provides a theoretical and computational framework for finding app...
In recent years Deep Neural Networks (DNNs) have achieved state-of-the-art results in many fields su...
During the past decade, machine learning techniques have achieved impressive results in a number of ...
Deep Neural Networks (DNNs) are increasingly being used in software engineering and code intelligenc...
Increasing the size of overparameterized neural networks has been shown to improve their generalizat...
The search for effective and robust metrics has been the focus of recent theoretical and empirical w...
While there has been progress in developing non-vacuous generalization bounds for deep neural networ...
Deep learning has transformed computer vision, natural language processing, and speech recognition. ...
Over-parameterized deep neural networks (DNNs) with sufficient capacity to memorize random noise can...
This paper provides theoretical insights into why and how deep learning can generalize well, despite...
Modern deep neural networks (DNNs) represent a formidable challenge for theorists: according to the ...
Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to c...
This is the final version. Available from ICLR via the link in this recordDeep neural networks (DNNs...
Existing generalization bounds fail to explain crucial factors that drive generalization of modern n...
Deep networks are usually trained and tested in a regime in which the training classification error ...
The Information Bottleneck theory provides a theoretical and computational framework for finding app...
In recent years Deep Neural Networks (DNNs) have achieved state-of-the-art results in many fields su...
During the past decade, machine learning techniques have achieved impressive results in a number of ...
Deep Neural Networks (DNNs) are increasingly being used in software engineering and code intelligenc...
Increasing the size of overparameterized neural networks has been shown to improve their generalizat...
The search for effective and robust metrics has been the focus of recent theoretical and empirical w...