Add to ORKGIt has been intensively investigated that the local shape, especially flatness, of the loss landscape near a minimum plays an important role for generalization of deep models. We developed a training algorithm called PoF: Post-Training of Feature Extractor that updates the feature extractor part of an already-trained deep model to search a flatter minimum. The characteristics are two-fold: 1) Feature extractor is trained under parameter perturbations in the higher-layer parameter space, based on observations that suggest flattening higher-layer parameter space, and 2) the perturbation range is determined in a data-driven manner aiming to reduce a part of test loss caused by the positive loss curvature. We provide a theoretical analysis that...
The recent success of large and deep neural network models has motivated the training of even larger...
Deep networks are typically trained with many more parameters than the size of the training dataset....
Model compression by way of parameter pruning, quantization, or distillation has recently gained pop...
Contour detection serves as the basis of a variety of com-puter vision tasks such as image segmentat...
The understanding of generalization in machine learning is in a state of flux. This is partly due to...
In the last decade or so, deep learning has revolutionized entire domains of machine learning. Neura...
Deep neural networks often consist of a great number of trainable parameters for extracting powerful...
The great success of deep learning heavily relies on increasingly larger training data, which comes ...
Neural networks are more expressive when they have multiple layers. In turn, conventional training m...
"Feature representations are the backbone of computer vision.They allow us to summarize the overwhel...
Recent legislation has led to interest in machine unlearning, i.e., removing specific training sampl...
In the past decade, neural networks have demonstrated impressive performance in supervised learning....
How to train deep neural networks (DNNs) to generalize well is a central concern in deep learning, e...
The success of deep learning has revealed the application potential of neural networks across the sc...
In deep learning, optimization plays a vital role. By focusing on image classification, this work in...
The recent success of large and deep neural network models has motivated the training of even larger...
Deep networks are typically trained with many more parameters than the size of the training dataset....
Model compression by way of parameter pruning, quantization, or distillation has recently gained pop...
Contour detection serves as the basis of a variety of com-puter vision tasks such as image segmentat...
The understanding of generalization in machine learning is in a state of flux. This is partly due to...
In the last decade or so, deep learning has revolutionized entire domains of machine learning. Neura...
Deep neural networks often consist of a great number of trainable parameters for extracting powerful...
The great success of deep learning heavily relies on increasingly larger training data, which comes ...
Neural networks are more expressive when they have multiple layers. In turn, conventional training m...
"Feature representations are the backbone of computer vision.They allow us to summarize the overwhel...
Recent legislation has led to interest in machine unlearning, i.e., removing specific training sampl...
In the past decade, neural networks have demonstrated impressive performance in supervised learning....
How to train deep neural networks (DNNs) to generalize well is a central concern in deep learning, e...
The success of deep learning has revealed the application potential of neural networks across the sc...
In deep learning, optimization plays a vital role. By focusing on image classification, this work in...
The recent success of large and deep neural network models has motivated the training of even larger...
Deep networks are typically trained with many more parameters than the size of the training dataset....
Model compression by way of parameter pruning, quantization, or distillation has recently gained pop...