Vstupní data a jejich význam pro vrstevnaté neuronové sítě

In the present work we study In some areas, artificial feed forward neural networks are still a competitive machine learning model. Unfortunately they tend to overfit the training data, which limits their ability to generalize. We study methods for regularization based on enforcing internal structure of the network. We analyze internal representations using a theoretical model based on information theory. Based on this study, we propose a regularizer that minimizes the overall entropy of internal representations. The entropy-based regularizer is computationally demanding and we use it primarily as a theoretical motivation. To develop an efficient and flexible implementation, we design a Gaussian mixture model of activations. In the experimental part, we compare our model with the existing work based on enforcement of internal representations. The presented Gaussian mixture model regularizer yields better results especially for classification tasks