IEEE 1 A Multiobjective Sparse Feature Learning Model for Deep Neural Networks

Abstract—Hierarchical deep neural networks are currently popular learning models for imitating the hierarchical archi-tecture of human brain. Single layer feature extractors are the bricks to build deep networks. Sparse feature learning models are popular models that can learn useful representations. But most of those models need a user defined constant to control the sparsity of representations. In this paper, we propose a multiobjective sparse feature learning model based on the autoencoder. The parameters of the model are learnt by optimizing two objectives, reconstruction error and sparsity of hidden units simultaneously to find a reasonable compromise between them automatically. We design a multiobjective induced learning procedure for this model based on a multiobjective evolutionary algorithm. In the experiments, we demonstrate that the learning procedure is effective and the proposed multiobjective model can learn useful sparse features. Index Terms—Deep neural networks, sparse feature learning, multiobjective optimization, evolutionary algorithm. I