An Evolutionary Approximation to Contrastive Divergence in Convolutional Restricted Boltzmann Machines

Deep learning is an emerging area in machine learning that exploits multi-layered neural networks to extract invariant relationships from large data sets. Deep learning uses layers of non-linear transformations to represent data in abstract and discrete forms. Several different architectures have been developed over the past few years specifically to process images including the Convolutional Restricted Boltzmann Machine. The Boltzmann Machine is trained using contrastive divergence, a depth-first gradient based training algorithm. Gradient based training methods have no guarantee of reaching an optimal solution and tend to search a limited region of the solution space. In this thesis, we present an alternative method for synthesizing deep networks using evolutionary algorithms. This is a breadth-first stochastic search process that utilizes reconstruction error along with additional properties to encourage evolution of unique features. Using this technique, potentially a larger region of the solution space is explored allowing identification of different types of solutions using less training data. The process of developing this method is discussed along with its potential as a viable replacement to contrastive divergence