Deep Learning and Subspace Segmentation: Theory and Applications

The first goal of this research is to improve the mathematical understanding of deep convolutional networks by exploring its relation to subspace segmentation. This research develops a set of algorithms that improve the subspace separation. The training stage generated set of automatic features are more separable compared to that of the traditional deep convolutional networks. The new algorithms are to be applied to the object classification problem. The second goal is to develop a novel technique for the segmentation of data W = [w1 · · · wN] ⊂ RD drawn from a union U = ∪Mi = 1 Si of subspaces {Si}Mi = 1. First, an existing subspace segmentation algorithm is used to perform an initial data clustering {Ci}Mi = 1, where Ci = {wi1 ···wik} ⊂ W is the set of data from the ith cluster. Then, a local subspace LSi is matched for each Ci and the distance dij between LSi and each point wij 2 Ci is computed. A data-driven threshold η is computed and the data points (in Ci) whose distances to LSi are larger than η are eliminated since they are considered as outliers or erroneously clustered data points in Ci. The remaining datapoints Ci ⊂ Ci are considered to be coming from the same subspace with high confidence. Then, {Ci}Mi = 1 are used in unsupervised way to train a convolution neural network to obtain a deep learning model, which is in turn used to re-cluster W. The system has been successfully implemented using the MNIST dataset and it improved the segmentation accuracy of a particular algorithm (EnSC-ORGEN) from 93.79% to 96.52%