Lococode Performs Nonlinear ICA Without Knowing The Number Of Sources

Low-complexity coding and decoding (Lococode), a novel approach to sensory coding, trains autoassociators (AAs) by Flat Minimum Search (FMS), a recent general method for finding low-complexity networks with high generalization capability. FMS works by minimizing both training error and required weight precision. We find that as a by-product Lococode separates nonlinear superpositions of sources without knowing their number. Assuming that the input data can be reduced to few simple causes (this is often the case with visual data), according to our theoretical analysis the hidden layer of an FMS-trained AA tends to code each input by a sparse code based on as few simple, independent features as possible. In experiments Lococode extracts optimal codes for difficult, nonlinear versions of the "noisy bars" benchmark problem, while traditional ICA and PCA do not