Abstract

Real world objects have persistent structure. However, as we move about in the world the spatio-temporal patterns coming from our sensory organs vary continuously. How the brain creates invariant representations from the always-changing input patterns is a major unanswered question. We propose that the neocortex solves the invariance problem by using a hierarchical structure. Each region in the hierarchy learns and recalls sequences of inputs. Temporal sequences at each level of the hierarchy become the spatial inputs to the next higher regions. Thus the entire memory system stores sequences in sequences. The hierarchical model is highly efficient in that object representations at any level in the hierarchy can be shared among multiple higher order objects, therefore, transformations learned for one set of objects will automatically apply to others. Assuming a hierarchy of sequences, and assuming that each region in the hierarchy behaves equivalently, we derive the optimal Bayes inference rules for any level in the cortical hierarchy and we show how feedfoward and feedback can be understood within this probabilistic framework. We discuss how the hierarchical nested structure of sequences can be learned. We show that static group formation and probability density formation are special cases of remembering sequences. Thus, although normal vision is a temporal process we are able to recognize flashed static images as well. We use the most basic form of one of these special cases to train an object recognition system that exhibits robust invariant recognition.