Time-causal and time-recursive spatio-temporal receptive fields for computer vision and computational modelling of biological vision

When operating on time-dependent image information in real time, a fundamental constraint originates from the fact that image operations must be both time-causal and time-recursive. In this talk, we will present an improved model and theory for time-causal and time-recursive spatio-temporal receptive fields, obtained by a combination of Gaussian filters over the spatial domain and first-order integrators or equivalently truncated exponential filters coupled in cascade over the temporal domain. This receptive field family obeys scale-space axiomatics in the sense of non-enhancement of local extrema over the spatial domain and non-creation of new local extrema over time for any purely temporal signal and does in these respects guarantee theoretically well-founded treatment of spatio-temporal image structures at different spatial and temporal scales. By a logarithmic distribution of the temporal scale levels in combination with the construction of a time-causal limit kernel based on an infinitely dense distribution of the temporal scale levels towards zero temporal scale, it will be shown that this family allows for temporal scale invariance although the temporal scale levels by the theory have to be discrete. Additionally, the family obeys basic invariance or covariance properties under other classes of natural image transformations including spatial scaling transformations, rotations/affine image deformations over the spatial domain, Galilean transformations of space time and local multiplicative intensity transformations. Thereby, this receptive field family allows for the formulation of multi-scale differential geometric image features with invariance or covariance properties under basic classes of natural image transformations over space-time. It is shown how this spatio-temporal scale-space concept (i) allows for efficient computation of different types of spatio-temporal features for purposes in computer vision and (ii) leads to predictions about biological receptive fields with good qualitative similarities to the results of cell recordings in the lateral geniculate nucleus (LGN) and the primary visual cortex (V1) in biological vision. References: T. Lindeberg (2016) ”Time-causal and time-recursive spatio-temporal receptive fields”, Journal of Mathematical Imaging and Vision, 55(1): 50-88. T. Lindeberg (2013) ”A computational theory of visual receptive fields”, Biological Cybernetics, 107(6): 589–635. T. Lindeberg (2013) ”Invariance of visual operations at the level of receptive fields”, PLOS One, 8(7): e66990. T. Lindeberg (2011) ”Generalized Gaussian scale-space axiomatics comprising linear scale space, affine scale space and spatio-temporal scale space”, Journal of Mathematical Imaging and Vision, 40(1): 36–81.QC 20160603